Varnish doesnt PURGE urls
Guillaume Quintard
guillaume at varnish-software.com
Mon Dec 9 20:31:30 UTC 2019
Purge uses the hash of the objects to clear them. In turn, the hash, by
default depends on the host of the requests that created the object.
So, when you purge using localhost/some/path.mp4, you dont purge the object
referring to my.website.com/some/path.mp4
--
Guillaume Quintard
On Mon, Dec 9, 2019 at 3:24 AM hamidreza hosseini <hrhosseini at hotmail.com>
wrote:
> This is purge to another server:
>
> $curl -v -k -X PURGE http://localhost:8080/Naserfeiz.mp4
>
> * Trying ::1...
> * Connected to localhost (::1) port 8080 (#0)
> > PURGE /Naserfeiz.mp4 HTTP/1.1
> > Host: localhost:8080
> > User-Agent: curl/7.47.0
> > Accept: */*
> >
> < HTTP/1.1 200 Purged
> < Date: Mon, 09 Dec 2019 11:20:52 GMT
> < Server: Varnish
> < X-Varnish: 10
> < Content-Type: text/html; charset=utf-8
> < Retry-After: 5
> < Content-Length: 237
> < Accept-Ranges: bytes
> < Connection: keep-alive
> <
> <!DOCTYPE html>
> <html>
> <head>
> <title>200 Purged</title>
> </head>
> <body>
> <h1>Error 200 Purged</h1>
> <p>Purged</p>
> <h3>Guru Meditation:</h3>
> <p>XID: 10</p>
> <hr>
> <p>Varnish cache server</p>
> </body>
> </html>
> * Connection #0 to host localhost left intact
>
> And varnishlog resault:
>
> $ sudo varnishlog
>
> * << Request >> 32773
> - Begin req 32772 rxreq
> - Timestamp Start: 1575890593.662132 0.000000 0.000000
> - Timestamp Req: 1575890593.662132 0.000000 0.000000
> - ReqStart ::1 63184
> - ReqMethod PURGE
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: localhost:8080
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader X-Forwarded-For: ::1
> - VCL_call RECV
> - VCL_acl MATCH purge "localhost"
> - VCL_return purge
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call PURGE
> - VCL_return synth
> - Timestamp Process: 1575890593.662268 0.000135 0.000135
> - RespHeader Date: Mon, 09 Dec 2019 11:23:13 GMT
> - RespHeader Server: Varnish
> - RespHeader X-Varnish: 32773
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespReason Purged
> - VCL_call SYNTH
> - RespHeader Content-Type: text/html; charset=utf-8
> - RespHeader Retry-After: 5
> - VCL_return deliver
> - RespHeader Content-Length: 240
> - Storage malloc Transient
> - RespHeader Accept-Ranges: bytes
> - Debug "RES_MODE 2"
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1575890593.662484 0.000352 0.000216
> - ReqAcct 93 0 93 218 240 458
> - End
>
> * << Session >> 32772
> - Begin sess 0 HTTP/1
> - SessOpen ::1 63184 :8080 ::1 8080 1575890593.662044 16
> - Link req 32773 rxreq
> - SessClose REM_CLOSE 0.001
> - End
>
> * << Request >> 12
> - Begin req 11 rxreq
> - Timestamp Start: 1575890600.799371 0.000000 0.000000
> - Timestamp Req: 1575890600.799371 0.000000 0.000000
> - ReqStart ::1 63186
> - ReqMethod PURGE
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: localhost:8080
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader X-Forwarded-For: ::1
> - VCL_call RECV
> - VCL_acl MATCH purge "localhost"
> - VCL_return purge
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call PURGE
> - VCL_return synth
> - Timestamp Process: 1575890600.799463 0.000092 0.000092
> - RespHeader Date: Mon, 09 Dec 2019 11:23:20 GMT
> - RespHeader Server: Varnish
> - RespHeader X-Varnish: 12
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespReason Purged
> - VCL_call SYNTH
> - RespHeader Content-Type: text/html; charset=utf-8
> - RespHeader Retry-After: 5
> - VCL_return deliver
> - RespHeader Content-Length: 237
> - Storage malloc Transient
> - RespHeader Accept-Ranges: bytes
> - Debug "RES_MODE 2"
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1575890600.799611 0.000240 0.000147
> - ReqAcct 93 0 93 215 237 452
> - End
>
> * << Session >> 11
> - Begin sess 0 HTTP/1
> - SessOpen ::1 63186 :8080 ::1 8080 1575890600.799261 16
> - Link req 12 rxreq
> - SessClose REM_CLOSE 0.001
> - End
>
> ------------------------------
> *From:* shafeeque aslam <shafeequeaslam at gmail.com>
> *Sent:* Monday, December 9, 2019 3:19 AM
> *To:* hamidreza hosseini <hrhosseini at hotmail.com>
> *Cc:* Guillaume Quintard <guillaume at varnish-software.com>;
> varnish-misc at varnish-cache.org <varnish-misc at varnish-cache.org>
> *Subject:* Re: Varnish doesnt PURGE urls
>
> This is purge to 1st Varnish server only. Give another purge request to
> 2nd Varnish server giving it's port number.
>
> On Mon 9 Dec, 2019, 4:45 PM hamidreza hosseini, <hrhosseini at hotmail.com>
> wrote:
>
> Yes, surely i purge both servers
> This is my purge request:
>
> $curl -v -k -X PURGE http://localhost/Naserfeiz.mp4
>
> * Trying ::1...
> * Connected to localhost (::1) port 80 (#0)
> > PURGE /Naserfeiz.mp4 HTTP/1.1
> > Host: localhost
> > User-Agent: curl/7.47.0
> > Accept: */*
> >
> < HTTP/1.1 200 Purged
> < Date: Mon, 09 Dec 2019 11:12:54 GMT
> < Server: Varnish
> < X-Varnish: 98306
> < Content-Type: text/html; charset=utf-8
> < Retry-After: 5
> < Content-Length: 240
> < Accept-Ranges: bytes
> < Connection: keep-alive
> <
> <!DOCTYPE html>
> <html>
> <head>
> <title>200 Purged</title>
> </head>
> <body>
> <h1>Error 200 Purged</h1>
> <p>Purged</p>
> <h3>Guru Meditation:</h3>
> <p>XID: 98306</p>
> <hr>
> <p>Varnish cache server</p>
> </body>
> </html>
> * Connection #0 to host localhost left intact
>
>
> And this is varnishlog resault:
>
> * << BeReq >> 32771
> - Begin bereq 32770 fetch
> - Timestamp Start: 1575889894.491843 0.000000 0.000000
> - BereqMethod GET
> - BereqURL /
> - BereqProtocol HTTP/1.1
> - BereqHeader Host: 192.168.200.13
> - BereqHeader User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64;
> rv:70.0) Gecko/20100101 Firefox/70.0
> - BereqHeader Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> - BereqHeader Accept-Language: en-US,en;q=0.5
> - BereqHeader Upgrade-Insecure-Requests: 1
> - BereqHeader X-Forwarded-For: 172.16.1.3
> - BereqHeader Accept-Encoding: gzip
> - BereqHeader X-Varnish: 32771
> - VCL_call BACKEND_FETCH
> - VCL_return fetch
> - BackendOpen 18 boot.varnish_malloc_01 192.168.200.12 8080
> 192.168.200.13 14088
> - Timestamp Bereq: 1575889894.492444 0.000601 0.000601
> - Timestamp Beresp: 1575889894.493276 0.001433 0.000832
> - BerespProtocol HTTP/1.1
> - BerespStatus 200
> - BerespReason OK
> - BerespHeader Server: nginx
> - BerespHeader Date: Mon, 09 Dec 2019 10:00:29 GMT
> - BerespHeader Content-Type: text/html
> - BerespHeader Content-Length: 340
> - BerespHeader Last-Modified: Tue, 12 Nov 2019 13:01:40 GMT
> - BerespHeader ETag: "5dcaad34-154"
> - BerespHeader X-Varnish: 8 6
> - BerespHeader Age: 4270
> - BerespHeader Via: 1.1 varnish-v4
> - BerespHeader Accept-Ranges: bytes
> - BerespHeader Connection: keep-alive
> - TTL RFC 120 10 -1 1575889894 1575885624 1575885629 0 0
> - VCL_call BACKEND_RESPONSE
> - TTL VCL 177070 10 0 1575885624
> - VCL_return deliver
> - Storage malloc s0
> - ObjProtocol HTTP/1.1
> - ObjStatus 200
> - ObjReason OK
> - ObjHeader Server: nginx
> - ObjHeader Date: Mon, 09 Dec 2019 10:00:29 GMT
> - ObjHeader Content-Type: text/html
> - ObjHeader Content-Length: 340
> - ObjHeader Last-Modified: Tue, 12 Nov 2019 13:01:40 GMT
> - ObjHeader ETag: "5dcaad34-154"
> - ObjHeader X-Varnish: 8 6
> - ObjHeader Via: 1.1 varnish-v4
> - Fetch_Body 3 length stream
> - BackendReuse 18 boot.varnish_malloc_01
> - Timestamp BerespBody: 1575889894.493474 0.001631 0.000198
> - Length 340
> - BereqAcct 336 0 336 279 340 619
> - End
>
> * << Request >> 32770
> - Begin req 32769 rxreq
> - Timestamp Start: 1575889894.491659 0.000000 0.000000
> - Timestamp Req: 1575889894.491659 0.000000 0.000000
> - ReqStart 172.16.1.3 42502
> - ReqMethod GET
> - ReqURL /
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.13
> - ReqHeader User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64;
> rv:70.0) Gecko/20100101 Firefox/70.0
> - ReqHeader Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> - ReqHeader Accept-Language: en-US,en;q=0.5
> - ReqHeader Accept-Encoding: gzip, deflate
> - ReqHeader Connection: keep-alive
> - ReqHeader Cookie: _ga=GA1.1.1476719154.1572941620;
> _gid=GA1.1.117112441.1575877103
> - ReqHeader Upgrade-Insecure-Requests: 1
> - ReqHeader If-Modified-Since: Tue, 12 Nov 2019 13:01:40 GMT
> - ReqHeader If-None-Match: "5dcaad34-154"
> - ReqHeader Cache-Control: max-age=0
> - ReqHeader X-Forwarded-For: 172.16.1.3
> - VCL_call RECV
> - ReqUnset Cookie: _ga=GA1.1.1476719154.1572941620;
> _gid=GA1.1.117112441.1575877103
> - VCL_return hash
> - ReqUnset Accept-Encoding: gzip, deflate
> - ReqHeader Accept-Encoding: gzip
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call MISS
> - VCL_return fetch
> - Link bereq 32771 fetch
> - Timestamp Fetch: 1575889894.493501 0.001842 0.001842
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespHeader Server: nginx
> - RespHeader Date: Mon, 09 Dec 2019 10:00:29 GMT
> - RespHeader Content-Type: text/html
> - RespHeader Content-Length: 340
> - RespHeader Last-Modified: Tue, 12 Nov 2019 13:01:40 GMT
> - RespHeader ETag: "5dcaad34-154"
> - RespHeader X-Varnish: 8 6
> - RespHeader Via: 1.1 varnish-v4
> - RespHeader X-Varnish: 32770
> - RespHeader Age: 4270
> - RespHeader Via: 1.1 varnish-v4
> - VCL_call DELIVER
> - VCL_return deliver
> - Timestamp Process: 1575889894.493541 0.001881 0.000039
> - RespProtocol HTTP/1.1
> - RespStatus 304
> - RespReason Not Modified
> - RespReason Not Modified
> - RespUnset Content-Length: 340
> - Debug "RES_MODE 0"
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1575889894.493599 0.001940 0.000058
> - ReqAcct 503 0 503 285 0 285
> - End
>
> * << Request >> 65538
> - Begin req 65537 rxreq
> - Timestamp Start: 1575889962.124164 0.000000 0.000000
> - Timestamp Req: 1575889962.124164 0.000000 0.000000
> - ReqStart ::1 62064
> - ReqMethod PURGE
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: localhost
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader X-Forwarded-For: ::1
> - VCL_call RECV
> - VCL_acl MATCH purge "localhost"
> - VCL_return purge
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call PURGE
> - VCL_return synth
> - Timestamp Process: 1575889962.124300 0.000136 0.000136
> - RespHeader Date: Mon, 09 Dec 2019 11:12:42 GMT
> - RespHeader Server: Varnish
> - RespHeader X-Varnish: 65538
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespReason Purged
> - VCL_call SYNTH
> - RespHeader Content-Type: text/html; charset=utf-8
> - RespHeader Retry-After: 5
> - VCL_return deliver
> - RespHeader Content-Length: 240
> - Storage malloc Transient
> - RespHeader Accept-Ranges: bytes
> - Debug "RES_MODE 2"
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1575889962.124593 0.000428 0.000292
> - ReqAcct 88 0 88 218 240 458
> - End
>
> * << Session >> 65537
> - Begin sess 0 HTTP/1
> - SessOpen ::1 62064 :80 ::1 80 1575889962.124081 13
> - Link req 65538 rxreq
> - SessClose REM_CLOSE 0.001
> - End
>
> * << Request >> 98306
> - Begin req 98305 rxreq
> - Timestamp Start: 1575889974.116017 0.000000 0.000000
> - Timestamp Req: 1575889974.116017 0.000000 0.000000
> - ReqStart ::1 62066
> - ReqMethod PURGE
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: localhost
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader X-Forwarded-For: ::1
> - VCL_call RECV
> - VCL_acl MATCH purge "localhost"
> - VCL_return purge
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call PURGE
> - VCL_return synth
> - Timestamp Process: 1575889974.116128 0.000111 0.000111
> - RespHeader Date: Mon, 09 Dec 2019 11:12:54 GMT
> - RespHeader Server: Varnish
> - RespHeader X-Varnish: 98306
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespReason Purged
> - VCL_call SYNTH
> - RespHeader Content-Type: text/html; charset=utf-8
> - RespHeader Retry-After: 5
> - VCL_return deliver
> - RespHeader Content-Length: 240
> - Storage malloc Transient
> - RespHeader Accept-Ranges: bytes
> - Debug "RES_MODE 2"
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1575889974.116288 0.000271 0.000160
> - ReqAcct 88 0 88 218 240 458
> - End
>
> * << Session >> 98305
> - Begin sess 0 HTTP/1
> - SessOpen ::1 62066 :80 ::1 80 1575889974.115935 15
> - Link req 98306 rxreq
> - SessClose REM_CLOSE 0.002
> - End
>
> * << Session >> 32769
> - Begin sess 0 HTTP/1
> - SessOpen 172.16.1.3 42502 :80 192.168.200.13 80
> 1575889894.491572 14
> - Link req 32770 rxreq
> - Link req 32772 rxreq
> - VSL timeout
> - End synth
>
> * << Request >> 32772
> - Begin req 32769 rxreq
> - Timestamp Start: 1575889894.845556 0.000000 0.000000
> - Timestamp Req: 1575889894.845556 0.000000 0.000000
> - ReqStart 172.16.1.3 42502
> - ReqMethod GET
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.13
> - ReqHeader User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64;
> rv:70.0) Gecko/20100101 Firefox/70.0
> - ReqHeader Accept:
> video/webm,video/ogg,video/*;q=0.9,application/ogg;q=0.7,audio/*;q=0.6,*/*;q=0.5
> - ReqHeader Accept-Language: en-US,en;q=0.5
> - ReqHeader Range: bytes=205560-
> - ReqHeader Connection: keep-alive
> - ReqHeader Referer: http://192.168.200.13/
> - ReqHeader Cookie: _ga=GA1.1.1476719154.1572941620;
> _gid=GA1.1.117112441.1575877103
> - ReqHeader X-Forwarded-For: 172.16.1.3
> - VCL_call RECV
> - ReqUnset Cookie: _ga=GA1.1.1476719154.1572941620;
> _gid=GA1.1.117112441.1575877103
> - VCL_return hash
> - VCL_call HASH
> - VCL_return lookup
> - Hit 3
> - VCL_call HIT
> - VCL_return deliver
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespHeader Server: nginx
> - RespHeader Date: Mon, 09 Dec 2019 11:06:51 GMT
> - RespHeader Content-Type: text/plain
> - RespHeader Content-Length: 23521499
> - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> - RespHeader ETag: "3a1794b0-166e8db"
> - RespHeader X-Varnish: 32770
> - RespHeader Via: 1.1 varnish-v4
> - RespHeader X-Varnish: 32772 3
> - RespHeader Age: 288
> - RespHeader Via: 1.1 varnish-v4
> - VCL_call DELIVER
> - VCL_return deliver
> - Timestamp Process: 1575889894.845683 0.000127 0.000127
> - RespHeader Accept-Ranges: bytes
> - RespHeader Content-Range: bytes 205560-23521498/23521499
> - RespProtocol HTTP/1.1
> - RespStatus 206
> - RespReason Partial Content
> - RespReason Partial Content
> - RespUnset Content-Length: 23521499
> - RespHeader Content-Length: 23315939
> - Debug "RES_MODE 2"
> - RespHeader Connection: keep-alive
> - Debug "Hit idle send timeout, wrote = 10899908/23316330;
> retrying"
> - Debug "Hit idle send timeout, wrote = 258432/12416422;
> retrying"
> - Debug "Write error, retval = -1, len = 12157990, errno =
> Resource temporarily unavailable"
> - Timestamp Resp: 1575890074.846999 180.001443 180.001316
> - ReqAcct 419 0 419 391 23315939 23316330
> - End
>
> ##########################
> ------------------------------
> *From:* Guillaume Quintard <guillaume at varnish-software.com>
> *Sent:* Monday, December 9, 2019 11:57 AM
> *To:* hamidreza hosseini <hrhosseini at hotmail.com>
> *Cc:* varnish-misc at varnish-cache.org <varnish-misc at varnish-cache.org>
> *Subject:* Re: Varnish doesnt PURGE urls
>
> Are you sure that you purge both Varnish layers (file and RAM)?
>
> can you pastebin a varnishlog of the purge and of the subsequent request?
>
> --
> Guillaume Quintard
>
>
> On Mon, Dec 9, 2019 at 2:55 AM hamidreza hosseini <hrhosseini at hotmail.com>
> wrote:
>
> because its steel in cach and i can download and see th urls that i purged
> before!
> its Interesting that it return 200 code from varnish!!!
>
> Best regards.
> ------------------------------
> *From:* Guillaume Quintard <guillaume at varnish-software.com>
> *Sent:* Monday, December 9, 2019 11:48 AM
> *To:* hamidreza hosseini <hrhosseini at hotmail.com>
> *Cc:* varnish-misc at varnish-cache.org <varnish-misc at varnish-cache.org>
> *Subject:* Re: Varnish doesnt PURGE urls
>
> Hi,
>
> How do you tell that it's not purging?
>
> Cheers,
>
> --
> Guillaume Quintard
>
>
> On Mon, Dec 9, 2019 at 2:45 AM hamidreza hosseini <hrhosseini at hotmail.com>
> wrote:
>
> Hi,
>
> I've two varnish, that one of them use ram as backend and another file as
> backend, requests first come to varnish(ram) and then go to another server
> in varnish(file) and then go to nginx, this is my diagram:
>
>
> Req=====> [varnish ram] =======> [varnish file] =====> [Nginx]
>
> And following file is my vcl configuration file that i'm using,
> but when i want to purge a url it doesnt work and it steel remains in
> cache,
> Should i chage my config file?
> What's wrong with varnish?
>
> Vcl config file:
>
> ### Naming note: use [0-9a-zA-Z_] only. no -. Or use camelCase ;-)
> vcl 4.0;
>
> import directors;
>
>
> backend varnish_malloc_01 { .host = "192.168.200.12"; .port = "8080"; }
>
>
> sub vcl_init {
> new varnish_cluster = directors.round_robin();
> varnish_cluster.add_backend(varnish_malloc_01);
>
> }
>
>
> acl purge {
> "localhost";
> "192.168.200.0"/24;
> ;
> }
>
>
> sub vcl_recv {
>
> sub vcl_recv {
> #set req.backend_hint = varnish_cluster.backend();
> unset req.http.Cookie;
> if (req.method == "PURGE") {
> ### TODO: also purge from the backend
> if (!client.ip ~ purge) {
> return(synth(405,"Not allowed."));
> }
> return (purge);
> }
> }
>
> sub vcl_backend_response {
> set beresp.ttl=48h;
> # Don't cache 404 responses
> if ( beresp.status == 404 ) {
> set beresp.ttl = 120s;
> set beresp.uncacheable = true;
> return (deliver);
> }
> }
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20191209/d014e384/attachment-0001.html>
More information about the varnish-misc
mailing list