Varnish doesnt PURGE urls

hamidreza hosseini hrhosseini at hotmail.com
Mon Dec 9 11:24:16 UTC 2019


This is purge to another server:

$curl -v -k -X PURGE http://localhost:8080/Naserfeiz.mp4

*   Trying ::1...
* Connected to localhost (::1) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 Purged
< Date: Mon, 09 Dec 2019 11:20:52 GMT
< Server: Varnish
< X-Varnish: 10
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 237
< Accept-Ranges: bytes
< Connection: keep-alive
<
<!DOCTYPE html>
<html>
  <head>
    <title>200 Purged</title>
  </head>
  <body>
    <h1>Error 200 Purged</h1>
    <p>Purged</p>
    <h3>Guru Meditation:</h3>
    <p>XID: 10</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>
* Connection #0 to host localhost left intact

And varnishlog resault:

$ sudo varnishlog

*   << Request  >> 32773
-   Begin          req 32772 rxreq
-   Timestamp      Start: 1575890593.662132 0.000000 0.000000
-   Timestamp      Req: 1575890593.662132 0.000000 0.000000
-   ReqStart       ::1 63184
-   ReqMethod      PURGE
-   ReqURL         /Naserfeiz.mp4
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Host: localhost:8080
-   ReqHeader      User-Agent: curl/7.47.0
-   ReqHeader      Accept: */*
-   ReqHeader      X-Forwarded-For: ::1
-   VCL_call       RECV
-   VCL_acl        MATCH purge "localhost"
-   VCL_return     purge
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       PURGE
-   VCL_return     synth
-   Timestamp      Process: 1575890593.662268 0.000135 0.000135
-   RespHeader     Date: Mon, 09 Dec 2019 11:23:13 GMT
-   RespHeader     Server: Varnish
-   RespHeader     X-Varnish: 32773
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespReason     Purged
-   VCL_call       SYNTH
-   RespHeader     Content-Type: text/html; charset=utf-8
-   RespHeader     Retry-After: 5
-   VCL_return     deliver
-   RespHeader     Content-Length: 240
-   Storage        malloc Transient
-   RespHeader     Accept-Ranges: bytes
-   Debug          "RES_MODE 2"
-   RespHeader     Connection: keep-alive
-   Timestamp      Resp: 1575890593.662484 0.000352 0.000216
-   ReqAcct        93 0 93 218 240 458
-   End

*   << Session  >> 32772
-   Begin          sess 0 HTTP/1
-   SessOpen       ::1 63184 :8080 ::1 8080 1575890593.662044 16
-   Link           req 32773 rxreq
-   SessClose      REM_CLOSE 0.001
-   End

*   << Request  >> 12
-   Begin          req 11 rxreq
-   Timestamp      Start: 1575890600.799371 0.000000 0.000000
-   Timestamp      Req: 1575890600.799371 0.000000 0.000000
-   ReqStart       ::1 63186
-   ReqMethod      PURGE
-   ReqURL         /Naserfeiz.mp4
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Host: localhost:8080
-   ReqHeader      User-Agent: curl/7.47.0
-   ReqHeader      Accept: */*
-   ReqHeader      X-Forwarded-For: ::1
-   VCL_call       RECV
-   VCL_acl        MATCH purge "localhost"
-   VCL_return     purge
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       PURGE
-   VCL_return     synth
-   Timestamp      Process: 1575890600.799463 0.000092 0.000092
-   RespHeader     Date: Mon, 09 Dec 2019 11:23:20 GMT
-   RespHeader     Server: Varnish
-   RespHeader     X-Varnish: 12
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespReason     Purged
-   VCL_call       SYNTH
-   RespHeader     Content-Type: text/html; charset=utf-8
-   RespHeader     Retry-After: 5
-   VCL_return     deliver
-   RespHeader     Content-Length: 237
-   Storage        malloc Transient
-   RespHeader     Accept-Ranges: bytes
-   Debug          "RES_MODE 2"
-   RespHeader     Connection: keep-alive
-   Timestamp      Resp: 1575890600.799611 0.000240 0.000147
-   ReqAcct        93 0 93 215 237 452
-   End

*   << Session  >> 11
-   Begin          sess 0 HTTP/1
-   SessOpen       ::1 63186 :8080 ::1 8080 1575890600.799261 16
-   Link           req 12 rxreq
-   SessClose      REM_CLOSE 0.001
-   End

________________________________
From: shafeeque aslam <shafeequeaslam at gmail.com>
Sent: Monday, December 9, 2019 3:19 AM
To: hamidreza hosseini <hrhosseini at hotmail.com>
Cc: Guillaume Quintard <guillaume at varnish-software.com>; varnish-misc at varnish-cache.org <varnish-misc at varnish-cache.org>
Subject: Re: Varnish doesnt PURGE urls

This is purge to 1st Varnish server only. Give another purge request to 2nd Varnish server giving it's port number.

On Mon 9 Dec, 2019, 4:45 PM hamidreza hosseini, <hrhosseini at hotmail.com<mailto:hrhosseini at hotmail.com>> wrote:
Yes, surely i purge both servers
This is my purge request:

$curl -v -k -X PURGE http://localhost/Naserfeiz.mp4

*   Trying ::1...
* Connected to localhost (::1) port 80 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: localhost
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 Purged
< Date: Mon, 09 Dec 2019 11:12:54 GMT
< Server: Varnish
< X-Varnish: 98306
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
<!DOCTYPE html>
<html>
  <head>
    <title>200 Purged</title>
  </head>
  <body>
    <h1>Error 200 Purged</h1>
    <p>Purged</p>
    <h3>Guru Meditation:</h3>
    <p>XID: 98306</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>
* Connection #0 to host localhost left intact


And this is varnishlog resault:

*   << BeReq    >> 32771
-   Begin          bereq 32770 fetch
-   Timestamp      Start: 1575889894.491843 0.000000 0.000000
-   BereqMethod    GET
-   BereqURL       /
-   BereqProtocol  HTTP/1.1
-   BereqHeader    Host: 192.168.200.13
-   BereqHeader    User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0
-   BereqHeader    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
-   BereqHeader    Accept-Language: en-US,en;q=0.5
-   BereqHeader    Upgrade-Insecure-Requests: 1
-   BereqHeader    X-Forwarded-For: 172.16.1.3
-   BereqHeader    Accept-Encoding: gzip
-   BereqHeader    X-Varnish: 32771
-   VCL_call       BACKEND_FETCH
-   VCL_return     fetch
-   BackendOpen    18 boot.varnish_malloc_01 192.168.200.12 8080 192.168.200.13 14088
-   Timestamp      Bereq: 1575889894.492444 0.000601 0.000601
-   Timestamp      Beresp: 1575889894.493276 0.001433 0.000832
-   BerespProtocol HTTP/1.1
-   BerespStatus   200
-   BerespReason   OK
-   BerespHeader   Server: nginx
-   BerespHeader   Date: Mon, 09 Dec 2019 10:00:29 GMT
-   BerespHeader   Content-Type: text/html
-   BerespHeader   Content-Length: 340
-   BerespHeader   Last-Modified: Tue, 12 Nov 2019 13:01:40 GMT
-   BerespHeader   ETag: "5dcaad34-154"
-   BerespHeader   X-Varnish: 8 6
-   BerespHeader   Age: 4270
-   BerespHeader   Via: 1.1 varnish-v4
-   BerespHeader   Accept-Ranges: bytes
-   BerespHeader   Connection: keep-alive
-   TTL            RFC 120 10 -1 1575889894 1575885624 1575885629 0 0
-   VCL_call       BACKEND_RESPONSE
-   TTL            VCL 177070 10 0 1575885624
-   VCL_return     deliver
-   Storage        malloc s0
-   ObjProtocol    HTTP/1.1
-   ObjStatus      200
-   ObjReason      OK
-   ObjHeader      Server: nginx
-   ObjHeader      Date: Mon, 09 Dec 2019 10:00:29 GMT
-   ObjHeader      Content-Type: text/html
-   ObjHeader      Content-Length: 340
-   ObjHeader      Last-Modified: Tue, 12 Nov 2019 13:01:40 GMT
-   ObjHeader      ETag: "5dcaad34-154"
-   ObjHeader      X-Varnish: 8 6
-   ObjHeader      Via: 1.1 varnish-v4
-   Fetch_Body     3 length stream
-   BackendReuse   18 boot.varnish_malloc_01
-   Timestamp      BerespBody: 1575889894.493474 0.001631 0.000198
-   Length         340
-   BereqAcct      336 0 336 279 340 619
-   End

*   << Request  >> 32770
-   Begin          req 32769 rxreq
-   Timestamp      Start: 1575889894.491659 0.000000 0.000000
-   Timestamp      Req: 1575889894.491659 0.000000 0.000000
-   ReqStart       172.16.1.3 42502
-   ReqMethod      GET
-   ReqURL         /
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Host: 192.168.200.13
-   ReqHeader      User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0
-   ReqHeader      Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
-   ReqHeader      Accept-Language: en-US,en;q=0.5
-   ReqHeader      Accept-Encoding: gzip, deflate
-   ReqHeader      Connection: keep-alive
-   ReqHeader      Cookie: _ga=GA1.1.1476719154.1572941620; _gid=GA1.1.117112441.1575877103
-   ReqHeader      Upgrade-Insecure-Requests: 1
-   ReqHeader      If-Modified-Since: Tue, 12 Nov 2019 13:01:40 GMT
-   ReqHeader      If-None-Match: "5dcaad34-154"
-   ReqHeader      Cache-Control: max-age=0
-   ReqHeader      X-Forwarded-For: 172.16.1.3
-   VCL_call       RECV
-   ReqUnset       Cookie: _ga=GA1.1.1476719154.1572941620; _gid=GA1.1.117112441.1575877103
-   VCL_return     hash
-   ReqUnset       Accept-Encoding: gzip, deflate
-   ReqHeader      Accept-Encoding: gzip
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       MISS
-   VCL_return     fetch
-   Link           bereq 32771 fetch
-   Timestamp      Fetch: 1575889894.493501 0.001842 0.001842
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespHeader     Server: nginx
-   RespHeader     Date: Mon, 09 Dec 2019 10:00:29 GMT
-   RespHeader     Content-Type: text/html
-   RespHeader     Content-Length: 340
-   RespHeader     Last-Modified: Tue, 12 Nov 2019 13:01:40 GMT
-   RespHeader     ETag: "5dcaad34-154"
-   RespHeader     X-Varnish: 8 6
-   RespHeader     Via: 1.1 varnish-v4
-   RespHeader     X-Varnish: 32770
-   RespHeader     Age: 4270
-   RespHeader     Via: 1.1 varnish-v4
-   VCL_call       DELIVER
-   VCL_return     deliver
-   Timestamp      Process: 1575889894.493541 0.001881 0.000039
-   RespProtocol   HTTP/1.1
-   RespStatus     304
-   RespReason     Not Modified
-   RespReason     Not Modified
-   RespUnset      Content-Length: 340
-   Debug          "RES_MODE 0"
-   RespHeader     Connection: keep-alive
-   Timestamp      Resp: 1575889894.493599 0.001940 0.000058
-   ReqAcct        503 0 503 285 0 285
-   End

*   << Request  >> 65538
-   Begin          req 65537 rxreq
-   Timestamp      Start: 1575889962.124164 0.000000 0.000000
-   Timestamp      Req: 1575889962.124164 0.000000 0.000000
-   ReqStart       ::1 62064
-   ReqMethod      PURGE
-   ReqURL         /Naserfeiz.mp4
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Host: localhost
-   ReqHeader      User-Agent: curl/7.47.0
-   ReqHeader      Accept: */*
-   ReqHeader      X-Forwarded-For: ::1
-   VCL_call       RECV
-   VCL_acl        MATCH purge "localhost"
-   VCL_return     purge
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       PURGE
-   VCL_return     synth
-   Timestamp      Process: 1575889962.124300 0.000136 0.000136
-   RespHeader     Date: Mon, 09 Dec 2019 11:12:42 GMT
-   RespHeader     Server: Varnish
-   RespHeader     X-Varnish: 65538
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespReason     Purged
-   VCL_call       SYNTH
-   RespHeader     Content-Type: text/html; charset=utf-8
-   RespHeader     Retry-After: 5
-   VCL_return     deliver
-   RespHeader     Content-Length: 240
-   Storage        malloc Transient
-   RespHeader     Accept-Ranges: bytes
-   Debug          "RES_MODE 2"
-   RespHeader     Connection: keep-alive
-   Timestamp      Resp: 1575889962.124593 0.000428 0.000292
-   ReqAcct        88 0 88 218 240 458
-   End

*   << Session  >> 65537
-   Begin          sess 0 HTTP/1
-   SessOpen       ::1 62064 :80 ::1 80 1575889962.124081 13
-   Link           req 65538 rxreq
-   SessClose      REM_CLOSE 0.001
-   End

*   << Request  >> 98306
-   Begin          req 98305 rxreq
-   Timestamp      Start: 1575889974.116017 0.000000 0.000000
-   Timestamp      Req: 1575889974.116017 0.000000 0.000000
-   ReqStart       ::1 62066
-   ReqMethod      PURGE
-   ReqURL         /Naserfeiz.mp4
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Host: localhost
-   ReqHeader      User-Agent: curl/7.47.0
-   ReqHeader      Accept: */*
-   ReqHeader      X-Forwarded-For: ::1
-   VCL_call       RECV
-   VCL_acl        MATCH purge "localhost"
-   VCL_return     purge
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       PURGE
-   VCL_return     synth
-   Timestamp      Process: 1575889974.116128 0.000111 0.000111
-   RespHeader     Date: Mon, 09 Dec 2019 11:12:54 GMT
-   RespHeader     Server: Varnish
-   RespHeader     X-Varnish: 98306
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespReason     Purged
-   VCL_call       SYNTH
-   RespHeader     Content-Type: text/html; charset=utf-8
-   RespHeader     Retry-After: 5
-   VCL_return     deliver
-   RespHeader     Content-Length: 240
-   Storage        malloc Transient
-   RespHeader     Accept-Ranges: bytes
-   Debug          "RES_MODE 2"
-   RespHeader     Connection: keep-alive
-   Timestamp      Resp: 1575889974.116288 0.000271 0.000160
-   ReqAcct        88 0 88 218 240 458
-   End

*   << Session  >> 98305
-   Begin          sess 0 HTTP/1
-   SessOpen       ::1 62066 :80 ::1 80 1575889974.115935 15
-   Link           req 98306 rxreq
-   SessClose      REM_CLOSE 0.002
-   End

*   << Session  >> 32769
-   Begin          sess 0 HTTP/1
-   SessOpen       172.16.1.3 42502 :80 192.168.200.13 80 1575889894.491572 14
-   Link           req 32770 rxreq
-   Link           req 32772 rxreq
-   VSL            timeout
-   End            synth

*   << Request  >> 32772
-   Begin          req 32769 rxreq
-   Timestamp      Start: 1575889894.845556 0.000000 0.000000
-   Timestamp      Req: 1575889894.845556 0.000000 0.000000
-   ReqStart       172.16.1.3 42502
-   ReqMethod      GET
-   ReqURL         /Naserfeiz.mp4
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Host: 192.168.200.13
-   ReqHeader      User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0
-   ReqHeader      Accept: video/webm,video/ogg,video/*;q=0.9,application/ogg;q=0.7,audio/*;q=0.6,*/*;q=0.5
-   ReqHeader      Accept-Language: en-US,en;q=0.5
-   ReqHeader      Range: bytes=205560-
-   ReqHeader      Connection: keep-alive
-   ReqHeader      Referer: http://192.168.200.13/
-   ReqHeader      Cookie: _ga=GA1.1.1476719154.1572941620; _gid=GA1.1.117112441.1575877103
-   ReqHeader      X-Forwarded-For: 172.16.1.3
-   VCL_call       RECV
-   ReqUnset       Cookie: _ga=GA1.1.1476719154.1572941620; _gid=GA1.1.117112441.1575877103
-   VCL_return     hash
-   VCL_call       HASH
-   VCL_return     lookup
-   Hit            3
-   VCL_call       HIT
-   VCL_return     deliver
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespHeader     Server: nginx
-   RespHeader     Date: Mon, 09 Dec 2019 11:06:51 GMT
-   RespHeader     Content-Type: text/plain
-   RespHeader     Content-Length: 23521499
-   RespHeader     Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-   RespHeader     ETag: "3a1794b0-166e8db"
-   RespHeader     X-Varnish: 32770
-   RespHeader     Via: 1.1 varnish-v4
-   RespHeader     X-Varnish: 32772 3
-   RespHeader     Age: 288
-   RespHeader     Via: 1.1 varnish-v4
-   VCL_call       DELIVER
-   VCL_return     deliver
-   Timestamp      Process: 1575889894.845683 0.000127 0.000127
-   RespHeader     Accept-Ranges: bytes
-   RespHeader     Content-Range: bytes 205560-23521498/23521499
-   RespProtocol   HTTP/1.1
-   RespStatus     206
-   RespReason     Partial Content
-   RespReason     Partial Content
-   RespUnset      Content-Length: 23521499
-   RespHeader     Content-Length: 23315939
-   Debug          "RES_MODE 2"
-   RespHeader     Connection: keep-alive
-   Debug          "Hit idle send timeout, wrote = 10899908/23316330; retrying"
-   Debug          "Hit idle send timeout, wrote = 258432/12416422; retrying"
-   Debug          "Write error, retval = -1, len = 12157990, errno = Resource temporarily unavailable"
-   Timestamp      Resp: 1575890074.846999 180.001443 180.001316
-   ReqAcct        419 0 419 391 23315939 23316330
-   End

##########################
________________________________
From: Guillaume Quintard <guillaume at varnish-software.com<mailto:guillaume at varnish-software.com>>
Sent: Monday, December 9, 2019 11:57 AM
To: hamidreza hosseini <hrhosseini at hotmail.com<mailto:hrhosseini at hotmail.com>>
Cc: varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org> <varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>>
Subject: Re: Varnish doesnt PURGE urls

Are you sure that you purge both Varnish layers (file and RAM)?

can you pastebin a varnishlog of the purge and of the subsequent request?

--
Guillaume Quintard


On Mon, Dec 9, 2019 at 2:55 AM hamidreza hosseini <hrhosseini at hotmail.com<mailto:hrhosseini at hotmail.com>> wrote:
because its steel in cach and i can download and see th urls that i purged before!
its Interesting that it return 200 code from varnish!!!

Best regards.
________________________________
From: Guillaume Quintard <guillaume at varnish-software.com<mailto:guillaume at varnish-software.com>>
Sent: Monday, December 9, 2019 11:48 AM
To: hamidreza hosseini <hrhosseini at hotmail.com<mailto:hrhosseini at hotmail.com>>
Cc: varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org> <varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>>
Subject: Re: Varnish doesnt PURGE urls

Hi,

How do you tell that it's not purging?

Cheers,

--
Guillaume Quintard


On Mon, Dec 9, 2019 at 2:45 AM hamidreza hosseini <hrhosseini at hotmail.com<mailto:hrhosseini at hotmail.com>> wrote:
Hi,

I've two varnish, that one of them use ram as backend and another file as backend, requests first come to varnish(ram) and then go to another server in varnish(file) and then go to nginx, this is my diagram:

Req=====> [varnish ram] =======> [varnish file] =====> [Nginx]

And following file is my vcl configuration file that i'm using,
but when i want to purge a url it doesnt work and it steel remains in cache,
Should i chage my config file?
What's wrong with varnish?

Vcl config file:

### Naming note: use [0-9a-zA-Z_] only. no -. Or use camelCase ;-)
vcl 4.0;

import directors;


backend varnish_malloc_01 { .host = "192.168.200.12"; .port = "8080"; }


sub vcl_init {
    new varnish_cluster = directors.round_robin();
    varnish_cluster.add_backend(varnish_malloc_01);

}


acl purge {
    "localhost";
    "192.168.200.0"/24;
;
}


sub vcl_recv {

sub vcl_recv {
    #set req.backend_hint = varnish_cluster.backend();
    unset req.http.Cookie;
    if (req.method == "PURGE") {
        ### TODO: also purge from the backend
        if (!client.ip ~ purge) {
            return(synth(405,"Not allowed."));
        }
        return (purge);
    }
}

sub vcl_backend_response {
    set beresp.ttl=48h;
    # Don't cache 404 responses
    if ( beresp.status == 404 ) {
        set beresp.ttl = 120s;
        set beresp.uncacheable = true;
        return (deliver);
    }
}


_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20191209/7e8b06cf/attachment-0001.html>


More information about the varnish-misc mailing list