[Varnish] #1753: Bakend flapping and High load

Varnish varnish-bugs at varnish-cache.org
Wed Jul 1 01:00:22 CEST 2015


#1753: Bakend flapping and High load
-----------------------+----------------------------------
 Reporter:  zaterio@…  |       Owner:
     Type:  defect     |      Status:  new
 Priority:  normal     |   Milestone:  Varnish 4.0 release
Component:  build      |     Version:  trunk
 Severity:  normal     |  Resolution:
 Keywords:             |
-----------------------+----------------------------------

Comment (by zaterio@…):

 {{{
 varnishd -V
 varnishd (varnish-trunk revision 7585d89)
 Copyright (c) 2006 Verdens Gang AS
 Copyright (c) 2006-2015 Varnish Software AS
 }}}

 backend flapping: no

 high load: no

 varnish stop answering to curl after 30 min to 2 hours uptime.

 No firewall rules.

 logging method:
 {{{
 varnishlog -g raw > file
 }}}

 VSL tag "Backend_health" always shows "Still healthy" for all probes, in
 all cases.

 Reversing to trunk e05ac94a893090707ea95bf16578b13388917165 not show this
 problem.

 In vcl I do not have any status 909 (backends only responds with standars
 status codes), however:

 {{{
 cat file|grep -w 909|grep Debug
         0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
          0 Debug          - "909 HTTP/1.1"
 }}}

 initially, SeessClose is primary by RX_TIMEOUT, when the problem begins to
 manifest, SessClose changes to RESP_CLOSE, and then only REM_CLOSE.


 {{{
 cat file|grep SessClose
   28181773 SessClose      c RX_TIMEOUT 8.288
   29230644 SessClose      c RX_TIMEOUT 10.043
   30017083 SessClose      c RX_TIMEOUT 5.969
   27427924 SessClose      c RX_TIMEOUT 11.128
   30115316 SessClose      c RX_TIMEOUT 5.018
   27034843 SessClose      c REM_CLOSE 0.058
   29361159 SessClose      c RX_TIMEOUT 6.682
   26018627 SessClose      c RX_TIMEOUT 6.862
   28051101 SessClose      c RX_TIMEOUT 5.007
   27886957 SessClose      c RX_TIMEOUT 10.863
   29689242 SessClose      c RX_TIMEOUT 5.008
   28051103 SessClose      c RX_TIMEOUT 5.023
   30213501 SessClose      c RX_TIMEOUT 5.009
   24314349 SessClose      c RX_TIMEOUT 5.840
   27034839 SessClose      c RX_TIMEOUT 5.457
   24839580 SessClose      c RX_TIMEOUT 7.157
   27721798 SessClose      c RX_TIMEOUT 5.009
   29426488 SessClose      c RX_TIMEOUT 9.512
   27460946 SessClose      c TX_PIPE 79.635
   28968293 SessClose      c RX_TIMEOUT 6.148
   29361165 SessClose      c RX_TIMEOUT 6.041
   27034847 SessClose      c RESP_CLOSE 0.527
   29689240 SessClose      c RX_TIMEOUT 6.073
   29361153 SessClose      c RX_TIMEOUT 8.122
   29329091 SessClose      c REM_CLOSE 1.721
   29591055 SessClose      c RX_TIMEOUT 5.974
   28542401 SessClose      c REM_CLOSE 15.868
   28116504 SessClose      c RX_TIMEOUT 9.612
   30017085 SessClose      c RX_TIMEOUT 7.173
   29754997 SessClose      c RX_TIMEOUT 12.243
   27034835 SessClose      c RX_TIMEOUT 7.454
   29787524 SessClose      c RX_TIMEOUT 8.181
   30017087 SessClose      c RX_TIMEOUT 5.001
   28509327 SessClose      c RX_TIMEOUT 7.706
   29230634 SessClose      c RX_TIMEOUT 23.206
   24839582 SessClose      c RX_TIMEOUT 6.868
   27886963 SessClose      c RX_TIMEOUT 5.024
   29361163 SessClose      c RX_TIMEOUT 7.618
   27526567 SessClose      c RX_TIMEOUT 5.010
   28181776 SessClose      c RX_TIMEOUT 5.026
   28509325 SessClose      c RX_TIMEOUT 8.446
   30311461 SessClose      c RX_TIMEOUT 9.973
   30343214 SessClose      c RX_TIMEOUT 6.021
   27721803 SessClose      c RX_TIMEOUT 6.663
   27721795 SessClose      c RX_TIMEOUT 7.225
   28574964 SessClose      c RX_TIMEOUT 7.488
   29951370 SessClose      c RX_TIMEOUT 8.841
   30311466 SessClose      c RX_TIMEOUT 7.357
   28116507 SessClose      c RX_TIMEOUT 6.894
   30115325 SessClose      c REM_CLOSE 0.057
   17368373 SessClose      c RX_TIMEOUT 9.951
   29689244 SessClose      c RX_TIMEOUT 7.652
   24314350 SessClose      c RX_TIMEOUT 8.642
   29525555 SessClose      c RX_TIMEOUT 5.764
   24839585 SessClose      c RX_TIMEOUT 6.207
   29525557 SessClose      c RX_TIMEOUT 5.008
   28051107 SessClose      c RX_TIMEOUT 8.053
   30148308 SessClose      c RX_TIMEOUT 5.618
   30246079 SessClose      c RX_TIMEOUT 5.009
   28935006 SessClose      c RX_TIMEOUT 271.592
   29787527 SessClose      c RX_TIMEOUT 7.326
   27721808 SessClose      c RX_TIMEOUT 5.879
   27721800 SessClose      c RX_TIMEOUT 8.358
   29034079 SessClose      c RESP_CLOSE 5.992
   28968297 SessClose      c RX_TIMEOUT 8.133
   27526546 SessClose      c RX_TIMEOUT 23.798
   29852821 SessClose      c RX_TIMEOUT 7.227
   27526565 SessClose      c RX_TIMEOUT 7.299
   29525560 SessClose      c RX_TIMEOUT 5.009
   30115328 SessClose      c REM_CLOSE 1.334
   29164565 SessClose      c RX_TIMEOUT 8.844
   25035834 SessClose      c RX_TIMEOUT 5.023
   30279078 SessClose      c RX_TIMEOUT 8.482
   29394129 SessClose      c RX_TIMEOUT 5.009
   30343210 SessClose      c RX_TIMEOUT 11.158
   25986792 SessClose      c RX_TIMEOUT 11.840
   28116509 SessClose      c RX_TIMEOUT 7.006
   27755577 SessClose      c RX_TIMEOUT 5.033
   28639985 SessClose      c RX_TIMEOUT 5.034
   30049545 SessClose      c RX_TIMEOUT 7.580
   27526570 SessClose      c RX_TIMEOUT 5.973
   30343217 SessClose      c RX_TIMEOUT 7.161
   30311468 SessClose      c RX_TIMEOUT 7.866
   28542411 SessClose      c RX_TIMEOUT 11.771
   28542414 SessClose      c RX_TIMEOUT 5.009
   29787531 SessClose      c RX_TIMEOUT 6.515
   30049547 SessClose      c RX_TIMEOUT 5.001
   29885736 SessClose      c RX_TIMEOUT 6.875
   30049548 SessClose      c RX_TIMEOUT 5.007
   27721806 SessClose      c RX_TIMEOUT 8.294
   28181779 SessClose      c RX_TIMEOUT 7.268
   25035837 SessClose      c RX_TIMEOUT 5.888
   17368376 SessClose      c RX_TIMEOUT 7.159
   29787529 SessClose      c RX_TIMEOUT 6.923
   29591058 SessClose      c RX_TIMEOUT 7.026
   30148311 SessClose      c RX_TIMEOUT 7.122
   28051109 SessClose      c RX_TIMEOUT 7.540
   17368382 SessClose      c RX_TIMEOUT 5.000
   30311471 SessClose      c RX_TIMEOUT 5.681
   28803460 SessClose      c RX_TIMEOUT 5.000
   28574967 SessClose      c RX_TIMEOUT 7.845
   27755574 SessClose      c RX_TIMEOUT 8.741
   17368385 SessClose      c RX_TIMEOUT 5.001
   29591060 SessClose      c RX_TIMEOUT 5.534
   27034841 SessClose      c RX_TIMEOUT 6.681
   29755000 SessClose      c RX_TIMEOUT 5.954
   29689246 SessClose      c RX_TIMEOUT 8.551
   28246672 SessClose      c RX_TIMEOUT 14.269
   29394131 SessClose      c RX_TIMEOUT 7.027
   27721810 SessClose      c RX_TIMEOUT 8.100
   29787532 SessClose      c RX_TIMEOUT 7.368
   29230655 SessClose      c RX_TIMEOUT 8.801
   27526575 SessClose      c RX_TIMEOUT 5.014
   26443810 SessClose      c RESP_CLOSE 0.006
   29197598 SessClose      c REM_CLOSE 120.276
   27328552 SessClose      c RESP_CLOSE 0.001
   26018636 SessClose      c RESP_CLOSE 7.402
   26148873 SessClose      c RESP_CLOSE 0.004
   30474243 SessClose      c RESP_CLOSE 0.004
   26148875 SessClose      c RESP_CLOSE 0.004
   29230508 SessClose      c REM_CLOSE 183.285
   24248396 SessClose      c RESP_CLOSE 0.005
   26836999 SessClose      c RESP_CLOSE 5.606
   25460789 SessClose      c RESP_CLOSE 5.261
   17760298 SessClose      c RESP_CLOSE 6.270
   19071084 SessClose      c RESP_CLOSE 5.795
   28869524 SessClose      c REM_CLOSE 120.014
   28968261 SessClose      c REM_CLOSE 122.986
   25100292 SessClose      c RESP_CLOSE 5.864
   27853914 SessClose      c REM_CLOSE 180.076
   23363627 SessClose      c RESP_CLOSE 0.001
   30572545 SessClose      c RESP_CLOSE 7.637
   17956935 SessClose      c REM_CLOSE 0.053
   22478889 SessClose      c RESP_CLOSE 4.894
   22511625 SessClose      c RESP_CLOSE 6.387
   21856270 SessClose      c REM_CLOSE 2.135
   20217896 SessClose      c RESP_CLOSE 5.854
   21692422 SessClose      c REM_CLOSE 4.879
   21299207 SessClose      c RESP_CLOSE 6.373
   19398706 SessClose      c RESP_CLOSE 5.090
   19791900 SessClose      c REM_CLOSE 1.515
   20086797 SessClose      c RESP_CLOSE 5.741
   16810059 SessClose      c REM_CLOSE 0.036
   18415647 SessClose      c RESP_CLOSE 5.386
   30932993 SessClose      c RESP_CLOSE 6.758
   17268746 SessClose      c RESP_CLOSE 6.856
   30965761 SessClose      c RESP_CLOSE 5.294
   16580613 SessClose      c RESP_CLOSE 5.987
   15007779 SessClose      c RESP_CLOSE 6.436
   31064067 SessClose      c RESP_CLOSE 6.832
   31096835 SessClose      c RESP_CLOSE 6.364
   13271061 SessClose      c RESP_CLOSE 6.068
   13172746 SessClose      c REM_CLOSE 0.025
   11239436 SessClose      c REM_CLOSE 0.024
   12910609 SessClose      c RESP_CLOSE 6.656
   12484614 SessClose      c RESP_CLOSE 5.623
   11534350 SessClose      c RESP_CLOSE 7.308
   31358979 SessClose      c RESP_CLOSE 5.770
    9175068 SessClose      c RESP_CLOSE 5.707
    7634962 SessClose      c RESP_CLOSE 6.553
    7634966 SessClose      c RESP_CLOSE 6.927
    8945682 SessClose      c RESP_CLOSE 6.848
    7340070 SessClose      c RESP_CLOSE 5.315
    6324241 SessClose      c RESP_CLOSE 4.749
    6225936 SessClose      c RESP_CLOSE 5.050
    4849677 SessClose      c REM_CLOSE 1.917
    3506225 SessClose      c RESP_CLOSE 6.466
    4522007 SessClose      c RESP_CLOSE 6.754
    3964957 SessClose      c RESP_CLOSE 5.354
    2031678 SessClose      c REM_CLOSE 0.025
    4128772 SessClose      c REM_CLOSE 0.026
    3637251 SessClose      c RESP_CLOSE 5.736
    3244046 SessClose      c RESP_CLOSE 6.622
    2326567 SessClose      c RESP_CLOSE 5.311
   31850497 SessClose      c REM_CLOSE 0.024
    2326569 SessClose      c REM_CLOSE 0.024
    2326571 SessClose      c RESP_CLOSE 6.405
    2129932 SessClose      c REM_CLOSE 1.393
    1671195 SessClose      c RESP_CLOSE 5.623
    1441809 SessClose      c RESP_CLOSE 5.513
     917535 SessClose      c REM_CLOSE 0.652
    1015817 SessClose      c RESP_CLOSE 5.739
     786434 SessClose      c REM_CLOSE 0.515
   32374785 SessClose      c REM_CLOSE 0.000
   32768001 SessClose      c REM_CLOSE 0.000
   32931845 SessClose      c REM_CLOSE 0.000
   33226753 SessClose      c REM_CLOSE 0.000
   33226754 SessClose      c REM_CLOSE 0.000
   33193985 SessClose      c REM_CLOSE 1.984
   33554433 SessClose      c REM_CLOSE 0.000
   33685507 SessClose      c REM_CLOSE 0.037
   34078721 SessClose      c REM_CLOSE 0.000
   34701313 SessClose      c REM_CLOSE 0.000
   34766849 SessClose      c REM_CLOSE 0.000
   34766850 SessClose      c REM_CLOSE 0.000
   34766851 SessClose      c REM_CLOSE 0.000
   35061763 SessClose      c REM_CLOSE 0.002
   35061769 SessClose      c REM_CLOSE 0.000
   35323909 SessClose      c REM_CLOSE 0.000
   35389441 SessClose      c REM_CLOSE 0.000
   35717123 SessClose      c REM_CLOSE 0.000
   35749891 SessClose      c REM_CLOSE 0.000
   35782659 SessClose      c REM_CLOSE 0.000
   35880961 SessClose      c REM_CLOSE 0.000
 }}}


 Another interesting log:

 {{{
 cat file.log | grep Debug|grep -v RES|grep -v Broken|grep -v retrying|grep
 -v "Connection reset by peer"|grep -v 909
   27526281 Debug          c "Write error, retval = -1, len = 140631, errno
 = Resource temporarily unavailable"
   29197599 Debug          c "Write error, retval = -1, len = 700846, errno
 = Resource temporarily unavailable"
   29820301 Debug          c "Write error, retval = -1, len = 1109783,
 errno = Resource temporarily unavailable"
   28869525 Debug          c "Write error, retval = -1, len = 152421, errno
 = Resource temporarily unavailable"
   28378768 Debug          c "Write error, retval = -1, len = 958763, errno
 = Resource temporarily unavailable"
   27853916 Debug          c "Write error, retval = -1, len = 549933, errno
 = Resource temporarily unavailable"
 }}}


 another example (after restart varnish):

 {{{
 cat file|grep SessClose
   13697065 SessClose      c RX_TIMEOUT 10.752
    9896076 SessClose      c RX_TIMEOUT 21.189
   15728645 SessClose      c RX_TIMEOUT 5.999
   15368203 SessClose      c RX_TIMEOUT 5.000
   10813550 SessClose      c RX_TIMEOUT 5.000
   14614548 SessClose      c RX_TIMEOUT 27.981
   10616975 SessClose      c RX_TIMEOUT 6.484
   11698293 SessClose      c RX_TIMEOUT 6.110
   13697071 SessClose      c RX_TIMEOUT 9.640
   14614569 SessClose      c RX_TIMEOUT 7.206
   12451907 SessClose      c RX_TIMEOUT 6.656
   10223758 SessClose      c REM_CLOSE 23.312
   12451915 SessClose      c RX_TIMEOUT 6.239
   14811178 SessClose      c TX_PIPE 0.045
   13828163 SessClose      c RX_TIMEOUT 7.665
   14221353 SessClose      c RX_TIMEOUT 5.010
   15826949 SessClose      c RX_TIMEOUT 5.019
    1638558 SessClose      c RX_TIMEOUT 6.976
   14745673 SessClose      c TX_PIPE 0.037
   14155791 SessClose      c TX_PIPE 42.360
   13828166 SessClose      c TX_PIPE 5.336
   15663111 SessClose      c RX_TIMEOUT 5.871
   14811170 SessClose      c RX_TIMEOUT 6.891
    6848788 SessClose      c RX_TIMEOUT 8.531
   12582985 SessClose      c RX_TIMEOUT 8.832
   15269916 SessClose      c RX_TIMEOUT 6.397
    7176360 SessClose      c REM_CLOSE 7.211
   12451913 SessClose      c RX_TIMEOUT 7.059
    1638561 SessClose      c RX_TIMEOUT 7.368
   14745676 SessClose      c TX_PIPE 0.286
   10813546 SessClose      c RX_TIMEOUT 15.983
   14450716 SessClose      c RX_TIMEOUT 10.746
   14811152 SessClose      c RX_TIMEOUT 15.734
   15761414 SessClose      c RX_TIMEOUT 5.007
   15138832 SessClose      c RX_TIMEOUT 7.806
   11403424 SessClose      c RX_TIMEOUT 16.144
   11403435 SessClose      c RX_TIMEOUT 15.877
   15368210 SessClose      c RX_TIMEOUT 5.000
   15859720 SessClose      c RX_TIMEOUT 5.000
   14516261 SessClose      c RX_TIMEOUT 5.017
   14319688 SessClose      c RX_TIMEOUT 19.693
   15564818 SessClose      c RX_TIMEOUT 6.239
   15859721 SessClose      c RX_TIMEOUT 5.003
   14221355 SessClose      c RX_TIMEOUT 5.004
   15302678 SessClose      c RX_TIMEOUT 12.262
   12353661 SessClose      c RX_TIMEOUT 7.196
   14516263 SessClose      c RX_TIMEOUT 5.513
   15302692 SessClose      c RX_TIMEOUT 6.276
   14450727 SessClose      c RX_TIMEOUT 5.009
   15368206 SessClose      c RX_TIMEOUT 8.205
   12419190 SessClose      c RX_TIMEOUT 7.357
   14745670 SessClose      c REM_CLOSE 9.053
    6848795 SessClose      c RX_TIMEOUT 6.026
   10223768 SessClose      c RX_TIMEOUT 12.442
   14319692 SessClose      c RX_TIMEOUT 6.306
   14450719 SessClose      c RX_TIMEOUT 7.926
   14221357 SessClose      c RX_TIMEOUT 6.714
   14221359 SessClose      c RX_TIMEOUT 5.001
   15663114 SessClose      c RX_TIMEOUT 6.867
   11960446 SessClose      c RX_TIMEOUT 11.245
   14450729 SessClose      c RX_TIMEOUT 6.668
   15859717 SessClose      c RX_TIMEOUT 7.886
   15663108 SessClose      c RX_TIMEOUT 10.451
   15302697 SessClose      c RX_TIMEOUT 7.992
   15106089 SessClose      c RX_TIMEOUT 15.843
   14319701 SessClose      c RX_TIMEOUT 6.969
   14221360 SessClose      c RX_TIMEOUT 5.961
   15990785 SessClose      c RX_TIMEOUT 5.076
   14450724 SessClose      c RX_TIMEOUT 7.928
   15958017 SessClose      c RX_TIMEOUT 7.158
   15958022 SessClose      c RX_TIMEOUT 5.013
   15368212 SessClose      c RX_TIMEOUT 6.159
   15794181 SessClose      c RX_TIMEOUT 5.025
   15958029 SessClose      c RX_TIMEOUT 5.001
   14516268 SessClose      c RX_TIMEOUT 7.593
   12877868 SessClose      c RX_TIMEOUT 23.554
   15794186 SessClose      c RX_TIMEOUT 5.009
   15859724 SessClose      c RX_TIMEOUT 5.035
   15859728 SessClose      c RX_TIMEOUT 5.008
   15794189 SessClose      c RX_TIMEOUT 5.007
   15859730 SessClose      c RX_TIMEOUT 5.055
   15794191 SessClose      c RX_TIMEOUT 5.015
   15663117 SessClose      c RX_TIMEOUT 7.499
   16089089 SessClose      c RX_TIMEOUT 5.615
   15302703 SessClose      c RX_TIMEOUT 7.730
    6848798 SessClose      c RX_TIMEOUT 10.057
   15204381 SessClose      c RX_TIMEOUT 9.821
   14778385 SessClose      c RX_TIMEOUT 7.749
   15106097 SessClose      c RX_TIMEOUT 10.413
   12353633 SessClose      c RX_TIMEOUT 34.255
   15794184 SessClose      c RX_TIMEOUT 10.964
    9175194 SessClose      c REM_CLOSE 0.048
   14778388 SessClose      c RX_TIMEOUT 6.667
   15106101 SessClose      c RX_TIMEOUT 6.855
   15106107 SessClose      c RX_TIMEOUT 5.001
   16220161 SessClose      c RX_TIMEOUT 6.373
   15368219 SessClose      c RX_TIMEOUT 7.956
   15106111 SessClose      c RX_TIMEOUT 6.038
   15368221 SessClose      c RX_TIMEOUT 6.502
   15368225 SessClose      c RX_TIMEOUT 5.002
   15368230 SessClose      c RX_TIMEOUT 6.896
   16416771 SessClose      c RX_TIMEOUT 6.389
   16449537 SessClose      c RX_TIMEOUT 6.652
   16121876 SessClose      c RX_TIMEOUT 5.001
   16351239 SessClose      c RX_TIMEOUT 5.824
   16351241 SessClose      c RX_TIMEOUT 6.161
   16547841 SessClose      c RX_TIMEOUT 7.863
   16351243 SessClose      c RX_TIMEOUT 6.342
   15007762 SessClose      c REM_CLOSE 47.493
   16416783 SessClose      c RESP_CLOSE 0.002
   13074447 SessClose      c RESP_CLOSE 0.002
   17235973 SessClose      c RESP_CLOSE 0.006
   13074449 SessClose      c RESP_CLOSE 0.005
   17530881 SessClose      c RESP_CLOSE 0.007
   18251781 SessClose      c REM_CLOSE 0.000
   18382849 SessClose      c REM_CLOSE 0.000
   18350083 SessClose      c REM_CLOSE 0.001
   18382851 SessClose      c REM_CLOSE 0.001
   18350085 SessClose      c REM_CLOSE 0.000
   18382853 SessClose      c REM_CLOSE 0.001
   18415617 SessClose      c REM_CLOSE 0.000
   10158138 SessClose      c REM_CLOSE 0.000
   18415619 SessClose      c REM_CLOSE 0.000
   10158140 SessClose      c REM_CLOSE 0.001
   18415621 SessClose      c REM_CLOSE 0.000
   10158142 SessClose      c REM_CLOSE 0.000
   18415623 SessClose      c REM_CLOSE 0.001
   10158144 SessClose      c REM_CLOSE 0.002
   18415625 SessClose      c REM_CLOSE 0.047
   18415627 SessClose      c REM_CLOSE 0.000
   18513921 SessClose      c REM_CLOSE 0.000
   18415629 SessClose      c REM_CLOSE 0.000
   18513923 SessClose      c REM_CLOSE 0.000
   18415631 SessClose      c REM_CLOSE 0.001
   18513925 SessClose      c REM_CLOSE 0.000
   18546691 SessClose      c REM_CLOSE 0.000
   18579459 SessClose      c REM_CLOSE 0.000
   19890177 SessClose      c REM_CLOSE 3.239
   20185089 SessClose      c REM_CLOSE 0.258
   11829273 SessClose      c REM_CLOSE 0.050
   20578305 SessClose      c REM_CLOSE 1.879
   20709377 SessClose      c REM_CLOSE 11.948
   10321987 SessClose      c REM_CLOSE 18.187
   21626881 SessClose      c REM_CLOSE 11.227
   22183939 SessClose      c REM_CLOSE 1.251
   22478851 SessClose      c REM_CLOSE 0.049
   22511619 SessClose      c REM_CLOSE 0.609
   23166977 SessClose      c REM_CLOSE 0.050
   23625729 SessClose      c REM_CLOSE 0.354
   24543233 SessClose      c REM_CLOSE 0.048
   25559043 SessClose      c RESP_CLOSE 0.048
   25591812 SessClose      c RESP_CLOSE 0.025
   25559045 SessClose      c RESP_CLOSE 1.206
   25591815 SessClose      c RESP_CLOSE 1.226
   25591817 SessClose      c RESP_CLOSE 0.754
   25591819 SessClose      c RESP_CLOSE 0.803
   25755653 SessClose      c RESP_CLOSE 1.179
   25853953 SessClose      c RESP_CLOSE 2.062
   25853955 SessClose      c RESP_CLOSE 1.590
   26411009 SessClose      c REM_CLOSE 4.109
    5341188 SessClose      c REM_CLOSE 0.021
   30015489 SessClose      c REM_CLOSE 0.088
   32079873 SessClose      c REM_CLOSE 0.000
   32079874 SessClose      c REM_CLOSE 0.000
   32374787 SessClose      c REM_CLOSE 0.000
   32374788 SessClose      c REM_CLOSE 0.000
   32374790 SessClose      c REM_CLOSE 0.000
   32374791 SessClose      c REM_CLOSE 0.000
   32866308 SessClose      c REM_CLOSE 0.000
   32866309 SessClose      c REM_CLOSE 0.000
   32899073 SessClose      c REM_CLOSE 0.000
   32931841 SessClose      c REM_CLOSE 0.000
   32931842 SessClose      c REM_CLOSE 0.000
   32964609 SessClose      c REM_CLOSE 0.000
   33062913 SessClose      c REM_CLOSE 0.000
   33193985 SessClose      c REM_CLOSE 0.000
   33193986 SessClose      c REM_CLOSE 0.000
   33193987 SessClose      c REM_CLOSE 0.000
   33226753 SessClose      c REM_CLOSE 0.000
   33292291 SessClose      c REM_CLOSE 0.000
   33292296 SessClose      c REM_CLOSE 0.000
   33357825 SessClose      c REM_CLOSE 0.000
   33456129 SessClose      c REM_CLOSE 0.001
   33456131 SessClose      c REM_CLOSE 0.000
   33456132 SessClose      c REM_CLOSE 0.000
   33685512 SessClose      c REM_CLOSE 0.001
   33685518 SessClose      c REM_CLOSE 0.000
   33685520 SessClose      c REM_CLOSE 0.000
   33685522 SessClose      c REM_CLOSE 0.000
   33685523 SessClose      c REM_CLOSE 0.000
   34373633 SessClose      c REM_CLOSE 0.000
   34537473 SessClose      c REM_CLOSE 0.001
   34701315 SessClose      c REM_CLOSE 0.000
   34701317 SessClose      c REM_CLOSE 0.000
   34701321 SessClose      c REM_CLOSE 0.000
   34799617 SessClose      c REM_CLOSE 0.000
   34799618 SessClose      c REM_CLOSE 0.000
   34799619 SessClose      c REM_CLOSE 0.000
   34865153 SessClose      c REM_CLOSE 0.000
   34897925 SessClose      c REM_CLOSE 0.000
   34897926 SessClose      c REM_CLOSE 0.000
   34897927 SessClose      c REM_CLOSE 0.000
   34897928 SessClose      c REM_CLOSE 0.000
   34930689 SessClose      c REM_CLOSE 0.000
   35160069 SessClose      c REM_CLOSE 0.000
   35160070 SessClose      c REM_CLOSE 0.000
   35160076 SessClose      c REM_CLOSE 0.000
   35160077 SessClose      c REM_CLOSE 0.000
   35291137 SessClose      c REM_CLOSE 0.000
   35323905 SessClose      c REM_CLOSE 0.001
   35422209 SessClose      c REM_CLOSE 0.001
   35454982 SessClose      c REM_CLOSE 0.000
   35553281 SessClose      c REM_CLOSE 0.000
   35782657 SessClose      c REM_CLOSE 0.000
   36306947 SessClose      c REM_CLOSE 0.000
   36306950 SessClose      c REM_CLOSE 0.000
   36339713 SessClose      c REM_CLOSE 0.040
   36405255 SessClose      c REM_CLOSE 0.107
   36470787 SessClose      c REM_CLOSE 0.000
   36470788 SessClose      c REM_CLOSE 0.000
   36438023 SessClose      c REM_CLOSE 0.127
   36438031 SessClose      c REM_CLOSE 0.000
   36438032 SessClose      c REM_CLOSE 0.000
   36438035 SessClose      c REM_CLOSE 0.000
   36438036 SessClose      c REM_CLOSE 0.000
   36503563 SessClose      c REM_CLOSE 0.020
   36503565 SessClose      c REM_CLOSE 0.000
   36503566 SessClose      c REM_CLOSE 0.000
   36503569 SessClose      c REM_CLOSE 0.000
   36601859 SessClose      c REM_CLOSE 0.000
   36601860 SessClose      c REM_CLOSE 0.000
   36601861 SessClose      c REM_CLOSE 0.000
   36569095 SessClose      c REM_CLOSE 0.119
   36929543 SessClose      c REM_CLOSE 0.000
   36929546 SessClose      c REM_CLOSE 0.075
   37224457 SessClose      c REM_CLOSE 0.086
   37257217 SessClose      c REM_CLOSE 0.000
   37257220 SessClose      c REM_CLOSE 0.000
   37355523 SessClose      c REM_CLOSE 0.332
   37486597 SessClose      c REM_CLOSE 0.125
   37617669 SessClose      c REM_CLOSE 0.017
   37617675 SessClose      c REM_CLOSE 0.000
   37617676 SessClose      c REM_CLOSE 0.000
   37617677 SessClose      c REM_CLOSE 0.000
   37683201 SessClose      c REM_CLOSE 0.073
   38109189 SessClose      c REM_CLOSE 0.000
   38141953 SessClose      c REM_CLOSE 0.108
   38469633 SessClose      c REM_CLOSE 0.025
   38469637 SessClose      c REM_CLOSE 0.000
   38469644 SessClose      c REM_CLOSE 0.000
   38535169 SessClose      c REM_CLOSE 0.000
   38535172 SessClose      c REM_CLOSE 0.000
   38535173 SessClose      c REM_CLOSE 0.121
   38699009 SessClose      c REM_CLOSE 0.077
   38699025 SessClose      c REM_CLOSE 0.046
   38699031 SessClose      c REM_CLOSE 0.000
   38731783 SessClose      c REM_CLOSE 0.000
   38731784 SessClose      c REM_CLOSE 0.000
   38731785 SessClose      c REM_CLOSE 0.000
   38731786 SessClose      c REM_CLOSE 0.000
   38764545 SessClose      c REM_CLOSE 0.097
   38764549 SessClose      c REM_CLOSE 0.000
   38764550 SessClose      c REM_CLOSE 0.000
   38764551 SessClose      c REM_CLOSE 0.000
   38862849 SessClose      c REM_CLOSE 0.001
   38928385 SessClose      c REM_CLOSE 0.001
   38993921 SessClose      c REM_CLOSE 0.000
   39124993 SessClose      c REM_CLOSE 0.057
   39550977 SessClose      c REM_CLOSE 0.094
   39550981 SessClose      c REM_CLOSE 0.000
   39550982 SessClose      c REM_CLOSE 0.000
   39550983 SessClose      c REM_CLOSE 0.000
   39747585 SessClose      c REM_CLOSE 0.000
   39780355 SessClose      c REM_CLOSE 0.106
   39780361 SessClose      c REM_CLOSE 0.016
   39813125 SessClose      c REM_CLOSE 0.000
   39813126 SessClose      c REM_CLOSE 0.000
   39813127 SessClose      c REM_CLOSE 0.000
   39878657 SessClose      c REM_CLOSE 0.000
   39911427 SessClose      c REM_CLOSE 0.000
   39911428 SessClose      c REM_CLOSE 0.000
   39944193 SessClose      c REM_CLOSE 0.284
   39944195 SessClose      c REM_CLOSE 0.000
   39944202 SessClose      c REM_CLOSE 0.000
   39944203 SessClose      c REM_CLOSE 0.000
   40009729 SessClose      c REM_CLOSE 0.000
   40042507 SessClose      c REM_CLOSE 0.000
   40108033 SessClose      c REM_CLOSE 0.099
   40108035 SessClose      c REM_CLOSE 0.000
   40140801 SessClose      c REM_CLOSE 0.000
   40140806 SessClose      c REM_CLOSE 0.000
   40173569 SessClose      c REM_CLOSE 0.000
   40173572 SessClose      c REM_CLOSE 0.000
   38928389 SessClose      c REM_CLOSE 0.000
   40271875 SessClose      c REM_CLOSE 0.000
   40271876 SessClose      c REM_CLOSE 0.000
   40304641 SessClose      c REM_CLOSE 0.000
   38928393 SessClose      c REM_CLOSE 0.001
   40435713 SessClose      c REM_CLOSE 0.000
   40468481 SessClose      c REM_CLOSE 0.001
   40501249 SessClose      c REM_CLOSE 0.001
   40534017 SessClose      c REM_CLOSE 0.000
   40566785 SessClose      c REM_CLOSE 0.000
   40566789 SessClose      c REM_CLOSE 0.000
   40566793 SessClose      c REM_CLOSE 0.000
   40566795 SessClose      c REM_CLOSE 0.000
   40632321 SessClose      c REM_CLOSE 0.000
   40632323 SessClose      c REM_CLOSE 0.175
   40665091 SessClose      c REM_CLOSE 0.000
   40665092 SessClose      c REM_CLOSE 0.000
   40665095 SessClose      c REM_CLOSE 0.013
   40665097 SessClose      c REM_CLOSE 0.000
   40665098 SessClose      c REM_CLOSE 0.182
   40566796 SessClose      c REM_CLOSE 0.000
   40697857 SessClose      c REM_CLOSE 0.142
   40730625 SessClose      c REM_CLOSE 0.000
   40632325 SessClose      c REM_CLOSE 0.001
   40632327 SessClose      c REM_CLOSE 0.000
   40632328 SessClose      c REM_CLOSE 0.000
   40763393 SessClose      c REM_CLOSE 0.000
   40796161 SessClose      c REM_CLOSE 0.000
   40828929 SessClose      c REM_CLOSE 0.001
   40828931 SessClose      c REM_CLOSE 0.000
   40828932 SessClose      c REM_CLOSE 0.001
   40861697 SessClose      c REM_CLOSE 0.001
   40894465 SessClose      c REM_CLOSE 0.001
   40894467 SessClose      c REM_CLOSE 0.059
   40894469 SessClose      c REM_CLOSE 0.000
   40992769 SessClose      c REM_CLOSE 0.000
   41058307 SessClose      c REM_CLOSE 0.054
   41058309 SessClose      c REM_CLOSE 0.000
   41058310 SessClose      c REM_CLOSE 0.000
   41091073 SessClose      c REM_CLOSE 0.000
   41189379 SessClose      c REM_CLOSE 0.025
   41189381 SessClose      c REM_CLOSE 0.000
   41189383 SessClose      c REM_CLOSE 0.000
   41189386 SessClose      c REM_CLOSE 0.000
   41189389 SessClose      c REM_CLOSE 0.000
   41320449 SessClose      c REM_CLOSE 0.000
   41320452 SessClose      c REM_CLOSE 0.000
   41353217 SessClose      c REM_CLOSE 0.000
   41385987 SessClose      c REM_CLOSE 0.049
   41385991 SessClose      c REM_CLOSE 0.058
   41385995 SessClose      c REM_CLOSE 0.000
   41418753 SessClose      c REM_CLOSE 0.000
   41418757 SessClose      c REM_CLOSE 0.000
   41451523 SessClose      c REM_CLOSE 0.000
   41517059 SessClose      c REM_CLOSE 0.000
   41517062 SessClose      c REM_CLOSE 0.013
   41549825 SessClose      c REM_CLOSE 0.043
   41582593 SessClose      c REM_CLOSE 0.000
   41582594 SessClose      c REM_CLOSE 0.000
   41582596 SessClose      c REM_CLOSE 0.000
   41582599 SessClose      c REM_CLOSE 0.000
   41713665 SessClose      c REM_CLOSE 0.000
   41811969 SessClose      c REM_CLOSE 0.000
   41779203 SessClose      c REM_CLOSE 0.000
   42008577 SessClose      c REM_CLOSE 0.000
   42008580 SessClose      c REM_CLOSE 0.124
   42041349 SessClose      c REM_CLOSE 0.000
   42270721 SessClose      c REM_CLOSE 0.308
   42270739 SessClose      c REM_CLOSE 0.013
   42303491 SessClose      c REM_CLOSE 0.000
   42303492 SessClose      c REM_CLOSE 0.000
   42303493 SessClose      c REM_CLOSE 0.000
   42336257 SessClose      c REM_CLOSE 0.000
   42369025 SessClose      c REM_CLOSE 0.000
   42500103 SessClose      c REM_CLOSE 0.000
   42532865 SessClose      c REM_CLOSE 0.000
   42532867 SessClose      c REM_CLOSE 0.000
   42532868 SessClose      c REM_CLOSE 0.108
   42565635 SessClose      c REM_CLOSE 0.000
   42663937 SessClose      c REM_CLOSE 0.175
   42663939 SessClose      c REM_CLOSE 0.000
   42663942 SessClose      c REM_CLOSE 0.000
   42696705 SessClose      c REM_CLOSE 2.000
   42729473 SessClose      c REM_CLOSE 0.000
   42827787 SessClose      c REM_CLOSE 0.111
   42991619 SessClose      c REM_CLOSE 0.000
   42991620 SessClose      c REM_CLOSE 0.000
   43024385 SessClose      c REM_CLOSE 0.101
   43155457 SessClose      c REM_CLOSE 0.000
   43155460 SessClose      c REM_CLOSE 0.000
   43155470 SessClose      c REM_CLOSE 0.000
   43155473 SessClose      c REM_CLOSE 0.000
   43188227 SessClose      c REM_CLOSE 0.089
   43384841 SessClose      c REM_CLOSE 0.000
   43384846 SessClose      c REM_CLOSE 0.000
   43417603 SessClose      c REM_CLOSE 0.000
   43417606 SessClose      c REM_CLOSE 0.065
   43417608 SessClose      c REM_CLOSE 0.014
   43483137 SessClose      c REM_CLOSE 0.133
   43515905 SessClose      c REM_CLOSE 0.000
   43548675 SessClose      c REM_CLOSE 0.000
   43548678 SessClose      c REM_CLOSE 0.000
   43548681 SessClose      c REM_CLOSE 0.000
   43548682 SessClose      c REM_CLOSE 0.107
   43548684 SessClose      c REM_CLOSE 0.000
   43548686 SessClose      c REM_CLOSE 0.000
   43548687 SessClose      c REM_CLOSE 0.000
   43548688 SessClose      c REM_CLOSE 0.000
   43548691 SessClose      c REM_CLOSE 0.063
   43581443 SessClose      c REM_CLOSE 0.000
   43614211 SessClose      c REM_CLOSE 0.000
   43646977 SessClose      c REM_CLOSE 0.053
   43909123 SessClose      c REM_CLOSE 0.000
   43909124 SessClose      c REM_CLOSE 0.000
   43909125 SessClose      c REM_CLOSE 0.000
   43909126 SessClose      c REM_CLOSE 0.000
   43941889 SessClose      c REM_CLOSE 0.000
   44007427 SessClose      c REM_CLOSE 0.121
   44498945 SessClose      c REM_CLOSE 0.102
   44564483 SessClose      c REM_CLOSE 0.000
   44564494 SessClose      c REM_CLOSE 0.000
   44597251 SessClose      c REM_CLOSE 0.000
   44564495 SessClose      c REM_CLOSE 0.076
   44597256 SessClose      c REM_CLOSE 0.041
   44728321 SessClose      c REM_CLOSE 0.000
   44728322 SessClose      c REM_CLOSE 0.135
   44728328 SessClose      c REM_CLOSE 0.000
   44761089 SessClose      c REM_CLOSE 0.000
   44793879 SessClose      c REM_CLOSE 0.000
   44826625 SessClose      c REM_CLOSE 0.000
   44826626 SessClose      c REM_CLOSE 0.048
   45285377 SessClose      c REM_CLOSE 0.000
   45285384 SessClose      c REM_CLOSE 0.012
   45285388 SessClose      c REM_CLOSE 0.000
   45547527 SessClose      c REM_CLOSE 0.324
   45547529 SessClose      c REM_CLOSE 0.000
   45547540 SessClose      c REM_CLOSE 0.000
   45547543 SessClose      c REM_CLOSE 0.013
   45547559 SessClose      c REM_CLOSE 0.020
   45547567 SessClose      c REM_CLOSE 0.000
   45547570 SessClose      c REM_CLOSE 0.000
   45547573 SessClose      c REM_CLOSE 0.000
   45580291 SessClose      c REM_CLOSE 0.000
   45580296 SessClose      c REM_CLOSE 0.055
   45580304 SessClose      c REM_CLOSE 0.000
   45613057 SessClose      c REM_CLOSE 0.000
   45875201 SessClose      c REM_CLOSE 0.000
   45973505 SessClose      c REM_CLOSE 0.000
   46039041 SessClose      c REM_CLOSE 0.000
   46039048 SessClose      c REM_CLOSE 0.000
   46104581 SessClose      c REM_CLOSE 0.000
   46170113 SessClose      c REM_CLOSE 0.034
   46301189 SessClose      c REM_CLOSE 0.000
   46301190 SessClose      c REM_CLOSE 0.000
   46301197 SessClose      c REM_CLOSE 0.115
   46563331 SessClose      c REM_CLOSE 0.000
   46596099 SessClose      c REM_CLOSE 0.000
   46694403 SessClose      c REM_CLOSE 0.276
   46694405 SessClose      c REM_CLOSE 0.000
   46694406 SessClose      c REM_CLOSE 0.000
   46694407 SessClose      c REM_CLOSE 0.000
   46694408 SessClose      c REM_CLOSE 0.000
   46694409 SessClose      c REM_CLOSE 0.000
   46727175 SessClose      c REM_CLOSE 0.225
   46727187 SessClose      c REM_CLOSE 0.168
   46727197 SessClose      c REM_CLOSE 0.000
   46727198 SessClose      c REM_CLOSE 0.000
   46727205 SessClose      c REM_CLOSE 0.000
   46727210 SessClose      c REM_CLOSE 0.191
   46727220 SessClose      c REM_CLOSE 0.000
   46792705 SessClose      c REM_CLOSE 0.000
   46792706 SessClose      c REM_CLOSE 0.000
   46792711 SessClose      c REM_CLOSE 0.000
   47120385 SessClose      c REM_CLOSE 0.153
   47218689 SessClose      c REM_CLOSE 0.074
   47284233 SessClose      c REM_CLOSE 0.000
   47415301 SessClose      c REM_CLOSE 0.109
   47579143 SessClose      c REM_CLOSE 0.000
   47579146 SessClose      c REM_CLOSE 0.060
   47611905 SessClose      c REM_CLOSE 0.000
   47644673 SessClose      c REM_CLOSE 0.000
   47644674 SessClose      c REM_CLOSE 0.000
   47677441 SessClose      c REM_CLOSE 0.000
   47677442 SessClose      c REM_CLOSE 0.000
   47710209 SessClose      c REM_CLOSE 0.063
   47742977 SessClose      c REM_CLOSE 0.000
   47742978 SessClose      c REM_CLOSE 0.097
   47775745 SessClose      c REM_CLOSE 0.000
   47775761 SessClose      c REM_CLOSE 0.000
   47775770 SessClose      c REM_CLOSE 0.047
   47775780 SessClose      c REM_CLOSE 0.000
   47808513 SessClose      c REM_CLOSE 0.000
   47808514 SessClose      c REM_CLOSE 0.000
   44498953 SessClose      c REM_CLOSE 179.998
 }}}


 {{{
 cat file.log| grep Debug|grep -v RES|grep -v Broken|grep -v retrying|grep
 -v "Connection reset by peer"|grep -v 909
    6979596 Debug          c "Write error, retval = -1, len = 836290, errno
 = Resource temporarily unavailable"
    1638486 Debug          c "Write error, retval = -1, len = 717552, errno
 = Resource temporarily unavailable"
   44498954 Debug          c "Write error, retval = -1, len = 1330963,
 errno = Resource temporarily unavailable"
 }}}

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1753#comment:3>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator



More information about the varnish-bugs mailing list