[Varnish] #1100: Assert error in stv_alloc(), stevedore.c line 144
Varnish
varnish-bugs at varnish-cache.org
Mon Feb 27 18:01:51 CET 2012
#1100: Assert error in stv_alloc(), stevedore.c line 144
-------------------+--------------------------------------------------------
Reporter: Roze | Type: defect
Status: new | Priority: normal
Milestone: | Component: varnishd
Version: trunk | Severity: normal
Keywords: |
-------------------+--------------------------------------------------------
Comment(by Roze):
I had checked out the 5773 revision (2011-01-20 09:07:38 +0200 (Thu, 20
Jan 2011)) (since the stable release had similar issues).
After the restart I am not able to see the panic.show ouput (I guess its
because the master process kills the child):
{{{
CLI connected to 127.0.0.1:8000
200 213
-----------------------------
Varnish Cache CLI 1.0
-----------------------------
Linux,3.2.0-rc2-2-default,x86_64,-sfile,-sfile,-smalloc,-hcritbit
Type 'help' for command list.
Type 'quit' to close CLI session.
panic.show
101 44
Unknown request.
}}}
Varnishstat output kinda resets on each child restart.
The interesting thing is I was able to reproduce this even with a small
file storage (like 10MB) but then after some SVN checkout it went away,
but now it still triggers when the storage is big(ger) and since it
happens only when it fills 100% it's hard to catch the right moment.
The current output:
{{{
client_conn 77600805 422.14 Client connections accepted
client_drop 18915 0.10 Connection dropped, no
sess/wrk
client_req 94600173 514.61 Client requests received
cache_hit 90765387 493.75 Cache hits
cache_hitpass 112181 0.61 Cache hits for pass
cache_miss 3885947 21.14 Cache misses
backend_conn 373957 2.03 Backend conn. success
backend_unhealthy 0 0.00 Backend conn. not attempted
backend_busy 0 0.00 Backend conn. too many
backend_fail 194947 1.06 Backend conn. failures
backend_reuse 3443070 18.73 Backend conn. reuses
backend_toolate 272840 1.48 Backend conn. was closed
backend_recycle 3715939 20.21 Backend conn. recycles
backend_unused 0 0.00 Backend conn. unused
backend_retry 14101 0.08 Backend conn. retry
fetch_head 0 0.00 Fetch head
fetch_length 3791440 20.62 Fetch with Length
fetch_chunked 0 0.00 Fetch chunked
fetch_eof 0 0.00 Fetch EOF
fetch_bad 0 0.00 Fetch had bad headers
fetch_close 0 0.00 Fetch wanted close
fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed
fetch_zero 0 0.00 Fetch zero len
fetch_failed 0 0.00 Fetch failed
fetch_1xx 0 0.00 Fetch no body (1xx)
fetch_204 0 0.00 Fetch no body (204)
fetch_304 0 0.00 Fetch no body (304)
n_sess_mem 21987 . N struct sess_mem
n_sess 18247 . N struct sess
n_object 3459291 . N struct object
n_vampireobject 0 . N unresurrected objects
n_objectcore 3460308 . N struct objectcore
n_objecthead 3460484 . N struct objecthead
n_waitinglist 63846 . N struct waitinglist
n_vbc 12 . N struct vbc
n_wrk 1030 . N worker threads
n_wrk_create 9482 0.05 N worker threads created
n_wrk_failed 0 0.00 N worker threads not created
n_wrk_max 48776 0.27 N worker threads limited
n_wrk_lqueue 0 0.00 work request queue length
n_wrk_queued 620633 3.38 N queued work requests
n_wrk_drop 21564 0.12 N dropped work requests
n_backend 1 . N backends
n_expired 169782 . N expired objects
n_lru_nuked 0 . N LRU nuked objects
n_lru_moved 51162886 . N LRU moved objects
losthdr 0 0.00 HTTP header overflows
n_objsendfile 0 0.00 Objects sent with sendfile
n_objwrite 92153039 501.30 Objects sent with write
n_objoverflow 0 0.00 Objects overflowing workspace
s_sess 77581773 422.03 Total Sessions
s_req 94600173 514.61 Total Requests
s_pipe 82 0.00 Total pipe
s_pass 112209 0.61 Total pass
s_fetch 3791440 20.62 Total fetch
s_hdrbytes 25728025086 139956.29 Total header bytes
s_bodybytes 1262279919691 6866598.41 Total body bytes
sess_closed 2211949 12.03 Session Closed
sess_pipeline 69946 0.38 Session Pipeline
sess_readahead 28511 0.16 Session Read Ahead
sess_linger 93978889 511.23 Session Linger
sess_herd 99705857 542.38 Session herd
shm_records 3738829095 20338.62 SHM records
shm_writes 435239256 2367.63 SHM writes
shm_flushes 0 0.00 SHM flushes due to overflow
shm_cont 4901047 26.66 SHM MTX contention
shm_cycles 1427 0.01 SHM cycles through buffer
sms_nreq 0 0.00 SMS allocator requests
sms_nobj 0 . SMS outstanding allocations
sms_nbytes 0 . SMS outstanding bytes
sms_balloc 0 . SMS bytes allocated
sms_bfree 0 . SMS bytes freed
backend_req 3817046 20.76 Backend requests made
n_vcl 1 0.00 N vcl total
n_vcl_avail 1 0.00 N vcl available
n_vcl_discard 0 0.00 N vcl discarded
n_ban 1 . N total active bans
n_ban_add 1 0.00 N new bans added
n_ban_retire 0 0.00 N old bans deleted
n_ban_obj_test 0 0.00 N objects tested
n_ban_re_test 0 0.00 N regexps tested against
n_ban_dups 0 0.00 N duplicate bans removed
hcb_nolock 94759999 515.48 HCB Lookups without lock
hcb_lock 3727914 20.28 HCB Lookups with lock
hcb_insert 3727854 20.28 HCB Inserts
esi_parse 0 0.00 Objects ESI parsed (unlock)
esi_errors 0 0.00 ESI parse errors (unlock)
esi_warnings 0 0.00 ESI parse warnings (unlock)
accept_fail 0 0.00 Accept failures
client_drop_late 2649 0.01 Connection dropped late
uptime 183829 1.00 Client uptime
dir_dns_lookups 0 0.00 DNS director lookups
dir_dns_failed 0 0.00 DNS director failed lookups
dir_dns_hit 0 0.00 DNS director cached lookups
hit
dir_dns_cache_full 0 0.00 DNS director full dnscache
vmods 0 . Loaded VMODs
LCK.sms.creat 1 0.00 Created locks
LCK.sms.destroy 0 0.00 Destroyed locks
LCK.sms.locks 0 0.00 Lock Operations
LCK.sms.colls 0 0.00 Collisions
LCK.smp.creat 0 0.00 Created locks
LCK.smp.destroy 0 0.00 Destroyed locks
LCK.smp.locks 0 0.00 Lock Operations
LCK.smp.colls 0 0.00 Collisions
LCK.sma.creat 1 0.00 Created locks
LCK.sma.destroy 0 0.00 Destroyed locks
LCK.sma.locks 566851 3.08 Lock Operations
LCK.sma.colls 0 0.00 Collisions
LCK.smf.creat 2 0.00 Created locks
LCK.smf.destroy 0 0.00 Destroyed locks
LCK.smf.locks 7597579 41.33 Lock Operations
LCK.smf.colls 0 0.00 Collisions
LCK.hsl.creat 0 0.00 Created locks
LCK.hsl.destroy 0 0.00 Destroyed locks
LCK.hsl.locks 0 0.00 Lock Operations
LCK.hsl.colls 0 0.00 Collisions
LCK.hcb.creat 1 0.00 Created locks
LCK.hcb.destroy 0 0.00 Destroyed locks
LCK.hcb.locks 3997483 21.75 Lock Operations
LCK.hcb.colls 0 0.00 Collisions
LCK.hcl.creat 0 0.00 Created locks
LCK.hcl.destroy 0 0.00 Destroyed locks
LCK.hcl.locks 0 0.00 Lock Operations
LCK.hcl.colls 0 0.00 Collisions
LCK.vcl.creat 1 0.00 Created locks
LCK.vcl.destroy 0 0.00 Destroyed locks
LCK.vcl.locks 115440 0.63 Lock Operations
LCK.vcl.colls 0 0.00 Collisions
LCK.stat.creat 1 0.00 Created locks
LCK.stat.destroy 0 0.00 Destroyed locks
LCK.stat.locks 21987 0.12 Lock Operations
LCK.stat.colls 0 0.00 Collisions
LCK.sessmem.creat 1 0.00 Created locks
LCK.sessmem.destroy 0 0.00 Destroyed locks
LCK.sessmem.locks 77627872 422.28 Lock Operations
LCK.sessmem.colls 0 0.00 Collisions
LCK.wstat.creat 1 0.00 Created locks
LCK.wstat.destroy 0 0.00 Destroyed locks
LCK.wstat.locks 1339768 7.29 Lock Operations
LCK.wstat.colls 0 0.00 Collisions
LCK.herder.creat 1 0.00 Created locks
LCK.herder.destroy 0 0.00 Destroyed locks
LCK.herder.locks 124570 0.68 Lock Operations
LCK.herder.colls 0 0.00 Collisions
LCK.wq.creat 8 0.00 Created locks
LCK.wq.destroy 0 0.00 Destroyed locks
LCK.wq.locks 205392462 1117.30 Lock Operations
LCK.wq.colls 0 0.00 Collisions
LCK.objhdr.creat 3736407 20.33 Created locks
LCK.objhdr.destroy 276178 1.50 Destroyed locks
LCK.objhdr.locks 379256304 2063.09 Lock Operations
LCK.objhdr.colls 0 0.00 Collisions
LCK.exp.creat 1 0.00 Created locks
LCK.exp.destroy 0 0.00 Destroyed locks
LCK.exp.locks 3982075 21.66 Lock Operations
LCK.exp.colls 0 0.00 Collisions
LCK.cli.creat 1 0.00 Created locks
LCK.cli.destroy 0 0.00 Destroyed locks
LCK.cli.locks 61272 0.33 Lock Operations
LCK.cli.colls 0 0.00 Collisions
LCK.ban.creat 1 0.00 Created locks
LCK.ban.destroy 0 0.00 Destroyed locks
LCK.ban.locks 3982428 21.66 Lock Operations
LCK.ban.colls 0 0.00 Collisions
LCK.vbp.creat 1 0.00 Created locks
LCK.vbp.destroy 0 0.00 Destroyed locks
LCK.vbp.locks 0 0.00 Lock Operations
LCK.vbp.colls 0 0.00 Collisions
LCK.vbe.creat 1 0.00 Created locks
LCK.vbe.destroy 0 0.00 Destroyed locks
LCK.vbe.locks 1138562 6.19 Lock Operations
LCK.vbe.colls 0 0.00 Collisions
LCK.backend.creat 1 0.00 Created locks
LCK.backend.destroy 0 0.00 Destroyed locks
LCK.backend.locks 9818919 53.41 Lock Operations
LCK.backend.colls 0 0.00 Collisions
SMF.s0.nreq 3629114 19.74 Allocator requests
SMF.s0.nobj 3459482 . Outstanding allocations
SMF.s0.nbytes 0 . Outstanding bytes
SMF.s0.balloc 71045181440 . Bytes allocated
SMF.s0.bfree 36329000960 . Bytes free
SMF.s0.n_smf 3459526 . N struct smf
SMF.s0.n_smf_frag 0 . N small free smf
SMF.s0.n_smf_large 44 . N large free smf
SMF.s1.nreq 3629041 19.74 Allocator requests
SMF.s1.nobj 3459109 . Outstanding allocations
SMF.s1.nbytes 0 . Outstanding bytes
SMF.s1.balloc 71121489920 . Bytes allocated
SMF.s1.bfree 36252692480 . Bytes free
SMF.s1.n_smf 3459153 . N struct smf
SMF.s1.n_smf_frag 0 . N small free smf
SMF.s1.n_smf_large 44 . N large free smf
SMA.Transient.nreq 566851 3.08 Allocator requests
SMA.Transient.nobj 566851 . Outstanding allocations
SMA.Transient.nbytes 707675125 . Outstanding bytes
SMA.Transient.balloc 707675125 . Bytes allocated
SMA.Transient.bfree 0 . Bytes free
VBE.default(10.0.10.2,,8080).vcls 18446744073705461594 . VCL
references
VBE.default(10.0.10.2,,8080).happy 0 . Happy health
probes
}}}
--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1100#comment:2>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator
More information about the varnish-bugs
mailing list