assertion error in 2.1.5
Jon Morby
jon at fido.net
Thu May 5 15:49:54 CEST 2011
Hi
We're seeing a number of segfaults / assertion errors in varnish
Panic message: Assert error in HSH_Lookup(), cache_hash.c line 376:
Condition((o)->magic == 0x32851d42) not true. thread = (cache-worker)
ident = Linux,2.6.18-238.9.1.el5,x86_64,-sfile,-hcritbit,epoll
Backtrace: 0x424446: /usr/sbin/varnishd [0x424446] 0x41e097:
/usr/sbin/varnishd(HSH_Lookup+0x317) [0x41e097] 0x4121a0:
/usr/sbin/varnishd [0x4121a0] 0x414744:
/usr/sbin/varnishd(CNT_Session+0x4a4) [0x414744] 0x426898:
/usr/sbin/varnishd [0x426898] 0x425b7d: /usr/sbin/varnishd
[0x425b7d] 0x3bf580673d: /lib64/libpthread.so.0 [0x3bf580673d]
0x3bf4cd44bd: /lib64/libc.so.6(clone+0x6d) [0x3bf4cd44bd] sp =
0x2aaaeb642008 { fd = 114, id = 114, xid = 1781230731, client =
80.163.34.184 53244, step = STP_LOOKUP, handling = hash, restarts
= 0, esis = 0 ws = 0x2aaaeb642080 { id = "sess", {s,f,r,e} =
{0x2aaaeb642cd8,+488,(nil),+65536}, }, http[req] = { ws =
0x2aaaeb642080[sess] "GET", "/skin/frontend/default/bk-denmark/f
Is this a known issue, or can we provide more info (if so what?) in
order to help track it down further
client_conn 19077 7.58 Client connections accepted
client_drop 0 0.00 Connection dropped, no sess/wrk
client_req 61788 24.55 Client requests received
cache_hit 1666 0.66 Cache hits
cache_hitpass 0 0.00 Cache hits for pass
cache_miss 55025 21.86 Cache misses
backend_conn 60071 23.87 Backend conn. success
backend_unhealthy 0 0.00 Backend conn. not attempted
backend_busy 0 0.00 Backend conn. too many
backend_fail 0 0.00 Backend conn. failures
backend_reuse 0 0.00 Backend conn. reuses
backend_toolate 0 0.00 Backend conn. was closed
backend_recycle 0 0.00 Backend conn. recycles
backend_unused 0 0.00 Backend conn. unused
fetch_head 3 0.00 Fetch head
fetch_length 45421 18.05 Fetch with Length
fetch_chunked 9747 3.87 Fetch chunked
fetch_eof 0 0.00 Fetch EOF
fetch_bad 0 0.00 Fetch had bad headers
fetch_close 2 0.00 Fetch wanted close
fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed
fetch_zero 0 0.00 Fetch zero len
fetch_failed 0 0.00 Fetch failed
n_sess_mem 121 . N struct sess_mem
n_sess 17 . N struct sess
n_object 53856 . N struct object
n_vampireobject 0 . N unresurrected objects
n_objectcore 53856 . N struct objectcore
n_objecthead 53920 . N struct objecthead
n_smf 107673 . N struct smf
n_smf_frag 0 . N small free smf
n_smf_large 1 . N large free smf
n_vbe_conn 2 . N struct vbe_conn
n_wrk 20 . N worker threads
n_wrk_create 42 0.02 N worker threads created
n_wrk_failed 0 0.00 N worker threads not created
n_wrk_max 0 0.00 N worker threads limited
n_wrk_queue 0 0.00 N queued work requests
n_wrk_overflow 422 0.17 N overflowed work requests
n_wrk_drop 0 0.00 N dropped work requests
n_backend 1 . N backends
n_expired 1169 . N expired objects
n_lru_nuked 0 . N LRU nuked objects
n_lru_saved 0 . N LRU saved objects
n_lru_moved 1587 . N LRU moved objects
n_deathrow 0 . N objects on deathrow
losthdr 0 0.00 HTTP header overflows
n_objsendfile 0 0.00 Objects sent with sendfile
n_objwrite 49906 19.83 Objects sent with write
n_objoverflow 0 0.00 Objects overflowing workspace
s_sess 19075 7.58 Total Sessions
s_req 61788 24.55 Total Requests
s_pipe 4949 1.97 Total pipe
s_pass 148 0.06 Total pass
s_fetch 55173 21.92 Total fetch
s_hdrbytes 15551147 6178.45 Total header bytes
s_bodybytes 483089050 191930.49 Total body bytes
sess_closed 5683 2.26 Session Closed
sess_pipeline 12 0.00 Session Pipeline
sess_readahead 6 0.00 Session Read Ahead
sess_linger 56242 22.34 Session Linger
sess_herd 53038 21.07 Session herd
shm_records 4306688 1711.04 SHM records
shm_writes 290053 115.24 SHM writes
shm_flushes 0 0.00 SHM flushes due to overflow
shm_cont 746 0.30 SHM MTX contention
shm_cycles 2 0.00 SHM cycles through buffer
sm_nreq 110143 43.76 allocator requests
sm_nobj 107672 . outstanding allocations
sm_balloc 826855424 . bytes allocated
sm_bfree 246886400 . bytes free
sma_nreq 0 0.00 SMA allocator requests
sma_nobj 0 . SMA outstanding allocations
sma_nbytes 0 . SMA outstanding bytes
sma_balloc 0 . SMA bytes allocated
sma_bfree 0 . SMA bytes free
sms_nreq 0 0.00 SMS allocator requests
sms_nobj 0 . SMS outstanding allocations
sms_nbytes 0 . SMS outstanding bytes
sms_balloc 0 . SMS bytes allocated
sms_bfree 0 . SMS bytes freed
backend_req 55172 21.92 Backend requests made
n_vcl 1 0.00 N vcl total
n_vcl_avail 1 0.00 N vcl available
n_vcl_discard 0 0.00 N vcl discarded
n_purge 1 . N total active purges
n_purge_add 1 0.00 N new purges added
n_purge_retire 0 0.00 N old purges deleted
n_purge_obj_test 0 0.00 N objects tested
n_purge_re_test 0 0.00 N regexps tested against
n_purge_dups 0 0.00 N duplicate purges removed
hcb_nolock 56684 22.52 HCB Lookups without lock
hcb_lock 54986 21.85 HCB Lookups with lock
hcb_insert 54986 21.85 HCB Inserts
esi_parse 0 0.00 Objects ESI parsed (unlock)
esi_errors 0 0.00 ESI parse errors (unlock)
accept_fail 0 0.00 Accept failures
client_drop_late 0 0.00 Connection dropped late
uptime 2517 1.00 Client uptime
backend_retry 0 0.00 Backend conn. retry
dir_dns_lookups 0 0.00 DNS director lookups
dir_dns_failed 0 0.00 DNS director failed lookups
dir_dns_hit 0 0.00 DNS director cached lookups hit
dir_dns_cache_full 0 0.00 DNS director full dnscache
fetch_1xx 0 0.00 Fetch no body (1xx)
fetch_204 0 0.00 Fetch no body (204)
fetch_304 0 0.00 Fetch no body (304)
being called as
/usr/sbin/varnishd -n pavodo -P /var/run/varnish-pv.pid -a
93.188.179.124:80 -f /etc/varnish/pavodo.vcl -T 127.0.0.1:6083 -t 120 -w
4,1000,120 -u varnish -g varnish -S /etc/varnish/secret -s
file,/var/lib/varnish/varnish_storage.bin,1G
More information about the varnish-misc
mailing list