Debugging of Child dies under heavy load and empty cache

[Audun Ytterdal]

Symtoms: When restarting all frontend varnishes at once they keep 
crashing (child dies,parent lives) with a frequncy of once every second 
to maybe once every 50 seconds. After mabye 10-15 minuttes they stop 
crashing.

Running trunk revision 2519 2524.

Running with -d -d revealed following


Child said (2, 32302): <<Child starts>>
Child said (2, 32302): <<managed to mmap 32212254720 bytes of 32212254720>>
Child said (2, 32302): <<Ready>>
Child said (2, 32302): <<Assert error in SES_Delete(), cache_session.c 
line 340:>>
Child said (2, 32302): <<  Condition((sp->vcl) == 0) not true.>>
Cache child died pid=32302 status=0x6
Clean child
Child cleaned

By changing


AZ(sp->vcl);

to

spassert(sp->vcl == 0)

I got a ton of these

start child pid 30356
Child said (2, 30356): <<Child starts>>
Child said (2, 30356): <<managed to mmap 32212254720 bytes of 32212254720>>
Child said (2, 30356): <<Ready>>
Child said (2, 30356): <<panic in SES_Delete() at cache_session.c:340>>
Child said (2, 30356): <<assertion failed: sp->vcl>>
Child said (2, 30356): <<sp = 0x327cf5ec08 {>>
Child said (2, 30356): <<  fd = -1, id = 1134, xid = 0,>>
Child said (2, 30356): <<  client = 81.166.235.84:13142,>>
Child said (2, 30356): <<  step = STP_DONE,>>
Child said (2, 30356): <<},>>
Cache child died pid=30356 status=0x6
Clean child
Child cleaned

*****************************************************************
Denne fotnoten bekrefter at denne e-postmeldingen ble
skannet av MailSweeper og funnet fri for virus.
*****************************************************************
This footnote confirms that this email message has been 
swept by MailSweeper for the presence of computer viruses.
*****************************************************************