logging problems - "Pipe Shut"
gaute at pht.no
Thu Jul 5 14:21:56 CEST 2007
> Ah, OK, you won't be able to process log files larger than about 2 GB on
> a 32-bit machine. I should probably figure out a way to get around
Ok, now I have set up /var/log/varnish/varnish.log to rotate every hour, and a
postrotate action on that to pipe through varnishncsa and append
to /var/log/varnish/varnish.access_log which in turn gets rotated and
processed by awstats dayly.
That should take care of the 2G limit in varnishncsa at least.
I was hoping it would help with the logging it self but apparently not..
I have about 20 logfiles by now, all under 100M, and have made various
Everything went well during the night. The accesstimes are about a minute past
the hour, and the last date string in the log corresponds with that.
Then at 09:13 suddenly it's all "Pipe Shut read(read)" and "Pipe Shut
write(read)" again. The situation when I noticed this, about 12:30, this was
in 'log.1' whith modtime 09:13 and 'log' was still 0 bytes.
I guess that just means warnishlog had stopped responding to SIGHUP.
Wil try to do a restart insted in postrotate to see if runningtime has
anything to do with it.
That's the logging problem, now for the processing problem:
I made a small script to run varnish(log|ncsa) on every file and get the
returnstatus. log works fine, and a tail of that diplays sensible output and
timestamps, but ncsa segfaults on about half of them.
No pattern discernible..
I tried to make a small python script to feed a log chunk by chunk into ncsa
on stdin, but I was not able to make that work reliably enough to get any
hint what precisely is causing the problem. Don't even know if ncsa can work
Tried to look for the last output lines from nscalogs in the varnishlog as
well, but nothing jumps out so far...
Starting to run out of things to try here...
More information about the varnish-misc