Varnish logging and data merging

[Rob S]

pub crawler wrote:
> We have a lot of logging going on in our applications. Logs pages, IP
> info, time date, URL parameters, etc.  Since many pages are being
> served out of Varnish cache,  they don't get logged by our
> application.
>
> How is anyone else out there working around this sort of problem with
> an existing application?  Considering a 1x1 graphic file inclusion
> into our pages to facilitate logging and ensuring Varnish doesn't
> cache it
We've not reported analytics based on raw Apache logs for a long time - 
they're far too polluted by spiders etc.  So, instead, for RSS we use 
Feedburner (who provide stats, and reduce our traffic load), and for 
general web access we either use Google Analytics (or commercial 
alternatives).  Google Analytics allows you to add and report on custom 
parameters, so it is very flexible.  However, as you suggested, an 
alternative is to use a tracking pixel.  Depending on how you've 
previously processed your information, you may find it very useful to 
encourage Varnish to route all hits to your tracking pixel to a specific 
server (obviously with failover).  This'd save you having to aggregate 
logs across multiple servers.

Rob