Varnish logging and data merging

[pub crawler]

Thanks again Rob!

Tracking pixel it is then :)  Off to cobble some code together and test.


On Mon, Jan 11, 2010 at 3:42 AM, Rob S <rtshilston at gmail.com> wrote:
> pub crawler wrote:
>>
>> We have a lot of logging going on in our applications. Logs pages, IP
>> info, time date, URL parameters, etc.  Since many pages are being
>> served out of Varnish cache,  they don't get logged by our
>> application.
>>
>> How is anyone else out there working around this sort of problem with
>> an existing application?  Considering a 1x1 graphic file inclusion
>> into our pages to facilitate logging and ensuring Varnish doesn't
>> cache it
>
> We've not reported analytics based on raw Apache logs for a long time -
> they're far too polluted by spiders etc.  So, instead, for RSS we use
> Feedburner (who provide stats, and reduce our traffic load), and for general
> web access we either use Google Analytics (or commercial alternatives).
>  Google Analytics allows you to add and report on custom parameters, so it
> is very flexible.  However, as you suggested, an alternative is to use a
> tracking pixel.  Depending on how you've previously processed your
> information, you may find it very useful to encourage Varnish to route all
> hits to your tracking pixel to a specific server (obviously with failover).
>  This'd save you having to aggregate logs across multiple servers.
>
> Rob
>