reg: not getting ReqAcct tag in varnishlog

[Hardik]

Hi Dridi / Team,

Gentle reminder..

Thank you
Hardik

On Thu, 4 Apr 2019 at 11:33, Hardik <hetardik.p at gmail.com> wrote:

> Thanks Dridi.
>
> I was ill and off work for few days so did not reply.
>
> Hey Dridi, I am already using varnishncsa. And use case perspective I am
> much familiar. I am using below configuration for varnishncsa. This logs I
> am calling as raw-logs/Apache and providing to customer directly without
> any modification.Means no processing on this.
>
> #the format of the log lines
> FORMAT='-F "%h %v %u %t \"%m %U%q %H\" %s %O \"%{Referer}i\"
> \"%{User-agent}i\" %S [%{VCL_Log:x-product}x lg=\"%{Accept-Language}i\"
> fwd=\"%{X-Forwarded-For}i\" %{VCL_Log:namespace}x
> %{VCL_Log:x-cache-hit}x/%{VCL_Log:x-revalidate-cache}x
> %{VCL_Log:sc-substatus}x]
>
> Note- few parameter I have added in varnishncsa to get that information
> ex:%S.
>
> I have written other utility because want to parse all the fields based on
> billing requirements. I have tried removing "-g session" option and now log
> loss reduced a lot(I can say 95%).
>
> If you can help on my previous reply/doubts then will be really helpful (
> because billing really difficult to change billing setup).I am only facing
> two issues here, 1. log loss 2. missing of ReqAcct ( which is also case in
> varnishncsa).
>
> If I am doing something wrong other then "-g session" option in the code I
> have pasted before from utility please point out.
>
> Thank you
> Hardik
>
> On Wed, 20 Mar 2019 at 21:02, Dridi Boukelmoune <dridi at varni.sh> wrote:
>
>> On Wed, Mar 20, 2019 at 10:40 AM Hardik <hetardik.p at gmail.com> wrote:
>> >
>> > Hi Dridi,
>> >
>> > Do you need all timestamps or a specific metric?
>> > Regarding timestamp, want to read two tags,
>> > Timestamp      Start: 1516269224.184112 0.000000 0.000000
>> > Timestamp      Resp: 1516269224.184920 0.000808 0.000087
>> >
>> > Do you need the BereqAcct records for all transactions? Including cache
>> hits?
>> > Sorry it is my mistake. I am not reading any of the beck-end records.
>> So can ignore BereqAcct.
>> > I need fields from Req records only.
>>
>> Ok, in this case you can probably get away with just varnishncsa to
>> collect all you need.
>>
>> No grouping (the default -g vxid), client mode (-c) only, with a
>> custom -F format to grab only what you need.
>>
>> This should help reduce the churn to the point where you lose data.
>>
>> If you struggle with this, I can help you later with that, but start
>> by reading the following manuals:
>>
>> - varnishncsa
>> - vsl
>> - vsl-query
>>
>> For example, the format for the timestamps you wish to collect would
>> look like this:
>>
>> > %{VSL:Timestamp:Start[1]}x %{VSL:Timestamp:Resp[2]}x
>> %{VSL:Timestamp:Resp[3]}x
>>
>> Rinse and repeat for all the things you need to capture for the logs,
>> put them in the order you prefer and off you go. No need to write your
>> own utility.
>>
>> > What does FD mean here? File descriptor? From ReqStart?
>> > Yes, Its file descriptor. And yes reading from ReqStart till ReqAcct.
>> Using switch case to read needed records.
>>
>> If you already work with VXIDs, the FD becomes redundant.
>>
>>
>>
>> Dridi
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20190411/90ebd951/attachment-0001.html>