reg: not getting ReqAcct tag in varnishlog

[Hardik]

Thanks Dridi.

I was ill and off work for few days so did not reply.

Hey Dridi, I am already using varnishncsa. And use case perspective I am
much familiar. I am using below configuration for varnishncsa. This logs I
am calling as raw-logs/Apache and providing to customer directly without
any modification.Means no processing on this.

#the format of the log lines
FORMAT='-F "%h %v %u %t \"%m %U%q %H\" %s %O \"%{Referer}i\"
\"%{User-agent}i\" %S [%{VCL_Log:x-product}x lg=\"%{Accept-Language}i\"
fwd=\"%{X-Forwarded-For}i\" %{VCL_Log:namespace}x
%{VCL_Log:x-cache-hit}x/%{VCL_Log:x-revalidate-cache}x
%{VCL_Log:sc-substatus}x]

Note- few parameter I have added in varnishncsa to get that information
ex:%S.

I have written other utility because want to parse all the fields based on
billing requirements. I have tried removing "-g session" option and now log
loss reduced a lot(I can say 95%).

If you can help on my previous reply/doubts then will be really helpful (
because billing really difficult to change billing setup).I am only facing
two issues here, 1. log loss 2. missing of ReqAcct ( which is also case in
varnishncsa).

If I am doing something wrong other then "-g session" option in the code I
have pasted before from utility please point out.

Thank you
Hardik

On Wed, 20 Mar 2019 at 21:02, Dridi Boukelmoune <dridi at varni.sh> wrote:

> On Wed, Mar 20, 2019 at 10:40 AM Hardik <hetardik.p at gmail.com> wrote:
> >
> > Hi Dridi,
> >
> > Do you need all timestamps or a specific metric?
> > Regarding timestamp, want to read two tags,
> > Timestamp      Start: 1516269224.184112 0.000000 0.000000
> > Timestamp      Resp: 1516269224.184920 0.000808 0.000087
> >
> > Do you need the BereqAcct records for all transactions? Including cache
> hits?
> > Sorry it is my mistake. I am not reading any of the beck-end records. So
> can ignore BereqAcct.
> > I need fields from Req records only.
>
> Ok, in this case you can probably get away with just varnishncsa to
> collect all you need.
>
> No grouping (the default -g vxid), client mode (-c) only, with a
> custom -F format to grab only what you need.
>
> This should help reduce the churn to the point where you lose data.
>
> If you struggle with this, I can help you later with that, but start
> by reading the following manuals:
>
> - varnishncsa
> - vsl
> - vsl-query
>
> For example, the format for the timestamps you wish to collect would
> look like this:
>
> > %{VSL:Timestamp:Start[1]}x %{VSL:Timestamp:Resp[2]}x
> %{VSL:Timestamp:Resp[3]}x
>
> Rinse and repeat for all the things you need to capture for the logs,
> put them in the order you prefer and off you go. No need to write your
> own utility.
>
> > What does FD mean here? File descriptor? From ReqStart?
> > Yes, Its file descriptor. And yes reading from ReqStart till ReqAcct.
> Using switch case to read needed records.
>
> If you already work with VXIDs, the FD becomes redundant.
>
>
>
> Dridi
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20190404/d7c599a3/attachment.html>