<div dir="ltr"><div>Hi,</div><div>I'm sorry, I wasn't being clear. Thank you for your patience.</div><div>I'm trying to estimate the response header from backend, if the "content-type:" is " image/*", we know that the response body is image. Then I'll rewrite</div>
<div> the image and insert it into varnish cache. Subsequent request will hit the rewritten image.</div><div>That's I wanna to do. Fetch backend response, rewrite image and insert into varnish cache.</div><div><br></div>
<div>Regards,</div><div>--</div><div>Shawn Wang</div><div style><br></div><div style><font color="#000000" face="Consolas, Lucida Console, monospace"><span style="font-size:12px;white-space:pre-wrap"><br></span></font></div>
<div style><font color="#000000" face="Consolas, Lucida Console, monospace"><span style="font-size:12px;white-space:pre-wrap"><br></span></font></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/2/1 Per Buer <span dir="ltr"><<a href="mailto:perbu@varnish-software.com" target="_blank">perbu@varnish-software.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>I don't quite understand what you're trying to do. Varnish will store the jpg together with the response headers in memory. When you request the object Varnish will deliver it verbatim along with the HTTP headers. What exactly are you trying to do?</div>
<div><br></div><div><br></div><div>PS: I see we haven't built packages of 3.0.3-plus, yet. This should pop up in the repo next week. Until then 3.0.2s might suffice.</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra">
<br><br><div class="gmail_quote">On Fri, Feb 1, 2013 at 8:01 AM, Xianzhe Wang <span dir="ltr"><<a href="mailto:wxz19861013@gmail.com" target="_blank">wxz19861013@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div><div> Hi,</div><div>Thanks for clarification. What you say is very clear.</div><div>I am sorry to show my poor English, but I have tried my best to communicate.</div><div><br></div><div>There is another question.For example, if we request a .jpg file(cacheable), varnish will encapsulation it as an object and insert into memory. How can we get the .jpg file from the object?</div>
<div><br></div><div>Thank you for help again.</div><div><br></div><div>-Shawn Wang</div></div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/1/30 Per Buer <span dir="ltr"><<a href="mailto:perbu@varnish-software.com" target="_blank">perbu@varnish-software.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi,<br></div><div><br></div><div>I was a bit quick and I didn't read the whole email the first time. Sorry about that. You're actually using the streaming branch, already I see. What you're writing is really, really odd. There is a slight lock while the "first" object is being fetched where other requests will be put on the waiting list. However, when the hit-for-pass object is created these should be released and pass'ed to the clients.<div>
<div><div><br></div><div>If the backend takes forever coming back with the response headers then the situation would be something like what you describe. However, that would be odd and doesn't make much sense.</div>
<div><br></div><div>PS: The streaming branch was renamed "plus" when it got other experimental features. You'll find source <a href="https://github.com/mbgrydeland/varnish-cache" target="_blank">https://github.com/mbgrydeland/varnish-cache</a> and packages at <a href="http://repo.varnish-cache.org/test" target="_blank">repo.varnish-cache.org/test</a> if I recall correctly.</div>
<div><br></div><div><br></div></div></div></div></div><div class="gmail_extra"><div><div><br><br><div class="gmail_quote">On Wed, Jan 30, 2013 at 3:32 AM, Xianzhe Wang <span dir="ltr"><<a href="mailto:wxz19861013@gmail.com" target="_blank">wxz19861013@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><font face="arial, sans-serif"><div><span style="font-size:14px">Hi,</span></div><div><span style="font-size:14px">Thanks a lot.</span></div>
<div><span style="font-size:14px"><br></span></div><div><span style="font-size:14px">I tried option </span></div><div>
<div><span style="font-size:14px">"set req.hash_ignore_busy = true;"</span></div><div><span style="font-size:14px">in vlc_recv. </span></div></div><div><span style="font-size:14px">I think it works. But there are side effects: it would increase backend load.</span></div>
<div><span style="font-size:14px"><br></span></div><div><span style="font-size:14px">I have an idea about it in my previous email. what do you think about it?</span></div><div><span style="font-size:14px"><br></span></div>
<div><span style="font-size:14px">Another question is that where can I find the "plus" branch of Varnish which matches this issue.</span></div><div><div><span style="font-size:14px"><br></span></div>
<div><span style="font-size:14px"> Any suggestions will be appreciate. </span></div>
</div><div><span style="font-size:14px">Thanks again for help.</span></div><div><span style="font-size:14px"><br></span></div><div><span style="font-size:14px">Regards,</span></div><div><span style="font-size:14px">--</span></div>
<div><span style="font-size:14px">Shawn Wang</span></div><div><span style="font-size:14px"><br></span></div></font><div><div><div><div><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Xianzhe Wang</b> <span dir="ltr"><<a href="mailto:wxz19861013@gmail.com" target="_blank">wxz19861013@gmail.com</a>></span><br>
Date: 2013/1/30<br>Subject: Re: How to make multiple clients can get the response at the same time by stream.<br>To: Jakub Słociński <<a href="mailto:kuba@ovh.net" target="_blank">kuba@ovh.net</a>><br><br><br><div dir="ltr">
<div>Hi Jakub S.</div>
<div>Thank you very much.</div><div>I tried, and take a simple test, two client request the big file at the same time, they get the response stream immediately, so it works.</div><div>
In that case, multiple requests will go directly to "pass", they do not need to wait, but it would increase backend load. </div><div>We need to balance the benefits and drawbacks.</div><div><br></div><div>I wanna is that:</div>
<div> Client 1 requests url /foo</div><div> Client 2..N request url /foo</div><div> Varnish tasks a worker to fetch /foo for Client 1</div><div> Client 2..N are now queued pending response from the worker</div>
<div> Worker fetch response header(just header not include body) from backend, and find it non-cacheable, then make the remaining requests(Client 2..N) go directly to "pass". And creat the hit_for_pass object synchronously in the first request(Client 1).</div>
<div> Subsequent requests are now given the hit_for_pass object instructing them to go to the backend as long as the hit_for_pass object exists.</div><div><br></div><div>As I mentioned below, is it feasible? Or do you have any Suggestions? </div>
<div><br></div><div>Thanks again for help.</div><div><br></div><div>Regards,</div><div>--</div><div>Shawn Wang</div><div><div style="text-align:justify">
</div><div style="text-align:justify"><font color="#444444" face="sans-serif"><span style="font-size:17px;line-height:21px"><br></span></font></div></div></div><div><div><div><div><div class="gmail_extra"><br>
<br>
<div class="gmail_quote">2013/1/29 Jakub Słociński <span dir="ltr"><<a href="mailto:kuba@ovh.net" target="_blank">kuba@ovh.net</a>></span><br>
<blockquote class="gmail_quote" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hi Xianzhe Wang,<br>
you should try option<br>
"set req.hash_ignore_busy = true;"<br>
in vlc_recv.<br>
<br>
Regards,<br>
--<br>
Jakub S.<br>
<br>
<br>
Xianzhe Wang napisał(a):<br>
<div><div>> Hello everyone,<br>
> My varnish version is 3.0.2-streaming release.And I set<br>
> "beresp.do_stream = true" in vcl_fetch in order to "Deliver the object to<br>
> the client directly without fetching the whole object into varnish";<br>
><br>
> This is a part of my *.vcl file:<br>
><br>
> sub vcl_fetch {<br>
> set beresp.grace = 30m;<br>
><br>
> set beresp.do_stream = true;<br>
><br>
> if (beresp.http.Content-Length && beresp.http.Content-Length ~<br>
> "[0-9]{8,}") {<br>
> return (hit_for_pass);<br>
> }<br>
><br>
> if (beresp.http.Pragma ~ "no-cache" || beresp.http.Cache-Control ~<br>
> "no-cache" || beresp.http.Cache-Control ~ "private") {<br>
> return (hit_for_pass);<br>
> }<br>
><br>
> if (beresp.ttl <= 0s ||<br>
> beresp.http.Set-Cookie ||<br>
> beresp.http.Vary == "*") {<br>
><br>
> set beresp.ttl = 120 s;<br>
> return (hit_for_pass);<br>
> }<br>
><br>
> return (deliver);<br>
> }<br>
><br>
> Then I request a big file(about 100M+) like "xxx.zip" from clients.There is<br>
> only one client can access the object.because "the object will marked as<br>
> busy as it is delivered."<br>
><br>
> But if the request goes directly to “pass” ,multiple clients can get the<br>
> response at the same time.<br>
><br>
> Also if I remove<br>
> if (beresp.http.Content-Length && beresp.http.Content-Length ~<br>
> "[0-9]{8,}") {<br>
> return (hit_for_pass);<br>
> }<br>
> to make the file cacheable,multiple clients can get the response at the<br>
> same time.<br>
><br>
> Now I want "multiple clients can get the response at the same time." in all<br>
> situations("pass","hit","hit_for_pass").<br>
><br>
> What can I do for it?<br>
> Any suggestions will be appreciate.<br>
> Thank you.<br>
><br>
> -Shawn Wang<br>
<br>
</div></div>> _______________________________________________<br>
> varnish-misc mailing list<br>
> <a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br>
> <a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br>
<br>
</blockquote></div><br></div>
</div></div></div></div></div><br></div></div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span><font color="#888888">-- <br></font></span><div style="color:rgb(34,34,34);font-size:12.727272033691406px;font-family:arial,sans-serif;margin:5px 15px 0px 0px;padding-bottom:5px">
<span><font color="#888888">
</font></span><span><font color="#888888">
</font></span><span><font color="#888888"></font></span><table border="0" cellpadding="0" cellspacing="0" style="text-align:start;font-size:12px;line-height:1.5em;font-family:'Helvetica Neue',Arial,sans-serif;color:rgb(102,102,102);width:550px;border-top-width:1px;border-top-style:solid;border-top-color:rgb(238,238,238);border-bottom-width:1px;border-bottom-style:solid;border-bottom-color:rgb(238,238,238);margin-top:20px;padding-top:5px;padding-bottom:5px">
<tbody><tr><td width="100" style="font-family:arial,sans-serif;margin:0px"><a href="http://www.varnish-software.com/" style="color:rgb(17,85,204)" target="_blank"><img style="float:left;margin-left:10px"></a></td>
<td style="font-family:arial,sans-serif;margin:0px"><span><font color="#888888"><b style="font-size:14px;color:rgb(34,34,34)">Per Buer</b></font></span><div><br>CEO | Varnish Software AS<br>Phone: <a href="tel:%2B47%20958%2039%20117" value="+4795839117" target="_blank">+47 958 39 117</a> | Skype: per.buer<br>
<span style="font-weight:bold">We Make Websites Fly!<br>
<br></span></div></td></tr></tbody></table><div></div></div><div style="clear:both;color:rgb(34,34,34);font-size:12.727272033691406px;font-family:arial,sans-serif;margin:15px 0px"></div>
</div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div style="color:rgb(34,34,34);font-size:12.727272033691406px;font-family:arial,sans-serif;margin:5px 15px 0px 0px;padding-bottom:5px">
<table border="0" cellpadding="0" cellspacing="0" style="text-align:start;font-size:12px;line-height:1.5em;font-family:'Helvetica Neue',Arial,sans-serif;color:rgb(102,102,102);width:550px;border-top-width:1px;border-top-style:solid;border-top-color:rgb(238,238,238);border-bottom-width:1px;border-bottom-style:solid;border-bottom-color:rgb(238,238,238);margin-top:20px;padding-top:5px;padding-bottom:5px">
<tbody><tr><td width="100" style="font-family:arial,sans-serif;margin:0px"><a href="http://www.varnish-software.com/" style="color:rgb(17,85,204)" target="_blank"><img style="float:left;margin-left:10px"></a></td>
<td style="font-family:arial,sans-serif;margin:0px"><b style="font-size:14px;color:rgb(34,34,34)">Per Buer</b><br>CEO | Varnish Software AS<br>Phone: +47 958 39 117 | Skype: per.buer<br><span style="font-weight:bold">We Make Websites Fly!<br>
<br></span></td></tr></tbody></table><div></div></div><div style="clear:both;color:rgb(34,34,34);font-size:12.727272033691406px;font-family:arial,sans-serif;margin:15px 0px"></div>
</div>
</div></div></blockquote></div><br></div>