Hey Kristian,<br><br>Thanks for your answer, it was really clarifying. <br><br>2011/10/14 Kristian Lyngstol <span dir="ltr"><<a href="mailto:kristian@varnish-software.com">kristian@varnish-software.com</a>></span><br>
<div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Greetings,<br>
<div class="im"><br>
On Thu, Oct 13, 2011 at 02:20:48PM -0300, Thiago Moraes wrote:<br>
> I have one server that provides access to some hundreds of files. All<br>
> of them are really big (some have more than 10GB). These files won't<br>
> change and are read only, but I'll need to provide access to them via<br>
> a WAN.<br>
><br>
> I want to make the access faster by using a reverse proxy server<br>
> running near my users. For example:<br>
><br>
> -User X wants to access something on server A.<br>
> -User X access a reverse proxy server on his LAN which causes a cache<br>
> miss. The file is downloaded to this proxy server.<br>
> -The next time user X wants the same file, he doesn't need to go get<br>
> it in my main server.<br>
<br>
</div>My first thought is why you need a reverse proxy and not a regular one.<br></blockquote><div><br>I'm not really experienced in this kind of work so the choice was made based on what I read about cache servers in the past weeks. As the server would only cache data from my application, it appeared to me that the reverse proxy alternative would be less "intrusive". I don't have the power to change the network topology from my users so I don't want to redirect all their traffic through my proxy. Probably, there already exists a proxy server in their networks. Am I missing something with this approach? Is it necessary?<br>
</div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div class="im"><br>
> I read something about Squid having problems caching files larger than<br>
> 2GB. Does Varnish face the same kind of problem?<br>
<br>
</div>Short answer: No.<br>
<br>
>From Varnish 3, we do regular tests to see that files bigger than 4GB<br>
are dealt with sensibly. 2/4GB being the barrier, and beyond that, the<br>
logical limit is your memory manager/hardware/network/client, etc. 10GB<br>
/shouldn't/ be a big problem.<br>
<br>
But keep in mind:<br>
<br>
- You need a 64bit system.<br>
- Use the latest Varnish version. 3.0.1 or even 3.0.2 when that<br>
finishes. 2.1 is not recommended.<br></blockquote><div><br>As the project is new I'll use the current stable version, probably. Are there any advantages for me to use 3.0.2 right now? How's the stability of the developer's version? Would you recommendo to start already using it?<br>
</div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
- If your users will access these files concurrently, you will really<br>
want to try out Martin's streaming branch, as that will ensure that<br>
multiple clients get the content as it arrives at the cache. Without<br>
Martin's branch, the first user that gets a cache miss will get the<br>
content "streamed" while Varnish downloads it, while other users have<br>
to wait for the download to finish before they get any data. </blockquote><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
- If your data is compressible, make sure you use gzip. Even if your<br>
clients don't support it (e.g: scripts?), Varnish does. If it isn't<br>
compressible, consider disabling it.<br></blockquote><div><br>I don't really know how to answer that, but I'll ask you something about this anyway=p.<br>My application is a server with a REST interface to a bunch of scientific data files. The interface allows you to slice the data and to get only a subset of it transparently. As the server was developed by someone else, I don't know how it handles data compression, but as it is possible to access data through some really ugly applications, I believe it's not possible to use compression due to the clients.<br>
<br>How would varnish compression work? It would send compressed data to the cache server and them uncompress it before sending to the clients?<br> </div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
- Make sure the cache is big enough, or that evictions are<br>
well-controlled. If you're mixing small and large files, you might end<br>
up evicting hundreds of small files to fit room for a single large<br>
one.<br></blockquote><div><br>I'll have mostly large files. There are some little ones with just a few MB, but they're the exception in my case.<br> </div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
With all of that said, I'd like to emphasis that support for large<br>
files is relatively new. I expect there's room for optimizations.<br>
<br>
The less used features are also the less optimized features. As a<br>
consequence of the lack of feedback.<br>
<br>
I would greatly appreciate any feedback you have if/when you test this.<br>
<font color="#888888"><br>
- Kristian<br>
<br>
</font></blockquote></div><br>I'll provide some feedback when I get it. I'm learning how to configure varnish right now and figuring out some way of testing if my requisites are fulfilled. By the way, would you recommend some way to test the server? Are there any automated tool to do this kind of test?<br>
<br>Thank you very much,<br><br>Thiago Moraes - EnC07 - UFSCar<br><br>