[Varnish] #410: Varnish uses up all system RAM on OpenVZ VPS
Varnish
varnish-bugs at projects.linpro.no
Sat Jan 3 00:08:38 CET 2009
#410: Varnish uses up all system RAM on OpenVZ VPS
---------------------+------------------------------------------------------
Reporter: eugaia | Owner:
Type: defect | Status: reopened
Priority: normal | Milestone:
Component: build | Version: 2.0
Severity: blocker | Resolution:
Keywords: |
---------------------+------------------------------------------------------
Changes (by eugaia):
* status: closed => reopened
* resolution: worksforme =>
Comment:
Replying to [comment:2 phk]:
> I'm slightly surprised that Varnish would use 8 times as much RAM as you
have given it storage, but absent information about what amount of traffic
you throw at it or any other details, it is awfully hard for me to say
something intelligent.
So was I!
I couldn't throw any traffic at Varnish. It basically rendered my VPS
(i.e. the whole server) unusable when it started because it used up so
much RAM, and I had to restart the server.
I can get Varnish working perfectly ok on my home computer, and I have
exactly the same configuration installed on my remote server (though they
are compiled separately). I am only experiencing a problem on the remote
server, which is a virtual private server, and therefore using a
virtualized OS.
I tried again, setting the SHM file to 10MB. Again it used up 875 of 883MB
(there were 34MB used before I launched Varnish).
I believe that it probably has to do with the virtualized environment that
is used on OpenVZ, and how Varnish's SHM operations work with it.
I don't know what information you're looking for. With a 10MB SHM file
(I've not tried with higher), I am able to log into the admin server using
telnet. I can start/stop Varnish (when I stop it, the RAM usage goes down
from 881MB to 30MB). The stats for two requests reads as follows:
==================
4 Client connections accepted
4 Client requests received
0 Cache hits
0 Cache hits for pass
4 Cache misses
0 Backend connections success
0 Backend connections not attempted
0 Backend connections too many
4 Backend connections failures
0 Backend connections reuses
0 Backend connections recycles
0 Backend connections unused
1 N struct srcaddr
0 N active struct srcaddr
2 N struct sess_mem
1 N struct sess
0 N struct object
2 N struct objecthead
1 N struct smf
0 N small free smf
1 N large free smf
0 N struct vbe_conn
1 N struct bereq
80 N worker threads
80 N worker threads created
769 N worker threads not created
0 N worker threads limited
0 N queued work requests
0 N overflowed work requests
0 N dropped work requests
1 N backends
0 N expired objects
0 N LRU nuked objects
0 N LRU saved objects
0 N LRU moved objects
0 N objects on deathrow
0 HTTP header overflows
0 Objects sent with sendfile
4 Objects sent with write
0 Objects overflowing workspace
4 Total Sessions
4 Total Requests
0 Total pipe
0 Total pass
0 Total fetch
676 Total header bytes
1808 Total body bytes
4 Session Closed
0 Session Pipeline
0 Session Read Ahead
0 Session Linger
0 Session herd
1105 SHM records
980 SHM writes
0 SHM flushes due to overflow
0 SHM MTX contention
0 SHM cycles through buffer
8 allocator requests
0 outstanding allocations
0 bytes allocated
10485760 bytes free
0 SMA allocator requests
0 SMA outstanding allocations
0 SMA outstanding bytes
0 SMA bytes allocated
0 SMA bytes free
4 SMS allocator requests
0 SMS outstanding allocations
0 SMS outstanding bytes
1808 SMS bytes allocated
1808 SMS bytes freed
0 Backend requests made
1 N vcl total
1 N vcl available
0 N vcl discarded
1 N total active purges
1 N new purges added
0 N old purges deleted
0 N objects tested
0 N regexps tested against
0 N duplicate purges removed
===========================
The VCL file is:
===========================
backend default {
.host = "127.0.0.1";
.port = "8000";
}
sub vcl_recv {
if (req.http.host ~ "^www.simpl.it$") {
set req.http.host = "simpl.it";
error 301;
}
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
remove req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
remove req.http.Accept-Encoding;
}
}
if (req.http.host == "simpl.it" || req.http.host == "simpl") {
if (req.url ~ "^/__/") {
error 403;
}
if (req.url ~ "\.(png|jpg|gif|js|css)$") {
lookup;
}
if (req.http.Cookie ~ "lang=[^;]+;" && ! req.url ~
"lang=[^;]+" ) {
set req.url = req.url regsub (req.http.Cookie,
"lang=([^;]+)(;.*)?", "&lang=\1");
}
if (req.http.Cookie ~ "ssid=[^;]+" ) {
set req.url = req.url regsub (req.http.Cookie,
"ssid=([^;]+)(;.*)?", "&ssid=\1");
}
if (req.request != "GET" && req.request != "HEAD" ) {
set req.url = "/__/page.php?url=" req.url;
pass;
}
}
lookup;
}
sub vcl_error {
if ( obj.status == 301 ) {
set obj.http.Location = "http://" req.http.host req.url;
}
}
sub vcl_miss {
if (req.http.host == "simpl.it" || req.http.host == "simpl") {
if (req.url ~ "^/__/") {
error 404;
}
if (req.url ~ "^/_/images/text/") {
if ( ! req.url ~
"^/_/images/text/[^/]+/[^/]+/[^/]+/[^/]+/[^/]+$" ) {
error 404;
}
set bereq.url = "/__/text_image.php?" regsub (
req.url , "^/_/images/text/([^/]+)/([^/]+)/([^/]+)/([^/]+)/([^/]+)$" ,
"font=\1&size=\2&col=\3&bg=\4&word=\5" );
remove bereq.http.Cookie;
### CHANGE - send to PBMS (automatically?) - but
cache them for say 30mins - so that cache doesn't become ridiculously
large with text images
} elsif (! req.url ~ "\.(png|jpg|gif|js|css)$") {
set bereq.url = "/__/page.php?url=" req.url;
}
}
fetch;
}
sub vcl_fetch {
if ( ! obj.http.Cache-Control && ! obj.http.Expires ) {
set obj.ttl = 10s; ### CHANGE - set to a
higher amount later
}
if (req.http.host ~ "boslowen") {
set obj.ttl = 1h;
}
deliver;
}
sub vcl_deliver {
remove resp.http.Age;
remove resp.http.Server;
remove resp.http.Via;
remove resp.http.X-Varnish;
}
=======================
What other information would be useful to you?
--
Ticket URL: <http://varnish.projects.linpro.no/ticket/410#comment:3>
Varnish <http://varnish.projects.linpro.no/>
The Varnish HTTP Accelerator
More information about the varnish-bugs
mailing list