bitrate and memory allocated

Miguel González miguel_3_gonzalez at yahoo.es
Sat Sep 3 12:14:39 CEST 2016


Why I don´t get a copy of my messages and I do get the rest of emails? I
have just double checked my settings and I should get a copy of my messages

Miguel

On 09/03/16 11:59 AM, Miguel González wrote:
> Dear all,
> 
>   I have Varnish 4.0.3 in front of a WHM/Cpanel hosting around 40
> Wordpress sites. Only 4-5 sites are quite visited but the rest are blogs
> that don´t have many visits.
> 
>   Varnish seems to work fine, I have increased the RAM of the VPS from 7
> to 21 Gb of RAM because from time to time the RAM of the system got
> exhausted and I had to restart the OS.
> 
>   I have signed up in datadoghq to get a handle of how well Varnish is
> doing and try to squeeze the performance of it. Mostly because I have
> realized that when I test any site with a tool like pingdom, the first
> test gets a worse performance than consecutive tests (when I assume
> Varnish starts caching).
> 
>   I have assigned in malloc 6 Gb of RAM but I have been checking the
> numbers provided by varnishstat and after 12 hours I don´t even get
> close to 1 Gb of RAM (I assume c_bytes is what stands for) used by varnish:
> 
>  varnishstat -1 |grep SMA
> SMA.s0.c_req                             33969         0.76 Allocator
> requests
> SMA.s0.c_fail                                0         0.00 Allocator
> failures
> SMA.s0.c_bytes                       984052835     21896.06 Bytes allocated
> SMA.s0.c_freed                       395844358      8807.89 Bytes freed
> SMA.s0.g_alloc                           19833          .   Allocations
> outstanding
> SMA.s0.g_bytes                       588208477          .   Bytes
> outstanding
> SMA.s0.g_space                      5854242467          .   Bytes available
> 
>  I had a ttl of 2 hours and now I have changed it to 24 hours, maybe I
> will increase it if I see the RAM allocated to Varnish is not entirely
> used. The bitrate is erratic, reaching almost 1 and then dropping to
> less than 0.5.
> 
> Are my assumptions right?
> 
> Another question, I know that running varnishtop I get the current most
> visited site, but how do I aggregate that info over the hours? I would
> like to get a picture of which are the most used sites (I´m considering
> separating storages and dedicate only RAM for the most used and have
> disk storage for the rest).
> 
> Thanks in advance
> 
> Miguel
> 
> ps: Here is my default.vcl
> 
> #
> # This is an example VCL file for Varnish.
> #
> # It does not do anything by default, delegating control to the
> # builtin VCL. The builtin VCL is called when there is no explicit
> # return statement.
> #
> # See the VCL chapters in the Users Guide at
> https://www.varnish-cache.org/docs/
> # and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.
> 
> # Marker to tell the VCL compiler that this VCL has been adapted to the
> # new 4.0 format.
> vcl 4.0;
> 
> import std;
> 
> # Default backend definition. Set this to point to your content server.
> backend default {
>     .host = "MYIP";
>     .port = "82";
>     .connect_timeout = 600s;
>     .first_byte_timeout = 600s;
>     .between_bytes_timeout = 600s;
> 
> 
> }
> 
> acl purge {
>         "localhost";
>         "127.0.0.1";
>         "MYIP";
> }
> 
> # This function is used when a request is send by a HTTP client (Browser)
> sub vcl_recv {
> 
>         # remove ?ver=xxxxx strings from urls so css and js files are
> cached.
>         # Watch out when upgrading WordPress, need to restart Varnish or
> flush cache.
>         set req.url = regsub(req.url, "\?ver=.*$", "");
> 
>         # Remove "replytocom" from requests to make caching better.
>         set req.url = regsub(req.url, "\?replytocom=.*$", "");
> 
>         #We pass real IP to the backend
> 
>         if (req.restarts == 0) {
>             if (req.http.X-Forwarded-For) {
>                set req.http.X-Forwarded-For = req.http.X-Forwarded-For +
> ", " + client.ip;
>            } else {
>             set req.http.X-Forwarded-For = client.ip;
>            }
>         }
> 
> 
>         # Normalize the header, remove the port (in case you're testing
> this on various TCP ports)
> 
>         set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
> 
>         # Remove has_js and CloudFlare/Google Analytics __* cookies.
>         set req.http.Cookie = regsuball(req.http.Cookie,
> "(^|;\s*)(_[_a-z]+|has_js)=[^;]*", "");
>         # Remove a ";" prefix, if present.
>         set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
> 
> 
>         # Allow purging from ACL
>         if (req.method == "PURGE") {
>                 # If not allowed then a error 405 is returned
>                 if (!client.ip ~ purge) {
>                         return(synth(405, "This IP is not allowed to
> send PURGE requests."));
>                 }
>                 # If allowed, do a cache_lookup -> vlc_hit() or vlc_miss()
>                 return (purge);
>         }
> 
>         # Post requests will not be cached
>         #if (req.http.Authorization || req.method == "POST") {
>         #       return (pass);
>         #}
> 
>         # Pass anything other than GET and HEAD directly.
>         if (req.method != "GET" && req.method != "HEAD") {
>                 return( pass );
>         }      /* We only deal with GET and HEAD by default */
> 
>         # --- WordPress specific configuration
> 
>         # Did not cache the admin and login pages
>         if (req.url ~
> "nocache|cart|my-account|checkout|addons|tienda|carro|wp-admin|wp-(comments-post|login|signup|activate|mail|cron)\.php|preview\=true|admin-ajax\.php|xmlrpc\.php|bb-admin|whm-server-status|server-status|control\.php|bb-login\.php|bb-reset-password\.php|register\.php")
> {
>             return (pass);
>         }
> 
>         if (req.url ~ "(ajax|dynamic|custom)") {
>              return(pass);
>         }
> 
>         # Remove the "has_js" cookie
>         set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(;
> )?", "");
> 
>         # Remove any Google Analytics based cookies
>         set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(;
> )?", "");
> 
>         # Remove the Quant Capital cookies (added by some plugin, all __qca)
>         set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(;
> )?", "");
> 
>         # Remove the wp-settings-1 cookie
>         set req.http.Cookie = regsuball(req.http.Cookie,
> "wp-settings-1=[^;]+(; )?", "");
> 
>         # Remove the wp-settings-time-1 cookie
>         set req.http.Cookie = regsuball(req.http.Cookie,
> "wp-settings-time-1=[^;]+(; )?", "");
> 
>         # Remove the wp test cookie
>         set req.http.Cookie = regsuball(req.http.Cookie,
> "wordpress_test_cookie=[^;]+(; )?", "");
> 
>         # Are there cookies left with only spaces or that are empty?
>         if (req.http.cookie ~ "^ *$") {
>                     unset req.http.cookie;
>         }
> 
>         # Cache the following files extensions
>         if (req.url ~ "\.(txt|css|js|png|gif|jp(e)?g|swf|ico)") {
>                 unset req.http.cookie;
>         }
> 
>         # Normalize Accept-Encoding header and compression
>         # https://www.varnish-cache.org/docs/3.0/tutorial/vary.html
>         if (req.http.Accept-Encoding) {
>                 # Do no compress compressed files...
>                 if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
>                                 unset req.http.Accept-Encoding;
>                 } elsif (req.http.Accept-Encoding ~ "gzip") {
>                         set req.http.Accept-Encoding = "gzip";
>                 } elsif (req.http.Accept-Encoding ~ "deflate") {
>                         set req.http.Accept-Encoding = "deflate";
>                 } else {
>                         unset req.http.Accept-Encoding;
>                 }
>         }
> 
>         # Check the cookies for wordpress-specific items
>         if (req.http.Cookie ~ "wordpress_" || req.http.Cookie ~
> "comment_") {
>                 return (pass);
>         }
>         if (!req.http.cookie) {
>                 unset req.http.cookie;
>         }
> 
>         # --- End of WordPress specific configuration
> 
>         # Did not cache HTTP authentication and HTTP Cookie
>         if (req.http.Authorization || req.http.Cookie) {
>                 # Not cacheable by default
>                 return (pass);
>         }
> 
>         # Cache all others requests
>         return (hash);
> }
> 
> sub vcl_pipe {
>         return (pipe);
> }
> 
> sub vcl_pass {
>         return (fetch);
> }
> 
> # The data on which the hashing will take place
> sub vcl_hash {
>         hash_data(req.url);
>         if (req.http.host) {
>         hash_data(req.http.host);
>         } else {
>         hash_data(server.ip);
>         }
> 
>         # If the client supports compression, keep that in a different cache
>         if (req.http.Accept-Encoding) {
>                 hash_data(req.http.Accept-Encoding);
>         }
> 
>         return (lookup);
> }
> 
> # This function is used when a request is sent by our backend (Nginx server)
> sub vcl_backend_response {
>         # Remove some headers we never want to see
>         unset beresp.http.Server;
>         unset beresp.http.X-Powered-By;
> 
>         # For static content strip all backend cookies
>         if (bereq.url ~ "\.(css|js|png|gif|jp(e?)g)|swf|ico") {
>                 unset beresp.http.cookie;
>         }
>         # Don't store backend
>         if (bereq.url ~ "wp-(login|admin)" || bereq.url ~ "preview=true") {
>                 set beresp.uncacheable = true;
>                 set beresp.ttl = 30s;
>                 return (deliver);
>         }
> 
>         # Only allow cookies to be set if we're in admin area
>                 if (!(bereq.url ~
> "(wp-login|cart|my-account|checkout|addons|tienda|carro|wp-admin|preview=true)"))
> {
>                 unset beresp.http.set-cookie;
>         }
> 
>         # don't cache response to posted requests or those with basic auth
>         if ( bereq.method == "POST" || bereq.http.Authorization ) {
>                 set beresp.uncacheable = true;
>                 set beresp.ttl = 120s;
>                 return (deliver);
>         }
> 
>         # don't cache search results
>         if ( bereq.url ~ "\?s=" ){
>                 set beresp.uncacheable = true;
>                 set beresp.ttl = 120s;
>                 return (deliver);
>         }
> 
>         # only cache status ok
>         if ( beresp.status != 200 ) {
>                 set beresp.uncacheable = true;
>                 set beresp.ttl = 120s;
>                 return (deliver);
>         }
> 
>         # A TTL of 24h
>         set beresp.ttl = 24h;
>         # Define the default grace period to serve cached content
>         #set beresp.grace = 30s;
>         set beresp.grace = 1h;
> 
>         return (deliver);
> }
> 
> # The routine when we deliver the HTTP request to the user
> # Last chance to modify headers that are sent to the client
> sub vcl_deliver {
>         if (obj.hits > 0) {
>                 set resp.http.X-Cache = "cached";
>         } else {
>                 set resp.http.x-Cache = "uncached";
>         }
> 
>         # Remove some headers: PHP version
>         unset resp.http.X-Powered-By;
> 
>         # Remove some headers: Apache version & OS
>         unset resp.http.Server;
> 
>         # Remove some heanders: Varnish
>         unset resp.http.Via;
>         unset resp.http.X-Varnish;
> 
>         return (deliver);
> }
> 
> sub vcl_init {
>         return (ok);
> }
> 
> sub vcl_fini {
>         return (ok);
> }
> 
> 
> 
> 
> 
> 
> 
> 




More information about the varnish-misc mailing list