Varnish CPU Usage
Ayberk Kimsesiz
ayberk.kimsesiz at gmail.com
Tue Aug 2 15:18:15 CEST 2016
I used Default.VCL in two different ways. The first with default settings
and the second with settings customized for Wordpress (either case, the CPU
usage increases). That is as follows:
Also i couldn't find a Centos 6 installation guide for Prometheus.
*/* SET THE HOST AND PORT OF WORDPRESS*
* * *********************************************************/*
*vcl 4.0;*
*import std;*
*backend default {*
* .host = "SERVER IP";*
* .port = "8080";*
* .first_byte_timeout = 60s;*
* .connect_timeout = 300s;*
*}*
*# SET THE ALLOWED IP OF PURGE REQUESTS*
*# ##########################################################*
*acl purge {*
* "localhost";*
* "127.0.0.1";*
*}*
*#THE RECV FUNCTION*
*# ##########################################################*
*sub vcl_recv {*
*# set realIP by trimming CloudFlare IP which will be used for various
checks*
*set req.http.X-Actual-IP = regsub(req.http.X-Forwarded-For, "[, ].*$",
""); *
* # FORWARD THE IP OF THE REQUEST*
* if (req.restarts == 0) {*
* if (req.http.x-forwarded-for) {*
* set req.http.X-Forwarded-For =*
* req.http.X-Forwarded-For + ", " + client.ip;*
* } else {*
* set req.http.X-Forwarded-For = client.ip;*
* }*
* }*
* # Purge request check sections for hash_always_miss, purge and ban*
* # BLOCK IF NOT IP is not in purge acl*
* # ##########################################################*
* # Enable smart refreshing using hash_always_miss*
*if (req.http.Cache-Control ~ "no-cache") {*
* if (client.ip ~ purge || !std.ip(req.http.X-Actual-IP, "1.2.3.4") ~
purge) {*
* set req.hash_always_miss = true;*
* }*
*}*
*if (req.method == "PURGE") {*
* if (!client.ip ~ purge || !std.ip(req.http.X-Actual-IP, "1.2.3.4") ~
purge) {*
* return(synth(405,"Not allowed."));*
* }*
* return (purge);*
* }*
*if (req.method == "BAN") {*
* # Same ACL check as above:*
* if (!client.ip ~ purge || !std.ip(req.http.X-Actual-IP, "1.2.3.4")
~ purge) {*
* return(synth(403, "Not allowed."));*
* }*
* ban("req.http.host == " + req.http.host +*
* " && req.url == " + req.url);*
* # Throw a synthetic page so the*
* # request won't go to the backend.*
* return(synth(200, "Ban added"));*
*}*
*# Unset cloudflare cookies*
*# Remove has_js and CloudFlare/Google Analytics __* cookies.*
* set req.http.Cookie = regsuball(req.http.Cookie,
"(^|;\s*)(_[_a-z]+|has_js)=[^;]*", "");*
* # Remove a ";" prefix, if present.*
* set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");*
* # For Testing: If you want to test with Varnish passing (not caching)
uncomment*
* # return( pass );*
* # FORWARD THE IP OF THE REQUEST*
* if (req.restarts == 0) {*
* if (req.http.x-forwarded-for) {*
* set req.http.X-Forwarded-For =*
* req.http.X-Forwarded-For + ", " + client.ip;*
* } else {*
* set req.http.X-Forwarded-For = client.ip;*
* }*
* }*
*# DO NOT CACHE RSS FEED*
* if (req.url ~ "/feed(/)?") {*
* return ( pass ); *
*}*
*## Do not cache search results, comment these 3 lines if you do want to
cache them*
*if (req.url ~ "/\?s\=") {*
* return ( pass ); *
*}*
*# CLEAN UP THE ENCODING HEADER.*
* # SET TO GZIP, DEFLATE, OR REMOVE ENTIRELY. WITH VARY ACCEPT-ENCODING*
* # VARNISH WILL CREATE SEPARATE CACHES FOR EACH*
* # DO NOT ACCEPT-ENCODING IMAGES, ZIPPED FILES, AUDIO, ETC.*
* # ##########################################################*
* if (req.http.Accept-Encoding) {*
* if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {*
* # No point in compressing these*
* unset req.http.Accept-Encoding;*
* } elsif (req.http.Accept-Encoding ~ "gzip") {*
* set req.http.Accept-Encoding = "gzip";*
* } elsif (req.http.Accept-Encoding ~ "deflate") {*
* set req.http.Accept-Encoding = "deflate";*
* } else {*
* # unknown algorithm*
* unset req.http.Accept-Encoding;*
* }*
* }*
* # PIPE ALL NON-STANDARD REQUESTS*
* # ##########################################################*
* if (req.method != "GET" &&*
* req.method != "HEAD" &&*
* req.method != "PUT" && *
* req.method != "POST" &&*
* req.method != "TRACE" &&*
* req.method != "OPTIONS" &&*
* req.method != "DELETE") {*
* return (pipe);*
* }*
* # ONLY CACHE GET AND HEAD REQUESTS*
* # ##########################################################*
* if (req.method != "GET" && req.method != "HEAD") {*
* return (pass);*
* }*
* # OPTIONAL: DO NOT CACHE LOGGED IN USERS (THIS OCCURS IN FETCH TOO,
EITHER*
* # COMMENT OR UNCOMMENT BOTH*
* # ##########################################################*
* if ( req.http.cookie ~ "wordpress_logged_in" ) {*
* return( pass );*
* }*
* # IF THE REQUEST IS NOT FOR A PREVIEW, WP-ADMIN OR WP-LOGIN*
* # THEN UNSET THE COOKIES*
* # ##########################################################*
* if (!(req.url ~ "wp-(login|admin)") *
* && !(req.url ~ "&preview=true" ) *
* ){*
* unset req.http.cookie;*
* }*
* # IF BASIC AUTH IS ON THEN DO NOT CACHE*
* # ##########################################################*
* if (req.http.Authorization || req.http.Cookie) {*
* return (pass);*
* }*
* # IF YOU GET HERE THEN THIS REQUEST SHOULD BE CACHED*
* # ##########################################################*
* return (hash);*
* # This is for phpmyadmin*
*if (req.http.Host == "ki1.org <http://ki1.org>") {*
*return (pass);*
*}*
*if (req.http.Host == "mysql.ki1.org <http://mysql.ki1.org>") {*
*return (pass);*
*}*
*}*
*# HIT FUNCTION*
*# ##########################################################*
*sub vcl_hit {*
* # IF THIS IS A PURGE REQUEST THEN DO THE PURGE*
* # ##########################################################*
* if (req.method == "PURGE") {*
* #*
* # This is now handled in vcl_recv.*
* #*
* # purge;*
* return (synth(200, "Purged."));*
* }*
* return (deliver);*
*}*
*# MISS FUNCTION*
*# ##########################################################*
*sub vcl_miss {*
* if (req.method == "PURGE") {*
* #*
* # This is now handled in vcl_recv.*
* #*
* # purge;*
* return (synth(200, "Purged."));*
* }*
* return (fetch);*
*}*
*# FETCH FUNCTION*
*# ##########################################################*
*sub vcl_backend_response {*
* # I SET THE VARY TO ACCEPT-ENCODING, THIS OVERRIDES W3TC *
* # TENDANCY TO SET VARY USER-AGENT. YOU MAY OR MAY NOT WANT*
* # TO DO THIS*
* # ##########################################################*
* set beresp.http.Vary = "Accept-Encoding";*
* # IF NOT WP-ADMIN THEN UNSET COOKIES AND SET THE AMOUNT OF *
* # TIME THIS PAGE WILL STAY CACHED (TTL)*
* # ##########################################################*
* if (!(bereq.url ~ "wp-(login|admin)") && !bereq.http.cookie ~
"wordpress_logged_in" ) {*
* unset beresp.http.set-cookie;*
* set beresp.ttl = 52w;*
*# set beresp.grace =1w;*
* }*
* if (beresp.ttl <= 0s ||*
* beresp.http.Set-Cookie ||*
* beresp.http.Vary == "*") {*
* set beresp.ttl = 120 s;*
* # set beresp.ttl = 120s;*
* set beresp.uncacheable = true;*
* return (deliver);*
* }*
* return (deliver);*
*}*
*# DELIVER FUNCTION*
*# ##########################################################*
*sub vcl_deliver {*
* # IF THIS PAGE IS ALREADY CACHED THEN RETURN A 'HIT' TEXT *
* # IN THE HEADER (GREAT FOR DEBUGGING)*
* # ##########################################################*
* if (obj.hits > 0) {*
* set resp.http.X-Cache = "HIT";*
* # IF THIS IS A MISS RETURN THAT IN THE HEADER*
* # ##########################################################*
* } else {*
* set resp.http.X-Cache = "MISS";*
* }*
*}*
2016-08-02 15:07 GMT+03:00 Stig Bakken <stig at zedge.net>:
> Could it be that Varnish is not caching those POST requests, and that they
> are what makes Apache consume a lot of CPU?
>
> On MPMs: with PHP prefork is the safest choice, and it will work just fine
> as long as you have a reasonable config, especially since you have Varnish
> in front offloading the job of shuffling bytes back to the users.
>
> If I were you I'd try to get more data on which requests take a long time,
> some variant of "varnishncsa -F %D"..
>
> If you're into using Prometheus for metrics, you can try out this exporter
> which will give you a lot of insight into which requests take a long time
> to process: https://github.com/stigsb/varnish_request_exporter
> (Prometheus is awesome!)
>
> - Stig
>
>
> On Thu, Jul 28, 2016 at 6:57 PM, Ayberk Kimsesiz <
> ayberk.kimsesiz at gmail.com> wrote:
>
>> Hi,
>>
>> *CPU Monitor: *
>>
>> http://i.imgur.com/5KT1xRu.jpg
>>
>> *Apache status:*
>>
>> SrvPIDAccMCPUSSReqConnChildSlotClientProtocolVHostRequest
>> *0-0* - 0/0/9766 . 134.59 37 0 0.0 0.00 64.40 ::1 http/1.1
>> ns1.***com:8080 OPTIONS * HTTP/1.0
>> *1-0* 14612 0/16/9058 _ 17.83 13 1498 0.0 0.02 53.29 176.***.10 http/1.1
>> www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1
>> *2-0* 10863 0/179/9795 _ 185.14 6 1424 0.0 0.58 60.32 176.***.10 http/1.1
>> www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1
>> *3-0* 13127 0/127/9435 _ 119.80 4 1419 0.0 0.42 56.51 176.***.10 http/1.1
>> www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1
>> *4-0* - 0/0/9187 . 0.00 50 0 0.0 0.00 56.60 ::1 http/1.1 ns1.***.com:8080 OPTIONS
>> * HTTP/1.0
>> *5-0* 14851 0/9/8761 _ 8.95 13 1559 0.0 0.01 57.90 176.***.10 http/1.1
>> www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1
>> *6-0* 14852 0/6/8130 _ 6.67 4 1482 0.0 0.01 51.88 176.***.10 http/1.1
>> www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1
>> *7-0* 14192 11/57/8355 *K* 72.73 0 1363 106.6 0.44 52.79 176.***.10
>> http/1.1 www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1
>> *8-0* 13067 0/125/7795 _ 121.19 13 1759 0.0 0.65 68.27 176.***.10
>> http/1.1 www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1
>>
>> Apache error logs don't show anything about CPU.
>>
>>
>> 2016-07-28 19:31 GMT+03:00 Guillaume Quintard <
>> guillaume at varnish-software.com>:
>>
>>> You seem to have a good hit ratio, are you seeing anything on the apache
>>> logs that would explain the cpu usage?
>>>
>>> --
>>> Guillaume Quintard
>>>
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
>
>
> --
> Stig Bakken
> CTO, Zedge.net - free your phone!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20160802/e6423530/attachment-0001.html>
More information about the varnish-misc
mailing list