Persistent mode statistics

[Anders Nordby]

Hi,

On the same server, I tried to kill the child process to see how well it
reloads the data.

In the logs, while restarting:

Jan  7 15:44:25 cache12 varnishd[61987]: Child cleanup complete

Jan  7 15:45:24 cache12 varnishd[61987]: Manager got SIGINT
Jan  7 15:45:24 cache12 varnishd[72808]: child (72809) Started
Jan  7 15:45:25 cache12 varnishd[72808]: Child (72809) said Closed fds:
6 7 8 11 12 14 15
Jan  7 15:45:25 cache12 varnishd[72808]: Child (72809) said Child starts
Jan  7 15:45:25 cache12 varnishd[72808]: Child (72809) said Dropped 0
segments to make free_reserve
Jan  7 15:45:25 cache12 varnishd[72808]: Child (72809) said Ready
Jan  7 15:45:27 cache12 varnishd[72808]: Child (72809) said Silo
completely loaded

On the console (I had auto_restart set to off):

# /usr/local/etc/rc.d/varnishd restart
Stopping varnishd.
Starting varnishd.
sizeof(struct smp_ident) = 112 = 0x70
sizeof(struct smp_sign) = 32 = 0x20
sizeof(struct smp_segptr) = 32 = 0x20
sizeof(struct smp_object) = 64 = 0x40
min_nseg = 10, max_segl = 2147063801
max_nseg = 104851, min_segl = 204772
aim_nseg = 1023, aim_segl = 20987915
aim_nobj = 536765
free_reserve = 209879150
sizeof(struct smp_ident) = 112 = 0x70
sizeof(struct smp_sign) = 32 = 0x20
sizeof(struct smp_segptr) = 32 = 0x20
sizeof(struct smp_object) = 64 = 0x40
min_nseg = 10, max_segl = 2147063801
max_nseg = 104851, min_segl = 204772
aim_nseg = 1023, aim_segl = 20987915
aim_nobj = 536765
free_reserve = 209879150
sizeof(struct smp_ident) = 112 = 0x70
sizeof(struct smp_sign) = 32 = 0x20
sizeof(struct smp_segptr) = 32 = 0x20
sizeof(struct smp_object) = 64 = 0x40
min_nseg = 10, max_segl = 2147063801
max_nseg = 104851, min_segl = 204772
aim_nseg = 1023, aim_segl = 20987915
aim_nobj = 536765
free_reserve = 209879150
Classic hash: 500009 buckets
Using old SHMFILE

Before I killed it:

# varnishstat -1 | grep ^n_object
n_object              1099549          .   N struct object
n_objectcore          1100129          .   N struct objectcore
n_objecthead          1099640          .   N struct objecthead

After I restarted it:

# varnishstat -1 | grep ^n_object
n_object                    4          .   N struct object
n_objectcore          1094434          .   N struct objectcore
n_objecthead          1093945          .   N struct objecthead

# ps axww -o pid,vsz,rss,command | grep varnish
72808 63008980 85316 varnishd: Varnish-Mgr cache12.finn.no (varnishd)
72809 63712260 821744 varnishd: Varnish-Chld cache12.finn.no (varnishd)

So it seems that n_object is emptied, while n_objectcore and
n_objecthead is not? Also I notice that resident memory usage is down by
around 96%, I don't know if that is to be expected?

Anders.

On Thu, Jan 07, 2010 at 04:39:41PM +0100, Anders Nordby wrote:
> I'm running Varnish trunk 4434 in FreeBSD/amd64 7.2-RELEASE with
> persistent mode and data spread on three filesystems:
> 
> -s persistent,/data00/varnishd.dat,20g -s persistent,/data01/varnishd.dat,20g -s persistent,/data02/varnishd.dat,20g
> 
> root at cache12:~# varnishstat -1 | egrep ^sm
> sm_nreq                     0         0.00 allocator requests
> sm_nobj                     0          .   outstanding allocations
> sm_balloc                   0          .   bytes allocated
> sm_bfree                    0          .   bytes free
> sma_nreq                    0         0.00 SMA allocator requests
> sma_nobj                    0          .   SMA outstanding allocations
> sma_nbytes                  0          .   SMA outstanding bytes
> sma_balloc                  0          .   SMA bytes allocated
> sma_bfree                   0          .   SMA bytes free
> sms_nreq                41969         5.71 SMS allocator requests
> sms_nobj                    0          .   SMS outstanding allocations
> sms_nbytes                  0          .   SMS outstanding bytes
> sms_balloc           26826511          .   SMS bytes allocated
> sms_bfree            26826511          .   SMS bytes freed
> root at cache12:~# ps axww -o pid,vsz,rss,command | grep varnish
> 61987 63008980 84844 varnishd: Varnish-Mgr cache12.finn.no (varnishd)
> 61988 63819396 19615712 varnishd: Varnish-Chld cache12.finn.no
> (varnishd)
> 
> As you can see, PID 61988 has allocated 19615712 kilobytes - 18,7
> gigabytes of memory resident. This does not show in varnishstat. I can
> see that sms_balloc grows steadily, but it does not have the number I
> expect for used memory. How do I check how much memory/space Varnish
> spent with persistent mode?
> 
> Cheers,
> 
> -- 
> Anders.
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc


-- 
Anders.