varnish 2.0.4 questions - no IMS, no persistence cache - please help
Michael S. Fischer
michael at dynamine.net
Tue Nov 10 20:47:28 CET 2009
amd64 refers to the architecture (AKA x86_64), not the particular CPU
vendor. (As a matter of fact, I was unaware of this limitation;
AFAIK it does not exist in FreeBSD.)
In any event, mmap()ing 340GB even on a 64GB box is a recipe for
disaster; you will probably suffer death by paging if your working set
is larger than RAM. If it's smaller than RAM, then, well, there's no
harm in making it just under the total RAM size.
--Michael
On Nov 11, 2009, at 1:04 AM, GaneshKumar Natarajan wrote:
> Thanks.
> I checked /proc/cpuinfo and it shows intel processor.
> So even with Intel, we see this limitation of 340 GB. This is a
> serious limitation to me, since in Squid, we were using 1.5 TB of
> storage and i thought i could mmap and use all the space for Varnish.
> Any workarounds or working kernel version in linux, please let me
> know.
>
> mylinux version: RH4
> 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:33:05 EDT 2009 x86_64 x86_64
> x86_64 GNU/Linux
>
> ulimit -a:
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> file size (blocks, -f) unlimited
> pending signals (-i) 1024
> max locked memory (kbytes, -l) 32
> max memory size (kbytes, -m) unlimited
> open files (-n) 65535
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> stack size (kbytes, -s) 10240
> cpu time (seconds, -t) unlimited
> max user processes (-u) 278528
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
>
> cat /proc/cpufinfo
>
> processor : 0
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz
> stepping : 6
> cpu MHz : 2992.505
> cache size : 6144 KB
> physical id : 0
> siblings : 2
> core id : 0
> cpu cores : 2
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
> nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm
> bogomips : 5989.00
> clflush size : 64
> cache_alignment : 64
> address sizes : 38 bits physical, 48 bits virtual
> power management:
>
> processor : 1
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz
> stepping : 6
> cpu MHz : 2992.505
> cache size : 6144 KB
> physical id : 3
> siblings : 2
> core id : 6
> cpu cores : 2
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
> nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm
> bogomips : 5985.03
> clflush size : 64
> cache_alignment : 64
> address sizes : 38 bits physical, 48 bits virtual
> power management:
>
> processor : 2
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz
> stepping : 6
> cpu MHz : 2992.505
> cache size : 6144 KB
> physical id : 0
> siblings : 2
> core id : 1
> cpu cores : 2
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
> nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm
> bogomips : 5984.96
> clflush size : 64
> cache_alignment : 64
> address sizes : 38 bits physical, 48 bits virtual
> power management:
>
> processor : 3
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz
> stepping : 6
> cpu MHz : 2992.505
> cache size : 6144 KB
> physical id : 3
> siblings : 2
> core id : 7
> cpu cores : 2
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
> nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm
> bogomips : 5985.04
> clflush size : 64
> cache_alignment : 64
> address sizes : 38 bits physical, 48 bits virtual
> power management:
>
> Ganesh
>
>
> On Tue, Nov 10, 2009 at 1:48 AM, cripy <ccripy at gmail.com> wrote:
>> GaneshKumar Natarajan writes:
>> Tue, 20 Oct 2009 12:35:00 -0700
>>
>> 3. mmap storage : max i can configure is 340 GB.
>> I was able to use only 340 GB of cache. any size after this, i
>> would get error.
>> child (25790) Started
>> Pushing vcls failed: dlopen(./vcl.1P9zoqAU.so): ./vcl.1P9zoqAU.so:
>> failed to map segment from shared object: Cannot allocate memory
>> --
>>
>> I was having this issue too. After some googling it appears this
>> is a
>> AMD64 Linux 2.6 issue. According to
>> http://lists.humbug.org.au/pipermail/general/2004-July/024139.html
>>
>> "It maybe important to note that as of the latest 2.6 kernels,
>> Linux on
>> the AMD64 platform can only memory map a 340GB per process. This is
>> due
>> mainly to a VM paging system ported from the ia32 platform that
>> should
>> have been left on the hillside at birth to die. I have not tested
>> *BSD
>> because we have not done enough research to confirm if the Linux
>> emulation works on AMD64 for AMD64."
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at projects.linpro.no
>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>>
>
>
>
> --
> Regards,
> Gany
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
More information about the varnish-misc
mailing list