varnish 2.0.4 questions - no IMS, no persistence cache - please help
Ken Brownfield
kb+varnish at slide.com
Tue Nov 10 21:11:57 CET 2009
Note that the linked article is from 2004. The kernels that RedHat uses are a bag of hurt, not to mention ancient.
If you can upgrade to RHELl5 that may be the easiest fix (I can only assume that the mmap limitation has been removed). Perhaps RedHat has newer RHELl4 kernels in a bleeding edge repository, or perhaps Fedora/CentOS have packages that could upgrade your kernel.
That being said, there may be other pratfalls in the libc on RHELl4; bad 64-bit judgment calls persist in libc to this day (e.g., MAP_32BIT).
Ken
On Nov 10, 2009, at 11:47 AM, Michael S. Fischer wrote:
> amd64 refers to the architecture (AKA x86_64), not the particular CPU
> vendor. (As a matter of fact, I was unaware of this limitation;
> AFAIK it does not exist in FreeBSD.)
>
> In any event, mmap()ing 340GB even on a 64GB box is a recipe for
> disaster; you will probably suffer death by paging if your working set
> is larger than RAM. If it's smaller than RAM, then, well, there's no
> harm in making it just under the total RAM size.
>
> --Michael
>
>
> On Nov 11, 2009, at 1:04 AM, GaneshKumar Natarajan wrote:
>
>> Thanks.
>> I checked /proc/cpuinfo and it shows intel processor.
>> So even with Intel, we see this limitation of 340 GB. This is a
>> serious limitation to me, since in Squid, we were using 1.5 TB of
>> storage and i thought i could mmap and use all the space for Varnish.
>> Any workarounds or working kernel version in linux, please let me
>> know.
>>
>> mylinux version: RH4
>> 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:33:05 EDT 2009 x86_64 x86_64
>> x86_64 GNU/Linux
>>
>> ulimit -a:
>> core file size (blocks, -c) 0
>> data seg size (kbytes, -d) unlimited
>> file size (blocks, -f) unlimited
>> pending signals (-i) 1024
>> max locked memory (kbytes, -l) 32
>> max memory size (kbytes, -m) unlimited
>> open files (-n) 65535
>> pipe size (512 bytes, -p) 8
>> POSIX message queues (bytes, -q) 819200
>> stack size (kbytes, -s) 10240
>> cpu time (seconds, -t) unlimited
>> max user processes (-u) 278528
>> virtual memory (kbytes, -v) unlimited
>> file locks (-x) unlimited
>>
>> cat /proc/cpufinfo
>>
>> processor : 0
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 23
>> model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz
>> stepping : 6
>> cpu MHz : 2992.505
>> cache size : 6144 KB
>> physical id : 0
>> siblings : 2
>> core id : 0
>> cpu cores : 2
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 10
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
>> nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm
>> bogomips : 5989.00
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 38 bits physical, 48 bits virtual
>> power management:
>>
>> processor : 1
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 23
>> model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz
>> stepping : 6
>> cpu MHz : 2992.505
>> cache size : 6144 KB
>> physical id : 3
>> siblings : 2
>> core id : 6
>> cpu cores : 2
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 10
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
>> nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm
>> bogomips : 5985.03
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 38 bits physical, 48 bits virtual
>> power management:
>>
>> processor : 2
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 23
>> model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz
>> stepping : 6
>> cpu MHz : 2992.505
>> cache size : 6144 KB
>> physical id : 0
>> siblings : 2
>> core id : 1
>> cpu cores : 2
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 10
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
>> nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm
>> bogomips : 5984.96
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 38 bits physical, 48 bits virtual
>> power management:
>>
>> processor : 3
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 23
>> model name : Intel(R) Xeon(R) CPU L5240 @ 3.00GHz
>> stepping : 6
>> cpu MHz : 2992.505
>> cache size : 6144 KB
>> physical id : 3
>> siblings : 2
>> core id : 7
>> cpu cores : 2
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 10
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
>> nx lm pni monitor ds_cpl est tm2 cx16 xtpr lahf_lm
>> bogomips : 5985.04
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 38 bits physical, 48 bits virtual
>> power management:
>>
>> Ganesh
>>
>>
>> On Tue, Nov 10, 2009 at 1:48 AM, cripy <ccripy at gmail.com> wrote:
>>> GaneshKumar Natarajan writes:
>>> Tue, 20 Oct 2009 12:35:00 -0700
>>>
>>> 3. mmap storage : max i can configure is 340 GB.
>>> I was able to use only 340 GB of cache. any size after this, i
>>> would get error.
>>> child (25790) Started
>>> Pushing vcls failed: dlopen(./vcl.1P9zoqAU.so): ./vcl.1P9zoqAU.so:
>>> failed to map segment from shared object: Cannot allocate memory
>>> --
>>>
>>> I was having this issue too. After some googling it appears this
>>> is a
>>> AMD64 Linux 2.6 issue. According to
>>> http://lists.humbug.org.au/pipermail/general/2004-July/024139.html
>>>
>>> "It maybe important to note that as of the latest 2.6 kernels,
>>> Linux on
>>> the AMD64 platform can only memory map a 340GB per process. This is
>>> due
>>> mainly to a VM paging system ported from the ia32 platform that
>>> should
>>> have been left on the hillside at birth to die. I have not tested
>>> *BSD
>>> because we have not done enough research to confirm if the Linux
>>> emulation works on AMD64 for AMD64."
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at projects.linpro.no
>>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>>>
>>
>>
>>
>> --
>> Regards,
>> Gany
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at projects.linpro.no
>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
--
kb
More information about the varnish-misc
mailing list