1333244321 Q * hparker Quit: I've fallen off the 'net and can't get up
1333244474 J * hparker ~hparker@2001:470:1f0f:32c:beae:c5ff:fe01:b647
1333246065 Q * misc-- Ping timeout: 480 seconds
1333247061 Q * hparker Remote host closed the connection
1333247187 J * hparker ~hparker@2001:470:1f0f:32c:beae:c5ff:fe01:b647
1333248549 M * Bertl_oO Guy-: doesn't look Linux-VServer related to me, sounds more like an overcommit setting (i.e. change in heuristics or config issue)
1333248735 M * Bertl_oO also if you have different results from enter and init, then double check the ulimits configured explicitely or implicitely by that init process and/or the script/session starting that java process
1333248926 M * Bertl_oO off to bed now ... have a good one everyone!
1333248930 N * Bertl_oO Bertl_zZ
1333260096 Q * clopez Ping timeout: 480 seconds
1333261538 M * Guy- Bertl_zZ: overcommit shouldn't be an issue, there is plenty of RAM, much of it free; the init process shouldn't be messing with ulimits (it didn't so far)
1333264004 Q * Alex[fob] Remote host closed the connection
1333264915 Q * theocrite Ping timeout: 480 seconds
1333264930 J * theocrite ~Hubert@kim.theocrite.org
1333269434 J * bonbons ~bonbons@2001:960:7ab:0:7568:e100:1178:daa
1333276988 Q * ensc|w Remote host closed the connection
1333276997 J * ensc|w ~ensc@www.sigma-chemnitz.de
1333282284 N * Bertl_zZ Bertl
1333282298 M * Bertl morning folks!
1333282325 M * Bertl Guy-: what I meant was that 'heuristic overcommitment' might be enabled and cause certain (huge) allocations to fail
1333284409 M * Guy- Bertl: but the heuristic mode is only supposed to refuse "obvious overcommits", which a request for 3G on a 16G system is not
1333284414 M * Guy- (I'd think)
1333284485 M * Guy- (but yes, overcommit_memory is 0 - just as it was before the kernel upgrade
1333285472 J * roymath ~user@cpe-66-108-129-98.nyc.res.rr.com
1333285662 M * roymath Hi - Is it possible to run an ubuntu desktop from a vserver guest? This seems much more flexible, but I wonder if there are any limitations.
1333286146 M * Bertl guess it should be possible
1333286237 M * roymath Bertl: great, thanks.
1333286333 M * roymath Also, I'm in a bind - I on a 3.0 kernel (ubuntu oneiric) and my util-vserver has the vc_set_sched() bug. So I downloaded trunk, but the built version gets a Segfault, because apparently, gcc 4.x has an issue with some sort of boundary thingy. Any ideas what I might do here?
1333286387 M * roymath The recommendation is to build with gcc3.4, but that's not possible without pinning and downgrading my ubuntu release.
1333286453 M * Bertl what util-vserver are you building?
1333286502 M * roymath util-vserver-0.30.216-pre3029
1333286599 M * Bertl and that doesn't build on ubuntu?
1333286601 N * BobR_oO BobR
1333286639 M * roymath It builds fine, but running 'vserver start' results in a seg fault.
1333286673 M * roymath https://bugs.launchpad.net/ubuntu/+source/util-vserver/+bug/87075
1333286676 M * Bertl that sounds interesting, could you show me the output you get when configuring util-vserver?
1333286738 M * Bertl ah, so it is a toolchain bug in ubuntu .. nice
1333286750 M * roymath Bertl: sure - I'll send you a paste of the config output.
1333287082 M * roymath Bertl: http://pastebin.com/QBnDPP7v
1333287127 M * roymath I'd appreciate any workarond suggestions you might have. Been quite frustrated the last couple of days
1333287143 M * Bertl looks good, thanks, so yeah, it seems a down/upgrade of your toolchain is in order
1333287230 M * roymath I'd prefer to upgrade. Since I think I'm at the latest of everything, not sure how I might do that.
1333289522 Q * roymath Remote host closed the connection
1333295499 J * pmenier ~pmenier@86.220.71.156
1333295669 J * Pazzo ~thomas@host-88-217-227-235.customer.m-online.net
1333295767 M * Pazzo Hi @ll, hi Bertl! Since a very long time I'm playing around with Linux vServer again :)
1333295803 M * Bertl wb Pazzo!
1333295818 M * Pazzo How are you Bertl, everything fine?
1333295838 M * Bertl eveyrthing fine, thanks! how are you?
1333295870 M * Pazzo Fine too, thanks. I moved from northern Italy to Germany in the meantime :)
1333295922 M * Pazzo As you might guess, I'm here as I stumbled over a little issue ;-) Using Kernel 3.3, tried to run vServers on XFS, haven't even been able to build a single guest, the firest mkdir hung. Googling around I found the following discussion: http://www.spinics.net/lists/xfs/msg10309.html
1333295933 M * Pazzo Are you aware of this, is this still an issue?
1333296022 M * Bertl we did some fixes on xfs lately, they should be in the latest kernel patch though
1333296058 M * Pazzo I'm running 3.3.0-vs2.3.3.1 -> is there a newer one?
1333296078 M * Bertl nope, that's the most recent one
1333296125 M * Pazzo Yep, there is nothing else to be found below /Experimental/ :)
1333296144 M * Bertl give me a few mintues to check if the patch made it into 3.3
1333296160 M * Pazzo Sure, thank you!
1333296769 M * Bertl okay, obviously I missed to add that one to 3.3, please apply http://vserver.13thfloor.at/ExperimentalT/delta-xfs-fix01.diff for now, I'll add it to the next patch
1333297045 M * Pazzo Great, thanks Bertl!
1333297087 M * Pazzo Is it possible to recompile just xfs_vnodeops?
1333297173 M * Bertl the kernel will only recompile the necessary part
1333297194 M * Bertl i.e. doing a 'make' in the kernel tree will not touch unaffected objects
1333297230 M * Pazzo make does so, yes - but do you know whether make-kpkg (Debian) does also behave like this?
1333297289 M * Bertl I doubt it
1333297919 M * Pazzo Bertl: cp usr/src/linux-3.3/fs/xfs/xfs_vnodeops.o /lib/modules/3.3.0-vs2.3.3.1/kernel/fs/xfs/xfs.ko
1333297928 M * Pazzo FATAL: Error inserting xfs (/lib/modules/3.3.0-vs2.3.3.1/kernel/fs/xfs/xfs.ko): Invalid module format
1333297946 M * Pazzo I did just "make", any idea what went wrong?
1333297964 M * Pazzo Forget it
1333297968 M * Pazzo Silly me 
1333298094 M * Pazzo Bertl: vServer installation on XFS is running fine right now!
1333300050 Q * pmenier 
1333300322 M * Bertl excellent!
1333304228 Q * fback Ping timeout: 480 seconds
1333305770 J * fback fback@red.fback.net
1333308226 J * Walex ~Walex@87-194-99-40.bethere.co.uk
1333311677 M * Guy- Bertl: any other pointers re mmap2() = -1 ENOMEM after kernel upgrade?
1333311784 M * Guy- Bertl: it fails even if I set overcommit_memory to 1
1333311795 M * Guy- (which should always overcommit)
1333311994 M * Bertl no, no ideas, probably a cgroup issue, did you try to just disable cgroups (at least the memory management) for that guest (for a test)?
1333312126 M * Guy- no, but that's certainly something to try
1333312133 M * Guy- interestingly, memory.failcnt stays 0
1333312604 Q * Walex Remote host closed the connection
1333312791 Q * bonbons Quit: Leaving
1333312958 M * Guy- Bertl: shouldn't touching /etc/vservers/guest/nocgroup prevent the creation of a cgroup for the guest?
1333312975 M * Guy- I did that, stopped the guest, started it again, and its cgroup is there
1333313032 M * Bertl nocgroup 
1333313033 M * Bertl If this file exists, .defaults/cgroup will be ignored for this guest.
1333313060 M * Bertl (that's from the great flower page :)
1333313130 M * Guy- yes, that's what I've been looking at
1333313156 M * Bertl doesn't say that guest specific cgroup configurations will be ignored
1333313158 M * Guy- this is apparently not the way to turn of cgroup usage for a guest
1333313173 M * Guy- it doesn't have a guest specific cgroup configuration
1333313212 M * Bertl well, then it looks like a bug, latest util-vserver I presume?
1333313258 M * Guy- 0.30.216-pre3000
1333313273 M * Guy- so no
1333313289 M * Guy- I'll compile the new package to be sure
1333313295 M * Bertl hmm, please double check with latest and if the issue persists contact daniel_hozac 
1333313353 M * Guy- which issue? cgroup usage despite nocgroup? or mmap2() failing?
1333313365 M * Bertl cgroup usage despite nocgroup
1333313387 M * Bertl we still have to check on the mmap part, and that is unlikely util-vserver related
1333313453 M * Guy- hmmm
1333313457 M * Guy- I stopped the guest
1333313462 M * Guy- and its cgroup is still there
1333313469 M * Guy- what should cause it to go away?
1333313492 M * Guy- I rmdired it
1333313493 M * Bertl ah, did you add the nocgroup while the guest was running?
1333313503 M * Guy- yes, then stopped it and started it again
1333313515 M * Guy- ah, I see
1333313518 M * Bertl then it's all your fault! :)
1333313524 M * Guy- it ignored the cgroup on stopping as well
1333313529 M * Bertl precisely
1333313534 M * Guy- OK
1333313557 M * Bertl so, no bug, no need to update .. although I don't want to keep you from updating
1333313559 M * Guy- is rmdiring the cgroup directory the correct way of removing the cgroup?
1333313566 M * Bertl AFAIK, yes
1333313598 M * Guy- OK, the guest doesn't have a cgroup now, and java still fails the same way
1333313636 M * Bertl good, let's get a complete strace first ...
1333313689 M * Bertl btw, can you try with different kernel versions easily?
1333313690 Q * Pazzo Ping timeout: 480 seconds
1333313704 M * Bertl if so, a bisection would be perfect with the very same guest setup
1333313774 M * Guy- relatively easily, for a few days
1333313782 M * Guy- should I pastebin the strace?
1333313789 M * Bertl yes, please
1333313848 J * Pazzo ~thomas@host-88-217-227-235.customer.m-online.net
1333313967 M * Guy- the util-vserver pastebin thinks it's spam :)
1333314012 M * Guy- http://pastebin.com/CUPPh27n
1333314048 M * Guy- Bertl: this is just the specific thread that's hitting the mmap2() failure
1333315931 M * Bertl we have an util-vserver pastebin?
1333315946 M * Bertl anyway pastebin is perfect
1333316151 M * Guy- sorry, I meant linux-vserver pastebin
1333316155 M * Guy- (it's getting late)
1333316234 M * Bertl this is on x86_64?
1333316276 M * Guy- yes, kernel is x86_64
1333316284 M * Guy- the userspace in this particular guest is i386
1333316292 M * Bertl tada!
1333316295 M * Guy- tada?
1333316299 M * Bertl 3GB limit
1333316306 M * Guy- but but. why did it work before?
1333316319 M * Pazzo Bertl: have a good night, and thanks again!
1333316325 M * Bertl Guy-: did you set a 32bit personality
1333316331 M * Guy- no
1333316334 M * Bertl Pazzo: thanks, u2!
1333316339 M * Bertl try that please
1333316354 Q * Pazzo Quit: ...
1333316358 M * Guy- right away
1333316363 M * Bertl and if that doesn't help, try to limit the memory (with cgroups) accordingly (with VIRT_MEM)
1333316430 M * Guy- that was something I did try (I tried limiting it to 10G)
1333316492 M * Bertl well, x86 is limited to 3GB per process
1333316497 M * Guy- Bertl: should I use PER_LINUX32 or PER_LINUX32_3GB?
1333316517 M * Bertl either one should be fine, PER_LINUX32_3GB will give you the full 3GB
1333316727 M * Guy- OK, it didn't help
1333316743 M * Bertl really? what does free show inside the guest now?
1333316773 M * Guy-              total       used       free     shared    buffers     cached
1333316773 M * Guy- Mem:      16401996   10154388    6247608          0     185196     615476
1333316773 M * Guy- -/+ buffers/cache:    9353716    7048280
1333316773 M * Guy- Swap:      4194300          0    4194300
1333316805 M * Bertl could you upload /proc/meminfo for me?
1333316813 M * Guy- sure
1333316868 M * Guy- Bertl: http://paste.linux-vserver.org/20787
1333316931 M * Guy- this is without a cgroup memory limit applied though
1333316938 M * Guy- should I limit it to e.g. 4G via cgroup?
1333316981 M * Guy- doesn't seem to make a difference
1333316985 M * Bertl no, the problem here is, you cannot map more than 3GB (minus some magical value)
1333317006 M * Bertl that's why the mmap2 on 32bit (compatibility mode) fails
1333317023 M * Guy- that I understand; what I don't understand is why it only started to fail after the kernel upgrade
1333317023 M * Bertl for whatever reason, java seems to calculate the wrong value here
1333317028 M * Guy- (and switching from SLUB to SLAB)
1333317064 M * Bertl let's compare the straces, specifically the amount of memory requested, between the working and failing kernel versions
1333317083 M * Guy- OK
1333317227 M * Guy- [pid 19579] mmap2(NULL, 3221225472, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x33667000
1333317230 M * Guy- [pid 19579] mmap2(NULL, 3221291008, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x33657000
1333317233 M * Guy- [pid 19579] mmap2(0x33660000, 3221225472, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x3366000
1333317237 M * Guy- these were the ones I could find
1333317266 M * Guy- (this is with 3.0.4-vs2.3.1-pre10.1 now)
1333317325 M * Guy- the first value is the exact same value that's failing with the new kernel
1333317510 J * Aiken ~Aiken@2001:44b8:2168:1000:21f:d0ff:fed6:d63f
1333317522 M * Guy- (and the second one, which still succeeds, is actually larger)
1333317564 M * Bertl yep, so it might indeed be a problem with the memory allocator
1333317599 M * Guy- oh, and 3221225472 is exactly 3GB
1333317635 M * Guy- 3221291008 is 3GB+64k...
1333317644 M * Bertl yup
1333317657 M * Guy- it almost looks like the 3GB limit doesn't apply
1333317668 M * Bertl let's try with SLUB or SLAB on both kernels
1333317686 M * Bertl makre sure to keep the personality set for 32bit
1333317716 M * Guy- OK, this will have to wait until tomorrow (NB. the personality isn't set on the old kernel now)
1333317743 M * Guy- (these are two distinct boxes that both have the same guest on them, one has a new kernel, the other an old one)
1333317756 M * Bertl okay, np, also try to set it there as well, if possible test with the exact same guest on the same system
1333317773 M * Bertl (just with different kernel configs)
1333317803 M * Guy- I only have one such test so far: the box where java fails now used to run a 3.2 kernel and java didn't fail with that
1333317834 M * Guy- but I can play around with the old box
1333317834 M * Bertl I'm pretty confident the issue is with the allocator and the huge (for 32bit) amount of memory allocated
1333317846 M * Guy- sound plausible
1333317849 M * Guy- +s
1333317893 J * misc-- ~misc@202.171.160.4
1333317897 M * Bertl the missing personality might just add some obfuscation ...
1333317923 M * Bertl i.e. allocations might get mapped according to 64bit limits instead of 32bit ones, etc
1333317946 M * Guy- but isn't that a "good" thing? avoiding unnecessary limitations?
1333317976 M * Bertl well, not really, because the 32bit limitations are there for a reason :)
1333318003 M * Bertl i.e. you might end up with memory which cannot be reached ...
1333318055 M * Guy- I see
1333318062 M * Bertl in general, setting a 32bit personality on 32bit guests makes them behave more consistant (because implicit expectations will be met)
1333318095 M * Bertl OTOH, you could just upgrade that guest to 64bit and be done
1333318233 M * Guy- I'm planning that, but it's going to be a pain :)
1333321869 J * clopez ~clopez@82.25.60.213.dynamic.mundo-r.com