1306281680 Q * dowdle Remote host closed the connection 1306285134 M * Bertl off to bed now ... have a good one everyone! 1306285138 N * Bertl Bertl_zZ 1306289238 J * ghislain ~AQUEOS@adsl2.aqueos.com 1306290463 Q * hparker Quit: Quit 1306290907 Q * ghislain Quit: Leaving. 1306303765 J * petzsch ~markus@dslb-092-078-119-125.pools.arcor-ip.net 1306304427 Q * petzsch Quit: Leaving. 1306305175 Q * derjohn_mob Ping timeout: 480 seconds 1306306649 J * derjohn_mob ~aj@213.238.45.2 1306307196 J * ghislain ~AQUEOS@adsl2.aqueos.com 1306308269 Q * quasisane Ping timeout: 480 seconds 1306309240 P * kir Leaving. 1306309381 M * Mr_Smoke Ok 1306309387 M * Mr_Smoke I had yet another kernel panic 1306309408 M * Mr_Smoke with a 2.6.38.2-vs2.3.0.37-rc9 kernel 1306309423 M * Mr_Smoke i'm upgrading to 2.6.38.6 or .7 now, with debug and netconsole 1306309427 M * Mr_Smoke This is wearing me out 1306311562 M * Mr_Smoke daniel_hozac: are you around ? 1306312087 M * Mr_Smoke I'd like to know which kernel debugging options might be useful to the vserver team in the hopes that my problems can be investigated 1306313355 J * quasisane ~sanep@c-76-24-80-97.hsd1.nh.comcast.net 1306313611 J * quasisane_ ~sanep@c-76-24-80-97.hsd1.nh.comcast.net 1306315405 M * ard the panics I had with < 2.6.38.4 were network vanilla kernel related... 1306316020 J * petzsch ~markus@dslb-092-078-119-125.pools.arcor-ip.net 1306316438 M * Mr_Smoke ard: ah 1306316466 M * Mr_Smoke Im not sure whether they were panics or oopses, I mean I'm not sure what the exact term should be 1306316485 M * Mr_Smoke the host was still responding to ping, but I couldn't launch a single process, not even ls 1306316513 M * Mr_Smoke ard: what did you experience ? 1306316527 M * Mr_Smoke Im getting quite desperate over a possible explanation 1306316581 M * ard silent freezes (nothing, really nothing at the serial console) and sometimes a panic 1306316589 M * ard as in dead... 1306316604 M * ard do you have a serial console? 1306316629 M * Mr_Smoke nope 1306316634 M * Mr_Smoke I just brought up a netconsole though 1306316654 M * Mr_Smoke Since the crashes are random, I can't even hope to reproduce them at home 1306316661 M * Mr_Smoke Well, there is a serial console at the data center 1306316678 M * Mr_Smoke But im usually too much in a hurry to get the service back online to wait for a tech to look at it 1306316710 M * Mr_Smoke In your case, was the host still responding to ping or not even that ? 1306316745 M * ard It was really dead... no response on anything, and that with multiple chassis... Anyway got to go... O/~ 1306316749 M * Mr_Smoke Also, did you notice any change in load before it occurred ? 1306316751 M * Mr_Smoke Ah ok 1306316760 M * Mr_Smoke Ah :/ 1306316776 M * Mr_Smoke I'm considering going back to 2.6.36 now :/ 1306317078 Q * petzsch Quit: Leaving. 1306317680 M * _are_ Mr_Smoke: I run 2 bigger Servers on 2.6.38.6-vs2.3.0.37-rc15 since saturday. 17 VServers for office infrastructure stuff and 6 KVM-instances, it seems stable 1306317778 M * Mr_Smoke okay then 1306317803 M * Mr_Smoke i've had issues with all 2.6.38 so far up to and including .2 1306317867 M * _are_ I run 2.6.38.4 on my laptop and I have issues with 'sync' not coming back, but I have not noticed these on teh 2.6.38.6-servers 1306317870 M * Mr_Smoke but maybe they were mainline 1306317874 M * Mr_Smoke mainline issues* 1306318144 M * Mr_Smoke I hope they were, in fact. 1306320108 N * Bertl_zZ Bertl 1306320115 M * Bertl morning folks! 1306320417 M * Mr_Smoke Morning Bertl 1306320426 M * Mr_Smoke I've got a question for you 1306320445 M * Mr_Smoke Yesterday I got yet another freeze (host pinged, but no more) 1306320456 M * Mr_Smoke I'm going to build a kernel with KALLSYMS ande DEBUG 1306320465 M * Mr_Smoke Anything in particular that might help you ? 1306320476 M * Mr_Smoke even if it's just to say whether the bug is vserver related or nto ? 1306320477 M * Mr_Smoke not* 1306320582 M * Bertl well, you should definitely get a serial console (or some other realiable means to inspect the kernel log) 1306320596 M * Mr_Smoke I've set up a netconsole so far 1306320613 M * Bertl with DEBUG_INFO and magic sysreq enabled, you should be able to trigger a kernel stack dump 1306320621 M * Bertl that is what we need to look at 1306320624 M * Mr_Smoke ok 1306320659 M * Mr_Smoke on a side note, I just tried the 2.6.36.6 patch against 2.6.36.7 1306320664 M * Bertl I suggest to test it out on a responsive system first, i.e. how to trigger a stack dump and similar 1306320668 M * Mr_Smoke aht there's only 1 minor rej 1306320671 M * Mr_Smoke and* 1306320679 M * Bertl yep, should be fine 1306320697 M * Bertl (and I presume you mean 2.6.38.6/7 1306320703 M * Mr_Smoke yes, sorry 1306320988 M * Mr_Smoke Bertl: also, I notice something this time 1306320992 M * Mr_Smoke Dunno how important it is 1306321020 M * Mr_Smoke About an hour before the freeze, two of the vservers simultaneously had their load increased x3 or x4 1306321036 M * Mr_Smoke Nothing huge in the end, ie way below a load of 1, but still 1306321051 M * Mr_Smoke Is that a clue to anything ? 1306321070 M * Bertl not without a kernel stack trace to see where it happens 1306321086 M * Mr_Smoke ok 1306321091 M * Mr_Smoke It's really odd 1306321113 M * Mr_Smoke There's a huge relative spike just before the freeze too 1306321160 M * Bertl most freezes are hardware related, i.e. something in the hardware goes wrong and the kernel locks up 1306321200 M * Bertl but in any case, if the kernel is doing something, you should be able to get a trace, and that trace will shed some light on the issue 1306321261 M * Mr_Smoke I might not be using the word freeze correctly 1306321266 M * Mr_Smoke I think it's an oops rather 1306321282 M * Bertl even better, capture that and we have some clues ... 1306321284 M * Mr_Smoke The server was responsive to ping, I could type stuff into ssh, but that's about it 1306321291 M * Mr_Smoke no response, no processes running 1306321322 J * petzsch ~markus@dslb-092-078-119-125.pools.arcor-ip.net 1306321403 M * Mr_Smoke making new kernel now 1306321422 M * Mr_Smoke Now, maybe it's a mainline bug, and 2.6.38.7 doesn't have it anymore 1306321444 M * Mr_Smoke But I'm now knowledgeable enough to check the changelog andd get clues from that 1306321454 M * Bertl maybe, just make sure that you can capture a stack trace 1306321475 M * Mr_Smoke Well I dunno whether to hope for that, or that it doesn't crash anymore :) 1306321498 M * Mr_Smoke If only it weren't random 1306321531 M * Bertl well, if it doesn't crash/freeze/hang/whatever ... then you're fine, but if it does, you want to make damn sure you can capture the trace :) 1306321566 M * Mr_Smoke Boy is it troubling to see RSS > VSZ 1306321580 M * Mr_Smoke I'm not used to it 1306322079 Q * petzsch Quit: Leaving. 1306322235 J * mtg ~mtg@vollkornmail.dbk-nb.de 1306329526 M * Bertl nap attack ... bbl 1306329539 N * Bertl bertl_zZ 1306329545 N * bertl_zZ bertl 1306329549 N * bertl Bertl_zZ 1306329875 Q * mtg Quit: Verlassend 1306330602 J * petzsch ~markus@dslb-092-078-119-125.pools.arcor-ip.net 1306330938 Q * petzsch Quit: Leaving. 1306334353 J * hparker ~hparker@2001:470:1f0f:32c:beae:c5ff:fe01:b647 1306334631 J * dowdle ~dowdle@scott.coe.montana.edu 1306337231 Q * bsingh Ping timeout: 480 seconds 1306337764 J * bsingh ~balbir@122.172.195.151 1306339764 J * bonbons ~bonbons@2001:960:7ab:0:ecba:dcaa:57a:bcbd 1306340783 Q * ryker Quit: Leaving. 1306340805 J * ryker ~Adium@c-76-16-115-27.hsd1.in.comcast.net 1306341194 N * Bertl_zZ Bertl 1306341202 M * Bertl back now ... 1306341998 J * kir ~kir@swsoft-msk-nat.sw.ru 1306342315 M * arekm does anyone have a clue how to debug this? https://lkml.org/lkml/2011/5/23/398 1306342331 M * arekm looking for a idea on how to figure out what causes this 1306342411 M * arekm (happens on 2.6.38.6+grsec+vserver) 1306342530 J * s0undt3ch s0undt3ch@80.69.34.153 1306342987 J * manana ~mayday090@nat049-252-205-109.tvoe.tv 1306343195 Q * derjohn_mob Ping timeout: 480 seconds 1306343554 J * petzsch ~markus@dslb-092-078-119-125.pools.arcor-ip.net 1306344050 M * Bertl arekm: hmm, well, you can search for the byte sequence in your kernel code 1306344100 M * Bertl but to me it looks like there is sufficient time to print a proper oops, so I wonder why you just get a small part of the information 1306344171 M * Bertl does the IPMI work as 'normal' serial console? 1306344233 M * Bertl and what debug options do you have enabled (kernel config and at runtime) 1306344556 M * arekm Bertl: ipmi as normal console; debug options - let me ask, likely none 1306344765 M * arekm none ;/ 1306346398 M * Bertl so, you basically want to enable kernel debugging and debug info then 1306346412 M * Bertl and make sure that the loglevel is reasonably high 1306346759 P * kir Leaving. 1306347934 Q * Guy- Ping timeout: 480 seconds 1306348141 Q * ghislain Quit: Leaving. 1306349751 N * ensc Guest2185 1306349761 J * ensc ~irc-ensc@p5DF2E70D.dip.t-dialin.net 1306349887 J * ecapriolo ~kvirc@209.249.216.2 1306350170 Q * Guest2185 Ping timeout: 480 seconds 1306350466 M * ecapriolo I am doing a vserver-build by yum on a fresh machine. (We have our own local yum repo) Will the vserver being built use the network settings of the host or the guest for yum ? 1306350488 M * daniel_hozac the host. 1306350530 M * daniel_hozac note that you'll need to configure your own repos in /etc/vservers/.distributions/ 1306350557 M * ecapriolo Ok so it does not just use /etc/yum.repos.d then. That is what the issue is. 1306350632 M * ecapriolo It must take a long time for this stuff to timeout...fyi vserver build can not be ctrl+c ed in this state. 1306350654 M * daniel_hozac yum is really hard to ctrl+c. 1306350686 M * ecapriolo Right. After this one if is all going to be build by clone anyway :) 1306350777 M * Marillion arekm: do you have got your 2.6.38.6+grsec+vserver online available to download? 1306350886 M * Marillion arekm: and tarball too? :) 1306350895 M * ecapriolo daniel_hozac: I am on a centos5 trying to build a centos5 I do not see a folder for this in .distributions. Any hints ? 1306350926 M * daniel_hozac create it. 1306351272 M * ecapriolo Right I was just curious what the default settings were and where it was getting those from. 1306351366 M * daniel_hozac /usr/lib*/util-vserver/distributions/ 1306352062 M * ecapriolo daniel_hozac: Awesome! Thanks. ! 1306354096 M * arekm Marillion: ftp://ftp1.pld-linux.org/dists/th/ready/SRPMS/kernel* there 1306354166 M * Marillion arekm: thank you very much 1306355056 J * fleischergesell ~fleischer@dslb-088-077-213-168.pools.arcor-ip.net 1306355092 M * fleischergesell Hey there - how can I run a command (e.g. "apt-get update" or similar) in all running guests in one go? 1306355128 M * fleischergesell vsomething doesn't do the trick - or I do not understand the syntax correctly 1306355484 M * daniel_hozac vsomething will do it. 1306355525 M * daniel_hozac vsomething vserver --running -- exec apt-get update 1306355557 M * fleischergesell Ah thank you so much, now I understand this tool 1306356121 Q * fleischergesell 1306356347 Q * bonbons Quit: Leaving 1306356392 M * ryker I changed the IP for a guest and the host still seems to be hanging on to the IP. In the past, I just reboot the host and the IP is gone, but I can't reboot this host. Any idea how I can remove that IP I no longer need? 1306356412 M * Bertl ip a del 1306356413 M * ryker Looking at the wiki faq, I see naddress used for adding an address, and it has a —remove option 1306356428 M * Bertl I presume you changed the IP before shutting down the guest 1306356506 M * ryker yes 1306356510 M * ryker let me give that a try 1306356526 J * Guy- ~korn@elan.rulez.org 1306356701 M * ryker Bertl: I guess I'm stuck on syntax. should I be using 'ip addr del ' ? 1306356729 M * daniel_hozac and dev 1306356764 M * Bertl use 'ip a l' pick the one you want to remove and use 'ip a d' with the same arguments 1306356786 M * ryker excellent, thank you 1306356838 M * ryker ip a l ? I used ip addr del dev eth0 1306356855 M * ryker oh, it actually lists them with that? that's cool 1306356901 M * ryker i still use ifconfig all the time instead of ip 1306356908 M * Bertl your loss :) 1306356917 J * derjohn_mob ~aj@d073240.adsl.hansenet.de 1306356970 Q * petzsch Quit: Leaving. 1306357032 M * ryker :) 1306357040 M * ryker yeah, I know. i need to learn it. 1306357117 Q * Piet Remote host closed the connection 1306357136 J * Piet_ ~Piet__@04ZAABWTI.tor-irc.dnsbl.oftc.net 1306357757 N * Piet_ Piet 1306362535 M * ecapriolo I am trying to limit memory to a vserver I have done . echo 100000 > /etc/vservers/scacti/rlimits/as.hard for rs|ass . hard.soft and restarted. I also added VIRT_MEM into cflags but inside the guest free is showing alot of memory. Am I missing something? 1306362958 M * daniel_hozac do you have anything resembling a recent kernel? 1306362964 M * ecapriolo http://pastebin.ca/2069195 1306362995 M * ecapriolo Check out -/+ buffers/cache: after I turn on virt_mem 1306363199 M * daniel_hozac what does your cgroup's memory.stat contain? 1306363394 M * ecapriolo I am not using cgroups (I do not think). I am getting back into vserver after a vacation. So I may be doing this the old fashioned way. I just installed vserver kernel and now I am trying to use rlimits to lock down memory. 1306363406 Q * hparker Ping timeout: 480 seconds 1306363412 M * daniel_hozac you need to use cgroups on that kernel. 1306363424 M * daniel_hozac should default to it. 1306363443 Q * manana Remote host closed the connection 1306363693 M * ecapriolo I know everything about memory.stat.. http://www.mjmwired.net/kernel/Documentation/cgroups/memory.txt except where it is :) 1306363706 M * daniel_hozac /dev/cgroup/ 1306363823 M * ecapriolo Updated pastie . http://pastebin.ca/2069199 1306364021 M * daniel_hozac that's the root cgroup 1306364024 M * daniel_hozac i.e. host 1306364030 M * daniel_hozac you want the guest's subdirectory. 1306364170 M * ecapriolo The guest does not have one. 1306364605 M * ecapriolo http://linux-vserver.org/util-vserver:Cgroups#using_cgroup_to_enforce_memory_limits.. make sure /dev/cgroup is mounted with -o...,memory to be able to use this feature. Does this directory still need to be specifically mounted ? 1306364860 M * ecapriolo daniel_hozac: The rpm does not do this. mkdir /etc/vservers/.defaults/cgroup now I see the cgroup 1306364964 M * Bertl which util-vserver version do you use? 1306365068 M * ecapriolo http://pastebin.ca/2069202 1306365152 M * Bertl we are currently at pre2967, so you might want to update, although I think that version did already mount cgroups properly (in one of the runlevel scripts) 1306365214 M * ecapriolo [root@rs05 scacti]# chkconfig --list vservers-default 1306365214 M * ecapriolo vservers-default 0:off 1:off 2:on 3:on 4:on 5:on 6:off 1306365262 M * Bertl that is for starting guests marked as 'default' 1306365299 M * Bertl util-vserver and vprocunhide are the setup scripts 1306365301 M * ecapriolo Bertl: I mirrored this. http://rpm.hozac.com/dhozac/centos/5/vserver/x86_64/ and Followed http://linux-vserver.org/Installation_on_CentOS 1306365420 J * hparker ~hparker@2001:470:1f0f:32c:beae:c5ff:fe01:b647 1306365470 M * ecapriolo The good news is after that directory is created the output of free is correct. 1306365634 M * ecapriolo And memory.limit_in_bytes works correctly! 1306365691 M * Bertl good 1306365729 M * ecapriolo daniel_hozac: Bertl: Thank you for your help as always. You guys are top notch. Also a big part of the reason I got into open source. 1306365835 M * Bertl you're welcome! 1306366568 Q * Piet Quit: Piet 1306366832 J * Piet ~Piet__@82VAABQPB.tor-irc.dnsbl.oftc.net 1306367146 Q * Piet Quit: Piet 1306367753 J * Piet ~Piet__@82VAABQPE.tor-irc.dnsbl.oftc.net