1305763549 M * Bertl off to bed now .. have a good one everyone! 1305763553 N * Bertl Bertl_zZ 1305765140 P * Leolo_2 1305777160 Q * hparker Quit: Quit 1305780426 J * ghislain ~AQUEOS@adsl2.aqueos.com 1305786789 Q * derjohn_foo Ping timeout: 480 seconds 1305788098 J * derjohn_foo ~aj@213.238.45.2 1305788223 J * petzsch ~markus@dslb-088-075-122-189.pools.arcor-ip.net 1305791008 Q * cehteh Ping timeout: 480 seconds 1305791764 J * cehteh ~ct@pipapo.org 1305793135 Q * petzsch Quit: Leaving. 1305797402 M * Mr_Smoke Morning 1305797445 M * Mr_Smoke I'm trying to make some sense out of debian's munin vserver plugins (thanks geb !), but it seems the resource plugin is looking for the info in the wrong place 1305797474 M * Mr_Smoke it checks either /proc/virtual//nsproxy or cvirt, and looks for info such as RSS or VM there 1305797491 M * Mr_Smoke Obviously it's not there (not on recent kernels at least), and I'm wondering where to look 1305797494 M * Mr_Smoke Any ideas ? 1305797652 Q * Piet Remote host closed the connection 1305797887 J * Piet ~Piet__@659AABRJN.tor-irc.dnsbl.oftc.net 1305798514 N * Bertl_zZ Bertl 1305798520 M * Bertl morning folks! 1305798534 M * Bertl Mr_Smoke: in recent kernels you have to consult the cgroup 1305798623 M * Mr_Smoke Bertl: ah, thought so 1305798635 M * Mr_Smoke But are there cgroup values for all that was in /proc/virtual before ? 1305798642 M * Mr_Smoke There seemed to be quite a lot of stuff back then 1305798699 M * Bertl there is no AS/VM accounting in cgroups (yet) 1305798727 M * Bertl but OTOH, you can get pretty accurate RSS and even swap values if configured properly 1305798831 M * Mr_Smoke rss is now memory.usage_in_bytes right ? 1305798883 Q * Wonka Ping timeout: 480 seconds 1305798891 M * Bertl modulo the multiplication factor, yes 1305798904 M * Mr_Smoke yeah, 4096 and so on 1305798931 M * Mr_Smoke How does swap work then ? 1305798946 M * Mr_Smoke ah memsw 1305798989 M * Mr_Smoke Bertl: but just to make one thing clear, when the guest is using memory in amounts above mem.limit but below memsw.limit, it will "think" it's swapping, but the host might not be swapping at all, right ? 1305799014 J * nospoonuser ~nospoonus@shell.net23.de 1305799032 M * daniel_hozac no 1305799035 M * daniel_hozac it will actually swap. 1305799045 M * Mr_Smoke Ah. 1305799054 M * Mr_Smoke Ok then 1305799131 M * Bertl that's a mainline 'feature' 1305799161 M * Mr_Smoke hehe :) 1305799178 M * Mr_Smoke Ok so I need to trip the debian script down a little, and make it use cgroup values instead of proc stuff 1305799183 M * Mr_Smoke I'll see what I can do about that 1305799192 M * Mr_Smoke Then maybe I can post it somewhere too 1305799207 M * Mr_Smoke What can be of interest in there ? 1305799237 M * nospoonuser hello guys, anyone here who can help me solve a issue with token bucket? no context consume tokens, sched_hard flag is set. 1305799241 M * Mr_Smoke just usage_in_bytes maybe 1305799267 M * Bertl nospoonuser: kernel/util-vserver version? 1305799308 M * Mr_Smoke is VSZ an "interesting" value to monitor and if so, where can one get it from cgroups ? 1305799310 M * nospoonuser Bertl: Kernel: 2.6.32-5-vserver-amd64 from the debian repo 1305799317 M * Mr_Smoke On a side note, is it normal to get VSZ < RSS ? 1305799334 M * nospoonuser util-vserver: 0.30.215 1305799336 M * daniel_hozac VSZ cannot be less than RSS per definition. 1305799350 M * daniel_hozac and no, it is not interesting to monitor VSZ. 1305799354 M * Bertl nospoonuser: you don't have hard CPU limits there, as a matter of fact, you do not even have scheduler limits, you need to use cgroups on that kernel 1305799386 M * nospoonuser Bertl: thanks 1305799402 M * Mr_Smoke daniel_hozac: well 1305799411 M * Mr_Smoke 182 114 678.8M 0.9G 5h52m09 42m02s52 26d13h45 www 1305799427 M * Mr_Smoke I dunno how that has come to be, but there it is 1305799456 M * Mr_Smoke Unless there is some weird rounding mechanism that causes 678M to be bigger than .9G 1305799720 M * daniel_hozac could you paste the memory.stat and ps faux of that guest? 1305799725 M * Mr_Smoke Sure 1305799730 M * Mr_Smoke Just a sec 1305799738 Q * AndrewLee Ping timeout: 480 seconds 1305799774 M * Mr_Smoke daniel_hozac: http://pastebin.com/79UvQ3wT 1305799983 J * AndrewLee ~andrew@210.240.39.201 1305800080 M * Mr_Smoke Are /dev/cgroups/ subdirectories necessarily vserver ids or can they be something else ? 1305800119 M * daniel_hozac hmm? 1305800122 M * Bertl they can be anything 1305800179 M * Mr_Smoke Hm 1305800223 M * Mr_Smoke What would be a cheap way to list vservers names then ? /proc/virtual/nsproxy's NodeName seems to report the *hostname*, not the vserver name 1305800353 M * daniel_hozac vuname -g --xid context 1305800477 M * Mr_Smoke daniel_hozac: so basically the context info is retrieved from /etc ? Since this is going to be in the munin script, I'd rather read the raw value than use an external tool. Unless it's a minimum overhard of course, maybe in this case :) 1305800520 M * daniel_hozac use the util-vserver Python bindings then. 1305800576 M * Mr_Smoke That's what vuname does ? 1305800629 M * Mr_Smoke Hm apparently not 1305800630 M * daniel_hozac no. 1305800650 M * Mr_Smoke BTW what do you make of my RSS > VSZ thing ? 1305800670 M * daniel_hozac cgroups include cached in the RSS value. 1305800704 M * Mr_Smoke but not in VSZ ? 1305800737 M * daniel_hozac VSZ is the sum of all processes 1305800840 M * Mr_Smoke So when you said ... "VSZ cannot be less than RSS per definition" ... what did I miss ? 1305800912 M * Bertl that RSS associated with memory allocated by processes will always be less or equal to the sum over the address space 1305800929 M * Bertl (note that caches and buffers are not included here) 1305800940 J * kir ~kir@swsoft-msk-nat.sw.ru 1305800968 M * Mr_Smoke ... I thought daniel just said that cgroups included cache in RSS ? 1305800976 A * Mr_Smoke is *very* confused 1305801085 Q * trippeh Ping timeout: 480 seconds 1305801279 M * Mr_Smoke I'm wondering whether I should just graph the output of vserver-stat instead of reading /dev/cgroups 1305801667 J * trippeh atomt@uff.ugh.no 1305801969 M * Mr_Smoke daniel_hozac: in that guest, I can indeed see that used+cache ≃ .9G 1305802001 M * Mr_Smoke Must I conclude that VSZ *can* in fact be less than RSS ? or that there is a bug there ? 1305802038 M * daniel_hozac with cgroups the way that they are used today, it can. 1305802061 M * Mr_Smoke Ok 1305802678 M * Mr_Smoke Better use memory.stat for my plugin then 1305802729 M * daniel_hozac depends on what data you want, i suppose. 1305802737 M * Mr_Smoke Yup 1305802744 M * Mr_Smoke .stat has cache and RSS set apart 1305802749 M * Mr_Smoke That might prove useful 1305805181 M * Mr_Smoke Ok I should now be able to graph proc, rss and swap from cgroup 1305805921 P * kir Leaving. 1305806693 J * hparker ~hparker@2001:470:1f0f:32c:beae:c5ff:fe01:b647 1305811027 J * Piet_ ~Piet__@659AABRLU.tor-irc.dnsbl.oftc.net 1305811437 Q * Piet Ping timeout: 480 seconds 1305812592 J * Romster ~romster@202.168.100.149.dynamic.rev.eftel.com 1305815081 M * Mr_Smoke Bertl: question regarding localhost in guests 1305815107 M * arekm 2.6.39 released 1305815116 M * Mr_Smoke How does one "properly" binds stuff to 127.0.0.1 in guest ? (or whatever that is not an "actual" interface, for that matter) 1305815141 M * daniel_hozac you bind to 127.0.0.1 1305815141 M * daniel_hozac ? 1305815147 M * Mr_Smoke Well some things do 1305815153 M * Mr_Smoke pgsql, fastcgi 1305815162 M * Mr_Smoke you don't always want that to listen on a public interface 1305815172 M * Mr_Smoke Ah 1305815182 M * Mr_Smoke You mean there's no trickery involved ? 1305815190 M * Mr_Smoke no ~single_ip necessary ? 1305815198 M * Bertl well, that depends on your config 1305815209 M * Mr_Smoke How so ? 1305815241 M * Bertl if you select single_ip (either via guest config or as auto default via kernel config) and assign a single IP to the guest, then you have two options: 1305815259 M * Bertl a) use the guest's loopback address as single IP 1305815275 M * Bertl b) add ~single_ip or remove single_ip if not the default 1305815288 M * Mr_Smoke by a) do you mean e.g. 127.0.123.1 ? 1305815294 M * Bertl if you assign more than one IP to the guest, single_ip will be off by default 1305815294 M * Mr_Smoke or plain old 127.0.0.1 ? 1305815298 M * daniel_hozac i thought single_ip wouldn't trigger if you bound to 127.0.0.1 1305815325 M * Bertl if you explicitely bind to 127.0.0.1, it won't 1305815340 M * Mr_Smoke Ok 1305815348 M * Bertl but if something binds to 0.0.0.0, then you need to be careful 1305815360 M * Mr_Smoke So I might have set ~single_ip in cases where it was not necessary. It that bad ? 1305815396 M * Bertl you won't benefit from the single_ip special casing, has advantages and disadvantages 1305815458 M * Mr_Smoke Ok then 1305815518 M * Bertl there is no deep mystery in this, although it seems like (at least judging from the number of questions in this regard), let me explain, and maybe add it on a wiki page then? 1305815546 M * Mr_Smoke Yep 1305815548 M * Mr_Smoke Soudns good :) 1305815551 M * Mr_Smoke sounds* 1305815563 M * Bertl first, apps can bind to a specific IP or to * aka 0.0.0.0 aka ANY_ADDR 1305815573 M * Mr_Smoke hm sorry to interrupt but 1305815581 M * Mr_Smoke can you also include ipv6 in your explanation ? 1305815585 M * Mr_Smoke ie 1305815590 M * daniel_hozac IPv6 doesn't have that. 1305815596 M * Mr_Smoke is :: == ANY_ADDR ? 1305815596 M * daniel_hozac so it is not relevant. 1305815609 M * Mr_Smoke mkay 1305815622 M * Mr_Smoke Bertl: fire away then :) 1305815644 M * Bertl so, when the app binds to a specific IP, that IP will be recorded in the socket/connection 1305815659 M * Bertl inside a guest, an additional check is done, if that IP is allowed 1305815699 M * Bertl now, if an app binds to 0.0.0.0, no IP can be associated at the time the socket is created 1305815730 M * Bertl on the host, this naturally means, match everything 1305815759 M * Bertl in the guest, with IP isolation in place, we want that to be restricted to the set of assigned IPs 1305815768 M * Mr_Smoke So far, so good 1305815798 M * Bertl to do that, we need to check against the set of guest IPs on every connect/packet action 1305815833 M * Bertl naturally this adds some overhead the host doesn't have, because the host simply doesn't check at all 1305815861 J * dowdle ~dowdle@scott.coe.montana.edu 1305815876 M * Mr_Smoke right 1305815878 M * Bertl to improve on that for setups where you only have a single IP assigned to a guest, we introduced a special case 1305815923 M * Bertl which instead of recording the 0.0.0.0 at socket creation time, simply replaces the 0.0.0.0 with the one and only assigned IP 1305815938 M * Bertl thus eliminating any further checks against the guest IP set 1305815943 M * Mr_Smoke I see 1305815953 M * Bertl the consequences of this are: 1305815953 M * Mr_Smoke But that's to the exclusion of 127.0.0.1 then 1305815968 M * Bertl 0.0.0.0 -> 1305815980 M * Bertl i.e. it won't check at a later time 1305816006 M * Bertl which means, you cannot change that IP after the socket was bound and you cannot extend the IP set (in regard of that socket) 1305816032 M * Bertl and of course, binding to 0.0.0.0 will _not_ include 127.x 1305816052 M * Bertl so, typical setups are: 1305816066 M * Bertl - single IP, no 127.x, localhost points to that IP 1305816081 M * Bertl this is the fastes but most inflexible setup 1305816085 M * Bertl *fastest 1305816105 M * Bertl - single IP, 127.0.0.1 1305816123 M * Bertl this requires explicit binding to 127.0.0.1 for local services 1305816160 M * Bertl - single IP with ~single_ip or more IPs 1305816180 M * Bertl this is the most flexible setup, as you can change everything at runtime after the binding 1305816203 M * Bertl of course, any explicit bindings will still be one IP only 1305816214 M * Mr_Smoke right 1305816223 M * Mr_Smoke That's a little clearer now 1305816224 M * Bertl completely independant from that, there is a kernel config option 1305816250 M * Bertl which allows to auto enable the single_ip special casing for guests with a single IP assigned 1305816311 M * Bertl now, the default probably should be off for that feature ... 1305816328 M * Bertl (at least nowadays, where nobody cares about performance :) 1305816351 M * Bertl any questions left regarding single_ip? 1305816473 M * Mr_Smoke Well 1305816480 M * Mr_Smoke You mentioned a performance hit 1305816490 M * Mr_Smoke How big a hit are we talking ? 1305816558 M * Bertl that really depends on the connections 1305816574 M * Mr_Smoke Hm 1305816581 M * Bertl if you get a million connections per second to that port, the overhead will be significant 1305816586 M * Mr_Smoke Ah 1305816616 M * Mr_Smoke So it's better to keep single_ip and explicitly bind to 127.0.0.1 + (public_IP) for example 1305816637 M * Bertl in the typical setup, especially with a reasonable amount of IPs assigned to a guest, you probably won't be able to even detect the overhead 1305816644 M * Mr_Smoke Ok 1305816660 M * Mr_Smoke So scenarii 2 and 3 in your description are quite interchangeable really 1305816665 M * Mr_Smoke in terms of user experience 1305816695 M * Bertl depends on the services and how they handle that 1305816716 M * Mr_Smoke Of course 1305816733 M * Bertl for example, if a service requires a separate thread for each bound port, it will consume more resources for separate bindings 1305816819 M * Bertl for the typical hosting setup, I'd suggest to go for the ~single_ip, even if you only assign a single IP to the guest 1305816836 Q * ncopa Quit: Leaving 1305816838 M * Bertl together with lback remapping this gives the most authentic experience 1305816872 M * Bertl for setups where you isolate services, like mail or web or whatever, you might consider trimming the setup down to the minimal 1305816904 M * Bertl (which includes using single_ip for single IP setups and avoiding 127.x where possible) 1305816931 M * Mr_Smoke Well I have one setup in mind where ~single_ip is necessary, and that's an app that will only bind to 0.0.0.0 and that I need to talk to on "localhost" 1305816944 M * Mr_Smoke but then maybe setting a localhost alias for the public IP should work 1305816962 M * Mr_Smoke at any rate, there would probably be no measurable performance impact in either case 1305817050 J * ncopa ~ncopa@3.203.202.84.customer.cdi.no 1305817051 M * Mr_Smoke Had the same problem with bind, but now I'll just tell it to listen to 127.0.0.1 as well 1305817191 M * Mr_Smoke hmm maybe that's not enough 1305817206 M * Mr_Smoke rndc can't seem to be able to talk to bind, even though it's listening on 127.0.0.1 1305817289 M * Mr_Smoke Ah, it needs a source adress to work properly in these conditions 1305817811 M * Mr_Smoke Yup, that works. A little more convoluted than expected, but very bearable 1305817839 J * Wonka ~produzier@chaos.in-kiel.de 1305819328 M * Bertl off for now ... bbl 1305819339 N * Bertl Bertl_oO 1305820678 Q * derjohn_foo Ping timeout: 480 seconds 1305821415 J * bonbons ~bonbons@2001:960:7ab:0:59c4:e835:25a5:387a 1305823954 Q * mcp Remote host closed the connection 1305824028 N * Piet_ Piet 1305826879 J * mcp ~mcp@wolk-project.de 1305827384 J * petzsch ~markus@p57B66006.dip.t-dialin.net 1305828316 J * petzsch1 ~markus@p57B63BF5.dip.t-dialin.net 1305828769 Q * petzsch Ping timeout: 480 seconds 1305830682 N * BobR_zZ BobR 1305830852 Q * petzsch1 Quit: Leaving. 1305831070 Q * ghislain Quit: Leaving. 1305831281 Q * MooingLemur Ping timeout: 480 seconds 1305833349 N * BobR BobR_zZ 1305833586 J * MooingLemur ~troy@ipv4.pinchaser.com 1305834510 Q * hparker Quit: Quit 1305835723 M * Radiance hello 1305835756 M * Radiance can anyone advise on how to force a stop if the vserver stop command doesn't complete, it shows this: Asking all remaining processes to terminate... 1305835813 M * Bertl_oO well, kill off all the processes in the context, then the context will disappear 1305835847 M * Bertl_oO thing is, that is what util-vserver does/tries, and if it fails, chances are good that one process is in 'D' state, i.e. locked inside the kernel 1305835877 M * Radiance ah ok 1305835888 M * Radiance i guess i'll not escape a full server reboot 1305835996 M * Bertl_oO depends, if you find what keeps the process in 'D' state, you might be able to 'fix' it 1305836049 M * Radiance gotta be something as wait state percentage is about 48 % 1305836317 M * Radiance can't see what it is as i can't enter the guest anymore 1305836361 M * Bertl_oO use vps and vtop, or vcontext 1305836403 J * derjohn_foo ~aj@d045175.adsl.hansenet.de 1305836802 M * Radiance looks like mysqld is in D state 1305837130 M * Radiance is it possible to list parent processes with vtop ? 1305837175 J * hparker ~hparker@2001:470:1f0f:32c:beae:c5ff:fe01:b647 1305838281 Q * bonbons Quit: Leaving 1305838313 Q * fLoo Remote host closed the connection 1305838853 J * aj__ ~aj@d074228.adsl.hansenet.de 1305839293 Q * derjohn_foo Ping timeout: 480 seconds 1305840492 Q * hparker Quit: Quit 1305841254 M * Bertl_oO vtop is the same as 'top just in xid=1 1305841304 Q * aj__ Ping timeout: 480 seconds 1305841608 J * hparker ~hparker@2001:470:1f0f:32c:beae:c5ff:fe01:b647 1305842434 J * aj__ ~aj@d074228.adsl.hansenet.de 1305844233 J * ghislain ~AQUEOS@adsl2.aqueos.com 1305844566 M * Bertl_oO off to bed now ... have a good one everyone! 1305844570 N * Bertl_oO Bertl_zZ 1305845414 Q * ghislain Quit: Leaving. 1305845473 J * ghislain ~AQUEOS@adsl2.aqueos.com 1305845851 Q * ghislain Quit: Leaving. 1305849423 Q * dowdle Remote host closed the connection