1635468361 M * Bertl_oO off to bed now ... have a good one everyone! 1635468363 N * Bertl_oO Bertl_zZ 1635485952 Q * ntrs Remote host closed the connection 1635485954 J * ntrs ~ntrs@vault08.rosehosting.com 1635486974 J * Ghislain ~ghislain@apollo-4300-51.titandc.io 1635503017 N * Bertl_zZ Bertl 1635503020 M * Bertl morning folks! 1635506208 J * arven ~arven4@h-81-170-212-57.A163.corp.bahnhof.se 1635506637 Q * arven 1635506703 J * slarvern ~slarvern4@h-81-170-212-57.A163.corp.bahnhof.se 1635506852 M * slarvern Any vserver users here? I have a problem with running on Buster. It eats my /proc directory on the host. 1635506899 M * Bertl sounds interesting ... please elaborate 1635506967 M * slarvern I did a fresh buster installation (amd64). Then, a bit short, "apt-get install linux-image-vserver-4.9-beng linux-headers-vserver-4.9-beng util-vserver util-vserver-build" 1635507067 M * slarvern After this (and some smaller adjustments of the install) I patched vserver.functions (seem to be required for Buster). Installed debootsrap (apt-get install debootstrap). 1635507085 M * slarvern Finally: vserver buildbuster64 build -n buildbuster64 -m debootstrap --i-know-its-there --context 1000 --hostname buildbuster64.twinspot.net -- -d buster -- --arch=amd64 --exclude=systemd-sysv,systemd,libsystemd0 --include=cron,sysvinit-core,sysvinit-utils 1635507166 M * Bertl okay, so far so good it seems ... 1635507175 M * slarvern Then, when starting up (vserver buildbuster64 start) I got some error messages (e.g ERROR: could not open /proc/stat: No such file or directory), got kicked out back to the shell. 1635507237 M * slarvern at this time "ls /proc" produces nothing, and the host is crazy. (meaning I can't evenr shut it down remotely). I have to hold CTRL-ALT-DELETE at the console. 1635507273 M * slarvern I may be doing something obviously wrong, so I'd better ask here first. 1635507283 M * Bertl okay, so the first question is, did the vprocunhide script run on the host before you start the guest? 1635507317 M * slarvern Forgot to say; before starting the vserver I also ran "/etc/init.d/vprocunhide start". 1635507358 M * slarvern Yes. 1635507367 M * Bertl okay, so /proc should be set up correctly, that's at least a start 1635507401 M * Bertl then, what 'modifications 1635507409 M * slarvern (But even if it didn't, it's a bit scary that vserver start could unmount /proc on the host). 1635507429 M * Bertl well, it looks like something in the tool startup went wrong 1635507444 M * Bertl normally util-vserver unshares the namespace early on 1635507457 M * slarvern Hm. Did I install the right packages? 1635507458 M * Bertl so anything which happens should only happen in a separate namespace 1635507486 M * Bertl the fact that it seems to happen on the host suggests that something is going terribly wrong there 1635507515 M * Bertl this could, of course, be a bad configuration on your side as well 1635507543 M * slarvern I also applied this patch http://linux-vserver.org/Installation_on_Debian#Buster, but I think /proc got unmounted before that. 1635507544 M * Bertl and yes, it could also be the 'wrong' packages 1635507572 M * slarvern Install commands: 1635507581 M * slarvern wget -q -O - http://repo.psand.net/pubkey.txt | apt-key add - 1635507591 M * slarvern echo "deb http://repo.psand.net/ buster main" | tee /etc/apt/sources.list.d/kernel-psand.list 1635507598 M * slarvern apt-get update 1635507605 M * slarvern (ok so far?) 1635507631 M * slarvern apt-get install curl 1635507647 M * slarvern apt-get install linux-image-vserver-4.9-beng linux-headers-vserver-4.9-beng util-vserver util-vserver-build 1635507720 M * Bertl you have to check with beng regarding those packages, I have no clue what they contain ... 1635507746 M * slarvern Is there other pre-build images for buster somewhere? 1635507762 M * Bertl no, we don't do pre-built images 1635507769 M * Bertl but usually they seem to work just fine and beng is quite responsive 1635507780 M * Bertl so I'd suggest to send an email to the mailing list 1635507813 M * Bertl best include what you did for the install and build process, and also the result you get 1635507852 M * Bertl double check that indeed the host mounts are affected and not just the shell you used to start the guest (but I guess you already checked that) 1635507931 M * Bertl I'd also check the guest, specifically the /dev/ entries there for stuff which doesn't belong 1635507942 M * slarvern OK, I'll do that, thanks. I'll wipe the host and reinstall again just to make sure nothing got messed up on the way. No, this is the /proc on the host (it get REALLY crazy whne /proc is empty - can't log in to it by ssh etc). 1635508001 M * Bertl did you try (regarding shutdown) to simply mount /proc again? 1635508024 M * Bertl something like 'mount -t procfs none /proc' should do 1635508062 M * slarvern yes, but only half-heartedly by "mount /proc" which didn't work. Then I figured rebooting is the faster option. 1635508090 M * Bertl probably is, no problem 1635508098 M * slarvern Hm, I can test it right now before wiping the system, hang on. 1635508146 M * slarvern (logged in, ls /proc shows all is fine...) 1635508191 M * slarvern root@j2dc7800:/home/j2admin# /etc/init.d/vprocunhide start 1635508191 M * slarvern : Fixing /proc entries visibility.... 1635508191 M * slarvern 1635508221 M * slarvern root@j2dc7800:/home/j2admin# vserver buildbuster64 start 1635508221 M * slarvern Using makefile-style concurrent boot in runlevel 3. 1635508221 M * slarvern ERROR: could not open /proc/stat: No such file or directory 1635508221 M * slarvern ERROR: could not open /proc/stat: No such file or directory 1635508221 M * slarvern Starting enhanced syslogd: rsyslogdrsyslogd: imklog: cannot open kernel log (/proc/kmsg): No such file or directory. 1635508223 M * slarvern rsyslogd: activation of module imklog failed [v8.1901.0 try https://www.rsyslog.com/e/2145 ] 1635508225 M * slarvern . 1635508227 M * slarvern Starting periodic command scheduler: cron. 1635508229 M * slarvern ERROR: could not open /proc/stat: No such file or directory 1635508231 M * slarvern vshelper.init: can not determine xid of vserver 'buildbuster64'; returned value was '' 1635508233 M * slarvern This usually means that you're using an init-less init-style, but the 1635508235 M * slarvern guest isn't configured to start any service. Try enabling a service, 1635508237 M * slarvern changing the init-style, or making the contexts persistent. 1635508239 M * slarvern An error occured after executing the vserver startup sequence. This 1635508241 M * slarvern means that some processes may exist in the created context and the 1635508242 M * slarvern manual execution of 1635508244 M * slarvern /usr/sbin/vserver '/etc/vservers/buildbuster64' stop 1635508248 M * slarvern is recommended to fix this. 1635508250 M * slarvern (back at the prompt of the host) 1635508253 M * slarvern root@j2dc7800:/home/j2admin# ls /proc 1635508254 M * slarvern root@j2dc7800:/home/j2admin# 1635508312 M * slarvern root@j2dc7800:/home/j2admin# mount -t procfs none /proc 1635508312 M * slarvern mount: /proc: unknown filesystem type 'procfs'. 1635508346 M * Bertl try proc instead of procfs 1635508354 M * Bertl mount -t proc none /proc 1635508367 M * slarvern That worked. 1635508400 M * Bertl does /proc/stat exist now and is accessable? 1635508418 M * slarvern yes. 1635508431 M * Bertl okay, so you are back at the host then 1635508439 M * Bertl let's try with isolation 1635508489 M * Bertl chcontext --xid 100 -- ls /proc/ 1635508492 M * slarvern Yes, thanks for that. Now I don't have to run downstairs to the server and brute-force reboot it. 1635508521 M * slarvern root@j2dc7800:/home/j2admin# chcontext --xid 100 -- ls /proc/ 1635508521 M * slarvern New security context is 100 1635508521 M * slarvern 1068 execdomains ioports locks mounts stat thread-self 1635508521 M * slarvern cpuinfo filesystems kcore meminfo net swaps tty 1635508521 M * slarvern crypto interrupts kmsg misc self sys uptime 1635508522 M * slarvern devices iomem loadavg modules slabinfo sysvipc version 1635508537 M * Bertl so stat seems to be there 1635508540 M * Bertl chcontext --xid 100 -- wc /proc/stat 1635508560 M * slarvern root@j2dc7800:/home/j2admin# chcontext --xid 100 -- wc /proc/stat 1635508561 M * slarvern New security context is 100 1635508561 M * slarvern 10 513 1188 /proc/stat 1635508568 M * Bertl an it is also readable 1635508584 M * Bertl which means it is not vprocunhide related 1635508628 M * Bertl try to run the guest start with --debug 1635508638 M * Bertl and upload the output to some pastebin service 1635508655 M * slarvern I admit I'm at a loss of the inner cogwheels of vserver. 1635508720 M * slarvern Hm, that produced a lot of text. 1635508729 M * Bertl indeed :) 1635508755 M * slarvern Give me a moment to take a look... 1635508764 M * Bertl sure 1635508955 M * slarvern First odd thing: ... ++ lock /var/lock/vserver.etcvserversbuildbuster64.startup 1635508955 M * slarvern +++ /bin/mktemp -t vserver-lock.XXXXXX 1635508955 M * slarvern ++ local tmp=/tmp/vserver-lock.O7YoBT 1635508955 M * slarvern ++ /bin/rm -f /tmp/vserver-lock.O7YoBT 1635508955 M * slarvern ++ /usr/bin/mkfifo -m600 /tmp/vserver-lock.O7YoBT 1635508956 M * slarvern ++ /bin/grep -q true /tmp/vserver-lock.O7YoBT 1635508957 M * slarvern ++ /usr/lib/x86_64-linux-gnu/util-vserver/lockfile /var/lock/vserver.etcvservers buildbuster64.startup /tmp/vserver-lock.O7YoBT 1635508961 M * slarvern lockfile: open(): No such file or directory 1635509018 M * slarvern (that doesn't stop it though) 1635509092 M * slarvern (actually repeated a bit down in the dump) 1635509162 M * slarvern ... further down: 1635509164 M * slarvern +++ getPhysicalDir /var/lock 1635509164 M * slarvern +++ set -P 1635509164 M * slarvern +++ cd /var/lock 1635509201 M * slarvern "/usr/share/util-vserver/functions: line 83: cd: /var/lock: No such file or direc tory 1635509266 M * slarvern ...and at the end: vcontext: vc_ctx_create(): File exists 1635509289 M * Bertl means that a context with this specific context id already exists 1635509291 M * slarvern Thats all the obviously odd things I could see. 1635509310 M * Bertl well, good that you know what's odd and what's not ;) 1635509346 M * slarvern When you say it like that I realize everything about this is odd, yes :-) 1635509361 M * slarvern But it keeps talking about a lockfile. 1635509381 M * Bertl as I said, upload the entire thing to a pastebin, feel free to anonymize IPs or similar sensitive information 1635509684 M * slarvern https://pastebin.ubuntu.com/p/Dz3yfybKhM/ 1635509749 M * slarvern Sorry it took a while. Your help is appreciated. 1635509872 M * slarvern Hmm... there's a broken symlink in /var on the host, pointing to /run/lock. 1635510030 M * slarvern "/var/lock -> /run/lock, should possibly be /var/lock -> /var/run/lock. 1635510298 M * slarvern Hey, some progress. "mkdir /var/run/lock" and then "vserver buildbuster64 start" at least got me to keep /proc. 1635510581 M * slarvern Now it's a more reasonable problem. It still won't start and leaves me with some loose ends causing next try to generate another error, but it should be easier to find now. 1635510598 M * slarvern "/etc/vservers/buildbuster64/fstab:1:1: failed to mount fstab-entry" 1635510629 M * Bertl the pastebin seems to be somewhat broken as well, there are huge spaces in some lines ... but it looks like trhe /var/run/lock was definitely missing 1635510638 M * Bertl *the 1635510657 M * slarvern Sorry, copy-pasted from a putty window. 1635510779 M * slarvern Ouch, /vservers/buildbuster64/proc/ is populated so it's left in a half-running state. 1635510831 M * slarvern But it's better! Less cause for panic. :-) 1635510849 M * Bertl context will disappear when the last process exits 1635510876 M * Bertl together with the context, all context mounts (like proc in the guest) will disappear as well 1635510938 M * slarvern is there a way to list the processes for context 1000? 1635511010 M * Bertl chcontext --xid 1000 -- ps 1635511039 M * Bertl you can also use vps and look for the context 1635511070 M * slarvern chcontext --xid 1000 -- ps -ax 1635511070 M * slarvern PID TTY STAT TIME COMMAND 1635511070 M * slarvern 1 ? Ss 0:01 /sbin/init 1635511070 M * slarvern 1018 ? Ssl 0:00 /usr/sbin/rsyslogd 1635511070 M * slarvern 1027 ? Ss 0:00 /usr/sbin/cron 1635511072 M * slarvern 1623 ? R+ 0:00 ps -ax 1635511093 M * Bertl so the guest seems to be running 1635511108 M * slarvern ...but I can't stop it. 1635511116 M * Bertl there is vkill 1635511973 M * slarvern After some experimentation I've come a bit closer. " 1635512043 M * slarvern "vserver buildbuster64 start" makes me loose /proc. If I reclaim it with "mount -t proc none /proc" I can see tha vserver is actually running. I can't enter it, but I can stop it. 1635512166 M * slarvern root@j2dc7800:/var# vserver-stat 1635512166 M * slarvern open(memory.stat): No such file or directory 1635512166 M * slarvern CTX PROC VSZ RSS userTIME sysTIME UPTIME NAME 1635512166 M * slarvern 1000 4 157.9M 0 0m00s00 0m00s00 1m09s70 buildbuster64 1635512687 M * slarvern I'll do a complete wipe and try again from scratch. Thanks again for the mount /proc tip, that got me a little bit further! 1635513562 M * Bertl you're welcome! 1635514409 M * Bertl off for now ... bbl 1635514410 N * Bertl Bertl_oO 1635517559 M * AlexanderS slarvern: You are running systemd on the host? 1635517598 M * AlexanderS slarvern: You may need this patch for util-verserver: https://github.com/linux-vserver/util-vserver/pull/26 1635517615 M * AlexanderS s/util-verserver/util-vserver/ 1635518004 M * slarvern Just at the end of reinstalling. I'll check it out! 1635518869 M * AlexanderS slarvern: There are some prebuild binaries available from beng: http://repo.psand.net/util-vserver_AlexanderS/ 1635518938 M * slarvern Shouldn't that be the same as wget -q -O - http://repo.psand.net/pubkey.txt | apt-key add - 1635518938 M * slarvern echo "deb http://repo.psand.net/ buster main" | tee /etc/apt/sources.list.d/kernel-psand.list 1635518938 M * slarvern apt-get update 1635518938 M * slarvern 1635518938 M * slarvern apt-cache search linux | grep beng 1635518939 M * slarvern 1635518941 M * slarvern apt-get install curl 1635518943 M * slarvern apt-get install linux-image-vserver-4.9-beng linux-headers-vserver-4.9-beng 1635518945 M * slarvern 1635518947 M * slarvern apt-get -t kernel-psand install util-vserver util-vserver-build 1635518949 M * slarvern 1635518972 M * slarvern ..or are the util-vserver taken from elsewhere? 1635519094 M * AlexanderS AFAIK are the packages in util-vserver_AlexanderS special packages with the pull request mentioned above. 1635519120 M * AlexanderS The default util-vserver packages from the repo are without this patch. 1635519170 M * slarvern I just finished the build of a vserver and tested it. E.g "vserver buildbuster64 start" still unmounts /proc on the host. 1635519282 M * slarvern So it's just to uninstall util-vserver and util-vserver-build and reinstall them from http://repo.psand.net/util-vserver_AlexanderS/ ? 1635519376 M * slarvern I'm too much of a newbie to know how to do that. Can you help? 1635521494 M * AlexanderS slarvern: Just download the *.deb files from http://repo.psand.net/util-vserver_AlexanderS/ and install them with: "dpkg --install *.deb" 1635521514 M * AlexanderS slarvern: No need to uninstall the current version. 1635521589 M * AlexanderS You may need util_verserver, util_vserver-core, util_vserver-build and libvserver0 1635536483 Q * Ghislain Quit: Leaving.