1559101375 J * fstd_ ~fstd@xdsl-81-173-178-56.nc.de 1559101844 Q * fstd Ping timeout: 480 seconds 1559116053 J * hijacker ~nikolay@149.235.255.3 1559118421 N * Bertl_zZ Bertl 1559118425 M * Bertl morning folks! 1559118776 M * gnarface morning Bertl! 1559119546 M * Bertl you had some problems with LXC a few days ago? 1559119611 M * gnarface no, not me, Guy- 1559119672 M * Bertl ah, sorry, mixed that up ... 1559119723 M * gnarface no worries 1559119800 M * Bertl everything fine on your end? 1559119838 M * gnarface yea, no problems here 1559119855 M * gnarface (i haven't been using lxc) 1559120570 M * Guy- well, it's not entirely fair to say I had a problem with lxc :) 1559120578 M * Guy- I had a problem running lxc on a vserver-patched kernel 1559120588 M * Guy- both work well enough separately 1559120711 M * Bertl yep, that's what I meant 1559120745 M * Bertl I'm using LXC as well on several (trusted/secure) machines 1559120801 M * Bertl but there should be no adverse interaction between LXC and Linux-VServer from the kernel PoV 1559120840 M * Bertl care needs to be taken to mount the cgroup system properly (in the right place) and let util-vserver know about it 1559120886 M * Bertl as long as no Linux-VServer context is involved, LXC should behave as usual 1559120975 M * Guy- Bertl: that was what I thought too; I got the cgroup thing sorted (found a bug in util-vserver and even submitted a patch for it) 1559120996 M * Guy- Bertl: but when I start an lxc guest, its /proc/self symlink is messed up and I have no idea why 1559121014 M * Bertl messed up in what way? 1559121021 M * Guy- it's empty 1559121026 M * Guy- lrwxrwxrwx 1 root root 0 May 26 21:42 /proc/self 1559121049 M * Bertl ls -la /proc/self 1559121060 M * Guy- instead of pointing to /proc/PID, it points nowhere, can't even be read 1559121067 M * Guy- ls: cannot read symbolic link '/proc/self': No such file or directory 1559121071 M * Bertl interesting 1559121094 M * Bertl what kernel/patch? 1559121116 M * Guy- 4.4.169-vs2.3.9.8 1559121131 M * Guy- lstat("/proc/self", {st_mode=S_IFLNK|0777, st_size=0, ...}) = 0 1559121137 M * Guy- readlink("/proc/self", 0x555787c29a30, 1) = -1 ENOENT (No such file or directory) 1559121167 M * Bertl any other (security) patches involved? 1559121171 M * Guy- another anomaly: in the lxc guest I can mount -t proc proc /proc and get a full view of the host's /proc 1559121178 M * Guy- no other patches 1559121207 M * Guy- I can also send signals to the host's processes from inside lxc 1559121237 M * Bertl and both doesn't work on a mainline kernel I presume? 1559121243 M * Guy- indeed 1559121258 M * Guy- but I haven't tested with mainline 4.4, only 4.19+ 1559121259 M * Bertl what if you resolve /proc/self yourself? 1559121263 M * Guy- resolve how? 1559121267 M * Guy- readlink() fails on it 1559121277 M * Bertl well, it just points to $PID 1559121287 M * Guy- but it doesn't :) it points nowhere 1559121294 M * Guy- it _should_ point to $PID 1559121313 M * Bertl i.e. is /proc/ exist? 1559121315 M * Guy- yes 1559121326 M * Bertl and it is otherwise fine too? 1559121326 M * Guy- before I mount -t proc proc /proc in the lxc guest, it looks normal for a guest (limited number of processes visible) 1559121349 M * Bertl okay, let's check the content of /proc/ then for unusual settings 1559121366 M * Bertl specifically the cgroup and context related stuff 1559121393 M * Bertl ideally dump the entire tree (/proc/) to pastebin 1559121399 M * Guy- hang on 1559121430 M * Guy- ah, /proc/PID doesn't exist for the shell I got via lxc-attach 1559121468 M * Bertl try with ssh or similar 1559121484 M * Bertl also check that the issue actually exists there and not just with lxc-attach 1559121551 M * Guy- since there is no /proc/net/dev, the guest can't configure its networking, so I can't ssh in 1559121567 M * Guy- however, I noticed another thing: the PIDs I see in the guest don't exist when viewed from the host 1559121574 M * Guy- so it seems there is some pid remapping going on 1559121579 M * Guy- which is incomplete 1559121605 M * Bertl the problem is, all the pid remapping should be disabled in context 0 1559121615 M * Guy- I could read the /proc/PID of the lxc-attach shell from the host 1559121624 M * Bertl and in theory, all LXC guests should end up with context 0 too 1559121648 M * Bertl but of course some pid remapping magic might miss the relevant checks 1559121888 M * Guy- the dump gets hung on pagemap, which is and endless stream of zeroes 1559121909 M * Bertl okay, maybe focus on cgroup and context related entries 1559121971 M * Guy- http://sprunge.us/mcsFRn 1559122038 M * Bertl so first we are in context 0, both VxID and NxID are 0 1559122099 M * Bertl there are a ton of mounts listed though 1559122116 M * Guy- yes, that's expected 1559122145 M * Guy- data point: what I see as pid 1564 on the host seems to be pid 426 in the guest 1559122146 M * Bertl so you have that many inside the container? 1559122168 M * Guy- yes 1559122177 M * Bertl okay, just checking 1559122199 M * Guy- this is just an experiment on a home box, not "production" 1559122230 M * Bertl no problem, just wanted to verify that there are not host mounts leaking through 1559122235 M * Guy- the shell that lxc-attach spawned seems to have no /proc/pid entry in the guest (no /proc/*/cmdline matches "zsh") 1559122252 M * Bertl yes, IIRC, that is expected 1559122290 M * Guy- it's not how it works on my lxc-only host -- there, /proc/$$ exists in lxc-attach 1559122306 M * Bertl okay? 1559122332 M * Bertl so it would be really great if you could test without lxc-attach 1559122353 M * Bertl just to verify that the problem is not related to the way the container is entered 1559122389 M * Bertl from the Linux-VServer side everything seems normal and I'd expect the same behaviour as on the host 1559122408 M * Bertl (doesn't mean that there isn't a problem with the pid remapping) 1559122432 M * Guy- I'll have to make lxc configure the networking from the host side somehow, because it can't work from the inside this way 1559122462 M * Bertl easiest way is to have LXC setup the networking but without bridge 1559122479 M * Guy- I use macvlan 1559122482 M * Bertl then you can simply assign an IP to the other end of the virtual network pair 1559122510 M * Bertl just create a simple test container 1559122531 M * Bertl (no need to change your setup) 1559122680 M * Guy- OK, this will work, the guest just needs some audit capability because ssh otherwise shows me the door 1559122701 M * Guy- can't drop audit_write apparently 1559122740 M * AlexanderS Guy-: The lxc container uses a network namespace? 1559122762 M * Guy- yes 1559122765 M * AlexanderS You may have to call vprocunhide inside this namespace. (see http://linux-vserver.org/Frequently_Asked_Questions#When_using_network_namespaces_and_vserver_together.2C_netstat_does_not_work_in_the_vserver._What.27s_wrong.3F ) 1559122777 M * Guy- AlexanderS: I'm doing that 1559122789 M * Guy- Bertl: when I enter via ssh, things look a lot better 1559122807 M * Guy- the pid remapping is in effect but it works 1559122824 M * Guy- AlexanderS: lxc.hook.start-host = /usr/lib/util-vserver/vprocunhide 1559122845 M * Guy- netstat works when I enter via ssh 1559122869 M * Guy- apparently lxc-attach doesn't enter the pid namespace, or something 1559123116 M * Guy- Bertl: is there any hope of getting lxc-attach working? 1559123145 M * Bertl well, we need to figure out why lxc-attach has problems 1559123295 M * Bertl unless you use -e, it should be added to the containers cgroups and drop priviledges 1559123323 M * Bertl the question now is, why does that fail? 1559123342 M * Guy- one perhaps relevant difference between the pure-lxc box and this one: the vserver box doesn't have /proc/self/ns/cgroup 1559123425 M * Bertl that is a change in 4.6+ 1559123440 M * Bertl so it just means you have different kernels 1559123521 M * Guy- I have an strace of lxc-attach; what should I look for? 1559123578 M * Guy- maybe the problem is not with lxc-attach but with lxc-start? 1559123614 M * Bertl check for namespace transitions and priviledge changes 1559123676 M * Guy- can't see a setns() call 1559123705 M * Guy- ah yes 1559123729 M * Guy- they're in a child process, but they all succeed 1559123768 M * Guy- http://sprunge.us/XG3gcq 1559123817 M * Bertl so that looks fine too then 1559123863 M * Guy- the child 13844 drops capabilities and starts my shell inside the container 1559123901 M * Guy- it also does a setsid() which returns 13844 (even though it's supposed to be in a different pid namespace) 1559123948 M * Guy- but the id of the PID namespace matches what I see when I enter the guest via ssh 1559123985 M * Bertl hmm, do you have any chance to make a comparison with an identical kernel without Linux-VServer patches? 1559124005 M * Guy- not really :( 1559124022 M * Guy- it would take a day 1559124043 M * Bertl so we need to do some qemu/kvm setup with variable kernel 1559124068 M * Bertl I'll give it a try when I find some spare time :) 1559124115 M * Guy- thanks :) 1559131243 Q * Aiken Remote host closed the connection 1559147183 Q * DLange Ping timeout: 480 seconds 1559149502 Q * hijacker 1559149568 M * Bertl off for now ... bbl 1559149569 N * Bertl Bertl_oO 1559149621 J * DLange ~DLange@dlange.user.oftc.net 1559158370 J * Aiken ~Aiken@b951.h.jbmb.net 1559163922 M * Bertl_oO off to bed now ... have a good one everyone! 1559163923 N * Bertl_oO Bertl_zZ