"Cannot allocate memory" even the free space is sufficient

General support questions
Post Reply
Jianchun
Posts: 1
Joined: 2018/10/17 03:18:39

"Cannot allocate memory" even the free space is sufficient

Post by Jianchun » 2018/10/19 07:24:01

Hello, I cannot create any new net namespace on centos7.3 after running a lot of docker containers (up to 300). It gives "Failed to create a new network namespace "ns0": Cannot allocate memory":

[root@docker-125 docker]# ip netns add ns0
Failed to create a new network namespace "ns0": Cannot allocate memory
[root@docker-125 docker]#


While the free memory space is definitely sufficent.

[root@docker-125 docker]# free -h
total used free shared buff/cache available
Mem: 125G 25G 90G 75M 9.1G 94G
Swap: 4.0G 0B 4.0G
[root@docker-125 docker]#
[root@docker-125 docker]# cat /proc/buddyinfo
Node 0, zone DMA 1 1 0 0 0 1 0 1 0 1 3
Node 0, zone DMA32 1327 2671 1147 486 247 119 68 17 150 7 89
Node 0, zone Normal 12887 5801 1979 2912 2045 1074 587 287 77 965 10385
Node 1, zone Normal 8952 14349 6247 3145 1387 682 239 184 219 859 11351
[root@docker-125 docker]#
[root@docker-125 docker]# cat /proc/meminfo
MemTotal: 131418620 kB
MemFree: 95205956 kB
MemAvailable: 99051424 kB
Buffers: 948 kB
Cached: 2584240 kB
SwapCached: 0 kB
Active: 20370964 kB
Inactive: 2093064 kB
Active(anon): 19904196 kB
Inactive(anon): 61132 kB
Active(file): 466768 kB
Inactive(file): 2031932 kB
Unevictable: 673784 kB
Mlocked: 673784 kB
SwapTotal: 4194300 kB
SwapFree: 4194300 kB
Dirty: 16 kB
Writeback: 0 kB
AnonPages: 20553476 kB
Mapped: 82652 kB
Shmem: 76988 kB
Slab: 6925624 kB
SReclaimable: 1972792 kB
SUnreclaim: 4952832 kB
KernelStack: 480992 kB
PageTables: 327024 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 69903608 kB
Committed_AS: 237172272 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 2921124 kB
VmallocChunk: 34290683900 kB
HardwareCorrupted: 0 kB
AnonHugePages: 9410560 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 325584 kB
DirectMap2M: 5572608 kB
DirectMap1G: 127926272 kB


Now, I find that it is the systemcall "unshare" which throws the exception. But I do not know how to debug it further.

[root@docker-125 docker]# unshare --net
unshare: unshare failed: Cannot allocate memory
[root@docker-125 docker]#


and the kernel info:

[root@docker-125 docker]# uname -r
3.10.0-514.el7.x86_64


Any one who can help me?
Thanks.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: "Cannot allocate memory" even the free space is sufficient

Post by TrevorH » 2018/10/19 07:35:33

CentOS 7.3 is 2 years old and totally unsupported. First step therefore is to run yum update and reboot into the new kernel and libraries. Then retest and see if your issue still exists.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

Post Reply