Locked out by process using 100%

General support questions
Post Reply
Latenights
Posts: 7
Joined: 2018/01/25 08:51:48

Locked out by process using 100%

Post by Latenights » 2019/11/07 02:04:18

I have occasionally had this problem but it has happened twice this week. I have the most recent Centos 7 updates. The GUI and user input are frozen. Ctrl-Alt-F2 does not switch me to another console. The SSD hard disk light is on and the CPU fan is screaming. I do not believe it is the hard disk because if I act fast I can recover. I've watched the problem start, and it worsens exponentially over a few seconds. Using the "top" command I see that it seems both Firefox and Chome randomly cause this, because when it starts I have about 10 seconds to kill the browser then suddenly the fan speed goes back to normal, I get full mouse control and the HD LED turns off.

From my Unix days I know that OS gave limited resources to different processes and no process was allowed to use 100%, in which case this would never happen. Is Linux behind the curve on this or is there a way I can set Centos to always reserve resources so I can get control back besides pulling the power plug?

anthonynorth
Posts: 7
Joined: 2019/11/01 17:21:30
Location: Brighton, UK

Re: Locked out by process using 100%

Post by anthonynorth » 2019/11/07 09:53:08

All Linux variants still become slow and unresponsive if given too much load, although it would be unusual if this was driven by a web browser. Could it be that your machine has been compromised?

User avatar
jlehtone
Posts: 2430
Joined: 2007/12/11 08:17:33
Location: Finland

Re: Locked out by process using 100%

Post by jlehtone » 2019/11/07 15:03:12

There is usually some swap space on disk; extension of RAM. When processes use lots of memory, swap will be used. The actively executed processes' memory has to be in RAM. There will be transfer between RAM and swap as different processes get CPU-time.

If processes use all RAM and all swap, then oom_killer (Out Of Memory) kills some process to reduce usage. Hopefully the user processes rather than system processes.

There is cgroups (control groups) system that can limit both memory and CPU usage of a process/session. The configuration has to be added; there is no default.


Extensive swapping does not tax CPU. If the CPU is hot, then there are some processes that tax it.
Package sysstat is probably installed. It logs to /var/log/sa/ the usage (CPU, mem, swap, I/O, network) of the system (daily files, for a month). There one could see whether CPU, memory, or both were at 100%.

User avatar
Errosion
Posts: 25
Joined: 2014/12/03 19:58:02

Re: Locked out by process using 100%

Post by Errosion » 2019/11/07 15:19:23

This sounds to be me like you have something else going on with your system.

There is no reason your browser should be chewing up IOs and CPU time at that fast a rate.

systat should help identify when this happens and maybe the root cause. I'd recommend dropping the rate that stats are pulled to every minute. (you could probably even go lower by playing with that cron) I believe it defaults to every 10 minutes.

Does it happen on an idle system?
Is it only if you are running a browser?
When the problem starts, you catch it and close the browser. Does the system go back to normal?
If it does go back to normal, what happens if you reopen that browser?

It seems like you might have something else happening (hardware or software) on the system that is causing this issue. Identifying that might be the better option than forcing your system to limit resources.

Could you also be running all of this on a system that barely meets the system requirements?
And if/when an app goes to do some automatic update, it pushes you over the limit on resources?

Latenights
Posts: 7
Joined: 2018/01/25 08:51:48

Re: Locked out by process using 100%

Post by Latenights » 2019/11/07 23:43:44

Errosion, generally I use this system for web-browsing, Chrome. Yesterday I came back into the room and the screen was locked because I had been gone over 15 minutes. The HD LED was continuous on, the fan was at max, and I couldn't log in. When it starts it ramps up then the mouse pointer gets jerky and sticks. If I can close the browser before it totally freezes then yes, everything calms down and it goes back to normal. I might have 10-12 tabs open in the browser. I have a dual core 3Ghz with 4GB ram.

I agree I might be straining the system resources, but I would expect the system not to let an application take 100%. That includes preventing automatic updates take 100%. Or that something else might be going on, and I am here looking for ideas. I have found sysstat and I will see how to use that to identify the problem process. This problem happens maybe once a month.

desertcat
Posts: 662
Joined: 2014/08/07 02:17:29
Location: Tucson, AZ

Re: Locked out by process using 100%

Post by desertcat » 2019/11/08 11:01:16

Latenights wrote:
2019/11/07 23:43:44
Errosion, generally I use this system for web-browsing, Chrome. Yesterday I came back into the room and the screen was locked because I had been gone over 15 minutes. The HD LED was continuous on, the fan was at max, and I couldn't log in. When it starts it ramps up then the mouse pointer gets jerky and sticks. If I can close the browser before it totally freezes then yes, everything calms down and it goes back to normal. I might have 10-12 tabs open in the browser. I have a dual core 3Ghz with 4GB ram.

I agree I might be straining the system resources, but I would expect the system not to let an application take 100%. That includes preventing automatic updates take 100%. Or that something else might be going on, and I am here looking for ideas. I have found sysstat and I will see how to use that to identify the problem process. This problem happens maybe once a month.
I've run into this problem once or twice, and has happened because of multiple spawning of the same program, or because I am trying to run a VM running Windows 10 (A resource hog for sure) plus several other programs simultaneously. As a comparison I have a hexacore (6 core) CPU and have 32 GB of RAM, a 500 GB SSD, and dual monitors. Yes, it is a light duty research workstation. Way overkill for day-to-day activity... until I start to need it because I am doing research, have 10 programs open and running and am jumping back and forth from one monitor to the other. A live system monitor tells me how many of the 6 cores are being used and things such as temperatures etc.

There are two things that jump out at me:

1) You have 10-12 (!!) tabs open in your browser at the same time. I really start pushing the limits when I have 8 open at the same time -- limits not so much hardware wise as being to follows the flow of where everything is. My 24" monitor simply does not have enough space on top to show the contents of every tab.

2) The second thing that jumps out at me is that you running a rather wimpy (sorry) computer with the dual core processor, when the default is now a quad core CPU.

3) And third you have only 4 GB of RAM. Again rather wimpy. Today the default is 8 GB.

In short you are over taxing the system. Can you solve the problem? Maybe. One quick thing to do is to close the number of open tabs down to no more than 4. Second if possible increase the amount of system RAM to 8 GB or more. Finally you never answered the question of how much SWAP you have. In the "old days" it would be double the amount of RAM, which in your case would be about 8GB. One other thing you could do would be to turn off the screen locker and simply use a simple Screen Saver.

For surfing the web, answering email, etc., your system is more than enough; but if you are trying to do research on it your system is way under powered. Depending on the vintage of your computer, and depending the intended use of the computer, it is worth while to at least upgrade the amount of system RAM, to simply retiring the system and investing in a new system. You can build a new system with room for growth for ~ $600 or less. Most off the shelf computers cost at least that amount and have next to zero room for growth.

Immediately in the very sort term close the number of open tabs down to 4 or less, and second kill the screen locker and switch to a simple screen saver.

Post Reply

Return to “CentOS 7 - General Support”