CentOS slows down when copying large files
Re: CentOS slows down when copying large files
I'm not sure what "copying issue" you are refering to and heard is fixed in later kernels. I've never heard of this. What Linux does here is a feature not a bug. It buffers writes and then does them asynchronously to improve performance. Only problem is that per default a lot of writes are buffered and this is problematic for external and or/slow drives. If you want to unplug your external drive, all the copying you thought is done actually isn't. And if the drive is slow, the dirty buffers bind the memory they are using for a long time.
I've put the three lines mentioned above in /etc/rc.local on every desktop/laptop I use. They are even useful on servers, that have a RAID controller with lots of cache. Writes are better buffered there than in the server's RAM.
I've put the three lines mentioned above in /etc/rc.local on every desktop/laptop I use. They are even useful on servers, that have a RAID controller with lots of cache. Writes are better buffered there than in the server's RAM.
-
- Posts: 35
- Joined: 2017/08/26 22:32:23
Re: CentOS slows down when copying large files
The accepted answer in the following Stack Exchange (which suggested the same solution you offered in your previous post) mentioned that the buffering writes behavior that causes the slow down I am experiencing should be fixed in later kernels.
https://unix.stackexchange.com/question ... a-pendrive
As I said I am not sure if that is true or not. It is just something I read while researching the issue.
https://unix.stackexchange.com/question ... a-pendrive
As I said I am not sure if that is true or not. It is just something I read while researching the issue.
Re: CentOS slows down when copying large files
A nice test is mentioned there. Type
dd if=/dev/zero of=foo bs=1M count=10k
in one terminal and start some large program (not used since last reboot, it must come frome disk) in another, e.g.
oowriter
It will not pop up on my CentOS 7 machine until dd is through.
Seems I'm not doing this normally.
You might want to try a more recent kernel (from elrepo perhaps) to check if things got better.
dd if=/dev/zero of=foo bs=1M count=10k
in one terminal and start some large program (not used since last reboot, it must come frome disk) in another, e.g.
oowriter
It will not pop up on my CentOS 7 machine until dd is through.
Seems I'm not doing this normally.
You might want to try a more recent kernel (from elrepo perhaps) to check if things got better.
-
- Posts: 35
- Joined: 2017/08/26 22:32:23
Re: CentOS slows down when copying large files
I tried updating the kernel with elrepo's kernel-ml. The installation went through successfully, but the kernel boots to the CLi. I thought it might be an issue with blacklisting nouveau, since I have the nvidia driver installed. I checked the boot options in the GRUB menu for the elrepo's kernel and CentOS kernel and they are both identical. I even reinstalled kmod-nvidia after booting with the kernel-ml to make sure that whatever it does to nouveau is done properly with the new kernel installed, but I still can't get the GUI to load.
Any ideas what might be causing that?
Re: CentOS slows down when copying large files
When you update the kernel you must reinstall the nvidia drivers. The ELRepo kmod-nvidia packages that allow the drivers to survive kernel updates only work with the distro kernels so if you were using those before you'll have to switch to the ones from nvidia themselves.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
-
- Posts: 35
- Joined: 2017/08/26 22:32:23
Re: CentOS slows down when copying large files
That’s a shame. I never managed to get the drivers from nvidia to work properly. Tried different solutions but it never worked. Don’t think it is worth trying it again as I can’t afford my machine going down now.TrevorH wrote: ↑2018/09/23 16:54:51When you update the kernel you must reinstall the nvidia drivers. The ELRepo kmod-nvidia packages that allow the drivers to survive kernel updates only work with the distro kernels so if you were using those before you'll have to switch to the ones from nvidia themselves.
Guess I will have to stick with the district kernel and continue to rely on that dirty_bytes solution that chemal provided.
Re: CentOS slows down when copying large files
Code: Select all
# don't buffer too many writes
echo $((16*1024*1024)) > /proc/sys/vm/dirty_background_bytes
echo $((48*1024*1024)) > /proc/sys/vm/dirty_bytes
# enable completely fair queuing
echo cfq > /sys/block/sda/queue/scheduler
-
- Posts: 2019
- Joined: 2015/02/17 15:14:33
- Location: Bulgaria
- Contact:
Re: CentOS slows down when copying large files
If CFQ is helping, then you can set it as a default scheduler, but keep in mind that is not suitable for SSDs.
Re: CentOS slows down when copying large files
For consumer-grade SATA SSDs, CFQ is ok. It has SSD optimizations (see kernel docs). And it's the only way to solve problem #2 (background writes hog drive) with the distro kernel as far as I can see.
Re: CentOS slows down when copying large files
Some additional tests with an NVME SSD (Samsung SM961). NVME SSDs have multiple hardware submission queues and RH has backported multiqueue I/O scheduling. So even with the default 'none' scheduler, the background writes do not monopolize the SSD. Oowriter starts in parallel to heavy dd-ing. But the number of parallel reads that come through is low, it takes a lot of time until oowriter pops up. Things are much better with the kyber scheduler.