I just deleted a single file that was 60GB from a ext4 filesystem and so far it has taken 30 mins. It has also made our WordPress sites VERY slow. We have another more traditional site and this seems OK.
Whats going on and is there anyway round this?
Deleting one very big file brought server to its knees
-
- Posts: 2019
- Joined: 2015/02/17 15:14:33
- Location: Bulgaria
- Contact:
Re: Deleting one very big file brought server to its knees
How did you do that?
Most probably during the removal, the Linux kernel tried to cache the whole contents and thus you might have swapped.
Check the sar to find the swap & memory usage during the deletion.
I prefer to delete my files via:
But I have never deleted such a large file on a memory-constrained machine.
Most probably during the removal, the Linux kernel tried to cache the whole contents and thus you might have swapped.
Check the sar to find the swap & memory usage during the deletion.
I prefer to delete my files via:
Code: Select all
truncate -s 0 /filename
Last edited by hunter86_bg on 2019/02/16 17:12:20, edited 1 time in total.
Re: Deleting one very big file brought server to its knees
What device is the underlying filesystem sitting on? If it's an SSD then have you mounted the filesystem with the 'discard' option? That would cause it to trim all sectors that are allocated to that file as it's deleted.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Re: Deleting one very big file brought server to its knees
I am interested in an answer to this, over the years in work environment (on servers > 256GB RAM not using CentOS) I've never had this happen specifically but sometimes when deleting files/folders of significant size, with many users using server at same time, I sometimes see a lot of flush processes when doing a top and general performance of the server is severely degraded when this is going on. I don't use any SSD's so trim should not be relevant in my case, but even if a TRIM were happening shouldn't that happen behind the scenes and have no effect? Can you not issue a command and force TRIM to happen, would that then [always] bring your server to its knees?
if I had to guess, you have 8, 16 or 32 GB or RAM and trying to delete 60gb file causes some kind of disk caching to happen that exceed RAM capacity, went back to disk i/o using swap, system got crippled waiting on disk i/o. I don't see that kind of severity because I'm running 128, 256, or 512 gb of ram.
if I had to guess, you have 8, 16 or 32 GB or RAM and trying to delete 60gb file causes some kind of disk caching to happen that exceed RAM capacity, went back to disk i/o using swap, system got crippled waiting on disk i/o. I don't see that kind of severity because I'm running 128, 256, or 512 gb of ram.
Re: Deleting one very big file brought server to its knees
Deleting a file should just involve manipulating directory data. The file is still present on disk - that's why you never ebay a disk after just doing an rm. I think there is more going on here than we know about.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Re: Deleting one very big file brought server to its knees
It was a disk problem, the raid array was dying.