Page 1 of 1
weird logwatch report - can anybody explain ?
Posted: 2018/08/08 11:13:52
by aegersz
Hi, i run a forum on a CentOS 6.9 30 GB VPS and check logwatch frequently but once i a while i see something BIZARRE like this:
Code: Select all
--------------------- httpd Begin ------------------------
450508.69 MB transferred in 116259 responses (1xx 0, 2xx 103076, 3xx 10374, 4xx
2559, 5xx 250)
22258 Images (386.32 MB),
8 Documents (1.36 MB),
4554 Sound files (448962.15 MB),
87847 Content pages (1091.79 MB),
21 Redirects (0.01 MB),
1571 Other (67.07 MB)
Note the: 4554 Sound files (448962.15 MB) - how could a whopping 400+ GB amount of data get transferred from a system whose disk footprint is only 30 GB:
Code: Select all
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 30G 22G 6.4G 78% /
tmpfs 499M 0 499M 0% /dev/shm
how can this happen ? i am very curious as how to have it explained, if possible.
Re: weird logwatch report - can anybody explain ?
Posted: 2018/08/08 11:50:04
by MartinR
There were 103076 successful transfers. 400GB / 100k transfers = 4MB per transfer on average. 4MB for a sound file seems reasonable, it's probably the same few files being transferred may times.
Re: weird logwatch report - can anybody explain ?
Posted: 2018/08/08 12:10:07
by aegersz
Thank you MartinR, I really appreciate your explanation and also felt that it could've been the same file transferred many times but couldn't understand why.
bots and search engine crawlers index my site but this particular amount of data is HUGE.
any thoughts as to why this would happen ? (a rogue bot, perhaps ?)
Re: weird logwatch report - can anybody explain ?
Posted: 2018/08/08 12:35:04
by avij
You could check the log files directly to see what is going on. They are probably in /var/log/httpd
One possibility is that the bot(?) that made the requests only actually downloaded the first few kilobytes of each file, but your httpd has been configured to log as if the entire file got downloaded. There is a config option for this, but I don't remember the details offhand.
Re: weird logwatch report - can anybody explain ?
Posted: 2018/08/08 13:12:16
by aegersz
thanks again !, i am actually installing the ELK stack to monitor log files so may be that will help one day.
I didn't know of such an option in appache.
i might let this one go unless any other thoughts come into play.
Re: weird logwatch report - can anybody explain ?
Posted: 2018/08/08 17:22:36
by MartinR
http://httpd.apache.org/docs/2.4/logs.html seems to imply that the number returned is the data actually sent. The
http service in
logwatch uses the
http logfile group which searches for
*access{_,.}log in multiple places. The figures returned are therefore those in the access log.
Have you tried getting more detail from logwatch? Try
logwatch --service http --detail=high
Re: weird logwatch report - can anybody explain ?
Posted: 2018/08/09 02:32:28
by aegersz
i did, yesterday but no help on where the data is going.
i do host a little bit of music on my site and it's about 5 GB worth in total.
if only i could see what IPs transferred what .wav/.mp3 ... *sigh*.
my forum's topics are very sensitive so i get curious about glaring inconsistencies.
a lot of data moves daily and it's getting me a little paranoid.
Re: weird logwatch report - can anybody explain ?
Posted: 2018/08/09 04:58:35
by Whoever
aegersz wrote: ↑2018/08/09 02:32:28
if only i could see what IPs transferred what .wav/.mp3 ... *sigh*.
Install awstats, it is in the EPEL repo.
Re: weird logwatch report - can anybody explain ?
Posted: 2018/08/09 06:00:12
by avij
grep mp3 /var/log/httpd/access_log | more or something similar should help.