Use come command-line fu to analyse your website log in real-time and find out which IP addresses are making the most requests.
Who’s Hitting on Me?
There are many reasons that you might want to analyse your website access log in order to find out which IP addresses are making the most requests. The most common reason would be if your content is getting scraped. You can identify and block offending IPs.
Here’s a quick command that will analyse the last 50,000 lines of your access log and show the top 10 most frequent visitors, along with the number of requests from each:
tail -50000 access_log | awk '{print $1}' | sort | uniq -c | sort -n | tail |
For the original source and a few other ideas check out the discussion at WebmasterWorld.