tech.sweetnam.eu

I’m pretty attentive when it comes to securing my server and I can be exceptionally pedantic when deciding what constitutes bad behaviour or misuse of my webserver. Typically this would include crawlers that ignore robots.txt and script kiddies looking for certain versions of software etc.

To that end I tend to use a combination of blacklisted networks which are blocked on the webserver using shorewall and user agent matching which is performed on my reverse proxy.

For the past two days I have had repeated attempted crawls from Chinese netspace. All looking for the same vulnerabilities and almost all using libwww-perl as their useragent, an example of which is below:

61.151.239.75 – - [28/Jan/2009:18:08:38 +0000] “GET http://blog.sweetnam.eu:80//blogtest/xmlsrv/xmlrpc.php HTTP/1.1″ 403 2371 “-” “libwww-perl/5.803″ TCP_DENIED:NONE

The crawler above appears as being from CHINANET Shanghai province network but I have had literally thousands of scan attempts from many different Chinese addresses. So once more I have decided to completely block all of China from accessing my webserver in addition to the other hosts that I block as well.

The link below contains a list of IP addresses that my firewall rejects requests from. The first 763 lines are from China alone.

http://tech.sweetnam.eu/blacklists/blacklist.txt

The Irish Government have signed new legislation that means it is now against the law to send unsolicited text messages, emails or faxes. Anyone convicted will be fined up to €250,000 or 10% of their business’s turnover.

It’s a noble gesture and I’m sure one or two Irish based companies may fall foul of the new regulations but given that most spam comes from botnets I wonder how they figure on prosecuting them?

Source

I’ve written before about how I use my reverse proxy to block various bad bots and crawlers. At this stage I am blocking so much stuff that it would be way to much to post here. So if you are interested here are two links for you. The first is a copy of my squid regex file I use as per my tutorial and the second is a list of IP address and IP block that I have blocked on the proxy using IP tables.

Blockedbots.txt
Blockedip.txt