tech.sweetnam.eu

Recently I decided to go into the hosting business. Not in a big way mind you. Just to cover the costs of hosting primarily with perhaps enough left over for a night on the town once a month or to pay the phone bill.

As it happens I was lucky enough in that I had two clients ready to go with over 100 sites each. Starting up a business without having to look for clients is an absolute luxury especially in the middle of one of the worst recessions this country has ever seen. After making all the agreements with my future clients it was time to come up with a concrete migration plan.

Before a single site could be migrated several things needed to be taken into consideration:

• Hardware
• Software (Operating System and Control Panel)
• Support/Knowlegebase/Helpdesk
• Bandwidth requirements
• Data import/migration/compatibility

The hardware was the easiest. I had a spare Dell poweredge 1850 hanging around here at home doing nothing with a not too inconsiderate 6GB of RAM.  The operating system was going to be a simple matter too. It was going to be Linux or nothing. However which Linux distribution to choose?

Choosing the Linux distrubution was going to be directly related to my choice of control panel software. I wrote here sometime back about Virtualmin and I decided that it would be absolutely perfect for my control panel requirements. Having decided on the control panel software it was now up to me to choose between CentOS or Ubutu 8.04LTS as the host OS. In the end out of familiarity I opted for Ubuntu.

One of the key reasons for choosing Virtualmin was its ability to import backups from cPanel. As I would be migrating almost 200 sites from a cPanel server in the U.S. the ability to seamlessly migrate would be an absolute bonus. Another important reason was that the control panel interface itself is very easy to use. Considering that my two future clients were coming from years of using cPanel I was confident that they would easily find their way around Virtualmin.

In February I installed the server into the datacentre and pulled across a couple of cPanel backups to test the import functionality. Of primary concern to me was the existing server was running CentOS and using Exim as its MTA. The new server was the already mentioned Ubuntu and I had decided on Postfix as the MTA. In addition the home folders of the existing server were split between two disks mounted as /home and /home2

So I pulled across two backups, one with /home as its location on the old server and the other with /home2, used the import function and in about 5 minutes both sites had been migrated flawlessly.

I couldn’t have been that lucky I thought to myself. Normally something goes awry especially when there are significant differences in software as well as software versions. I poked around all the config files and was astonished to see that everything looked as it should be.

Over the next week I made out a schedule for migration and before long all sites were up and running. My clients were happy as too were my clients clients. The only issues that cropped up were minor and were as a result of differences in the way Virtualmin handles user accounts compared to cPanel.

So that was over a month ago and with everyone happy I can reflect on what was accomplished. Most notably was there anything I could have done differently or more efficiently.

As it happens yes, there was something I could have done had I thought about it.

The hardware of the server is absolutely overkill for what it is doing. 6GB of RAM is way too much. Over the past month the most I have ever seen in use was just short of 800MB however having lots of free RAM is not a bad thing as Linux likes to use lots of it for cache.

The load on the old server was constantly around the 1.00 to 2.50 mark, the new server with it’s dual core Xeons is barely even breaking a sweat with load averages between 0.03 and 0.12.

What I should have done, and it’s obvious now, is that I should have installed a hypervisor like VMware ESX server and paritioned the physical machine in to three virtual machines. One for each of my two clients and the third for myself.

In any event I have another Poweredge 1850 and a pair of 1750′s that I intend on installing into the datacentre in the not too distant future so now it is time to start planning for that.

To finish off here’s a screenshot:

It’s been absolutely ages since I last wrote here and I had been meaning to do so for some time. Since my last entry I have been tinkering away with my setup as I do on a regular basis and plenty of things have changed here on the back end. I suppose I will begin with my my connection and move up the chain from there.

First up is a new router. I picked up a nice Cisco 837 for a very reasonable price. I had a bit of fiddling to get it up and running with eircom broadband but I got there in the end and I have documented the procedure over on my wiki.

Next up is my reverse proxy. It is still running on the same hardware but I decided to move from Linux over to OpenBSD. My primary reason for this is that I was getting sick and tired of Linux iptables. In addition I concluded that Squids performance wasn’t optimal either. A move to OpenBSD seems to have resolved both those issues for me. PF on BSD is very configurable and easy to understand as well. Needless to say I have documented my experiences with PF over on my wiki also.

I’ve also gotten my hands on some new hardware. Well new hardware to me that is. I picked up three Sun Netra T1 servers, a Dell PowerEdge 1750 and two Symantec 5420 Firewall appliances.

So far I have been busy setting up the three Netras. They don’t have CD-ROM drives or display adapters so I had to install them over the network which was a process that was actually relatively simple. However it was only simple in that I already have a couple of other Sun machines here one of which I was able to use as an install server. Of course I have this process documented on my wiki also.

I haven’t decided what I will use the Netras for yet. One of them seems to have CPU problems and has been cannabilised to increase the specs. of the other two. They have 500Mhz UltraSparc IIe CPUs so by todays standards are not at all that powerful but they do draw very little power, certainly less than a Pentium IV machine so I may set them up as file servers. Initially I was hoping to install OpenBSD on one of them to use as my reverse proxy but unfortunately an OpenBSD network install does not seem to be as straight forward as Solaris.

The Poweredge 1750 I had hoped to set up as a webserver running either IIS or Apache. Unfortunately it only has 512MB of RAM so until I get some more for it that rules out Server 2008. However it is incredibly loud and that rules out keeping it running in my computer room. I am currently planning on setting up a dedicated computer room as it were in another part of the house so check back here for update on that.

Having being obsessed with computers and networks for many years I am always interested to find out the infrastructure behind some of the more popular sites out there. Quite often it is possible to glean bits of information here and there and occasionally an error may occur that offers a glimpse as to what is happening in the back end.

Today it is the turn of the Irish Independent. I got the error pictured here while browsing through their site. What is interesting about it is the domain name; externalcontent.independent.ie. As we can see it is an served by an Apache web server running on a Red Hat machine. However if you look at the error more closely it is a bit more telling.

The server in question (externalcontent.independent.ie) attempted to serve up an ad or content related to  loadzajobs.ie but was unable to contact the back end server. So this tells me that externalcontent.independent.ie is configured as a reverse proxy server and according to Netcraft is located in Ireland.

The primary domain; www.independent.ie, is hosted in the Netherlands and runs Apache Coyote, again according to Netcraft. Apache Coyote is a connector for Apache Tomcat. I find it curious that the main www site is located in the Netherlands but I suspect that it might have something to do with being connected to the Amsterdam Internet Exchange which is largest Internet Exchange in the world.

It does make me wonder why they chose AMS-IX given that here in Ireland we have INEX.

So what about the other national daily online newspapers?

The Irish Times is hosted on Linux and Apache and hosted in Dublin and The Irish Examiner is hosted on Windows Server 2003 and Microsoft IIS/6.

Although there are Irish editions of the Sun, The Star, The Daily Mail and The Mirror, none of them have specific Irish orientated sites but I will include them here nonetheless.

The Sun claims to be hosted on Linux and Apache however they use Akamai for content delivery so this could be inaccurate.

The Daily Star claims to be hosted on an unknown Unix and Apache and the Daily Mirror is hosted on RedHat Linux and Apache.

Finally the Daily Mail, like the Sun also uses Akamai for content delivery and claims to be running Linux and Apache.

It’s clear that Apache and Linux are the front runners.

I’m pretty attentive when it comes to securing my server and I can be exceptionally pedantic when deciding what constitutes bad behaviour or misuse of my webserver. Typically this would include crawlers that ignore robots.txt and script kiddies looking for certain versions of software etc.

To that end I tend to use a combination of blacklisted networks which are blocked on the webserver using shorewall and user agent matching which is performed on my reverse proxy.

For the past two days I have had repeated attempted crawls from Chinese netspace. All looking for the same vulnerabilities and almost all using libwww-perl as their useragent, an example of which is below:

61.151.239.75 – - [28/Jan/2009:18:08:38 +0000] “GET http://blog.sweetnam.eu:80//blogtest/xmlsrv/xmlrpc.php HTTP/1.1″ 403 2371 “-” “libwww-perl/5.803″ TCP_DENIED:NONE

The crawler above appears as being from CHINANET Shanghai province network but I have had literally thousands of scan attempts from many different Chinese addresses. So once more I have decided to completely block all of China from accessing my webserver in addition to the other hosts that I block as well.

The link below contains a list of IP addresses that my firewall rejects requests from. The first 763 lines are from China alone.

http://tech.sweetnam.eu/blacklists/blacklist.txt

I tend to have multiple ssh sessions open at once to my multiple *nix machines. However despite all the best attempts to keep some order all the windows tend to clutter up the desktop. Gnome terminal goes some way to easing that pain by supporting tabs but it would be even nicer if I could have multiple consoles all in a single window.

After a bit of digging I came across Gnome Terminator.

It’s a rather nice simple terminal console app but where it wins for me is that if you right click inside the terminal you can split the terminal horizontally or vertically or any combination of both. If you see my screenshot below it will give you a good idea. And yes almost all my machines are names after characters in Stargate!

Chances are that there are already pre-built binaries available for your distribution. At least there is for my Ubuntu workstation so it was a simple

sudo apt-get install terminator

It’s certainly worth checking out if like me you like to have lots of console windows open.

Terminator in action

I had a hard drive failure on my main workstation recently and after installing the new one I decided to partition it with Windows XP and Ubuntu 8.04. That was about six weeks ago and I have only booted into Windows once.

I would consider myself a pretty advanced Linux/Unix user. My first encounter with Linux was in 1995 with LinuxFT and not long after that with an early Slackware release. As the years have rolled by since then I have setup literally hundreds of Linux servers and scores of Solaris and HP-UX servers but Linux has never impressed me on the desktop.

Through one job I had eight years ago I had to use RedHat 6.1 as a desktop OS and I absolutely hated it. The look and feel was terrible. The early GTK widgets were hideous. Gnome was at version 1.4 and it was almost completely unusable. Around the same time I installed SuSE 6.0 on a spare PC at home and while it used KDE by default, it wasn’t much better. Installing fonts was a nightmare and the only decent browser available then was Netscape Navigator which used the Motif Widget set and was a nightmarisly ugly looking beast with which to browse the web.

How things have changed in eight years! A quick roll call of my computer room here at home comes up with the following:

1 HP 9000 running HP-UX
1 Reverse Proxy server running Ubuntu 6.06 LTS
1 Web Server running Debian Lenny 64bit
2 Sun Blade 2000 workstations running Solaris 10
1 Apple MacBook running OS X 10.5 Leopard
1 Dell Poweredge running Ubuntu 6.06 LTS
1 Main workstation running Ubuntu 8.04 LTS (Dual booting with Windows XP)
1 Firewall running Smoothwall Linux

And finally:

1 File server running Windows Server 2003.

I’m now down running just one single Windows machine from a maximum of about 8 over the years. And unlike before I don’t think I will be turning back. Most of the applications that I used on Windows were open source to begin with and naturally they have Linux versions. Firefox and Thunderbird being the most obvious two. WINE has matured to such an extent that when I’m no longer comfortable using The Gimp for certain tasks, Photoshop CS2 works under WINE like a charm.

However if there is a problem with Linux it is that there is probably too much choice. Particularly when it comes to your default desktop. I’ve finally decided on Gnome mainly because I couldn’t get Compiz working properly with XFCE. I had a look at KDE4 and I will probably have to wrte a seperate post about it. Suffice to say for the moment I don’t like the look of it one little bit.

So after six weeks I finally have my desktop looking the way I want it to. Gone is the Ubuntu Brown default theme replaced instead with one I liked from art.gnome.org . I’m still trying to find window decorations that I like but for the moment the default Ubuntu one is tolerable.

I still do think however that Linux is still no where even near ready for use as a mainstram desktop OS for the average user. But for the moment I’m finally impressed enough that I can eschew Windows at long last.