So I take no credit for this but simply repeat it here so that the search gods may recognise it’s usefulness and +1 it’s importance.

Start by downloading VMware Server 2.0.2 from the official VMware site. If you haven’t already got a few licenses, get one now. (They’re free so you might as well get a few) I’m going to assume the downloaded file is in your home directory.

You also need to update the header files for your current kernel so that the configuration scripts from VMware can build the appropriate modules.

sudo apt-get install linux-headers-`uname -r` build-essential

Now just run the following commands.

cd /usr/local/src
sudo wget [http://codebin.cotescu.com/vmware/vmware-server-2.0.x-kernel-2.6.3x-install.sh]
sudo tar xvzf raducotescu-vmware-server-linux-2.6.3x-kernel-592e882.tar.gz
cd raducotescu-vmware-server-linux-2.6.3x-kernel-592e882/
sudo cp /home/<your_username>/VMware-server-2.0.2-203138.i386.tar.gz .
sudo tar xvzf VMware-server-2.0.2-203138.i386.tar.gz
sudo chmod +x vmware-server-2.0.x-kernel-2.6.3x-install.sh
./vmware-server-2.0.x-kernel-2.6.3x-install.sh

If you have a previous installation of VMware Server, you’ll be prompted that it’ll be removed as part of the install. Don’t worry, any guest VMs you had should still be there afterwards. The script will run through the usual prompts and you’ll see references to the patched files from Radu Cotescu. After a few minutes you should have a working install of VMware Server 2.0.2 on your Ubuntu 10.04 server.

In this post I’ll pass on the methods I found to get PHP and your Linux boxes talking out through a corporate ISA proxy server. You can then bring in RSS feeds, updates and other things in WordPress and use apt-get to update Ubuntu.

cntlm

To start with we need to install cntlm on a Linux box (I’ll mention Ubuntu here but any flavour should work with a couple of tweaks). cntlm is “an NTLM / NTLM Session Response / NTLMv2 authenticating HTTP proxy“. Basically your non-Windows machines and apps can now talk to cntlm which will then talk to the ISA proxy in your organisation. So you stay within your corporate infrastructure and keep the security guys happy.

Download the deb installation package to your Ubuntu machine and install it with,

sudo dpkg -i cntlm_0.35.1-1_i386.deb

Open the config file for cntlm and edit it to add in your ISA proxy information and a valid user, domain and password that has web access with download rights. Further down the settings make sure gateway is enabled and you set suitable restrictions on the IP ranges that can use cntlm.

sudo nano /etc/cntlm.conf

Restart cntlm to make sure it gets the new settings,

sudo /etc/init.d/cntlm restart

Test your cntlm configuration by attempting to get to an external website,

sudo cntlm -M http://www.google.com -u your_user@your_domain -p your_password your_isa_proxy:port

apt-get

Configure apt-get to use the new proxy by editing the apt-get config,

sudo nano /etc/apt/apt.conf.d/70debconf

and add this line pointing to your cntlm proxy gateway. Change the IP and port if you’ve installed cntlm on another machine or different port.

Acquire::http::proxy "http://127.0.0.1:3128";

WordPress

Add these extra lines to the wp-config.php file in your WordPress directory. This works from version 2.8 onward and means you can keep your install and plugins up to date and secure. Again, make sure you change the IP and port to that of your cntlm proxy gateway. You may be able to comment out the username and password if things don’t work as expected. It’ll depend on your cntlm configuration.

define('WP_PROXY_HOST', '127.0.0.1');
define('WP_PROXY_PORT', '3128');
define('WP_PROXY_USERNAME', 'my_user_name');
define('WP_PROXY_PASSWORD', 'my_password');
define('WP_PROXY_BYPASS_HOSTS', 'localhost, www.example.com');

With any luck you now have a Linux machine and WordPress install that can easily be kept up to date inside your corporate environment.

This post will show you how to create two MySQL servers that automatically replicate all their databases using DRBD. With Heartbeat installed on a third machine you’ll have basic fail over protection as well (we’ll do this in another post). Only one of the database servers will be active at any one time.

To follow along exactly, you’ll need the following items set up or readily at hand:

• VMware Server or ESXi, etc to host the virtual machines we use. You could just as easily use physical machines if you have the hardware lying around.
• Ubuntu 9.04 server – other versions or distros should work just as well but you may have to make some changes to the steps as you go through.
• Some basic skills in cmd-line usage of Ubuntu and MySQL.
• Being Ubuntu we’re using a whole heap of sudo prefixed commands here. If you want to drop them out, you can switch to a root session with sudo -s.

Start things off by creating a virtual machine with 512MB RAM and two 2GB disks attached. One disk will be for the OS and one will be used as the data location for MySQL. You can change the RAM or disk sizes as you like but we will be duplicating this machine so don’t use up all your RAM and disk space with the first one.

Install Ubuntu 9.04 server onto this machine and name it node0 (that’s n-o-d-e-zero if the font isn’t clear). Just use one of the disks for the OS and don’t format or prepare the second one yet. You can also leave out the LAMP and other installation packages for now as we’ll grab what we need later.

Once the install is complete, reboot the node0 server, login, run a package update and install MySQL and DRBD with,

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install mysql-server mysql-client drbd8-utils heartbeat

You’ll have to download roughly 45MB of packages for those four items and the extras that come with them.

You now have one server configured with the software we need to do the rest. Depending on your luck, you now have two options for creating the node1 virtual machine.

1. Duplicate the Virtual machine in VMware

To make things easy, we’ll simply copy the node0 machine in VMware to create our node1. Make sure you shutdown the node0 machine in VMware and (assuming your VMware server host is Ubuntu too) open up a cmd prompt and run,

cd /var/lib/vmware/Virtual\ Machines
sudo cp -r node0 node1

Once that’s completed, you should be able to open the node0.vmx file that’s in the new node1 folder using VMware console. VMware will add the ‘new’ virtual machine to it’s inventory and you’ll probably have to rename it as VMware will have kept the same node0 name.

Start up your new node1 machine and watch for VMware to prompt you about the machine ID. You’ll need to say you copied the VM to create a new virtual machine ID in VMware.

Log in to node1 and change the server name in /etc/hostname and /etc/hosts files, so the two servers don’t get confused over their identities. Shutdown node1 for the changes to take effect. While the server is offline, edit it’s VMware configuration to remove the NIC and then re-add it. You need to do this to keep the virtual networking in VMware happy.

Depending on how you network is configured, it may be a good idea to set both servers to have static IP addresses. If you don’t want to do this, make sure you know their IPs and add them to the “other” server’s /etc/hosts file. This will mean that node0 can reach node1 by it’s DNS name and vice versa. Test it’s all working with a ping from each machine to the other.

2. Repeat the Ubuntu Installation

OK, so maybe your virtual machine just didn’t want to be duplicated like mine. If so, just create a new VMware guest config with the same settings as the first and install Ubuntu server the same way.

3. Disk Configuration

Now that we have our two node servers configured and the required apps installed, let’s practice safe-virtualisation by taking a VMware snapshot of both servers. Then if it all blows up in the next few steps, at least we’ll have something to go back to.

Let’s create a new disk partition on that extra virtual hard disk we added to both our machines. On both node0 and node1 use fdisk to add a Linux type 83 partition.

sudo fdisk /dev/sdb

Press p to make sure you have the right disk and there are no current partitions. Press n and create a primary partition with the ID of 1, accept the other default options. Press w to save the changes and exit fdisk.

4. Configure DRBD

We can now configure DRBD to use this empty partition space we’ve created. Like many other Linux apps we need to set up DRBD by editing a config file – /etc/drbd.conf

On node0, make your drbd.conf file look something like the file linked below. There are plenty a massive number of other settings and comments but these are the key ones – assuming you’ve used the same disk names as above and apply your own network addresses. My server IPs were node0:192.168.0.17 and node1:192.168.0.18 so replace those with your information. Take your time to read through at least some of this file as other things like disconnects on failures may come in handy.

drbd.conf

From node0, copy that drbd.conf file to your home directory (~) on node1 using the scp command,

sudo scp /etc/drbd.conf node1:~

You’ll have to accept the RSA key for node1 if you haven’t SSH’d between those servers yet.

And on node1 we need to move that file to the correct place,

sudo mv ~/drbd.conf /etc/

DRBD will now start, but we need to prepare the partition we created earlier so that DRBD can use it. Turn off node1 and on node0 do this,

sudo drbdadm create-md mysql

Restart node0 and log back in again. Typing ‘yes’ to DRBD’s timeout will speed things up. You’ll see some warnings about using the ‘drbd-peer-outdater’ in a mode where something else needs root privileges. Just ignore that for now and we’ll make those changes later.

Run cat /proc/drbd on node0 to have a look at the messages logged when drbd started up. We can see that node0 is in an unknown state as it can’t see what’s happening on node1 properly. So lets sort out node1.

On node1 create the drbd file system just like a couple of minutes ago,

sudo drbdadm create-md mysql

And restart DRBD on node1 to make sure it’s using that new data,

sudo /etc/init.d/drbd restart

Run cat /proc/drbd again to check the status on both node0 and node1 and you should see that they are both connected and set as secondary with an inconsistent status. If you don’t see that then give both nodes a reboot and try again. If you still don’t see that output, start back tracking and check the previous steps.

At this stage both nodes are essentially looking at each other but don’t know who is the primary so they’re not going to do anything. By running the following command on node0 you make that machine the primary and data will start to be replicated straight away.

sudo drbdadm -- --overwrite-data-of-peer primary mysql

You can either use cat /proc/drbd or watch -n1 cat /proc/drbd to view the status of the replication as it happens. This may take some time, even for the relatively small 2GB of empty space we are using, as DRBD will replicate every block in the device /dev/mysql. Between the VMs on an old P4 with a PATA disk I was seeing about 2.8 MB/s.

When you ran that command to set the primary, you probably got some messages about the heartbeat program requiring root privileges to run drbdsetup and drbdmeta. Do that now to get rid of those warnings.

sudo chgrp haclient /sbin/drbdsetup
sudo chmod o-x /sbin/drbdsetup
sudo chmod u+s /sbin/drbdsetup
sudo chgrp haclient /sbin/drbdmeta
sudo chmod o-x /sbin/drbdmeta
sudo chmod u+s /sbin/drbdmeta

Prepare, Mount & Test the DRBD Device

Now although we have our two nodes talking and set as primary and secondary, the disk space is not formatted or mounted for any apps (i.e. MySQL) to use. So we first format our /dev/drbd0 device with ext3 and a nice filesystem label.

sudo mkfs.ext3 -L mysql /dev/drbd0

Then we create the mount point on node0 and mount the new filesystem. Whatever gets dropped into /mnt/mysql will magically be synchronised from the primary node to the secondary.

sudo mkdir /mnt/mysql
sudo mount /dev/drbd0 /mnt/mysql

To test our new setup copy some files and folders into /mnt/mysql and unmount the DRBD device. We then switch the primary node0 to be the secondary and jump onto the node1 server.

sudo cp -r ~ /mnt/mysql
sudo umount /mnt/mysql
sudo drbdadm secondary mysql

Now move to the node1 server and create the same mount point, mount our DRBD device and we’ll see what’s inside.

sudo mkdir /mnt/mysql
sudo mount /dev/drbd0 /mnt/mysql

Like magic you should now see that same data you copied on node0, has appeared in the same device on node1.

Now switch the roles back again so node0 is the primary and node1 is the secondary, and delete the test data in /mnt/mysql.

Configure MySQL

The core settings for MySQL is in a file called /etc/mysql/my.cnf and it’s this file that we are going to edit to tell MySQL to use our DRBD device to store it’s databases, indexes and logs.

At this point it’s very important that you do not have any MySQL databases already running on node0 or node1 – none that you want to keep anyway. Let’s stop the MySQL server and backup our original MySQL settings so we can then make changes on node0.

sudo /etc/init.d/mysql stop
cd /etc/mysql
sudo cp my.cnf myold.cnf

Open up the my.cnf file in your favorite text editor like vi or nano, and change the datadir option as below. If you have logging enabled, or will at some point, you may want to synchronise the log files as well. For more information on other options in this file see the MySQL documentation.

datadir = /mnt/mysql/data

Save my.cnf and make the same changes on node1. We have to repeat this as these files aren’t in out replicating DRBD device.

Run the following commands to set the correct permissions on the new location and also update the AppArmor settings to allow access. This is important else the mysql user and group will not be able to access the new directory properly and you will get errors. Trying to restart the MySQL daemon at the moment will fail.

sudo chown -R mysql:mysql /mnt/mysql
sudo nano /etc/apparmor.d/usr.sbin.mysqld

On the bottom of the list of locations, add the following (within the curly closing bracket } ),

/mnt/mysql/data/ r,
/mnt/mysql/data/** rwk,

Restart AppArmor to pick up the new locations for MySQL,

sudo /etc/init.d/apparmor restart

Make the same changes to AppArmor on node1 – again these changes are server based and not automatically replicated.

We should try and move the previous databases into the new location with the following commands. Repeat the first copy command for any other databases you might have. (That mysql directory we’re copying first is the MySQL system database.)

sudo cp -r /var/lib/mysql/mysql /mnt/mysql/data/
...
sudo chown -R mysql:mysql /mnt/mysql/data/*

The last thing to do is restart MySQL and check it’s working.

sudo /etc/init.d/mysql start

If this gives permission errors about the debian-sys-maint user, check your permissions and/or restart the server. Restarting MySQL might not be enough.

Connect to mysql as the root user and let’s just create a new database and table to test things before we do another manual fail-over to node1.

mysql -u root -p
create database drbdtest;
use drbdtest;
create table users (id INTEGER(9), name VARCHAR(255));
exit;

You can now see your new database files in /mnt/mysql/data/drbdtest. So we fail-over from node0 to node1 again, just for practice. We must stop the mysql server before we unmount the DRBD device else very bad things may happen.

sudo /etc/init.d/mysql stop
sudo umount /mnt/mysql
sudo drbdadm secondary mysql

and over on node1 . . .

sudo drbdadm primary mysql

And check those database files are there with the mysql client or looking in /mnt/mysql/data/drbdtest.

Conclusion

So we can see that this will save us (with some manual intervention), when our MySQL storage falls over. But wouldn’t it be nice if there was an automated way to make this happen? See the next article on using Heartbeat to make this happen.

Potential Issues

If you try and mount /dev/drbd0 and get an error about the device being read-only, check that the DRBD node you’re on is in the correct mode i.e. primary or secondary.

I found an older post in the Ubuntu forums that had the fix for USB devices and it also works for PS2 with the simplest of changes. So follow these steps and you should be saving power and getting back to work faster.

Open a Terminal and type,

cat /proc/acpi/wakeup

Note the entries that come back and you should see a device called “PS2K” toward the top if you have a PS2 keyboard. For those with USB, it’ll be one of the USB items toward the bottom. The entry will probably also have “Disabled” on the same line, hence your problem.

To enable this entry, switch to a root session by typing,

sudo -s

and enter your password. Now type the following to update the acpi file and toggle “disabled” to “enabled”, (those with USB devices can try USB0, USB1, etc)

echo PS2K > /proc/acpi/wakeup

That should have now enabled your PS2 keyboard to wakeup your PC for this session. Give it a test by putting your machine to sleep and then tapping a key on your keyboard. Probably a good idea to save stuff first, just in case.

If you tried changing a USB device, it may take a few guesses until you find the KB. My mouse was USB0 and clicking any mouse button can also do the wakeup task.

To make this change permanant, you need to add that line to a script and run it when Ubuntu starts. So we create a file called wakeup.sh with the following contents,

#!/bin/bash
echo PS2K > /proc/acpi/wakeup

Save it and from a Terminal make it executable so it runs properly as a script and not just a text file,

chmod +x wakeup.sh

Now to add it to the startup area go back to your Terminal that’s running as root. We need to copy the file to the correct location and add it to the startup processes. You’ll need to run the cp command in the same folder as where you saved your wakeup.sh file.

cp wakeup.sh /etc/init.d/wakeup.sh
update-rc.d wakeup.sh defaults

Now when you reboot, the script will run and enable your PS2 keyboard in ACPI so you can wakeup your PC.

For the others in the audience that are installing on Ubuntu 8.04 or another Linux system that doesn’t have packages, you should have a working VMware Server install with web interface and a client console by the bottom of the page.

Downloads

There are a few things you will need from VMware’s site before we kick off. Download these to your server machine.

VMware Server 1.06 is the latest version of the free product, released in late May 2008.

The web based Management User Interface (MUI) onto your VMware Server install. The files for VMware Server 1.06 are available here.

System Extras

Your Linux (in this case Ubuntu 8.04) system will require some extra libraries and bits if you haven’t already done so. Run the following on both the server and the client (if you’re installing the VMware console too).

sudo apt-get install build-essential linux-headers-`uname -r` xinetd

As you progress through the install you may find that your system has extra requirements. It all depends on the setup and libraries already installed on each machine. So you have the option of installing these ones on your server now or just keep going and come back if needed.

sudo apt-get install libx11-6 libx11-dev libxtst6 libICE6 libxt6 libxrender1 libxi6 xfsprogs

Install

Extract both the tarballs you’ve download,

tar -xvzf VMware-server-1.0.6-*.tar.gz
tar -xvzf VMware-mui-1.0.6-*.tar.gz

switch to the Server install folder,

cd vmware-server-distrib

and run the installation script as root,

sudo ./vmware-install.pl

You may see errors during the install. If so, run these two commands to cover for a couple of missing libraries for cairo and gcc (tip courtesy of Ubuntu Tutorials)

sudo ln -sf /usr/lib/gcc/i486-linux-gnu/4.2.3/libgcc_s.so /usr/lib/vmware/lib/libgcc_s.so.1/libgcc_s.so.1

sudo ln -sf /usr/lib/libpng12.so.0 /usr/lib/vmware/lib/libpng12.so.0/libpng12.so.0

At the end of the install you’ll be asked for the registration key that you can get from the VMware website. I usually request 10 and copy and paste them to a safe place. Be aware that the Linux and Windows keys are different and can’t be interchanged.

When the key has been accepted the install tidies up and you should see the VMware Server starting and giving the OK message. To double check, just restart it again with,

sudo /etc/init.d/vmware restart

You now have a working VMware Server but no easy way to control it. So we need the MUI and the Console.

Switch folder back to the MUI source folder that you extracted earlier and run the install script,

sudo ./vmware-install.pl

Once that’s completed, you should be able to connect from your client system’s web browser to https://<server>:8333 as long as you haven’t changed the port.

Again, depending on the state of your system, you may need to run the following if you have connection problems. I’ve found this fixes the SSL generation error (“starting httpd.vmware:-ne failed“) you may have at the end of the MUI install.

sudo ln -s -f /bin/bash /bin/sh

sudo vmware-config-mui.pl

Log on to https://<server>:8333 as your server user and you’ll see you can’t do too much. No options to create new VMs or change too many options. You need to install the VMWare console on your client machine.

Client Console Install

Installing the client console software is pretty easy. If you’re on a Windows machine just grab the client ZIP file from VMware, extract, double-click and you’re done.

For the Linux (specifically Ubuntu 8.04) world, use the links on those VMware MUI pages to download the console tarball and extract to a folder on your client machine. Remember that the rest of these instructions should be run on your Linux client and not your server.

cd vmware-server-console-distrib

sudo ./vmware-install.pl

Just follow the instructions through and accept the defaults. Look out for any ominous error messages, but you shouldn’t see any.

Test things out by starting the console with,

/usr/bin/vmware-server-console

and log on as your server username and password. You can now create a VMware guest and start it up – as long as you’ve used a registration key. If you receive errors when starting your guest system via the console, make sure you’ve installed the libraries noted in the System Extras section. If you still have problems, run the following on the server to see if you have anything specific missing,

ldd /usr/lib/vmware/bin/vmware-vmx

Hopefully by now you’ve got a smile on your face and if not, keep plugging away and feel free to ask any questions in the comments.