Installing Emulab Software on boss

This procedure is similar to installing the testbed software on ops. Before beginning, make sure that ops is installed, running, and reachable through the network by your new boss machine.

Step 1 - Installing Packages

Unpack the Emulab Server Package someplace. Let's say you unpacked it to /usr/tmp. You would then run:

ln -s /usr/local/bin/perl5.8.8 /usr/bin/perl
env PKG_PATH=/usr/tmp/packages pkg_add emulab-boss-2.1

Of course, if you unpacked it somewhere else, put in the correct path.

We provide all of the binary packages necessary to build boss in the packages tarball. If you would like to install additional software on boss, unpack the 'approved' ports collection to a place such as /usr/ports (rename or delete the old directory, if it exists). The package unpacks to approximately 175MB. Install whichever software you would like.

Step 1a - A fixup

A number of sites are having problems related to a broken perl installation. Briefly, the installed perl is not built with the SETUID option, which causes the Emulab installation to fail in odd ways. Lets check:

pkg_info -L perl-5.8.8_1.tbz | grep sperl

Make sure that it shows /usr/local/bin/sperl5.8.8 in the output. If not, then:

pkg_delete -f perl-5.8.8_1
env PKG_PATH=/usr/tmp/packages pkg_add perl-5.8.8_1.tbz

Step 2 - Unpacking and running 'configure'

Unpack the Emulab source tree to /usr/testbed/src/testbed. Then run its configure script, providing the local path of your definitions file:

mkdir -p /usr/testbed/src/testbed
mkdir -p /usr/testbed/obj/testbed
cd /usr/testbed/src/testbed
tar ...
cd /usr/testbed/obj/testbed
/usr/testbed/src/testbed/configure --with-TBDEFS=/path/to/your/defs-file

Remember, you have to use the same defs-file that you used on ops.

There is a FreeBSD-6.3-specific bug in the named configuration files. We have provided a patch which should now be located in /usr/testbed/src/testbed/named/rc.named.6.3.patch. You must apply this patch if you are running FreeBSD 6.3. To do so, run the following command:

patch -p0 < /usr/testbed/src/testbed/named/rc.named.6.3.patch

Step 3 - Running the boss installation script

In the object tree you've configured (say, /usr/testbed/obj/testbed), there's an 'install' subdirectory with a script called 'boss-install'. Run this script as root (not the same package directory argument as above):

cd install
perl boss-install -p /usr/tmp/packages

Like the ops-install script, boss-install sets up passwordless sudo for anyone in the wheel group. If this script fails, send the output to Utah so we can fix it. Note that boss-install requires internet access so it can grab some files.

The source installation is performed by the boss-install script and should be completely automatic. However, you may check it or fix up problems by running the makes yourself, as follows:

gmake
gmake boss-install
gmake post-install

The post-install target needs to be run as root, so that it can make certain scripts setuid, etc.

Step 4 - Verify SSH configuration

The boss-install script should have set up root ssh from boss to ops. To confirm this, make sure this works:

boss> sudo ssh ops ls /

If this fails, you will need to set this up by hand, typing a password:

scp /root/.ssh/identity.pub ops:/root/.ssh/authorized_keys

Step 5 - Setting up named

The testbed software manipulates DNS zone files for two reasons. First, it adds your nodes to them so that you don't have to. Second, it creates CNAMEs for all the nodes in every experiment (so that you can use, for example, 'nodeA.myexp.myproj.emulab.net' to refer to your node regardless of which physical node it got mapped to).

The named_setup script does this by generating zone files. In general, it concatenates a '.head' file, written by you, with its own generated entries. The main zone file is /etc/namedb/OURDOMAIN.db, where OURDOMAIN is from your defs file. (OURDOMAIN, unless explicitly specified, is taken to be the domain portion of BOSSNODE). We also generate reverse zone files (for inverse lookups, i.e. turning IP addresses back into names) in /etc/named/reverse/SUBNET.db, where SUBNET is the class-C subnet in which the addresses reside (i.e. 10.0.0.db). This value is defined in the defs file, as is TESTBED_NETWORK.

boss-install makes a reasonable attempt to create a set of named configuration files for you, placing them in /etc/namedb. If your testbed consists of a single class-C network, then these files will most likely be correct, although you'll want to look at them to make sure. Look at these files to make sure:

/etc/named/OURDOMAIN.db.head
/etc/named/reverse/SUBNET.db.head
/etc/named/named.conf

If you have more than one class-C subnet for your testbed, you'll need a copy of the reverse zone file for each one. You want to put boss, ops, and any 'infrastructure' equipment such as routers and switches into the zone files. These zone files do not need to include the nodes - they will be added automatically. Be sure to edit /etc/named/namedb.conf if you add any reverse map files (follow the format for the existing entry).

Once you think you've got things set up, run make sure the following command doesn't give you any error messages:

/usr/testbed/sbin/named_setup

This command will generate the following files:

/etc/namedb/OURDOMAIN.db
/etc/namedb/reverse/SUBNET.db

*** If you are using unroutable private IP addresses for part of the testbed: ***

In order to handle this situation, we'll need to use a feature of bind called 'views' so that the private addresses are only exposed to clients within the testbed. See the bind documentation for more details on this feature. You'll want to make sure that loopback queries from boss itself see the internal view - you want boss to resolve its own hostname to its private address, not its public one.

In order to use multiple views, we generate multiple zone files. In addition to OURDOMAIN.db, which will be used for the 'external' view, we create OURDOMAIN.internal.db for use with the 'internal' view. So, you'll also need to create OURDOMAIN.internal.db.head. When we generate the zone files, only publically-routable addresses are put into OURDOMAIN.db. OURDOMAIN.internal.db contains all addresses, both public and private. So basically, you'll want to put the public addresses for boss, ops, etc. into OURDOMAIN.db.head, and their private addresses into OURDOMAIN.internal.db.head.

Step 6 - Miscellaneous configuration

There are a few things that we haven't been able to completely automate just yet, though we hope to soon.

hosts file - You want to put boss/ops name/IP addresses in /etc/hosts on both boss and ops to avoid boottime circular dependencies (because of NFS cross mounts). This is done for you in ops-install and boss-install, but you should confirm that it was done properly. If you change the IP addresses of boss/ops later, you will want to be sure to update /etc/hosts on both machines.

SSL Certificates - Our apache config file expects to find SSL certificates in:

/usr/local/etc/apache/ssl.crt/www.<sitename>.crt
/usr/local/etc/apache/ssl.key/www.<sitename>.key

Where <sitename> is OURDOMAIN from the configure defs file, which defaults to boss' domain name.

boss-install already generated a temporary no-passphrase certificate for you and placed it in the locations specified above. However, we recommend that you get a "real" certificate from Verisign (or one of the others).

DHCPD - boss-install generated a dhcpd.conf.template file and installed it in /usr/local/etc (the template is derived from information you provided in the defs file). It then generated an actual dhcpd.conf file and started up dhcpd for you. Do not edit the dhcpd.conf file directly! If you need to make changes, change the template instead and then run:

/usr/testbed/sbin/dhcpd_makeconf -i -r

locate database - It can be useful to update the 'locate' database to help you find files as you're learning the system. This normally happens nightly, but you can force it to happen now by running the following as root:

/usr/libexec/locate.updatedb &

This will run in the background for several minutes. You can then find 'foo.conf' by running 'locate foo.conf'.

dhclient - If you don't control the public network to which boss and ops (and perhaps tipserv) are attached, your network admins may want you to run dhclient on those interfaces (and you may want to as well so that your boss /etc/resolv.conf contains the correct nameserver returned by dhclient, to avoid configuring it statically). However, boss runs its own nameserver that ops and the experimental nodes must contact to resolve internal names, so you must get dhclient to correctly write /etc/resolv.conf (i.e., boss must be the first nameserver). The easiest way to solve this problem is to add the following line to boss' /etc/dhclient.conf file:

prepend domain-name-servers 127.0.0.1

and add the following line to ops' (and any other non-experimental nodes, such as tipservs) /etc/dhclient.conf:

prepend domain-name-server <bossip>

where <bossip> is replaced by a valid IP address for your boss. Make sure to restart dhclient on these machines since dhclient will continue to overwrite /etc/resolv.conf incorrectly.

Step 7 - Reboot boss

Okay, go ahead and reboot boss now, and make sure it comes up okay.