
		Monday December 23, 2024 01:14:59 UTC

Folding && BOINC - PXE / TFTP Diskless Network Boot Image

Introduction

This project consists of a bootable PXE image supporting the BOINC and folding clients. The goal of this project is to be able to setup a centralized server that will contain a bootable image that systems on the network can boot via the network using PXE functionality. This opens up the possibility of running diskless distributed computing systems that can boot simply by enabling PXE functionality in BIOS once the server-side is setup. A centralized server setup will eliminate having to install and maintain multiple OS installs and makes kernel and file system upgrade rollout simple.

The image uses a configuration file based on MAC address setup on the TFTP server and stores the distributed computing related data via a NFS share that overlays the read-only file system (aufs) that is loaded into memory via the network. The image also supports serial console and SSH for headless configuration. The image is console-only for a small footprint.

Features

Folding@home client support
BOINC 32-bit and 64-bit client support

Diskless folding and BOINC crunching
Client configuration via TFTP configuration
NVIDIA CUDA support
Wine support
NFS support
Samba, SSH, NTP, and Cron services
Screen
Sensors for temperature monitoring
3x consoles (ALT F1-F3)
2x serial port consoles for headless configuration (ttyS0-ttyS1 - 115200 8N1 No Flow Control)
Optional IA32 and 32-bit BOINC support for projects like Einstein@home that are 32-bit only

Kernel support:

Version: 3.0.1
Optimized builds for AMD Barcelona, AMD Bobcat, Intel Atom, Intel Core 2, Intel Core i7, and Intel Core i7-AVX processors
Up to 256 processors
NUMA
Supports 100/1000/10000 Mbit/s Ethernet cards

Image

Below is a link to a compressed image consisting of the bootloader, kernels, image, and associated files:

Linux64_DC PXE Bootable Image v1.8.1 (34.7MB)

Modules

The following file contains nvidia modules for all available kernels. Nvclock and nvidia-smi are also included. This file should be stored in the top-level directory along with initrd.img (typically /tftpboot/dc). It is downloaded and installed if CUDA=1 in the TFTP configuration for the host.

NVIDIA Driver 280.13 and Libraries (Image v1.8.0 64-bit)

The following file contains the Wine software used for running the Windows folding client in Linux. This file should be stored in the top-level directory along with initrd.img (typically /tftpboot/dc). It is downloaded and installed if WINE=1 in the TFTP configuration for the host.

Wine 1.2.1

ChangeLog

DC PXE Image ChangeLog

Notes and updates

08/13/11 - Built NVIDIA driver 280.13. Added support for adding rc and cron file to boinc or fah directory to allow for running extra commands on bootup and for overriding the default crontab. Updated lmsensors to the latest version.

04/23/11 - Built a 32-bit image based off the 64-bit image as recently requested. This image supports 32-bit FAH client 6.02 and BOINC 32-bit and includes optimized kernels for AMD Geode/K6/K7, Intel Atom, and Intel Pentium 3/4/M processors. This image can coexist on the same servers as the 64-bit image using the same PXELinux bootloader. Updated the 64-bit image to include the recent stable 270.41.06 driver release from NVIDIA which supports GPUs up to the most recent GTX 590. Fixed several bugs in fahcontrol and made a few updates to the init scripts. Also enabled RTC support in the kernels so that hwclock works properly.

04/16/11 - Added latest NVIDIA beta driver 270.26 and kernel 2.6.38 builds. Updated the folding related init and control scripts to the latest version similar or equal to what the other images use.

02/23/11 - Built NVIDIA Beta driver 270.18. This driver is needed to run the new Einstein@home CUDA app that does not require the usage of a full CPU per GPU work unit. Updated the download URL for the folding client to download the new version 1.34. Disabled bigadv via Wine with the release of the new Core A5 application that supports Linux64 bigadv. Added a nvidia version option to the TFTP configuration for simpler rollout of newer drivers.

02/01/11 - Updated the kernel to 2.6.37 with the latest image. The new Einstein@home Linux BRP3 CUDA app has been running stable on my test system for the past several weeks using the latest image and a GTX 295.

01/12/11 - Built several new kernels optimized for different architectures using experimental build of GCC 4.6. The new builds include support for AMD Barcelona, AMD Bobcat, Intel Atom, Intel Core 2, Intel Core i7, and Intel Core i7-AVX (Sandy Bridge). Upgraded the drivers to the latest 260.19.29 driver to support GTX 570 cards.

11/29/10 - Wine support has been added to the latest image making it possible to run the Windows folding client in Linux for folding bigadv support. Also, the latest image now supports running one or more folding uni-processor work units. Options for both clients are available via the TFTP configuration. If the bigadv option is disabled, the image uses the Linux folding client instead.

11/24/10 - The latest image now includes CUDA support for running Linux enabled CUDA projects like Einstein@home, Milkyway@home, and Seti@home. Tested successfully with Milkyway via a EVGA Dual-PCB GTX 295.

Modifying the image

Modifying the DC PXE Image

Server-side configuration

The image requires a Linux server configured with the following services:

DHCP service with BOOTP support (The isc dhcp version that comes with most Linux distributions supports this)
TFTP
NFS

Some basic Linux knowledge is required. The services above are generally available with most Linux distributions and may need to be installed with a 
package management tool.

DHCP setup

I have included a sample DHCP configuration that is based on the one I use. This file is often located in the /etc directory.

DHCP Configuration

In this example, 192.168.0.0 is the network and 192.168.0.12 is the TFTP server. If you already have a DHCP server on your network, you may consider running this one as your primary DHCP server to avoid having two DHCP servers on the network. The hosts in the group will have access to the image.

TFTP setup

Extract the image file to the TFTP root directory. This is often /tftpboot. Example:

cd /tftpboot

64-bit:
tar xzf ~/linuxdcpxe-1.7.2.tar.gz

32-bit:
tar xzf ~/linux32dcpxe-1.7.2.tar.gz

The 32-bit image can coexist on the same server as the 64-bit image. The init image for the 32-bit version is called initrd32.img and the 32-bit kernels are located separately from the 64-bit kernels in kernels/32. The PXE config file in pxelinux.cfg should reference the correct init image and kernel depending on whether the system is 32-bit or 64-bit. Separate PXE config files can be setup by MAC address to support 32-bit and 64-bit systems.

NFS setup

The folding and BOINC data will be stored in a NFS share. The /etc/exports file is where you setup your NFS shares. Here is an example:

NFS Exports Configuration

The no_root_squash option may present a security issue as any files written to the NFS mount are written as if they were the root user. You can leave out this option but make sure that the directory has the appropriate permissions so that the client system can write to the directory. By default this would be uid:gid 65534. You may consider setting up IP restrictions using /etc/hosts.allow and /etc/hosts.deny for further security and a firewall script to only allow access to specific hosts.

TFTP client configuration

Once the archive is extracted to the tftp directory, you will find a sub-directory under dc called config. This directory contains a file called default. For each client, copy this default file to fahMACADDR and adjust the client paremeters accordingly. MACADDR is the hardware address of the Ethernet adapter in the client system without colons and in lower case. The below file is an example configuration file set with a sample MAC address:

Sample client configuration

I would suggest using NFS instead of CIFS at this time. I ran into kernel crashes when using CIFS in conjunction with a Samba share and aufs or unionfs. The CIFS variables in the config can be left blank.

Bootloader configuration

The pxelinux bootloader configuration is kept in fah/pxelinux.cfg. In this directory you will find a default file. The default file will be used by the client systems unless a specific file is put in place based on MAC address. This file will let you set the default kernel that boots up. Also, you will want to update the TFTP server address (TFTPSERVER=) in the default file or other files which is needed by the client. The below file is an example of the bootloader configuration.

Sample bootloader configuration

If you decide to setup bootloader configurations for specific systems, you would want to format the file based on MAC address starting with 01 and including dashes. This can be beneficial when you want one or more systems to boot a different kernel. Here is an example file name:

01-00-0c-29-a7-32-f4

Login information

The password for the folding and boinc logins is set via the TFTP configuration. Login information is found below:

Folding@home

Username: fah

BOINC

Username: boinc

If you enable SSH, the same credentials will work.

There is also the option to start a screen session and manually start the client from there:

Start a session:
screen -R -D -S fah

Detach from a session:
CTRL A+D

Attach to a session:
screen -rx fah

Shutdown and reboot

To shutdown the system, run these commands:

halt

To reboot:

reboot

GNU License

The core Linux system image is covered under the GNU General Public License Version 2

Other information

The image uses pxelinux for the bootloader. This link has much more information regarding configuration of pxelinux for custom configuration:

PXELINUX Information

Screenshots