|Intro | Install | Using | Monitor | Tuning

Teambuilder

[Home] [Next: Installation]

Introduction

Teambuilder is a distributed compilation system for C/C++. It allows compilation to be assigned to any machine participating in the compilation farm to take advantage of unused processor cycles. Combining three machines into a farm often gives three-fold gains, and larger gains are possible with larger farms.

Teambuilder does not need modifications to the build system already in use. Unlike general purpose clustering solutions, Teambuilder requires no kernel or system changes apart from running the Teambuilder Daemon on each participating machine. In particular, there is no need for shared source trees or NFS.

Teambuilder handles local processing (such as linking) as well as remote compilations. This ensures that the load on all machines stays within specified limits.

Teambuilder also monitors the speed of each participating machine and assigns jobs to the fastest machines first. If jobs are queued, Teambuilder assigns machines based on the size of jobs.

Teambuilder includes three base components:

• Teambuilder Scheduler

  The Teambuilder Scheduler runs on one machine, scheduling and distributing the work for all the machines in the compiler farm. Work is distributed based on the speed and load of the machines involved, to ensure that machines stay responsive and that the load is distributed fairly. The scheduler tries to assign jobs to faster machines first, and when jobs are queued, assigns larger jobs to faster machines.

• Teambuilder Daemon

  Each machine in the compiler farm runs a Teambuilder Daemon. The Teambuilder Daemon handles the actual compilations that the Teambuilder Scheduler assigns. It accepts and processes jobs, and provides status information that the Teambuilder Scheduler uses in its scheduling decisions. The maximum number of concurrent jobs to run on each computer is configurable. Jobs are usually run at reduced priority to ensure that computers remain responsive, but the priority can be adjusted to suit each computer. Developers are usually oblivious to Teambuilder jobs being run on their machine.

• Teambuilder Compiler

  Each machine has one or more Teambuilder Compilers. A Teambuilder Compiler is run instead of the normal compiler and has the same name and usage as the normal compiler it replaces. This means that no changes to the build environment or working practices are required to make use of Teambuilder. The Teambuilder Compiler uses the Teambuilder Scheduler to select machines to use for compilation. Each selected machine's Teambuilder Daemon uses the appropriate Teambuilder Compilers to compile the jobs it is given.

The Teambuilder Monitor shows the state of the compilation farm, and can be used to adjust various Teambuilder settings.

Most Linux/Unix operating systems and C/C++ compilers work with Teambuilder. At the time of writing, the platforms which have been tested and are known to work are these:

• FreeBSD gcc 2.95.x

• Linux gcc 2.95.x

• Linux gcc 2.96.x

• Linux gcc 3.0.x

• Solaris 7 gcc 2.95.x

• Solaris 7 Sun WorkShop 6 update 2 (C compiler only)

• Solaris 7 Workshop Compilers 5.0 (C compiler only)

All the machines in a Teambuilder farm must run the same operating system, but they can use cross-compilers if required. Teambuilder requires a make that is capable of spawning multiple jobs. GNU make is suitable, but Sun's make does not have this capability. Teambuilder should work with almost any Unix/Linux with the GNU C/C++ compiler and with many commercial C compilers.^*

Currently, Teambuilder has only been released for Linux/GCC. If you want to use Teambuilder with a different operating system--compiler combination, please contact Trolltech (info@trolltech.com).

^* The reason that some commercial C++ compilers cannot currently benefit from Teambuilder is because of the way they compile templates. This problem is under investigation.

See the Teambuilder whitepaper for examples of the performance benefits of Teambuilder.

[Home] [Next: Installation]