Author: Mugabi Siro

Category: GNU/Linux Toolchain

Summary:

This entry discusses general considerations when using crosstool-NG, an automated toolchain build utlility. For purposes of illustration, Ubuntu is used as the build platform.

Tags: gnu/linux toolchain gnu linux crosstool-ng cross-compiling

Overview

Crosstool New Generation (crosstool-NG) is a fork of the original crosstool toolchain build facility. It offers, among other things, a friendlier configuration interface and build system than its predecessor. It includes support for:

  • a number of archictures including Alpha, ARM, AVR32, Blackfin, m68k, MIPS, PowerPC, s390, SH, and x86(_64).

  • both (e)glibc and uClibc.

  • inclusion of debug utilities such as gdb, ltrace, strace, dmalloc and duma into the final toolchain

Crosstool-NG also presents a useful platform for studying toolchain build. For example, check out the instructions in the file docs/9 - How is a toolchain constructed.txt in the crosstool-NG source tree.

This entry covers general considerations when using the crosstool-NG facility. The objective here is to provide the general working background required when building a toolchain via crosstool-NG. For examples of using crosstool-NG to build a specific toolchain, follow the links at the bottom in Section link.

Crosstool-NG Package Build dependencies

At least on Ubuntu 12.04 and 13.04:

$ sudo apt-get update
$ sudo apt-get install build-essential autoconf automake libtool
$ sudo apt-get install flex bison libncurses5-dev gettext texinfo 
$ sudo apt-get install gperf gawk curl cvs subversion

Obtaining crosstool-NG

To obtain the latest development version, use Mercural:

$ sudo apt-get install mercurial
$ hg clone http://crosstool-ng.org/hg/crosstool-ng/

With this tree, you may have to generate the GNU configure script before proceeding with the toolchain configuration and build process:

$ cd crosstool-ng
$ ./bootstrap
Running autoconf...
Done. You may now run:
  ./configure

Alternatively, a direct download of a specific archive release can be made via this link.

Crosstool-NG Configuration and Build Process

Although crosstool-NG uses the GNU configure and make build systems, it does so in a somewhat unconventional manner. It works in two phases:

Phase 1

Generation of the ct-ng utility and miscellaneous build scripts. This is done via the conventional

$ ./configure 
$ make
$ make install

command sequence. The end result being the generation of a toolchain build tree that contains the ct-ng2 tool and other build files. This toolchain build tree is what is required to perform Phase 2. In other words, the downloaded crosstool-NG sources are only required to produce this second phase toolchain build tree.

Phase 2

This is the actual toolchain configuration and build process via the generated ct-ng and miscalleneous build files/scripts (i.e. the generated toolchain build tree).

PHASE 1: Generating and Locating the Toolchain Build Tree

Depending on the user's toolchain configuration settings, crosstool-NG will, correspondingly, download a number of toolchain component archive files from the Internet for the automated toolchain build process. During toolchain build (i.e. Phase 2), all build actions and output (around 5GB) is done in a working directory before the final toolchain files (binaries, libraries, headers, etc - typically less than 500MB) are packaged and installed in a separate target directory.

This working directory is, by default, located at the root of the generated toolchain build tree3. It is recommended that these default settings be used as-is. Also recall that the generated toolchain build tree is the location from where Phase 2 is performed.

Now, it is often desirable to separate the locations of the crosstool-NG sources, downloaded archives, toolchain build tree, and final toolchain binaries:

  • In the event of a toolchain build error, a simple deletion of the toolchain build tree guarantees a clean restart without affecting the sources or downloaded archive files (and thus the need to perform lengthy re-downloads).

  • Once toolchain build is complete, the toolchain build tree (now around 5GB in size) need not be kept around and may be deleted to save disk space.

Assuming that the following steps have been completed:

$ KAZIDIR=$PWD
$ wget http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.17.0.tar.bz2
$ tar xjf crosstool-1.17.0.tar.bz2
$ CT_SOURCE=${PWD}/crosstool-ng-1.17.0
$ cd $CT_SOURCE

then the following standard commands will generate the toolchain build tree:

./configure [--prefix]
make 
make install

The --prefix option to configure specifies where the generated toolchain build tree will be installed:

  • Default installation of the generated toolchain build tree:

    By default (i.e. if the --prefix option is omitted) crosstool-NG will attempt to install the toolchain build tree under /usr/local when make install is executed.

  • Installing the generated toolchain build tree within source tree:

    $configure --prefix=${PWD}
    

    or

    $ configure --enable-local
    

    In this case, the root of the toolchain build tree will coincide with the root of the crosstool-NG source tree.

  • Installing the generated toolchain build tree somewhere else:

    Unless hacking crosstool-NG, this may be the recommended approach. Specify an absolute path to a (non-system) directory outside the crosstool-NG source tree e.g:

    $ CT_BUILDTREE=${KAZIDIR}/ct_build
    $ ./configure --prefix=$CT_BUILDTREE
    $ make
    $ make install
    

    Note that $CT_BUILDTREE will be created if it didn't exist.

At this point, if the toolchain build tree was generated outside the source tree, then the crosstool-NG sources are now no longer required for the rest of the toolchain build process (i.e. Phase 2).

In any case, after running make install, the hierarchy of the generated toolchain build tree should resemble:

phase1 build tree

Pay special attention to the relative location of the ct-ng utility.

Staging the toolchain build tree

So far, the installation of the toolchain build tree has been in the context of performing toolchain compilation from the current build machine. It is also possible to package the toolchain build tree for distribution via the use of the DESTDIR directive to make install. See Building Packages with configure for a detailed explanation on staging and a crosstool-NG case study on the subject.

PHASE 2: Toolchain Configuration, Build and Install

Central to this process is the generated ct-ng utility. It is a "bash-make" script and functions similar to the make command.

Preliminaries

Prior to using ct-ng, the following optional steps may be performed. For most practical purposes, these steps can be skipped:

  • Include ct-ng's location in $PATH

  • Shell completion:

    Crosstool-NG comes with a shell script fragment that defines bash-compatible completion. As of the version of crosstool-NG used in this entry, the shell fragment is not installed automatically. To install the shell script fragment, you have two options:

    • Install system-wide e.g. by copying ct-ng.comp into /etc/bash_completion.d/

    • Install for a single user e.g. by copying ct-ng.comp into ${HOME}/ and sourcing this file from your ${HOME}/.bashrc

Configuration

The instructions in this section assume that the $PWD is the root of the toolchain build tree generated in phase 1 above.

To view the available ct-ng options:

$ bin/ct-ng --help #

To enter the toolchain configuration menu:

$ bin/ct-ng menuconfig

To use, or begin with, a default configuration:

  • List available sample configurations:

    $ bin/ct-ng list-samples
    Status  Sample name
    [G.X]   alphaev56-unknown-linux-gnu
    [G.X]   alphaev67-unknown-linux-gnu
    [G.X]   arm-bare_newlib_cortex_m3_nommu-eabi
    [G.X]   arm-cortex_a15-linux-gnueabi
    [G..]   arm-cortex_a8-linux-gnueabi
    ...
    [G..]   arm-unknown-linux-gnueabi
    [G.X]   arm-unknown-linux-uclibcgnueabi
    ...
    [G.X]   mips64el-n32-linux-uclibc
    [G.X]   mips64el-n64-linux-uclibc
    [G.X]   mips-ar2315-linux-gnu
    [G..]   mipsel-sde-elf
    [G..]   mipsel-unknown-linux-gnu
    ...
    [G..]   powerpc-unknown-linux-gnu
    [G..]   powerpc-unknown-linux-uclibc
    [G..]   powerpc-unknown_nofpu-linux-gnu
    [G.X]   s390-ibm-linux-gnu
    [G.X]   s390x-ibm-linux-gnu
    [G..]   sh4-unknown-linux-gnu
    [G..]   x86_64-unknown-linux-gnu
    [G..]   x86_64-unknown-linux-uclibc
     L (Local)       : sample was found in current directory
     G (Global)      : sample was installed with crosstool-NG
     X (EXPERIMENTAL): sample may use EXPERIMENTAL features
     B (BROKEN)      : sample is currently broken
    
  • Select a sample configuration:

    $ bin/ct-ng arm-unknown-linux-gnueabi
    

    At this point you may run menuconfig, to make any modifications.

Environment Variables

There are three environment variables that are computed by crosstool-NG and available for use in the configuration menu e.g. in string options:

  • CT_TARGET

    Expands to the GNU triplet that forms the toolchain tuple (i.e. minus the trailing hyphen ``-") of the target4. In other words, the compiler and \emph{Binutils} components of the generated toolchain will be prefixed with the this tuple.

  • CT_TOP_DIR

    The root of the toolchain build directory.

  • CT_VERSION

    The version of crosstool-NG being used.

Entering String options

String and number options can refer to environment variables. In such a case, you must use the shell syntax e.g. $VAR. DO NOT single-quote or double-quote the string/number options, for example:

Entering String Options

Common Config Options

The following configuration options are common regardless of the specific toolchain being built:

  • Under Paths and misc options

    • Local tarballs directory (CT_LOCAL_TARBALLS_DIR)

      This is the local directory that will contain downloaded package tarballs. If a given tarball is already present, then no corresponding download of the archive will be performed. Note:

      • the value specified here should be an absolute path.

      • the directory should exist prior to executing the toolchain build command below. Otherwise, the following warning will be issued:

        $ bin/ct-ng build
        ...
        [INFO ]  Building environment variables
        [WARN ]  Directory '${CT_LOCAL_TARBALLS_DIR}' does not exist.
        [WARN ]  Will not save downloaded tarballs to local storage.
        [EXTRA]  Preparing working directories
        ...
        
    • Save new tarballs (CT_SAVE_TARBALLS)

      If selected, newly downloaded tarballs will be saved in the directory specified in CT_LOCAL_TARBALLS_DIR above. Recommended.

    • Working directory (CT_WORK_DIR)

      This is where all build actions will be done. Default is ${CT_TOP_DIR}/.build. As explained in Section link, this is the recommended setting (i.e. working directory within build tree).

    • Prefix directory (CT_PREFIX_DIR)

      Final toolchain installation directory. Note that the value given here should be an absolute path.

    • Remove the prefix dir prior to building (CT_RM_RF_PREFIX_DIR)

      If selected, and if CT_PREFIX_DIR exists, then it will be deleted before the toolchain is (re-)built.

    • Stop after downloading tarballs (CT_ONLY_DOWNLOAD)

      Only download tarballs. Do not proceed with toolchain build. Useful for pre-retrieving the required tarball archives before going off-line.

    • Number of parallel jobs (CT_PARALLEL_JOBS)

      Number of make jobs that will run concurrently. By default, (value of 0), crosstool-NG will set the number of jobs based on how many processors the host has. This value should be, generally, set to twice the number of processors of the build machine.

  • Under Toolchain options

    • Use sysroot'ed toolchain (CT_USE_SYSROOT)

      Recommended.

    • Tuple's vendor string (CT_TARGET_VENDOR)

      This corresponds to the vendor/manufacture field in the tuple of the toolchain being built5. It is an optional field that may be omitted. If, say, the value none is specified here when building a GNU/Linux ARM cross-toolchain generating the armhf EABI, then the resultant tuple will be arm-none-linux-gnueabihf-. Otherwise, if left at the default value, then the cross-toolchain tuple will be arm-unknown-linux-gnueabihf-.

      Note, the specified value must not contain spaces or dashes. Use a one-word string, or use underscores to separate words if need be. Avoid dots, commas, and special characters.

    • Type

      Select the toolchain type being generated. See docs/6 - Toolchain types.txt of the crosstool-NG sources for a description of toolchain types. Also check out the note on Linux kernel version versus Minimum supported kernel version in Appendix link

Build and Install

The following command builds and installs the final toolchain

$ bin/ct-ng build

It is possible to override the number of parallel jobs set in configuration option CT_PARALLEL_JOBS. For example, to reset to 4 parallel jobs:

$ bin/ct-ng build.4

Upon a successful build, the build tree may be deleted to save disk space, e.g:

$ cd $KAZIDIR
$ rm -r $CT_BUILDTREE

Performing Re-builds

From the root of the (Phase 1 generated) toolchain build directory:

  • distclean (if applicable):

    $ bin/ct-ng distclean
    
  • Retrieve the configuration file.

    Previously installed toolchains contain the ct-ng configuration file used to build them. To retrieve it:

    $ ${CT_PREFIX_DIR}/bin/${CT_TARGET}-ct-ng.config > .config
    

    Alternatively,

    $ bzcat ${CT_PREFIX_DIR}/build.log.bz2 | bin/ct-ng extractconfig > .config
    
  • Reconfigure

    Once the configuration file, .config, has been restored at the root of the clean/fresh toolchain build tree, run:

    $ bin/ct-ng oldconfig
    

    to restore the toolchain configuration. Otherwise, to modify the configuration:

    $ bin/ct-ng menuconfig
    
  • Re-build:

    $ bin/ct-ng build[.N]
    

Also See

Resources

Appendix A: Known Issues

Notes in this section taken from the free-electrons embedded linux training labs notes

Archive Download Failure

It is frequent that Crosstool-ng aborts because it can’t find a source archive on the Internet, when such an archive has moved or has been replaced by more recent versions. New Crosstool- ng versions ship with updated URLs, but in the meantime, you need work-arounds. If this happens to you, what you can do is look for the source archive by yourself on the In- ternet, and copy such an archive to the src directory in your home directory. Note that even source archives compressed in a different way (for example, ending with .gz instead of .bz2) will be fine too. Then, all you have to do is run ./ct-ng build again, and it will use the source archive that you downloaded.

ppl-0.10.2 compiling error with gcc 4.7.1

If you are using gcc 4.7.1, for example in Ubuntu 12.10, compilation will fail in ppl-0.10.2 with the below error:

error: ’f_info’ was not declared in this scope

One solution is to add the -fpermissive flag to the CT_EXTRA_FLAGS_FOR_HOST setting (in Path and misc options -> Extra host compiler flags).

Appendex B: Linux kernel version vs. Minimum supported kernel version


NOTE: This section mainly applies to (e)glibc

The Linux kernel version prompt is used to specify the version of Linux kernel headers that the C library will be compiled against. The Minimum supported kernel version menu entry (CT_LIBC_GLIBC_MIN_KERNEL) is used to specify the oldest kernel that userspace binaries built with this toolchain can function with. See An introduction to the GNU/Linux toolchain for a discussion on the implications of these settings.

The following "contextual" account of a toolchain build aims to clarify the use of these settings. The crosstool-NG tree used here was crosstool-ng-linaro-1.13.1-4.8-2013.10.

Configuration

$ cat .config
...
207 # Operating System
208 #
209 CT_KERNEL_SUPPORTS_SHARED_LIBS=y
210 CT_KERNEL="linux"
211 CT_KERNEL_VERSION="3.1.1"
# CT_KERNEL_bare_metal is not set
213 CT_KERNEL_linux=y
214 CT_KERNEL_bare_metal_AVAILABLE=y
215 CT_KERNEL_linux_AVAILABLE=y
216 CT_KERNEL_V_3_1_1=y
...
463 CT_LIBC_GLIBC_KERNEL_VERSION_CHOSEN=y
464 CT_LIBC_GLIBC_MIN_KERNEL_VERSION="2.6.9"
465 CT_LIBC_GLIBC_MIN_KERNEL="2.6.9"
...

Build Log

Upon successful build completion:

$ cat build.log
...
162437 [CFG  ]    checking installed Linux kernel header files... 2.6.19 or later
162438 [CFG  ]    configure: WARNING: minimum kernel version reset to 2.6.16
162439 [CFG  ]    checking for kernel header at least 2.6.16... ok
...

i.e the minimum kernel version was reset to v2.6.16 from the user specified value v2.6.9.

Checking Kernel Version Info

  • Kernel version that this eglibc was built against:

    $ qemu-arm -L ${CT_PREFIX_DIR}/arm-linaro-linux-gnueabihf/sysroot/${CT_PREFIX_DIR}/arm-linaro-linux-gnueabihf/sysroot/lib/libc-2.18-2013.10.so 
    Compiled on a Linux 3.1.1 system ...
    
  • Checking minimum supported kernel version by this eglibc:

    The minimum supported kernel version can be read from the .note.ABI-tag section of the libc ELF object file:

    $ arm-linux-gnueabi-readelf -n ${CT_PREFIX_DIR}/arm-linaro-linux-gnueabihf/sysroot/lib/libc-2.18-2013.10.so | grep Linux
    OS: Linux, ABI: 2.6.16
    

    Alternatively, since file(1)6 also reads this section of an ELF object file:

    $ file ${CT_PREFIX_DIR}/arm-linaro-linux-gnueabihf/sysroot/lib/libc-2.18-2013.10.so  
    ... for GNU/Linux 2.6.16 ...
    

Footnotes

2. The ct-ng utility is actually a "bash-make" script. [go back]

3. See item "Working directory" in Section link and named .build. [go back]

4. See Using a GNU/Linux Cross-Toolchain for an explanation of GNU triplets and the meaning of target in the context of the GNU Autotools [go back]

5. See Introduction to GNU/Linux Toolchain for an explanation of toolchain tuple names. [go back]

6. See src/readelf.c of its source code. [go back]