Embedded Linux Conference Europe 2011
Using Buildroot for real projects
Thomas Petazzoni Free Electrons thomas.petazzoni@freeelectrons.com
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
Thomas Petazzoni
Embedded Linux engineer and trainer at Free Electrons since 2008
Embedded Linux development: kernel and driver development, system integration, boot time and power consumption optimization, consulting, etc. Embedded Linux and driver development training, with materials freely available under a Creative Commons license. http://www.free-electrons.com
Major contributor to Buildroot, an open-source, simple and fast embedded Linux build system Living in Toulouse, south west of France
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
Agenda
What is Buildroot ? How does it work ? Example systems generated by Buildroot Recommendations for real projects
Toolchain recommendations Project specic conguration and les Project specic packages Enabling application developers Misc best practices
Features in 2011.11 making things better Conclusion
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
What is Buildroot ? (1/2)
Buildroot is an embedded Linux build system Its goal is to build
a cross-compiling toolchain a root lesystem with multiple cross-compiled libraries and applications a kernel image and bootloader images
or any combination of these It has a kcong conguration mechanism, identical to the one used in the kernel It is completely written in make It builds only whats necessary. A base system, containing just Busybox, takes less than 2 minutes to be built from scratch.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
What is Buildroot ? (2/2)
It is designed with simplicity in mind
Standard languages used, relatively lightweight infrastructure. Very easy to add packages or customize the build system behaviour.
It is best-suited for small to medium-sized embedded systems
In the generated root lesystem, Buildroot doesnt track which source package installed what. You need to do complete rebuilds for a clean root lesystem after conguration changes. Generates systems with no package management mechanism on the target (no ipkg, or apt)
700+ packages Stable releases published every three months Active user/developer community
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
Basic usage (1/2)
$ make menuconfig
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
Basic usage (2/2)
$ make ... ... ... $ ls output/images/ dataflash_at91sam9m10g45ek.bin rootfs.ubi u-boot.bin uImage
rootfs.tar rootfs.ubifs u-boot-env.bin
Ready to use !
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
How does it work ?
Conguration options dened in Config.in les, and stored in a .config le Buildroot starts with the toolchain: either it generates it, or imports an existing toolchain, or uses crosstool-NG to generate one It creates a basic root lesystem from a skeleton (just a few conguration les) Once the toolchain is ready, Buildroot goes through the list of selected packages. It simply fetches, congures, builds and installs all packages, respecting their dependencies. It builds the kernel image and/or bootloader images, if requested It creates one or more root lesystem images, using fakeroot
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 8
Top-level source code organization (1/2)
board/ contains hardware-specic and project-specic les. Covered later. boot/ contains cong options and recipes for various bootloaders configs/ the default congurations. Covered later. docs/ Yes, we have some documentation. fs/ contains cong options and makeles to generate the various lesystem images (js2, ubifs, ext2, iso9660, initramfs, tar and more). Also contains the root lesystem skeleton
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
Top-level source code organization (2/2)
linux/ contains the cong options and makele to generate the Linux kernel, and also the kernel part of real-time extensions (Xenomai, RTAI, etc.) package/ contains the cong options and makeles for all userspace packages support/ various misc stu needed for the build (kcong code, etc.) target/ mostly historic directory. No longer contains anything useful besides the default device table. toolchain/ cong options and makeles to build or import the toolchain
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 10
What does it generate?
In the output directory (output by default, but out-of-tree build with O= is supported): build/, a directory with one sub-directory per component built. The directory contains the source of that component, and this is where it is built. host/, a directory that contains the utilities built for the host machine, including the toolchain
A subdirectory, host/usr/<tuple>/sysroot/ contains the toolchain sysroot, with all libraries and headers needed to build applications for the target.
staging/, a historical symbolic link to host/usr/<tuple>/sysroot target/, the target root lesystem (without device les) images/, where nal images are stored
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 11
Example 1: Multi-function device
The device is an ARM AT91-based platform with GPS, RFID readers, GSM modem, Ethernet and USB. The Buildroot conguration:
CodeSourcery ARM glibc toolchain Linux kernel Busybox for the basic system Dropbear for SSH access (debugging) Qt with only QtCore, QtNetwork and QtXml, no GUI QExtSerialPort zlib, libxml2, logrotate, pppd, strace, a special RFID library, popt library The Qt application JFFS2 root lesystem
Filesystem size: 11 MB. Could be reduced by using uClibc. Build time: 10 minutes on a fast build server (quad-core i7, 12 GB of RAM)
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 12
Example 2: vehicle navigation system
An x86-based system, with an OpenGL application for vehicle navigation system.
External glibc toolchain generated with crosstool-NG The Grub bootloader Linux kernel, of course Busybox A part of the X.org stack (the server, a few drivers, and some client libraries), including libdrm, Mesa The fglrx ATI proprietary OpenGL driver ALSA utils, ALSA library, V4L library, Flashrom, LM Sensors, Lua, Dropbear, Ethtool The OpenGL application and its data
Filesystem size: 95 MB, with 10 MB of application (binary + data) and 45 MB (!) of fglrx driver. Build time: 27 minutes on a fast build server (quad-core i7, 12 GB of RAM)
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 13
Toolchain
Three choices: Buildroot builds a toolchain. This is limited to uClibc based toolchains, and the toolchain is rebuilt completely at every complete Buildroot rebuild. Buildroot uses crosstool-NG as a back-end to generate the toolchain. More choices available (glibc, eglibc, uClibc supported, more gcc versions, etc.). However, requires toolchain rebuild at every complete Buildroot rebuild. Buildroot can import external toolchains. This is denitely the mechanism I recommend.
Allows to re-use CodeSourcery toolchains, or custom toolchains built with crosstool-NG or Buildroot Importing the toolchain into Buildroot takes just a few seconds, which saves the toolchain build time at every Buildroot rebuild The toolchain rarely needs to be changed compared to the root lesystem, so this mechanism makes sense
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 14
External toolchains: toolchain prole
Buildroot contains presets for well-known binary toolchains such as the CodeSourcery ones, for which Buildroot knows the properties (C library used, etc.) and the download location.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
15
External toolchains: custom toolchain
Buildroot can also use custom, locally installed toolchains, in which case one must tell Buildroot a few details about the toolchain (C library being used, location, prex, etc.)
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
16
External toolchains HOWTO
You can either:
Use an existing pre-built toolchain, available publicly from the Web Generate a custom toolchain, and store it pre-built next to the Buildroot directory on the local lesystem, or in some internal FTP/HTTP server
And then point your Buildroot conguration to this toolchain:
As a custom toolchain Or by adding a new prole so that the toolchain gets downloaded automatically and all its conguration is already known to Buildroot
This way, users of the Buildroot environment dont have to worry about the toolchain and dont have to wait for it to build.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 17
Project specic les
For each project, the recommendation is to create a directory board/<company>/<project>/ which will be used to store: Conguration les Root lesystem additions Project-specic kernel/bootloader patches All other project specic les
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
18
Kernel and bootloader changes
The kernel and bootloaders often require modications for a particular hardware platform. My typical workow is the following :
Have a branch in a kernel Git tree in which I commit the necessary changes Generate patches with git format-patch Import them into Buildroot in the board/<company>/<project>/linux-patches/ directory Congure Buildroot so that it applies the patches before building the kernel (option BR2 LINUX KERNEL PATCH)
The advantage is that the Buildroot environment is easy to use for others (no external dependency on a kernel Git tree) The drawback is that Buildroot does not directly use your kernel Git tree. This has been improved recently, covered later.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
19
Kernel and bootloaders example (1/2)
$ ls board/<company>/<project>/ linux-2.6.39.config samba-script.tcl linux-2.6.39-patches/ u-boot-1.3.4-patches/ at91bootstrap-1.16-patches/ $ ls board/<company>/<project>/linux-2.6.39-patches/ linux-0001-foobar.patch linux-0002-barfoo.patch $ ls board/<company>/<project>/u-boot-1.3.4-patches/ u-boot-0001-barfoo.patch u-boot-0002-foobar.patch $ ls board/<company>/<project>/at91bootstrap-1.16-patches/ at91bootstrap-0001-bleh.patch at91bootstrap-0002-blah.patch
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
20
Kernel and bootloaders example (2/2)
BR2_TARGET_UBOOT=y BR2_TARGET_UBOOT_BOARDNAME="company_project" BR2_TARGET_UBOOT_1_3_4=y BR2_TARGET_UBOOT_CUSTOM_PATCH_DIR= "board/company/project/u-boot-1.3.4-patches/" # BR2_TARGET_UBOOT_NETWORK is not set BR2_TARGET_AT91BOOTSTRAP=y BR2_TARGET_AT91BOOTSTRAP_BOARD="at91sam9m10g45ek" BR2_TARGET_AT91BOOTSTRAP_CUSTOM_PATCH_DIR= "board/company/project/at91bootstrap-1.16-patches/" BR2_LINUX_KERNEL=y BR2_LINUX_KERNEL_CUSTOM_VERSION=y BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="2.6.39" BR2_LINUX_KERNEL_PATCH= "board/company/project/linux-2.6.39-patches/" BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE= "board/company/project/linux-2.6.39.config"
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 21
Root lesystem customization (1/3)
The Buildroot process to create the root lesystem is: Copy a skeleton to the output/target directory. The default one is in fs/skeleton, but a dierent one can be used. Copy base libraries (C libraries and al.) from the toolchain sysroot into the target root lesystem Install all packages Execute a conguration-specied post build script Generate the lesystem images in the selected formats (js2, ubifs, etc.)
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
22
Root lesystem customization (2/3)
Depending on which modications are necessary, you might need to: Use a dierent lesystem skeleton if the default skeleton is really not appropriate. If only tiny modications are needed, do them in the post build script. I dont recommend this solution as it consists in duplicating the default skeleton, which would prevent from taking advantage of future improvements of the default skeleton. Create packages to install applications, data les, etc. The skeleton is really only for basic cong les. Create a post build script that adjusts the root lesystem as needed.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
23
Root lesystem customization (3/3)
Create a post-build script in board/<company>/<project>/post-build.sh and tell Buildroot to run it with the BR2 ROOTFS POST BUILD SCRIPT. This script is executed from the Buildroot main directory and receives the target lesystem directory as argument. Typically, my script does:
Tune the /etc/inittab, /etc/fstab or /etc/securetty les Copy the contents of board/<company>/<project>/rootfs-additions/ into the target root lesystem. In this directory, I put additional init scripts, conguration les for network services, or override existing les.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
24
Root lesystem customization: example script
TARGETDIR=$1 # Set root password to root. Password generated with # mkpasswd, from the whois package in Debian/Ubuntu. sed -i s%^root::%root:8kfIfYHmcyQEE:% $TARGETDIR/etc/shadow # Application/log file mount point mkdir -p $TARGETDIR/applog grep -q "^/dev/mtdblock7" $TARGETDIR/etc/fstab || \ echo "/dev/mtdblock7\t\t/applog\tjffs2\tdefaults\t\t0\t0" \ >> $TARGETDIR/etc/fstab # Copy the rootfs additions cp -a $BOARDDIR/rootfs-additions/* $TARGETDIR/
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
25
/dev managament
Buildroot oers four solutions for /dev management: static, where device les are created at build time. They are listed in a congurable device table. devtmpfs, the kernel lesystem which creates dynmically device nodes. Buildroot makes sure that your kernel has the right options selected. Recommended solution. devtmpfs+mdev, which allows to trigger scripts or applications on device insertion/removal. devtmpfs+udev, the full-edge solution. Whichever solution is choosen, there is still a device table used, which is used to set specic permissions and ownership on certain les or directories. The list of device tables is congured through BR2 ROOTFS DEVICE TABLE.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 26
Project-specic packages (1/5)
A project typically requires one or more specic packages, containing project-specic applications or libraries. In order to make the build process fully integrated, those can be integrated in Buildroot. In order to isolate your packages from all other packages, create a sub-directory for all your packages:
Create a package/<company>/ directory This allows to isolate your specic packages from all other packages. Put a package/<company>/<company>.mk les into it, with just: include package/<company>/*/*.mk This tells Buildroot to parse the makeles of your packages.
Obviously, if you need to package an existing open-source component, package it under package/ and submit the patch to the Buildroot community.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 27
Project-specic packages (2/5)
For each package: Create a package/<company>/<yourpkg>/ directory Create a package/<company>/<yourpkg>/<yourpkg>.mk makele that contains the recipe to build your package Create a package/<company>/<yourpkg>/Config.in le that details the conguration options of your package. At least one is mandatory, in order to enable/disable your package. From package/Config.in, source your package/<company>/<yourpkg>/Config.in, preferably in a company-specic menu to make merges with future Buildroot versions easier.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
28
Project-specic packages (3/5)
To create the .mk le, three infrastructures are available: The AUTOTARGETS infrastructure, for autotools-based packages. You describe the tarball URL and version, the dependencies and conguration options, and Buildroot does all the rest. The CMAKETARGETS infrastructure, for CMake-based packages. Here as well, Buildroot does most of the work. The GENTARGETS infrastructure, for other packages not using a well-known build system. Here, Buildroot has no knowledge of the build system, so you have to specify what needs to be done to congure, build and install the component. See the Buildroot documentation for more details on using these package infrastructures.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 29
Project-specic packages (4/5)
Typically, the package source code is downloaded as a tarball from HTTP/FTP or from a Git/Mercurial/Subversion repository. For project-specic applications or libraries, it might be useful to store them locally.
For small utilities, it can be directly in the package directory, in an src/ subdirectory. For larger programs or libraries having their own version control, it is possible to override the package extraction commands to copy their source code in the build directory
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
30
Project-specic packages (5/5)
The application source code is stored in ../company-application/ relative to the Buildroot source code.
MY_APPLICATION_VERSION = 1.0 MY_APPLICATION_DEPENDENCIES = \ qextserialport host-pkg-config define MY_APPLICATION_EXTRACT_CMDS cp -a $(TOPDIR)/../company-application/* $(@D)/ endef define MY_APPLICATION_INSTALL_TARGET_CMDS cp $(@D)/myapp $(BINARIES_DIR) endef $(eval $(call CMAKETARGETS))
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
31
During application development
Buildroot is really an integration utility. Once a package has been built, it is not rebuilt, even if its source code changes. When working on the development of a component, it is usually more convenient to build it outside of Buildroot, for a quicker compile/test/debug cycle. The upcoming source directory override feature, covered later, makes it easier to use Buildroot during development.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
32
Using Buildroot to build external components
The output/host directory is the Buildroot SDK. output/host/usr/bin/ARCH-linux-* are the cross-compilation utilities. A wrapper is used so that the compiler is passed the appropriate --sysroot option. From the user perspective, the compiler therefore automagically nds the libraries and headers. output/host/usr/bin also contains other useful utilities for the build process
pkg-config, which is congured to look for libraries in the right location by default. qmake, if Qt is used, also congured with the correct paths
output/toolchainfile.cmake is a CMake description of the toolchain. Can be passed to cmake using -DCMAKE TOOLCHAIN FILE=....
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 33
Using Buildroot to build external components
A simple application
BRPATH/output/host/usr/bin/arm-unknown-linux-gcc -o prog prog.c $(BRPATH/output/host/usr/bin/pkg-config --libs --cflags glib-2.0)
Autotools component
export PATH=$PATH:BRPATH/output/host/usr/bin/ ./configure --host=arm-unknown-linux
CMake component
cmake -DCMAKE_TOOLCHAIN_FILE=BRPATH/output/toolchainfile.cmake make
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
34
Forcing Buildroot to rebuild a package
Buildroot keeps stamp les in the package build directory output/build/pkg-version/. They are named .stamp extracted, .stamp configured, .stamp built, .stamp target installed, etc. Removing a stamp le and re-executing make will force Buildroot to do the corresponding step again. Removing the complete package build directory will force Buildroot to rebuild the package from scratch. This will be improved in the upcoming Buildroot version.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
35
Using NFS during application development
Mounting the root lesystem over NFS is very practical during development The output/target directory created by Buildroot cannot directly be used for NFS mount, because it does not contain any device le and the permissions may not be correct. This is because Buildroot runs as non-root. The recommended way is:
Ask Buildroot to generate a tarball image of the root lesystem Uncompress this tarball image, as root, in some directory exported by NFS. make && tar -xf -C /nfsroot/ output/images/rootfs.tar
/nfsroot should be exported with the NFS options rw and no root squash in /etc/exports
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 36
Storing the project conguration
Once you have dened a Buildroot conguration for your project, you want to save it somewhere and allow others to use it. Just do:
make savedefconfig Creates a defconfig le in the top Buildroot source directory. mv defconfig configs/company project defconfig
Others can then load this conguration very easily:
make company project defconfig
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
37
Summarizing the project-specic bits
In the end, your project-specic bits are in:
board/<company>/<project>/ for the kernel and bootloader patches and conguration, post-build script and root lesystem overlay configs/company project defconfig for the Buildroot conguration. package/company/ for your specic packages
Your modications are cleanly isolated from the Buildroot core, making it easy to upgrade Buildroot when needed.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
38
Be prepared for oine builds
When working on a project, youll want to make sure that your build can be done oine in order to not depend on resources that may disappear from the Net. Here are some useful Buildroot features to do so:
make source will download into the Buildroot download cache all the les required to do the build make external-deps will list the name of all the les (essentially tarballs) that are required to do the build. Identies the useful tarballs in the Buildroot download cache. With the BR2 PRIMARY SITE conguration option, Buildroot will download les from a specied HTTP/FTP server before trying to reach the ocial site for each package. Useful to create an internal server. The location of the local download cache can be congured with BR2 DL DIR or with the BUILDROOT DL DIR environment variable.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com 39
In 2011.11: local method
The upcoming 2011.11 release will natively support packages whose source code is in a local directory. Simply need to use: MYPKG SITE = /some/local/directory MYPKG SITE METHOD = local Buildroot will rsync the source code from the specied location into its build directory in output/build/pkg-version.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
40
In 2011.11: source directory override
A package typically species a HTTP or FTP location for its tarball You might want to use a local source code for this package, instead of the ocial tarball The upcoming 2011.11 allows to specify an override le
This le is simply a makele with variable assignments Those variable assignments allows you to tell Buildroot: for package foo, use the source code in this directory rather than the normal tarball location ZLIB OVERRIDE SRCDIR = /home/foo/my-zlib/ LINUX OVERRIDE SRCDIR = /home/foo/linux-2.6/
The source code is rsynced from the given source directory into the Buildroot build directory.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
41
In 2011.11: other features
Addition of make <pkg>-reconfigure and make <pkg>-rebuild
Instead of manually manipulating the stamp les, those commands restart the package build process at the conguration stage or at the compilation stage. Makes using Buildroot during kernel or application development a lot easier.
Support for fetching from Mercurial repository Support for fetching using scp, both for packages and for the primary download site
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
42
Conclusion
Buildroot has multiple features making it easy to customize the generated system
Addition of kernel and bootloader patches and conguration Addition of project specic conguration les and scripts Addition of project specic packages
These features makes Buildroot suitable to generate embedded Linux systems for a wide range of projects, with a preference on moderately large systems. From our experience, Buildroot also remains simple enough to be used and understood by non-Linux experts.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
43
Questions?
Thomas Petazzoni
thomas.petazzoni@free-electrons.com
Special thanks to the Buildroot community, including Peter Korsgaard, Baruch Siach, Thomas de Schampheleire, Arnout Vandecappelle and Will Moore for their comments and suggestions on this presentation. Slides under CC-BY-SA 3.0.
Free Electrons. Embedded Linux development, consulting, training and support. http://free-electrons.com
44
