sbuild: a source build environment for debian based systems
In the past i was using a qemu/kvm based virtual host to build my deb-packages from source. I liked the complete separation between my workstation and the build host. There where no source-packages, gcc, libsomething-dev etc installed on the workstation but only in the buildmachine kvm-guest.
But i disliked that the buildmachine was not very fast and my builds where not reproduceable. There where leftovers on the build host after building sourcecode that needed another package to be installed as dependency. At last i manually needed to track all dependencies which had to be installed (manually) before issueing configure/make on the packages source.
I needed a completely new virtual machine for each and every package i build from source. That was not very convenient. Then i came around sbuild.
Sbuild is used by debian developers to build their packages locally and on the build-network. It builds binary packages (can package src-debs as well) for a lot of architectures.
The base is a minimal bootstrapped debian (~200MB) that will be used as a golden filesystem that never gets modified (you still can if you want to). All modifications are made on top and do not get persisted. That leads to a predictable and clean environment one can start with. That buildsystem is executed within a chroot jail in my workstation thus shares my direct access to hardware and has no complex virt-io hard/software in between. Sbuild also manages dependencies of the package so i don’t have to.
Let’s install and setup a sbuild build environment. For further details refer to the sbuild manpage, the wiki.debian page, etc.
Installation
NOTICE: sbuild > 0.66.x has the –build-dir parameter which is needed for ccache, otherwise ccache has no effect, as sbuild uses a different (random) build-directory each time. It will still work, but cannot benefit from ccache. This is for debian 8 jessie.
We will install sbuild and ccache. Replace USERNAME with your loginname. I want everything in my home folder, thus i took a more complex way of setting up everything.
Once setup is done, we will have the following directory structure:
/home/USERNAME/Projects/buildmachine/apt-cache
/home/USERNAME/Projects/buildmachine/ccache
/home/USERNAME/Projects/buildmachine/jessie-amd64.tar.gz
Here comes the installation:
root:# apt-get install sbuild ccache
root:# mkdir /root/.gnupg
root:# sbuild-update --keygen
root:# sbuild-adduser USERNAME
root:# newgrp sbuild
root:# cp /usr/share/doc/sbuild/examples/example.sbuildrc /home/USERNAME/.sbuildrc
root:# chown USERNAME:USERNAME /home/USERNAME/.sbuildrc
Create a debian-jessie minimal chroot
root:# mkdir -p /home/USERNAME/Projects/buildmachine
root:# sbuild-createchroot \
--arch=amd64 \
--components=main,contrib,non-free \
--make-sbuild-tarball=/home/USERNAME/Projects/buildmachine/jessie-amd64.tar.gz \
jessie \
`mktemp -d` \
http://ftp.de.debian.org/debian
That creates a debian-jessie minimal bootstrapped environment/fileystem into the specified folder (~/Projects/buildmachine/…tar.gz). As /etc/apt/sources.list components i registered main, contrib and non-free.
Create a apt-cache folder for the chroot
Next we create a special folder holding the deb-packages which get loaded from within the chroot build environment so that they do not get loaded from the internet over and over again for each build we start. We also do not want to fill/spam our host-machines apt-cache (/var/cache/apt/…) and therefore define a specific folder here to be used.
root:# mkdir -p /home/USERNAME/Projects/buildmachine/apt-cache
root:# chown USERNAME:USERNAME /home/USERNAME/Projects/buildmachine/apt-cache
root:# echo "/home/USERNAME/Projects/buildmachine/apt-cache /var/cache/apt/archives none rw,bind 0 0" >> /etc/schroot/sbuild/fstab
Customize the chroot, install a compiler and stuff
All modifications will be saved when exiting the chroot. First, enter the golden image to make modifications in it:
root:# schroot --chroot=source:jessie-amd64-sbuild
then modify the apt/sources.list file to match the followin:
vim /etc/apt/sources.list and add jessie-updates repository
---------------------
deb http://ftp.de.debian.org/debian jessie main contrib non-free
deb-src http://ftp.de.debian.org/debian jessie main contrib non-free
---------------------
Everything is done. exit the golden image for now.
root:# exit
update an existing chroot environment, apt-get upgrade in the chroot
root:# sbuild-update --update --dist-upgrade --clean --autoclean --autoremove jessie-amd64
install basic software in the golden image
Install basic needed development tools and libs / sources all my packages may depend on. I won’t build a complete userspace, so i’ll not build libc or zlib. Therefore those can be pre-installed in the chroot and this will save time when starting the chroot.
root:# schroot --chroot=source:jessie-amd64-sbuild
apt-get install \
autoconf \
automake \
autopoint \
bison \
build-essential \
bzip2 \
checkinstall \
cmake \
curl \
debhelper \
devscripts \
dh-make \
dh-autoreconf \
dpkg-dev \
fakeroot \
flex \
fontconfig-config \
fp-compiler \
g++ \
gawk \
gdc \
gettext \
gperf \
gzip \
make \
nasm \
ncurses-bin \
patch \
patchutils \
pkg-config \
pmount \
shtool \
swig \
tar \
unzip \
vim \
wget \
xtrans-dev \
yasm \
zip \
zlib1g-dev
exit
tune the buildenvironment
Let’s further tune the buildenvironment by
- disabling fsync-calls using eatmydata
- install ccache as compiler-cache
Issue the following commands:
root:# schroot --chroot=source:jessie-amd64-sbuild apt-get install eatmydata ccache
root:# echo "command-prefix=/var/cache/ccache-sbuild/sbuild-setup,eatmydata" >> /etc/schroot/chroot.d/jessie-amd64-sbuild-*
Install ccache in host environment:
root:# mkdir -p /home/USERNAME/Projects/buildmachine/ccache
root:# chgrp sbuild /home/USERNAME/Projects/buildmachine/ccache
root:# chmod g+rws /home/USERNAME/Projects/buildmachine/ccache
root:# env CCACHE_DIR=/home/USERNAME/Projects/buildmachine/ccache ccache --max-size 4G
root:# vim /home/USERNAME/Projects/buildmachine/ccache/sbuild-setup
--------------------
#!/bin/sh
export CCACHE_DIR=/var/cache/ccache-sbuild
export CCACHE_UMASK=002
export CCACHE_COMPRESS=1
unset CCACHE_HARDLINK
export PATH="/usr/lib/ccache:$PATH"
#exec "$@"
cmd=$1 ; shift
exec "$cmd" "$@"
--------------------
root:# chmod +x /home/USERNAME/Projects/buildmachine/ccache/sbuild-setup
root:# chown -R USERNAME /home/USERNAME/Projects/buildmachine/ccache
Make ccache folder and setup-script available in build-environments:
root:# echo "/home/USERNAME/Projects/buildmachine/ccache /var/cache/ccache-sbuild none rw,bind 0 0" >> /etc/schroot/sbuild/fstab
It this point we are finished installing our sbuild environment. See my other articles (like this one) for details on how to build debian deb packages from source using that sbuild environment.
Some commands useful to interact with the chroot
List all available chroots:
root:# schroot --list
Display detailed information to a specific chroot:
root:# schroot --info jessie-amd64
To enter a golden image to make some modifications, type exit to leave the image and save the changes:
root:# schroot --chroot=source:jessie-amd64-sbuild