Dariusz on Software

There are people who prefer bleeding-edge software versions, other prefer stability over new features. I'm an advanced Linux user, however I fall into the latter category of users. Recently I discovered my current mixed-version Ubuntu is not as stable as I would like and I decided to revert to older version of Ubuntu (lucid).

Debian apt system allows to set preferences for packages and this method can be used to downgrade packages. First of all you have to declare sources of software of desired distro (lucid) in my case:

$ grep ^deb /etc/apt/sources.list
deb http://pl.archive.ubuntu.com/ubuntu/ lucid main restricted
deb http://pl.archive.ubuntu.com/ubuntu/ lucid-updates main restricted
deb http://pl.archive.ubuntu.com/ubuntu/ lucid universe
deb http://pl.archive.ubuntu.com/ubuntu/ lucid-updates universe
deb http://pl.archive.ubuntu.com/ubuntu/ lucid multiverse
deb http://pl.archive.ubuntu.com/ubuntu/ lucid-updates multiverse
deb http://security.ubuntu.com/ubuntu lucid-security main restricted
deb http://security.ubuntu.com/ubuntu lucid-security universe
deb http://security.ubuntu.com/ubuntu lucid-security multiverse

Then we can declare we prefer lucid even if current package in the system if newer:

$ cat /etc/apt/preferences.d/downgrade-to-lucid
Package: *
Pin: release a=lucid
Pin-Priority: 1010

Package: *
Pin: release a=lucid-updates
Pin-Priority: 1011

Package: *
Pin: release a=lucid-security
Pin-Priority: 1012

As you can see I added updates and security source pacages into account to have such updates already in my system.

OpenEmbedded is a framework for building (linux based) distributions. It's something like "Gentoo for embedded systems".

Ccache is a tool that allows to reuse building results thus gives big speed improvements for frequent rebuilds (>10x faster or more then full rebuild).

For the current project I was given responsiblity for existing build system reorganisation. We have many hardware targets and sometimes full rebuild of whole operating system can take many hours even on strong machine (16 cores in our case).

Of course you can reuse existing build artefacts (*.o files), but sometimes it can be dangerous (we cannot gurantee header dependencies are properly tracked between projects). Thus "whole rebuild" sometimes is necessary.

I need every build will be created using ccache: native (for x86) and for target platform (sh4, mipsel). Ccache allows "integration by symlink": you create symlink from /usr/bin/ccache to /usr/local/bin/cc and ccache will discover compiler binary on PATH (/usr/bin/cc) and call it if necessary (rebuild of given artifact is needed). I used that method of ccache integration.

On the other hand OpenEmbedded allows to insert arbitrary commands before every build using so called "bbclasses" and "*_prepend" methods. Below you will find integration methid applied to very basic OE base class: base.bbclass, thus ensuring every build will use ccache:

openembedded/classes/base.bbclass:

(...)
do_compile_prepend() {
    rm -rf ${WORKDIR}/bin
    ln -sf `which ccache` ${WORKDIR}/bin/`echo ${CC} | awk '{print $1}'`
    ln -sf `which ccache` ${WORKDIR}/bin/`echo ${CXX} | awk '{print $1}'`
    export PATH=${WORKDIR}/bin:$PATH

    export CCACHE_BASEDIR="`pwd`"
    export CCACHE_LOGFILE=/tmp/ccache.log
    export CCACHE_SLOPPINESS="file_macro,include_file_mtime,time_macros"
    export CCACHE_COMPILERCHECK="none"
}

As a result you will receive pretty good hit rate during frequent rebuilds:

$ ccache -s
cache directory                     /home/sdk/.ccache
cache hit (direct)                 50239
cache hit (preprocessed)              27
cache miss                           889
called for link                     3965
no input file                      33506
files in cache                    182673
cache size                           9.6 Gbytes
max cache size                      20.0 Gbytes

Recently I've hit problem that some package was silently not installed in rootfs. The reason was visible only in debug mode:

$ bitbake -D
(...)
Collected errors:
 * opkg_install_cmd: Cannot install package libnexus.
 * opkg_install_cmd: Cannot install package libnexus-dbg.
 * resolve_conffiles: Existing conffile /home/sdk/master/wow.mach/tmp/rootfs/sdk-image/etc/default/dropbear
is different from the conffile in the new package. The new conffile will be placed
at /home/sdk/master/wow.mach/tmp/rootfs/sdk-image/etc/default/dropbear-opkg.
 * opkg_install_cmd: Cannot install package libpng.
 * opkg_install_cmd: Cannot install package libpng-dbg.
 * opkg_install_cmd: Cannot install package directfb.
 * opkg_install_cmd: Cannot install package directfb-dbg.
 * opkg_install_cmd: Cannot install package libuuid-dbg.
 * check_data_file_clashes: Package comp-hal wants to install file /home/sdk/master/wow.mach/tmp/rootfs/sdk-image/etc/hosts But that file is already provided by package * netbase-dcc
 * opkg_install_cmd: Cannot install package package1.

/etc/hosts file was clashes with one from netbase-dcc package. The solution was to re-arrange file list for comp-hal (not include /etc/hosts there):

FILES_${PN} = "\
               (...)
               /etc \
               /usr/local/lib/cecd/plugins/*.so* \
               /usr/local/lib/qtopia/plugins/*.so* \
               (...)

/etc specification was replaced by more detailed description:

FILES_${PN} = "\
               (...)
               /etc/init.d \ /etc/rc3.d \
               /usr/local/lib/cecd/plugins/*.so* \
               /usr/local/lib/qtopia/plugins/*.so* \
               (...)