Dariusz on Software Quality & Performance


OpenEmbedded ccache integration

Filed under: en — Tags: , — dariusz.cieslak @

OpenEmbedded is a framework for building (linux based) distributions. It's something like "Gentoo for embedded systems".

Ccache is a tool that allows to reuse building results thus gives big speed improvements for frequent rebuilds (>10x faster or more then full rebuild).

For the current project I was given responsiblity for existing build system reorganisation. We have many hardware targets and sometimes full rebuild of whole operating system can take many hours even on strong machine (16 cores in our case).

Of course you can reuse existing build artefacts (*.o files), but sometimes it can be dangerous (we cannot gurantee header dependencies are properly tracked between projects). Thus "whole rebuild" sometimes is necessary.

I need every build will be created using ccache: native (for x86) and for target platform (sh4, mipsel). Ccache allows "integration by symlink": you create symlink from /usr/bin/ccache to /usr/local/bin/cc and ccache will discover compiler binary on PATH (/usr/bin/cc) and call it if necessary (rebuild of given artifact is needed). I used that method of ccache integration.

On the other hand OpenEmbedded allows to insert arbitrary commands before every build using so called "bbclasses" and "*_prepend" methods. Below you will find integration methid applied to very basic OE base class: base.bbclass, thus ensuring every build will use ccache:


do_compile_prepend() {
    rm -rf ${WORKDIR}/bin
    ln -sf `which ccache` ${WORKDIR}/bin/`echo ${CC} | awk '{print $1}'`
    ln -sf `which ccache` ${WORKDIR}/bin/`echo ${CXX} | awk '{print $1}'`
    export PATH=${WORKDIR}/bin:$PATH

    export CCACHE_BASEDIR="`pwd`"
    export CCACHE_LOGFILE=/tmp/ccache.log
    export CCACHE_SLOPPINESS="file_macro,include_file_mtime,time_macros"
    export CCACHE_COMPILERCHECK="none"

As a result you will receive pretty good hit rate during frequent rebuilds:

$ ccache -s
cache directory                     /home/sdk/.ccache
cache hit (direct)                 50239
cache hit (preprocessed)              27
cache miss                           889
called for link                     3965
no input file                      33506
files in cache                    182673
cache size                           9.6 Gbytes
max cache size                      20.0 Gbytes


Large C++ Project Build Time Optimisation

Filed under: en — Tags: — dariusz.cieslak @

When you hit some level of code size in a project you starting to observe the following sequence:

  1. Developer creates and tests a feature
  2. Before submitting commit to repository update/fetch/sync is done
  3. Developer builds project again to check if build/basic functionality is not broken
  4. Smoke tests
  5. Submit

During step 3 you hear "damn slow rebuild!". One discovers that synchronization with repository forces him to rebuild 20% of files in a project (and it takes time when project is really huge). Why?

The answer here is: header dependencies. Some header files are included (directly and indirectly) in many source code files, that's rebuild of so many files is needed. You have the following options:

  • Skip build dependencies and pray resulting build is stable / working at all
  • Reduce header dependencies

I'll explain second option.



Assert: To Abort Or Not To Abort, That's The Question

Filed under: en — Tags: , , — dariusz.cieslak @

Everyone agrees that internal state checking using assert(), Q_ASSERT(), assert are good. Programmer can declare expected input (asserting parameters), internal state (invariants) and verify return values (postconditions) and runtime will verify such expectations. There are languages with direct support for assertions in those three variants (Eiffel with his Design By Contract philosophy).

Those assertions typically will show filename/line number/message information and abort program / raise an exception if the condition is not met. Runtime environment then can collect current stacktrace to give developer more information on failed expectation.

One can disable assertions entirely (C++, Java) or select subset of assertions (Eiffel) for production releases. Resulting code will be faster, but failed expectations will not be verified by software – problem reports may be harder to interpret.

On the other hand if an assertion is too strict (the assumption may be invalid) it may abort program giving user negative impression about software stability.

What to do then? How can we keep problem-diagnosing power of enabled assertions and prevent minor of invalid failed assertions from aborting whole program?

The answer is: weak assertions (assertion without abort).



Automatic random testing for QT-based projects

Filed under: en — Tags: , , , , — dariusz.cieslak @

My current project I'm working on is based on embedded systems and QT platform. Of course the very first task in the project is to implement some kind of testing method to get feedback on software quality. The test system is composed from few components:

  • Automatic crash reports collected on central server
  • Automatic random test runners connected to always-running (24/7) devices to catch crashes

First channel collects all crashes (from human and automated tests), second channel is performed fully automatically. Second channel allows to measure MMTF (mean time between failures) and analyse changes in time, probably helping with estimating current software quality state.

Second testing channel requires automatic test driver to inject random UI events (key presses, remote in my case). I used QT message queue for that purpose:



Collecting crash reports over UDP using netcat

Filed under: en — Tags: , , , , — dariusz.cieslak @

Collecting runtime errors (crashes, failed assertions, …) is very important part of software quality efforts. If you know crash details from your testing team you can handle them even before a tester writes first line of error report (!). That improves development speed.

Probably the fastest method how to create KISS (Keep It Simple Stupid) central crash report repository is to use:

  • netcat – command line UDP server
  • crontab – for daily logs rotation


Older Posts »

Powered by WordPress