Raspbian Buster and Cross Toolchain for Linux and Mac OS 8.3.0

Raspbian Buster is out. It seems it updated gcc from version 6.3.0 (used for Stretch) to gcc version 8.3.0. This is not always needed, but I had a lot of problems in the past related to using an outdated crosscompiler, or the one provided in the raspberry repo: https://github.com/raspberrypi/tools. I therefore built my own toolchain for Stretch, which I provided here, both for Linux and Mac OS.

I did the same for Buster: I built my own toolchain and I built many projects with it, like Qt, ffmpeg etc... Just place the toolchain in /opt/rpi (probably position independent) and you should be done. This toolchain should be 100% compatible with gcc provided by Buster.

Download cross toolchain GCC 6.3.0 for Stretch here.
Download Linux x64 cross toolchain GCC 8.3.0 here.
Download Mac OS cross toolchain GCC 8.3.0 here.

Bye ;-)

Raspbian Stretch and Cross Toolchain for Linux and Mac OS

Raspbian Stretch is out for Raspberry Pi. I needed a cross toolchain with the same exact version used to build the system so I built my own. As I had my script ready, I also tried to build one for Mac OS and it seems to work for the moment. I tested these cross toolchains for my projects and seemed to be able to build everything that I tried: Qt libs, apps, omxplayer, ffmpeg etc...

Download Linux x64 cross toolchain GCC 6.3.0 for Stretch here.
Download Mac OS cross toolchain GCC 6.3.0 for Stretch here.

NOTE: The Mac OS version requires some packages to be installed from macports.
NOTE: The toolchain is built to crossbuild for armv7-a. I could also build for armv8 but not sure if it also works for armv6.

Have fun! Bye! ;-)

Light Logging Facilities for Android/iOS/Linux/MacOS/Windows

I don't like to reinvent the wheel, and there are many logging codes out there, but still I couldn't find what I wanted. Those were all either too complex or not exactly what I wanted. That is why I took my old macros and added some ideas taken from this good tutorial: http://www.drdobbs.com/cpp/logging-in-c/201804215.

The result of a few hours is this: https://github.com/carlonluca/LightLogger. I'm using it on Windows/Linux/Mac OS/iOS/Android. This is an overview:

1. Simple: include the header and build.
2. Logging to Android results in logs sent to logcat (consider having a look at logcat-colorize, pretty project: https://bitbucket.org/brunobraga/logcat-colorize).
3. Supports the usual feature of log levels.
4. Supports coloring of logs: coloring is implemented using ANSI Escape sequences on Linux and MacOS. On Windows I disabled it, but you can enable if you're using something like cygwin or similar. For iOS I use XcodeColors, a great project: https://github.com/robbiehanson/XcodeColors. So a specific implementation is reserved for that platform.
5. Selecting debug levels with a macro at build-time enables/disables logs. Compiler optimizations should, in most cases, simply strip logs entirely from the binary, resulting in no/minimal overhead.
6. It is possible to reimplement the "sink" of the logs by reimplementing an output.
7. I preferred the printf way of formatting logs to the stream implementation. Anyway I tried to provide both. The usage of a "null sink" when logs are disabled should, together with compiler optimizations, make the overhead minimal.
8. Each log is flushed to avoid issues related to buffering. This might increase the overhead, but it is simple to remove it.
9. On Windows/Linux/iOS a stack trace function is also available to show the current call stack.
10. Should be entirely thread-safe.
11. Each log can be associated to a tag; I commonly use this with grep to filter logs by module.

There a still things I don't like about this approach, like being impossible to use in C sources, needing Objective-C++ instead of simple Objective-C and so on. Still someone may find it useful, so I uploaded to github: https://github.com/carlonluca/LightLogger.

How to Use

Just include in your sources and you're done. Most useful functions are those level-based:

inline bool log_info_t_v(const char* log_tag, const char* format, va_list args);
inline bool log_info_t(const char* log_tag, const char* format, ...);
inline bool log_info_v(const char* format, va_list args);
inline bool log_info(const char* format, ...);

These functions work differently according to the platform: on Android send INFO logs to logcat, on iOS print colored text to Xcode (and thus the XcodeColors plugin is needed if you keep colors enabled), on Windows simply print text and on Mac OS/Linux print text with ANSI colors to the shell. I use the return type to do something like:

if (!condition)
   return log_warn("Ooops, something bad happened, returning failure.");

These functions are "wrappers" for the LC_Log template. You choose a delegate and call:

LC_Log(...).printf(...);
LC_Log(...) << "Some stream based " << "text.";

or use the default logger:

LC_LogDef(...).printf(...);
LC_LogDef(...) << "Some stream based " << "text.";

For specific cases I also needed to write text with specific attributes in the past, so I added something like:

inline bool log_formatted_t_v(
   const char* log_tag,
   LC_LogAttrib a,
   LC_LogColor c,
   const char* format,
   va_list args
   );
inline bool log_formatted_t(
   const char* log_tag,
   LC_LogAttrib a,
   LC_LogColor c,
   const char* format,
   ...
   );
inline bool log_formatted_v(
   const char* format,
   va_list args
   );
inline bool log_formatted(
   LC_LogAttrib a,
   LC_LogColor c,
   const char* format,
   ...
   );
inline bool log_formatted(
   LC_LogColor c,
   const char* format,
   ...
   );

There are a few macros I use to configure for each project: COLORING_ENABLED to enable colors, BUILD_LOG_LEVEL_* to set the logging verbosity and XCODE_COLORING_ENABLED to enable/disable XcodeColors support.

In github you'll find Qt, iOS/XCode and Android sample projects.

How it Works

Pretty simple: wrapper functions like log_info(...) use the template class LC_Log to print the string. The LC_Log delegates actual log call to the class of type T, which can be implemented according to the needs. Delegates I currently implemented are:

1. LC_Output2Std: outputs to standard output, adding ANSI escape codes.
2. LC_Output2FILE: write the logs to file (no escape codes here).
3. LC_OutputAndroid: implements logging to logcat.
4. LC_Output2XCodeColors: implements logging using XcodeColors format.

Hope you can find this useful. If you improve this, share your work! If you find issues, report them! Bye! ;-)

Raspberry Pi Wheezy Image With OpenMAX-based Multimedia Backend

Please take into consideration that this firmware is based on an old firmware and an old PiOmxTextures version.

As I see still someone has troubles building PiOmxTextures, I built a new image based on the Raspbian wheezy image released the 26th of July (http://www.raspberrypi.org/downloads). This version of the image also contains Qt version 5.1.2 (35fe76d9c343f98ec55c7cacd8c402e90d91ef38) and an OpenMAX-based Qt Multimedia plugin (PiOmxTextures). The available Qt modules are:

• qtbase
• qtscript
• qtxmlpatterns
• qtjsbackend
• qtdeclarative
• qtsensors
• qtquick1
• qtgraphicaleffects
• qtwebkit
• qtwayland
• qtsvg
• qtserialport
• qttolls
• qttranslations
• qtquickcontrols
• qtmultimedia
• qtimageformats

The mediaservice plugin installed is the openmaxil plugin (available here: https://github.com/carlonluca/pi). It is based on ffmpeg version 2.0 and that version of ffmpeg is statically built inside the PiOmxTextures library available in /usr/lib.
Note that the wayland module and the wayland and xkbcommon libs are available in the image, but I never tested those. Also, PiOmxTextures won't work with wayland but only using the eglfs platform plugin.

How to Test

Of course all this is experimental and if you seriously intend to use in production I suggest you start working on the sources in https://github.com/carlonluca/pi. I placed a build of the POCPlayer (sources are available in the same repository under tools/POCPlayer) in the home directory of the pi user. You can run it and see how it works:

./POCPlayer [video_file]

By pressing the "l" button, you can see the possible options. The POC player is experimental as well and it is entirely implemented in QML.
You can help improve and fix both PiOmxTextures lib and the openmaxil plugin by simply replacing libPiOmxTextures in /usr/lib and/or libopenmaxilmediaplayer.so in /usr/local/Qt-rasp-5.1.2/plugins/mediaservice.

How to Use

Refer to Raspberry Pi Wheezy Image With Qt and Wayland Support for instructions on how to use. The procedure is almost identical. Then, you can simply build your QtMultimedia-based application and test.

Where to Download

Unfortunately the download is over 1GB. You'll have to download from this ed2k link. You can download it with this excellent torrent. Thanks to the author.

Please let me know if you experience troubles downloading or with the image itself. Bye!

Cross-building ICU for Applications on Embedded Devices

There are some pretty common libraries that come very handy in some situations when developing for common embedded systems like Android and iOS. One of this is the ICU library, which is useful when you need to work on code page conversion, collation, transliteration etc...

ICU is available as library for some systems, but it is quite large (libicudata for instance is more than 23MB alone). It is possible to reduce the size considerably reading this, and rebuilding.

ICU is perfectly portable on Linux, MacOS, Android, iOS, Linux Embedded etc... The process of cross-building is very simple, but still it took me a couple of hours to build for Linux, MacOS, iOS device and simulator and Android. So, I found some scripts around and fixed those bit (maybe the originals were a little outdated). I therefore write here a couple of notes on how to do it quickly (tested on 51.1).

Download the sources

I commonly download the sources from the repo directly:

export MY_DIR=some building directory
cd $MY_DIR
svn export http://source.icu-project.org/repos/icu/icu/tags/release-51-2 icu-51.2

You might want now to modify uconfig.h or data to avoid including data which is useless for your application: http://userguide.icu-project.org/icudata.

Build for Android

Cross-building ICU requires to build it first for the system where the cross-build is run, then for the target system. So, if we're using Linux when building for Android, let's first build ICU for Linux:

cd $MY_DIR
mkdir build_icu_linux
cd build_icu_linux

and use this script to build from there (change the variables and build options according to your needs):

export ICU_SOURCES=$MY_DIR/icu-51.2
export CPPFLAGS="-O3 -fno-short-wchar -DU_USING_ICU_NAMESPACE=1 -fno-short-enums \
-DU_HAVE_NL_LANGINFO_CODESET=0 -D__STDC_INT64__ -DU_TIMEZONE=0 \
-DUCONFIG_NO_LEGACY_CONVERSION=1 -DUCONFIG_NO_BREAK_ITERATION=1 \
-DUCONFIG_NO_COLLATION=1 -DUCONFIG_NO_FORMATTING=1 -DUCONFIG_NO_TRANSLITERATION=0 \
-DUCONFIG_NO_REGULAR_EXPRESSIONS=1"

sh $ICU_SOURCES/source/runConfigureICU Linux --prefix=$PWD/icu_build --enable-extras=no \
--enable-strict=no -enable-static --enable-shared=no --enable-tests=no \
--enable-samples=no --enable-dyload=no
make -j4
make install

Now you can build for Android:

cd $MY_DIR
mkdir build_icu_android
cd build_icu_android

and use this script:

export ICU_SOURCES=$MY_DIR/icu-51.2
export ANDROIDVER=8
export AR=/usr/bin/ar
export BASE=$MY_DIR
export HOST_ICU=$BASE/build_icu_android
export ICU_CROSS_BUILD=$BASE/build_icu_linux
export NDK_STANDARD_ROOT=your toolchain root
export CPPFLAGS="-I$NDK_STANDARD_ROOT/sysroot/usr/include/ \
-O3 -fno-short-wchar -DU_USING_ICU_NAMESPACE=1 -fno-short-enums \
-DU_HAVE_NL_LANGINFO_CODESET=0 -D__STDC_INT64__ -DU_TIMEZONE=0 \
-DUCONFIG_NO_LEGACY_CONVERSION=1 -DUCONFIG_NO_BREAK_ITERATION=1 \
-DUCONFIG_NO_COLLATION=1 -DUCONFIG_NO_FORMATTING=1 -DUCONFIG_NO_TRANSLITERATION=0 \
-DUCONFIG_NO_REGULAR_EXPRESSIONS=1"
export LDFLAGS="-lc -lstdc++ -Wl,-rpath-link=$NDK_STANDARD_ROOT/sysroot/usr/lib/"

export PATH=$PATH:$NDK_STANDARD_ROOT/bin

$ICU_SOURCES/source/configure --with-cross-build=$ICU_CROSS_BUILD \
--enable-extras=no --enable-strict=no -enable-static --enable-shared=no \
--enable-tests=no --enable-samples=no --enable-dyload=no \
--host=arm-linux-androideabi --prefix=$PWD/icu_build
make -j4
make install

In the icu_build directory you should have all you need to build your new application.
Note that I didn't use the NDK here, but the standard toolchain that results from the make-standalone-toolchain.sh script in the NDK.
Also note that part of ICU is already in /system/lib in some Android devices but I don't think there is any guarantee that it will be in every device (not part of the standard Android interface) and don't know exactly what is included inside that build.

Building for iOS

The same approach can be applied to cross-build for iOS. First, I built for MacOS in this case, and then for iOS device and simulator.

cd $MY_DIR
mkdir build_icu_mac
cd build_icu_mac

The script is similar to the Linux one:

ICUSRC_PATH=$MY_DIR/icu-51.2
export CPPFLAGS="-O3 -DU_USING_ICU_NAMESPACE=1 -fno-short-enums \
-DU_HAVE_NL_LANGINFO_CODESET=0 -D__STDC_INT64__ -DU_TIMEZONE=0 \
-DUCONFIG_NO_LEGACY_CONVERSION=1 -DUCONFIG_NO_BREAK_ITERATION=1 \
-DUCONFIG_NO_COLLATION=1 -DUCONFIG_NO_FORMATTING=1 -DUCONFIG_NO_TRANSLITERATION=0 \
-DUCONFIG_NO_REGULAR_EXPRESSIONS=1"

sh $ICUSRC_PATH/source/runConfigureICU MacOSX --prefix=$PWD/icu_build --enable-extras=no \
--enable-strict=no -enable-static --enable-shared=no --enable-tests=no \
--enable-samples=no --enable-dyload=no
make -j4
make install

Then I built for the iOS simulator in:

cd $MY_DIR
mkdir build_icu_simulator
cd build_icu_simulator

with this script:

DEVROOT=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer
SDKROOT=$DEVROOT/SDKs/iPhoneSimulator6.1.sdk
SYSROOT=$SDKROOT

ICU_SOURCES=$MY_DIR/icu-51.2
ICU_FLAGS="-I$ICU_PATH/source/common/ -I$ICU_PATH/source/tools/tzcode/ -O3 \
-DU_USING_ICU_NAMESPACE=1 -fno-short-enums -DU_HAVE_NL_LANGINFO_CODESET=0 \
-D__STDC_INT64__ -DU_TIMEZONE=0 -DUCONFIG_NO_LEGACY_CONVERSION=1 \
-DUCONFIG_NO_BREAK_ITERATION=1 -DUCONFIG_NO_COLLATION=1 -DUCONFIG_NO_FORMATTING=1 \
-DUCONFIG_NO_TRANSLITERATION=0 -DUCONFIG_NO_REGULAR_EXPRESSIONS=1"

export CPPFLAGS="-I$SDKROOT/usr/include/ -I$SDKROOT/usr/include/ -I./include/ \
-miphoneos-version-min=2.2 $ICU_FLAGS -pipe -arch i386 -no-cpp-precomp \
-isysroot $SDKROOT"

export LDFLAGS="-arch i386 -L$SDKROOT/usr/lib/ -isysroot $SDKROOT \
-Wl,-dead_strip -miphoneos-version-min=2.0"

sh $ICU_PATH/source/configure --host=i686-apple-darwin11 --enable-static --disable-shared \
-with-cross-build=/Users/luca/tmp/icu_build_mac --prefix=$PWD/icu_build
make -j4
make install

It works similarly for the arm device itself:

cd $MY_DIR
mkdir build_icu_device
cd build_icu_device

and this is the script to build:

DEVROOT=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer
SDKROOT=$DEVROOT/SDKs/iPhoneOS6.1.sdk
SYSROOT=$SDKROOT

ICU_PATH=$MY_DIR/icu-51.2
ICU_FLAGS="-I$ICU_PATH/source/common/ -I$ICU_PATH/source/tools/tzcode/ -O3 \
-fno-short-wchar -DU_USING_ICU_NAMESPACE=1 -fno-short-enums \
-DU_HAVE_NL_LANGINFO_CODESET=0 -D__STDC_INT64__ -DU_TIMEZONE=0 \
-DUCONFIG_NO_LEGACY_CONVERSION=1 -DUCONFIG_NO_BREAK_ITERATION=1 \
-DUCONFIG_NO_COLLATION=1 -DUCONFIG_NO_FORMATTING=1 -DUCONFIG_NO_TRANSLITERATION=0 \
-DUCONFIG_NO_REGULAR_EXPRESSIONS=1"

export CPPFLAGS="-I$SDKROOT/usr/include/ -I$SDKROOT/usr/include/ -I./include/ \
-miphoneos-version-min=2.2 $ICU_FLAGS -pipe -no-cpp-precomp -isysroot $SDKROOT"

export CC="$DEVROOT/usr/llvm-gcc-4.2/bin/arm-apple-darwin10-llvm-gcc-4.2"
export CXX="$DEVROOT/usr/llvm-gcc-4.2/bin/arm-apple-darwin10-llvm-g++-4.2"

export LDFLAGS="-L$SDKROOT/usr/lib/ -isysroot $SDKROOT -Wl,-dead_strip \
-miphoneos-version-min=2.0"

sh $ICU_PATH/source/configure --host=arm-apple-darwin --enable-static \
--disable-shared -with-cross-build=$MY_DIR/icu_build_mac --prefix=$PWD/icu_build
make -j4
make install

Post a comment if you find something wrong!
Of course the same approach might work similarly for other embedded devices with a proper toolchain :-)
Not too difficult, but still might speed up your work! ;-)