Synthesize Qt Settings

I frequently create classes that I use as an interface to settings in my apps. Using a single class with accessors makes the code readable, safer and simple to maintain. Unfortunately, doing this requires to create getters and setters for every entry in the settings file, which is a bit bothering. Also, QSettings is not a QObject and cannot provide signals to notify changes.

My use cases are typically very simple, so I created a couple of macros that synthesise classes for me. Macros synthesise a reentrant class that can be used to access settings and a notifier, that can be used to get notifications of the changes. An example of definition of the class is:

L_DECLARE_SETTINGS(LSettingsTest, new QSettings("settings.ini", QSettings::IniFormat))

L_DEFINE_VALUE(QString, string1, QString("string1"), toString)

L_DEFINE_VALUE(QSize, size, QSize(100, 100), toSize)

L_DEFINE_VALUE(double, temperature, -1, toDouble)

L_DEFINE_VALUE(QByteArray, image, QByteArray(), toByteArray)

L_END_CLASS

L_DECLARE_SETTINGS(LSettingsTestSec1, new QSettings("settings.ini", QSettings::IniFormat), "SECTION_1")

L_DEFINE_VALUE(QString, string2, QString("string2"), toString)

L_END_CLASS

this will let you instantiate objects of class LSettingsTest and LSettingsTestSec1, and access entries with strong typed methods. Also, by calling LSettingsTest::notifier(), you can get a reference to the unique notifier. By setting an instance as a context property, you can get notifications and you can update settings from QML. You can find some more info in the repo https://github.com/carlonluca/lqtutils and an example using Qt Quick here: https://github.com/carlonluca/lqtutils/tree/master/LQtUtilsQuick.

Bye! ;-)

Synthesize Qt properties

The are some utils that I use in most of my Qt projects. I'm a bit fed up of copy-pasting those every time I need something, so I created a project containing tools that are of frequent use when I write Qt apps, regardless of the type of app.

The only interesting tool I added yet is the synthesizer of Q_PROPERTY's. You can use the macros:

L_RW_PROP
L_RO_PROP

to synthétise props, with getter, setter and notifications. Each macro is overloaded: with 3 params, you do not have initialisation, adding a 4th param, also initialises the variable to the provided value. Also, I frequently need QObject that are just containers of properties, to be used in QML. I added these two macros to spare some lines:

L_BEGIN_CLASS(<class_name>)
L_END_CLASS

A class like:

class Fraction : public QObject
{
    Q_OBJECT
    Q_PROPERTY(double numerator READ numerator WRITE setNumerator NOTIFY numeratorChanged)
    Q_PROPERTY(double denominator READ denominator WRITE setDenominator NOTIFY denominatorChanged)
public:
    Fraction(QObject* parent = nullptr) : QObject(parent) {}
    double numerator() const {
        return m_numerator;
    }
    double denominator() const {
        return m_denominator;
    }
public slots:
    void setNumerator(double numerator) {
        if (m_numerator == numerator)
            return;
        m_numerator = numerator;
        emit numeratorChanged(numerator);
    }
    void setDenominator(double denominator) {
        if (m_denominator == denominator)
            return;
        m_denominator = denominator;
        emit denominatorChanged(denominator);
    }
signals:
    void numeratorChanged(double numerator);
    void denominatorChanged(double denominator);
private:
    double m_numerator;
    double m_denominator;
};

can be simplified to:

L_BEGIN_CLASS(Fraction)
L_RW_PROP(double, numerator, setNumerator)
L_RW_PROP(double, denominator, setDenominator)
L_END_CLASS

I'm sure there are other similar approaches out there, but if you need it, you can simply add this repo as a submodule like I do: https://github.com/carlonluca/lqtutils.

Bye! ;-)

Hardware Decoding in Chromium through QtWebEngine on Raspberry Pi

A few months back I decided to try to implement hardware decoding in WebKit. Unfortunately this task is always pretty long and complex for many reasons. I found the time to draft an implementation for WebKit1, which is pretty useless as WebKit1 in Qt is only used outside QML and JS is executed in the main thread. Unfortunately I never found the time to implement this in WebKit2, which runs in QML and is suitable for more fluid UIs. This was the result:

This is how YouTube was running with this implementation:

Now Qt has deprecated QtWebKit and is working heavily on QtWebEngine, which is built on Chromium, so I wanted to try this road. Unfortunately these kind of things always claim a lot of time, and I don't typically have that much, but I was able to start and get something done already.

Writing a complete solution in Chromium to decode and render video takes much time, so I thought of a shortcut: creating a custom VDA (Video Decode Accelerator) that loads the POT library and reusing its entire codebase to implement decode and rendering with little modifications. This proved to be possible and now I get something on the screen.

So, to summarise: a little patch to Chromium is needed to create a VDA that dynamically loads POT library into memory and uses it with a common interface. Data and calls are translated to POT structures and are sent to POT, which then processes the buffers properly. The result of the decode operation is then sent back to the VDA through the same interface and textures are then sent to Chromium for rendering.

Still many problems remain open, there is much to be done yet as you can see from the video, but something is drawn. Have a look at the demo:

Have fun! ;-)

Binaries for POT 5.5.0-beta1 on Qt 5.8.0-rc1 and Experimental Accelerated QtWebKit

A new version of Qt is out and some interesting changes are being developed. Interesting work was done on WebEngine. This package contains an experimental build of Qt 5.8.0-rc1 with the POT driver to provide hardware acceleration to video decoding and rendering. POT was updated to build on the new plugin architecture of Qt and includes ffmpeg 3.2.2. For info on the content of the package refer to this article. The youtube player still runs on WebKit 1 and so is still hardly useful.

Please report issues on github. Have fun ;-)

Download POT 5.5.0-beta1 for Raspbian Jessie Lite Pi2 (also tested on Pi3) here (md5: 7ca1b961ab8c70176f6aeb13d0bc4f9a).

Binaries for POT 5.3.0-beta1 on Qt 5.6.0 and Experimental Accelerated QtWebKit

This is an experimental version of POT that includes a few new features. The driver now includes an experimental version of playback of remote content including proper buffering of the media and the QtWebKit library is able to play videos using proper hardware acceleration.

Features

The driver can now be used through the regular Qt/QML interface to play content over protocols different from file://. I only tested HTTP but others should be available.
This new feature allowed me to also implement a new player in QtWebKit using the POT driver to stream content to the regular Qt QWebView. Unfortunately I could only find the time to implement this in the WebKit 1 branch, not in WebKit 2 which requires more work. This makes the implementation hardly useful, but it is a start. If anyone wanted to contribute this is where to start. This video shows the new implementation in action:

As QtWebKit now uses the GPU properly, I could also implement a YouTube player app using the same QWebView with just a few lines of code of regular Qt code:

In the video you can also see an application I wrote to implement a YouTube fullscreen player using the YouTube iframe API. Such a sample app is also included in the package.

Of course the real target would be to implement this in QtWebEngine. I was never able to find a version of QtWebEngine with video acceleration for Pi. Anyone who knows one?

Package

The package now includes:

• build_valgrind.tar: a build of valgrind to analyse your code;
• libpiomxtextures_qmlutils.so: QML plugin to provide a video probe in QML;
• piomxtextures_browser_we: an application that tests the QtWebEngine module (run passing a URL, ./piomxtextures_browser_we http://www.youtube.com);
• piomxtextures_browser_wk: an application that tests the QtWebKit module:
• you can run on WebKit 1 like ./piomxtextures_browser_wk http://www.youtube.com;
• or you can run on WebKit 2 like ./piomxtextures_browser_wk --wk2 http://www.youtube.com;
• piomxtextures_pocplayer: a sample QML player;
• piomxtextures_pocplayer_yt: a youtube sample player (run passing a video ID like ./piomxtextures_pocplayer_yt 71UvXMzVgx4);
• qtdeps.tar: the usual libs needed by Qt and POT;
• Qt-rasp2-5.6.0.tar: a build of Qt 5.6.0 stable including:
• regular Qt 5.6.0 modules;
• untested bluetooth module including BLE support;
• untested MySQL plugin;
• untested QtFtp module;
• untested Qt hat tools module;
• QtWebEngine;
• QtWebKit.

Please report any bug you find on github. Don't use in production. Sources will be available shortly. Have fun! ;-)

Download POT 5.3.0-beta1 for Raspbian Jessie Lite Pi2 (also tested on Pi3) here (md5: edbe0ad2a552a5c8280e3876d16b237d).

Binaries for Hardware Accelerated Qt 5.6.0-rc1 Multimedia Backend on Raspbian Jessie Lite (POT 5.2.2)

This is a new build including a video probe to monitor frames being rendered. I also included a plugin to access video probe functionalities from QML. You can use it like this:

POT_VideoProbe {
   source: mediaPlayer
   onVideoFrameProbed: console.log("Frame probed!")
}

where mediaPlayer is an element of type MediaPlayer. Modifying the frame is not probably possible at the moment.

Have fun! Bye ;-)

Download PiOmxTextures 5.2.2 for Raspbian Jessie Pi2 here (extraction code is: 2bd6, md5:
   8be82eee7498e63e63ecbe17cb3eb7ef).

Fully Optimised Raspbian Jessie Lite with Accelerated Qt Multimedia Backend (Pi2)

This is a follow-up to the image provided in this article. The concept is identical, but in this case this firmware is smaller, it is based on Raspbian Jessie Lite and includes updated software. The firmware was 1.9GB while now is 759MB. This is the current setup (but I encourage to update to the latest possible firmware):

pi@raspberrypi:~ $ vcgencmd version
Feb 25 2016 14:25:47 
Copyright (c) 2012 Broadcom
version dea971b793dd6cf89133ede5a8362eb77e4f4ade (clean) (release)
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.1.18-v7+ #846 SMP Thu Feb 25 14:22:53 GMT 2016 armv7l GNU/Linux

Download the image here (torrent) (md5: 06d75d03f63674350be6cb807850bf09).
(Please be patient while downloading and decompressing)

The image includes:

• Qt 5.6.0-rc1: built for armv7 with neon optimisations, including support for touch screens, gstreamer, libinput, tslib, and Bluetooth/Bluetooth BLE using bluez (complete configuration below).
• QtWebKit 5.6.0: includes WebKit 1 and WebKit 2.
• QtWebEngine 5.6.0-rc1: multimedia only based on ffmpeg.
• Dependant libs: evdev, icu (not the one provided by raspbian), mtdev, libts.
• Samples: samples from the repo to test POT.
• POT: POT version 5.2.1 including ffmpeg 2.8.6.

Samples

To have a quick look at multimedia in Qt/QML test with this commands:

• /usr/local/Qt-rasp2-5.6.0-rc1/bin/qmlscene "some_video_file" samples/video_simple.qml
• /usr/local/Qt-rasp2-5.6.0-rc1/bin/qmlscene "some_url" samples/webkit_simple.qml
• /usr/local/Qt-rasp2-5.6.0-rc1/bin/qmlscene "some_url" samples/webengine_simple.qml
• /home/pi/piomxtextures_pocplayer --multipleanimtest "720p_video_1" "720p_video_2"

Please let me know if you find something wrong and file a bugreport on https://github.com/carlonluca/pi.

Have fun! ;-)

Fully Optimised Raspbian Jessie with Accelerated Qt Multimedia Backend (Pi2)

This is an image of Raspbian Jessie ready to test (and use) the Qt Multimedia Backend POT. This image is optimised for maximum performance (see the demos below). This is the current setup:

pi@raspberrypi ~ $ sudo /opt/vc/bin/vcgencmd version
Nov 11 2015 21:31:07 
Copyright (c) 2012 Broadcom
version 54011a8ad59a9ae1c40bd07cddd9bcf90e779b66 (clean) (release)
pi@raspberrypi ~ $ uname -a
Linux raspberrypi 4.1.13-v7+ #826 SMP PREEMPT Fri Nov 13 20:19:03 GMT 2015 armv7l GNU/Linux
pi@raspberrypi ~ $ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 8.0 (jessie)
Release:        8.0
Codename:       jessie

Download image here.

The image includes:

• Qt 5.5.1 built for armv7 with neon optimisations, including support for touch screens, gstreamer, libinput, X11 (no multimedia), tslib, and Bluetooth/Bluetooth BLE using bluez (complete configuration below). Modules provided include QtWebKit (no multimedia support).
• Dependant libs: evdev, icu (not the one provided by raspbian), mtdev, libts.
• POCPlayer and QML samples to show the performance of the backend and test bugs.
• POT library including ffmpeg 2.7.2.

Everything is built for armv7 including neon optimisations. To quickly test the performance there are a few things you can launch:

/home/pi/piomxtextures_pocplayer --multipleanimtest /home/pi/sintel_trailer_720p.mp4 /home/pi/big_buck_bunny_720p_h264.mov

This is the result you should expect with a proper overclocking:

It will show you a continuous animation while running two 720p videos concurrently. To improve the frame rate of the animations you can also overclock your Pi2 (see below).

Other samples to test the features of the plugin are stored in /home/pi/samples:

/usr/local/Qt-rasp2-5.5.1/bin/qmlscene /home/pi/samples/video_simple.qml file:///home/pi/big_buck_bunny_1080p_h264.mov

Another interesting test is running qmlvideofx example included in the Qt sources. This will show you the perfomance of the effects applied on the video using shaders. You can find an executable in /usr/local/Qt-rasp2-5.5.1/examples/multimedia/video/qmlvideofx/qmlvideofx. This shows approximately what you should expect (the video shows the result on Pi1, Pi2 is just a little better):

Overclocking

The Pi can be overclocked by tuning parameters in /boot/config.txt. I provided a configuration that I find useful:

#force_turbo=1  # Voids Warranty!

#boot_delay=1   # Helps to avoid sdcard corruption when force_turbo is enabled.

#arm_freq=1100

#sdram_freq=450

#core_freq=550

#over_voltage=4

#temp_limit=80  # Will throttle to default clock speed if hit.

In the image this is all disabled. If you intent to enable please keep in mind I don't take responsibility for any consequence. Please refer to the documentation for this procedure.

Have fun! Bye ;-)

Qt Configuration

   Configure summary

Building on:   linux-g++ (x86_64, CPU features: mmx sse sse2)

Building for:  devices/linux-rasp-pi2-g++ (arm, CPU features: neon)

Platform notes:

            - Also available for Linux: linux-clang linux-kcc linux-icc linux-cxx

        

qmake vars .......... styles += mac fusion windows QT_CFLAGS_GLIB = -pthread -I/opt/rpi/sysroot/usr/include/glib-2.0 -I/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf/glib-2.0/include  QT_LIBS_GLIB = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lgthread-2.0 -pthread -lglib-2.0  QT_CFLAGS_PULSEAUDIO = -D_REENTRANT -I/opt/rpi/sysroot/usr/include/glib-2.0 -I/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf/glib-2.0/include  QT_LIBS_PULSEAUDIO = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lpulse-mainloop-glib -lpulse -lglib-2.0  QMAKE_CFLAGS_FONTCONFIG = -I/opt/rpi/sysroot/usr/include/freetype2  QMAKE_LIBS_FONTCONFIG = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lfontconfig -lfreetype  QMAKE_INCDIR_LIBUDEV =  QMAKE_LIBS_LIBUDEV = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -ludev  QMAKE_LIBINPUT_VERSION_MAJOR = 1 QMAKE_LIBINPUT_VERSION_MINOR = 0 QMAKE_INCDIR_LIBINPUT = /opt/rpi/sysroot/home/pi/qtdeps/include  QMAKE_LIBS_LIBINPUT = -L/opt/rpi/sysroot/home/pi/qtdeps/lib -linput  QMAKE_LIBXI_VERSION_MAJOR = 1 QMAKE_LIBXI_VERSION_MINOR = 7 QMAKE_LIBXI_VERSION_PATCH = 4 QMAKE_X11_PREFIX = /usr QMAKE_XKB_CONFIG_ROOT = /usr/share/X11/xkb QMAKE_CFLAGS_XCB =  QMAKE_LIBS_XCB = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lxcb-sync -lxcb-xfixes -lxcb-render -lxcb-randr -lxcb-image -lxcb-shm -lxcb-keysyms -lxcb-icccm -lxcb-shape -lxcb  INCLUDEPATH +=  "/opt/rpi/sysroot/home/pi/qtdeps/include" LIBS +=  -L"/opt/rpi/sysroot/home/pi/qtdeps/lib" sql-drivers =  sql-plugins =  sqlite qmake switches ......... 

Build options:

  Configuration .......... accessibility accessibility-atspi-bridge alsa audio-backend c++11 clock-gettime clock-monotonic compile_examples concurrent cross_compile cups dbus egl eglfs eglfs_brcm enable_new_dtags evdev eventfd fontconfig full-config getaddrinfo getifaddrs glib gstreamer-1.0 harfbuzz iconv icu inotify ipv6ifname large-config largefile libinput libproxy libudev linuxfb medium-config minimal-config mremap mtdev neon nis opengl opengles2 openssl pcre png posix_fallocate pulseaudio qpa qpa reduce_exports release rpath shared small-config system-freetype system-jpeg system-png system-zlib tslib use_gold_linker xcb xcb-glx xcb-plugin xcb-render xcb-xlib xinput2 xkbcommon-qt xlib xrender 

  Build parts ............  libs examples

  Mode ................... release

  Using sanitizer(s)...... none

  Using C++11 ............ yes

  Using gold linker....... yes

  Using new DTAGS ........ yes

  Using PCH .............. no

  Target compiler supports:

    Neon ................. yes

Qt modules and options:

  Qt D-Bus ............... yes (loading dbus-1 at runtime)

  Qt Concurrent .......... yes

  Qt GUI ................. yes

  Qt Widgets ............. yes

  Large File ............. yes

  QML debugging .......... yes

  Use system proxies ..... no

Support enabled for:

  Accessibility .......... yes

  ALSA ................... yes

  CUPS ................... yes

  Evdev .................. yes

  FontConfig ............. yes

  FreeType ............... yes (system library)

  Glib ................... yes

  GStreamer .............. yes (1.0)

  GTK theme .............. no

  HarfBuzz ............... yes (bundled copy)

  Iconv .................. yes

  ICU .................... yes

  Image formats: 

    GIF .................. yes (plugin, using bundled copy)

    JPEG ................. yes (plugin, using system library)

    PNG .................. yes (in QtGui, using system library)

  journald ............... no

  libinput................ yes

  mtdev .................. yes (system library)

  Networking: 

    getaddrinfo .......... yes

    getifaddrs ........... yes

    IPv6 ifname .......... yes

    libproxy.............. yes

    OpenSSL .............. yes (loading libraries at run-time)

  NIS .................... yes

  OpenGL / OpenVG: 

    EGL .................. yes

    OpenGL ............... yes (OpenGL ES 2.0+)

    OpenVG ............... no

  PCRE ................... yes (bundled copy)

  pkg-config ............. yes 

  PulseAudio ............. yes

  QPA backends: 

    DirectFB ............. no

    EGLFS ................ yes

      EGLFS i.MX6....... . no

      EGLFS KMS .......... no

      EGLFS Mali ......... no

      EGLFS Raspberry Pi . yes

      EGLFS X11 .......... no

    LinuxFB .............. yes

    XCB .................. yes (system library)

      EGL on X ........... no

      GLX ................ yes

      MIT-SHM ............ yes

      Xcb-Xlib ........... yes

      Xcursor ............ yes (loaded at runtime)

      Xfixes ............. yes (loaded at runtime)

      Xi ................. no

      Xi2 ................ yes

      Xinerama ........... yes (loaded at runtime)

      Xrandr ............. yes (loaded at runtime)

      Xrender ............ yes

      XKB ................ yes

      XShape ............. yes

      XSync .............. yes

      XVideo ............. yes

  Session management ..... yes

  SQL drivers: 

    DB2 .................. no

    InterBase ............ no

    MySQL ................ no

    OCI .................. no

    ODBC ................. no

    PostgreSQL ........... no

    SQLite 2 ............. no

    SQLite ............... yes (plugin, using bundled copy)

    TDS .................. no

  tslib .................. yes

  udev ................... yes

  xkbcommon-x11........... yes (bundled copy, XKB config root: /usr/share/X11/xkb)

  xkbcommon-evdev......... no

  zlib ................... yes (system library)

Binaries for Hardware Accelerated Qt 5.5.1 Multimedia Backend on Raspbian Jessie (POT 5.1.0)

This is a build of version 5.1.0 for Raspbian Jessie of POT that includes Qt 5.5.1. As many asked in the past, this version also includes the QtWebKit module (WebKit1 and WebKit2). The changes are:

• Fixed bugs reported on github.
• New environment variables used for controlling deinterlacing:
• POT_HALF_FRAMERATE_MODE: if set to 1 the number of frames sent to the renderer are halved. This is needed mostly when using deinterlacing.
• POT_TEXTURE_COUNT: number of texture to be used when buffering. Default is 4.
• POT_DEINTERLACE_MODE: the type of deinterlacing; 0 means off, 1 means auto and 2 means on.
• POT_DEINTERLACE_QPU: 0 or 1; indicates whether to use the QPU for deinterlacing.
• Added watchdogs to monitor the video capabilities and discover instability. These are currently disable but you can enable if needed.
• Some other minor fixes and improvements.

Please remember to update the firmware to the latest possible version as it includes a fix relevant to POT.
Please file bugs to github if you experience issues.
Bye! ;-)

Download PiOmxTextures 5.1.0 for Raspbian Jessie Pi2 here (extraction code is: bd4f).
Download PiOmxTextures 5.1.0 for Raspbian Jessie Pi2 with Bluetooth (BLE) here.

This is the configure summary of the Qt build:
   Configure summary

Building on:   linux-g++ (x86_64, CPU features: mmx sse sse2)
Building for:  devices/linux-rasp-pi2-g++ (arm, CPU features: neon)
Platform notes:

            - Also available for Linux: linux-clang linux-kcc linux-icc linux-cxx
        
qmake vars .......... styles += mac fusion windows QT_CFLAGS_GLIB = -pthread -I/opt/rpi/sysroot/usr/include/glib-2.0 -I/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf/glib-2.0/include  QT_LIBS_GLIB = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lgthread-2.0 -pthread -lglib-2.0  QT_CFLAGS_PULSEAUDIO = -D_REENTRANT -I/opt/rpi/sysroot/usr/include/glib-2.0 -I/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf/glib-2.0/include  QT_LIBS_PULSEAUDIO = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lpulse-mainloop-glib -lpulse -lglib-2.0  QMAKE_CFLAGS_FONTCONFIG = -I/opt/rpi/sysroot/usr/include/freetype2  QMAKE_LIBS_FONTCONFIG = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lfontconfig -lfreetype  QMAKE_INCDIR_LIBUDEV =  QMAKE_LIBS_LIBUDEV = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -ludev  QMAKE_LIBINPUT_VERSION_MAJOR = 1 QMAKE_LIBINPUT_VERSION_MINOR = 0 QMAKE_INCDIR_LIBINPUT = /opt/rpi/sysroot/home/pi/qtdeps/include  QMAKE_LIBS_LIBINPUT = -L/opt/rpi/sysroot/home/pi/qtdeps/lib -linput  QMAKE_LIBXI_VERSION_MAJOR = 1 QMAKE_LIBXI_VERSION_MINOR = 7 QMAKE_LIBXI_VERSION_PATCH = 4 QMAKE_X11_PREFIX = /usr QMAKE_XKB_CONFIG_ROOT = /usr/share/X11/xkb QMAKE_CFLAGS_XCB =  QMAKE_LIBS_XCB = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lxcb-sync -lxcb-xfixes -lxcb-render -lxcb-randr -lxcb-image -lxcb-shm -lxcb-keysyms -lxcb-icccm -lxcb-shape -lxcb  INCLUDEPATH +=  "/opt/rpi/sysroot/home/pi/qtdeps/include" LIBS +=  -L"/opt/rpi/sysroot/home/pi/qtdeps/lib" sql-drivers =  sql-plugins =  sqlite qmake switches ......... 

Build options:
  Configuration .......... accessibility accessibility-atspi-bridge alsa audio-backend c++11 clock-gettime clock-monotonic compile_examples concurrent cross_compile cups dbus egl eglfs eglfs_brcm enable_new_dtags evdev eventfd fontconfig full-config getaddrinfo getifaddrs glib gstreamer-1.0 harfbuzz iconv icu inotify ipv6ifname large-config largefile libinput libproxy libudev linuxfb medium-config minimal-config mremap mtdev neon nis opengl opengles2 openssl pcre png posix_fallocate pulseaudio qpa qpa reduce_exports release rpath shared small-config system-freetype system-jpeg system-png system-zlib tslib use_gold_linker xcb xcb-glx xcb-plugin xcb-render xcb-xlib xinput2 xkbcommon-qt xlib xrender 
  Build parts ............  libs examples
  Mode ................... release
  Using sanitizer(s)...... none
  Using C++11 ............ yes
  Using gold linker....... yes
  Using new DTAGS ........ yes
  Using PCH .............. no
  Target compiler supports:
    Neon ................. yes

Qt modules and options:
  Qt D-Bus ............... yes (loading dbus-1 at runtime)
  Qt Concurrent .......... yes
  Qt GUI ................. yes
  Qt Widgets ............. yes
  Large File ............. yes
  QML debugging .......... yes
  Use system proxies ..... no

Support enabled for:
  Accessibility .......... yes
  ALSA ................... yes
  CUPS ................... yes
  Evdev .................. yes
  FontConfig ............. yes
  FreeType ............... yes (system library)
  Glib ................... yes
  GStreamer .............. yes (1.0)
  GTK theme .............. no
  HarfBuzz ............... yes (bundled copy)
  Iconv .................. yes
  ICU .................... yes
  Image formats: 
    GIF .................. yes (plugin, using bundled copy)
    JPEG ................. yes (plugin, using system library)
    PNG .................. yes (in QtGui, using system library)
  journald ............... no
  libinput................ yes
  mtdev .................. yes (system library)
  Networking: 
    getaddrinfo .......... yes
    getifaddrs ........... yes
    IPv6 ifname .......... yes
    libproxy.............. yes
    OpenSSL .............. yes (loading libraries at run-time)
  NIS .................... yes
  OpenGL / OpenVG: 
    EGL .................. yes
    OpenGL ............... yes (OpenGL ES 2.0+)
    OpenVG ............... no
  PCRE ................... yes (bundled copy)
  pkg-config ............. yes 
  PulseAudio ............. yes
  QPA backends: 
    DirectFB ............. no
    EGLFS ................ yes
      EGLFS i.MX6....... . no
      EGLFS KMS .......... no
      EGLFS Mali ......... no
      EGLFS Raspberry Pi . yes
      EGLFS X11 .......... no
    LinuxFB .............. yes
    XCB .................. yes (system library)
      EGL on X ........... no
      GLX ................ yes
      MIT-SHM ............ yes
      Xcb-Xlib ........... yes
      Xcursor ............ yes (loaded at runtime)
      Xfixes ............. yes (loaded at runtime)
      Xi ................. no
      Xi2 ................ yes
      Xinerama ........... yes (loaded at runtime)
      Xrandr ............. yes (loaded at runtime)
      Xrender ............ yes
      XKB ................ yes
      XShape ............. yes
      XSync .............. yes
      XVideo ............. yes
  Session management ..... yes
  SQL drivers: 
    DB2 .................. no
    InterBase ............ no
    MySQL ................ no
    OCI .................. no
    ODBC ................. no
    PostgreSQL ........... no
    SQLite 2 ............. no
    SQLite ............... yes (plugin, using bundled copy)
    TDS .................. no
  tslib .................. yes
  udev ................... yes
  xkbcommon-x11........... yes (bundled copy, XKB config root: /usr/share/X11/xkb)
  xkbcommon-evdev......... no
  zlib ................... yes (system library)

Raspbian Wheezy 05.05.2015 including Qt 5.5.0 and Accelerated Qt Multimedia Backend

As some asked in the past, this is a Raspbian Wheezy image 05.05.2015 with fully working POT built for armv6 (which therefore works on both Pi 1 and Pi 2). It includes Qt 5.5.0, POT, ffmpeg 2.7.2 and all the libs needed by Qt not provided by Raspbian.

Download Raspbian 05.05.2015 with Qt, ffmpeg, POT and all dependencies here (extraction code is: 22e9).

Sorry but the download will probably take very long.

Flash it on a micro SD-card and test by running some of the QML samples I placed in there:

/usr/local/Qt-rasp-5.5.0/bin/qmlscene samples/video_position.qml file:///home/pi/big_buck_bunny_1080p_h264.mov

Those are mostly tests for debugging so they won't make much sense. Also you can run a command like this:

./piomxtextures_pocplayer --multipleanimtest /home/pi/big_buck_bunny_720p_h264.mov /home/pi/big_buck_bunny_720p_h264.mov

(You'll need to transfer a 720p video for this demo as it will be running both videos concurrently.) On Pi 2 the result you'd get with full optimisations is this:

The version of POT included is 5.0.0. I don't test the samples for every release so it is possible some do not work. You can go to https://github.com/carlonluca/pi/tree/master/piomxtextures_samples and get updates.

This image only makes sense for a quick test, it is not fully optimised and it is not even built with armv7 optimisations for Pi 2. I may provide updates in the future if I got a considerable amount of requests (unfortunately uploading full images requires a considerable amount of time).

You can however write your own qml samples and run right away. You can rebuild just the library and replace it to strip away debug messages or to build a newer version. Also you can write your Qt apps, cross compile using your environment and run on the pi.

As usual report bugs to github.

Bye ;-)

Binaries for Hardware Accelerated Qt Multimedia Backend on Raspberry Pi (4.4.0)

This is a package containing binaries for PiOmxTextures 4.4.0. The relevant improvements here over 4.3.0 are:

• Fixed horizontal tearing effect.
• Fixed leaks when starting/stopping and restarting video.
• Added support for 3.5mm analog audio output. It is now possible to select which one to use (hdmi, analog or both) by setting a AUDIO_OUT env variable (possible values are both, local and hdmi; default: hdmi).
• Fixed garbage at the beginning of the video.
• Merged code from omxplayer code base to revision 74aac. This includes update to ffmpeg 2.6.
• Other minor fixes and improvements.
• Port to Qt 5.4.1.

The package contains:

• piomxtextures_pocplayer: sample player which includes tests for animations, loops and commands. Have a look at the sources to understand how it works.
• Qt-rasp-5.4.1.tar: the entire tree of Qt 5.4.1 binaries built for Raspbian (firmware
     b71d7b6, rpath is set to /usr/local/Qt-rasp-5.4.1).
• Qt-rasp-5.4.1_host.tar: the utility to be used on the host when cross-building (built for x86 Kubuntu 13.10).

Current version of the package is 4.4.0: download.

To have a quick look, just place Qt-rasp-5.4.1 in /usr/local into your Pi. Now you can test running qmlvideo, qmlvideofx or piomxtextures_pocplayer from the Qt Multimedia examples. Something similar to this should come up: 

To build using the host tools:

1. Place the Qt libs provided in /usr/local in your Pi filesystem.
2. Place the sysroot of your Pi into /opt/rpi/sysroot (you should then have Qt libs into /opt/rpi/sysroot/usr/local/Qt-rasp-5.4.1).
3. Place the host tools in /usr/local.
4. Place the Linaro toolchain (https://github.com/raspberrypi/tools/tree/master/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian) in /opt/rpi (which means you should have /opt/rpi/gcc-linaro-arm-linux-gnueabihf-raspbian).
5. You should be done. 

I tested on a Ubuntu 15.04 64 bit and the procedure worked, just be sure to use the 64 bit toolchain (in the same position) and add compatibility libs (qmake is a 32 bit binary).

Bye! ;-)

NOTE: Please notice that the provided binaries contain both the gstreamer plugin (the default unaccelerated Qt multimedia plugin) and the PiOmxTexures (POT) plugin. You'll have to ensure that your app uses POT to run and not gstreamer. To avoid confusion, you can simply remove (or move away) the gstreamer plugin from the default path (/usr/local/Qt-rasp-5.4.1/plugins/mediaservice).

Updates on Hardware Accelerated Qt Multimedia Backend on Raspberry Pi (01.05.2015)

Many complained about an horizontal tearing issue when playing videos on the Pi using the PiOmxTextures driver. The issue was caused by a missing switch of buffers backing the textures on the output of the renderer component. I took a little time to quickly improve the rendering and the new result is interesting. I also fixed an OpenMAX setup that caused my Pi to crash when stressed (have a look at the commit logs). These were the videos showing the old implementation:

Pay attention to the fact that video was only 720p when shaders were applied. I couldn't do it on 1080p.

Consider that in these videos animations involving both 1080p graphics and 1080p source video were very difficult. In fact you can see that video is sometimes only 720p.

After recent changes I tried again the same sample apps and created a small demo of what you can achieve with Pi. Now source video, images and UI are all 1080p resolution.

Source code can be found in the usual github repo. At the moment the latest changes are all committed to the branch no_tearing. Keep in mind that those changes may break the rest of the implementation, so some work is needed if you really want to use it in your project. The modifications were only made to show what performance Pi can achieve. You're welcome to send patches to fix the branch as well if you're interested.

Updates on Hardware Accelerated Qt Multimedia Backend on Raspberry Pi (23.12.2014)

As I see some are still working on the Pi, and someone asked to "port" to Qt 5.4.0, this is a small update to the Qt Multimedia Backend POC.

This is what I updated so far:

• Merged code from omxplayer until a4ee074.
• Updated to work with ffmpeg 2.5.
• Updated to work on the latest Pi firmware ba43047.
• Updated some build scripts to speed up build.
• Updated to work with the new Qt 5.4.0.

The build procedure is mostly unchanged. I summarise it here with the new updates (I assume you're in the root of the repo sources):

export RPI_SYSROOT=
export COMPILER_PATH=
cd tools
./compile_ffmpeg.sh
export QMAKE_PATH=
./prepare_openmaxil_backend.sh


This should build ffmpeg, build the PiOmxTextures lib (needed to get the hardware interface to video and audio playback) and prepare the sources of the Qt multimedia backend.
NOTE: I had to modify the libssh pkgconfig file ($RPI_SYSROOT/usr/lib/arm-linux-gnueabihf/pkgconfig/libssh.pc) to:


prefix=/usr
exec_prefix=${prefix}
libdir=${prefix}/lib/arm-linux-gnueabihf
includedir=${prefix}/include

Name: libssh
Description: The SSH Library
Version: 0.5.4
Libs: -L{libdir} -lssh
Cflags: -I{includedir}

Now you have to copy the directory of the plugin to the src/plugins subdirectory of your Qt source tree. Once finished, this plugin will be built.
Unfortunately, still a patch to QtMultimedia is needed, so go into the qtmultimedia dir of your Qt source tree (I tested on the Qt 5.4.0 tree) and patch using the patch provided in tools:

cp -a /openmaxil_backend /qtmultimedia/src/plugins/openmaxil
patch -p1 < /tools/qt_multimedia_patch_5.4.0.patch
$QMAKE_PATH CONFIG+=raspberry
make -j
make install


Now you should have the new openmaxil plugin in your Qt prefix. When using, remember that libPiOmxTextures must be in the runtime library path when using the plugin. If an error like this is returned when running an application using the plugin:

luca@raspberrypi ~ $ ./POCPlayer
* failed to open vchiq instance

then set the permissions appropriately:

sudo chmod a+rw /dev/vchiq

Or better add your user to the video group. Same stands for input devices:

sudo usermod -a -G video
sudo usermod -a -G input

As usual remember this is just a POC.
If anything went wrong, file an issue on github. Someone may have a look.
Bye! ;-)

NOTE: It seems that 1080p video on 1080p output really stresses the GPU at the point that the video starts to have issues and blocks. I tried to test Pi turbo mode, that should increase GPU performance but my Pi seems not to agree on the overvolt parameter. USB ports completely crash and then the system goes bye-bye. If anyone was able to test somehow animations and 1080p playout on 1080p resolution with turbo mode and like 450MHz GPU, please let me know!

Qt on Linux i.MX 6

i.MX 6 is a good device. Qt runs pretty well on it. Also a plugin for hardware accelerated and zero-copy rendering seems to work pretty well. Simply building Qt for i.MX 6 makes it possible to use the Qt Multimedia plugin. This is the result (on Linux):

I also tested a provided Android port but the result was not as good.

Video with Animations on Raspberry Pi and A31s Compared

For many uses, playing video while animations are running is pretty useful and it is also considerably stressful for the GPU. I therefore tested a couple of cheap chips with the same Qt/QML sample app. I created a small demo video comparing Qt on A31s and Raspberry Pi (PiOmxTextures):

In the demo, all the media (video and images) are 720p, and the output is 720p for A31s and 1080p for Raspberry Pi (16bpp). It is pretty hard to tell from the video, but Raspberry seems to still be above A31s.
Bye! ;-)

A Tribute to Qt Portability :-)

The new ports of Qt to mobile systems like Android, iOS and WinRT are getting more and more reliable. I just wanted to have a look at the current stage of development and I therefore wrote a sample app. This is the result:

This is a quick app to test the portability of Qt on desktop, mobile and embedded.

The video shows client/server code, communicating via WebSockets (using the new WebSockets module in Qt), with a server in Qt/C++ running on Pi and with a fluid hardware accelerated interface in C++/QML running on Raspberry Pi, Mac OS X Maverick, Android Phone and Tablet, Windows 8 (Modern UI), Windows Phone 8, iOS and Linux Kubuntu. The server (on Raspberry Pi) stores information about tasks and the client provides a remote interface to manage. A client written using HTML5 canvas and WebSockets from the browser is also shown.

Unfortunately I don't own iOS and Winphone ARM devices, otherwise I'd add those as well :-) Good start anyway: this is really "write once, deploy anywhere".

Using the ports is simple: configure Qt Creator with the correct Qt build, setup the toolchain to use, qmake and build (for iOS and WinRT I preferred Xcode and Visual Studio respectively yet). Your done!
Pretty cool! Bye! ;-)

Updates on Hardware Accelerated Qt Multimedia Backend on Raspberry Pi (4.15.2014)

For those of you still working on this exciting subject, I did some new changes to PiOmxTextures and it still works pretty good. To have more information on PiOmxTextures refer to this older article. PiOmxTextures is the code behind the creation of a Qt Multimedia backend to provide hardware accelerated video (and software-decoded audio using ffmpeg) decoding and rendering from/in QML scenes on Raspberry Pi. PiOmxPlayer leverages the code from omxplayer. These are the new additions:

1. Imported code from omxplayer to revision 9b0793faf2c12f49b743ae3778ece03e31ed1538.
2. Now using ffmpeg 2.2.
3. Now it can be built using gcc 4.8 using the recent Raspberry Pi Linaro toolchain released in 2014.
4. Tested using Qt 5.3.0 beta.
5. Tested on the most recent Raspberry Pi firmware: it seems Broadcom has made some changes (fixes?) to the egl_render component. Use the new patch to Qt Multimedia and rebuild. The hack I was forced to add to compensate is no more needed.

Everything seems to keep working correctly, both in 1080p and 720p. Enjoy: https://github.com/carlonluca/pi. Bye! ;-)

A Sample to Use for Debugging and Testing Performance of the QtMultimedia Backend

In the github project for PiOmxTextures I recently uploaded also a sample player (tools/POCPlayer) that can be used for debugging and testing the performance. This is supposed to be a sample for the performance of the QtMultimedia backend and Qt itself on the Pi. It is entirely Qt/QML based, with the GUI written entirely in QML/Javascript. It shows the common controls for a media player and the service for metadata extraction. This is a demo of the current state:

How to Build

This is simple: it is a standard Qt application, with C++/Qt and QML code. A simple build of Qt version >= 5.1 should be sufficient. There is no interaction whatsoever with the plugin during build time.

Limitations

The current version can handle images and videos. Unfortunately it seems to perform pretty good in 1080p with 720p video, but 1080p videos are quite difficult to handle when other animations are running concurrently. Animations not involving video perform really good.
What you see in the video is running on the eglfs plugin: running on some more interesting layers like Wayland would require some changes in how EGL images are handled in the QtMultimedia plugin and PiOmxTextures. Unfortunately, it seems the QtWayland module is not currently perfectly working, so a hard work might be needed.

Raspberry Pi Wheezy Image With OpenMAX-based Multimedia Backend

Please take into consideration that this firmware is based on an old firmware and an old PiOmxTextures version.

As I see still someone has troubles building PiOmxTextures, I built a new image based on the Raspbian wheezy image released the 26th of July (http://www.raspberrypi.org/downloads). This version of the image also contains Qt version 5.1.2 (35fe76d9c343f98ec55c7cacd8c402e90d91ef38) and an OpenMAX-based Qt Multimedia plugin (PiOmxTextures). The available Qt modules are:

• qtbase
• qtscript
• qtxmlpatterns
• qtjsbackend
• qtdeclarative
• qtsensors
• qtquick1
• qtgraphicaleffects
• qtwebkit
• qtwayland
• qtsvg
• qtserialport
• qttolls
• qttranslations
• qtquickcontrols
• qtmultimedia
• qtimageformats

The mediaservice plugin installed is the openmaxil plugin (available here: https://github.com/carlonluca/pi). It is based on ffmpeg version 2.0 and that version of ffmpeg is statically built inside the PiOmxTextures library available in /usr/lib.
Note that the wayland module and the wayland and xkbcommon libs are available in the image, but I never tested those. Also, PiOmxTextures won't work with wayland but only using the eglfs platform plugin.

How to Test

Of course all this is experimental and if you seriously intend to use in production I suggest you start working on the sources in https://github.com/carlonluca/pi. I placed a build of the POCPlayer (sources are available in the same repository under tools/POCPlayer) in the home directory of the pi user. You can run it and see how it works:

./POCPlayer [video_file]

By pressing the "l" button, you can see the possible options. The POC player is experimental as well and it is entirely implemented in QML.
You can help improve and fix both PiOmxTextures lib and the openmaxil plugin by simply replacing libPiOmxTextures in /usr/lib and/or libopenmaxilmediaplayer.so in /usr/local/Qt-rasp-5.1.2/plugins/mediaservice.

How to Use

Refer to Raspberry Pi Wheezy Image With Qt and Wayland Support for instructions on how to use. The procedure is almost identical. Then, you can simply build your QtMultimedia-based application and test.

Where to Download

Unfortunately the download is over 1GB. You'll have to download from this ed2k link. You can download it with this excellent torrent. Thanks to the author.

Please let me know if you experience troubles downloading or with the image itself. Bye!

Hardware Accelerated QtMultimedia Backend for Raspberry Pi and OpenGL Shaders on Video

EDIT: This article is completely outdated. Refer to newer posts for properly working builds, instructions on how to build etc... In particular: "Using POT Builds" and "Build Procedure for PiOmxTextures".

In some previous posts I developed a custom QML component to render video in a QML scene using hardware accelerated decoding capabilities and rendering without passing on the ARM side. This resulted in good performance of the Raspberry Pi even with 1080p high profile h264 videos.

Many bugs need to be fixed, code should be refactored a little but still it shows it is possible and that it works good. So I decided to move the following step: modifying Qt to make it possible to use the "standard" QtMultimedia module to access the same decoding/rendering implementation. This would make it possible to better integrate with Qt and allow users to recompile without changing anything on their implementation.

The QtMultimedia module uses gstreamer on Linux to provide multimedia capabilities: gstreamer is unfortunately not hardware accelerated on the Pi unless you use something like gst-omx.

Thus, I started to look at the QtMultimedia module sources in Qt5 and found out (as I was hoping), that the Qt guys have done, as usual, a very good job in designing the concept, providing the classic plugin structure also for multimedia backends. Unfortunately, also as usual, not much documentation is provided on how to implement a new backend, but it is not that difficult anyway by looking at the other implementations.

Design

At the end, I came up with a structure like this: I implemented a new QtMultimedia backend providing the MediaPlayer and VideoOutput minimal functionalities leveraging a "library-version" of the PiOmxTextures sample code which in turn uses a "bundled" version of omxplayer implemented using the OpenMAX texture render component as a sink for the video.

As said, Qt guys have done a good job! I didn't have to change almost nothing of the Qt implementation; all the implementation is inside the plugin (apart from a minimal modification on the texture mapping, for some reason it was upside-down and inverted).

Results

The result is pretty good, I don't see many differences from the previous custom QML component (the decoding and rendering code is the same and the QML component is implemented using the same exact principle, so nothing really changed).
I'm only beginning to play a little bit with this, I just tried a couple of things. In the video you can see the "standard" qmlvideo and qmlvideofx examples provided with the Qt sources.

How to Build

Clone the repo somewhere, then use the prepare_openmaxil_backend.sh script in tools. It will compile PiOmxTextures as a shared lib and will place everything you need in the openmaxil_backend directory. Copy that directory recursively into your Qt source tree in qt_source_tree/qtmultimedia/src/plugins naming it simply openmaxil.

Some changes are needed to the Qt tree to make it compile the new backend automatically instead of the gstreamer backend, for the texture mapping and to make the "standard" qmlvideo and qmlvideofx examples work. No real modification to the code is needed: sufficient to instantiate the QQuickView those examples use with a specific class definition. This is needed. and to provide the plugin the instance of the QQuickWindow containing the media player.
These changes can be applied with a patch to the qtmultimedia tree using the patch in the tools in git. Then build the qtmultimedia module with:

path_to_qmake/qmake "CONFIG+=raspberry"

You'll find all you need here: https://github.com/carlonluca/pi.

How to Use

After you have built the plugin, you can simply use the "standard" Qt API for MediaPlayer and VideoOutput. Only restriction is that the plugin needs to access a QQuickView to access the renderer thread of the Qt Scene Graph. This might be an issue, but I've not found another solution to this yet.

What you have to do is to simply provide your application the QQuickView by using the exact class, which must be included in your application:

class RPiQuickView
{
public:
   static QQuickView* getSingleInstance();
};

Q_DECL_EXPORT QQuickView* RPiQuickView::getSingleInstance() {
   static QQuickView instance;
   return &instance;
}

This is needed because the plugin will look for the RPiQuickView::getSingleInstance() symbol, which should be found after the dynamic linker has linked to plugin the the executable. Also, you'll need to add -rdynamic to the LFLAGS of your application, so we ensure that the linker will add the symbol to the symbol table.
This is what I added to the qmlvideo and qmlvideofx examples to make those work. This is of course not elegant, but still I couldn't find a better way in reasonable time.

Of cuorse, you'll have to copy the Qt libraries that are built to your Pi, together with libPiOmxTextures.so (unless you build it statically) and the ffmpeg libraries (do not use the ffmpeg libs you have in your Pi, it is likely those won't work; use those compiled by the compile_ffmpeg.sh script in tools.

What Remains to Be Done

Most the calls are not implemented, just the minimal to get video on the screen. Also, still the audio implementation is missing (but OMX_MediaProcessor class should be ready to play audio as well) and only the QtQuick side is taken into consideration: I've never had the time to look at the widget implementation.

In case you find bugs, try to report an issue on github. If I'll find the time I'll answer.

Edit 6.25.2013

Instantiation of the QQuickView using the RPiQuickView class is no more needed from 30e24106c5dd7a5998d49d7093baef49f332b1d2. I tested this revision with Qt 5.1.1 and everything seems to keep working correctly.