Hardware Accelerated Video with Qt 6 on the Raspberry Pi

Getting hardware acceleration into Qt eglfs is tricky. Doing so on a Raspberry Pi is, unfortunately, still tricky after many years. Qt claimed to have reimplemented the Qt Multimedia module entirely, and one of their target was getting hardware acceleration where possibile. So, I thought I could start with a quick look.

Qt 5

Since Raspberry Pi was born, I had to solve the problem of hardware accelerated video in Qt. At the beginning, I wrote POT (PiOmxTextures) to solve this problem: https://github.com/carlonluca/pot. It used OpenMAX to stream decoded video into an OpenGL texture, which was then showed through a custom backend of Qt Multimedia in Qt 5. This approach worked fine, but won’t work on Pi 4/Qt 6. On the other hand, there is another component in the same repo, that includes a custom Qt Quick item to render video through omxplayer. This is the most performant approach, but has its limitations.

Qt Multimedia

I quickly tested Qt Multimedia in Qt 6 on the rpi. My build from this article should support gstreamer. All I got was a a warning on the console. I didn’t investigate further. Maybe I’ll spend more time on this in the future.

POTVL

As the classical POT is no more usable on Raspberry Pi 4, I started to have a look at POTVL, which is very simple to port to Qt 6. With a small patch, it is possible to build it. You’ll find updates on the repo.

Demo

The video is a 1080p video. As you can see, the framerate is acceptable up to a certain weight of the graphics. It seems that Qt 6 OpenGL backend still is a bit less performant than Qt 5 in this specific demo, as you can see from this test from a previous article:

so the result may even improve in the future.
The benchmark app can be found here: https://github.com/carlonluca/Fall.
Unfortunately POTVL is still not future proof, but it is the simplest and most efficient element to port to Qt 6. I may try something better in the near future.
Have fun! Bye 😉

Qt 6 on the Raspberry Pi on eglfs

Read this article on WordPress on a Pi! :-)

It has been some time since Qt 6 was released. Now that Qt 6.2 is almost out, I guess it is time to have a look at it.

Building Qt 6

Qt 6 moved from qmake to cmake. This makes a difference when trying to build it and cross-build it. Instructions on how to build are already available online from many sources, so I won’t repeat the theory here. I just want to note a few key points.

Configuration

I configured this build in the simplest form. To start, I want to use eglfs on OpenGL ES.

Toolchain

Building Qt requires a toolchain for building on the host and a toolchain to build for the target (the Raspberry Pi in this case). The toolchain from the host is typically provided by your distro (I used Manjaro in this case), while the toolchain to cross-build for the Raspberry Pi is officially provided here. In the past years I had a lot of weird problems building with those toolchains, so I started to create my own. Never had any problem anymore. Here you can find the toolchains:

• Raspberry OS Stretch: here;
• Raspberry OS Buster: here.

Please note that those toolchains are not always able to target every arch. The one for Buster is the one I used in my build, and the target arch is armv8-a.

Dependencies

Dependencies are already listed elsewhere. I would only like to note one thing here: I had problems linking because of a missing symbol: qt_resourceFeatureZstd. According to the sources, it seems that resources are compiled by the rcc tool. My guess is that the host tools use zstd by default to compress when available, but this is a problem when zstd is available on the host and not available on the target. Therefore, I suspect you have to decide if you want zstd support or not before building Qt 6 for the host, and be coherent with the build for the target. I decided to add zstd.

Build Errors

I had a few build errors building Qt 6.2 beta3. Nothing particularly relevant, but I had to apply a few changes. I did not keep track of those changes, they were all trivial.

Build Qt Test App

Once the Qt builds are ready, it is time to run. For the moment, I only want to do a quick test of QML. In my build, I forgot to set eglfs as the default platform, but it can be set it in the environment. I wrote a simple app to test Qt Quick here. The animations are very simple, but I wrote it like this to see how well many items are handled and how well the uniform movement on the y axis is rendered.

NOTE: Unlike Qt5, Qt6 includes host tools and tools built for the target. This means you can build the test app on the target directly or cross-build it from your host. For the package below anyway, you'd need to have a system compatible with those binaries.

First App

This video shows a comparison of this Qt 6 build against Qt 5. It also shows the same benchmark app on Intel UHD and nVidia.

Download

If you want to test the build of Qt 6 on your rpi, you can download it from here:

Download Qt 6.2.0-rc1 for Raspberry OS Buster armv8-a

The package does not include the toolchain, that you can download from the other links, but includes the host build. I don’t think it will be of help, as it is built for my Manjaro context, but I placed it there anyway.

The positions of the directories are relevant:

• /opt/rpi/rpi-8.6.3: toolchain;
• /opt/Qt-x64-6.2.0-beta3: host build;
• /usr/local/Qt-rasp-6.2.0-beta3: position in the target;
• /opt/rpi/sysroot: sysroot for cross-building.

Runtime Context

The is the 3D environment as seen by Qt 6:

qt.rhi.general: Created OpenGL context QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize 24, redBufferSize 8, greenBufferSize 8, blueBufferSize 8, alphaBufferSize 0, stencilBufferSize 8, samples 0, swapBehavior QSurfaceFormat::DefaultSwapBehavior, swapInterval 1, colorSpace QColorSpace(), profile  QSurfaceFormat::NoProfile)
qt.rhi.general: OpenGL VENDOR: Broadcom RENDERER: VC4 V3D 2.1 VERSION: OpenGL ES 2.0 Mesa 19.3.2
qt.scenegraph.general: MSAA sample count for the swapchain is 1. Alpha channel requested = no.
qt.scenegraph.general: rhi texture atlas dimensions: 2048x2048

Bye 😉

Docker Image for Qt Development and Continuous Integration on x64 and arm64

Intro

It happens from time to time that I would benefit from having a simple Docker image with all the deps needed to build and run Qt apps on specific Qt versions. For instance, I may need to build some binary on Mac OS for Linux or I may need to have an image for some continuous integration system.

Problem

I had a look around and there are quite a few projects: some do not provide an arm build, some rely on the Qt official installer, that is a bit tricky to automate and tends to change often, some rely on distro-provided packages, that do not provide every version etc… I’d prefer to have a solid build procedure to build images for any Qt version I want. Also, my CI system runs on arm64, so I’d need a multiarch image, and therefore two different Qt builds. I thought it could be simpler to just build Qt on-the-fly in a Dockerfile but… turned out it is not exactly simple.

Solution

At first I created a Dockerfile to:

1. install all the needed deps on Ubuntu focal;
2. download Qt sources;
3. configure;
4. build;
5. install;
6. cleanup.

This proved to be a simple solution, but building Qt for arm on qemu or building on arm took way too much. So I though of a different solution. I created two images:

• carlonluca/qt-builder is an image with the proper tools to build Qt for x64 and cross-build Qt for arm64;
• carlonluca/qt-dev is an image with the proper tools to build and run Qt apps.

With the first image Qt can be built and cross-built in a few hours for both x64 and arm64, directly on x64. The builds are then injected into a second multiarch image. This is way faster than building on qemu.

Only one problem left: in Qt 5, crossbuilding Qt results in some Qt tools like qmake to be built for the host arch, which is unacceptable in this case. So, in the second image, a build of the only qtbase module is also needed to build arm64 binaries. Qt 6 works differently instead.

Result

The code to build Qt and all the images are available at: https://github.com/carlonluca/docker-qt.

The image to build Qt is available at: https://hub.docker.com/repository/docker/carlonluca/qt-builder.

The image to use for app development is available at: https://hub.docker.com/repository/docker/carlonluca/qt-dev.

Continuous Integration on GitLab with Qt

An example of usage of the dev image is running continuous integration and unit tests on your code on Jenkins or GitLab. For example, I can now run my unit tests in my GitLab instance running on Raspberry Pi 4 with the following code (taken from my project https://github.com/carlonluca/lqobjectserializer):

variables:

GIT_SUBMODULE_STRATEGY: recursive

stages:
  - test_qt5
  - test_qt6

Test_qt5:
  stage: test_qt5
  image:
    name: "carlonluca/qt-dev:5.15.2"
    entrypoint: [""]
  script:
    - cd LQObjectSerializerTest
    - mkdir build
    - cd build
    - cmake ../qt5
    - make
    - ./LQObjectSerializerTest
    - ./LGithubTestCase

Test_qt6:
  stage: test_qt6
  image:
    name: "carlonluca/qt-dev:6.1.2"
    entrypoint: [""]
  script:
    - cd LQObjectSerializerTest
    - mkdir build
    - cd build
    - cmake ../qt6
    - make
    - ./LQObjectSerializerTest
    - ./LGithubTestCase

These images are still experimental though.

Have fun 😉

Bye!

Serialize/Deserialize JSON Representations to Qt Objects (QObject's) in C++

Calls to HTTP servers are very frequent nowadays, and JSON is frequently used when dealing with structured data. When I write applications in Java/Kotlin, I always use retrofit2 with gson integration, or similar approaches. It is very comfortable to be able to serialize/deserialize to Java objects automatically. I tried to find something similar for Qt, but I couldn't find much. JSON support in Qt is good, but does not seem to be able to map QOject's to QJSonObject. I also could not find other C++ libraries doing this. Probably the difficulty of implementing reflection in C++ would make the implementation a bit convoluted. A good candidate I found is this: https://github.com/Loki-Astari/ThorsSerializer, which seems not to require much boilerplate code, not too verbose and no generated code.

However, Qt includes the meta object system (https://doc.qt.io/qt-5/metaobjects.html), and therefore I tried to use it to get the desired result. In a couple of hours I got to a pretty decent result: https://github.com/carlonluca/lqobjectserializer. It is far from perfect, but it seems to work. Have a look at the readme or the unit tests to know more.

The draft is free to use: you can contribute, report bugs etc...

For instance, a JSON object like this:

{"menu": {

    "header": "SVG Viewer",

    "items": [

        {"id": "Open"},

        {"id": "OpenNew", "label": "Open New"},

        null,

        {"id": "ZoomIn", "label": "Zoom In"},

        {"id": "ZoomOut", "label": "Zoom Out"},

        {"id": "OriginalView", "label": "Original View"},

        null,

        {"id": "Quality"},

        {"id": "Pause"},

        {"id": "Mute"},

        null,

        {"id": "Find", "label": "Find..."},

        {"id": "FindAgain", "label": "Find Again"},

        {"id": "Copy"},

        {"id": "CopyAgain", "label": "Copy Again"},

        {"id": "CopySVG", "label": "Copy SVG"},

        {"id": "ViewSVG", "label": "View SVG"},

        {"id": "ViewSource", "label": "View Source"},

        {"id": "SaveAs", "label": "Save As"},

        null,

        {"id": "Help"},

        {"id": "About", "label": "About Adobe CVG Viewer..."}

    ]

}}

can be deserialized to a QObject simply by defining the classes (macros here are inherited by my other project https://github.com/carlonluca/lqtutils, but this is not mandatory):

L_BEGIN_CLASS(Item)

L_RW_PROP(QString, id, setId, QString())

L_RW_PROP(QString, label, setLabel, QString())

L_END_CLASS

L_BEGIN_CLASS(Menu)

L_RW_PROP(QString, header, setHeader)

L_RW_PROP_ARRAY_WITH_ADDER(Item*, items, setItems)

L_END_CLASS

L_BEGIN_CLASS(MenuRoot)

L_RW_PROP(Menu*, menu, setMenu, nullptr)

L_END_CLASS

and by writing these few lines:

LDeserializer<MenuRoot> deserializer(factory);

QScopedPointer<MenuRoot> g(deserializer.deserialize(jsonString));

Please leave a comment if you know of other tools in this context, serializing and deserializing JSON is very frequent.

Bye! ;-)

Synthesize Qt Settings

I frequently create classes that I use as an interface to settings in my apps. Using a single class with accessors makes the code readable, safer and simple to maintain. Unfortunately, doing this requires to create getters and setters for every entry in the settings file, which is a bit bothering. Also, QSettings is not a QObject and cannot provide signals to notify changes.

My use cases are typically very simple, so I created a couple of macros that synthesise classes for me. Macros synthesise a reentrant class that can be used to access settings and a notifier, that can be used to get notifications of the changes. An example of definition of the class is:

L_DECLARE_SETTINGS(LSettingsTest, new QSettings("settings.ini", QSettings::IniFormat))

L_DEFINE_VALUE(QString, string1, QString("string1"), toString)

L_DEFINE_VALUE(QSize, size, QSize(100, 100), toSize)

L_DEFINE_VALUE(double, temperature, -1, toDouble)

L_DEFINE_VALUE(QByteArray, image, QByteArray(), toByteArray)

L_END_CLASS

L_DECLARE_SETTINGS(LSettingsTestSec1, new QSettings("settings.ini", QSettings::IniFormat), "SECTION_1")

L_DEFINE_VALUE(QString, string2, QString("string2"), toString)

L_END_CLASS

this will let you instantiate objects of class LSettingsTest and LSettingsTestSec1, and access entries with strong typed methods. Also, by calling LSettingsTest::notifier(), you can get a reference to the unique notifier. By setting an instance as a context property, you can get notifications and you can update settings from QML. You can find some more info in the repo https://github.com/carlonluca/lqtutils and an example using Qt Quick here: https://github.com/carlonluca/lqtutils/tree/master/LQtUtilsQuick.

Bye! ;-)

Synthesize Qt properties

The are some utils that I use in most of my Qt projects. I'm a bit fed up of copy-pasting those every time I need something, so I created a project containing tools that are of frequent use when I write Qt apps, regardless of the type of app.

The only interesting tool I added yet is the synthesizer of Q_PROPERTY's. You can use the macros:

L_RW_PROP
L_RO_PROP

to synthétise props, with getter, setter and notifications. Each macro is overloaded: with 3 params, you do not have initialisation, adding a 4th param, also initialises the variable to the provided value. Also, I frequently need QObject that are just containers of properties, to be used in QML. I added these two macros to spare some lines:

L_BEGIN_CLASS(<class_name>)
L_END_CLASS

A class like:

class Fraction : public QObject
{
    Q_OBJECT
    Q_PROPERTY(double numerator READ numerator WRITE setNumerator NOTIFY numeratorChanged)
    Q_PROPERTY(double denominator READ denominator WRITE setDenominator NOTIFY denominatorChanged)
public:
    Fraction(QObject* parent = nullptr) : QObject(parent) {}
    double numerator() const {
        return m_numerator;
    }
    double denominator() const {
        return m_denominator;
    }
public slots:
    void setNumerator(double numerator) {
        if (m_numerator == numerator)
            return;
        m_numerator = numerator;
        emit numeratorChanged(numerator);
    }
    void setDenominator(double denominator) {
        if (m_denominator == denominator)
            return;
        m_denominator = denominator;
        emit denominatorChanged(denominator);
    }
signals:
    void numeratorChanged(double numerator);
    void denominatorChanged(double denominator);
private:
    double m_numerator;
    double m_denominator;
};

can be simplified to:

L_BEGIN_CLASS(Fraction)
L_RW_PROP(double, numerator, setNumerator)
L_RW_PROP(double, denominator, setDenominator)
L_END_CLASS

I'm sure there are other similar approaches out there, but if you need it, you can simply add this repo as a submodule like I do: https://github.com/carlonluca/lqtutils.

Bye! ;-)

Hardware Decoding in Chromium through QtWebEngine on Raspberry Pi

A few months back I decided to try to implement hardware decoding in WebKit. Unfortunately this task is always pretty long and complex for many reasons. I found the time to draft an implementation for WebKit1, which is pretty useless as WebKit1 in Qt is only used outside QML and JS is executed in the main thread. Unfortunately I never found the time to implement this in WebKit2, which runs in QML and is suitable for more fluid UIs. This was the result:

This is how YouTube was running with this implementation:

Now Qt has deprecated QtWebKit and is working heavily on QtWebEngine, which is built on Chromium, so I wanted to try this road. Unfortunately these kind of things always claim a lot of time, and I don't typically have that much, but I was able to start and get something done already.

Writing a complete solution in Chromium to decode and render video takes much time, so I thought of a shortcut: creating a custom VDA (Video Decode Accelerator) that loads the POT library and reusing its entire codebase to implement decode and rendering with little modifications. This proved to be possible and now I get something on the screen.

So, to summarise: a little patch to Chromium is needed to create a VDA that dynamically loads POT library into memory and uses it with a common interface. Data and calls are translated to POT structures and are sent to POT, which then processes the buffers properly. The result of the decode operation is then sent back to the VDA through the same interface and textures are then sent to Chromium for rendering.

Still many problems remain open, there is much to be done yet as you can see from the video, but something is drawn. Have a look at the demo:

Have fun! ;-)

Binaries for POT 5.5.0-beta1 on Qt 5.8.0-rc1 and Experimental Accelerated QtWebKit

A new version of Qt is out and some interesting changes are being developed. Interesting work was done on WebEngine. This package contains an experimental build of Qt 5.8.0-rc1 with the POT driver to provide hardware acceleration to video decoding and rendering. POT was updated to build on the new plugin architecture of Qt and includes ffmpeg 3.2.2. For info on the content of the package refer to this article. The youtube player still runs on WebKit 1 and so is still hardly useful.

Please report issues on github. Have fun ;-)

Download POT 5.5.0-beta1 for Raspbian Jessie Lite Pi2 (also tested on Pi3) here (md5: 7ca1b961ab8c70176f6aeb13d0bc4f9a).

Binaries for POT 5.4.0 on Qt 5.7.0 and Experimental Accelerated QtWebKit

So far, no real complains about POT 5.4.0-beta1 so I suppose it is sufficiently safe to consider it POT 5.4.0. Refer to this article for more info.

Binaries for POT 5.4.0-beta1 on Qt 5.7.0 and Experimental Accelerated QtWebKit

This is a new binary package including the same content of POT 5.3.0-beta1 package, with some updates:

• new improvements in POT taken from omxplayer code;
• bump to ffmpeg 3.1;
• based on Qt 5.7.0.

The build includes support for network protocols and WebKit 1 supporting video html5 elements (including youtube).

This version is still experimental. Please report bugs on github.

Have fun! Bye! ;-)

Download POT 5.4.0-beta1 for Raspbian Jessie Lite Pi2 (also tested on Pi3) here (md5: e6b6a1f92b71ac2bdd3d9756efd7669c).

Streaming from VLC to a Qt App on HTTP Using POT

The latest version of POT (5.3.0-beta1) implements some kind of buffering which makes it also possible to do some streaming. ffmpeg provides what is needed to work on protocols like HTTP. This video is a little demo showing how simple it is to stream from VLC on HTTP, remuxing to mpeg-ts, and decoding/rendering on Raspberry Pi in an accelerated Qt application using POT.

Such feature can be useful when dealing with streams coming from many sources, like an IP camera for instance.
Have fun! Bye ;-)

Binaries for POT 5.3.0-beta1 on Qt 5.6.0 and Experimental Accelerated QtWebKit

This is an experimental version of POT that includes a few new features. The driver now includes an experimental version of playback of remote content including proper buffering of the media and the QtWebKit library is able to play videos using proper hardware acceleration.

Features

The driver can now be used through the regular Qt/QML interface to play content over protocols different from file://. I only tested HTTP but others should be available.
This new feature allowed me to also implement a new player in QtWebKit using the POT driver to stream content to the regular Qt QWebView. Unfortunately I could only find the time to implement this in the WebKit 1 branch, not in WebKit 2 which requires more work. This makes the implementation hardly useful, but it is a start. If anyone wanted to contribute this is where to start. This video shows the new implementation in action:

As QtWebKit now uses the GPU properly, I could also implement a YouTube player app using the same QWebView with just a few lines of code of regular Qt code:

In the video you can also see an application I wrote to implement a YouTube fullscreen player using the YouTube iframe API. Such a sample app is also included in the package.

Of course the real target would be to implement this in QtWebEngine. I was never able to find a version of QtWebEngine with video acceleration for Pi. Anyone who knows one?

Package

The package now includes:

• build_valgrind.tar: a build of valgrind to analyse your code;
• libpiomxtextures_qmlutils.so: QML plugin to provide a video probe in QML;
• piomxtextures_browser_we: an application that tests the QtWebEngine module (run passing a URL, ./piomxtextures_browser_we http://www.youtube.com);
• piomxtextures_browser_wk: an application that tests the QtWebKit module:
• you can run on WebKit 1 like ./piomxtextures_browser_wk http://www.youtube.com;
• or you can run on WebKit 2 like ./piomxtextures_browser_wk --wk2 http://www.youtube.com;
• piomxtextures_pocplayer: a sample QML player;
• piomxtextures_pocplayer_yt: a youtube sample player (run passing a video ID like ./piomxtextures_pocplayer_yt 71UvXMzVgx4);
• qtdeps.tar: the usual libs needed by Qt and POT;
• Qt-rasp2-5.6.0.tar: a build of Qt 5.6.0 stable including:
• regular Qt 5.6.0 modules;
• untested bluetooth module including BLE support;
• untested MySQL plugin;
• untested QtFtp module;
• untested Qt hat tools module;
• QtWebEngine;
• QtWebKit.

Please report any bug you find on github. Don't use in production. Sources will be available shortly. Have fun! ;-)

Download POT 5.3.0-beta1 for Raspbian Jessie Lite Pi2 (also tested on Pi3) here (md5: edbe0ad2a552a5c8280e3876d16b237d).

Binaries for Hardware Accelerated Qt 5.6.0-rc1 Multimedia Backend on Raspbian Jessie Lite (POT 5.2.1)

This the same build provided in the firmware from the previous post. It includes a couple of fixes over 5.2.0 and the new Qt 5.6.0-rc1. Refer to this article for more information.
Please file issues on github if needed.
Bye! ;-)

Download PiOmxTextures 5.2.1 for Raspbian Jessie Pi2 here (extraction code is: 62c9, md5: 356e2004dfeb9017a482e2448750121e).

Fully Optimised Raspbian Jessie Lite with Accelerated Qt Multimedia Backend (Pi2)

This is a follow-up to the image provided in this article. The concept is identical, but in this case this firmware is smaller, it is based on Raspbian Jessie Lite and includes updated software. The firmware was 1.9GB while now is 759MB. This is the current setup (but I encourage to update to the latest possible firmware):

pi@raspberrypi:~ $ vcgencmd version
Feb 25 2016 14:25:47 
Copyright (c) 2012 Broadcom
version dea971b793dd6cf89133ede5a8362eb77e4f4ade (clean) (release)
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.1.18-v7+ #846 SMP Thu Feb 25 14:22:53 GMT 2016 armv7l GNU/Linux

Download the image here (torrent) (md5: 06d75d03f63674350be6cb807850bf09).
(Please be patient while downloading and decompressing)

The image includes:

• Qt 5.6.0-rc1: built for armv7 with neon optimisations, including support for touch screens, gstreamer, libinput, tslib, and Bluetooth/Bluetooth BLE using bluez (complete configuration below).
• QtWebKit 5.6.0: includes WebKit 1 and WebKit 2.
• QtWebEngine 5.6.0-rc1: multimedia only based on ffmpeg.
• Dependant libs: evdev, icu (not the one provided by raspbian), mtdev, libts.
• Samples: samples from the repo to test POT.
• POT: POT version 5.2.1 including ffmpeg 2.8.6.

Samples

To have a quick look at multimedia in Qt/QML test with this commands:

• /usr/local/Qt-rasp2-5.6.0-rc1/bin/qmlscene "some_video_file" samples/video_simple.qml
• /usr/local/Qt-rasp2-5.6.0-rc1/bin/qmlscene "some_url" samples/webkit_simple.qml
• /usr/local/Qt-rasp2-5.6.0-rc1/bin/qmlscene "some_url" samples/webengine_simple.qml
• /home/pi/piomxtextures_pocplayer --multipleanimtest "720p_video_1" "720p_video_2"

Please let me know if you find something wrong and file a bugreport on https://github.com/carlonluca/pi.

Have fun! ;-)

Binaries for POT 5.1.0 With Remote Streaming Support

This is an experimental build of POT 5.1.0 including support for remote streaming. I tested it for HTTP/HTTPS with YouTube but it should work with other protocols supported by ffmpeg as well.

In case someone is interested in testing it please let me know. This is a build for Pi2 on Qt 5.6.0-beta1. To quickly test you can use POCPlayer:

./piomxtextures_pocplayer "https://something.com/..."

The streaming should start right away.

Download here the build for Pi2 on Qt 5.6.0-beta1 (extraction code is: bfc7).

Have fun! Bye ;-)

Binaries for Hardware Accelerated Qt 5.6.0-beta1 Multimedia Backend on Raspbian Jessie (POT 5.2.0)

This new build is identical to the previous one but includes a few changes in the library:

• Bump to ffmpeg from 2.7.2 to 2.8.5 to fix some rare crashes.
• Multiple fixes that were causing bug #51.
• Other minor fixes.

Download PiOmxTextures 5.2.0 for Raspbian Jessie Pi2 here (extraction code is: cd4f).

Please file bug reports in case of issues and have fun! Bye ;-)

NOTE: If you need to test the samples please keep in mind that Qt 5.6.0 seems to have changed the order of the params to provide to qmlscene: the qml file must appear at the end.

Binaries for Hardware Accelerated Qt 5.6.0-beta1 Multimedia Backend on Raspbian Jessie (POT 5.1.0)

This is a experimental build of POT 5.1.0 running on Qt 5.6.0-beta1 built for Raspbian Jessie for armv7 (Raspberry Pi 2). Do not use in production. The package includes:

• Qt 5.6.0-beta1 optimised for armv7.
• QtConnectivity including BLE support.
• QtWebKit including both WebKit 1 and WebKit 2. Video decoding is implemented in WebKit with gstreamer (won't even do 720p properly).
• QtWebEngine with unaccelerated video decoding using ffmpeg.
• QtFtp.
• valgrind-3.12.0.SVN: using it on PiOmxTextures requires to disable libarmmem.so in /etc/ld.so.preload. Also the default valgrind in Raspbian was not working for me.
• POT 5.1.0 for accelerated playback in Qt.

No accelerated video playback is implemented in QtWebKit nor QtWebEngine in this build. CSS animations instead are properly accelerated when OpenGL rendering is used.

Download PiOmxTextures 5.1.0 for Raspbian Jessie Pi2 here (extraction code is: 508d).

Hands-on WebKit and HTML5 Media Elements on Raspberry Pi

I tried to do this a few months ago actually, but I got lost in the huge amount of code and architecture variants of WebKit. Now I convinced myself it would be too interesting to see it working. I don't have much spare time, so I wanted to start with the path of least-resistance and then, depending on the time available, continue climbing. This activity is very time consuming, so I do not know where I'll get.

This is however the first stage: using POT to implement 1080p hardware accelerated zero-copy video playback in the Qt port of WebKit1. No QML here. Of no use, but it somehow works. I read there are far better results out there, but I learned much by doing this, and I had fun :-)
Of course, as you can see, there is much much work being done in the UI thread. JavaScript is mostly run in the main thread, making the framerate pretty bad on heavy websites. That is why WebKit2 is there after all :-)

Demo

Bye! ;-)

Bluetooth (BLE) Support with Qt on Raspberry Pi

There are many ways to access Bluetooth capabilities on Raspberry Pi. Qt has its own API to use Bluetooth and to implement a BLE central device. The last build I provided a few days ago didn't support Bluetooth: this package should support it. I tested the low energy scanner example and it worked properly.
Have fun! Bye ;-)

Download POT 5.1.0 with Qt 5.5.1 for Pi2 Raspbian Jessie here (extraction code: 96aa).

Fully Optimised Raspbian Jessie with Accelerated Qt Multimedia Backend (Pi2)

This is an image of Raspbian Jessie ready to test (and use) the Qt Multimedia Backend POT. This image is optimised for maximum performance (see the demos below). This is the current setup:

pi@raspberrypi ~ $ sudo /opt/vc/bin/vcgencmd version
Nov 11 2015 21:31:07 
Copyright (c) 2012 Broadcom
version 54011a8ad59a9ae1c40bd07cddd9bcf90e779b66 (clean) (release)
pi@raspberrypi ~ $ uname -a
Linux raspberrypi 4.1.13-v7+ #826 SMP PREEMPT Fri Nov 13 20:19:03 GMT 2015 armv7l GNU/Linux
pi@raspberrypi ~ $ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 8.0 (jessie)
Release:        8.0
Codename:       jessie

Download image here.

The image includes:

• Qt 5.5.1 built for armv7 with neon optimisations, including support for touch screens, gstreamer, libinput, X11 (no multimedia), tslib, and Bluetooth/Bluetooth BLE using bluez (complete configuration below). Modules provided include QtWebKit (no multimedia support).
• Dependant libs: evdev, icu (not the one provided by raspbian), mtdev, libts.
• POCPlayer and QML samples to show the performance of the backend and test bugs.
• POT library including ffmpeg 2.7.2.

Everything is built for armv7 including neon optimisations. To quickly test the performance there are a few things you can launch:

/home/pi/piomxtextures_pocplayer --multipleanimtest /home/pi/sintel_trailer_720p.mp4 /home/pi/big_buck_bunny_720p_h264.mov

This is the result you should expect with a proper overclocking:

It will show you a continuous animation while running two 720p videos concurrently. To improve the frame rate of the animations you can also overclock your Pi2 (see below).

Other samples to test the features of the plugin are stored in /home/pi/samples:

/usr/local/Qt-rasp2-5.5.1/bin/qmlscene /home/pi/samples/video_simple.qml file:///home/pi/big_buck_bunny_1080p_h264.mov

Another interesting test is running qmlvideofx example included in the Qt sources. This will show you the perfomance of the effects applied on the video using shaders. You can find an executable in /usr/local/Qt-rasp2-5.5.1/examples/multimedia/video/qmlvideofx/qmlvideofx. This shows approximately what you should expect (the video shows the result on Pi1, Pi2 is just a little better):

Overclocking

The Pi can be overclocked by tuning parameters in /boot/config.txt. I provided a configuration that I find useful:

#force_turbo=1  # Voids Warranty!

#boot_delay=1   # Helps to avoid sdcard corruption when force_turbo is enabled.

#arm_freq=1100

#sdram_freq=450

#core_freq=550

#over_voltage=4

#temp_limit=80  # Will throttle to default clock speed if hit.

In the image this is all disabled. If you intent to enable please keep in mind I don't take responsibility for any consequence. Please refer to the documentation for this procedure.

Have fun! Bye ;-)

Qt Configuration

   Configure summary

Building on:   linux-g++ (x86_64, CPU features: mmx sse sse2)

Building for:  devices/linux-rasp-pi2-g++ (arm, CPU features: neon)

Platform notes:

            - Also available for Linux: linux-clang linux-kcc linux-icc linux-cxx

        

qmake vars .......... styles += mac fusion windows QT_CFLAGS_GLIB = -pthread -I/opt/rpi/sysroot/usr/include/glib-2.0 -I/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf/glib-2.0/include  QT_LIBS_GLIB = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lgthread-2.0 -pthread -lglib-2.0  QT_CFLAGS_PULSEAUDIO = -D_REENTRANT -I/opt/rpi/sysroot/usr/include/glib-2.0 -I/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf/glib-2.0/include  QT_LIBS_PULSEAUDIO = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lpulse-mainloop-glib -lpulse -lglib-2.0  QMAKE_CFLAGS_FONTCONFIG = -I/opt/rpi/sysroot/usr/include/freetype2  QMAKE_LIBS_FONTCONFIG = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lfontconfig -lfreetype  QMAKE_INCDIR_LIBUDEV =  QMAKE_LIBS_LIBUDEV = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -ludev  QMAKE_LIBINPUT_VERSION_MAJOR = 1 QMAKE_LIBINPUT_VERSION_MINOR = 0 QMAKE_INCDIR_LIBINPUT = /opt/rpi/sysroot/home/pi/qtdeps/include  QMAKE_LIBS_LIBINPUT = -L/opt/rpi/sysroot/home/pi/qtdeps/lib -linput  QMAKE_LIBXI_VERSION_MAJOR = 1 QMAKE_LIBXI_VERSION_MINOR = 7 QMAKE_LIBXI_VERSION_PATCH = 4 QMAKE_X11_PREFIX = /usr QMAKE_XKB_CONFIG_ROOT = /usr/share/X11/xkb QMAKE_CFLAGS_XCB =  QMAKE_LIBS_XCB = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lxcb-sync -lxcb-xfixes -lxcb-render -lxcb-randr -lxcb-image -lxcb-shm -lxcb-keysyms -lxcb-icccm -lxcb-shape -lxcb  INCLUDEPATH +=  "/opt/rpi/sysroot/home/pi/qtdeps/include" LIBS +=  -L"/opt/rpi/sysroot/home/pi/qtdeps/lib" sql-drivers =  sql-plugins =  sqlite qmake switches ......... 

Build options:

  Configuration .......... accessibility accessibility-atspi-bridge alsa audio-backend c++11 clock-gettime clock-monotonic compile_examples concurrent cross_compile cups dbus egl eglfs eglfs_brcm enable_new_dtags evdev eventfd fontconfig full-config getaddrinfo getifaddrs glib gstreamer-1.0 harfbuzz iconv icu inotify ipv6ifname large-config largefile libinput libproxy libudev linuxfb medium-config minimal-config mremap mtdev neon nis opengl opengles2 openssl pcre png posix_fallocate pulseaudio qpa qpa reduce_exports release rpath shared small-config system-freetype system-jpeg system-png system-zlib tslib use_gold_linker xcb xcb-glx xcb-plugin xcb-render xcb-xlib xinput2 xkbcommon-qt xlib xrender 

  Build parts ............  libs examples

  Mode ................... release

  Using sanitizer(s)...... none

  Using C++11 ............ yes

  Using gold linker....... yes

  Using new DTAGS ........ yes

  Using PCH .............. no

  Target compiler supports:

    Neon ................. yes

Qt modules and options:

  Qt D-Bus ............... yes (loading dbus-1 at runtime)

  Qt Concurrent .......... yes

  Qt GUI ................. yes

  Qt Widgets ............. yes

  Large File ............. yes

  QML debugging .......... yes

  Use system proxies ..... no

Support enabled for:

  Accessibility .......... yes

  ALSA ................... yes

  CUPS ................... yes

  Evdev .................. yes

  FontConfig ............. yes

  FreeType ............... yes (system library)

  Glib ................... yes

  GStreamer .............. yes (1.0)

  GTK theme .............. no

  HarfBuzz ............... yes (bundled copy)

  Iconv .................. yes

  ICU .................... yes

  Image formats: 

    GIF .................. yes (plugin, using bundled copy)

    JPEG ................. yes (plugin, using system library)

    PNG .................. yes (in QtGui, using system library)

  journald ............... no

  libinput................ yes

  mtdev .................. yes (system library)

  Networking: 

    getaddrinfo .......... yes

    getifaddrs ........... yes

    IPv6 ifname .......... yes

    libproxy.............. yes

    OpenSSL .............. yes (loading libraries at run-time)

  NIS .................... yes

  OpenGL / OpenVG: 

    EGL .................. yes

    OpenGL ............... yes (OpenGL ES 2.0+)

    OpenVG ............... no

  PCRE ................... yes (bundled copy)

  pkg-config ............. yes 

  PulseAudio ............. yes

  QPA backends: 

    DirectFB ............. no

    EGLFS ................ yes

      EGLFS i.MX6....... . no

      EGLFS KMS .......... no

      EGLFS Mali ......... no

      EGLFS Raspberry Pi . yes

      EGLFS X11 .......... no

    LinuxFB .............. yes

    XCB .................. yes (system library)

      EGL on X ........... no

      GLX ................ yes

      MIT-SHM ............ yes

      Xcb-Xlib ........... yes

      Xcursor ............ yes (loaded at runtime)

      Xfixes ............. yes (loaded at runtime)

      Xi ................. no

      Xi2 ................ yes

      Xinerama ........... yes (loaded at runtime)

      Xrandr ............. yes (loaded at runtime)

      Xrender ............ yes

      XKB ................ yes

      XShape ............. yes

      XSync .............. yes

      XVideo ............. yes

  Session management ..... yes

  SQL drivers: 

    DB2 .................. no

    InterBase ............ no

    MySQL ................ no

    OCI .................. no

    ODBC ................. no

    PostgreSQL ........... no

    SQLite 2 ............. no

    SQLite ............... yes (plugin, using bundled copy)

    TDS .................. no

  tslib .................. yes

  udev ................... yes

  xkbcommon-x11........... yes (bundled copy, XKB config root: /usr/share/X11/xkb)

  xkbcommon-evdev......... no

  zlib ................... yes (system library)