Binaries for Hardware Accelerated Qt Multimedia Backend on Raspberry Pi

As someone still have troubles building PiOmxTextures, I prepared a little package that contains the binaries needed to make it run and have a look. In the past I tried to provide an entire firmware, but it was large to upload and not very comfortable to update. This package only contains:

• PiOmxTextures.tar: library to provide the playback capabilities, including ffmpeg 2.5.
• Qt-rasp-5.4.0.tar: the entire tree of Qt 5.4.0 binaries built for Raspberry Pi (firmware ba43047, rpath is set to /usr/local/Qt-rasp-5.4.0).
• Qt-rasp-5.4.0_host.tar: the utility to be used on the host when cross-building (built for x86 Kubuntu 13.10).

Current version of the package is 4.3.0: download.

To have a quick look, just place Qt-rasp-5.4.0 in /usr/local into your Pi, then place PiOmxTextures somewhere and add it to the runtime library path. Now you can test running qmlvideo or qmlvideofx from the Qt Multimedia examples. Something similar to this should come up: 

Bye! ;-)

Updates on Hardware Accelerated Qt Multimedia Backend on Raspberry Pi (23.12.2014)

As I see some are still working on the Pi, and someone asked to "port" to Qt 5.4.0, this is a small update to the Qt Multimedia Backend POC.

This is what I updated so far:

• Merged code from omxplayer until a4ee074.
• Updated to work with ffmpeg 2.5.
• Updated to work on the latest Pi firmware ba43047.
• Updated some build scripts to speed up build.
• Updated to work with the new Qt 5.4.0.

The build procedure is mostly unchanged. I summarise it here with the new updates (I assume you're in the root of the repo sources):

export RPI_SYSROOT=
export COMPILER_PATH=
cd tools
./compile_ffmpeg.sh
export QMAKE_PATH=
./prepare_openmaxil_backend.sh


This should build ffmpeg, build the PiOmxTextures lib (needed to get the hardware interface to video and audio playback) and prepare the sources of the Qt multimedia backend.
NOTE: I had to modify the libssh pkgconfig file ($RPI_SYSROOT/usr/lib/arm-linux-gnueabihf/pkgconfig/libssh.pc) to:


prefix=/usr
exec_prefix=${prefix}
libdir=${prefix}/lib/arm-linux-gnueabihf
includedir=${prefix}/include

Name: libssh
Description: The SSH Library
Version: 0.5.4
Libs: -L{libdir} -lssh
Cflags: -I{includedir}

Now you have to copy the directory of the plugin to the src/plugins subdirectory of your Qt source tree. Once finished, this plugin will be built.
Unfortunately, still a patch to QtMultimedia is needed, so go into the qtmultimedia dir of your Qt source tree (I tested on the Qt 5.4.0 tree) and patch using the patch provided in tools:

cp -a /openmaxil_backend /qtmultimedia/src/plugins/openmaxil
patch -p1 < /tools/qt_multimedia_patch_5.4.0.patch
$QMAKE_PATH CONFIG+=raspberry
make -j
make install


Now you should have the new openmaxil plugin in your Qt prefix. When using, remember that libPiOmxTextures must be in the runtime library path when using the plugin. If an error like this is returned when running an application using the plugin:

luca@raspberrypi ~ $ ./POCPlayer
* failed to open vchiq instance

then set the permissions appropriately:

sudo chmod a+rw /dev/vchiq

Or better add your user to the video group. Same stands for input devices:

sudo usermod -a -G video
sudo usermod -a -G input

As usual remember this is just a POC.
If anything went wrong, file an issue on github. Someone may have a look.
Bye! ;-)

NOTE: It seems that 1080p video on 1080p output really stresses the GPU at the point that the video starts to have issues and blocks. I tried to test Pi turbo mode, that should increase GPU performance but my Pi seems not to agree on the overvolt parameter. USB ports completely crash and then the system goes bye-bye. If anyone was able to test somehow animations and 1080p playout on 1080p resolution with turbo mode and like 450MHz GPU, please let me know!

Having Fun with OpenGL, Android and Hardware Decoding

OpenGL is a good technology, and playing around with it on Android can be real fun! :-) By streaming a video to texture it is possible to create interesting effects, like it is possible to do with Qt. With a simple shader, for instance, lightning effects can be added:

#extension GL_OES_EGL_image_external : require

precision mediump float;

uniform samplerExternalOES u_Texture;

uniform vec3 u_LightPos;      // The position of the light in eye space.

varying vec2 v_TexCoordinate; // Interpolated texture coordinate per fragment.

varying vec3 v_Position;      // Interpolated position for this fragment.

varying vec4 v_Color; 

varying vec3 v_Normal;        // Interpolated normal.

void main()

{

   float distance = length(u_LightPos - v_Position);

   vec3 lightVector = normalize(u_LightPos - v_Position);

   float diffuse = max(dot(v_Normal, lightVector), 0.0);

   diffuse = diffuse*(1.0/(1.0 + (0.10*distance)));

   diffuse = diffuse + 0.3;

   gl_FragColor = v_Color*diffuse*texture2D(u_Texture, v_TexCoordinate);

}

This is the result on a couple of different devices (Nexus 7 with Qualcomm Snapdragon and an Allwinner A31s chip):

Qt on Linux i.MX 6

i.MX 6 is a good device. Qt runs pretty well on it. Also a plugin for hardware accelerated and zero-copy rendering seems to work pretty well. Simply building Qt for i.MX 6 makes it possible to use the Qt Multimedia plugin. This is the result (on Linux):

I also tested a provided Android port but the result was not as good.

Video with Animations on Android using VLC for software decoding

Again, video with animations: this time video is demuxed and decoded using the awesome VLC library. VLC is provided buffers from Java that are then drawn on a view. The result is pretty good if you think all the decoding runs on the CPU. And I'm sure it can be improved.

Video with Animations on Raspberry Pi and A31s Compared

For many uses, playing video while animations are running is pretty useful and it is also considerably stressful for the GPU. I therefore tested a couple of cheap chips with the same Qt/QML sample app. I created a small demo video comparing Qt on A31s and Raspberry Pi (PiOmxTextures):

In the demo, all the media (video and images) are 720p, and the output is 720p for A31s and 1080p for Raspberry Pi (16bpp). It is pretty hard to tell from the video, but Raspberry seems to still be above A31s.
Bye! ;-)