Monday, November 30, 2015

Hands-on WebKit and HTML5 Media Elements on Raspberry Pi

I tried to do this a few months ago actually, but I got lost in the huge amount of code and architecture variants of WebKit. Now I convinced myself it would be too interesting to see it working. I don't have much spare time, so I wanted to start with the path of least-resistance and then, depending on the time available, continue climbing. This activity is very time consuming, so I do not know where I'll get.

This is however the first stage: using POT to implement 1080p hardware accelerated zero-copy video playback in the Qt port of WebKit1. No QML here. Of no use, but it somehow works. I read there are far better results out there, but I learned much by doing this, and I had fun :-)
Of course, as you can see, there is much much work being done in the UI thread. JavaScript is mostly run in the main thread, making the framerate pretty bad on heavy websites. That is why WebKit2 is there after all :-)

Demo

Bye! ;-)

Monday, November 16, 2015

Bluetooth (BLE) Support with Qt on Raspberry Pi

There are many ways to access Bluetooth capabilities on Raspberry Pi. Qt has its own API to use Bluetooth and to implement a BLE central device. The last build I provided a few days ago didn't support Bluetooth: this package should support it. I tested the low energy scanner example and it worked properly.
Have fun! Bye ;-)

Download POT 5.1.0 with Qt 5.5.1 for Pi2 Raspbian Jessie here (extraction code: 96aa).

Fully Optimised Raspbian Jessie with Accelerated Qt Multimedia Backend (Pi2)

This is an image of Raspbian Jessie ready to test (and use) the Qt Multimedia Backend POT. This image is optimised for maximum performance (see the demos below). This is the current setup:

pi@raspberrypi ~ $ sudo /opt/vc/bin/vcgencmd version
Nov 11 2015 21:31:07 
Copyright (c) 2012 Broadcom
version 54011a8ad59a9ae1c40bd07cddd9bcf90e779b66 (clean) (release)
pi@raspberrypi ~ $ uname -a
Linux raspberrypi 4.1.13-v7+ #826 SMP PREEMPT Fri Nov 13 20:19:03 GMT 2015 armv7l GNU/Linux
pi@raspberrypi ~ $ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 8.0 (jessie)
Release:        8.0
Codename:       jessie

Download image here.

The image includes:
  • Qt 5.5.1 built for armv7 with neon optimisations, including support for touch screens, gstreamer, libinput, X11 (no multimedia), tslib, and Bluetooth/Bluetooth BLE using bluez (complete configuration below). Modules provided include QtWebKit (no multimedia support).
  • Dependant libs: evdev, icu (not the one provided by raspbian), mtdev, libts.
  • POCPlayer and QML samples to show the performance of the backend and test bugs.
  • POT library including ffmpeg 2.7.2.
Everything is built for armv7 including neon optimisations. To quickly test the performance there are a few things you can launch:

/home/pi/piomxtextures_pocplayer --multipleanimtest /home/pi/sintel_trailer_720p.mp4 /home/pi/big_buck_bunny_720p_h264.mov

This is the result you should expect with a proper overclocking:


It will show you a continuous animation while running two 720p videos concurrently. To improve the frame rate of the animations you can also overclock your Pi2 (see below).
Other samples to test the features of the plugin are stored in /home/pi/samples:

/usr/local/Qt-rasp2-5.5.1/bin/qmlscene /home/pi/samples/video_simple.qml file:///home/pi/big_buck_bunny_1080p_h264.mov

Another interesting test is running qmlvideofx example included in the Qt sources. This will show you the perfomance of the effects applied on the video using shaders. You can find an executable in /usr/local/Qt-rasp2-5.5.1/examples/multimedia/video/qmlvideofx/qmlvideofx. This shows approximately what you should expect (the video shows the result on Pi1, Pi2 is just a little better):

Overclocking

The Pi can be overclocked by tuning parameters in /boot/config.txt. I provided a configuration that I find useful:

#force_turbo=1  # Voids Warranty!
#boot_delay=1   # Helps to avoid sdcard corruption when force_turbo is enabled.
#arm_freq=1100
#sdram_freq=450
#core_freq=550
#over_voltage=4
#temp_limit=80  # Will throttle to default clock speed if hit.

In the image this is all disabled. If you intent to enable please keep in mind I don't take responsibility for any consequence. Please refer to the documentation for this procedure.

Have fun! Bye ;-)

Qt Configuration

   Configure summary

Building on:   linux-g++ (x86_64, CPU features: mmx sse sse2)
Building for:  devices/linux-rasp-pi2-g++ (arm, CPU features: neon)
Platform notes:

            - Also available for Linux: linux-clang linux-kcc linux-icc linux-cxx
        
qmake vars .......... styles += mac fusion windows QT_CFLAGS_GLIB = -pthread -I/opt/rpi/sysroot/usr/include/glib-2.0 -I/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf/glib-2.0/include  QT_LIBS_GLIB = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lgthread-2.0 -pthread -lglib-2.0  QT_CFLAGS_PULSEAUDIO = -D_REENTRANT -I/opt/rpi/sysroot/usr/include/glib-2.0 -I/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf/glib-2.0/include  QT_LIBS_PULSEAUDIO = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lpulse-mainloop-glib -lpulse -lglib-2.0  QMAKE_CFLAGS_FONTCONFIG = -I/opt/rpi/sysroot/usr/include/freetype2  QMAKE_LIBS_FONTCONFIG = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lfontconfig -lfreetype  QMAKE_INCDIR_LIBUDEV =  QMAKE_LIBS_LIBUDEV = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -ludev  QMAKE_LIBINPUT_VERSION_MAJOR = 1 QMAKE_LIBINPUT_VERSION_MINOR = 0 QMAKE_INCDIR_LIBINPUT = /opt/rpi/sysroot/home/pi/qtdeps/include  QMAKE_LIBS_LIBINPUT = -L/opt/rpi/sysroot/home/pi/qtdeps/lib -linput  QMAKE_LIBXI_VERSION_MAJOR = 1 QMAKE_LIBXI_VERSION_MINOR = 7 QMAKE_LIBXI_VERSION_PATCH = 4 QMAKE_X11_PREFIX = /usr QMAKE_XKB_CONFIG_ROOT = /usr/share/X11/xkb QMAKE_CFLAGS_XCB =  QMAKE_LIBS_XCB = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lxcb-sync -lxcb-xfixes -lxcb-render -lxcb-randr -lxcb-image -lxcb-shm -lxcb-keysyms -lxcb-icccm -lxcb-shape -lxcb  INCLUDEPATH +=  "/opt/rpi/sysroot/home/pi/qtdeps/include" LIBS +=  -L"/opt/rpi/sysroot/home/pi/qtdeps/lib" sql-drivers =  sql-plugins =  sqlite qmake switches ......... 

Build options:
  Configuration .......... accessibility accessibility-atspi-bridge alsa audio-backend c++11 clock-gettime clock-monotonic compile_examples concurrent cross_compile cups dbus egl eglfs eglfs_brcm enable_new_dtags evdev eventfd fontconfig full-config getaddrinfo getifaddrs glib gstreamer-1.0 harfbuzz iconv icu inotify ipv6ifname large-config largefile libinput libproxy libudev linuxfb medium-config minimal-config mremap mtdev neon nis opengl opengles2 openssl pcre png posix_fallocate pulseaudio qpa qpa reduce_exports release rpath shared small-config system-freetype system-jpeg system-png system-zlib tslib use_gold_linker xcb xcb-glx xcb-plugin xcb-render xcb-xlib xinput2 xkbcommon-qt xlib xrender 
  Build parts ............  libs examples
  Mode ................... release
  Using sanitizer(s)...... none
  Using C++11 ............ yes
  Using gold linker....... yes
  Using new DTAGS ........ yes
  Using PCH .............. no
  Target compiler supports:
    Neon ................. yes

Qt modules and options:
  Qt D-Bus ............... yes (loading dbus-1 at runtime)
  Qt Concurrent .......... yes
  Qt GUI ................. yes
  Qt Widgets ............. yes
  Large File ............. yes
  QML debugging .......... yes
  Use system proxies ..... no

Support enabled for:
  Accessibility .......... yes
  ALSA ................... yes
  CUPS ................... yes
  Evdev .................. yes
  FontConfig ............. yes
  FreeType ............... yes (system library)
  Glib ................... yes
  GStreamer .............. yes (1.0)
  GTK theme .............. no
  HarfBuzz ............... yes (bundled copy)
  Iconv .................. yes
  ICU .................... yes
  Image formats: 
    GIF .................. yes (plugin, using bundled copy)
    JPEG ................. yes (plugin, using system library)
    PNG .................. yes (in QtGui, using system library)
  journald ............... no
  libinput................ yes
  mtdev .................. yes (system library)
  Networking: 
    getaddrinfo .......... yes
    getifaddrs ........... yes
    IPv6 ifname .......... yes
    libproxy.............. yes
    OpenSSL .............. yes (loading libraries at run-time)
  NIS .................... yes
  OpenGL / OpenVG: 
    EGL .................. yes
    OpenGL ............... yes (OpenGL ES 2.0+)
    OpenVG ............... no
  PCRE ................... yes (bundled copy)
  pkg-config ............. yes 
  PulseAudio ............. yes
  QPA backends: 
    DirectFB ............. no
    EGLFS ................ yes
      EGLFS i.MX6....... . no
      EGLFS KMS .......... no
      EGLFS Mali ......... no
      EGLFS Raspberry Pi . yes
      EGLFS X11 .......... no
    LinuxFB .............. yes
    XCB .................. yes (system library)
      EGL on X ........... no
      GLX ................ yes
      MIT-SHM ............ yes
      Xcb-Xlib ........... yes
      Xcursor ............ yes (loaded at runtime)
      Xfixes ............. yes (loaded at runtime)
      Xi ................. no
      Xi2 ................ yes
      Xinerama ........... yes (loaded at runtime)
      Xrandr ............. yes (loaded at runtime)
      Xrender ............ yes
      XKB ................ yes
      XShape ............. yes
      XSync .............. yes
      XVideo ............. yes
  Session management ..... yes
  SQL drivers: 
    DB2 .................. no
    InterBase ............ no
    MySQL ................ no
    OCI .................. no
    ODBC ................. no
    PostgreSQL ........... no
    SQLite 2 ............. no
    SQLite ............... yes (plugin, using bundled copy)
    TDS .................. no
  tslib .................. yes
  udev ................... yes
  xkbcommon-x11........... yes (bundled copy, XKB config root: /usr/share/X11/xkb)
  xkbcommon-evdev......... no
  zlib ................... yes (system library)

Wednesday, November 11, 2015

Binaries for Hardware Accelerated Qt 5.5.1 Multimedia Backend on Raspbian Jessie (POT 5.1.0)

This is a build of version 5.1.0 for Raspbian Jessie of POT that includes Qt 5.5.1. As many asked in the past, this version also includes the QtWebKit module (WebKit1 and WebKit2). The changes are:
  • Fixed bugs reported on github.
  • New environment variables used for controlling deinterlacing:
    • POT_HALF_FRAMERATE_MODE: if set to 1 the number of frames sent to the renderer are halved. This is needed mostly when using deinterlacing.
    • POT_TEXTURE_COUNT: number of texture to be used when buffering. Default is 4.
    • POT_DEINTERLACE_MODE: the type of deinterlacing; 0 means off, 1 means auto and 2 means on.
    • POT_DEINTERLACE_QPU: 0 or 1; indicates whether to use the QPU for deinterlacing.
  • Added watchdogs to monitor the video capabilities and discover instability. These are currently disable but you can enable if needed.
  • Some other minor fixes and improvements.
Please remember to update the firmware to the latest possible version as it includes a fix relevant to POT.
Please file bugs to github if you experience issues.
Bye! ;-)

Download PiOmxTextures 5.1.0 for Raspbian Jessie Pi2 here (extraction code is: bd4f).
Download PiOmxTextures 5.1.0 for Raspbian Jessie Pi2 with Bluetooth (BLE) here.

This is the configure summary of the Qt build:
   Configure summary

Building on:   linux-g++ (x86_64, CPU features: mmx sse sse2)
Building for:  devices/linux-rasp-pi2-g++ (arm, CPU features: neon)
Platform notes:

            - Also available for Linux: linux-clang linux-kcc linux-icc linux-cxx
        
qmake vars .......... styles += mac fusion windows QT_CFLAGS_GLIB = -pthread -I/opt/rpi/sysroot/usr/include/glib-2.0 -I/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf/glib-2.0/include  QT_LIBS_GLIB = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lgthread-2.0 -pthread -lglib-2.0  QT_CFLAGS_PULSEAUDIO = -D_REENTRANT -I/opt/rpi/sysroot/usr/include/glib-2.0 -I/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf/glib-2.0/include  QT_LIBS_PULSEAUDIO = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lpulse-mainloop-glib -lpulse -lglib-2.0  QMAKE_CFLAGS_FONTCONFIG = -I/opt/rpi/sysroot/usr/include/freetype2  QMAKE_LIBS_FONTCONFIG = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lfontconfig -lfreetype  QMAKE_INCDIR_LIBUDEV =  QMAKE_LIBS_LIBUDEV = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -ludev  QMAKE_LIBINPUT_VERSION_MAJOR = 1 QMAKE_LIBINPUT_VERSION_MINOR = 0 QMAKE_INCDIR_LIBINPUT = /opt/rpi/sysroot/home/pi/qtdeps/include  QMAKE_LIBS_LIBINPUT = -L/opt/rpi/sysroot/home/pi/qtdeps/lib -linput  QMAKE_LIBXI_VERSION_MAJOR = 1 QMAKE_LIBXI_VERSION_MINOR = 7 QMAKE_LIBXI_VERSION_PATCH = 4 QMAKE_X11_PREFIX = /usr QMAKE_XKB_CONFIG_ROOT = /usr/share/X11/xkb QMAKE_CFLAGS_XCB =  QMAKE_LIBS_XCB = -L/opt/rpi/sysroot/usr/lib/arm-linux-gnueabihf -lxcb-sync -lxcb-xfixes -lxcb-render -lxcb-randr -lxcb-image -lxcb-shm -lxcb-keysyms -lxcb-icccm -lxcb-shape -lxcb  INCLUDEPATH +=  "/opt/rpi/sysroot/home/pi/qtdeps/include" LIBS +=  -L"/opt/rpi/sysroot/home/pi/qtdeps/lib" sql-drivers =  sql-plugins =  sqlite qmake switches ......... 

Build options:
  Configuration .......... accessibility accessibility-atspi-bridge alsa audio-backend c++11 clock-gettime clock-monotonic compile_examples concurrent cross_compile cups dbus egl eglfs eglfs_brcm enable_new_dtags evdev eventfd fontconfig full-config getaddrinfo getifaddrs glib gstreamer-1.0 harfbuzz iconv icu inotify ipv6ifname large-config largefile libinput libproxy libudev linuxfb medium-config minimal-config mremap mtdev neon nis opengl opengles2 openssl pcre png posix_fallocate pulseaudio qpa qpa reduce_exports release rpath shared small-config system-freetype system-jpeg system-png system-zlib tslib use_gold_linker xcb xcb-glx xcb-plugin xcb-render xcb-xlib xinput2 xkbcommon-qt xlib xrender 
  Build parts ............  libs examples
  Mode ................... release
  Using sanitizer(s)...... none
  Using C++11 ............ yes
  Using gold linker....... yes
  Using new DTAGS ........ yes
  Using PCH .............. no
  Target compiler supports:
    Neon ................. yes

Qt modules and options:
  Qt D-Bus ............... yes (loading dbus-1 at runtime)
  Qt Concurrent .......... yes
  Qt GUI ................. yes
  Qt Widgets ............. yes
  Large File ............. yes
  QML debugging .......... yes
  Use system proxies ..... no

Support enabled for:
  Accessibility .......... yes
  ALSA ................... yes
  CUPS ................... yes
  Evdev .................. yes
  FontConfig ............. yes
  FreeType ............... yes (system library)
  Glib ................... yes
  GStreamer .............. yes (1.0)
  GTK theme .............. no
  HarfBuzz ............... yes (bundled copy)
  Iconv .................. yes
  ICU .................... yes
  Image formats: 
    GIF .................. yes (plugin, using bundled copy)
    JPEG ................. yes (plugin, using system library)
    PNG .................. yes (in QtGui, using system library)
  journald ............... no
  libinput................ yes
  mtdev .................. yes (system library)
  Networking: 
    getaddrinfo .......... yes
    getifaddrs ........... yes
    IPv6 ifname .......... yes
    libproxy.............. yes
    OpenSSL .............. yes (loading libraries at run-time)
  NIS .................... yes
  OpenGL / OpenVG: 
    EGL .................. yes
    OpenGL ............... yes (OpenGL ES 2.0+)
    OpenVG ............... no
  PCRE ................... yes (bundled copy)
  pkg-config ............. yes 
  PulseAudio ............. yes
  QPA backends: 
    DirectFB ............. no
    EGLFS ................ yes
      EGLFS i.MX6....... . no
      EGLFS KMS .......... no
      EGLFS Mali ......... no
      EGLFS Raspberry Pi . yes
      EGLFS X11 .......... no
    LinuxFB .............. yes
    XCB .................. yes (system library)
      EGL on X ........... no
      GLX ................ yes
      MIT-SHM ............ yes
      Xcb-Xlib ........... yes
      Xcursor ............ yes (loaded at runtime)
      Xfixes ............. yes (loaded at runtime)
      Xi ................. no
      Xi2 ................ yes
      Xinerama ........... yes (loaded at runtime)
      Xrandr ............. yes (loaded at runtime)
      Xrender ............ yes
      XKB ................ yes
      XShape ............. yes
      XSync .............. yes
      XVideo ............. yes
  Session management ..... yes
  SQL drivers: 
    DB2 .................. no
    InterBase ............ no
    MySQL ................ no
    OCI .................. no
    ODBC ................. no
    PostgreSQL ........... no
    SQLite 2 ............. no
    SQLite ............... yes (plugin, using bundled copy)
    TDS .................. no
  tslib .................. yes
  udev ................... yes
  xkbcommon-x11........... yes (bundled copy, XKB config root: /usr/share/X11/xkb)
  xkbcommon-evdev......... no
  zlib ................... yes (system library)