Tuesday, January 6, 2015

Updates on Hardware Accelerated Qt Multimedia Backend on Raspberry Pi (01.05.2015)

Many complained about an horizontal tearing issue when playing videos on the Pi using the PiOmxTextures driver. The issue was caused by a missing switch of buffers backing the textures on the output of the renderer component. I took a little time to quickly improve the rendering and the new result is interesting. I also fixed an OpenMAX setup that caused my Pi to crash when stressed (have a look at the commit logs). These were the videos showing the old implementation:
Pay attention to the fact that video was only 720p when shaders were applied. I couldn't do it on 1080p.

Consider that in these videos animations involving both 1080p graphics and 1080p source video were very difficult. In fact you can see that video is sometimes only 720p.

After recent changes I tried again the same sample apps and created a small demo of what you can achieve with Pi. Now source video, images and UI are all 1080p resolution.

Source code can be found in the usual github repo. At the moment the latest changes are all committed to the branch no_tearing. Keep in mind that those changes may break the rest of the implementation, so some work is needed if you really want to use it in your project. The modifications were only made to show what performance Pi can achieve. You're welcome to send patches to fix the branch as well if you're interested.

18 comments:

  1. Great work once again! Will you be providing binary files for this fix? Your previously provided binaries work beautifully but I do have the tearing issue.

    Thanks.

    ReplyDelete
    Replies
    1. Unfortunately this is just a demo of the performance/result. The changes broke many other "features" so it makes no sense to provide bins. It can't be used without working on it.

      Delete
  2. Hi Again Luca

    It's really awesome what you have achieved with the raspberry pi.

    Thanks again for helping me to make work the PiOmxTextures plugin. But I have an issue again, I don't get audio when reproducing a video with Qt using the PiOmxTextures Plugin...

    Is there some extra configuration needed?

    ReplyDelete
    Replies
    1. No video or just a specific video? There is no particular action you should take to play audio.

      Delete
    2. No video... I know its really weird...
      To Discard other possibilities I tried using OmxPlayer, aplay. etc.... I get audio via the 3.5mm jack with all of them, just can't get audio with QT...

      Delete
    3. Did you try the standard big buck bunny mov format from the official Web site with POCPlayer (sources in the same repo)?

      Delete
    4. I tried again with the video suggested and with the POCPlayer... there's no audio output via the 3.5mm jack. (I even forced the audio output via raspi-config)...
      I used your binaries from 12/2014, and also compiled Qt 5.3.2 from Git and added the OpenMaxil Backend as you described in your posts...
      I'm working on a Raspberry Pi 2, I don't know if there's something related to the kernel version that could make the difference to get the audio.
      If I place again the libgstmediaplayer.so in order to make Qt use the gstreamer plugin I got normal audio, but obviously not Hardware Accelerated Video...

      Any Ideas?

      Delete
    5. I'm sorry. Can'the help you with that. I never used that audio output.

      Delete
    6. I'm sorry. Can'the help you with that. I never used that audio output.

      Delete
    7. Thanks Luca, I didn't have a monitor with audio output from the HDMI, but I tried in another one and I got audio... this means that audio through HDMI is forced by the Openmaxil backend.

      I have other 2 observations:
      - The MediaPlayer component doesn't follows the loops property if I use the Openmaxil Backend. In order to get a loop I can use the onPlaybackStateChanged, but then you get a lag of about 5 seconds between loops (I think it is due to the data interchange between CPU and GPU)
      - Is there any way to avoid the console logs that the Openmaxil Backends deliver?

      Delete
    8. It is possible that the loop feature does not work. I never bothered to implement that. You can probably fix that and patch easily.
      All the logs are made up using https://github.com/carlonluca/LightLogger. Have a look in the header and you'll see that there are macros (BUILD_LOG_LEVEL_*) to switch on and off logs efficiently.

      Delete
    9. Those macros are very useful!

      I'm trying to make a video banner like application. That's why I need the loop property.
      However I manage to overcome this using two mediaplayer components that switch one into the other, with fading effect using the opacity property. The switch is activated 3 seconds before the ending of the video being played in order to overcome the lag I told you before.

      Now I'm testing the performance to assure a 24/7 working duty of the banner.... and I found a bug... It seems that the GPU doesn't dispose the textures correctly, and as I change the video being played it starts to fill the memory. It doesn't matter if the video is long (20min) or short (20 sec) it can only achieve an specific amount of loops.
      If I use only one mediaplayer component it's about 93 video changes, but as I'm using two the number decreases to about 52 video changes (128Mbts memory split to GPU).

      I'm getting into your code to find a way to assure the GPU frees the memory resources... any help would be appreciated...

      These are some of the reports I got using the command sudo vcdbg reloc
      - Just after reboot: http://pastebin.com/BEW0gcfC
      - After 1 play of Big Buck Bunny : http://pastebin.com/jrTsQuAJ
      - When GPU memory gets filled up: http://pastebin.com/TQgfr4Um

      Delete
    10. Yes, probably the same here: https://github.com/carlonluca/pi/issues/13. You can maybe ask if someone fixed it.

      Delete
    11. Hi Nicolás! If you are in a hurry, this summer I made a similar project. You could try to control Omxplayer from qt through QProcess and using "transparent" as background color in your app . It worked well to me.

      It would be awesome if people fork the project and upload their improvements.

      Delete
    12. Hi Atron! That's kind of the solution I have right now, but maybe it's not a good control as the one I can have trough mediaPlayer... maybe I'm doing it wrong...
      Can you share your project or some part of it?

      Right now I'm trying to solve the memory leak problem, but I'm slow at that because I have little experience... I'm Nightmare204 at https://github.com/carlonluca/pi/issues/13

      Delete
    13. "but then you get a lag of about 5 seconds between loops (I think it is due to the data interchange between CPU and GPU)"

      There was a wrong sequence in the setup of the omx graph when opening a video. The time to start is now a little lower. I'll push this change shortly.

      Delete