Tuesday, January 6, 2015

Updates on Hardware Accelerated Qt Multimedia Backend on Raspberry Pi (01.05.2015)

Many complained about an horizontal tearing issue when playing videos on the Pi using the PiOmxTextures driver. The issue was caused by a missing switch of buffers backing the textures on the output of the renderer component. I took a little time to quickly improve the rendering and the new result is interesting. I also fixed an OpenMAX setup that caused my Pi to crash when stressed (have a look at the commit logs). These were the videos showing the old implementation:
Pay attention to the fact that video was only 720p when shaders were applied. I couldn't do it on 1080p.

Consider that in these videos animations involving both 1080p graphics and 1080p source video were very difficult. In fact you can see that video is sometimes only 720p.

After recent changes I tried again the same sample apps and created a small demo of what you can achieve with Pi. Now source video, images and UI are all 1080p resolution.

Source code can be found in the usual github repo. At the moment the latest changes are all committed to the branch no_tearing. Keep in mind that those changes may break the rest of the implementation, so some work is needed if you really want to use it in your project. The modifications were only made to show what performance Pi can achieve. You're welcome to send patches to fix the branch as well if you're interested.
Posted by at
Labels: , , , , , , ,

22 comments:

  1. AnonymousJanuary 7, 2015 at 3:50 PM

    Great work once again! Will you be providing binary files for this fix? Your previously provided binaries work beautifully but I do have the tearing issue.

    Thanks.

    ReplyDelete
    Replies
    1. Luca CarlonJanuary 7, 2015 at 3:56 PM

      Unfortunately this is just a demo of the performance/result. The changes broke many other "features" so it makes no sense to provide bins. It can't be used without working on it.

      Delete
  2. AnonymousFebruary 1, 2015 at 4:19 PM

    This is awesome.

    ReplyDelete
  3. UnknownFebruary 19, 2015 at 11:19 PM

    Hi Again Luca

    It's really awesome what you have achieved with the raspberry pi.

    Thanks again for helping me to make work the PiOmxTextures plugin. But I have an issue again, I don't get audio when reproducing a video with Qt using the PiOmxTextures Plugin...

    Is there some extra configuration needed?

    ReplyDelete
    Replies
    1. Luca CarlonFebruary 20, 2015 at 12:25 AM

      No video or just a specific video? There is no particular action you should take to play audio.

      Delete
    2. UnknownFebruary 20, 2015 at 12:57 AM

      No video... I know its really weird...
      To Discard other possibilities I tried using OmxPlayer, aplay. etc.... I get audio via the 3.5mm jack with all of them, just can't get audio with QT...

      Delete
    3. Luca CarlonFebruary 20, 2015 at 1:04 AM

      Did you try the standard big buck bunny mov format from the official Web site with POCPlayer (sources in the same repo)?

      Delete
    4. UnknownFebruary 20, 2015 at 7:02 PM

      I tried again with the video suggested and with the POCPlayer... there's no audio output via the 3.5mm jack. (I even forced the audio output via raspi-config)...
      I used your binaries from 12/2014, and also compiled Qt 5.3.2 from Git and added the OpenMaxil Backend as you described in your posts...
      I'm working on a Raspberry Pi 2, I don't know if there's something related to the kernel version that could make the difference to get the audio.
      If I place again the libgstmediaplayer.so in order to make Qt use the gstreamer plugin I got normal audio, but obviously not Hardware Accelerated Video...

      Any Ideas?

      Delete
    5. Luca CarlonFebruary 20, 2015 at 8:31 PM

      I'm sorry. Can'the help you with that. I never used that audio output.

      Delete
    6. Luca CarlonFebruary 21, 2015 at 12:13 AM

      I'm sorry. Can'the help you with that. I never used that audio output.

      Delete
    7. UnknownFebruary 23, 2015 at 9:13 PM

      Thanks Luca, I didn't have a monitor with audio output from the HDMI, but I tried in another one and I got audio... this means that audio through HDMI is forced by the Openmaxil backend.

      I have other 2 observations:
      - The MediaPlayer component doesn't follows the loops property if I use the Openmaxil Backend. In order to get a loop I can use the onPlaybackStateChanged, but then you get a lag of about 5 seconds between loops (I think it is due to the data interchange between CPU and GPU)
      - Is there any way to avoid the console logs that the Openmaxil Backends deliver?

      Delete
    8. Luca CarlonFebruary 23, 2015 at 10:46 PM

      It is possible that the loop feature does not work. I never bothered to implement that. You can probably fix that and patch easily.
      All the logs are made up using https://github.com/carlonluca/LightLogger. Have a look in the header and you'll see that there are macros (BUILD_LOG_LEVEL_*) to switch on and off logs efficiently.

      Delete
    9. UnknownMarch 2, 2015 at 2:38 PM

      Those macros are very useful!

      I'm trying to make a video banner like application. That's why I need the loop property.
      However I manage to overcome this using two mediaplayer components that switch one into the other, with fading effect using the opacity property. The switch is activated 3 seconds before the ending of the video being played in order to overcome the lag I told you before.

      Now I'm testing the performance to assure a 24/7 working duty of the banner.... and I found a bug... It seems that the GPU doesn't dispose the textures correctly, and as I change the video being played it starts to fill the memory. It doesn't matter if the video is long (20min) or short (20 sec) it can only achieve an specific amount of loops.
      If I use only one mediaplayer component it's about 93 video changes, but as I'm using two the number decreases to about 52 video changes (128Mbts memory split to GPU).

      I'm getting into your code to find a way to assure the GPU frees the memory resources... any help would be appreciated...

      These are some of the reports I got using the command sudo vcdbg reloc
      - Just after reboot: http://pastebin.com/BEW0gcfC
      - After 1 play of Big Buck Bunny : http://pastebin.com/jrTsQuAJ
      - When GPU memory gets filled up: http://pastebin.com/TQgfr4Um

      Delete
    10. Luca CarlonMarch 2, 2015 at 2:43 PM

      Yes, probably the same here: https://github.com/carlonluca/pi/issues/13. You can maybe ask if someone fixed it.

      Delete
    11. AtronMarch 2, 2015 at 9:23 PM

      Hi Nicolás! If you are in a hurry, this summer I made a similar project. You could try to control Omxplayer from qt through QProcess and using "transparent" as background color in your app . It worked well to me.

      It would be awesome if people fork the project and upload their improvements.

      Delete
    12. UnknownMarch 5, 2015 at 3:28 PM

      Hi Atron! That's kind of the solution I have right now, but maybe it's not a good control as the one I can have trough mediaPlayer... maybe I'm doing it wrong...
      Can you share your project or some part of it?

      Right now I'm trying to solve the memory leak problem, but I'm slow at that because I have little experience... I'm Nightmare204 at https://github.com/carlonluca/pi/issues/13

      Delete
    13. Luca CarlonMarch 15, 2015 at 3:07 AM

      "but then you get a lag of about 5 seconds between loops (I think it is due to the data interchange between CPU and GPU)"

      There was a wrong sequence in the setup of the omx graph when opening a video. The time to start is now a little lower. I'll push this change shortly.

      Delete
    14. UnknownMarch 4, 2018 at 12:49 AM

      Hi Nicolás and Luca! I know that this is an old topic, but I can not loop the playlist. After the first video playlist goes to the next video without playing the video and so on.
      How can I fix this?

      Thank you in advance.

      Delete
    15. Luca CarlonMarch 4, 2018 at 1:52 AM

      Can you provide a minimal sample showing the problem?

      Delete
    16. UnknownMarch 4, 2018 at 11:18 AM

      Luca thanks for the answer!
      Minimal sample: https://pastebin.com/ksifMv6v
      Log: https://pastebin.com/s0BG38wU

      Delete
    17. UnknownMarch 6, 2018 at 11:47 AM

      I was able to loop the playlist as in your piomxtextures_pocplayer/qml/main_loop.qml. At the moment, it's enough for me.

      Delete
  4. UnknownMarch 5, 2015 at 3:28 PM

    This comment has been removed by the author.

    ReplyDelete

Subscribe to: Post Comments (Atom)