I am trying to read an image sequence (.dds) and an audio file (.wav) separately. They both start on the same trigger. Unfortunetly, both files get desynchronized after a few seconds. It is really noticeable with an image sequence where someone is speaking and the audio file is played on the side. You clearly see the lips not in sync with the audio.
The Position value outputed from the ImagePlayer and the AudioPlayer is different even though they were started at the same moment.
I also tried to use AudioBuffer. In this case, the Position value is somehow identical, but both files are still desynchronized.
It works if the mainloop is not blocked. But if frames drop, obviously the AudioPlayer and ImagePlayer freeze. At least, the image and the audio are correctly synchronized. So I guess to avoid the freezes, that’s where your idea of adding / substracting N frames comes in. Am I right?
avoiding frame drops is key. to keep the last bit of sync then you could also go the other way round and slow down/speed up the audio. doing it with a very long filter and super small values can keep the sync and should not be noticable in most cases. but if you drop a lot frames constantly this might not suffice or speed change of audio might be too much to overhear.
Ah good idea, didn’t think about using a filter!
I was trying to do it the other way around but you can’t just say to the AudioPlayer: “play that frame”.
I will give it a try with a filter.
Thanks to the both of you for leading me into some directions!
Just a quick update on this topic to share my latest working setup, in case it is helpful for someone later. I use the recommendation from @tgd to slow down or speed up the AudioPlayer based on the ImagePlayer. Changing the frame rate of the mainloop doesn’t affect the synchronization. Small framedrops are ok too.
It works for my current setup, so I share it like this for everyone.
Thanks again Tebjan and tgd for the insights!
