YoloDotNet Bug

[Raphael]

Hello Guys, just spot a bug on the yoloDotNet Nugget,
All the Non Async doesnt work properly anymore for me…
I using the the preview 320
And the version 0.2.2-alpha of the nuget

Thanks

[joreg]

that’s best directed at the packs author @dani21s

[dani21s]

Hi Raphael,
it is related to a change in VideoSourceToSKImage behavior.
Just disable the PreferGpu input pin on the VideoSourceToSKImage node.

[Raphael]

Thanks you @dani21s
I love when its so simple :)
shouldn’t be disable by default in the patch exemple?

[dani21s]

Yes, it should be disabled by default, but VideoSourceToSKImage has changed after the package was released. I will fix it in the next release.

[Elias]

@dani21s
Keep in mind that the SKImage returned by VideoSourceToSkImage is only valid for this one frame. Once it receives a new image from the camera it will dispose the previous one. If you process it in background later you should make a copy first. Currently you seem to do this on the background thread as well which is not entirely correct - bit surprised that you didn’t run into any issues there.

Also note that images can be GPU or CPU based (called raster image). In your case you want to deal with CPU based images because you access the pixels. Setting the “Prefer GPU” to false on the VideoSourceToSkImage node is one way to achieve that. If you already have a texture backed image, you can use ToRasterImage.

Now regarding the bug described here between 6 stable and 7 preview - what did change between those two versions is that Skia and Stride now share one Direct3D11 device. Each app / entry point / root patch comes with its own device. Sadly the Skia API does not keep track of this hidden dependency and assumes that when accessing GPU based images the corresponding underlying graphics context is current. In its current form of Skia it uses OpenGL for rendering / GPU based resource allocation. The OpenGL implementation we initialize Skia with is implemented by ANGLE which in turn translates OpenGL calls to Direct3D. This is how we can use the same device for Skia and Stride. But that also means that it’s now up to us programmers to ensure the correct graphics context is current before accessing such GPU backed images which is a bit cumbersome. It could be that we find a more compatible / more aligned to version 6 solution here, by making the graphics context current for at least the Update call on the root patch. Can’t give you any estimate though whether or not we go that route.

Maybe the easiest approach for you is to clone the image up-front and ensure its CPU based. Once you have your private copy backed by CPU memory you can do what you want with it.

[dani21s]

@Elias
Hi Elias,
thank you very much for your explanation, which, as always, is precise and very helpful. I’ve started working on this package, even though my knowledge of the tools involved is not advanced. Now, everything is much clearer to me.

If I understand correctly, using the “ToRasterImage” node before processing the image should resolve the issue, even if Skia’s internal handling changes in the future.

I also noticed that when opening and closing the “Yolo Object Detection Async” help patch multiple times, an exception occurs upon closing the patch on the fifth or sixth consecutive attempt. Here is the exception message:

“System.AccessViolationException - Attempted to read or write protected memory. This is often an indication that other memory is corrupt.”

This happens in the “InferenceSession.cs” module of the Microsoft.ML.OnnxRuntime library. Could this be due to improper handling of the asynchronous task when the patch is closed?

Just for reference, are there any plans to upgrade SkiaSharpfrom version 2.88
to 3.xx?

[Elias]

AccessViolation - yes, this is most probably because you access memory which has already been deleted. I do not know what “InferenceSessions.cs” does exactly. If it makes access to a SkImage which in turn doesn’t fully belong to your library / you didn’t copy it initially on the main thread, then yes, this could be the reason. But it might be about something completely different as well - could get tricky to figure out.

And yes, we want to update to SkiaSharp 3.0 after our 7.0 release. Preliminary tests looked good already.