This is not the first time I have encountered this problem: if I need to save a file and then load it, it is impossible if I am writing to the same (temporary) file.
All nodes related to texture loading are either asynchronous or non-blocking. I am trying to load using TextureReader. This is a blocking function, but it does not have a ‘reload’ option and simply will not ‘reload’ the file when the same name is passed. At the same time, FileTexture has Reload, but I cannot use it in a blocking scenario (waiting for loading, for example, in an observable pipeline).
Can you please add a ‘Reload’ function to TextureReader? Or is there another way?
In general, I discovered something quite unpleasant. If you use TextureReader in an Observable pipeline, you will notice that it is also asynchronous and non-blocking.
The texture is created and only then is the image loaded into it. As a result, I simply cannot process it because I don’t know when the texture was actually loaded.
I suspect that this is some kind of feature of Stride.
I noticed this the other day, toggling is sRGB triggers a reload, so you can make use of that as a workaround, but adding a reload pin makes sense to me
I simplified things a bit by writing an example illustrating the problem. In your case, it may or may not work, and you may need to ‘tinker’ with it. But I think the essence is clear. The problem is that at some stage it is impossible to guarantee that the pixels will be obtained and only then move on, which in some cases makes loading and processing images impossible, and managing ‘whether the image has loaded or not’ very difficult.
This is not the first time I have encountered this problem, and I have tried to understand what it is all about. As I understand it, Stride has a ‘resource engine’ and uploading an image in the way I am trying to do it is either impossible or unavailable in VVVV specifically. In any case, in such cases, you have to invent some kind of delay structures, complex checks or something else to make it work, and this should not be the case for such a basic operation.
I will show you a small part of my patch. This is my third attempt to load pixels received via the Internet. Basically, it’s the same problem every time. I try to save the received image to a file, then load it as a texture, then get the pixel bytes from the texture and process them. It seems like nothing special, but this is not the first time I’ve realised that it’s simply impossible. At some point, it becomes impossible to capture pixels from the texture. It is impossible to tell whether it has loaded, and often the processing is done on an empty, transparent texture before the texture finally ‘loads’ into its object. Sometimes this ‘manages’ to happen before processing begins, sometimes not.
Ok I see what you want to achieve here. You need to understand first that the GPU is an asynchronous resource per se. Commands are sent to it and only at some point later they will be completed. I think we have a help patch doing something similar in the TPL package. Type “TPL” in help browser, there is a “Run rendering tasks in parallel”. Maybe have a look first.
@Elias Wow, TPL is huge! I will look into the package and its capabilities, and I will get back to you after I have researched it. Thank you very much for the insight.
I would be very grateful if you could provide some more examples on the topic.
I tried to analyze the example, and it’s quite interesting. As I understand it, we can ignore TPL for now.
What remains is that we can copy the texture as a staging texture. At this point, I don’t really completely understand the terms and what is happening. But the CopyToStagingTexture command performs the copy, and thanks to the signature (with Task), the result can be awaited. But further possibilities are limited. I can only save it. How can I convert a staging texture into, for example, a pixel buffer? I need bytes from this texture, but I can’t get them because it’s a staging texture.
It’s not urgent for me because I’m just experimenting. But this isn’t the first time this problem has come up. In Beta, it was much easier and clearer, but here it’s clearly not that simple.
I can do CopyToStagingAsync and then GetData and get the bytes. That’s a big step forward.
But there is still a problem. I have to load the image. When the image is “loaded,” I have to draw it (crop it or apply a shader). And then I have to get the bytes from the render. And the whole scheme breaks down at the first stage, where I cannot guarantee that the texture is loaded.
It seems that Advanced nodes are needed to guarantee texture loading. Something like LoadTexture (Task). If such a node existed, it would solve all the problems and my attempts to find a workaround.
ah, I also have a copy of texturereader with the graphicsdevice exposed as a pin. using it for quite some time to load textures async without problems.
@sebl Am I correct in understanding that you have a similar setup that works without any problems? Could you share your version, please?
Indeed, maybe the problem is only on my end and I need to look for a solution elsewhere? Or maybe it’s some kind of rare bug? Don’t get me wrong, in 9 out of 10 cases this behaviour wouldn’t be a big deal, but I literally can’t solve the problem.
make sure your observables are on an background thread with ToBackground, for example. ToObservable will NOT put the call on the a background thread, it will execute immediately and wait until the complete observable chain is done.
The TPL help patch does all that what you asked for. Not sure why you dismissed it right away. If you go back to that patch and instead of GetStagingTextureAsync you place GetDataAsync you would in the bottom have access to the texture data. See attached slightly modified version of said help patch.
I need to study the example more carefully. But I will leave the original message under the spoiler.
@Elias Try to understand what I’m talking about. I wrote about this above, I’ll try to explain in more detail and where exactly the problem lies.
I need to load the texture. <<< This is a key element!
Process this texture using shaders. Basically, just crop and adjust the gamma and brightness.
Next, convert the resulting texture into bytes and pass it on.
So, the problem starts not at the processing stage, but at the LOADING stage. Loading is asynchronous. No matter what I do, I can’t first load the texture, then process it, and then get its data. I’ve already checked GetDataAsync, it works fine on the staging texture, but the staging texture cannot be processed in any way. You can load and process, but you can’t get the data. You can load and get the data, but you can’t process it. The result is a stalemate: you can do either one or the other, but not both. Moreover, neither option is obvious, nor are there any alternative options. It is worth noting that I tried many different combinations, but I will not go into detail here; some of them are shown in the screenshots.
@sebl Thanks, but it’s the same problem. The loaded textures load their data in the background, which is very fast, but this is what happens. As a result, I cannot obtain the bytes of the loaded and processed texture at the same time.
@tonfilm I’m not completely sure what it means. I tried “ToBackground” in different places, but with no success. Please provide more context; it would be very helpful, and I would be very grateful.
I feel that I need to give an example to illustrate the problem. Perhaps the problem seems ‘too simple,’ but in fact it is much more complicated than that. I will make an example and come back to it.
@sebl Pretty sure your attempt will cause random crashes. Rule of thumb: don’t put Stride nodes in background threads. Most of them access the immediate context (command list) which must only be done from the main thread. Exception to that rule are nodes which only create / destroy resources, like LoadTexture. In other words: while the device can be used to create resources on multiple threads, the device context / immediate context is only meant to be operated by one thread at a time. One could build up rendering commands on a deferred contexts and play them back in the immediate context, but that’s a whole different story.
