How to: Map texture based on 4 source and 4 target points in vvvv gamma, ideally with VL.Stride

[domj]

Hey people of vvvv land!

Having a bit of a pickle here

Trying to set-up my point cloud based virtual studio with the gift of real-time captured textures.

For that, I need to turn the original camera image into a straight front view of various objects.

I tried using the various Homography nodes and using the resulting Matrix to transform textures. The results were rather strange. With the transformation being limited to a sort of skew. I’ll try to grab a screenshot soon.

I tried various corner ordering, eventually settling for that from the Homography help patch.

I also attempted modifying the UV coordinates in the Dynamic mesh example IIRC, but the stretching was only applied per-triangle.

Probably missing some detail here, maybe need to apply homography to camera instead of texture but could definitely use some tips so I don’t waste time next time I get around to it.

Thanks!

[motzi]

i don’t think this is feasible just using a 2D (texture) transform as your scene objects are 3-dimensional (and probably not aligned in a plane) and therefore need different stretchings.

just look at your illustration for clarification:

from your point of view the red part should probably disappear (being perpendicular to the wanted “virtual” position) while the yellow part would need some stretching.

you’d probably need some reprojection approach where you need a 3D geometry representation of your scene, virtually project the image on them from the correct position onto the scene and recapture it from the intended position.

the “Calibrate a projector and Reproject” patches from VL.OpenCV should give you and idea how to do it. (VL.Badmapper also appears to have a similar patch but I’m not familiar with that).

[domj]

Hey @motzi, thanks so much for the answer, I loved how you built on top of the illustration :D

It does seem like that might be the way to go in situations with considerable lens warping or when the goal is to have a full 3D representation with a dynamic texture.

I did give Homography another go before delving into OpenCV or Badmapper, and it does seem to work actually. Based on the included Homography example, I’ve created this patch that stretches target points to full view using a Transform applied on a QuadRenderer.

homography-from-to.vl (28.3 KB)
homography-test.png

(Note: Using the pad for distributing Texture is causing flicker with a live texture, eg. a camera feed)

(Note 2: UV approach also seems to work but the mesh requires subdivisions, 20 in each axis seem enough for a clear image, tested in blender)

(Note 3: Homography calculation fails if shape of original points is too thin

---

Now to find what is the easiest way to drag rendered points around by mouse in Stride.

[domj]

As for the intended usage: