Kinect -> Orbbec: The Femto Family

vvvv gamma question
21

Good question, @maxs, actually I haven’t tried it yet.

Search your error I came across this post by Orbbec . It concerns running bodytracking with Unity but it seems like its another forensic copy-paste job.

22

Well it seems like you got this covered @Hadasi :)

I wanted to add that we recently discovered the Bolt in particular seems to have an issue where it jumps from USB 3 to USB 2 and introduces a lot of lag. So far there seems to be no known fix: Femto Bolt sensors frequently appearing connected as USB2.1 while using USB 3.1 ports · Issue #45 · orbbec/OrbbecSDK · GitHub

We ended up switching to a Femto Mega which so far has not shown the problem.

As for the BodyTracking, I have not tried it myself, but I have come across this document which lists a few extra steps involving the registry, it might be related (read sections 2 and 3): https://orbbec.github.io/OrbbecSDK-K4A-Wrapper/src/orbbec/docs/Access_AKDK_Application_Software_with_Femto_Bolt.pdf

Give it a try and let us know how it goes :)

23

To continue adding to this, it seems that the USB issue is only happening on AMD machines.

24

I have the strange issue, that on one pc here in the studio the Femto Bolt is working in the standard Kinect DSK Viewer, but not inside VVVV. Getting this warning:

image
image1280×499 74 KB

Seems not like an Orbbec issue, but a overall windows-.net-something issue. Any ideas what i need to reinstall or update?

25

Hello there,
I tried to use a femto mega and I followed the instruction of @Hadasi but nothing happen in vvvv … liked no camera was connected…
I do have all the feed back in the k4viewer.exe
I install the driver, I have copied the dll in the nuget file, I used the win x 64 for the sdk…

Other question, is there a plan to developpe the use of this camera throught the Poe connection (big advantage for long cable installation)

Thanks you :)

26

Hello,
just to let you know
i managed to get the AzureKinect.Body working with the Femto Bolt using:

  • Orbbec Release v1.10.16
  • OrbbecSDK-K4A-Wrapper running the obsensor_metadata_win10.md commands in powershell
  • CUDA V11.2
  • cuDNN 9.6.0 (not sure if it’s mandatory)
  • Azure Kinect Body Tracking SDK v1.1.2
    and the dll copy-paste from OrbbecSDK-K4A-Wrapper to MS Azure Kinect Body Tracking SDK
4 Likes
27

Headsup: Thanks to sponsoring by Supply and Demand we’ve started work on VL.Devices.Orbbec which is now in an initial useful state in that you get a world-texture out of it. Obviously it will also get RGB, depth and IMU outputs at some point.

Tested with:

  • Femto Bolt
  • Femto Mega (usb and ethernet)
  • Gemini 335

Basis of this pack is the Orbbec v2 CSharp SDK meaning it should support all devices listed there.

11 Likes
28

hello,
works great with here with

  • Femto Bolt

looks like there is also a V2 branch for the body tracking part:

29

I also have AzureKinect.Body running with the Femto Bolt following @circuitb 's description. But I do get periodic frame drops and eventually crashes just running the help patch.

This is what the log says before it crashes:

result = K4A_RESULT_FAILED
Microsoft.Azure.Kinect.Sensor.AzureKinectException: result = K4A_RESULT_FAILED
at Microsoft.Azure.Kinect.Sensor.AzureKinectException.ThrowIfNotSuccess[T](Func1 function) at _VL_Devices_AzureKinect_Body_.Devices.AzureKinect.BodyImage_Reactive_P.<>c__DisplayClass3_0.<Update>b__0(Object s_7, Frame Input_1_ In_8) at VL.Lib.Reactive.ForEach3.OnNext(TIn input)
at VL.Lib.Reactive.ObservableInputBase`1.SafeOnNext(T input)
— End of stack trace from previous location —
at VL.Lang.Platforms.RuntimeHost.SharedStep(TimeSpan timeSinceStart)

AzureKinectException: “result = K4A_RESULT_FAILED”
StackTrace:
Microsoft.Azure.Kinect.Sensor.AzureKinectException { internal static void ThrowIfNotSuccess(System.Func<> function) { … } }
VL_Devices_AzureKinect_Body.Devices.AzureKinect.BodyImage_Reactive_P+<>c__DisplayClass3_0 { internal System.Tuple<object, VL.Li
b.Basics.Imaging.IImage> b__0(object s_7, Microsoft.Azure.Kinect.BodyTracking.Frame Input_1_In_8) { … } }
VL.Lib.Reactive.ForEach3 { protected virtual void OnNext(TIn input) { ... } } VL.Lib.Reactive.ObservableInputBase1 { private void SafeOnNext(T input) { … } }
System.Runtime.ExceptionServices.ExceptionDispatchInfo { public void Throw() { … } }
VL.Lang.Platforms.RuntimeHost { private void SharedStep(System.TimeSpan timeSinceStart) { … } }