Revvvvamped support for Depth Cameras

July 27, 2020 posted by: joreg

Still life with cactus

In a quest to get more basic things working out of the box with VL (ie. using vvvv beta>=40 or the all-new vvvv gamma), we took on to support your favorite depth cameras. Most of the cameras and their APIs share basically the same features as a baseline and then some of them have a few extra features. This means that using them in vvvv works mostly the same for all of them.

You have the main device node that you connect ColorImage, DepthImage, PointCloud, Skeleton,… nodes to, to get the desired info out of them. See the help patches coming with the packs for details.

Here is a list and comparison of all available depth cameras with links to the respective packs on nuget.org. To learn how to use nuget packs with vvvv please watch HowTo use Nugets.

Kinect

The original Microsoft Kinect or the XBOX 360 that was released a bit later. Get the VL pack on nuget.org. Created with support by chaupowuser.

Pros

  • everyone has one at home
  • good enough for quick prototyping
  • can connect multiple to one PC

Cons

  • no longer produced
  • complicated cabling
  • skeleton tracking requires T-Pose to initialize

Kinect v2

The second version of the Microsoft Kinect. Get the VL pack on nuget.org. Created with support by ravazquezuser.

Pros

  • everyone has one at home
  • quite robust and performant skeleton tracking

Cons

  • no longer produced
  • complicated cabling
  • can only connect one per PC

Azure Kinect

The third version. AzureKinect. Get the VL pack on nuget.org. Get the VL pack for skeleton tracking on nuget.org.

Pros

  • easy cabling
  • does not require extra drivers to be installed

Cons

  • to this point rather slow skeleton tracking as compared to Kinect v2

Orbbec Astra

Orbbec Astra. Get the VL pack on nuget.org.

Pros

  • easy cabling
  • small form factor

Cons

  • requires extra license for skeleton tracking

Intel RealSense

Intel RealSense. Get the VL pack on nuget.org.

Pros

  • easy cabling
  • smallest form factor
  • highly configurable
  • built-in depth image filters

Cons

  • Skeleton tracking for now only in connection with Nuitrack (see below)

Nuitrack API

Nuitrack is a piece software that works with all of the above cameras and provides skeleton, hand and face tracking. Get the VL pack on nuget.org. Created with support by ravazquezuser.

Pros

  • a unified API for all of the above cameras

Cons

  • requires extra license for skeleton tracking
  • can only handle one camera at a time
  • weird setup and configuration

Leap Motion Controller

The Leap Motion Controller device provides hand and finger tracking. Get the VL pack on nuget.org.

Pros

  • for precise and close-up hand and finger tracking

Cons


Please help us improve this list of pros and cons. Know any other or disagree with some mentioned, please add them in the comments! This could eventually grow into a page of the gray book.


Comments:

Comments are no longer accepted for this post.

knoeterich
09.11.2020 - 19:21
Thanks a lot for implementing those cameras!!! For my course of studies I am currently exploring the use of depthcameras in a creative sense. I also got access to a zed. I found that the zed has far more reach then orbec ( like up to 20m!!!) which would be a great plus in terms of beeing able to cover greater distances. Is there a possibility that the zed can be implemented in here too?
joreg
09.11.2020 - 21:37
reminds me, there is actually: https://github.com/m-box-de/VL.ZEDCamera @rayment can you tell us about the status of this?
Rayment
11.11.2020 - 15:08
Hi! With this one you can do the basics only - retrieve an image and/or depth in any provided format. It returns OpenCV image. Unfortunately I haven’t test with latest gamma versions. You can give it a try - if something is not working, please open an issue. I’m also planning to update it to Stride and latest SDK with full functionality since we are getting a ZED 2, but it will take some time.

Contact


Imprint

vvvv - Dießl & Gregor GbR
Oranienstrasse 10
10997 Berlin/Germany
VAT: DE275566955

groupӘvvvv.org

Follow us

Mastodon.social
Pixelfed.social
Github
Vimeo
Youtube

Sign up for our Newsletter

Your subscription could not be saved. Please try again.
Your subscription has been successful.