Multiple Azure Kinects for body tracking

Question

Hello,
Does the Azure Kinect SDK allow multiple devices for body tracking? Does the obtained pose work with all of the inputs, or just with one? I have read about calibration of multiple devices in the docs but haven't found anything regarding body tracking with multiple A4K's.

Answer 1

Hello @jatorrasagasti , Welcome to Microsoft Q&A Platform!

Below is the response from Microsoft Product Team, Hope this helps with your query.

The body tracking engine tracks on images from 1 camera. You can have multiple cameras with an instance of Body Tracking running on each and (you do the work) merge the skeletons. We have developers doing this and using join confidence as part of the merging process.

Sorry about the document not yet published or samples available on this topic, I hope they are part of the documentation in the future.

Below are two related Github discussions on Multiple cameras used for Body tracking.
I hope these discussions will give some basic information about your query.

Github issue: Link 1

Technically you can hardware synchronize two Azure Kinect cameras and impose a 1/2 frame delay between the two cameras. If you intention is to interleave the frames from the two cameras you would need to translate them due to the physical offsets of the two cameras (since the two cameras will not be on the same axis).

Github Issue : Link 2

The Body Tracking SDK only supports images from a single camera. That said I am aware of some customers (including Microsoft internal teams) that are using multiple instances of body tracking with overlapping cameras and merging the skeletons taking into account the joint confidence. Running multiple cameras at 30fps requires understanding of the PCs USB controller architecture and bandwidth limitations. Running multiple instances of body tracking requires sufficient GPU (the Microsoft internal team is running 3 instances on a 2080).

Please let us know if you need further help in this matter.