Discrepency between bone_list, JSON output and online documentation of joint connections, which to consider?

Ridwan Whitehead 0

Hi all,

I am a researcher using Azure kinects to analyze posture behavior. I cloned the Azure Kinect Samples repo from GitHub, then built and used the offline_processor to work on my own .mkv videos captured using azure kinects to obtain body tracking output. On a test example I was working on, I realized some sort of issue. The JSON output for the body tacking includes a bone_list matrix array that has names of joints. The JSON output gave me 31 joints, however, the joint positions that is outputted includes 32 rows with values, indicating that there are 32 joints. When I looked up the official documentation (https://learn.microsoft.com/en-us/azure/kinect-dk/body-joints) there seems to be 32 joints (0 to 31). Now, the reason I am confused is I am not sure what to consider while determine what coordinates refer to which joints. I assume as well that the bone_list is probably not incomplete as well as the ordering is different, which is needed to identify corresponding coordinates.

I would appreciate any insight on what to consider. Below I have some screenshots, I am using R to process my data.

User's image

Thanks!

1 answer

LeelaRajeshSayana-MSFT 13,621 Reputation points

2023-10-11T20:52:02.83+00:00

Hi @Ridwan Whitehead Greetings! Welcome to Microsoft Q&A forum. Thank you for posting this question here.

The JSON output gave me 31 joints, however, the joint positions that is outputted includes 32 rows with values, indicating that there are 32 joints.

Can you share more details on how you are capturing the body tracking joints though your code and pushing into a JSON file. What I believe is happening is that you are getting two sets of data here. One is the joint name data which would have 32 records in it. The other set of data is the parent joint data which would have 31 records in it. This is indicated in the Joint hierarchy section of the documentation.

I also recommend reviewing the documentation on Get body tracking results where you can access body frame details which would give you the joint position and orientation details. Kindly observe the results using this approach and let us know if you notice any improvements.
Please sign in to rate this answer.
Ridwan Whitehead 0 Reputation points

2023-10-11T21:44:20.34+00:00

Hi @LeelaRajeshSayana-MSFT Thank you for the response!

I am not using any code for the body tracking. I am building from this repo: https://github.com/microsoft/Azure-Kinect-Samples and using the offline_processor to run my analysis using the offline_processor.exe <input_mkv_file> <output_json_file> code. The output I get is a single nested JSON file with frame by frame coordinates for both "joint_orientations" and "joint_positions". The nested JSON file also included a "bone_list", which is what I showed in the snippet in my original post, with the two columns V1 and V2. Unfortuantely, I could not make much sense of the documentation as they seem to not be matching.

I would take the "bone_list" as reference for my joint names, but it seems that the joint output has 32 rows of values and this list has only 31. So I can't as well take what is provided nested in the JSON file as a resource.

What I would be delighted to know is how do I determine which joints the output of coordinates are referring to, for example, the second snippet I have in my original post, where I print an output for a frame.

One additional note, I do have two separate outputs that I get, "joint_orientations" and "joint_positions" - but both these still have 32 rows each, while my bone_list has 31. Below you can see a snippet of the joint_orientation output:

> print(bodyid2_positions$joint_orientations[1]) [[1]] [,1] [,2] [,3] [,4] [1,] 0.67693049 0.06140104 -0.0456469655 -0.73205972 [2,] 0.73520595 0.05741172 -0.0505729765 -0.67351210 [3,] 0.76884764 -0.03232342 0.1497995257 -0.62079680 [4,] 0.81615520 -0.02020640 0.1519091874 -0.55714095 [5,] 0.64218664 -0.50682449 -0.4967746139 -0.28972456 [6,] 0.60214347 -0.63296419 -0.4834025502 0.05569324 [7,] -0.44455165 0.79644293 -0.0003732199 -0.40994203 [8,] 0.42674690 0.87810409 -0.1553762853 -0.15059395 [9,] 0.29387709 0.83779424 0.3054658175 -0.34413335 [10,] 0.29387709 0.83779424 0.3054658175 -0.34413335 [11,] -0.40975547 0.89328241 -0.0032858974 -0.18476005 [12,] 0.33128697 0.71943635 -0.4973602891 0.35396761 [13,] -0.16855192 -0.68087524 0.6992070079 -0.13823472 [14,] -0.07628259 -0.23740761 0.9466356635 -0.20420465 [15,] -0.30033091 0.20535789 0.6933548450 0.62200379 [16,] -0.10424017 -0.00750976 0.7230880857 0.68280399 [17,] -0.10424017 -0.00750976 0.7230880857 0.68280399 [18,] 0.59384590 -0.05058148 -0.7986960411 -0.08290535 [19,] 0.53642893 0.10854553 0.0201923735 -0.83669245 [20,] 0.89069170 0.10338095 -0.0385898426 -0.44101194 [21,] 0.89297980 0.09003723 -0.0301073398 -0.43997043 [22,] 0.94308347 0.04448269 -0.0901398659 0.31700113 [23,] 0.95335233 0.11967187 0.0874838755 0.26295373 [24,] 0.86027920 0.13741314 0.0554926768 0.48780936 [25,] 0.80181444 0.29489246 0.1063490212 0.50874543 [26,] 0.21020357 0.28969759 -0.1333576441 0.92417836 [27,] 0.89099288 0.10297281 0.1907866299 -0.39890945 [28,] 0.89099288 0.10297281 0.1907866299 -0.39890945 [29,] 0.91209871 0.20771930 0.0620937534 0.34795550 [30,] 0.39890945 0.19078663 -0.1029728130 0.89099288 [31,] 0.91209871 0.20771930 0.0620937534 0.34795550 [32,] 0.89099288 0.10297281 0.1907866299 -0.39890945

LeelaRajeshSayana-MSFT 13,621 Reputation points

2023-10-24T23:22:20.68+00:00

Hi @Ridwan Whitehead I am not sure what is causing the SDK to return this result set. I would advise you to create an issue on GitHub repository link Azure Kinect Sensor SDK page and see if the product team can help you with additional feedback.

Please do note that the SDK is no longer actively maintained by the team. There might be significant delays in getting a response on this issue. I understand that this situation may have caused some inconvenience, and I want to express my gratitude for your efforts and patience in handling it.
Sign in to comment