Is there a way to clear Azure Kinect DK dlls from the memory once my application is done using the sensor, even though the application is still running?

Anirban Mookherjee 0

For a more detailed context I'm trying to run a .NET application with several 'Health check assessments' which use the Azure Kinect, but when none of the assessments are running, the sensor is not in use even though the application may be running. However, the Azure Kinect dlls seem to remain in memory and permanently increase the RAM usage by ~2GB until the entire application is shut down. This is a problem because we are running on a system without a lot of RAM to spare and our application also needs to launch Unity executables, which will have their own memory requirements and load it's own set of Azure Kinect dlls. When we do this, we go through app crashes or sensor inconsistencies due to insufficient memory.

We do uninitialize and dispose sensor after our assessments, but that does not seem to clear up memory. Is there any way this can be achieved?

LeelaRajeshSayana-MSFT 13,621 Reputation points

2023-08-03T13:06:26.4066667+00:00

Hi @Anirban Mookherjee Greetings! Welcome to Microsoft Q&A forum. Thank you for posting this question here.

I appreciate it if you can share more details on the system you are using and it's specifications. to help us better understand the issue you are facing. May I also know the dll's which you notice to be in memory?
Anirban Mookherjee 0 Reputation points

2023-08-03T20:23:57.27+00:00

The system is a Lenovo IdeaPad Gaming 3 15ACH6 and the specs are

Processor: AMD Ryzen 5 5600H, 3301 MHz, 6 Core

Total Physical Memory: 5.86 GB

Installed Physical Memory: 8 GB

GPU: GeForce RTX 3050

Adapter RAM: 1048576 bytes

Attached pic shows most of the mentioned Azure Kinect dlls
LeelaRajeshSayana-MSFT 13,621 Reputation points

2023-08-09T01:02:09.7+00:00
Hi @Anirban Mookherjee Thank you you sharing these details. I would like to know how you are using the Azure Kinect Sensor SDK from a .Net application. You can perhaps try the follwoing implementation where the resources will be dropped automatically once the code block is executed.

using (var kinect = new AzureKinectSensor()) { // Use the kinect sensor } // At this point, the kinect sensor will be automatically disposed

Let me know if this helps.

Anirban Mookherjee 0

Hi @LeelaRajeshSayana-MSFT , my implementation is below and after execution flows out of the using scope, the resources are not dropped.

using (Device device = Device.Open(id))
				{
					ImuSample imuSample = null;
					device.StartCameras(new DeviceConfiguration()
					{
						CameraFPS = TrackerSettings.fps == 15 ? FPS.FPS15 : FPS.FPS30,
						ColorFormat = ImageFormat.ColorMJPG,
						ColorResolution = ColorResolution.R720p,
						DepthMode = TrackerSettings.binned ? DepthMode.NFOV_2x2Binned : DepthMode.NFOV_Unbinned,
						WiredSyncMode = WiredSyncMode.Standalone,
						SynchronizedImagesOnly = false
					});
					device.StartImu();

					Logger.WriteLine("Open K4A device successful. id " + id + "sn:" + device.SerialNum);

					var deviceCalibration = device.GetCalibration();
					var transform = deviceCalibration.CreateTransformation();

					RiseSensorOpen(deviceCalibration, transform);
					var floorDetector = new FloorDetector(deviceCalibration);
					var floorDetected = false;
					Plane4 floorVst = new Plane4();

					using (Tracker tracker = Tracker.Create(deviceCalibration, new TrackerConfiguration()
					{
						ProcessingMode = TrackerProcessingMode.Cuda,
						ModelPath = TrackerSettings.model_filename,
						SensorOrientation = SensorOrientation.Default,
					}))
					{
						tracker.SetTemporalSmooting((float)TrackerSettings.smoothing_factor);
						Logger.WriteLine("Body tracker created.");
						while (!token.IsCancellationRequested)
						{

							if (!IsTestLaunched)
							{
								continue;
							}

							using (Capture sensorCapture = device.GetCapture())
							{
								// Queue latest frame from the sensor.

								if (sensorCapture == null)
								{
									continue;
								}

								if (sensorCapture.Depth != null)
								{
									tracker.EnqueueCapture(sensorCapture);
								}

								if (sensorCapture.Color != null)
								{
									using (var colorImage = sensorCapture.Color.Reference())
									{
										var colorWidth = colorImage.WidthPixels;
										var colorHeight = colorImage.HeightPixels;
										byte[] colorJpeg = new byte[colorImage.Size];
										var jpegLength = colorImage.Size;
										colorImage.Memory.CopyTo(colorJpeg);
										
										// Check if the jpeg data has start of the image and end of the image markers 
										// For more information : Look at `File Format Structure` section of https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format

										if (jpegLength > 4 &&  colorJpeg[0] == 255 && colorJpeg[1] == 216 && colorJpeg[jpegLength - 2] == 255 && colorJpeg[jpegLength - 1] == 217)
										{
											var colorFrame = new ColorFrameYuy()
											{
												jpeg = colorJpeg,
												timestamp = colorImage.SystemTimestampNsec / 1000 / 1000
											};

											SetColorFrame(ref colorFrame);
										}
									}

								}

							}

							if (!floorDetected)
							{
								if (!TryGetImuSample(device, out imuSample))
								{
									continue;
								}
							}

							// Try getting latest tracker frame.
							using (Frame frame = tracker.PopResult(TimeSpan.Zero, throwOnTimeout: false))
							{
								if (frame == null)
								{
									Logger.WriteLine("Pop result from tracker timeout!");
								}
								else
								{

									IsRunning = true;

									BackgroundData currentFrameData = new BackgroundData();
									var depthImage = frame.Capture.Depth.Reference();


									if (!floorDetected)
									{
										var floor = floorDetector.DetectFloorPlane(depthImage, imuSample);
										floorVst = new Plane4() { d = -floor.D, normal = new Vector3D(floor.Normal.X, -floor.Normal.Y, floor.Normal.Z) };
										//new Vector3D(0, 1, 0) };
										if (floorVst.normal.Length > 0 && floorVst.d > 0.3)
										{
											floorDetected = true;
											device.StopImu();
										}
									}

									var bodyIndexMap = frame.BodyIndexMap.Reference();
									currentFrameData.SkeletonFrame = dataConverter.ToSkeletonFrame(frame, floorVst);
									currentFrameData.BodyIndexFrame = dataConverter.ToVSTBodyIndexFrame(bodyIndexMap);
									currentFrameData.DepthImage = dataConverter.ToDepthFrame(depthImage);
									currentFrameData.Timestamp = depthImage.SystemTimestampNsec / 1000 / 1000;

									// Update data variable that is being read in the UI thread.
									SetCurrentFrameData(ref currentFrameData);
								}

							}
						}
						Logger.WriteLine("dispose of tracker now!!!!!");
						tracker.Dispose();

					}
					device.StopCameras();
					device.StopImu();
					device.Dispose();
				}

LeelaRajeshSayana-MSFT 13,621 Reputation points

2023-08-11T20:21:37.7566667+00:00

Hi @Anirban Mookherjee Thank you for sharing these details. We will investigate this further and get back to you with our findings. Please stay tuned to this thread for more updates.