您现在访问的是微软AZURE全球版技术文档网站,若需要访问由世纪互联运营的MICROSOFT AZURE中国区技术文档网站,请访问 https://docs.azure.cn.

使用 Azure Kinect 传感器 SDK 图像转换Use Azure Kinect Sensor SDK image transformations

遵循特定的函数在 Azure Kinect DK 中的协调相机系统之间使用和转换图像。Follow the specific functions to use and transform images between coordinated camera systems in your Azure Kinect DK.

k4a_transformation 函数k4a_transformation functions

所有带有 k4a_transformation 前缀的函数针对整个图像运行。All functions prefixed with k4a_transformation operate on whole images. 它们需要通过 k4a_transformation_create() 获取的转换句柄 k4a_transformation_t,并通过 k4a_transformation_destroy() 取消分配.They require the transformation handle k4a_transformation_t obtained via k4a_transformation_create() and are unallocated via k4a_transformation_destroy(). 你还可以参考 SDK 转换示例,其中演示了如何使用本主题所述的三个函数。You can also refer to the SDK Transformation Example that demonstrates how to use the three functions in this topic.

本文将介绍以下函数:The following functions are covered:

k4a_transformation_depth_image_to_color_camerak4a_transformation_depth_image_to_color_camera

概述Overview

函数 k4a_transformation_depth_image_to_color_camera() 将深度图从深度相机的视点转换为彩色相机的视点。The function k4a_transformation_depth_image_to_color_camera() transforms the depth map from the viewpoint of the depth camera into the viewpoint of the color camera. 此函数旨在生成所谓的 RGB-D 图像,其中,D 表示录制深度值的附加图像通道。This function is designed to produce so-called RGB-D images, where D represents an additional image channel recording the depth value. 在下图中可以看到,k4a_transformation_depth_image_to_color_camera() 的彩色图像和输出如同它们取自同一视点(即,彩色相机的视点)。As seen in the figure below, the color image and the output of k4a_transformation_depth_image_to_color_camera() look as if they were taken from the same viewpoint, that is, the viewpoint of the color camera.

图像转换

实现Implementation

此转换函数比单纯针对每个像素调用 k4a_calibration_2d_to_2d() 更为复杂。This transformation function is more complex than simply calling k4a_calibration_2d_to_2d() for every pixel. 它将深度相机几何结构中的三角网格扭曲成彩色相机的几何结构。It warps a triangle mesh from the geometry of the depth camera into the geometry of the color camera. 使用三角网格可以避免转换的深度图像出现孔洞。The triangle mesh is used to avoid generating holes in the transformed depth image. Z 缓冲区确保正确处理遮挡物。A Z-buffer ensures that occlusions are handled correctly. 默认已为此函数启用 GPU 加速。GPU acceleration is enabled for this function by default.

parametersParameters

输入参数是转换句柄和深度图像。Input parameters are the transformation handle and a depth image. 深度图像的分辨率必须与创建转换句柄时指定的 depth_mode 相匹配。The depth image resolution must match the depth_mode specified at creation of the transformation handle. 例如,如果转换句柄是使用 1024x1024 K4A_DEPTH_MODE_WFOV_UNBINNED 模式创建的,则深度图像的分辨率必须是 1024x1024 像素。For example, if the transformation handle was created using the 1024x1024 K4A_DEPTH_MODE_WFOV_UNBINNED mode, the resolution of the depth image must be 1024x1024 pixels. 输出是需要由用户调用 k4a_image_create() 分配的已转换深度图像。The output is a transformed depth image that needs to be allocated by the user via calling k4a_image_create(). 已转换的深度图像的分辨率必须与创建转换句柄时指定的 color_resolution 相匹配。The resolution of the transformed depth image must match the color_resolution specified at creation of the transformation handle. 例如,如果颜色分辨率设置为 K4A_COLOR_RESOLUTION_1080P,则输出图像的分辨率必须是 1920x1080 像素。For example, if the color resolution was set to K4A_COLOR_RESOLUTION_1080P, the output image resolution must be 1920x1080 pixels. 输出图像步幅设置为 width * sizeof(uint16_t),因为图像存储 16 位深度值。The output image stride is set to width * sizeof(uint16_t), as the image stores 16-bit depth values.

k4a_transformation_depth_image_to_color_camera_customk4a_transformation_depth_image_to_color_camera_custom

概述Overview

函数 k4a_transformation_depth_image_to_color_camera_custom() 将深度图和自定义图像从深度相机的视点转换为彩色相机的视点。The function k4a_transformation_depth_image_to_color_camera_custom() transforms the depth map and a custom image from the viewpoint of the depth camera into the viewpoint of the color camera. 作为 k4a_transformation_depth_image_to_color_camera() 的扩展,此函数旨在生成对应的自定义图像,其中每个像素与除了转换的深度图像之外的彩色相机的相应像素坐标相匹配。As an extension of k4a_transformation_depth_image_to_color_camera(), this function is designed to produce a corresponding custom image for which each pixel matches the corresponding pixel coordinates of the color camera additional to the transformed depth image.

实现Implementation

此转换函数生成转换的深度图像的方式与 k4a_transformation_depth_image_to_color_camera() 相同。This transformation function produces the transformed depth image the same way as k4a_transformation_depth_image_to_color_camera(). 若要转换自定义图像,此函数提供了使用线性内插或最近的邻域内插的选项。To transform the custom image, this function provides options of using linear interpolation or nearest neighbor interpolation. 使用线性内插可以在转换后的自定义图像中创建新值。Using linear interpolation could create new values in the transformed custom image. 使用最近的邻域内插将防止原始图像中不存在的值出现在输出图像中,但会导致图像不太平滑。Using nearest neighbor interpolation will prevent values not present in the original image from appearing in the output image but will result in less smooth image. 自定义图像应为单通道 8 位或 16 位。The custom image should be single channel 8-bit or 16-bit. 默认已为此函数启用 GPU 加速。GPU acceleration is enabled for this function by default.

parametersParameters

输入参数为转换句柄、深度图像、自定义图像和内插类型。Input parameters are the transformation handle, a depth image, a custom image and the interpolation type. 深度图像和自定义图像的分辨率必须与创建转换句柄时指定的 depth_mode 相匹配。The depth image and custom image resolution must match the depth_mode specified at creation of the transformation handle. 例如,如果转换句柄是使用 1024x1024 K4A_DEPTH_MODE_WFOV_UNBINNED 模式创建的,则深度图像和自定义图像的分辨率必须是 1024x1024 像素。For example, if the transformation handle was created using the 1024x1024 K4A_DEPTH_MODE_WFOV_UNBINNED mode, the resolution of the depth image and custom image must be 1024x1024 pixels. interpolation_type 应为 K4A_TRANSFORMATION_INTERPOLATION_TYPE_LINEARK4A_TRANSFORMATION_INTERPOLATION_TYPE_NEARESTThe interpolation_type should be either K4A_TRANSFORMATION_INTERPOLATION_TYPE_LINEAR or K4A_TRANSFORMATION_INTERPOLATION_TYPE_NEAREST. 输出是转换后的深度图像和转换后的自定义图像,需要由用户通过调用 k4a_image_create() 来分配。The output is a transformed depth image and a transformed custom image that need to be allocated by the user via calling k4a_image_create(). 转换后的深度图像和转换后的自定义图像的分辨率必须与创建转换句柄时指定的 color_resolution 相匹配。The resolution of the transformed depth image and transformed custom image must match the color_resolution specified at creation of the transformation handle. 例如,如果颜色分辨率设置为 K4A_COLOR_RESOLUTION_1080P,则输出图像的分辨率必须是 1920x1080 像素。For example, if the color resolution was set to K4A_COLOR_RESOLUTION_1080P, the output image resolution must be 1920x1080 pixels. 输出深度图像步幅设置为 width * sizeof(uint16_t),因为该图像存储 16 位深度值。The output depth image stride is set to width * sizeof(uint16_t), as the image stores 16-bit depth values. 输入的自定义图像和转换后的自定义图像的格式必须为 K4A_IMAGE_FORMAT_CUSTOM8K4A_IMAGE_FORMAT_CUSTOM16,应相应设置对应的图像步幅。The input custom image and transformed custom image must be of format K4A_IMAGE_FORMAT_CUSTOM8 or K4A_IMAGE_FORMAT_CUSTOM16, corresponding image stride should be set accordingly.

k4a_transformation_color_image_to_depth_camerak4a_transformation_color_image_to_depth_camera

概述Overview

函数 k4a_transformation_color_image_to_depth_camera() 将彩色图像从彩色相机的视点转换为深度相机的视点(参阅上图)。The function k4a_transformation_color_image_to_depth_camera() transforms the color image from the viewpoint of the color camera into the viewpoint of the depth camera (see figure above). 使用此函数可以生成 RGB-D 图像。It can be used to generate RGB-D images.

实现Implementation

对于深度图的每个像素,该函数使用像素的深度值来计算彩色图像中的相应子像素坐标。For every pixel of the depth map, the function uses the pixel’s depth value to compute the corresponding subpixel coordinate in the color image. 然后,我们将在彩色图像中的此坐标处查找颜色值。We then look up the color value at this coordinate in the color image. 在彩色图像中执行双线性内插,以获取子像素精度的颜色值。Bilinear interpolation is performed in the color image to obtain the color value at subpixel precision. 没有关联的深度读数的像素将分配到输出图像中的 BGRA 值 [0,0,0,0]A pixel that does not have an associated depth reading is assigned to a BGRA value of [0,0,0,0] in the output image. 默认已为此函数启用 GPU 加速。GPU acceleration is enabled for this function by default. 由于此方法会在转换的彩色图像中产生孔洞,并且不会处理遮挡物,因此我们建议改用函数 k4a_transformation_depth_image_to_color_camera()As this method produces holes in the transformed color image and does not handle occlusions, we recommend using the function k4a_transformation_depth_image_to_color_camera() instead.

parametersParameters

输入参数是转换句柄、深度图像和彩色图像。The input parameters are the transformation handle, a depth image, and a color image. 深度图像和彩色图像的分辨率必须与创建转换句柄时指定的 depth_mode 和 color_resolution 相匹配。The resolutions of depth and color images must match the depth_mode and color_resolution specified at creation of the transformation handle. 输出是需要由用户调用 k4a_image_create() 分配的已转换彩色图像。The output is a transformed color image that needs to be allocated by the user via calling k4a_image_create(). 已转换的彩色图像的分辨率必须与创建转换句柄时指定的 depth_resolution 相匹配。The resolution of the transformed color image must match the depth_resolution specified at creation of the transformation handle. 输出图像存储四个 8 位值(表示每个像素的 BGRA)。The output image stores four 8-bit values representing BGRA for every pixel. 因此,图像的步幅为 width * 4 * sizeof(uint8_t)Therefore, the stride of the image is width * 4 * sizeof(uint8_t). 数据顺序为像素交错式,即,蓝色值 - 像素 0,绿色值 - 像素 0,红色值 - 像素 0,alpha 值 - 像素 0,蓝色值 - 1,依此类推。The data order is pixel interleaved, that is, blue value - pixel 0, green value - pixel 0, red value - pixel 0, alpha value - pixel 0, blue value - pixel 1 and so on.

k4a_transformation_depth_image_to_point_cloudk4a_transformation_depth_image_to_point_cloud

概述Overview

函数 k4a_transformation_depth_image_to_point_cloud() 将相机拍摄的 2D 深度图转换为同一相机的坐标系中的 3D 点云。The function k4a_transformation_depth_image_to_point_cloud() converts a 2D depth map taken by a camera into a 3D point cloud in the coordinate system of the same camera. 因此,相机可以是深度相机或彩色相机。The camera can thereby be the depth or color camera.

实现Implementation

该函数的结果与针对每个像素运行 k4a_calibration_2d_to_2d() 的结果相同,不过计算效率更高。The function gives equivalent results to running k4a_calibration_2d_to_2d() for every pixel, but is computationally more efficient. 调用 k4a_transformation_create() 时,我们会预先计算一个所谓的 xy 查找表,用于存储每个图像像素的 x 和 y 比例因子。When calling k4a_transformation_create(), we precompute a so-called xy-lookup table that stores x- and y-scale factors for every image pixel. 调用 k4a_transformation_depth_image_to_point_cloud() 时,我们会通过将像素的 x 比例因子与像素的 Z 坐标相乘,来获取像素的 3D X 坐标。When calling k4a_transformation_depth_image_to_point_cloud(), we obtain a pixel’s 3D X-coordinate by multiplying the pixel’s x-scale factor with the pixel’s Z-coordinate. 类似地,与 y 比例因子相乘可以计算出 3D Y 坐标。Analogously, the 3D Y-coordinate is computed by multiplication with the y-scale factor. SDK 的快速点云示例演示了如何计算 xy 表。The fast point cloud example of the SDK demonstrates how the xy-table is computed. 例如,用户可以遵循示例代码实现其自有版本的此函数,以加速其 GPU 管道。Users can follow the example code to implement their own version of this function, for example, to speed up their GPU pipeline.

parametersParameters

输入参数是转换句柄、相机说明符和深度图像。The input parameters are the transformation handle, a camera specifier, and a depth image. 如果相机说明符设置为深度,则深度图像的分辨率必须与创建转换句柄时指定的 depth_mode 相匹配。If the camera specifier is set to depth, the resolution of the depth image must match the depth_mode specified at creation of the transformation handle. 否则,如果说明符设置为彩色相机,则分辨率必须与所选 color_resolution 的分辨率相匹配。Otherwise, if the specifier is set to the color camera, the resolution must match the resolution of the chosen color_resolution. 输出参数是需要由用户调用 k4a_image_create() 分配的 XYZ 图像。The output parameter is an XYZ image that needs to be allocated by the user via calling k4a_image_create(). 该 XYZ 图像的分辨率必须与输入的深度图的分辨率相匹配。The XYZ image resolution must match the resolution of the input depth map. 我们将为每个像素存储三个带符号的 16 位坐标值(以毫米为单位)。We store three signed 16-bit coordinate values in millimeter for every pixel. 因此,XYZ 图像步幅设置为 width * 3 * sizeof(int16_t)The XYZ image stride is therefore set to width * 3 * sizeof(int16_t). 数据顺序为像素交错式,即,X 坐标 – 像素 0,Y 坐标 – 像素 0,Z 坐标 – 像素 0,X 坐标 – 像素 1,依此类推。The data order is pixel interleaved, that is, X-coordinate – pixel 0, Y-coordinate – pixel 0, Z-coordinate – pixel 0, X-coordinate – pixel 1 and so on. 如果无法将某个像素转换为 3D,该函数将为该像素分配值 [0,0,0]If a pixel cannot be converted to 3D, the function assigns the values [0,0,0] to the pixel.

示例Samples

转换示例Transformation example

后续步骤Next steps

了解如何使用 Azure Kinect 传感器 SDK 图像转换函数后,接下来还可以了解Now you know how to use Azure Kinect sensor SDK image transformation functions, you also can learn about

此外,可以了解Also you can review

坐标系Coordinate systems