Azure Kinect

In terms of hardware, Azure Kinect is actually a “bundle” of 4 devices:

  • A 4K RGB camera (Color data)
  • A wide-angle depth sensor (Depth data)
  • An IMU - inertial measurement unit (Accelerometer – Gyroscope)
  • A microphone array

Spec Details

  • Dimensions: 103 x 39 x 126 mm
  • Weight: 440 g

Camera FOV (Field of View)

  • The angles that the sensors “see”.
  • This diagram shows the RGB camera in a 4:3 mode.

Camera FOV

Motion sensor (IMU)

  • The embedded Inertial Measurement Unit (IMU) is an LSM6DSMUS.
  • includes both an accelerometer and a gyroscope.
  • The accelerometer and gyroscope are simultaneously sampled at 1.6 kHz.
  • The samples are reported to the host at a 208 Hz.

Microphone Array

  • identifies as a standard USB audio class 2.0 device
  • All 7 channels can be accessed.
  • Sensitivity: -22 dBFS (94 dB SPL, 1 kHz)
  • Signal to noise ratio > 65 dB
  • Acoustic overload point: 116 dB

Azure Kinect Development Kit

Azure Kinect DK is a developer kit with advanced AI sensors that provide sophisticated computer vision and speech models. Kinect contains a depth sensor, spatial microphone array with a video camera, and orientation sensor as an all in-one small device with multiple modes, options, and software development kits (SDKs).

Supported operating systems and architectures

  • Windows 10 April 2018 (Version 1803, OS Build 17134) release (x64) or a later version
  • Linux Ubuntu 18.04 (x64), with a GPU driver that uses OpenGLv4.4 or a later version

Setup of Azure Kinect DK (Ubuntu 18.04)

Remember to stictly follow sudo command otherwise the command will produce errors.

Sensor SDK - via source

Make sure your linux is AMD64.

1
dpkg --print-architecture

For ARM64, the setup is a bit different so please refer to offical document.

You need to have cmake version >3.9

1
2
3
4
5
6
git clone https://github.com/microsoft/Azure-Kinect-Sensor-SDK.git
cd Azure-Kinect-Sensor-SDK
mkdir build && cd build
cmake .. -GNinja
ninja
sudo ninja install

You might encounter ninja: error: ‘LIBSOUNDIO_LIB-NOTFOUND’, needed by ‘bin/k4aviewer’, missing and no known rule to make it

Solve it by installing the leftover dependency and cmake .. -GNinja again. Then ninja and ninja install.

Configure Microsoft’s Package Repository

1
curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
1
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
1
sudo apt-get update
1
2
sudo apt install libk4a1.3
sudo apt install libk4a1.3-dev

The depth engine is closed source and comes with the apt package and is not included in this repo. The depth engine (DE) is a closed source binary shipped with the Linux Debian package. As an example, run apt install libk4a1.3 to install the Azure Kinect 1.3 and get the depth engine. See using the depth engine for information about versioning and adding the Microsoft’s Package Repository to your machine. NOTE This step is not need for building, but is required running the SDK.

Invoke k4aviewer in bin

Before writing a test program, lets check if we setup correctly.

To see if we setup correctly , start the Azure Kinect Viewer

1
2
whereis k4aviewer
sudo k4aviewer

Invoke k4aviewer without Root

On Linux, once attached, the device should automatically enumerate and load all drivers. However, in order to use the Azure Kinect SDK with the device and without being ‘root’, you will need to setup udev rules. We have these rules checked into this repo under ‘scripts/99-k4a.rules’.

To do so:

  • Copy ‘scripts/99-k4a.rules’ into ‘/etc/udev/rules.d/’.
  • Detach and reattach Azure Kinect devices if attached during this process.
1
2
cd /etc/udev/rules.d/
sudo vi 99-k4a.rules

99-k4a.rules File:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# Bus 002 Device 116: ID 045e:097a Microsoft Corp.  - Generic Superspeed USB Hub
# Bus 001 Device 015: ID 045e:097b Microsoft Corp.  - Generic USB Hub
# Bus 002 Device 118: ID 045e:097c Microsoft Corp.  - Azure Kinect Depth Camera
# Bus 002 Device 117: ID 045e:097d Microsoft Corp.  - Azure Kinect 4K Camera
# Bus 001 Device 016: ID 045e:097e Microsoft Corp.  - Azure Kinect Microphone Array

BUS!="usb", ACTION!="add", SUBSYSTEM!=="usb_device", GOTO="k4a_logic_rules_end"

ATTRS{idVendor}=="045e", ATTRS{idProduct}=="097a", MODE="0666", GROUP="plugdev"
ATTRS{idVendor}=="045e", ATTRS{idProduct}=="097b", MODE="0666", GROUP="plugdev"
ATTRS{idVendor}=="045e", ATTRS{idProduct}=="097c", MODE="0666", GROUP="plugdev"
ATTRS{idVendor}=="045e", ATTRS{idProduct}=="097d", MODE="0666", GROUP="plugdev"
ATTRS{idVendor}=="045e", ATTRS{idProduct}=="097e", MODE="0666", GROUP="plugdev"

LABEL="k4a_logic_rules_end"

Once complete, the Azure Kinect camera is available without being ‘root’ or ‘sudo’.

1
k4aviewer

YOU MUST DO THIS STEP IN ORDER TO USE WITH ROS.

Using the SDK

Since Visual Studio only support Windows, we need to use VSCode or other alternatives in Linux.

Adding Path in VSCode

If we include the k4a library, we should see error pops up, because we haven’t configure the includePath.

Click the Lightblob icon to edit c_cpp_properties.json file.

After successfully installing the SDK, we should see the k4a library in /usr/local/k4a directory. Therefore we add the path /usr/local/include/** into includePath.

back to main.cpp, now the error highlight should be gone.

Verifying the Library

Lets try this simple code in main.cpp.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#pragma comment(lib, "k4a.lib")
#include <k4a/k4a.h>

#include <stdio.h>
#include <stdlib.h>

int main()
{
    uint32_t count = k4a_device_get_installed_count();
    if (count == 0)
    {
        printf("No k4a devices attached!\n");
        return 1;
    }

    // Open the first plugged in Kinect device
    k4a_device_t device = NULL;
    if (K4A_FAILED(k4a_device_open(K4A_DEVICE_DEFAULT, &device)))
    {
        printf("Failed to open k4a device!\n");
        return 1;
    }

    // Get the size of the serial number
    size_t serial_size = 0;
    k4a_device_get_serialnum(device, NULL, &serial_size);

    // Allocate memory for the serial, then acquire it
    char *serial = (char*)(malloc(serial_size));
    k4a_device_get_serialnum(device, serial, &serial_size);
    printf("Opened device: %s\n", serial);
    free(serial);

    // Configure a stream of 4096x3072 BRGA color data at 15 frames per second
    k4a_device_configuration_t config = K4A_DEVICE_CONFIG_INIT_DISABLE_ALL;
    config.camera_fps       = K4A_FRAMES_PER_SECOND_15;
    config.color_format     = K4A_IMAGE_FORMAT_COLOR_BGRA32;
    config.color_resolution = K4A_COLOR_RESOLUTION_3072P;

    // Start the camera with the given configuration
    if (K4A_FAILED(k4a_device_start_cameras(device, &config)))
    {
        printf("Failed to start cameras!\n");
        k4a_device_close(device);
        return 1;
    }

    // Camera capture and application specific code would go here

    // Shut down the camera when finished with application logic
    k4a_device_stop_cameras(device);
    k4a_device_close(device);

    return 0;
}

To run the file, open a new terminal in vscode and type in:

1
g++ main.cpp -lk4a

If you are running gcc manually from the command line you have to specify the library path and library file with the -L and -l options. You may also need to specify an Include path so that gcc finds the header files.

Tutorial: Compling with g++

Recommended reading: CMake Tutorial

Recommended reading: GCC and Make - Compiling, Linking and Building C/C++ Applications

then a file a.out should be generated.

1
./a.out

Then if you have everything correctly setup, you should see the message:

1
Opened device: <your device ID>

Update Firmware of Azure Kinect DK

We might want to update our firmware.

List connected devices

1
sudo AzureKinectFirmwareTool -l

Then the shell should print something like this:

1
2
3
4
== Azure Kinect DK Firmware Tool ==
Found 2 connected devices:
0: Device "000036590812"
1: Device "000274185112"

check device firmware version

1
sudo AzureKinectFirmwareTool -q

Then the shell should print something like this:

1
2
3
4
5
6
7
8
9
== Azure Kinect DK Firmware Tool ==
Device Serial Number: 000036590812
Current Firmware Versions:
    RGB camera firmware:      1.5.92
    Depth camera firmware:    1.5.66
    Depth config file:        6109.7
    Audio firmware:           1.5.14
    Build Config:             Production
    Certificate Type:         Microsoft

Now get the Latest Firmware here(Pinned Issue)

Lets say you downloaded the firmware bin file into Downloads folder:

Inspect the firmware

1
sudo AzureKinectFirmwareTool -i Downloads/AzureKinectDK_Fw_1.6.110079014.bin

Update device firmware

1
sudo AzureKinectFirmwareTool -u Downloads/AzureKinectDK_Fw_1.6.110079014.bin

Sidenote

Actually, it is better to use a Dockerfile to ensure your machine has required dependencies.

https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/building.md

Install Sensor SDK - via apt

DONT DO THIS IF YOU HAVE ALREADY DONE Sensor SDK - via source

Besides getting it via source, you can also install it from apt.

Firstly, configure Microsoft’s Package Repository

1
curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
1
sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
1
sudo apt-get update

Then We can install the necessary packages.

1
sudo apt install k4a-tools

It will automatically downloads the latest version of libk4a1.X as well.

Don’t install Sensor SDK via both apt and source

In case you built the k4a-tools through both apt and source, you may probably encounter problem (since you have 2 k4a-tools in your system) and you need to delete one of them.

https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/1453

References

If You install in Ubuntu 20.04

Very risky, I suggest you get 18.04 instead because Azure kinect is not supported on Ubuntu 20.04 at all.
Some people found ways to get it installed on 20.04, but I have tested it and most of the things don’t work properly.

https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/1263

Install 1.3.0 instead of the latest one

1
2
3
4
5
6
7
8
$ curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
$ sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod
$ curl -sSL https://packages.microsoft.com/config/ubuntu/18.04/prod.list | sudo tee /etc/apt/sources.list.d/microsoft-prod.list
$ curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
$ sudo apt-get update
$ sudo apt install libk4a1.3-dev
$ sudo apt install libk4abt1.0-dev
$ sudo apt install k4a-tools=1.3.0

Other things should work the same.

Error you might encounter :
Error: Azure Kinect SDK Version numbers contain exactly 3 components (major.minor.rev). Requested number of components: 2
https://github.com/microsoft/Azure_Kinect_ROS_Driver/issues/143
This link show the solution.