Setup Azure Kinect on Ubuntu 18.04
Azure Kinect
In terms of hardware, Azure Kinect is actually a “bundle” of 4 devices:
- A 4K RGB camera (Color data)
- A wide-angle depth sensor (Depth data)
- An IMU - inertial measurement unit (Accelerometer – Gyroscope)
- A microphone array
Spec Details
- Dimensions: 103 x 39 x 126 mm
- Weight: 440 g
Camera FOV (Field of View)
- The angles that the sensors “see”.
- This diagram shows the RGB camera in a 4:3 mode.
Motion sensor (IMU)
- The embedded Inertial Measurement Unit (IMU) is an LSM6DSMUS.
- includes both an accelerometer and a gyroscope.
- The accelerometer and gyroscope are simultaneously sampled at 1.6 kHz.
- The samples are reported to the host at a 208 Hz.
Microphone Array
- identifies as a standard USB audio class 2.0 device
- All 7 channels can be accessed.
- Sensitivity: -22 dBFS (94 dB SPL, 1 kHz)
- Signal to noise ratio > 65 dB
- Acoustic overload point: 116 dB
Azure Kinect Development Kit
Azure Kinect DK is a developer kit with advanced AI sensors that provide sophisticated computer vision and speech models. Kinect contains a depth sensor, spatial microphone array with a video camera, and orientation sensor as an all in-one small device with multiple modes, options, and software development kits (SDKs).
Supported operating systems and architectures
- Windows 10 April 2018 (Version 1803, OS Build 17134) release (x64) or a later version
- Linux Ubuntu 18.04 (x64), with a GPU driver that uses OpenGLv4.4 or a later version
Setup of Azure Kinect DK (Ubuntu 18.04)
Remember to stictly follow sudo
command otherwise the command will produce errors.
Sensor SDK - via source
Make sure your linux is AMD64.
1 | dpkg --print-architecture |
For ARM64, the setup is a bit different so please refer to offical document.
You need to have cmake version >3.9
1 | git clone https://github.com/microsoft/Azure-Kinect-Sensor-SDK.git |
Solve it by installing the leftover dependency and
cmake .. -GNinja
again. Thenninja
andninja install
.
Configure Microsoft’s Package Repository
1 | curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add - |
1 | sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod |
1 | sudo apt-get update |
1 | sudo apt install libk4a1.3 |
The depth engine is closed source and comes with the apt package and is not included in this repo. The depth engine (DE) is a closed source binary shipped with the Linux Debian package. As an example, run
apt install libk4a1.3
to install the Azure Kinect 1.3 and get the depth engine. See using the depth engine for information about versioning and adding the Microsoft’s Package Repository to your machine. NOTE This step is not need for building, but is required running the SDK.
Invoke k4aviewer in bin
Before writing a test program, lets check if we setup correctly.
To see if we setup correctly , start the Azure Kinect Viewer
1 | whereis k4aviewer |
Invoke k4aviewer without Root
On Linux, once attached, the device should automatically enumerate and load all drivers. However, in order to use the Azure Kinect SDK with the device and without being ‘root’, you will need to setup udev rules. We have these rules checked into this repo under ‘scripts/99-k4a.rules’.
To do so:
- Copy ‘scripts/99-k4a.rules’ into ‘/etc/udev/rules.d/’.
- Detach and reattach Azure Kinect devices if attached during this process.
1 | cd /etc/udev/rules.d/ |
99-k4a.rules
File:
1 | # Bus 002 Device 116: ID 045e:097a Microsoft Corp. - Generic Superspeed USB Hub |
Once complete, the Azure Kinect camera is available without being ‘root’ or ‘sudo’.
1 | k4aviewer |
YOU MUST DO THIS STEP IN ORDER TO USE WITH ROS.
Using the SDK
Since Visual Studio only support Windows, we need to use VSCode or other alternatives in Linux.
Adding Path in VSCode
If we include the k4a library, we should see error pops up, because we haven’t configure the includePath.
Click the Lightblob icon to edit c_cpp_properties.json
file.
After successfully installing the SDK, we should see the k4a library in /usr/local/k4a
directory. Therefore we add the path /usr/local/include/**
into includePath
.
back to main.cpp
, now the error highlight should be gone.
Verifying the Library
Lets try this simple code in main.cpp
.
1 |
|
To run the file, open a new terminal in vscode and type in:
1 | g++ main.cpp -lk4a |
If you are running gcc manually from the command line you have to specify the library path and library file with the -L and -l options. You may also need to specify an Include path so that gcc finds the header files.
Recommended reading: CMake Tutorial
Recommended reading: GCC and Make - Compiling, Linking and Building C/C++ Applications
then a file a.out
should be generated.
1 | ./a.out |
Then if you have everything correctly setup, you should see the message:
1 | Opened device: <your device ID> |
Update Firmware of Azure Kinect DK
We might want to update our firmware.
List connected devices
1 | sudo AzureKinectFirmwareTool -l |
Then the shell should print something like this:
1 | == Azure Kinect DK Firmware Tool == |
check device firmware version
1 | sudo AzureKinectFirmwareTool -q |
Then the shell should print something like this:
1 | == Azure Kinect DK Firmware Tool == |
Now get the Latest Firmware here(Pinned Issue)
Lets say you downloaded the firmware bin file into Downloads
folder:
Inspect the firmware
1 | sudo AzureKinectFirmwareTool -i Downloads/AzureKinectDK_Fw_1.6.110079014.bin |
Update device firmware
1 | sudo AzureKinectFirmwareTool -u Downloads/AzureKinectDK_Fw_1.6.110079014.bin |
Sidenote
Actually, it is better to use a Dockerfile to ensure your machine has required dependencies.
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/building.md
Install Sensor SDK - via apt
DONT DO THIS IF YOU HAVE ALREADY DONE Sensor SDK - via source
Besides getting it via source, you can also install it from apt.
Firstly, configure Microsoft’s Package Repository
1 | curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add - |
1 | sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod |
1 | sudo apt-get update |
Then We can install the necessary packages.
1 | sudo apt install k4a-tools |
It will automatically downloads the latest version of
libk4a1.X
as well.
Don’t install Sensor SDK via both apt and source
In case you built the k4a-tools through both apt and source, you may probably encounter problem (since you have 2 k4a-tools in your system) and you need to delete one of them.
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/1453
References
- Azure Kinect DK documentation on Github(READ THIS FIRST)
- Azure Kinect DK documentation
- Notes on Setting up the Microsoft Azure Kinect on Ubuntu 18.04
If You install in Ubuntu 20.04
Very risky, I suggest you get 18.04 instead because Azure kinect is not supported on Ubuntu 20.04 at all.
Some people found ways to get it installed on 20.04, but I have tested it and most of the things don’t work properly.
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/1263
Install 1.3.0 instead of the latest one
1 | curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add - |
Other things should work the same.
Problem might related to Azure Kinect ROS Driver
Error you might encounter :
Error: Azure Kinect SDK Version numbers contain exactly 3 components (major.minor.rev). Requested number of components: 2
https://github.com/microsoft/Azure_Kinect_ROS_Driver/issues/143
This link show the solution.