Linux Installation Guide
Set up AVA SDK on native Linux (Ubuntu 24.04) with Docker and NVIDIA Container Toolkit. No WSL2, no Windows — pure Linux.
AVA SDK on Native Linux
While Lesson 01 covers WSL2 setup on Windows, running AVA SDK directly on Linux offers better performance, lower latency, and direct hardware access. This guide covers the complete installation on Ubuntu 24.04 LTS, the recommended operating system for AVA SDK development.
If you use Windows, check out Lesson 01: Execution Environment for the WSL2 setup guide.
Prerequisites
Step-by-Step Installation
Step 1: Install Docker Engine
Install Docker Engine directly on Ubuntu. Unlike WSL2, you don't need Docker Desktop or additional network configuration.
# Remove old versions if they exist
sudo apt remove docker docker-engine docker.io containerd runc
# Install dependencies
sudo apt update
sudo apt install -y ca-certificates curl
# Add Docker's official GPG key
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
# Add repository
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Install Docker
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
# Add your user to the docker group (avoids using sudo for every command)
sudo usermod -aG docker $USER
newgrp dockerStep 2: Install NVIDIA Container Toolkit
The NVIDIA Container Toolkit allows Docker to access your NVIDIA GPU. This is what makes local AI model execution with hardware acceleration possible.
# Add NVIDIA repository
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
# Install
sudo apt update
sudo apt install -y nvidia-container-toolkit
# Configure Docker to use the NVIDIA runtime
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
# Verify installation
docker run --rm --gpus all nvidia/cuda:12.1.0-base-ubuntu22.04 nvidia-smiStep 3: Clone and Configure AVA SDK
Download the Razer AI Kit repository and set up the execution environment with Docker Compose.
# Clone the official repository
git clone https://github.com/razerofficial/aikit.git
cd aikit
# Build and start services with Docker Compose
docker compose up -d
# Verify containers are running
docker compose psStep 4: Run Your First Inference
Once the containers are running, execute the test script to verify everything works correctly.
# Enter the development container
docker compose exec dev bash
# Inside the container, run the inference test
python scripts/inference_test.py
# If everything works, you'll see the model's response in the terminalNative Linux vs WSL2
Running AVA SDK on native Linux offers several advantages over WSL2 on Windows. If you have the option, Linux is the recommended environment for development and production.
Direct GPU Access
No virtualization layer between the OS and GPU. Lower latency and better performance.
Memory Management
Linux manages RAM without WSL2's default 50% cap. Large models benefit significantly from this.
Native Docker
No need for Docker Desktop or bridged network configurations. Docker runs directly on the kernel.
Fewer Layers
No Windows→WSL→Linux→Docker translation chain. Every removed layer is one less failure point and a performance gain.
Troubleshooting
Error: "could not select device driver"
This means the NVIDIA Container Toolkit is not properly configured. Run sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker and try again.
Docker won't start after installation
Check that the service is active: sudo systemctl status docker. If inactive, start it with sudo systemctl enable --now docker.
Permission denied when running docker without sudo
Make sure you added your user to the docker group: sudo usermod -aG docker $USER. Log out and back in for changes to take effect.
GPU not showing in nvidia-smi inside the container
Check that NVIDIA drivers are installed on the host: nvidia-smi. If it doesn't work on the host, install drivers first: sudo apt install nvidia-driver-550.
Out of disk space during model download
Language models can be 5-30 GB each. Make sure you have at least 50 GB free. You can change the model download directory in the Docker Compose configuration.
All set?
Continue with the technical documentation and learn to customize AVA SDK.
Explore DocumentationLinux FAQ
Can I use a distribution other than Ubuntu?
Yes, although Ubuntu 24.04 LTS is the recommended and tested distribution. Fedora, Debian, and Arch Linux also work, but package installation commands will vary. The Docker and NVIDIA Container Toolkit setup is common across all distributions.
Does AVA SDK work on Linux without an NVIDIA GPU?
Not practically. The Razer AI Kit requires CUDA acceleration, which is only available on NVIDIA GPUs. AMD GPUs (ROCm) and Intel GPUs (OneAPI) are not currently supported.
What performance can I expect on Linux vs WSL2?
On native Linux you can expect 5-15% better inference performance compared to WSL2, mainly due to the removal of the virtualization layer and more efficient memory management.
Can I run AVA SDK on a VPS or cloud server?
Yes, as long as the VPS has a dedicated NVIDIA GPU. Providers like Lambda Labs, Vast.ai, and RunPod offer GPU instances at competitive prices. Just install Docker and NVIDIA Container Toolkit following this same guide.