Razer AI Kit · Local AI

The Development Hub forRazer AVA

Community for developers and enthusiasts of the Razer AI Kit: local AI, Linux, and Razer AVA customization. Join to receive updates and be ready when Razer opens its ecosystem.

WHAT IS THE RAZER AI KIT?

Razer AIKit is an open-source, preview-stage AI development environment that brings cloud-grade performance to your desktop. Run large language models (LLMs) locally on NVIDIA GPUs, scale inference across a GPU cluster with Ray and vLLM, fine-tune with LoRA via LlamaFactory, and explore 300,000+ models from Hugging Face — all through a Docker Compose stack with Open WebUI, Jupyter Lab, Grafana and Prometheus.

▸Local-first AI: run LLMs on-device for total privacy and zero latency

▸Multi-GPU scaling with Ray for distributed inference

▸vLLM engine with PagedAttention for memory-efficient serving

▸LoRA fine-tuning with LlamaFactory

▸NVIDIA GPUs (Compute Capability 7.0+): RTX 20/30/40/50, Blackwell, Hopper, Ada, Ampere

▸Ubuntu 22.04/24.04, Windows 11 via WSL2, x86-64 & ARM64

THE CORE ARCHITECTURE

vLLM Engine

High-throughput, memory-efficient LLM inference. vLLM uses PagedAttention to serve local models at state-of-the-art speed on a single GPU or a cluster.

Ray

Distributed computing framework for scaling AI across GPUs. Ray orchestrates multi-GPU inference and manages resources efficiently for smooth multitasking.

LlamaFactory

Parameter-efficient fine-tuning toolkit. Predefined recipes to fine-tune Llama and other open models with LoRA for gaming and assistance contexts.

Open WebUI

Built-in local chat interface on port 1919. Interact with your models on-device, fully offline, through a clean web UI.

Docker & Dev Tools

Docker Compose stack with Jupyter Lab (8888), Grafana (3000) and Prometheus (9090) for notebooks, monitoring and observability.

AIKit CLI

Model lifecycle management from the terminal with rzr-aikit. Clone, run and fine-tune models pulled from 300,000+ Hugging Face checkpoints.

GET READY FOR THE ERA OF HOLOGRAPHIC ASSISTANCE

Join the AVA Hologram network. Access exclusive material, availability alerts, and customization and AI guides.

SDK Documentation:Technical integration guides and AI model architecture.

Step-by-step Tutorials:Learn to configure your holographic assistant from scratch and without code.

Skins & Prompts:Early access to aesthetic customization and behavior libraries.

Availability:Be the first to know when units are available in your region.