LM Studio Integration Guide
Run local language models without a dedicated GPU. Connect LM Studio with AVA SDK for fast, private local inference.
What is LM Studio?
LM Studio is a desktop application that lets you download, run, and serve local language models (LLMs) without needing an NVIDIA GPU or complex setup. With LM Studio you can load models like Llama 3, Mistral, Qwen, and hundreds more directly from HuggingFace, and expose them through an OpenAI-compatible API. For AVA SDK, it is the simplest alternative to vLLM when you don't have access to a dedicated GPU or prefer a graphical experience.
Why LM Studio for AVA SDK?
No Dedicated GPU Needed
Works with modern CPUs and integrated GPUs. You don't need an NVIDIA RTX to get started.
Graphical Interface
Download, configure, and run models from a visual interface. No terminal, no Docker required.
OpenAI-Compatible API
Exposes models via http://localhost:1234/v1 with the same interface as ChatGPT.
Great for Prototyping
Test different models and configurations before deploying to production with vLLM.
System Requirements
- Windows 10+, macOS 12+, or Linux (recent distribution)
- 8 GB RAM (16 GB recommended for 7B models)
- 4+ core CPU
- Optional GPU (Metal, CUDA, or Vulkan acceleration)
- 10 GB free disk space
- Internet connection for downloading models
Installation and Setup
1. Download and Install LM Studio
Visit lmstudio.ai and download the version for your operating system (Windows, macOS, or Linux). Install it like any desktop application.
2. Search and Download a Model
Open LM Studio, use the built-in search to browse models on HuggingFace. We recommend starting with:
| Model | Why |
|---|---|
| Llama 3.2 3B Instruct | Fast, lightweight, ideal for testing AVA SDK. ~2 GB. |
| Mistral 7B Instruct v0.3 | Excellent quality/performance balance. ~4 GB. |
| Qwen 2.5 7B Instruct | Great for structured responses. ~4 GB. |
| Phi-3.5 mini 3.8B | Surprisingly capable for its size. ~2.5 GB. |
3. Load the Model
Select the downloaded model in the LM Studio interface. Load it in the chat tab to verify it works before starting the server.
4. Start the Local API Server
In the "Local Inference Server" tab of LM Studio:
- Select your downloaded model
- Configure the port (default: 1234)
- Adjust max context (recommended: 4096 tokens)
- Click "Start Server"
LM Studio lets you search and download models directly from HuggingFace. GGUF format models are the lightest and most compatible. Start with a small one like Llama 3.2 3B (~2 GB) to get familiar.
Connect LM Studio with AVA SDK
Once the LM Studio server is running, connect it to AVA SDK using the OpenAI client, exactly as with vLLM but pointing to the LM Studio port:
1from openai import OpenAI
2
3# AVA SDK - Conexion con LM Studio
4client = OpenAI(
5 base_url="http://localhost:1234/v1", # Puerto de LM Studio
6 api_key="not-needed" # LM Studio no requiere API key
7)
8
9response = client.chat.completions.create(
10 model="local-model", # LM Studio ignora este valor
11 messages=[
12 {"role": "system", "content": "Eres AVA, un asistente tactico de gaming."},
13 {"role": "user", "content": "Analiza esta partida y sugiere mejoras."}
14 ],
15 temperature=0.7,
16 max_tokens=512
17)
18
19print(response.choices[0].message.content)The only difference is the port: LM Studio uses :1234, vLLM uses :8000. By changing base_url you can switch between both without touching anything else.
LM Studio vs vLLM: Which to Choose?
Both are excellent, but designed for different use cases:
| Feature | LM Studio | vLLM |
|---|---|---|
| GPU Required | No (CPU + integrated GPU) | NVIDIA with 8 GB+ VRAM |
| Installation | Download and install | Docker + NVIDIA Container Toolkit |
| Performance | Moderate | High (PagedAttention) |
| Interface | Graphical (GUI) | Terminal / API |
| Supported Models | GGUF (hundreds) | AWQ/GPTQ/Safetensors (dozens) |
| Recommended Use | Prototyping, CPU, limited GPUs | Production, powerful GPUs |
You don't have to choose just one. Many developers use LM Studio for rapid prototyping and vLLM for production. Both are compatible with AVA SDK.
Optimization Tips
GPU Offloading
In LM Studio you can control how many model layers run on the GPU. For low-VRAM GPUs, lower offloading to 50%.
Quantization
GGUF models come pre-quantized. Start with Q4_K_M for a good quality/performance balance.
Context Length
Reduce max context if the model responds slowly. 2048 tokens is enough for most AVA tasks.
Prompt Caching
LM Studio caches processed prompts. Repeated similar queries will be faster.
Troubleshooting
Server won't start
Check that port 1234 is not in use. Change the port in settings if needed.
Very slow responses
Try a smaller model (3B instead of 7B) or reduce max context to 2048.
Out of memory error
Close other applications. 7B models require at least 16 GB RAM.
API not responding
Confirm the server shows green in LM Studio. Test with curl http://localhost:1234/v1/models
Want to access the Academy?
Sign up to access development guides, tutorials, and exclusive AVA SDK resources.
Access the Academy