⚠️Work in progress.
My research on cheap AI infra for RAG.
Small Local Models
| Model | Params | Storage | Layers | Context | dimensions | MRL |
|---|---|---|---|---|---|---|
| qwen3-embedding:8b | 8B | 4.7 GB | 36 | 32K | 4096 | Yes |
| qwen3-embedding:4b | 4B | 2.5 GB | 36 | 32K | 2560 | Yes |
| harrier-oss-v1-0.6b | 0.6B | 1.1 GB | 28 | 32K | 1024 | No |
| qwen3-embedding:0.6b | 0.6B | 639 MB | 28 | 32K | 1024 | Yes |
| harrier-oss-v1-270m | 270M | 512 MB | 18 | 32K | 640 | No |
API Models
| Model | Cost / 1M tokens | Provider | Open weights | Notes |
|---|---|---|---|---|
| qwen3-embedding-8b | $0.0099 | DeepInfra | Yes | |
| text-embedding-3-small | $0.020 | OpenAI | No | |
| llama-nemotron-embed-vl-1b-v2 | free | NVIDIA (OpenRouter free) | Yes | NVIDIA may use prompts for training and retain prompt data. |
| llama-nemotron-embed-vl-1b-v2 | $0.010 | DeepInfra | Yes | Paid alternative to OpenRouter free tier. |
Cheap LLMs for role play
Input and output price per per 1M
| Model | Input | Output | provider | Notes |
|---|---|---|---|---|
| DeepSeek V4 Flash | $0.09 | $0.18 | or/deepInfra | long responses |
| DeepSeek V4 Flash | $0.14 | $0.28 | deepseek | This provider may use prompts for training and may retain prompt data. |
| Gemma4 31B | $0.10 | $0.35 | or/openInterference | tool calling problems |
| Mimi 2.5 | $0.105 | $0.28 | or/DigitalOcean | |
| gpt-oss-120b | $0.03 | $0.17 | or/CoreWeave |