Run My AI Model: The Ultimate AI Hardware Calculator

Overview

Run My AI Model is a high-performance hardware calculator designed to help users determine which AI models (LLMs) can run on their specific computer hardware. It simplifies the complex task of matching parameter size, architecture, and memory needs with local hardware capabilities like GPU VRAM and System RAM.

The Challenge

The primary technical hurdle was engineering a calculation engine capable of mapping thousands of AI model variations—spanning different parameter sizes (8B, 70B, etc.), quantization levels (Q4_K_M, IQ4_XS), and context windows—against heterogeneous hardware configurations. Ensuring sub-second filtering through this dense technical data while maintaining a smooth user experience was critical.

The Solution

I built a hardware-aware calculator using SvelteKit that accounts for:

• Quantization Logic: Precise memory overhead estimates for various GGUF/EXL2 quantization levels.
• Performance Estimation: Real-time analysis of "CPU offloading" penalties when models exceed VRAM.
• Unified Memory Support: Special handling for Apple Silicon architectures.

Key Features

• Hardware-Based Filtering: Select your GPU or input custom VRAM values to see compatible models instantly.
• Quantization Guides: Detailed breakdowns of how bit-depth affects both model size and quality.
• Educational Resource: Integrated guides on VRAM vs. System RAM and PCIe bottlenecks.

Tech Stack

• SvelteKit: For a lightning-fast, reactive user interface.
• TypeScript: Ensuring type safety across complex calculation logic.
• Tailwind CSS: For a clean, professional, and responsive design.
• Vite: Empowering the development workflow with blazing-fast builds.