All projects
PrototypeArtificial Intelligencevisibility: publicIndependent project
Local AI & Model Experimentation
Running and troubleshooting local LLM inference on Linux
Experimental.
- Role
- Experimenter
- Links
- Source is private or sanitized; no public repository is linked.
- Reading mode
Project overview
Hands-on experimentation with running open models locally: installing inference runtimes, managing GPU and driver configuration on Linux, comparing quantized model variants and exploring how local models could back private automation workflows.
Problem and context
Understanding model deployment constraints firsthand informs better AI workflow design and data-privacy decisions.
Objectives
- Run models locally
- Diagnose environment issues
- Compare latency and quality across variants
My contribution
- Environment setup, troubleshooting and comparison notes
Technology stack
- Linux
- Python
- Local LLM runtimes
- GPU drivers
- Quantized models
- Bash
Results
- Working local inference environment
Limitations
- Experimental; no formal benchmarks published