Glossary · Phase 6 · Modern LLM engineering
Hugging Face Hub
The main website for sharing open models and datasets, each with a model card.
Try it now
The formula from this section, with about 1 GB for the runtime. Negative room to spare means it won't fit.
Learn it properly
Running open models locally · Download, quantize and run a model on your own machine. About 30 minutes.
Other terms from the same lesson
- llama.cpp: A fast open-source program for running quantized models on ordinary computers, including laptops without a big GPU.
- Ollama: A free app for downloading and running open models on your own computer with one command.
- unified memory: Memory shared by the CPU and GPU, as on Apple Silicon Macs, so a large model can use most of the computer's memory.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.