Running an AI locally means installing a language model directly on your hard drive and chatting with it with no internet connection, no monthly subscription, and without a single one of your sentences leaving for an outside server. Ollama is the tool that makes this doable: one file to install, one command to type, and a model like Llama, Mistral, or Gemma is answering you at home within seconds.
The real question isn't «is this possible» (it is, even on a 2021 laptop) but «is it worth it for what I want to do». Here you'll find the actual minimum specs, a step-by-step install, which models to pick based on your RAM, and an honest comparison with the other ways of using AI. The one-line summary: local AI is excellent for privacy and for understanding how these models work, weaker on raw answer quality, and useless if your goal is to produce images, videos, or music.
What exactly is a local AI?
A local AI is a language model downloaded onto your computer that computes its answers using your processor and your graphics card, without going through the internet. When you use ChatGPT or Claude, your question travels to a data center, a model with several hundred billion parameters processes it, and the answer comes back. Locally, everything happens between your keyboard and your hard drive.
Two terms to know before we go further, without the pointless jargon:
- Parameters: this is the «size» of the model, expressed in billions (written 3B, 7B, 20B). The more parameters a model has, the more relevant it is, the more it weighs, and the more memory it needs.
- Quantization: a compression of the model so it fits in memory. By default, Ollama downloads so-called Q4 versions, four times lighter than the original, with a moderate loss of quality. That's what makes it possible to run a 7-billion-parameter model on an ordinary computer.
Ollama itself is not an AI. It's the program that downloads models, launches them, manages memory, and gives you an interface to talk to them. It's free, open source, and it runs on macOS, Windows, and Linux.
The three ways to use an AI when you're starting out
Before you install anything, you should know that local AI is only one option out of three, and not the fastest one for getting concrete results. Here are all three, ordered by what works best for a beginner.
Method 1: a learning environment with the real tools included
This is Skilzy's approach. You follow a guided program (writing good prompts, creating images, automating with n8n, coding with AI) and you use professional tools directly inside the built-in «AI Lab», credits included. You install nothing, you don't stack up six €20 subscriptions just to experiment, and you don't lose three evenings getting a graphics driver to cooperate.
Why this matters when you're a beginner: you learn the method at the same time as the tool. Skilzy's 15+ AI training programs start from absolute zero, and two of them lead to a state-recognized, fundable certification (RS7439 in AI content marketing, RS6792 in AI and sales).
What this method doesn't do: it won't teach you how to administer an inference server. If your goal is specifically to run a model offline on your own machine, go to method 2.
Method 2: Ollama, the model on your own machine
Free, offline, private, no message limits. This is the method the rest of this article covers in detail. It requires a decent computer, a bit of patience, and accepting one simple fact: a 7-billion-parameter model on your laptop does not reason like a cutting-edge model hosted in a data center.
Method 3: the free versions of the big chatbots
ChatGPT, Claude, and Gemini all offer free access in the browser, with models far more powerful than anything you'll run locally. The trade-off: daily quotas, and your conversations going through a server. If you're torn between the three, the detailed comparison of ChatGPT, Claude, and Gemini in their free versions will save you some time.
These three methods combine very well. Plenty of people learn with a structured program, use an online chatbot day to day, and keep Ollama for sensitive documents.
What hardware you need to run Ollama
RAM is the deciding factor: Ollama's official documentation recommends at least 8 GB of RAM for a 7-billion-parameter model, 16 GB for a 13B model, and 32 GB for a 33B model. Everything else is secondary.
In practice, here's what counts, in order of importance:
- RAM (or your graphics card's VRAM). The model has to fit entirely in memory, otherwise your system falls back on the disk and speed collapses. Budget the size of the model file plus roughly 1 to 2 GB for the rest.
- The graphics card. A recent NVIDIA card (RTX 3060 and up) or an Apple Silicon Mac (M1 through M4, where memory is shared between processor and graphics card) gives you smooth answers, somewhere around 20 to 60 words per second. Ollama detects your hardware on its own.
- CPU only. It works, genuinely, but it's slow: on a laptop with no dedicated graphics card, expect 3 to 10 words per second with a small model. Fine for proofreading a text, painful for a long conversation.
- Disk space. Each model weighs between 1 and 15 GB. Plan on 20 GB free if you want to try several.
A concrete benchmark: a MacBook Air M2 with 16 GB, a PC with 16 GB of RAM and an RTX 3060, or even an office laptop with 8 GB for the small models. Below 8 GB of RAM, stick to models in the 1-to-3-billion-parameter range.
Installing Ollama and launching your first model in 10 minutes
The install comes down to three steps: download Ollama, type one command, wait for the model to download. No programming skills required.
Step 1: install Ollama. Go to the official site and grab the version that matches your system. On macOS and Windows, it's a regular application with a chat window. On Linux, it's a single line to paste into the terminal, provided on the download page.
Step 2: open a terminal. On Mac, the Terminal app. On Windows, PowerShell or Command Prompt. Type ollama --version and hit enter: if a version number appears, you're all set.
Step 3: launch a model. Type ollama run llama3.2 then Enter. Ollama downloads about 2 GB the first time, then shows a prompt where you can type your question just like in any chat. To leave the conversation, type /bye.
Four commands are all you need day to day:
- ollama list: see your installed models
- ollama pull mistral: download a model without launching it
- ollama rm mistral: delete a model and free up disk space
- ollama ps: see what's running and how much memory it's currently using
If the terminal puts you off, Ollama also opens a small local web interface at localhost, port 11434, and free projects like Open WebUI add a ChatGPT-like presentation on top, with conversation history and file uploads. The details are in Ollama's official documentation.
Which model to choose for your machine
Always start with the smallest model that's usable on your setup, then work your way up in size for as long as the speed stays bearable. The classic beginner mistake is downloading a 20-billion-parameter model right away, watching it crawl, and concluding that local AI doesn't work.
| Model | Download size | Recommended RAM | Good for |
|---|---|---|---|
| Llama 3.2 (3B) | about 2 GB | 8 GB | First test, summaries, rewording |
| Gemma 3 (4B) | about 3 GB | 8 GB | Short texts, reading images |
| Mistral (7B) | about 4 GB | 16 GB | Writing in French, general questions |
| Qwen 3 (8B) | about 5 GB | 16 GB | Reasoning, code, multilingual |
| gpt-oss (20B) | about 14 GB | 32 GB or 16 GB of VRAM | Complex tasks, serious offline use |
Two useful notes. Models under 4 billion parameters write decent French but make up facts easily: don't ask them for dates, figures, or quotes. And none of these models has internet access: their knowledge stops at their training cutoff, and they won't read a web page for you without extra configuration.
What it's really worth: benefits and limits
Local AI wins on privacy, cost, and offline availability; it loses on answer quality, speed, and anything involving images, video, or audio. Given the question at hand, let's be direct.
What Ollama is a very good choice for:
- Documents you don't want to send to anyone: contracts, medical records, client data, internal notes. Nothing leaves your machine.
- Volume: you can process 500 product descriptions without counting credits or paying per use.
- Understanding how it works: swapping models, playing with temperature, seeing the effect of size on quality. It's genuinely educational, and it follows on from what the guide to learning AI as a beginner covers.
- Connection dropouts: trains, planes, dead zones.
What it's not the right solution for:
- Reasoning quality. A 7B model gets things wrong far more often than a cutting-edge online model on a long analysis or a multi-step calculation.
- Visual and audio creation. Ollama runs text models. For images, UGC videos, voiceovers, or music, you need other tools, often heavy to install and hungry for VRAM.
- Setup time. Budget one to two hours if your setup fights back (drivers, not enough memory, badly chosen model).
- The absence of guardrails. Nobody is going to tell you your prompt is bad. And that's where 80% of the result is decided, whatever the model.
One more thing to keep in mind if you want to turn these skills into something marketable: no local install goes on a résumé. A certification does. Skilzy's two certifications are registered in the Répertoire spécifique at France Compétences and can be funded through training-finance schemes, which changes the picture for a career-change application. If that's your situation, the article on switching careers into AI at 30, 40, or 50 lays out the possible paths.
Final word
Install Ollama, launch Llama 3.2, ask it three questions: in fifteen minutes you'll know whether your machine can keep up and whether the experience suits you. Keep local for what it does better than anyone else — your sensitive data and high volume — and use a guided environment with real tools to learn how to produce results. The two approaches aren't in competition; they cover different needs.
