2 days · Session

Ship Local AI in a Weekend

Two days. One offline AI tool that actually does something for you. Not a tech demo.

For: Builders who want to try local AI hands-on without renting a GPU or trusting a cloud API.

The timeline

  1. Saturday morningSaturday AM

    Install Ollama, pull a small model, and pick one real use case.

  2. Saturday afternoonSaturday PM

    Build the thinnest possible loop that completes the use case end to end.

  3. SundaySunday

    Run it fully offline and write down the honest speed and quality verdict.

The metrics

MetricHowCadence
Model sizeNote the parameter count and quantization of the model you pulled (e.g. 3B, Q4).Once, at setup
Tokens/sec observedWatch the terminal or app output while it generates; note the rate.Per test run
Shipped use caseOne sentence describing the real thing it now does for you.Once, at the end

Track it

Model size
Tokens/sec observed
Shipped use case

Logged in your browser — and to your account if you save one.

The protocol

4 steps

  1. Install Ollama, Pull a Small Model

    Standard practice

    Install Ollama on whatever machine you have. A laptop is fine, no GPU required for a small model. Pull a small model to start: something in the 3B–8B range runs on modest hardware without melting your fan. This is the common starting setup most people use to get a first local model talking, nothing custom yet. If Ollama doesn't fit your stack, llama.cpp directly is the same idea with more control and more setup. Either way, the goal of day one morning is this: type a prompt, get a response, entirely on your own machine, no API key, no cloud call.

    Do this

    One prompt, one real response, wifi off, before lunch on day one.

    Not this

    Spent the whole morning comparing seven models before running any of them.

    llama.cpp — LLM inference in C/C++ (GGUF format, ggml-org)

  2. Pick ONE Real Personal Use Case

    ☬ Tested on myself

    Don't build a demo. Pick one specific, boring, real thing you'd actually use: summarizing your own notes, drafting a reply, answering questions about a document you already have on disk. This is the exact philosophy behind BrainBox: useful beats impressive, every time. A local AI tool nobody uses because it was built to look clever in a screenshot is a wasted weekend. Write the use case down in one sentence before you touch any code. If you can't state it in one sentence, it's not specific enough yet to build against.

    Do this

    One sentence: summarize my meeting notes into three bullet points.

    Not this

    Built a general-purpose assistant that does a bit of everything.

  3. Build the Smallest Loop That Works

    ☬ Tested on myself

    Build the thinnest possible version that actually completes your use case end to end: input goes in, the model runs, output comes back, done. No UI polish, no extra features, no 'while I'm at it' additions. This is the same discipline behind every system I've shipped: get one full loop working before you improve any single piece of it. A working ugly loop beats a half-built beautiful one every time. Only the working loop tells you whether the idea was worth building in the first place.

    Do this

    Ugly script, one file, input to output, working end to end by Saturday night.

    Not this

    Spent Saturday designing a settings screen nobody had used yet.

  4. Run It Fully Offline, Note Latency and Quality

    Standard practice

    Turn off your wifi, or physically disconnect, and run the loop again. If it works exactly the same with no network, day two's job is done. While you're at it, note two things honestly: how fast it felt (tokens per second, or just a gut 'fast enough' or 'too slow') and how good the output actually was for your specific use case, not compared to a giant cloud model. Small local models trade some raw capability for full offline control. This is where you find out, on your own use case, whether that trade was worth it for you.

    Do this

    Wifi off, same result, tokens per second written down honestly.

    Not this

    Assumed it was offline-ready without ever testing without wifi.

← All sessions