Local-First AI Starter

BrainBox breaks the quiet deal most AI apps make with your data.

Most AI apps today have a quiet, unspoken deal built into them: send us your conversations, your habits, your journal entries, and we'll process them somewhere in a data centre you'll never see.

I built BrainBox specifically to break that deal — on-device chat, personas, habit and routine tracking, journaling, where the model runs on your phone, not in my cloud, because there is no cloud in the loop at all.

"Privacy-friendly cloud" is a marketing phrase, not an architecture; if your data ever leaves the device, it exists somewhere else, under someone else's control.

Ask one question first: where does the data actually go when you hit send?

As a starting point for evaluating any AI feature, ask one question before anything else: where does the data actually go when the user hits send? If the honest answer involves a server you don't control, that's an architecture decision being made for the user, not by them.

There's also a practical reason beyond privacy — a model running on-device answers the moment it's asked, no round trip, no dependency on signal strength.

Offline-first isn't a nice-to-have feature bullet; on a train or with the wifi down, it's the difference between the app working and the app being furniture.

One engineer, no cloud budget, shipped because nothing needed a server.

BrainBox is a local-first AI lifestyle app, live on Google Play, sitting on WatermelonDB for fully offline storage. It shipped as one person, no cloud budget, no backend team.

None of that was possible because I'm unusually talented at any single one of those pieces — it worked because none of them required a server round-trip to function, which meant one engineer could actually finish it.

• On-device LLM chat and personas
• Habit and routine tracking
• Journaling
• A gamified wellness engine rendered at 60fps
• Computer vision components that never phone home

Audit which parts of your idea genuinely need a server — most don't.

As a starting point when scoping your own build: audit which parts of your idea genuinely need a server and which don't.

Habit tracking doesn't need the network. Journaling doesn't need the network. Even chat doesn't need it if the model lives on the same device as the data it's reasoning about.

Every piece you can remove from the "needs a backend" column is a piece a small team, or a solo builder, can actually ship without external infrastructure cost.

Don't build your own inference engine — the open-source stack already exists.

As a starting point, don't design your own inference engine — the open-source ecosystem already solved the hard part.

• llama.cpp (Georgi Gerganov) — runs LLMs efficiently in plain C/C++ on ordinary consumer hardware
• GGUF — the compressed, quantized model format that fits a model on a phone or laptop
• llama.rn — wraps llama.cpp for React Native, for mobile builds
• Ollama — desktop model management with a simple command-line interface, the fastest way to try this first

Read llama.cpp's own supported-models list before picking a model.

This is real, actively maintained open-source infrastructure, not a toy project — worth checking the repository directly before building on top of it, since the ecosystem moves fast and model support changes often.

It's the same project underpinning most of the current on-device LLM tooling, including the stack BrainBox runs on.

Read the repository's own supported-models list before picking a model to experiment with; new architectures land there faster than any third-party summary can track, and a model that isn't yet supported will simply fail to load instead of degrading gracefully.

Start with one narrow use case, not "an AI app" as the goal.

As a starting point, don't start with "an AI app" as the goal — start with one real, narrow use case you'd actually use yourself, the same way BrainBox started as a habit tracker with a chat layer, not a general-purpose assistant.

Pick a small model that fits your target device's memory comfortably rather than the largest one that technically runs; a model that's fast and always available beats a bigger one that lags or drains the battery.

Ship the smallest working version — one screen, one on-device feature, one clear use case — before adding personas, journaling, or anything else layered on top.

Narrow core first, proven offline, before any feature got layered on.

That's the actual order BrainBox was built in, not the marketing order it's described in now: a narrow habit-and-chat core first, proven to work fully offline, before the wellness engine, the computer vision components, or the gamification layer got added on top.

Building it any other way — starting broad and hoping to make it all work offline later — is how most "local-first" projects quietly end up depending on a server anyway, because the hard constraint gets treated as a bolt-on instead of the starting design decision.