Saving Tokens (and My Privacy) by Running Local Models on Commodity Hardware¶

May 8, 2026

A while back I built an n8n workflow that goes through my Gmail and auto-labels whether an email requires a response or not. Let's face it, the majority of email we get doesn't. But that's a whole other rant.

I originally set the workflow up on OpenAI, back when I was still using OpenAI. It was helpful, but it would end up wasting tokens that were precious back then, and it would add to my quota too quickly. I also worried about the privacy concerns of sending everything to a company like OpenAI. So I turned it off.

Six months later, my email is filled to the brim again, and it's driving me nuts. I need to get to Inbox Zero. So I dusted off the n8n workflow and converted it to use Ollama running Gemma locally on my M1 MacBook Pro. The only real change to the flow itself was the LLM node — instead of calling OpenAI, it now hits the local Ollama server. Same prompt logic, same Gmail labeling on the back end, no SaaS, no API key, no usage dashboard.

It takes 30 seconds per email, and that's fine¶

Each email takes about 30 seconds to analyze and decide on. But it's happening in the background, so it doesn't really matter. By the time I open Gmail, the triage is done. And honestly — if an email takes 30 extra seconds to land, who's actually gonna notice? Nobody's sitting on the other end with a stopwatch.

What's really neat is the backfill. I've been going through thousands of emails to catch up, and I can hear my MacBook fans humming along like the computer is having trouble. But it's all on the GPU. Ollama is using about 89% of the GPU and everything else stays running fine — anything that uses the CPU, at least. I wouldn't try gaming on it, but I don't game much on this computer anyway.

The part I really like¶

Locally, all of my data is completely private. It never leaves my infrastructure. I don't have to worry about emails leaking data to a big cloud provider.

It makes me want to invest in a local AI model server, and I think I might do that soon. But I still don't think I need anything top of the line. Commodity hardware can get me a long way, especially something like a Mac Studio with an M3 Max or M4 Max chip.

Why this matters¶

It's just one small way I wean myself off of the vendor lock-in of the big cloud providers. I'm all in on Anthropic right now — I think it's the best out there — but I'm also hesitant about getting locked in and relying too much on the cloud, just like everything else we end up doing.

As a home lab purist and a Linux nerd at heart, it makes sense that this is the next step. I'll keep using Claude, especially Opus, for tasks that require deep reasoning and very fast processing. But for the little day-to-day things where I'm leveraging AI to make decisions about something, Ollama and Gemma have found a place in my home lab.