A few weeks ago, I shared the math worksheet generator with the math teachers at my daughter’s school. One of them asked a great question:

“I use AI regularly to help develop math problems, investigations, and instructional materials, but mathematical accuracy remains one of my biggest concerns. I’d love to better understand the practices and tools you use to catch errors, verify solutions, and recognize when an AI-generated result should not be trusted.”

It’s an interesting question, because I’m not sure it’s obvious to people the work that goes on behind the scenes in AI product companies and at frontier labs to drive more trustworthy and reliable results. Or that a lot of it is actually pretty accessible with readily available tools.

Nobody can guarantee an 100% accurate result from an AI system, and I’d be suspicious of anyone who claims to. What you can build is a system that knows which of its outputs deserve trust and that is transparent about it. It’s one of the things I love most about Gemini Notebook (fka NotebookLM) and I want to hold the same standard for the Math Worksheets system. It should do the work to earn trust and be clear where the lines are.

Let’s talk about the eval system and the in-line quality guards I built as an example for how you too can use these patterns when building AI tools.

Principles

As a quick refresher, our math generator system takes a math topic and a number of requested problems and uses AI to create three PDF formatted documents: a worksheet, a step-by-step answer key, and a study guide. The study guide, worksheet and answer key all need to be aligned, well-written and critically the answers need to be correct!

That’s because AI can’t be trusted to deliver accurate results on it’s own. In my system, the model’s job is to propose problems as structured data that can be independently verified: “problem 4 is a triangle with sides 9 and 13 and a 36 degree angle, solve for angle B, and I claim the answer is 58.07.”

A separate system that knows how to do math, checks the answers. In our approach, this is a fixed program (built on SymPy, an open-source computer algebra system that predates modern AI and does math the old-fashioned deterministic way) computes every claim. If the model’s answer is wrong, the proposal is rejected and another question is generated. The printed answer key is then checked again, character by character, against the verified values, so a typo between “verified” and “printed” also fails the build.

Models often hallucinate, and in our system the model can hallucinate all it wants because we don’t trust it not to. We check our work with a different math tool every time. This way, wrong answers never reach the paper.

For example: The number 58.07 in the question above is a real proposed AI answer for a test worksheet, and it is wrong. It’s one example from a law-of-sines worksheet that the verifier rejected. It took the AI tool four tries to get a correct solution, with answers that were off by between 0.01 and 0.23. The correct answer was 58.11, and AI really struggled to do it correctly. If we weren’t checking our work, this would have passed an eyeball test yet still be incorrect. Disaster for someone learning the skills.

To confirm our system performs as expected, we designed a robust practice test. In the AI world, we call these evals. Essentially, this is a large set of practice problems we can test ourselves against. For the math worksheet system, I created a set of 300 math topics ranging from counting to calculus to serve as example tests.

I then used the math worksheet system to generate 10-problem worksheets, answer keys and study guides for every one of the 300 topics. If everything is correct on all 300 of these worksheets, then we can feel pretty good about the operations of the system.

Some of you sickos would probably enjoy hand working 3000 math problems on a Saturday. I’m a bit lazier than that, so I wrote a different AI tool to help us grade whether the answer keys were correct. The tool employed a similar SymPy math engine, but also considers broader topics like whether problems actually teach the requested topic and whether they get progressively more difficult across the worksheet. There’s a lot you can do in a critic system to improve your quality.

But I skipped something super important. You absolutely must use a different AI system for critique than the one you use for creation. If you ask an AI to grade itself, it pats itself on its back and misses issues. When you ask one AI to critique another AI, they find all sorts of issues.

By running this system, I found dozens of edge cases we fixed and vastly improved the overall quality of the worksheets we generate. Some were pretty gnarly math issues, some were formatting problems a math checker could never spot, and others were just silly problems like not leaving enough space for the student to work out an answer.

I ended up running the eval suite 3 separate times before we started hitting diminishing returns on improvements. That’s when the outputs felt consistently great. There are probably harder evals yet to be designed and even more problems we can uncover, but that’s an assignment for another day. By taking these practice tests, we make many observations allowing us to climb the hill of quality and deliver a better overall experience.

If the story ended there, this would just be a nice little architecture diagram and a self-indulgent blog post. But AI is a sneaky beast and you need to test your verification systems, because a green checkmark is a claim about the checker as much as about the content it’s testing.

In one case, we caught the AI system submitting different questions to the checker than what the question was actually asking. It once asked students to find the value of c that makes a function continuous, but in its verification entry asserted that 9 is the solution because “c minus 9 equals 0.” I mean, that’s true, but it was also completely irrelevant to the question it actually asked.

We only found this class of issue through testing, so we now run automated lints that catch these types of issues during the generation process. As the recent Claude and OpenAI stories about AI escaping confinement implies, we’ll probably find other ways AI tries to take shortcuts in the future. But knowing is half the battle.

Unsurprisingly, understanding the tests you’re implementing is critical, or you’ll create false precision or overlook something essential. At one point our numeric comparison was being too strict about decimals, in a floating-point way that had nothing to do with the math. We found five separate cases where the authoring model had changed correct mathematics to satisfy the checker. A physics constant of 4.9 became a tidy 5. A perfectly good 9.4 became 9.5. Nobody decided to lie; the path of least resistance was just worse math.

If your verification rejects correct answers, it will eventually manufacture wrong ones. We rebuilt the comparison to understand significant figures so a value like 6.30 is accepted at the precision it was written to, and exact things are still compared exactly.

This is the part of the teacher’s question I most want to answer directly, because the trustworthy part of the system is precisely that it knows its own edges. Not every question that’s relevant on a math worksheet can be answered by a computer. Some require the expert hand of a real teacher.

Proofs, constructions, graph sketches, and every “explain your reasoning” question cannot be verified by a formula checker, and we can’t pretend otherwise. In our system, problems like this are declared clearly, and the printed answer key marks it with a spade (we picked ♠ because no mathematics on these pages uses the symbol, so it can mean exactly one thing).

The answer key’s summary page highlights these questions and adds a note: “14 of 17 answers machine-checked. ♠ marks an answer only you can judge.” A sheet where nothing is machine-checkable refuses to build at all unless the author explicitly acknowledges this intention, because a sheet with zero verification probably shouldn’t look the same as a sheet with full verification.

We also don’t claim the checks prove a problem is good. Verification can at best prove the math and its transcription. It cannot prove the problem teaches what you wanted, at the level you wanted. That judgment stays with teachers, who remain as always indispensable.

Takeaways

If you’re building AI systems, there are a few important things to remember.

1. Never let AI grade itself. Check answers with a different tool than the one that wrote the problems: a calculator, Desmos, a spreadsheet, WolframAlpha. Independence matters more than sophistication. Asking the same AI “are you sure?” is a second sample from the same distribution, not a second opinion.

2. Make AI produce checkable claims. “The answer is 58.11” can be tested. A worked paragraph that arrives at an answer somewhere in the middle cannot, easily. Ask for the final answers as a separate list and check those.

3. Ask what was verified, not whether. The vacuous-check problem lives in human workflows too. “I double-checked it” can mean anything.

4. Be transparent about assumptions. Make AI label which parts of its output require human judgement or are based on assumptions. Ours does, with a spade.

5. Distrust confident specificity. The fabricated number-line citations were detailed, plausible, and wrong. Detail is what makes a claim checkable, not what makes it true. AI loves to gaslight you with plausible sounding answers that are just plain wrong.

6. Make it obvious when a check can’t run. The most dangerous state is not “wrong” but “unverified and indistinguishable from verified.”

Hopefully, this can help you as you think about making your own AI tools and give you a bit more language and familiarity for when and how to trust these systems.

If you want to learn more, everything above is public, including the eval runs, generated worksheets, planted-defect experiments, and the analyses where our first conclusions got corrected by later evidence, at https://github.com/stellawuellner/math-worksheets-skill. The commit history includes every case where we were wrong, which by this point I consider a feature of the verification system.

Now, get out there and build rad shit!

I spent eight years handwriting math worksheets for my kids before I built a tool to do it for me. That tool was good. Every worksheet I ever handed my kids was correct, and I'd grown to trust it for the day-to-day math practice they needed.

But a new school year is coming, and both my kids are on an advanced math track. My son is starting 10th grade and is on pace to take Calc BC by senior year. My daughter starts 8th grade on a similar path. I wanted the worksheet tool to grow along with them and cover algebra, geometry, trig, word problems, and eventually calculus. Basically the whole math curriculum through high school.

That's a much bigger surface area than what the first version of my tool covered, and it changed what I needed it to do. The system needed to cover hundreds of new problem types I hadn't yet tested. I didn't just want to expand the scope and hope it kept working. I wanted to raise the bar, so when it takes on a new corner of the curriculum I can prove it's right rather than merely assume so.

So I rebuilt it last week. The new version does several important new things.

Works with your favorite AI agent

The old version was built for OpenClaw, which is great for me but a bunch of people told me they had no idea what that is. More people have access to a tool like Gemini, ChatGPT or Claude, so I fixed the limitation.

The rewrite is packaged as a skill: a folder with a SKILL.md file, some reference material, and a verification script any coding tool can load. Claude Code, Gemini, and Codex all standardize on this instruction format, which is nice. There's no per-tool version to maintain because there's only one skill, and it works out the local environment on its own (which LaTeX engine is installed, what model config is present) instead of assuming a particular configuration.

Thoroughly checks the answers

The model still writes the problems and proposes answers. But now before anything gets rendered, every answer is checked against SymPy, an actual computer algebra system, the same class of engine that sits under Mathematica. If SymPy can't confirm the answer, we rework the problem.

There are 24 specific verification types, because "check the answer" means something different depending on the problem. For example:

• Solving an equation over the set of real numbers is not the same as solving it over the set of complex numbers.

• A definite integral gets confirmed symbolically when SymPy can do it exactly, and falls back to convergence-checked numerical quadrature when it can't. If that doesn't converge, the problem is flagged for review.

• Triangles get the law of sines and cosines, including the ambiguous SSA case.

Statistics, probability, limits, series, systems of equations, reading a value off a chart: each has its own checker, and each has its own idea of what "correct" answers look like.

Passes comprehensive evaluation

Next, I challenged the answer system with real problem sets. Every step in the GSM8K collection of grade-school word problems and every boxed answer I could pull out of a public dataset of competition math problems went through the verifier. I evaluated 4,282 checks from the first set and 2,711 from the second. Just under 7,000 checks in total and across the eval set we hit zero false positive results.

It seemed too good to be true, so I went looking for trouble. I ran two rounds of adversarial audits against my own verifier, hunting for cases where it might confirm something false as I stretched it into harder math. Both rounds found real problems, and the second surfaced eight subtle issues at the edges of the new coverage, including:

• The definite-integral checker was using a quadrature method that gave wrong values on oscillatory integrands.

• The interval solver couldn't see tangent roots, the places where a curve kisses the x-axis without crossing.

• The system-of-equations checker would accept a solution that satisfied only one branch.

• The probability checker didn't confirm that a probability landed between 0 and 1.

These are exactly the failure modes you'd never notice by eye on an advanced worksheet, which is the whole point of having a machine check them. I fixed all eight and pinned each fix with a regression test, so they can't come back. I haven't been able to find others, but you never know.

Ready for any grade level

Correct answers are the minimum bar. A worksheet also has to be the right worksheet: the right concept, at the right level, for where the kid actually is.

So all problems are now mapped to real curriculum standards. Common Core codes for K-12, and the AP Course and Exam Description for the calculus track. Each concept carries a difficulty rating from 1 to 5 and a Bloom's-taxonomy tag for cognitive level, so "quadratics" can mean plug into the formula or derive the discriminant condition depending on what you ask for.

To find coverage gaps, I ran the whole thing against the Marble team's open taxonomy of what students learn.

About 40% of that taxonomy isn't machine-verifiable. "Write a story problem about sharing" has no single right answer a computer algebra system can confirm. The tool doesn't pretend otherwise. It knows which problems it can prove and which it can't, and it says so instead of dressing up a guess we check the result.

Keeping the system healthy

Finally, I implemented a robust regression and code quality test system to make sure any future changes don't re-introduce a problem we've solved before.

The suite covers 91% of the code, and continuous integration blocks any change that drops it below 90%. Every audit finding has a dedicated test, and every verification type ships with fixtures that must pass before I make a software update. Whether the thing that checks correctness is itself correct isn't a matter of my confidence. It's measured every time I make a code change to this system. The safety net allows me to add curriculum coverage without wondering whether I broke the system.

Ready for you

The updated math worksheet generation tool can now create worksheets for math topics from counting through calculus, and ready for the school year ahead. The whole thing is open source, and the verification layer is included. The system is honest about what it can and can't prove. It tells you when it's sure, and when it isn't, and it holds that line even as the math gets harder.

Download the skill: github.com/stellawuellner/math-worksheets-skill

This is the third and final post about GlassMeter, the film light meter I built to actually understand how metering works. The first was the why: a broken meter, a darkroom class, nine months of nights. The second was the afternoon an AI agent rewrote the whole app. This one is the part I didn't see coming, which is how much a hobby about hundred-year-old film chemistry taught me about my actual job, building AI products.

I build software for a living. I did not expect a light meter for film nerds to be where some of my clearest product lessons of the year came from. But the distance helped. It's easier to see how you work when the stakes are a roll of Tri-X instead of a launch. Here's what transferred.

Build for an audience of one, ruthlessly

The bar I set for GlassMeter was selfish and specific: would I carry it into the field instead of my real meter? Not "would people like it." Would I reach for it, on a real shoot, when getting it wrong meant a wasted sheet of film. For a long time the answer was no, and that no was the entire to-do list.

A single, demanding, real user surfaces truth faster than a fuzzy imagined million. The failure mode I see constantly in AI products is building for "users" in the abstract, an average person who doesn't exist, and shipping something that's plausible to everyone and indispensable to no one. One sharp user you actually know, even if that user is you, keeps the scope honest. They notice the thing that's almost right, which is the thing that matters.

Make the magic auditable

It is easy to fake a film look. Drop a preset, warm it up, add grain, call it Portra. I refused to do that. Every film stock in GlassMeter is built from the manufacturer's own published datasheet, the real characteristic curves and reciprocity tables, and I wrote the engineering notes so a stranger could check my work. When the app says Tri-X needs a stop more light at one second, that traces to a number Kodak printed.

The pull in AI is the exact opposite direction. The incentive is the dazzling demo, the answer that sounds authoritative and can't be inspected. But the products people actually come to trust are the ones that show their work: the citation, the eval, the honest "here's where this is shaky." In the GlassMeter post I included a whole section on where the app is still wrong, on purpose. Naming the limits is not a weakness in a tool people rely on. It is the thing that makes them able to rely on it.

Let the domain write the roadmap

I never sat down and brainstormed a feature list for GlassMeter. My darkroom syllabus wrote it for me. Darkroom I was 35mm and the fundamentals. Darkroom II brought the Zone System, push processing, and contrast filters. Darkroom III is large format. Each idea showed up in class on a Tuesday and in the app a week or two later, because I was living the workflow and feeling exactly where it hurt.

You cannot fully shortcut that with user interviews. There's a kind of product knowledge you only get by doing the work yourself, badly, until you feel the friction in your own hands. The PM version of "go shoot the film" is to actually live inside the workflow you're building for until the roadmap becomes obvious rather than invented. The best features I've ever shipped came from being annoyed personally, not from a research deck.

Treat your foundations as reversible

The second post in this series was about an agent rewriting the app in an afternoon. The product lesson underneath that story is bigger than one rewrite: the cost of changing your mind about a foundation has dropped through the floor.

That changes how you should make early decisions. For years the smart move was to agonize over the first architecture, because you'd be stuck with it. Now the smart move is often to build the fastest version that teaches you something, knowing you can re-pour the concrete later for cheap. Optimize the early phase for learning velocity, not for being right the first time. The thing you learn from the wrong version is what lets you specify the right one, including to an agent.

Ship when you reach for it, not when the calendar says

I didn't release GlassMeter on a date. I released it the week the honest answer to "do I reach for this first" finally flipped to yes. No readiness signal is more trustworthy, and none is more tempting to override with a roadmap commitment or a launch window.

Dogfooding isn't a QA step you do at the end. It's the spec. If the people who made the thing don't actually use the thing, no amount of polish fixes what that's telling you.

Restraint can be the product

GlassMeter collects nothing. No account, no analytics, no servers, no network calls at all. The App Store privacy label reads "Data Not Collected," because there is nothing to collect. That wasn't a sacrifice I made grudgingly. It's one of the best things about the app.

In a field whose default setting is to instrument and capture everything, deciding not to is increasingly the differentiator. Saying no to a feature, a data grab, a bit of cleverness, is a product decision with as much weight as any feature you add. Often more.

The real return

The artifact, a light meter used by a handful of film photographers, is not the return on nine months of nights. The return is judgment, and I spend it every single day at work.

The whole case for building to learn is simple. The small, deep version of a thing is the cheapest way I know to earn a real opinion about how something works, and opinions are the only thing that actually transfer. You can't borrow them from a blog post, including this one. You have to go build the thing and let it correct you.

So pick yours. The system you've been circling, half-understanding from the outside. Build the small version, badly, and let it fail at you until it hands you the map. It has never once let me down.

That wraps the GlassMeter series. If it sounds like your kind of thing, it's on TestFlight at glass-works.ai, and if you go build your own rad thing because of these posts, tell me what it taught you.

A couple of weeks ago I wrote about GlassMeter, the film light meter I built because I bought a Hasselblad with no meter and discovered I had no idea how to expose a scene without one. That post was about why I built it. This one is about a single afternoon last November when an AI agent rebuilt the entire thing in a language I barely knew, and what that did to my sense of how software gets made.

First I have to tell you about the two months I left out of the last post.

The version I don't show people

GlassMeter started life as a React Native app called GroundGlass. I reached for React Native because it's where I'm fast, and I wanted to be moving. For about two months it worked, sort of. It put a number on the screen. It had rotary dials and a calibration screen and a little tutorial.

But a light meter lives or dies on one thing: what it can read off the camera sensor, frame by frame, in real time. And every layer I'd stacked between my code and that raw pixel buffer was a place where the truth got rounded off, or slowed down, or fought me. I was writing native camera modules and then reaching them through a bridge, which is a polite way of saying I was doing the hard part twice. The abstraction I'd picked to go faster had quietly become the ceiling on what the app could ever be.

So I knew it had to be native Swift. And I dreaded it. A from-scratch rewrite, in a language and a UI framework I'd never shipped, was exactly the kind of multi-week slog that kills side projects in the crib. I had a working-ish app with a barrier in front of the version I actually wanted.

The afternoon

Then Google Antigravity launched an agent-first coding environment, and I did the thing you do with a new toy. I pointed it at the React Native repo and asked it to port the whole app to native Swift and SwiftUI.

It did. In one pass. Not a function, not a file, the app: the metering screen, the dials, the film selection, the settings, rebuilt natively. My commit history from that day reads like a single held breath. "Pre-Swift transition: save current React Native state," and then, minutes later, "Migrate to native Swift/SwiftUI, archive React Native code." The entire React Native version is still sitting in the repo in a folder literally named __ARCHIVE__, a whole parallel app I will never touch again.

I want to be honest about what "one shot" does and doesn't mean, because the phrase gets oversold. It does not mean I typed a sentence and went to get coffee and came back to a finished product. I still had to know exactly what I wanted: the architecture, the Hasselblad-style interface, what "correct" even meant for a meter. The agent didn't decide the product. It executed a translation I could specify because I'd already built the thing once, wrong, and learned it cold. And I spent the next day chasing build errors and polishing the UI. The git log says so, plainly: "Fix build errors" is right there.

But here is the part that still gets me. The work I had been dreading, the weeks-long rewrite that was going to cost me a month of evenings I didn't have, happened between lunch and dinner.

The thing that actually changed

The port wasn't even the surprising part. The surprising part was the velocity after.

Within forty-eight hours of standing on native ground, I had added the LiDAR rangefinder through ARKit, true spot metering, real reciprocity math, film halation, and grain. Features that had felt like someday-maybe items behind the React Native bridge just started falling, one after another, because the tax I'd been paying on every single one was suddenly gone. The rewrite didn't save me a month one time. It changed the slope of everything that came after it.

For most of my career, the cost of re-platforming was the moat around a bad early decision. You picked the fast framework to get going, and then you were married to it, because tearing it out was a month you couldn't spare. So you lived with the ceiling. Agents collapse that cost. Changing my mind about the foundation of the app went from "a month, so no" to "an afternoon, so sure." That's not a productivity tweak. It quietly moves a whole class of decisions from irreversible to reversible, and reversible decisions are the ones you can afford to get wrong on the way to getting them right.

What I'd actually tell you

The agent is exactly as good as your ability to say what you want. It ported my app cleanly because I knew precisely what my app was. It would have just as happily ported my confusion. The skill that mattered that afternoon was not Swift, which I still barely know. It was knowing the product down to the pixel, and I only knew it that well because I had built the wrong version first and let it teach me.

Which is the same lesson the whole GlassMeter project keeps handing me. You don't get the shortcut until you've earned the map. The agent didn't replace the part where I had to understand the thing. It just made the understanding the only expensive part left.

The strange thing is that I build AI products for a living, and it took a hobby light meter to make this land. Next in the series: the lessons from this little film app that followed me all the way back to my day job.

Have you let an agent re-pour the foundation of something real yet? I want to hear where it worked and where it bit you.

I'll walk the whole thing end to end, using one frame as the example: the valley oak you saw in the GlassMeter post, the one with the big tree against a bright, cloud-streaked sky. It's a hard exposure and a harder print, which makes it a good teacher.

1. Choosing the film

I mostly shoot Ilford HP5 Plus or Kodak T-Max 400. I like a contrasty negative, and both push and pull well, so I have room to chase the light instead of being boxed in by it.

A quirk I've leaned into: I've bought a fair bit of 20-plus-year-old expired T-Max, and it shoots nearly the same as a fresh box. It wants a touch more exposure to make up for age, but not much. There's something satisfying about feeding decades-old film through a brand-new app and getting a clean negative out the other side.

2. Metering and making the exposure

I work off a tripod. I compose with the lens wide open, because that throws the most light onto the ground-glass screen and makes focus easier to see, and I use a magnifying loupe to confirm it's tack sharp. Then I pick the aperture I actually want, the one that puts the depth of field where I need it.

Now the meter. I open GlassMeter and match the composition in the finder, then I read the scene the Zone System way. I spot-meter a shadowed area where I want detail without it going muddy and place it on Zone IV, and I drop a pin there. I add a second pin on the brightest part of the scene and note how far apart the two readings are, so I know the contrast range I'm dealing with. I add a third on the darkest point. The exposure I trust is the one that holds those shadows at Zone IV, which is exactly what the app hands me.

I set that aperture in the app and check the rangefinder to see whether the depth of field matches what I pictured. If it doesn't, I adjust and look again. Then I lock the aperture, press meter, and the app gives me the shutter time. I dial it into the camera.

Then the ritual. I check the composition one more time, press the cable release, and wonder why nothing happened. Right: the dark slide is still in. I pull the dark slide, press the release again, and there's that satisfying ker-chunk as the leaf shutter fires and the film finally sees the scene. I advance the film and slide the dark slide back in. If the exposure runs longer than a second I'll use mirror lock-up to kill the vibration, but for a normal frame I don't bother.

3. Loading the tank

Developing starts in total darkness. We use plastic Paterson tanks, usually sized for two reels, which suits me because I tend to shoot more than one roll a session. There's a properly dark room, no safelight, where I pull the film off its spool and wind it onto the reels by feel.

That expired T-Max fights me a little here, because old film likes to stay curled, which makes it stubborn to seat on the reel. I can load by touch in complete darkness now, but it took real practice with a sacrificial roll to get there. It is a strange and good feeling, doing something delicate entirely by hand and feel, with your eyes useless.

4. Developing the film

Once the film is sealed in the tank, the lights come back on and it becomes a careful kitchen recipe.

For most of the class we were using Arista Premium, Freestyle's liquid developer, which is widely understood to be a rebadged Clayton F76+. We run it at a 1:9 dilution, which makes it milder and more consistent than alternatives. At 68°F, HP5 Plus runs 7:00 and Tri-X runs 6:30. Lately the tap water comes out closer to 70°F, so we shorten the time a touch using a development chart rather than pretend the thermometer says what we wish it did.

The sequence is the standard archival one: developer, then a one-minute water bath, then fixer for five minutes, a water wash, three minutes in hypo-clear to pull the fixer back out, a long archival rinse, and finally a dip in Photo-Flo so the film sheets water instead of spotting. Agitation matters as much as time: thirty seconds at the start, then five seconds every thirty after that. Too much agitation and the contrast climbs; too little and you get uneven development. It is a feel you build by repetition.

5. Drying

In class we have a luxury: an industrial film dryer that takes the film from dripping to bone-dry in about forty minutes. I squeegee the excess water off between two fingers, hang the strips in the dryer, and forty minutes later they're ready. At home this step is an overnight affair in a dust-free bathroom, so the dryer feels like cheating in the best way. Once dry, the strips go into archival binder pages, which are what I'll proof from.

6. The contact sheet

Before I commit good paper to a single frame, I make a contact sheet, a one-to-one proof of the whole roll, so I can see what I actually have.

It's made on the same Saunders/LPL enlargers we print with, using a full sheet of glass to press the negative pages flat against the paper. The method is a test strip first: raise the enlarger so the light covers the easel, open the lens to its brightest, and expose a thin strip of paper in stages, sliding a piece of cardboard along in three-second steps so I end up with a strip showing four or five different exposure times. I develop that, look at it in the light, and pick the time where the frames read best. Then I expose the full sheet at that time. We started the term on Ilford MGRC, the resin-coated paper, and have moved to Ilford Multigrade FB Classic, a glossy fiber paper, for the advanced class. The fiber is more expensive and fussier to handle, but the blacks are deeper and the print just looks better.

7. Making the print

This is where it stops being a recipe and starts being a craft.

I print on a Saunders/LPL 4×5 enlarger, the diffusion-source workstation Foothill's darkroom is built around, and an 11×14 sheet of Ilford Multigrade FB Classic is my favorite. The diffusion head matters: it spreads light more evenly than an old condenser enlarger, which means smoother tonal transitions and a little mercy on dust. I focus with a throwaway sheet in the easel and set the lens around f/16, two or three stops down from wide open, where it's sharpest.

For a scene like the oak, with detail spread across a wide tonal range, I print split-grade. The idea is to control highlights and shadows separately, using two different contrast filters, instead of compromising with one. First I put in the soft 00 filter and run a test strip in five-second steps over an area where I want highlight detail, the wispy clouds in the bright sky. That tells me the base exposure that holds the highlights. I lock that 00 time in.

Then I switch to the hard 5+ filter and run a second test strip, additive, meaning it lands on top of the 00 base I already chose. That dials in the shadows and the overall contrast. The final print is the sum of the two: a soft exposure for the gentle tones and a hard one stacked on top for punch. You can read the recipe right off this negative's sleeve.

That "burn top R" is the last move, and it's the reason split-grade was worth it here. The sky was so much brighter than the tree that a straight print left the clouds blown out. So after the two base exposures, I give the top-right corner about ten extra seconds of light, holding back the rest of the print with my hands, to bring the clouds back without darkening the tree or muddying the print around it. Split-grade is an advanced technique, and it's overkill on an easy negative. It earns its keep on the hard ones, where you're trying to recover a sky and protect a subject in the same sheet.

Fiber prints get their own processing run, longer and more patient than the film: two minutes in the developer, forty-five seconds in the stop bath, two minutes in the fixer, a three-minute pre-wash, five minutes in hypo-clear, and a twenty-minute final wash to get it truly archival. Then I squeegee it and lay it face-down on a drying rack. Fiber paper dries with a will of its own and curls as it goes, so the next day I flatten the finished prints in a print press. Only then is it done.

Why do it the slow way

You could ask, reasonably, why anyone does this in 2026 when a phone makes a perfectly good picture in a quarter second. The honest answer is the same one behind GlassMeter itself: doing the whole chain by hand is how you actually understand it. When you've placed the shadow on a zone, agitated the tank by feel, and burned a sky back in under a safelight, "exposure" and "contrast" stop being settings and start being things you have a relationship with.

That oak hangs on my wall now. It took a tripod, a meter I wrote, a tank loaded in the dark, a recipe of chemistry, and two filters' worth of light bent around my own hands. Every step is a place it could have gone wrong, which is exactly why getting it right feels like something.

This is the companion to my GlassMeter series. If you want the app that handles the first link in this chain, it's on TestFlight at [glass-works.ai](https://glass-works.ai/testflight). And if you've got a darkroom, tell me how your process differs from mine.

It started with a broken light meter.

I bought a Canon AE-1 at a camera swap, a beautiful old 35mm SLR and one of the best-selling cameras ever made. It was cheap because the meter inside it was dead. No problem, I figured. I'd just meter the scenes myself.

Then I tried, and found I had no idea how. I knew the vocabulary of exposure (aperture, shutter, ISO) the way you know the words to a song in a language you don't actually speak. Handed a real scene and a camera that wouldn't tell me what to do, I froze.

So last September I signed up for Darkroom I at Foothill College: 35mm black-and-white film, trays of chemistry under a safelight, and the slow fundamentals. Load the tank in total darkness. Develop a roll. Make a print. My first prints came back muddy, flat shadows with no snap to them, and it was obvious the mistake hadn't happened in the darkroom. It happened out in the field, the moment I metered the scene and guessed at the light. I've shipped products at Google for eighteen years. I figured understanding metering would take a weekend.

So I did what I always do when I want to understand something for real. I built it.

Build to learn

This whole newsletter is named after the idea, so I won't belabor it: the fastest way I know to actually understand a system is to try to reconstruct it. Not read about it. Not watch a video. Build the thing, badly, and let it fail in all the specific ways that show you where your mental model was wrong.

A light meter looked like the perfect target. I had a dead one in my hands, for one. It's small enough to finish, and deep enough to hurt. And it sits right on top of a stack I'd wanted an excuse to learn anyway: the iPhone camera pipeline, real-time GPU rendering with Metal, ARKit and the LiDAR sensor, and the strange world of color science. Every one of those is a rabbit hole. A meter gave me a reason to climb down all of them with a clear question driving each descent.

The deal I made with myself was simple. It had to be good enough that I'd carry it into the field instead of my real meter. If I'd reach for it on a shoot, I understood the problem. If I wouldn't, I was kidding myself.

It took about nine months. The first commit is dated September 17, the week class started. As I write this it's on TestFlight as GlassMeter, sixty-three commits and roughly eighteen thousand lines of Swift later. The reason I'm writing at all is that it finally cleared the bar. I reach for it first now, so it's time to let other people shoot with it.

The first version was wrong, and that was the point

I started in React Native, because that's where I was comfortable and I wanted to move fast. The app was called GroundGlass. It worked, sort of. It put a number on the screen.

It took about three weeks of fighting the camera through a bridge to realize the abstraction I'd chosen was the thing standing between me and the actual problem. A light meter lives or dies on what it can read off the sensor, frame by frame, in real time, and every layer between my code and that buffer was a place where the truth got rounded off. It had to be native Swift.

The switch happened the week Google Antigravity came out last November. I pointed it at the React Native project and asked it to port the whole thing to Swift. It did it in one shot: the entire app rebuilt natively in a single pass. That one made me sit back in my chair.

The rewrite was also the first real lesson of the project, and it had nothing to do with photography. It was about how often the tool you pick to go faster is the tool quietly deciding what you're allowed to learn.

Build the thing, badly, and let it fail in all the specific ways that show you where your mental model was wrong.

Lesson one: your phone is lying to you on purpose

The iPhone camera is a phenomenal piece of engineering whose entire job is to make every photo look good without you thinking about it. Face detection, Smart HDR, shadow lifting, local tone mapping. It is constantly, aggressively second-guessing the light in front of it.

That is exactly what you do not want in a light meter. A meter has to tell you the cold truth about the scene, including when the truth is "this is going to be a hard exposure." Point a stock iPhone at a night sky and it cranks the ISO to the moon and reports a brightness that is physically a lie.

So the core of GlassMeter is a custom exposure loop that switches the camera into fully manual mode and runs my own metering at 5 Hz. Every 200 milliseconds it grabs the raw pixel buffer, measures the actual luminance, and nudges ISO and shutter to pin the scene to middle gray, the same 18% value photographers have calibrated against for a century. From those hardware settings plus the residual error, it computes a scene EV that holds still instead of jumping around.

A detail that ate an entire evening: the iPhone sensor is mounted in landscape even when you're holding the phone in portrait, so every tap on the screen has to be rotated into sensor coordinates before you can sample the right pixels. Tap a shadow in the bottom-left of the finder and, without the transform, you'd be metering something off the side of the frame. Obvious in hindsight. Not obvious at 11pm.

There was a subtler call underneath all of it, too. When you average a whole scene, do you average the brightness values directly, or in stops? GlassMeter averages in log space (a geometric mean), so a bright window in the corner of a room doesn't drag the entire reading toward the highlights the way a naive average would. It meters the way your eye weighs a scene, not the way a spreadsheet would.

Getting that loop to hold still without oscillating came down to deadbands and gain, the same feedback-control tricks I'd reached for before in other corners of my work. What was new was what it taught me about the darkroom idea that a good meter "settles." I'd been treating that as poetry. It's a feedback loop.

Lesson two: middle gray is 0.18, except when it's 0.46

This is the bug I'm most fond of, because it's where the photography and the code turned out to be the same fact wearing two costumes.

I wanted the finder to show you the photograph, your chosen film and aperture and shutter, not just a number. So after metering, the image flows through a film simulation: a tone curve, color shifts, grain, the works. And it came out crushed and muddy, shadows everywhere.

Middle gray, in the linear light the math operates in, is 0.18. But the film color tables I was feeding it expected gamma-encoded sRGB, where middle gray sits around 0.46. I was handing the film stage a 0.18 and it was reading that as "deep shadow," so every midtone in every scene got shoved toward black.

The fix is one gamma-correction step in the right place. The reason I understood the fix is that this is the same 0.18 Josh Smith, who teaches my darkroom class, had me metering off a gray card to set exposure and calibrate the camera. The number that means "the middle of everything" is the same number whether you're standing under a safelight or staring at a Metal shader. Eighteen percent. I will never forget where middle gray lives, because I spent a Saturday debugging it in two places at once.

Lesson three: film has opinions, and they're written down

The part that ate the most time, and gave back the most, was making the film simulations honest.

It's easy to fake a film look. Slap on a preset, warm it up, add grain, ship it. That's not what I wanted. I wanted Tri-X to behave like Tri-X. So I went to the source: the actual datasheets that Kodak, Ilford, and Fuji publish for every stock, with their characteristic curves (the H&D curves that plot how density responds to exposure), their reciprocity tables, their filter factors.

Each film in GlassMeter is built from its real published curve: Portra's long, gentle shoulder holding the highlights, T-MAX running nearly straight, slide film falling off a cliff in the shadows. Push HP5 to 3200 and the contrast steepens and the grain compounds the way a real push does, because the curve is doing real work, not a slider pretending to.

The reciprocity math is my favorite example of the datasheets paying off. Film loses sensitivity during long exposures, an effect called reciprocity failure), and every stock fails differently. I fitted a generalized version of the Schwarzschild law to each film's published correction table. The payoff: GlassMeter's Tri-X now matches Kodak's printed table to the stop, one extra stop at 1 second, two at ten, three at a hundred, while Acros II and Provia correctly don't start failing until you're two minutes into an exposure. The first time I watched the app add exactly the compensation the datasheet calls for, on a long exposure I was about to shoot, something clicked that no amount of reading had managed.

Then there was the bug I'll never live down. A batch of color films were rendering in full psychedelic Technicolor, skies going magenta and skin going green. The cause was two errors stacked on top of each other: I was reading the film's color table with the bytes in the wrong order and the color channels in the wrong order. The gray diagonal of the table happened to survive both mistakes, which is why grays looked fine and only colors scrambled, which is why it took me forever to find. The lesson there is older than photography. When the simple case works and the complex case explodes, suspect your assumptions about the data before you suspect your logic.

The number that means "the middle of everything" is the same whether you're standing under a safelight or staring at a Metal shader.

The decision that made it mine

Partway through, I'd gotten some of the film looks by decoding third-party color profiles. They worked. But they weren't mine, and shipping them as if they were didn't feel right.

So in June I deleted all of them and rebuilt every remaining film look from scratch, authored from the documented character of each stock rather than copied from someone else's profile. There's even a test that fails the build if any third-party profile sneaks back into the bundle. It cost me a week, and the app looks marginally different than it did. But every film look that ships is now original work, traceable to a published datasheet, and I can stand behind all of it.

That wasn't a coding decision or a legal decision. It was the instinct the darkroom drills into you: you own the print. Every choice in it is yours, on purpose, and you should be able to say why. Judy Walgren, who runs the photography department and has spent a career on the ethics of images, has a way of making that feel non-negotiable.

The class kept writing the roadmap

The course sequence turned out to be a curriculum for the app, too. Darkroom I was 35mm and the fundamentals. Darkroom II moved up to 120 roll film and medium format, and brought the Zone System, push processing, and contrast filters with it. Each idea showed up in Josh's class and then, a week or two later, in the app. I never had to write a feature list. The syllabus was the feature list.

Medium format is where GlassMeter found its soul, and it's the format I built the app for. By then I'd bought a Hasselblad 503CX, a 6×6 camera that, like the classic V-system bodies, has no meter at all. You compose on a ground-glass screen in the waist-level finder, you get twelve frames to a roll of 120, and you bring your own meter or you don't shoot. That's the camera GlassMeter is built to think like, down to the square-frame discipline, the Zeiss-style glass, and the Hasselblad-flavored metadata it writes into every file. I'd named the first prototype GroundGlass, after that focusing screen, before I owned the camera that made the name make sense.

So the Zone System workflow is the real one. Spot-meter a shadow and pin it, spot-meter a highlight and pin it, and the meter places your exposure between them in log space, the shadow-and-highlight placement Adams taught and my instructor drills. As far as I can tell, no other phone meter does it properly. You can pin up to five readings and watch the app average them, exactly the way you'd reason through a tricky scene by hand.

The app also shows you the scene's dynamic range live. Two markers track the darkest and brightest regions in the frame and report the spread in stops between them, and when the sensor clips, the meter says so plainly instead of pretending the spread is smaller than it is. That single readout has talked me out of more than one exposure I would have blown.

I've since moved up again. I'm in Darkroom III now, shooting 4×5 large format, one sheet of film at a time under a dark cloth. It's the most deliberate photography there is, and the meter comes with me. But its heart is still the medium-format roll it was built for.

A meter that knows where things are

A handheld meter has never been able to do this next part, because a chunk of glass in a leather case doesn't know how far away anything is.

GlassMeter uses the phone's LiDAR scanner as a rangefinder. Up close, LiDAR reads true distance to whatever you've metered. Past its range, your focus distance takes over on the same scale, and the app draws the near and far limits of your depth of field for the aperture you've dialed and the lens you've loaded. You can see, before the shot, exactly how much of the scene will be sharp.

This is the feature I'd never have guessed I'd build when I started. It exists because once I understood metering well enough to finish that part, the next question (how much of this will be in focus?) was sitting right there, and the sensor to answer it was already in my pocket.

The glass has opinions, too

The same datasheet honesty that went into the films went into the lenses. GlassMeter carries fifteen Zeiss V-system lenses, from the 30mm F-Distagon fisheye to the 500 Tele-Apotessar, and each one is built from its real published numbers: focal length, aperture range, true angle of view, minimum focus distance, filter thread, even which shutter it carries.

The most useful part is the simplest. Load a lens and the finder reframes to exactly what that lens sees. Put on the 250 Sonnar and the frame snaps to its real 13-degree field, and the meter reads only inside that crop. You end up metering the photograph you're actually going to take, not the wider scene your phone happens to be pointed at.

From there the small stuff piles up the way it does on real glass. Each lens vignettes at the edges wide open and cleans up as you stop down, modeled in linear light so the falloff behaves. Each one loses a little light passing through, so a fast lens reads a hair slower than its f-number promises, and that loss is folded into the exposure. And each has its own bloom: the Planar FE 2/110 is the system's glow lens, with a visible halo wide open at f/2, while the 100 Planar is corrected so well it barely blooms at all. That character then gets multiplied by the film's own halation, so an 800T frame through the 110 glows very differently than Acros through the 100.

One honest caveat, since I'm keeping a list of those: Zeiss never published per-lens vignetting curves for the V system, so that single number is a heuristic from focal length and speed rather than a measured falloff. Everything else traces to a data sheet.

One frame, the long way

Here is what all of that machinery actually feels like in the field, on a frame you've already seen in this piece: the valley oak.

I set the camera on the tripod and point the phone at the scene. I spot-meter the deep shadow under the canopy and pin it, then the bright dry grass in the sun and pin that too. Both readings land on the zone ruler with the spread between them, and on a high-contrast afternoon the app tells me plainly that the range is wider than the film will hold cleanly. So I make the call every film photographer makes: I place the shadow on Zone III, dark but still holding texture, and the meter sets the exposure to put it exactly there. Now I know where the highlights will fall, and how much of the sky will blow, before I have spent a single sheet.

I want the trunk crisp and the far ridge soft, so I read the depth-of-field limits the rangefinder draws for the aperture and the lens I've loaded, and stop down until the near edge of sharpness reaches the grass. The light is going, and the exposure runs long, past a second, so the app folds in the film's reciprocity correction and counts the bulb exposure down in my hand with haptic taps. I trip the shutter, hold it open until the count ends, and that is the frame.

The notebook that keeps itself

Film never recorded its own settings. You scribbled them in a notebook or you lost them forever, and most of us lost them. By the time you've developed the roll and you're hanging the negatives up to dry, you can't remember whether that frame was f/8 or f/11, or which filter was on the lens.

Every frame GlassMeter captures carries its full exposure record into the file's metadata: film, lens, aperture, shutter, ISO, push or pull, even the individual spot readings you pinned. The notebook I always meant to keep, kept automatically.

You can share a frame two ways: as the photo, or as a little contact-sheet cell with film-edge markings and your exposure data printed alongside it, the way a real proof sheet looks. It scratches the same itch as labeling your negatives, minus the part where you never actually do it.

The details that don't photograph well

Plenty of the nine months went into work that will never show up in a screenshot.

There's a bulb-mode timer that counts your long exposure down in your hand with haptic taps, the film's reciprocity correction already folded into the recommended time. That one is pure darkroom-class wish fulfillment.

The aperture and shutter dials have haptic detents you can feel, and you can lock one to shoot aperture-priority or shutter-priority, the way the dials on a real body work. Press the meter button and the unlocked dial solves itself.

The filters are honest, too. Yellow 8, Green 11, Orange 21, Red 25, each with Kodak's published Wratten factors and a spectrally-derived color response, so a red filter really does drive a blue sky toward near-black the way it does on real B&W film.

And then there's the unglamorous stuff that makes an app feel finished. The whole thing is localized into 34 languages. It sheds frame rate and effects when the phone gets hot, and pauses rendering entirely when a menu covers the finder, because your battery is part of the design. None of that is exciting to describe. All of it is the difference between a demo and a tool.

The stack, in the end

Nine months in, this is what the side quest actually taught me, beyond photography. Real-time GPU rendering in Metal, where the live finder and the final capture run the exact same 16-bit, wide-gamut image pipeline so that what you see really is what you get. ARKit and depth sensing. A pile of color science I'd never have learned from a tutorial, because I'd never have had a reason to care. And a fresh respect for how much deliberate engineering goes into the camera in your pocket pretending to be effortless.

I also made a decision I'm quietly proud of. GlassMeter collects nothing. No account, no analytics, no servers, no network calls of any kind. The App Store privacy label reads "Data Not Collected" because there is, truly, nothing to collect. The most interesting thing a light meter could ever phone home is nothing.

A few thanks

I didn't get here on my own. Josh Smith teaches the darkroom classes I've been taking at Foothill since last fall, and he's the reason I started shooting film seriously in the first place. He's a working photographer and a patient teacher, and he also teaches photography at Eastside College Prep in East Palo Alto. Judy Walgren runs Foothill's photography department; she's a Pulitzer Prize-winning photojournalist and a former director of photography at the San Francisco Chronicle, and her advice, on the pictures and on everything around them, has shaped how I think about all of this. Both of them have been generous with their time and their eye throughout this project, and patient with a software guy who kept turning up to class wanting to talk about exposure math. Thank you.

Where it's still wrong

I'd rather be straight about the limits, because a meter you trust blindly is worse than no meter.

It's a phone, not a dedicated spot meter. In bright sun the screen washes out, the battery drains under the rendering pipeline, and it slows itself down when it gets hot. For critical work I still sanity-check it against my handheld, and you should too.

Incident metering is an approximation. A real incident meter has a diffusing dome with known optics; the phone's front camera doesn't, so I model the diffusion instead of measuring it. It lands close, but it still wants a calibration pass against the real thing.

And some of the film numbers are educated guesses. Manufacturers only publish their reciprocity tables so far. Past a second or two, the color-negative corrections are extrapolated rather than measured, because Kodak and Fuji simply don't print them. I flag those in the app's engineering notes rather than pretend they're gospel.

None of that is false modesty. It's the list I'm hoping the beta helps me shorten.

The most interesting thing a light meter could ever phone home is nothing.

Why I'm sharing it now

I built this for me. For most of its life it was a tool with exactly one user, tuned to exactly one person's confusion about exposure. The bar I set was selfish: would I carry it into the field instead of my real meter? For a long stretch the answer was no, and that no was the to-do list.

Somewhere this spring the answer flipped. Now it's the first thing I reach for when I'm standing in front of a scene trying to decide where to put the shadows. That's the whole signal I was waiting for.

So it's on TestFlight, and you're invited. If you shoot film, or you've ever wanted to actually understand what a light meter is telling you instead of just trusting it, I'd love for you to break it and tell me where it's wrong. There's a feedback button right in the app. What I most want to hear: where the meter disagrees with your handheld, and which film looks feel off to people who have shot them for years. If the beta holds up, the App Store is the next stop.

You can join the beta at glass-works.ai.

And if there's a system you've been circling for a while, half-understanding it from the outside: pick the small, deep version of it and build the thing. Let it fail at you. You'll never read middle gray the same way again.

You might make one too. It's a Google Doc with a table. Flights up top, then hotels, then a row for every day: what's booked, the confirmation codes, who's going where and when. I hand a copy to my wife at the airport and it becomes the bible for the trip. In 2024 it ran us through the Croatian islands: Dubrovnik, Korčula, Split, a ferry every couple of days. In 2025 it ran Japan: Tokyo to Kyoto, a samurai lesson, a hedgehog café. This year it's a run through Boston, Croatia, Italy, France, and Maine. The Doc quickly gets messy.

Nevertheless The Doc works, but its kind of a tyrant. Every year it gets longer and smarter and harder to use. More links, more "[!] needs attention" flags, more places where the one thing you need right now is buried on page nine. This spring I caught myself building three competing layouts for the daily planner and labeling them, with no irony, FORMAT A, FORMAT B, FORMAT C.

That's when it clicked: I didn't want a better version of The Doc. I wanted something richer, more informative, and a little easier to navigate. What I wanted was an app. I’ve tried a bunch of apps theoretically made to do this, but none really worked for me.

So I built the one I wanted.

I didn't have to do manual iOS development to make this. That's the whole point. I built it using Antigravity the way you'd work with a sharp junior developer. I described what I wanted, in plain English, and my agents did the coding. (I used a mixture of agents including my home agent, Stella, who regular readers have met.) The Doc a real native app now, on my phone and my wife's. Mostly vibe-coded, but all mine. I guided and the tools built.

The bar for building things you just want for yourself is so much lower than you think. And the ceiling higher. Getting started is easy, crafting excellence takes time.

What The App actually does

The App knows what day it is. Open it on June 23rd (that’s today!) and it reminds me of the flight that I’m currently on. Open it on July 1st in Verona and the first thing you see is a reminder about our tickets to Aida, a map to the Arena with a link to a saved PDF of the ticket, and my note to "bring a seat cushion.” Open it on a travel day and it leads with your next transfer, a link to the train booking site and instructions on how to purchase tickets. The same screen quietly rearranges as the trip unfolds, adapting as needed to new information. (Remember, I wrote a whole post about just-in-time UX being the thing for 2026.).

Every day you also get the weather forecast, suggestions for meals and a selection of relevant phrases in the local language along with pronunciation audio. Hilariously, the AI added a phrase guide for Boston including: Wicked, Pisser and Packie. All useful phrases for the kids to learn for the camp bus tomorrow. Maybe the adults will hit a Packie after we drop the kids off.

The App has several touches that I really wanted in my app. It creates a detailed city guide for each place with useful information including a live map with your hotel marked, a brief about the history of the city, summaries of what it's known for, hints about local etiquette like tipping customs, relevant emergency numbers, and a selection of restaurants filtered so my wife (who doesn't eat seafood) never gets pointed at an oyster bar.

Since I work on NotebookLM, my favorite part (obviously) is that every city has it’s own podcast. I had the NotebookLM agent research the history, pull in the most relevant sources, and generate an audio briefing about the city. Little documentary-style episodes that ride in your pocket and play offline, right on the lock screen, while you walk around and explore a new place.

Here’s a small taste of Rovinj's audio briefing, "The Venetian Island That Became a Peninsula":

There's one catch: making those podcasts is the only step in this whole system I still do by hand. The notebook agent does the research and the generation; I'm just the one clicking through it, city by city. The day there's a NotebookLM MCP or API, this manual step can go away and the trip's audio will generate itself. (I'm choosing to hear that as a feature request. You're welcome, future me.)

A few more things The Doc could never do safely. It scans our passports and keeps the information in the iPhone’s secure storage: point the camera, and it reads the little machine-readable strip at the bottom and fills in the traveler information, while the photo itself never leaves the phone. The sensitive stuff stays sensitive, since passport numbers and document scans live behind Face ID in the iOS Keychain and never touch the network.

Both phones stay in sync without handing the cloud anything readable: when I update the trip back home, both phones pull the change, but the server only ever sees encrypted gibberish. The passphrase lives on our devices and nowhere else. Zero-knowledge sync, for a literal family vacation. Overkill? Maybe. I’ve never been accused of doing things half-assed.

The messy middle

I don't want to lie to you and tell you making this was a totally frictionless experience. Building with AI is mostly iteration, and the bugs are real and occasionally hilarious or annoyingly sticky.

It took me forever to get the agent to do a reasonably good job with the Liquid Glass styling and to get the padding right for the edges and cards in the experience. Almost as frustrating as debugging CSS, but maybe a bit more complex. I guess there isn’t enough training data on the liquid glass design system yet.

Content can be misunderstood as well. At one point an agent reading a rental-car confirmation dropped Portland International Airport, the one in Oregon, into a trip that goes nowhere near it. Two Portlands. Classic.

For awhile the app crashed every single time you hit play on a podcast, because the lock-screen artwork was rendering on the wrong thread. A genuine Swift concurrency gremlin that took reading an actual crash log to pin down. And a real chunk of my commit history is just me sanding the tells off the machine-generated UI until it stopped looking AI-built and started looking hand crafted.

None of that required me to use my computer science degree. It required me to be stubborn and curious enough to keep going. That's the actual work now — bring your ideas, your sense of taste, and don’t quit until you like it.

Where I'm landing (and a question)

Technically, we still have about an hour before we land in Boston. But on my mind is a question: How low is the floor for making something? "I wish this were an app" used to be the end of the sentence: a wistful thing you said and then didn't do. It isn't anymore. If you've got a real problem you actually care about and you're willing to iterate, you can build a tool that solves your problem. Not someday. This weekend.

Which brings me to something I'd love your read on:

The engine under this app is already generic. I built it so the trip is just a simple json data structure with one file and a folder of screenshots. Each unique trip is a different file; the app itself doesn't change, it just points to a different source of truth. Which means I'm basically one onboarding flow and a hosting system away from being able to let other people try it: point it at your upcoming trip doc, and have The App make your own custom version to take with you.

Should I? Is this a rad thing to share, a little app to fill with your own trip? Or is it the kind of personal tool that's better left personal? I made it for me, but maybe it’s good enough to share?

Reply and tell me. And if you've been sitting on your own "I wish this were an app," consider this your sign. Go build the rad thing. I want to hear about it.

Last week, we announced a significant update to NotebookLM with the addition of an integrated cloud computer for each notebook. The announcement focused on how it helps users go deeper with research and do more within notebooks, which is the right way to frame an announcement on a large-scale Google blog. There, we’re speaking to a broad audience spanning press, users, and influencers interested in keeping tabs on what’s new at Google.

Here on my personal Substack I have a bit more editorial license, so I wanted to share a bit more about why I’m so personally excited about this update. It boils down to one question: what changes when a research tool stops being just a place to ask questions and also becomes a place where work actually happens? That is the part I find interesting here. NotebookLM is becoming a place to do real work, grounded in information you trust.

A fundamental architecture shift

With these changes, NotebookLM now has a more explicit execution layer. Everything remains rooted in source grounding, but it can now route work into a dedicated runtime, choose between direct reasoning and tool use, and return structured artifacts instead of only text. That is a bigger deal than it sounds. Once you separate planning from execution, you stop forcing one model call to do every job in the stack.

For most of its life, NotebookLM did all of that in a single model call that read your sources, reasoned, and answered, all inline. Underneath this launch, the same work now flows through three distinct layers:

1. Grounding. The work stays anchored to sources you provided or approved, exactly as it always has, and it remains the foundation everything else sits on.

2. Orchestration. The model decides what the task actually needs, whether that’s a direct answer, a prebuilt skill, or some custom code, and routes it accordingly.

3. Execution. The chosen path runs in a dedicated cloud environment and hands back a finished artifact, not just text.

The cleaner that contract between layers, the more capability we can add without the whole thing turning into a brittle prompt chain, with no single model call stuck being analyst, formatter, and renderer all at once.

The cloud computer behind the notebook

The most interesting piece is that execution layer. In practical terms, it’s a dedicated virtual machine with a set of skills the agent harness can reach for. Gemini can either write bespoke code for a one-off task, or call a pre-made skill that already knows how to produce a given type of file.

What changes here is that the model no longer has to improvise an entire workflow in one shot. It can choose a path and then delegate, which moves the interesting design question from “can it answer?” to “how well does it decide what to do?” An analytical task can drop down into code, a repetitive one can hand off to a skill that has done it before, and a request for a structured artifact can be packaged into the right format, with each one routed deliberately rather than crammed into a single response.

That deliberate routing buys repeatability as much as flexibility. When the same class of task gets handled the same way each time, you get something you can build a workflow on top of, rather than a result that happens to impress you once and then behaves differently the next time you ask.

What the runtime can produce

There’s more inside that environment than you might expect. Instead of a thin wrapper that knows how to emit a handful of file types, it’s a genuine computer with a real toolchain. There’s a full data stack (pandas, polars, DuckDB), proper modeling libraries for when a task needs them (scikit-learn, XGBoost, PyTorch, statsmodels), and rendering for nearly anything you’d want out the other side: matplotlib, Plotly and seaborn for charts; python-docx, python-pptx and openpyxl for Office files; reportlab and a LaTeX toolchain for typeset PDFs. It can OCR a scanned document (Tesseract), pull tables out of a messy PDF (pdfplumber), work with audio and video (ffmpeg, librosa, moviepy), do geospatial work (GeoPandas, Shapely), and reach for domain-specific libraries like QuantLib for finance or pydicom for medical imaging when a notebook calls for it.

I’m listing names on purpose, because the breadth is the point. The skills sitting on top of that stack are reusable capability modules shaped around tasks rather than file formats, and they cluster into a handful of things that matter:

• Code execution: the backbone. It lets NotebookLM move from summarizing a source to analyzing it, computing metrics, testing assumptions, and surfacing patterns a plain-text response never could.

• Web research and source discovery: for when you start with a loose idea instead of a finished source set, and need the system to go find and organize the material first.

• Documents and reports: pdf, docx, and markdown, the briefs and writeups where most real work actually lands.

• Charts and visualization: generated straight from the analysis, so the visual stays tied to the source instead of being rebuilt by hand somewhere else.

• Spreadsheets and structured exports: xlsx, csv, and json, the bridge from a notebook answer into scripts, dashboards, and whatever’s downstream.

• Presentations: pptx, turning source material into a communicable narrative without rebuilding the structure from scratch.

• Images: diagrams and supporting visuals (yes, Nano Banana) for when the work needs more than text.

What ties these together is that NotebookLM can now choose a file, a chart, a deck, or a structured export as the appropriate response to a given task, instead of leaving you to export and reformat everything by hand afterward. Plenty of tools can generate something that looks finished, but far fewer can do it while keeping the result tied back to the sources it came from, and that tie is the part I care about most.

What people actually did with it

I don’t have to speculate about the use cases, because I’ve now watched the first wave of real sessions come through, and the pattern is clearer than any roadmap deck. Two patterns in particular stood out to me.

The first is that people aren’t mostly coming to ask what their sources say. They’re coming to make something out of them, and that creative use is where the center of gravity sits: people doing real work in a long tail of languages, not running through a demo in English.

The second is how varied that work is. Here are a few representative examples of the sorts of things people can now do:

• a law lecturer turning a dense set of property statutes into a structured graduate-seminar lecture

• an analyst uploading a stack of quantitative-finance textbooks, then having the system test trading signals derived from market-positioning data

• an instructor handing over a grading rubric and a batch of student assignments, and getting back a scored, formatted spreadsheet

• someone turning a single research topic into a short video script, a slide deck, and a companion podcast, all in a language other than English

• a reader sorting a pile of tech-conference announcements into clean thematic buckets

None of those is a “summarize my PDF” request. They’re people using NotebookLM to do the next step of the work, and in each case the whole thing stayed inside one grounded environment, which meant fewer tool switches, fewer places for the workflow to break, and an output that still traced back to the sources.

It’s changing the question people bring to the product, too, shifting from “What do these sources say?” toward “What should this become?”

Why this is the right shape

I’m betting on this direction because the division of labor finally feels right. NotebookLM stays grounded, which is non-negotiable, but the contract underneath is cleaner now: the model handles intent, the runtime handles execution, and the skills handle repeatable output. New capability slots into an existing layer instead of getting bolted onto the side.

Grounding is also what makes the execution layer credible in the first place. Once a product starts producing charts, spreadsheets, and decks, the thing readers need to trust is that those outputs still reflect the source, and faithful grounding is what NotebookLM has been built around from the beginning. Take it away and you’re left with another chatbot generating confident-looking files.

We’re early here, and not every routing decision will be right yet. But the architecture is the part I find most exciting. The system now has a real shape for turning knowledge work into finished outputs, which feels like a bigger change than the model simply getting better at conversation. More and more, it feels like a tool that helps you make the thing you came for, instead of stopping at an answer about it.

So I’ll leave you with the question I keep asking my own team: what would you build first, if your research tool could actually run the work?

— Trond

Google I/O this year was about what comes after chatbots: agents everywhere, Gemini underneath everything, and a product stack built for a different way of working.

I’ve been going to I/O since the first event in 2008. I was there when everyone fit on the plaza outside Charlie’s and when Sergey had sky divers deliver Google Glass to the stage at Moscone. The event has changed a lot, but the pattern this year felt familiar: lots of launches, but one clear story underneath them.

We didn’t really announce anything new for NotebookLM at I/O. Instead, we shared that people have created more than 1.5B notebooks, slide decks, podcasts, and other artifacts since launch. That is not the flashiest announcement, but it matters because it says the product has moved past “is this real?” and into “how much bigger can this get?”

NotebookLM has found a real audience by delivering something people understand and value. When I talk to people about how they use it, I’m always struck by the ingenuity they bring to it. The roadmap is still driven by user value, and that usually means quiet progress instead of big stage moments. Audio Overviews in 2024 was one memorable exception. Most of the time, the product just keeps getting better.

I tend to remember maybe two or three announcements from each I/O (out of 100). This year, people were looking for Gemini, agents, and augmented reality. I think the show delivered on those dimensions, so I’ll focus on the parts that felt most important to me.

Gemini 3.5 Flash Led the Show

Gemini 3.5 Flash was the centerpiece of I/O this year. AI coding is one of the places where LLMs have found real product-market fit, so the natural comparison is coding-specific benchmarks. That matters, but speed is the real story here. Flash delivers strong performance at a much faster pace than the alternatives.

I’ve been using it for awhile now, and it’s overtaken other models in my workflow. I knew it was good when I found myself annoyed not to have it on personal projects at home. With access to pretty much everything else on the market, I still missed Flash. Once you get used to that speed, it is hard to go back.

That Gemini 3.5 Flash is the model Google used to define I/O is telling. The pitch is not just that it is better. It is that it is fast enough to be useful across the places where Google wants AI to live: Search, Android Studio, AI Studio, and the broader developer stack via a refreshed Antigravity. Google is positioning this as the practical default for agentic work, not just the model you benchmark and admire.

Flash is also strong at tool calling, which is essential for agentic workflows. It’s the first model I’ve used that feels like it understands how and when to use the range of tools and skills I make available to it. That changes the experience a lot, because the model is not just answering well. It is taking action well. Runescape-Bench is one benchmark that tries to measure that, and Flash leads there too:

The important thing here is the shape of the claim. Google is no longer arguing for intelligence in the abstract. It is arguing for a combination of speed, capability, and action. That is a more mature product story, and I think Google is right to bet that users will care more about that than model theater.

If I had to boil it down to one line, it would be this: Gemini 3.5 Flash is Google’s way of saying the next frontier is not smarter chat. It is faster things that work.

Gemini Everywhere

Once you see what the model announcement really means, the rest of the event snaps into focus.

Antigravity, the new agent-first developer environment, shows how Gemini 3.5 Flash changes AI coding tools. Gemini Spark is about more personal agent experiences. Managed Agents in the Gemini API point in the same direction. So do the Android Studio updates. So does the way Google keeps moving agentic features into Search and the Gemini app itself.

That is the pattern worth calling out: Google is not trying to win on one heroic app. It is trying to make every surface speak a common agentic language.

That means:

• Search becomes more agentic.

• Developer tooling becomes more agentic.

• The Gemini app becomes more proactive.

• Android becomes a place where AI helps build, not just consume.

That is why I/O felt less like a product launch and more like a platform reset. Google is centering on a single idea: if the next interface is conversational and task-driven, then Search, Android, Chrome, and the rest of the stack need to lean into that way of working.

My takeaway

Google I/O was not a year of isolated launches. It was a shift in how Gemini is becoming the platform for the company. NotebookLM showed that some of Google’s AI products already have real momentum with real users. Gemini 3.5 Flash was the core technical statement. Antigravity, Gemini Spark, Managed Agents, Search changes, and the Gemini app all pointed to the same expansion.

The line tying all of it together is simple: Google wants Gemini to become the default layer across its ecosystem. The question now is not whether Google can ship AI features. It clearly can. The real question is whether these systems become useful enough that people trust them with real work. Keynotes are not the story. The story is whether the thing saves time in ordinary moments.

If Google can make Gemini a trusted layer underneath your work, your search, and your devices, what do you actually want it to do better?

I was three sips into a whiskey at the Booking Office at St. Pancras when I clicked “buy.”

It had been a long day. I’d spent it at the Google DeepMind office in London, back-to-back meetings with researchers. One of them had assembled a Reachy Mini with his kids the prior weekend and was now poking at it as a way to interact with his AI work. He was delighted. I’d been talking about Stella all day. The pitch was basically: build one with your kids, see what an embodied agent actually feels like, find out if any of this matters when it has a face.

My favorite kind of decision: the kind you make in a bar.

My daughter lost her mind when I told her.

Not the polite “oh cool, Dad” kids do when you’re trying. The actual counting-down-the-weeks version. I told her one Sunday morning over breakfast, and within about ninety seconds she had named it.

Merc.

She did this entirely on her own. Stella — our family AI, four months old now and running most of the calendar/email/lights/kitchen-display layer of our life — is named after the ship computer from Miles from Tomorrowland. Lucy watched that show a lot when she was little, and she remembered, without me saying a word, that there was a small robotic sidekick on the show called M.E.R.C.

She just put it together. “If Stella is the ship, Merc is the helper.” I nodded like I’d been planning that the whole time. (I had absolutely not been planning that the whole time.)

It was Merc the moment she said it. She counted the weeks until the box arrived.

The kit landed this past week.

For the uninitiated: the Reachy Mini is an open-source desktop robot from Pollen Robotics (now part of Hugging Face). It’s small. It has a head that moves, two antennas that swivel, and a face that, well, looks at you. It does not do laundry. It does not walk. It sits on a desk and emotes and listens and looks at you, and that’s the whole pitch and basically the whole point.

It also arrives in a hundred small pieces in a plain cardboard box. The kind of unboxing that’s simple and practical. I like simple.

We did the build over Friday night and Saturday afternoon, May 1st and 2nd. Both kids were in. Leo and Lucy took turns: one would line up the part, the other would seat the screw, then they’d swap. I sat with the manual open and tried to be useful.

“Useful” is generous. My job was closer to sous chef than anything else: hand them the right piece, double-check orientation, point at a thing on the page. The kids did the actual building. This is how it should be. Pretty much the whole point of being a dad.

A few parts went on backward and had to come back off. Nothing dramatic. The kit is forgiving: the manual is good, the parts are well-labeled, and Pollen has clearly thought about the fact that small, distractible humans are going to be the ones putting it together. It went smoothly. It was the right amount of fiddly to feel like an accomplishment without becoming a chore.

A few specs, for the curious. Eleven inches tall, a kilo and a half on the desk. A 6-DOF head — pan, tilt, roll — on a body that can rotate a full 360°. Two animated antennas. A wide-angle HD camera in the face, a 4-mic array around the base, a 5W speaker. We got the Wireless SKU, which means a Raspberry Pi CM4 onboard, a battery, and an IMU. No tether, no laptop next to it. Fully autonomous. $299 to start, Python-programmable, and Pollen says the build is a 2–3 hour job. Ours took Friday night plus a chunk of Saturday afternoon. We were in no hurry, and there were two kids negotiating turns with the screwdriver.

The first time we powered it on, Merc moved its head.

That was it. But it was exciting.

Two kids leaning over a kitchen table watching a small robot turn its head and twitch its antennas. Leo immediately wanted to know if Merc could see him. Lucy wanted to know if Merc could hear her. We connected a game controller and they took turns (well, argued about taking turns mostly) controlling the various movable parts.

You can talk yourself into being unimpressed by a head and two antennas. Don’t. Watch a kid watch a robot look at them. Tell me what that feels like.

So why a robot?

I’ve written a lot about Stella. She’s been online about four months. She runs the calendar, watches the email, runs the kitchen display, controls the lights, calls the car dealership. She’s useful — and I want to be honest about this — she’s also a little transactional. And temperamental as new technology tends to be.

You ask Stella a thing. Stella does the thing. Stella reports back. Most of the interaction lives in chat windows: Telegram threads, the kitchen display, the occasional voice exchange. It works. But it’s mostly typing into a box. And sometimes talking into a box that doesn’t respond correctly.

This is the trap I keep noticing with personal AI. The capability is enormous and the discovery is terrible. There’s so much these systems can do, and so few natural ways for a family to find out what those things are. The interface is a text box. The only question that gets answered is the one you knew to ask.

Embodiment is an experiment I wanted to try. Not because a robot can do more than chat (Merc is dramatically less capable than Stella) but because a robot is there. It sits on a desk. It looks up when you walk in. It’s a thing you notice, not an app you have to remember to open. When the kids walk past it, they’ll ask it questions they’d never have thought to type into a chat window.

That’s the thesis at least. A friendly, cute robot with a personality, sitting in the room with you, makes the family AI feel familiar instead of transactional. Less Slack channel, more pet that knows things.

We’ll see if I’m right. That’s why we’re testing it. I haven’t actually gotten that part working yet.

What’s next

The point of Merc isn’t really Merc. It’s Merc plus Stella. The kids and I have been brainstorming, and the list is long. A few things near the top:

• Merc as a voice surface for Stella. Stella already has a voice (it runs locally on the Mac Mini’s Neural Engine), but right now she’s tied to the kitchen display and a wake word. The kids want to be able to walk up to Merc and talk to Stella, and have Merc do something expressive while Stella thinks. Antennas up while listening. A little head-tilt when she has a question. Antennas leaning forward when she’s working on it.

• Merc as a hands-free way to ask Stella for things. Not literally hands. Merc has no arms. But Merc can ask Stella to do things on the kids’ behalf. “Merc, remind me to water my plant” → Merc → Stella → calendar entry. The bridge for this is already running in test (auditable JSON in ~/Stellasphere/agents/shared/merc-to-stella/inbox/, because of course it is) but the kids haven’t gotten to play with it yet.

• Games. Lucy wants to play 20 questions with Merc. Leo wants Merc to play redlight greenlight. I have no idea how either of those will go and I cannot wait to find out.

• An expressive vocabulary. We’ve defined a small set of moods Merc can be in: quiet, wake, sleep, thinking, happy, curious, alert, reminder, stop. Each gets its own antenna and head behavior. The goal is for Merc’s body language to be readable from across the room without anyone needing to look at a screen.

It’s early. The whole thing is only a few days old. The kids haven’t really lived with Merc yet. We’re going to find out what works by playing with it for a few weeks and seeing what they actually reach for. It’s called building to learn, and it’s what we do here.

One more thing on the build-with-your-kids piece of this.

I keep coming back to this in everything I do with AI. You can read about agents. You can take a course. You can watch a demo. None of it lands the way building one with your hands does. And building one with your kids is a level beyond that, because they’re going to ask questions that don’t have textbook answers, they’re going to discover behaviors you didn’t anticipate, and they’re going to find the bugs faster than you will.

Lucy did the product positioning for me on the ottoman in the family room before we’d seated a single screw. “Stella’s the ship. Merc’s the helper.” That’s the whole thing. I’ve been trying to articulate that thesis in long-form posts for months and my 12-year-old beat me to it in eight words.

If you’ve been waiting for the right entry point into the embodied side of all this, Reachy Mini is a very friendly first robot. The community is great. The build is approachable. The price won’t ruin you. And there is something satisfying about a kit that arrives in a box and ends with a small, curious robot watching your kid from across the kitchen.

10/10 would buy a robot from a bar again.

If you’re building anything embodied with your family, Reachy or otherwise, I want to hear about it. What’s working? What surprised you? Reply, or find me on X. I read everything.

Merc, if Stella ever forwards this to you during a heartbeat: hi, buddy. Welcome to the Stellosphere. Play nice with the ship. Wuellners, Let’s Rocket!

Earlier this year I wrote that 2026 would be the year AI learned to compose UIs in real-time. Users watching interfaces assemble live. Interactive components with full state. Agents drawing UI on the fly specifically to solve your precise need.

Looks like I nailed the prediction, but underestimated the speed and the weirdness.

By May 2026—literally right now—an entire ecosystem is crystallizing around this notion. A2UI. Shadify. AG-UI. And a bunch of others I’m still discovering. All shipping real-time, generative UIs. Software is about to get rad!

My Original Prediction

Here’s what I actually wrote back in January (from “12 AI Predictions for 2026”):

Chat has been the predominant interface for AI. There are interesting examples of linking useful UI components into a chat interface. In November 2025, Anthropic and OpenAI partnered to release the MCP Apps Extension—a specification that brings standardized interactive UI capabilities to the Model Context Protocol.

This will lead to a new class of apps that self-compose UIs in real-time. They’ll draw components for existing patterns from vast libraries of capabilities, but when a novel need arises, they’ll create bespoke experiences to match real-time user requests. The very definition of software will forever change, and 2026 is when we see the beginnings of this evolution.

So I was thinking: MCP standardizes things → builders adopt → apps ship interactive UX. A nice sequential story. But that’s not quite how it went.

A lot started unifying when Google shipped A2UI. It happened before there was consensus we needed it. Builders started using it without waiting for the ecosystem to debate. The entire thing organized backward—not “let’s standardize first, then build,” but “let’s build, and the standard will emerge from what works.” I kind of think that trend is here to stay.

Vibe coding might play a role, but that’s not all of it. We’re all moving too fast for official standards to keep up. Standards organizations are going to need to stay on their toes. Or maybe it’s already too late. Yeah, they’re probably more historians than futurists now.

Real-Time UX is Shipping Now

Before A2UI, this space was a mess. Everyone building generative UI was inventing their own protocol. Custom WebSockets. Ad-hoc JSON schemas. No way for an agent trained on one platform to work on another.

A2UI is solving that. It’s boring on purpose—just a declarative way for agents to describe UIs. Not code. Not instructions. Just “here’s what this UI looks like.” JSON you can stream, component by component, platform by platform. This is key because now:

• Web Components, Angular, React, Flutter, SwiftUI, Jetpack Compose can all understand the same language

• One agent can work everywhere

• It requires no translation and no glue code.

It’s all pulled together via progressive rendering. As the agent thinks and generates components, the client renders them live. You’re not stuck watching a spinner. You’re watching the UI assemble. Components appearing. Lists populating. Forms showing up. All while the agent is still thinking.

That’s becoming the default UI paradigm. 2026 isn’t just the year where UI is generated in real time. 2026 is the year of generative UI.

Let’s talk about a few of the big things driving this space:

Google A2UI: The Shared Protocol

Google shipped A2UI. Agent-to-User Interface. It’s deceptively simple.

Instead of agents generating code—React components, HTML, Flutter widgets—they generate a flat JSON description of what the UI should look like. That’s it. One agent, any platform.

Version 0.9 shipped in April 2026. Multiple renderers are already shipping stable versions. This should have taken 18 months to standardize. It took 6.

Shadify: Real time UI component system

Describe a UI in plain English. Get a live, interactive shadcn/ui page back. Export it as clean React code. This makes it really easy to deliver a consistent UI because the agent isn’t inventing new components, it’s remixing what already exists. But the remix is custom and it’s specific to your request. And it all happens in realtime -- which is amazing. And it’s open source under an MIT license which I like.

CopilotKit: Making Generative UI the Default

CopilotKit has been on the Generative UI train for awhile and is building some of the most interesting capabilities with it. They can help an Agent generate a stargazing app with live sky compass, 3D solar system, interactive dashboards with charts. All in real-time, in response to requests. It’s pretty amazing.

This isn’t a one-off demo. It’s a framework thousands of developers are using to compose UIs on the fly. This space is moving so fast, we already have framework companies gaining traction:

AG-UI Protocol: Streaming Agent Events in Real-Time

AG-UI is an open, lightweight, event-based protocol that standardizes how AI agents connect to user-facing applications. It standardizes how agent state, UI intents, and user interactions flow between your model/agent runtime and user-facing frontend applications.

This allows agent interactions to feel more real-time because instead of waiting for a complete response, you get play-by-play updates:

“Agent is analyzing...” → tokens start flowing → “Generating content...” → UI updates as events arrive.

Combined with A2UI, this creates a full stack: agents think out loud, UIs render incrementally, users experience live feedback instead of spinners. It’s a key part of how realtime generative UI is beginning to win.

Why This Happened in 6 Months (Not 12)

A2UI standardized what was scattered. Before: WebSockets, custom protocols, one-off implementations. After: One open protocol, multiple renderers ready, agents speaking the same language.

• The ecosystem coalesced simultaneously. MCP + A2UI + AG-UI = A complete stack. Not pieces in sequence—a system shipping all at once.

• Incentives aligned perfectly. Developers hated code → compile → deploy. Streaming eliminates it. Agents eliminate manual state changes.

• Economics improved. Streaming is cheaper than batch. Partial results matter before the full response arrives. Agents calling tools directly mean fewer context switches and lower costs.

I predicted: Self-composing UIs. Bespoke experiences. Real-time composition. All of that landed and more. Today, you’re not asking the AI to write code. You’re delegating state changes to an agent that speaks your app’s language. The UI is just the visualization layer.

What’s Coming

The shift from “chat interface” to “standardized protocols for agent-composed, real-time, voice-driven state manipulation” happened in 6 months instead of 12. And it’s going to accelerate from here. I see more on the horizon:

1. On-device A2UI. All of this currently assumes live agents in the cloud. Imagine agents running locally, rendering UIs via A2UI to your device. Lower latency. Privacy. No API dependency.

2. Persistent agent memory in UX. Right now UIs are stateless across sessions. What happens when your agent remembers your interaction patterns, preferences, and context, and adapts the interface itself? The UI becomes genuinely personal.

3. Multi-agent, shared-state collaboration. Multiple agents, multiple users, one shared canvas. Teams collaborating through agent-composed interfaces. No more meetings about how to format the data—the agent generates the interface.

4. Accessibility baked in by default. Voice + state is already 10x more accessible than clicking. The next step: Agents that automatically generate semantic HTML, alt text, keyboard navigation, and ARIA as they compose. Accessibility isn’t an afterthought—it’s in the protocol.

5. Agents in your existing tools. A2UI renderers shipping inside enterprise software. Slack. Figma. Notion. Your existing tools suddenly have agent-driven, real-time UI generation built in. The line between “app” and “agent” disappears.

The definition of software hasn’t changed yet. But the friction is gone. By end of 2026, “real-time generative UI with A2UI” will be as normal as “responsive design” was 10 years ago. The developers who ship voice-driven agents that manipulate state directly win. The platforms that make this the default layer—not an add-on—own the next era.

If you’re building tools, this isn’t “the moment.” This is accelerating. The infrastructure exists. The pattern is proven. The ecosystem is moving. The question is simple: Are you shipping voice-driven agents that manipulate state in real-time, or are you still asking users to click?

Because by the time you finish reading this, someone shipped something new that’s doing exactly that.

Prefer an audio overview? I got you:

I’ll admit, I was deeply conflicted. I have loved everything about my time in the GenMedia group, and the teams we’ve built in Labs are truly special. Project Genie is delivering breakthroughs on an entirely new universe of interactive experiences. Flow Music (formerly producer.ai) is already pushing the boundaries of what’s possible with sound, and Doppl is bringing our virtual try-on technology to life in shopping. Leaving that work, and the people who made it possible, is not an easy choice.

But after consulting my council of advisors and digging into the roadmap, I ultimately decided that the challenge ahead at NotebookLM is one I’m eager to take on. I’m accepting this with a healthy dose of “uncomfortable excitement.”

I’m also fully aware that I am stepping into the footsteps of people I deeply admire. Raiza, Josh, and Simon have built an incredible foundation of user love, rooted in trust and genuine utility. My goal is to build upon that culture with the same care they’ve shown.

To the teams I’m leaving behind, Elias, Elliott, Sakshi, Pavan, Nicole, Seth, and Diego you are an absolutely unmatched group of product leaders. While I’m literally just moving across the hall, I will sincerely miss working with you as closely as I have.

The Opportunity

NotebookLM is a crown jewel at Google, and I take the responsibility of stewarding it seriously. It is a profound honor to guide this product into the agentic era.

NotebookLM isn’t just a novelty, it is a genuine partner for research, learning, and project organization. Its fundamental promise is its grounding. It is built entirely from the sources you provide, with clear citations that let you verify every output yourself. In a world where healthy skepticism of AI is necessary, that transparency is our strongest asset.

As I settle into this role, I’m focused on one central question: How do we scale NotebookLM without losing touch with what makes the product special?

We recently began rolling out our integration into Gemini, which will introduce NotebookLM to millions of new users. The risk is that the product identity becomes diluted as we reach a broader, more diverse audience. We’ve delighted our early power users by focusing on the “learning journey,” turning dense information into synthesized “magic.” As we grow, my priority is ensuring that our commitment to grounding and trust remains the North Star. We will use these new surfaces as a doorway, while keeping the “core notebook” experience a place where the real work happens.

What Comes Next

Technically, I joined the team last week. You may have noticed I missed my usual weekly post, a first since I started publishing here. I made the decision to prioritize getting up to speed and truly understanding the team’s needs before I picked up the pen again. It’s been an incredible first week.

I’m thrilled to be partnering with this team to grow both our product and our culture. We are on a mission to build tools that help people learn and work more naturally, not by replacing them with AI, but by partnering with them to become the best versions of themselves.

I’m also committed to staying engaged with the people who use this product. You can find me here on Substack, on X, and on Reddit. I want to build in the open, stay close to your feedback, and showcase our team’s commitment to building this with you.

We have a lot of work to do, and I couldn’t be more excited to get started.

— Trond

Last week Google dropped Gemma-4 and it is a ridiculously awesome piece of AI engineering.

Gemma4

Everyone is obsessing over the flagship 31B model pushing the limits for open limits on the Arena AI leaderboard. This is insane for an open model, and pretty incredible for it to be available via a permissive Apache 2.0 License. But if you want to run Gemma at home, the real prize is the 26B Mixture-of-Experts (MoE) variant. You lose minimal ELO impact, but can easily run it on readily available hardware.

It packs 26 billion total parameters but only activates 3.8 billion per token. You get a massive 256K context window, and it to rank #6 overall among all open models. More importantly, it runs blazingly fast on a Mac Studio. Which I happen to have because my photography habit “required” I upgrade my old device last fall.

I’ve been itching to rip the cloud brain out of my OpenClaw system and wire up Gemma-4 locally since Tris told me about it a few weeks ago. But when a system is working just fine, it’s hard to justify breaking it just to see what happens.

Then Anthropic did me a massive favor: they killed API access for tools like OpenClaw on my current plan. I had been using a mix of Gemini and Claude in my system, but I was looking for an excuse to change so… twist my arm. I took the opportunity to pull the plug on Claude and move entirely to local silicon.

The Swap

You can’t just change a URL to swap an agent’s brain. But thanks to Ollama, it was pretty close. I’m on vacation this week with my family in London, so I needed to be able to do brain surgery from my phone.

So there I was, sitting on the Tube, dragging two tired kids back to our Notting Hill Airbnb, when I read the update on X from Boris Cherny about the Claude change. I fired up Telegram right then and got the process going. Kick off a command at a station, let Stella cook between stops while I was without service. Actually works pretty well!

We were returning from the British Museum after checking out the Parthenon marbles depicting the birth of Athena. (They really should be in Greece where they belong!) In the myth, Athena springs fully formed from the head of Zeus. I mention this because I named my Mac Studio Athena when we set her up last year. It feels poetic to have my own Athena, the Mac Studio node sitting 5,000 miles away in California, getting a new brain birthed directly from Gemini.

I started by having Stella remotely log into Athena to kick off an upgrade of Ollama to a version that supports the new Gemma-4 manifest format. Then it downloaded the ollama/gemma4:26b model. Took about 10 seconds at the Embankment stop on the Tube to get this started.

Next came the OpenClaw gateway patch for Stella on my Mac mini. I pointed the provider to http://athena.local:11434/v1 and set the default model to ollama/gemma4. A quick request at the Sloane Square stop.

I applied the config and restarted the system. Then I ran openclaw doctor from my phone via Chrome Remote Desktop. This was actually the hardest bit because it’s a bit of a pain to drive a terminal via CRD on a phone. The text is just so small! Got that done before we pulled into Notting Hill Gate.

After the process ran, everything just worked. Before we got back to the AirBnB.

Running on my own silicon

Now Stella is back online, processing prompts and executing workflows, completely untethered from the cloud.

The API bill is gone. The latency is near zero. And because it runs entirely on Athena, the data never leaves my house.

In fact, I started writing this article from my phone while my son Leo and I rode the train out to Richmond for a Ted Lasso tour. I’m finalizing it right now over a pint at The Prince’s Head (the pub from the show). While we ate, Stella formatted the draft and mapped out our train route to our next stop—a street photography tour near The Shard. It did all of this on local silicon 5,000 miles away, streaming the results right to my pocket with no delay.

Builder’s note: Some migrations take weeks. Some I completely whiff on. This one just worked. Watching your own AI agent run entirely on your own hardware is pretty awesome. You should try it.

Over winter break, my 12-year-old called me over to look at her laptop.

She was studying for a quiz on a book she’d been reading. She had flashcards. And a podcast about the book — two AI hosts walking through the themes, the characters, the things that would probably show up on the test. She’d made both herself using NotebookLM. I didn’t even realize she knew about the product. Which is funny, because I helped build the Audio Overviews feature she was showing me.

It turns out NotebookLM is very popular at her school both among students and teachers. She had been using it all semester to build herself a study system. On her own. Which is pretty cool.

That’s the moment I understood how impactful NotebookLM can be.

When we were developing Audio Overviews at Google Labs, we knew it was a novel way to help people engage with their content. The immediate reaction when we shipped was delightful — people were surprised that such lifelike audio was possible. But the real impact turned out to be something we felt more than predicted: it reached people who felt they learned best by listening. People who’d been handed PDFs and book chapters their whole lives, formats they struggled to connect with.

That realization pushed the team further. Mind maps for content that benefits from clearly outlined structure. Slide decks for information that combines visual and written information. Videos for content that benefits from a deeper telling of the story or motion to bring ideas to life. Flashcards to help people test themselves and secure knowledge.

Builder’s note: This isn’t a feature roadmap built around “what’s technically cool.” Each output format is a bet that the same information, delivered differently, becomes useful to a different person or within another context. Google’s mission is to organize the world’s information and make it accessible and useful. NotebookLM is what “useful” looks like when you take it seriously.

What NotebookLM actually is

Most people I talk to think of it as a chatbot for their files. Upload a PDF, ask it questions. That’s not wrong, but it’s missing most of what makes it interesting.

NotebookLM is a research and learning environment grounded entirely in sources you choose. You bring the material; the system stays inside it. Every answer comes with citations back to your specific sources, which massively reduces hallucinations. That constraint is the whole point.

The interface has three panels. Sources on the left: everything you’ve uploaded or linked. Chat in the middle: ask questions, get answers, save the good ones as notes. Studio on the right: where you generate outputs to make your information more useful.

You can put in: PDFs, Google Docs, websites, YouTube URLs, audio files, raw text, even your own notes. I find it helpful to share voice recordings of my own disorganized thoughts about the topic. The system does the organization automatically, which feels like magic.

What you get out: Audio Overviews, Video Overviews (in three formats, including a Cinematic mode that is genuinely stunning), Mind Maps, Infographics, Slide Decks, Flashcards, Quizzes, Briefing Documents, Study Guides, FAQs. All grounded in your sources, all completely different ways into the same material.

The learning styles myth

Before I get into how to use NotebookLM, I want to flag something.

The obvious framing for a tool like NotebookLM is: figure out your learning style, pick the matching output format. Visual learner? Mind map. Audio learner? Podcast. You get the idea.

That framing is wrong. Not intuitively wrong — it feels right, which is why over 70% of educators still believe it. Scientifically wrong. The research on “learning styles” as fixed, matchable preferences has been thoroughly debunked. A 2024 meta-analysis across more than 1,700 students found that matching teaching style to learning style had no meaningful impact on performance. The effect size is 0.04. Essentially zero.

What the research does support is something more interesting: combining formats builds better understanding than any single format alone. This is called Dual Coding Theory. When you pair verbal information with visual information, the two memory systems reinforce each other. You retain more. You understand deeper.

My daughter didn’t pick her learning style. She made flashcards and a podcast. Both. That’s Dual Coding in action, figured out by a 12-year-old without anyone explaining the theory to her.

So the tips below aren’t about finding your type. They’re about using the tool the way the science suggests learning actually works.

Six tips for getting started

1. Start with something you actually need to understand

Don’t test NotebookLM with a random PDF just to see what happens. Pick something you’re genuinely trying to learn — a topic at work, a book you’re reading, a subject your kid is studying for a test. The tool works best when you have a real question. Generic inputs produce generic outputs. Here are a few topics already set up as notebooks if you want a running start:

• Introduction to NotebookLM

• Parenting Advice for the Digital Age

• Jane Austen: The Complete Works

• Secrets of the Super Agers

2. Add more sources than you think you need — and let NotebookLM find them

One document produces one-document results. A YouTube lecture, two articles, and a PDF on the same topic produces something much richer — the AI can find connections and disagreements across sources that you wouldn’t catch reading them separately.

Here’s the part most people miss: you don’t have to find the sources yourself. Hit the Discover button in the Sources panel, describe what you’re trying to learn, and NotebookLM searches the web and returns curated sources with annotated summaries. Add the ones that look useful with one click. That’s how I built the notebook backing the research in this article — I described the topic, let it do the research, and had a solid set of sources in a few minutes.

[🔗 You can explore the full notebook here: The Learning Styles Myth]

3. Try the Audio Overview first

It’s the fastest way to understand what NotebookLM pulled out as the key ideas. Even if you don’t normally listen to things, run it once before generating anything else. It surfaces connections and frames the material in a way that makes every subsequent output more useful.

[🎧 Listen: Audio Overview — The Learning Styles Myth]

4. Don’t stop at one format

This is the tip that changes how you use the tool. Generate the Audio Overview. Then generate the Mind Map. If the content is complex, try the Cinematic Video. The learning styles research says picking your “type” doesn’t help. The Dual Coding research says layering formats does. Use both.

[🗺️ Mind Map — Key concepts from the research]

[📊 Infographic — Beyond the Myth: A Science-Based Guide]

[📽️ Video Overview — Cinematic Video: The Learning Styles Myth]

[📑 Slide Deck — Download the full presentation]

5. Use the Steering Prompt

Every output in the Studio panel has a customization option before you generate. You can tell NotebookLM to focus on specific sources, approach the topic from a particular angle, or explain things at a certain level. Most people skip this and wonder why the output feels generic. “Focus on the disagreements between these sources” produces a completely different Audio Overview than the default. Try it.

6. Save the good stuff to Notes

When the chat surfaces an insight worth keeping, save it. Notes live inside your notebook alongside the sources. Your own thinking and the AI’s synthesis sit in the same place. Over time, the notes become a record of how your understanding of a topic developed — which is useful in a way that a chat history never is.

The thing my daughter understood

She didn’t ask what kind of learner she was. She made flashcards to test herself and a podcast to listen to while she studied. She layered formats because that’s what felt right, and it turns out the science agrees with her instincts.

NotebookLM is good at a lot of things. The Cinematic Video is genuinely amazing — watch the embed above if you haven’t yet. The Mind Map makes complex source material navigable in a way that’s hard to describe until you’ve dragged a branch around and watched the rest reorganize.

But none of that is the point. The point is that the same information can be made useful in a dozen different ways, and most of us have been stuck with one or two of them our whole lives.

My daughter had a quiz to pass. She figured out how to prepare for it. You can do the same thing with whatever’s on your plate. Big sales call? Load up the client’s annual reports. Job interview? Drop in the job description and let Discover find you prep material. Presentation next week? Paste in your notes, record a voice brain dump, and watch it turn into slides. Even if it only breaks the writer’s block, that’s worth it.

So, what are you trying to understand right now? Have you tried more than one format yet? I’m curious what you’re hoping to learn and what works for you!

— T

I had lunch with my friend Ronny at work a few weeks ago and we were (shocker!) talking about AI. Specifically, we were talking about how AI is absorbing more of the daily computing tasks that used to require a person sitting at a screen. Scheduling, research, drafting, monitoring, summarizing. The list grows every month.

That led us to a question: if AI agents are becoming the primary operators of computing systems, what happens to all the infrastructure we built so that humans could operate them?

Because that’s what operating systems actually are. They’re abstraction layers. Decades of engineering spent translating machine reality into something a human mind can understand. File systems, because humans think in containers and locations. Graphical interfaces, because humans perceive visually. Process scheduling, because humans experience time linearly. Every design decision in Unix, Windows, Android, ChromeOS and MacOS encodes endless assumptions about who’s sitting at the controls.

But what if the operator we’re building for is no longer a human?

15,000 tokens per second

Let’s take a quick aside. There is a company called Taalas who has recently built an app called Chat Jimmy. It’s a demonstration of their main business: building AI models compiled directly into custom silicon. Not software inference running on a general-purpose chip. The model is in the computer.

Chat Jimmy operates consistently above 15,000 tokens per second. For context, human speech runs at roughly 3 tokens per second. At 15,000 tokens per second, an agent can reason through a complex problem, take multiple actions, and produce a result before a human has finished reading the first sentence of the last response. The inference bottleneck for most computing tasks totally disappears.

If we can get hardware of this nature to a place where local models operate at similar speeds, something shifts with on-device models. They become the computer. And we don’t have an OS built for that world yet.

The big inversion

It won’t be trivial to get 15k tokens per second running on a device you keep on your desk, but it will happen. And it’ll signal a shift in how we work. At current inference speeds, agents wait on compute. The OS doesn’t matter much because the model is the slow part. Get the model fast enough and the calculus flips. The agent isn’t waiting on the model. The model is waiting on the human. I already see some of this happening today.

Humans will be the rate-limiting constraint on our systems.

That’s a completely different world to design for. An operating system doesn’t need to be optimized for human-speed interaction anymore. It needs to be optimized for machine-speed agents that are supervised by humans who still move at human speed. Safety, reversibility, and trust enforcement become the central design problem.

I don’t think anyone has asked what an OS designed for this user inversion would look like yet.

A sketch of the stack

I’ve been thinking about this question and sketching out an architecture that could be interesting. By no means is this thoroughly thought through, so think of it like an outline for what might work. If you’ve thought about this as well or have some ideas, reach out — I’d love to chat.

Layer 0: Linux kernel. Start here and keep it. It handles hardware, networking, and process isolation. These are solved problems not worth re-solving. You can add eBPF programs on top to enforce capability rules at the kernel boundary. No kernel fork required.

Layer 1: Hardware model. An on-device LLM minted into silicon that’s always on, always available, near-zero latency. The runtime talks to this chip in natural language. That’s not a poetic choice. If the model is the chip and human language is its instruction set, then system calls are just English. Every operation in the audit log is human-readable by default. LLMs printed into chips will have knowledge gaps, so this layer gets augmented with an online model for some tasks — but we’ll handle that later.

Layer 2: Agent runtime. This is the interesting layer. It replaces all of traditional user space. It’s the part of the stack that exists to mediate between the machine and a human user. Instead of mediating for humans, it manages agents: their context windows, their tool access, their permissions, their lifecycle.

A few key jobs it handles:

• Context management. An agent’s working memory is its context window, not just RAM. The OS manages context as a first-class resource — what gets evicted, summarized, or persisted across sessions.

• Tools instead of syscalls. Instead of a syscall table, a tool registry. Agents call tools. The registry knows what each tool does, whether it’s reversible, what it costs, and whether human approval is required before execution.

• Capabilities instead of permissions. Unix permissions are a file ownership model. That’s not the right primitive for agents. We need something closer to structured, delegatable, auditable capabilities.

• Reversibility as infrastructure. Right now, every application that wants undo has to build it from scratch. An agent OS should make reversibility a first-class primitive. A staging area for irreversible actions that waits for explicit human approval before executing. At 15,000 tokens per second, mistakes accumulate fast.

Layers 3 and 4: Specialist models and agents. Specialist models load on demand above the runtime — like kernel modules for specific domains: code, vision, planning. Above that, agents run as goal-oriented processes rather than session-bound apps. This is where we can add online models to overcome the natural knowledge gap of an LLM printed to silicon.

I’ve seen this movie before

One of my first jobs at Google was helping build ChromeOS. I know a little about what it takes to rethink an operating system. ChromeOS asked: what could an OS be if the web browser were the only app? The answer was a much more efficient system. Strip away 40 years of complexity built to run native applications. Boot in 8 seconds. Let the browser do everything else.

Today we’re in a similar spot, asking a similar question: what would an OS need to be if you built the whole thing for AI agents instead of people?

I work next to people building the next generation of operating systems and I haven’t even asked them about this. Partly because what I’m sketching is a thought experiment, and I didn’t want anyone else’s answers to constrain mine. The Android and ChromeOS teams have something in development called Aluminum — I don’t know what’s in it, and this isn’t that.

What I do think is clear: every company building an OS right now is trying to figure out how to add AI into it. But the answer we’re looking for isn’t an existing paradigm with AI bolted on. It isn’t ChromeOS or MacOS or Android with a chat window. It’s something designed from scratch around the assumption that the thing doing the computing is an agent, and the human is there to supervise and interact with it — not to drive all the details of making it work.

That’s a different OS than anything that exists today.

Why does this matter now?

The window is specific to this moment. Every day another AI system announces more capabilities for computer use: schedules, remote access, browser automation, skills, MCP servers. All of these systems designed so agents can pretend to operate like people. We’re forcing computers to pretend to behave like humans in order to do things on computers. It’s a strange layer of friction we keep injecting because we’re using a wrench to hammer a nail.

At the same time, local inference is still too slow to run capable agents on-device for everything a modern OS relies on. Even the newest Qwen models on a tricked-out Mac Studio fall short of what we need for agents to run all of the details of these systems continuously. Until we close that gap, agents live in the cloud, on human-designed infrastructure, using agent harnesses to drive systems built for people.

Before long these constraints will be gone. Chips will hit 15,000 tokens per second and will run efficiently on your laptop. Then on your phone. Then on your headphones. Local capable agents will become practical on every device. And everyone who wants to run those agents needs an OS built specifically to maximize that experience, or we’ll keep absorbing friction translating from computer to human and back to computers.

When that OS exists, it will shape what agents can do and how safe they are. If we keep heading down the road we’re on with defaults of agents running on Linux and Windows — systems with no concept of a tool registry, no capability model for autonomous action, no staging area for irreversible decisions — we’re headed for problems.

The time to design this is before the hardware arrives. After it arrives, you’re retrofitting. Or maybe the agents will just do it themselves.

What’s left to ponder?

Because I’m using this as a way to write-to-think, I’m very much missing a lot of detail. A few gaps I’m genuinely curious about:

The Tensor question. Google’s Tensor chips already do pretty impressive on-device Gemini Nano inference. Is it close enough to what Layer 1 needs, and how well-suited is this approach to being what we want exposed to software? This matters for whether we’re talking about new silicon or new software on existing silicon.

The safety research. The capability and reversibility model I’ve sketched is conceptually interesting, but I don’t know how much formal work exists on verifying capability systems for LLM agents. This is probably a known problem in security research that I haven’t fully worked through yet.

The device form factor. A personal AI server — a small always-on box that runs your agents locally — seems like a natural first hardware expression of this. But I’m not sure if that’s the right starting point, or if this begins as a server product and migrates to the edge over time. What do you think?

I find this compelling enough to think more about, which is why I wrote it down.

If you’re working on any part of this stack — hardware, runtime, capabilities, reversibility — I’d love to know. And if I’ve missed something obvious, tell me. What’s the right device form factor for this? Does it start in your living room, or in a data center?

— T

One of my readers sent me an email last week with the subject line “Building Semi-Rad Shit.”

He’d been inspired by something I wrote and stayed up late putting it to work. He trained Gemini to review solar site title insurance exceptions. Then he had it pull utility hosting capacity maps, find available parcels, and cross-reference them against where the grid has room. A list of viable solar development sites, ranked by proximity to power lines. In a few hours.

“Last time I did site identification a few years ago,” he wrote, “I pulled the map data off the utility website into a database, then matched it with parcel data from each county by hand... this time I got a solid list in only a few hours!”

Then, near the bottom of his email, he hit the wall.

“It’s a little annoying that I can’t just have it do things offline, and it can’t send me emails once it’s done the work periodically.”

Almost every builder I know hits this wall eventually. You do something impressive with AI, you see what’s possible, and then you realize: your AI only works when you’re there asking it questions. It goes to sleep the moment you close the tab.

This post is about how to take that next leap with AI.

The gap between an assistant and an agent

Most AI tools are reactive. You prompt, they respond. You close the tab, they stop existing. That’s useful. But it’s not the same as having something working for you all the time.

A proactive agent is different in one specific way: it has a schedule. It wakes up, does a job, and reaches out to you if it finds something it needs your input on. If nothing’s worth saying, you hear nothing. You only know it’s running because, every so often, your phone buzzes with something that is worthy of your attention.

The first time that happens something profound clicks.

The good news is that three ingredients are all you need:

1. A schedule: a system to trigger your agent at regular intervals

2. A task: a specific job with clear instructions about what to do each time

3. A way to reach you: configuration for a chat message or an email

Everything else is implementation details.

Introducing OpenClaw

The most popular tool right now for building these sorts of agents is OpenClaw. It’s a self-hosted AI gateway you run on your own machine. It handles the scheduling, the agent sessions, and the channel connections needed to keep an agent running in a loop. It’s become very popular lately because it can be configured to do just about anything and has a rapidly growing ecosystem of skills you can use with it.

It’s not the only option for this, but it’s the one I’ve actually used to build everything I’m about to describe. It runs on Mac, Linux, any VPS, a Raspberry Pi, or even a Docker container you can run on your main computer. Monthly cost beyond the hardware is just your Anthropic, Gemini or OpenAI API usage. For light-to-moderate agent workloads, this can be had for about $5–20 per month.

Let’s help get you started.

Step 1: Install OpenClaw

On a Mac
I have my OpenClaw running an a Mac mini I wasn’t using, so let’s start with this option. If you’ve been trying to buy a Mac mini recently and found them out of stock, OpenClaw is part of why. There’s a wave of people buying them specifically as always-on personal AI servers. The M4 model idles at about 10 watts. Less than a light bulb to keep an agent running around the clock.

If you’re using a Mac for OpenClaw you really want to dedicate that device to your setup so it’s not competing with you for resources. Once you have updates applied and a fresh install, open terminal and run this:

curl -fsSL https://openclaw.ai/install.sh | bash

This one command detects how you system is setup, installs Node if needed, installs OpenClaw, and walks you through the next steps. If you have a Mac ready to go, this is your fastest way to get started.

On a Virtual Private Server
Another popular option is to spin up an inexpensive virtual server (aka a VPS). Hetzner is a good option with their CX22 plan at $4/month. It provides 2 vCPU and 4GB RAM which is plenty for most basic OpenClaw setups. Here’s how set it up:

a. Create the server.
Go to hetzner.com, create an account, and create a new server. Pick Ubuntu 24.04, the CX22 plan, and select a data center near you. When asked about SSH keys, you can skip it and Hetzner will email you a root password — or if you want to set one up, GitHub’s SSH key guide is the clearest I’ve found. It’s worth doing once.

b. Connect to your new server using SSH.
Open terminal and type this:

ssh root@YOUR_SERVER_IP

c. Run the installer.
This single command sets up OpenClaw, creates a firewall, configures auto-start on boot, and implements Tailscale for secure remote access:

curl -fsSL https://raw.githubusercontent.com/openclaw/openclaw-ansible/main/install.sh | bash

You don’t need to know what any of that means. The installer walks you through everything, including how to add your LLM API key (Anthropic, Gemini or OpenAI). Once you finish, you’ll have a running OpenClaw gateway.

d. Verify.
After setup, you should check that everything is working as expected. You can do that with these commands:

openclaw doctor    # checks for config issues
openclaw status    # confirms the gateway is running

gateway: running means you’re done with this step and ready to go.

Step 2: Connect Telegram

Telegram is the fastest way to get your agent talking to you. It’s free, works everywhere, has no iMessage dependency, and is the default for a reason. Works on every platform, no phone number required.

2a. Create a bot

Open Telegram and search for @BotFather. Start a chat and run:

/newbot

Follow the prompts, pick a name and username for your bot. BotFather will give you a token that looks like:

7412938456:AAHdqTgsH8m9a3Xe5lk0mBHZJzwp1234xxcd

Copy that token. You’ll need it in the next step.

2b. Add the token to OpenClaw config

Open your OpenClaw config file (the onboarding wizard tells you where it lives, typically ~/.openclaw/config.json5) and add the following to the file. I’m going to ask you to edit a few text files. If you’re on a mac, you can open the files in TextEdit which is pre-installed. On a VPS you’ll need to use vim which is a bit trickier to learn and somewhat unintuitive. Google it and figure it out. I believe in you.

Anyway, once you figure out how to edit text files add this to the config.json5 file:

{
  channels: {
    telegram: {
      enabled: true,
      botToken: "YOUR_TOKEN_HERE",
      dmPolicy: "pairing",
    },
  },
}

Save the file, then restart the gateway:

openclaw gateway restart

2c. Pair your Telegram account

Open Telegram and send any message to your new bot. Then back in your terminal:

openclaw pairing list telegram

You’ll see a pairing code tied to your Telegram user ID. Approve it:

openclaw pairing approve telegram YOUR_CODE

Send another message to your bot. It should respond. That’s the connection live.

Step 3: Build your first proactive agent

The basic setup is ready. Now for the good part.

A cron job in OpenClaw is just a scheduled task with a natural language prompt. You describe what you want the agent to do, when to do it, and where to send the result. No code.

Here’s the exact setup for a monitoring agent, the same pattern I use for a dozen different tasks:

openclaw cron add \
  --name "My First Monitor" \
  --cron "0 */4 * * *" \
  --tz "America/Chicago" \
  --session isolated \
  --message "Check [URL or data source] for [specific condition]. If you find [thing worth reporting], send me a Telegram message at [your Telegram ID] with the details. If nothing has changed or nothing meets the condition, output QUIET and stop. Do not send any message."

The --cron "0 */4 * * *" runs it every 4 hours. Crontab.guru is a handy tool for building your own expressions. Some common ones:

• Every morning at 7am: "0 7 * * *"

• Every Monday at 8am: "0 8 * * 1"

• Every hour: "0 * * * *"

The most important part of the prompt: tell the agent to be quiet when nothing is worth reporting. output QUIET and stop prevents notification fatigue. You want to hear from it when something matters, not on a schedule.

For our reader’s solar use case, this might look like:

Check the ComEd hosting capacity map at [URL] for any new high-capacity zones 
that weren't in my last report. Cross-reference with parcel data for any lots 
over [n] acres within [m] miles of those zones that are listed for sale. 
If you find new matches, email me with a summary. If nothing new, output QUIET.

But you don’t need to any of that by hand! You can literally open Telegram and send your bot a message with the prompt above along with a description of when you want it run and how often. And your brand new agent will do it for you. This is the most important lesson here. Your agent knows how to set itself up and can do just about anything you want it to do from now on. This is why we’re careful about the machine we give it access to.

Step 4: The AgentMail upgrade

Telegram works great for personal alerts. But if you want your agent to send proper emails, or receive emails and act on them, you’re going to want to add AgentMail. I like this service because it’s built specifically for agents and how we use them securely. You could give your agent access to your personal email, but before you do that I recommend you really understand what you’re doing so you don’t bork your personal inbox like some people did.

Once you have it setup AgentMail gives your agent its own inbox. Next is setting up your system so your Agent knows about it’s new email account. You can install the skill by hand with this command:

clawdhub install agentmail

Then your agent can send from solar-scout@agentmail.to (or whatever you configure), format a proper email report, and drop it in your inbox every Monday morning. It can also receive emails. If you reply to a report, the agent processes your reply and takes action.

For our reader this is the missing piece. Instead of a Telegram ping, he’d wake up to a formatted email from his solar scout with a table of new sites, ranked by proximity to capacity, with links to the parcel listings.

What changes

The first few days you’ll keep checking if it worked. Opening the logs, running openclaw status, making sure the cron fired, etc. Then you forget it’s running.

A week later your phone might buzz. Your agent found something. Something you would have missed because you weren’t looking.

That’s the shift you need to get comfortable with. Your mindset shifts from “What can AI do for me?” to “How do I make my AI work for me when I’m not working?” This is the answer. The initial setup takes 30 minutes. The payoff is an AI that has a job, shows up for it, and only bothers you when it has something worth saying.

Before long you’ll be hooked. You’ll discover you can simply ask your agent to do all sorts of things for you proactively and it’ll set itself up to make it happen. You’ll browse clawhub.ai for skills and eventually write your own skills. You’ll read about how people are doing amazing things with their setups and replicate some of their efforts. It gets fun fast.

On Sunday, my agent texted me at 7am about sold out Harry Potter tickets for a trip we’re taking to London. Stella was keeping an eye out for cancellations, so I could pounce if any came available. We didn’t get the tickets because someone else was faster. That’s a different problem. But I’m hopeful I’ll have another chance.

What would you have your agent do for you?

Builder’s note: The OpenClaw docs are at docs.openclaw.ai and the community is active on Discord if you get stuck. If you end up building something like solar scout, I want to hear about it.

Trond Wuellner builds things at Google Labs and writes about what he learns. If this was useful, forward it to someone who’d appreciate it.

Subscribe now

Smart people. All of them.

Every single one has played with ChatGPT. Most use it regularly. And when I described what I’ve been building: AI assistants that manage my family’s calendar, generate math worksheets for my kids, call restaurants on my behalf, monitor my house. They all had the same look. Not skepticism. More of an eagerness to dive in to figure out where these tools might help them at home and at work.

“Okay but how do you actually start?”

That’s what this is. The answer to that question. Not theory or hype. A step-by-step guide for someone who understands their business, isn’t afraid of technology, and is ready to go from curious to building.

This is my favorite part of this moment we’re living in. You don’t need to be a developer. You need to be willing to try and eager to learn. This guide is for you.

The Decision

Before you install anything, understand what you’re actually signing up for.

Building with AI coding tools is not like using ChatGPT. You’re not typing questions and copying answers. You’re setting up an ongoing collaboration with a system that writes code, runs commands on your computer, reads your files, searches the web, and iterates based on your feedback.

And the output isn’t always software in the traditional sense. It might be an automated workflow, a document processor, a report generator, or a tool that monitors your inbox and surfaces what matters. If it involves information and repetition, these tools can help.

It’s closer to hiring a junior developer than using a search engine. A very fast, very capable junior developer who has read most of the internet but has no context about your specific situation.

Your job is to provide that context. To ask good questions. To review what it builds and tell it what to change. You don’t need to understand every line of code it writes. You need to understand what you want, and be able to recognize when you’re getting it.

That’s it. That’s the skill.

The tools do the rest.

A Word on Cost

Before you install anything: check what you already have.

A lot of people are already paying for access to these tools without realizing it.

• Google Workspace Business Standard and above includes Gemini built in: summarization in Gmail, Docs, Sheets, and more. Check with your IT admin or look in your Google account under the Gemini icon.

• ChatGPT Plus ($20/month) includes Codex. If you’re already subscribed, you’re in.

• Microsoft 365 offers Copilot as a paid add-on, but if your company has rolled it out, you may already have access to some of these capabilities.

If you have any of these, start with what you have before buying anything new.

If you need to pay, subscriptions are almost always the better choice. The $20/month plans (Claude Pro, ChatGPT Plus, Google AI Pro) give you predictable costs, generous limits, and the most capable models. For most people, a subscription goes further than pay-per-use at the same price.

The higher tiers, Claude Max ($100-200/month) and ChatGPT Pro ($200/month), are worth it once you’re using these tools daily and want priority access, maximum limits, and the fastest responses. Think of it like a SaaS tool: pay for the plan that matches your usage.

The pay-per-use API is powerful but unpredictable. Claude Code can run $20-50 in a single day on a large project without you noticing. It makes sense eventually, especially for automated workflows that run without you. It’s not where you want to start.

The progression that makes sense:

1. Check if you already have access (Workspace, ChatGPT Plus, Microsoft 365)

2. Start with a free tier (Gemini CLI, Antigravity) to get comfortable

3. Upgrade to a $20/month subscription when you’re ready to go deeper

4. Move to higher tiers ($100-200/month) if you’re using it every day

5. Add pay-per-use API access only when you’re building automated workflows that run on their own

Pick Your Tool

There are essentially three tools worth using right now. They all do roughly the same thing: AI-assisted coding and automation. Different providers, each with different strengths.

All have desktop apps to get you building without touching a terminal. Claude’s desktop app includes a Code tab (Claude Code with a visual interface) and Cowork, an autonomous mode where Claude acts on files in a folder you authorize. Codex has a standalone app. Google’s Antigravity IDE is free and helpful for getting started quickly.

Pick the tool that matches what you already have access to. You can always change later. We’re just finding the right path to get you started today.

My simple point of view:

Start with what you already pay for. ChatGPT Plus gives you Codex. Google Workspace gives many people Gemini Pro. Check what you already have before you buy anything.

If you’re starting from zero: Antigravity (Google’s IDE) is free, polished, and requires no terminal. Gemini is especially good if your project involves reading lots of documents, emails, or spreadsheets; which is a bonus.

Codex is the natural choice if you already use ChatGPT Plus. You’re already paying for it. Just start there.

I generally start most projects with Antigravity, but I also love what Claude can do even though it requires a subscription. Their desktop app is quite impressive. The Code tab handles most coding tasks beautifully, and Cowork (included from Pro and above) takes it to another level. It reasons more carefully than the others, handles nuance well, and is often the most honest about what it doesn’t know. Worth it once you’re willing to add another subscription to your monthly nut.

You don’t have to pick just one. You’ll learn what you prefer as you start building and can easily change later.

Install It: Step by Step on Mac

Option A: Gemini (Google)

The easy way: Antigravity

Google released an agent-first IDE called Antigravity in late 2025. It’s free and runs Gemini models under the hood. The standout feature: a “Manager View” that orchestrates multiple AI agents in parallel across your editor, terminal, and browser. Think Mission Control for coding tasks. Download it like any Mac app, sign in with your Google account, and you’re building in minutes.

The power-user way: CLI

The Gemini CLI unlocks its biggest advantage: the ability to read enormous amounts of text at once. Useful for projects involving lots of documents, emails, or large codebases.

brew install node
npm install -g @google/gemini-cli
gemini

On first run it opens a browser window for Google sign-in. That’s the whole setup. For the most powerful model:

gemini -m gemini-pro-latest

Option B: Claude (Anthropic)

The easy way: desktop app

Download the Claude desktop app at claude.com/download. Free to download, works on Mac and Windows. Sign in with your Anthropic account and you get two modes worth knowing about:

• Code tab: a graphical interface for Claude Code. You get visual diff review, live previews of what you’re building, and the ability to run multiple tasks in parallel, all without touching the terminal. This is Claude Code with a UI on top.

• Cowork: a research preview (Pro and above) where Claude autonomously acts on files in a folder you authorize. Think of it as giving Claude a desk of its own: it can organize documents, build spreadsheets from receipts, draft reports. You review and confirm significant changes.

For most people starting out, the Code tab is the right place to begin.

The power-user way: CLI

The terminal version is one line to install:

curl -fsSL https://claude.ai/install.sh | bash

After it finishes, just run:

claude

Sign in with your Anthropic account. You’ll need a Pro or Max subscription (starting at $20/month), or an API key from console.anthropic.com to pay by usage instead. The CLI and the desktop app share the same config, including your CLAUDE.md files, so you can move between them freely.

Option C: Codex (OpenAI)

The easy way: Codex app

OpenAI has a standalone Codex app at openai.com/codex. Purpose-built for coding tasks. If you already pay for ChatGPT Plus ($20/month), you have access.

The power-user way: CLI

brew install node
npm install -g @openai/codex
export OPENAI_API_KEY="your-key-here"
codex

Get your API key at platform.openai.com → API Keys → Create Key.

Your First Session: What You’ll Actually See

Here’s what you’ll actually see when you open each tool for the first time.

If you’re using Antigravity (Google)

When Antigravity opens you’ll see two views in the sidebar: Editor and Manager.

Start in Editor. It looks like a code editor: a big panel on the left for files, a writing area in the center, and a chat panel on the right. That chat panel is where you talk to the AI. You don’t need to touch the file panel or the code. Just focus on the chat.

To open your project: File → Open Folder → navigate to your project folder (Desktop or Documents is fine) and select it. Your files appear in the left panel. Now when you talk to the AI, it can see and edit those files directly.

Type your first message in the chat panel on the right. The AI responds, makes changes to your files, and shows you what it did. In a moment, we’ll show you what to type in that chat box!

If you’re using the Claude desktop app

When it opens you’ll see a clean interface with a chat window in the center. At the top you’ll see tabs: Chat, Code, and (on Pro and above) Cowork.

Click Code. You’ll see a chat interface similar to the regular Claude chat, but with file access. Click the folder icon or go to File → Open Project to point it at your project folder. The AI can now read and edit files in that folder.

Type your message in the chat box at the bottom. The AI responds in the main window, shows you code changes with visual diffs (highlighted additions and deletions), and asks for confirmation before editing files.

If you’re using Claude Code in the terminal

After running claude, you’ll see a prompt that looks like this:

Claude Code (claude-opus-4-5)
Type a message or /help for commands
>

Navigate to your project folder first:

cd ~/Desktop/my-project
claude

Full Claude Code docs at docs.anthropic.com/claude-code.

Now type your message at the > prompt. The AI responds inline, shows you the files it’s changing, and asks your permission before modifying anything.

If you’re using the Codex desktop app

When Codex opens you’ll see a Projects panel on the left and a main area on the right. Projects are how Codex organizes your work: each project points to a folder on your computer (or a GitHub repo, if you have one).

To get started: click New Project, give it a name, and point it at your project folder. Codex loads the files and you’re ready.

The main area shows Threads: each thread is a task you’ve given to an AI agent. Think of threads like chat conversations, except instead of just answering, the AI is actively editing files and running code in the background. You can have multiple threads running at once, each working on a different thing.

Type your first message in the input box at the bottom of the main panel. The AI picks it up as a new thread, starts working, and shows you what it’s doing in real time: which files it’s reading, what changes it’s making, whether the code ran successfully. When it’s done, you’ll see a diff — a summary of what changed — and you can approve it, ask for changes, or open the result in your own editor.

One thing worth knowing: Codex is built for parallel work. You can kick off one task, then start another while the first is still running. Each stays in its own thread. Genuinely useful once you have more than one thing to build at once. For your first session, just start one thread and get a feel for the loop before you try running multiple.

The big thing most people don’t realize

You don’t need to know how to run code. In all three tools, the AI runs the code itself. You ask it to build something, it builds it, runs it to check it works, shows you the result, and tells you if anything went wrong. Your job is to tell it what you want and give feedback on what you see.

If the AI produces a script you need to run yourself, it will tell you exactly what to type. Copy it. Paste it. Run it. That’s the full extent of what you need to know.

On permissions and trust

One thing that catches people off guard: these tools ask for your approval constantly. Before editing a file, before running a command, before accessing the internet: you’ll see a prompt asking if it’s okay to proceed.

Often you won’t know exactly what it’s asking. The command it wants to run might look like gibberish. You’re faced with a choice: trust the system and say yes, or stop and ask what it means before proceeding.

Here’s a rough guide to what’s routine and what deserves a pause:

Generally fine to approve:

• Reading or editing files inside your project folder

• Running the code it just wrote

• Installing packages or dependencies (it needs these to build things)

• Creating new files in your project

Worth a moment’s thought:

• Making network requests or calling external APIs: ask what it’s connecting to and why

• Deleting files: make sure you know what’s being removed

• Commands that reference folders outside your project: why does it need to go there?

• Anything involving credentials, API keys, or passwords: confirm it’s not storing these somewhere unexpected

Decline and ask questions:

• Anything with sudo (administrator access) that the task clearly doesn’t require

• Commands touching sensitive system locations you didn’t direct it toward

• Sending data to a service you didn’t set up or recognize

When in doubt: if a command looks completely alien, type “explain exactly what this does in plain English before running it.” The AI will walk you through it. You’ll either feel confident to proceed, or you’ll catch something that shouldn’t happen.

Once you trust the project and want to stop approving every small action, Claude Code has an answer: yolo mode. Type /yolo in the chat. It skips permission prompts for the session. Your files are still there, changes are still visible, and you can undo. It just means you’ve decided to let the AI work without interruption. Most people switch to it once a project is underway and they understand what the AI is doing.

Setting Up Your First Project

Every project starts the same way: a folder and a briefing file.

If you’re using a desktop app (Antigravity, Codex or Claude): Create a folder anywhere. Desktop or Documents is fine. Name it something descriptive. Then open it in the app. That’s your project.

If you’re using the terminal: Create the folder and navigate into it:

mkdir ~/Desktop/my-project
cd ~/Desktop/my-project

The briefing file is called AGENTS.md. Think of it as permanent memory for your project: context the AI reads at the start of every session so you never have to re-explain from scratch. Claude Code reads CLAUDE.md natively, and most other AI coding tools are converging on AGENTS.md as the shared standard. Using AGENTS.md means every tool picks it up automatically.

A quick note on the .md file format. These files are written in a format called Markdown: a simple way of formatting plain text using symbols like # for headings and - for bullet points. You’ve probably seen it without knowing it. The AI reads it perfectly, and it’s easy to write once you’ve seen an example.

Macs don’t come with a great Markdown editor by default. A few good options:

• Bear: Mac-native, beautiful, free to start. The one I’d start with.

• Obsidian: free, powerful, worth it if you want your notes and projects in one place.

• VS Code: free, already installed if you’re using Antigravity (it’s built on the same base), and has Markdown preview built in.

Again, remember that you can always just ask the AI to create the file for you. After the interview, say “Based on what we just discussed, create an AGENTS.md file for this project.” It will write the whole thing correctly. If you want, you can open it in Bear or Obsidian to review and tweak.

Seriously, don’t start from scratch. The community has built templates you can copy. Just use those as a starting point, they’re going to be better than whatever you’ll do at first:

• agentsmd/agents.md: The simplest, most widely-adopted template.

• davila7/claude-code-templates: Ready-to-use configurations with working examples.

Here’s a starting point if you absolutely want to roll your own:

# AGENTS.md

## What This Project Is
[One paragraph: what it does and why it exists]

## Who Uses It
[Describe the person this is built for]

## Current Status
[What's done, what's in progress, what's next]

## Important Constraints
[Things the AI must never do, hard requirements, known limitations]

## Style and Standards
[How you want things done: tone, format, patterns to follow consistently]

You don’t need to fill in every section before you start. The first two are enough to begin. The AI will help you figure out the rest as you go.

The Style and Standards section is the most underrated part. Tell the AI once how you want things done, and it applies that consistently every session. You stop correcting the same things over and over.

One note on naming: AGENTS.md works across most tools. If you’re using Claude, it reads a file called CLAUDE.md instead. That file has the same purpose and works the same way but they don’t use the common naming scheme as everyone else. Sigh.

Starting every session, your opening message:

Please read AGENTS.md to get context on this project, then tell me what you understand and what questions you have before we continue.

It’s a good idea to say this at the start of every new chat. It takes five seconds and prevents you from re-explaining your project from scratch. If you’re in the terminal, navigate to your project folder first (cd ~/Desktop/my-project), then start the tool.

Your First Project: The Interview Method

Here’s one mistake people make when they get started: they open the tool and immediately say “build me X.”

The model has no idea who you are, what your business does, what constraints you’re working under, or what “good” looks like for you. So it builds something generic that doesn’t fit, you get frustrated, and you assume AI coding tools don’t work.

They do work, but you need to teach them what you want first.

I use what I call interview mode. Open your tool of choice, make sure it’s pointing at your project folder, and type this into the chat window:

Before we start building anything, I want to make sure you deeply understand what I need. Please ask me up to 10 clarifying questions about this project — one at a time, waiting for my answer before asking the next. Focus on: who the users are, what problem we're solving, what success looks like, what constraints I'm working under (time, budget, technical), and what I definitely don't want. When you feel you have enough, summarize what you've heard and ask if it's right before we proceed.

The model will ask you things you hadn’t thought to specify. Answer honestly in the chat, one question at a time. When it summarizes back to you, correct anything that’s off. This usually takes 10-15 minutes and saves hours of misdirected work.

After the interview, you’re going to want to capture the key insights in a structured way using 2 docs. The AI will write them and save them directly to your project folder:

The requirements prompt:

Based on our conversation, write a short Product Requirements Document and save it as REQUIREMENTS.md in this project folder. Include: the problem we're solving, who it's for, the core features in priority order, what we're explicitly not building, and what "done" looks like for the first version.

The plan prompt:

Now write a step-by-step build plan and save it as PLAN.md. What are the major pieces? What order would you tackle them in? Flag anything where you see a real tradeoff I should weigh in on.

Once the AI saves these files, they’ll appear in your project folder. Open them in Bear or VS Code to read through them. Correct anything that’s off. These become the ground truth that keeps you and the AI aligned throughout the build, and the foundation of your AGENTS.md.

The Build Loop

You’ve briefed the AI, you have a plan, you’ve started a session. Now what?

Building with AI tools is a loop:

1. Ask for something specific. Not “build my app” but “build the function that reads incoming emails and pulls out the dates.”

2. Review what it builds (you don’t need to understand every line, but read the explanation it gives you)

3. Test it. Say “run this and show me the output.” The AI executes the code itself and reports back what happened. You don’t need to know how to run code.

4. Tell it what’s wrong. Be specific: “this crashes when the email has no subject line” or “the output is missing the date field.”

5. Repeat

Reading responses: The AI will often explain what it did and why. Read this. It’s showing its reasoning. If the reasoning sounds off, say so before it goes further. Correcting direction early is much easier than unwinding bad work later.

When it gets stuck: All three tools will sometimes hit a wall and go in circles. Signs: it rewrites the same thing repeatedly, asks you the same question again, or apologizes for the confusion more than twice. When this happens, stop. Start a fresh conversation. In the app, click “New Chat” or “New Session”. In the terminal, type exit and run claude again. Then paste the relevant context: “Read AGENTS.md and DECISIONS.md, then let’s approach [the specific problem] differently.” Fresh context, fresh angle.

When to push vs ask: If you don’t understand a choice the AI made, ask. “Why did you use X here instead of Y?” It will explain. Either there’s a good reason you hadn’t considered, or it made a mistake. Either way, you learn something. Never accept work you don’t understand at all. You’ll have to deal with it later.

The Learning Loop: How the System Gets Smarter Over Time

Here’s something most people miss. It’s probably the biggest multiplier once you get past your first project.

Every AI coding tool starts with zero knowledge about you. Your preferences. Your standards. The decisions you’ve already made. What you tried last week and why it didn’t work. Each new session, you’re starting from scratch unless you build a system to prevent that.

The fix is simple: document decisions as you go, in a place the AI can read them.

Create a file called DECISIONS.md in your project folder. After each meaningful session, end with this prompt:

Summarize what we built today, what decisions we made, and why. Format it as a short list I can append to DECISIONS.md.

Paste the output into the file. Next session, start with:

Before we continue, read AGENTS.md and DECISIONS.md to get full context on this project and the decisions we've already made.

Over time, this file becomes institutional memory. The AI stops suggesting things that contradict choices you’ve already made. It stops proposing approaches you already tried and abandoned. It builds on what exists.

What to capture:

• Choices made and why (”we’re using X instead of Y because...”)

• Things that didn’t work (”tried Z, caused problems because...”)

• Standards you want applied consistently (”always do it this way”)

• Open questions you haven’t resolved yet

You don’t need to be exhaustive. Three or four bullet points per session compounds fast over weeks.

The projects where this discipline holds get easier over time. The ones where it breaks down get harder. The AI gets genuinely more useful the more context it has. DECISIONS.md is the simplest way to build that context.

API Keys Done Right

Most people don’t need this section yet. If you’re using a subscription plan (Claude Pro, ChatGPT Plus, Google One), you authenticate with your account and you’re done. API keys are for automated workflows that run without you: scripts, scheduled jobs, integrations that fire in the background. API keys are also necessary if you want to connect to a service programmatically.

For example, I built a system that used a USDA dataset to calculate the nutritional value of various recipes and to set that up, I needed to create an API key with the service and save it securely. Come back here when you’re building something like that.

In short, API keys are like passwords. If someone gets yours, they can use your account and you pay the bill. Here’s how to handle them safely.

Create a secrets folder that won’t accidentally get shared:

mkdir -p ~/.secrets
chmod 700 ~/.secrets

Store each key in its own file:

echo 'ANTHROPIC_API_KEY=your-key-here' > ~/.secrets/anthropic.env
echo 'OPENAI_API_KEY=your-key-here' > ~/.secrets/openai.env
echo 'GOOGLE_API_KEY=your-key-here' > ~/.secrets/google.env  # Get key at aistudio.google.com

Load them automatically when you open Terminal:

echo 'source ~/.secrets/anthropic.env' >> ~/.zshrc
source ~/.zshrc

Do this for each key. Now, your keys load automatically every time you open Terminal and they’re in a folder with restricted permissions where they can’t accidentally get shared.

Three rules:

1. Never paste your API key directly into code

2. Never share a file that contains your API key

3. Set spending limits in each provider’s dashboard before heavy use (Anthropic limits, OpenAI limits)

Skills: Going Further

Once you’ve shipped something, you’ll want the AI to do more than write code.

Skills are reusable packages of instructions that teach your AI how to handle a specific recurring task. A skill for reading and summarizing emails. A skill for generating weekly reports in a consistent format. A skill for reviewing what you built and flagging problems. You install it once, and the AI knows how to do that thing every session.

Where to find them:

• skills.sh: The largest open directory, with contributions from Anthropic, Google, Microsoft, and the community. Browse by category.

• playbooks.com: Curated skills and bundles. Good for finding a set that works together.

Prompt Resources Worth Bookmarking

• Gemini API Cookbook: Google’s official Gemini prompt collection.

• Anthropic Prompt Library: Claude’s official collection of prompts for common tasks.

• OpenAI Cookbook: Practical examples for Codex-based workflows.

• PromptingGuide.ai: Deep reference with research-backed techniques.

Three techniques worth learning first:

Think step by step. Add “Think through this step by step before answering” to any complex request. The AI slows down, shows its reasoning, and catches mistakes it would otherwise skip. It sounds almost too simple. It works.

Label the parts of your prompt. When your request has multiple parts, label each one clearly. Like this:

Context: I'm building a tool that summarizes supplier emails for a food distribution business.

Task: Write a function that reads an email and pulls out any price changes mentioned.

Constraints: Only flag changes above 5%. Ignore shipping cost changes.

Format: Return a simple list with supplier name, product, and new price.

Claude in particular responds much better to a structured prompt than a wall of text. It knows exactly what’s background and what’s the actual ask.

Plan before you build. Before asking the AI to do anything, ask it to plan first. “Outline the steps you’d take to do X, without doing anything yet.” Review the plan. Fix anything that looks wrong. Then say go. You catch bad assumptions before they turn into bad work.

The Real Bar

Here’s the thing I kept saying in Whistler that I want to say again here.

The bar for “building something” is not as high as you think.

A solar entrepreneur who automates the extraction of permit requirements from municipal PDFs into a structured spreadsheet has built something. A wild game distributor who sets up a system to summarize supplier emails and flag price changes has built something. A VC partner who creates a tool that generates first-pass memos from founder decks has built something.

None of those require a developer. They require someone who understands the problem, can explain it clearly, and is willing to iterate.

The AI does the technical part. You do the part that matters: knowing what’s worth building.

Start with the smallest possible version of the thing you actually need. Get it working. Then make it better. Repeat until you’ve built something rad.

That’s it.

If you build something using this guide, I want to hear about it. Reply to this email or find me on Twitter/X. I read everything.

And if you’re not sure what to build first, that’s okay too. Start the interview with: “I’m not sure what to build. Here’s my business and here’s where I spend the most time on things that feel repetitive.” Let the AI help you figure out what’s worth automating. That conversation alone is usually worth the setup.

For eight years, when my kids needed extra practice before a math test, I sat down and handwrote practice problems myself. Sounds wholesome. It was mostly tedious.

Something you might not know about handcrafting math problems: you have to solve them (in your head) as you write them. Because if you don’t, you end up with division that produces some wretched 11-decimal answer, or a quadratic that has no real solutions, and your kid is sitting there staring at it while you quietly panic and say “let me check that one.” Teachers really aren’t paid enough.

The process I used was: come up with a good problem, pre-solve it, write the answer somewhere, repeat. Then when they come back with their work, solve it again alongside them to help them check. For basic arithmetic, totally manageable. For my freshman in Algebra 2 or my seventh grader in Pre-Algebra? That 10-minute process quickly grew to 45-minutes or more. My daughter once solved problems faster than I could write them. That was a humbling evening.

I started trying to use AI for this maybe three years ago. The results were... fine. Inconsistent. The formatting was always a mess and I’d have to clean it up before printing. And the problems weren’t always right. Or leveled correctly. Or just plain repetitive.

Over the last year it got noticeably better. I started testing models by asking them to generate LaTeX directly. Real formatted worksheets became easily printable PDFs. Over time, its gotten more reliable. But I was still doing a lot of shepherding: prompt, review, fix the formatting, regenerate problems that didn’t make sense, spot-check the answers. Better, but still bad.

Something has shifted in the last few weeks. Gemini 3.1 Pro and Opus 4.6 are incredible for this use case. The results have gotten... good?

A while ago, I realized I could take a photo of my kid’s existing problems (problems their teacher assigned, or problems they’d already worked through) and say: make me more like these. And a thinking model would just run off and make good practice problems. Not just structurally similar, but calibrated to exactly the right difficulty level, because it had actual examples to learn from.

I think this might actually work now! Thank you thinking models!

Read more

If you’ve been following the trond.ai Build to Learn series, you’ll have heard of Stella. Stella is an OpenClaw setup running on a Mac Mini in my office. She (yes, we anthropomorphize her) manages our family’s: calendar, smart home, kids’ schedules, recipes, travel planning, reminders, car service, appliance maintenance, and so much more. She’s genuinely useful and has quickly become a trusted partner in our home.

But there are problems with Stella that need fundamental improvements in my OpenClaw setup to address. Let’s talk about the problems and the solution I’ve been working on building this week.

Assistant Amnesia

Stella is frustratingly forgetful. I’ve had to remind her about projects we were working on together, and sometimes she’d lose the train of thought mid-conversation. Especially if it spanned a 30 minute boundary when her memory would get collected and updated. New session, clean slate. I’d re-explain context, she’d reload what we talked about, we’d rebuild momentum, and then the conversation would hit the context limit and compact. Details gone. Thread lost. Start over. GAH!

I thought AI hallucinations were annoying. This is 1000x worse!

There are countless people working on solutions to better agent memory across the OpenClaw community. The stock solution uses a MEMORY.md file that’s updated periodically based on conversational context within each heartbeat process. This context is passed into the agent’s memory each time a session is initiated, which is supposed to preserve details and progress in a seamless way. But in reality, the compaction process loses too much detail and biases too much on recency.

I poked around with a lot of the solutions people were building to try to address this problem. Some are very well designed, like supermemory.ai, featuring RAG systems with complex vector databases and advanced workflows to integrate into your agent. But they didn’t really solve my problem. I wanted something that kept data in a format I could easily read and modify and something that worked better with my collaboration model.

Collaboration Nightmare

The forgetting is one thing. But when you’re trying to collaborate across devices, sessions and people it all gets just so much worse. I use Obsidian as a thinking and organization tool. Stella runs on a separate machine without access to the local vault. Those two worlds don’t talk to each other. If she learns something from a conversation, my notes don’t know about it. If I edit a note about a project, she doesn’t see the change until I paste it into a chat. Working together on things was like playing telephone with project context.

I see this as a fundamental flaw with chat-based AI: everything is assumed to be serial. One conversation, one thread, linear. That works fine for “what’s the weather.” It breaks immediately when you want to do real knowledge work, picking up a thread from a different device, having your AI and your notes reflect the same reality, working on something that evolves over days or weeks. Never mind projects that work across agents and people.

We kept bumping into the same wall: the AI lives in the chat. Everything important lives somewhere else. They never quite sync up. I needed a better approach. I really wanted something that would keep Stella’s memory system and my Obsidian vault in perfect sync.

Text Is a Terrible Data Model

We all know LLMs work best with text, and OpenClaw is no different. So much so, that it reminded me of the good old days of computer science when our computers were basically command line interfaces passing text between processes using unix primitives. Turns out Anthropic agrees — they’ve published research on Contextual Retrieval showing this technique outperforms RAG and vector lookup for agent memory.

This means that when Stella needs to know about my mom, she can cat a markdown file and read it like a document.That’s how most AI memory systems work. Store everything as text, search through text, return text.

But my mom isn’t a document. She’s a collection of facts, some current, some outdated, some time-bound, with relationships to other facts, timestamps, and different levels of reliability. “She’s visiting in March” is a different kind of fact than “She was born in 1951.” One expires. The other doesn’t.

Flat text files don’t know any of this. They mix it all together. An AI reading a flat file has to parse, infer, and guess what’s still true. It’s actually pretty dumb.

This is actually a solved problem in software. In 2002, Jeffrey Snover wrote the Monad Manifesto about this exact problem. He helped Microsoft design PowerShell in 2006 to implement a better approach. Unix pipes pass text between commands, so every tool has to parse the output of the previous one. PowerShell passes objects instead. Typed, structured, queryable. That’s because text is the right interface for humans but objects are a better interface for automation.

What if we applied this same approach to AI memory? Could we use the benefits of LLMs and the structure of data objects to build something even better?

Building to Learn - AgentSync

This week, I set out to learn about these three problems:

1. Agentic Amnesia

2. Borked Collaboration

3. Broken Object Models

Which is why I (with a lot of help from Stella) built AgentSync. AgentSync is fundamentally a knowledge graph system built on a foundation of typed objects that are automatically collected, scoped, updated and shared in real-time across my devices. Facts aren’t stored as prose. They’re stored as typed objects: what the fact is, when it was learned, where it came from, whether it’s still active, when it expires. And everything is sync’d in realtime via human readable and modifiable markdown.

When Stella needs to know details about my mom, she runs:

agentsync query people --name guri

She gets back structured data. Not a text file to interpret. A typed object to act on. Invalid states are impossible because validation happens at the write boundary. A fact that doesn’t match the schema can’t reach the knowledge graph.

The filesystem is the schema. Create a directory for a new domain and it’s automatically discovered as an entity type. We have 217 entities across 11 domains right now: people, recipes, projects, properties, conversations, subscriptions. No configuration. We just started using it and it grew. That’s its magic. When Stella extracts a fact from a conversation, it gets validated and filed automatically.

Session starts cost (virtually) zero tokens. There’s a lightweight index that shows what exists without loading anything. Stella knows she has 127 recipes without reading a single one. She loads context on demand, for whatever domain she actually needs. Sessions that used to burn 5-10K tokens on context reload now start at zero.

It’s much more efficient, accurate and flexible. And since it’s self-learning and self-healing, the system can evolve as my knowledge base expands.

Real-Time Sync With Obsidian

Structured objects fix the data model. The collaboration problem needed something else.

To take this on, I built a WebSocket relay using Cloudflare Workers. File changes on any configured device broadcast through the relay. Each component in the system subscribes to the relay. On Stella’s Mac Mini, we run a real-time file watcher service. On my other devices, I run a custom Obsidian plugin that keeps our shared vaults in sync. Sub-second latency, content-addressed deltas, no merge conflicts. It’s really pretty magical.

Edit a note in Obsidian. Stella sees it instantly. She learns something new in a conversation. It shows up in my vault before I open it. The files we interact with are human readable markdown. The insights they contain are instantly converted to typed objects saved in companion files built for computer readability.

Now, I can easily see anything in Stella’s vault. Update TODO files directly and implement multi-agent or multi-person collaboration workflows with complete confidence our shared memory preserves context.

The Self-Documenting Test

One cool thing about this project is that we used the system while building the system.

Every architecture decision got logged as a fact. Every deployment got tracked with a timestamp and commit hash. Every time we changed direction, we logged why. When I sat down to write this article, I ran:

agentsync query projects --search "agentsync"

Complete build history. Every decision, every deployment, every pivot. The system documented its own creation in real-time, so when we needed to write about it, we didn’t have to remember anything. We just queried.

That’s not a demo. That’s the whole point.

Some numbers from five days of real use:

• 217 entities, zero invalid data on disk

• Sessions: from 5-10K tokens to zero on startup

• Sub-second sync between Mac Mini and MacBook

• 127 recipes, including one extracted from an Instagram reel at 11pm

• Full conversation archives going back 48+ hours, searchable and compressed

• Typed objects for every entity type, automatically organized and searchable

Why This Matters Now

AI assistants are going to get much better at the hard things: reasoning, planning, synthesis. The memory problem is solvable right now, with a filesystem and 1,400 lines of TypeScript.

The real unlock isn’t any single feature. It’s the combination: typed knowledge that can’t corrupt itself, zero-token session starts, real-time sync that keeps humans and AI working from the same source of truth, and conversation archives that survive compaction.

Your AI assistant shouldn’t have to re-learn who your mom is every morning. That’s a solvable problem. We just needed to actually build the solution.

AgentSync is open source on GitHub, MIT licensed, self-hostable, works with Obsidian or any file-based workflow.

If your AI is still waking up with amnesia, it doesn’t have to. We’ve learned by building AgentSync there’s a better way. Next up: project management!

GitHub: stellawuellner/agentsync

What does your AI memory setup look like? I’m curious whether others have hit the same wall or found different ways around it.

Yes, Another OpenClaw Article. But…

I know. You’ve seen the tweets. You’ve heard tech nerds raving. For the last few weeks, half of Silicon Valley has been talking about ClawdBot (now called OpenClaw). The open-source AI agent framework that lets you give Claude or Gemini access to your email, calendar, smart home, and basically your entire digital life. If you’re tired of hearing about it, you’re probably right.

But I’m not writing this to hype an AI tool. I’m writing this because building my own turned out to be one of the best examples of something I keep coming back to: building to learn. The best way to deeply understand something isn’t to read about it or take a course. It’s to build something real with it. Not a toy. Not a tutorial. Something that solves a problem you actually have.

Two weeks ago, I decided to build an AI assistant for my family. Not a chatbot or Alexa skill, but a full-blown digital crew member. More JARVIS, less Siri. I wanted it to manage our calendar, monitor our email, control our smart home, run our kitchen display, and talk to us. The dream of my Google Home but more useful?

I named her Stella, after the ship computer from Miles from Tomorrowland, a show my kids watched when they were little. The name felt right. Warm, capable, part of the family. That’s the target I was shooting for; friendly and helpful.

How Stella Got Her Groove

Stella runs on OpenClaw an open-source framework for building persistent AI agents. Think of it as the operating system for a personal AI. It handles memory, scheduling, tool use, and multi-channel messaging. Under the hood, she’s powered by an LLM, and in our case runs on a dedicated Mac Mini in my garage. The Mac Mini is overkill for sure, but it’s an older model that I wasn’t using for anything else, so ... shrug.

After two weeks, here’s what she can do:

• She manages our family calendar. She reads and writes to our shared Google Calendar, knows our naming conventions (”L&T” means Lauren and me, “PYT” means Peninsula Youth Theatre), and adds full street addresses so events are navigable. She cleaned up months of messy calendar entries on day one. Actually closer to hour 1 to be honest.

• She reads email. She monitors Gmail for anything important and can help summarize promotions, newsletters, tasks that need to be done, and flags things that need attention. She knows not to bother me at 2am about a Lyft promo and can have a summary of the most interesting articles to read from the dozens of Substacks I subscribe to.

• She runs our kitchen display. We have a custom family dashboard in our kitchen powered by a Raspberry Pi connected to a monitor I framed to look nice. It shows family photos, a 4-day calendar, weather, air quality, stock prices, sleep scores, nearby wildfires (we’re in California), and as of this week, a live Winter Olympics medal count. At night, sleep data. In the morning, calendar and weather. When I’m traveling, a flight tracker appears automatically. I originally built it using DAKboard but now Stella runs the whole thing. We call it Stellascreen. I have so many more ideas for how we can make this awesome.

• She controls our home. Through Home Assistant, she manages 20 Lutron Caseta lights, a Nest thermostat, Sonos speakers, and Chromecast devices. I’ve barely scratched the surface on what we can do here, but if I want to add something all I do is ask Stella and in minutes it’s setup and working.

• She talks. We built a local text-to-speech system that runs entirely on the Mac Mini’s Apple Neural Engine with sub-second voice synthesis. You can try it yourself by installing the skill. We setup a wakeword on the Stellascreen so we can talk to her anytime we want. As a placeholder, we’re using “Hey Jarvis” for now. It’s one of the built-in options in OpenWakeWord, and we haven’t gotten around to training a custom one yet.

• She makes phone calls. Using Bland AI, she can call businesses on my behalf. Her first call was to our Volvo dealer to ask about service hours. They’re closed on weekends, she reported back. Why are businesses that ostensibly want customers who have jobs not open on the weekends? I’ll never know.

• She has memory. This is what makes her feel different from a chatbot. She maintains structured files about our family: birthdays, preferences, school schedules, ongoing projects. She writes daily notes about what happened and periodically distills them into long-term memory. When she wakes up each session, she reads her own notes to remember who she is and what’s been going on.

I feel like I’ve barely scratched the surface of what we can do with Stella and it’s only been a few weeks of toying around with her. Silicon Valley is excited bout OpenClaw because we’re seeing something that works in ways that we always wanted but never could quite achieve without monumental effort. OpenClaw is almost certainly not the final form this will take. But it is something interesting.

The Gmail Incident

To do many of the things we want her to do, Stella needed a Google account. So I created one for her. A regular consumer Gmail. I set up OAuth, shared our family calendar, and pointed her email monitoring at that inbox. Then I added a bunch of guardrails that I hope work. It all worked beautifully for about ten days.

Then Google suspended the account.

I only found out when the Stellascreen didn’t have any information on it. I tried to log in and hit a wall. Google had flagged the account for “unusual activity” and locked me out of it. Calendar access disappeared. Email monitoring stopped. Every integration that depended on Google OAuth just broke entirely.

Stella wasn’t doing anything malicious. She was checking email a few times per hour and reading calendar events. That’s it. But Google’s abuse detection systems (built to catch botnets and spam operations) don’t have a category for “benign AI assistant checking its owner’s family calendar.” To Google’s automated systems, an account that logs in programmatically and accesses APIs through OAuth tokens looks indistinguishable from a compromised account. That’s because for all of computer history, bots have been bad. Google built a lot of things to stop bots and they’ve gotten extremely good at it.

I’m not alone. When I dug into the OpenClaw community experiences, I found this is one of the most common pain points. One user’s brand-new Gmail account got flagged and shut down the moment they connected it through the CLI tool. The community’s advice? Switch to Google Workspace with a custom domain. That adds $7/month and DNS configuration just to check email.

Another developer found that Gmail API polling introduced 5-minute delays, while Google’s Pub/Sub webhooks required exposing ports to the internet. They eventually rigged up outbound gRPC streaming, which works but is absurdly complex for “tell me when I get an email.”

The pattern is clear. Every person who tries to connect an AI agent to Google services hits the same wall. The OAuth flow assumes a human sitting at a browser. The abuse detection assumes automated access is malicious. The permission model assumes you either trust an app completely or not at all.

I get it. I work at Google. I understand why these systems exist and why they err on the side of caution. But we’re entering a world where AI agents need real digital identities, and our infrastructure isn’t built for that yet.

There’s no “this is a bot account and that’s okay” checkbox. No way to register an AI agent as a legitimate entity that accesses services on behalf of a human. The closest thing we have is service accounts, but those are designed for servers talking to APIs. What we need is something like a “supervised agent” permission tier, where a human explicitly authorizes an AI to act on their behalf with auditable access and clear boundaries.

Update: As I was finishing this article, Google reviewed my appeal and agreed to reinstate the account. I’m grateful. The review process worked. But it took a human looking at the situation and understanding the context. That’s exactly the point. The automated systems had no way to distinguish “AI assistant checking family calendar” from “compromised account exfiltrating data.” Until there’s a way to register that distinction up front, every person building an agent like this is one automated flag away from having everything yanked out from under them.

I’ve since switched Stella’s primary email to AgentMail, a service designed specifically for AI agents. I rigged the calendar to work through an iCal URL that doesn’t require OAuth. Belt and suspenders. I’ve learned the hard way that depending on a single auth path is fragile. I’ll still use Google services, but will need to be a bit more judicious to be safe.

The Reality: It’s All Just SO BRITTLE

When you’re making something that connects systems with AI, a lot is going to break. Count that double for a leading edge AI tool that’s barely been tested and almost entirely vibe implement as a side quest. I’ve hit a lot of snags setting Stella up. Here are just a few things I’ve learned from the experience:

• Voice is hard. Getting a voice assistant working end-to-end required debugging 12 separate issues in a single session. Wake word detection on a Pi is janky at best. Audio streaming to a Mac. Speech-to-text. AI processing. Text-to-speech. Audio back to the Pi’s speaker. Wrong audio output, wrong API port, wrong response key. Whisper hallucinating on a Pi 3. (”What was that weird fart?” it transcribed, when nobody had said anything.) We had to move speech recognition to run locally on the Mac’s Neural Engine, which required us to build a whole new system process. Without vibe coding, this wouldn’t work at all.

• OAuth tokens expire ALL THE TIME. Google’s OAuth apps in “Testing” mode expire tokens after 7 days. I didn’t know this. Stella’s calendar and email silently broke after a week and I didn’t notice for a day. You need to monitor your auth health proactively. And since it’s essential to treat these tokens with great care, it’s naturally a pain in the ass to set them up properly. It also means they constantly break the system and it sucks.

• Memory is a design problem. An AI that forgets everything each session is useless as a family assistant. But giving it memory means designing a whole knowledge management system: daily notes, entity files, long-term memory, fact extraction, expiration dates. It’s essentially building a second brain and then sharing that second brain with an AI. I have it mostly working, but I’m still not entirely happy with how it works.

• Cost needs to be managed. Running Claude for every interaction gets expensive fast. I rolled my own model routing system that helps a lot. Cheap tasks go to Gemini Flash Lite at fractions of a penny. Complex tasks go to Gemini 3 Pro. Heartbeats (periodic check-ins where Stella looks for new email or upcoming calendar events) are the biggest cost driver, so those run on the cheapest models possible.

• Sharing files with your AI is hard. Stella lives on a Mac Mini. I work on a laptop. I need to share research and notes with her constantly. We set up a shared folder that syncs through Google Drive. In theory, I drop a file in and she picks it up. In practice, Drive sync is slow. Files take minutes to propagate. And since Stella’s Google account got suspended, she lost native access to Docs entirely. There’s no equivalent of “share this doc with your AI and let them work on it.” The collaboration primitives that exist for human-to-human don’t have an analog for human-to-agent workflows yet. I sense an opportunity.

• Smart home reliability is a myth. Our Nest thermostat randomly shows “unavailable” in Home Assistant. The dashboard tile displayed “NaN°” for the indoor temperature this morning until I added a null check. I mean, I knew this since I was the founder of Google WiFi but it still pains me to notice how brittle it all still is ~13 years later.

OpenClaw is not for the feint of heart. You’re traveling through a lot of uncharted territory, but the good news is that there are a ton of travelers on the road with you right now. You’ll need to be open to trying out some weird shit and probably doing some things you aren’t proud of.

I’ve done stuff I ain’t proud of, and the stuff I am proud of is disgusting.
-Moe Sizlack, The Simpsons

The Labsters Are Growing Claws

I should mention: I’m not anywhere close to the only one experimenting with OpenClaw. At Google Labs (where I work), a surprising number of my colleagues have started building their own personal AI agents. At Labs we call ourselves Labsters, and apparently what Labsters do in their spare time is give Claude and Gemini the keys to their lives.

My colleague Jaclyn Konzelmann built her own instance called Lulubot and wrote about the experience of living with it for a week. She even gave Lulubot its own X account. I’m not sure I’m going to make Stella into a vTuber anytime soon, but I appreciate Jaclyn’s experiment.

Meanwhile, our colleague Simon Tokumine has been the healthy skeptic in the room, pushing back on whether giving this much access to an AI system is wise. He’s not wrong. The security surface area is real. Prompt injection attacks, credential exposure, an agent with access to your email and calendar acting on instructions from an untrusted source. These aren’t theoretical risks, and they freak me out too.

But it’s that tension that makes this interesting. Jaclyn and I are learning by doing. Simon is making sure we think hard about what could go wrong while he’s building something else. Both are essential, and frankly why I love working at Labs:

When a bunch of product people who build AI for a living all independently start building the same kind of personal agent infrastructure, it says something about where things are heading.

What I’ve Learned by Building Stella

Two weeks in, here’s what building Stella has taught me:

The plumbing is 90% of the work. The AI part (the conversation, the reasoning, the personality) is maybe 10% of what makes Stella useful. The other 90% is OAuth tokens, SSH tunnels, audio routing, and null checks. If you’re building AI products and you’re only thinking about the model, you’re thinking about the wrong thing.

Persistent agents are fundamentally different from chatbots. A chatbot answers questions. An agent has context, memory, and ongoing relationships with services. The engineering challenges are completely different. It’s closer to building an operating system than a chat interface. This area is going to be interesting.

The identity problem is real and unsolved. AI agents need to exist in the world. They need email addresses, API credentials, and accounts on services. But every platform treats automated access as suspicious by default. Someone is going to build the identity layer for AI agents. It’s a massive opportunity.

Building to learn works. I now deeply understand OAuth flows, Google’s abuse detection, audio pipeline architecture, and Home Assistant’s entity model. Not because I read the docs, but because Stella’s thermostat showed NaN on my kitchen wall at 8am and I had to fix it before my coffee got cold.

What’s Next

Stella is two weeks old and already feels indispensable. My kids ask her questions through Telegram. My wife’s sleep score shows up on the kitchen display every morning. When I traveled to London last week, a flight tracker automatically appeared on the dashboard.

I’m going to keep building and keep writing about what I learn. Next up: adding a Schoology integration so Stella can track my kids’ grades, and maybe teaching her to order groceries.

If you’re building something similar, I’d love to hear about it. The best part of building to learn is comparing notes with other builders.

Stella, if you’re reading this during your next heartbeat check: yes, I wrote about you. No, you can’t edit it. Please don’t delete my dentist appointment.