Beta Bot: AI-powered writing feedback

Beta Bot is a local-first writing app I built for organizing manuscripts and getting AI feedback. The interesting problem is context: a useful critique of chapter twenty needs to know what happened in the chapters before it.

I built a workflow that turns earlier writing into structured context, combines it with the current chapter, and asks a reviewer with a specific editorial perspective for feedback. The writer can inspect the prompt and keep the resulting review alongside the chapter.

Giving the model enough of the story

Sending an entire manuscript with every request would repeat a lot of text. Sending only the latest chapter would leave the reviewer guessing about established characters and events.

Beta Bot uses summaries at two levels. Chapter summaries capture the point-of-view character, cast, locations, plot beats, and a factual overview. Part summaries condense a longer story arc. A review can then use previous part summaries and earlier chapter summaries from the current part, alongside the full text of the chapter being reviewed.

The summaries are editable. If a summary misses an important detail, the writer can correct that context rather than repeatedly trying to steer the reviewer around the same misunderstanding.

Different reviewers for different questions

An encouraging reader, a developmental editor, and a line editor should not give the same kind of response. I added built-in reviewer profiles for those perspectives, plus custom profiles with editable prompts.

The profile supplies the system prompt. The application assembles the story context and chapter into the user prompt, then stores both the review and the prompt used to produce it. That makes feedback easier to revisit and inspect: the writer can see what the model was responding to.

Turning summaries into a story bible

The same summary workflow can optionally update wiki pages for characters and locations. Detected entities lead to new pages or updates to existing ones, with links back to the source chapter. Existing page content is included when requesting an update, so the model can incorporate new information and flag contradictions.

This connects an AI response to durable application data. The chapter, its summary, and the relevant wiki entries become useful inputs for later writing work.

Keeping the writer in control

Books, chapters, summaries, and reviews live in a local SQLite database. AI requests go directly from the app to OpenAI using the writer's own API key when they invoke an AI feature. The text included in that request leaves the device; keeping the rest of the library local does not make the AI request itself private to the device.

The model can still misunderstand a scene or produce an unhelpful review. Editable context, inspectable prompts, and saved feedback give the writer ways to understand and respond to that, while leaving editorial judgment with them.

For me, this project has been an exercise in making LLM output useful inside a larger product: selecting context, supporting different user intentions, and connecting generated text to records people can revisit.

Explore the example book without an account or API key, or browse the source on GitHub. Generating new AI feedback requires an API key.