Building an AI Story App: Architecture and Lessons from Inky

Building an AI story app is often framed as a prompt engineering exercise. The reality is different. When I started building Inky, my multi-agent storytelling engine, I realized quickly that the LLM is the least interesting part of the stack. The challenge isn't getting the AI to write; the challenge is building a system that maintains state, enforces narrative logic, and scales without turning into a token-burning mess.

I am building Inky in public as part of my multi-product studio. This isn't a side project; it is a testbed for agentic engineering—the practice of using AI as the operating layer of a product rather than just a feature. Here is how the system is architected and what I have learned the hard way while shipping today.

The Architecture of a Narrative Engine

Most people building an AI story app start with a single prompt. They ask a model to "write a story about a cat." This works for a paragraph, but it fails for a novel. To build something durable, you have to move away from the chat interface and toward a structured system of agents.

Inky runs on a monorepo architecture. I use a custom agent orchestration layer I built called VERA. VERA doesn't just send prompts; it manages the lifecycle of a story through a series of specialized agents.

The Agentic Pipeline

Instead of one model doing everything, Inky uses four distinct agents:

1. The Architect: This agent handles the high-level plot points and world-building rules. It doesn't write prose. It writes the "truth" of the world into a JSON schema.

1. The Author: This agent takes the Architect's instructions and generates the narrative. It is constrained by the current state of the world.

1. The Editor: This agent reviews the Author's output for consistency. If the Author says a character is wearing a red hat in chapter one and a blue hat in chapter two without explanation, the Editor flags it.

1. The Archivist: This agent manages the long-term memory. It summarizes previous chapters and updates the global state to ensure the context window doesn't overflow.

By separating these concerns, I can swap models for different tasks. I might use Claude 3.5 Sonnet for the Author because of its creative nuance, while using a faster, cheaper model for the Archivist's summarization tasks. This is the core of agentic engineering: picking the right tool for the specific sub-task.

Handling State and Persistence

The biggest hurdle in building an AI story app is state. LLMs are stateless by nature. If you want a story to remain coherent over 50,000 words, you cannot rely on the model to remember what happened in chapter three.

I solved this by treating the story as a database, not a document. Every character, location, and plot point is an entry in a relational schema. Before the Author agent generates a single word, VERA queries the database for the relevant context.

For example, if a scene takes place in a tavern, the system pulls the tavern's description, the NPCs currently located there, and any active plot threads involving those NPCs. This context is injected into the prompt as a "World State" block. This ensures the AI isn't hallucinating details; it is interpreting data.

Lessons Learned the Hard Way

Shipping Inky has taught me that the hype around AI often ignores the friction of production. Here are three specific lessons from the build:

1. Abstractions are Expensive

I started with some of the popular AI frameworks. I ripped them out within two weeks. They added too much overhead and made debugging agent loops nearly impossible. I moved back to raw API calls and custom orchestration. If you are building an AI story app, build your own plumbing. You need to see exactly how the data is moving between the model and your database.

2. The Context Window is a Trap

Just because a model has a 200k context window doesn't mean you should use it. The more noise you put in the prompt, the more the model's performance degrades. I found that "surgical context"—providing only the 500 words of relevant world-building data—produced better prose than dumping the entire story history into the prompt.

3. Latency is the Enemy of Flow

Waiting 30 seconds for a chapter to generate kills the user experience. I had to architect a streaming system that allows the Editor and Author to work in parallel. The user sees the prose as it's being vetted, not after the entire loop finishes. This required a significant shift in how I handle backend events, moving from standard REST endpoints to a more robust WebSocket-based architecture.

Working in Public

I am not interested in building in a vacuum. The studio model works because it allows for rapid iteration and shared patterns across products. The lessons I learn on Inky's state management are already being applied to the logistics tools I'm building for other branches of the studio.

Building an AI story app today is about more than just generative text. It is about architecting a system that can handle the unpredictability of AI while maintaining the rigor of a traditional software product. It is a balance of craft and engineering.

If you are working on something similar or want to look at the specific schemas I'm using for VERA, I am happy to talk.

Full implementation details and the logic behind the Archivist agent are available in the Builder's Playbook.

justintsugranes.dev/resources/builders-playbook