Domain-driven design: An unexpected kryptonite for LLM applications

Flexibility breeds opportunity

For most of my career, I’ve been a domain-driven design zealot. In complex, operational industries, robust domain models are invaluable. A well-designed data model allows you to build simple customer workflows stitched gracefully atop concise representations of highly complex businesses. Deterministic software engineering is an exercise in materializing simplicity from complexity, often achieved by constraining your representation of real-world entities. Freight tech companies often spend years optimizing their “shipment model” to support multiple legs, track live progress, and enable easy querying of derived data properties. Data models are tightly coupled to workflows, ensuring reliability and clarity within their domains.

When application data is input to or output by large language models, highly structured data representations become a crutch. For more localized generations, such as generating JSON or "enum" values, this isn’t a significant problem—it’s relatively easy to constrain model outputs to fit any desired representation (tools like Instructor or OpenAI’s “Structured Outputs” feature make this trivial). For larger workflows, workflows, which fundamentally wouldn’t be possible without the power of LLMs, restricting the output space of generations severely limits the value you can extract from your application. Intelligent models, with adequate context, continually expand their capabilities in ways that are hard to predict. Application data that is stored and accessed with excessive structure can narrow the aperture for opportunity available to your application.

Harkening again to logistics, the "killer application" for LLMs in that space has been email processing. Models can handle processing data with a variety of types and adapt to determine an appropriate next action given input in various modalities (parsing free-text emails, structured attachments, and metadata fields). Supporting all possible use cases requires saving partial data in widely varied formats, enabling the system to process incomplete or inconsistent inputs. If the pipeline for reading and writing from a data model is extremely opinionated—requiring every invoice document to have a direct reference to an invoice amount or allowing “proof of delivery” ID numbers to be labeled only with one of a few predefined labels. Such rigid constraints would hinder the system's ability to process the diverse and unpredictable nature of unstructured data, thereby severely limiting the automation potential of the application.

When data models have strict constraints—requiring many fields or minimal data duplication—developers face significant challenges in tuning model prompts for optimal performance. For example, relational data models often span 5-10 “generations” of parent-child relationships, making it extremely difficult to ensure that LLMs appropriately "focus" on critical leaf properties in deeply nested data structures. Rigid approaches inhibit developers' ability to experiment with model performance leveraging a variety of data structures.

After a few too many engineering decisions I’d come to regret, I’ve begun to incorporate a few main principles to develop LLM applications that can handle the long tail of input data—impossible to model by even the most comprehensive engineer design.

• Consider defining data objects primarily at the application layer, employing in-memory validation and other techniques rather than enforcing strict schemas at the persistence layer. For example, storing raw documents (images, input text, transcripts) more frequently can support post-processing and enable greater flexibility.

• Applications can pass raw, unstructured data versions between one another, allowing LLMs to reformat and contextualize outputs dynamically for different user groups and use cases.

• Embrace loose coupling between data storage and workflows by leveraging formats like JSON blobs or raw markdown. This allows systems to evolve dynamically, making it easier to experiment with new LLM generation approaches without completely reworking foundational data schemas.