Prompts Are Code: Versioning, Structure, and Testing

Prompts that live in string literals rot fast. Treating them as versioned, structured, tested artifacts is what keeps LLM features maintainable after launch.

Every team I have seen ship an LLM feature goes through the same arc: the prompt starts as a one-line string, grows into a paragraph, gets copy-pasted into a second feature, and six months later nobody can change a word of it without fear. That is not a model problem. That is what happens to any code you refuse to treat as code.

The practices that fix it are the ones you already know: structure, version control, and tests.

Give the prompt a structure

A production prompt has distinct parts: the role and constraints, the task, the context you inject, the output format, and examples. Keep them visibly separate, with clear delimiters around anything user-provided so instructions and data cannot blur together. A prompt built from named sections can be reviewed; a wall of text cannot.

Keep prompts in files, not in string literals

Move each prompt into its own file with a name and a version, and load it where needed. You get diffs in code review, blame history when quality shifts, and one place to edit instead of five call sites. When a prompt changes behavior in production, "git log" on that file is your changelog.

Few-shot examples are your spec

Two or three worked examples in the prompt do more than any amount of instruction prose, because they pin down edge cases: what to do with empty input, how to format the tricky case, when to refuse. Choose examples that encode your hardest decisions, not your easiest ones.

Test prompts like you test functions

Build a small evaluation set: 20 to 50 real inputs with expected outputs or scoring criteria. Run it on every prompt change and on every model upgrade. It does not need a framework; a script that prints a pass rate is enough to turn "I think the new prompt is better" into a number.

• Include the inputs that failed in production; they are your regression tests.
• Score structure programmatically (did it return valid JSON, did it cite a source) and quality by spot check.
• Rerun the set when the provider ships a new model version; silent drift is real.

Takeaways

• Prompts are load-bearing code; give them files, versions, and reviews.
• Structure plus delimiters beats clever wording.
• A small eval set makes prompt changes safe instead of superstitious.