Somewhere along the way, I started making my manually-written tests do more that just verify features.
I started using them to document how features worked, where buttons lived, and what users could expect to see on their screens. What began as a technical artifact for quality assurance quietly turned into my main documentation process.
Once I started working on AI-generated acceptance tests, it quickly became obvious that I could have the AI agent carry on this documentation process.
How an AI-Generated Test Doubles as Documentation
Each acceptance test the agent generates is actually a method within a Codeception test class. One thing I do a lot is use a single class to represent a single user objective. Then each method is a different way of achieving that same objective. This means that the documentation produced by running that one class is one or more sets of instructions for achieving the same end goal.
At the top of each class, the agent adds a title and introduction to the accompanying markdown file, summarizing the feature or problem the test addresses. Then, for each test method, it writes command to output a subheading and a paragraph explaining that specific test. This, of course, is of followed by all the step code documentation.
When I run the full test suite, the output is a markdown document that reads less like a raw log and more like a guided tutorial. For example, if for a user outcome there is both a settings-based method and a code-based method to achieve the same result, the markdown will describe both in turn. The reader can see two solutions, each fully verified through the testing process.
It’s not just that something works — it’s how it works, demonstrated in a sequence that both the computer and a human can follow.
The test ensures correctness; the markdown makes it readable.
From Test Run to Tutorial
That dual nature — verification and explanation — makes these files ideal for generating user-facing documentation. Each test’s goal is typically aligned with a feature a real user would care about: enabling a setting, changing a layout, or validating a visual effect.
The only issue is granularity. The AI writes with the precision of a tester — including every setup action and click, even those a user wouldn’t need to perform. A user trying to make a style change to a site probably doesn’t need a step-by-step guide on how to log into WordPress along the way.
So I added a refinement layer. Another AI process takes the raw markdown file and abridges it — keeping the essential steps, trimming the setup, and rewriting the text in a tone appropriate for documentation. The result reads like a friendly, human tutorial: “Here’s how to hide the gallery arrows,” not “Click this selector, wait for that element.”
Once the markdown is at the right level of detail, a script automatically uploads it as a new post on my website. Each post is a verified, illustrated walkthrough, generated directly from a test that was run successfully on a real system.
The entire path — from AI-generated test, to raw markdown, to abridged markdown, to published tutorial — is automated end to end.
Turning Posts into Videos
Of course, not everyone likes to read text-based tutorials. Many users would rather watch a video demonstration than read an article. The nice thing is that the markdown’s step-by-step structure lends itself perfectly to automation in that medium, too.
So I wrote a script to convert the markdown to a video.
For each step, the AI generates a short narration script, which I convert to speech using a text-to-speech model. Each step’s screenshot becomes a visual slide, with the narration overlaid. Then, using ffmpeg, the slides and audio files are stitched together into a single cohesive video — a kind of AI-generated PowerPoint-style presentation.
The final product is uploaded to YouTube and embedded into the documentation post, offering two synchronized formats: one for readers, one for viewers.
And because the entire process is driven by the acceptance test, the video isn’t just illustrative — it’s guaranteed to reflect the actual working behavior of the plugin as tested.
The Overlap Between QA and Knowledge Management
What started as a system for verifying correctness has evolved into a system for capturing and communicating knowledge.
The acceptance test, once a purely internal artifact, now serves as a single source of truth — a canonical description of how a feature behaves. If the plugin or its UI changes, I can simply rerun the test. The system will re-generate the screenshots, confirm that the steps still pass, and update the documentation accordingly.
It’s a powerful form of synchronization: testing and documentation updating in tandem.
This closes a long-standing gap in software projects. Developers often treat QA and documentation as separate domains — one about correctness, the other about communication. But in reality, they describe the same thing from different perspectives: what the software does, and how it does it.
For a solo developer like me, such automated documentation is transformative. Producing detailed, illustrated documentation for every use case by hand would be prohibitive. But when the same process that guarantees correctness can also generate a clear explanation, the economics change. I can now afford — in time, cost, and effort — to document everything.
And that’s not just convenient. It’s a kind of transparency that benefits both developer and user: confidence that what’s documented isn’t hypothetical, but demonstrated; not described, but tested.