When you start building an AI agent for engineering work, the first decision is almost always the same: write a big system prompt. You want to describe everything in it. How to work with Playwright. How to run tests. How to analyze errors. How to write assertions. How to work with APIs. How to avoid unstable selectors. How to create test data. How to format the result.
At first this feels right. The more rules we give the agent, the better it will work. But in practice you quickly see the opposite: a big prompt doesn’t make an agent more reliable. It makes it overloaded.
This became one of the main takeaways from experiments with AI tools for testing. The model may know a lot, but it doesn’t need to know everything at once. If the agent is writing a UI test right now, it doesn’t need all the API rules. If it’s analyzing a Playwright error, it doesn’t need a full unit-test strategy. If it’s working with CI logs, it doesn’t need a detailed DOM description.
A human engineer doesn’t work through one endless prompt in their head. We pull the knowledge we need for the specific context. That’s the idea I wanted to bring into Cairn.
Why the mega-agent starts to break
A mega-agent is a tempting idea. One agent that knows everything, sees everything, solves everything. It analyzes the repository, writes tests, runs them, reads errors, fixes them, assesses quality, and produces a report.
But in practice such an agent quickly hits cognitive overload. It starts mixing up levels of responsibility. For example, instead of using a ready-made tool to run tests, it may try to generate its own runner code. Instead of focusing on check logic, it burns context on infrastructure details. Instead of working with a specific error, it recites general best practices.
And the longer the session, the worse it gets. Context accumulates: previous attempts, log fragments, code snippets, old decisions. The original goal gradually blurs. In the end the agent may produce something formally correct but not very useful.
This is a lot like giving a junior engineer all the project documentation, all team rules, all possible edge cases at once and asking them to “just write a good test.” There’s plenty of information, but little focus.
The Skills-First idea
Instead of building one agent with a huge prompt, I started thinking in terms of a skills architecture. Roughly: not “the agent knows everything,” but “the agent gets the right skill at the right moment.”
In Cairn, a skill isn’t just a text instruction. It’s a separate module that can contain a short description for the model, activation rules, solution examples, constraints, and executable code for deterministic operations.
For example, a Playwright skill shouldn’t just say “use good locators.” It should give the agent a practical frame: when to use role-based locators, how to work with expectations, how to avoid hard waits, how to collect a trace, how to return test-run results in structured form.
More importantly: some work shouldn’t go to the LLM at all. If you can deterministically run a test, collect stdout, stderr, a screenshot, or a trace — code should do that, not the model. The model should make decisions, not manually replay infrastructure steps.
That’s where Skills-First becomes practical. It separates what the LLM should do from what ordinary code does better.
How this looks in a real process
Imagine Cairn gets a task: create tests for user registration.
The first step isn’t to write a test right away. First you need to orient yourself. What kind of project is this? React, Vue, Next.js? What test framework is already in use? Playwright or Cypress? Where do existing tests live? How does the team structure fixtures? Is there an API for data setup? Is there CI configuration?
After that Cairn should load only the relevant skills. If the project uses Playwright — the Playwright skill activates. If data needs an API — the API skill is added. If there’s a separate reporting format — the reporting skill connects.
In this approach the agent isn’t dragging the whole world along. It gets a compact set of knowledge for this task only.
Then it can build a plan: happy path, negative scenarios, email validation, weak password, existing user, successful redirect. Then generate the test, run it, read errors, and loop back to fixes if needed.
The key is that each step happens within the right skill. That reduces chaos and makes agent behavior more predictable.
Why this matters for QA
In QA automation, small mistakes often cost a lot. A test can be flaky because of one wrong wait. It can create data and not clean it up. It can pass only on a local machine. It can be tied to text that changes after the first UI update. It can check implementation instead of user behavior.
An AI agent without clear skills easily makes these mistakes, because its main job is to generate a plausible solution. A plausible solution isn’t always a good one.
Skills-First architecture lets you embed engineering habits in the agent, not just syntax knowledge. For example:
- think about test isolation;
- don’t trust a single happy path;
- check business outcome, not just a UI element;
- use stable locators;
- account for asynchrony;
- return structured results;
- don’t hide failures behind “the test passed.”
This matters because Cairn isn’t just supposed to write tests. It’s supposed to help build trust in changes.
From eval to skills
The link to the previous article on eval is direct here. If eval is a mechanism of truth, skills are how you make that mechanism stable.
You can’t ask the model every time to “please assess test quality” in free form and expect equally useful answers. You need structure: which criteria to use, what response format, which fix rules, when a test is good enough.
Same for generation. You can’t rely every time on the model’s general Playwright knowledge. You need a skill that reflects how the team works or specific best practices.
In that sense Cairn isn’t just an agent with LLM access. It’s a system that tries to formalize engineering experience as skills, loops, and checks.
For another angle on Cairn — verifying AI-generated code after the fact — see From Eval to Your Own QA Agent.
Why this isn’t “replace QA”
I don’t like the idea of selling these systems as a replacement for QA engineers. On real teams, QA isn’t only writing tests. It’s understanding risk, communicating with product, analyzing requirements, prioritization, test design, working with uncertainty.
Cairn isn’t meant to replace that thinking. But it can help where there’s a lot of repeatable work: create a first version of test coverage, run scenarios, find obvious weak spots, generate check variants, analyze errors, point out where a test is insufficient.
So Cairn’s role isn’t to take the job away. It’s to shrink the distance between an idea and a verified result.
If a team designed a feature, AI helped implement it, and Cairn helped quickly create and verify a test set, the QA engineer doesn’t get an empty desk. They get material already prepared for review, improvement, and integration into the overall quality strategy.
Conclusion
After trying different AI approaches, it became clear: a big prompt isn’t architecture. It can help at the start, but it scales poorly when the system has to work with a real project, different technologies, and the cyclical nature of QA.
Skills-First seems like a healthier path. It gives the agent not all knowledge at once, but exactly the skill needed now. It separates decision from execution. It moves deterministic work into code and leaves the LLM where flexibility is needed.
For Cairn this became one of the core principles. If we want an autonomous QA agent to not just generate tests but help move from idea to evidence-backed implementation, it doesn’t need more instructions. It needs the right skills architecture.