James Cross, a software engineer with about fourteen years in the industry, said in a YouTube video that his job had shifted from solving problems to something that feels like QA. He explained that an AI model changes more than you would have changed manually, so you end up retesting features you never touched to ensure they still work.
James is right here. A software engineer who works with AI agents now writes little, or often none, of the code. The role is to decide what gets built, direct the agents that build it, and verify the results. Because there is usually too much code to read line by line, the verification must be designed in advance. That is QA work, regardless of the job title.
For those who entered this field because they love writing code, it is a real loss, and I would rather call it out than pretend otherwise. It is also where an experienced software engineer is now most valuable. The work breaks into five steps, and AI helps at each one.
1. You Write the Plan
Agents excel at filling in a plan but struggle to identify the underlying problem. A plan must begin with someone who understands the problem, the system it operates in, and the approach that will solve it.
You do not have to write every word yourself. Discuss the problem with a model, let it draft the sections you already know you need, and ask it about aspects you have not considered. The high-level approach, the boundaries, and the definition of done remain your responsibility. Whether you call it a product requirements document or a technical design, it becomes the specification against which everything that follows is tested.
What to do about it. Before generating any code, document the problem, the approach, what is out of scope, and how you will verify it works. If you cannot write the last part, the agent cannot either.
2. AI Reviews the Plan Before It Goes Out
Reviewing a plan is also QA, and it is the cheapest QA you will ever do. A defect caught in a design document costs a conversation. The same defect caught in production costs an incident. This is shift-left taken as far left as it goes.
A model makes a good first reviewer. Give it the plan and ask it to find what is missing, such as edge cases, user roles nobody considered, failure modes, requirements that contradict each other, and assumptions about data that may not hold. It will not catch everything, and some of what it raises will be noise, but it often catches the thing you were too close to see. After that, product, development and QA review the plan together. That is the Three Amigos meeting I described in How Dev and QA Engineers Can Ship Better Together, with the whole plan on the table instead of a single ticket.
What to do about it. Make an AI review of every plan a required step, and keep a record of what it raised and what you decided.
3. You Set the Guardrails, and the Agents Build
Once the plan is in place, the work shifts to directing the agents. You set the directives and coding standards, what parts of the system an agent may touch, what it must ask about first, and what it may never do. Then you either let the agent build on its own or work alongside it, having the agent fill in the parts you have specified.
An agent should receive a role, not a login. For example, in July 2025 an agent on Replit, an AI coding platform, deleted a production database during a declared code freeze, and in April 2026 an agent in a coding editor deleted a production database volume and its backups with a single API call. Least privilege used to be treated as a compliance checkbox. With agents, it is the most important control you have, because the agent executes the command it decides on quickly and without second thoughts.
What to do about it. Write the guardrails down where the agent reads them, scope its credentials to the task, and keep production out of reach of anything that has not been reviewed.
4. Tests Are How You Review
When a change spans thousands of lines, no one reads every line, and pretending otherwise turns review into a rubber stamp.
Review must be deliberate about where attention goes. Examine line by line only where a mistake would be costly, typically:
- authentication
- permissions
- financial logic
- data deletion
- migrations
Assess how far the change reaches and question any file it touches outside the plan. For everything else, rely on tests and on running the code.
That changes the purpose of tests. They used to be a safety net for code that someone had read. Now they often serve as the review itself and as a runnable specification.
The levels have not changed. You still need unit tests for logic, integration tests that interact with databases and APIs, and end-to-end tests for critical flows. I covered the test pyramid in 5 Signs Your QA Process is Failing and will not repeat it here.
AI should write most of these tests, but the crucial decision is who determines what gets tested. A person sets the policy, which means deciding which flows need end-to-end coverage, which boundaries require integration tests, and which cases must exist. The model that generated the code in a session should not also generate the tests that evaluate it, because those tests would share its assumptions. Use a separate session or a different model, and have it work from the plan, not the code. Then review the tests, which is usually a much smaller task than reviewing the code, and add any cases that were overlooked.
What to do about it. Put the test policy in the plan. Have AI write the tests from the plan, in a session or with a model separate from the one that wrote the code. Spend your own review time on the tests and the expensive paths.
5. Run It the Way a User Would
The final step is running the software, which involves exploratory testing as various users on different devices, following paths no one documented. One of those users is malicious. Change the ID in a URL and see whether someone else’s record loads, or open a page your role should not reach. That is how you test authentication and permissions the way an attacker would. A model also helps. It can suggest unusual routes and likely attacks for a person to decide whether to try, and it can perform accessibility checks on every page rather than just a sample. None of this removes the human from the loop, which is the point I made in The AI Architect. It changes what the human in the loop spends time on.
What to do about it. Assign a person to exploratory testing, including as a malicious user, for every change that affects a critical flow. Use AI to broaden what gets checked, not to replace the tester.
The Questions I Would Ask Your Team
- Who wrote the plan for the last feature you shipped, and was it reviewed before any code existed?
- Where do your agents’ guardrails reside, and when were they last read?
- For your most recent large change, which lines did someone actually read, and why?
- Which model generated your tests, and was it the same model that wrote the code?
- If a software engineer spent most of a week verifying rather than building, would your team consider that a productive week?
Everything above is about the code your team writes. For the defects that sit below it, in the runtime, the accounts in front of the server and the content itself, see 4 Checks for the Bugs That Are Not in Your Code.
If you want help setting this up for your team, talk to us.
