Testing a Workflow
Use past work to check a workflow's results, test how it handles problem cases, and decide what is safe to put into use.
Before a workflow touches new work, test it on work your team already knows well. Past orders are ideal: you have the original request and the result your team reached, so you can see where the agent agrees and where it does not.
Finishing is not passing
The goal is not to see whether the agent can complete a run. It is to see whether it gets the job right, spots the cases that need a person, and leaves the result where your team expects it.
The six steps
Write a clear test prompt
Describe the job, the information the agent should use, the checks it should make, and where it must stop. Say clearly what the test must not do. If you have already written the job down with the Define a Workflow template, use it here.
For example:
Help us test a workflow that prepares an order from a customer request. Use only the past requests we select and the information we provide. Extract the items, quantities, and requested date. Check them against that information. Flag missing or conflicting details for a person instead of guessing. Prepare a proposed order for review. Do not send messages, create real orders, change existing records, or start handling new requests. Before running anything, explain how this test will make sure none of those things can happen.
A prompt like this is a starting point, not a guarantee that the test is safe. Before you run it, check how the setup keeps real actions out. If the test is connected to your real tools and includes a step that saves or sends something, it can still change real records or send real messages.
Old input does not make a run harmless
A past order can still cause a new order or message if the workflow acts on it again. Make sure the test cannot do anything for real before you start.
Pick past work that covers the real job
Start with a routine request, then add the cases that made your team stop and think:
- A request with everything needed
- A request missing a detail
- A request that does not match what is in your records
- A request the workflow is not meant to handle
For each one, keep the original request, the information available at the time, and what the team actually did. If the team made an unusual call, note why.
Do not test only on clean examples. A workflow that handles an ordinary request but guesses when a detail is missing is not ready to work with less supervision.
Compare against what your team knew then
A past result only makes sense next to the facts your team had then. Prices, stock levels, and customer details may have changed since the original job.
If the test looks at today's records, a different answer may come from changed information, not a mistake by the agent. Write that difference down, and where you can, use the information as it was at the time of the original job.
Do not assume the old result was right, either. Check both results against the rules you have agreed. Sometimes a test shows a mistake in the old work, or a rule nobody ever wrote down.
Check the result and the path
For each case, compare what the agent proposed with what your team did, and with the rules for the job. A simple notes sheet of your own works well for keeping track.
| What to check | What a good result looks like |
|---|---|
| Details pulled from the request | Items, quantities, dates, and anything else required match the request. |
| Your rules | Mismatches and requests the workflow should not handle are noticed, not passed through. |
| Missing information | The workflow asks for attention rather than inventing an answer. |
| Human review | The steps reserved for a person stay with a person. |
| Finished result | The proposed result is correct, and the test has not changed anything for real. |
Use the Activity log to see what the agent did and how it ended, and check the original request and the place the result went. A run that finishes is not proof that every detail is right. See Logs & Audit Trails for what the record covers and what it does not.
Fix the cause, then repeat
When a result is wrong, find the cause before you change anything. Were the instructions unclear? Was some information missing? Did a rule fail to catch the problem? Could the agent not get what it needed from one of your tools?
After a change, run the case that failed again, along with the ones that already worked. Fixing one problem should not break everyday work.
If the problem was a technical failure, check what already happened before trying again. See Runs, Retries & Failures. Trying a run again is not the same as fixing the rules behind it.
Decide what is ready
Readiness is your team's decision about a defined job. It is not a promise that the agent will handle every possible request.
Before you start using the workflow for real, confirm that:
- Everyday cases produce the right result
- Known problem cases are handled, or sent to the right person
- Any difference from your team's results in the tests has an explanation
- Your team knows who will review new problem cases and how to check the outcome
- You have reviewed what the workflow will do for real, which tools it uses, and where a person approves
Do not let a good overall score hide the cases that still fail. One wrong promise to a customer matters, even if many other cases went well.
Start with a person approving the steps that matter, and let the workflow do more on its own only after it has handled the unusual cases your team actually sees. See Human in the Loop.
Example: an order with no requested date
Your team once received an order request that listed items and quantities but no requested date. They asked the customer for the date before finishing the order.
In a safely bounded test, the workflow should extract the items and quantities, notice the missing date, and flag the gap. It should not borrow a date from another order or pick one on its own.
Compare that with how your team handled it. If the workflow fills in a date, fix the instructions or the rule and test the case again. Then try a different request with the same kind of gap, to check that the fix works in more than one case.
Look for the right flag
The result you want is the right flag raised, not a finished order at any cost.
Where to go next
Your First Workflow
A worked example that tests a reply and ends with an approved, unsent draft.
Validation & Exceptions
How your rules catch work that needs attention.
Human in the Loop
How to keep a person involved in the steps that matter.
Logs & Audit Trails
How to look back at what the agent did and how it ended.
Logs & Audit Trails
How the Activity log records what an agent did, how to use it to review a piece of work, and what it does not cover.
Validation & Exceptions
How Trelium agents check their work against your business rules before acting. Covers what a check is, what an exception is, where exceptions go, and how checks decide what needs a person.