Building the Solution Right
Begin with Testing in Mind
What makes a solution “right?” Thorough testing, starting with discovering needs and curating them into requirements. At this stage, stakeholders determine acceptance criteria, the final arbiter of a “right” solution. While quality assurance (QA) finds and corrects defects during development, the organization accepting the solution for production makes it right.
After a solution version passes final quality assurance, the business analyst leads user acceptance testing (UAT). People who will use the solution verify that it meets the acceptance criteria and real-life use cases. A successful UAT confirms that the solution version is ready for production.
Business analysts create use cases for users to test the solution in UAT. They should also create use cases to cover exceptions, and alternative process flows to stress the solution. For instance, if a testing user takes several steps to complete a transaction, cancels the transaction, and performs a different set of steps for another transaction, the data should reflect the same state as if the canceled transaction never happened (except in the logs).
End-to-End Validation
Business analysts and architects validate the solution to confirm it fulfills its requirements. First, they run it through use cases, looking for expected results. If validation returns an unexpected result, they report the issue to the development team.
Validation sometimes reveals gaps where the solution doesn’t fulfill one or more requirements. Business analysts check each gap to see if it stemmed from the requirement or implementation. If it arose from implementation, they log a task for the development team to close the gap. In some cases, the requirement needs revision or is no longer needed. The validation process checks the requirements as well as the solution.
Verifying that the solution meets requirements also provides an opportunity to revise and refine the use cases. Since UAT users don’t routinely test software, the cases should clearly show them what to do and expect at each step. Validating well-written use cases sets the stage for a successful UAT.
Testing Deliverables
The first three stages of solution development have deliverables for testing, as shown in the table below.
| Stage | Done When | Testing Deliverables |
|---|---|---|
| Discovery and Analysis | Requirements cover the version’s scope | Initial acceptance criteria |
| Design | Development team understands requirements and specifications | Use cases for quality assurance Acceptance testing cases |
| Development | The solution passes quality assurance and user acceptance tests | Unit testing, quality assurance, and user acceptance testing |
The Design and Development testing deliverables reflect four levels of testing:
| Test | Confirms | Run by |
|---|---|---|
| Unit Tests | Code works properly | Developers |
| Quality Assurance | Solution works properly | QA staff |
| Validation | Solution should pass UAT | Business analyst |
| UAT | Solution meets acceptance criteria | Users |
Testing begins with the end in mind. It starts with eliciting acceptance criteria with requirements during discovery. Fulfilling these criteria in UAT concludes development. Quality assurance and validation follow use cases created before building the solution. The software development “V” model below shows testing deliverable creation and execution are over time.
User Acceptance Testing
UAT confirms that the solution fulfills its requirements by having users test it against the acceptance criteria. Ideally, the testing users will go “off-script” from the use cases and stress the solution based on their real-life experience.
For instance, a retailer wants to create an online shopping experience based on customers’ browsing habits. During UAT, selected users browse through merchandise, then verify the solution recommends the right related items. Another user with little online shopping experience browses and purchases items then returns some. The business analyst and architect look for opportunities to improve the new user’s experience.
A savvy marketing manager browses a wide range of merchandise then checks the recommended items. They come from each of the product categories she browsed. She says it should show only products related to the most recently viewed item. The business analyst confirms the requirement for this behavior and logs a task for the development team to implement the required behavior.
If UAT uncovers gaps where the solution doesn’t meet the acceptance criteria, the business analyst captures all of the gaps. Then, they have the development team close those gaps.
When a solution passes UAT, it is ready for production.
How Did We Do?
Testing starts in a project's discovery and analysis stage by creating acceptance criteria from well-understood requirements. It continues with writing use cases for those testing the solution - quality assurance staff and users testing for acceptance.
Business analysts validate the solution with the use cases, checking solution behavior against the requirements while ensuring the use cases will make sense for users running acceptance tests. Validation can also uncover requirements gaps or inaccuracies, extending quality assurance back to the requirements.
Well-managed user acceptance testing provides the ultimate metric for business analysis. If gaps emerge in the testing, a business analyst determines where communications broke down to create the gap. For instance, a discovery oversight provides an opportunity for the analyst to improve future discoveries. Likewise, if the development team misunderstood a requirement, the analyst takes the opportunity to improve team communications to prevent future misunderstandings.
End-to-end testing based on well-understood requirements leads to building the solution right.