I often get asked, “why can’t I update any of the fields on the Associated Automation tab of a Test Case?”
TL;DR: These fields are read-only in the web portal and can be updated via Test Explorer in Visual Studio.
Why would you want to associate automated tests with Test Cases, a manual testing tool?
- End-to-end traceability. By linking your requirements (PBI’s, User Stories) to test cases, you can establish the quality of the requirements based on test results. Ideally a test case is created for each of the acceptance criteria listed for the requirement. Are any of the test cases failing? If so, the requirement is not complete.
- More user-friendly. Some of your testers may not be familiar with Azure Pipelines (CI/CD). The automated tests can be run by triggering a Test Case, rather than a pipeline.
Setting up the automated tests
Before we can associate an automated test with a Test Case, we’ll need to do a couple of things. First, the automated tests will need to be stored in source control. This can be GitHub, Azure Repos, Bitbucket, or any other source control provider of your choice. Azure Pipelines can consume code from just about anywhere. Next, we’ll need to build and package our tests by creating a continuous integration build. Finally, our release can consume the build artifact and execute the tests using the VsTest task:
Create a new release definition (or use the application’s release that’s being tested):
Add the build artifact that contains the automated tests:
Add 2 tasks to a stage. In my example I’m using my “Test” environment for the stage. The first task will install the Visual Studio Test Platform. The second task will execute the tests using VsTest. Set the second task’s platform version to “Installed by Tools Installer” so you don’t need to manually install anything on the environment/server that’s running the tests. We’ll be triggering these tests through a Test Case, so set the “Select tests using” field to “Test Run”.
Configuring Azure Test Plans
Next, we can configure our Test Plan to use our build and release definitions. This can be done in Test Plan settings:
We can now execute automated tests in our Test Plan using our automated tests pipeline.
Using Visual Studio to associate to Test Case
The last step is actually associating the tests with Test Cases using Visual Studio. Open the solution containing the tests, then use Test Explorer to right click “Associate to Test Case”:
Find the Test Case:
After saving, you can then find the fields in the Test Case’s Associated Automation tab populated:
Running the tests
Finally, testers can now trigger the automated Test Case!
Then see the results in Test Runs:
My examples use the open source TailSpin Space Game web application that can be found on GitHub here: https://github.com/MicrosoftDocs?q=spacegame