Welcome back to the final part, part 3 of 3, in this Azure DevOps tutorial series. In the previous two articles, I explained how to set up a repository, create a build pipeline, and set up an Azure release pipeline. Now that all your pipelines are in place, it’s time for testing. In this tutorial, I’ll guide you through the basics of setting up a testing plan that you can run automatically with your pipelines. That way, you can ensure your code is tested and meets your requirements.
Why You Need a Testing Plan
When you develop an application, you must ensure your code is reliable. To test your code, you have all sorts of testing methods you can use. One of the most well-known methods is unit tests. Developers write these tests to check the code they wrote. Another method is UI/functional tests, which the QA team writes too. These two testing methods build a solid foundation for testing.
Generally, you’ll want these tests to run every time you have a merge request. That ensures the new code going into the repo is quality. It also prevents the new code from introducing errors or breaking already existing functionality. Now, let’s build a testing plan which will set you up for automation in the future!
How to Create a Testing Plan and Define Your Backlog Requirements
First thing’s first, you’ll want to ensure your backlog requirements are in order. Depending on your role on the team, you might also need to work closely with a business member, a project manager (PM), or a product owner (PO) on this task. That’s because your backlog items will define your testing plan. Each backlog item equals one test case.
STEP 1: Go to the left-hand sidebar menu and click on Tests → Test Plans → create New Test Plan.
STEP 2: Give your testing plan a proper name and iteration. Your iteration will be a part of your release cycle, so it might look like “myProject/Release 1/Sprint 2”. Then click on Create.
STEP 3: Add a Requirement-based test suite. It’s important to know that requirement tracking is supported only for test cases linked through a Requirement-based test suite. That means you’ll need work items that include a User Story (Agile), Product Backlog Item (Scrum), Requirement (CMMI), and Issue (Basic). These are the 4 Software Development Methodologies that Azure DevOps will accept. You can form an association between a work item and test execution only when you link the test case using a requirement-based test suite. If you don’t link them properly, they won’t show correctly in the testing plan.
Keep in mind that you’ll want to ensure you only include the work items for each sprint. That’s because you’ll generally want to run tests for each sprint. When releasing time comes later, you’ll also want to group all your tests from the sprints that comprise your release into one. This will generate your testing reports for that particular release.
Next, I’ll discuss how you turn your work items into actual tests.
How to Turn Your Work Items into Tests
In this section, I’ll show you how to turn your work items into test cases. Once you have your test cases ready, you can run them against your application.
STEP 1: Click on Requirement-based test suite. Then, on the right-hand side of the page, click on New Test Case. If you don’t have existing test cases, a button to create them will appear at the bottom right. Click on that to start creating your first test case.
STEP 2: Inside the new work item, enter a title for the test case and then click on Click or type here to add a step. Each of these lines will equal a test you want to measure your application against.
STEP 3: Once you write all your test cases, you should also ensure they’re added to your test suite. If not, you can manually add them.
You’ve now created your testing suite, congratulations!
Series Overview
In this article series, you learned the following skills in Azure DevOps:
- Create a repo
- Add code to the repo
- Create a build pipeline for your repo
- Add tasks to the pipeline
- Create an artifact from that build pipeline
- Create a release pipeline and 4 environments to deploy to
- Set up a testing plan
- Add test cases to the testing plan
I hope these skills help you enjoy your time with Azure DevOps.
Final Thoughts
In this tutorial, I stepped outside the pipelines to look at the testing plans. You now can configure a basic testing plan based on your work items in the backlog. Testing is a very important part of software development. Without it, you’ll have bugs that detract from the user experience. Azure DevOps also makes it much easier to create tests and connect them with their work items.
Finally, this three-part series should help you hone your Azure DevOps knowledge. All in all, I hope you can use it for reference in your future projects.
Got more questions? Check out the FAQ and Resources sections below!
FAQ
Can I copy test cases from one project to another?
Yes, you can copy test cases from one project to another. For Azure DevOps Server 2020 and later versions, you can also copy test cases from within a project or another project to a test suite. Optionally, you can bulk, import, and export test cases.
Are parameters the best way to specify test runs on different operating system platforms?
It’s better to use test configurations. You run the different parameter values one after another with test case parameters. This also makes it difficult to switch from one platform to another.
Can I use test cases in an automated pipeline?
Yes, you can use the “Visual Studio Test” task to run any of your test cases in a pipeline. When you’re building your pipeline and using this task, you’ll need to specify your display name and test selection. You’ll also need to specify the names of your test files, which folder to search, and the test results folder. Once you specify this information, you can run your testing task.
Should I automate all tests?
No. While you can automate almost anything, it isn’t advisable to automate every test. Some tests require manual validation. Some complex features within an application also need a human to validate them.
What tests should I not automate?
Before automating a test, you’ll want to ensure it has been thoroughly tested manually. You’ll also want to ensure you know all the possible outcomes and document them well before automating. Otherwise, you may produce a false positive when you run automated tests.
Resources
TechGenix: Part 1 Azure DevOps: Setting up a Repo and Creating a build Pipeline Article
Check out the first article in this three-part series on getting started in Azure DevOps.
TechGenix: Part 2 Azure DevOps: How to Create a Release Pipeline
Learn how to create a release pipeline in Azure DevOps in the second part of the series.
TechGenix: Article on Azure Blueprints
Learn more about using automation to improve your deployments on Azure.
TechGenix: Guide on How to Setup Azure DevOps for Blue/Green Deployments
Learn how to get your evenings back and do day deployments with a blue/green approach.
TechGenix: Article on Moving From Azure Portal/Powershell to Azure DevOps
Experience the change when moving from Portal to DevOps and unleash your true potential.
