Continuous integration (CI) is the practice of merging all developers’ working copies to shared mainline several times a day.
One of the most important aspects of CI is keeping the mainline healthy. That’s why it’s crucial to have proper tests, both unit and UI, that can run against any PR and/or commit to the mainline. At the same time, these tests should also be fast. It’s usually not a problem with unit tests, but UI tests, especially if you want to run them on multiple devices, can take a rather long time.
As a balance between speed and reliability, for PRs into the mainline, we use smoke UI tests that only prove the main functionality is working. Proper UI testing with manual QA verification is done only when we’re preparing a new release candidate.
In this article, I will show you how we’ve set up Azure Pipelines for running automated UI smoke tests for a Flutter app.