Here are some useful observations I have picked up along the way. I also have a couple of Cucumber-specific tips.
Tip #1 – Execute frequently
Run your automated scripts every day. If you neglect them and re-run them after too many factors have changed, it can be very difficult to troubleshoot. E.g. if your browser updated, the website was modified, and an error crept into your automation code, it could be very difficult to pinpoint the source of the problem. For this reason, I also like to run my code after the smallest change, sometimes just one line, to get to the cause of the failure quickly.
Tip #2 – Address flakiness
Flaky tests are ones that that are ‘brittle’, which sometimes pass and sometimes fail. End-to-end UI tests are often flaky. E.g. a popup might unexpectedly appear, hiding a button that you wanted to click, which could cause your test to fail in error. As soon as you encounter such a test, fix it if possible. If not possible, delete it or archive it for later so that it isn’t included as part of your standard regression run.
Tip #3 – Target IDs or data-testids
Targeting xpaths will result in flaky tests when the HTML of a web page changes. IDs are unique and much more reliable.
Tip #4 – Test in multiple environments
Running your automation in as many different environments as possible (different Operating Systems, different browsers, headless and non-headless) helps to ensure your code is robust, and that the website works on all platforms too.
Tip #5 – Do a full run before committing your code
It’s possible for a new test to run perfectly in isolation, but have an unexpected effect on other test cases, so it’s a good idea to also do a complete regression run to ensure that there are no dependencies between tests, etc.
Tip #6 – Don’t automate everything
The best time to automate a feature (or write any test case for that matter) is when you are currently working on it. When selecting things to automate, start with known bugs, rather than functionality that has always been working.
Tip #7 – Don’t rely too much on automation
It’s perfectly possible for all automated tests to pass with a missing stylesheet, while the website is unusable. Additionally, things like captcha, 2FA and inspecting the console for errors can’t be automated. Just because a full regression run has passed doesn’t mean that everything is working.
Tip #8 – Record a history of passing tests
Automation isn’t just about finding regression bugs – it can be very useful to see proof of when something last worked, so that you have an idea of when the bug crept in.