Attention Ruby/Cucumber people. Don’t use PageObject. There’s something better. See for yourself and click [here] or read on…
Let me tell you about a team that’s felt a lot of pain centered around things many Automated Test Engineers driving a web application can relate to.
A team devoted to triaging 22,000 automated Cucumber tests for a complex insurance application.
The tests were farmed across over 300 nodes.
They would have taken a week and a half to run if the team didn’t run them in parallel.
Still they were running into problems that are all too common among automated testing efforts.
They drew up a fishbone and noted that
- Tests took too much time/effort/money to maintain
- Reporting was useless. Test results told them little. There were many ‘failures’ every run, but less than 10% of these were ‘failures’ in the app. Most were indications they needed to do more maintainance.
- Etc.
These two things alone should have been addressed long ago and put in libraries/gems/test suites hidden in repos at corporations or startups.
Living in the Ruby/Cucumber webapp test automation world, the standard has been
- Watir
- PageObject
These libraries have been great in many ways, but there’s something better.
The story above is about my team. That team I was on started a design phase to address all the aforementioned pain points, I heard of a project at another company that had great success with reducing false positives from end-to-end tests. Even long ones. The kind any sensible TE/Developer thinks we should avoid, but all too often the business absolutely has to have them.
If these tests can be maintainable and provide useful results then it’s possible to reach an economy yet unknown and have great ROI on tests that are not so abstract that the business can’t relate to them.
The test pyramid has traditionally said that Cucumber/drive-the-app/end-to-end/monster tests are at the top, but as Steve Fenton has mentioned in his excellent set of essays, The Humans Are Dead - Essays on Automated Testing, the real important bit is that expensive tests is what you are sparing with.
So let’s have a framework that
- provides organization
- has abstractions for TE-important things (examples: ValidationPoint - A thing you call out as something you check, Ledger - an audit trail of everything your automated test has done)
- emphasizes calling out what you’re validitating and creating exception classes to separate app errors from ‘test framework errors’
- leverages design principles to prevent you from being WET (that’s the opposite of DRY)
- makes it easy to get from place to place in an app (represent routing with a graph)
Here’s a link to a great project that is rapidly spinning up: [OZ Automated Test Framework on GitHub]
Give it a look and at least run the demo in the readme to kick the tires.