If you decide not to design some parts of TAF at the very beginning, you are guaranteed to have HUGE problems in the long run. Do not deceive yourself by thinking that the future is far away. Issues that stem from poor design can turn out to be nasty. It will not be just 'a bit expensive' to get rid of them. Failing to design TAF layers properly may render the whole system virtually unusable. I always invest a lot into the initial design, and I have never regretted it. It saved me time, effort and sanity.
And, as I mentioned earlier in the previous post, TAF consists of layers:
Tests
As test layer is supposed to be used by people without advanced technical skills, it is best to design it in such a way, so the users do not have to implement new functionality. Instead there should be an opportunity to import or inherit objects, methods and variables from the upper layer, which is helpers layer.
On a side note, I have observed that it is easier to use bare strings, not variables with HTTP paths, even if they are repeated. We did try do move them into a special routes file, but it turned out to be very inconvenient during troubleshooting and defect retest. Thing is, whenever you need to retest something, you need to make sure that all the input data is perfectly correct. New nicely looking routes file cost us a couple of extra clicks and a full text search, instead of quickly looking it up in-place. This is yet another proof that not all the rules should be applied universally.
Helpers: boilerplate code
The lego bits for the tests layer come from that part of helpers that contains boilerplate code and everything repeatable.
The approach I like the best follows the natural structure of a system and builds on its basic elements. If we are testing a site, the base element is a page. If we are testing a REST service, it is an endpoint. The rest depends on how complex the solution under test is, and on your common sense. But the rule of a thumb says that the structure of your helpers layer should match the structure of the solution under test.
Sometimes it makes sense to add yet another level of abstraction, and to implement wrappers for HTTP verbs separately. At the same time, I can think of a couple of cases where it would be inconvenient and unnecessary. Same applies to other parts of HTTP protocol and system configuration. The decision is up to you. Just try to keep everything simple.
Helpers: test lab
Even with well-defined helpers we still need a separate component for managing test runs and configurations. It is necessary for you to be able to run your tests in different environments and with different settings. It also helps with quick switching between configurations. Basically, 'test lab' feeds configuration into the tests for a given test run, and acts like an adapter.
Configuration
Configuration is a vertical that contains data for every layer in TAF and is implemented by means of configuration files, code and environment variables. For security reasons we never store passwords in Git, we use secrets for storing passwords on Jenkins, and we apply similar approaches wherever possible.
Also, TAF and its configuration must support an ability to run on different components and environments. For instance, it needs to be able to run on a barebone and in a container, on a local machine and on Jenkins. It should be able to setup a valid environment for itself (Python envs for instance or installing nodejs packages).
Switching between configurations should not cost you more than changing just one value -- the code name for an environment.
Core
There is a simple reason, why the core part needs to be implemented in full. The Core is what makes TAF valuable. Without logging, db connectors, generators of test data and other tools it is hardly a labour-saving device. Only a properly implemented Core layer makes your TAF a good investment of effort.
A good Core includes at least some of these components:
- logging,
- reporting,
- saving logs, test data, screenshots and whatever is necessary,
- support of ci/cd (such as jenkinsfile or ansible),
- support of docker and docker-compose,
- support of working with clouds (reading from S3, for instance),
- working with a third-party API,
- linting if applicable,
- db connectors and queries,
- generators of test data,
- shell wrappers,
- support of running in different environments.
Estimates
In case you are intersted if it is expensive, the answer is relatively. But it pays off in the long run. Personally, I was able to implement a fully functional TAF, as it is described here, within about five months. About 80% of my working time was spent on TAF.
Additionally, it was possible to implement about 30% of test coverage at the same time. Work on TAF and on coverage was performed in parallel. Some of the coverage was done by a colleague working together with me, which proves that it is convenient to share work on test coverage across the team.
We didn't have to wait until all the TAF functionality was in place to start implementing and running tests. Test design, execution & collecting metrics on test coverage started immediately after the initial outline was defined, which proves that this kind of design allows you to add new functionality to TAF without interrupting your work on test coverage.
Conclusion
If your project is expected to last longer than 6 months, you should invest in your TAF. Otherwise the test will be nothing but a huge cost.
Good luck with your automation.
-------------------------------
Related posts:
Related posts:
No comments:
Post a Comment