The layers
The majority of test frameworks solve the same problem, so it is only natural that even on different projects you end up using the same architecture. The variable part is limited to:-- tests (they depend on requirements);
-- support of the project-specific infrastructure (you may need to implement support for AWS when assigned to project A, and it will not be required on project B, because the stack is completely different).
Balancing team effort
Personally, I developed a habit of thinking of TAF architecture in terms of layers:
-- a layer of test;
-- a layer of helpers (mainly boilerplate, moved out of tests to avoid duplication);
-- core layer and configuration (start up and shutdown, envrironmnet, logging, working with API of third-pary systems, etc.)
This approach has a lot of advantages, because it allows to share workload across the team without stepping on each other toes.
For
instance, the layer of test can be given to a manual QA or to
automation juniors (and, incidentally, you do not have to implement BDD,
if you know how to design the second layer properly). The second layer
can be given to slightly more experienced people. And the third layer
can be given to a SDET or a developer. This approach allows you to
manage your costs better, and to preserve the peace and balance in the
team. It is also very useful when you need to quickly increase your test
coverage, because independent layers allow different tasks to be done
at the same time and using the same repository.
Internal dependencies
And another important thing. You need to be very careful with how the layers are related to each other. If you want to keep your TAF stable and easy to manage, you MUST make sure that all the dependencies are STRICTLY UNIDIRECTIONAL. For instance, the layer of helpers can use methods and variables from the core layer, but cannot borrow anything from the test layer. And the core layer should not borrow anything from helpers or tests. If you run into a situation where your core needs to borrow something from helpers or from tests, it means you have a serious design problem. And you need to fix it as soon as you can. Separating configuration from code goes without saying.
The importance of real-life workflows
Test automation should always stick with real-life workflows as closely as possible. Otherwise it will not be fullfilling its major purpose: being a labour-saving device. This is where your critical thinking comes in, because, as we all know, 'it is important and popular fact that things are not always what they seem'. And popular (and too often too flawed) notions may lead you astray. If you want to build a system that is convenient and effective, you need to be prepared to break the rules to the point of blasphemy.
For instance, it is believed that chaining is always a good thing. I strongly disagree. In fails rather badly if you use it with test layer. It is only good if you need to read the code, not if you work with it constantly. Because, in reality, test workflows are never limited to just reading. In fact, they also include:
- design (when test is being written);
- regular usage (execution);
- maintenance (TAF updates, test updates, technical debt);
- troubleshooting (bugs of application, bugs of TAF, miscelaneous issues) and
- defect retest.
It is important to realize that test layer is not your regular piece of sofware. It is more like an interface a user interacts with. And as such it should support this requirement. Practice proves that chained code may be rather inconvenient to modify. And you will have to do a lot of extra work just to check something or fix an internal defect quicky. Convenient systems should look for a balance between how easy it is to read and how cheap it is to use and maitain. It is important to always keep in mind that if you simplify it in one place you are likely to make it overly complicated in another.
Cognitive effort and the number of clicks
There is one kind of a cost that has been overlooked for too long a time when it comes to TAFs. It is the cost of a cognitive effort.
Have you ever noticed that researching an unfamiliar subject hurts more then a routine job? Or maybe you hate routine jobs because they are not exciting enough and you are looking for a position in R&D to have more fun on a daily basis? That's because of the difference in cognitive effort you need to make. And not unlike its physical brother, it is not ok if it is not enough, and it is not ok if it is too much either.
And it is a complete waste to spend it on doing extra clicking or decision making. If you sense that this colorful and alledgedly good design is inconvenient, it is your brain is signalling that the UX is bad. Same goes for TAFs. If you want your TAF to be a labour-saving device, you need to design it in such a way that the end user (a manual tester, a junior or a developer) would need to perform as little number of clicks and decisions as possible when working with their respective layers. And it may be a tricky business, since project time budget is usually too tight to allow creating a real UI for a TAF.
But if you are able to grasp and nail all the above, you will be able to build a good tool for your team.
Internal dependencies
And another important thing. You need to be very careful with how the layers are related to each other. If you want to keep your TAF stable and easy to manage, you MUST make sure that all the dependencies are STRICTLY UNIDIRECTIONAL. For instance, the layer of helpers can use methods and variables from the core layer, but cannot borrow anything from the test layer. And the core layer should not borrow anything from helpers or tests. If you run into a situation where your core needs to borrow something from helpers or from tests, it means you have a serious design problem. And you need to fix it as soon as you can. Separating configuration from code goes without saying.
The importance of real-life workflows
Test automation should always stick with real-life workflows as closely as possible. Otherwise it will not be fullfilling its major purpose: being a labour-saving device. This is where your critical thinking comes in, because, as we all know, 'it is important and popular fact that things are not always what they seem'. And popular (and too often too flawed) notions may lead you astray. If you want to build a system that is convenient and effective, you need to be prepared to break the rules to the point of blasphemy.
For instance, it is believed that chaining is always a good thing. I strongly disagree. In fails rather badly if you use it with test layer. It is only good if you need to read the code, not if you work with it constantly. Because, in reality, test workflows are never limited to just reading. In fact, they also include:
- design (when test is being written);
- regular usage (execution);
- maintenance (TAF updates, test updates, technical debt);
- troubleshooting (bugs of application, bugs of TAF, miscelaneous issues) and
- defect retest.
It is important to realize that test layer is not your regular piece of sofware. It is more like an interface a user interacts with. And as such it should support this requirement. Practice proves that chained code may be rather inconvenient to modify. And you will have to do a lot of extra work just to check something or fix an internal defect quicky. Convenient systems should look for a balance between how easy it is to read and how cheap it is to use and maitain. It is important to always keep in mind that if you simplify it in one place you are likely to make it overly complicated in another.
Cognitive effort and the number of clicks
There is one kind of a cost that has been overlooked for too long a time when it comes to TAFs. It is the cost of a cognitive effort.
Have you ever noticed that researching an unfamiliar subject hurts more then a routine job? Or maybe you hate routine jobs because they are not exciting enough and you are looking for a position in R&D to have more fun on a daily basis? That's because of the difference in cognitive effort you need to make. And not unlike its physical brother, it is not ok if it is not enough, and it is not ok if it is too much either.
And it is a complete waste to spend it on doing extra clicking or decision making. If you sense that this colorful and alledgedly good design is inconvenient, it is your brain is signalling that the UX is bad. Same goes for TAFs. If you want your TAF to be a labour-saving device, you need to design it in such a way that the end user (a manual tester, a junior or a developer) would need to perform as little number of clicks and decisions as possible when working with their respective layers. And it may be a tricky business, since project time budget is usually too tight to allow creating a real UI for a TAF.
But if you are able to grasp and nail all the above, you will be able to build a good tool for your team.
-------------------------------
Related posts:
Related posts:
No comments:
Post a Comment