Real Life Requirements For A Test Framework

The layers

The majority of test frameworks solve the same problem, so it is only natural that even on different projects you end up using the same architecture. The variable part is limited to:
-- tests (they depend on requirements);
-- support of the project-specific infrastructure (you may need to implement support for AWS when assigned to project A, and it will not be required on project B, because the stack is completely different).

Balancing team effort

Personally, I developed a habit of thinking of TAF architecture in terms of layers:
-- a layer of test;
-- a layer of helpers (mainly boilerplate, moved out of tests to avoid duplication);
-- core layer and configuration (start up and shutdown, envrironmnet, logging, working with API of third-pary systems, etc.)
This approach has a lot of advantages, because it allows to share workload across the team without stepping on each other toes.

Test Framework Is A Software Product

When you intend to write a test framework you need to remember the following:

- test framework is a software product, which means everything that applies to a commercial product, also applies to a test framework;

- test framework is not less important than the software under test, it is actually more important (and sometimes more complicated) than the software under test.

Test frameworks are created to help you check if you still meet the requirements (and to do it with less effort than manual test requires). Because if you do not meet the requirements, you do not achieve the project goals and|or create more problems than you solve. For instance, you spend enormous amount of time on manual defect management and dealing with architectural issues when it is too late and too expensive.

So if you decide to create a test framework, you have to:
-- define the requirements;
-- define priorities for the requirements;
-- define stages of maturity (what is to be implemented first and what next);
-- find out how much resource you have (time, technical knowledge, congitive ability, ability to delegate);
-- build the schedule of implementation (preferably by using a Gantt diagram).

Otherwise, I am sorry to say, your test framework will be nothing but a cost.

-------------------------------

Related posts:
Real Life Requirements For A Test Framework
More On Designing TAF Layers

Requirements for Requirements

This text is not exactly mine. It is a very high-level abstract of what a book Requirements Engineering by Elizabeth Hull, Ken Jackson, Jeremy Dick, 2005 has to say on the subject of requirements for requirements.

So, how can we understand that your requirements are ok, or not ok? If they are another source of quality risks? To answer these questions, I've prepared some sets of criteria that we can use for assessment: