In the first and second part, we successfully completed the agile transition of the test manager’s responsibilities, and we proved that small scrum teams no longer require a test manager. Several scrum teams working together on the same product pose another challenge. Can the previous observations made with respect to a small scrum project with a single team also be adopted for large projects?

Do we need a test manager for large scrum projects?

In many companies that use scrum, several teams work together on the development of the same product. There have already been some thoughts regarding scaled scrum on the company level and how it is possible to comply with agile principles even as the level of complexity increases. In this context, Boris Glober identifies two problematic areas: “compliance with the scaled scrum framework” and “scaling of the requirements process”. To solve this, he introduces additional roles: the company scrum master and the company product owner, who assist all the scrum masters and product owners of the project. Both roles coordinate with their counterparts in the individual scrum teams, and they manage the scrum tools on the company level, such as the company product backlog and the company scrum board.

Can this concept be adopted for testing as well, i.e. do we need something like a company test owner or a company QA master? The purpose of this new tool would be the coordination of and responsibility for the overall, integrative testing process, establishing the necessary common test structures, and filling the backlog with stories, tasks and incidents relating to QA.

For projects with a manageable number (2 to 8) of scrum teams, this kind of coordination can be done within the framework of the general coordination meeting, the scrum of scrums. If the coordination becomes more test-specific and time-consuming, it is advisable to establish a separate test meeting as required. Each team sends a team member with the necessary test know-how to this meeting. If the number of teams is higher, or if coordination between several projects is required with respect to QA issues, the guilds can again be used for this purpose. The guilds collect examples of best practices that they provide to all the projects, or they appoint coaches who bring agile testing methods to new projects. The guild masters agree on important decisions and moderate the scrum teams wherever it is necessary to define common rules and solutions.

We can conclude that agile companies no longer need a test manager, even for large projects. However, this is possible only by way of a complete agile transition of the test manager’s responsibilities because each of the test manager’s responsibilities—in particular with respect to communication and coordination—has to be matched with an agile tool.

In the first part, we successfully completed the agile transition of the test manager’s operative responsibilities, and we proved that small scrum teams no longer require a test manager. But will this work for the test manager’s strategic responsibilities, too? And who assumes the test manager’s strategic responsibilities?

Who assumes the test manager’s strategic responsibilities in agile companies?

A test manager’s strategic area of responsibility includes quality management (QM), which, according to ISO 8402, comprises “all activities of the overall management function that determine the quality policy, objectives and responsibilities and implement them by means such as quality planning, quality control, quality assurance and quality improvement within the quality system”.

Irrespective of whether a company uses traditional or agile methods in development, the company’s management is responsible for the quality management. The scrum team cannot, should not and must not assume this responsibility. At first glance, nothing in the organization and the division of responsibilities changes. At second glance, however, a problem arises for agile companies. In the traditional world, the test manager used to act as the interface between the operative and the strategic level. They transmitted information between the management and the operative level, passed strategic specifications and methods on to the test teams, or forwarded developments and improvements to the management.

But how does information get transmitted from the strategic to the operative level and vice versa in an agile environment? The answer: The agile transition does not stop with the allocation of the test manager’s operative responsibilities to the scrum team. The solution for the agile transition of the “interface” between the two levels is similar to the agile transition of the test manager’s operative responsibilities. New concepts and tools are available for the current corporate communication tasks.

One of the new concepts is the so-called guild. Guilds (called competence teams in our company) are a tool that puts and keeps knowledge and information management on track. They are organized in a matrix structure apart from the normal company structure. The objective of the guilds is to pool the company’s know-how carriers, offer the staff a platform to exchange knowledge or carry out training programs, or agree general project decisions such as the development of test environments or rules for code quality. Depending on the company’s goals, the structure and composition of the guilds can differ: For example, guilds can be divided according to competence areas such as Java development, .NET development, test or process analysis. Or entire scrum teams can be aggregated in guilds, working together on a specific subject in the project, such as QA, database connection, GUI, or interfaces (see Figure 1).

The guilds work according to the following pattern: The primary role that each employee assumes in their daily work (e.g. developer, tester [QAlas], product owner and scrum master) is determined. With this role, they are allocated to a corresponding guild. Within the guild, the members exchange information relating to their areas of responsibility, carry out in-house training seminars, collect the available knowledge in portals, or discuss best practices in individual projects. A guild master is responsible for the moderation and coordination within the guild. In accordance with the motto “primus inter pares”, they do not have more rights than their colleagues in the guild, and they are elected from among the guild members. The guild master is the principal contact for the members of their own guild, the other guild masters and the management. This is necessary because the guilds engage in active interdisciplinary exchange, and they are also supposed to forward feedback from the operative level—such as new developments or technologies—to sales, or suggestions regarding training seminars to HR.

In the last part, we will look at what happens when several scrum teams work together on the same project and the effort required to coordinate them increases for the field of QA as well.

Some time ago, a colleague of mine asked me whether we still needed a test manager in an agile development process like scrum. My first response was no because the Agile Manifesto and the scrum framework only know three roles: product owner, development team, and scrum master. Accordingly, the scrum team—i.e. the three scrum roles mentioned above—does not provide for a test manager. But on second thought, I wondered who among the scrum team was supposed to assume the test manager’s responsibilities in and around the sprint?

Studies such as the 2014 ASQF Industry Report^[1] and the 2011 Standish Chaos Report show that agile methods have already become a permanent fixture in companies. Furthermore, the Standish Chaos Report shows that projects using agile processes are more likely to be successful than “conventional projects”.  The Agile Manifesto of Ken Schwaber and Jeff Sutherland was the basis for this development. It defines basic principles and specifications that uncover “better ways of developing software by [the parties involved in the process] doing it and helping others do it” [²].

The key principles from the Agile Manifesto are:

• Individuals and interactions are more important than processes and tools.
• Working software is more important than comprehensive documentation.
• Client collaboration is more important than the original specifications.
• Responding to change is more important than following a plan.

Companies that switch to the agile method of development have a competitive advantage compared to companies using traditional methods. However, transitioning the processes to development methods such as scrum—also called agile transition—presents a great challenge. Agility is not achieved by divvying the development milestones up into sprints and appointing a product owner (see Figure 1). Comprehensive changes in the organization are required to achieve an agile way of working and living.

The challenge of the agile transition becomes very clear when you look at the example of the test manager. The question:  If we do not need a test manager in scrum, who assumes their responsibilities? The product owner? The scrum master? The team?

According to the Scrum Alliance, the product owner is the person responsible for developing the product accurately and on time. The product owner fills and refines the product backlog, and ensures that everybody knows what it contains and what is given which priority. Consequently, they are usually closest to the “business side” of the project. Scrum requires the development team to be a group with mixed functions—pooling all the necessary skills—that are required for the development of the product. The team organizes itself, i.e. it independently chooses the content to be implemented in the sprint and takes care of the planning, management, and implementation. The scrum master is the “pilot” guiding them through the depths of the scrum framework, helping the rest of the scrum team to comply with scrum principles. Another task of the scrum master is removing obstacles that hamper the team’s progress

Even after studying the scrum roles, we do not know who assumes the test manager’s responsibilities, or how they are allocated. To answer these questions, we must first determine the test manager’s responsibilities in the traditional testing and quality assurance process. According to the International Software Testing Qualifications Board (ISTQB), the certification board for testers, the test manager’s role and responsibilities include more than just supervising the test project. They manage the testing department or the test team, and thus, the resources for the tests. They prepare reports; escalate to development, technical department and project management; assess test projects; ensure compliance with the company’s quality processes; procure the testing tools for the organization; and review the test plans and the test cases.

The responsibilities can be allocated to two fields: strategic and operative (see Figure 2). The operative level includes the planning and conceptual design of the test cases and tests, monitoring the execution of the tests, and the communication within the project. The strategic level includes the quality management tasks.

The operative responsibilities cannot be assumed by the scrum master or the product owner. The product owner does not get involved in the implementation, nor the scrum master in the development. In agile development, the testing tasks are assumed by the team. Following the definition and allocation of the test manager’s responsibilities, we can examine how they can be integrated into the agile process. However, there is a problem: It is not possible to allocate them to a specific person because within the scrum framework, all tasks are distributed across the agile team.

The solution to the problem lies within the scrum framework itself. It provides a comprehensive package of tools and artifacts that can be matched with the test manager’s responsibilities. We did a complete agile transition of the responsibilities in our scrum teams, and we found that scrum provides a tool or artifact for every one of the test manager’s responsibilities.

For example, the test strategy and the paramount quality characteristics are considered during the planning of the sprint and the backlog grooming. The pass-fail criteria, i.e. the criteria that determine whether a sprint has been successful or the test has been completed, are defined in the definition of done (see Figure 3).

In the sprint review, the implementation of the specifications is verified and validated (see Figure 4).

In addition, the stories and their representation on the scrum board and in the backlogs facilitate documentation of the progress and the assessment of the quality level (see Figure 3).

Accordingly, there is an agile tool or artifact for each of the test manager’s responsibilities. And thus, complete agile transition of the test manager’s responsibilities is achieved. Provided that the scrum team has the testing know-how required for the implementation of the upcoming development and testing tasks, small projects do not require a test manager. If the know-how is not yet given, the team has to be coached accordingly.

In the next part, we will examine who can assume the responsibilities on the strategic level, and what happens in projects where several scrum teams work on the same product together.