Digitalization and Industry 4.0 entail new requirements for processes and software systems in all company divisions and business areas. Companies that outsource the development of their software or purchase it from third parties face an additional challenge. Considering the interconnected work in the companies’ business operation, the different systems of various manufacturers are required to exchange ever more data. Despite the tests by the internal and external development teams who validate the software in various development-related levels of testing before handing it over to the client, and despite the subsequent approval by way of departmental testing, errors occur when the individual components interact. A test center with a focus on comprehensive integration tests could possibly solve this problem, but it has to meet specific requirements to be successful.
Critical errors that become evident only in live operation mean negative publicity both for the product and the companies involved. To prevent this, testing is a fundamental, integral part of modern software development. Only a sufficient number of tests and prompt feedback of the results allow for the quality and maturity of the product to be appropriately documented and confirmed. In the course of large software development projects, the number of new and/or upgraded functions is often in the hundreds. Development teams use component, integration and system tests to test the software before it is handed over to the client. The department approves the delivered software by way of the acceptance test (see Figure 1: Test pyramid).
Companies have several information systems for different tasks such as logistics, accounting, sales, etc., all of which were built using a wide variety of technologies. These information systems are already exchanging data today. The requirements of the digitalization and Industry 4.0 amplify these effects: New requirements such as increased networking throughout the entire value chain lead to a higher number or extension of the existing interfaces between the information systems. Thus, the overall system becomes more and more complex, as does the life cycle of the software: Dependencies have to be taken into account from the identification of the requirements through to the testing.
The effort required for the integrative tests increases enormously, in particular for companies that have their software developed by various service providers. In most constellations, the software systems are developed by several third-party software manufacturers and/or possibly an in-house IT organization. The providers themselves perform more or less in-depth component, integration and system tests, and verify the quality for the individual information system they create. The departments are now responsible for testing the interconnected information systems as they interact (see Figure 2: Challenges in testing due to digitalization and Industry 4.0).
The worst problems, or errors with a high risk, occur in the interaction of the information systems. However, most companies fail to perform the necessary comprehensive, integrative tests, or the testing done is insufficient, resulting in an insufficient quality statement and errors in live operation. This has various reasons. A comprehensive test at the development level is impossible due to the organizational and geographical separation of the service providers involved, and performing the necessary tests for each release is impossible for the expert users or testers from the department because that is too costly in terms of time and resources. Furthermore, the employees tasked with these tests lack the experience and the know-how necessary for optimal test planning and covering all the requirements for integration tests. The physical distance between the respective testers in the various departments further impede consultations and knowledge-building.
For the company to be successful, it is therefore becoming ever more important to outsource the necessary tests to dedicated testers, significantly increasing both the degree of coverage achieved in testing and the frequency of testing (regression). A possible solution is a test center that oversees the comprehensive integration test that takes place after the tests of the service providers and before the acceptance test of the department (see Figure 3: Comprehensive integration test by a test center). The test center verifies that the information systems interact correctly, and the department ultimately focuses on the approval of the requirements it specified.
A test team or test center of dedicated and trained testers has several advantages:
- The quality of the information systems is the dedicated test team’s primary objective.
- The test results are collected and communicated to the parties involved in an objective manner.
- There is a test manager who focuses on quality issues and who is responsible for the management of the test group.
- The test manager coordinates with the technical and development departments, determines the requirements to be tested, coordinates the test team, integrates the testers from the departments, communicates with the project management, and documents the results in test reports.
However, there are also disadvantages to an in-house test team or test center: Longer release cycles or delays in the provision of the software to be tested can cause the workload to fluctuate. The in-house test team or test center continuously generates costs, but does not always have enough work. On the other hand, in the case of peaks in the testing work, the team may not, or only with great difficulty, be able to cover them.
Companies that already use service providers for the development of their software can also call on an external provider offering integration testing as a service for the test center. Using a test center does not merely mean outsourcing the testing. A test center based on a test service agreement is a solution where the responsibilities, duties and settlement terms are customized for the individual client.
The third-party test team or test center is as independent as possible from the software development, highly specialized, and due to the nature of the service, adaptable to the client’s testing requirements. This resolves the above-mentioned disadvantages of an in-house test team or test center, and allows the company to focus on its processes.
In order for a test center to be able to optimally respond to the client’s wishes, certain prerequisites need to be fulfilled. The test center must not be a detached organizational unit, but has to establish open channels of communication and information with all the parties involved. Proximity to the client is of particular importance. Based on our experience, the test team should preferably be located on the client’s premises or at a distance of no more than 5 to 10 minutes on foot. This ensures knowledge transfer and target-oriented coordination with the departments.
The service managers and test managers are responsible for coordinating with the client and/or the department. The service manager agrees the planning of the test services with the client. This includes defining the content of the test services and the responsibilities of the test center. As every client has different requirements and processes, the assumption of the testing requires individual coordination with each client, and an individual transition. If the transition, and thus the assumption of the testing, has been successful, the test manager agrees the testing period, the operative test content and the test cases with the department and/or the client’s test coordination for each test release. But the communication is not limited to these two roles. The test experts in the test center and the department need to be in immediate, close contact in order to create, adapt and review the test cases in the best possible way, and to coordinate when deviations are discovered.
The result of the tests largely depends on the know-how of the testers, which has to comprise at least three aspects: Firstly, the technical know-how regarding the applications to be tested, and secondly, comprehensive knowledge of the testing methods. This ensures that optimal coverage of the requirements is achieved, both in technical terms and with respect to the definition of test cases. Thirdly, the testers also need to know the way the developers work. This enables them to better identify and analyze errors and communicate them to the software developers in the best possible way.
On the other side, the test center has to exchange information with the third-party providers and the development. The objective of such coordination is, for example, comparing the content of the tests already done to the downstream integration tests in order to identify any gaps in the testing or redundancies, if any. Furthermore, the delivery of new software releases to the test systems is planned, and the analysis and follow-up of deviations are discussed with the client.
In addition to the planning and execution of the test activities of the comprehensive integration test, the test center can also take over the technical support of the testing in the company. This includes, for example, the development and maintenance of the test infrastructure and test environments, and the development of comprehensive test data management. It is important that all the software systems to be tested are installed in the integration test environments, ensuring that the entire business process can be comprehensively tested.
An additional aspect of the test center is the continuous optimization of the test processes. This not only includes the optimization of the operations that have already been established, but also the introduction and operation of test automation, the dissolution of current interdependencies within and between the test stages, and the review of the level of development of the software developers at an early stage by way of so-called pre-integration tests.
For this purpose, additional test environments besides the test environments for the comprehensive integration tests are created. The service providers provide pre-release versions of the software in the pre-integration environments, giving the test center’s pre-integration team the opportunity to perform tests with interaction with the other applications at an early stage. Thus, the pre-integration tests help to identify possible deviations between the different information systems of the various providers more quickly.
For companies that have their software developed by various providers and still have complex, interconnected system environments, an external test center offers a quick and in-depth quality statement regarding all the software systems. The objective of the external test center is the establishment of an integrative test process that includes not only the interconnection of the test systems, but an interconnected test organization and interconnected test processes as well. This way, the test center responds to the companies’ requirements regarding a more integrative test focus and flexibility through scalability, communication, and concentrated testing expertise.
This post was written by:
