The Test Analyst – Just a Reviewer for Automated Test Cases? A Field Report from an Agile Environment

Introduction

In an agile environment, regression testing helps to maintain a high level of quality. With each user story, newly developed functions are added, while the old functions still have to work. By the tenth sprint, the effort involved in the regression is so high that it is impossible to test everything manually. The only solution is test automation.

If a project is built from scratch, it is possible to properly integrate the test automation from the start. At the same time, the tester often feels like a lone warrior facing several developers. Now then, how can we realize the time-consuming automation of the functions in the daily work of a test analyst?

Project environment

In our project, we are creating new software in a JavaScript environment. It is implemented by means of the Electron framework. Consequently, Spectron is the preferred tool for the automation of test cases. Jira is used for the project platform, and the project is implemented based on the scrum model. The project team (based on FTE) consists of:

  • 6 developers, incl. 1 architect
  • 1 scrum master
  • 1 business analyst
  • 1.5 testers

Concept

It was obvious from the project kick-off that the testers would not be able to do the test automation. Therefore, the team came up with the following solution:

  • the test automation is done by the developers
  • the review for the test cases is done by the testers
  • the creation and approval of the Spectron test cases are codified in the Definition of Done

Advantages

  • Time saved in testing: The real reason for this procedure is the scarcity of resources on the testers’ part. If they had had to take on the automation as well, the entire project would not have been possible.
  • Change of perspective in testing: The testers can learn quite a lot in the discussion and review. For example, the implementation becomes clearer when questions are asked about why a test has been written this way. This can result in test cases that would otherwise not have been thought of.
  • Development of know-how: Since writing tests while the development is underway is routine work for programmers, the basic understanding regarding the creation of automated tests is generally very good. In our project, this has already proven useful for several reasons:
    • Parts of the applications could be covered using technical tricks that a tester would not have been readily able to provide, e.g. automated testing of the correct representation of a point cloud in a chart and the display of the details of a selected point.
    • Technical refinements enabled us to significantly improve the performance and stability of the Spectron tests.
    • After changing the tools used, the duration of a complete Spectron run was reduced by half an hour (time savings of 25%)
  • Change of perspective in development: Due to the fact that the developers concerned themselves with the user’s perspective regarding the functions and the interface of the software, a large number of errors could be avoided, and the basic understanding increased due to the intense interaction with the testers.

Disadvantages

  • More time required for the developers: The time saved in one place is needed elsewhere. However, the work can be shouldered by several people in this case.
  • Structure: Developers classify test cases into logical areas from a technical point of view. As they are not always identical to the functional logic, testers may have trouble finding and reviewing specific test cases.

Challenges and solutions

  • Traceability DEV QA: The review is done in Git (diff tool) in the project. In the project, the test team reviewed modified and newly created Spectron test cases, but not deleted ones, based on the assumption that those had been replaced in the course of the modification. Consequently, some requirements were no longer covered.
    Solution: To solve the problems with the review, training everybody who has to work in and review using Git on how to use Git is particularly helpful. Doing a walkthrough with the test team and the development team in the case of major modifications is useful as well to allow the testers to better understand the implementation by the developers.
Figure 1: Example of a Git review
  • Traceability of Spectron requirements: This is a challenge that was specific to our project environment. The agile team uses Jira for the requirements and test management, but in the client’s environment, the requirements and test cases have to be specified by means of different requirements management software for legal reasons. Since the systems do not know each other, automatic traceability cannot be ensured.
    Solution: To overcome this obstacle, we established a direct allocation of the Req ID to the Spectron clusters.
Figure 2: Example of direct allocation

Conclusion

In conclusion, we can say that our concept of only having the test analyst review automated test cases instead of writing them proved to be effective in our project. The division of tasks between the testers and the developers fits very well with the agile method (scrum). The advantages far outweigh the disadvantages.

This approach is perfect for an agile project that is being built from scratch with a small staff and a high standard of quality. However, you should use this method from the start. Integrating it at a later time is almost impossible because gradually expanding the test cases after each user story is much more manageable and easier than creating the test cases en bloc. Furthermore, decisions regarding the implementation, structure, architecture, and most importantly, the processes (Definition of Done, …) are made at the beginning.

WCF Alternatives (Part 4) – Summary

In the last blog post of the series on alternatives for the Windows Communication Foundation (WCF), we are going to recapitulate and compare them once again. 

Decision matrix 

CharacteristicWeb APIgRPC
Bidirectional communication Possible with restrictions 
Has to be realized with SignalR. 		Possible with restrictions 
Only stream response is supported. 
Scope of changes to existing codeSmall 
Due to the adoption of the service methods into the controllers and the generation of the client, few adjustments are required. 		Large 
Mapping is required because specific data types are prescribed in some instances: An input parameter and a return value are required.
Necessary prior knowledge Technology-specific 
Web API knowledge about the use of controllers and actions, HTTP verbs 		Technology-specific 
Particularities of gRPC, creation of the *.proto file 
Platform independence Yes 
When you use .NET Core, client and server can run on different platforms. 		Yes 
When you use .NET Core, client and server can run on different platforms. 
InteroperabilityYes 
Client and server can be created using different programming languages. 		Yes 
Client and server can be created using different programming languages. 
Browser support YesOptional 
Currently possible only with third-party libraries 
Self-describing interfaces Optional 
OpenAPI is possible by integrating third-party libraries. 
No 
You have to create the *.proto file for the description of the interface yourself. 
Payload sizeHigher 
JSON, human-readable 		Lower 
Binary format 
SpeedLowerHigher
SustainabilityYes 
Microsoft currently recommends Web API as a sustainable alternative. 		Yes 
Microsoft currently recommends gRPC as a sustainable alternative. 
DebuggingEasily possible Possible with restrictions 
The transferred data cannot be examined due to the compression. 

Advantages and disadvantages 

Web API

gRPC

Advantages

  • Transmission data are readable 
  • Fewer code adjustments required for the migration 
  • More flexible design of the endpoints and calls with respect to input parameters and return values 
  • Faster transmission 
  • Typed by means of Protocol Buffers interface description 
  • Simple generation of client classes 

Disadvantages

  • Slower transmission compared to gRPC 
  • Generation of client classes only by means of third-party libraries 
  • Typing of the interface not possible 
  • Transmission data not readable 
  • Mapping code required because no standard .NET types are used 
  • Greater effort involved in the migration due to more necessary code adjustments 

Conclusion

In our blog post series on WCF, we have presented both ASP.NET Core Web API and gRPC. We have seen that both options have advantages and disadvantages. 

With Web API, the interfaces can easily be used by anybody due to the content-first approach and the use of HTTP. The transmitted data are accessible and can be read at any time. 

gRPC abstracts the interface calls using the contract-first approach, making them faster and very easy to address by developers. However, the transmitted data cannot be accessed. 

In principle, migration to both options is possible, and both are recommended by Microsoft. Still, we cannot definitively recommend one of the alternatives. The decision should always be project-specific and based on various criteria such as the project scope, experience with the respective technology or the existing architecture. 

Christmas party in times of Corona: The 2nd ZEISS Digital Innovation Online Campus Event

As early as spring we at ZEISS Digital Innovation (ZDI) started to plan our Christmas party which normally takes place on the Friday before the 1st Sunday of Advent. In 2020 we quickly came to realize that a traditional Christmas party where all the employees normally meet in Dresden and celebrate together would unfortunately not be possible this time. Cancelling the Christmas party was not an option for us either, so we started to search for a new format.

We came back to the idea of our 1st ZDI online campus event which had already received a lot of positive feedback. It was especially important to us this time that we created an all-encompassing festive mood, in addition to the intern knowledge sharing. But how could we bring this home to our colleagues?

A map showing all the locations from which our employees were connected.
Figure 1: Our colleagues participated in the online campus event from a wide variety of locations.

A small Christmas surprise for all employees

To enhance a Christmas atmosphere all employees received a small Christmas parcel by mail with the request that it should only be opened on the day of our common online campus event.

All 345 parcels were personally packed by the organizational team to a background of festive Christmas music. We were happy that the Christmas surprise arrived in time at the homes of all our colleagues in Germany and Hungary.

So all were on time for our 2nd ZDI online campus event with Christmas goodies, mulled wine and traditional Christmas spices to create a delicious punch, an Advent calendar, a Christmas party hat and other small gifts and off we go!

2nd online campus event including online team game and Christmas celebration

Using the positive experiences from our 1st ZDI online campus event this time again we offered different lecture slots from and for our colleagues during the whole day. The technical realization was done again via Microsoft Teams. We had 29 lectures altogether covering a variety of different topics such as Java, .NET, Cloud, Usability, Agile and Web. Through lessons learned the first time around we adapted the length of the lecture slots increasing them to 45 minutes instead of 30 with 15-minute breaks in between to change. So everyone had time to breathe and mentally prepare for the next one.

Timetable of all presentations at our 2nd ZEISS Digital Innovation online campus event.
Figure 2: A total of 29 presentations in 6 parallel slots were given on related topics.

After a longer lunch break our online team game took place, 28 teams took part, consisting of nine to ten players in each one. When we put together the teams we paid great attention to the make-up of each team ensuring that they were as heterogeneous as possible regarding the level of maturity, location and business area. Each group had their own team room for solving the tasks, it was a super way to get to know each other. The activities included among other things general knowledge questions, charades and quizzes as well as practicing the Christmas team contribution, which every team presented live at the end of our online campus event within the framework of our collective Christmas party.

We kicked off our Christmas Celebration together by drinking a toast! After that each team started to show their live Christmas contributions. There were musical and poetic presentations and performances including Christmas greetings in different languages together with Christmas film tips and recommendations. All teams were extremely creative and the team chat was also available and used actively adding another dimension to our party. So despite the wide distribution and accompanying physical distance, there was nevertheless a contemplative Christmas mood that was felt by all.  

Conclusion

We all knew that an online format and a virtual Christmas celebration could never replace the feeling of a real on-site Christmas party with ice skating, delicious food, dancing and the relaxed atmosphere amongst one other. Still, we were pleasantly surprised how much festive, community spirit could nonetheless be felt online. It has shown us once more that we are a strong team acting in unison and despite Corona we are ready to make the best of the current situation breaking new ground together.

It is uncertain if we may celebrate a summer or Christmas party with all the colleagues in Dresden on-site this year. But one thing is already for sure: our new online campus event format with team games is certain to outlast Corona.

WCF Alternatives (Part 1) – Introduction

The Windows Communication Foundation (WCF) is a communication platform for the creation of distributed applications developed by Microsoft for the .NET Framework. It was introduced with the .NET Framework 3.0 in 2006, replacing .NET Remoting. With WCF, the various aspects of distributed communications are logically separated, and different communication technologies are combined into a standardized programming interface. This makes it possible to focus on the business logic without having to deal with the connection of the different communication technologies.

The structure of WCF

The following communication technologies were unified with the release of WCF.

WCF alternatives
Figure 1: Unified communication technologies

The structure of a WCF application is based on three questions (where, how, what).

The first question, “Where”, describes the address at which the application can be found in order to communicate with it, e. g.:

  • http://localhost:81/DataInputService
  • net.tcp://localhost:82/TcpDataInputService
  • net.pipe://localhost/PipeDataInputService
  • net.msmq://localhost/MsMqDataInputService

The second question, “How”, describes which protocols and which encoding, the so-called bindings, are to be used for the communication. They are defined in the *.config of the application, allowing them to be modified at any time without the need to recreate the application. WCF supports nine different bindings:

  • Basic Binding (BasicHttpBinding)
  • TCP Binding (NetTcpBinding)
  • Peer Network Binding (NetPeerTcpBinding)
  • IPC Binding (NetNamedPipeBinding)
  • Web Service Binding (WSHttpBinding)
  • Federated WS Binding (WSFederationHttpBinding)
  • Duplex WS Binding (WSDualHttpBinding)
  • MSMQ Binding (NetMsmqBinding)
  • MSMQ Integration Binding (MsmqIntegrationBinding)  

The last question, “What”, uses various contracts to define the endpoints and data types to be provided. The endpoints are defined by ServiceContracts, and the data types by DataContracts.

Example of a WCF ServiceContract and DataContract:

[ServiceContract]
public interface IDataInputService
{
    [OperationContract]
    int CreateUser(User user);
 
    [OperationContract]
    int Login(User user);
 
    [OperationContract]
    List<Time> GetTimes(int userId);
 
    [OperationContract]
    void AddTime(Time time, int userId);
 
    [OperationContract]
    List<string> Projects();
}
 
[DataContract]
public class User
{
    [DataMember]
    public string Name { get; set; }
 
    [DataMember]
    public string Passwort { get; set; }
}
 
[DataContract]
public class Time
{
    [DataMember]
    public DateTime Start { get; set; }
 
    [DataMember]
    public DateTime End { get; set; }
 
    [DataMember]
    public string Project { get; set; }
 
    [DataMember]
    public int uId { get; set; }
 
    [DataMember]
    public int Id { get; set; }
}

WCF is very popular because of its flexibility achieved with the separation and its versatility, which is why the platform is readily used in numerous projects.

Why is a migration necessary?

When .NET Core was first announced in 2016, WCF was already no longer included. WCF is not part of the subsequent .NET Core releases, including the most recent .NET 5.0.

For Microsoft, the “W” in WCF, which stands for Windows, would probably be the greatest issue in porting. In order to do justice to .NET Core, a cross-platform solution would have to be found for this purpose. One of the problems in this context are the Windows-specific operating system libraries currently used, e.g. for Socket Layers or cryptography.

Even though the developer community is asking for the integration of WCF in .NET Core, it is unlikely that Microsoft will provide this in the foreseeable future.

The future with gRPC and Web API

To make an existing project sustainable, or generally use the advantages of .NET Core, porting to .NET Core should be the aim. This is particularly useful for projects that are being actively and continuously developed. If WCF is used in a project, this poses an additional challenge in porting. First, an alternative needs to be found, and then a preparatory transition of WCF is required. Microsoft generally recommends two alternatives, gRPC and Web API, to replace WCF.

We are going to present these two alternatives in a series of blog posts, discussing the respective particularities and challenges of the migration. More articles will follow shortly.