This was the motto of the annual Hackathon of the ZEISS Digital Innovation Partners from 8 to 10 February 2019 in Munich. “Bringing Data Science to the Nano Age” was the title of the invitation to Munich, which was accepted by seven employees of Saxonia Systems AG (since 03/2020 ZEISS Digital Innovation). We were a colourful team of developers from various specialist groups, designers and software architects and were supported by an AI expert friend.
The hackathon started on Friday evening with a meetup of the Datageeks Usergroup from Munich and three exciting lectures on the topics “Word Embeddings for multilingual language processing”, “Anomaly detection with autoencoders” and “Machine Learning in the ZEISS company”.
The actual hackathon started on Saturday morning. After a short welcome, the topic was introduced and the possible tasks were presented: There were two tasks from the ZEISS Semiconductor division and one from ZEISS Microscopy, all from the field of image processing, as well as a fourth task on the subject of time series processing and CPU monitoring.
We split into two teams and chose the time lapse topic and the task for defect detection on photomasks. The clock was running from one o’clock – we had 24 hours. A total of 75 hackers, organized in 16 teams, rushed into work and sank behind their laptops. In addition to a fridge full of caffeinated soft drinks, employees who brought crisps and chocolate provided the right hacker atmosphere.
Task #1 – Time series processing
For the time series task, approximately 1.8 million data points from 235 machines were available, containing hourly temperature measurements from two CPUs. All these machines broke down at some point, so the remaining lifetime was added to the data points afterwards. The goal of the task was to enable a prediction of the remaining lifetime from the given temperature values to be able to warn the user and to arrange for appropriate maintenance work in time. Furthermore, a special focus was to be placed on the UX design and a user-friendly presentation of the prediction results.
So let’s get to work and visualize the data first. Once we had a clear picture of the problem and the goal of the analysis, we chose LSTM (Long Short-Term Memory) as the neural network to solve the problem. Due to their “storage capability”, these are ideally suited for time series processing.
At the same time, we developed a monitoring interface that displays the temperature curves in a clear diagram and warns the user in case of abnormalities. A red line and a warning message should signal the user the need for action.
Task #2 – Defect detection
The first subtask of our second team was to classify microscope images and to identify elliptical features on the images. These were defects on photomasks from semiconductor production. For this purpose, a total of 111 images were available in the three categories “defect completely visible”, “defect partially visible in the image” and “no defect”. The second work step, the segmentation, had the goal to mark the defects in the image in colour and as a bonus task a matching ellipse should be drawn around the defect.
Classification
Due to the small data set, this was first augmented by rotating and mirroring the images. Since the first step was a classic image classification task, we decided to use a ResNet 50 and trained this model on a local GPU to use the exported model for inference in our cloud solution.
At the same time, we developed a web interface in this team, where the microscope images could be uploaded and the classification result for the first subtask could be viewed.
The result of our training was a 100% accuracy in the classification of the validation data. This is a fantastic result, but with such a small amount of data it doesn’t say much at first. As so often in the use of neural networks, more data would have been an advantage here.
Segmentation
For the second subtask we decided to label the training data independently. For this purpose, we marked the data with the central points as well as the long and short axis of the ellipses, so that this additional information allows the training of a second net. Unfortunately, we ran out of time right here and we did not manage to complete the second task.
In both teams we used the Python framework Keras for the development and training of the machine learning algorithms. This enables a comparatively fast and intuitive creation of the desired network architectures and their export into common formats.
Combination of the components
In addition to the technical solution, the jury’s assessment criteria included the business value and UI design of our solutions. So, we also had to think about the deployment of the machine learning models, because a console application is known to sell badly. In both teams we decided to develop an Angular web application that communicates with the model in the cloud via an HTTP request and displays the result clearly. As one of the sponsors of the event, Microsoft provided each team of the Hackathon with an Azure Cloud access and a budget of 170 €. In addition, a team of employees provided us with advice and support.
For the applications in our first team, we used the Azure Machine Learning service to host the machine learning models on the one hand and to enable communication with the web application on the other. To host the model, it must first be registered in the Machine Learning Service workspace and then packaged in a docker image that contains all the necessary dependencies to run the model. This image can then be made available through an Azure Container instance to act as a web service. However, since Azure does not send CORS headers in this service and these cannot be configured, it was not possible to address this web service with a simple HTTP request from our Angular application. This communication required a detour via Azure Functions with a separate HTTP endpoint.
Unfortunately, our second team, which was working on defect detection on photomasks, did not succeed in making a breakthrough until a few minutes after the end of the processing time, which made it possible to send an image to the HTTP endpoint from a web application and receive a classification result as a response. Unfortunately, we were not able to officially present the overall solution, but the success still helped our inner well-being and we learned a lot about the deployment of machine learning models.
Result presentation
The hackathon ended on Sunday afternoon with the team presentations. In a pitch, each team had three minutes to convince the five-member jury of their ideas and their implementation.
It was extremely interesting to see the colourful range of possible solutions. From a sophisticated business concept or a particularly elegant UI to highly sophisticated technical solutions, everything was there. Unfortunately, our two teams came away empty-handed at the awards ceremony.
Our return luggage wasn’t just filled with some promotional gifts and some lack of sleep, but above all valuable experience in the development of machine learning solutions for time series and image processing, in the implementation of machine learning in the Azure Cloud and in the development of an interface between web application and Azure machine learning services. We got an insight into what it means to prepare and label data, were able to refresh our CSS skills and had a lot of fun in teamwork.
We would like to thank the organiser ZEISS Digital Innovation Partners for the successful organisation, the conception of the tasks, the support and the excellent catering. It was a great pleasure for us.
Imagery: © ZEISS Hackathon Munich 2019
This post was written by:
