Accelerated innovation cycle of a medical robotic device through test automation

Many medical technology companies face the challenge of making their product innovation cycles shorter and accelerating market launches. However, modern medical devices are highly complex and must meet the highest security and regulatory requirements for the quality of the software. For this reason, quality assurance during development plays a crucial role, but is also time-consuming and costly. The increased use of test automation reduces this effort considerably, but standard tools for customer-specific requirements are often missing.

We used the 'Scalable Automated Framework for Firmware Tests'. This is a test automation tool, that fully meets the customer-specific requirements of Carl Zeiss Meditec AG. The tool was developed and validated for the firmware test of robotic devices. Using test automation, time and costs for development and integration tests for the client could be significantly reduced. At the same time the level of software quality could be further increased. Conceived as a platform solution, the framework can be expanded to other components and products at low cost and can therefore be used for a broad product portfolio.

Due to the strong interdisciplinary character of developing medical devices, a deep technical understanding from different areas such as e.g. embedded engineering, software development, software architecture, testing and test automation is needed. In addition, the high regulatory requirements of medical technology had to be considered during development and documented in a meaningful way.

Our test team, which specializes in medical technology, was responsible for the complete conception and development of the test automation solution. In close cooperation with the embedded software developers, architects, and reliability managers at Carl Zeiss Meditec AG, we analyzed the customer requirements and planned and implemented the technical implementations in an agile manner. Efficient scalability and expandability of the test automation were considered right from the start. The framework was validated as a test tool in the form of IEC 62304 and ISO 13485 in cooperation with the customer. The high quality and complete documentation of the framework was ensured throughout the development phase. It can therefore be used both during development and for acceptance tests.

The 'Scalable Automated Framework for Firmware Tests' is used to verify and validate the embedded firmware of robotic surgical microscopes. For this purpose, classic black box test cases were implemented, which run in a hardware-in-the-loop test environment.

The framework communicates directly with the device under test via CAN bus and sends control signals to the device firmware or receives data and signals from the device and records them. The complex device logic was abstracted into an easy-to-use interface so that test cases can be implemented quickly and efficiently.

Two types of test cases - fully automated and semi-automated tests - were implemented based on the framework.

• Fully automated tests do not require any user interaction and can e.g. be run overnight as regression tests. This gives the developers direct feedback on their changes in a short time. For this purpose, the test framework was integrated into the customer's continuous integration environment and can thus be triggered with each update of the firmware or by a timed trigger.
• Semi-automated test cases are largely automated, but also have individual test steps that require user interaction. For this purpose, the framework was extended by a Graphical User Interface (GUI). The test GUI guides the tester through the test case and informs him about necessary manual interventions. The evaluation of the test results and the generation of reports continue to be fully automatic. The tester is thus relieved of a large part of the manual steps, which significantly reduces the complexity and duration of the test execution.

Test reports and associated data, such as e.g. recorded measurement data and logs, are automatically stored in the customer's test management tool. This ensures that the test results are archived and traceable.