Automated Testing is an important cornerstone of Continuous Integration/Continuous Delivery (CI/CD). Codelab implemented a solution for an industrial client based on automated testing, agile development and CI/CD in order to significantly reduce the time to market for their product.
In smaller, less complex projects, the time to market is relatively short. Industrial projects with many functionalities and connections are a different story. The complexity in testing advanced projects is growing rapidly as each functionality requires additional test scenarios.
One of our industrial customers was facing this exact problem. The product required about three months of manual system tests before the product could be released and distributed to the market. Needless to say, our customer wanted to drastically improve this situation and reduce the time needed for testing.
Our solution was to change the methodology by implementing both an agile framework and a CI/CD environment, which enabled the customer to perform quality checks and execute automated unit, module and system tests. This project was based on a bare-metal embedded system controlling motors using analog signals, the CANopen protocol and industrial ethernet protocols (EthernetIP, EtherCAT, Profinet).
Previously, our client’s model development was setup as shown in the following graphic:
In order to streamline this process and become more efficient the V‑model product-development process was used:
Our first step was to switch to agile project management, conducting 3‑week sprints provided a significant increase in efficiency. Jira was used for project management and for the requirements description. The project content and the documentation was stored in Confluence.
The implementation of CI/CD was based on the Jenkins automation server and Gitlab as a code repository. Jenkins offered the appropriate plugins needed for working with the embedded code:
- Jobs for the development build (after each push)
- Night tests (all from the selected branch)
The code build was changed to Cmake for increased portability and streamlining the project building process. The compilation takes place in two stages: the libraries are compiled first, followed by the compilation of the source files. The implementation of the unit tests (Gtest, Gmock) as well as static analysis using SonarQube was performed after each push. The implementation of VTestStudio system tests (VT System, CANoe and vTestStudio) facilitated manual on request tests, night system tests and automatic target code deployment and test runs. The results of the testing were integrated into Jira using JIRA Xray and the final package was dropped on Artifactory (Jfrog).
Increasing efficiency with CI/CD and test automation customized for industrial requirements
By switching to agile development and implementing CI/CD and automated testing, we managed to ensure that each new functionality is covered with unit tests and automatically performed system tests.
Additionally, the entire release process was shortened to a few days by covering the previous functionalities with automatic tests. A few functionalities still required manual tests, otherwise the timeframe could have been reduced even further.
Our goal is to fully automate all the tests the client had to perform manually after each software release. Using CI/CD it is possible to check each change on an ongoing basis in the form of regression (incremental) tests. The automatic tests deliver a higher quality of test results and are more easily repeatable, thus significantly improving the development time and accelerating the implementation of tasks, which furthermore reduces the manual testing workload of the engineers. Our solution can also be integrated in an industrial network using various other communication protocols (Ethernet IP/Profinet/EtherCAT) or operate independently.