Auto­mated Test­ing is an import­ant corner­stone of Con­tinu­ous Integration/Continuous Deliv­ery (CI/CD). Codelab imple­men­ted a solu­tion for an indus­tri­al cli­ent based on auto­mated test­ing, agile devel­op­ment and CI/CD in order to sig­ni­fic­antly reduce the time to mar­ket for their product.

In smal­ler, less com­plex pro­jects, the time to mar­ket is rel­at­ively short. Indus­tri­al pro­jects with many func­tion­al­it­ies and con­nec­tions are a dif­fer­ent story. The com­plex­ity in test­ing advanced pro­jects is grow­ing rap­idly as each func­tion­al­ity requires addi­tion­al test scen­ari­os.

One of our indus­tri­al cus­tom­ers was facing this exact prob­lem. The product required about three months of manu­al sys­tem tests before the product could be released and dis­trib­uted to the mar­ket. Need­less to say, our cus­tom­er wanted to drastic­ally improve this situ­ation and reduce the time needed for testing.

Our solu­tion was to change the meth­od­o­logy by imple­ment­ing both an agile frame­work and a CI/CD envir­on­ment, which enabled the cus­tom­er to per­form qual­ity checks and execute auto­mated unit, mod­ule and sys­tem tests. This pro­ject was based on a bare-met­al embed­ded sys­tem con­trolling motors using ana­log sig­nals, the CAN­open pro­tocol and indus­tri­al eth­er­net pro­to­cols (Eth­er­ne­tIP, Eth­er­CAT, Profinet).

Pre­vi­ously, our client’s mod­el devel­op­ment was setup as shown in the fol­low­ing graphic:

In order to stream­line this pro­cess and become more effi­cient the V‑model product-devel­op­ment pro­cess was used:

Our first step was to switch to agile pro­ject man­age­ment, con­duct­ing 3‑week sprints provided a sig­ni­fic­ant increase in effi­ciency. Jira was used for pro­ject man­age­ment and for the require­ments descrip­tion. The pro­ject con­tent and the doc­u­ment­a­tion was stored in Confluence.

The imple­ment­a­tion of CI/CD was based on the Jen­kins auto­ma­tion serv­er and Git­lab as a code repos­it­ory. Jen­kins offered the appro­pri­ate plu­gins needed for work­ing with the embed­ded code:

  • Jobs for the devel­op­ment build (after each push) 
  • Night tests (all from the selec­ted branch)

The code build was changed to Cmake for increased port­ab­il­ity and stream­lin­ing the pro­ject build­ing pro­cess. The com­pil­a­tion takes place in two stages: the lib­rar­ies are com­piled first, fol­lowed by the com­pil­a­tion of the source files. The imple­ment­a­tion of the unit tests (Gtest, Gmock) as well as stat­ic ana­lys­is using Son­ar­Qube was per­formed after each push. The imple­ment­a­tion of VTest­Stu­dio sys­tem tests (VT Sys­tem, CANoe and vTest­Stu­dio) facil­it­ated manu­al on request tests, night sys­tem tests and auto­mat­ic tar­get code deploy­ment and test runs. The res­ults of the test­ing were integ­rated into Jira using JIRA Xray and the final pack­age was dropped on Arti­fact­ory (Jfrog).

Increas­ing effi­ciency with CI/CD and test auto­ma­tion cus­tom­ized for indus­tri­al requirements

By switch­ing to agile devel­op­ment and imple­ment­ing CI/CD and auto­mated test­ing, we man­aged to ensure that each new func­tion­al­ity is covered with unit tests and auto­mat­ic­ally per­formed sys­tem tests.

Addi­tion­ally, the entire release pro­cess was shortened to a few days by cov­er­ing the pre­vi­ous func­tion­al­it­ies with auto­mat­ic tests. A few func­tion­al­it­ies still required manu­al tests, oth­er­wise the time­frame could have been reduced even fur­ther.

Our goal is to fully auto­mate all the tests the cli­ent had to per­form manu­ally after each soft­ware release. Using CI/CD it is pos­sible to check each change on an ongo­ing basis in the form of regres­sion (incre­ment­al) tests. The auto­mat­ic tests deliv­er a high­er qual­ity of test res­ults and are more eas­ily repeat­able, thus sig­ni­fic­antly improv­ing the devel­op­ment time and accel­er­at­ing the imple­ment­a­tion of tasks, which fur­ther­more reduces the manu­al test­ing work­load of the engin­eers. Our solu­tion can also be integ­rated in an indus­tri­al net­work using vari­ous oth­er com­mu­nic­a­tion pro­to­cols (Eth­er­net IP/Profinet/EtherCAT) or oper­ate independently.