Auto­ma­ted Testing is an impor­tant cor­ner­sto­ne of Con­ti­nu­ous Integration/Continuous Deli­ve­ry (CI/CD). Code­lab imple­men­ted a solu­tion for an indu­strial client based on auto­ma­ted testing, agi­le deve­lop­ment and CI/CD in order to signi­fi­can­tly redu­ce the time to mar­ket for the­ir product.

In smal­ler, less com­plex pro­jects, the time to mar­ket is rela­ti­ve­ly short. Indu­strial pro­jects with many func­tio­na­li­ties and con­nec­tions are a dif­fe­rent sto­ry. The com­ple­xi­ty in testing advan­ced pro­jects is gro­wing rapi­dly as each func­tio­na­li­ty requ­ires addi­tio­nal test sce­na­rios.

One of our indu­strial custo­mers was facing this exact pro­blem. The pro­duct requ­ired abo­ut three mon­ths of manu­al sys­tem tests befo­re the pro­duct could be rele­ased and distri­bu­ted to the mar­ket. Needless to say, our custo­mer wan­ted to dra­sti­cal­ly impro­ve this situ­ation and redu­ce the time needed for testing.

Our solu­tion was to chan­ge the metho­do­lo­gy by imple­men­ting both an agi­le fra­me­work and a CI/CD envi­ron­ment, which ena­bled the custo­mer to per­form quali­ty checks and exe­cu­te auto­ma­ted unit, modu­le and sys­tem tests. This pro­ject was based on a bare-metal embed­ded sys­tem con­trol­ling motors using ana­log signals, the CANo­pen pro­to­col and indu­strial ether­net pro­to­cols (Ether­ne­tIP, Ether­CAT, Profinet).

Pre­vio­usly, our client’s model deve­lop­ment was setup as shown in the fol­lo­wing graphic:

In order to stre­am­li­ne this pro­cess and beco­me more effi­cient the V‑model pro­duct-deve­lop­ment pro­cess was used:

Our first step was to switch to agi­le pro­ject mana­ge­ment, con­duc­ting 3‑week sprints pro­vi­ded a signi­fi­cant incre­ase in effi­cien­cy. Jira was used for pro­ject mana­ge­ment and for the requ­ire­ments descrip­tion. The pro­ject con­tent and the docu­men­ta­tion was sto­red in Confluence.

The imple­men­ta­tion of CI/CD was based on the Jen­kins auto­ma­tion server and Gitlab as a code repo­si­to­ry. Jen­kins offe­red the appro­pria­te plu­gins needed for wor­king with the embed­ded code:

  • Jobs for the deve­lop­ment build (after each push) 
  • Night tests (all from the selec­ted branch)

The code build was chan­ged to Cma­ke for incre­ased por­ta­bi­li­ty and stre­am­li­ning the pro­ject buil­ding pro­cess. The com­pi­la­tion takes pla­ce in two sta­ges: the libra­ries are com­pi­led first, fol­lo­wed by the com­pi­la­tion of the sour­ce files. The imple­men­ta­tion of the unit tests (Gtest, Gmock) as well as sta­tic ana­ly­sis using Sona­rQu­be was per­for­med after each push. The imple­men­ta­tion of VTe­st­Stu­dio sys­tem tests (VT Sys­tem, CANoe and vTe­st­Stu­dio) faci­li­ta­ted manu­al on requ­est tests, night sys­tem tests and auto­ma­tic tar­get code deploy­ment and test runs. The results of the testing were inte­gra­ted into Jira using JIRA Xray and the final pac­ka­ge was drop­ped on Arti­fac­to­ry (Jfrog).

Incre­asing effi­cien­cy with CI/CD and test auto­ma­tion custo­mi­zed for indu­strial requirements

By swit­ching to agi­le deve­lop­ment and imple­men­ting CI/CD and auto­ma­ted testing, we mana­ged to ensu­re that each new func­tio­na­li­ty is cove­red with unit tests and auto­ma­ti­cal­ly per­for­med sys­tem tests.

Addi­tio­nal­ly, the enti­re rele­ase pro­cess was shor­te­ned to a few days by cove­ring the pre­vio­us func­tio­na­li­ties with auto­ma­tic tests. A few func­tio­na­li­ties still requ­ired manu­al tests, other­wi­se the time­fra­me could have been redu­ced even fur­ther.

Our goal is to ful­ly auto­ma­te all the tests the client had to per­form manu­al­ly after each softwa­re rele­ase. Using CI/CD it is possi­ble to check each chan­ge on an ongo­ing basis in the form of regres­sion (incre­men­tal) tests. The auto­ma­tic tests deli­ver a higher quali­ty of test results and are more easi­ly repe­ata­ble, thus signi­fi­can­tly impro­ving the deve­lop­ment time and acce­le­ra­ting the imple­men­ta­tion of tasks, which fur­ther­mo­re redu­ces the manu­al testing wor­klo­ad of the engi­ne­ers. Our solu­tion can also be inte­gra­ted in an indu­strial network using vario­us other com­mu­ni­ca­tion pro­to­cols (Ether­net IP/Profinet/EtherCAT) or ope­ra­te independently.