For the last 3 years I’m working as a software developer at Codelab on a particular project. My responsibility was to implement a new user interface to already existing back-end application using Spring and a proprietary database which was a commercial solution for full text search. Docker is one of the main tools in this project. This article is intended to point out the issues encountered, the solutions provided with focus on how docker assisted us during this project.
Docker is a program which supports system-level virtualization. The first version of Docker was released in 2013. It got quickly adapted by many companies which noticed the enormous value of this new model. Nowadays Docker is almost everywhere and supports developer’s daily work to create better applications. ‘Unit of work’ in this environment is a container which simulates operating system functionalities, while the container is built — it can be executed on any platform and hardware infrastructure which support Docker technology e.g. Windows, Linux, Cloud, on premise servers. It is not only a great platform for development, but also for shipping and running application. It also supports faster application delivery.
Containers are created based on images that contain all dependencies which application needs to be executed e.g. to operating system as well to external components. An Image is a single package which simplifies it’s distribution. Containers are small and cheap and can be used in the whole lifecycle of the applications. The main advantage of the containers is that they simulate environments in which applications are running well. Even on different machines/servers, because of operating system level virtualization, we can be sure that our application will work as expected. This is a big step forward for supporting the concept “build once, run anywhere”. Many CI/CD environments adapt docker as their first-class citizen.
It is easy to build automated deployment processes with docker under the hood. Moreover, docker is a heart of systems for automating deployment, scaling and managing computer applications. It is also the main feature chosen by the cloud providers to allow customers deploy and scale their applications based on their needs.
In our project we have decided to use docker for CI/CD purpose. Heart of our system was a proprietary database, providing very efficient full text search of big mount of various documents, where we implement the middle layer to access the data and user friendly interface.
In order to speed up the development we wanted to create a CI/CD system which:
- Every stage can be executed independently to other stages according single responsibility principle
- We can introduce changes to the configuration of every stage without breaking and needing to adapt other stages
- Assures that application is tested by developers and QA on the same environment that it’s used on the production
- Each test can be executed independently to others and can be parallelized.
- Allow to create and execute automated integration tests which are independent from data in the DB and it’s state, structure and configuration.
We agreed to use gitlab for preparing the CI/CD environment because we already had some experience with this open-source product. We were aware of its advantages which are very active community and competitiveness in the commercial market. Furthermore, gitlab has the possibility to define all the steps how the application is built, tested, analyzed and deployed in YAML configuration file, which can be incrementally adopted to project needs. . In our case every step in the gitlab was executed within a separate docker container. All the stages were defined with appropriate docker image that needs to be used during the step execution. The CI/CD environment was configured to build new software image after every commit so that every change was automatically tested and after proper verification delivered to the development servers. The built images were additionally stored in the gitlab container registry, and were tagged with their own unique commit hash ID. This setup let us easily verify which version was running on the servers if needed. While some problems were detected it was very easy for developers to get the right version to reproduce the errors.
In our case, testing of the front-end integration with prioprietary DB solution was crucial part for the effective and high quality sotware development. We have decided to prepar our own image of the database which was used only during testing. Creating production ready database image is much more complicated and time-consuming, mainly due to multiplicity of cases that must be tested.
We used Testcontainers library that natively support docker. It is a fantastic library that can be smoothly added to the project, even by those, who would like to adapt docker features incrementally. The library allows to create a new container type designed for databases and provides functionality for configuration of the DB and for effective usage of the image in the tests. During tests execution new container with instance of the database was created and closed automatically after all tests were executed. As a next step, we configured that each single test was run separately and indecently, in parallel because each of them has it is own independent database container.
It is also easier to new developers in the project to build and run locally whole software because the majority of tasks can be automated with docker support. Last but not least, using docker images allows whole environment to migrate easily to new servers. All the definitions e.g. gitlab YAML file are automatically supported because they are connected to docker, not to host on which they are running. To sum up, docker is widely used by many companies including Codelab. It simplifies the process of building and delivering software, as well as supports applications in their lifecycle. Docker, owing to it independence from various frameworks, programming languages and libraries, is an excellent tool, especially for developers and devops. With the support of external tools such as docker swarm or kubernetes or libraries like test containers, it has been very helpful in process of creating software. Although, docker is so popular, it is still hard to use without experience. One needs to familiarize themselves with it before using it at production. Regardless of all the effort that docker requires from its users I strongly recommend this tool because it is successfully used in many of our projects.