Elsevier

Data ingestion project

the number of security vulnerabilities reduced from 149 to 9, serving as a solid foundation for upgrading other microservices

Project Synopsis

Elsevier, a leading provider of scientific, technical, and medical information products and services, required Yobibyte Solutions to assist with the completion of an application due for release in four months that allowed ingestion of data into another system.

The project required additional functionality, the upgrading of old libraries, new environments within the Kubernetes cluster creating and the maintainance of the build pipeline to support ongoing development.

The Challenge

To support the ongoing development and the provisioning of new environments, the Jenkins instance had to be updated using Jenkins-configuration-as-code (JCasC) which used Groovy which Yobibyte Solutions had no prior experience with.

Prior to the release, the project required the upgrade of all libraries in use, including the upgrade of Java from version 8 to 11, to address reported vulnerabilities. This included upgrading to major versions of Spring Boot, Apache Camel, and the AWS Java SDK. To ensure a smooth transition across various microservices, Yobibyte Solutions had to identify a way to gradually implement this upgrade, as all dependencies were stored centrally in a Maven bill-of-materials (BOM).

The Process

To facilitate the version upgrade, Yobibyte Solutions selected a single microservice to identify the necessary modifications and thoroughly documented the process for the development team to tackle the other microservices during an upcoming sprint.

Yobibyte Solutions created a new major version of the Maven BOM that encompassed all aspects of the upgrade which isolated the work from other developers and microservices. This included new shared libraries/components that were designed to minimise the copy/paste work required on other services by utilising convention over configuration to provide the necessary functionality.

Yobibyte Solutions had to quickly understand the Jenkins configuration-as-code written in Groovy which meant a bit of a learning curve. As part of the version upgrade work, a separate build pipeline was created for the choosen service so the work would be isolated from the rest of the team and other microservices.

The Success

Yobibyte Solutions accomplished the required work with success. The selected microservice was upgraded to the latest libraries and comprehensively documented. Consequently, the number of security vulnerabilities reduced from 149 to 9, serving as a solid foundation for upgrading other microservices.

Moreover, Yobibyte Solutions efficiently updated the Jenkins pipeline to offer all the necessary environments for ongoing development, while enhancing the Groovy code.

Tech Stack

Java
Spring Boot
Apache Camel
AWS EKS (Kubernetes)
AWS SQS
AWS S3
Helm
Docker
Terraform
JUnit
Mockito
Jenkins JCasC