Automating Software Quality Certification on eBay
The eBay Application Platform team has started using Kubernetes Operators, Helm Charts, and Jenkins to ensure software quality across the organization. In order to make changes safely within containers and associated environments, the team created a self-service “certification” solution.
Certification is required for changes such as framework versions, runtime updates, operating system updates, etc. To avoid a bad version, the certification solution provides an automated model for handling complex certification requests. Zhong Shen, Rocky sheng, and Ramit Bedi, members of eBay’s application platform team, provided a detailed overview of the solution’s architecture.
The solution consists of stand-alone “certification units” made up of application test cases, performance tests, test applications, pipelines and traffic mirrors. For a given certification category, there are Helm Charts containing models for certification units and certification bodies.
Helm Chart and user input parameters are used to create certification instances and chart custom resource definition (CRD) instances. Certification instances specify the Jenkins pipeline git repository and pipeline settings. Chart CRD instances define standard parameters, certification unit groups, and respective dependencies.
Using the Certification Service, the certification controller manages Jenkins pipeline jobs. Depending on the requirement, Jenkins jobs are created and deleted. The Application Platform team reuses modularized Jenkins pipeline scripts based on individual certification type.
The App Platform team appears to be tracking the creation, maintenance, and delivery of an organized platform experience for all communities using it on eBay. This team performed full automation of software quality certification and automated test comparison with the certification solution. Team members can trigger certification tasks, and configurable notification mechanisms allow for quick troubleshooting. As a result, the certification time is reduced from a few days to a few hours.
Platform teams provide an easy-to-use “paved road” by reducing the cognitive load on the development team and allowing the common platform to become the path of least resistance. In keeping with this mindset, eBay’s Application Platform team is working on future improvements to software quality certification. The team plans to use Tekton, an open source framework for building CI / CD systems, and leverage machine learning to categorize failures, analyze root causes, and predict future failures.