Skip to content

Deploy Serverless V2

Background

As part of (Overview) E2E: Have MRs run E2E during MRs. (#1212 - closed), we have integrated the execution of the E2E during the MRs pipeline, and for this initial version, we implemented a server spawning procedure through by means of creating a script that act as wrapper over the SW cli. A couple of the problem we have seen with this approach is that in case of a failure in the pipeline, there is no easy way to add hooks, meaning that a particular script or job would run. When it fails, or it is cancelled, it simply disappears. This poses a challenge for cleaning up resources of the serverless containers.

In oder to have a more solid solution, we are going to incorporate a simple webserver that will be in charge of controlling the state of the containers.

What to do

Server

  • Create repository for project, it should be in the group https://git.recoverycat.de/recoverycat.ops
  • Check if https://git.recoverycat.de/recoverycat.ops/serverless-e2e-image can be used or if better a new repo
  • Create simple Webserver in GO Lang
  • Service should offer 3 endpoints. 1 for the spawning of a serverless-container. 2 for the listing of the running containers. and 3. for the deletion of serverless containers.
  • All endpoints must be protected, an access token must be validated and use in all endpoints.
  • Spawning a new container will store its reference and timestamp in a simple sqlite3 db.
  • Deploy in kubernetes cluster using helm charts

Pipeline

  • Update current rcc-client pipeline to user server instead of script in the job serverless-create.
  • Remove serverless-delete as not needed anymore here as containers shutdown will be controlled in the server as such.

Question: the deletions of containers could take place either by an external call like a scheduled pipeline job, or should it have a background process that takes care of it.