Hugo Masclet
Software Engineer
ABOUT

Automatically build and deploy latest near core on a kubernetes cluster

August 15, 2020

Introduction

This article will explain how Gielve is automating its NEAR Validator node with GitHub Actions, Docker and Kubernetes. You can find the repo with the sources here: Gielve/NearDeploy

Prerequisites

You must be familiar with

  • NEAR ecosystem
  • Docker and Kubernetes

You must have access to a Docker registry (Docker Hub…)

Building Docker image

To run the near software in Kubernetes, we first need to create a custom Docker image which we will later use in Kubernetes. We want this image to be “as ready as possible”, thus, we don’t want to compile the latest nearcore version in the Docker image itself. Otherwise, everytime the docker image starts, it will have to compile again the source code of nearcore.

GitHub Actions

The build process will be handled in our CI/CD provider. There are lot of CI/CD providers but in this tutorial, we will use Github Actions because it has a nice Free Tier plan. I usually like to work with CircleCI but the build process of nearcore is very long and the free tier won’t be enough for our needs. A first draft of CircleCI configuration is available in the .circleci/ folder.

Compiling latest nearcore

You can find the GitHub Actions code here. The steps of the CI are quite straightforward:

  1. Define a matrix to perform one job per release.
  2. Find latest tag for the current release.
  3. Check if image already exists in our docker registry. If image does not exist, send a Slack message and continue.
  4. Clone the repository. We fetch with --depth 1 parameter because we don’t need a full copy of the repo.
  5. Install dev dependencies and rust. You can also refer to nearcore/Dockerfile to check the packages used by the main nearcore repo.
  6. Build nearcore. This is the longest step.
  7. Build and publish the Docker image (see next section)

Build Dockerfile

Once nearcore has been built in the CI, we can now copy it into our Dockerfile. That way, the Dockerfile won’t have to handle the compilation process at each start.

  • The first lines allow us to define custom environment variables which we can easily update later in the deployment process.
  • Copy the build nearcore into the docker image
  • Install nearup
  • Expose ports and start nearcore 💁‍♂️

This image has everything needed to run nearcore. It has been pushed in our private Docker registry and will be used in the next deployment step.

Deploy on a Kubernetes cluster

Once we have our custom Docker image, we can now deploy it in our Kubernetes cluster.

The configuration file is located in the kubernetes/ folder.

The first half of the YAML (before the ---) defines a Service. It is used to expose the ports outside of the cluster.

The second half defines a Deployment.

Finally, apply this YAML file with:

kubectl apply -f path-to-yaml.yml

References