To deploy BuildBuddy Remote Build Execution on-prem, we recommend using the BuildBuddy Enterprise Helm charts.
First add the BuildBuddy Helm repository:
Finally install BuildBuddy enterprise to your Kubernetes cluster:
This will deploy a minimal BuildBuddy enterprise install to your Kubernetes cluster.
You can verify your install by waiting a minute or two for your deployment to complete, then running:
This will return an IP address that you can visit in a browser and verify that you install was successful. The basic deployment doesn't configure RBE, so you'll only see options for BES and Cache endpoints.
Now that you have a basic BuildBuddy Enterprise install deployed. Let's configure it to enable Remote Build Execution.
You can do so this by passing a
values.yaml file to Helm that enables RBE. Here's a simple
values.yaml file with RBE enabled. This will deploy RBE with 3 executors and Redis enabled:
This configuration with 1 app instance, 3 executors, and a Redis instance will fit on a machine/cluster with at least 5 vCPUs and 24 GB of RAM.
GCP's n2-standard-8 machines or similar are a good place to start. For information on scaling up and down your deployments, see the Configuring resource section.
You can now upgrade your install with the following command:
Once your upgrade has completed (and the rollout has finished), you can reload the IP address you obtained from the kubectl command above.
You should now see a remote build execution checkbox and can try your first remote build execution. Make sure your RBE toolchains and platforms are correctly configured.
Now that you've got a working RBE deployment, you can configure resources requested by BuildBuddy app instances and executors to scale up and down your cluster.
By default BuildBuddy app instances request 1 CPU and 4 GB of RAM, executors request 1 CPU and 5 GB of RAM per instance, and Redis requests 1 CPU and 5GB of RAM.
Here's a values.yaml file that specifies the replica and resource settings you can use to scale your cluster up and down:
Note: if you scale beyond one app replica, make sure you're using MySQL instead of SQLite, and GCS/S3 instead of disk storage. For information on how to configure these, see the enterprise configuration guide. Scaling the number of executor replicas does not have these requirements.
The default values support both memory hungry Java tests and CPU-intensive Tensorflow builds. Fine tuning these parameters for peak performance depends a lot on your workload, and we're happy to help.
For a full overview of what can be configured via our enterprise Helm charts, see the buildbuddy-enterprise values.yaml file, and the buildbuddy-executor values.yaml file. Values for the executor deployment are nested in the
executor: block of the buildbuddy-enterprise yaml file.
If you don't want to use Helm, you can write your Kubernetes deployment configuration to a file:
You can then check this configuration in to your source repository, or manually apply it to your cluster with: