Update clustering docs

This commit is contained in:
José Valim 2024-07-02 11:43:21 +02:00
parent cbf0bea249
commit 6bf2788ec8
5 changed files with 23 additions and 13 deletions

View file

@ -202,8 +202,8 @@ The following environment variables can be used to configure Livebook on boot:
* `LIVEBOOK_CLUSTER` - configures clustering strategy when running multiple
instances of Livebook using either the Docker image or an Elixir release.
See the "Clustering" section of our Docker Deployment guide for more
information: https://hexdocs.pm/livebook/docker.html
See the "Clustering" docs for more information:
https://hexdocs.pm/livebook/clustering.html
* `LIVEBOOK_COOKIE` - sets the cookie for running Livebook in a cluster.
Defaults to a random string that is generated on boot.

View file

@ -1,19 +1,27 @@
# Clustering
If you plan to run several Livebook instances behind a load balancer, you need to enable clustering via the `LIVEBOOK_CLUSTER` environment variable. In addition you must set `LIVEBOOK_SECRET_KEY_BASE` and `LIVEBOOK_COOKIE` to different random values (use `openssl rand -base64 48` to generate said values).
If you plan to run several Livebook instances behind a load balancer, you need to enable clustering via the `LIVEBOOK_CLUSTER` environment variable. This page describes how to configure the relevant environment variables.
Depending on the clustering strategy of your choice, you must set additional environment variables, oftentimes, at runtime. When using the Livebook Docker image, you can create a file at `/app/user/env.sh` that exports the necessary environment variables. This file is invoked right before booting Livebook.
If you are using [Livebook Teams](https://livebook.dev/teams/), you can deploy with the click of a button by running Livebook servers inside your infrastructure. To get started, open up Livebook and click "Add Organization" on the sidebar. Once completed, open up the Application pane on the sidebar (with a rocket icon), click "Deploy with Livebook Teams". We provide templates for clustering inside Fly.io and Kubernetes, without a need to follow the steps below.
## Setting `LIVEBOOK_CLUSTER`
You may set `LIVEBOOK_CLUSTER` to one of the following values.
## `auto`
### `auto`
Detects the hosting platform and automatically sets up a cluster using DNS configuration. Currently the only supported platform is Fly.io (and Kubernetes when using Livebook Teams).
Detects the hosting platform and automatically sets up a cluster using DNS configuration. Currently the only supported platform is Fly.io.
## `dns:QUERY`
### `dns:QUERY`
Sets up a cluster using DNS for queries for A/AAAA records to discover new nodes. Additionally, you must additionally set the following env vars:
* `LIVEBOOK_NODE=livebook_server@MACHINE_IP`, where `MACHINE_IP` is the machine IP of each deployed node
* If your cloud requires IPv6, also set `ERL_AFLAGS="-proto_dist inet6_tcp"`
## Setting other env vars
In addition you must set `LIVEBOOK_SECRET_KEY_BASE` and `LIVEBOOK_COOKIE` to different random values (use `openssl rand -base64 48` to generate said values).
You may need to set additional environment variables at runtime. When using the Livebook Docker image, you can create a file at `/app/user/env.sh` that exports the necessary environment variables. This file is invoked right before booting Livebook.

View file

@ -1,6 +1,6 @@
# Docker
There are two main use cases to deploy Livebook in the cloud. The first is to read and write notebooks in the cloud, instead of your machine. The second is to deploy notebooks as applications. This guide covers both as well other details such as clustering.
There are two main use cases to deploy Livebook in the cloud. The first is to read and write notebooks in the cloud, instead of your machine. The second is to deploy notebooks as applications.
## Livebook in the cloud
@ -44,7 +44,7 @@ services:
### Kubernetes
If using k8s the following template is a good starting point:
If using k8s the following template is a good starting point. It includes a load balancer and preset clustering:
```yml
apiVersion: v1
@ -136,6 +136,8 @@ data:
## Deploy notebooks as applications
It is possible to deploy any notebook as an application in Livebook. Inside the notebook, open up the Application pane on the sidebar (with a rocket icon), click "Deploy with Docker", and follow the required steps. You will be able to choose a Livebook image, preset clustering options, and more.
It is possible to deploy any notebook as an application in Livebook. Inside the notebook, open up the Application pane on the sidebar (with a rocket icon), click "Manual Docker deployment", and follow the required steps.
If you are using [Livebook Teams](https://livebook.dev/teams/), you will also have access to airgapped notebook deployment with pre-configured Zero Trust Authentication, shared team secrets and file storages. To get started, open up Livebook, click "Add Organization" on the sidebar, and visit the "Airgapped Deployment" section of your organization.
If you are using [Livebook Teams](https://livebook.dev/teams/), you can also deploy with the click of a button by running Livebook servers inside your infrastructure. To get started, open up Livebook and click "Add Organization" on the sidebar. Once completed, open up the Application pane on the sidebar (with a rocket icon), click "Deploy with Livebook Teams".
Livebook Teams also support airgapped deployments, pre-configured Zero Trust Authentication, shared team secrets, file storages, and more.

View file

@ -134,7 +134,7 @@ defmodule LivebookWeb.AppComponents do
See the
<a
class="text-blue-600 hover:text-blue-700"
href="https://hexdocs.pm/livebook/docker.html#clustering"
href="https://hexdocs.pm/livebook/clustering.html"
>
Clustering docs
</a>

View file

@ -257,7 +257,7 @@ defmodule Livebook.Hubs.DockerfileTest do
assert {"XLA_TARGET", "cuda118"} in env
end
test "deploying with fly.io cluster setup" do
test "deploying with auto cluster setup" do
config = dockerfile_config(%{clustering: :auto})
hub = team_hub()
agent_key = Livebook.Factory.build(:agent_key)