<liclass="md-nav__item"><ahref="#highly-available-installation-kubernetes"class="md-nav__link">Highly Available Installation (Kubernetes)</a><navclass="md-nav">
<liclass="md-nav__item"><ahref="#highly-available-installation-vms-bare-metal"class="md-nav__link">Highly Available Installation (VMs/Bare Metal)</a><navclass="md-nav">
<liclass="md-nav__item"><ahref="#highly-available-installation-kubernetes"class="md-nav__link">Highly Available Installation (Kubernetes)</a><navclass="md-nav">
<liclass="md-nav__item"><ahref="#highly-available-installation-vms-bare-metal"class="md-nav__link">Highly Available Installation (VMs/Bare Metal)</a><navclass="md-nav">
<h1id="server-installation--page-root">Advanced Server Installation<aclass="headerlink"href="#server-installation--page-root"title="Permalink to this headline">¶</a></h1>
<p>This section outlines installing the Netmaker server, including Netmaker, Netmaker UI, rqlite, and CoreDNS</p>
<p>Netmaker will require elevated privileges to perform network operations. Netmaker has similar limitations to <aclass="reference internal"href="client-installation.html"><spanclass="doc">netclient</span></a> (client networking agent).</p>
<p>Typically, Netmaker is run inside of containers (Docker). To run a non-docker installation, you must run the Netmaker binary, CoreDNS binary, database, and a web server directly on the host. Each of these components have their own individual requirements.</p>
<h2id="server-configuration-reference">Server Configuration Reference<aclass="headerlink"href="#server-configuration-reference"title="Permalink to this headline">¶</a></h2>
<p>Netmaker sets its configuration in the following order of precendence:</p>
<p><strong>Description:</strong> Specifies if GRPC is going over secure GRPC or SSL. This is a setting for the clients and is passed through the access token. Can be set to “on” and “off”. Set to on if SSL is configured for GRPC.</p>
<p><strong>Description:</strong> Allows specification of the string used to connect to the server api. Format: IP:PORT or DOMAIN:PORT. Defaults to SERVER_HOST if not specified.</p>
<p><strong>Description:</strong> Allows specification of the string used to connect to grpc. Format: IP:PORT or DOMAIN:PORT. Defaults to SERVER_HOST if not specified.</p>
</dd>
<dt>SERVER_HOST: <em>(depreciated, use SERVER_API_CONN_STRING and SERVER_GRPC_CONN_STRING)</em></dt><dd><p><strong>Default:</strong> Server will perform an IP check and set automatically unless explicitly set, or DISABLE_REMOTE_IP_CHECK is set to true, in which case it defaults to 127.0.0.1</p>
<p><strong>Description:</strong> Sets the SERVER_HTTP_HOST and SERVER_GRPC_HOST variables if they are unset. The address where traffic comes in.</p>
</dd>
<dt>SERVER_HTTP_HOST: <em>(depreciated, use SERVER_API_CONN_STRING and SERVER_GRPC_CONN_STRING)</em></dt><dd><p><strong>Default:</strong> Equals SERVER_HOST if set, “127.0.0.1” if SERVER_HOST is unset.</p>
<p><strong>Description:</strong> Set to make the HTTP and GRPC functions available via different interfaces/networks.</p>
</dd>
<dt>SERVER_GRPC_HOST: <em>(depreciated, use SERVER_API_CONN_STRING and SERVER_GRPC_CONN_STRING)</em></dt><dd><p><strong>Default:</strong> Equals SERVER_HOST if set, “127.0.0.1” if SERVER_HOST is unset.</p>
<p><strong>Description:</strong> Set to make the HTTP and GRPC functions available via different interfaces/networks.</p>
<p><strong>Description:</strong> Specifies if server should deploy itself as a node (client) in each network. May be turned to “off” for more restricted servers.</p>
<h3id="config-file-reference">Config File Reference<aclass="headerlink"href="#config-file-reference"title="Permalink to this headline">¶</a></h3>
<p>A config file may be placed under config/environments/<env-name>.yml. To read this file at runtime, provide the environment variable NETMAKER_ENV at runtime. For instance, dev.yml paired with ENV=dev. Netmaker will load the specified Config file. This allows you to store and manage configurations for different environments. Below is a reference Config File you may use.</p>
<spanclass="nt">apihost</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to 127.0.0.1 or remote ip (SERVER_HOST) if DisableRemoteIPCheck is not set to true. SERVER_API_HOST if set</span>
<spanclass="nt">apiport</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to 8081 or HTTP_PORT (if set)</span>
<spanclass="nt">grpchost</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to 127.0.0.1 or remote ip (SERVER_HOST) if DisableRemoteIPCheck is not set to true. SERVER_GRPC_HOST if set.</span>
<spanclass="nt">grpcport</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to 50051 or GRPC_PORT (if set)</span>
<spanclass="nt">masterkey</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to 'secretkey' or MASTER_KEY (if set)</span>
<spanclass="nt">allowedorigin</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to '*' or CORS_ALLOWED_ORIGIN (if set)</span>
<spanclass="nt">restbackend</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to "on" or REST_BACKEND (if set)</span>
<spanclass="nt">agentbackend</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to "on" or AGENT_BACKEND (if set)</span>
<spanclass="nt">clientmode</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to "on" or CLIENT_MODE (if set)</span>
<spanclass="nt">dnsmode</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to "on" or DNS_MODE (if set)</span>
<spanclass="nt">disableremoteipcheck</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># defaults to "false" or DISABLE_REMOTE_IP_CHECK (if set)</span>
<p>All environment variables and options are enabled in this file. It is the equivalent to running the “full install” from the above section. However, all environment variables are included, and are set to the default values provided by Netmaker (if the environment variable was left unset, it would not change the installation). Comments are added to each option to show how you might use it to modify your installation.</p>
<spanclass="nt">netmaker</span><spanclass="p">:</span><spanclass="c1"># The Primary Server for running Netmaker</span>
<spanclass="nt">privileged</span><spanclass="p">:</span><spanclass="l l-Scalar l-Scalar-Plain">true</span><spanclass="c1"># Necessary to run sudo/root level commands on host system. Take out if not running with CLIENT_MODE=on</span>
<spanclass="nt">volumes</span><spanclass="p">:</span><spanclass="c1"># Volume mounts necessary for CLIENT_MODE to control wireguard networking on host (except dnsconfig, which is where dns config files are stored for use by CoreDNS)</span>
<spanclass="p p-Indicator">-</span><spanclass="l l-Scalar l-Scalar-Plain">dnsconfig:/root/config/dnsconfig</span><spanclass="c1"># Netmaker writes Corefile to this location, which gets mounted by CoreDNS for DNS configuration.</span>
<spanclass="nt">network_mode</span><spanclass="p">:</span><spanclass="l l-Scalar l-Scalar-Plain">host</span><spanclass="c1"># Necessary for CLIENT_MODE. Should be removed if turned off, but then need to add port mappings</span>
<spanclass="nt">SERVER_HOST</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># All the Docker Compose files pre-populate this with HOST_IP, which you replace as part of the install instructions. This will set both HTTP and GRPC host.</span>
<spanclass="nt">SERVER_HTTP_HOST</span><spanclass="p">:</span><spanclass="s">"127.0.0.1"</span><spanclass="c1"># Overrides SERVER_HOST if set. Useful for making HTTP and GRPC available via different interfaces/networks.</span>
<spanclass="nt">SERVER_GRPC_HOST</span><spanclass="p">:</span><spanclass="s">"127.0.0.1"</span><spanclass="c1"># Overrides SERVER_HOST if set. Useful for making HTTP and GRPC available via different interfaces/networks.</span>
<spanclass="nt">API_PORT</span><spanclass="p">:</span><spanclass="l l-Scalar l-Scalar-Plain">8081</span><spanclass="c1"># The HTTP API port for Netmaker. Used for API calls / communication from front end. If changed, need to change port of BACKEND_URL for netmaker-ui.</span>
<spanclass="nt">GRPC_PORT</span><spanclass="p">:</span><spanclass="l l-Scalar l-Scalar-Plain">50051</span><spanclass="c1"># The GRPC port for Netmaker. Used for communications from nodes.</span>
<spanclass="nt">CLIENT_MODE</span><spanclass="p">:</span><spanclass="s">"on"</span><spanclass="c1"># on if netmaker should run its own client, off if not.</span>
<spanclass="nt">MASTER_KEY</span><spanclass="p">:</span><spanclass="s">"secretkey"</span><spanclass="c1"># The admin master key for accessing the API. Change this in any production installation.</span>
<spanclass="nt">CORS_ALLOWED_ORIGIN</span><spanclass="p">:</span><spanclass="s">"*"</span><spanclass="c1"># The "allowed origin" for API requests. Change to restrict where API requests can come from.</span>
<spanclass="nt">REST_BACKEND</span><spanclass="p">:</span><spanclass="s">"on"</span><spanclass="c1"># Enables the REST backend (API running on API_PORT at SERVER_HTTP_HOST). Change to "off" to turn off.</span>
<spanclass="nt">AGENT_BACKEND</span><spanclass="p">:</span><spanclass="s">"on"</span><spanclass="c1"># Enables the AGENT backend (GRPC running on GRPC_PORT at SERVER_GRPC_HOST). Change to "off" to turn off.</span>
<spanclass="nt">DNS_MODE</span><spanclass="p">:</span><spanclass="s">"on"</span><spanclass="c1"># Enables DNS Mode, meaning config files will be generated for CoreDNS. Note, turning "off" does not remove CoreDNS. You still need to remove CoreDNS from compose file.</span>
<spanclass="nt">DISABLE_REMOTE_IP_CHECK</span><spanclass="p">:</span><spanclass="s">"off"</span><spanclass="c1"># If turned "on", Server will not set Host based on remote IP check. This is already overridden if SERVER_HOST is set. Turned "off" by default.</span>
<spanclass="nt">GRPC_SSL</span><spanclass="p">:</span><spanclass="s">"off"</span><spanclass="c1"># Tells clients to use SSL to connect to GRPC. Switch to on to turn on.</span>
<spanclass="nt">COREDNS_ADDR</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># Address of the CoreDNS server. Defaults to SERVER_HOST</span>
<spanclass="nt">DISPLAY_KEYS</span><spanclass="p">:</span><spanclass="s">"on"</span><spanclass="c1"># Show keys permanently in UI (until deleted) as opposed to 1-time display.</span>
<spanclass="nt">SERVER_API_CONN_STRING</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># Changes the api connection string. IP:PORT format. By default is empty and uses SERVER_HOST:API_PORT</span>
<spanclass="nt">SERVER_GRPC_CONN_STRING</span><spanclass="p">:</span><spanclass="s">""</span><spanclass="c1"># Changes the grpc connection string. IP:PORT format. By default is empty and uses SERVER_HOST:GRPC_PORT</span>
<spanclass="nt">BACKEND_URL</span><spanclass="p">:</span><spanclass="s">"http://HOST_IP:8081"</span><spanclass="c1"># URL where UI will send API requests. Change based on SERVER_HOST, SERVER_HTTP_HOST, and API_PORT</span>
<spanclass="nt">coredns</span><spanclass="p">:</span><spanclass="c1"># The DNS Server. Remove this section if DNS_MODE="off"</span>
<spanclass="nt">command</span><spanclass="p">:</span><spanclass="l l-Scalar l-Scalar-Plain">-conf /root/dnsconfig/Corefile</span><spanclass="c1"># Config location for Corefile. This is the path of file which is also mounted to Netmaker for modification.</span>
<spanclass="p p-Indicator">-</span><spanclass="s">"53:53/udp"</span><spanclass="c1"># Likely needs to run at port 53 for adequate nameserver usage.</span>
<h2id="dns-mode-setup">DNS Mode Setup<aclass="headerlink"href="#dns-mode-setup"title="Permalink to this headline">¶</a></h2>
<p>If you plan on running the server in DNS Mode, know that a <aclass="reference external"href="https://coredns.io/manual/toc/">CoreDNS Server</a> will be installed. CoreDNS is a light-weight, fast, and easy-to-configure DNS server. It is recommended to bind CoreDNS to port 53 of the host system, and it will do so by default. The clients will expect the nameserver to be on port 53, and many systems have issues resolving a different port.</p>
<p>However, on your host system (for Netmaker), this may conflict with an existing process. On linux systems running systemd-resolved, there is likely a service consuming port 53. The below steps will disable systemd-resolved, and replace it with a generic (e.g. Google) nameserver. Be warned that this may have consequences for any existing private DNS configuration.</p>
<p>With the latest docker-compose, it is not necessary to perform these steps. But if you are running the install and find that port 53 is blocked, you can perform the following steps, which were tested on Ubuntu 20.04 (these should be run prior to deploying the docker containers).</p>
<p>Port 53 should now be available for CoreDNS to use.</p>
<h2id="docker-compose-install">Docker Compose Install<aclass="headerlink"href="#docker-compose-install"title="Permalink to this headline">¶</a></h2>
<p>The most simple (and recommended) way of installing Netmaker is to use one of the provided <aclass="reference external"href="https://github.com/gravitl/netmaker/tree/master/compose">Docker Compose files</a>. Below are instructions for several different options to install Netmaker via Docker Compose, followed by an annotated reference Docker Compose in case your use case requires additional customization.</p>
<h3id="test-install-no-dns-no-secure-grpc">Test Install - No DNS, No Secure GRPC<aclass="headerlink"href="#test-install-no-dns-no-secure-grpc"title="Permalink to this headline">¶</a></h3>
<p>This install will run Netmaker on a server without HTTPS using an IP address. This is not secure and not recommended, but can be helpful for testing.</p>
<p>It also does not run the CoreDNS server, to simplify the deployment</p>
<li><p>You can change the port mappings in the Docker Compose if the listed ports are already in use.</p></li>
</ul>
</dd>
</dl>
<p>Assuming you have Docker and Docker Compose installed, you can just run the following, replacing <strong>< Insert your-host IP Address Here ></strong> with your host IP (or domain):</p>
<h3id="traefik-proxy">Traefik Proxy<aclass="headerlink"href="#traefik-proxy"title="Permalink to this headline">¶</a></h3>
<p>To install with Traefik, rather than Nginx or the default Caddy, check out this repo: <aclass="reference external"href="https://github.com/bsherman/netmaker-traefik">https://github.com/bsherman/netmaker-traefik</a></p>
<h3id="no-dns-coredns-disabled">No DNS - CoreDNS Disabled<aclass="headerlink"href="#no-dns-coredns-disabled"title="Permalink to this headline">¶</a></h3>
<p>DNS Mode is currently limited to clients that can run resolvectl (systemd-resolved, see <aclass="reference internal"href="architecture.html"><spanclass="doc">Architecture docs</span></a> for more info). You may wish to disable DNS mode for various reasons. This installation option gives you the full feature set minus CoreDNS.</p>
<p>To run without DNS, follow the <aclass="reference internal"href="quick-start.html"><spanclass="doc">Quick Install</span></a> guide, omitting the steps for DNS setup. In addition, when the guide has you pull (wget) the Netmaker docker-compose template, use the following link instead:</p>
<spanid="nodocker"></span><h2id="linux-install-without-docker">Linux Install without Docker<aclass="headerlink"href="#linux-install-without-docker"title="Permalink to this headline">¶</a></h2>
<p>Most systems support Docker, but some do not. In such environments, there are many options for installing Netmaker. Netmaker is available as a binary file, and there is a zip file of the Netmaker UI static HTML on GitHub. Beyond the UI and Server, you need to install MongoDB and CoreDNS (optional).</p>
<p>To start, we recommend following the Nginx instructions in the <aclass="reference internal"href="quick-start.html"><spanclass="doc">Quick Install</span></a> guide to enable SSL for your environment.</p>
<p>Once this is enabled and configured for a domain, you can continue with the below. The recommended server runs Ubuntu 20.04.</p>
<li><p>Install rqlite on your server: <aclass="reference external"href="https://github.com/rqlite/rqlite">https://github.com/rqlite/rqlite</a></p></li>
<li><p>If any settings are incorrect such as host or mongo credentials, change them under /etc/netmaker/config/environments/< your env >.yaml and then run <codeclass="docutils literal notranslate"><spanclass="pre">sudo</span><spanclass="pre">systemctl</span><spanclass="pre">restart</span><spanclass="pre">netmaker</span></code></p></li>
</ol>
<h3id="ui-setup">UI Setup<aclass="headerlink"href="#ui-setup"title="Permalink to this headline">¶</a></h3>
<spanid="kubeinstall"></span><h2id="kubernetes-install">Kubernetes Install<aclass="headerlink"href="#kubernetes-install"title="Permalink to this headline">¶</a></h2>
<h3id="server-install">Server Install<aclass="headerlink"href="#server-install"title="Permalink to this headline">¶</a></h3>
<p>This template assumes your cluster uses Nginx for ingress with valid wildcard certificates. If using an ingress controller other than Nginx (ex: Traefik), you will need to manually modify the Ingress entries in this template to match your environment.</p>
<p>This template also requires RWX storage. Please change references to storageClassName in this template to your cluster’s Storage Class.</p>
<p>Replace the NETMAKER_BASE_DOMAIN references to the base domain you would like for your Netmaker services (ui,api,grpc). Typically this will be something like <strong>netmaker.yourwildcard.com</strong>.</p>
<h3id="netclient-daemonset">Netclient Daemonset<aclass="headerlink"href="#netclient-daemonset"title="Permalink to this headline">¶</a></h3>
<p>The following instructions assume you have Netmaker running and a network you would like to add your cluster into. The Netmaker server does not need to be running inside of a cluster for this.</p>
<p>For a more detailed guide on integrating Netmaker with MicroK8s, <aclass="reference external"href="https://itnext.io/how-to-deploy-a-cross-cloud-kubernetes-cluster-with-built-in-disaster-recovery-bbce27fcc9d7">check out this guide</a>.</p>
<h2id="nginx-reverse-proxy-setup-with-https">Nginx Reverse Proxy Setup with https<aclass="headerlink"href="#nginx-reverse-proxy-setup-with-https"title="Permalink to this headline">¶</a></h2>
<p>The <aclass="reference external"href="https://github.com/linuxserver/docker-swag">Swag Proxy</a> makes it easy to generate a valid ssl certificate for the config bellow. Here is the <aclass="reference external"href="https://docs.linuxserver.io/general/swag">documentation</a> for the installation.</p>
<p>The following file configures Netmaker as a subdomain. This config is an adaption from the swag proxy project.</p>
<spanclass="kn">set</span><spanclass="nv">$upstream_app</span><spanclass="s">netmaker-ui</span><spanclass="p">;</span><spanclass="c1"># The internal URL</span>
<spanclass="kn">set</span><spanclass="nv">$upstream_port</span><spanclass="mi">80</span><spanclass="p">;</span><spanclass="c1"># The internal Port</span>
<spanclass="kn">set</span><spanclass="nv">$upstream_proto</span><spanclass="s">http</span><spanclass="p">;</span><spanclass="c1"># the protocol that is being used</span>
<spanclass="kn">proxy_pass</span><spanclass="nv">$upstream_proto://$upstream_app:$upstream_port</span><spanclass="p">;</span><spanclass="c1"># combine the set variables from above</span>
<spanclass="kn">set</span><spanclass="nv">$upstream_app</span><spanclass="s">netmaker</span><spanclass="p">;</span><spanclass="c1"># The internal URL</span>
<spanclass="kn">set</span><spanclass="nv">$upstream_port</span><spanclass="mi">8081</span><spanclass="p">;</span><spanclass="c1"># The internal Port</span>
<spanclass="kn">set</span><spanclass="nv">$upstream_proto</span><spanclass="s">http</span><spanclass="p">;</span><spanclass="c1"># the protocol that is being used</span>
<spanclass="kn">proxy_pass</span><spanclass="nv">$upstream_proto://$upstream_app:$upstream_port</span><spanclass="p">;</span><spanclass="c1"># combine the set variables from above</span>
<spanclass="c1"># Forces the header to be the one that is visible from the outside</span>
<spanclass="kn">proxy_set_header</span><spanclass="s">Host</span><spanclass="s">backend.netmaker.example.org</span><spanclass="p">;</span><spanclass="c1"># Please cange to your URL</span>
<spanclass="c1"># Pass all headers through to the backend</span>
<spanid="hainstall"></span><h2id="highly-available-installation-kubernetes">Highly Available Installation (Kubernetes)<aclass="headerlink"href="#highly-available-installation-kubernetes"title="Permalink to this headline">¶</a></h2>
<divclass="highlight-default notranslate"><divclass="highlight"><pre><span></span><spanclass="n">helm</span><spanclass="n">install</span><spanclass="n">netmaker</span><spanclass="o">/</span><spanclass="n">netmaker</span><spanclass="o">--</span><spanclass="n">generate</span><spanclass="o">-</span><spanclass="n">name</span> \ <spanclass="c1"># generate a random id for the deploy</span>
<spanclass="o">--</span><spanclass="nb">set</span><spanclass="n">baseDomain</span><spanclass="o">=</span><spanclass="n">nm</span><spanclass="o">.</span><spanclass="n">example</span><spanclass="o">.</span><spanclass="n">com</span> \ <spanclass="c1"># the base wildcard domain to use for the netmaker api/dashboard/grpc ingress</span>
<spanclass="o">--</span><spanclass="nb">set</span><spanclass="n">replicas</span><spanclass="o">=</span><spanclass="mi">3</span> \ <spanclass="c1"># number of server replicas to deploy (3 by default)</span>
<spanclass="o">--</span><spanclass="nb">set</span><spanclass="n">ingress</span><spanclass="o">.</span><spanclass="n">enabled</span><spanclass="o">=</span><spanclass="n">true</span> \ <spanclass="c1"># deploy ingress automatically (requires nginx or traefik and cert-manager + letsencrypt)</span>
<spanclass="o">--</span><spanclass="nb">set</span><spanclass="n">ingress</span><spanclass="o">.</span><spanclass="n">className</span><spanclass="o">=</span><spanclass="n">nginx</span> \ <spanclass="c1"># ingress class to use</span>
<spanclass="o">--</span><spanclass="nb">set</span><spanclass="n">ingress</span><spanclass="o">.</span><spanclass="n">tls</span><spanclass="o">.</span><spanclass="n">issuerName</span><spanclass="o">=</span><spanclass="n">letsencrypt</span><spanclass="o">-</span><spanclass="n">prod</span> \ <spanclass="c1"># LetsEncrypt certificate issuer to use</span>
<spanclass="o">--</span><spanclass="nb">set</span><spanclass="n">dns</span><spanclass="o">.</span><spanclass="n">enabled</span><spanclass="o">=</span><spanclass="n">true</span> \ <spanclass="c1"># deploy and enable private DNS management with CoreDNS</span>
<spanclass="o">--</span><spanclass="nb">set</span><spanclass="n">dns</span><spanclass="o">.</span><spanclass="n">clusterIP</span><spanclass="o">=</span><spanclass="mf">10.245</span><spanclass="o">.</span><spanclass="mf">75.75</span><spanclass="o">--</span><spanclass="nb">set</span><spanclass="n">dns</span><spanclass="o">.</span><spanclass="n">RWX</span><spanclass="o">.</span><spanclass="n">storageClassName</span><spanclass="o">=</span><spanclass="n">nfs</span> \ <spanclass="c1"># required fields for DNS</span>
<spanclass="o">--</span><spanclass="nb">set</span><spanclass="n">postgresql</span><spanclass="o">-</span><spanclass="n">ha</span><spanclass="o">.</span><spanclass="n">postgresql</span><spanclass="o">.</span><spanclass="n">replicaCount</span><spanclass="o">=</span><spanclass="mi">2</span> \ <spanclass="c1"># number of DB replicas to deploy (default 2)</span>
</pre></div>
</div>
<p>The below command will install netmaker with two server replicas, a coredns server, and ingress with routes of api.nm.example.com, grpc.nm.example.com, and dashboard.nm.example.com. CoreDNS will be reachable at 10.245.75.75, and will use NFS to share a volume with Netmaker (to configure dns entries).</p>
<p>The below command will install netmaker with three server replicas (the default), <strong>no coredns</strong>, and ingress with routes of api.netmaker.example.com, grpc.netmaker.example.com, and dashboard.netmaker.example.com. There will be one UI replica instead of two, and one database instance instead of two. Traefik will look for a ClusterIssuer named “le-prod-2” to get valid certificates for the ingress.</p>
<p>Below, we discuss the considerations for Ingress, Kernel WireGuard, and DNS.</p>
<h3id="ingress">Ingress<aclass="headerlink"href="#ingress"title="Permalink to this headline">¶</a></h3>
<p>To run HA Netmaker, you must have ingress installed and enabled on your cluster with valid TLS certificates (not self-signed). If you are running Nginx as your Ingress Controller and LetsEncrypt for TLS certificate management, you can run the helm install with the following settings:</p>
<li><p><cite>–set ingress.annotations.cert-manager.io/cluster-issuer=<your LE issuer name></cite></p></li>
</ul>
<p>If you are not using Nginx or Traefik and LetsEncrypt, we recommend leaving ingress.enabled=false (default), and then manually creating the ingress objects post-install. You will need three ingress objects with TLS:</p>
<p>If deploying manually, the gRPC ingress object requires special considerations. Look up the proper way to route grpc with your ingress controller. For instance, on Traefik, an IngressRouteTCP object is required.</p>
<p>There are some example ingress objects in the kube/example folder.</p>
<h3id="kernel-wireguard">Kernel WireGuard<aclass="headerlink"href="#kernel-wireguard"title="Permalink to this headline">¶</a></h3>
<p>If you have control of the Kubernetes worker node servers, we recommend <strong>first</strong> installing WireGuard on the hosts, and then installing HA Netmaker in Kernel mode. By default, Netmaker will install with userspace WireGuard (wireguard-go) for maximum compatibility, and to avoid needing permissions at the host level. If you have installed WireGuard on your hosts, you should install Netmaker’s helm chart with the following option:</p>
<p><strong>This address will only be reachable from hosts that have access to the cluster service CIDR.</strong> It is only designed for use cases related to k8s. If you want a more general-use Netmaker server on Kubernetes for use cases outside of k8s, you will need to do one of the following:
- bind the CoreDNS service to port 53 on one of your worker nodes and set the COREDNS_ADDRESS equal to the public IP of the worker node
- Create a private Network with Netmaker and set the COREDNS_ADDRESS equal to the private address of the host running CoreDNS. For this, CoreDNS will need a node selector and will ideally run on the same host as one of the Netmaker server instances.</p>
<h3id="values">Values<aclass="headerlink"href="#values"title="Permalink to this headline">¶</a></h3>
<p>To view all options for the chart, please visit the README in the code repo <aclass="reference external"href="https://github.com/gravitl/netmaker/tree/master/kube/helm#values">here</a> .</p>
<h2id="highly-available-installation-vms-bare-metal">Highly Available Installation (VMs/Bare Metal)<aclass="headerlink"href="#highly-available-installation-vms-bare-metal"title="Permalink to this headline">¶</a></h2>
<p>For an enterprise Netmaker installation, you will need a server that is highly available, to ensure redundant WireGuard routing when any server goes down. To do this, you will need:</p>
<p>These documents outline general HA installation guidelines. Netmaker is highly customizable to meet a wide range of enterprise environments. If you would like support with an enterprise-grade Netmaker installation, you can <aclass="reference external"href="https://gravitl.com/book">schedule a consultation here</a> .</p>
<p>The main consideration for this document is how to configure rqlite. Most other settings and procedures match the standardized way of making applications HA: Load balancing to multiple instances, and sharing a DB. In our case, the DB (rqlite) is distributed, making HA data more easily achievable.</p>
<p>If using PostgreSQL, follow their documentation for <aclass="reference external"href="https://www.postgresql.org/docs/14/high-availability.html">installing in HA mode</a> and skip step #2.</p>
<h3id="load-balancer-setup">1. Load Balancer Setup<aclass="headerlink"href="#load-balancer-setup"title="Permalink to this headline">¶</a></h3>
<p>Your load balancer of choice will send requests to the Netmaker servers. Setup is similar to the various guides we have created for Nginx, Caddy, and Traefik. SSL certificates must also be configured and handled by the LB.</p>
<h3id="id1">2. RQLite Setup<aclass="headerlink"href="#id1"title="Permalink to this headline">¶</a></h3>
<p>RQLite is the included distributed datastore for an HA Netmaker installation. If you have a different corporate database you wish to integrate, Netmaker is easily extended to other DB’s. If this is a requirement, please contact us.</p>
<p>Assuming you use Rqlite, you must run it on each Netmaker server VM, or alongside that VM as a container. Setup a config.json for database credentials (password supports BCRYPT HASHING) and mount in working directory of rqlite and specify with <cite>-auth config.json</cite> :</p>
<p>Once your servers are set up with rqlite, the first instance must be started normally, and then additional nodes must be added with the “join” command. For instance, here is the first server node:</p>
<p>Netmaker will be started on each node with default settings, except with DATABASE=rqlite (or DATABASE=postgress) and SQL_CONN set appropriately to reach the local rqlite instance. Rqlite will maintain consistency with each Netmaker backend.</p>
<p>If deploying HA with PostgreSQL, you will connect with the following settings:</p>
<h3id="other-considerations">4. Other Considerations<aclass="headerlink"href="#other-considerations"title="Permalink to this headline">¶</a></h3>
<p>This is enough to get a functioning HA installation of Netmaker. However, you may also want to make the Netmaker UI or the CoreDNS server HA as well. The Netmaker UI can simply be added to the same servers and load balanced appropriately. For some load balancers, you may be able to do this with CoreDNS as well.</p>