Setting up the Controller & Registry

Welcome! This tutorial guides you through setting up your Anka Build Cloud Controller & Registry on either Linux/Docker or MacOS.

This guide is for first time users and shouldn’t be used for upgrading. Please use the upgrading guide instead.

Linux / Docker

Necessary Hardware

  1. A machine running Linux to install the Anka Controller & Registry.
  2. A Mac to install Anka CLI as a Node.
While it’s possible to run Docker on mac, it’s not recommended. An Anka Controller & Registry package exists for native macOS if absolutely necessary.

Necessary Software

  1. Docker

  2. Docker-Compose – Be sure to follow the Post Installation setup in order to run docker-compose without using sudo.

What we are doing

  1. Install the Anka CLI + Create your first VM Template
  2. Install the Anka Build Cloud Controller & Registry
  3. Link the Anka CLI Node to the Controller
  4. Start a VM instance using the Controller UI

Step 1: Get familiar with Anka Virtualization

Please go over our Getting Started guide before proceeding.

Step 2: Install the Anka Build Cloud Controller & Registry

Perform the following steps on the machine intended to run the Controller & Registry and not the node running the Anka Virtualization software.

Download and extract the Controller & Registry

FULL_FILE_NAME=$(echo $(curl -Ls -r 0-1 -o /dev/null -w %{url_effective} https://veertu.com/downloads/ankacontroller-registry-docker-latest) | cut -d/ -f5)
PARTIAL_FILE_NAME=$(echo $FULL_FILE_NAME | awk -F'.tar.gz' '{print $1}')
mkdir -p $PARTIAL_FILE_NAME
cd $PARTIAL_FILE_NAME
curl -Ls https://veertu.com/downloads/ankacontroller-registry-docker-latest -o $FULL_FILE_NAME
tar -xzvf $FULL_FILE_NAME

You can also manually download the file called “Cloud Controller & Registry (Run on Linux Instance)” from the Anka Build Download page.

Configuration

We’ll need to do two things:

  1. Set the external registry address – This address is passed to the anka nodes so they can pull VM Templates from the Registry.

  2. Mount a volume for the Registry data – The directory containing VM Template/Tag layers/files and configuration metadata.

First, edit the controller/controller.env:

  1. Find the variable ANKA_ANKA_REGISTRY and set it to the proper URL (remove the comment). It should look like:

    ANKA_ANKA_REGISTRY: http://<ip/fqdn>:8089

Next, edit the docker-compose.yml (in the package root, not under the registry directory):

  1. Under anka-registry > volumes, find the line that says # - ****EDIT_ME****:/mnt/vol. Change this to include the host directory you wish to mount into the container and which will be used to store the data. It should look like:

    - /var/registry:/mnt/vol
If you’re running these containers on mac (which you should avoid), you need to also change volume source from /var/etcd-data and /var/registry to a writable location on your mac.
Start the containers

In the root package directory, execute:

docker-compose up -d

Verify the containers are running

docker ps -a

CONTAINER ID        IMAGE                 COMMAND                  CREATED              STATUS              PORTS                    NAMES
aa1de7c150e7        test_anka-controller   "/bin/bash -c 'anka-…"   About a minute ago   Up About a minute   0.0.0.0:80->80/tcp       test_anka-controller_1
0ac3a6f8b0a1        test_anka-registry     "/bin/bash -c 'anka-…"   About a minute ago   Up About a minute   0.0.0.0:8089->8089/tcp   test_anka-registry_1
03787d28d3a3        test_etcd              "/usr/bin/etcd --dat…"   About a minute ago   Up About a minute                            test_etcd_1

Anka Controller should be listening on port 80 (HTTP). Try pointing your browser to the machine’s IP or hostname. You can use localhost or 127.0.0.1 if you’re on the Controller machine.

Your new dashboard should look like the picture below

How your new dashboard looks like

Orientation

Let’s take a look at what is now running on your machine:

  1. Anka Controller is serving web UI and REST API on port 80.
  2. Anka Registry is serving REST API on port 8089.
  3. ETCD database server serving on ports 2379 and 2380 (used by Anka Controller).
Configuration and scripts

Any non-default configuration changes are done by editing the .env files, or directly in docker-compose.yml.

A full configuration reference is available.

Logging

Containers are writing logs to STDOUT+ERR, making them available to Docker.

To see the Controller’s logs:

docker logs --tail 100 -f test_anka-controller_1

To see the Registry’s logs:

docker logs --tail 100 -f test_anka-registry_1

By default, docker does not do log-rotation. As a result, log-files stored by the default json-file logging driver logging driver can cause a significant amount of disk space to be used for containers that generate much output, which can lead to disk space exhaustion.

Troubleshooting tip

The log level can be modified from the default 0 value. The higher the number, the more verbose the logging. (reference)

Great! Now that we have our Anka Controller & Registry up and running, let’s add Nodes!

Perform the following steps on the host machine/node where you created your first VM.

Add the Registry

We now need to configure the Registry on this machine so we can push/upload the local VM Template we created earlier. Uploading the Template to the Registry makes it possible to download and run it from other nodes.

Assuming you haven’t changed the default port configuration, your Registry is serving requests on port 8089.

sudo anka registry add <registry name here> http://<ip>:8089

Verify the configuration:

sudo anka registry list-repos
++
++

<registry name you set> (default)

+--------+------------------+
| host   | <the ip you set> |
+--------+------------------+
| scheme | http             |
+--------+------------------+
| port   | 8089             |
+--------+------------------+

Then, confirm the registry list command doesn’t throw any failures:

sudo anka registry list

Push the VM to the Registry

sudo anka registry push 12.X -t vanilla
Terminology note: A “VM” becomes a “Template” once it has been pushed to the registry.

After the push completes, you should see your new Template in the Templates section of the Controller UI.

Your first template

Join to the Controller & Registry

In order for the host/node to perform controller tasks (pull, start, delete, etc), it must be joined to the Anka Build Cloud. You can do this by executing:

If you have any issues with this step, please see our troubleshooting guide.
sudo ankacluster join http://<ip>
Password:
Testing connection to controller...: Ok
Testing connection to the registry...: Ok
Ok
Cluster join success
  • Replace <ip> with the IP of the machine hosting your controller:
  • If you changed the default port for the controller from 80, you’ll need to use the new port at the end of the IP. Otherwise, leave it off.

The command will hang for a few moments before displaying Cluster join success.

Step 4: Start a VM instance using the Controller UI

  1. Go to your Controller dashboard and click on the Instances tab:

    your instances view

  2. Click on Create Instance(s), and the Create New Instances view displays:

    new instances view

  3. Select the VM Template and click Start. The Create New Instances view closes and returns you to the Instances view. You should now see the Instance in a Scheduling or Pulling State: a scheduling instance

  4. After the Scheduling and Pulling finishes, the VM starts on one of the Nodes and shows a Started State in the Controller UI:

    a started instance

You can now confirm the Instance is running from inside the Node:

  • JSON output is available for scripting/automation using anka --machine-readable
sudo anka --machine-readable list | jq
{
  "status": "OK",
  "body": [
    {
      "status": "suspended",
      "name": "catalina",
      "stop_date": "2020-04-01T21:30:59.798697Z",
      "creation_date": "2020-04-01T00:00:13.656296Z",
      "version": "base",
      "uuid": "10c720eb-dcce-46f7-baa3-28bacef0ec0f"
    },
    {
      "status": "running",
      "name": "mgmtManaged-catalina-1585776660490226000",
      "stop_date": "2020-04-01T21:36:11.742662Z",
      "creation_date": "2020-04-01T21:31:01.055250Z",
      "version": "",
      "uuid": "dcbeb319-421a-4d30-8466-194eb7fa5f75"
    }
  ],
  "message": ""
}
Timestamp format: https://www.ietf.org/rfc/rfc3339.txt

Step 5: Orientate to your new environment

Anka Controller Container

  • Default Ports: 80
  • Binary in the container: /bin/anka-controller
  • Configuration files: Configuration is done through docker-compose file, the controller/controller.env, or through environment variables.
  • Logs will be written to: STDOUT/ERR. It’s possible to get the logs through docker logs command.
  • Data Storage: No data is saved on disk.

Anka Registry Container

  • Default Ports: 8089
  • Binary in the container: /bin/anka-registry
  • Configuration files: Configuration is done through docker-compose file, the registry/registry.env, or through environment variables.
  • Logs will be written to: STDOUT/ERR. It’s possible to get the logs through docker logs command.
  • Data will be written to: No default; You must set it in docker-compose.yml.

ETCD Container

  • Default Ports: N/A
  • Binary in the container: /usr/bin/etcd
  • Configuration files: Configuration is done through docker-compose file, the etcd/etcd.env, or through environment variables.
  • Logs will be written to: STDOUT/ERR. It’s possible to get the logs through docker logs command.
  • Data will be written to: /var/etcd-data

ETCD Snapshotting

It’s always a good idea to take frequent snapshots of the etcd data. This helps if you have to recover from data loss or a crash of your Build Cloud host. In this guide, we’ll not only show you how to take a snapshot, but also how to restore it to a different etcd.

The official ETCD snapshotting docs can be found at https://etcd.io/docs/v3.5/op-guide/recovery/#snapshotting-the-keyspace.

Taking a snapshot

docker exec -it anka-etcd /bin/bash -c "ETCDCTL_API=3 etcdctl snapshot save snapshot.db" && docker cp anka-etcd:/snapshot ./

Output should be something like:

{"level":"info","ts":1646315443.450814,"caller":"snapshot/v3_snapshot.go:68","msg":"created temporary db file","path":"snapshot.db.part"}
{"level":"info","ts":1646315443.4519353,"logger":"client","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":1646315443.451973,"caller":"snapshot/v3_snapshot.go:76","msg":"fetching snapshot","endpoint":"127.0.0.1:2379"}
{"level":"info","ts":1646315443.4534197,"logger":"client","caller":"v3/maintenance.go:219","msg":"completed snapshot read; closing"}
{"level":"info","ts":1646315443.5064054,"caller":"snapshot/v3_snapshot.go:91","msg":"fetched snapshot","endpoint":"127.0.0.1:2379","size":"41 kB","took":"now"}
{"level":"info","ts":1646315443.5067463,"caller":"snapshot/v3_snapshot.go:100","msg":"saved","path":"snapshot.db"}
Snapshot saved at snapshot.db

Recovering from snapshot

Let’s say that you’ve created a brand new server and unpacked the docker package’s tar.gz.

The default data directory/volume the docker-compose.yml sets for etcd is /var/etcd-data. Before you start, you want to restore the snapshot from the server/host level into the data directory for etcd:

We include the etcdctl binary for you to use inside of the package.
cd ~/anka-controller-registry-1.22.0-5dc750f1/
./etcd/etcdctl snapshot restore --data-dir /var/etcd-data ./snapshot.db

Output should look something like:

2022-03-03T16:04:16+02:00	info	snapshot/v3_snapshot.go:251	restoring snapshot	{"path": "snapshot.db", "wal-dir": "etcd-data/member/wal", "data-dir": "etcd-data", "snap-dir": "etcd-data/member/snap", "stack": "go.etcd.io/etcd/etcdutl/v3/snapshot.(*v3Manager).Restore\n\t/tmp/etcd-release-3.5.1/etcd/release/etcd/etcdutl/snapshot/v3_snapshot.go:257\ngo.etcd.io/etcd/etcdutl/v3/etcdutl.SnapshotRestoreCommandFunc\n\t/tmp/etcd-release-3.5.1/etcd/release/etcd/etcdutl/etcdutl/snapshot_command.go:147\ngo.etcd.io/etcd/etcdctl/v3/ctlv3/command.snapshotRestoreCommandFunc\n\t/tmp/etcd-release-3.5.1/etcd/release/etcd/etcdctl/ctlv3/command/snapshot_command.go:128\ngithub.com/spf13/cobra.(*Command).execute\n\t/home/remote/sbatsche/.gvm/pkgsets/go1.16.3/global/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:856\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/home/remote/sbatsche/.gvm/pkgsets/go1.16.3/global/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:960\ngithub.com/spf13/cobra.(*Command).Execute\n\t/home/remote/sbatsche/.gvm/pkgsets/go1.16.3/global/pkg/mod/github.com/spf13/cobra@v1.1.3/command.go:897\ngo.etcd.io/etcd/etcdctl/v3/ctlv3.Start\n\t/tmp/etcd-release-3.5.1/etcd/release/etcd/etcdctl/ctlv3/ctl.go:107\ngo.etcd.io/etcd/etcdctl/v3/ctlv3.MustStart\n\t/tmp/etcd-release-3.5.1/etcd/release/etcd/etcdctl/ctlv3/ctl.go:111\nmain.main\n\t/tmp/etcd-release-3.5.1/etcd/release/etcd/etcdctl/main.go:59\nruntime.main\n\t/home/remote/sbatsche/.gvm/gos/go1.16.3/src/runtime/proc.go:225"}
2022-03-03T16:04:16+02:00	info	membership/store.go:141	Trimming membership information from the backend...
2022-03-03T16:04:16+02:00	info	membership/cluster.go:421	added member	{"cluster-id": "cdf818194e3a8c32", "local-member-id": "0", "added-peer-id": "8e9e05c52164694d", "added-peer-peer-urls": ["http://localhost:2380"]}
2022-03-03T16:04:16+02:00	info	snapshot/v3_snapshot.go:272	restored snapshot	{"path": "snapshot.db", "wal-dir": "etcd-data/member/wal", "data-dir": "etcd-data", "snap-dir": "etcd-data/member/snap"}

Once restored, you can now start with docker-compose up -d and you will see a similar list of instances and nodes as the previous controller.

Be sure that the FQDN or IP you’re using for the Anka Nodes to join now points to the new machine or else they will need to be disjoined and rejoined to the new server.

MacOS

The macOS package will install and run using Apple’s Rosetta. There is no native arm package at this time.

Necessary Hardware

  1. A Mac to install the Anka Controller & Registry.
  2. A second Mac to install the Anka CLI (the “Node”).

You can complete this tutorial with only one machine running Mac OS, but it’s not recommended.

What we are doing

  1. Install the Anka CLI + Create your first VM Template
  2. Install the Anka Build Cloud Controller & Registry
  3. Link the Anka CLI Node to the Controller
  4. Start a VM instance using the Controller UI

Step 1: Get familiar with Anka Virtualization

Please go over our Getting Started guide before proceeding.

Step 2: Install the Anka Build Cloud Controller & Registry

Perform the following steps on the machine intended to run the Controller & Registry and not the node running the Anka Virtualization software.

Download the Controller & Registry PKG

Download the file called “Cloud Controller & Registry (Run on Mac)” from Anka Build Download page. If you are more comfortable with the command line, you can download the file with curl:

curl -S -L -o ~/Downloads/$(echo $(curl -Ls -r 0-1 -o /dev/null -w %{url_effective} https://veertu.com/downloads/ankacontroller-registry-mac-latest) | cut -d/ -f5) https://veertu.com/downloads/ankacontroller-registry-mac-latest

Install the Controller & Registry PKG

Double click on the .pkg to start the UI install process.

  • Or, you can install the package using the command line:
    sudo installer -pkg ~/Downloads/$(echo $(curl -Ls -r 0-1 -o /dev/null -w %{url_effective} https://veertu.com/downloads/ankacontroller-registry-mac-latest) | cut -d/ -f5) -target /

Verify your installation

Two methods are available:

  • Use the CLI status command:
    sudo anka-controller status
  • Use curl:
    curl http://<ip>:8080/api/v1/status
Configuration and scripts (reference)

The Anka Controller AND Registry command is installed into /usr/local/bin/anka-controller. To see what functions it has, execute the script with root privileges:

sudo anka-controller
usage: /usr/local/bin/anka-controller [start|stop|restart|status|logs]

When sudo anka-controller start is executed, the script will use launchd to load the daemon: /Library/LaunchDaemons/com.veertu.anka.controller.plist.

  • The Anka Controller & Registry run script is /usr/local/bin/anka-controllerd. This file acts as a run script and configuration file. You can modify it to change the default ports used by adding the proper option or ENV. For example, if you want to run the Registry on a different port and use 127.0.0.1, you would add the following above the "$CONTROLLER_BIN" line (reference):
    export ANKA_ANKA_REGISTRY="http://127.0.0.1:8081"
export ANKA_REGISTRY_LISTEN_ADDRESS=":8081"
Logging

Logs are written to /Library/Logs/Veertu/AnkaController by default:

/Library/Logs/Veertu/AnkaController/anka-controller.INFO
/Library/Logs/Veertu/AnkaController/anka-controller.WARNING
/Library/Logs/Veertu/AnkaController/anka-controller.ERROR

You can modify the destination in the /usr/local/bin/anka-controllerd file (reference).

You can also watch the logs live (similar to tail -f):

sudo anka-controller logs

The log level can be modified from the default 0 value. The higher the number, the more verbose the logging. (reference)

Great! Now that we have our Anka Controller & Registry up and running, let’s add Nodes!

Perform the following steps on the Node where you created your first VM Template.

Add the Registry

We now need to configure the Registry on this machine so we can push/upload the local VM Template we created earlier. Uploading the Template to the Registry makes it possible to download and run it from other nodes.

Assuming you haven’t changed the default port configuration, your Registry is serving requests on port 8089.

sudo anka registry add <registry name here> http://<ip>:8089

Verify the configuration:

sudo anka registry list-repos
++
++

<registry name you set> (default)

+--------+------------------+
| host   | <the ip you set> |
+--------+------------------+
| scheme | http             |
+--------+------------------+
| port   | 8089             |
+--------+------------------+

Then, confirm the registry list command doesn’t throw any failures:

sudo anka registry list

Push the VM to the Registry

sudo anka registry push 12.X -t vanilla
Terminology note: A “VM” becomes a “Template” once it has been pushed to the registry.

After the push completes, you should see your new Template in the Templates section of the Controller UI.

Your first template

Join to the Controller & Registry

If you have any issues with this step, please see our troubleshooting guide.
sudo ankacluster join http://<ip>
Password:
Testing connection to controller...: Ok
Testing connection to the registry...: Ok
Ok
Cluster join success
  • Replace <ip> with the IP of the machine hosting your controller:
  • If you changed the default port for the controller from 80, you’ll need to use the new port at the end of the IP. Otherwise, leave it off.

The command will hang for a few moments before displaying Cluster join success.

Step 4: Start a VM instance using the Controller UI

  1. Go to your Controller dashboard and click on the Instances tab:

    your instances view

  2. Click on Create Instance(s), and the Create New Instances view displays:

    new instances view

  3. Select the VM Template and click Start. The Create New Instances view closes and returns you to the Instances view. You should now see the Instance in a Scheduling or Pulling State: a scheduling instance

  4. After the Scheduling and Pulling finishes, the VM starts on one of the Nodes and shows a Started State in the Controller UI:

    a started instance

You can now confirm the Instance is running from inside the Node:

  • JSON output is available for scripting/automation using anka --machine-readable
sudo anka --machine-readable list | jq
{
  "status": "OK",
  "body": [
    {
      "status": "suspended",
      "name": "catalina",
      "stop_date": "2020-04-01T21:30:59.798697Z",
      "creation_date": "2020-04-01T00:00:13.656296Z",
      "version": "base",
      "uuid": "10c720eb-dcce-46f7-baa3-28bacef0ec0f"
    },
    {
      "status": "running",
      "name": "mgmtManaged-catalina-1585776660490226000",
      "stop_date": "2020-04-01T21:36:11.742662Z",
      "creation_date": "2020-04-01T21:31:01.055250Z",
      "version": "",
      "uuid": "dcbeb319-421a-4d30-8466-194eb7fa5f75"
    }
  ],
  "message": ""
}
Timestamp format: https://www.ietf.org/rfc/rfc3339.txt

Step 5: Orientate to your new environment

Anka Controller and Registry package

  • Default Ports: Anka Controller: 80 / Anka Registry: 8089
  • Binaries and scripts: Anka Controller has only one combined binary. It can run Anka Controller, Anka Registry and ETCD Server.
    Installed at: /Library/Application Support/Veertu/Anka/bin/anka-controller
    Start script: /usr/local/bin/anka-controllerd
    Start/Stop script: /usr/local/bin/anka-controller
    Launchd daemon: /Library/LaunchDaemons/com.veertu.anka.controller.plist
  • Configuration files: Configuration for this package is done by altering the start script at /usr/local/bin/anka-controllerd.
  • Logs: /Library/Logs/Veertu/AnkaController
  • Data: ETCD data will be saved in: /Library/Application Support/Veertu/Anka/anka-controller
    Registry data will be saved in: /Library/Application Support/Veertu/Anka/registry

Standalone Anka Registry

  • Default Ports: 80
  • Binaries and scripts: Anka Registry for Mac binary is installed at: /Library/Application Support/Veertu/Anka/bin/ankaregd
    Launchd daemon: /Library/LaunchDaemons/com.veertu.anka.registry.plist
  • Configuration files: Configuration for this package is done by altering the Launchd daemon xml file at /Library/LaunchDaemons/com.veertu.anka.registry.plist. Be sure to unload it first.
  • Data will be saved in: /Library/Application Support/Veertu/Anka/registry

What next?