Mist Cloud Management Platform - Community Edition

Mist simplifies multicloud management. It offers a unified interface from where you can manage public clouds, private clouds, hypervisors, containers and bare metal servers.

With Mist you can perform common management tasks like provisioning, orchestration, monitoring, automation and cost analysis.

It comes with a RESTful API and a CLI, so you can easily integrate it into your existing workflows.

Mist users include organizations like Juniper Networks, SevOne, Windstream, National Bank of Greece, Shoprite and more. They all report faster roll-outs while reducing their bills and management overheads by 40%-60%.

NOTE: In December 2023, Mist.io Inc was acquired by Dell Technologies. Since then, the Enterprise Edition (EE) and Hosted Service (HS) are no longer available, while the Community Edition (CE) has not been maintained besides some security fixes contributed by the community.

Mist Community Edition (CE) is licensed under the Apache License v2. It is ideal for teams with a DIY approach.

~~Mist Enterprise Edition (EE) and Hosted Service (HS) are commercial editions which offer additional plugins for governance, role-based access control & cost insights. You can check them out on our website.~~

~~The easiest way to try Mist is to sign up for a 14-day free trial at https://mist.io/sign-up.~~

Table of Contents

• List of features
• Basic terms and concepts
• Mist's architecture
• How to install Mist
• On Kubernetes with Helm
• On a single host with Docker Compose
• Local dev environment with Docker Compose

Features

Mist's features include:

• Support for more than 20 infrastructure technologies.
• Instant visibility of all the available resources across clouds, grouped by tags.
• Instant reporting/estimation of the current infrastructure costs.
• ~~Compare current & past costs, correlate with usage, provide right-sizing recommendations (EE & HS only).~~ (No longer available)
• Provision new resources on any cloud including machines, clusters, volumes, networks, zones and DNS records.
• Deploy and scale Kubernetes clusters on any supported cloud.
• Perform lifecycle actions on existing resources, e.g. stop, start, reboot, resize, destroy, etc.
• Upload scripts and run them on any machine while enforcing audit logging and centralized control of SSH keys.
• SSH command shell on any machine within the browser or through the CLI, enforcing audit logging and centralized control of SSH keys.
• Instant audit logging for all actions performed through Mist or detected through continuous polling.
• Monitor machines, display real time system & custom metrics and store them for long term access.
• Set rules on metrics or logs that trigger notifications, webhooks, scripts or lifecycle actions.
• Set schedules that trigger scripts or machine lifecycle actions.
• ~~Set fine-grained access control policies per team, tag, resource and/or action (EE & HS only).~~ (No longer available)
• ~~Set governance constraints: e.g. quotas on cost per user/team, required expiration dates (EE & HS only).~~ (No longer available)
• ~~Upload infrastructure templates that may describe complex deployments and workflows (EE & HS only).~~ (No longer available)

Terminology

Some terms are used very often in Mist. Below is a list of the most basic ones to help you avoid any confusion:

• Cloud. Any service that provides on-demand access to resources, e.g. public clouds, private clouds, hypervisors, container hosts, Kubernetes clusters, bare metal servers, etc.
• Machine. Any computing resource. There are many types of machines and some machines may contain other machines.
• Volume. Any physical or virtual data storage device, e.g. physical HDD/SSD, cloud disks, EBS volumes etc. Volumes may be attached on machines. Volumes may be provisioned along with machines or independently.
• Network. Private network spaces that machines can join, e.g. AWS VPCs.
• Script. An executable (e.g. bash script) or an Ansible playbook that can run on machines over SSH. Scripts may be added inline or by a reference to a tarball or a Git repository.
• Template. A blueprint that describes the full lifecycle of an application that may require multiple computing resources, network, storage and additional configurations. For example, the provided Kubernetes template enables the deployment of a Kubernetes cluster on any cloud and provides workflows to easily scale the cluster up or down. Currently, Mist supports Cloudify blueprints. Helm and Terraform support is coming soon.
• Stack. The deployment of a template is a stack. A stack may include resources (e.g. machines, networks, volumes etc) and provides a set of workflow actions that can be performed. A stack created by the Kubernetes template refers to a Kubernetes cluster. It includes references to all control and data plane nodes. It provides scale up & down workflows that can be applied to the cluster.
• Tunnel. A secure, point-to-point VPN tunnel enabling Mist to access infrastructure that is not on a publicly addressable network space.

Architecture

Mist is a cloud native application split into microservices which are packaged as Docker containers. It can be deployed on a Kubernetes cluster using Helm or a single host with Docker Compose.

The most notable components are the following:

• Mist UI, a web application built with Web Components and Polymer.
• REST API that serves requests from clients.
• WebSocket API, sends real-time updates to connected clients and proxies shell connections.
• Hubshell service, opens SSH connections to machines or shell connections using the Docker API.
• Dramatiq workers, running asynchronous jobs.
• APScheduler based scheduler that schedules polling tasks, rule checks, as well as user defined scheduled actions.
• RabbitMQ message queue service.
• MongoDB as the main database.
• Elasticsearch for storing and searching logs.
• Logstash for routing logs to Elasticsearch.
• Telegraf as a data collection agent, installed on monitored machines.
• Gocky as the relay to receive and pre-process monitoring metrics.
• InfluxDB or VictoriaMetrics as a time series database.

The user interacts with the RESTful Mist API through client apps like the Mist UI in the browser or command line tools (e.g. cURL, Mist CLI, etc.).

The Mist UI, apart from invoking the RESTful API, also establishes a WebSocket connection. This is used to receive real time updates and to proxy shell connections to machines.

The Mist API server interacts with the respective API's of the target clouds, either directly, or by adding tasks that get executed asynchronously by Dramatiq workers. The messaging is following the AMQP protocol and gets coordinated by RabbitMQ.

The main data store is MongoDB. Logs are being stored in Elasticsearch. Time series data go to either VictoriaMetrics or InfluxDB, depending on the installation.

Rule checks, polling tasks & user tasks are triggered by the scheduler service. Whenever a shell connection is required (e.g. SSH, Docker shell, etc.), Sheller establishes the connection and makes it available through the WebSocket API.

Installation

You can install Mist in several ways, depending on your needs:

• On Kubernetes with Helm
• On a single host with Docker Compose
• Local dev environment with Docker Compose

Kubernetes

To get started, you will need:

1. A working and up-to-date Kubernetes cluster, able to allocate 8 CPUs and 16GB of RAM to Mist.
2. Access rights to run Helm on your cluster.

Run the following commands to install Mist:

helm repo add mist https://dl.mist.io/charts
helm repo update
helm install mist-ce mist/mist-ce

Finally, follow the on-screen instructions after the installation is completed to configure an ingress IP and create the required Mist admin user.

Linode and Vultr users can find detailed installation videos in the respective, official YouTube channels.

Important configuration options

Domain and TLS

The quick installation method described above does not set up TLS. This is done in order to keep things simple and get you to test Mist quickly. However, we strongly recommend using TLS. This requires a domain for your Mist installation.

First, configure your DNS to point to your cluster's IP.

If you want to issue a new certificate, configure the cluster issuer that will be used, e.g.:

helm install mist-ce mist/mist-ce --set http.host=foo.bar.com  \
  --set http.tlsClusterIssuer=letsencrypt-prod \
  --set http.tlsSecret=secretName

For instructions on how to install and configure cert-manager read the docs here.

If you have configured a TLS certificate for this hostname as a Kubernetes secret, you should use the http.tlsSecret option, e.g.:

helm install mist-ce mist/mist-ce --set http.host=foo.bar.com \
  --set http.tlsSecret=secretName

Email

In some cases, such as user registration, forgotten passwords, user invitations etc., Mist needs to send emails. By default, Mist uses a mock mailer.

To see emails sent by Mist, get the relevant pod name:

kubectl get pods -l app=mailmock

Now, view the logs of this pod, e.g.:

kubectl logs -f mailmock-pod-name

If you wish to use an SMTP server, do something like this:

helm install mist-ce mist/mist-ce --set smtp.host=smtp.foo.bar.com \
  --set smtp.username=foo
  --set smtp.password=bar
  --set smtp.port=25
  --set smtp.tls=false
  --set smtp.starttls=true

External Docker host

Mist's orchestration plugin needs to deploy Docker containers. By default, Mist deploys an in-cluster dockerhost pod in privileged mode.

To use an external Docker host, set the following:

helm install mist-ce mist/mist-ce --set docker.host=dockerIP \
  --set docker.port=dockerPort \
  --set docker.key=TLSKey \
  --set docker.cert=TLSCert \
  --set docker.ca=TLSCACert

All configuration options

To review and customize all available configuration options:

1. Export the default chart values.

helm show values mist/mist-ce > values.yaml

1. Edit the exported values.yaml.
2. Run helm install with values.yaml as input.

helm install mist-ce mist/mist-ce -f values.yaml

The following table lists all the configurable parameters in Mist's Helm chart and their default values.

Parameter	Description	Default
http.host	FQDN or IP of Mist installation.	localhost
http.http2	Use HTTP/2.	false
http.tlsSecret	Kubernetes secret containing the tls.crt and tls.key data.	''
http.tlsHosts	Array of TLS hosts for ingress record.	[]
http.tlsAnnotations		{}
http.tlsClusterIssuer	TLS cluster issuer.	''
smtp.host	SMTP mail server address.	''
smtp.port	SMTP port.	8025
smtp.username	SMTP username.	''
smtp.password	SMTP password.	''
smtp.tls	Use TLS with SMTP.	false
smtp.starttls	Send the starttls command. Typically, it is not used with smtp.tls=true.	false
portalAdmin.enabled	Create a Mist admin user upon chart installation.	true
portalAdmin.organization	Mist organization name.	example.com
portalAdmin.mail	Mist admin's email address.	admin@example.com
portalAdmin.password	Mist admin's password.	example.com
portalAdmin.createApiToken	Cr