Kubernetes Migration Factory User Guide

Kubernetes Migrations Factory (KMF) is a tool developed for migrating docker containers to Amazon EKS. KMF is written in Golang and offers command line interface. The Kubernetes Migration Factory solution is an orchestration platform for migrating containers to Amazon EKS at scale. It is designed to coordinate and automate many of the manual processes, eliminating human error and speeding migration phases down to minutes from weeks of planning and data collection. With just a few inputs provided to the KMF command line interface(CLI), customers will be able to set, launch, and track the migration plan for their current container platform configuration into Amazon EKS. This solution is able to scan the source platform and generate a blueprint of the running workload. After the scan is done, you can also see list of all the Kubernetes resources that will be created onthe destination. This solution is built to understand the source platform implementation details and then manipulate the obtained manifest files such that it is valid and accepted by the destination Kubernetes implementation, Amazon EKS. It is able to provide recommendations to add relevant AWS native services by determining the type of services used in the source and finding a similar type of AWS resources. The solution will enable the customer with autonomous recommendation for their data plane setup based on their current cluster meta data.

Architecture

Kubernetes Migration Factory (KMF) tool can be used to migrate Kubernetes resources from a Kubernetes cluster running anywhere(1) to Amazon EKS via a Command Line Interface(CLI). It can run on a server that can authenticate to the source Kubernetes cluster and target Kubernetes cluster, which is Amazon EKS in this case.

Prerequisites

• This tool migrates from any Kubernetes cluster to Amazon EKS, so it needs a destination EKS Cluster. For more information, please refer Creating an Amazon EKS Cluster documentation
• On the workstation where the KMF CLI will be used, setup the kubeconfig for the destination cluster. For more information on authenticating and accessing an Amazon EKS cluster, please refer to Cluster Authentication documentation
• Setup kubeconfig file to access the EKS Cluster using the kubectl tool and the same setup is sufficient for KMF to get access. Please refer setting kubeconfig file for EKS cluster access documentation
• Similarly setup the kubeconfig file for the source Kubernetes cluster. For Google Kubernetes Engine(GKE), you may refer cluster access to GKE just like how you setup access for kubectl tool
• Download KMF for your platform from here [TO COME]. If your platform is not among the available releases, you can install GoLang and setup the workspace. Please refer Download and Install for more information about installing golang to your workstation
• Amazon EKS Cluster used as destination for migrating the Kubernetes workload should have access to the docker registry used in the source. You may follow the documentation Pull an Image from a Private Registry . If you create the secret for private registry access in the source kubernetes cluster and also attach to all the container specifications in all resources, kubernetes migration factory will migrate that as well
• KMF tool also helps to migrate images from 3rd party repositories such as GCR, Dockerhub, Gitlab private registry to Amazon Elastic Container Registry. On the workstation where the KMF CLI will be used, ensure to setup docker login for Amazon ECR and also add docker login to list of supported repositories (GCR, Gitlab, Dockerhub) that are intended for migration prior to executing KMF
• KMF tool uses aws privileges assigned to the execution id in order to create Amazon elastic container registry and push images to it. The following IAM policy statement is the minimum required permission

  {
      "Version":"2012-10-17",
      "Statement":[
      {
          "Sid":"ListImagesInRepository",
          "Effect":"Allow",
          "Action":[
              "ecr:ListImages"
          ],
          "Resource":"arn:aws:ecr:<AWS-REGION>:<AWS-ACCOUNT>:repository/*"
      },
      {
          "Sid":"GetAuthorizationToken",
          "Effect":"Allow",
          "Action":[
              "ecr:GetAuthorizationToken"
          ],
          "Resource":"*"
      },
      {
          "Sid":"ManageRepositoryContents",
          "Effect":"Allow",
          "Action":[
                  "ecr:BatchCheckLayerAvailability",
                  "ecr:GetDownloadUrlForLayer",
                  "ecr:GetRepositoryPolicy",
                  "ecr:DescribeRepositories",
                  "ecr:ListImages",
                  "ecr:DescribeImages",
                  "ecr:BatchGetImage",
                  "ecr:InitiateLayerUpload",
                  "ecr:UploadLayerPart",
                  "ecr:CompleteLayerUpload",
                  "ecr:PutImage"
          ],
          "Resource":"arn:aws:ecr:<AWS-REGION>:<AWS-ACCOUNT>:repository/*"
      }
      ]
  }
  

KMF CLI Setup

Use an already build binary: Steps:

1. Download KMF CLI for your os from the release section to your workstation where you have authenticated to both source and destination clusters. There are compiled binaries for Mac OS, Linux and Windows operating system.
2. For Mac OS, the binary name is kmf_mac. This document is narrated to reference binary by the name kmf, hence edit the commands accrodingly.
3. For Linux OS, the binary name is kmf.
4. For Windows OS, the binary name kmf.exe. This document is narrated to reference binary by the name kmf, hence edit the commands accrodingly.

Note: The compiled binaries for 64 bit executables with GOARCH=amd64 go build commands used are as below for compilation:

For linux: CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o ./bin/linux/kmf

For windows CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -o ./bin/windows/kmf.exe

For mac go build -o ./bin/mac/kmf since it was compiled on a mac machine.

For more information you may refer cross-compiling made easy with Golang documentation

1. Optional: Add the tool to your OS PATH

Building the binary from source: Steps:

1. Clone the kubernetes-migration-factory repository
2. Change directory to containers-migration-factory

$cd kubernetes-migration-factory/

1. Run the below command to build the binary for the KMF

$go build

The above command will build the binary for KMF CLI within current directory to a file named containers-migration-factory If you want this to be a shorter name say kmf, then run the command as below:

$go build -o ./bin/kmf

Running KMF CLI:

Running with a Config config.ini

From the operating system path from where you are making calls to ./bin/kmf, make sure that you have a file with the name exactly called config.ini and provide all the required information. A sample config.ini file looks like below:

Explanation for each parameter is provided in the config.ini file as comments.

[COMMON]
# common configuration params required for migration.
# Local path where generated helm charts to be saved
HELM_CHARTS_PATH=/Users/username/kuberenetes-pocs/helm
RESOURCES=all
# Valid Value for ACTION Deploy/Delete
ACTION=Delete
# Namespaces from which the resources need to migrated
# comma seperated list of namespace or "all"
NAMESPACES=all

[SOURCE]
# Source Cloud Provider valid values are GKE,AKE,KOPS
CLOUD=GKE
# Source kube config file
# Refer the documentation for the respective source cluster provider to create the kubeconfig file. For GKE, you may refer https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl
KUBE_CONFIG=/Users/username/.kube/gcp.config
CONTEXT=<Kube-Context-Name>

[TARGET]
# Target Cloud Provider valid value is AWS only
CLOUD=AWS
# Target kube config file
# Refer the AWS documentation for creating the kubeconfig file https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html
KUBE_CONFIG=/Users/username/.kube/config
CONTEXT=<Kube-Context-Name>

[MIGRATE_IMAGES]
# This section is used for passing values to the KMF CLI to perform container registry images migration to Amazon ECR and this is optional
# Do you wish to migrate images from 3rd party repositories to Amazon Elastic Container Registry? Supply either "Yes" or "No"
USERCONSENT=Yes
# Comma separated list of 3rd party registries. Tool supports migration from gcr, gitlab, dockerhub registries.
REGISTRY=GCR

Explanation of each supported parameter for the KMF CLI tool

COMMON Section

common configuration params required for migration

HELM_CHARTS_PATH (Required): Local path to store the Helm charts obtained from the source cluster example value /Users/username/kuberenetes-pocs/helm

RESOURCES (Required): Kubernetes resources to migrate from source to destination cluster valid values are: * "all" for all resources supported by KMF CLI * Comma separated resources, for example "services, deployments, cronjobs"

Action (Required) : Action to perform on the destination cluster valid values are: ACTION=Delete: To delete the kubernetes resource matching the source cluster Note: Use this only if it is necessary as this is a destruction feature Action=Deploy: To deploy the kubernetes resource matching the source cluster

Namespaces (Required): Kubernetes Namespaces from which the KMF tool should migrate resources valid values are: "all" for migrating Kubernetes resources from all namespaces you can also provide comma separated values of namespaces, for example if the namespaced from which your want to migrate are dev, test, stage, then this will be "dev,test,stage"

SOURCE Section

CLOUD (Required): Cloud provider for the source Kubernetes cluster Valid values: any one of GKE, AKE, KOPS

KUBE_CONFIG (Required): Kubeconfig file path on the local machine for the destination cluster

CONTEXT (Required): Kubeconfig context. This helps to choose the Kubernetes cluster if there is a combined kubeconfig file with multiple clusters

TARGET Section

CLOUD (Required): Cloud provider for the source Kubernetes cluster Valid values: EKS

KUBE_CONFIG (Required): Kubeconfig file path on the local machine for the target cluster

CONTEXT (Required): Kubeconfig context. This helps to choose the Kubernetes cluster if there is a combined kubeconfig file with multiple clusters

MIGRATE_IMAGES Section

USERCONSENT (Required): User consent to migrate container images to Amazon ECR

Valid values: Yes, No

REGISTRY (Required): Source container registry name. This requires the user to perform docker login to those registry on the machine where KMF-CLI is getting executed

Valid Values: gcr, gitlab, dockerhub

If any argument is missing in the config.ini file or if not using a config.ini file, follow the prompt and give all the information asked.

NOTE: This tool supports a merged kubeconfig file with both the source and destination configurations. Use the same kubeconfig file location for source and destination when answering the prompts from the tool

Some examples while interacting with KMF CLI

$ ./bin/kmf
Please pass the location of source kubernetes cluster kubeconfig file: 


Please pass the source context: (default: <current-context>): <Leave Blank for default or source context>
Please pass comma separated list of resources to migrate from source cluster to destination cluster. For all resources enter 'all':
Please pass comma separated list of namespaces for source cluster. For all namespaces enter 'all': 
Please pass path to save Helm charts from source cluster:
Do you want to migrate images from 3rd party registries to ECR? Supply either Yes or No: 
Tool supports migration from gcr, gitlab, dockerhub registries. Please pass comma separated list of 3rd party registries: 
Please pass the location of destination EKS cluster kubeconfig file: <destination kubeconfig file location>
Please pass the destination context (default: <current-context>): <Leave Blank for default or destination context>
Please pass what action the tool needs to perform. Accepted values are Deploy or Delete :

Based on the information provided to the CLI, it will poll the source cluster and get the details about the Kubernetes resources. KMF evaluates the source Kubernetes objects and manipulates them to be compatible with Amazon EKS, and deploys the modified manifests to its destination.

Running as one line:

Migrating all resources from all namespaces

$ ./bin/kmf --source_kubeconfig  /Users/<user-name>/.kube/gcp.config \
--destination_kubeconfig /Users/<user-name>/.kube/aws.config \
--namespaces "all" \
--resources "all" \
--migrate_images "yes" \
--reg_names "gcr, dockerhub, gitlab" \
--source_context <source_kubernetes_context> \  
--destination_context <destination_kubernetes_context> \
--helm_path <local-file-system-path> \
--action Deploy