Table of Contents generated with DocToc

• Composable Operator
• Installing Composable
• Removing Composable
• Examples
  • ConfigMap created based on a Kubernetes Service
  • IBM Cloud Service plan specified dynamically
• getValueFrom elements
• The input object group and version discovery algorithm
• Format transformers
• Namespaces
• Deletion
• Field path discovery
  • Limitations

Note: The project uses Golang modules, in order to activate module support, please set the the GO111MODULE environment variable to on. How to Install and Activate Module Support

This project also contains the Composable SDK, a library that allows users to add cross-resource references to their own CRD and enable dynamic configuration. For more information and a tutorial see here.

Composable Operator

Kubernetes object specifications often require constant values for their fields. When deploying an entire application with many different resources, this limitation often results in the need for staged deployments, because some resources have to be deployed first in order to determine what data to provide for the specifications of dependent resources. This undermines the declarative nature of Kubernetes object specification and requires workflows, manual step-by-step instructions and/or brittle automated scripts for the deployment of applications as a whole.

The Composable operator alleviates this problem by wrapping any resource (native Kubernetes or CRD instance) and allowing it to be specified with references to fields of other objects. These references are resolved dynamically by the Compsable controller when the data becomes available. This allows the yaml for the entire application to be deployed at once regardless of dependencies and leverages Kubernetes native mechanisms to stage the deployment of different resources.

For example, consider a Knative KafkaSource resource:

apiVersion: sources.eventing.knative.dev/v1alpha1
kind: KafkaSource
metadata:
  name: kafka-source
spec:
  consumerGroup: knative-group
  bootstrapServers: my-cluster-kafka-bootstrap.kafka:9092,my-cluster0-kafka-bootstrap.kafka:9093
  topics: knative-demo-topic
  sink:
    apiVersion: serving.knative.dev/v1
    kind: Service
    name: event-display

The KafkaSource resource requires a field bootstrapServers whose value can only be known if a Kafka service has already been deployed successfully. So one must first deploy Kafka, obtain this data, then create the above yaml and deploy it.

With the Composable operator, this yaml can be deployed at the same time as the rest of the application, as follows:

apiVersion: ibmcloud.ibm.com/v1alpha1
kind: Composable
metadata:
  name: kafka-source
spec:
  template:
    apiVersion: sources.eventing.knative.dev/v1alpha1
    kind: KafkaSource
    metadata:
      name: kafka-source
    spec:
      consumerGroup: knative-group
      bootstrapServers:
        getValueFrom:
          kind: Secret
          name: my-kafka-binding
          path: '{.data.kafka_brokers_sasl}'
          format-transformers:
          - "Base64ToString" 
          - "JsonToObject"
          - "ArrayToCSString"
      topics: knative-demo-topic
      sink:
        apiVersion: serving.knative.dev/v1
        kind: Service
        name: event-display

With Composable as the wrapper object, the field bootstrapServers can be specified with a reference getValueFrom to another object, in this case a secret named my-kafka-binding that contains binding information for the Kafka service (created by a different operator). When the Composable object is deployed, the Composable controller tries to resolve this value and keeps trying if the secret has not been created yet. Once the secret is created and the data becomes available, the Composable operator then deploys the underlying object.

Often there is data formatting mismatch between a source and a referencing field, so Composable also provides a series of handy data transformers that can be piped together in order to obtain the correct format. In this case, the base64 secret data is first decoded to obtain a Json string, which is then parsed producing an array of urls. Finally this array is transformed into a comma-separated string.

The Composable operator allows all yamls of an application to be deployed at once, in one step, by supporting cross-resource references that are resolved dynamically, and leverages native Kubernetes mechanisms to stage the deployment of a collection of resources.

Installing Composable

To install the latest release of Composable, run the following script:

curl -sL https://raw.githubusercontent.com/IBM/composable/master/hack/install-composable.sh | bash 

Composable will be installed in the composable-operator namespace

Removing Composable

To remove Composable, run the following script:

curl -sL https://raw.githubusercontent.com/IBM/composable/master/hack/uninstall-composable.sh | bash 

Examples

Here we provide several small examples of Composable usage. See samples for more samples.

ConfigMap created based on a Kubernetes Service

Let's assume that we have a Kubernetes Service, which is part of another deployment, but we would like to create an automatic binding of our deployment objects with this Service. With help of Composable we can automatically create a ConfigMap with a Service parameter(s), e.g. the port number, whose name is http.

The Service yaml file might looks like:

apiVersion: v1
kind: Service
metadata:
  name: myservice
  namespace: default
spec:
  sessionAffinity: None
  type: ClusterIP
  selector:
    app: MyApp
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 9376

The following file contains the Composable definition:

apiVersion: ibmcloud.ibm.com/v1alpha1
kind: Composable
metadata:
  name: to-cm
spec:
  template:
    apiVersion: "v1"
    kind: ConfigMap
    metadata:
      name: myconfigmap
    data:
      servicePort:
        getValueFrom:
          kind: Service
          name: myservice
          namespace: default
          path: '{.spec.ports[?(@.name=="http")].port}}'
          format-transformers:
            - ToString

You can see the detail explanation of the getValueForm fields below, but the purpose of the object is to create a
ConfigMap named myconfigmap and set servicePort to be equal to the port named http in the Service object named myservice in the default namespace.
A Composable and a created object (myconfigmap) will be in the same namespace, but input objects can be in any namespaces.

IBM Cloud Service plan specified dynamically

The Composable operator project works tightly with 2 other related projects: SolSA - Solution Service Architecture and cloud-operators. The samples directory has 3 different examples of creation/configuration of Service.ibmcloud.ibm.com from the cloud-opertors project.

Here is one of them

apiVersion: ibmcloud.ibm.com/v1alpha1
kind: Composable
metadata:
  name: comp
spec:
  template: 
    apiVersion: ibmcloud.ibm.com/v1alpha1
    kind: Service
    metadata:
      name: mymessagehub
    spec:
      instancename: mymessagehub
      service: Event Streams
      plan:
        getValueFrom:
            kind: Secret
            name: mysecret
            path: '{.data.plan}'
            format-transformers:
            - "Base64ToString" 

In this example, the field plan of the Service.ibmcloud instance is specified by referring to a secret. When the composable operator is created, its controller tries to read the secret and obtains the data needed for this field. If the secret is available, it then creates the Service.ibmcloud resource with the proper configuration. If the secret does not exist, the Composable controller keeps re-trying until it becomes available.

Here is another example: yaml apiVersion: ibmcloud.ibm.com/v1alpha1 kind: Composable metadata: name: comp spec: template: apiVersion: ibmcloud.ibm.com/v1alpha1 kind: Service metadata: name: getValueFrom: kind: ConfigMap name: myconfigmap namespace: default path: {.data.name} spec: instancename: getValueFrom: kind: ConfigMap name: myconfigmap namespace: default path: {.data.name} service: Event Streams plan: getValueFrom: kind: Secret name: mysecret namespace: default path: {.data.planKey} In this example, the name of the underlying Service.ibmcloud instance is obtained from a configmap and the same name is used for the field instancename. This allows flexibility in defining configurations, and promotes the reuse of yamls by alleviating hard-wired information. Moreover, it can be used to configure with data that is computed dynamically as a result of the deployment of some other resource.

The getValueFrom element can point to any K8s and its extensions object. The kind of the object is defined by the kind element; the object name is defined by the name elements, and finally, the path to the data is defined by the value of the path element, which is a string with dots as a delimiter.

getValueFrom elements

The getValueFrom element should be a single child of the parent element and can contain the following sub-fileds:

Notes: * Ether name or labels of the input object should be defined. If neither of both the fields are defined, an error will be generated. * The labels based search should return a single input object.

The input object group and version discovery algorithm

• If apiVersion of the input object is specified, Composable controller will try to discover an input object with the given group and version.
  • If the given Kind is not part of the provided group, an error will be generated, despite of the Kind existence in other groups.
  • If the provided version is not supported, an error will be generated, despite of other versions existence.
  • Kubernetes core objects don't have group, so only version should be specified, e.g. v1.
• If apiVersion is not provided and the given Kind is part of the Kubernetes core group, the core group will be used, despite of the Kind existence in other groups.
• If apiVersion is not provided and the given Kind exists in several groups, and doesn't exist in the Kuberntes Core group, an error will be generated.

Format transformers

Sometimes, types of an input value and expected output value are not compatable, in order to resolve this issue, Composable supports several predefined transformers. They can be defined as a string array, so output of the previous transformer's will be input to next one. When you define a Composable object, it is your responsibility to put in a correct order the transformers.

Currently Composable supports the following transformers: