PaaSTA - Build, Deploy, Connect, and Monitor Services

PaaSTA is a highly-available, distributed system for building, deploying, and running services using containers and Kubernetes.

PaaSTA has been running production services at Yelp since 2016. It was originally designed to run on top of Apache Mesos but has subsequently been updated to use Kubernetes. Over time the features and functionality that PaaSTA provides have increased but the principal design remains the same.

PaaSTA aims to take a declarative description of the services that teams need to run and then ensures that those services are deployed safely, efficiently, and in a manner that is easy for the teams to maintain. Rather than managing Kubernetes YAML files, PaaSTA provides a simplified schema to describe your service and in addition to configuring Kubernetes it can also configure other infrastructure tools to provide monitoring, logging, cost management etc.

Want to know more about the opinions behind what makes PaaSTA special? Check out the PaaSTA Principles.

Components

Note: PaaSTA is an opinionated platform that uses a few un-opinionated tools. It requires a non-trivial amount of infrastructure to be in place before it works completely:

• Docker for code delivery and containment
• Kubernetes for code execution and scheduling (runs Docker containers)
• Tron for running things on a timer (nightly batches)
• SmartStack and Envoy for service registration and discovery
• Sensu for monitoring/alerting
• Jenkins (optionally) for continuous deployment
• Prometheus and HPA for autoscaling services

One advantage to having a PaaS composed of components like these is you get to reuse them for other purposes. For example, at Yelp Sensu is not just for PaaSTA, it can be used to monitor all sorts of things. We also use Kubernetes to run other more complex workloads like Jolt and Cassandra. Our service mesh, which is a heavily customised version of SmartStack and Envoy, allows many systems at Yelp to communicate with PaaSTA services and each other.

On the other hand, requiring lots of components, means lots of infrastructure to setup before PaaSTA can work effectively! Realistically, running PaaSTA outside of Yelp would not be sensible, because in addition to the integrations mentioned above we also have strong opinions encoded in other tooling that you would need to replicate. Nevertheless, we code PaaSTA in the open because we think it is useful to share our approach and hope that the code can at least help others understand or solve similar problems.

Integrations and Features

In addition to the direct integrations above PaaSTA also relies on other components to provide PaaSTA users with other features and to manage compute capacity at Yelp.

• We use Karpenter to autoscale pools of EC2 instances to run PaaSTA. Formerly we used our own autoscaler Clusterman
• We bake AMIs using Packer
• We collect logs from services and send them via Monk to Kafka
• We use StatefulSets to run a few stateful PaaSTA services
• We autotune the resources needed by each service by monitoring usage (similar to VPA)

Design Goals

• Declarative, rather than imperative, control
• Fault tolerance
• Service isolation
• Efficient use of resources
• No single points of failure
• Pleasant interface

Getting Started

See the getting started documentation for how to deploy PaaSTA. This reference is intended to help understand how PaaSTA works but we don't advise that you use PaaSTA in production.

Debugging PaaSTA (in VS Code)

To debug PaaSTA in VS Code, please refer to the internal PaaSTA wiki page "Debugging PaaSTA (in VS Code)".

Documentation

Read the documentation at Read the Docs.

Yelp-internal Documentation/Links

• HPA
• Service Deployment

Videos / Talks About PaaSTA

• EvanKrall speaks at QCon NYC 2015 (Oct 2015)
• EvanKrall, solarkennedy, and jnb give a behind the scenes tour of PaaSTA at Yelp (Oct 2015)
• Rob-Johnson talks about PaaSTA at MesosCon 2015 (Nov 2015)
• solarkennedy presents at Box to give a Theory of PaaSes (Jan 2016)
• nhandler speaks at OSCON about Running Applications at Yelp (Slides / Video) (May 2016)

License

PaaSTA is licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Contributing

Everyone is encouraged to contribute to PaaSTA by forking the Github repository and making a pull request or opening an issue.