cote — A Node.js library for building zero-configuration microservices

cote lets you write zero-configuration microservices in Node.js without nginx, haproxy, redis, rabbitmq or anything else. It is batteries — and chargers! — included.

Join us on for anything related to cote.

Features

• Zero dependency: Microservices with only JavaScript and Node.js
• Zero-configuration: No IP addresses, no ports, no routing to configure
• Decentralized: No fixed parts, no "manager" nodes, no single point of failure
• Auto-discovery: Services discover each other without a central bookkeeper
• Fault-tolerant: Don't lose any requests when a service is down
• Scalable: Horizontally scale to any number of machines
• Performant: Process thousands of messages per second
• Humanized API: Extremely simple to get started with a reasonable API!

Develop your first microservices in under two minutes:

in time-service.js...

const cote = require('cote');
const timeService = new cote.Responder({ name: 'Time Service' });

timeService.on('time', (req, cb) => {
    cb(new Date());
});

in client.js...

const cote = require('cote');
const client = new cote.Requester({ name: 'Client' });

client.send({ type: 'time' }, (time) => {
    console.log(time);
});

You can run these files anyway you like — on a single machine or scaled out to hundreds of machines in different datacenters — and they will just work. No configuration, no third party components, no nginx, no kafka, no consul and only Node.js. cote is batteries — and chargers — included!

Microservices case study

Make sure to check out the e-commerce case study that implements a complete e-commerce application with microservices using cote. It features;

• a back-office with real-time updates for managing the catalogue of products and displaying sales with a RESTful API (express.js)
• a storefront for end-users with real-time updates to products where they can buy the products with WebSockets (socket.io)
• a user microservice for user CRUD
• a product microservice for product CRUD
• a purchase microservice that enables users to buy products
• a payment microservice that deals with money transactions that occur as a result of purchases
• Docker compose configuration for running the system locally

cote plays very well with Docker, taking advantage of its network overlay features. The case study implements a scalable microservices application via Docker and can scale to multiple machines.

Table of Contents

1. Motivation
2. Getting started
   1. Introduction to cote
   2. Installation
   3. Using cote for the first time
   4. Implementing a request-response mechanism
      1. Creating a requester
      2. Creating a responder
   5. Tracking changes in the system with a publish-subscribe mechanism
      1. Creating the arbitration service
      2. Creating a publisher
      3. Creating a subscriber
3. Components Reference
   1. Requester
   2. Responder
   3. Publisher
   4. Subscriber
   5. Sockend
   6. Monitor
   7. Monitoring Tool
4. Advanced Usage
   1. Environments
   2. Keys
   3. Namespaces
   4. Multicast address
   5. Broadcast address
   6. Controlling cote with environment variables
5. Deploying with Docker Cloud
6. Using centralized discovery tools
7. FAQ
8. Contribution
9. License

Motivation

Tomorrow belongs to ~~distributed software~~ microservices. As CPU performance is heavily dictated by the number of cores and the power of each core is already at its limits, distributed computing will decide how your application performs. ~~Distributed systems~~ Microservices also pose great architectural benefits such as fault-tolerance and scalability.

Components of such ~~a distributed system~~ microservices should be able to find other components zeroconf and communicate over a set of conventions. Sometimes they may work as a cluster, may include a pub/sub mechanism, or a request/response mechanism.

cote brings you all the advantages of ~~distributed software~~ microservices. Think of it like homing pigeons.

Getting Started

Introduction to cote

cote allows you to implement hassle-free microservices by utilizing auto-discovery and other techniques. Typically, in a microservices system, the application is broken into smaller chunks that communicate with each other. cote helps you build such a system by providing you several key components which you can use for service communication.

In a way, cote is the glue that's most necessary between different microservices. It replaces queue protocols and service registry software by clever use of IP broadcast/IP multicast systems. It's like your computer discovering there's an Apple TV nearby. This means, cote needs an environment that allows the use of IP broadcast or multicast, in order to scale beyond a single machine. Most bare-metal systems are designed this way, however, cloud infrastructure like AWS needs special care, either an overlay network like Weave, or better yet, just, Docker — which is fortunately the way run all of our software today anyway. That's why Docker is especially important for cote, as it enables cote to work its magic.

cote also replaces HTTP communication. Microservices architecture is meant for hundreds of internal services communicating with each other. That being the case, a protocol like HTTP is cumbersome and heavy for communication that doesn't need 90% of HTTP's features. Therefore, cote uses a very light protocol over plain old TCP sockets for communication, making it fast, effective and most importantly, cheap.

Installation

cote is a Node.js library for building microservices applications. It's available as an npm package.

Install cote locally via npm:

npm install cote

Using cote for the first time

Whether you want to integrate cote with an existing web application — e.g. based on express.js as exemplified here — or you want to rewrite a portion of your monolith, or you want to rewrite a few microservices with cote, all you need to do is to instantiate a few of cote's components (e.g. Responder, Requester, Publisher, Subscriber) depending on your needs, and they will start communicating automatically. While one component per process might be enough for simple applications or for tiny microservices, a complex application would require close communication and collaboration of multiple microservices. Hence, you may instantiate multiple components in a single process / service / application.

Implementing a request-response mechanism

The most common scenario for applications is the request-response cycle. Typically, one microservice would request a task to be carried out or make a query to another microservice, and get a response in return. Let's implement such a solution with cote.

First, require cote;

const cote = require('cote');

Creating a requester

Then, instantiate any component you want. Let's start with a Requester that shall ask for, say, currency conversions. Requester and all other components are classes on the main cote object, so we instantiate them with the new keyword.

const requester = new cote.Requester({ name: 'currency conversion requester' });

All cote components require an object as the first argument, which should at least have a name property to identify the component. The name is used mainly as an identifier in monitoring components, and it's helpful when you read the logs later on as each component, by default, logs the name of the other components they discover.

Requesters send requests to the ecosystem, and are expected to be used alongside Responders to fulfill those requests. If there are no Responders around, a Requester will just queue the request until one is available. If there are multiple Responders, a Requester will use them in a round-robin fashion, load-balancing among them.

Let's create and send a convert request, to ask for conversion from USD into EUR.

const request = { type: 'convert', from: 'usd', to: 'eur', amount: 100 };

requester.send(request, (err, res) => {
  console.log(res);
});

You can save this file as client.js and run it via node client.js.

Click to see the complete <code>client.js</code> file.

const cote = require('cote');

const requester = new cote.Requester({ name: 'currency conversion requester'});

const request = { type: 'convert', from: 'usd', to: 'eur', amount: 100 };

requester.send(request, (err, res) => {
  console.log(res);
});

Now this request will do nothing, and there won't be any logs in the console, because there are no components to fulfill this request and produce a response.

Keep this process running, and let's create a Responder to respond to currency conversion requests.

Creating a responder

We first instantiate a Responder with the new keyword.

const responder = new cote.Responder({ name: 'currency conversion responder' });

As detailed in Responder, each Responder is also an instance of EventEmitter2. Responding to a certain request, let's say convert, is the same as listening to the convert event, and handling it with a function that takes two parameters: a request and a callback. The request parameter holds information about a single request, and it's basically the same request object the requester above sent. The second parameter, the callback, expects to be called with the actual response.

Here's how a simple implementation might look like.

const rates = { usd_eur: 0.91, eur_usd: 1.10 };

responder.on('convert', (req, cb) => {
    cb(null, req.amount * rates[`${req.from}_${req.to}`]);
});

Now you can save this file as conversion-service.js and run it via node conversion-service.js on a separate terminal.

Click to see the complete <code>conversion-service.js</code> file.

const cote = require('cote');

const responder = new cote.Responder({ name: 'currency conversion responder' });

const rates = { usd_eur: 0.91, eur_usd: 1.10 };

responder.on('convert', (req, cb) => {
    cb(null, req.amount * rates[`${req.from}_${req.to}`]);
});

As you run the service, you will immediately see the first request in client.js being fulfilled and logged to the console. Now you can take this idea and build your services on it.

Notice how we didn't have to configure IP addresses, ports, hostnames, or anything else.

Note: By default, every Requester will connect to every Responder it discovers, regardless of the request type. This means, every Responder should respond to the exact same set of requests, because Requesters will load-balance requests between all connected Responders regardless of their capabilities, i.e, whether or not they can handle a given request.

If you have multiple Responders with varying response handlers, you will experience lost requests. In cote, this separation between responsibilities is called segmentation, or partitioning. If you wish to segment your requests in groups, you can use keys. Check out keys for a detailed guide on how and when to use segmentation.

Tracking changes in the system with a publish-subscribe mechanism

One of the benefits of a microservices approach is its ease of use as a tool for tasks that previously required serious infrastructural investments. Such a task is managing updates and tracking changes in a system. Previously, this required at least a queue infrastructure with fanout, and scaling and managing this technological dependency would be a hurdle on its own.

Fortunately, cote solves this problem in a very intuitive and almost magical way.

Say, we need an arbitration service in our application which decides currency rates, and whenever there's a change wit