Goomerang

A protocol buffers over websocket communications library.

Gopher art by @lidiackr

Status

This project is still a proof of concept. It has none or very little production experience. But that doesnt mean it wasnt made with ❤ . Maintainers of the project reserve the right of breaking the public API in new versions.

Motivation

Goomerang is an effort to provide an easy, application friendly way for communicating clients and servers. Because some of us only need to send some protos over a fast pipe. Here is a high level of the idea (bar napkin drawing):

Possible status transitions for servers:

stateDiagram-v2
 direction LR
 [*] --> NEW
 NEW --> RUNNING
 RUNNING --> CLOSING
 CLOSING --> CLOSED  
 CLOSED --> [*]

Possible status transitions for clients:

stateDiagram-v2
direction LR
[*] --> NEW
NEW --> RUNNING
RUNNING --> CLOSING
CLOSING --> CLOSED
CLOSED --> RUNNING
CLOSED --> [*]

Main features

• Simple, modular, embeddable. Use only the parts of interest. Have full control over the dependency.
• Websockets as underlying transport (currently built on top of gorilla/websocket).
• Protocol buffers as first class citizen. Register message handlers at clients and servers.
• Middleware support, inspired by the Go HTTP standard lib.
• Message Broadcasting feature.
• Publish and subscribe model.
• Send synchronous messages from the client side, following a request/response pattern.
• Support for concurrency at a message handler level.
• Support for configurable hooks for certain actions.
• Graceful shutdown of the websocket connection/processing.
• Ping/pong messages out of the box in clients and server. Keep alive connections.
• Customizable TLS configuration.
• Ability to introduce custom HTTP routes.
• Custom errors are exposed, ability to build retry systems on top of the public API.
• Observability tools, that will help instrumenting clients and servers.

Installation

go get go.eloylp.dev/goomerang

Basic usage

If you are unfamiliar with protocol buffers, all the examples on this readme plus a little guide can be found here.

Let's create a basic server. We just need to import the server package from the library. The server has multiple available options, we encourage the user to review them.

package main

import (
    "log"

    "go.eloylp.dev/goomerang/server"
)

func main() {
    s, err := server.New(
        server.WithListenAddr("127.0.0.1:8080"),
    )
    if err != nil {
        log.Fatal(err)
    }
    if err := s.Run(); err != nil { // Will block the program till exit.
        log.Fatal(err)
    }
    //...
}

The client side package its quite symmetric with the server one. The client has also multiple available options, Let's see an example:

package main

import (
    "context"
    "log"

    "go.eloylp.dev/goomerang/client"
)

func main() {
    //...
    c, err := client.New(
        client.WithServerAddr("127.0.0.1:8080"),
    )
    if err != nil {
        log.Fatal(err)
    }
    if err := c.Connect(context.TODO()); err != nil {
        log.Fatal(err)
    }
    //...
}

That's it ! we just have connected a client and a server ! But not in a very useful way. Both, client and server have support for handlers. See the next sections for more information.

Messages

Goomerang facilitates the creation of custom message protocols. Messages can have 2 parts, headers and a payload. Headers are just a map[string]string. The payload allows holding any type which accomplishes the proto.Message interface, so any protocol buffer message type.

package main

import (
    "go.eloylp.dev/goomerang/message"

    "go.eloylp.dev/goomerang/example/protos"
)

func main() {
    msg := message.New().
        SetHeader("Key", "Value").
        SetPayload(&protos.MessageV1{
            Message: "my message !",
        })
}

It's encouraged to use the message builder, so all complexities and needed data will be fulfilled.

The message.Message type it's normally used in message handlers. It can also be used in the public API of client and server instances for simple operations like Send(msg). See more in the following documentation.

Messages registration

Both parts, clients and servers, need to know all kind of messages they are going to receive beforehand. There are 2 ways of registering message types:

• Using the handler registration functions.
• In clients, the convenient client.RegisterMessage(msg) can be used.

They are exclusive. If a message is already registered in the moment of the handler registration, such message doesn't need to be registered by other methods.

An history of handlers and middlewares

There's support for handlers and middlewares for both, client and server sides. This part of the project comes inspired by the Go HTTP standard library. The intention is to make this part familiar and versatile.

Handlers and middlewares can only be registered before starting the client or the server.

Message Handlers

Let's see how to register a message handler.

package main

import (
    "go.eloylp.dev/goomerang/message"
    "go.eloylp.dev/goomerang/server"
    "go.eloylp.dev/goomerang/example/protos"
)

func main() {

    // Create the server
    s, _ := server.New(server.WithListenAddr("127.0.0.1:8080"))

    // protos.MessageV1 represents a hypothetical message in your proto repo.
    s.Handle(&protos.MessageV1{}, message.HandlerFunc(func(sender message.Sender, msg *message.Message) {
        // We need to cast the message, from the proto interface, to the concrete message type.
        msgT := msg.Payload.(*protos.MessageV1)

        // Do whatever with your message,
        // your handling logic goes here.

        // Alternatively, reply with another message.
        payload := &protos.ReplyV1{}
        reply := message.New().
            SetPayload(payload).
            SetHeader("status", "200")

        _, err := sender.Send(reply) // Replies to the client connection.
        // check for errors ...
    }))
}

Symmetrically, the same handler registering signature can be found at the client public API.

As handlers only need to accomplish the message.Handler interface, It's very common to use structs for holding handler dependencies, which need to be thread safe:

package main

import (
    "database/sql"

    "go.eloylp.dev/goomerang/message"
)

type Handler struct {
    DB *sql.DB
}

func (m *Handler) Handle(sender message.Sender, msg *message.Message) {
    //  handling logic goes there.
}

Middlewares

Middlewares are just message handlers that always get executed no matter the kind of message, alternatively providing the ability to execute the next handler in the chain. They are executed just before the matched message handler, bringing the opportunity to add preprocessing or postprocessing logic to the message handling operation. i.e metrics, logging or panic handlers. Let's see how to register a middleware:

package main

import (
    "fmt"

    "go.eloylp.dev/goomerang/message"
    "go.eloylp.dev/goomerang/server"
)

func main() {
    s, _ := server.New(server.WithListenAddr("127.0.0.1:8080"))
    s.Middleware(func(h message.Handler) message.Handler {
        return message.HandlerFunc(func(sender message.Sender, msg *message.Message) {
            fmt.Printf("received message of kind: %q", msg.Metadata.Kind)
            h.Handle(sender, msg) // Continue with the next handler in chain.
        })
    })
}

For many of you that sounds really familiar ! Yes, Goomerang tries to preserve same aspects of the standard library. Again, the same symmetric interface can be found in the client public API.

It's important to note that any number of middlewares can be registered. The order of registration will drive the order of execution . So for example, if we had:

package main

import (
    "go.eloylp.dev/goomerang/message"
    "go.eloylp.dev/goomerang/server"
)

func main() {
    s, _ := server.New(server.WithListenAddr("127.0.0.1:8080"))
    s.Handle(Handler())
    s.Middleware(Middleware1())
    s.Middleware(Middleware2())
}

The order of execution would be:

graph LR;
Middleware1-->Middleware2;
Middleware2-->Handler;

This library facilitates some middleware implementations that could be used directly by library users. Let's see the example of the panic handler one:

package main

import (
    "fmt"

    "go.eloylp.dev/goomerang/message"
    "go.eloylp.dev/goomerang/middleware"
    "go.eloylp.dev/goomerang/server"
)

func main() {
    s, _ := server.New(server.WithListenAddr("127.0.0.1:8080"))
    s.Middleware(middleware.Panic(func(p interface{}) {
        fmt.Printf("panic detected: %v", p)
    }))
}

As a rule of thumb, it's recommended to implement the panic middleware as first in the chain (so it protects the rest of handlers), in order to not crash the entire process if a panic it's raised at some point in the handler chain.

Broadcasts

This library allows sending a message to all connected clients via the broadcast interface. This interfaces can be found in both implementations, the client and the server.

From the client side perspective, the only thing to do is to call the c.Broadcast(msg) method:

package main

import (
    "context"
    "log"

    "go.eloylp.dev/goomerang/client"
    "go.eloylp.dev/goomerang/example/protos"
    "go.eloylp.dev/goomerang/message"
)

func main() {
    //...
    c, err := client.New(
        client.WithServerAddr("127.0.0.1:8080"),
    )
    if err != nil {
        log.Fatal(err)
    }
    if err := c.Connect(context.TODO()); err != nil {
        log.Fatal(err)
    }
    msg := message.New().SetPayload(&protos.MessageV1{
        Message: "my message !",
    })
    if _, err := c.Broadcast(msg); err != nil {
        log.Fatal(err)
    }
}

The above call to c.Broadcast(msg) sends a command to the server, which will send the message to all connected clients. That means the message being broadcasted needs to be properly registered in all peers.

graph LR;
Client1-->Server;
Server-->Client1;
Server-->Client2;
Server-->Client3;

From the server side perspective, it's also possible to send a message to all connected clients. This can be useful for a number of cases, like push notifications. Here's and example on how to do it from the server side:

package main

import (
    "context"
    "log"

    "go.eloylp.dev/goomerang/example/protos"
    "go.eloylp.dev/goomerang/message"
    "go.eloylp.dev/goomerang/server"
)

func main() {
    s, _ := server.New(server.WithListenAddr("127.0.0.1:8080"))

    msg := message.New().SetPayload(&protos.MessageV1{
        Message: "a message for everyone !",
    })

    _, err := s.BroadCast(context.TODO(), msg)
    if err != nil {
        log.Fatal(err)
    }
}

In contrast with client side broadcasts, server side invoked broadcasts will start the broadcasting operation immediately.

graph LR;
Server-->Client1;
Server-->Client2;
Server-->Client3;

Publish and subscribe

Clients can send asynchronous messages among them by publishing and subscribing to topics. Currently, each topic will conform a diffusion domain, which means that any message published in a given topic will be sent to all subscribed clients, without taking place any kind of load balancing.

On the client side, all messages intended to be received by a subscription, should be properly handled.

The server must be aware of all message kinds going through the pub/sub system. See message registration for more details.

Let's check and example of the client side API:

package main

import (
    "context"
    "log"

    "go.eloylp.dev/goomerang/client"
    "go.eloylp.dev/goomerang/example/protos"
    "go.eloylp.dev/goomerang/message"
)

func main() {
    //...
    c, err := client.New(
        client.WithServerAddr("127.0.0.1:8080"),
    )
    if err != nil {
        log.Fatal(err)
    }
    if err := c.Connect(context.TODO()); err != nil {
        log.Fatal(err)
    }

    // Client subscribes to a topic. Now it will receive all messages
    // for that topic.
    if err := c.Subscribe("topic.a"); err != nil {
        log.Fatal(err)
    }

    // At any moment, any client can publish a message to a specific topic.
    msg := message.New().SetPayload(&protos.MessageV1{})
    if _, err := c.Publish("topic.a", msg); err != nil {
        log.Fatal(err)
    }
    // Finally, client can unsubscribe from a topic at any moment, the
    // client will stop receiving messages for that topic.
    if err := c.Unsubscribe("topic.a"); err != nil {
        log.Fatal(err)
    }
}

We can notice the Subscribe(topic string), Publish(topic string, msg *message.Message) and Unsubscribe(topic string) interfaces. This three calls will send a command to the server. After that, the server will accomplish the required operation.

From the server side API, there's also a method `Publish(topic string, msg *message