Zoom

A blazing-fast datastore and querying engine for Go built on Redis.

Requires Redis version >= 2.8.9 and Go version >= 1.2. The latest version of both is recommended.

Full documentation is available on godoc.org.

Table of Contents

• Development Status
• When is Zoom a Good Fit?
• Installation
• Initialization
• Models
• What is a Model?
• Customizing Field Names
• Creating Collections
• Saving Models
• Updating Models
• Finding a Single Model
• Finding Only Certain Fields
• Finding All Models
• Deleting Models
• Counting the Number of Models
• Transactions
• Queries
• The Query Object
• Using Query Modifiers
• A Note About String Indexes
• More Information
• Persistence
• Atomicity
• Concurrent Updates and Optimistic Locking
• Testing & Benchmarking
• Running the Tests
• Running the Benchmarks
• Contributing
• Example Usage
• License

Development Status

Zoom was first started in 2013. It is well-tested and going forward the API will be relatively stable. We are closing in on Version 1.0.

At this time, Zoom can be considered safe for use in low-traffic production applications. However, as with any relatively new package, it is possible that there are some undiscovered bugs. Therefore we would recommend writing good tests, reporting any bugs you may find, and avoiding using Zoom for mission-critical or high-traffic applications.

Zoom follows semantic versioning, but offers no guarantees of backwards compatibility until version 1.0. We recommend using a dependency manager such as godep or glide to lock in a specific version of Zoom. You can also keep an eye on the Releases page to see a full changelog for each release. In addition, starting with version 0.9.0, migration guides will be provided for any non-trivial breaking changes, making it easier to stay up to date with the latest version.

When is Zoom a Good Fit?

Zoom might be a good fit if:

1. You are building a low-latency application. Because Zoom is built on top of Redis and all data is stored in memory, it is typically much faster than datastores/ORMs based on traditional SQL databases. Latency will be the most noticeable difference, although throughput may also be improved.
2. You want more out of Redis. Zoom offers a number of features that you don't get by using a Redis driver directly. For example, Zoom supports a larger number of types out of the box (including custom types, slices, maps, complex types, and embedded structs), provides tools for making multi-command transactions easier, and of course, provides the ability to run queries.
3. You want an easy-to-use datastore. Zoom has a simple API and is arguably easier to use than some ORMs. For example, it doesn't require database migrations and instead builds up a schema based on your struct types. Zoom also does not typically require any knowledge of Redis in order to use effectively. Just connect it to a database and you're good to go!

Zoom might not be a good fit if:

1. You are working with a lot of data. Redis is an in-memory database, and Zoom does not yet support sharding or Redis Cluster. Memory could be a hard constraint for larger applications. Keep in mind that it is possible (if expensive) to run Redis on machines with up to 256GB of memory on cloud providers such as Amazon EC2.
2. You need advanced queries. Zoom currently only provides support for basic queries and is not as powerful or flexible as something like SQL. For example, Zoom currently lacks the equivalent of the IN or OR SQL keywords. See the documentation for a full list of the types of queries supported.

Installation

Zoom is powered by Redis and needs to connect to a Redis database. You can install Redis on the same machine that Zoom runs on, connect to a remote database, or even use a Redis-as-a-service provider such as Redis To Go, RedisLabs, Google Cloud Redis, or Amazon Elasticache.

If you need to install Redis, see the installation instructions on the official Redis website.

To install Zoom itself, run go get -u github.com/albrow/zoom to pull down the current master branch, or install with the dependency manager of your choice to lock in a specific version.

Initialization

First, add github.com/albrow/zoom to your import statement:

import (
     // ...
     github.com/albrow/zoom
)

Then, you must create a new pool with NewPool. A pool represents a pool of connections to the database. Since you may need access to the pool in different parts of your application, it is sometimes a good idea to declare a top-level variable and then initialize it in the main or init function. You must also call pool.Close when your application exits, so it's a good idea to use defer.

var pool *zoom.Pool

func main() {
    pool = zoom.NewPool("localhost:6379")
    defer func() {
        if err := pool.Close(); err != nil {
            // handle error
        }
    }()
    // ...
}

The NewPool function accepts an address which will be used to connect to Redis, and it will use all the default values for the other options. If you need to specify different options, you can use the NewPoolWithOptions function.

For convenience, the PoolOptions type has chainable methods for changing each option. Typically you would start with DefaultOptions and call WithX to change value for option X.

For example, here's how you could initialize a Pool that connects to Redis using a unix socket connection on /tmp/unix.sock:

options := zoom.DefaultPoolOptions.WithNetwork("unix").WithAddress("/tmp/unix.sock")
pool = zoom.NewPoolWithOptions(options)

Models

What is a Model?

Models in Zoom are just structs which implement the zoom.Model interface:

type Model interface {
  ModelID() string
  SetModelID(string)
}

To clarify, all you have to do to implement the Model interface is add a getter and setter for a unique id property.

If you want, you can embed zoom.RandomID to give your model all the required methods. A struct with zoom.RandomID embedded will generate a pseudo-random id for itself the first time the ModelID method is called iff it does not already have an id. The pseudo-randomly generated id consists of the current UTC unix time with second precision, an incremented atomic counter, a unique machine identifier, and an additional random string of characters. With ids generated this way collisions are extremely unlikely.

Future versions of Zoom may provide additional id implementations out of the box, e.g. one that assigns auto-incremented ids. You are also free to write your own id implementation as long as it satisfies the interface.

A struct definition serves as a sort of schema for your model. Here's an example of a model for a person:

type Person struct {
     Name string
     Age  int
     zoom.RandomID
}

Because of the way Zoom uses reflection, all the fields you want to save need to be exported. Unexported fields (including unexported embedded structs with exported fields) will not be saved. This is a departure from how the encoding/json and encoding/xml packages behave. See issue #25 for discussion.

Almost any type of field is supported, including custom types, slices, maps, complex types, and embedded structs. The only things that are not supported are recursive data structures and functions.

Customizing Field Names

You can change the name used to store the field in Redis with the redis:"<name>" struct tag. So for example, if you wanted the fields to be stored as lowercase fields in Redis, you could use the following struct definition:

type Person struct {
     Name string    `redis:"name"`
     Age  int       `redis:"age"`
     zoom.RandomID
}

If you don't want a field to be saved in Redis at all, you can use the special struct tag redis:"-".

Creating Collections

You must create a Collection for each type of model you want to save. A Collection is simply a set of all models of a specific type and has methods for saving, finding, deleting, and querying those models. NewCollection examines the type of a model and uses reflection to build up an internal schema. You only need to call NewCollection once per type. Each pool keeps track of its own collections, so if you wish to share a model type between two or more pools, you will need to create a collection for each pool.

// Create a new collection for the Person type.
People, err := pool.NewCollection(&Person{})
if err != nil {
     // handle error
}

The convention is to name the Collection the plural of the corresponding model type (e.g. "People"), but it's just a variable so you can name it whatever you want.

NewCollection will use all the default options for the collection.

If you need to specify other options, use the NewCollectionWithOptions function. The second argument to NewCollectionWithOptions is a CollectionOptions. It works similarly to PoolOptions, so you can start with DefaultCollectionOptions and use the chainable WithX methods to specify a new value for option X.

Here's an example of how to create a new Collection which is indexed, allowing you to use Queries and methods like FindAll which rely on collection indexing:

options := zoom.DefaultCollectionOptions.WithIndex(true)
People, err = pool.NewCollectionWithOptions(&Person{}, options)
if err != nil {
    // handle error
}

There are a few important points to emphasize concerning collections:

1. The collection name cannot contain a colon.
2. Queries, as well as the FindAll, DeleteAll, and Count methods will not work if Index is false. This may change in future versions.

If you need to access a Collection in different parts of your application, it is sometimes a good idea to declare a top-level variable and then initialize it in the init function:

var (
    People *zoom.Collection
)

func init() {
    var err error
    // Assuming pool and Person are already defined.
    People, err = pool.NewCollection(&Person{})
    if err != nil {
        // handle error
    }
}

Saving Models

Continuing from the previous example, to persistently save a Person model to the database, we use the People.Save method. Recall that in this example, "People" is just the name we gave to the Collection which corresponds to the model type Person.

p := &Person{Name: "Alice", Age: 27}
if err := People.Save(p); err != nil {
     // handle error
}

When you call Save, Zoom converts all the fields of the model into a format suitable for Redis and stores them as a Redis hash. There is a wiki page describing how zoom works under the hood in more detail.

Updating Models

Sometimes, it is preferable to only update certain fields of the model instead of saving them all again. It is more efficient and in some scenarios can allow safer simultaneous changes to the same model (as long as no two clients update the same field at the same time). In such cases, you can use UpdateFields.

if err := People.UpdateFields([]string{"Name"}, person); err != nil {
    // handle error
}

UpdateFields uses "last write wins" semantics, so if another caller updates the same field, your changes may be overwritten. That means it is not safe for "read before write" updates. See the section on Concurrent Updates for more information.

Finding a Single Model

To retrieve a model by id, use the Find method:

p := &Person{}
if err := People.Find("a_valid_person_id", p); err != nil {
     // handle error
}

The second argument to Find must be a pointer to a struct which satisfies Model, and must have a type corresponding to the Collection. In this case, we passed in Person since that is the struct type that corresponds to our People collection. Find will mutate p by setting all its fields. Using `Find