Eclipse Zenoh

The Eclipse Zenoh: Zero Overhead Pub/sub, Store/Query and Compute.

Zenoh (pronounce /zeno/) unifies data in motion, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

Check the website zenoh.io and the roadmap for more detailed information.

---

InfluxDB backend

In Zenoh a backend is a storage technology (such as DBMS, time-series database, file system...) alowing to store the keys/values publications made via zenoh and return them on queries. See the zenoh documentation for more details.

This backend relies on an InfluxDB server to implement the storages. Its library name (without OS specific prefix and extension) that zenoh will rely on to find it and load it is zenoh_backend_influxdb.

:point_right: Install latest release: see below

:point_right: Build "main" branch: see below

---

:warning: Documentation for previous 0.5 versions

The following documentation related to the version currently in development in "main" branch: 0.6.x.

For previous versions see the README and code of the corresponding tagged version:

• 0.5.0-beta.9
• 0.5.0-beta.8

---

Examples of usage

Prerequisites:

• You have a zenoh router (zenohd) installed, and the zenoh_backend_influxdb library file is available in ~/.zenoh/lib.
• You have an InfluxDB service running and listening on http://localhost:8086

You can setup storages either at zenoh router startup via a configuration file, either at runtime via the zenoh admin space, using for instance the REST API.

Setup via a JSON5 configuration file

• Create a zenoh.json5 configuration file containing for example:

```json5 { plugins: { // configuration of "storage_manager" plugin: storage_manager: { volumes: { // configuration of a "influxdb" volume (the "zenoh_backend_influxdb" backend library will be loaded at startup) //this should be named influxdb for v1 and influxdb2 for v2 influxdb: { // URL to the InfluxDB service. The example below is for plugin v1.x. For plugin 2.x, we need to append /api/v2/ to the url url: "http://localhost:8086", private: {

          //For detailed explanation, see individual README files for v1 and v2
          //For Influxdb v1.x:
          // If needed: InfluxDB credentials, preferably admin for databases creation and drop
            //username: "admin",
            //password: "password"

          //For Influxdb v2.x:
          // If needed: InfluxDB credentials, preferably ALL-ACCESS for databases creation and drop
          //if not ALL-ACCESS then atleast with authorization to create/delete buckets
          //Note: this should not be left empty; if you have no admin creds, you can copy the user creds instead
           // org_id: "organization ID",
          //  token: "access token"

        }
      }
    },
    storages: {
      // configuration of a "demo" storage using the "influxdb" volume
      demo: {
        // the key expression this storage will subscribes to
        key_expr: "demo/example/**",
        // this prefix will be stripped from the received key when converting to database key.
        // i.e.: "demo/example/a/b" will be stored as "a/b"
        // this option is optional
        strip_prefix: "demo/example",
        volume: {
          //this will be influxdb for v1 and influxdb2 for v2 (exactly the same name as in volumes section)
          id: "influxdb",
          // the database/bucket name within InfluxDB
          db: "zenoh_example",
          // if the database doesn't exist, create it
          create_db: false,
          // strategy on storage closure
          on_closure: "do_nothing",
          private: {

            //For Influxdb v1.x:
            //Required: InfluxDB credentials to read-write on the bucket
            //username: "admin",
            //password: "password"

          //For Influxdb v2.x:
            //Required 
            //InfluxDB credentials, with read/write privileges for the database
            //the org_id value should be the same as for admin
             // org_id: "organization ID",
              //this is a token with either:
              //a.) Read-Write access to the existing DB named above in the config (case where db/bucket already exists)
              //b.) Read-write access to ALL buckets in the organization so it can access the new bucket created by zenoh;
                          //(case where a new db/bucket has to be created)
             // token: "user access token" 
          }
        }
      }
    }
  },
  // Optionally, add the REST plugin
  rest: { http_port: 8000 }
}

} ```

• Run the zenoh router with: zenohd -c zenoh.json5

Setup at runtime via curl commands on the admin space

• Run the zenoh router, with write permissions to its admin space and with the REST plugin:
  zenohd --adminspace-permissions=rw --rest-http-port=8000
• Add the "influxdb" volume (the "zenoh_backend_fs" library will be loaded), connected to InfluxDB service on http://localhost:8086: curl -X PUT -H 'content-type:application/json' -d '{url:"http://localhost:8086"}' http://localhost:8000/@/local/router/config/plugins/storage_manager/volumes/influxdb
• Add the "demo" storage using the "influxdb" volume: curl -X PUT -H 'content-type:application/json' -d '{key_expr:"demo/example/**",volume:{id:"influxdb",db:"zenoh_example",create_db:true}}' http://localhost:8000/@/local/router/config/plugins/storage_manager/storages/demo

Tests using the REST API

Using curl to publish and query keys/values, you can:

# Put some values at different time intervals
curl -X PUT -d "TEST-1" http://localhost:8000/demo/example/test
curl -X PUT -d "TEST-2" http://localhost:8000/demo/example/test
curl -X PUT -d "TEST-3" http://localhost:8000/demo/example/test

# Retrive them as a time serie where '_time=[..]' means "infinite time range"
curl -g 'http://localhost:8000/demo/example/test?_time=[..]'

---

Volume configuration

InfluxDB-backed volumes need some configuration to work:

• "url" (required) : a URL to the InfluxDB service. Example for plugin v1: http://localhost:8086. For plugin v2, we need to append /add/v2 to the url.

admin level credentials

for v1.x

• "username" (optional) : an InfluxDB admin user name. It will be used for creation of databases, granting read/write privileges of databases mapped to storages and dropping of databases and measurements.

• "password" (optional) : the admin user's password.

Both username and password should be hidden behind a private object, as shown in the example above. In general, if you wish for a part of the configuration to be hidden when configuration is queried, you should hide it behind a private object.

for v2.x

• "org_id" (optional) : an InfluxDB organizations organization ID.

• "token" (optional) : the admin user's token. It will be used for creation and dropping of databases. In Influxdb2.x, you can use an ALL ACCESS token for this (https://docs.influxdata.com/influxdb/cloud/admin/tokens/#all-access-token)

Both org_id and token should be hidden behind a private object, like the "username" and "password" shown in the example above. In general, if you wish for a part of the configuration to be hidden when configuration is queried, you should hide it behind a private object.

---

Volume-specific storage configuration

Storages relying on a influxdb backed volume may have additional configuration through the volume section:

• "db" (optional, string) : the InfluxDB database name the storage will map into. If not specified, a random name will be generated, and the corresponding database will be created (even if "create_db" is not set).

• "create_db" (optional, boolean) : create the InfluxDB database if not already existing. By default the database is not created, unless "db" property is not specified. (the value doesn't matter, only the property existence is checked)

• "on_closure" (optional, string) : the strategy to use when the Storage is removed. There are 3 options:

• unset or "do_nothing": the database remains untouched (this is the default behaviour)
• "drop_db": the database is dropped (i.e. removed)
• "drop_series": all the series (measurements) are dropped and the database remains empty. This is currently not supported in v2.

Volume-specific user credentials

for v1.x

• "username" (optional, string) : an InfluxDB user name (usually non-admin). It will be used to read/write points in the database on GET/PUT/DELETE zenoh operations.

• "password" (optional, string) : the user's password.

for v2.x

• "org_id" (optional, string) :the user's organization. It should be same as the admin's organization.

• "token" (optional, string) : an InfluxDB access token, usually non-admin. It will be used to read/write points in the database on GET/PUT/DELETE zenoh operations.

---

Behaviour of the backend

Mapping to InfluxDB concepts

Each storage will map to an InfluxDB database. Each key to store will map to an InfluxDB measurement named with the key stripped from the "strip_prefix" property (see below). Each key/value put into the storage will map to an InfluxDB point reusing the timestamp set by zenoh (but with a precision of nanoseconds). The fileds and tags of the point is are the following:

• "kind" tag: the zenoh change kind ("PUT" for a value that have been put, or "DEL" to mark the deletion of the key)
• "timestamp" field: the original zenoh timestamp
• "encoding" field: the value's encoding flag
• "base64" field: a boolean indicating if the value is encoded in base64
• "value"field: the value as a string, possibly encoded in base64 for binary values.

Behaviour on deletion

On deletion of a key, all points with a timestamp before the deletion message are deleted. A point with "kind"="DEL" is inserted (to avoid re-insertion of points with an older timestamp in case of un-ordered messages). In influxdb 1.x, after a delay (5 seconds), the measurement corresponding to the deleted key is dropped if it still contains no points. In influxdb 2.x, dropping measurement is not supported.

Behaviour on GET

On GET operations, by default the storage returns only the latest point for each key/measurement. This is to be coherent with other backends technologies that only store 1 value per-key. If you want to get time-series as a result of a GET operation, you need to specify a time range via the "_time"argument in your Selector.

:note: Right now, wild chunks like * and ** works only for Infl