github.com/louischatriot/nedb @v1.6.0 sqlite

The JavaScript Database

Embedded persistent or in memory database for Node.js, nw.js, Electron and browsers, 100% JavaScript, no binary dependency. API is a subset of MongoDB's and it's plenty fast.

IMPORTANT NOTE: Please don't submit issues for questions regarding your code. Only actual bugs or feature requests will be answered, all others will be closed without comment. Also, please follow the bug reporting guidelines and check the change log before submitting an already fixed bug :)

Support NeDB development

No time to help out? You can support NeDB development by sending money or bitcoins!

Money:

Bitcoin address: 1dDZLnWpBbodPiN8sizzYrgaz5iahFyb1

Installation, tests

Module name on npm and bower is nedb.

npm install nedb --save    # Put latest version in your package.json
npm test                   # You'll need the dev dependencies to launch tests
bower install nedb         # For the browser versions, which will be in browser-version/out

API

It is a subset of MongoDB's API (the most used operations).

• Creating/loading a database
• Persistence
• Inserting documents
• Finding documents
• Basic Querying
• Operators ($lt, $lte, $gt, $gte, $in, $nin, $ne, $exists, $regex)
• Array fields
• Logical operators $or, $and, $not, $where
• Sorting and paginating
• Projections
• Counting documents
• Updating documents
• Removing documents
• Indexing
• Browser version

Creating/loading a database

You can use NeDB as an in-memory only datastore or as a persistent datastore. One datastore is the equivalent of a MongoDB collection. The constructor is used as follows new Datastore(options) where options is an object with the following fields:

• filename (optional): path to the file where the data is persisted. If left blank, the datastore is automatically considered in-memory only. It cannot end with a ~ which is used in the temporary files NeDB uses to perform crash-safe writes.
• inMemoryOnly (optional, defaults to false): as the name implies.
• timestampData (optional, defaults to false): timestamp the insertion and last update of all documents, with the fields createdAt and updatedAt. User-specified values override automatic generation, usually useful for testing.
• autoload (optional, defaults to false): if used, the database will automatically be loaded from the datafile upon creation (you don't need to call loadDatabase). Any command issued before load is finished is buffered and will be executed when load is done.
• onload (optional): if you use autoloading, this is the handler called after the loadDatabase. It takes one error argument. If you use autoloading without specifying this handler, and an error happens during load, an error will be thrown.
• afterSerialization (optional): hook you can use to transform data after it was serialized and before it is written to disk. Can be used for example to encrypt data before writing database to disk. This function takes a string as parameter (one line of an NeDB data file) and outputs the transformed string, which must absolutely not contain a \n character (or data will be lost).
• beforeDeserialization (optional): inverse of afterSerialization. Make sure to include both and not just one or you risk data loss. For the same reason, make sure both functions are inverses of one another. Some failsafe mechanisms are in place to prevent data loss if you misuse the serialization hooks: NeDB checks that never one is declared without the other, and checks that they are reverse of one another by testing on random strings of various lengths. In addition, if too much data is detected as corrupt, NeDB will refuse to start as it could mean you're not using the deserialization hook corresponding to the serialization hook used before (see below).
• corruptAlertThreshold (optional): between 0 and 1, defaults to 10%. NeDB will refuse to start if more than this percentage of the datafile is corrupt. 0 means you don't tolerate any corruption, 1 means you don't care.
• nodeWebkitAppName (optional, DEPRECATED): if you are using NeDB from whithin a Node Webkit app, specify its name (the same one you use in the package.json) in this field and the filename will be relative to the directory Node Webkit uses to store the rest of the application's data (local storage etc.). It works on Linux, OS X and Windows. Now that you can use require('nw.gui').App.dataPath in Node Webkit to get the path to the data directory for your application, you should not use this option anymore and it will be removed.

If you use a persistent datastore without the autoload option, you need to call loadDatabase manually. This function fetches the data from datafile and prepares the database. Don't forget it! If you use a persistent datastore, no command (insert, find, update, remove) will be executed before loadDatabase is called, so make sure to call it yourself or use the autoload option.

Also, if loadDatabase fails, all commands registered to the executor afterwards will not be executed. They will be registered and executed, in sequence, only after a successful loadDatabase.

// Type 1: In-memory only datastore (no need to load the database)
var Datastore = require('nedb')
  , db = new Datastore();


// Type 2: Persistent datastore with manual loading
var Datastore = require('nedb')
  , db = new Datastore({ filename: 'path/to/datafile' });
db.loadDatabase(function (err) {    // Callback is optional
  // Now commands will be executed
});


// Type 3: Persistent datastore with automatic loading
var Datastore = require('nedb')
  , db = new Datastore({ filename: 'path/to/datafile', autoload: true });
// You can issue commands right away


// Type 4: Persistent datastore for a Node Webkit app called 'nwtest'
// For example on Linux, the datafile will be ~/.config/nwtest/nedb-data/something.db
var Datastore = require('nedb')
  , path = require('path')
  , db = new Datastore({ filename: path.join(require('nw.gui').App.dataPath, 'something.db') });


// Of course you can create multiple datastores if you need several
// collections. In this case it's usually a good idea to use autoload for all collections.
db = {};
db.users = new Datastore('path/to/users.db');
db.robots = new Datastore('path/to/robots.db');

// You need to load each database (here we do it asynchronously)
db.users.loadDatabase();
db.robots.loadDatabase();

Persistence

Under the hood, NeDB's persistence uses an append-only format, meaning that all updates and deletes actually result in lines added at the end of the datafile, for performance reasons. The database is automatically compacted (i.e. put back in the one-line-per-document format) every time you load each database within your application.

You can manually call the compaction function with yourDatabase.persistence.compactDatafile which takes no argument. It queues a compaction of the datafile in the executor, to be executed sequentially after all pending operations.

You can also set automatic compaction at regular intervals with yourDatabase.persistence.setAutocompactionInterval(interval), interval in milliseconds (a minimum of 5s is enforced), and stop automatic compaction with yourDatabase.persistence.stopAutocompaction().

Keep in mind that compaction takes a bit of time (not too much: 130ms for 50k records on a typical development machine) and no other operation can happen when it does, so most projects actually don't need to use it.

Compaction will also immediately remove any documents whose data line has become corrupted, assuming that the total percentage of all corrupted documents in that database still falls below the specified corruptAlertThreshold option's value.

Durability works similarly to major databases: compaction forces the OS to physically flush data to disk, while appends to the data file do not (the OS is responsible for flushing the data). That guarantees that a server crash can never cause complete data loss, while preserving performance. The worst that can happen is a crash between two syncs, causing a loss of all data between the two syncs. Usually syncs are 30 seconds appart so that's at most 30 seconds of data. This post by Antirez on Redis persistence explains this in more details, NeDB being very close to Redis AOF persistence with appendfsync option set to no.

Inserting documents

The native types are String, Number, Boolean, Date and null. You can also use arrays and subdocuments (objects). If a field is undefined, it will not be saved (this is different from MongoDB which transforms undefined in null, something I find counter-intuitive).

If the document does not contain an _id field, NeDB will automatically generated one for you (a 16-characters alphanumerical string). The _id of a document, once set, cannot be modified.

Field names cannot begin by '$' or contain a '.'.

var doc = { hello: 'world'
               , n: 5
               , today: new Date()
               , nedbIsAwesome: true
               , notthere: null
               , notToBeSaved: undefined  // Will not be saved
               , fruits: [ 'apple', 'orange', 'pear' ]
               , infos: { name: 'nedb' }
               };

db.insert(doc, function (err, newDoc) {   // Callback is optional
  // newDoc is the newly inserted document, including its _id
  // newDoc has no key called notToBeSaved since its value was undefined
});

You can also bulk-insert an array of documents. This operation is atomic, meaning that if one insert fails due to a unique constraint being violated, all changes are rolled back.

db.insert([{ a: 5 }, { a: 42 }], function (err, newDocs) {
  // Two documents were inserted in the database
  // newDocs is an array with these documents, augmented with their _id
});

// If there is a unique constraint on field 'a', this will fail
db.insert([{ a: 5 }, { a: 42 }, { a: 5 }], function (err) {
  // err is a 'uniqueViolated' error
  // The database was not modified
});

Finding documents

Use find to look for multiple documents matching you query, or findOne to look for one specific document. You can select documents based on field equality or use comparison operators ($lt, $lte, $gt, $gte, $in, $nin, $ne). You can also use logical operators $or, $and, $not and $where. See below for the syntax.

You can use regular expressions in two ways: in basic querying in place of a string, or with the $regex operator.

You can sort and paginate results using the cursor API (see below).

You can use standard projections to restrict the fields to appear in the results (see below).

Basic querying

Basic querying means are looking for documents whose fields match the ones you specify. You can use regular expression to match strings. You can use the dot notation to navigate inside nested documents, arrays, arrays of subdocuments and to match a specific element of an array.

```javascript // Let's say our datastore contains the following collection // { _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false, satellites: ['Phobos', 'Deimos'] } // { _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true, humans: { genders: 2, eyes: true } } // { _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false } // { _id: 'id4', planet: 'Omicron Persei 8', system: 'futurama', inhabited: true, humans: { genders: 7 } } // { _id: 'id5', completeData: { planets: [ { name: 'Earth', number: 3 }, { name: 'Mars', number: 2 }, { name: 'Pluton', number: 9 } ] } }

// Finding all planets in the solar system db.find({ system: 'solar' }, function (err, docs) { // docs is an array containing documents Mars, Earth, Jupiter // If no document is found, docs is equal to [] });

// Finding all planets whose name contain the substring 'ar' using a regular expression db.find({ planet: /ar/ }, function (err, docs) { // docs contains Mars and Earth });

// Finding all inhabited planets in the solar system db.find({ system: 'solar', inhabited: true }, function (err, docs) { // docs is an array containing document Earth only });

// Use the dot-notation to match fields in subdocuments db.find({ "humans.genders": 2 }, function (err, docs) { // docs contains Earth });

// Use the dot-notation to navigate arrays of subdocuments db.find({ "completeData.planets.name": "Mars" }, function (err, docs) { // docs contains document 5 });

db.find({ "completeData.planets.name": "Jupiter" }, function (err, docs) { // docs is empty });

db.find({ "completeData.planets.0.name": "Earth" }, function (err, docs) { // docs contains document 5 // If we had tested against "Mars" docs would be empty because we are matching against a specific array element