ground:db

GroundDB is a fast and thin layer providing Meteor offline database and methods - Taking cloud data to the ground.

1  // Return a grounded Meteor.Collection
2  var list = new Ground.Collection('list');

Meteor Collection Interface

GroundDB is like a normal Meteor.Collection - but changes and outstanding methods are cached and resumed. Turn off resume by setting the option resume: false

Live basic debug test

Features:

Light footprint
Broad browser support Chrome, Safari, Firefox and Internet Explorer 9
Fallback to normal Meteor.Collection if no local storage
Resume of changes in collections
Resume of methods
Works offline updating cross window tabs
Support for SmartCollection
Support for offline client-side only databases
Uses EJSON.minify and EJSON.maxify to compress data in localstorage
In the future there will be a customizable conflict handler on the server-side

Current development

Current development is the Meteor 1.2 ES2015 localforage version ground:db@1.0.0-alpha.x branch is tracked in #118 It supports nearly all types of storage even sqlite on cordova and will have better performance and general resumeability - that said it's also an api breaking version.

Creating a Ground.Collection object (variants)

1
2  // Return a grounded Meteor.Collection
3  var list = new Ground.Collection('list');
4
5  or
6
7  // Get the groundDB of existing Meteor.Collection
8  var list = new Meteor.Collection('list');
9  var groundList = new Ground.Collection(list);
10
11  or
12
13  // Ground an existing Meteor.Collection  
14  var list = new Meteor.Collection('list');
15  // just ground the database:
16  Ground.Collection(list);

Example of different patterns. Grounding a Meteor.Collection will attach the cache, resume and cross tabs update offline

Pure client-side offline databases (variants)

Ground.Collection can be applied on client-side only eg.: new Meteor.Collection(null);

1
2  // Creates client-side only database, this one maps on suffix `null`
3  var list = new Ground.Collection(null);
4
5  // Creates client-side only database, this one maps on suffix `list` *(Meteor 0.6.5+)*
6  var list = new Ground.Collection('list', { connection: null });
7  
8  or
9
10  // Get the groundDB of existing Meteor.Collection
11  var list = new Meteor.Collection(null);
12  var groundList = new Ground.Collection(list, 'list');
13
14  or
15
16  // Ground an existing Meteor.Collection  
17  var list = new Meteor.Collection(null);
18  // just ground the database and map on suffix `list`
19  Ground.Collection(list, 'list');

You can only have one grounded collection with name null

Support

Tested on Chrome, Safari, Firefox and IE9 (though appcache is not supported in IE9 tabs are updated when offline) - but all browsers that support localstorage contains a FF safe test of localstorage

If localstorage is not supported the groundDB simply work as a normal Meteor.Collection

Concept

Localstorage is simple and widely supported - but slow - Thats why we only use it for caching databases and methods + trying to limit the read and writes from it.

GroundDB saves outstanding methods and minimongo db into localstorage - The number of saves to localstorage is minimized. Use Ground.resumeMethods

When the app loads GroundDB resumes methods and database changes - made when offline and browser closed.

Ground user details

It's possible to mount an allready existing collection on a groundDB eg.:

1  Ground.Collection(Meteor.users);

The example will keep Meteor.user() returning correct user details - even if offline

Ground SmartCollections

It's possible to ground an allready existing smartCollectin on a groundDB eg.:

1  var mySmartCollection = new SmartCollection('foo');
2  Ground.Collection(mySmartCollection);
3
4or
5
6  var mySmartCollection = Ground.Collection(new SmartCollection('foo'));
7
8  // use the smart collection
9  mySmartCollection.insert(/* stuff */);

Resume of outstanding methods

Database changes and methods will be sent to the server just like normal. The methods are sent to server after relogin - this way this.userId isset when running on the server. In other words: Just like normal

Publish and subscription

Online

Subscription behavior when using GroundDB - When online it's just like normal Meteor so nothing new. If you unsubscribe a collection you can still insert etc. but the data will not be visible on the client.

Offline

When offline the data remains in the local database - since the publish is a server thing. Use the query selector for filtering unwanted data. When reconnected the database will update client subscription and changes will be resumed

Events - client-side

The event api is as follows:

1Ground.lookup = function(collectionName) {};
2Ground.methodResume = function(names, connection) {};
3
4Ground.addListener // Listen to general events
5foo.addListener // Add listener specific to the foo collection
6
7// Reactive status of all subscriptions, ready or not:
8Ground.ready();

DEPRECATED API: ~~Ground.onQuotaExceeded = function() {};~~ ~~Ground.onResumeDatabase = function(name) {};~~ ~~Ground.onResumeMethods = function() {};~~ ~~Ground.onMethodCall = function(methodCall) {};~~ ~~Ground.onCacheDatabase = function(name) {};~~ ~~Ground.onCacheMethods = function() {};~~ ~~Ground.onTabSync = function(key) {};~~ ~~Ground.skipMethods = function(methodsToSkipObject)~~

Cache methods

Use the Ground.methodResume to cache method calls on a collection. It takes the method name or array of names. The connection is optional if not set the default connection is used:

1  // This is how grounddb uses this internally
2  Ground.methodResume([
3    '/' + self.name + '/insert',
4    '/' + self.name + '/remove',
5    '/' + self.name + '/update'
6  ], self.connection);

The Ground.skipMethods is deprecated

Conflict handling IN the works - not ready for use yet

The conflict handling api is as follows:

1Ground.now(); // Returns server timestamp works on both client and server

Additional api

Normally Ground Collections are cleaned up for local only data when subscriptions are ready. But sometimes we might want to local the data later eg. if the db is already populated.

1  var groundList = new Ground.Collection('list', {
2    cleanupLocalData: false
3  });
4
5  // Manually triggering a clean up of local only data
6  // parameter query is optional, if passed, only local items that match query will be removed 
7  groundList.removeLocalOnly(query); 
8
9  // Clear local data - This will empty all local data
10  groundList.clear();

Future

At the moment the conflict resolution is pretty basic last change recieved by server wins. This could be greatly improved by adding a proper conflict handler. For more details look at comment in server.js
Intelligent subscriptions - A way for the groundDB to keep the data most important for the user - and letting less important data go to match quota limit

Contributions

Feel free to send issues, pull requests all is wellcome

Kind regards Morten