Meteor Grid-Factory

Create FilesCollections with GridFS storage. Lightweight. Simple.

With this package you can easily create multiple ostrio:files collections (FilesCollections) that work with MongoDB's GridFS system out-of-the-box.

It can be a real hassle to introduce gridFS as storage to your project. This package aims to abstract common logic into an easy and accessible API while ensuring to let you override anything in case you need a fine-tuned custom behavior.

The abtract factory allows you to create configurations on a higher level that apply to all your FilesCollections, while you still can fine-tune on the collection level. Supports all constructor arguments of FilesCollection.

Why such a specialized package?

This package is designed for projects, that can't rely on third-party storages, because one or more of the following applies:

• there are concerns about privacy or security when using a third party storage
• the application has to be shipped in a "all-in-one" monolith
• the app is intended for intranet use and connection to the "outside" is prohibited
• whatever you can think of....

The use case may be not very common (Meteor + ostrio:files + GridFS) but if it's for you, this package makes file handling much easier and consistent.

What is covered / what is not (yet)

This package has some out-of-the-box functionality that covers the following points

Creation

• [ x ] Creating new FilesCollection instances
• [ x ] Supporting full FilesCollection constructor
• [ x ] Allows to override package internals by explicitly passing the hooks (onAfterUpload etc.)
• [ x ] Code-splitting (server/client)
• [ x ] Using GridFS buckets instead of deprecated gridfs-stream
• [ x ] Adapter for translation
• [ x ] Creating new FilesCollection instances

onBeforeUpload

This package has some some default behavior defined for the onBeforeUpload hook. You can override it completely or hook into it's behavior using the following parameters:

• [ x ] check file size via maxSize (Number)
• [ x ] check file extensions via extensions ([String])
• [ x ] check permissions via validateUser (Function)
• [ x ] check permissions via validateUser (Function)

onAfterUpload

The default behavior for onAfterUpload is to check the mime of the uploaded file and move it to the Grid. Hoever, you can hook into this process, too:

• [ x ] validate user via validateUser (Function)
• [ x ] validate mime via validateMime (Function)
• [ x ] transform additional versions (e.g. thumbnails, converted videos, etc.) via transformVersions (Function)

protected

• [ x ] validate user via validateUser (Function)

interceptDownload

• [ x ] falls back to find a valid version, if request to a non-existent version fails
• [ x ] streams the file from the GridFS bucket
• [ x ] handles errors with an error response
• [ x ] sets the correct content disposition, depending on download query attribute

onBeforeRemove

• [ x ] validate user via validateUser (Function)

afterRemove

• [ x ] removes file, including all versions, from the GridFS and the FilesCollection

Getting started

1. Install this package via

$ meteor add leaonline:files-collection-factory ostrio:files

We decoupled this package from ostrio:files so your host project can manage the versioning.

2. Optionally install packages for mime-check and transformations

If you want to check the mime you can use packages like mmmagic and mime-types to check for the correct mime type. Of course you can implement your mime-check a total different way, too.

$ meteor npm install --save mmmagic mime-types

If you want to transform your images or videos you also need the respective packages for that. Often you will also have to install additional software / packages on your host OS, since the npm packages (e.g. for image magick / graphics magic) are usually just wrappers for the OS-level packages.

3. Import the abstract factory

The package exports different APIs for client and server but you import it the same way on server and client:

1import { createGridFilesFactory } from 'meteor/leaonline:grid-factory'

From here you have to consider the FilesCollection architecture in order to manage access, post processing, removal etc.

4. Create a server side FilesCollection factory

On the server side you can use the following abstract factory api:

1({
2  i18nFactory: Function, // translator function, str => translatedStr
3  fs: Object, // the node file-system module
4  bucketFactory: Function, // a function that returns a gridFS bucket
5  defaultBucket: String, // a default name for the bucket to be used
6  createObjectId: Function, // a function that creates an Object Id by a given GridFS id
7  onError: Function, // logs errors for all collections across all factories
8  debug: Boolean,
9    ...config // all valid config, that can be passed to the FilesCollection server constructor
10}) => Function => FilesCollection

The factory Function that is returned contains the following api:

1({
2  bucketName: String, // override the defaultBucket, if desired
3  maxSize: Number, // number in bytes to limit the maximum size for files of this collection
4  extensions: [String], // a list of supported extensions
5  validateUser: Function, // a Function that checks permission of the current user/file and returns falsy/truthy
6  validateMime: Function, // async Function that checks permission of the current file/mime and returns falsy/truthy
7  transformVersions: Function, // async Function that transforms the file to different versions
8  onError: Function // logs errors, overrides onError from abstract factory
9}) => FilesCollection

Minimal example

The following example shows a minimal abstract GridFactory:

1import { MongoInternals } from 'meteor/mongo'
2import { createGridFilesFactory } from 'meteor/leaonline:grid-factory'
3import { i18n } from '/path/to/i8n'
4import fs from 'fs'
5
6const debug = Meteor.isDevelopment
7const i18nFactory = (...args) => i18n.get(...args)
8const createObjectId = ({ gridFsFileId }) => new MongoInternals.NpmModule.ObjectID(gridFsFileId)
9const bucketFactory = bucketName => 
10  new MongoInternals.NpmModule.GridFSBucket(MongoInternals.defaultRemoteCollectionDriver().mongo.db, { bucketName })
11const defaultBucket = 'fs' // resolves to fs.files / fs.chunks as default
12const onError = error => console.error(error)
13
14const createFilesCollection = createGridFilesFactory({ i18nFactory, fs, bucketFactory, defaultBucket, createObjectId, onError, debug })
15const ProfileImages = createFilesCollection({
16  collectionName: 'profileImages',
17  bucketName: 'images', // put image collections in the 'images' bucket
18  maxSize: 3072000, // 3 MB max
19  validateUser: function (userId, file, type) {
20    // is this a valid and registered user?
21    if (!userId || Meteor.users.find(userId).count() !== 1) {
22      return false
23    }
24    
25    const isOwner = userId === file.userId
26    const isAdmin = ...
27    const isAllowedToDownload =  ...
28    
29    if (type === 'upload') {
30      return Roles.userIsInRole(userId, 'can-upload', 'mydomain.com') // example of using roles
31    }
32    
33    if (type === 'download') {
34      return isOwner || isAdmin || isAllowedToDownload // custom flags
35    }
36    
37    if (type === 'remove') {
38     // allow only owner to remove the file
39     return isOwner || isAdmin
40    }
41    
42    throw new Error('unexpected code reach')
43  }
44})

5. Create a client side factory

On the server side you have less options to pass to the API:

1({
2  i18nFactory: Function, // translator function, str => translatedStr
3  debug: Boolean,
4  ...config // all valid config, that can be passed to the FilesCollection client constructor
5}) => Function => FilesCollection

The factory Function that is returned contains the following api:

1({
2  bucketName: String, // override the defaultBucket, if desired
3  maxSize: Number, // number in bytes to limit the maximum size for files of this collection
4  extensions: [String], // a list of supported extensions
5  validateUser: Function, // a Function that checks permission of the current user/file and returns falsy/truthy
6  validateMime: Function, // async Function that checks permission of the current file/mime and returns falsy/truthy
7  transformVersions: Function, // async Function that transforms the file to different versions
8  onError: Function // logs errors, overrides onError from abstract factory
9}) => FilesCollection

Changelog

• 1.0.2

  • getGridFsFileId fix bug searching fallback versions

• 1.0.1

  • allow skipping user validation for prototyping but raise a server warning
  • README fix export name
  • standardjs lint fix