edemaine:file-collection
This project is a fork/rewrite of vsivsi:file-collection. See HISTORY.md for a summary of the changes, especially since V2 where much of the code and API changed.
Introduction
file-collection is a Meteor.js package that cleanly extends Meteor's Collection metaphor to efficiently manage collections of files and their data, backed by MongoDB's GridFS so everything ends up in your Mongo database. File Collections are fully reactive, and if you know how to use Meteor Collections, you already know most of what you need to begin working with this package.
Major features:
- HTTP upload and download including support for Meteor authentication
- Client and server integration of resumable.js for robust chunked uploading
- Also compatible with traditional HTTP POST or PUT file uploading
- HTTP range requests support random access for resumable downloads, media seeking, etc.
- External changes to the underlying file store automatically synchronize with the Meteor collection
- Designed for efficient handling of millions of small files as well as huge files 10GB and above
These features (and more) are possible because file-collection tightly integrates MongoDB GridFS with Meteor Collections, without any intervening plumbing or unnecessary layers of abstraction.
Quick server-side example
1myFiles = new FileCollection('myFiles', 2 { resumable: true, // Enable built-in resumable.js chunked upload support 3 http: [ // Define HTTP route 4 { method: 'get', // Enable a GET endpoint 5 path: '/:md5', // this will be at route "/gridfs/myFiles/:md5" 6 lookup: function (params, query) { // uses express style url params 7 return { md5: params.md5 }; // a query mapping url to myFiles 8}}]}); 9 10// You can add publications and allow/deny rules here to securely 11// access myFiles from clients. 12// On the server, you can access everything without limitation: 13 14// Find a file document by name 15thatFile = myFiles.findOne({ filename: 'lolcat.gif' }); 16 17// or get a file's data as a node.js Stream2 18thatFileStream = myFiles.findOneStream({ filename: 'lolcat.gif' }); 19 20// Easily remove a file and its data 21result = myFiles.remove(thatFile._id);
Feature summary
Under the hood, file data is stored entirely within the Meteor MongoDB instance using a Mongo technology called GridFS. Your file collection and the underlying GridFS collection remain perfectly in sync because they are the same collection. The file-collection package also provides a simple way to enable secure HTTP (GET, POST, PUT, DELETE) interfaces to your files, and additionally has built-in support for robust and resumable file uploads using the excellent Resumable.js library.
Related work
A major other library for supporting file uploads for Meteor is Meteor-Files. The main difference is that file-collection makes it really easy to use GridFS (and that's all it supports), whereas Meteor-Files requires significant extra code to integrate, especially for advanced features like HTTP streaming. So while Meteor-Files is a more flexible solution, file-collection is easier to get up and running with your existing MongoDB database.
Example
The block below implements a FileCollection
on server, including support for owner-secured HTTP file upload using Resumable.js
and HTTP download. It also sets up the client to provide drag and drop chunked file uploads to the collection. The only things missing here are UI templates and some helper functions.
1// Create a file collection, and enable file upload and download using HTTP 2myFiles = new FileCollection('myFiles', 3 { resumable: true, // Enable built-in resumable.js upload support 4 http: [ 5 { method: 'get', 6 path: '/:md5', // this will be at route "/gridfs/myFiles/:md5" 7 lookup: function (params, query) { // uses express style url params 8 return { md5: params.md5 }; // a query mapping url to myFiles 9 } 10 } 11 ] 12 } 13); 14 15if (Meteor.isServer) { 16 17 // Only publish files owned by this userId, and ignore 18 // file chunks being used by Resumable.js for current uploads 19 Meteor.publish('myData', 20 function (clientUserId) { 21 if (clientUserId === this.userId) { 22 return myFiles.find({ 'metadata._Resumable': { $exists: false }, 23 'metadata.owner': this.userId }); 24 } else { // Prevent client race condition: 25 return null; // This is triggered when publish is rerun with a new 26 // userId before client has resubscribed with that userId 27 } 28 } 29 ); 30 31 // Allow rules for security. Should look familiar! 32 // Without these, no file writes would be allowed 33 myFiles.allow({ 34 // The creator of a file owns it. UserId may be null. 35 insert: function (userId, file) { 36 // Assign the proper owner when a file is created 37 if (!file.metadata) file.metadata = {}; 38 file.metadata.owner = userId; 39 return true; // always allow uploading 40 return Boolean(userId) // or require being logged 41 return (file.metadata.cool === true) // or depend on metadata 42 }, 43 // Only owners can delete a file 44 remove: function (userId, file) { 45 return (userId === file.metadata.owner); 46 }, 47 // Only owners can retrieve a file via HTTP GET 48 read: function (userId, file) { 49 return (userId === file.metadata.owner); 50 }, 51 }); 52} 53 54if (Meteor.isClient) { 55 56 Meteor.startup(function() { 57 58 // This assigns a file upload drop zone to some DOM node 59 myFiles.resumable.assignDrop($(".fileDrop")); 60 61 // This assigns a browse action to a DOM node 62 myFiles.resumable.assignBrowse($(".fileBrowse")); 63 64 // When a file is added via drag and drop 65 myFiles.resumable.on('fileAdded', function (file) { 66 // Assign custom metadata if needed 67 file.metadata = { 68 cool: true 69 }; 70 71 // Start uploading 72 myFiles.resumable.upload(); 73 }); 74 75 // This autorun keeps a cookie up-to-date with the Meteor Auth token 76 // of the logged-in user. This is needed so that the read/write allow 77 // rules on the server can verify the userId of each HTTP request. 78 Tracker.autorun(function () { 79 // Sending userId prevents a race condition 80 Meteor.subscribe('myData', Meteor.userId()); 81 // $.cookie() assumes use of "jquery-cookie" Atmosphere package. 82 // You can use any other cookie package you may prefer... 83 $.cookie('X-Auth-Token', Accounts._storedLoginToken(), { path: '/' }); 84 }); 85 }); 86}
For more complete examples, see:
- Coauthor is an actively maintained use of file-collection (including userId+group-based authentication via metadata when uploading files).
- meteor-file-sample-app is old and unmaintained, so may need tweaking to get working again.
Installation
To add to your project, run:
meteor add edemaine:file-collection
The package exposes a global object FileCollection
on both client and server.
To run tests (using Meteor tiny-test):
git clone --recursive https://github.com/edemaine/meteor-file-collection FileCollection cd FileCollection meteor test-packages ./
Load http://localhost:3000/
and the tests should run in your browser and on the server.
Use
Below you'll find the MongoDB GridFS files
data model. This is also the schema used by file-collection because a FileCollection is a GridFS collection.
1{ 2 "_id" : <ObjectId>, 3 "length" : <number>, 4 "chunkSize" : <number> 5 "uploadDate" : <Date> 6 "md5" : <string> 7 8 "filename" : <string>, 9 "contentType" : <string>, 10 "aliases" : <array of strings>, 11 "metadata" : <object> 12}
Here are a few things to keep in mind about the GridFS file data model:
- Some of the attributes belong to GridFS, and you may lose data if you mess around with these.
- For this reason,
_id
,length
,chunkSize
,uploadDate
andmd5
are read-only. - Some of the attributes belong to you. Your application can do whatever you want with them.
filename
,contentType
,aliases
andmetadata
are yours. Go to town.contentType
should probably be a valid MIME Typefilename
is not guaranteed unique._id
is a better bet if you want to be sure of what you're getting.
Sound complicated? It really isn't and file-collection is here to help.
First off, when you create a new file you use myFiles.insertStream(...)
or myFiles.insertEmpty(...)
and just populate whatever attributes you care about. The file-collection package does the rest. You are guaranteed to get a valid GridFS file.
Likewise, when you run myFiles.update(...)
on the server, file-collection tries really hard to make sure that you aren't clobbering one of the "read-only" attributes with your update modifier. For safety, clients are never allowed to directly update
, although you can selectively give them that power via Meteor.methods
.
Limits and performance
There are essentially no hard limits on the number or size of files other than what your hardware will support.
At no point in normal operation is a file-sized data buffer ever in memory. All of the file data import/export mechanisms are stream based, so even very active servers should not see much memory dedicated to file transfers.
File data is never copied within a collection. During chunked file uploading, file chunk references are changed, but the data itself is never copied. This makes file-collection particularly efficient when handling multi-gigabyte files.
file-collection offers no locking, so you shouldn't modify files after uploading them; just read from them (or delete them).
Security
You may have noticed that the GridFS files
data model says nothing about file ownership. That's your job. If you look again at the example code block above, you will see a bare bones Meteor.userId
based ownership scheme implemented with the attribute file.metadata.owner
. As with any Meteor Collection, allow/deny rules are needed to enforce and defend that document attribute, and file-collection implements that in almost the same way that ordinary Meteor Collections do. Here's how they're a little different:
- The
insert
allow/deny rules work just as you would expect for clientinsertEmpty
calls, but more likely you want to upload a nonempty file via POST/PUT or Resumable, which are similarly protected by theinsert
rule. - The
remove
allow/deny rules work just as you would expect for client calls, and they also secure the HTTP DELETE method when it's used. - The
read
allow/deny rules secure access to file data requested via HTTP GET. These rules have no effect on clientfind()
orfindOne()
methods; these operations are secured byMeteor.publish()
as with any meteor collection. - There are no
update
allow/deny rules because clients are always prohibited from directly updating a file document's attributes. - All HTTP methods are disabled by default. When enabled, they can be authenticated to a Meteor
userId
by using a currently valid authentication token passed either in the HTTP request header or using an HTTP Cookie.
API
The FileCollection
API is essentially an extension of the Meteor Collection API, with many similar methods and a few new file-specific ones mixed in.
The big losers are insert()
and upsert()
, which are not supported by FileCollection
. If you try to call them, you'll get an error. Instead, you might want insertEmpty()
or insertStream
. update()
is also disabled on the client side, but it can be safely used on the server to implement Meteor.methods
calls for clients to use.
fc = new FileCollection([name], [options])
Create a new FileCollection
object - Server and Client
1 2// create a new FileCollection with all default values 3 4fc = new FileCollection('fs', // base name of collection 5 { resumable: false, // Disable resumable.js upload support 6 resumableIndexName: undefined, // Not used when resumable is false 7 chunkSize: 2*1024*1024 - 1024, // Use 2MB chunks for GridFS and resumable 8 baseURL: '\gridfs\fs', // Default base URL for all HTTP methods 9 http: [] // HTTP method definitions, none by default 10 } 11);
Note: The same FileCollection
call should be made on both the client and server.
name
is the root name of the underlying MongoDB GridFS collection. If omitted, it defaults to 'fs'
, the default GridFS collection name. Internally, three collections are used for each FileCollection
instance:
[name].files
- This is the collection you actually see when using file-collection[name].chunks
- This collection contains the actual file data chunks. It is managed automatically.[name].locks
is no longer used since v2; you can drop this collection.
FileCollection
is a subclass of Meteor.Collection
, however it doesn't support the same [options]
.
Meteor Collections support connection
, idGeneration
and transform
options. Currently, file-collection only supports the default Meteor server connection, although this may change in the future. All _id
values used by FileCollection
are MongoDB style IDs. The Meteor Collection transform functionality is unsupported in FileCollection
.
Here are the options FileCollection
does support:
options.resumable
-<boolean>
Whentrue
, exposes the Resumable.js API on the client and the matching resumable HTTP support on the server.options.resumableIndexName
-<string>
When provided andoptions.resumable
istrue
, this value will be the name of the internal-use MongoDB index that the server-side resumable.js support attempts to create. This is useful because the default index name MongoDB creates is long (94 chars out of a total maximum namespace length of 127 characters), which may create issues when combined with long collection and/or database names. If this collection already exists the first time an application runs using this setting, it will likely have no effect because an identical index will already exist (under a different name), causing MongoDB to ignore request to create a duplicate index with a different name. In this case, you must manually drop the old index and then restart your application to generate a new index with the requested name.options.chunkSize
-<integer>
Sets the GridFS and Resumable.js chunkSize in bytes. The default value of a little less than 2MB is probably a good compromise for most applications, with the maximum being 8MB - 1. Partial chunks are not padded, so there is no storage space benefit to using small chunk sizes. If you are uploading very large files over a fast network and upload spped matters, then achunkSize
of 8MB - 1KB (= 8387584) will likly optimize upload speed. However, if you elect to use such largechunkSize
values, make sure that the replication oplog of your MongoDB instance is large enough to handle this, or you will risk having your client and server collections lose synchronization during uploads. Meteor's development mode only uses an oplog of 8 MB, which will almost certainly cause problems for high speed uploads to apps using a largechunkSize
.
For more information on Meteor's use of the MongoDB oplog, see: Meteor livequery.
options.baseURL
-<string>
Sets the base route for all HTTP interfaces defined on this collection. Default value is/gridfs/[name]
option.maxUploadSize
-<integer>
Maximum number of bytes permitted for any HTTP POST, PUT or resumable.js file upload.option.http
-<array of objects>
HTTP interface configuration objects, described below:
Configuring HTTP methods
Each object in the option.http
array defines one HTTP request interface on the server, and has these three attributes:
obj.method
-<string>
The HTTP request method to define, one ofget
,post
,put
,delete
(oroptions
with a custom handler).obj.path
-<string>
An express.js style route path with parameters. This path will be added to the path specified byoptions.baseURL
.obj.lookup
-<function>
A function that is called when an HTTP request matches themethod
andpath
. It is provided with the values of the route parameters and any URL query parameters, and it should return either:- GET/HEAD/OPTIONS/DELETE: a mongoDB query object which can be used to find a file that matches those parameters
- POST/PUT: a
file
object forinsertStream
, which can include_id
,aliases
,chunkSizeBytes
,contentType
, andmetadata
, butcontentType
is set automatically from HTTPContent-Type
header. For POST requests, the lookup function is also provided any with MIME/multipart parameters and other file information from the multipart headers.
obj.handler
-<function>
OPTIONAL! This is an advanced feature that allows the developer to provide a custom "express.js style" request handler to satisfy requests for this specific request interface. For an example of how this works, please see the resumable.js upload support implementation in the source fileresumable_server.coffee
.
When arranging http interface definition objects in the array provided to options.http
, be sure to put more specific paths for a given HTTP method before more general ones. For example: /hash/:md5
should come before /:filename/:_id
because "hash"
would match to filename, and so /hash/:md5
would never match if it came second. Obviously this is a contrived example to demonstrate that order is significant.
Note that an authenticated userId is not provided to the lookup
function. UserId-based permissions should be managed using the allow/deny rules described later on.
Here are some example HTTP interface definition objects to get you started:
1// GET file data by md5 sum 2{ method: 'get', 3 path: '/hash/:md5', 4 lookup: function (params, query) { 5 return { md5: params.md5 } } } 6 7// DELETE a file by _id. Note that the URL parameter ":_id" is a special 8// case, in that it will automatically be converted to a Meteor ObjectID 9// in the passed params object. 10{ method: 'delete', 11 path: '/:_id', 12 lookup: function (params, query) { 13 return { _id: params._id } } } 14 15// GET a file based on a filename or alias name value 16{ method: 'get', 17 path: '/name/:name', 18 lookup: function (params, query) { 19 return {$or: [ {filename: params.name }, 20 {aliases: {$in: [ params.name ]}} ]} }} 21 22// PUT data to a file based on _id and a secret value stored as metadata 23// where the secret is supplied as a query parameter e.g. ?secret=sfkljs 24{ method: 'put', 25 path: '/write/:_id', 26 lookup: function (params, query) { 27 return { _id: params._id, "metadata.secret": query.secret} }} 28 29 30// POST data to a file based on _id and a secret value stored as metadata 31// where the secret is supplied as a MIME/Multipart parameter 32{ method: 'post', 33 path: '/post/:_id', 34 lookup: function (params, query, multipart) { 35 return { _id: params._id, "metadata.secret": multipart.params.secret} }} 36 37// GET a file based on a query type and numeric coordinates metadata 38{ method: 'get', 39 path: '/tile/:z/:x/:y', 40 lookup: function (params, query) { 41 return { "metadata.x": parseInt(params.x), // Note that all params 42 "metadata.y": parseInt(params.y), // (execept _id) are strings 43 "metadata.z": parseInt(params.z), 44 contentType: query.type} }}
CORS / Apache Cordova Support
The HTTP access in file-collection can be configured for compatibility with Cross Origin Resource Sharing (CORS) via use of a custom handler for the 'options'
request method.
This provides a simple way to support accessing file-collection files in Apache Cordova client applications:
1myFiles = new FileCollection('myFiles', 2 { resumable: true, // Enable built-in resumable.js chunked upload support 3 http: [ // Define HTTP route 4 { method: 'get', // Enable a GET endpoint 5 path: '/:md5', // this will be at route "/gridfs/myFiles/:md5" 6 lookup: function (params, query) { // uses express style url params 7 return { md5: params.md5 }; // a query mapping url to myFiles 8 }, 9 handler: function (req, res, next) { 10 if (req.headers && req.headers.origin) { 11 res.setHeader('Access-Control-Allow-Origin', 'http://meteor.local'); // For Cordova 12 res.setHeader('Access-Control-Allow-Credentials', true); 13 } 14 next(); 15 } 16 }, 17 { method: 'put', // Enable a PUT endpoint 18 path: '/:md5', // this will be at route "/gridfs/myFiles/:md5" 19 lookup: function (params, query) { // uses express style url params 20 return { md5: params.md5 }; // a query mapping url to myFiles 21 }, 22 handler: function (req, res, next) { 23 if (req.headers && req.headers.origin) { 24 res.setHeader('Access-Control-Allow-Origin', 'http://meteor.local'); // For Cordova 25 res.setHeader('Access-Control-Allow-Credentials', true); 26 } 27 next(); 28 } 29 }, 30 { method: 'options', // Enable an OPTIONS endpoint (for CORS) 31 path: '/:md5', // this will be at route "/gridfs/myFiles/:md5" 32 lookup: function (params, query) { // uses express style url params 33 return { md5: params.md5 }; // a query mapping url to myFiles 34 }, 35 handler: function (req, res, next) { // Custom express.js handler for OPTIONS 36 res.writeHead(200, { 37 'Content-Type': 'text/plain', 38 'Access-Control-Allow-Origin': 'http://meteor.local', // For Cordova 39 'Access-Control-Allow-Credentials': true, 40 'Access-Control-Allow-Headers': 'x-auth-token, user-agent', 41 'Access-Control-Allow-Methods': 'GET, PUT' 42 }); 43 res.end(); 44 return; 45 } 46 } 47 ] 48 } 49);
If using resumable endpoint use this instead:
1myFiles = new FileCollection('myFiles', 2 { resumable: true, // Enable built-in resumable.js chunked upload support 3 http: [ // Define HTTP route 4 { 5 method: 'POST', // Enable a POST endpoint 6 path: '/_resumable', // this will be at route "/gridfs/images/_resumable" 7 lookup: function (params, query) { // uses express style url params 8 return {}; // a dummy query 9 }, 10 handler: function (req, res, next) { 11 if (req.headers && req.headers.origin) { 12 res.setHeader('Access-Control-Allow-Origin', req.headers.origin); // For Cordova 13 res.setHeader('Access-Control-Allow-Credentials', true); 14 } 15 next(); 16 } 17 }, 18 { 19 method: 'head', // Enable an HEAD endpoint (for CORS) 20 path: '/_resumable', // this will be at route "/gridfs/images/_resumable/" 21 lookup: function (params, query) { // uses express style url params 22 return { }; // a dummy query 23 }, 24 handler: function (req, res, next) { // Custom express.js handler for HEAD 25 if (req.headers && req.headers.origin) { 26 res.setHeader('Access-Control-Allow-Origin', req.headers.origin); // For Cordova 27 res.setHeader('Access-Control-Allow-Credentials', true); 28 } 29 next(); 30 } 31 }, 32 { 33 method: 'options', // Enable an OPTIONS endpoint (for CORS) 34 path: '/_resumable', // this will be at route "/gridfs/images/_resumable/" 35 lookup: function (params, query) { // uses express style url params 36 return { }; // a dummy query 37 }, 38 handler: function (req, res, next) { // Custom express.js handler for OPTIONS 39 res.writeHead(200, { 40 'Content-Type': 'text/plain', 41 'Access-Control-Allow-Origin': req.headers.origin, // For Cordova 42 'Access-Control-Allow-Credentials': true, 43 'Access-Control-Allow-Headers': 'x-auth-token, user-agent', 44 'Access-Control-Allow-Methods': 'GET, POST, HEAD, OPTIONS' 45 }); 46 res.end(); 47 return; 48 } 49 } 50 ] 51 } 52);
Note!: Reportedly due to a bug in Cordova, you need to add the following line into your mobile-config.js
App.accessRule("blob:*");
Please notice that this package will only work with "blob" types when using resumable on Cordova enviroment. If you are using a "file" type remember to convert it to blob before the upload.
HTTP authentication
Authentication of HTTP requests is performed using Meteor login tokens. When Meteor Accounts are used in an application, a logged in client can see its current token using Accounts._storedLoginToken()
. Tokens are passed in HTTP requests using either the HTTP header X-Auth-Token: [token]
or using an HTTP cookie named X-Auth-Token=[token]
. If the token matches a valid logged in user, then that userId will be provided to any allow/deny rules that are called for permission for an action.
For non-Meteor clients that aren't logged-in humans using browsers, it is possible to authenticate with Meteor using the DDP protocol and programmatically obtain a token. See the ddp-login npm package for a node.js library and command-line utility capable of logging into Meteor (similar libraries also exist for other languages such as Python).
HTTP request behaviors
URLs used to HTTP GET file data within a browser can be configured to automatically trigger a "File SaveAs..." download by using the ?download=true
query in the request URL. Similarly, if the ?filename=[filename.ext]
query is used, a "File SaveAs..." download will be invoked, but using the specified filename as the default, rather than the GridFS filename
as is the case with ?download=true
.
To cache files in the browser use the ?cache=172800
query in the request URL, where 172800 (48h) is the time in seconds. This will set the header response information to cache-control:max-age=172800, private
. Caching is useful when streaming videos or audio files to avoid unwanted calls to the server.
HTTP PUT requests write the data from the request body directly into the file. By contrast, HTTP POST requests assume that the body is formatted as MIME multipart/form-data (as an old-school browser form based file upload would generate), and the data written to the file is taken from the part named "file"
. Below are example cURL commands that successfully invoke each of the four possible HTTP methods.
# This assumes a baseURL of '/gridfs/fs' and method definitions with a path # of '/:_id' for each method, for example: # { method: 'delete', # path: '/:_id', # lookup: function (params, query) { # return { _id: params._id } } } # The file with _id = 38a14c8fef2d6cef53c70792 must exist for these to succeed. # The auth token should match a logged-in userId # GET the file data curl -X GET 'http://127.0.0.1:3000/gridfs/fs/38a14c8fef2d6cef53c70792' \ -H 'X-Auth-Token: 3pl5vbN_ZbKDJ1ko5JteO3ZSTrnQIl5g6fd8XW0U4NQ' # POST with file in multipart/form-data curl -X POST 'http://127.0.0.1:3000/gridfs/fs/38a14c8fef2d6cef53c70792' \ -F 'file=@"lolcat.gif";type=image/gif' \ -H 'X-Auth-Token: 3pl5vbN_ZbKDJ1ko5JteO3ZSTrnQIl5g6fd8XW0U4NQ' # PUT with file in request body curl -X PUT 'http://127.0.0.1:3000/gridfs/fs/38a14c8fef2d6cef53c70792' \ -H 'Content-Type: image/gif' \ -H 'X-Auth-Token: 3pl5vbN_ZbKDJ1ko5JteO3ZSTrnQIl5g6fd8XW0U4NQ' \ -T "lolcat.gif" # DELETE the file curl -X DELETE 'http://127.0.0.1:3000/gridfs/fs/38a14c8fef2d6cef53c70792' \ -H 'X-Auth-Token: 3pl5vbN_ZbKDJ1ko5JteO3ZSTrnQIl5g6fd8XW0U4NQ'
Below are the methods defined on the returned FileCollection
object
fc.resumable
Resumable.js API object - Client only
1fc.resumable.assignDrop($(".fileDrop")); // Assign a file drop target 2 3// When a file is dropped on the target (or added some other way) 4myData.resumable.on('fileAdded', function (file) { 5 // file contains a resumable.js file object, do something with it... 6}
fc.resumable
is a ready to use, preconfigured Resumable
object that is available when a FileCollection
is created with options.resumable == true
. fc.resumable
contains the results of calling new Resumable([options])
where all of the options have been specified by file-collection to work with its server side support. See the Resumable.js documentation for more details on how to use it.
fc.find(selector, [options])
Find any number of files - Server and Client
1// Count the number of likely lolcats in collection, this is reactive 2lols = fc.find({ 'contentType': 'image/gif'}).count();
fc.find()
is identical to Meteor's Collection.find()
fc.findOne(selector, [options])
Find a single file. - Server and Client
1// Grab the file document for a known lolcat 2// This is not the file data, see fc.findOneStream() for that! 3myLol = fc.findOne({ 'filename': 'lolcat.gif'});
fc.findOne()
is identical to Meteor's Collection.findOne()
fc.insertEmpty([file], [callback])
Insert a new zero-length file. - Server and Client
1// Create a new zero-length file in the collection 2// All fields are optional and will get defaults if omitted 3_id = fc.insert({ 4 _id: new Meteor.Collection.ObjectID(), 5 filename: 'nyancat.flv', 6 contentType: 'video/x-flv', 7 metadata: { owner: 'posterity' }, 8 aliases: [ ] 9 } 10 // Callback here, if you really care... 11);
fc.insertEmpty()
is similar as Meteor's Collection.insert()
, except that the document is forced to be a GridFS files
document, and it's not possible to specify the file contents (the file is permanently empty). All attributes not supplied get default values, non-GridFS attributes are silently dropped. Inserts from the client that do not conform to the GridFS data model will automatically be denied. Client inserts will additionally be subjected to any 'insert'
allow/deny rules (which default to deny all inserts).
fc.remove(selector, [callback])
Remove a file and all of its data. - Server and Client
1// Make it go away, data and all 2fc.remove( 3 { filename: 'nyancat.flv' } 4 // Callback here, if you want to be absolultely sure it's really gone... 5);
fc.remove()
is nearly the same as Meteor's Collection.remove()
, except that in addition to removing the file document, it also removes the file data chunks from the GridFS store. For safety, undefined and empty selectors (undefined
, null
or {}
) are all rejected. Client calls are subjected to any 'remove'
allow/deny rules (which default to deny all removes). Returns the number of documents actually removed on the server, except when invoked on the client without a callback. In that case it returns the simulated number of documents removed from the local mini-mongo store.
fc.update(selector, modifier, [options], [callback])
Update application controlled GridFS file attributes. - Server only
Note: A local-only version of update is available on the client. See docs for fc.localUpdate()
for details.
1// Update some attributes we own 2fc.update( 3 { filename: 'keyboardcat.mp4' }, 4 { 5 $set: { 'metadata.comment': 'Play them off...' } }, 6 $push: { aliases: 'Fatso.mp4' } 7 } 8 // Optional options here 9 // Optional callback here 10);
fc.update()
is nearly the same as Meteor's Collection.update()
, except that it is a server-only method, and it will return an error if:
- any of the GridFS "read-only" attributes would be modified
- any standard GridFS document level attributes would be removed
- the
upsert
option is attempted
Since fc.update()
only runs on the server, it is not subjected to any allow/deny rules.
fc.localUpdate(selector, modifier, [options], [callback])
Update local minimongo file attributes. - Client only
Warning! Changes made using this function do not persist to the server! You must implement your own Meteor methods to perform persistent updates from a client. For example:
1// Implement latency compensated update using Meteor methods and localUpdate 2Meteor.methods({ 3 updateFileComment: function (fileId, comment) { 4 // Always check method params! 5 check(fileId, Mongo.ObjectID); 6 check(comment, Match.Where(function (x) { 7 check(x, String); 8 return x.length <= 140; 9 })); 10 // You'll probably want to do some kind of ownership check here... 11 12 var update = null; 13 // If desired you can avoid this by initializing fc.update 14 // on the client to be fc.localUpdate 15 if (this.isSimulation) { 16 update = fc.localUpdate; // Client stub updates locally for latency comp 17 } else { // isServer 18 update = fc.update; // Server actually persists the update 19 } 20 // Use whichever function the environment dictates 21 update({ _id: _id }, { 22 $set: { 'metadata.comment': comment } 23 } 24 // Optional options here 25 // Optional callback here 26 ); 27 } 28});
fc.localUpdate()
is nearly the same as Meteor's server-side Collection.update()
, except that it is a client only method, and changes made using it do not propagate to the server. This call is useful for implementing latency compensation in the client UI when performing server updates using a Meteor method. This call can be invoked in the client Method stub to simulate what will be happening on the server. For this reason, this call can perform updates using complex selectors and the multi
option, unlike client side updates on normal Mongo Collections.
It will return an error if:
- any of the GridFS "read-only" attributes would be modified
- any standard GridFS document level attributes would be removed
- the
upsert
option is attempted
Since fc.localUpdate()
only changes data on the client, it is not subjected to any allow/deny rules.
fc.allow(options)
Allow client insert and remove, and HTTP data accesses and updates, subject to your limitations. - Server only
fc.allow(options)
is essentially the same as Meteor's Collection.allow()
, except that the Meteor Collection fetch
and transform
options are not supported by FileCollection
. In addition to returning true/false, rules may also return a (possibly empty) options object to indicate truth while affecting the behavior of the allowed request. See the maxUploadSize
option on 'insert'
allow rules as an example. Note that more than one allow rule may apply to a given request, but unlike deny rules, they are not all guaranteed to run. Allow rules are run in the order in which they are defined, and the first one to return a truthy value wins, which can be significant if they return options or otherwise modify state.
insert
rules are similar to ordinary Meteor collections, but they also
apply to HTTP PUT/POST requests for uploading files. insert
rules may optionally return an object with a positive integer maxUploadSize
attribute instead of true
. This indicates the maximum allowable upload size for this request. If this max upload size is provided, it will override any value provided for the maxUploadSize
option on the fileCollection as a whole. Nonpositive values of maxUploadSize
mean there will be no upload size limit for this request.
remove
rules also apply to HTTP DELETE requests.
In addition to Meteor's insert
and remove
rules, file-collection also uses read
rules.
These are used to secure access to file data via HTTP GET/HEAD requests for retrieving file data.
read
have the same parameters as all other rules.
The parameters for callback functions for all three types of allow/deny rules are the same:
1function (userId, file) { 2 // userId is Meteor account if authenticated 3 // file is the GridFS file record for the matching file, 4 // or for insert rules, the initial data describing the file 5}
fc.deny(options)
Override allow rules. - Server only
1fc.deny({ 2 remove: function (userId, file) { return true; } // Nobody can remove, boo! 3});
fc.deny(options)
is the same as Meteor's Collection.deny()
, except that the Meteor Collection fetch
and transform
options are not supported by FileCollection
. See fc.allow()
above for more deatils.
fc.findOneStream(selector, [options], [callback])
Find a file collection file and return a readable stream for its data. - Server only
1// Get a readable data stream for a known lolcat 2lolStream = fc.findOneStream({ 'filename': 'lolcat.gif'});
fc.findOneStream()
is like fc.findOne()
except instead of returning the files
document for the found file, it returns a Readable stream for the found file's data.
options.range
-- To get partial data from the file, use the range
option to specify an object with start
(inclusive) and end
(exclusive) attributes:
1stream = fc.findOneStream({ 'filename': 'lolcat.gif'}, { range: { start: 100, end: 200 }})
Other available options are options.sort
and options.skip
which have the same behavior as they do for Meteor's Collection.findOne()
.
The returned stream is a Mongo GridFSBucketReadStream.
When the stream has ended, the callback
is called with the GridFS file document.
fc.insertStream(file, [options], [callback])
Create a file collection file and return a writable stream to its data. - Server only
1// Get a writeable data stream to store all that is right and good 2nyanStream = fc.insertStream({ filename: 'nyancat.flv', 3 contentType: 'video/x-flv', 4 metadata: { caption: 'Not again!'} 5 });
fc.insertStream()
is similar to Meteor's Collection.prototype.insert()
, but returning a writable stream for file contents. Optionally, the file
parameter can specify an _id
field to specify what ID to use for the file, but it will fail if another file already has that ID. If no _id
is provided, then a new ID is created automatically. Any application-owned GridFS attributes (filename
, contentType
, aliases
, metadata
) that are present in the file
parameter will be used for the file.
The options
argument is currently ignored; it is allowed just for
interface similarly with Collection.prototype.insert
.
Once that is done, fc.insertStream()
returns a writable stream for the file.
NOTE! Breaking Change! Prior to file-collection v2.0, this function was named upsertStream
and it supported (re)writing data for existing files. insertStream
only works for nonexisting files.
When the write stream has closed, the callback
is called as callback(error, file)
, where file is the GridFS file document following the write.
fc.exportFile(selector, filePath, callback)
Export a file collection file to the local fileSystem. - Server only
1// Write a file to wherever it belongs in the filesystem 2fc.exportFile({ 'filename': 'nyancat.flv'}, 3 '/dev/null', 4 function(err) { 5 // Deal with it 6 });
fc.exportFile()
is a convenience method that pipes the readable stream produced by fc.findOneStream()
into a local file system writable stream.
The selector
parameter works as it does with fc.findOneStream()
. The filePath
is the String directory path and filename in the local filesystem to write the file data to. The value of the filename
attribute in the found GridFS file document is ignored. The callback is mandatory and will be called with a single parameter that will be either an Error
object or null
depending on the success of the operation.
fc.importFile(filePath, file, callback)
Import a local filesystem file into a file collection file. - Server only
1// Read a file into the collection from the filesystem 2fc.importFile('/funtimes/lolcat_183.gif', 3 { filename: 'lolcat_183.gif', 4 contentType: 'image/gif' 5 }, 6 function(err, file) { 7 // Deal with it 8 // Or file contains all of the details. 9 });
fc.importFile()
is a convenience method that pipes a local file system readable stream into the writable stream produced by a call to fc.upsertStream()
.
The file
parameter works as it does with fc.upsertStream()
. The filePath
is the String directory path and filename in the local filesystem of the file to open and copy into the GridFS file. The callback is mandatory and will be called with the same callback signature as fc.upsertStream()
.