Meteor Community Logo

Meteor Community

Packosphere

undefined's Avatar

jam:easy-schema

v1.1.1-alpha300.19•Published 8 months ago

Easy Schema

Easy Schema is an easy way to add schema validation for Meteor apps. It extends the functionality provided by Meteor's check to validate arguments passed to Meteor Methods and validates automatically on the server prior to write operations insert / update / upsert. It also automatically generates a JSON Schema and attaches it to the database's Collection. It's meant to be lightweight and fast. By default, it validates automatically but it is configurable.

This package can be used with jam:method, Meteor.methods, Validated Method, or any other package that includes a way to validate the method via a function. It also has built-in support for Validation Error for friendlier error messages.

Basics

When using this package, you create a schema once for each Collection and attach it with attachSchema. When a method is called, you'll use this package's check function to make sure the arguments passed from the client match what is expected by the schema you defined.

Then, right before the insert / update / upsert to the database, a validation will be automatically performed against the data that will be written to the database. By default, it will also be validated against the JSON Schema attached to the Collection via Mongo's JSON Schema support though you can disable this if you'd like.

Usage

Add the package to your app

meteor add jam:easy-schema

Define a schema and attach it to its Collection

1import { Mongo } from 'meteor/mongo';
2
3export const Todos = new Mongo.Collection('todos');
4
5const schema = {
6  _id: String,
7  text: String,
8  done: Boolean,
9  createdAt: Date,
10  authorId: String,
11  username: String
12};
13
14Todos.attachSchema(schema); // attachSchema is a function that's built into this package

Use the schema with jam:method Methods

1import { createMethod } from 'meteor/jam:method';
2
3export const insertTodo = createMethod({
4  name: 'todos.insert',
5  schema: Todos.schema,
6  run({ text }) {
7    const userId = Meteor.userId(); // can use this.userId instead
8    const todo = {
9      text,
10      done: false,
11      createdAt: new Date(),
12      authorId: userId,
13      username: Meteor.users.findOne(userId).username
14    }
15
16    const todoId = Todos.insert(todo);
17    return todoId;
18  }
19});

See jam:method for more info.

Use the check function with Validated Methods or Meteor.methods

You can use this package with ValidatedMethod or Meteor.methods if you prefer. Use the check function provided by this package.

1import { check } from 'meteor/jam:easy-schema';
2import { ValidatedMethod } from 'meteor/mdg:validated-method';
3
4export const insertTodo = new ValidatedMethod({
5  name: 'todos.insert',
6  validate(args) { // args should be an object that you pass in from the client. If you want to destructure here, then be sure to pass an object into the check function.
7    check(args, Todos.schema); // the package automatically compares the args only against the relative data inside the Todos.schema so no need to pick them out yourself.
8    // if you want, you can also pass in a custom schema, like this:
9    /* check(args, {text: {type: String, min: 1, max: 16}}) */
10  },
11  run({ text }) {
12    const userId = Meteor.userId();
13    if (!userId) {
14      throw new Meteor.Error('not-authorized');
15    }
16
17    const todo = {
18      text,
19      done: false,
20      createdAt: new Date(),
21      authorId: userId,
22      username: Meteor.users.findOne(userId).username
23    }
24
25    const todoId = Todos.insert(todo);
26    return todoId;
27  }
28});

Then import insertTodo method in your UI component and call it like you would any other Validated Method. See their docs for more info.

Defining Schemas

1import { Optional, Any, Integer, AnyOf } from 'meteor/jam:easy-schema';
2
3// Illustrating the various possibilities for a schema
4const schema = {
5  _id: String, // _id can technically be optional with inserts and upserts since it won't be created yet. this is handled automatically.
6  text: String,
7  emails: [String], // an array of strings
8  createdAt: Date,
9  private: Boolean,
10  thing: Number,
11  stuff: Object,
12  int: Integer,
13  digit: {type: Integer, min: 4, max: 12}, // min is >= and max is <=. automatically converted to JSON Schema "minimum / maximum"
14  address: {
15    street_address: Optional(String), // street address is optional
16    city: String,
17    state: {type: String, min: 0, max: 2}, // min is >= and max is <=. automatically converted to JSON Schema "minLength / maxLength"
18  },
19  messages: [{text: String, createdAt: Date}], // array of objects
20  people: [ // an array of objects with additionalProperties: true. additonalProperties is false by default.
21    {type: {name: String, age: Number, arrayOfOptionalBooleans: [Optional(Boolean)]}, additionalProperties: true}
22  ],
23  regexString: {type: String, regex: /.com$/}, // regex supported for Strings. should be a regex literal. automatically converted to JSON Schema "pattern"
24  optionalArray: Optional([String]),
25  optionalObject: Optional({thing: String, optionalString: Optional(String)}),
26  arrayOfInts: [Integer],
27  arrayOfOptionalInts: [Optional(Integer)],
28  arrayOfRegexStrings: [{type: String, regex: /.com$/}],
29  anyOf: AnyOf([String], [Date]), // AnyOf matches one or more of the items. In this example, it matches either an array of Strings or an array of Dates
30  arrayAnyOf: [AnyOf(String, Number)], // matches an array of Strings or an array of Numbers,
31  any: Any // anything, aka a blackbox
32};

Integer

Integer matches only signed 32-bit integers

Optional

By default, everything listed in the schema is assumed to be required. For anything optional, you need to specify it with Optional

1optionalArray: Optional([String])
2optionalObject: Optional({thing: String, optionalString: Optional(String)})
3arrayOfOptionalInts: [Optional(Integer)]

AnyOf

AnyOf matches one or more of the items. If you're coming from Meteor's Match, this is equivalent to Match.OneOf.

1anyOf: AnyOf([String], [Date]) // matches either an array of Strings or an array of Dates
2arrayAnyOf: [AnyOf(String, Number)] // matches an array of Strings or an array of Numbers

Conditions

You can add conditions to validate against. Here's how you do that:

min / max

Strings, Numbers, Integers, Arrays, Objects

min is greater than or equal to and max is less than or equal to. min / max map to the JSON Schema equivalent for the type.

1{type: String, min: 1, max: 16} // a string that is at least 1 character and at most 16 characters
2{type: Number, min: 0.1, max: 9.9} // a number greater than or equal to 0.1 and less than or equal to 9.9
3{type: Integer, min: 10, max: 25} // an integer greater than or equal to 10 and less than or equal to 25
4{type: Array, min: 1, max: 5} // an array with at least one item and no more than 5 items
5{type: [String], min: 1, max: 5} // an array of Strings with at least one item and no more than 5 items
6{type: Object, min: 1, max: 2} // an object with at least one property and no more than 2 properties
7{type: {name: String, age: Optional(Number)}, min: 1, max: 2} // an object with the properties name and age with at least one property and no more than 2 properties

allow

Any Type

You can specify an array of items that are allowed values with allow – it maps to JSON Schema's enum

1{type: String, allow: ['hello', 'hi']}
2{type: Number, allow: [1.2, 6.8, 24.5]}
3{type: Integer, allow: [145, 29]}
4{type: Boolean, allow: [true]}
5{type: Date, allow: [new Date('2021-12-17T03:24:00'), new Date('2022-01-01T03:24:00')]}
6
7// For arrays, recommend using it within the array, e.g. [{type: String, allow: ['hello', 'hi']}] as opposed to {type: [String], allow: [['hello'], ['hi']]}
8[{type: String, allow: ['hello', 'hi']}]
9{type: [String], allow: [['hello'], ['hi']]}
10{type: Array, allow: [['hello'], ['hi']]}
11// 2d arrays are supported too
12{type: [[String, Number]], allow: [['hi', 1], ['bye', 2]]}
13
14// Object examples
15{type: {hi: String, num: Optional(Number)}, allow: [{hi: 'hi', num: 2}]}
16{type: Object, allow: [{hi: 'hi', num: 2}]}

regex

Strings only

regex maps to JSON Schema's pattern.

1{type: String, regex: /.com$/}

unique

Arrays only

unique maps to JSON Schema's uniqueItems.

1{type: [Number], unique: true} // an array of numbers that must be unique, e.g. [1, 2, 3]. [1, 2, 1] would fail.

additionalProperties

Objects only

By default, additionalProperties is false, i.e. what you define in the schema is what is expected to match the data in the db. If you want to accept additionalProperties, you can do that like this:

1{type: {name: String, createdAt: Date}, additionalProperties: true}

where

Any Type

where is a custom function that you can use to validate logic and even create a dependency on another property of your schema. Throw the error message you want as a plain string. You can return true inside where if you want, but it's taken care of for you if you want to keep it concise.

Note: Currently, unlike the other conditions, there isn't a great way to map where to JSON Schema so the where function will not be translated to Mongo JSON Schema.

Here are some examples of how you might use this:

You can make a property conditionally required on its value.

1{
2  // ... //
3  text: {type: Optional(String), where: text => { if (text === 'world') throw EasySchema.REQUIRED }}, // you can also destructure text in the where function if you prefer
4  // ... //
5}

You can make a property of the schema dependent on the value of a sibling property. Important: you must destructure the params.

1{
2  // ... //
3  text: Optional(String),
4  status: {type: Optional(String), where: ({text, status}) => {
5    if (text && !status) throw EasySchema.REQUIRED
6  }},
7  // ... //
8}
1{
2  // ... //
3  password: String,
4  confirmPassword: {type: String, where: ({password, confirmPassword}) => {
5    if (confirmPassword !== password) throw 'Passwords must match'
6  }},
7  // ... //
8}

Blackboxes

In general, it's recommended to specify what you expect but sometimes it's helpful just to validate against a blackbox, i.e. validating the contents is not important or wanted.

1// For blackbox objects, you can do either of these
2{stuff: Object}
3{stuff: {type: Object}} // this can come in handy if you want to use conditions
1// For blackbox arrays, you can do either of these
2{things: Array}
3{things: {type: Array}} // this can come in handy if you want to use conditions
1// For a true blackbox, you can do either of these
2{something: Any}
3{something: {type: Any}}

Embedding Schemas

If you have a schema that repeats, you can define it once and then embed it.

1const addressSchema = {
2  street: String,
3  city: String,
4  state: {type: String, min: 2, max: 2}
5}
6
7const personSchema = {
8  _id: String,
9  name: String,
10  homeAddress: addressSchema,
11  billingAddress: Optional(addressSchema)
12}

Working with Numbers

Currently, numbers like 1 and 1.0 are both considered to be type Integer by the Node Mongo driver. Numbers like 1.2 are considered a Double as you might expect.

Javascript has the Number prototype which is technically a double-precision floating point numeric value but if you do Number.isInteger(1.0) in your console you'll see it’s true. The JSON Schema spec also says that 1.0 should be treated as an Integer.

What this means for you

Basically, if you can ensure that the numbers you'll store will never be or sum to .0 or .0000000 etc, and you don't require precision, then it’s fine to use the Number type, which will be mapped to bsonType: 'double' in the JSON Schema this package generates and attaches to the Mongo Collection. Depending on your situation, you might be better of using Integer or storing the numbers as String and convert them to Numbers when you need to perform some simple math. Otherwise if you use Number and you have a situation where some data of yours sums to, for example 34.000000, the Node Mongo driver will see that as an integer 34 and will complain that it was expecting bsonType: 'double'.

One way to get around this – and prevent raising an error when you're ok using floating numbers – would be to define your schema like:

1const schema = {
2  _id: String,
3  num: AnyOf(Number, Integer)
4}

Precise numbers

If you need precision for your numbers, and want to avoid weird floating point math where 0.1 + 0.2 = 0.30000000000000004 then you should probably store them as Decimal. Here's what Mongo has to say about modeling monetary data.

You can do that by adding the mongo-decimal package: meteor add mongo-decimal

mongo-decimal uses decimal.js so you can refer to its documentation.

Then when defining your schema:

1import { Decimal } from 'meteor/mongo-decimal';
2
3const schema = {
4  _id: String,
5  price: Decimal,
6}

Configuring (optional)

If you like the defaults, then you won't need to configure anything. But there is some flexibility in how you use this package.

By default, an automatic validation will be performed on the server prior to insert / update / upsert operations. If you don't want that, you can turn it off by setting autoCheck to false. The data will still be validated against the JSON Schema but you won't get as friendly of error messages and you won't know that a write fails until it's attempted on the database.

1// in /server/somewhere.js
2`Do not put in /server/main.js. Make sure it's in a different file on the server that is imported at the top of your /server/main.js`
3import { EasySchema } from 'meteor/jam:easy-schema';
4
5EasySchema.configure({autoCheck: false});

If you turn autoCheck off, you can still validate manually by using the check function

1import { check } from 'meteor/jam:easy-schema';
2// Use check inside of a Meteor Method before you perform a db write operation.
3// data - the object you want to validate. For an insert operation, this is the document object you're inserting. In the case of an update / upsert operation, this is the modifier, aka the 2nd argument of the operation.
4// schema - the schema object. Most likely you'll want to pass in the schema attached to your collection, e.g. Todos.schema, but you can also create a custom schema object.
5const data = {$set: {text: 'book flight to hawaii'}}; // example of an update modifier
6const schema = Todos.schema;
7check(data, schema);
8
9// perform db write operation. here's an example with update.
10Todos.update(_id, data);

If you choose to validate manually, you can force a full check against the entire schema by passing in {full: true} to the check function.

1import { check } from 'meteor/jam:easy-schema';
2
3const data = {text: 'hi', ...};
4const schema = Todos.schema;
5check(data, schema, {full: true}); // will perform a full check only on the server

You can also skip an auto check on a one-off basis by calling EasySchema.skipAutoCheck() inside the body of a Meteor Method.

1import { EasySchema } from 'meteor/jam:easy-schema';
2
3// Inside Meteor Method body
4EasySchema.skipAutoCheck();
5
6// optionally check manually
7// perform db write operation

You can prevent automatically attaching a JSON Schema to your Collection by setting autoAttachJSONSchema to false. If you stick with the default of automatically attaching a JSON Schema, you can configure the validationAction and validationLevel – see Mongo's Schema Validation for more info.

1import { EasySchema } from 'meteor/jam:easy-schema';
2
3EasySchema.configure({
4  // autoAttachJSONSchema: false,
5  validationAction: 'warn', // 'error' is default
6  valdiationLevel: 'moderate' // 'strict' is default
7});

Dynamically import schema (optional)

You can also dynamically import your schemas to reduce the initial bundle size on the client.

First you'll need to create your schema inside its own schema.js and put it inside its collection's folder, e.g. /imports/api/todos/schema.js

Then you'll need to add a file to your project's root, e.g. register-dynamic-imports.js and import this file on both the client and the server near the top of its mainModule. Here's an example of the file:

1// In order for the dynamic import to work properly, Meteor needs to know these paths exist. We do that by declaring them statically inside a if (false) block
2Meteor.startup(async () => {
3  if (false) {
4    await import('/imports/api/todos/schema');
5    // add additional schema paths here as well
6  }
7});

Then instead of using Todos.attachSchema(schema), you'd just use Todos.attachSchema()

1// By not passing in a schema explicitly, it will automatically dynamically import the schema and then attach it
2Todos.attachSchema();

This assumes your directory structure is /imports/api/{collection}/schema.js. If you have a different structure, e.g. /api/todos/schema.js, you can configure the base path with:

1EasySchema.configure({
2  basePath: `/api`
3});

Unsupported

Auto Check for bulk writes

If you're using rawCollection() for bulk writes, these will not be automatically validated prior to the operation but you will get validation from the JSON Schema on the db Collection. You can manually use the check function if you'd like to validate prior to writing to the db. I'm hoping that Meteor will one day support bulk writes without needing rawCollection(). There was some progress on this front but it seems to have stalled – https://github.com/meteor/meteor/pull/11222

Update accepting an aggregation pipeline

Though technically Mongo supports it, Meteor does not support an aggregation pipeline in update operations (https://github.com/meteor/meteor/issues/11276) and so this package does not.

JSON Schema omissions

See: MongoDB JSON Schema omissions

In addition to the MongoDB omissions, these are also unsupported by this package at this time:

  • dependencies
  • oneOf
  • allOf
  • not
  • Array keywords
    • contains
    • maxContains
    • minContains
  • Number keywords
    • multipleOf
    • exclusiveMinimum (you can use min instead to achieve the desired result just be aware that min is inclusive)
    • exclusiveMaximum (you can use min instead to achieve the desired result just be aware that max is inclusive)
  • String keywords
    • contentMediaType
    • contentEncoding