publish-lookups

Installation

meteor add kschingiz:publish-lookups

Usage

1Meteor.publish("subscription", () => {
2  return PrimaryCollection.lookup(selector, options, [
3    {
4      collection: SecondaryCollection, // required: collection to join
5      localField: "_id", // required: field in PrimaryCollection
6      foreignField: "postId", // required: field in SecondaryCollection
7      selector: {}, // optional: apply additional selector to SecondaryCollection
8      options: { fields: { text: 1 } } // optional: apply additional options to SecondaryCollection
9    }
10  ]);
11});

Usage example

You have collections:

Posts:
  _id
  authorId
  text

Comments:
  _id
  postId
  text
  status

Authors:
  _id
  name

And want to publish posts with comments and post author in join, you can do it like this:

1Meteor.publish("postsWithCommentsAndAuthors", () => {
2  return Posts.lookup({}, {}, [
3    {
4      collection: Comments,
5      localField: "_id",
6      foreignField: "postId",
7      selector: { status: "active" },
8      options: { fields: { text: 1 } }
9    },
10    {
11      collection: Authors,
12      localField: "authorId",
13      foreignField: "_id",
14      selector: {},
15      options: {}
16    }
17  ]);
18});

Comparison

We have several packages which allows us to publish collections in joins, let's compare how they work and what's differences:

publish-composite

Package url: https://github.com/englue/meteor-publish-composite

Usage:

1{
2  find() {
3    // Primary query
4    return Posts.find({});
5  },
6  children: [
7    {
8      find(topLevelDocument) {
9        // applied for each of the posts document
10        return Comments.find({ postId: topLevelDocument._id })
11      },
12    }
13  ]
14}

publish-composite does not scale well, because in the second level queries it will create N cursor observers, where N is the number of documents returned in Primary query. This behavour will overload your database.

Unlike publish-composite, publish-lookups package does not depend on the number of documents returned in Primary query, it will create M number of cursor observers, where M is the number of required lookups.

Let's assume we have 100 posts and 200 post comments and we are joining all of them with primary collection Posts:

1. publish-composite: will create 1 observer for primary query, then it will create 100 observers for comments, because posts returned 100 documents.
2. publish-lookups: will create 1 observer for primary query, then it will create 1 observer for the lookup query.

101 vs 2

publish-lookups wins.

publish-aggregations

Package url: https://github.com/kschingiz/publish-aggregations

The package was developed by me one year ago, internally it's using mongodb change streams feature which does not scale well. Creating +10 change streams (it's created on each subscription) can overlad your database and make it much slower. Proof: https://jira.mongodb.org/browse/SERVER-32946

publish-lookups uses regular mongodb db find queries.

publish-lookups wins.