meteor-sitemaps

Quickly create dynamic ("real-time") sitemaps using your own functions.

Quick Start

A simple example is shown below, with more complicated examples further down. Sitemaps are dynamic (generated "on-the-fly" at request time), intended for moderate use with the latest data from your database. If you have more than a few hundred pages and/or are crawled at high frequency, you may be better off creating a static sitemap.

meteor add gadicohen:sitemaps

Create server/sitemaps.js which contains something like:

1sitemaps.add('/sitemap.xml', function() {
2  // required: page
3  // optional: lastmod, changefreq, priority, xhtmlLinks, images, videos
4  return [
5    { page: '/x', lastmod: new Date() },
6    { page: '/y', lastmod: new Date(), changefreq: 'monthly' },
7    { page: '/z', lastmod: new Date().getTime(), changefreq: 'monthly', priority: 0.8 },
8    // https://support.google.com/webmasters/answer/178636?hl=en
9    { page: '/pageWithViedeoAndImages',
10      images: [
11        { loc: '/myImg.jpg', },        // Only loc is required
12        { loc: '/myOtherImg.jpg',      // Below properties are optional
13          caption: "..", geo_location: "..", title: "..", license: ".."}
14      ],
15      videos: [
16        { loc: '/myVideo.jpg', },      // Only loc is required
17        { loc: '/myOtherVideo.jpg',    // Below properties are optional
18          thumbnail_loc: "..", title: "..", description: ".." etc }
19      ]
20    },
21    // https://support.google.com/webmasters/answer/2620865?hl=en
22    { page: 'lang/english', xhtmlLinks: [
23      { rel: 'alternate', hreflang: 'de', href: '/lang/deutsch' },
24      { rel: 'alternate', hreflang: 'de-ch', href: '/lang/schweiz-deutsch' },
25      { rel: 'alternate', hreflang: 'en', href: '/lang/english' }
26    ]}
27  ];
28});

You can call sitemaps.add() as many times as you like. More details on the format below. Note that the url is automatically added to the data served from /robots.txt (since 0.0.4, using the robots.txt smart package).

Important: The above example uses a brand new Date() for every link. This is just for demonstration purposes. Of course you should use the real date of the last page update (updatedAt from the database?). If you always use the current time, Google will penalize you (or at the very least, ignore this field on future crawls).

Full Usage

1// To add a sitemap
2sitemaps.add(url, list);
3
4// To compress sitemap as gzip files. Note this will apply to all sitemap files
5sitemaps.config('gzip', true/false); // default to false

URL

The obvious example is /sitemap.xml. You can call the function more than once to have many different (types of) sitemaps. The URL is added to the output of /robots.txt automatically (since 0.0.4).

Note that the location is important. A sitemap can only reference other URLs in its own path or descendant paths. e.g. /sitemap.xml can reference all URLs on the site. /articles/sitemap.xml can only reference other pages in the /articles/ directory/path/route.

List (Array or Function)

The list can either be an array in the following format, or a function that returns an array in the following format (e.g. a function that iterates over information in a Collection).

1[
2  {
3    // Required.  http[s]://sitename.com automatically prepended */
4    page: '/pageName',
5    // Optional.  Timestamp of when the page was last modified.
6    lastmod: new Date(),         // or new Date().getTime()
7    // Optional.  always, hourly, daily, weekly, monthly, yearly, never
8    // http://www.sitemaps.org/protocol.html#changefreqdef
9    changefreq: 'monthly',
10    // Optional.  http://www.sitemaps.org/protocol.html#prioritydef
11    priority: 0.8
12    // Optional.  https://support.google.com/webmasters/answer/2620865
13    // Again, the base URL is automatically prepended to the href key
14    xHtmlLinks: [
15      { ref: 'alternate', 'hreflang': 'en', 'href': 'en/blah' },
16      { ref: 'alternate', 'hreflang': 'de', 'href': 'de/blah' }
17    ],
18    // Optional.  https://support.google.com/webmasters/answer/2620865?hl=en
19    // Again, the base URL is automatically prepended to the loc key
20    images: [
21      { loc: '/myImg.jpg' },      // Only loc is required
22      { loc: '/myOtherImg.jpg',   // Below properties are optional
23        caption: "..", geo_location: "..", title: "..", license: ".."}
24    ],
25    // Optional.  https://support.google.com/webmasters/answer/80472?hl=en
26    // Again, the base URL is automatically prepended to loc, *_loc
27    videos: [
28      { loc: '/myVideo.jpg' },    // Only loc is required
29      { loc: '/myOtherVideo.jpg'  // Below properties are optional
30        thumbnail_loc: "..", title: "..", description: "..", etc: ".." }
31    ]
32  }
33]

Other options might come soon, e.g. to automatically use routes in your app to build the sitemap.

Example (from Meteorpedia)

1// To compress all sitemap as gzip file
2sitemaps.config('gzip', true);
3
4sitemaps.add('/mw_AllPages_sitemap.xml.gz', function(req) {
5  // NOTE: req is the Node request object, use it when you need to retrieve information (host, port, protocol ...)
6  // check out more in ./example/example.js
7
8  var out = [], pages = WikiPages.find().fetch();
9  _.each(pages, function(page) {
10    out.push({
11      page: 'read/' + page.name,
12      lastmod: page.lastUpdated
13    });
14  });
15  return out;
16});

You can see this output here: http://www.meteorpedia.com/mw_AllPages_sitemap.xml

Locations (page, loc, href, etc)

Anywhere where a url can be provided, you can provide a "relative URL" (with or without a leading /), and Meteor.absoluteUrl() will be prepended. You can override this by calling sitemaps.config('rootUrl', 'myRootUrl'). For individual links, providing an absoluet URL (beginning with http:// or https://) will avoid this behaviour. URI components are escaped for you.

Contributors

Thanks to @zol, @tarang, @dandv, @DirkStevens, @picsoung, @SashaG for various PRs as listed in History.md.