spiderable

v1.0.14Published 3 years ago

This package has not had recent updates. Please investigate it's current state before committing to using it in your project.

spiderable

Source code of released version | Source code of development version


spiderable is part of Webapp. It's one possible way to allow web search engines to index a Meteor application. It uses the AJAX Crawling specification published by Google to serve HTML to compatible spiders (Google, Bing, Yandex, and more).

How it works

When a spider requests an HTML snapshot of a page the Meteor server runs the client half of the application inside phantomjs, a headless browser, and returns the full HTML generated by the client code.

In order to have links between multiple pages on a site visible to spiders, apps must use real links (eg <a href="/about">) rather than simply re-rendering portions of the page when an element is clicked. Apps should render their content based on the URL of the page and can use HTML5 pushState to alter the URL on the client without triggering a page reload. See the Todos example for a demonstration.

When running your page, spiderable will wait for all publications to be ready. Make sure that all of your publish functions either return a cursor (or an array of cursors), or eventually call this.ready(). Otherwise, the phantomjs executions will fail.

Notes

If you deploy your application with meteor bundle, you must install phantomjs (http://phantomjs.org) somewhere in your $PATH. If you use meteor deploy this is already taken care of.