Is a headless browser the answer?

What exactly is a headless browser?

It's just like any other web browser -- Firefox, Chrome, Safari, IE -- except there is no user interface. Nothing is shown on the screen and there is nothing to click on. Instead, you write scripts that interact with the browser. The particular script you need for javascript SEO is pretty basic. For every web page on your site, you want to open the page, wait for the javascript to run, then capture the entire DOM and save it to a file.

The most commonly used headless browser is PhantomJS, but HTMLUnit is also an option for Java websites.

How does this help you?

Now you have an html file. Unlike this html file you serve to users, this html has all the content already in place. This is exactly what Google needs. When requests come to your web server from Google-bot, you can send Google this file instead of the one you send to your users. Your headless browser already executed all the javascript and filled in the content so Google can read it.

Some Gotchas