Requiring Modules in CouchDB

Thanks to node.js, using the same code on the server and client is easier than ever. What about running application code in the database? If you have a function that is expensive to compute, but always returns the same result for the same input, then you could get a significant speedup by having your database cache the results of the computation ahead of time. This is especially true if you’re using CouchDB, which uses incremental MapReduce to make view computation very efficient.

Our realtime app at Getable is backed by CouchBase, a derivative of CouchDB. CouchBase and CouchDB share the same replication protocol, but differ in several important ways. One such difference is that CouchBase does not support require, while CouchDB does. One major caveat with CouchDB’s require is that you have to push the modules you want up as design documents. Since forcing humans to resolve dependency trees is outlawed in the Geneva Conventions, we need a better way of doing this.

Enter Browserify’s standalone option. We can use it to create a string of code that is easily prepended to the map function in our views. Browserify’s standalone option takes a string, which is the name of the exported module. Critically, the export will be added to the global object if require is not available, which is the case in CouchBase views. Therefore, if you create a browserify bundle with the {standalone: ‘mymodule’} option and prepend that string to your map function, you will now have global.mymodule available for use. The one gotcha is that the global object does not exist in CouchBase views either, so it must be initialized ahead of the bundle.

Create an entry script that just exports the module you want:

// entry.js
module.exports = require(‘mymodule’)  

Then bundle it with the standalone option and initialize global ahead of time:

// bundler.js
var bundle = new browserify({standalone: ‘mymodule’})  
bundle.add(‘entry.js’)  
bundle.bundle(function (err, src) {  
  var prelude = ‘var global={};’ + src.toString()
})

Now, if you have a map function, you can just insert this prelude right after the function header:

function mapFunction (doc) {  
  // prelude goes here!
  emit(doc._id, global.mymodule(doc))
}

As an implementation note, we have a small script that uses Function.toString() to manage our design documents. It turns our map functions into strings, searches for the use of application logic, and browserifies the appropriate standalone bundle for each function. It’s less prone to failure than manual updates, and makes the experience just a bit more magical.


The “One Year Later” update: we’ve seen vastly improved performance by pushing these standalone bundles up under the lib key.

Jankproof Javascript

Javascript is getting faster all the time, but things like long lists of complex cells are always going to be expensive to compute. To solve this, I wrote the unjank module last week. It helps you do expensive things in Javascript without causing the user experience to suffer.

I’m happy to report that after a week of production use, it’s clear that this technique is a significant improvement over what we were doing before for several reasons.

Device Agnostic

It doesn’t matter how fast or slow the task is; unjank benchmarks it on-the-fly and runs it as quickly as the device will allow. This means that your application is jank-free on all devices without you having to come up with magic numbers that determine how quickly a task should run.

Smooth Scrolling

An unexpected discovery was that kinetic scrolling in Webkit works very well even if the page is getting longer during the scroll. This means that if your user is scrolling down a long list as it is being rendered with unjank, they will not perceive it as slow at all. Webkit preserves the momentum of the scroll and keeps going as the page gets longer.

Aborting Tasks

The ability to abort an ongoing task is critical because most tasks are initiated by a user action. For example, if you have two tabs that have a long list each, quickly switching between the tabs will eventually crash the application unless the rendering of the lists is aborted when the tab becomes inactive.

Conclusion

I’m going to be using unjank a lot more going forward, especially where lists are involved. I pulled up the Getable app to experience it pre-unjank, and it has that signature lagginess associated with web apps, despite our use of requestAnimationFrame. With unjank, our longest lists no longer cause the browser to stutter — a small step out of the uncanny valley of hybrid apps.