A Faster Horse: The Future Of Web Development

Let’s talk about the rise and fall of technologies. There’s a neat graphical representation of the life-cycle of a technology called an S-Curve:

s-curve.jpeg

JavaScript is enjoying tremendous success right now, so I’m not going to bore you with the metrics. Exponential growth is fantastic, and as someone who uses JavaScript at almost every layer of the tech stack, I couldn’t be happier for my community. It’s certainly made my job as a web developer much easier.

Now, what about the future of JavaScript are you most excited about? I can think of a few off the top of my head:

  • New language features: ES6 generators, template strings, SIMD, …
  • Package manager/module ecosystem upgrades: parameterized scripts, private repositories, …
  • Framework updates: Angular 2, koa, …

Now, these are all quite exciting. New language features let us write more expressive code, and do faster computations. New frameworks help us write more robust applications. npm is amazing, and it is certainly the innovation that made node so successful, so any improvement to it is just icing on the cake.

However, these are all incremental improvements. We’re still sliding up the same S-Curve, and we’re going to reach maturity eventually because all technologies experience diminishing returns. You’re optimistic to a fault if you think that HTML+CSS+JavaScript is the holy grail of web development, and that we can’t do better. As much as we love our tools, we have to accept that they are far from perfect.

multiple-s-curves.jpeg

S-Curves don’t exist in isolation. Something else is on the horizon, and it’s not going to be an incremental improvement. This is why I think it was fantastic that TJ made a high-profile jump to a different S-Curve. Go has its own set of problems, but that’s not the point; he recognized the limits of the tools at hand, and tried something different. It’s far easier to pour more effort into the tools you are already familiar with than it is to try something completely different.

Pick any technology out there and you’ll find someone who can wax poetic about how its better than what you’re using right now. It doesn’t matter if you think they’re right or wrong, listen like you’re wrong, because eventually, you will be wrong. Eventually, you will be the cranky administrator who still believes that JSP is the holy grail of web development. Eventually, you will have to do something insane like write a new VM for the aging technology you’re locked into. Eventually, you will still be concatenating strings like this when everyone else is using the + operator.

What is next-generation web development going to look like? I don’t know, but I do have a small wish list:

  • Lower-level access to rendering: Lose the HTML/CSS, and go straight to the canvas
  • Multithreading support: We’re close to the limit of single-core performance, and even cell phones have multiple cores now
  • Lower-level access to memory: This is a compliment to multithreading, and its nice to not have to rely on garbage collection if you know what you’re doing
  • Static verification: This should be an option for applications where correctness is important
  • Better error handling: This is a real pain in node right now

What do you want to see next?

web-dev-s-curves.jpeg

Jankproof Javascript

Javascript is getting faster all the time, but things like long lists of complex cells are always going to be expensive to compute. To solve this, I wrote the unjank module last week. It helps you do expensive things in Javascript without causing the user experience to suffer.

I’m happy to report that after a week of production use, it’s clear that this technique is a significant improvement over what we were doing before for several reasons.

Device Agnostic

It doesn’t matter how fast or slow the task is; unjank benchmarks it on-the-fly and runs it as quickly as the device will allow. This means that your application is jank-free on all devices without you having to come up with magic numbers that determine how quickly a task should run.

Smooth Scrolling

An unexpected discovery was that kinetic scrolling in Webkit works very well even if the page is getting longer during the scroll. This means that if your user is scrolling down a long list as it is being rendered with unjank, they will not perceive it as slow at all. Webkit preserves the momentum of the scroll and keeps going as the page gets longer.

Aborting Tasks

The ability to abort an ongoing task is critical because most tasks are initiated by a user action. For example, if you have two tabs that have a long list each, quickly switching between the tabs will eventually crash the application unless the rendering of the lists is aborted when the tab becomes inactive.

Conclusion

I’m going to be using unjank a lot more going forward, especially where lists are involved. I pulled up the Getable app to experience it pre-unjank, and it has that signature lagginess associated with web apps, despite our use of requestAnimationFrame. With unjank, our longest lists no longer cause the browser to stutter — a small step out of the uncanny valley of hybrid apps.

Tips For Debugging JavaScript Race Conditions

Race conditions are tedious to debug, even in a single-threaded language like JavaScript. Here are three tips I hope you never need.

1. Breaking On DOM Changes

When dealing with nested views and asynchronous renders, it’s common to run into issues where a subview fails to appear reliably.

To debug these issues, I like using Chrome’s ability to break on DOM changes. Right click on a node in the Elements panel of your dev tools and you’ll be presented with different types of breakpoints you can set.

dom-breakpoint-screenshot.png

2. Spying On Object Properties

A particularly frustrating problem that results from improperly cloning objects is when properties change their values seemingly at random.

// One source of this problem is devious .toJSON methods
// that don't clone their source object
var myModel = new Backbone.Model({uhoh: {nested: 'somevalue'})}  
var copy = myModel.toJSON()  
copy.uhoh.nested = 'changed'  
myModel.get('uhoh').nested == 'changed'  

View on jsfiddle

Here the source of the mischief is clear, but you won’t always be so lucky. If the code that manipulates the object is in some asynchronous callback three modules deep, its not trivial to figure out where the source of the problem is since there are so many ways to manipulate the value:

// The many different ways to cause this problem
myModel.get('uhoh').nested = 'changed'  
myModel.toJSON().uhoh.nested = 'changed'  
myModel.attributes.uhoh.nested = 'changed'  

You can handle all these cases by defining a setter on the uhoh object with a debugger breakpoint in it:

myModel.attributes.uhoh.__defineSetter__('nested', function(val) {  
  debugger
  uhoh._nested = val
})

View on jsfiddle

Now you’ll be able to look at the call stack no matter how that property was changed.

3. Making It Worse

Debugging is hard because it’s the inverse of programming: “Given the code, pinpoint the source of this behavior”. This is especially true when dealing with race conditions, because the behavior doesn’t happen all the time. This is why when all else fails, I solve the inverse problem: writing code that makes the bug happen reliably.

If you can make the issue 100% reproducible, not only have you gained insight into the root cause of the problem, you’ve also shortened the debugging cycle.

In general, this is what I do:

  1. Locate the sections of code that manipulate the misbehaving data
  2. Wrap those sections in .setTimeout with large-ish intervals to force an deterministic order of operations
  3. Permute the order of operations until the problem is 100% reproducible
  4. Permute the order of operations until the problem is gone

I like to use prime numbers as intervals to reduce the odds that multiple sections of code run in close temporal proximity to each other because they happened to be repeating in phase.


I hope these tips save you as much time as they’ve saved me.dom-breakpoint-screenshot.png

Sleepsort

Sleepsort might be a joke, but its implementation demonstrates three very important things about javascript: closures, asynchronous functions, and variable hoisting.

It is therefore an excellent example for a beginner to study to understand javascript better.

The Naive Implementation

function sleepsort (input) {  
  for(var i=0; i<input.length; ++i) {
    setTimeout(function () {
      console.log(input[i]);
    }, input[i] * 1000);
  }
};

sleepsort([3,1,2]);  

Output:

undefined  
undefined  
undefined  

What happened here? Why did we get three undefined elements?

If we modify the example a little bit, the reason is clear:

function sleepsort (input) {  
  for(var i=0; i<input.length; ++i) {
    setTimeout(function () {
      console.log(input[i]);
    }, input[i] * 1000);

    console.log('i is ' + i);
  }
};

sleepsort([3,1,2]);  

Output:

i is 0  
i is 1  
i is 2  
undefined  
undefined  
undefined  

The for loop kept running, incrementing the value of i. By the time the callback functions ran, the value of i was 3, which was out of bounds of the input array.

This demonstrates closure, which is the ability of a function to “remember” the scope it was created in, and access the variables in that scope.

The Naive Fix

A beginner would likely try to fix the problem with this code:

function sleepsort (input) {  
  for(var i=0; i<input.length; ++i) {
    var j = i; // copy i into j.

    setTimeout(function () {
      console.log(input[j]);
    }, input[j] * 1000);
  }
};

sleepsort([3,1,2]);  

Output:

2  
2  
2  

Why did we get three 2s this time? The reason is because of hoisting. All variable declarations in javascript are “hoisted” to the top of the scope they are in. Since only functions create new scopes in javascript, this means that our code was equivalent to this:

function sleepsort (input) {  
  var j;

  for(var i=0; i<input.length; ++i) {
    j = i; // copy i into j.

    setTimeout(function () {
      console.log(input[j]);
    }, input[j] * 1000);
  }
};

sleepsort([3,1,2]);  

Unlike languages like Java and C, the for loop did not have its own scope. Thus, j was last assigned the value of 2 before the loop ended, and that was the element that was printed three times.

The Real Fix

The real solution to this problem is to create a new scope. By wrapping our timeout code in an immediately invoked function expression (IIFE), we create a variable j in a new scope and assign it the current value of i. Since j is in a new scope, it is untouched by future iterations of the loop.

function sleepsort (input) {  
  for(var i=0; i<input.length; ++i) {
    (function (j) {
      setTimeout(function () {
        console.log(input[j]);
      }, input[j] * 1000);
    })(i);
  }
};

sleepsort([3,1,2]);  

Output:

1  
2  
3  

Optimization

While optimizing sleepsort is somewhat of a laughing matter, creating functions inside a loop is almost always a bad idea. The following code is equivalent and resolves this problem:

function sleepsort (input) {

  function sort (j) {
    setTimeout(function () {
      console.log(input[j]);
    }, input[j] * 1000);
  };

  for(var i=0; i<input.length; ++i) {
    sort(i);
  }
};

sleepsort([3,1,2]);  

I hope that helped your understanding of closure, asynchrony, and hoisting!

Is this really O(n)?

No, it’s actually using your operating system’s insertion sort.

This code is all available on github and of course, npm.