tomdale.net

Every so often, we get requests to make Ember.js support AMD (asynchronous module definition). Until today, I had yet to hear anyone articulate why the advantages outweighed the (in my opinion) many disadvantages. Then, James Burke wrote an article called Simplicity and JavaScript modules that has so far done the best job outlining why AMD is good. However, I disagree with many of the assumptions and find many of the arguments outright contradictory. So, while James is both smart and I’m sure good-looking (and I agree with his comments on CommonJS), here are the reasons I think he is wrong about AMD.

Build Tools Are Okay

However, for those of us who came from Dojo, requiring a server tool or compile step to just develop in JS was a complication. I’m going to mangle Alex Russell’s quote on this, but “the web already has a compile step. It’s called Reload”.

I have a lot of respect for Alex but, if this is his current opinion, he’s wrong. The app development world of the Dojo era is different from the world of today, and it’s important that we are driven by current realities rather than stale institutional knowledge. Every serious application development environment in the world has a build step. If you are making a small application, then fine, I agree you don’t need build tools. But you probably don’t need AMD or a script loader, either.  If your app is truly simple, you will be fine with adding a few <script> tags to your page. I’m involved in several large web application projects right now and universally they use a build procedure of some kind. CoffeeScript compilation and minification are two examples of legitimate reasons to have a build step. Packaging your modules is fine too.

Many HTTP Requests

AMD expects every module to be contained in a separate file. As web app development gets more rigorous, developers want to be able to organize their files in the same way they might in Ruby or Cocoa. For example, all of the projects I’m working on easily have hundreds of files. Each view is a separate file, each template is a separate file, each controller is a separate file, and so on. With AMD, each additional file means additional HTTP overhead. But more importantly, many users are now on high-latency but high-bandwidth wireless connections. Minimizing the number of trips to the server is important. James addresses this in his blog post:

Loading individual modules piecemeal is a terrifically inefficient way to built a website. Because of this, there’s the great RequireJS optimizer, which will turn your modules into ordinary packages.

But wait, I thought we were just arguing that build tools are bad? Here’s the thing: if your app is simple, you don’t need build tools or module loading. If your app is more sophisticated, you’ll need both; so why not let the build tools handle the packaging for you? AMD proponents also argue that serving files individually makes it easier to debug. We did this in the SproutCore 1.0 days and, though it was extremely slow in development (perhaps because our HTTP server was single-threaded), it was effective. However, enough browsers support the sourceURL convention that in our current projects we just include a function like this:

  function appendSourceURL(data, path) {
    return data + " //@ sourceURL="+path;
  }

Too Much Ceremony

AMD requires you to wrap all of your code inside an anonymous function that is passed to the define method:

define(['dep1', 'dep2'], function (dep1, dep2) {
    //Define the module value by returning a value.
    return function () {};
});

In my opinion, having to write this out for every file sucks. I prefer to call require for each dependency. Perhaps it’s a trivial difference, but removing a dependency is as easy as deleting a line and adding a new dependency means adding a new line of code. It’s less error-prone than editing an array of strings.

You can achieve a similar syntax with AMD, like this:

define(function() {
 require('ember.js');
 window.MyApp = Ember.Application.create();
});

But at that point, why bother with loading a large AMD script loader? We use an implementation of a script loader that is under 50 lines of code.

The Alternative

What we’ve been using for our projects is a system that takes the source code for each file and wraps it as a string, as described by Tobie Langel in his post Lazy evaluation of CommonJS modules. All of the source code is downloaded in one HTTP request (great for high-latency 3G connections) and is saved in memory as a string so parsing is fast. Those strings are saved in a hash keyed on each individual file’s name. If you were to look at our application.js file, you would see something similar to this:

Files = {};
Files['main.js'] = "require(\"controllers/app_controller.js\");";
Files['controllers/app_controller.js'] = "alert(\"Hello world!\");";

The main JavaScript file is executed, and dependencies are resolved at runtime. This allows you to conditionally evaluate code based on the execution environment. For example, imagine you had a file that defined keyboard shortcuts. You could decide not to pay the parsing cost for that file if you detected that you were running on a touch platform.

We also have a system for packaging up many files into a single module, which can be loaded lazily. This helps us get the initial payload within the limits of mobile device’s cache limits.

The best part of this system is that, as an application developer, there is very little ceremony involved. If I need some functionality, I just place a require statement. If it has already been loaded, the require is a no-op. I make this argument a lot, but going from “a little friction” to “no friction” often makes the difference between good habits and bad habits. In this case, I can open a new file and start typing code, instead of worrying about setting it up as a module.

AMD has many features, but I think that the extra markup and complex loaders needed to support it outweighs its benefits. Out-of-the-box it is not ready for production, and needs build tools to make it truly useful. If build tools are required anyway, a much simpler solution should fit most developers’ needs.

I am looking forward to your blog post pointing out the flaws in my argument and excoriating me for making a fool of myself in public.

A very sincere thank you to James Burke, Tim Branyen and Yehuda Katz for reviewing this post and helping me better understand AMD. Please consider this a strong opinion, weakly held.

Yehuda Katz and I spend the majority of our time pair programming on client projects and on our open source projects like Ember.js. As in any profession, it’s important to be “foaming at the mouth crazy” about your tools.

Since Tilde moved into our new offices, we’ve finally put together what I think is our dream setup that I wanted to share.

Hardware

The crown jewel of the setup is a brand new 3.4GHz 27″ iMac with a 256GB SSD and 1TB hard drive. This thing absolutely screams, and is a joy to use. On our last pairing setup (a Mac mini with a 5400rpm hard drive), we would cringe when we had to bootup VMware to test Internet Explorer. On this thing, you can’t even tell that you’re running Windows virtualized. It’s insanely responsive.

We prefer being able to face each other, and modeled our setup after Josh Susser’s post about his pairing setup at Pivotal. Sitting side-by-side has always felt awkward to me; and the ability to easily have a face-to-face conversation is critical. This also allows us to more easily deliver high-fives after slaying a particularly hard bug.

We orient our desks at a right angle, and plug an external 27″ Thunderbolt Display into the iMac with mirroring turned on. This allows us both to see the same content at the same resolution. We both get a Bluetooth keyboard, and Yehuda uses a Magic Trackpad while I prefer the Magic Mouse. The ability to have your own space to keep drinks and other personal belongings is a big win. I can also customize the keyboard to my liking, since I prefer mapping Caps Lock to the Control key and Yehuda hates it. When we’re not pairing, we can use either the Thunderbolt Display or the iMac as external monitors for our laptops. The display also serves as a nice USB hub for charging iPhones or other devices.

Perhaps most critically, we hung the biggest whiteboard we could find on the wall next to our station. There are few problems a good whiteboard diagram can’t fix.

Software

Yehuda and I both use MacVIM with Janus as our editor. One major point of contention was file navigation: I prefer NERDtree and Yehuda prefers alloy’s fork of MacVim with a native file browser. (The native file browser drives me absolutely fucking crazy because I can’t navigate using just the keyboard; I have to pick up the mouse.)

We almost came to blows over this, but now that we have a 27″ display, we agreed to just use both.

Marital Bliss

We use zsh with oh-my-zsh. Our shell displays both the current git branch (if applicable), as well as the current Ruby version. Both of these have saved us countless hours of hair-pulling. We manage Ruby versions with Wayne Seguin’s indispensable rvm, the Ruby version manager.

For communcation, we use Propane to hang out in various Campfire rooms and iChat to log on to AIM, where we have separate pair accounts so that there is no feeling of invaded privacy. We use Dropbox and Lion’s AirDrop feature to shuttle files back and forth, and use the surprisingly powerful speakers in the Thunderbolt Display to do our standups via Skype.

I couldn’t be happier with our setup. If you haven’t tried pairing, I recommend it; while it sometimes feels slower, you avoid many of the obvious mistakes that one can make when spending hours in isolation. I also think that the software you write tends to be better if it’s constantly being sanity-checked by someone else.

If you have your own pairing setup, what improvements would you make to ours?

Edit: I forgot the most important component of a successful pairing setup:

Edit 2: Updated to clarify that the second display is mirrored.

Where is the web’s Loren Brichter?¹ Why does it take big teams, big budgets, and long timelines to deliver web apps that have functionality and UI that approaches that of an iPhone app put together by one or two people?

If you’re a developer who is obsessive about building a quality user experience, you are most likely to create an iOS application. One reason is that Cocoa is able to operate at a higher level of abstraction than raw web primitives. It means that iOS developers can think more about how their UI or features should work, instead of how they should be implemented. The human brain is like a buffer: at a certain point, every new concept pushes something else off the other end. For web developers, buffer overflow is common as they grapple with cross-browser bugs or APIs that are openly design-by-committee.

The web needs better abstractions. There are, however, only a limited number of abstractions you can implement in under 5k of code. I’m sensitive to cultivating an attitude in the community that the only valuable problems worth solving are those that can be done under 5k (or 10k, or 20k, or…). I will take my share of the blame, and admit that many frameworks that tried to take on large problems ended up out-of-touch and crufty.

The web won’t have a singular answer, and that’s fine. But we have to avoid fostering a culture where people are afraid to tackle hard problems because they’ll be seen as architecture astronauts. We shouldn’t let the mistakes of the past make us timid about learning and moving forward.

***

Yesterday on Twitter, I posted:

Just wanted to point out that Backbone is still listed on microjs.com, even though it’s >5k if you count dependencies.

Which is fine; let’s just be upfront about the fact that microjs.com is more akin to the cool kids table than a useful directory.

In retrospect, this came off crankier than I intended. Several people contacted me privately (and publicly!) and weren’t sure why I was making a stink over something that, to them, seemed trivial. And while in the scheme of things it is trivial, I think this arbitrary line in the sand we as a community have drawn is harmful to the future of the web.

I want the web to win. Everywhere.

So I want people to be aware of the abstractions we’re giving up to hit our 5k target. If some things get included that are over 5k, and other things get rejected that are under the 5k limit, it sends a very mixed message to new developers about what is possible. If we’re going to emphasize small codebases, let’s be honest about the limitations that entails.

If we don’t get our act together soon, proprietary platforms are going to become entrenched. Every new device that comes out is an opportunity that is ours to lose. The community has to figure out how we educate and recruit developers into the web fold. People will disagree with me about the best way to accomplish the task, and that’s great. But I’d like to compete on things like developer productivity and finished products, not insinuation that anything over a certain size is inherently unsuitable for the web.

It’s human nature that once we start assigning labels to things, we think about which side of the label we’re on. I think that positioning microlibraries as a separate “thing” is a bad idea. There are just tools that fall on different sides of the size spectrum.

Writing efficient code is important. But when we decide that certain web abstractions are a bridge too far, we’re actively discouraging precisely the developers that we need the most. When the next Loren Brichter comes along, I want to be able to say he or she is a web developer.

Thanks to Majd Taby and Yehuda Katz for reviewing this post. I tweet as @tomdale if you’d like more timely updates.

1. Loren Brichter’s company, atebits, created both Tweetie for Mac and Tweetie for iOS, which were acquired by Twitter. [↩]

Thomas Fuchs recently wrote about the advantages of using JavaScript micro-frameworks:

I for one welcome our new micro-framework overlords—small frameworks and libraries that do one thing only, not trying to solve each and every problem, just using pure JavaScript to do it. Easy to mix and match, and to customize to your needs.

He also pans “full-stack” frameworks:

These libraries are HUGE, being 100+ kilobytes in code size, and even minified and gzipped they are pretty big. … A whopping 100% of sites or apps using these libraries don’t use all the features they provide.

His argument sounds good, in theory. The idea that I can pick and choose among a number of well-written and focused JavaScript libraries to assemble the ultimate bespoke JavaScript environment seems very enticing indeed.

The golden rule of software, however, is that unless it is designed to work well together, it usually won’t. Mr. Fuchs has apparently never heard of dependency hell. Dustin Diaz has done a great job of putting together many of these micro-frameworks with Ender.js, but as a curator, he has to rely on the original author if he wants to make a change.

This attitude also continues the trend of giving developers overwhelming choice. Convention-over-configuration and emphasis on developer ergonomics are key to getting real work done. Of course Mr. Fuchs is able to tell you which JavaScript library will precisely match your requirements—he writes JavaScript libraries in his spare time! For the rest of us, we don’t really want choice; we just want what’s best.

If choice was more important than well-tested integration, the majority of attendees at last week’s CodeConf would have been carrying ThinkPads instead of MacBook Pros. Instead, developers want a happy path that they can be reasonably assured will work well. If it doesn’t meet their needs, they want the ability to augment or replace the components that make up their integrated system.

Here’s the reality: As web applications get more complex, developers end up needing all of the miscellaneous nuts and bolts that they thought they would never use. Take Flow for example. Flow is a killer task manager web application built on Backbone.js. Backbone.js often advertises that it is only 3.9kb minified and compressed. How much of Flow’s nearly 900k of (minified!) JavaScript do you think is the application developers filling in the deficiencies in Backbone?

SproutCore is often dismissed because of its size. Of course minimizing size is important. But if it takes a megabyte of JavaScript to ship the features you want, better the bulk of that code live in a framework instead of your application. Living in a framework means that it has been reviewed and optimized by an entire community of developers, and improves without any effort on your part. New Twitter currently clocks in at over a megabyte of JavaScript:

To the best of my knowledge, Twitter is not using any third-party libraries except for Mustache and jQuery. That means that a huge amount of JavaScript was written by Twitter engineers just to provide the framework in which the application runs. This isn’t stuff specific to Twitter. This is, for example, code that determines how to propagate changes to the DOM when the data model changes via XHR—a problem that all JavaScript developers face.

SproutCore may be big, but it’s because web applications are getting more complex every day, and we all have a common set of complex problems to solve. Oversimplifying the problem is disingenuous and forces each application to fend for itself. Mr. Fuchs says:

Small code is good code: the fewer lines in your source, the fewer bugs you’ll have, plus it will download and execute faster.

If creating a modern web app requires a megabyte of JavaScript, would you rather write, debug and optimize all of that code yourself?

It’s time to lay to rest the argument that full-stack libraries are unable or unfit to solve real problems. Instead, they are best suited to tackle the problems faced by modern web app developers. The same criticisms full-stack frameworks receive today are eerily similar to the same criticisms levied at jQuery several years ago. Unless you are planning to discontinue your application in six months, it’s time to start developing for the future.

Thanks to Charles Jolley for reviewing this post. I tweet at @tomdale if you want more timely updates.

The views expressed below are my own and not those of the SproutCore core team.

Recently, Yehuda Katz and I have been making some changes to the SproutCore view layer. We’ve pulled out basic functionality into a new class called SC.CoreView, and broken the old SC.View (a behemoth) into several files based on what features they add. We also introduced SC.TemplateView, a subclass of SC.CoreView, that allows you to use Handlebars templates. ^{1 2}

Changes like these can be scary because they introduce uncertainty, but I want to assure you that the project is still headed in the same direction; we just have two additional goals:

• Reduce the learning curve for new developers
• Improve load time on low-power, low-bandwidth devices (e.g., iPad and Android tablets)

It turns out that these view changes put us on the path towards meeting both goals.

The desktop framework, which can best be described as SproutCore’s widget library, is its largest by far. The major reason for its size is that we have re-implemented almost all desktop controls in JavaScript and HTML. For example, SproutCore menu items flash when selected, like they do in Mac OS X, and we have popovers that can anchor to another on-screen element, like in iOS.

The downside to this sophistication is both an increase in code size and an increase in cognitive overhead for developers new to SproutCore. In particular, traditional web developers have a hard time learning how views work; they are better off reading the Cocoa guide to views than to read a book on JavaScript and DOM.

Today, when we ask developers to write SproutCore applications, we ask them to throw away their existing expertise and commit entirely to the framework. If in six months they decide it won’t meet their needs, they have to abandon almost all of their work. And it’s almost impossible to test the waters with an existing codebase, since developers must throw away their entire view layer.

Many developers don’t want or need native-style controls. They want to throw together a quick application that still feels like a website. We can bring value to these developers, too. While SproutCore’s view layer complements the controller and model layers, it doesn’t require them—indeed, the point of MVC is to encapsulate these concerns. If we can bring the sophistication of the data store and the robustness of statechart-driven apps to everyone, we should.

Here’s the thing about templates, though: they’re a better way of doing what we’ve been doing all along.

At its core, a SproutCore view is a DOM representation plus a managed layout. You need to build that DOM representation somehow, which means either multiple DOM API calls, or lots of string concatenation operations.

For example, here’s the render method for SC.RadioView:

render: function(dataSource, context) {
  var theme = dataSource.get('theme');
 
  var isSelected = dataSource.get('isSelected'),
      width = dataSource.get('width'),
      title = dataSource.get('title'),
      value = dataSource.get('value'),
      ariaLabeledBy = dataSource.get('ariaLabeledBy'),
      ariaLabel     = dataSource.get('ariaLabel');
 
  context.setClass({
    active: dataSource.get('isActive'),
    mixed: dataSource.get('isMixed'),
    sel: dataSource.get('isSelected'),
    disabled: !dataSource.get('isEnabled')
  });
 
  //accessing accessibility
  context.attr('role', 'radio');
  context.attr('aria-checked', isSelected);
  if(ariaLabel && ariaLabel !== "") {
    context.attr('aria-label', ariaLabel);
  }
  if(ariaLabeledBy && ariaLabeledBy !== "") {
    context.attr('aria-labelledby', ariaLabeledBy);
  }
 
  if (width) context.css('width', width);
 
  context.push('<span class = "button"></span>');
 
  context = context.begin('span').addClass('sc-button-label');
  theme.labelRenderDelegate.render(dataSource, context);
  context = context.end();
}

For those keeping track at home, that’s 34 lines of code to generate this:

<div class="sc-radio-button mixed" aria-checked="true" role="radio" index="0">
  <span class="button"></span>
  <span class="sc-button-label">Item1</span>
</div>

And that’s just to create the DOM representation. Updating it takes another 27 lines of code, and it’s not even efficient; if a single property changes, it re-renders the entire label portion of the control.

Imagine if we could instead tell the radio view to use a template:

<div {{bindClass "classNames isActive isMixed isSelected isEnabled"}}
     {{bindAttr role="ariaRole"
                aria-checked="ariaChecked"
                aria-label="ariaLabel"
                aria-labelledby="ariaLabeledBy"}}>
  <span class="button"></span>
  <span class="sc-button-label">
    {{renderDelegate "label"}}
  </span>
</div>

This is both shorter and easier to understand. But perhaps most importantly, good abstractions allow us to make optimizations once. Instead of each view trying (and failing) to optimize DOM access, and in the process introducing a rat’s nest of complicated logic, we can move the complexity to the templating layer and have every view ever written benefit from it.

We could, for example, use different optimizations for Internet Explorer and Chrome. Because views are now stating intent instead of atomic DOM operations, we can choose the fastest path based on the performance characteristics of the current environment. Decoupling allows implementers to focus on behavior and appearance, not the intricacies of DOM.

My point is this: template views are not meant as a replacement for SC.View and our rich library of controls. In fact, template views can serve to make desktop-style controls even better. If controls are easier to implement, more people will make them, and then we all benefit.

Template views are designed to integrate with existing applications, or to stand on their own if you need them to. As always, it’s important to pick the right tool for the job.

Yehuda and I have recently made fixes to SC.TemplateView that allow it to be used inside traditional view hierarchies. An updated prerelease gem with these changes is in the works, or you can find them in the master branch on GitHub.

We think there will be a lot of opportunity for existing SproutCore apps to take advantage of templates. For example, think of the many modern Mac OS X applications that use a WebView to integrate HTML content.

If you’re starting a new SproutCore application, you can decide which works best for you. If you want a web-style application or a desktop-style application (or some combination of the two), we want you to feel like a first-class citizen. And if you opt for desktop-style, we will work hard to ensure SC.TemplateView works with Greenhouse when it ships.

I will be discussing this and more at the SF SproutCore meetup, Tuesday, March 15.

Thanks to Chris Swasey, Tyler Keating, Sudarshan Bhat and Peter Wagenet for reviewing this post. I tweet at @tomdale if you want more timely updates.