AMD is Not the Answer
by Tom Dale
Every so often, we get requests to make Ember.js support AMD (asynchronous module definition). Until today, I had yet to hear anyone articulate why the advantages outweighed the (in my opinion) many disadvantages. Then, James Burke wrote an article called Simplicity and JavaScript modules that has so far done the best job outlining why AMD is good. However, I disagree with many of the assumptions and find many of the arguments outright contradictory. So, while James is both smart and I’m sure good-looking (and I agree with his comments on CommonJS), here are the reasons I think he is wrong about AMD.
Build Tools Are Okay
However, for those of us who came from Dojo, requiring a server tool or compile step to just develop in JS was a complication. I’m going to mangle Alex Russell’s quote on this, but “the web already has a compile step. It’s called Reload”.
I have a lot of respect for Alex but, if this is his current opinion, he’s wrong. The app development world of the Dojo era is different from the world of today, and it’s important that we are driven by current realities rather than stale institutional knowledge. Every serious application development environment in the world has a build step. If you are making a small application, then fine, I agree you don’t need build tools. But you probably don’t need AMD or a script loader, either. If your app is truly simple, you will be fine with adding a few <script> tags to your page. I’m involved in several large web application projects right now and universally they use a build procedure of some kind. CoffeeScript compilation and minification are two examples of legitimate reasons to have a build step. Packaging your modules is fine too.
Many HTTP Requests
AMD expects every module to be contained in a separate file. As web app development gets more rigorous, developers want to be able to organize their files in the same way they might in Ruby or Cocoa. For example, all of the projects I’m working on easily have hundreds of files. Each view is a separate file, each template is a separate file, each controller is a separate file, and so on. With AMD, each additional file means additional HTTP overhead. But more importantly, many users are now on high-latency but high-bandwidth wireless connections. Minimizing the number of trips to the server is important. James addresses this in his blog post:
Loading individual modules piecemeal is a terrifically inefficient way to built a website. Because of this, there’s the great RequireJS optimizer, which will turn your modules into ordinary packages.
But wait, I thought we were just arguing that build tools are bad? Here’s the thing: if your app is simple, you don’t need build tools or module loading. If your app is more sophisticated, you’ll need both; so why not let the build tools handle the packaging for you? AMD proponents also argue that serving files individually makes it easier to debug. We did this in the SproutCore 1.0 days and, though it was extremely slow in development (perhaps because our HTTP server was single-threaded), it was effective. However, enough browsers support the sourceURL convention that in our current projects we just include a function like this:
function appendSourceURL(data, path) { return data + " //@ sourceURL="+path; } |
Too Much Ceremony
AMD requires you to wrap all of your code inside an anonymous function that is passed to the define method:
define(['dep1', 'dep2'], function (dep1, dep2) { //Define the module value by returning a value. return function () {}; }); |
In my opinion, having to write this out for every file sucks. I prefer to call require for each dependency. Perhaps it’s a trivial difference, but removing a dependency is as easy as deleting a line and adding a new dependency means adding a new line of code. It’s less error-prone than editing an array of strings.
You can achieve a similar syntax with AMD, like this:
define(function() { require('ember.js'); window.MyApp = Ember.Application.create(); }); |
But at that point, why bother with loading a large AMD script loader? We use an implementation of a script loader that is under 50 lines of code.
The Alternative
What we’ve been using for our projects is a system that takes the source code for each file and wraps it as a string, as described by Tobie Langel in his post Lazy evaluation of CommonJS modules. All of the source code is downloaded in one HTTP request (great for high-latency 3G connections) and is saved in memory as a string so parsing is fast. Those strings are saved in a hash keyed on each individual file’s name. If you were to look at our application.js file, you would see something similar to this:
Files = {}; Files['main.js'] = "require(\"controllers/app_controller.js\");"; Files['controllers/app_controller.js'] = "alert(\"Hello world!\");"; |
The main JavaScript file is executed, and dependencies are resolved at runtime. This allows you to conditionally evaluate code based on the execution environment. For example, imagine you had a file that defined keyboard shortcuts. You could decide not to pay the parsing cost for that file if you detected that you were running on a touch platform.
We also have a system for packaging up many files into a single module, which can be loaded lazily. This helps us get the initial payload within the limits of mobile device’s cache limits.
The best part of this system is that, as an application developer, there is very little ceremony involved. If I need some functionality, I just place a require statement. If it has already been loaded, the require is a no-op. I make this argument a lot, but going from “a little friction” to “no friction” often makes the difference between good habits and bad habits. In this case, I can open a new file and start typing code, instead of worrying about setting it up as a module.
AMD has many features, but I think that the extra markup and complex loaders needed to support it outweighs its benefits. Out-of-the-box it is not ready for production, and needs build tools to make it truly useful. If build tools are required anyway, a much simpler solution should fit most developers’ needs.
I am looking forward to your blog post pointing out the flaws in my argument and excoriating me for making a fool of myself in public.
A very sincere thank you to James Burke, Tim Branyen and Yehuda Katz for reviewing this post and helping me better understand AMD. Please consider this a strong opinion, weakly held.
Bravo!! A crisp and clean articulation of what was for me a weird smell around AMD. Adding “define” markup wrappers creates a system of meta-dependencies (possible errors, more layers). Test running an app can have verbose script tags so that debugging can indicate line numbers; minfied/packaged code (w/ closure compiler, etc) can run in production on very few http’age, and the air is more clear. Thanks Tom.
r.js is an excellent build tool. Anybody using AMD and not building is just as foolish as the next person not building. The difference is, AMD apps can get started w/o the build process figured out, and is a simple shell command away with r.js. Not a valid point.
See above.
You can’t require files/modules with a synchronous API, AMD or not unless you build. On that note, you can write your scripts like this
If you build with r.js
So, that’s 0/3 :\
I respectfully disagree and find your arguments pretty weak. My full response here:
http://geddesign.com/post/15994566577/amd-is-the-answer
Hi Tom, I enjoyed talking over the post with you. Thanks for sharing it. My reply is here:
http://tagneto.blogspot.com/2012/01/reply-to-tom-on-amd.html
I find the arguments pretty weak and would assume because there is a lack of experience with using AMD.
+1 I also don’t see a solution in AMD
See also interesting discussion on ES.next group: http://old.nabble.com/Harmony-modules-feedback-to33125975.html
I totally agree with James and Geddesign replies. AMD gives a better dev process since you don’t need to compile to test it locally and livereload is able to update the browser at each change, it takes under 1s to load a page on localhost and everyone should optimize files before deploy.
The only thing on this post that I almost agreed was the part that it says that if the project is simple you don’t need a build step and/or AMD loader, but nowadays I have to disagree, after I started using AMD the organization and amount of code reuse between projects increased drastically, it got way easier to share code since dependencies are resolved dynamically and you don’t rely on globals that much. Also all my helpers/widgets are written in the AMD format, so dropping the format is a bad choice even for simple projects. You can code an “AMD loader” in under 50 LOC if you remove support for plugins and lazy-load, but plugins are one of the reasons why AMD “kicks butts”.
Another huge benefit of using AMD is that you can have multiple devs working independently and you can expect everything will “glue together” without too much overhead, no name collisions, no missing dependencies, no race conditions, everything “just works”. Since creating modules is so easy it encourages you to think modularly and to decouple things, smaller files/functions are usually easier to understand.
http://blog.millermedeiros.com/2011/09/amd-is-better-for-the-web-than-commonjs-modules/
A lot of people already wrap their code into IIFE to avoid generating globals, it’s about the same amount of boilerplate, so the “ceremony” argument is invalid.
Protip: create a snippet on your editor so you don’t need to type the bloated wrapper, I only type “def” and press “ctrl+tab” and BOOM!! I’m ready to code an awesome module that can be shared between many projects and that avoids lots of headaches…
I agree.
Why use AMD then. Why not just use a synchronous call with the XMLHttpRequest object. I would imagine that it already exists, but I may be mistaken.
Saying AMD is bad because it loads a bunch of scripts is the same as saying using globals is bad because you have to write your entire app in one file.
All solutions require a build step. AMD makes it easy and interoperable (and unnecessary during development).
The ability to package was the main concern I had too.
I also don’t like the must have closure and return.
I have tried to take the best out of both worlds in yuno project but I still need to create a proper package builder.
Good to know I am not the only one with problems with both, current, solutions
http://webreflection.blogspot.com/2012/01/y-u-no-use-libraries-and-add-stuff.html
I disagree with those that say that by using AMD you should not use or care about build tools. I also disagree with those who say that if you are using a build tool then you shouldn’t care about AMD.
If you are using AMD with require.js you already have a build tool: http://requirejs.org/docs/optimization.html
Others loaders might have other solutions, and if not, you should still package things up for production properly.
If anyone who uses require.js thinks that they would not need to bundle files for production deployment, then they are bad developers and should be kept away from the web.
So what advantage do you get from AMD if you are bundling things up? Dependency management.
As you mentioned, people try to organize their code into many different files, and this leads to dependency issues, specially if you want to do lazy loading or more importantly, if you want to have reusable modules for different apps inside your company.
I agree with you when you say that the current way to declare modules is not optimal, but it’s a necessary evil we have to live with until JS has a proper module system.
Having all of this in mind, including that not having a build step is out of the question, I honestly haven’t seen a single alternative to this day that is easier to use or setup and that gives you the dependency handling advantage you get from AMD.
It would be good if AMD loaders would incorporate the lazy loading features you describe, which I also think are a really good thing, and that’s the good thing: these are not mutually exclusive things. They can and they should coexist.
I don’t think it’s a good thing to push for a non adoption of AMD instead of trying to improve it.
Both solutions are good, is not about right or wrong.
It depends of the project and the development environment.
One thing I notice is that when there is a need for some functionality in JavaScript many solutions pop up in the community; which is great. However, it would be nice if developers find a good solution and adopt it; it would seem that more adoption and development using the ‘good’ solution would lead to the said solution becoming event better or great and the use of the solution would become easier to adopt and implement. AMD is a good solution, some would argue ‘the’ solution. For now it is a viable solution and I’m using it.
Outside of the debate on what is better, I tend to look at things from experience. We ( my team and I) have built several very large applications for tablet and stb type of devices and both approaches were utilized in the different projects ( AMD being presented by requireJS and non-AMD being presented by closure tools).
From my personal point of view the code guidelines (consensually enforced by the compiler) are producing better code, meaning easier to read, work with and debug. On the other hand modularization of the such code is much harder and involves additional tweaks, while code using AMD is much simpler to separate as optional module.
One thing that makes it almost impossible to use large AMD/requireJS code base for a new project is the lack of support for the patter in any IDE. For example most javascript IDEs can handle the google inheritance model, so you can get pretty good idea on what is inherited and from where at a glance, while with AMD pattern the IDE has no idea what you are doing and you basically need to switch to help files and your code all the time. This makes it much harder to orient in large code base and makes the new developers in our team much less productive than they are with the other pattern.
This is all practical concerns and do not touch the other aspects like the quality of the produced code, footprint, possible optimizations and so on. Still, how fast you are able to produce something out of existing code base is important.
PS. while closure library requires “build” step this can be entirely automated with available tools ( like closure-script) so the edit/test cycle does not include the compilation from developer’s point of view, exactly as it is for requireJS development process.
AMD is a solution without a problem.
As is so much of the new stuff lately – I love the push for development in JS lately, its refreshing, but a side effect is that we are being bombarded with solutions that don’t have problems. Coffeescript is a prime example….
If AMD is not the answer does that mean that Intel is?
I like to use AMD because it increases reuse of code across projects, code gets modular out of the box when using AMD module definition, kind of dependency injection, good debugging capabilities as you get the correct url to the file and the correct line number where an error occoured AND good optimization/built possibilities even across different projects. It also switches the decision when and if to use globals to the users. I really like Ember and will use it though i will patch it.
But, what is the reason to set Ember globally – even on node.js? What is the benefit of that?
Is it possible to define a function without declare the dependencies, then the define function scan through the anonymous function and find out the modules need import?
Wondering why the writer and also many other people only see this build tool problem like this: “Build tool or no-build tool”.
No sensible front-end dev would deploy their web site/app to production with some JS loader (like RequireJS) downloading multiple (maybe dozens) files on first page load. There has to be a way to concat (and minify/compress) those files to a single file. Therefore a build tool is needed.
BUT I wanna shoot myself every time I need to “build to development”. While building the site/app this necessity of running build process/tool after every small change to code while developing is too time consuming.
There needs to be a way for working in development environment without having to use build tool after every edit. I’m just starting to look into AMD and RequireJS (so I am no expert on this), but the idea of loading resources “on the fly” while developing and then for production running the optimizer which concats/minifies/compresses JS seems the ideal way. Though not sure how in practice that will work out.
You are my HERO!! Proponents of Dojo and AMD are misguided by the retarded assumption that a server (or build step) is completely stupid and only capable of sending back static files.
Most of this AMD crap is just metainfo/annotations that could just as easily have been captured using some comment syntax, XML, or any other format besides something that appears to be actual javascript code.
In our environment, AMD-style javascript is completely rewritten so that calls to require or define never occur. Those functions don’t even exist at runtime.
Another mistake made by AMD was its loosely-defined plugin concept. Dojo has already abused plugins with their “domReady” module. So something intended as a way to express resource dependencies is now being used to hook event listeners. Plugins should only be asynchronous in the absence of an optimized build phase, they shouldn’t be asynchronous by definition!
This means it’s impossible for the dojo/domReady “resource” to ever be synchronously available in production, regardless of any optimizations an AMD implementation might want to use. So every downstream module must get shoved into a big dependency graph at runtime and then the graph is torn apart when the “resource” (read “event”) finally occurs.
When it comes to real problems, like the fact that there’s no way to know which CSS resources are required by each widgets, Dojo punts.
@Joshua: I can’t agree more on your saying “a solution without a problem”. That is exactly what it is. So is coffeescript. I think CS is for RoR/Sinatra aficionados that want the same language structure in JS as well. Although it might speed up development for less-js-knowledgeable, I think it is counter-intuitive to me. Pure JS is just great. I think some say “Solution without a problem”, others say “Evolution”. We also have to consider that we don’t necessarily need a problem to breakthrough a new technology.
On the AMD matter, I think that I cannot see far enough to know for sure what’s good from what’s bad. I think structuration is good. And I think dependency management ( although it can be a huge pain ) is a great thing.
I think your concerns are valid, but I’m not so sure its an issue with require, but perhaps how its used, particularly on large projects that may have user configurable\optional deployment of widgets.
For me the biggest area of contention is that the “module” unit we want to deploy is less granular than the “module” unit you develop with, for example, you may develop using a define per JS file, but then aggregate those files into a shared library (alla assembly, .jar, etc). The problem is, that when you do that dependent libraries can no long (easily) refer to the individual parts that make up that package (as they are no longer in separate files).
With out code, we have a background compiler that on the fly packages the code into the same structure that you deploy, we also generate source maps so we can debug the original files…, http://www.html5rocks.com/en/tutorials/developertools/sourcemaps/
With this approach our “defines” are at the package level, not the file level. I realize that for some people this will immediately sound like a hack, but think about it, the unit of development matches the unit of deployment, dependency management between packaged “modules” are easily accessible in a single place (rather than scattered across multiple files), the way a dependency is referenced is consistent across all files that make up the packaged module, we can still debug individual files. Also lets be honest, is it really that important to see which individual types are in use on a file, in java, c#, etc. we include a namespace, not each dam type we want to use, and even then, how many of us never look at the include section? and how many refactoring tools support remove unused namespaces!!
I created a similar loader to the one you mention. The big drag with that approach is that you have to worry about the ordering of dependencies. It’s really useful to have a dependency loading mechanism like most other programming languages have, because it makes it easier to apply the single responsibility principle. Using the requirejs optimizer and almond, your production code only issues one http request for the javascript as it should.
Another thing that’s great about AMD is that you don’t have to duplicate the list of files that comprise your application from your application, your unit testing environment, and your build script. So you can follow DRY, which really saves a lot of pain.
There’s room for all approaches though! Requirejs 2.0 is awesome in that you can get the benefits of not worrying about ordering or having a list of dependencies even with non AMD libraries by using shims.
And a quick clarification on the above comment. AMD / RequireJS makes it so that instead of having to duplicate your list of dependencies three times in order, you no longer even have a list of dependencies.
Tom, I am using AMD (RequireJS) but I would like to test your approach. Do you have some sample app using your approach?
I can’t emphasize how much the require anonymous calling makes me want to pull my hair out as a JavaScript developer. I’m dealing with it right now in DOJO and I honestly don’t understand why every library would be loaded asynchronously in this manner. It’s horrible inefficient to request libraries after you already loaded the page in my opinion. What other languages are set up so that your actual code is a call back from your dependencies?
It’s backwards and difficult to troubleshoot.
I agree with the other ways of doing it. I’d rather have a JavaScript equivalent of an import keyword, but until that time, it needs to be as non-asynchronous and non-wrapper based as possible.
I just feel like I need to emphasize it again. Why oh why would module loading always be asynchronous? Is it just because JS doesn’t have an import keyword, because I have to tell you, the idea of dynamically loading files piecemeal as you start to run code has to be one of the worst things I’ve seen.
I’m sure it will lead to lots and lots of situations where code starts breaking halfway through running because of chaining dependency resolution at runtime. Which, by the way, is precisely what I have seen with DOJO’s model of heavy AMD usage.
If DOJO is some sort of example of a way in which AMD is useful, I’m really starting to see this as a solution without a problem.
I really think the solution is to actually get an import keyword in JavaScript. Until then, just manually load your files like we’ve been doing for the last decade. At least you’ll know at page load that one of your JS files has a 404.
Yeah, I also disagree here. AMD with Require.JS is the answer to the SDLC process with any small to large sized 100% javascript based app.
If you have ANY back-end HTML rendering going on (PHP, ASP.NET, JSF, Ruby, HAML, etc) you should probably just get off AMD all together…
For an app that does pure HTML5 and only calls the server for REST operations or socket.io calls require.js is a godsend that has proven itself a timesaver and a powerful part of the javascript workflow. Live refresh and the development stream is drastically reduced.
People who have puppet pages being rendered by a server (A) don’t use it and (B) why are you living in the stone ages?
While I think AMD is good for fast project development, I heavily agree with the above.
I think alot of the problem with AMD is that it could be improved by contributions of technical writers improving the documentation.
For libraries I write, I will only support AMD in the same way that jQuery and lodash do, with a hybrid approach where they create one global using IIFE – and then also have a call to define() to make the AMD loader (e.g. require.js) happy.
For opponents of AMD, I say give us “code not words.” AMD works for dev & deploy, and it isn’t optimal but, give, us some thing that is.
For most websites and most web developers out there, not in conformity with performance optimization, using no build system and loading 12-18 scripts synchronously, an AMD loader is a huge improvement.
@Andrej Kaurin I’ve been working on an open-source implementation of the hybrid-string using the CommonJS model. You can find it here: http://github.com/TheHydroImpulse/Resolve.js
Tom about the build tools: I think that sometimes we need to trigger a compiler step by hand because of a limitation of our tools.
I worked with Smalltalk like 3 years.
In Smalltalk there is no difference between run/debug mode, you can run expressions in every part of the editor. And that ability is a perfect match for a dynamic language: I didn’t miss autocomplete at all; but when I went back to Java development… I really miss the ability to evaluate things while I’m coding (ie doing TDD was a pleasure in Smalltalk).
In some sense the browser and JavaScript brings a similar experience to Smalltalk. Which is not related on how big your application is.
See this Gilad Bracha post on the topic: http://gbracha.blogspot.com/2012/11/debug-mode-is-only-mode.html
Two things AMD pattern require beats CJS require or anything else:
Any other “improvements” or “differences” are fluff.
After porting Curl.js (both AMD and CJS-compatible loader) to support passing args to the required arg, I can coerce it to do POST for the data. This way I can get template, state-specific-data and the business logic in one bunch:
There AMD actually transcends the build by serving the real-time data arrival sync needs. You can’t “build” real-time data into compiled, minified js can you?
The “jsonrpc!” part above is our specific JSONRPC AMD plugin that does POST.
So, while both your and James’ arguments are about trivialities of static asset dress-up and compilation, the only thing that is actually good about AMD seems to be missing from the discussion – dead-simple, flexible-source async asset arrival sync.
I hate the AMD dress-up as much as the next guy, and boy am I disgusted with limitations of r.js (literal strings for required args only), but right now and until JS.Next is a practical reality in some 2-3 years, AMD is just freaken magic.
Hi, Tom!
Surprisingly, the LMD: Lazy Module Declaration corresponds to 100% of your requirements.
I develop LMD almost 2 years. I saw the title of this article, but did not read, it is surprising that our ideas are the same.
There is plugins for all “special resources”: async, CSS, AMD. CommonJS module wrapper. Lazy initialization. Static build analyzer and optimiser. Powerful CLI tool, and upcoming GUI.
Please take a look: https://github.com/azproduction/lmd