Imagine a Beowulf Cluster of JavaScript Frameworks
by Tom Dale
Thomas Fuchs recently wrote about the advantages of using JavaScript micro-frameworks:
I for one welcome our new micro-framework overlords—small frameworks and libraries that do one thing only, not trying to solve each and every problem, just using pure JavaScript to do it. Easy to mix and match, and to customize to your needs.
He also pans “full-stack” frameworks:
These libraries are HUGE, being 100+ kilobytes in code size, and even minified and gzipped they are pretty big. … A whopping 100% of sites or apps using these libraries don’t use all the features they provide.
His argument sounds good, in theory. The idea that I can pick and choose among a number of well-written and focused JavaScript libraries to assemble the ultimate bespoke JavaScript environment seems very enticing indeed.
The golden rule of software, however, is that unless it is designed to work well together, it usually won’t. Mr. Fuchs has apparently never heard of dependency hell. Dustin Diaz has done a great job of putting together many of these micro-frameworks with Ender.js, but as a curator, he has to rely on the original author if he wants to make a change.
This attitude also continues the trend of giving developers overwhelming choice. Convention-over-configuration and emphasis on developer ergonomics are key to getting real work done. Of course Mr. Fuchs is able to tell you which JavaScript library will precisely match your requirements—he writes JavaScript libraries in his spare time! For the rest of us, we don’t really want choice; we just want what’s best.
If choice was more important than well-tested integration, the majority of attendees at last week’s CodeConf would have been carrying ThinkPads instead of MacBook Pros. Instead, developers want a happy path that they can be reasonably assured will work well. If it doesn’t meet their needs, they want the ability to augment or replace the components that make up their integrated system.
Here’s the reality: As web applications get more complex, developers end up needing all of the miscellaneous nuts and bolts that they thought they would never use. Take Flow for example. Flow is a killer task manager web application built on Backbone.js. Backbone.js often advertises that it is only 3.9kb minified and compressed. How much of Flow’s nearly 900k of (minified!) JavaScript do you think is the application developers filling in the deficiencies in Backbone?
SproutCore is often dismissed because of its size. Of course minimizing size is important. But if it takes a megabyte of JavaScript to ship the features you want, better the bulk of that code live in a framework instead of your application. Living in a framework means that it has been reviewed and optimized by an entire community of developers, and improves without any effort on your part. New Twitter currently clocks in at over a megabyte of JavaScript:
To the best of my knowledge, Twitter is not using any third-party libraries except for Mustache and jQuery. That means that a huge amount of JavaScript was written by Twitter engineers just to provide the framework in which the application runs. This isn’t stuff specific to Twitter. This is, for example, code that determines how to propagate changes to the DOM when the data model changes via XHR—a problem that all JavaScript developers face.
SproutCore may be big, but it’s because web applications are getting more complex every day, and we all have a common set of complex problems to solve. Oversimplifying the problem is disingenuous and forces each application to fend for itself. Mr. Fuchs says:
Small code is good code: the fewer lines in your source, the fewer bugs you’ll have, plus it will download and execute faster.
If creating a modern web app requires a megabyte of JavaScript, would you rather write, debug and optimize all of that code yourself?
It’s time to lay to rest the argument that full-stack libraries are unable or unfit to solve real problems. Instead, they are best suited to tackle the problems faced by modern web app developers. The same criticisms full-stack frameworks receive today are eerily similar to the same criticisms levied at jQuery several years ago. Unless you are planning to discontinue your application in six months, it’s time to start developing for the future.
Thanks to Charles Jolley for reviewing this post. I tweet at @tomdale if you want more timely updates.
This is a helpful counterpoint: If there’s one thing that JS (i.e. web) developers are doing these days it’s trying to figure out how the hell to approach frameworks with any given project. The “set of many” does introduce tons of little API issues that could make something brittle. It does seem that Sproutcore’s recent goals have been à propos to Mr. Fuch’s ideas: We should all strive to eat more leafy green vegetables and exercise.
I agree. But I think the developer should just remember to pick the right tool for the job.
If I were to build a small website (maybe a blog or an informational website for a retail shop) which would need some DOM manipulation here and a bit of AJAX there, maybe I could get away with using on of the smaller micro framework which Thomas Fuchs advocates.
On the other side, if I were to build a large rich application, I would look for a full stack framework that supports my requirements.
I started to learn SproutCore and I like it for that sort of situation. With a very small amount of code I can build very complex features. When I looked at the sample Todo app from backbone.js, I was surprised to see what seemed to be a lot of code for a simple app (lot of code outside of the framework itself). I also like that SproutCore provides organization guidelines (what goes where). It also has some nice more advanced feature such as the Chance CSS framework (sprite my images for me out of the box without me having to do anything). This sort of thing allows me to focus on what my app does and not how to build it (Isn’t why we pick frameworks?).
Now I don’t think Thomas Fuchs is advocating against the large application frameworks. My takeaway was more like: “If you are building a small website, you don’t need a power hammer, a simple hammer is enough”. That would be my first example above.
Like Rebecca Murphey said in the Jquery Divide talk: analyze the need then pick the tool (slides: http://www.slideshare.net/rmurphey/the-jquery-divide-5287573 and video: http://jsconf.eu/2010/speaker/the_jquery_divide_by_rebecca_m.html).
About dependency hell: this can definitely be an issue. I maintain a large app using jquery and a myriad of plugins. Those plugins tend to break every other jquery release
Now, does large framework protect from that problem? Maybe, maybe not. What if I use a feature of SproutCore that gets deprecated? It might break my app if I try to upgrade.
Bottom line: I think there is room for small/single purpose libraries as well as larger/multi purposes one as well as bigger full stack frameworks.
I feel like the JavaScript-based applications explosion is only to come.
We’ll see what will happen to all these frameworks.
Also, with independent micro frameworks, the individual pieces aren’t able to take advantage of any core functionality. Much of jQuery is built on top of other parts of jQuery, so if some parts had to work independently, they would have to implement (perhaps duplicate) whatever utility methods each part needed, since it couldn’t rely on core always being there.
I think this could lead to a library that, when combined, is ultimately larger than if it had been built as a single unit. I still think the idea is a good one though, just something to consider.
Very well articulated.
“Of course Mr. Fuchs is able to tell you which JavaScript library will precisely match your requirements — his job is writing JavaScript libraries!”
[Emphasis, and presumed outrage, yours.]
Come now. That’s simply false, and if you think about it, silly. “He writes open source frameworks for free so he’s biased against open source frameworks”? Really? Thomas has never sold a JavaScript framework or written one for hire, and his libraries are MIT-licensed. It’s not his “job”. Furthermore, Prototype.js and Scriptaculous are not micro-frameworks. So Thomas is arguing against major examples of his own work. Yup, must be some kind of evil hidden agenda.
[Sidenote: I called Tom out about this misrepresentation on Twitter and he probably changed it by now, but I for one think it's important to know about the history of such things.]
“Mr. Fuchs has apparently never heard of dependency hell.” Very logical argument, that. Way to slyly insinuate he’s a bad and inexperienced programmer, without actually introducing any actual facts.
“Dustin Diaz has done a great job of putting together many of these micro-frameworks with Ender.js, but as a curator, he has to rely on the original author if he wants to make a change.”
Really? How is that a counter-argument? I’d be very surprised if someone has read Thomas’ little essay on micro-frameworks and genuinely come away with the idea that what he really supports is taking a bunch of other people’s OSS projects and mushing them together with an integration layer.
The whole point about micro-frameworks is you don’t have to make them “go together”. You wield them individually like scalpels instead of spinning them en masse like Edward Scissorhands.
There are a LOT of problems with an undertaking like Ender.js, but that’s not the fault of micro-frameworks “not being made to work together.” It’s the fault of doing something that, from the outset, is fairly expected to be more trouble than it’s worth — however noble it may be.
One might argue that an integration layer that hooks up a bunch of other micro-frameworks is no better than — and in a lot of ways, worse, and more complex, and less reliable — than a monolithic library. So I call “red herring” on the Ender.js argument. It doesn’t really support your attack on Thomas’ article against monolithic frameworks, because it is one… just made from parts, like Frankenstein.
I could go on, but I’m even boring myself at this point. What I’d like to know is: Why is the suggestion “You don’t need all that code all the time” considered so radical & threatening?
Twitter and Flow are great examples. You either implement the backbone of your app well, using something like SproutCore, or you wing it. There’s no way Twitter would be as large as it if they used SproutCore — and it would have taken a lot less time to develop, too.
@JP I somewhat agree, and somewhat disagree.
First, how many of those “small” projects grow into large projects over time? Many, if not most in my experience.
In my mind, either the project is small enough to just use jQuery for decorating the page, or it is going to end up being a RIA, in which case using a full fledged framework it likely the way to go.
I actually think that it is the fault of the framework developers that this problem even exists. When you have to choose between pulling everything in from a framework or nothing then using the micro-frameworks make sense. But if the framework is modular enough to only pull in what you need, then I don’t see a downside of allowing yourself to scale up over time.
It’s the difference between the Rails 2 VS Sinatra, and the Rails 3 VS Sinatra. I used Sinatra for small projects when I had to choose between the monolithic giant that was Rails and the tiny library that fit what I need. But now that I can pull in only what I need from Rails, I don’t need to fragment my work between frameworks. And when I want to add more features or functionality to the application, I can get all the benefits of the best practices, etc from the framework I generally work in.
So I’m all for “If you are building a small website, you don’t need a power hammer, a simple hammer is enough”. I’m just saying take the simple hammer from the set of tools that you know you generally work in, and allow yourself space to grow.
My two cents
@JP:
Part of the problem we’re trying to address with Amber is that simple applications don’t always stay simple. Applications that only get developed for a weekend may be ideal candidates for micro-frameworks. However, in practice most applications are developed further and in time there will be a growing amount of functionality. The framework you choose in that case should be there to help you bring your application forward.
I can’t speak for other frameworks, but we’re trying to make SproutCore the ideal solution for those types of applications. They can start HTML-based and iterate quickly towards something more complex, without needing to roll your own solutions or worry about integrating other potentially incompatible code. Our work is still ongoing and there are still weak spots which will need to be addressed, but this is where we’re heading.
@Amy
I have tremendous respect for both you and Thomas and I’m sorry if the anything in my post came off as disrespectful. My point in saying that his “job” was writing JavaScript frameworks (which I have since updated to better characterize his relationship to them) was not that there was anything untoward going on. My point is simply that he is comfortable making the tweaks and modifications necessary to wield the power of multiple disparate libraries. Quite simply, he is a JavaScript hacker of the first order and is more comfortable taking on that job himself than at least 80% of JavaScript developers out there.
I felt like I had to write this post when I read this:
The suggestion that “You don’t need all that code all the time” is not the problem. The insinuation in the JS community seems to be “You never need all that code,” which I believe to be false.
Again, no outrage on my part. I just think that we need to address the reality of how modern web apps are built.
Oh man, Tom, you are so right about all of this. Thank your this blog post, I will use it to refer to people asking for micro frameworks.
An interesting debate. But I think the problem is that both sides are trying to warp and narrow the playing field to suit themselves. The truth is there are plenty of projects that will never need all the machinery of SproutCore. And there are plenty of projects that start out too small and then struggle to grow more complex. Choosing your tools is hard. It should be hard and it should be taken seriously. Neither side should be trying to convince people that it’s an easy decision. - “This is why you should always just start with SproutCore”. - “This is why you should just start small and integrate micro-frameworks as needed”.
Both bogus arguments really.
I’m also quite sure that neither side really wants to disparage the other side. They just want to make sure their own side isn’t being disparaged or misrepresented. This is what most internet arguments are made of unfortunately
What would be more awesome IMO is if we started talking about when it’s a good idea to take either approach. Let’s hear some use cases. And let’s stop using Twitter as an example. There is only 1 Twitter and their requirements are not going to be representative of the web dev community at large.
Tom, sorry for leaving this mini rant in your comment box. I hope T. Fuchs reads it as well and we can actually get some constructive info flowing. It is a well written piece and I enjoyed it.
I think that the huge implication of all of this is that the market for JavaScript dev’s needs to move away from (insert generic framework) developer to JavaScript developer. By being flexible and being able to write in any framework because you know the language then you have the ability to do either.
Micros are good for small scale things, such as needing functionality on say 2 pages of a web application and creating a shim for any backwards compat. But for full scale web applications you don’t need to reinvent the wheel.
@Greg Moeck
I really like your analogy about Rails 2 vs Rails 3 and Sinatra.
About small projects growing into large projects over time: I think this is part of identifying the needs for the project. In my previous comment, the 2 small examples I was referring to (small retail shop website and blog) were obviously from the start not RIAs. You are spot on when you say “decorating the page” vs building a rich client. This is exactly what I was doing then, decorating the page.
I think it should be fairly straight forward to decide from the start if one is building a website or a rich web app
First case pick a small library, second case pick the big framework…
Well not so fast
Maybe the website is not simple and needs to handle a lot of user interactions and other complex widgets with some remote data fetching… which might justify the use of a large framework.
One of the issue I would see with using a big framework on a traditional website would be the javascript requirement. What if my user base does not have a JS enabled browser? Because all of the app markup might be generated by JS, this might be an issue. I am wondering if we are going to start seeing full stack JS framework render on the server side with node.js to provide an alternative? (kind of like that demo of YUI3 that renders HTML with node.js on the server and also provide client side behaviors with the same code base)
@Colin
I am really interested by what you are explaining. I got the concept behind SproutCore and started to build a toy app to learn. It all works great.
I also followed the story behind Amber and the new SC.TemplateView leveraging Handlebars.js.
What I am missing though is how to integrate SproutCore with legacy HTML apps. I was discussing this point this morning with a colleague and I was wondering how to go about that? I can not just add a script tag with src /js/libs/sproutcore.js and start using the SproutCore infrastructure. Part of the my problem might be that I have not yet looked at the build and deployment process of SproutCore.
Basically, imagine I have a complex app which use PHP or Ruby on the server to generate HTML and has a lot of client side enhancements implemented in JS. It also as a lot of specific CSS styles. How can I use SproutCore today?
I’d love to learn more about this, thanks for your reply!
[Reposted from HN]
As co-creator of the NOLOH Framework (http://www.noloh.com) it’s crazy for me to constantly read these posts, clearly we don’t do a good enough marketing job. One of the benefits of NOLOH is it’s lightweight and on-demand nature, that everyone in this thread, and in the blog post seems to be requesting, but without actually identifying it.
In the case of NOLOH, lightweight and on-demand means that the server sends the client only the necessary highly optimized client-side code for their application specific to the user’s device at that current point in time, resulting in faster initial and continued loads. Similarly, as the user continues to use the application, NOLOH continues to send only the necessary and optimized code. This elkminates the fat-client problem, as the user only has what they absolutely need, specific to them. Similarly, in the case of search engine robots, NOLOH sends standards compliant semantically rich content, without the other baggage.
This is accomplished by implementing different renderers for each target device, so rather than use a general client-side library that loads everything it could possibly need for all browsers and situations, NOLOH has specific variations for each browser, version, and device. The ever growing number of browsers and devices demands this.
In “Lightweight, On-demand, and Beyond” in this past November’s issue of php|architect, http://www.phparch.com/magazine/2010-2/november/ (sorry for the pay-wall, we’ll repost it this month as the exclusivitey period expires), we go in-depth explaining the next version of our lightweight and on-demand functionality, where we make it easier for us to maintain an ever growing number of target devices, while sending even better code to the client, without any drawbacks on either the client, or the server.
It’s been very interesting reading all the posts regarding event-driven programming, fat-clients, and unnecessary bloat. I remember initially thinking about these issues in 2005 when Philip Ross and I first created NOLOH, we were young and naive at the time and like all good solutions, didn’t have the status quo entrenched in our thinking. So when I read these posts and they have the same gist as our initial white-papers it saddens me that we haven’t made as much progress as I thought we would’ve, it also makes me feel old.
@Tom, I don’t think you need to spend a lot of time genuflecting and apologizing to social media gliterati for their poor reading comprehension skills. It’s obvious that you appreciate the continuum between the web app that is small now and will forever with complete certainty remain small on one end and the ginormous web app that lead to an invocation of Greenspun’s Tenth Rule—look it up, kids—on the other.
If you don’t know that a project is a bonzai tree that is going to yield to one’s elegantly-wielded scalpels for the rest of time, it may makes sense to take the hit in terms of download footprint to pick something that hangs together at least somewhat elegantly now, lest you be trying to battle the kudzu of web client library dependency hell.
Taking the time to meticulously curate your web site’s library dependencies is a waste of time. (Dare I call it a perfect faux-nerd hipster hobby?) It’s re-arranging your record collection chronologically by date of acquisition on the burning deck of the Titanic with a fiddle in your hands as other people pick a framework and get to work and kick your product’s butt.
I disagree. Frameworks – small or large – all have a major problem: if your framework doesn’t support something you need, you’re either out of luck or you’ll have to figure out a way around the walls of your framework. I’ve seen developers spend tons of time hurdling themselves against these imposed barriers. In addition, choosing a framework that meets all our requirements is pretty much impossible unless you know all your requirements at the beginning of the project and they never change.
Frameworks are also pretty much “all or nothing”, throwing everything our industry has learned about modularity out the window. So yeah, you’re gonna have lots of code on your page that is never used.
But here’s the thing. The web IS a framework. The integration level is just good old JavaScript running in the browser. Integration is especially painless when you organize your project as AMD modules using RequireJS or another loader. Then you simply pick the best tools for the job as dependencies. Need history management? Use Sammy.js. Need cross-browser DOM manipulation/events/ajax? Use any of the popular libraries (jQuery, mootools, dojo etc) Need templating? Give mustache a try. And never forget you can create your own custom modules to solve new problems, like pub/sub or CSS3 transition queuing.
The point is, if you treat the web as a framework, you can cherry pick the best of breed solutions and create a truly custom solution. Even if that solution isn’t 100% defined when the project begins.
Both micro and full-stack frameworks have their place. In my experience, however, most of the time I’m much happier and much more productive working with one well-designed framework where all the parts were designed to work with one another. I like the approach of having a suite of micro-frameworks from the same group of developers, enabling you to pick the pieces you need and add more pieces later while ensuring that they all work together. Combining lots of different micro frameworks from different authors to do the job sounds good in theory, but in practice you simply need communication and a common roadmap and goal to make for a pleasant development experience.
And one last point: While most (actually, all) apps start small and might just need a tiny piece of functionality from a “big” framework in the beginning, the huge majority of applications quickly grows in complexity. So dismissing big frameworks because they have lots of features you don’t need right now might not be the best way to go.
So pick the right tool for the job, keeping in mind that your app will grow in complexity.
Micro JS frameworks are good, but they always take me a lot of time looking into their features to make sure they have what I need.
In some cases, when I have time, it’s good because I’ll have the right tool with minimal resource. But in some cases, I need a bunched tool that has all things I need, and that leads me to jQuery.
From this point, I think authors of these frameworks should make a good documentation for them. It will help other people understanding the idea, philosophy, and how it works. Open source community can improve it, leave comment, etc. And users, like me, can have time for doing the main job, while using the frameworks.
You response to the “small code is good code” is exactly what I thought. The larger libraries that are widely supported will already have few bugs due to the testing and improvements provided by all the developers that use those libraries. You won’t have to worry about you or someone else recreating those features (and possibly introducing bugs), either in your apps or in a microframework. Plus, those bigger libraries will take care of a lot of code you would have written, resulting in you having to write and debug less code.
So we don’t want choice; we just want what you’ve chosen is best for us ? No thanks I think I know my application requirements better than you do Mr. Tom Dale.
The true failure of your argument is there is no definition of “best”. Balance and causality are a universal laws and there is no one single approach that can possibly be the “best” for solving every web problem. You give something away to gain something, simple as that. Some people don’t want to use your framework despite its high quality and suitability for some problems, get over it.
Your effectively advocating “the world is flat and anyone who diverges from the truth is a heretic”. Such philosophy has been proven to hinder progression and innovation. Diversity and new ideas is a good thing.
This post is so obviously more about Mr. Dale pushing his own agenda rather than providing any pragmatic information for web developers.
This thread is a bit old but it keeps coming across my radar via twitter, etc, so I’m compelled to respond as someone who is considering a switch to a micro-framework based approach for a new app.
First of all, the argument that a more cohesive library can provide a better experience for users (not developers) isn’t very compelling to me. I’ve found that tools such as jQueryUI, ExtJS, etc never provide more than an adequate experience out of the box. If I want to change customize something there’s a steep learning curve, and usually I could implement aspects of the UI from scratch in less time. More insidiously, I think libraries like these then encourage a “good enough” approach to UI design that over-leverages the library. For me, UI design is the most important competitive advantage an app can have so libraries that provide these sorts of packages are useless to me.
Limiting then the discussion to developer-facing functionality, I think the case for micro-frameworks is pretty strong. It sets up an incentive structure for library developers that didn’t exist before. In the micro-framework model I can create a small but useful project on github that might be easy to care for, document, test, and promote myself. I can scratch my own itch without trying to insert myself into a certain community and get buy-in for a change. I can also get much more credit for he project. I can still interact with other developers using the library, or bring on contributors, but it’s a much easier way to start sharing code. I don’t have to reinvent every wheel either, because I can use tools like npm to include popular dependencies like underscore.
This scheme might be a bit disorganized, but it could also lead to a more vibrant ecosystem of tools that thrive based on merit rather than the politics of a particular community. And there would be strong selective pressure for the most popular libraries to fit together cohesively.