Case study: How & why to build a consumer app with Node.js

Node.js has been getting great press for being used to build real-time web applications and fast networking tools that help big web sites run and scale.

But is Node just as good a fit for web agencies and developers that build dozens, scores, or even hundreds of sites a year?

Making the switch away from tried-and-tested platforms such as Ruby, PHP, Python or .NET for a fresh technology barely out of its infancy, especially when it’s not your product and when you’re working for global brands that simply can’t fail, is a risky proposition.

So why would you want to use Node.js? As a high-volume web agency, here are the reasons we chose to go with Node and the tips and tricks we learned along the way.

Why we chose Node.js

» It’s JavaScript!

Every developer also knows at least some JavaScript. Introducing Node.js, then, is relatively easy.

Everyone in a given web shop knows the basics and just has to learn about the event loop, callbacks and to how to use async flow control.

At our own agency, as people got to know Node, we actually saw that our browser-side JavaScript code improved in quality and structure.

» Code re-use at every level: browser, back end & database

JavaScript is the language of the browser, but JavaScript also powers many of the new NoSQL databases. We tried a couple of them for building content management system and quick fell in love with MongoDB.

MongoDB uses JavaScript for querying data, which means at the very worst that we can copy and paste code and use it in different layers of the system. What might be written as a parser for the browser might be used to format a report executed on the database.

Taking this one step further, we are standardising the include mechanism to actually reuse code and modules across the layers. This means all layers can include the same file, massively reducing the maintenance needed of code and cutting down the time required to write tests.

» Strong, responsive and enthusiastic community

IRC, meetups, bloggers, Twitter and Github are all alive with the chatter and support of Node.js. But not only are they alive; there is an excitement which you don’t normally see on this scale.

So for our developers and many others, this is an invaluable resource when bugs, issues and obstacles do occur.

» Large productivity gains in HTML & CSS using Jade & Stylus

HTML and CSS guys love working on Node because using Node means we’ll be using Jade and Stylus by TJ Holowaychuck. We loved HAML and SASS before Node.js, and now we can’t imagine using anything  other than Jade and Stylus.

» Performance and scalability

We have found that Node.js scales really well. The non-blocking event loop allows for a phenomenal amount of traffic compared to our old, highly-optimized PHP stack running through Apache.

On our first project for a national newspaper, we used a front-end nginx proxy for load balancing to the various node instances and were ready to add extra instances during the peaks caused by TV and national newspaper advertising. The peaks came but the load on the first instance stayed low.

The single Node.js instance hardly broke a sweat, despite seeing one of the highest requests per seconds we’d ever seen.

With PHP and PostgreSQL we could scale up, but it felt really hard and gave us many sleepless nights. Using Node.js with a MongoDB backend scaling up is quick and easy; but because Node can handle more traffic, you don’t need to as quickly.

» Wealth of hosting options: No.de, Joyent’s SmartMachine, Heroku, Nodejitsu

We host our production sites on our own private cloud, but for smaller companies there are many cost effective and easy to setup hosting providers. We’ve got a couple of new Node.js projects in development and will be deploying them on Joyent’s SmartMachines.

» Make your developers famous

I always try to ensure our developers work on open-source projects in contracted hours as much as possible. Encouraging them to start projects that benefit the community as well as the business.

When building projects for our clients, we look for modules that have common functionality which could be packaged up and made into open source projects:

For example, Gzippo is connect middle-ware developed by one of our staff to perform gzip compression of static assets.

We like to manage compression and expiry times of static assets in our Node.js layer because it gives us the freedom to either serve directly from Node or to stick a caching proxy in front.

This way it is served from the caching proxy for all other requests and keeps all the configuration in the Node app, meaning you only need node.js on your development machine to work on projects and do QA using Pagespeed or ySlow.

» Developer happiness

There is something intoxicating about coding JavaScript on the server side; throw in the event-loop and it’s heaven for developers. They just can’t get enough of it!

Maybe it’s because we’ve been coding PHP for such a long time, but Node.js has really inspired our developers. If you were to ask any one of them, “How do you find coding in Node.js?”, you can bet your bottom dollar you’d hear “I love it!” right back.

A sample Node development stack

This is a snapshot of our ever-evolving development stack. There are a couple of staples, but for the most part if we find something better we can just switch it out.

• Node.js

0.4.12 and 0.6.6

• Binary management: n

Point releases often break backwards compatibility. We manage our version using the excellent tool n, also by TJ Holowaychuk.

• Flow control: Async.js

Anyone wandering into the world of Node.js quickly make a choice how they are going to handle flow control. There are a number of methods out there. We’ve stuck with one of the most popular Async.js and depend on it in nearly all modules.

• HTTP, routing and view rendering: Express.js & Connect

The de facto standard Node.js web application framework is the Sinatra inspired Express.js which builds on top of the Connect middle-ware framework by Sencha Labs.

• Presentation: Jade & Stylus

We use Jade as our template engine in the view layer to generate the HTML, and we use Stylus for creating CSS. If you’ve not checked them out then you should, they are a compelling reason for using Node.js. Here is an example of the sort of thing that our designer are doing with Stylus.

• Homebrewed Node modules: Serby on Github
• We’ve built a number of Node modules which we reuse from project to project. Validation, data mapping, et cetera — Basic utility stuff that epitomizes our style of development.
• Logging: Winston

We use Winston writing to files and also pushing log data to the excellent Loggly cloud logging service, which consolidates the logs from each of our node.js instances.

• Testing: Mocha

There are a lot of testing frameworks for Node.js. Knowing which to chose it hard. We looked at a few and made the decision to go with something very simple, in this case Expresso and Should.js. New projects are now using Mocha, the successor to Expresso.

Conclusion

We are on our fifth large project with Node.js (for us, “large” is around 150 worker-days) and are continuing to support the previous projects whilst developing a handful of small and medium-size projects in between.

Less than a year has passed since we adopted Node.js, but the project stats are already looking really good. Initially, the projects we undertook did take longer as our developers learned the intricacies of Node and as we found our feet with the hosting, editors and tools needed to develop and deploy a large scale applications. However, our last project in node was actually under budget, something that rarely occurs at Clock.

Once you get used to event-based async programming, settle on a set of standard development patterns and an architectural style. You quickly start to reap the benefits of working solely in JavaScript. This makes Node.js not only fun but extremely effective to program in.

Paul Serby is CTO of Clock, a UK web agency, where he directs technology choices, architects software, manages large development projects, writes code, makes tea and preaches the virtues of good software design. Clock developed Sun Perks, one of Britian’s largest consumer facing Node.js implementations for News International. Followed shortly by another ground breaking service, the Eat Out Dining Card for The Times. Both leveraging Node.js and a whole host of young and exciting supplementary technology.<