Developers name their top tools: GitHub, Heroku, and yes, Azure

Companies making developer tools are in a cutthroat battle for customers, revenue, and mindshare.

The battle has many fronts: The growing army of developers, the need for speed on software development, the popularity of work-from-anywhere and remote team models, and the growth of mobile coding all add up to a broad market ripe for innovation and monetization.

An army of devs: The surging need for developers in a software-oriented economy has led to the rise of coding bootcamps and online training courses. The learn-to-code craze has spread its roots into working-class high schools, corporate cubicles, and every place where individuals seek more exciting careers with an almost infinite ceiling for salaries. The explosion of new developers has led to a correlating explosion in the need for easy-to-use tools for beginners and moderately skilled coders.

Rapid development: Software development has also increased in pace significantly due to trends such as the lean startup movement, minimum viable product/rapid prototyping, and continuous deployment. In the 1990s, you could release a boxed software package once a year, taking the time to debug, test, and thoroughly perfect the product prior to shipping. Today, flaw-filled “alpha” and “beta” releases often occur within weeks of one another, with final and relatively polished products following by a few months. Subsequent releases occur in fast succession, each showing measurable improvements in design, flow, and overall usability and experience. With consumer demand for product improvements accelerating in turn, developers have had to outsource or automate parts of the software-creation process that were once handled in-house, whether those parts be multiplatform expansion, server maintenance, or backend stack architecture. This has led to a fast-growing need for more and better tools to maintain the rapid pace of innovation.

Remote working: Not only is speed a requirement of modern developer tooling. Cloud-based features such as collaboration, code-pairing, and analytics dashboards speak to the popularity of distributed teams (a financial boon for lean startups) as well as work-from-home (or work-from-cafe, work-from-the-train, and so on) models. But developers still derive a great deal of benefit from constant, transparent communication with their coworkers, including business and design team members. This means that startups and divisions within larger companies alike need always-on tools for communication and collaboration. These teams also need remote, real-time access to their own technology stacks, from codebases to servers.

Mobile development: Finally, mobile coding is still a nascent niche in developer tooling. But as the future of computing lies not in desktop boxes but in the palm of the user’s hand, devs need to create for and from mobile devices, including tablets. Cloud-based tools that enable software developers to work on iPads or to manage code repositories from smartphones are becoming more than desirable; they are quickly becoming absolutely necessary. Pioneers in this field stand to define a new segment of the developer-tools marketplace, taking the unique features of mobile devices not as challenges but as opportunities.

With all those factors and needs in mind, we surveyed 200 developers on the most exciting, forward-looking developer tools based in the cloud. These services cover a wide range of needs, from code hosting to application monitoring and backends-as-a-service. All the services have compelling web-based components that impact the nature of the services they offer. Some, such as Adobe’s Creative Cloud and GitHub, manage to merge desktop software and cloud-based features. Others, such as Cloud9 IDE, are entirely web-based with no downloadable components.

Parts of the survey are quantitative; we considered the pure numbers to determine the popularity of a 10 company list of tools.

---

Editor’s note: A lot of these companies will be present at our upcoming CloudBeat event, Sept. 9-10 in San Francisco, where we’ll be tackling revolutionary cases of cloud adoption. Register today for CloudBeat! They’ll also be present DevBeat 2013, VentureBeat’s conference for developer topics in November.

---

But more than that, we strove to deliver a qualitative view of these services based on four main criteria:

1. Is the service innovative? Does it provide something unique that pairs with developers’ needs for modern tooling and support? Does it build on and reach beyond the services of the recent past? Does it leverage existing technologies in new, unexpected ways?
2. Does the service contribute to the future of development? Is it both forward-looking and extensible? Does it speak to the trends of the present as well as anticipate the needs of the near future? Does it move development forward, help to speed its pace, or remove roadblocks and bottlenecks for software creators? Does the company show signs of long-term stability and viability, or is it a flash in the pan?
3. In a multiplatform economy, does the service bridge the gaps between multiple platforms? Using code translation and the power of web-based technologies, does it make it easier and faster for developers to create products for many devices, browsers, and operating systems, regardless of individual skill sets?
4. Is the service validated by market momentum? Does the developer audience respond favorably to the product? Has the company proven its financial viability, either through revenues or the confidence of knowledgeable investors?

With these criteria and our own data-gathering in mind, here are our findings on the best cloud-based developer tools right now. These findings will change and will be accordingly updated to reflect changes in momentum and developer sentiment throughout the months to come.

Total quantitative survey results

Methodology

We chose 10 services from 10 companies for our initial survey: Google Cloud Playground, Adobe Creative Suite, GitHub, Heroku, Joyent, New Relic, Parse, Urban Airship, Kinvey, and Windows Azure. We asked respondents to identify the services they had used and indicate whether or not they would recommend that service. Invalid and questionable responses (such as responses that rated every service on the list) were discarded. We ended up with 200 valid responses for this round of surveying.

We realize that in this survey, there are some apples/oranges relationships between some of the companies named. The survey is intended to be a very broad overview of a large and growing industry. Rather than direct comparisons of very different services, we hope to provide a general picture of the developer tools landscape as it stands in 2013.

Numeric results

This graph shows the exact number of developers who said they had used and would or would not recommend a particular service. The positive responses are shown in green, with negative responses represented in red. Here, you can see which services generated the strongest sentiment among a developer audience, whether that sentiment was positive or negative.

Percent results

For each service, we also broke down the percentage of positive and negative responses, showing not just the overall sentiment but also which services are the best-liked in this market. For example, while GitHub generated the most sentiment by far, the huge majority of that sentiment was positive. And Adobe’s services, which elicited greater overall sentiment than Joyent’s, were decidedly less liked among developers.

Tools with the most recommendations

If our survey were a mere popularity contest, these contenders would win. However, each one shows qualitative attributes of momentum, market fit, and innovation, as well.

GitHub

GitHub is a service that hosts code repositories online. It especially works well for open-source projects, as it provides for forking of repos and encourages social collaboration on large or ambitious projects such as new programming languages.

The service includes individual, team, and enterprise pricing options. Each tier allows for different options and features for creating private repositories and collaborating within internal teams. Other interesting features include cover pages, which enable devs to showcase their work in a user-friendly introductory page; desktop and mobile tools for a wide variety of operating systems; and peripheral documentation on topics such as software licensing and startup finances.

The service boasts more than 4 million developer users and more than 8 million code repositories, making it one of the largest developer services on the web. Momentum in user acquisition has been strong, with one million developers signing up for new accounts between August 2012 and January.

Financially, GitHub was bootstrapped from its launch in 2008 until July 2012. The majority of the company’s revenue comes from enterprise customers that use the service to coordinate and collaborate on large, internal codebases. In July 2012, GitHub took a huge $100 million first round of funding led by Silicon Valley superfirm Andreessen-Horowitz. This round stands as the largest investment that Andreessen has made to date.

For GitHub’s roadmap and future plans, cofounder Tom Preston-Werner told VentureBeat the company will focus on monetizing the enterprise, creating more clients for different operating systems, and continuing to create useful features for developers as well as para-development personnel such as technical writers and web designers. “Specifically, there’s a strategy called ‘GitHub Everywhere.’ We want everyone in software to be using GitHub,” said Preston-Werner, “individuals by themselves, small teams, students, as well as big, massive enterprises.”

User stories:

“We have geographically disperse teams and do a lot of distributed development. We tried several version control systems, including SVN. We even tried to install Git on our server and manage it ourselves. None were as convenient to use as GitHub. Creating branches, marking repositories are all really easy to do with GitHub. This way, we can focus on what we do best: sling code.”

“It is quite a headache to host and manage your own code repository. It completely takes away time and resources from what’s really important, which is your product! GitHub makes it so easy to share and collaborate on development, especially when your team is distributed all over the globe. Not to mention all the available open source projects that are on GitHub. They can save teams hundreds if not thousands of hours of development.”

“Excellent product. Excellent documentation and community. Easy to use, and definitely a vital part of the open-source community.”

Heroku

Heroku is a platform-as-a-service (PaaS) company for users to deploy, run, and manage applications written in Ruby, Node.js, Java, PHP, Perl, Python, Clojure, and Scala. The Heroku platform began as a service for Ruby developers and continues to add more languages and technologies to its suite of tools and services.

The company also runs a standalone Postgres database-as-a-service (DaaS) product.

As of May 2012, Heroku hosts 1.5 million applications, with that figure having increased fifteenfold since January 2011. When the company announced new capabilities for Facebook developers in September 2011, it recorded 34,000 new users within 24 hours. And after adding an initial cohort of 12 add-ons, the company announced it had seen four million installs between 2009 and 2012.

The company was acquired by Salesforce.com, a publicly traded company, in 2010 for $250 million. While Salesforce does not break out financials on Heroku, the parent company’s quarterly revenues for Q1 2014 totaled $893 million.

In the near future, Heroku plans to continue expanding its products to include the longer tail of developers and technologies. “Heroku started about Ruby on Rails, that was absolutely our community,” said general manager Oren Teich in an interview. “One of the things we’ve done in the process of building out Heroku, we launched a new stack, Cedar, that enabled us to support anything. There’s a few open-source packs people have built. … So it’s not a technical question; it’s a community question. That Middle American dad programmer using .NET, how do we respect and engage with that community?”

User stories:

“We use Heroku less frequently [than GitHub], but we definitely think PaaS in general are helpful for small and/or startup teams that just don’t have the time and resources to be messing around with server administration and configuration. Again, this allows time to focus on the product and feature development. PaaS companies also allow small startups/teams to handle scaling for their initial growth without the hassle of having to learn how to scale infrastructure if they have no experience.”

“Ultra-easy to use. Free plan to test things around. Great service, and good support.”

Windows Azure

Windows Azure is both a infrastructure-as-a-service (IaaS) and PaaS created and sold by Microsoft. Developers can use it to create, deploy, and manage applications powered by the company’s worldwide network of datacenters. Azure supports both Microsoft languages and technologies as well as popular languages created by third parties. Supported technologies include Python, Java, and Node.js.

Azure was first launched in February 2010. The infrastructure portion, featuring Azure virtual machines, was released in mid-2012. Moreover, in 2013, Microsoft announced the availability of Azure Web Sites and Azure Mobile Services.

Currently, Azure processes nearly 5 billion authentications per week. The service, which offers pay-as-you-go and subscription options, Azure generates around $19 billion in annual revenue.

For the future, Azure chief Satya Nadella told us, “There are four specific things that we’re working on: One is the modern datacenter, because now you’re managing compute, storage, and network as a unit at datacenter and multi-datacenter scale. People refer to it as a software driven data center, and to us that’s a key part. Next layer is the data … we need to have a very first-class data platform. Then above that, you have to have a very modern application platform for building mobile, social, and data driven applications. Last, we want to make all these applications available on any device while giving IT the ability to secure the access and the data.”

Other executives have indicated Microsoft plans to do better at competitive pricing and “top-to-bottom” customer service.

User stories:

“We have worked as a six-person startup headquartered in Bengaluru, India, to use Azure to power our innovative solution aimed at sports lovers worldwide. We are fortunate to have been actively supported by Microsoft, hence this experience is one which we recommend to the developer community.”

New Relic

New Relic is an analytics service delivered in a SaaS model. Without counting plugins, the service supports applications built with Ruby, Rails, PHP, Java, .NET, and Python. The product analyzes and reports metrics around performance, latency, experience, server resources, and much more. All the data is rendered in a graphically simple dashboard UI.

In June 2013, the company announced it was opening a plugin platform so developers could create tools for and monitor any kind of metrics they wanted, including business performance metrics.

In addition to web app monitoring, New Relic also provides analytics for iOS and Android applications, capabilities that were announced in March.

In terms of momentum, this startup is used by 1.5 million application creators. The company told VentureBeat in August that it had grown 130 percent in the most recent quarter and was poised to hit a $100 million run rate.

New Relic raised an $80 million round of funding in February. Its founder, Lew Cirne, has stated that he is looking forward to taking the company public next. “We’d love New Relic to be for IT management market what Salesforce is to sales and CRM,” Cirne said.

User stories:

“New Relic has been invaluable for us to help us tune our backend. When we put our first version of the services out, we did not exactly know how they will be used. With New Relic, we were able to quickly determine the bottlenecks and fix them. We were also able to get some invaluable analytics and know, in near real time, the latencies experienced by our users. In fact, we have not been able to harness all the data that New Relic offers. We are actively in the process of pursuing this to help enhance our users’ experience.”

Joyent

Joyent is in the business of virtualization and cloud computing. Its JoyentCloud product is a direct competitor of Amazon EC2, offering IaaS and PaaS tools.

In addition to hosting, Joyent is also responsible for several notable software projects, including SmartOS, an open-source Unix competitor; and Node.js, a wildly popular project for server-side JavaScript.

The company’s current release is Joyent7, a “fabric-based cloud infrastructure platform” intended for large enterprises and cloud providers. This product provides tools such as Node.js debugging, unified directory services, billing and other financial features, a cloud ops dashboard, workflow APIs, and datacenter local storage.

In January 2012, Joyent took an $85 million round of funding led by Weather Investment II Group with participation from Telefónica, Greycroft Partners, Intel Capital, and several other entities.

A note on Node

Node.js has without question been Joyent’s biggest standalone success in the developer community. While the company has filled its client roster with big-name enterprises, it also fostered the growth of this open-source programming technology by employing its creator, Ryan Dahl, and several other core contributors. In maintaining the project over the years, Joyent has stabilized the project from a hacker’s plaything to an enterprise-ready power tool.

Today, an entire ecosystem of consultancies, security tools, platforms, and applications exist around Node.js, which itself has become one of the most popular open-source projects on the Internet.

Up-and-comers

When we considered the services at the shallow end of the popularity pool, we looked both at the total number of votes each had received (i.e., how many developers out of 200 had used those tools) as well as the “approval rating” (i.e., how many of those developers would recommend that tool to a peer). Based on those criteria, we identified three services that are still relatively nascent in defining their benefits to the developer community.

We do note that these are not necessarily bad services or struggling companies. Rather, especially as many of these tools and features are new, we would suggest that they are not yet as familiar to the developer audience, nor have they had as much time to be crafted to their users’ preferences and needs.

We’re keeping an eye on these services and think they’ll have interesting products to show in the near future.

Adobe Creative Cloud and Edge

Adobe relaunched its entire suite of development, video, design, and animation tools this year, calling the new offering “Creative Cloud.” While most of the products still have downloadable software, a large emphasis has been placed on collaboration, syncing of settings and assets, and other cloud-enabled features.

An important component of Adobe’s Creative Cloud is the Edge suite of services, which are intended to appeal to developers and designers who work primarily or entirely with digital media. Edge Animate is a product for browser-based multimedia creation using HTML5, JavaScript, CSS3, and jQuery. Edge Reflow is a tool for responsive design, especially for the current range of mobile devices and mobile-optimized websites. Edge Code is an HTML editor, and Edge Inspect is a product for testing out native mobile applications before deploying.

Adobe’s primary audience for decades has been traditional, nonweb designers. As many of those professionals are forced to transition their skill sets to include digital media creation and even programming, Adobe is ensuring its products continue to meet their needs as well as the needs of more advanced developers.

Creative Cloud, including the Edge suite, are available through a subscription model, with prices starting at $20 per month for single-product use. Subscriptions for teams needing powerful collaboration tools are available starting at $30 per month per user. As of the second quarter of 2013, Creative Cloud has more than 700,000 subscribers, with Adobe generating more than $1 billion for the quarter from Creative Cloud and other avenues.

While Creative Cloud received some of the most strongly negative sentiment in the survey, Adobe is far from giving up on courting developers. Executives at the company have told us Adobe plans to continue releasing “lightweight, task-specific” tools for web and mobile developers and will continue to bridge the gaps between creative and technical workflows. The company also plans to speed up its release cycle to more quickly respond to developers’ requests.

Kinvey

Kinvey is a backend-as-a-service (BaaS) for mobile developers to set up and operate a cloud-based backend for their applications. The company’s stated goal is to provide developers with plug-and-play “plumbing” for their mobile apps. The service takes care of scaling and maintenance automatically. Developers use any SDK they like and then connect their applications to Kinvey using auto-generated APIs and libraries.

To date, Kinvey has raised $7 million in two rounds of funding. It charges customers on a monthly basis, with free and paid tiers available and scaled pricing for larger instances.

“PaaS offerings were designed and built for a web-centric world, where customers build and host a backend and a web tier on the PaaS technology stack,” said Kinvey investor Fred Destin in a VentureBeat post. “Mobile apps, however, live across native and HTML5 platforms that require libraries that connect to the backend. PaaS doesn’t make this easy. … BaaS provides mobile developers with the entire backend stack, including native mobile and web libraries, which ensure the app keeps the data secure and works online and offline. As a result, developers have a consistent user experience across all devices.”

Parse

Parse is another mobile BaaS. Like Kinvey, it offers a cornucopia of SDKs and a full backend stack. The service includes features for data storage, social media integration, and push notifications.

The company claims it powers more than 100,000 applications. It offers free use for lightweight customers along with monthly price plans for larger and enterprise customers.

In April, Facebook acquired Parse for a reported $85 million. Parse continues to operate as a standalone business, launching features and serving customers independent of Facebook. In May, the company launched a hosting service.

User stories:

“Parse has allowed us to focus on the user-experience and leave the backend to them. It’s been a great experience so far. Parse is always pushing their platform forward, and we feel that it is a huge benefit to be working with a company that is innovating at such a fast rate.”

“I took a look at the documentation and within minutes, I clearly understood Parse’s functionality. Easily some of the best documentation we’ve seen.”

“In my opinion, developers may find better solutions. … Other BaaS solutions (like Kinvey) are more friendly for developers. When I work with Kinvey, if I have problem, I can ask and always get a quick answer. From Parse support, I received answers always, too; but in my opinion, they were minimalistic.”

Recommendations from the C-suite

We saw a few differences in the recommendations from CIOs, CTOs, COOs, CEOs, and founders. Specifically, this demographic showed a small spike in preferences for New Relic and Azure.

Operating systems

With Linux’s provenance as an open-source operating system, Windows’ foothold in the Microsoft community, and Apple’s history in the world of creativity and design, we decided to put these stereotypes to the test. For the most part, they played out just as you might expect.

Our Linux-loving respondents had little use for Adobe (which doesn’t make Linux-compatible versions of its software) and the virtual machines of Joyent. Mac users showed their designer stripes in a marked preference for Adobe’s Creative Cloud suite of tools. And the Windows users can always be counted upon to stick up for Azure.

International and U.S.

U.S. residents accounted for less than half of the total respondent pool.

Overall, U.S. respondents were slightly more likely to recommend Joyent, Urban Airship, and Parse:

Write-ins: Tools with significant popularity

There were several tools that didn’t make our initial list but which each received several write-ins. These services each meet some but not all of our qualitative criteria. However, strong developer sentiment in favor of these offerings will likely put some of them on our next survey.

Amazon

Amazon has several developer offerings collectively known as Amazon Web Services. Together, AWS provide hosting, networking, database resources, deployment, management, and application services to an army of developers. AWS powers some of the most popular consumer applications on the web.

Bitbucket

Bitbucket is a code repository host and a GitHub competitor. While it’s tough going up against one of the most popular developer services around, Bitbucket fans were ardent in their support for the service. While GitHub favors Git as a technology, Bitbucket supports both Git and Mercurial.

Cloud9 IDE

Cloud9 IDE is what it sounds like — a web-based integrated development environment. Many developers we consulted for this survey indicated a lack of confidence in the performance and robustness of cloud-based IDEs. However, Cloud9 is near the head of a small but dogged pack of startups in this space.

SendGrid

SendGrid is a service that helps developers send e-mail, whether it’s transactional updates, marketing missives, or enterprise communications. The best devs we know swear by this service, which counts among its clients many of Silicon Valley’s most notable startups.

Conclusion

Cloud-based developer tooling is still a very new and highly volatile marketplace. As development itself continues to evolve at an astonishing rate, we will continue to follow both the heavyweights in this industry as well as the up-and-comers to see who’s on the rise, who’s on the downswing, and who is capturing the future of development.

We hope you’ll continue to participate in our updated survey as we track the best, most innovative cloud-based developer tools.