Mozilla launches Test Pilot, a Firefox add-on for trying experimental new features

Mozilla today launched Test Pilot, a program for trying out experimental Firefox features. To try the new functionality Mozilla is offering for its browser, you have to download a Firefox add-on from testpilot.firefox.com and enable an experiment. The main caveat is that experiments are currently only available in English (though Mozilla promises to add more languages “later this year”).

Test Pilot is supposed to help Mozilla figure out which features should ship and how they should work, by letting users provide feedback and suggestions to the teams behind each one. You can turn each experiment on and off at any time (there will be bugs, so this will naturally come in handy), and the add-on explains what information you’re sharing with Mozilla to help the team understand how the feature is used.

Nick Nguyen, Firefox’s vice president of product, told VentureBeat that Mozilla will push Test Pilot to users by placing a promotion on the Firefox start page, emailing newsletter users, and of course via the press. The company hopes to have “tens of thousands of users” install the add-on, which initially ships with three experiments.

Experiments

The first is called Activity Stream, and it was the original reason why Test Pilot was created: This experiment is a “radical change” that isn’t easy to test with a small number of users. In short, Activity Stream makes it easier to navigate through your browsing history and rediscover the sites you love the most on the Web.

Each time you open a new tab, you’ll see your top sites along with highlights from your bookmarks and history. You can browse the visual timeline to explore and find a site to check out again.

The second is called Tab Center. This experiment displays tabs vertically along the side of the screen instead of horizontally along the top of the browser.

There are plenty of add-ons that let you do this already. Still, the fact Mozilla is considering adding the feature directly into Firefox is surprising given how the placement of tabs has changed little since Google placed them at the very top with Chrome.

The third is called Universal Search. This experiment combines the Awesome Bar’s history with the Firefox Search drop-down menu. Firefox has had separate URL and search bars for years, even after other browsers adopted a single, merged URL and search bar.

The idea is to give you recommendations right in your address bar. If you have been to a site before, you will see it clearly highlighted as a search suggestion, and recommended results will include more information about the site suggestion.

Why?

These experiments look great, but Test Pilot is really the more interesting debut today. We talked to Mozilla to find out why the company deemed it necessary.

Mozilla doesn’t break out the exact numbers for Firefox, though the company does say “half a billion people around the world” use the browser. Firefox development starts with the Nightly builds, which consist of the latest Firefox code packaged up every night for bleeding-edge testers. It is then followed by the Firefox Developer Edition (or Aurora on mobile), which includes everything that is labeled “experimental.” After that, the code eventually finds its way to Beta and then finally Release for the broader public.

In short, Mozilla already has a system for testing Firefox features before they’re ready, and many can even be configured in the about:config section. Firefox testers and power users do this all the time, and each of the channels have a dedicated following. So what’s the point of the Test Pilot program?

Nguyen told VentureBeat that Test Pilot offers two advantages: onboarding and telemetry. Both help the company better determine what should change, and whether the feature should ship at all.

Unlike the preferences in about:config, experiments have an onboarding experience that explains to the user what exactly he or she can expect. In the Firefox channels it’s difficult to tie new features to crashes or retention, while feature-specific telemetry lets Mozilla understand how well a given experiment works. The goal is to “gather more data and be more transparent about it,” Nguyen said, adding that when the experiment is over, Mozilla no longer gathers that data. This simply isn’t possible with the existing channels.

Test Pilot is designed to work on the current stable release of Firefox (today that’s Firefox 46). It can, however, work outside the stable channel or even on older versions. “Test Pilot works on all Firefox channels now including in Developer Edition and Beta,” a Mozilla spokesperson told VentureBeat. “In the future we may constrain certain experiments or all of Test Pilot to certain versions or Firefox channels.”

Aside from the three experiments shared today, there are many more in the pipeline. Nguyen named two: a screenshotting tool and a collaboration with the Internet Archive to make 404 pages more useful.

Ultimately, Nguyen said Test Pilot is unlikely to offer more than six experiments at a time. Mozilla doesn’t want experiments to just “sit there for months and months without anything changing.” The team will hold a weekly mission control call that looks at feedback, implements changes, and decides whether a feature should be promoted or killed.

“Test Pilot” is actually a recycled name — Mozilla first announced a Test Pilot program, which consisted of user studies regarding the company’s products but didn’t actually let them test unreleased features, in March 2008 (and launched the add-on in August 2009) though it was retired years later. Before deciding to reuse Test Pilot again, Mozilla also considered the names “Idea Lab” and “Idea Town.”

It has been a long time since Mozilla offered hardcore Firefox users something to really get excited about. So far, based on what I’ve seen, this nails it. Test Pilot might just be what Firefox needs to no longer feel like a browser constantly playing catch-up.