Baidu apologizes for exceeding submission limit for ImageNet challenge

Chinese search giant Baidu today was revealed as the company that made more submissions than was permitted in the most recent ImageNet Large Scale Visual Recognition Challenge (ILSVRC), a major competition in the domain of image recognition.

Stanford University’s Vision Lab, which runs the ILSVRC, announced today that from November 28 to May 13, people associated with Baidu used at least 30 accounts to make at least 200 submissions. The rules for the competition permit no more than two submissions a week.

“This includes short periods of very high usage, for example with more than 40 submissions over 5 days from March 15th, 2015 to March 19th, 2015,” the competition’s organizers wrote in the announcement. The rules of the competition permit no more than two submissions per week, as submitting very frequently could provide an unfair advantage.

Baidu has issued an apology for the infraction in a note in the “Deep Image” research paper documenting its latest test results:

Note: Recently the ILSVRC organizers contacted the Heterogeneous Computing team to inform us that we exceeded the allowable number of weekly submissions to the ImageNet servers (over 200 submissions during the lifespan of our project). We apologize for this mistake, and have put processes into place to ensure it doesn’t happen again. We are working with the ILSVRC organizers to review the results and will continue to provide updates to this paper as our understanding of the results improves. We are staunch supporters of fairness and transparency in the ImageNet Challenge and are committed to the integrity of the scientific process.

The organizers of the challenge have told Baidu that the results it received from ImageNet’s test servers cannot be compared with other entrants’ test results, and that Baidu should “refrain from submitting to the evaluation server or the challenge for the next 12 months.”

This isn’t the best thing to happen to Baidu as the company looks to impress and lure researchers well versed in image recognition and, more generally, deep learning, a type of artificial intelligence that’s en vogue at the moment.

Baidu managed to recruit Google AI expert Andrew Ng to advance the company’s efforts around deep learning, which entails training artificial neural networks on large quantities of data and then using those trained neural nets to make inferences about new data.

Facebook, Google, Microsoft, and other companies have been improving the performance of their own neural networking systems, and talent counts for a lot in that task.

Companies naturally want their applications to perform better than any others on the Internet, but it’s still critical to maintain a clean reputation. That’s why it’s hardly surprising to see Baidu moving fast to say sorry about its transgressions.