MemSQL releases a tool to easily ship big data into its database

Like other companies pushing databases, San Francisco startup MemSQL wants to solve low-level problems, such as easily importing data from critical sources. Today MemSQL is acting on that impulse by releasing a tool to send data from the S3 storage service on the Amazon Web Services cloud and from the Hadoop open-source file system into its proprietary in-memory SQL database — or the open-source MySQL database.

Engineers can try out the new tool, named MemSQL Loader, today, now that it’s been released under an open-source MIT license.

The existing “LOAD DATA” command in MemSQL and MySQL can bring data in, although it has its shortcomings, as Wayne Song, a software engineer at the startup, wrote in a blog post today. Song and his colleagues ran into those snags and started coding.

“You can specify all of the files you want to load with one command, and MemSQL Loader will take care of deduplicating files, parallelizing the workload, retrying files if they fail to load, and more,” Song wrote.

Of course, the open-source release will help MemSQL get more people working on the tool, which could further increase adoption of MemSQL.

“Right now the loader supports MemSQL and MySQL (via the LOAD DATA command), but does not support other database systems. We would love for members of the community to add support for more systems,” MemSQL notes on the GitHub repository page for the new tool.

Microsoft, Oracle, and other SQL database vendors have previously come out with connectors for Hadoop in particular. For MemSQL, which announced a $35 million funding round in January, this could be the start of a new run at widespread adoption.

Meanwhile, NoSQL database companies like Couchbase and DataStax have been busy releasing features and taking on substantial rounds of funding in recent months, while MongoDB could be in a position to go public soon.