Amazon is constantly on the heels of innovating against other well known giants. We’re familiar with the Amazon Echo, one of its most innovative products, indicating how AI and deep learning are a core of Amazon’s current and future products. To build an organization upon a foundation of deep learning requires a great deal of careful consideration for the right partners for the job. After evaluating several different options, one clear winner emerged: the open-source MXNet platform.
What is MXNet?
MXNet is a scalable flexible programming framework that allows users to define, train, and deploy deep neural networks on a myriad of devices, from mobile to the cloud. With MXNet, which supports a number of programming languages and is also portable and lightweight, users can mix symbolic and imperative programming flavors for productive workflows. Built upon a dynamic dependency schedule that automatically parallelizes symbolic and imperative operations on the fly, MXNet also utilizes a graph optimization layer so to ensure for efficient execution and memory handling. It can also scale to multiple GPUs and devices. In Amazon’s official announcement, the true power of MXNet was illustrated: popular image recognition Resnet uses MXNet to receive 2x the throughput compared to all other engines, which means training happens in half the time.
MXNet in its open source glory can be deployed on Amazon Linux, Ubuntu/Debian, OS X, and Windows, as well as Docker or on the cloud (and of course, AWS). Further, it’s supported on embedded devices like Raspberry Pi.
Currently, MXNet can support Python, R, Julia, and Scala programming languages.
The true value in MXNet can be seen in AWS re:Invent 2016’s keynote presentation:
Currently, MXNet is supported by Intel, Dato, Baidu, Microsoft, Wolfram, and research institutions such as Carnegie Mellon, MIT, the University of Washington, and the Hong Kong University of Science and Technology.
Enter the Apache Incubator
MXNet was a top pick of Amazon’s, but it’s a top pick of the Apache Incubator too. As a member of the Apache Software Foundation’s incubator, MXNet is making a commitment to join the Foundation’s efforts. In return, MXNet will be able to take advantage of the Foundation’s processes, guidance, outreach, and events for the community at large.
The incubator structure is explained in this video which describes the life of an “incubatee” from acceptance to graduation:
[tg_youtube video_id=”KopPbWS87fw”]
Former graduates of the incubator include Bigtop (developer of packaging/tests for Hadoop), Empire-db (data persistence), Lenya (content management and publishing based on Cocoon), Nutch (web search), Wink (development or REST web services), and Zeppelin (data analytics), among countless others. MXNet joins 62 other projects in the incubator.
Not only does this ensure that MXNet is going to remain in the open source community for the near future, it will ensure that MXNet remains enterprise-grade and freely available for the benefit of everyone.
