Amazon Web Services has announced the availability of its new Amazon EC2 P3 instances, said to be dramatically faster and more powerful than previous instances. AWS considers this the next-generation of EC2 compute-optimized GPU instances and is said to make training machine learning models faster than ever.
Driven by customer demand, AWS is proud of how far they’ve come since their original m1.small instance launched in 2006. Now, instances are able to host much more compute power, memory, storage, and more.
These instances are made for very computationally advanced workloads, including machine learning (ML), high-performance computing (HPC), data compression, and cryptography. According to Amazon, “They are also ideal for specific industry applications for scientific computing and simulations, financial analytics, and image and video processing.”
They are specifically “designed to handle compute-intensive machine learning, deep learning, computational fluid dynamics, computational finance, seismic analysis, molecular modeling, and genomics workloads.”
Details of the EC2 P3 instances
Essentially, these new instances are great for anything that could require very high processing power. AWS explains that they leverage 64 vCPUs using custom Intel Xeon E5 processors, 488 GB of RAM, and up to 25 Gbps of aggregate network bandwidth leveraging Elastic Network Adapter technology.
The increase in power isn’t marginal. In fact, P3 instances actually offer 14x performance improvement over P2 instances for ML applications. This means that if you’re a developer working with machine learning models, you can train them in mere hours instead of the days it would have previously taken you.
Amazon claims that these are “the most powerful GPU instances available in the cloud.” As of right now, they’re only available in the U.S. East (north Virginia), U.S. West (Oregon), EU West (Ireland), and Asia Pacific (Tokyo) regions, although they do have plans to move into more markets in the future.
These instances are available in three different sizes (all VPC-only and EBS-only):
| Model | NVIDIA Tesla V100 GPUs | GPU Memory | NVIDIA NVLink | vCPUs | Main Memory | Network Bandwidth | EBS Bandwidth |
| p3.2xlarge | 1 | 16 GiB | n/a | 8 | 61 GiB | Up to 10 Gbps | 1.5 Gbps |
| p3.8xlarge | 4 | 64 GiB | 200 GBps | 32 | 244 Gib | 10 Gbps | 7 Gbps |
| p3.16xlarge | 8 | 128 GiB | 300 GBps | 64 | 488 GiB | 25 Gbps | 14 Gbps |
According to AWS, “Each of the Nvidia GPUs is packed with 5,120 CUDA cores and another 640 Tensor cores and can deliver up to 125 TFLOPS of mixed-precision floating point, 15.7 TFLOPS of single-precision floating point, and 7.8 TFLOPS of double-precision floating point.”
The two larger-sized GPUs are able to exchange intermediate results, as well as other types of data, quickly without the need to send it through the CPU or the PCI-Express fabric. This is because these GPUs are “connected together via Nvidia NVLink 2.0 running at a total data rate of up to 300 Gbps.”
Understanding the performance
The eight Nvidia Tesla V100 GPUs on just one p3.16xlarge are able to perform 125 trillion single-precision floating point multiplications per second, which makes this performance hard to compare to other options available right now.
Jeff Barr, chief evangelist for AWS, wrote a blog post about AWS’s new EC2 P3 instances and explained how the p3.16xlarge is a full 150 billion times faster than one of the first microprocessing chips, the Intel 8080A from 1977. We’ve certainly come a long way since then. As Barr puts it, “I can do 100x more calculations today in one second than my Altair could have done in the last 40 years.”
While we can look at the massive leap between now and 1977, Barr explains that it’s hard to compare the P3 with today’s scale-out supercomputers because “you can think of the P3 as a step-and-repeat component of a supercomputer that you can launch on an as-needed basis.”
AWS Deep Learning Amazon Machine Images (AMI)
AWS has also decided to release a new set of deep learning Amazon Machine Images (AMI). These include tools to help developers build AI systems on AWS like Nvidia’s CUDA toolkit, and includes popular frameworks such as Google’s TensorFlow or Caffe2; they promise to add more frameworks as they become available for Volta.
These AMIs come preinstalled “with deep learning frameworks optimized for the Nvidia Volta V100 GPUs in the new Amazon EC2 P3 family.” The AMIs are available on both Ubuntu and Amazon Linux, coming preinstalled and configured with Cuda 9.
AWS offers its customers a step-by-step guide to begin using the AWS Deep Learning AMI.
These AMIs are built to help give machine learning practitioners and researchers all of the infrastructure and tools necessary to help advance deep learning in the cloud. According to Amazon:
You can quickly launch EC2 instances pre-installed with popular deep learning frameworks such as Apache MXNet and Gluon, TensorFlow, Microsoft Cognitive Toolkit, Caffe, Caffe2, Theano, Torch, Pytorch, and Keras to train sophisticated, custom AI models, experiment with new algorithms, or to learn new skills and techniques.
AWS proves to be serious about wanting to advance deep learning on the cloud by not charging for any of the Deep Learning AMIs. Instead, they only require their customers pay for the AWS resources that are necessary to store and run the applications.
Choosing an AWS Deep Learning AMI isn’t easy, even for those experienced with machine learning. There are three different types of AMIs offered, each catering to different needs of developers. AWS has compiled a selection guide and offers more deep learning resources to help you decide.
Simply put:
- Their Conda AMI is for developers who would prefer preinstalled pip packages of deep learning frameworks in separate virtual environments.
- Their Base AMI is geared more towards those who would prefer a “clean slate to set up private deep learning engine repositories or custom builds of deep learning engines.”
- And finally, their AMI with source code is for those developers who prefer preinstalled deep learning frameworks and their source code in a shared Python environment.
It’s clear that AWS wants to stay at the head of the cloud wars, and they’re offering the features and power to do so. Even though their new EC2 P3 instances don’t come cheap, they’re certainly worth the power if you can swing it.
Photo credit: Shutterstock
