|
Today, we are announcing the general availability of Amazon Elastic Compute Cloud (Amazon EC2) G7e instances, which provide cost-effective performance for generative AI workloads and peak performance for graphics workloads.
G7e instances are accelerated by NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs and are suitable for a wide range of GPU-enabled workloads, including spatial computing and scientific computing. G7e instances provide up to 2.3x higher inference performance compared to G6e instances.
Improvements over predecessors:
- NVIDIA RTX PRO 6000 Blackwell GPU — The NVIDIA RTX PRO 6000 Blackwell Server Edition GPU offers twice the GPU memory and 1.85 times the GPU memory bandwidth compared to G6e instances. By using the higher GPU memory offered by the G7e instances, you can run medium-sized models with parameters up to 70B with FP8 accuracy on a single GPU.
- NVIDIA GPUDirect P2P — For models that are too large to fit in the memory of a single GPU, you can split the model or calculations across multiple GPUs. G7e instances reduce the latency of your multi-GPU workloads with NVIDIA GPUDirect P2P support, which enables direct communication between GPUs over a PCIe link. These instances offer the lowest peer-to-peer latency for GPUs on the same PCIe switch. In addition, the G7e instances offer up to four times the inter-GPU bandwidth compared to the L40 GPUs included in the G6e instances, increasing the performance of multi-GPU workloads. These improvements mean that on larger models you can run inference on multiple GPUs offering up to 768GB of GPU memory on a single node.
- networking — G7e instances offer four times the network bandwidth compared to G6e instances, meaning you can use the instance for small, multi-node workloads. In addition, multi-GPU G7e instances support NVIDIA GPUDirect Remote Direct Memory Access (RDMA) with Elastic Fabric Adapter (EFA), which reduces GPU-GPU remote communication latency for multi-node workloads. These instance sizes also support NVIDIA GPUDirectStorage with Amazon FSx for Luster, increasing instance throughput by up to 1.2 Tb/s compared to G6e instances, meaning you can load your models quickly.
EC2 G7e specifications
G7e instances include up to 8 NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs with up to 768 GB of total GPU memory (96 GB of memory per GPU) and Intel Emerald Rapids processors. They also support up to 192 vCPUs, up to 1,600 Gbps of network bandwidth, up to 2,048 GiB of system memory, and up to 15.2 TB of local NVMe SSD storage.
Here are the specs:
| The name of the instance |
GPU | GPU Memory (GB) | vCPU | Memory (GiB) | Storage (TB) | EBS Bandwidth (Gbps) | Network Bandwidth (Gbps) |
| g7e.2xlarge | 1 | 96 | 8 | 64 | 1.9 x 1 | Up to 5 | 50 |
| g7e.4xlarge | 1 | 96 | 16 | 128 | 1.9 x 1 | 8 | 50 |
| g7e.8xlarge | 1 | 96 | 32 | 256 | 1.9 x 1 | 16 | 100 |
| g7e.12xlarge | 2 | 192 | 48 | 512 | 3.8 x 1 | 25 | 400 |
| g7e.24xlarge | 4 | 384 | 96 | 1024 | 3.8 x 2 | 50 | 800 |
| g7e.48xlarge | 8 | 768 | 192 | 2048 | 3.8×4 | 100 | 1600 |
To get started with G7e instances, you can use the AWS Deep Learning AMI (DLAMI) for your machine learning (ML) workloads. You can use the AWS Management Console, the AWS Command Line Interface (AWS CLI), or the AWS SDKs to launch instances. For managed environments, you can use G7e instances with Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS), and AWS Parallel Computing Service (AWS PCS). Support for Amazon SageMaker AI is also coming soon.
Now available
Amazon EC2 G7e instances are available today in the US East (N. Virginia) and US East (Ohio) AWS regions. For regional availability and future schedule, search for instance type v CloudFormation on the Resources tab of the AWS options by region.
Instances can be purchased as On-Demand Instances, Savings Plan, and Instant Instances. G7e instances are also available in dedicated instances and dedicated hosts. To learn more, visit the Amazon EC2 Pricing page.
Try the G7e instances in the Amazon EC2 console. To learn more, visit the Amazon EC2 G7e instances page and submit feedback on AWS re:Post for EC2 or through your usual AWS support contacts.
— Channy