NVIDIA H100
Performance and security at scale
Why H100 on Together GPU Clusters?
The world’s most powerful AI infrastructure. Delivered faster. Tuned smarter.
Efficient training with Hopper GPUs
Each H100 cluster leverages fourth-generation Tensor Cores and the Transformer Engine with FP8 precision, enabling fast training for GPT-scale models.
Advanced multi-GPU connectivity
We deploy NVIDIA's NVLink Switch System for 900GB/s bidirectional bandwidth per GPU, providing unparalleled scalability for multi-node AI and HPC workloads.
Secure, multi-instance GPU configurations
Second-generation MIG technology securely partitions GPUs into isolated instances, maximizing resource utilization and quality of service across diverse teams.
Run by researchers who train models
Our research team actively runs and tunes training workloads on NVIDIA H100 systems. You're not just getting hardware — you're working with experts at the edge of what's possible.
What our customers are saying
"Delivering competitive pricing, strong reliability and a properly set up cluster is the bulk of the value differentiation for most AI clouds. The only differentiated value we have seen outside this set is from a Neocloud called Together AI, where the inventor of FlashAttention, Tri Dao, works. We don't believe the value created by Together can be replicated elsewhere."
"Together GPU Clusters provided a combination of amazing training performance, expert support, and the ability to scale to meet our rapid growth to help us serve our growing community of AI creators."
Demi Guo
CEO, Pika
“Together AI provides the performance and reliability we need for real-time, high-quality image and video generation at scale. We value that Together AI is much more than an infrastructure provider — they're a true innovation partner, enabling us to push creative boundaries without compromise.”
Victor Perez
Co-Founder, Krea
Outstanding specs of H100
4x
vs A100
30x
on Megatron 530B
7x
higher
"We’ve been thoroughly impressed with the Together Enterprise Platform. It has delivered a 2x reduction in latency (time to first token) and cut our costs by approximately a third. These improvements allow us to launch AI-powered features and deliver lightning-fast experiences faster than ever before."
Caiming Xiong
VP Salesforce AI Research
Technical specification
- Hopper GPUs 8 GPUs
- FP8 Tensor Core 3,958 TFLOPS
- FP16/BF16 Tensor Core 1,979 TFLOPS
- TF32 Tensor Core 989 TFLOPS
- GPU Memory 80 GB HBM3
- GPU Memory Bandwidth 3.35 TB/s
- Total NVLink Bandwidth 900 GB/s
- Multi-Instance GPU (MIG) 7
- Decoders 7 NVDEC, 7 JPEG
- Max Thermal Design Power (TDP) Configurable up to 700 W
- Interconnect NVLink: 900 GB/s, PCIe Gen5: 128 GB/s
- Server Options NVIDIA HGX H100 partner and Certified Systems with 4 or 8 GPUs
Infrastructure you can trust at scale.
Production-grade security.
We take security and compliance seriously, with strict data privacy controls to keep your information protected. Your data and models remain fully under your ownership, safeguarded by robust security measures.
NVIDIA preferred partner
AICPA SOC 2 Type II
Regions and availability zones
Choose from global regions to meet data residency and compliance requirements—HIPAA for healthcare, GDPR for Europe, or banking regulations.
USA2GW+ in the portfolio with 600MW of near-term capacity in US.
Europe150 MW+ available in Europe: UK, Spain, France, Portugal, and Iceland also.
Asia & Middle EastOptions available based on the scale of the projects in Asia and the Middle East.