Maximizing GPU Utilization: Heterogeneous Pipelines with Ray and Kubernetes
Data Engineering Podcast

Maximizing GPU Utilization: Heterogeneous Pipelines with Ray and Kubernetes

May 6, 202658 min

Summary In this episode Robert Nishihara, co-founder of Anyscale and co-creator of Ray, talks about maximizing hardware utilization for AI and data-intensive workloads. He explores Ray’s evolution alongside Kubernetes and PyTorch, and why consolidation at these layers has enabled a new generation of complex, heterogeneous workloads. Robert explains how data preparation has shifted to GPU- and inference-heavy, multimodal pipelines; where Ray fits compared to Spark and workflow orchestrators; and why Ray excels at composing heterogeneous pools of compute, handling failures, and scaling complex systems like multi-node LLM inference and reinforcement learning. He digs into practical strategies for boosting GP...

Play episode

No transcript for this episode yet

You can still listen to it in the player. On episodes with a transcript, you can tap any word to see its meaning.