NVIDIA and Ineffable Intelligence Forge Path to Next-Generation Reinforcement Learning Infrastructure

A New Era of AI Learning

Reinforcement learning (RL) agents—systems that improve through trial and error—have the unique ability to transform computational power into new knowledge. This principle lies at the heart of a fresh engineering collaboration between NVIDIA and Ineffable Intelligence, a London-based AI lab founded by AlphaGo architect David Silver. The partnership comes shortly after Ineffable emerged from stealth mode, signaling a major push to advance RL infrastructure.

“The next frontier of AI is superlearners—systems that learn continuously from experience,” said Jensen Huang, founder and CEO of NVIDIA. “We are thrilled to partner with Ineffable Intelligence to codesign the infrastructure for large-scale reinforcement learning as they push the frontier of AI and pioneer a new generation of intelligent systems.”

Silver, a pioneer in RL, sees this as a necessary shift from current AI paradigms. “Researchers have largely solved the easier problem of AI: how to build systems that know all the things humans already know,” Silver explained. “But now we need to solve the harder problem of AI: how to build systems that discover new knowledge for themselves.” This requires systems that learn from experience rather than from static datasets.

The Challenges of Reinforcement Learning at Scale

Training RL systems at scale is fundamentally different from pretraining large language models. In pretraining, a fixed dataset of human-generated data flows through the system in a relatively predictable manner. RL, by contrast, generates its own data on the fly. The system must continuously cycle through act, observe, score, and update steps in tight loops, placing extreme demands on interconnect, memory bandwidth, and serving infrastructure.

Moreover, RL systems often train on rich forms of experience—such as simulations of physical environments, game play, or robotic interactions—that are distinct from human language and other human-curated data. This may require novel model architectures and training algorithms that break away from current transformer-heavy designs.

Engineering the Pipeline for Continuous Learning

The core of the NVIDIA–Ineffable collaboration lies in designing a pipeline that can feed RL systems at scale. Engineers from both companies are jointly exploring how to optimize this training loop. The work begins on the NVIDIA Grace Blackwell platform and will be among the first to test the upcoming NVIDIA Vera Rubin platform. The aim is to anticipate the hardware and software needs of a future where AI moves beyond human data toward models that learn through simulation and direct experience.

This pipeline must handle high-throughput data generation, rapid inference, and immediate integration of new observations into the model. It requires careful orchestration of compute, memory, and networking to avoid bottlenecks that could slow down learning. The team is investigating techniques such as distributed experience replay and asynchronous actor-critic methods to maximize efficiency.

Hardware and Software Foundation for Future Breakthroughs

NVIDIA's next-generation architectures are being designed with RL workloads in mind. The Grace Blackwell platform, with its unified memory architecture and high-bandwidth interconnects, provides a suitable starting point. The upcoming Vera Rubin platform promises even tighter integration of CPU and GPU resources, which will be critical for the low-latency, high-throughput loops required by RL.

On the software side, the collaboration will leverage NVIDIA's AI frameworks, including NeMo and RAPIDS, to build a robust RL training stack. Ineffable brings deep expertise in RL algorithms and agent architectures, ensuring that the hardware is tailored to the specific demands of continual learning.

As Silver noted, “The system has to act, observe, score and update continuously. That puts pressure on interconnect, memory bandwidth and serving in ways that pretraining doesn’t.” Getting this right will unlock unprecedented scale for RL in complex and rich environments, allowing agents to discover breakthroughs across all fields of knowledge—from drug discovery to autonomous systems to fundamental science.

Ultimately, this partnership aims to answer a critical question: how can we build AI that doesn't just mimic human knowledge but actively creates new knowledge? The infrastructure being developed today could lay the groundwork for a generation of superlearners that continually expand the boundaries of what is known.