Author: Jin Xin, General Manager of IBM China Storage Business Sales
BeijingMay 19, 2026 /PRNewswire/ — Recently, IBM Research, in collaboration with NVIDIA and Samsung, demonstrated a content-aware storage system[1]. The system successfully supports the storage and retrieval of hundreds of billions of vectors on a single server, with an average query latency of 694 milliseconds and a recall accuracy of 90%. The system hardware combination includes the IBM Storage Scale System 6000 all-flash device, six NVIDIA H200 GPUs, and 48 Samsung 30.72TB PCIe Gen5 NVMe SSDs. The IBM Storage Scale System 6000 all-flash device decouples compute and storage, and leverages NVIDIA H200 GPUs to accelerate index rebuilding, reducing the index construction process from hours on CPUs to minutes on GPUs.
Breaking the “1% Data Dilemma,” Letting AI Go to Data
Let’s take a look at how IBM achieved with a single machine what typically requires large clusters.
Today, large model versions are updated on average every few days, and RAG (Retrieval-Augmented Generation) has become the core for unlocking the value of unstructured data. Enterprise CIOs generally face a core challenge: How to leverage general artificial intelligence (AI) and AI agents to improve daily operational efficiency? How to deliver accurate, high-value business decisions based on existing IT resources?
The core prerequisite for high-quality AI responses is the model’s ability to efficiently access original, trustworthy data, and Retrieval-Augmented Generation (RAG) is the key technology for optimizing reasoning effectiveness and improving response accuracy and timeliness. However, when vector data volumes surge to tens of billions, CIOs face the capacity and cost dilemmas of traditional in-memory vector index solutions. Soaring DRAM prices, unstable lead times, and the “memory wall” and “I/O wall” bottlenecks caused by frequent data movement between CPUs and storage are severely constraining the large-scale deployment of AI applications. Enterprises commonly encounter four major pain points during implementation:
- Unstructured data types are complex, with only 1% of data being effectively utilized by AI to create value;
- Data distortion and model hallucinations introduce compliance and decision-making risks for enterprises;
- The RAG process leads to redundant copies and repeated data transmission, keeping costs high;
- When facing petabytes of massive data, traditional architecture performance bottlenecks become apparent, making it difficult to achieve large-scale deployment.
Breaking the “1% Data Dilemma,” Letting AI Go to Data
Enterprises today are surrounded by massive amounts of unstructured data—PDFs, emails, audio/video, presentations, financial reports, etc.—which continue to grow, but less than 1% of this data is accessible to large models and generates value.
RAG technology, through data vectorization, optimizing batch refresh cycles, and leveraging GPU clusters for distributed processing, can break down data access barriers, allowing AI to cover a wider range of data sources. The core breakthrough of IBM Storage Scale lies in abandoning the traditional “migrate data to AI” model and realizing a new paradigm of “AI going to data.” Simply put, CAS technology performs document extraction and vectorization directly at the storage layer (even integrating NVIDIA microservices), enabling AI to go to data. This allows AI to quickly locate compliant, clean, usable data, reducing the risk of model hallucinations at the source. This capability is realized through IBM’s CAS (Content-Aware Storage) technology.
New AI Storage Paradigm: CAS Offloads Vector Processing to the Storage Layer
The disruptive innovation of CAS is transforming the storage system from a passive “data warehouse” into an active “AI participant”—storage no longer just saves data but quantitatively understands data items, offloading the document vectorization process, traditionally handled by vector databases, from the application layer directly down to the storage layer.
IBM CAS
In layman’s terms, traditional RAG requires first extracting data from storage, vectorizing it externally, and then importing it into a vector database. In contrast, CAS can complete the entire process within the storage system itself, without data migration or copying.
This technology stems from IBM Research’s long-term technical accumulation in natural language processing, vector embedding models, and hardware acceleration. The document data extraction process deeply integrates the NVIDIA NeMo Retriever microservice built on NVIDIA NIM (part of NVIDIA AI Enterprise), ensuring that AI assistants and AI agents respond based on the latest, most relevant context, simplifying RAG operations and enhancing the business value of AI applications.
IBM Storage Scale (formerly GPFS) builds a global unified data platform for enterprises, creating a single namespace across multi-site, multi-cloud, data center, and edge environments. It is compatible with third-party storage, breaking down data silos and enabling unified access to data across domains. CAS, as a new AI enhancement capability of Storage Scale, helps enterprises unlock greater value from existing data assets, significantly improving RAG accuracy, reducing model hallucinations, and allowing AI models to synchronize with the latest data without retraining, suitable for enterprise-grade scenarios like research, customer service, and knowledge-based applications.
Enterprise-Grade RAG at Scale: Breaking Performance Bottlenecks, Enhancing Security and Compliance
Mainstream vector databases supporting tens of billions of vectors typically require dozens or even hundreds of servers. As the node scale expands, issues like distributed index synchronization, failure recovery, and expansion migration become frequent, leading to immense operational and cost pressures.
IBM Storage Scale System
The IBM Storage Scale storage solution achieves 100 billion vectors on a single server. Based on typical enterprise document scenarios, it can comprehensively cover petabytes to tens of petabytes of unstructured data, delivering four core values for enterprise CIOs:
- Exponential reduction in infrastructure costs: No need to deploy dozens or hundreds of vector database servers;
- Significantly reduced operational complexity: A single storage cluster can support the entire RAG workflow;
- Enterprise-grade real-time assurance: End-to-end latency as low as 694 milliseconds, meeting core business real-time requirements;
- Enhanced data security capabilities: Inherits the permission control system of the original data source; derivative data, such as chatbot responses, uniformly adheres to security policies.
Underlying core advantage: Data is processed in place, without migration. Retrieval and computation are performed directly at the data storage location, naturally aligning with data compliance and security control requirements.
The technical confidence behind supporting 100 billion vectors on a single server: Core reliance on the IBM Storage Scale System 6000 all-flash storage appliance: A single node is configured with 48 NVMe drives, equipped with PCIe Gen5 and 400Gb InfiniBand high-speed interconnects. Combined with NVIDIA GPUDirect Storage technology, it enables GPUs to directly access SSD data, bypassing the CPU data transfer step.
The system splits the massive index into multiple independent sub-indices. Each sub-index can be independently optimized, rebuilt, and operates without interference, completely solving the “touching one part affects the whole” reconstruction pain point of traditional vector databases.
Measured data comparison: In a pure CPU environment, rebuilding a 100-billion-level vector index would take 120 days. With the IBM Storage Scale System 6000 equipped with 6 NVIDIA H200 GPUs, it takes only 4 days to complete.
Conclusion
In the era of artificial intelligence, the role of storage is being redefined. IBM provides a clear answer: Storage should not be an AI bottleneck but a core accelerator for AI infrastructure.
This solution is available in two delivery forms: a pure software version and an appliance version. It is fully compatible with the RHEL AI open-source data pipeline and deeply integrated with the NVIDIA AI Data Platform, making it an enterprise-grade solution ready for production deployment.
The AI storage solution centered on IBM Storage Scale is turning petabyte-scale enterprise RAG from a technical concept into reality. The scale limit of RAG is no longer constrained by vector quantity or storage performance, but by the boundaries of data that enterprises can access and utilize.
|
[1] IBM Introduces Content-Aware-Storage for RAG Workloads, Storage review, April 22, 2026. https://www.storagereview.com/news/ibm-introduces-content-aware-storage-for-rag-workloads |
IBM Corporation logo.