Rafay unveils serverless inference to power AI-as-a-Service for GPU cloud providers
Rafay launched a Serverless Inference offering to help NVIDIA Cloud Partners (NCPs) and GPU Cloud Providers deliver high-margin AI services quickly and cost-effectively.
The offering provides a token-metered API for running open-source and privately trained/tuned large language models (LLMs). Key features include seamless developer integration, intelligent infrastructure management, built-in metering and billing, enterprise-grade security, and observability tools.
It enables NCPs and GPU Clouds to transition from GPU-as-a-Service to AI-as-a-Service, addressing the growing demand in the AI inference market. The solution eliminates infrastructure complexity, allowing developers and enterprises to integrate generative AI workflows into applications rapidly.
“Having spent the last year experimenting with GenAI, many enterprises are now focused on building agentic AI applications that augment and enhance their business offerings,” says Haseeb Budhani, CEO and co-founder of Rafay Systems. “The ability to rapidly consume GenAI models through inference endpoints is key to faster development of GenAI capabilities. This is where Rafay’s NCP and GPU Cloud partners have a material advantage.”
This solution represents a shift towards more dynamic, scalable AI workloads that can operate closer to data sources, reducing latency and enhancing real-time processing. Furthermore, it could accelerate the adoption of edge-based machine learning applications across industries, driving growth in edge AI inference markets.
The global AI inference market is projected to grow significantly, reaching $106 billion by 2025 and $254 billion by 2030.
Rafay’s platform supports multi-tenant GPU/CPU infrastructure and will soon include fine-tuning capabilities for AI models. Rafay aims to simplify cloud-native and AI infrastructure management, with customers such as MoneyGram and Guardant Health leveraging its solutions.
New research exposes gaps as edge AI becomes mission-critical
Article Topics
AI/ML | cloud infrastructure | generative AI | GPU cloud | serverless inference
Comments