Buoyant has launched Kinference, an open-source project for building scalable inference platforms on Kubernetes.
“Enterprises need a plug-and-play solution to private AI that allows them to handle the wide variety of use cases and tenants within their organisation in a unified way,” said William Morgan, CEO and founder of Buoyant.
Kinference
Built on the open-source vLLM inference engine, Kinference enables organisations to deliver safe, reliable, and cost-effective inference across multiple models, GPUs, and machines. It supports use cases such as software engineering, application intelligence, and agentic tasks, and provides the governance and policies required for enterprise-scale deployment.
The open-source project allows organisations to offer a true internal inference platform capable of:
- Inference guardrails and controls. Kinference lets platform owners define boundaries around performance, cost, and compliance; developers then have maximum flexibility to consume inference within those constraints.
- Lowered cost of inference. Kinference improves GPU utilisation and inference efficiency by selecting endpoints that minimise GPU work, by taking into account KV cache data on the GPU, queueing status, and other real-time operational characteristics.
- Intent-based routing. Kinference provides semantic classification to enable policies such as “send simple prompts to the faster, smaller model”.
- Burst to cloud. Kinference can optionally send inference requests to cloud providers (including frontier models) under conditions defined by the platform owner, which can include GPU pool saturation, usage by specific tenants, or characteristics of the inference request itself.
- Per-tenant usage and cost attribution. Kinference can track GPU usage per tenant, group, or team and provide the resulting cost attribution, giving organisations a full picture of inference spend.
