Zero-Idle Scale-to-Zero
Automatically unloads inactive models from GPU VRAM and host RAM after a configurable idle timeout, with LRU & reactive OOM eviction.
SnoozeML — Bare-Metal Scale-to-Zero ML Inference Daemon
Serverless scale-to-zero economics meets bare-metal Rust inference performance for ONNX, Safetensors, and XGBoost.
Adaptive Dynamic Micro-Batching
Coalesces concurrent HTTP REST, gRPC, and Kafka/NATS requests along Axis(0) within sub-millisecond windows.
Multi-Runtime Execution
Serve ONNX Runtime (ort), Hugging Face Safetensors (candle), and pure-Rust GBDT trees (hessboost) from a single binary.
Official Python SDK
Convert Scikit-Learn, PyTorch, and XGBoost models to ONNX and query over HTTP or high-throughput gRPC.