# AIVory Smart Inference - Same AI, Cheaper Bill

We launched Smart Inference - an OpenAI-compatible gateway that routes every request to the cheapest GPU provider. Change one line of code.


Okay, so… we built another product for AIVory: Smart Inference.
And the reason is simple: GPU inference is expensive, but most of the time you’re overpaying for it. There are $765 billion worth of GPUs out there and roughly half sit idle at any given time. That’s why providers offer them at a discount - if you can find their offers and keep track of the actual prices across 60+ providers. That takes time. So we built a service around it.
Smart Inference is an OpenAI-compatible gateway. You swap your base URL, keep your SDK, keep your model names. Every request gets scored and routed to the cheapest healthy provider. If that one goes down, it falls over to the next automatically.
We also launched spot GPU rental alongside it. H100s, H200s, B300s - by the second, no commitment, up to 80% off. You can deploy a HuggingFace model on one and route it through Smart Inference automatically.
I pitched this at Google Frankfurt last week and honestly, the hardest part is explaining what AIVory is now. Guard does code compliance, Smart Inference does inference costs, Architect does cloud infra. Getting all of that across in three minutes is tough, but I’m working on it.
Check it out at aivory.net/smart-inference. You pay for what you use, nothing else.
Cheers, Paul