Introducing Mantis

Unlike Bifrost and LiteLLM, Mantis pairs the gateway itself with infrastructure that can be provisioned and deployed with a single terminal command. Mantis is built for small teams building multi-LLM-powered applications. It provides four core capabilities, namely, configurable routing, response caching, guardrails, and observability, whilst being easy to set up on one’s own AWS infrastructure. This enables teams to quickly benefit from an LLM Gateway with a quick, easy setup while remaining fully in control of their data.