Introduction

What is Mantis?

Mantis is an open-source, self-hosted Large Language Model (LLM) gateway that provides a single interface for interacting with multiple LLMs. It abstracts away the complexities of LLM integration through four core capabilities: configurable routing rules that direct requests to different models, response caching to reduce the latency and cost of repeated prompts, guardrails to control potentially harmful LLM requests or responses, and observability into the request lifecycle.