SELF_HOSTED: TRUECLOUD_PROVIDER: AWSAPI_STYLE: OPENAI

Mantis

A self-hosted LLM gateway for routing, caching, guardrails, and observability across model providers.

01 // UNIFIED_API

A stable chat completions endpoint that sits in front of multiple model targets. Switch providers without changing a single line of client code.

OPENAICLAUDE+MORE

02 // INFRASTRUCTURE

Built for the AWS ecosystem. Deploys via Terraform to ECS, using ElastiCache for fast caching and Bedrock for foundational models.

ECSCLOUDWATCHBEDROCK

03 // LOGIC_ENGINE

Routing, retry, and fallback logic. Define timeouts, cooldown periods, and cache behavior directly in your config.

ROUTINGRETRYCACHING

// Capabilities

Route requests by metadata, model aliases, weighted targets, and fallback chains.

multi-providerweighted routingfallback chains

Coordinate validation, cache checks, cooldowns, provider calls, retries, and terminal responses.

lifecycle controlshort circuitretry logic

Send chat completion requests through a single gateway endpoint with optional routing metadata.

streaming supportunified schemasingle endpoint

Call Mantis from application code without manually constructing each HTTP request.

pip installableasync clientrouting metadata

// Team

Hubert de Parseval Europe

Rey van den Berg Dublin, Ireland

Riz Syed London, UK

Samuel Meka Taipei, Taiwan

// Next Steps

Route. Cache. Guard. Observe.