oruni
Your models. Your data. Your rules.
One OpenAI compatible endpoint in front of Anthropic, Google, OpenAI, Mistral, Groq, DeepSeek, xAI, three enterprise hyperscalers and local Ollama models, running entirely on your own machine. No intermediary account sees a prompt, no markup on token prices, PII is masked locally and a real Z3 solver proves it.
Status and price
Runs in production today. We clarify onboarding, demo and terms in a direct conversation.
Core is free
Apache 2.0, BYOK with no token price markup. Enterprise license (SSO, team memory, compliance reports) from 4,900 EUR one time per instance, no subscription.
oruni vs. a cloud LLM gateway
A central prompt entry point is too valuable economically to be given away.
A cloud gateway routes your prompts through a third party's own infrastructure before they reach the model provider, with its own pricing model and its own interest in exactly that data stream. oruni runs as a single Rust binary of about 13 MB entirely on your own machine, no Docker requirement, no server setup, a direct connection to the provider you choose.
One endpoint, 13 provider kinds, practically every vendor
Fully connected instead of merely advertised.
Anthropic, Google Gemini, OpenAI, Mistral, Groq, DeepSeek, xAI, local Ollama models, plus the three enterprise hyperscalers AWS Bedrock, Oracle OCI and Azure OpenAI Service as well as IBM watsonx.ai. Any other vendor with a real OpenAI compatible endpoint works without a line of code through kind: "custom", including the Chinese frontier labs and the common serverless inference hosts.
Failover, fallback chains, EU only routing, local first
Four mechanisms against a single point of failure.
Automatic failover on network errors, 429 or 5xx, while a real 4xx ends the chain immediately. Configurable fallback chains can even switch the provider kind. EU only routing enforces exclusively EU or locally flagged providers. If both fail and the error looks like a real connection loss, local first fallback automatically reroutes to the first reachable local provider.
Cache: from 17.3 down to 0.3 seconds
Exact match and semantic matching, automatic for every request.
A hash of model and message serves identical requests straight from the local vector cache, an additional semantic match over local embeddings also catches a mere rephrasing. Measured against a real model call: 17.3 seconds without a hit versus 0.26 to 0.32 seconds with a hit.
Persistent memory across apps
A fact remembered once, available to every connected application.
After every answer, question and answer are stored locally with an embedding and automatically injected as context for a topically similar, independent request, across apps. Encrypted in the same vault as provider keys, with origin and confidence value in the audit log as a defense against memory poisoning, switchable per request via header.
PII anonymization with a proof instead of a promise
The same Z3 solver that verifies outputs also secures the masking.
76 standard entity types are masked locally before a request reaches a provider, IBAN detection with a real ISO 7064 mod 97 check digit instead of plain format matching. A real Z3 SMT solver then formally proves that no original value remains in the outgoing text, a violation is blocked hard, not just logged.
Key vault: never in plain text
Hardware and OS bound, on every platform.
DPAPI on Windows, login keychain on macOS, Secret Service on Linux, an AES 256 GCM file fallback as the last tier without a desktop session. Verifiable with a grep against the raw database file, not just claimed.
Guardrails and a hash chained audit log
Rate limit, budget kill switch, circuit breaker, prompt injection pre check.
A sliding window rate limit per gateway key, a monthly budget with a real kill switch based on real provider costs, a circuit breaker that automatically pauses unstable providers. Every routing decision lands in the audit log hash chained, a verification endpoint recomputes the whole chain and reports the first break.
MCP gateway for agent frameworks
No unprotected side path next to the regular endpoint.
POST /mcp speaks the Model Context Protocol over JSON RPC 2.0, the chat_completion tool runs through the same core as /v1/chat/completions: routing, cache, PII masking and memory apply identically. Real provider passthrough streaming with a PII safe buffer runs for all 13 provider kinds, tool calls included.
The dashboard: no mockup
Every screenshot is a real capture of the running application.
A chat test field sends one question to several models at once, with latency, token usage and PII counter per answer. Live traffic stacked by routing result, a cost dashboard built from real, already billed provider values, plus OpenTelemetry export and price catalog sync, both deliberately opt in instead of automatic background traffic. Since the dashboard rework: consistent toggle and slider controls instead of text fields, a three way theme switch (light, dark, system), an info page with version, license and open source components used, and enterprise features visible as a preview even without a license instead of hidden entirely.
Router metadata, zero data retention routing and presets
Three new features that complete productive gateway use.
The header X-Oruni-Metadata: enabled returns an oruni_metadata field in every response, including the last stream chunk: actual provider, cache hit, fallback that kicked in, masked PII spots, injected memory facts, duration. Zero data retention routing optionally restricts requests to providers that do not store data. Presets bundle a model and a system prompt under one name such as preset/support-de.
Management keys, prompt injection allowlist and SSO group mapping
Built for platforms that build on oruni and for enterprise logins.
Management keys, a separate lower privileged key type, automatically create gateway keys for your own customers without manual dashboard interaction. An allowlist permits targeted exceptions in prompt injection detection instead of blocking every hit across the board. SSO group to role mapping automatically maps IdP groups to oruni roles at login, reevaluated on every login.
Pricing: open core, no token price markup
1,000 dollars of model cost stay 1,000 dollars.
The core is Apache 2.0 open source and free, BYOK with no margin of its own. A widespread cloud gateway charges a 5.5 percent fee in credit mode, oruni has no such markup at any volume. Enterprise features are licensed separately per instance or team, Ed25519 signed, with no cloud activation server.
Enterprise: governance you can prove
SSO, team memory, ensemble voting, formal verification.
SSO/OIDC with PKCE against any standard provider, team memory with conflict checks, multi provider ensemble voting with a visible cost total, automatic lockout after repeated prompt injection hits, hub and spoke policy distribution without shared runtime state, compliance reports and cost centers straight from audit log data.
Radical transparency instead of a marketing facade
A developer named by name, AI assisted development stated openly.
Built by a single person named in the legal notice, Apache 2.0 core, no vendor lock in, the baseURL can be switched to another provider in five seconds. The role of AI systems in the development process is stated openly instead of hidden, human final responsibility for every line of code remains.
Roadmap
What gets built out next
- Precompiled Linux and macOS packages (currently Windows release only)
- Signed Windows MSI installer for rollout via group policy, SCCM or Intune
- Independent external security audit of the open core
Inquiry
Ask about oruni
oruni is one of 23 systems I built entirely myself. Send me a short note about your case, I answer personally.
justautomatemore@gmail.com