Rhesis AI
German open-source workspace where domain experts review agent behaviour and engineers turn it into tests
Quick Overview
| Company | Rhesis AI GmbH |
|---|---|
| Category | LLM Observability and Evaluation Tools |
| Headquarters | Potsdam, Germany |
| Founded | Not stated |
| EU Presence | Yes - Germany (EU) |
| Data Location | Cloud or self-hosted |
| Open Source | MIT, with an enterprise-licensed part (per README) |
| GDPR Compliant | Yes |
| Self-Hosting | Yes - open-source edition |
| Main Features | Expert annotation, test sets, conversation simulation, adversarial testing, DeepEval metrics, traces, SSO, audit log, SDK and MCP access |
| Pricing | Free cloud allowance and free self-hosted edition; see website for paid cloud plans |
| Best For | Teams where non-engineers know the right answers and must review the agent |
| Replaces | LangSmith, Arize |
Detailed Review
Rhesis starts one step before traces: the people who know what an agent should answer are rarely the people building it. Reviewers open a shared link, chat with the live agent, turn conversations into tests and leave pass or fail verdicts on single metrics or conversation turns.
The README says the project is made in Potsdam, Germany, and the imprint page names Rhesis AI GmbH. Code is on GitHub with an MIT badge and an enterprise-licensed part, and the documentation covers self-hosting guides.
Feedback attached to tests
Every review sits on the test case and the agent version that produced it, rather than in a chat thread. Recurring feedback becomes tests and metrics that can run on each change.
The documentation lists test generation, import from files and from Garak, generation from OWASP material, conversation simulation and adversarial testing.
Traces and metrics
A tracing section covers decorators, custom spans, auto-instrumentation for several agent frameworks, conversation tracing and token costs. Metrics include DeepEval-based and code metrics.
Teams and access
The documentation lists organisations, roles, single sign-on, API clients and audit-log streaming, and the product is reachable through a UI, an SDK or an MCP server.
Limitations to Consider
The pricing page text available without scripts only says that Rhesis Cloud has a monthly allowance and that the open-source edition is not metered, so paid prices were not verifiable. The licence is MIT plus an enterprise part, which GitHub does not recognise as a single standard licence. It is a collaboration and testing layer rather than a high-volume production trace store; LangWatch is closer to that.
Who Should Use Rhesis AI
Product teams that need domain experts to review and approve agent behaviour and want an open-source tool from a German company.
Pros and Cons
Pros
- Non-engineers can review through a shared link
- Feedback becomes tests and metrics
- Self-hostable open-source edition
- Test generation from OWASP and Garak sources
- SDK, UI and MCP access
Cons
- Paid cloud prices not verifiable without scripts
- Licence mixes MIT and enterprise terms
- Younger project, 402 GitHub stars
- Less focus on high-volume production monitoring
Alternatives to Rhesis AI
Other European llm observability and evaluation tools worth comparing before you decide:
LangWatch
Netherlands · Open-source agent tracing, evaluations and simulations from Amsterdam
Latitude
Spain · Open-source agent observability that groups failures into issues
Galtea
Spain · Test generation, simulation and evaluation for regulated industries
Giskard
France · Red teaming and evaluation for LLM agents before release
See all European llm observability and evaluation tools or compare them directly against LangSmith on our alternatives page.
Frequently Asked Questions
Yes. The README shows an MIT plus enterprise licence badge and describes the project as open source, SaaS or self-hosted.
Teams where domain experts, not engineers, know the right answers. They review the agent in a browser while engineers use the SDK or MCP.
Yes. The documentation has a tracing section with decorators, custom spans, auto-instrumentation and conversation tracing.