Metis Labs · Evals

Metis Labs Evals

In development

Accuracy testing for finance AI agents.

Our first focus is financial research: checking answers against company filings and verified calculations.

Compare the agent’s illustrative 2 percent answer with the calculation, reporting period and units. Planned method illustration · Illustrative values
Planned services

What we’re developing

We’re developing evaluations that show where an answer fails, the evidence behind the finding and what to test again after changes.

01

Focused review

Examine a small, agreed set of questions to find where answers fail.

02

Deeper evaluation

Test facts, calculations and sources across the agent’s intended tasks.

03

Findings and recommendations

Explain the failures, show the supporting evidence and suggest improvements.

04

Retesting

Repeat the agreed checks after the team changes its agent.

The accuracy story

A convincing answer is not enough.

An answer can sound convincing and still be wrong. We want to build useful AI and check its answers against evidence.

Our planned evaluation methodIn development
01 / 04

Known answers

Start with a reference answer supported by a company filing and a checked calculation.

Illustrative revenue inputs: 12 million this year and 10 million last year. The checked growth is 20 percent. Planned method illustration · Illustrative values
02 / 04

Clear checks

Compare the answer’s facts, reporting period, units and sources with that reference.

Compare the agent’s illustrative 2 percent answer with the calculation, reporting period and units. Planned method illustration · Illustrative values
03 / 04

Reviewed failures

Show where the answer differs, the evidence behind the finding and what needs attention.

The illustrative 2 percent answer differs from the reference calculation. Explain the failure before recommending a change. Planned method illustration · Illustrative values
04 / 04

Repeat testing

Run the agreed checks again after the team changes its agent.

After a change, repeat the agreed checks. A revised 20 percent answer is an example, not a measured agent result. Planned method illustration · Illustrative values

Illustrative example, not a completed client evaluation.

Discuss an evaluation.

Tell us what your agent does and which answers need careful checking.

Email instead ↗