Skip to content

Financial answers you can check.

FinRAG MCP gives Claude, Cursor, or any MCP client exact reported figures for any US-listed company and cited passages from SEC filings, plus a tool that checks every figure in the answer. Add it to Claude, no sign-in.

Recorded from the live server
|

Stated figures grounded, FinanceBench 150Q. Answers written mainly by the gpt-4o-mini fallback; to be re-measured.

1,082

SEC filings ingested

71

Companies

171,790

Chunks embedded

NVDAAAPLMSFTGOOGLMETATSLAFGMRIVNLCIDJPMBACGSMSVJNJPFEUNHABBVMRKXOMCVXCOPSLBOXYWMTCOSTHDTGTLOWBACATGEHONUPSTVZTMUSCMCSACHTRDISNFLXWBDPARAAMDINTCQCOMTXNAVGOMMMPEPAMCRBBYAXPMGMULTAADBEGLWCVSGISAESAMZNAWKSQKOLMTATVIFLKHCNKEPYPLNVDAAAPLMSFTGOOGLMETATSLAFGMRIVNLCIDJPMBACGSMSVJNJPFEUNHABBVMRKXOMCVXCOPSLBOXYWMTCOSTHDTGTLOWBACATGEHONUPSTVZTMUSCMCSACHTRDISNFLXWBDPARAAMDINTCQCOMTXNAVGOMMMPEPAMCRBBYAXPMGMULTAADBEGLWCVSGISAESAMZNAWKSQKOLMTATVIFLKHCNKEPYPL

Ask an LLM about revenue and you get a confident guess.

Every public company files the real numbers with the SEC every quarter. A plain chat model rarely has them, and rarely says so.

Stale training data

A model's knowledge cutoff is months old. It has no idea what a company reported last quarter.

Invented numbers

Ask for a specific figure and you get one, stated with full confidence and sourced from nothing.

No citation

Even a correct-sounding answer gives you nothing to check it against.

One question, every step shown.

  1. 01

    Claude picks a tool

    Your AI assistant decides which of the six tools fits your question.

  2. 02

    Limits and cache

    A shared limit keeps the server fair for everyone. If the same call was made in the last day, the saved result comes back right away.

  3. 03

    Ticker and year filters

    The server works out which company and which year you mean.

  4. 04

    GAAP expansion

    Your wording is translated into the words a filing actually uses.

  5. 05

    BM25 and Pinecone

    Two searches run at once, one by exact words and one by meaning, so a miss in one can be caught by the other.

  6. 06

    Full text and rerank

    Each hit is replaced by its complete passage, then the 5 best are kept.

  7. 07

    Evidence returned

    You get the source passages, each labeled with the filing it came from.

  8. 08

    Claude writes the answer

    Your own AI assistant writes the answer, using only what the passages say.

  9. 09

    verify_figures

    The dollar amounts and percentages in the answer are checked against the sources. Anything missing is flagged.

  10. 10

    Log

    A record of each call is kept for up to 90 days in case something needs checking.

The part that says no.

Edit the number below. If it doesn't match what's in the source chunk, the verifier flags it, live, in your browser. This is a TypeScript port of the production verifier's matching rule.

Retrieved source chunk

From 3M's real 10-K, as extracted by the pipeline. Table cells only, cleaned up for reading here; the verifier below runs against the unmodified chunk.

Years ended December 31
(Millions)201820172016
Major GAAP Cash Flow Categories
Net cash provided by operating activities$6439$6240$6662
Net cash provided by (used in) investing activities222(3,086)(1,403)
Net cash used in financing activities(6,701)(2,655)(4,626)
Free Cash Flow (non-GAAP measure)
Net cash provided by operating activities$6439$6240$6662
Purchases of property, plant and equipment (PP&E)(1,577)(1,373)(1,420)
Free cash flow$4862$4867$5242
Net income attributable to 3M$5349$4858$5050
Free cash flow conversion

Generated answer (editable)

3M's capital expenditure in fiscal year 2018 was $ million.

Grounded: this figure appears in the retrieved source.

The bug this exists to prevent

An early version of this verifier used substring matching: it checked whether "$1,234" appeared anywhere in the source text. That passed a fabricated $1,234 whenever the source actually said $1,234.56, since the shorter string is a literal substring of the longer one. The fix compares parsed numeric magnitude instead of text, which is what you're running above.

Six tools. Figures and evidence in, your model writes.

Requests and responses below are real calls to the live server, with passage text shortened. Figures work for any US-listed company; passage search covers 71.

search_sec_filings

Free-text search over every filing. Optional ticker and fiscal year filters.

{
  "question": "3M capital expenditure",
  "ticker": "MMM",
  "fiscal_year": 2018
}
{
  "passages": [
    {
      "citation": "[MMM 10-K 2019-02-07]",
      "text": "Company pension and postretirement contributions Company pension and postretirement expense Income taxes (deferred and accrued income taxes) Net cash provided by operating activities Cash fl ..."
    },
    "... 4 more"
  ]
}

get_company_financials

Exact reported figures for any US-listed company, read from the XBRL data in its filings, with period, filing link and citation. Q4 is labelled when derived.

{
  "ticker": "MMM",
  "metrics": [
    "capital expenditure"
  ],
  "period": "FY2018"
}
{
  "results": [
    {
      "label": "Payments to Acquire Property, Plant, and Equipment",
      "display": "$1,577.0 million",
      "period": {
        "fiscal_year": 2018,
        "fiscal_period": "FY",
        "start": "2018-01-01",
        "end": "2018-12-31"
      },
      "citation": "[MMM 10-K 2019-02-07]",
      "id": "xbrl:MMM:us-gaap:PaymentsToAcquirePropertyPlantAndEquipment:FY2018"
    }
  ]
}

compare_companies

The same figures for two to five companies, side by side.

{
  "tickers": [
    "TSLA",
    "F"
  ],
  "metrics": [
    "revenue"
  ],
  "period": "FY2024"
}
{
  "results": {
    "TSLA": [
      "$97,690.0 million [TSLA 10-K 2025-01-30]"
    ],
    "F": [
      "$184,992.0 million [F 10-K 2025-02-06]"
    ]
  }
}

get_latest_filing

Any company SEC lists, any form (10-K by default), looked up live from EDGAR, with a flag for whether it is already in the corpus.

{
  "ticker": "AAPL",
  "filing_type": "10-Q"
}
{
  "filing": {
    "ticker": "AAPL",
    "cik": 320193,
    "form": "10-Q",
    "filed": "2026-07-31",
    "accession": "0000320193-26-000020",
    "primary_document": "aapl-20260627.htm"
  },
  "in_corpus": true
}

list_covered_companies

Every ticker with first and last filing date per form. Call it to see exact years.

{}
{
  "total_filings": 1082,
  "companies": [
    {
      "ticker": "AAPL",
      "forms": {
        "10-K": {
          "first": "2024-11-01",
          "last": "2025-10-31",
          "filings": 2
        },
        "10-Q": {
          "first": "2025-08-01",
          "last": "2026-07-31",
          "filings": 4
        }
      }
    },
    "... 70 more"
  ]
}

verify_figures

Send your draft answer and the passage and figure ids. Each dollar amount, percentage and comma-grouped number comes back grounded or not_found; against a figure, units convert.

{
  "answer": "3M spent $1,577 million on capex in 2018.",
  "passage_ids": [
    "5d44fe78114e18919005a0d70ddd24e4",
    "92b3ab0ae49e688caee769614bc6c911",
    "..."
  ]
}
{
  "grounded": [
    "$1,577 million"
  ],
  "not_found": [],
  "checked_against": 5
}

Measured, not claimed.

End to end, the way a user meets it: a model calls the real tools in a loop, as Claude or Cursor would, and each answer is scored right, wrong or refused against FinanceBench's answer key. gpt-4o-mini plays the client.

Agent test, 2026-10-30Search only+ exact figures (live)
FinanceBench right (of 110)6792
Wrong2114
Refused224
Fourth-quarter set right / wrong (of 48)13 / 3239 / 7

Exact figures come from the XBRL data in each filing, for any US-listed company. Searching passages, the model had to find one table row among 171,790 chunks; most of its fourth-quarter answers reported the full year, since no filing states a Q4 on its own. The figures tool derives Q4 and says how.

The one-shot test isolates retrieval: one search, five passages, one answer. Re-measured on 2026-10-30: the same 150 FinanceBench questions through the live server's retrieval, answers written by gpt-4o-mini, September answers re-scored with the same code. Automated checks, not hand-graded. “Voyage rerank” is the same server with a neural reranker over 40 candidates per search instead of a word-overlap scorer over 10.

FinanceBench, 150 questionsSeptemberWord-overlap rerankVoyage rerank+ clean tables (live)
Refused to answer121995561
Right number (of 110)16265453
Wrong number (of 110)49149
Evidence passage in the top 5 (of 127)not measured13.4%45.7%46.5%
Every figure found in its sourcesnot measured96.7%84%85.3%

Clean tables raised refusals from 55 to 61 and cut wrong answers from 14 to 9: 17 of the 18 answers that became refusals had been wrong, mostly a full-year figure reported as a quarter. A wrong figure is worse than a refusal, so refusals alone are the wrong scoreboard. The figure check is lower than before because more answers compute ratios or convert units, which it does not do by design. The charts below are the September pipeline.

Where the metric lied

BM25-only scores higher here than the full pipeline, and for a while I published that as an honest loss. The metric is the problem. Numerical accuracy asks what fraction of the numbers an answer states are grounded, and an answer with no numbers in it scores a perfect 1.0, so refusing raises the score. BM25-only refused the most (133 of 150, against 121 for the other two). But that explains only part of its lead. On a rough check of whether the answer states the correct number, the three setups land within noise of each other (52, 48 and 52 of 110, a count that also included stray numbers inside refusals). So this benchmark cannot rank them either way. The lesson is to ask what a do-nothing system would score before trusting a metric.

What we can't explain yet

Answer relevancy, context precision and context recall score low (about 10 to 20%). Part of that is explained: about four in five answers were refusals, and ragas scores a refusal as zero relevancy; about 28 of 150 questions need years the corpus doesn't hold; and in a rough check the right chunk was in the top five only about half the time. Whether the metrics also fit this task badly (terse numeric answers judged against long table chunks) is untested.

Cost and latency.

Measured against the live server on 2026-10-30, 10 searches. A warm search takes at the median, and the slowest of the 10 took . One search costs about , an estimate from Lambda time plus one embedding, not a bill. There is no chat model cost, because the server runs none.

The cache race: 2.6x faster on a repeat call

Cache miss (warm)
Cache hit

The first call after a deploy is slow: server-side, because the Lambda loads the 964 MB keyword index. A health check every 5 minutes keeps one instance warm; a second instance, started when calls overlap, pays the same cost once.

Bugs that looked like answers.

Financial RAG fails quietly. A wrong number looks exactly like a right one until you check it. These are real bugs from building this system, not hypotheticals.

Read the full write-up on these bugs

Dense results silently discarded

Symptom
faithfulness scored 0.96 but every context metric scored ~0.0.
Root cause
merge_and_dedup concatenated BM25-then-dense and rerank truncated with chunks[:top_k], so the top-5 was always 100% BM25, and every dense result was discarded before reranking ever saw it.
Fix
Hybrid search now actually merges both sources before the top-k cut.

The substring verifier

Symptom
Fabricated figures passed the grounding check undetected.
Root cause
"$1,234" is a literal substring of "$1,234.56", so a naive text-containment check treats a fabricated $1,234 as grounded whenever the source happens to contain a longer number that starts the same way.
Fix
Compare parsed numeric magnitude instead of raw text. See the verifier demo above.

uuid4 chunk IDs

Symptom
Re-ingesting a filing duplicated every one of its chunks.
Root cause
chunk_id was uuid4(), so every re-ingest generated fresh IDs. BM25 and Pinecone IDs never matched, so cross-source dedup silently couldn't work at all.
Fix
Chunk IDs are now content-addressed (sha256), so the same chunk always gets the same ID.

The corpus held the wrong years

Symptom
147 of 150 FinanceBench questions were unanswerable.
Root cause
list_filings defaulted to "last 2 10-Ks and 4 10-Qs", and in 2026 that means 2025-2026 filings, while FinanceBench asks about 2015-2024. The corpus had the right companies, just the wrong years.
Fix
Benchmark companies now ingest {10-K: 8, 10-Q: 12, 8-K: 8}, 28 filings each instead of 6.

Unpinned temperature

Symptom
The identical question, through the identical fixed pipeline, was correct once and wrong once.
Root cause
A coin flip at each API's default temperature (1.0). The same query could return a correctly cited figure or "the context does not provide this figure" on two separate runs.
Fix
All four LLM call sites (Bedrock + OpenAI fallback, rewriter + generator) now pin temperature=0. All four have since been removed from the server.

The XBRL prior-year-comparative trap

Symptom
NVDA revenue was reported as $26,044M instead of the correct $44,062M.
Root cause
A 10-Q's XBRL tags the prior-year comparative figure under the same fy/fp as the current quarter, so a naive lookup grabbed last year's number for AAPL, NVDA, META, and GOOGL alike.
Fix
Ground-truth extraction now takes the maximum end date within the filing, not the first match.

The model on the label wasn't the model writing answers

Symptom
The site credited the 91.0% FinanceBench score to Claude Sonnet.
Root cause
The account's Bedrock quota was 0. Over 100 days CloudWatch shows 241 successful Bedrock calls against about 20,000 throttles, so the gpt-4o-mini fallback wrote most answers. The answer files never recorded which model wrote them.
Fix
Evals now play the client with a named model and record it on every answer. The 91.0% figure stays on this site with that caveat until it is re-measured.

Vector hits arrived cut to 1,000 characters

Symptom
Figures that sat late in a chunk never reached the answer when a hit came from Pinecone alone.
Root cause
Pinecone metadata stored only the first 1,000 characters of each chunk, but 75% of chunks are longer than that (the average chunk is 3,728 characters).
Fix
Each candidate is swapped for its full stored text before reranking. On the same index, evidence in the top 5 rose from 10.2% to 13.4% and the number in the top 5 from 47.8% to 52.2%.

And everything underneath.

OpenAI embeddings
MCP streamable HTTP
Pinecone
AWS Lambda
DynamoDB
EventBridge
CloudWatch
SNS alerts
SQLite FTS5
ragas
AWS CDK

Add it to Claude. No sign-in.

The server is public. Streamable HTTP, one URL.

  • Claude (claude.ai and Claude Desktop)

    Customize > Connectors > + Add > Add custom connector. Enter a name, paste the URL, and continue. Choose No sign in if asked. On Team and Enterprise plans an Owner adds it first under Organization settings > Connectors.

    https://ecvsxkeqdpyj5hkal7wplm2b4q0gacfh.lambda-url.us-east-1.on.aws/mcp
  • Claude Code

    One command in your terminal.

    claude mcp add --transport http finrag https://ecvsxkeqdpyj5hkal7wplm2b4q0gacfh.lambda-url.us-east-1.on.aws/mcp
  • Cursor and other MCP clients

    In Cursor, add this to .cursor/mcp.json in your project or ~/.cursor/mcp.json. Other clients that speak streamable HTTP take the same URL, though their config keys may differ.

    {
      "mcpServers": {
        "finrag": {
          "url": "https://ecvsxkeqdpyj5hkal7wplm2b4q0gacfh.lambda-url.us-east-1.on.aws/mcp"
        }
      }
    }

Deploy your own.

Want your own limits and your own corpus? These are the exact steps. You need an AWS account with the CLI configured, Node 20 or newer, Python 3.12 or newer, an OpenAI API key, a Pinecone account and a Voyage AI key with a payment method on file. Everything deploys to us-east-1.

  1. 01

    Create the Pinecone index

    Serverless index named finrag-filings, 1536 dimensions, cosine metric
    (it must match OpenAI text-embedding-3-small)
  2. 02

    Clone and install

    git clone https://github.com/karthikreddyyalala/FinRAG.git && cd FinRAG
    python3 -m venv .venv && source .venv/bin/activate
    pip install -r requirements.txt
  3. 03

    Rename the S3 buckets (names are global), then set SEC_USER_AGENT in pipeline/edgar_client.py to your name and email

    perl -pi -e 's/finrag-(raw|processed)-filings/yourname-$1-filings/g' $(git grep -lE 'finrag-(raw|processed)-filings' -- '*.py')
  4. 04

    Store your keys in SSM

    aws ssm put-parameter --region us-east-1 --name /finrag/openai-api-key --type SecureString --value <your-openai-key>
    aws ssm put-parameter --region us-east-1 --name /finrag/pinecone-api-key --type SecureString --value <your-pinecone-key>
  5. 05

    Deploy the CDK stacks (alert_email is optional: alarm emails and a $10 monthly budget alert)

    npx aws-cdk bootstrap aws://<account-id>/us-east-1
    npx aws-cdk deploy --all --require-approval never -c alert_email=you@example.com
  6. 06

    Build the corpus (takes hours, a few dollars of OpenAI embeddings), then seed the weekly refresh

    export OPENAI_API_KEY=<your-openai-key> PINECONE_API_KEY=<your-pinecone-key>
    PYTHONPATH=. python scripts/bootstrap_corpus.py
    aws s3 sync evals/results/chunk_cache s3://yourname-processed-filings/chunk_cache/
  7. 07

    Connect with the McpEndpoint stack output

    claude mcp add --transport http finrag <McpEndpoint, already ending in /mcp>