There is a particular kind of wrong answer that should frighten anyone using AI to research their money.
Ask a typical AI chatbot how a stock's current valuation compares to its history, and you will often get a confident, specific, well-formatted reply: a price-to-earnings multiple, a five-year average, a clean verdict about whether the stock looks cheap or expensive. It reads like analysis. It has numbers in it. It sounds exactly like the thing you asked for.
And a meaningful share of the time, some of those numbers were never real. The model didn't look them up. It produced figures that were statistically plausible — the kind of number that would fit — and presented them with the same confidence it uses for facts it actually has.
We built Aurvus as a portfolio-native investing companion, and the single hardest engineering problem we faced wasn't pulling data or computing risk metrics. It was getting an AI to stop making things up — and, just as importantly, to say "I don't have that" out loud instead of papering over the gap. This post is about why that problem exists, why it's especially dangerous with money, and how you can catch it in any tool you use, including ours.
How AI tools fabricate financial numbers
Large language models don't retrieve facts the way a database does. They generate the most probable next piece of text given everything before it. That's why they're fluent — and it's also why, when they lack a specific fact, they don't reliably stop. They generate the shape of an answer that fits.
In casual use this produces the familiar "hallucination." In financial research it produces something more insidious, because finance is full of numbers that have a plausible range. A model that doesn't know a company's historical P/E knows roughly what historical P/Es look like. So instead of refusing, it can emit a number in the believable zone — say, "the stock trades at 24x versus a five-year average of 18x" — that is entirely invented but utterly convincing.
The danger isn't that the answer looks crazy. It's that it looks exactly right. A fabricated valuation that lands in a sensible range is far more dangerous than an obvious error, because nothing about it triggers your skepticism. You read it, it confirms or challenges your thinking, and you act on a number that came from nowhere.
We saw this concretely while building Aurvus. Early on, when a user asked how their portfolio had performed against the S&P 500 and the underlying data wasn't present in that view, our own system would sometimes produce an answer like "your portfolio is up around 9% versus roughly 15% for the S&P." Reasonable-sounding. Specific. And completely manufactured — there was no return data in front of the model at all. It had filled the gap with numbers that merely seemed right.
That was the moment the real product problem came into focus. The job wasn't to make the AI smarter. It was to make it honest about the edge of its knowledge.
The red flags that you're looking at a guess
You don't need to see the code to catch a fabricating tool. The tells are in the output. Here is what we learned to look for — and what we built Aurvus to never do.
It never says "I don't know." A trustworthy financial tool refuses constantly, because real data has real gaps. If an AI tool answers every single question with the same confident fluency, that is not a sign of intelligence. It's a sign that it treats "I don't have this" and "here is the fact" identically — which means you can't tell them apart either.
The numbers have no visible source or timestamp. Real financial data is dated and sourced. "As of the most recent quarter," "trailing twelve months," "from the latest filing." When precise figures appear with no indication of where or when they came from, treat them as unverified until proven otherwise.
Precision without provenance. A model saying a stock is "expensive" is an opinion you can evaluate. A model saying it trades at "31.4x versus a 22.1x ten-year average" is a claim of fact — and if it can't tell you where the 22.1x came from, the decimal points are theater. Fabrication often hides behind false precision, because specific numbers feel more authoritative than vague ones.
It can't reproduce the answer. Ask the same question twice. A tool reading real cached data gives you the same figures. A tool generating plausible numbers will often drift between runs, because it's re-rolling the dice each time.
It answers questions it structurally can't. Some questions require data the tool simply may not have — a multi-decade valuation history, intraday detail on an obscure name, a forward estimate. A tool that always produces a confident answer to these, rather than sometimes saying "that lives elsewhere / I don't have that depth," is almost certainly filling gaps with invention.
Why "I don't know" is a feature, not a failure
Here is the counterintuitive part, and it's the principle Aurvus is built around: an investing tool that admits its limits is more useful than one that hides them — even though it feels worse in the moment.
When you ask Aurvus how your portfolio compares to the S&P this year and that return data isn't in the relevant view, it doesn't invent a comparison. It tells you it doesn't have the performance data in that view, tells you what it does have (your holdings, your weights, your concentration), and points you to where the real number lives. That's a less satisfying answer than a fabricated "+9% vs +15%." It's also the only honest one — and on decisions involving real money, honest beats satisfying every time.
This isn't a stylistic choice. It's a design discipline that runs through the whole system: every figure comes from real, dated data, and when the data isn't there, the tool says so rather than reaching for a number that "would fit." We test for this deliberately — we throw questions at the system specifically designed to tempt it into fabricating, and we treat any invented figure as a failure to fix, not a quirk to tolerate.
The reason is simple. A tool that's right 95% of the time and confidently fabricates the other 5% — with no way to tell which is which — is worse than useless for money, because you can't trust any of it. A tool that's honest about the 5% it doesn't know lets you trust the 95% it does.
How to research with AI without getting burned
You don't have to abandon AI for investment research. It's genuinely powerful for synthesis, for explaining concepts, for surfacing what you didn't think to ask. You just have to use it with the right guardrails:
- Treat every specific number as unverified until sourced. Use AI to understand and frame, then confirm hard figures against a primary source — the filing, the brokerage, the fund page.
- Reward refusal. A tool that tells you what it doesn't know is showing you its edges. That's the behavior you want. Be suspicious of the tool that never hits a limit.
- Prefer tools grounded in your real data. Generic stock chatter invites fabrication because the model is reaching into a vague universe. A tool reading your actual holdings has a far smaller surface to invent on — and a good one will tell you when even that data is missing.
- Re-ask and cross-check. If the numbers move between runs, you've found a guesser.
The bottom line
AI is going to be how a lot of people research their money from here on. That's not the problem. The problem is that fluency and accuracy feel identical from the outside, and finance is exactly the domain where a confident, plausible, fabricated number does the most damage.
We built Aurvus because we wanted an investing companion we could actually trust with our own portfolios — one that reads your real holdings, grounds every answer in real data, and has the discipline to say "I don't know" instead of making something up. If you want to see what that feels like, try the copilot with your portfolio and ask it something it might not be able to answer The honest answer is the whole point.
Aurvus provides portfolio analysis and research tools for informational purposes. It is not a registered investment advisor and does not provide personalized investment advice. Always consult a qualified professional about your specific situation before making investment decisions.
