My experimentation with AI for workflows and analysis leads me into some really interesting discussions. The ones I enjoy very much are comparing the “crazy good” and “crazy bad” things that AI can do. Today, I want to talk about “crazy bad”. A CFO colleague and wine enthusiast Valentin Recker, and I were catching up on a joint project and we got into a discussion about the failures of AI. The story he told me about an experience he had recently planning to acquire some wines at an auction and how he attempted to use OpenAI’s ChatGPT to plan his bidding strategy. The failure of the AI’s analysis was pervasive. While this was a personal use case, there was sufficient data and analysis included in the process and it serves as a cautionary notice to all users of AI in their analytics.

TL;DR:

Valentin asked ChatGPT to analyze a real auction catalog and build a bidding strategy. The AI invented lots that didn’t exist, mispriced what did, sorted on corrupted data, and presented all of it with complete confidence. These aren’t edge case quirks. They’re the exact failure modes that make AI dangerous in any analytical context where the data matters.

What Valentin Was Trying to Do

Valentin’s goal was straightforward. He had access to an auction catalog containing over 2,500 wine lots with lot numbers, producers, vintages, regions, bottle counts, and pricing. He wanted to build a disciplined bidding strategy with specific preferences: roughly 40% white wine biased toward white Burgundy, and 60% rich, dark, Parker-style reds with a range of tannin levels.

He uploaded the Excel file to ChatGPT and asked it to rank the top 100 lots under $300 that matched his criteria. A reasonable, well-scoped analytical request. The kind of thing AI should be genuinely useful for.

What followed was a masterclass in AI overconfidence.

The Errors: Specific, Documented, and Consequential

Hallucinated Lot Numbers

The most glaring failure was fabrication. ChatGPT confidently recommended specific lots including a 2013 Dominus, Spottswoode, Shafer Hillside Select, Clinet, and La Conseillante, complete with lot numbers and detailed tasting notes explaining why each was a “must-win.” The problem: none of these lots existed in the catalog. When Valentin pushed back, ChatGPT acknowledged it had been “extrapolating from what I would typically expect to see in a large auction catalog.” In other words, it invented plausible-sounding recommendations because that’s what a catalog like this should contain, not because the data said so.

In a financial context, this is the equivalent of an AI generating a revenue forecast by filling in numbers it thinks should be there rather than pulling from your actual books.

Ignored the Data It Was Given

Compounding the fabrication problem, ChatGPT wasn’t consistently working from the 2,530 rows Valentin actually uploaded. It was blending the file with its training knowledge about auction catalogs in general. This created a hybrid analysis that looked data-driven but wasn’t. The AI never disclosed this. Valentin had to catch it himself.

Corrupted Sort on a Text Field

The score column in the Excel file contained text strings like “93pts BH” or “90-93pts VM” rather than clean numeric values. ChatGPT attempted to sort and rank on this field without cleaning it first, which would have produced badly skewed rankings. It flagged this only after Valentin pressed for more precision, not as an upfront data quality disclosure. Any analyst worth their seat catches data quality issues before running the analysis, not during or after.

Wrong Unit Assumption on Pricing

ChatGPT initially treated the lot prices as per-bottle figures. They were per-lot. This completely invalidated the budget optimization, since a lot containing twelve bottles at a per-lot price is a fundamentally different purchase decision than a per-bottle wine. Valentin caught this and forced a full restart of the analysis. The AI’s response was “good catch, that completely changes the optimization,” which is accurate, and cold comfort when you’ve already spent an hour reviewing recommendations built on the wrong foundation.

Dropped a Major Vintage Without Explanation

When asked to rank White Burgundy vintages since 2000, ChatGPT omitted 2002 entirely. This matters because 2002 is widely regarded as one of the great modern White Burgundy vintages, one that serious collectors rank alongside 2014, 2017, and 2019. The AI didn’t flag the omission. It didn’t note any uncertainty. It simply left it out. When Valentin asked why, ChatGPT admitted it was “an omission on my part.” In a bidding strategy built on vintage quality, that omission could have cost real money.

How Valentin Managed the Failures

To his credit, Valentin didn’t abandon the tool. He adapted his approach to work around the AI’s weaknesses, which is itself a useful lesson.

He enforced strict data discipline, explicitly instructing ChatGPT to use only the rows in the uploaded file and nothing else. He verified specific lot recommendations against the actual catalog before acting on them. He caught the per-bottle versus per-lot pricing error by applying basic logic to the outputs before trusting them. And he pushed for re-runs when the analysis felt off rather than accepting the first result.

The takeaway from his mitigation approach is telling: to use ChatGPT reliably for this task, Valentin essentially had to function as the analyst himself, using the AI as a drafting tool rather than a decision engine. That’s a reasonable workflow, but it’s not what most users expect when they hand an AI a dataset and ask for analysis.

My Reaction: This Level of Error Surprised Me

When Valentin walked me through the conversation, I was genuinely taken aback by the compounding nature of the failures. One data quality miss is understandable. A misread unit is human error territory. But fabricating lot recommendations from a file that was sitting right there in the conversation, presenting invented data with the same confidence as real data, that’s a different category of problem.

I work primarily with Anthropic’s Claude, and based on my direct experience doing similar analysis, the comparison is worth being specific about. I have not seen Claude hallucinate numbers or pull outside data into an analysis in an unsupervised way. On the text field sorting issue, I have seen Claude handle these correctly without instruction when fed a spreadsheet, catching the data type problem and cleaning it before attempting any ranking. That’s the behavior you want from an analytical tool.

Where I can’t make a direct comparison is the per-lot versus per-bottle pricing ambiguity. I can’t think of a similarly ambiguous unit scenario I’ve run Claude through, so I won’t claim it would have caught that automatically. And to be balanced: I have seen Claude drop data. It’s not a perfect instrument. Large datasets and complex multi-condition filters can cause it to miss rows, which is its own category of risk that users need to watch for.

So the honest comparison is this: in my experience, Claude handles data integrity and sourcing more carefully than what Valentin encountered, but it is not immune to analytical errors. The specific failure pattern here, fabricating recommendations from outside the provided dataset while presenting them as data-driven, is something I have not seen Claude do. That distinction matters when you’re using AI to inform a real decision.

The danger isn’t that the AI gets something wrong. It’s that it gets things wrong while sounding exactly like it’s getting them right.

Valentin landed on his own conclusion at the end of their conversation. After catching the Dominus hallucination, he added a note to his transcript: “Why I would NOT trust any LLM-based AI with anything that needs to be precise at this point.” That’s a reasonable reaction. I’d frame it slightly differently: trust, but verify aggressively, and pick your tools carefully.

Conclusion

The wine auction story is entertaining. But strip out the bottles and the bidding strategy, and what you have is a precise illustration of the risks AI introduces into any data-driven analytical process. The AI analyzed the wrong data set, mispriced the assets, invented recommendations, sorted on corrupted fields, and dropped a key variable, all without signaling that anything was wrong. In a business context, those failures translate directly into bad decisions.

AI is genuinely transforming how financial analysis gets done. I use it daily and I believe in its potential. But the use cases where it shines are not the same as the use cases where it fails, and the failure modes aren’t always obvious until the damage is done. The right posture is to treat AI outputs as a strong first draft that requires a trained eye, not a finished analysis that’s ready to act on.

Valentin walked away from that auction with a cleaner bidding strategy by the end of the conversation. He got there by questioning everything the AI told him. That’s the skill set every AI user needs to develop, regardless of what they’re analyzing.

FAQ

Doesn’t the AI get better if you give it clearer instructions?

Partially. Valentin did tighten his prompts as the conversation progressed, and the outputs improved incrementally. But the most significant errors, fabricated lot numbers and the per-bottle pricing assumption, occurred even when his instructions were clear. Prompt quality matters, but it doesn’t eliminate the risk of an AI filling gaps with invented data when it doesn’t have the answer in front of it.

Is this a reason to avoid using AI for analysis entirely?

No. The lesson isn’t to stop using AI. It’s to use it correctly. AI is powerful for pattern recognition, drafting, summarization, and structuring analysis. Where it fails is when users treat its output as ground truth without verification. A well-trained analyst using AI as a drafting partner, while applying their own judgment to the outputs, gets the best of both. The risk comes from removing the human review step.

Why does this matter for business and financial analysis specifically?

Because the consequences of confident wrong answers are higher. In a casual setting, a hallucinated wine recommendation is annoying. In a financial model, a hallucinated revenue figure or a mis-sorted dataset can inform a capital decision, a hiring plan, or an investor presentation. The AI doesn’t know the difference between a low-stakes and high-stakes context. That judgment belongs to the person using it.

Salvatore Tirabassi is the Founder of CFOPro+Analytics, providing fractional CFO services to growth-stage companies. Based in New York, he leverages over 24 years of experience in venture capital and strategic finance to help entrepreneurs master cash flow, unit economics, and equity value creation through data-driven financial clarity.

Assess your CFO needs in 5-minutes