EP 389 Is ChatGPT Making Investors Lose Money?

APPLE PODCASTS | SPOTIFY

If you’re a real estate investor and you’ve started uploading offering memorandums, rent rolls, and T-12s into ChatGPT and asking:

“Is this a good deal?”

We need to talk.

Because AI can analyze in minutes what used to take an investment team hours.

It can summarize a 100-page offering memorandum.

Find trends in a rent roll.

Compare expenses.

Calculate returns.

Write an investment memo that looks like it came straight out of an institutional investment committee.

And that is exactly why it can be dangerous.

Because the more professional the answer looks, the easier it is to forget one very important thing:

AI can make something up and sound completely confident while doing it.

When you’re asking it to write an Instagram caption, that’s annoying.

When you’re deciding whether to wire hundreds of thousands—or millions—of dollars into a real estate deal?

That’s a very different problem.

One invented expense changes NOI.

NOI changes valuation.

Valuation changes what you’re willing to pay.

And suddenly one tiny AI hallucination has worked its way all the way into an investment decision.

So we want to show you what we believe the solution actually is.

And surprisingly?

It isn’t a better prompt.

AI Is Basically Autocomplete With Confidence

We know.

That’s a lot less exciting than the way AI gets described online.

But stay with us.

Think about autocomplete on your phone.

You type:

“On my way…”

And your phone suggests:

“home.”

Your phone didn’t check your GPS.

It didn’t look at your calendar.

It didn’t call your sister and ask where you’re headed.

It simply recognized a pattern.

Those words frequently appear together.

Large language models are obviously far more sophisticated, but the underlying concept is important to understand: they’re extraordinarily good at generating plausible language based on patterns.

And plausible isn’t the same thing as true.

We love this line from the original explanation:

Fluency is not understanding. It’s pattern completion that learned how to wear a suit.

That distinction matters enormously when you start using AI for real estate investing.

Because if you upload a rent roll and tell AI:

“Underwrite this.”

You may think it’s behaving like an analyst opening a file cabinet.

But it can also generate something that simply looks like underwriting.

And those are two very different things.

Here’s Where It Gets Expensive

Imagine AI sees a multifamily deal.

It knows that multifamily properties often have management fees.

It has encountered management fees thousands and thousands of times.

So it produces one.

Maybe 4%.

Maybe 5%.

Looks normal.

Sounds normal.

You keep reading.

Except…

That management fee wasn’t actually in your document.

AI filled in something plausible.

That’s a hallucination.

And the scariest hallucinations aren’t ridiculous.

They’re reasonable.

If AI told you your 200-unit apartment complex had a monthly unicorn-maintenance expense, you’d catch it.

But a 4.5% management fee?

That looks like it belongs there.

And that’s the problem.

Confident and correct are different hobbies.

“But We Have a Really Good Prompt”

This is usually where someone says:

“Okay, but our prompt is really detailed.”

Great prompts absolutely matter.

But they don’t fundamentally eliminate the problem.

A better prompt can tell the model which direction to go.

It doesn’t necessarily turn probabilistic generation into deterministic arithmetic.

And neither does giving AI more files.

We’ve heard this described as building a “second brain.”

Upload your SOPs.

Upload old deals.

Add your notes.

Build a custom GPT.

Connect everything.

Suddenly it feels like Jarvis showed up for work Monday morning.

Except…

Jarvis did not show up.

You built a better filing system.

And that’s useful!

But labeling every jar in the pantry doesn’t magically hire an investment partner who shares your judgment.

The model still has to generate the connections between the information you’ve given it.

And generation is exactly where drift can sneak in.

Files can give AI context. They don’t automatically give it judgment.

So we started thinking about the problem differently.

What If the Solution Isn’t a Better Prompt?

Instead of asking:

How do we get AI to hallucinate less?

We started asking:

How do we build the workflow so AI isn’t allowed to invent the numbers that matter?

That’s a very different question.

And it led us to what we call:

Closed-Loop Judgment

There are three pieces.

1. Lock the math.

2. Establish the rules of engagement.

3. Keep the investment decision human.

Simple.

But the difference it creates is enormous.

Lock #1: The Calculator Doesn’t Get Creative

Some things in real estate investing require judgment.

Math isn’t one of them.

If payroll on the T-12 is $317,426, we don’t want AI deciding that similar properties usually spend more and “helpfully” adjusting it.

We want the number in the document.

If NOI needs to be calculated, we want the same formula to calculate it every time.

If occupancy is calculated from a rent roll, we want the actual calculation.

Not the number AI thinks sounds reasonable.

That’s what we mean by deterministic math.

Same inputs.

Same formula.

Same answer.

Tomorrow.

Next week.

For us.

For our analyst.

For whoever reviews the deal next.

Because if the AI simply typed a number into existence?

That’s not underwriting. That’s autocomplete.

Lock #2: Give AI Rules, Not Just Information

Next comes the playbook.

This is different from uploading a folder full of notes.

We’re talking about explicit rules governing how AI is allowed to behave.

For example:

Only use information contained in the documents or calculated directly from those documents.

Never invent a fee.

Never invent vacancy.

Never invent CapEx.

Clearly label assumptions as assumptions.

Anything that materially changes a go/no-go decision goes back to a human.

Imagine AI sees a property without a stated vacancy assumption.

Maybe 5% vacancy is common for similar deals.

But we don’t want AI quietly inserting 5% and presenting it as though the seller gave us that number.

Instead, the system needs to say:

This is an assumption.

That’s a very small distinction with very large consequences.

Notes store information. A playbook establishes judgment boundaries.

Lock #3: We Still Make the Decision

This might be the most important one.

AI can prepare the file.

AI can surface anomalies.

AI can tell us insurance increased 40% year over year.

It can identify unusual repairs and maintenance.

It can compare the rent roll with the OM.

It can show us where historical expenses don’t match underwriting assumptions.

Wonderful.

Then we decide what those things mean.

Do we change reserves?

Do we adjust our price?

Do we call the broker?

Do we renegotiate?

Do we walk?

AI doesn’t wire the money.

And we don’t want it making the final investment decision either.

AI gives us speed.

It does not get our authority.

So We Tested Both Approaches on the Same Deal

This is where things got interesting.

We took an anonymized 96-unit multifamily property in the Phoenix metro.

Nothing exotic.

An offering memorandum.

A trailing-12-month income statement.

Basically the kind of package investors look at every day.

Then we analyzed it two different ways.

Path A: Open Chat

We uploaded the package into a general AI model and asked it to underwrite the property.

Income.

Expenses.

Risks.

Flags.

The response sounded great.

Then we found the freestyle.

4.5% management fee.

Seems reasonable, right?

There’s just one problem.

The offering memorandum never stated a management-fee percentage.

There was an expense total.

There wasn’t a rate.

But “multifamily,” “management fee,” and “4–5%” commonly appear together.

So AI generated something that fit.

And if you were moving quickly, it would’ve been incredibly easy to assume that number came from the documents.

It didn’t.

And remember what happens next.

That one expense affects NOI.

NOI affects valuation.

Valuation affects your offer.

A tiny hallucination can travel surprisingly far.

Then We Ran the Exact Same Deal Through a Closed System

Same property.

Same documents.

Different architecture.

Inside our AIREI OS, the first thing we want to know isn’t:

“What does AI think?”

It’s:

“Where did this number come from?”

The system reads the documents and extracts fields with citations back to the source.

Page.

Section.

Line.

If a number isn’t actually in the document, it doesn’t get to masquerade as extracted data.

That’s an important distinction.

Because now there’s a wall between:

Sourced

and

Generated.

Then the calculator takes over.

Year-one NOI.

Effective gross income.

Expense build.

Debt service.

IRR.

Those are calculations.

The language model doesn’t get to “write” them.

Same inputs tomorrow?

Same output.

And that changes AI’s job completely.

We Don’t Want AI Doing the Math. We Want It Explaining the Math.

This is where we think language models are incredibly useful.

Once the math produces a finding, AI can help explain it.

Maybe the system determines that this property’s expenses are significantly below what we’ve historically experienced in that market.

That’s a finding.

Now AI can translate it:

“The property’s reported operating expenses are materially below the firm’s historical experience for comparable assets in this market and should be investigated before underwriting them as sustainable.”

Great.

That’s exactly where we want AI.

The system found the issue.

AI explained it.

The language model isn’t inventing the finding and then explaining its own invention.

That distinction is everything.

And Assumptions Should Look Like Assumptions

Let’s say the OM doesn’t provide something we need.

Fine.

Sometimes underwriting requires assumptions.

The problem isn’t assumptions.

The problem is when assumptions quietly become facts.

So if we’re using a standard vacancy assumption because the seller didn’t provide one, we want it visibly labeled.

ASSUMPTION.

Not buried inside a paragraph.

Not blended into extracted information.

Not magically turned into something the seller supposedly said.

The same applies to investment criteria.

Market.

Vintage.

Unit count.

Basis per unit.

Those rules shouldn’t change because AI suddenly became enthusiastic about a deal.

That’s what closed-loop means to us:

Same inputs. Same math. Same boundaries.

But There’s Still One Thing the System Can’t Do

Eventually, somebody has to decide:

Pursue or pass?

That’s us.

Inside our workflow, the human decision gets recorded.

If we pass, we record why.

Who made the decision.

When.

What mattered.

AI can make the preparation dramatically faster.

But it can’t own the decision.

That’s the line we don’t want to blur.

Software can speed the path. It cannot own the wire.

“Okay, We’ll Just Verify Every Number”

We hear you.

Because that was our first instinct too.

We’ll just check everything AI produces.

And maybe you will.

On deal one.

And deal two.

But what happens after you’ve reviewed ten deals?

Or you’re tired?

Or you’re moving quickly?

Or the hallucination isn’t sitting in a neat little table with a red flashing sign saying:

HELLO! I MADE THIS UP!

What if it’s buried in paragraph six of a beautifully written analysis that sounds exactly like something an experienced underwriter would say?

That’s why we don’t want our system dependent on us maintaining perfect vigilance forever.

We’d rather design the system so the calculator can’t freestyle in the first place.

A better prompt can steer the model.

More files can give it context.

A custom GPT can improve the experience.

But none of those things, by themselves, lock the math.

If your workflow can’t survive “the model sounded sure,” it isn’t closed-loop yet.

The Question We Think Every Real Estate Investor Should Ask About AI

Not:

“What’s the best prompt?”

Not:

“Which AI model should we use?”

Not even:

“How can we make AI smarter?”

Ask:

Where is AI allowed to generate—and where must the answer be deterministic?

Because those are very different jobs.

Let AI summarize.

Let it explain.

Let it organize.

Let it help you investigate.

Let it help surface questions you might not have thought to ask.

But when the number determines NOI…

When NOI determines value…

And value determines whether you send the wire…

Don’t let autocomplete do the arithmetic.

That’s the shift.

AI can be extraordinarily useful without pretending it’s Jarvis.

We just need to give it the right job.

The Future Isn’t Better Prompts. It’s Better Systems.

We think a lot of real estate investors are currently in the first phase of AI.

We’re learning prompts.

Building custom GPTs.

Creating second brains.

Uploading documents.

And that’s useful.

But we think the next phase is much more interesting.

It’s building systems where AI has boundaries.

The calculator is locked.

The investment playbook is explicit.

Assumptions are labeled.

Sources are traceable.

And the human still owns the call.

Because at the end of the day:

Plausible doesn’t mean sourced.

Fluent doesn’t mean correct.

And AI sounding like an underwriter doesn’t make it one.

The goal isn’t to eliminate AI from your investment process.

It’s to build an investment process where AI can make you faster without quietly changing what you believe to be true.

That’s closed-loop judgment.

And for us, that’s where AI in real estate starts getting really interesting.

Want to See the Full Workflow?

If you’re already using AI to underwrite real estate deals—or you’re thinking about starting—we put together a free underwriting roadmap that shows you the process we use to keep AI from inventing the numbers that matter.

And if you want to see us walk through the full process—from a deal package all the way to human judgment—we also teach the workflow inside our free AI real estate investing masterclass.

Because you probably don’t need another list of clever prompts.

You need a system you can trust when real money is on the line.