IA en derecho: menos plausibilidad, más verificabilidad (Full Transcript)

[00:00:02] Speaker 1: Hey everyone, welcome back to another episode of Now Listen. Today, we're talking about legal AI. Not the flashy demos, not the, wow, it wrote a memo stuff.

[00:00:12] Speaker 2: With the pyramid, you have the connection to everything in time and space.

[00:00:25] Speaker 1: We're talking about the kind of AI that can quietly ruin a case if it gets even one thing wrong. Let's get into it. In early 2025, a man in California was held without bail on gun possession charges. Prosecutors filed an 11-page brief opposing his release. Later, his attorneys flagged something disturbing. The brief included fabricated quotations, misread law, even misstated constitutional provisions.

[00:00:56] Speaker 3: Mr. Madison, what you've just said is one of the most insanely idiotic things I have ever heard. At no point in your rambling, incoherent response were you even close to anything that could be considered a rational thought.

[00:01:12] Speaker 1: This wasn't a typo. It had all the fingerprints of generative AI, and it wasn't an isolated incident. By the end of 2025, courts were sanctioning AI-fabricated citations, and legal scholars were warning that unchecked AI could cause real due process harm. This wasn't a bad prompt problem. It's an architecture problem.

[00:01:35] Speaker 4: Okay, a simple wrong would have done just fine.

[00:01:37] Speaker 1: Now, listen. Generative AI is really good at sounding confident. That's literally what it's designed to do. But when you ask a generative model to support an argument, it tries to be helpful by producing something that looks authoritative. Even if that authority doesn't exist, or worse, says the opposite. Now, that's fine if you're writing a poem.

[00:02:02] Speaker 5: Bop boop beep, bop bop boop bop. That's for Cynthia. He's dead.

[00:02:11] Speaker 1: But in law, that's how people lose trust, or even their freedom. At the same time, legal AI adoption is exploding. Many firms are rolling out tools built on large language models, often wrapped in legal-friendly interfaces. And to be clear, these tools can be genuinely useful, but most of them share the same underlying issue. They're still generative-first systems. Even when they retrieve documents or reference internal knowledge, they're often allowed to free-generate around that information, which means hallucinations are still possible. And that's not a vendor problem. That's an architectural one. This is where the distinction really matters. Generative AI predicts what sounds right based on patterns. Closed-loop AI works differently. In a closed-loop system, the AI is constrained by design. It can only operate on verified source material that you provide. Transcripts, exhibits, interviews, case records. It can summarize, extract, organize, connect. It's designed to reduce fabricated citations, invented quotes, drifting beyond the record. Now listen, this doesn't mean it's perfect. But architecturally, hallucination risk is materially reduced because the system isn't allowed to scrape the internet and guests in the first place. This difference matters in real legal work. Legal teams are overwhelmed by default, huge evidence volumes, tight deadlines, high stakes. Generative systems can produce a clean, confident story of what a witness probably meant, even when the record doesn't fully support it. Closed-loop systems behave differently. When support is thin, they show uncertainty. When testimony conflicts, they surface it. And when evidence exists, they point you directly back to it. That's how hallucination risk gets reduced, by designing systems that can't wander off into the uncontrolled ether.

[00:04:30] Speaker 6: So we backtracked a tad. A tad.

[00:04:33] Speaker 1: A tad.

[00:04:34] Speaker 6: A tad, Lloyd. You drove almost a sixth of the way across the country in the wrong direction.

[00:04:43] Speaker 1: This creates a different trust foundation for legal work. First, verifiability. Every output ties back to a source. Second, auditability. A clear trail from result to record. Third, defensibility. You can show receipts, not just confidence. This isn't about banning AI. It's about using AI that respects how legal work actually functions. Every hallucinated citation erodes trust. Not just in AI, but in the legal system itself. The problem isn't artificial intelligence, it's AI optimized for plausibility instead of verifiability. And in law, plausibility just isn't enough. If this helped clarify what actually matters in legal AI, like, subscribe, and let me know what questions you still have in the comments below. And check out our deep dive on closed-loop AI linked here. I'll see you all in the next one.