Found it or answered it?

Sort each failure: was it retrieval or generation?

The support assistant got six answers wrong. Find where it failed.

Before blaming the model, look at what the search delivered. ⚗️

The right chunk never showed up in the results.

The right chunk came, but the answer cites another value.

The search returned only chunks about another product.

The answer has a date that no chunk mentions.

The new manual was never indexed.

The right chunk came, but the model said “I don't know.”