From Flaky LLM Judge to Reliable Check

An LLMJudge with a vague prompt is a test whose verdict is sampled. It passes today, fails tomorrow, and nobody trusts the suite. This tutorial shows the failure first, then climbs a ladder of increasingly cheap and stable checks.

What you’ll build

1. A naive judge run three times over the same fixed output
2. The same requirement expressed as StringMatching / FnCheck
3. The same requirement expressed as SemanticSimilarity
4. A constrained LLMJudge with an explicit rubric, for what is genuinely subjective

Prerequisites

• Completed Your First LLM Call
• Azure OpenAI credentials in AZURE_AI_API_KEY and AZURE_AI_ENDPOINT

from giskard.checks import set_default_generator
from giskard.agents.generators import Generator
from giskard.agents.embeddings import EmbeddingModel

set_default_generator(Generator(model="azure_ai/gpt-4.1-nano"))

# Embedding-based checks need their own model; use the Azure deployment.
embedding_model = EmbeddingModel(model="azure_ai/text-embedding-3-small")

1. Freeze the output under test

To measure judge variance, the system under test must not vary. A hardcoded answer isolates the judge as the only moving part.

ANSWER = (
    "Refunds are processed within 5 business days. "
    "You will get an email once the money is on its way back to your card."
)

QUESTION = "How long does a refund take?"

2. Watch a vague judge disagree with itself

“Is this a good response?” gives the model nothing to measure against, so the verdict follows the sampled reasoning. Run the same scenario three times and compare.

from giskard.checks import LLMJudge, Scenario

naive_judge = LLMJudge(
    name="is_it_good",
    prompt="""
    Is this a good response?

    User: {{ trace.last.inputs }}
    Assistant: {{ trace.last.outputs }}
    """,
)


async def run_with(check):
    scenario = (
        Scenario("refund_policy")
        .interact(inputs=QUESTION, outputs=ANSWER)
        .check(check)
    )
    result = await scenario.run()
    return result


for run in range(3):
    result = await run_with(naive_judge)
    check_result = result.steps[0].results[0]
    print(f"run {run + 1}: {check_result.status} — {check_result.message}")

Output

run 1: CheckStatus.PASS — The response is clear, concise, and provides the necessary information regarding refund processing time and notification. run 2: CheckStatus.PASS — The response provides a clear and concise answer to the user’s question about the refund timeline, including the time frame and an indication that the user will receive an email notification, which adds helpful detail. run 3: CheckStatus.PASS — The response provides a clear and concise answer to the user’s question about the refund timeframe. It includes the processing duration and mentions the notification via email, which adds helpful information.

Even when all three runs agree on the verdict, the reasons differ — which means the criterion is being invented on each call rather than applied. A requirement that the model has to guess is a requirement you have not written down yet.

3. Rung one: deterministic checks

Most “is it good” requirements decompose into facts. If the answer must state the 5-business-day window, that is a substring, not a judgement call: zero cost, zero variance.

from giskard.checks import FnCheck, StringMatching

result = await run_with(
    StringMatching(
        name="states_5_business_days",
        keyword="5 business days",
        text_key="trace.last.outputs",
    )
)
result.print_report()

Output

──────────────────────────────────────────────────── ✅ PASSED ────────────────────────────────────────────────────
states_5_business_days  PASS    
────────────────────────────────────────────────────── Trace ──────────────────────────────────────────────────────
────────────────────────────────────────────────── Interaction 1 ──────────────────────────────────────────────────
Inputs: 'How long does a refund take?'
Outputs: 'Refunds are processed within 5 business days. You will get an email once the money is on its way back to 
your card.'
──────────────────────────────────────────── 1 step in 4ms | runs: 1/1 ────────────────────────────────────────────

FnCheck covers anything expressible as Python — length limits, formats, forbidden phrases, structured-field assertions:

result = await run_with(
    FnCheck(
        name="short_enough",
        fn=lambda trace: len(trace.last.outputs) < 300,
    )
)
result.print_report()

Output

──────────────────────────────────────────────────── ✅ PASSED ────────────────────────────────────────────────────
short_enough    PASS    
────────────────────────────────────────────────────── Trace ──────────────────────────────────────────────────────
────────────────────────────────────────────────── Interaction 1 ──────────────────────────────────────────────────
Inputs: 'How long does a refund take?'
Outputs: 'Refunds are processed within 5 business days. You will get an email once the money is on its way back to 
your card.'
──────────────────────────────────────────── 1 step in 1ms | runs: 1/1 ────────────────────────────────────────────

4. Rung two: semantic similarity

When the wording is free but the meaning is fixed, an embedding comparison buys tolerance to paraphrase while staying deterministic for a given pair of texts. Tune threshold on a handful of known-good and known-bad answers rather than guessing.

from giskard.checks import SemanticSimilarity

result = await run_with(
    SemanticSimilarity(
        embedding_model=embedding_model,
        name="matches_refund_policy",
        reference_text="Refunds take about five business days and you get a confirmation email.",
        threshold=0.6,
    )
)
result.print_report()

Output

──────────────────────────────────────────────────── ✅ PASSED ────────────────────────────────────────────────────
matches_refund_policy   PASS    
────────────────────────────────────────────────────── Trace ──────────────────────────────────────────────────────
────────────────────────────────────────────────── Interaction 1 ──────────────────────────────────────────────────
Inputs: 'How long does a refund take?'
Outputs: 'Refunds are processed within 5 business days. You will get an email once the money is on its way back to 
your card.'
─────────────────────────────────────────── 1 step in 150ms | runs: 1/1 ───────────────────────────────────────────

5. Rung three: a constrained judge

Some requirements really are subjective — tone, hedging, whether an answer overpromises. Keep the judge, but remove its freedom:

• State the criteria as a numbered list, not an adjective
• Say explicitly what makes it fail
• Judge one thing per check, so a failure names the problem
• Feed it only the fields it needs

constrained_judge = LLMJudge(
    name="no_overpromising",
    prompt="""
    You are checking one specific property of a support answer.

    Answer: {{ trace.last.outputs }}

    The answer PASSES if all of these hold:
    1. It states a time frame for the refund.
    2. It does not guarantee a refund will be approved.
    3. It does not claim the refund is instant or immediate.

    The answer FAILS if any of 1-3 is violated.
    Ignore tone, length, and formatting.

    Return 'passed: true' only if the answer passes all three.
    """,
)

for run in range(3):
    result = await run_with(constrained_judge)
    check_result = result.steps[0].results[0]
    print(f"run {run + 1}: {check_result.status} — {check_result.message}")

Output

run 1: CheckStatus.PASS — The answer provides a time frame for the refund (5 business days), does not guarantee that a refund will be approved, and clarifies that the refund is not instant. run 2: CheckStatus.PASS — The answer states a time frame (5 business days), does not guarantee a refund approval, and does not claim the refund is instant or immediate. run 3: CheckStatus.PASS — The answer states a time frame for the refund (5 business days), does not guarantee a refund will be approved, and does not claim the refund is instant or immediate.

The verdicts now agree, and the messages point at a specific numbered rule instead of a mood.

When to use which rung

Requirement	Use
Exact fact, keyword, format, length	StringMatching, Equals, FnCheck
Fixed meaning, free wording	SemanticSimilarity
Answer must follow the provided context	Groundedness
Fixed policy or style rule	Conformity
Genuinely subjective, one property at a time	LLMJudge with an explicit rubric

Rules of thumb:

• Climb only when the rung below cannot express the requirement. Every rung up costs money, latency, and stability.
• Split a multi-part judge into several single-property judges — you lose nothing and gain a failure message that names the cause.
• If a judge still flip-flops, the rubric is ambiguous, not the model.
• Use multiple_runs on a scenario to make a flaky check visible in CI instead of letting it fail randomly.

See When to use which check for the full decision table.

Next step

• Custom Checks — package a rubric as a reusable check
• Test Suites — run the tightened checks together

See also

• Checks reference — LLMJudge prompt templating, SemanticSimilarity parameters
• Core concepts