wobbly

Overview

Metamorphic testing for AI outputs. It finds wrong LLM answers without an answer key by checking consistency across related inputs. Written in Python.

Best for

Best for
Developers testing LLM outputs without labeled data or answer keys

Use cases

• Detect inconsistent LLM responses across paraphrased prompts
• Validate AI outputs when no ground truth labels exist
• Test model behavior under input transformations

Notes

Metamorphic testing for AI outputs. It finds wrong LLM answers without an answer key by checking consistency across related inputs. Written in Python.

6 stars on GitHub. Last updated 2026-10-03. Licensed MIT.

Use cases

• Detect inconsistent LLM responses across paraphrased prompts
• Validate AI outputs when no ground truth labels exist
• Test model behavior under input transformations

Pros

• No answer key or labeled data required
• Catches subtle logical inconsistencies
• Open source with a simple Python interface

Cons

• Very low community adoption (6 stars)
• Limited documentation and examples
• Python only, no language bindings

Indexed from awesome-llmops and enriched against its public facts.