Tool
promptfoo
Open-source CLI for prompt evals, regressions, and red-teaming
About
CI-native prompt testing used widely in LLM eng
Features
- Prompt regression tests in CI
- Pricing
- Open Source
- Pricing detail
- MIT free; Team collab plan
- Platforms
- CLI / CI
Alternatives from our catalog
Tools on the same shelves as promptfoo, ordered by shared tags and our own comparisons. Not vendor endorsements.
- AgentaOpen Source · Web / Self-host
OSS playground for comparing prompt variants
same tags
- DSPyOpen Source · Python / CLI
Research-favorite automatic prompt optimization
same tags
- LangfuseOpen Source · Web / Self-host
Self-host favorite for traces + prompt versions
same tags
- AIPRMFreemium · Browser extension
Power-user ChatGPT prompt sidebar staple
same tags
- LangSmithFreemium · Web + SDK
Default observability for LangChain stacks
same tags
- Replicate PlaygroundUsage-based · Web + API
Easiest hosted UI for community ML model prompts
same tags