Simpli QA
Automated quality assurance scoring for support conversations.
Instead of manually reviewing a handful of conversations each week, QA scores every conversation against your rubrics automatically. Team leads get consistent, unbiased quality data and targeted coaching recommendations.
Simpli QA evaluates support conversations against configurable rubrics, producing quality scores and coaching notes for agents.
Configuration
| Variable | Default | Description |
|---|---|---|
APP_PORT | 8000 | Server port |
LITELLM_MODEL | openai/gpt-5-mini | LLM model for evaluation |
DATABASE_URL | — | Database connection string |
Start the server
simpli-qa serveAPI endpoints
POST /evaluate
Score a conversation against a quality rubric.
Request:
{
"conversation_id": "CV-001",
"agent_id": "A-005",
"messages": [
{"role": "customer", "content": "My order hasn't arrived"},
{"role": "agent", "content": "I'm sorry about that. Let me check the tracking for you."},
{"role": "agent", "content": "It looks like it was delayed in transit. I've initiated a replacement."},
{"role": "customer", "content": "Thank you, that's very helpful!"}
],
"rubric_id": "R-default"
}Response:
{
"conversation_id": "CV-001",
"overall_score": 0.92,
"dimensions": {
"empathy": 0.95,
"resolution": 0.90,
"communication": 0.91
},
"coaching_notes": ["Strong empathy shown", "Quick resolution provided"]
}GET /scorecards/{agent_id}
Get an agent's QA scorecard with aggregate metrics.
Response:
{
"agent_id": "A-005",
"average_score": 0.88,
"total_reviews": 47,
"trend": "improving",
"top_strengths": ["empathy", "product knowledge"],
"improvement_areas": ["follow-up timing"]
}The trend field is one of: improving, stable, or declining.
Rubric management
| Method | Endpoint | Description |
|---|---|---|
GET | /rubrics | List all rubrics |
GET | /rubrics/{id} | Get a specific rubric |
POST | /rubrics | Create a new rubric |
DELETE | /rubrics/{id} | Delete a rubric |
Create a rubric:
{
"name": "Standard QA",
"criteria": [
{"name": "empathy", "description": "Shows understanding of customer emotions", "weight": 0.3},
{"name": "resolution", "description": "Resolves the issue completely", "weight": 0.4},
{"name": "communication", "description": "Clear and professional language", "weight": 0.3}
]
}GET /health
Health check.
Next steps
- Quality Improvement Loop — Build a continuous improvement cycle with QA, Sentiment, and Pulse
- The Ticket Lifecycle — See how QA fits into the full service pipeline
- Integration Overview — Connect QA scoring to your helpdesk
- Model Selection — Choose the right model for evaluation