TemplatesTemplate · Tooling & Stack
Support vendor evaluation scorecard
A weighted 1-5 scorecard for comparing support platforms on the criteria that actually predict a good year — fit, usability, integrations, reporting, support, security, pricing/TCO, and roadmap — with demo questions and red flags.
Set the weights to total 100 before any demo, score each finalist 1-5 per criterion, then tally the weighted total out of 5.0 (and never let a high total hide a dealbreaker).
Use this to compare support platforms (help desk, ticketing, AI/agent-assist, CX suite) on the same terms so the loudest demo doesn't win by default. Score each finalist against weighted criteria, tally a single number, then read the notes — the number starts the argument, it doesn't end it.
How to use this scorecard
- Adjust the weights first, before you see a single demo. Weights encode what your team needs; if reporting doesn't matter to you, drop it and move the points elsewhere. Weights must total 100.
- Give every finalist the same scorer or the same panel. Score right after each demo, while it's fresh.
- Score each criterion 1-5 using the guide below. Compute the weighted score per row: (Weight ÷ 100) × Score.
- Sum the weighted-score column. That total is the vendor's overall rating out of 5.0.
- Do not average away a dealbreaker. A 1 on Security or a hard integration gap can disqualify a vendor no matter how high the total.
Scoring guide (1-5)
| Score | Meaning | What you saw |
|---|---|---|
| 5 | Excellent | Does this out of the box, better than we do today; proven at our scale. |
| 4 | Strong | Solid, minor gaps or light config needed. |
| 3 | Adequate | Works, but with workarounds, add-ons, or caveats. |
| 2 | Weak | Partial, immature, or on the roadmap "soon." |
| 1 | Dealbreaker / absent | Missing, broken, or blocks a must-have workflow. |
Score 1 on any must-have and the vendor is out, regardless of total.
The scorecard
Copy this table once per vendor. Fill Score and Weighted; keep a one-line note per row.
| Criterion | Weight | What "good" looks like | Score (1-5) | Weighted |
|---|---|---|---|---|
| Fit — solves our actual problem | 20 | Handles our real ticket types, volume, and channels without a rebuild | ___ | ___ |
| Usability — day-to-day for reps | 15 | Reps get productive in a shift; few clicks to resolve; low context-switching | ___ | ___ |
| Integrations — fits our stack | 15 | Native, supported connectors to our CRM, telephony, and data — not just an API to build against | ___ | ___ |
| Reporting & analytics | 10 | Answers our KPIs out of the box; export raw data; we own our numbers | ___ | ___ |
| Vendor support & onboarding | 10 | Real implementation help, clear SLAs, a human when it breaks at 2am | ___ | ___ |
| Security & compliance | 10 | SOC 2 Type II, SSO/SCIM, data residency, DPA — evidence, not adjectives | ___ | ___ |
| Pricing / TCO | 12 | Transparent, predictable at 2× our volume; implementation and add-ons included in the math | ___ | ___ |
| Roadmap & viability | 8 | Credible near-term roadmap; funded, growing, not about to be acquired-and-sunset | ___ | ___ |
| Total | 100 | __ / 5.0 |
What to actually check, per criterion
Fit (weight 20)
- Ran a realistic scenario with our ticket types, not the vendor's demo data
- Confirmed it handles our peak volume and all live channels (email, chat, voice, social, in-app)
- Modeling our workflow didn't require a professional-services rebuild
Usability (weight 15)
- A current rep, not a sales engineer, drove part of the demo
- Common actions (reply, reassign, merge, macro) take few clicks
- Agent view surfaces context (history, customer, order) without tab-hopping
- Admin config is self-serve, not a ticket to the vendor
Integrations (weight 15)
- Native connector exists for each system we named — and is supported, not community-maintained
- Data syncs both directions and near-real-time where we need it
- API rate limits and webhook reliability documented and adequate
Reporting & analytics (weight 10)
- Reproduced 2-3 of our current KPIs live in the demo
- Custom reports without a paid analytics add-on
- We can export raw event/ticket data (we own our numbers if we leave)
Vendor support & onboarding (weight 10)
- Named implementation plan, owner, and timeline
- Support SLA in writing; severity tiers defined; after-hours coverage confirmed
- Talked to a reference customer of similar size without the vendor on the call
Security & compliance (weight 10)
- Current SOC 2 Type II (or ISO 27001) report provided
- SSO (SAML/OIDC) and SCIM provisioning included, not upsold
- Data residency, retention, and deletion meet our policy; DPA and subprocessor list reviewed
- If AI features: clear data-use, training-opt-out, and PII-handling terms
Pricing / TCO (weight 12)
- Priced our real seat count and volume, not a starter tier
- Implementation, migration, add-ons, and overage fees itemized
- Modeled cost at 2× volume and at renewal (uplift caps in writing?)
Roadmap & viability (weight 8)
- Roadmap commitments dated, not "on the roadmap"
- Vendor is funded/profitable and not signaling an exit
- Contract has an exit ramp: data export, offboarding help, no auto-renew trap
Demo questions to ask
Ask these in the demo and make them drive the live product, not slides.
- "Load one of our real (redacted) tickets and resolve it end to end."
- "Show me the exact clicks a rep makes to handle our most common request."
- "Build one of our current reports right now, live."
- "Walk through your native integration with [our CRM] — where does it break?"
- "What happens at peak — Black Friday / an outage — to performance and support response?"
- "Show admin config: how do we add a workflow without calling you?"
- "What's genuinely not possible in the product today?"
- "Give me total year-one cost for our numbers, all fees included, in writing."
- "What shipped in the last two quarters, and what slipped?"
- "How do we get all our data out if we leave?"
Red flags
Any one of these is a reason to slow down and dig, not necessarily to walk — but stacked up, they predict a bad year.
- Won't demo with your data or your scenarios ("we'll cover that in the POC")
- The person clicking is always a sales engineer; a real rep never touches it
- Reporting needs a paid add-on to answer basic questions
- "Integration" means "we have an API" — you'd be building the connector
- Pricing is vague, per-resolution/usage without a cap, or "let's talk at renewal"
- Security answers are adjectives, not documents (no SOC 2 report on request)
- Key features are "coming soon" or gated behind a future release
- No named implementation plan or support SLA in writing
- References are hand-picked and only available with the vendor on the call
- Contract auto-renews, caps price increases nowhere, and makes data export hard
- Pressure to sign before the trial ends ("this quarter's discount expires Friday")
Reading the result
- 4.0-5.0 — Strong fit. Confirm security and references, then negotiate.
- 3.0-3.9 — Viable with caveats. List every workaround; price the gaps into the decision.
- Below 3.0 — Walk, unless a single fixable gap is dragging an otherwise strong score.
A high total with a 1 on any must-have is not a pass. Note the dealbreaker next to the total so it survives the meeting where everyone stares at the number.
Continue exploring