Vendor evaluation
Skill Amey-Thakur/AI-SKILLS/skills/big-tech-processes/vendor-evaluation
Plug-and-play skills and prompts for every AI coding agent
npx -y skills add Amey-Thakur/AI-SKILLS --skill vendor-evaluationAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
2 things to look at
- 19 days oldThe repository was created 19 days ago. New is not bad, but a brand new repository carrying a familiar-sounding name is the shape a typosquat arrives in, and there has been no time for anyone else to find a problem with it.
- 4 stars4 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Evaluate vendors against weighted criteria, a proof of concept that tests your real workload, and a clear-eyed accounting of exit costs before you sign. Use when choosing a paid tool or service you will depend on and a wrong pick is expensive to undo.
SKILL.md
3.1 KB, as published. Nobody here has run it
Vendor evaluation
Choosing a vendor is choosing a dependency you will live with for years and pay to leave. The evaluation goes wrong when it is run on demos and vibes: the polished sales walkthrough decides it, criteria get invented to justify the favorite, and no one asks what it costs to get back out. A disciplined evaluation tests the vendor on your work and prices the exit before it prices the entry.
Method
- Set weighted criteria before you see a single demo. List what matters: functionality, reliability, security posture, support, total cost, and how hard it is to leave. Assign weights that sum to a fixed total, and write them down first. Criteria invented after the demo just rationalize the vendor who demoed best.
- Score against requirements, not against each other. Rate every vendor on each criterion against your bar, so a weak field cannot make a mediocre option look strong by comparison. Separate must-haves, which are pass or fail, from nice-to-haves that earn weighted points.
- Design the proof of concept around your real workload. Feed it your actual data shapes, your peak volume, your awkward edge cases, and your integration points. A POC on the vendor's sample data proves their demo works. Define the success metrics and the time box before you start so the POC ends in a verdict, not an open-ended trial.
- Cost the exit, not just the entry. Estimate what it takes to migrate off: data export formats, proprietary interfaces you would rewrite, retraining, and contract lock-in. High switching cost is a real price even if it never appears on an invoice. Prefer vendors whose data you can get back in a usable form.
- Verify security and compliance with evidence, not the questionnaire alone. Ask for the SOC 2 or ISO report, the data processing terms, subprocessor list, breach history, and where data is stored. A vendor who cannot produce these for a serious deal is telling you something.
- Reference-check past the list they gave you. Talk to the references, then find a customer they did not hand you and ask what breaks at month six: support response times, hidden costs, the feature that was "on the roadmap." Sales sells the roadmap; a real customer tells you the product.
Checks
- Were the weighted criteria fixed before demos, or shaped to fit a favorite?
- Did the POC run on your workload and data, or on the vendor's canned scenario?
- Can you state, in dollars and weeks, what leaving this vendor would cost?
Boundaries
This picks the vendor; it does not negotiate the contract terms or run procurement and legal review, which have their own owners and thresholds. Approval limits and required security sign-offs are company policy: route spend and data agreements through the process your organization requires rather than deciding them here.