Evaluating a SaaS Management Platform
The demonstrations all look the same. What to test on your own data, and the questions that separate the products from the dashboards.
Checklist · 714 words
Every platform in this category demonstrates well, because the demonstration uses their data and their integrations.
Test it on your estate
A trial connected to your actual identity provider and your actual expense data. Not a sandbox.
Count what it finds against what you know exists. The gap is the product's real discovery coverage.
Check the reconciliation. How many of your applications does it identify correctly, and how much manual mapping is left.
Look at the residue. Unmatched transactions and unidentified applications are where you will spend your time.
The questions
Which discovery sources does it use, and which does it require? A platform that only reads your identity provider is telling you something about its coverage.
Does it read OAuth grants? Many do not, and it is the highest-value source for the security side.
How does it handle reconciliation and aliases? Can you correct it, and does the correction persist.
Can we export everything? The register, the mapping, the history. Test this during the trial, not at renewal.
Does it write anywhere or only read? Deprovisioning and seat reclamation require write access, which is a different security conversation.
What does it do with our data? Where processed, retained how long, used for benchmarking or training.
Can it hold hardware assets too, or is that a second system.
How are renewals and notice dates handled? Frequently a calendar field with no workflow behind it.
What to weight lightly
The dashboard, where demonstrations spend their time.
Benchmarking against other organisations, which is built on data of unknown comparability.
Savings estimates generated by the tool, which assume every dormant seat can be removed.
Integration counts. Two hundred integrations you will not use is not better than the twelve you need working properly.
Automated remediation, which sounds attractive and which most organisations disable within a month.
The cost that is not in the price
Integration effort, which is your time.
Reconciliation and mapping, ongoing.
The ownership and review process, which the platform does not create.
Change management for the request path and the approval flow.
Measure your own hours during the trial and extrapolate. This is usually the largest cost and it never appears in the business case.
Whether you need one at all
Under roughly fifty applications, a spreadsheet and a monthly export routine works and costs nothing. Many organisations buy too early.
The platform earns its place when the reconciliation is too large to do by hand, when several people need to work on the register simultaneously, or when the workflow — requests, approvals, reviews, renewals — is what you are actually buying.
Buy for the workflow, not for the discovery, in most cases. Discovery you can approximate with three exports and an afternoon; the workflow is what decays without a system.
The decision
Buy on: discovery coverage measured on your data, reconciliation quality, export completeness, workflow fit, and data terms.
Not on: dashboards, integration counts, or the savings number the trial produced.
The trial checklist
What to do during a trial so the decision rests on evidence rather than on the demonstration.
Connect it to your real identity provider and real expense data in the first week.
Record what it found and compare against your known application list.
Count the manual mapping you had to do, and extrapolate.
Export everything and open the export. This tests the exit provision while you still have leverage.
Run one real renewal through it, which tests the workflow rather than the dashboard.
Log your own hours, which is the cost that never appears in the business case.
Ask a second person to use it without a demonstration, and watch where they get stuck.
The exit test
The provision that matters most and the only one you can verify while you still have leverage.
Export the full register during the trial — applications, owners, costs, contracts, history.
Export the alias mapping, which is the artefact you cannot rebuild cheaply.
Open the exports in something else and check they are complete and usable.
Ask what happens to your data after termination and how deletion is confirmed.
Ask whether the export is available after the subscription ends, and for how long.
A platform whose export omits the mapping is a platform you cannot leave without repeating the original reconciliation from nothing.