Yes, folks – the rumors are true. HFS is officially out of the quadrant business.
We’re done, the whole quadrant craze is starting to smell pretty bad and we know the industry is fed up with it. Increasingly, many of these 2×2 matrices are missing several of the market leaders (who refuse to participate) and having them all stacked in the top right just smacks of pay-for-play (even if the analyst has fair intentions). Let’s be honest, noone trusts these matrices and they are harming the entire credibility of the analyst industry. Sure, there are many honest, quality analysts with integrity, but their craft is being soiled by several quacks who are basing their vendor placements purely on vendor briefings, whether they like a particular vendor, and whether some vendors pony up for their research services. There are many “analysts” out there who do not bother to do sufficient customer research and we all suspect who these characters (and their employers) are…
If we don’t change, we all – as analysts – might as well admit we’re no longer in the research business: we’re in the vendor PR business. Yes, it’s that bad… and let’s stop sugar coating it.
Enterprise executives tell us all the time they get zero value from these grids – they are purely for vendor marketing sales decks (and I talk to a helluva lot of these enterprise folks). However, enterprises desperately need to be informed on vendor performance – they just need a direct ranking that’s relevant for their needs, where a credible analyst puts a stake in the ground. That’s what everyone has told me, so that is what we are delivering: The HFS TOP 10.
Quadrants, Peaks, NEATS and Waves – and sadly Blueprints – are all sales tools for vendors as opposed to decision support tools for enterprise customers. At HFS, we are not in that business – we are in the research business to support informed enterprise decisions. At HFS, we are not ending our involvement in covering the hottest markets in the industry and producing the best competitive analyses, we are merely making our research more relevant, more timely and more impactful with the HFS TOP 10 and much more simplified to support the enterprise customer. What’s more, when some firms take six to nine months to get a quadrant to market, that market has often already moved on, and the data, despite its credibility, may already be stale. We are in a world that doesn’t stand still, where enterprise customers are thirsty for timely, credible data that clearly shows the winners, contenders and laggards in a given market.
Customers want rankings where the analyst took a stand, not merely a fuzzy matrix where everyone looks like a winner. Here is an example of how the HFS TOP 10 ranking looks (the RPA Products in 2018), and here you can download a full report example to see for yourself how we get to the point, how we inform decisions and we clearly profile where vendors are strong – and where they face challenges.
HFS TOP 10 reports remove the unhealthy involvement of vendors from the analyst evaluation process and are much more timely, relevant and less cumbersome to produce
The main difference with the HFS TOP 10 is the fact we’re running them purely on desk research, support from our research academy and from our vast repository of current user data. We are eliminating the whole laborious vendor lobbying and briefing processes so we can get these reports out the door faster than ever before, without being tied to vendors schedules and relying on references they provide. This does not mean analysts cannot do vendor briefings to support their research (if the analyst deems it necessary, or if the vendor requests a timely briefing), it just means we do the research in a timeframe that can’t be moved. It means vendors cannot complain that we “did not do reference calls with their customers” or give them a chance to be adequately represented in the market. Because HFS already has the data! We have reams of data on service vendor performances, or vertical markets, on RPA products, on blockchain platforms, on analytics firms, on FinTechs etc. And where we may occasionally not have sufficient customer data in a niche market, we will invest in gathering it using the HFS network. Yes, we actually set aside funds for user surveys where most of our competitors only perform custom research when their customers are funding it.
Here are some FAQs you probably want answering:
1. How is the methodology of the HFS Top 10 different from the Blueprint? There are several key differences in methodology:
a. We are Ranking vendors, not Gridding them. The HFS Top 10 is presented as a simple and clear ranking of assessed products / service vendors versus the 2X2 Blueprint grid
b. Voice of the Customer, execution success, and innovation capability. The HFS TOP 10 methodology is driven by customer experience with products / services (voice of the customer) in addition to vendor’s ability to execute and innovate.
c. Powered by HFS G2000 network. The primary source of data for the HFS TOP 10 reports is HFS’ extensive network of G2000 enterprise customers. HFS will gather information via surveys, analyst interviews, and ongoing dialog with customers versus relying on data inputs from service vendors. HFS conducts over 5,000 interviews a year with enterprise customers right across the six change agent areas of our research coverage: RPA, AI, Smart Analytics, Global Sourcing, Blockchain and Digital Business Models.
d. Not reliant on vendor RFI responses. The Top HFS TOP 10 report methodology does not rely on the use of old-school traditional approaches of collecting data through vendor RFIs. We welcome vendors to augment our analysts’ knowledge base through structured briefings, demos, and reference customers, but this not a necessary component in the process. We will not allow vendors to slow-down our research processes.
e. No opt-out. There is no opt-out for leading vendors given HFS is relying 100% on its own network and data sets. We never product vendor landscapes where half the leading players are absent.
2. Will there still be fact checks with the vendors?
Yes, vendor profiles, including strengths and development opportunities will be sent for fact-checks. However, rankings will not be shared in these fact-checks. An embargoed HFS TOP 10 will be released one-day prior to the actual release of the report, intended to be an FYI versus any negotiation on ranking etc. We are not in the lobbying business, we are in the research business.
3. What data will populate the HFS TOP 10 reports?
The data will be populated from multiple sources of information:
– The primary source of data for the HFS TOP 10 reports is HFS’ extensive network of G2000 enterprise customers. HFS gathers this information via surveys, analyst interviews, HFS roundtables and summits, and ongoing dialog with enterprise customers, versus relying on data inputs purely from service vendors.
– Providers can augment our analysts’ knowledge base through structured briefings, demos, and reference customers.
– Note that we will minimize the use of old-school traditional approaches of collecting data through vendor RFIs (unless covering a nascent / emerging market where most of the solutions are still in beta mode).
4. What is the minimum customer data-set needed to be able to guarantee a voice of the customer? What happens, if for whatever reason, there is not enough customer data?
A statistically significant sample set is 30 datapoints for a report across reference checks, our existing data sources, and our own customer conversations. While most of our current research has a significantly higher sample set than 30 there is rarely a lack of available data to use to source the rankings. Where a lack of customer data does occur, it may result in delays of the research publication as we make extra efforts to source customer data.
5. What can vendors do to maximize customer data access?
Real value usually comes through engaging with HFS analysts throughout the year by providing HFS analysts the opportunity to speak with more of their customers, sharing and collaborating on customer stories. As mentioned, we make it our business to do our own customer research – that is our purpose in the industry, but those vendors who can persuade many of their customers to showcase their experiences will benefit.
(Cross-posted @ Horses for Sources)