Can we trust the analysts?
26 FEBRUARY, 2014 BY IJCOX 7 COMMENTS
Every year IT industry analysts launch their themes, trends and predictions for the coming twelve months. These are the things that will shape much of their research, events and sales activity over the course of the next year. It is an important time for everyone involved in IT; IT professionals, vendors, consultants and journalists will all be impacted in some way by the announcements, press releases and survey findings released by Gartner, Forrester et al.
But is that necessarily a good thing? Analyst firms’ primary objective is to sell more reports, subscriptions, advisory services and places at events to delegates and sponsors. And the more they sell, the better. They are selling products and services and, just like any other organisation with something to sell, they need to make a compelling case for people to buy from them. They also need to create newsworthy headlines and sound bites to make it as easy as possible for the media to give their products and services the free publicity they need to maximise sales.
In a previous article, The CDO: a self-fulfilling prophecy?, I discussed whether the rise of the Chief Digital Officer (CDO) was at least in part caused by Gartner focusing on the role as one of theirthemes and therefore producing a number of products related to the CDO role. The Gartner headline that has been quoted heavily by the media, bloggers and industry commentators was that 25% of companies will have a CDO by 2015. My argument was that, given Gartner’s reputation and influence on executives across the world, its prediction about the CDO role could become a self-fulfilling prophecy. Interestingly Gartner are now predicting that the CDO role will be temporary and that by 2020 it will probably disappear altogether. Sounds a bit like retiring a product that has reached the end of its profitable life. I wonder what its replacement will be?
The prompt for revisiting the subject of analyst predictions though is the ever popular Gartner claim that CMOs will be spending more on IT than CIOs by 2017, which I saw once again in an article just a few days ago. This forecast was first made at the beginning of 2012 and is still being used by various parties today, over two years later. The return for Gartner on that prediction must be phenomenal; excited CMOs, concerned CIOs and interested CEOs, COOs, CFOs, etc, have no doubt purchased reports, attended conferences and hired Gartner analysts to learn about how to deal with this momentous change. And that’s not to mention the endless articles that have quoted the statistic along with Gartner’s name to give the company huge amounts of free publicity. It has also caused a frenzy of activity within the vendor community as technology providers rushed to shift their sales and marketing efforts to target the marketing function.
That one prediction has had a massive impact across the IT industry and beyond. Yet, it is not entirely correct. At best it is a misleading statistic that needs heavy qualification yet Gartner has been strangely silent on the subject. As Matt Ballantine noted in his blog last year, a link to awebinar on Gartner’s website appears to be the only formal reference to the prediction that Gartner has made. I wonder why that is?
In an interview with CXOTalk in June 2013 Mark P McDonald, who at the time was a Group Vice President and Gartner Fellow (he is now listed on the Gartner website as a former GVP and fellow following his move to Accenture), was asked to talk about that forecast. In a faulting reply he made the distinction between IT spend and technology spend, adding that if you included marketing expenditure on items such as Google AdWords within the definition of technology spend then yes, the CMO will spend more on technology than the CIO. He then went on to blame everyone else for misinterpreting his firm’s prediction and using it to their own advantage! It is worth noting, however, that the title of the webinar on Gartner’s website is “By 2017 the CMO will spend more on IT than the CIO”. I thought it was technology spend not IT spend? And the bullets below the title state that the prediction is specific to high-tech providers only. But that is not what McDonald said in his interview. Confused?
Given the lack of supporting data and clarification provided by the company I would say that if anyone has been guilty of knowingly misusing the prediction for its own benefit then it is Gartner itself. With the clarification from McDonald the prediction actually becomes pretty meaningless and certainly not one that should have received the attention, and caused the level of reaction, that it has.
All of which prompts the question whether we can trust analysts. Given that one of Gartner’s most successful (and quite possibly profitable) statistics of recent years is rendered meaningless when explained, why should we believe any of the other headline making, attention grabbing, revenue generating sound bites issued by analysts? How can we tell when a prediction is telling the full story? The lesson from Gartner’s CMO spending theme appears to be that, if there is no data or analysis to support what an analyst is saying, then it is probably best to treat the claim with a healthy dose of scepticism.
However, analysts do have their place and their uses; they serve a valuable purpose in helping to cut through the hype and noise in the IT industry. As a CIO myself I have used Gartner quadrants, Forrester waves, etc to research and understand the market for IT products and services. And many of the survey- and research-backed reports produced by analysts are a valuable source of insight and information on trends, activity and developments within the industry. But when the analysts themselves are contributing to the hype and noise their impartiality, credibility and value is called into question.
So can we trust the analysts? My advice is not to trust their headline predictions without first understanding how they were created. Always look behind the sound bite to find out where it came from and what data or research has been used to make the forecast. And, if there is little or no substance behind the headline, then then it may be just another revenue generating theme.