How the age of Big Data made statistics the hottest job around
(Illustration by Raymond Biesinger)
Two years ago, Ritchie Bros. Auctioneers faced an interesting dilemma: The Vancouver-based company, which organizes auctions for industrial equipment, was accumulating massive amounts of information on its customers and the items it was listing for sale, but it had no one on staff who could really dive deep and make sense of it all. The company wanted to take a more data-driven approach to pricing, because it needed a better idea of what the market would pay for equipment. When structuring a deal for a used oilfield drill for an upcoming auction in Texas, for example, staff look at what similar equipment sold for in that area in the recent past and use that as a guide. But final sale prices depend on other factors, too. The quality of used equipment can vary wildly, and the resource sector that often buys these products is volatile, meaning demand can fluctuate without warning. In order to develop a more accurate, responsive pricing model, Ritchie Bros. needed help. “The problems we were having were too complex for the skills we had,” says Jeremy Coughlin, director of business intelligence.
So Coughlin went on a hunt for data scientists: experts who can derive insights from large, complex data sets. He checked LinkedIn, scoured online forums and attended industry networking events. He experienced first-hand something that a growing number of Canadian companies are now learning: Data experts are in short supply. Many businesses are struggling to find talent, even as more people enter the field. The number of data professionals in Canada—people employed as statisticians, mathematicians and actuaries—has increased by 48% over the past five years, making it the fastest-growing job category in the country. And demand isn’t letting up. A survey conducted last year by IDC found that 53% of large Canadian organizations said lack of talent was the biggest impediment to successful completion of big data projects.