Exploring And Managing The Data Scientist Skill Gap

0Anil Kaul AbsolutdataLeveraging the promise of Big Data is a challenge for retailers as they balance the need to streamline processes and reduce costs while investing in tech and staff members capable of working with the data. Information is often a retailer’s most important asset, and utilizing it properly can be the difference-maker between success and failure.

To fulfill the promise of Big Data there is now the role of “data scientist,” a person who uses their technical and abstract thinking skills to look at data in new and impactful ways. Given the explosive interest in analysis of large interconnected data sets, it’s not a surprise that data scientists are in heavy demand. The demand is so great that it has developed a skill gap, with a McKinsey study estimating a 50% to 60% gap between supply and demand for analytic talent by 2018.

Glassdoor information ranked Data Scientist as one of the best jobs in 2015, a ranking that is expected to continue for years. Higher education institutions often react quickly to such situations, and they are developing data management programs to allow students to study algorithms and other techniques to spot trends in the data. Students are attracted to the field for the promise of high pay and benefits, and the opportunity to work with retailers and a broad spectrum of industries.


Retailers need to accept the current skill gap and be willing to pay for top talent, which can bring considerable rewards:

  • Insights from data allow companies to adjust their strategies or product mix in real time, so they can stay ahead of trends and retain market share;

  • Retailers’ marketing departments need data scientists to improve cost-per-acquisition and other similar metrics. Data scientists can segment markets or prospects for maximum gain, and uncover previously hidden correlations that can transform marketing tactics; and

  • Predictive modeling is possible when data scientists utilize data from multiple sources. They can, for example, provide finance with solid estimates on sales results, recognize shifts in audience tastes, and better predict seasonal and geographic trends.

Retailers must properly manage their information, and should follow several best practices and consider their options in order to get the most out of their scientists and information:

  • Recruit The Very Best Data Scientists

Using an “art and science” approach, the top data scientists utilize sound technical skills with an ability to see connections and trends. Retailers need data scientists who understand the market and customer motivators so they have proper context when looking at data. Focus on data scientists who not only bring in customers through their efforts but also can streamline internal processes and spot potential cost savings.

  • Develop Internal Staff Into Data Scientists

The best data scientist will still need time for onboarding and to learn the retailer’s specific platforms and industry niche. An option is to avoid the high salary demands of top data scientists and instead intensively train current tech-savvy staff. Look for individuals who are already involved in multiple departments and have a keen understanding of the company’s value proposition.

  • Leverage A Third-Party Vendor

Even a team of highly-qualified data scientists can struggle to manage the amount of information produced by a large retailer. Sales, customer data and even customer service interactions all come together and need to be unified. It often makes sense to bring in a third party that can help with data collection and producing reports that spur actions. Consider a third-party vendor to work hand-in-hand with your data scientists for maximum efficiency.

  • Insert Automation And Algorithms

Data scientists earn their high salaries when they find the unexpected. They should be involved in the technical aspects, but you don’t want them to be overwhelmed with process. The best approach is to have the data scientists develop algorithms that can automate the data collection and analytics process, using “predictive coding” to ensure data is converted to digestible formats.

Retailers must consider Big Data an integral part of their business, certainly not a “passing fad” that will fall out of favor in a few years. Properly using this data requires moving beyond the skills gap and finding the best data scientist expertise. Retailers that find qualified data scientists will get the most out of predictive analytics, customer satisfaction scores and the ability to jump into new unexplored markets.

Dr. Anil Kaul is the CEO and co-founder of Absolutdata. He has over 20 years of experience in marketing, strategic consulting and quantitative modeling. Before starting Absolutdata in 2001, Anil worked at Personify and McKinsey & Company. Kaul has a PhD in quantitative marketing from Cornell University. He is dedicated to the cause of exploring the best practices for implementing analytic solutions in progressive organizations and building capabilities towards robust decision-making and is a recognized thought leader in the industry, having published articles in leading management and academic journals such as the McKinsey Quarterly, Marketing Science, Journal of Marketing Research and International Journal of Research in Marketing.


Feature Your Byline

Submit an Executive ViewPoints.

Featured Event

Join the retail community as we come together for three days of strategic sessions, meaningful off-site networking events and interactive learning experiences.


Access The Media Kit


Access Our Editorial Calendar

If you are downloading this on behalf of a client, please provide the company name and website information below: