This is a subject I have been screaming about for a number of years. Finally, it is subject that the industry is starting to recognize as a big problem. Business Intelligence and data mining have been around for some time, but the issue has always been that the biggest, most expensive BI solutions are one size fits all. This has resulted in very poor results and disappointment to organizations that have made significant investments without realizing an appropriate ROI.
You must give the “power” users in an organization different tools than the casual user. You have to build a center of excellence where the analysts and data scientists who understand the data and at least recognize what the business users need and the issues they are trying to solve. This group produces the data analysis and more importantly, the format that allows the more casual Line of Business user to use that analysis to make better and more insightful business building recommendations.
It is not only retailers that have this hurdle, it is many manufacturers as well. I would say that the manufacturer community has a leg up in this area and has been working with loyalty card, web and other qualitative data and in many cases, combining that data with the typical demand data for better insights. I also firmly believe that this is an area where the manufacturer can take back some control and have a bigger seat at the table with their retail partners to even the playing field!
Retailers cannot be experts in every category and they certainly cannot be experts across all the data that is available to mine for greater sales. The data scientist is just a new name for the super power users in every company. The real issue is there are not many of them, and that is why a Center of Excellence makes so much sense, because when a data scientist builds a compelling analysis, it should be done in a way that it can be shared with others throughout the organization so if that person leaves or “gets hit by a bus,” it can be picked up by others.
Read the article at http://www.retailwire.com/discussion/17405/where-have-all-the-retail-data-scientists-gone