It happens all the time.  IT says to line of business leaders, “Tell me what you need for Business Intelligence (BI), and I will go find the right technologies.”  The issue is that we don’t know, and we will not know soon.  We only know that applications are changing and that the data is growing exponentially.  The answer to the question of: “What is the right data architecture for demand-driven value networks?  The answer is “It is evolving. We don’t know.”
All we know is that it will get even bigger and more complex.  We are facing a redefinition of applications for consumer products and retail.  It will require a rethinking of business intelligence strategies. 

The Discussion Today

When I talk to most people about BI and demand-driven value networks, the discussion quickly evolves to two topics:

• What is the best BI solution to layer on top of Enterprise Resource Planning?
• How do I contain and manage the volumes of data in Excel spreadsheets?

For me, these are the wrong discussion tracks.  It is like going to the wrong church with the wrong pew.  For a lady that is knee-deep in building a house, the analogy that I would like to apply is “that it is like a discussion of which window pane will be best for the view, when we should really be talking about the design of the house.”

Planning for Tomorrow

I think that we need to create conversations that don’t exist.  We need to free the data to answer the questions that we don’t know to ask.  Sensing technologies to support the evolution of pattern recognition technologies, advanced optimization and rules-based ontologies.  We also need flexible information architectures that will support changing application infrastructures.
Let’s take a closer look at six drivers:

• Geo-spatial Data. Maps enable new ways to engage the shopper. The list is endless but includes geo-mapping for gaming, new technologies to track shopping to drive in-store insights, and searches across banners to understand in-stock positions, pricing strategies and offers.  Mapping has grown in importance and generates a new type of data.
• Sentiment Analysis/Listening Posts:  I was speaking to a customer last week that is trying to syndicate data from 800 review sites, and another that was listening to customer sentiment at 500 listening sites.  This is unstructured data that needs to be cleansed, syndicated, and managed.  A new source of data that will be critical to the evolution of the future of demand-drive value networks.  It will allow us to better plan demand, execute assortment and run-out programs, and adjust new product launch programs.
• Loyalty Programs.  Traditionally, loyalty programs have been household level data.  As social/mobile/ecommerce programs converge, loyalty programs move from household to individual loyalty data sets.  The data explodes as we add shopper attributes to individual data to drive market insights.
• Engage.  Before we have loyalty data we have the tracking of engagement behavior.  How do I engage with fans like me?  In programs that make sense in my community?  The result is social behavior data.  Look for a convergence of social and shopping insights data as we try to merge information across the proliferation of channels.  The apparel retailer will soon have at least five channels: social, mobile, ecommerce, direct store purchases and purchases within other retail outlets.  The term cross-channel will take on a WHOLE new meaning.
• Point of Sale (POS)/Enrichment.   As we move from broad-brushing markets to localized assortments driven by crowd sourcing/social fan engagement, retailers will add enrichment data to POS data.  Recently, I attended a Retail Connections conference <BTW, Marc Millstein throws a great conference>, three retailers mentioned that they have over 100 attributes to add to their retail POS data.  Just think about the possibilities of how we can use this enriched data to sense, shape and respond to demand.
• Data Enrichment/Syndicated Data.  Despite the investment in downstream data, syndicated data is not going away.  Instead, it will become deeper and we will see a coelesceance of social, consumer and shopper insight data.

 Where to Turn….

My bet is that answers will come from a new set, and a mixture, of predictive analytics vendors that are largely Software as a Service (SaaS) suppliers – Bazaarvoice, Clarabridge, DemandTec, IRI, M-Factor, Predictix, Revionics, SAS, and Terra Technology—  in combination with vendor solutions from data analytics companies like GreenPlum, Netezza, and Teradata. 
In the short term, due to the large quantities of data, I am also seeing a Microstrategy resurgence and the use of Microsoft ProClarity for ad hoc reporting.   Business models for downstream data found for consumer products in solutions like Retail Solutions, Relational Solutions, Shiloh, Vision Chain and VMT will find their place within the Microsoft, Microstrategy and Teradata ecosystems. They will consolidate and mature, but the data models are needed to support retail-specific processes.  Prepare for IBM Cognos, Oracle BI, SAP BW, and SAP Business Objects approaches to be largely relegated to the world of reporting infrastructures.  When it comes to managing demand data, they have been proven to not be equal to the task.

My Take….

It is time to stop picking out window panes.  We need to build a new foundation—a data foundation—for the new enterprise architecture that is coming.  Yes, we do not know exactly what it looks like, but we do know that it will need to:

• Manage large quantities of data
• Most of the data is EXTERNAL
• Unstructured data will open up new frontiers for sensing true customer service
• Be built using a data strategy where data reuse will be key
• Be flexible to support changing applications as the vendor landscape rises, wanes and falls.  There will be a lot of change.
• Support line of business sandboxes that will become prevalent to run specialized analytics
• Have advanced master data management (MDM) tools for data maintenance

Am I missing anything?  What do you see?  Where will you turn?