Over the past six years, I have studied the use of downstream data, and watched consumer products companies inch along, ever-so-slowly, with pilot projects.  While I challenge my readers to take a leap of faith (reference prior blog post http://www.supplychainshaman.com/demanddriven/a-leap-of-faith/), and aggressively use downstream data (E.g. point of sale, warehouse withdrawal, loyalty and retail demographic data), I also want to equip them for the journey.  It is not as easy as it sounds.  There are pitfalls and landmines, and major obstacles to overcome. 
Last week, I was in Chicago working with clients on the use of downstream data and the design of Business Intelligence (BI) strategies to become demand driven.  Like many clients, these companies were in the middle of an Enterprise Resource Planning (ERP) upgrade cycle, and wanted to put downstream data into ERP to use the data to be more demand driven.  I ruined their day, when I told them that I could not endorse their approach.  Which led me to sharing the three things that I have learned about the usage of downstream data with these teams:
-It is not about Integration.  This is too simplistic. It is about the synchronization of demand data.  To synchronize and use the data, it must be cleaned, harmonized, and enriched based upon a carefully crafted data architecture and road map.  Unfortunately, for many companies, they learn too late that ERP is not the de facto enterprise data model.
Data coming in from retailers represents their data model.  To use the data (for anything other than sales reporting), the data must be converted to conform to the enterprise data model for product (hierarchies of selling units), customer ship-to locations, and corporate calendars and augmented with shipment and order data from Enterprise Resource Planning.  Since most data in ERP is the manufacturing unit—not the selling unit—and represents ship from (what should be made at a manufacturing plant)—not ship to or the retail channel—the ERP data must also be converted to a standardized logical data model.  Stuffing downstream data into ERP and ignoring this process is like stuffing a square peg into a round hole. 
-Design with the End in Mind.  In the organization, there are multiple use cases—sales reporting, category management, trade promotion management, replenishment, demand planning, improved transportation planning, cost to serve and client profitability modeling, inventory reduction and obsolescence reduction, item rationalization, shelf compliance sensing, new product launch success and demand orchestration processes (to translate the demand plan into buying strategies),   Each of these use cases requires a different frequency of data –daily data/daily, daily data received weekly, weekly data received weekly, etc—and data enrichment schema.  As a result, data must be held at the lowest level of granularity. 
The problem on usage happens when companies start with simple use cases in the front office and as they mature, the requirements to use the data change and mature.  To enable this process, companies should design a data model with the end in mind, and store the data at the lowest level of granularity to ensure flexibility in aggregation and reuse (note the Coca-Cola data model presented by Data Ventures on 12/17/2010 in figure 2).  In doing this design, teams will quickly find that the data model that works is not ERP.

-Cultural issues Abound. Attack them upfront!  While companies want to quickly point out the issues on the use of downstream data, the greatest issue lies in retooling the organization to “embrace new concepts, and think about the art of the possible.”  To do this, five change management issues need to be addressed up front:
1) A shift in focus from “ship in” to “sell through” in the channel.  The traditional approach encourages companies to push product into the channel in a very efficient push-based supply chain.  As companies move from push to pull, they realize the need for push-pull decoupling points and the need to sense and use multiple demand points in the channel simultaneously through new forms of predictive analytics.
2) Realization that syndicated data is wrong.  A barrier for the team is the traditional paradigm of “why do I need point of sale data, if I have syndicated data?”  After many weeks, it becomes clear that the sources of syndicated data are inaccurate and that there is enormous value in seeing channel movement 3-4 weeks earlier than you can see it through syndicated data usage.  A barrier is the high dependency of marketing and sales organizations on syndicated data, and the unfaltering belief that it is sufficient.
3) Redefine what good looks like.  The traditional definition of the supply chain changes.  Organizational incentives reward vertical silos.  Sales incentives are to sell volume, marketing’s are to improve market share, and supply chain is rewarded to reduce costs.  As companies use downstream data to build horizontal processes—a shift from north south to east west processes—the definition of supply chain excellence changes.  Companies realize that supply chain extends over go-to-market activities of sales and marketing, and can be used to improve revenue management, channel sensing, and new product launch.  However, companies can only get there if they give themselves the permission to change focus from inside-out processes to outside-in processes.
4) Right stuff.  The organization needs to reward inspiration, perspiration and innovation.  These initiatives are being driven at the director and manager levels not from the top down. These three characteristics are prevalent in  companies that have leap-frogged the competition.  In these organizations, there is a line of business leader with a clear vision, and the power to influence and persuade to get funding.  There is also investment in back office analysts to look at the data differently—to let the data answer the questions that we do not know to ask—so that companies can reap maximum value. 
5) New concepts.  Companies are not used to thinking about demand latency and inventory velocity.  Enterprise applications focus on data integration not the rate of change.  It is a different focus and requires a different mindset.
It is hard work.  It is cross-functional.  It is a new way of thinking.  At the core, it challenges traditional paradigms.  However, if you can cross these boundaries, companies find that the use of downstream data pays for itself in less than six weeks every six weeks, and companies that were good at the use of downstream data and sensing channel demand aligned and transformed their supply chains 5X faster than competition.  Procter and Gamble attributes their work with Terra Technology and the use of downstream data to a 2.5 billion dollar reduction in inventory.  Jim Temme, previously of Anheuser Busch attributes the use of downstream data to a 20 million dollar improvement in trade promotion spending.  Three companies have given recent testimonials at industry conferences on the use of downstream data to improve shelf sensing prevent voids and no-scan with a return in days.   And, if this is not enough, consider the changes coming in 2011:
-Wal-Mart is upping the ante for many manufacturers on penalities on trucks that are not delivered on time and in-full up to 3% of the value of the truck.
–Kroger and Safeway data programs are launching increasing the ability for grocery manufacturers to get signficiant channel concentration of data to manage the demand signal from the outside-in.  The increase in point of sale data in the channel is a great opportunity to improve the supply chain response.
-The need to tie digital and social programs to brick and mortar programs.  Retailers are asking companies to shape demand in the store through social programs.  Demand elasciticy for social couponing programs is 100X the response of store circulars necessitating a responsive supply chain.  But, how can you be responsive if you cannot sense?
With that, have a great holiday season and a good new year.  Maybe, just maybe, the serious use of downstream data will be on organizations’ New Year resolution for 2011. In the words of  The Night before Christmas, “now dash away, dash away, dash away all.”