Musings on Big Data and Supply Chains. Fireworks?

July 3, 2012
4:01 pm

“You are speaking a new language. Where do we go to learn these new concepts?”

A recent client comment during a session on Big Data Usage in Supply Chain

It is July 3rd. For most of us YANKS, tomorrow is a holiday. It is time for us to wave flags, sing patriotic songs and watch fireworks blaze magnificent patterns on night skies. For me, while I will watch the fireworks, it will be a calm day in the middle of a calm week. One that I hope will be a productive one.

It is welcome relief: a week with no travel. I am off the road! Today, there is no mad dash to the airport, or crazy searching in my purse for the lost hotel key. With my travel schedule I constantly struggle to unravel time zones, remember the number of the hotel room I checked into or the color of the rental car; but not today. This is the first week that I have not traveled in six months. WHEW! Where has time gone? My body is decompressing a bit, not quite knowing what to do next. With freedom, and being unshackled from the busy travel schedule, it is hard to decide. Shall it be a blog, a report, the design of a quantitative study, or more work on the book?

As I write this, the east coast of the United States is blistering hot. So, I am looking forward to spending the week writing with the air conditioner running full tilt. Hopefully, I will not blow a circuit breaker. With any luck, I will finish four reports, two blog posts, five quantitative studies and review two chapters of book edits for Bricks Matter (can be pre-ordered at Amazon.com ). Most of my team at Supply Chain Insights are kicked back enjoying vacation with their families. So, won’t they be surprised when they come back to FULL mailboxes?

Next week, I work with a global supply chain executive team on the Future of Supply Chain Management and the pending impacts of big data, digital business, and mobility. To get ready, I am compiling the data from the Supply Chain Insights survey on Big Data. It is fun. I am having a good time.

It was a tough survey to complete in an area that is not well understood by supply chain leaders. So much so, that only one-in-four respondents could finish the survey. While over 240 people started the survey, only 53 could finish it. I thought that the “open-end” comments from the study listed below summed up the current state well. They are shown in figure 1.

Bottom line, it is a new language and a new way of thinking at a time when we are not really using the data that we have. I particularly like the comment that “Big data seems like a silly term invented by a bunch of IT nerds trying to sell more ‘stuff’. Why not name it for the results not the data?” I could not agree more.

I firmly feel that “Big Data Concepts” will rock the boat of the nice and neat traditional supply chain application world that companies have worked on for two decades. So, as we start this discussion, light the fireworks. I feel that the discussion of Big Data is a revolution, not an evolution. The world of supply chain applications that has been defined by neat, nice packaged application definitions where the vendors are well-known and defined around traditional business models is changing. It not about cool technologies or the re-marketing of Web 2.0 concepts; instead, it is a fundamental rethinking of supply chain business problems. So, as we begin the discussion on Big Data, realize that much of what we have learned over the course of three decades gets thrown out the window.

These concepts are not some crazy waving of hands. It is here to stay; but, like the fireworks in your backyard, approach it with caution. Yes, the concepts are over-hyped, and everyone is using the terms; but the basis of the definitions are well-rooted in business need states. It started with eCommerce and telecommunication giants. Web search had a problem. It had the characteristics of very large, distributed aggregations of loosely structured data that was often incomplete and inaccessible (requiring inference from new forms of predictive analytics). The volume of the data was usually large (petabytes and exabytes of data), the variety of data was high (new forms of data including geolocation, unstructured data from social, customer call centers, warranty information, ratings and reviews and blogs) and the velocity of the data was high (streaming data, sensor data, and mobile data). Simply stated Big Data is a data set that is too large and complex to process using traditional methods. It requires new techniques and architectures to process the data and to sense data problems.

As we extend the concepts of supply chain from the customer’s customer to the supplier’s supplier, we are also facing this boundary. The data is no longer structured. We cannot listen, test and learn about shoppers without embracing unstructured data. While the largest complaint in enterprise supply chain systems is dirty data, we haven’t seen anything yet. The building of the end-to-end supply chain will give us new forms of different data (that will be called dirty) that will need to be embraced using new and different techniques. This is the world of Big Data. Here is the Shaman’s advice:

Big Data Needs to be Focused on Delivering New Value-based Outcomes. The discussion should start with the business need state and the impact on value-based outcomes. For example, retail demand insights, agroscience crop yield, food safety and pharmaceutical serialization/tracking are Big Data problems. I think that demand sensing, digital path-to-purchase and digital manufacturing will become Big Data problems. I also think that we will gradually solve master data issues in the enterprise through master data techniques; however, you don’t start with the data and work out. Instead, you start with the problem, analyze the data set requirements and then look for appropriate technology. The ah-ha moment comes when you find that these new data forms do not neatly fit into traditional enterprise applications. It is not data that neatly stuffs into an ERP or APS architecture. They are too large, too different, and moving at too fast of a rate of speed.

It Needs to be Led by Line of Business Leaders. The survey had 53 completes from 44 companies. The companies were primarily in retail and consumer products industries. In the study, I find that retailers are further ahead in their thinking on demand, and consumer products companies are thinking more about big data to unleash new opportunities in supply. 8% of companies have an ERP system that is currently at a Petabyte of data and 47% expect it to be a Petabyte within five years. In the survey, 38% of organizations had a cross-functional group chartered to better understand Big Data. This team was twice as likely to exist in the organization if the size of the ERP instance was believed to be a Petabyte of data within five years. However, it was most likely to be led by the CIO (47% of the time). I find this problematic. I feel that the greatest success in these new types of initiatives happens when it is focused on solving a business problem led by a line of business leader.

The Use Cases are Many. Start Simple. In fact, there are so many use cases that the problem for many teams may be where to start. In figure 2, I share the relative importance and perceived performance of twelve use cases from the survey responses from the study for companies that currently have cross-functional teams working on supply chain Big Data problems. Note that the sample size is small, and we can only take away high level trends. My insights are that the business cases are varied and many. And, that teams are confused. Today, the biggest focus is against two use cases: safe and secure supply chains and building listening capabilities for sentiment. I predict that it will soon include digital manufacturing and digital path to purchase initiatives.

I am a data-driven girl. I deplore hype. I love thinking about the art of the possible and helping clients to drive first mover-advantage. Getting data like this was hard. This survey was in the field for five months and I begged and pleaded with every advanced supply chain thinker that I know to fill it out. I was grateful to have help from WIS Publications, Red Prairie and Susan Scrupski, Founder of the Social Business Council. Each organization shared a link with their audience and helped me to gather the data.

My Fear

This week, I received notification that the Grocery Manufacturers Association (GMA) IT committee is sponsoring a project on Big Data. I received the information via a press release attached to an email to discuss the project with Deloitte. Reading the press release gave me a headache. Talking to Deloitte sent shivers down my spine. (Reference the press release at http://www.gmaonline.org/news-events/newsroom/gma-selects-deloitte-to-harness-big-data/) Why?

1) My Wish. These concepts are new and overhyped. I have traveled the last year to Big Data after Big Data conference and I have never seen folks from my GMA contacts or Deloitte at the conferences. I also just finished a study where I tried to get information from the GMA audience and found that the respondents could not fill out the survey.

I fear that GMA and the GMA IT committee have lost relevancy. As the organization has focused more on lobbying, the entry fees for membership have increased in price. It is too high for smaller manufacturers and technology providers. Most of the smaller manufacturers (less than 5 billion) question the value. (e.g. to participate as a member, I would have to spend $10,000/year. And, I am barred from helping the workgroups because I am not a member.) I find that the most advanced thinking is found in smaller consulting firms, not the large system integrators or technology providers.

As I read the press release, I worried about the project definition. As a data-driven gal, I fret about these kinds of things. The definition of Big Data as “the recent increase in largely external and unstructured business and consumer information” disturbs me. Big Data is SO much more than this definition. I am also worried by the definition of the statement of work to “create a Big Data and analytics roadmap for CPG companies focused on driving top and bottom-line growth.” I think that the evolution of the Big Data techniques and practice is too early to define a roadmap. Instead, I would like to see the GMA IT committee focus on education and insights from current pilots in the industry. What I would not like to see is slick consultants in nice suits talking about theoretical applications at the upcoming conferences. <And, while we are at it, please save the trees. Please do not give me another pretty brochure without any insight (I am finding the quality of the GMA publications done by big consultants to be declining in value.)>

If the committee wants to do real work, I would look at the early case studies and understand the barriers and enablers more completely. I would analyze how these teams got initiatives funded and how organizations are using Big Data systems to solve new type of problems and improve policies. I would define a handbook of terms and practices that companies are using. I would educate consumer products and retail executives on why Big Data concepts matter. I would start with:

Digital Path to Purchase. The work that McCormick is doing on Digital Path to Purchase is breakthrough thinking. I would start there.
eCommerce. Amazon‘s work on understanding demand insights of pantry shopping is exciting. They are an early leader in Big Data techniques. I would have them at the top of my list.
Supply Chain Visibility. Let’s face it, we have been talking about supply chain visibility and agile supply chains for many years, but it has just been talk. The use of rules-based ontologies and learning systems to redefine supply chain visibility at Conair is a new way to think about sensing supply chains. While early, it is a great case study on how to use Big Data techniques to solve a tough problem.
Listening. Text mining and ratings and review information at Bazaarvoice is a Big Data service. How could companies use this data? How could it help in sensing early product failure? Only one out of ten companies that I talk to have ever heard of Bazaarvoice and the great work that they are doing.
Supplier Sensing. The work at D&B on supplier sensing is a great use of Big Data. I would include them in the list and work with them on how consumer products companies and retailers can sense supplier failure early and use it to build stronger supplier relationships. We have talked about collaboration, but in reality, we have pushed costs and working capital back into the chain increasing risk. The further back in the supply chain that we go and sense supplier health, the weaker the players and the greater the risk to the brand. The work at D&B is a great start to better understand this.
Safe and Secure Supply Chains. The work at Eli Lilly on product serialization of pharmaceuticals has direct applicability of where we are headed on food safety. Food and beverage companies need to learn from this early work in Big Pharma.
Demand Insights. The work at Kraft on consumer insights, or the work at General Mills on downstream data are Big Data problems in the making. While both companies today are using more conventional techniques, the size of the data is growing and insights can be gained on where we are headed.
Large-scale ERP. The race is on for global companies to better serve emerging markets. These markets are fraught with disparate data that is often incomplete. These large consumer products companies are also the companies with BIG DATA ERP systems. What are best practices for companies in the ERP Petabyte club? How does the data change? How does it affect maintenance and upgrade cycles? And the definition of business analytics?

2) The second best thing that GMA can do is Rethink the Value Proposition. I am disappointed that GMA has largely become a lobbying organization losing relevance in helping the industry move forward. It is largely dominated by large consumer companies and technology providers that are jockeying for position. The presentations are stagnant. The studies are largely dominated by large system integrators that are delivering declining value.

Let’s face it. We are not moving the industry forward. A decade ago, the consumer products industry wasted 500 million dollars in the investments in Transora. While other industries moved forward with trading exchanges that were rooted in value — E2Open, GHX, Exostar — the consumer products industry lost momentum by focusing on data for the sake of data, not on driving value-based outcomes. I would hate to see this happen again.

Wrap-up:

Light the fireworks tomorrow and enjoy the show. Sit back and watch them light up the sky. As you stand back and look at the show, reflect that this is a form of pattern recognition. It is what we need to do with Big Data Analytics. It is about SO MUCH more than rows and columns and traditional analytics.

Data is piling up at the doorsteps of our organizations. It is in different forms offering new opportunities. We cannot just stuff the data into traditional applications. It is time to step up and accept the challenge while realizing that we are at the start of a long journey. What do you think? Please let me know your thoughts.

For more on this topic, reference my earlier posts on Big Data:

Boosting Your Vocabulary for Big Data