Big data and the chemical industry

Author: Bill Stringer


Chemical companies are always looking for new ways to predict future trends affecting demand and pricing. Could Big Data be another useful tool for market analysis?


 Copyright: Rex Features

Big Data is a small term for a big concept that will prove to be the next frontier for virtually every sector in the near future. The ability to effectively analyse increasingly large and complex data sets will challenge leaders across industry – in both private and public domains – as they seek to maximise competitiveness and harness new waves of growth, productivity and innovation.

The exponential growth in Big Data is worth stating: It is estimated that 2.5 quintillion bytes of data (1 followed by 18 zeros) are created every day, with 90% of the world’s data created in the last two years alone. The detail and complexity of data captured by industry and the consumer sector – coupled with the rise and rise of social media, multimedia and internet search – will continue to fuel exponential data growth. But what does it all mean? What opportunities, challenges and threats does Big Data afford? What value can harnessing Big Data generate and is the bounty available to all, including the chemicals industry?

Big Data makes previously hidden information transparent and useable and as organisations get smarter at analytics, they can more accurately assess performance and variability – and in theory improve decision making.

One of the challenges of leveraging Big Data is the sheer size of the sets – often arriving from diverse sources and unstructured in nature. Speedy development of advanced analytic techniques that can extract useful insights is key. In addition, we know there is a worldwide shortage of the talent necessary to take advantage of Big Data and several issues around privacy and security have to be addressed.

One example of a major corporate company using advanced data analytics to analyse unstructured data is Honda. The company wanted to detect and contain warranty and call centre issues before they became widespread. Prior to using advanced analytics, Honda relied on structured data from already categorised warranty and customer information. However, this approach was dependent on identifying what the problems already were.

By using techniques that could extract data from free text and unstructured sources such as technician and ad hoc customer feedback, Honda was able to determine potential future issues and take action before they became widespread.

Big Data in Chemicals
Other industries are clearly deriving benefit from Big Data – what about the chemicals and materials sector? To date the focus has largely been on consumer markets. However as chemical companies shift focus to application markets rather than product push, there is an opportunity to leverage Big Data and advanced analytics to make better demand forecasts and pricing decisions. This could ultimately drive growth and improve margins. The chemical industry has embraced the benefits of information technology for many years – in design, process management and process modelling (some readers will remember using ASPEN+ back in the early 1990s). As early as 2005 Dow Chemical was using advanced analytics to develop freight and logistics cost models as well as raw material spend analysis. The ability of procurement teams to make better decisions when renegotiating contracts led to considerable cost saving benefits.

Similarly, DSM and Sinpoec have been using information technology to enhance spend reporting and analysis to improve and speed up procurement decision making.

The new era of information technology and the advent of Big Data could create even further opportunities for the chemicals industry especially in the commercial side of the business.

Two areas of particular interest are pricing strategy and market forecasting.

Pricing Strategy
A robust pricing strategy is a critical commercial operation in the chemicals sector as it ultimately determines profitability. The diversity of raw material inputs and products offered along with markets served (geographic and application) makes pricing decisions extremely complex. In addition, factors such as market demand, raw material and energy pricing, exchange rates, competitor strategy and even weather can impact the price of chemicals. Traditionally, the strategy has been based on gut feel and experience. Would advanced analysis of these diverse data sets enable more informed pricing decisions?

Market Forecasting
Reliable short-and-long-term market forecasts are equally critical to effective production, procurement and investment planning in the chemicals industry. Market forecasting is a complex area and again, often reliant on experience and gut feel. The ability to mine vast historical data sets and uncover potential indicators for future demand and trends would be extremely beneficial.

Green Shoots Growth has first-hand experience of the potential for using data analytics in market forecasting. When a global chemical company assigned us the task of selecting an appropriate neutralisation chemistry required in a new process, we turned to data analytics. A variety of neutralising agents that produced a particular sulphate were available. Our client wanted to understand from an economic point of view, the most appropriate option based on the cost of the neutralisation agent and the potential revenue generated from the sales of the sulphate produced. Initially, we purchased off the shelf market studies, but it became apparent that through intelligent interrogation and basic analysis of freely available trade statistics (and trading sites like Alibaba), costs and potential revenues could be derived in real time without purchasing expensive and often out of date market reports.

Dow seems to be leading the way in using advanced analytics commercially. Widely reported examples include:

  • Improved sales forecasts with fewer errors
  • Early enough indication of monthly target achievement to enable corrective action
  • Cost base and exchange rate analysis enabling more effective timing of buying of raw materials and pricing of finished products.
  • Sophisticated data enhanced staffing models enabling acquisition of the right resources at the right time.

Not all plain sailing
Maximising the potential of Big Data requires new skills and ways of working. To generate the benefits cited above, Dow has recruited 10 PhDs in computer sciences supported by a team of advanced analytics experts to work alongside its business intelligence team.

Acknowledging the need for these new skills is the first step: taking advice from “geeks” – often in their twenties, with a more casual approach to business, unlike other corporate animals – can represent a challenge to the senior executive mindset.

The most obvious and immediate benefits of Big Data are around pricing and forecasting, but is there an opportunity to leverage big data in innovation management and research and development? Can consumer trends identified through social media generated data, for example, be used to determine chemicals and materials demanded in 20 to 30 years’ time?

Big Data and advanced analytics are hugely disruptive forces to our industry, but companies willing to embrace the challenge will create significant competitive advantage and growth opportunities.

Bill Stringer is managing director of Green Shoots Growth. Bill has had the opportunity to work within the IT sector for the past 12 months and this has in part generated interest in the topic of big data. Green Shoots Growth uses market insights to support chemicals and materials companies to develop and evaluate growth pipelines. 

What is “Big Data”?
Sets of data that are huge and difficult to analyse. Data from social networking sites and other publicly-available sources are good examples.

Adjusting to the “New Normal”
Consultant Paul Hodges, chairman of International E Chem, believes advanced analytics and Big Data can help chemical companies adjust to the “New Normal” of lower and less easy predictable growth driven by population demographics. Back in the 1970s and 1980s, chemical companies commissioned extensive research to understand consumer demographics, identify growth opportunities and forecast demand. In the 1990s and 2000s, however, the correlation of chemicals demand to GDP became so robust that these studies were abandoned. Hodges believes the challenge now is to understand specific segments in different regions in the same depth as in the 1970s and 1980s. He explains: “The rules of the recent past are no longer valid. Access to Big Data and the advanced analytics techniques now available enable chemical companies to build in-depth, robust insight quicker and at lower cost.

“Of particular interest is the buying behaviour of the over 55s – bearing in mind that when these Baby Boomers retire they will see their disposable income significantly reduce. In addition, the growing importance of domestic consumption in the developing world will generate opportunities, even if the income levels of this group do not match that of the developed world middle classes.”

The problem with understanding these markets is that their growth will not necessarily align with GDP and therefore in depth analysis is required. The good news is that Big Data can be exploited to build this understanding.

Process for leveraging big data and advanced analytics
Companies need to work through the following questions to understand potential opportunities for big data and advanced analytics in critical decision making:

  • What is the decision to be made?
  • What are the parameters that affect the decision?
  • What data sets are available to determine the parameters?
  • What analytics need to be applied to extract the relevant parameters?
  • Do we have the skills to do this internally or do we need to work with a specialist partner?