Big Data: big headache or big opportunity?

One of the hot topics du jour is Big Data – the admirably concise term used to describe the problem of managing very large, complex and potentially unstructured data sets.  

It does seem that we live in an era of ever-bigger numbers – not only do we have sovereign debts measured in trillions and rising, but it is now common for corporate databases to be measured in petabytes (millions of gigabytes) or even exabytes (billions of gigabytes).

For manufacturers, the idea of constant streams of data from their instrumented plants – the plethora of sensors, readers and other devices that track production operations, machine condition, material movement and so on – is not new. 

Here, though, the ways in which the data are processed and the actions to be taken on the outcomes are well-defined. The newer Big Data issues that confront manufacturers are the result of ‘instrumenting’ their business – the huge quantities of data collected by their suites of business systems. With these data sets, the ways in which the information is retrieved and analysed and what to do based on the results are much less deterministic.

Seeing the wood for the tress

There are certainly information technologies available now that are capable of mining large data sets quickly and efficiently (generally called Business Intelligence technologies). The challenge lies in the application of them.

For example, the data collection capabilities of modern day business systems (usually some combination of ERP, Supply Chain Management, Customer Relationship Management and Product Lifecycle Management) mean that it’s possible to collect data that don’t really need to be collected – giving visibility of lots of things that don’t really need to be visible.

Similarly, there is a time dimension – business data changes rapidly so it may be tempting to perform analyses on short time horizons when a longer average would provide a much better, more actionable view.  

It is also easy to be misled by data, an issue that is particularly associated with another aspect of business systems and Big Data in manufacturing – the view that manufacturing companies can be run by good data analysts rather people that really understand the business. Just like, say, engineering analysis and simulation technologies, it’s important to understand the fundamentals to avoid thinking correlation must imply causality.

Having said all of the above, it is important that manufacturers do have the ability to collect and analyse the right information about their business. While the most aggressive adopters of large scale data gathering and analytics are consumer companies (especially the internet based ones like Amazon), manufacturers are increasingly exploiting those capabilities in areas like analysing their markets, monitoring supply chain-wide operations and obtaining rapid feedback on the effectiveness of new initiatives. 

Of course, while both academic research and the IT industry promote the impact of fantastic data analytics as making companies more productive and responsive to market needs, there seems to be a danger that information processing becomes an end in itself, rather than serving the productive aspects of the business.

To avoid this, it’s important to define what the information is going to be used for – how does it link to either a business requirement or an improvement opportunity? This, even more than the technology, is the Big Data challenge.