Big Data: What's all the fuss about?

If you read the technology news, or even just put searches into Google, you surely can't have failed to notice the term big data being bandied around very frequently. Whilst it's easy to dismiss it out of hand as the latest buzzword in the technology industry, which does like its jargon, let's face it, big data is something that could prove very valuable to your business.

So without further ado, let's have a look at what big data actually is.

Big data is a collection of data sets that are too big and complex for the usual database management tools or data processing applications to manage.

In order to be able to use this data for various business purposes, the enterprise has to find a way to carry out key tasks such as:

  • Capture

  • Curation

  • Search

  • Transfer

  • Analysis

  • Visualisation

Successfully carrying this out means that companies can then use the data to pick up trends, determine the quality of research and even prevent disease, amongst many other things. Data can be collected from a variety of sources, both inside and outside of the company. Product sales, financial information, online and offline interaction channels, all of these can make up a part of a data set.

Structured and unstructured

It's important to understand that data sets can be made up of data that's both structured and unstructured, with the latter coming from sources that are difficult to organise, such as social media posts. Multi-structured data comes from a variety of sources and uses many different formats and can be categorised as having some form of human interaction with the machine.

This means that it can be made up of text, images, web logs, forms or transactional information. Basically it's talking about any data that comes about thanks to the interactions between people and machines, whether that be web-based or not.

It has plenty of uses and with the ever-growing amounts of information that are used on social and web communities, it's something that marketers are taking advantage of and this trend is only likely to continue.

Understand your customers

The key to using big data successfully for marketing purposes, is that it allows you to leverage big data technology to understand how your customers think. This means that you can then engage with them at a time that is relevant to them and with information on the products and services that they really want.

With big data, you can examine and act upon everything that you know about your customers and this can be a substantial amount of information that allows for more accurate targeting. If you look at a company like Amazon and consider the kind of emails that you receive from them, they're always targeted to you specifically, with suggestions that they think you may act on.

Likewise, when browsing Amazon's site, suggested products will appear at the bottom of the page as you're surfing. This is what big data can do, using technology to personalise the customer experience that increases the chance that they will buy significantly.

Maximise operating margins

According to the McKinsey Global Institute, “a retailer using big data to  the full could increase its operating margin by more than 60 percent” and harnessing it for the public sector has a huge amount of potential. Health care systems could benefit from vast cost-savings and could “ create more than $300 billion in value every year”. Certainly not a sum to be sniffed at and for the US, it could reduce the amount spent on health care by a whopping 8%.

Governments in Europe could also make huge savings, which McKinsey say could total as much as 100 bn Euros. Big data has numerous uses and there's no doubt at all that it will also aid research in such a way that's never been seen before. Of course, there are also always the scare stories and conspiracies that mean there will always be some media sensationalist story about the dangers (such as the NSA and how they use it), but that's a natural side-effect of progression.

More research

It's no surprise that IBM are one of the pioneering technology companies that are tackling big data and providing software to handle it either. Now, the company has also launched its new Accelerated Discovery Lab which aims to find connections among massive data sets, which can in turn be used for analytics.

“There’s a set of data challenges and lot of different expertise you need to tackle those problems,” Laura Haas, the labs director of technology and operations told Venture Beat in a recent interview.

“Everyone is talking about Big Data and analytics. We talk to our researchers, but it’s still really hard to get the right kind of insight out of the data.”

The new lab is located at the IBM Almaden Research Center in San Jose, California and already one researcher has used big data to decipher someone's personality just by studying 200 tweets.

Big data isn't for every company, if only because small businesses may struggle to find the time and money necessary to utilise it. But there's little doubt as to the value of it, in pretty much every industry across the board.