Tag Archives: big data

Big Data – Can’t ignore it?

Big data

2012 is almost over and I just realized that I have not yet posted a single entry about big data. Clearly a big mistake – right? Let’s see: Software vendors, media and industry analysts are all over the topic. If you listen to some of the messages, it seems that big data will create billions of jobs, solve all problems and will make us happier individuals. Really? Not really – at least in my humble opinion. It rather seems to me that big data fills a number of functions for a select group of people:

  • It provides analysts with a fresh and fancy-sounding topic
  • Media have something big to write about
  • BI companies obtain a ‘fresh’ marketing message
  • Professionals can have ‘smart’ discussions
  • Consultants can sell new assessment projects

Big data – really?

I do apologize for sounding so negative. But I have a hard time finding big value in this big data discussion. Please don’t get me wrong – I would be the last person to deny that there is a tremendous amount of value in big data. But it does not deserve the hype. On the contrary, I personally find that the current discussions ignore the fact that most of us do not have the skills to do big data. We need to get the foundation right and make sure that we can tame the ‘small data lion’ before we tackle the big data Gozzilla. Don’t believe me? Consider the following:

  • Spreadsheets are still the number one data analysis tool in most organizations.
  • Managers still argue about whose revenue and unit numbers are correct.
  • Knowledge workers have yet to learn how to make sense of even simple corporate data sets.
  • 3D pie charts are floating around boardrooms.
  • Companies spend over 6 months collecting and aggregating budgets only to find that a stupid formula mistake messed up the final report
  • Hardly any professional has ever read a book or attended a course about proper data analysis

Pie Chart

Here is the thing: Dealing with big data is a big challenge. It will require a lot more skills than most of us currently have (try finding meaning in gazillion TBs of data using a 3D pie chart!).

A big data problem

Earlier this year, I acquired a 36 megapixel camera. You can take some amazingly gorgeous photos with it. But it comes at a cost. Each photo consumes 65-75MB on my sad hard drive. Vacations now create a big data challenge for me. But guess what: this camera is anything but easy to handle. You have to really slow down and put 100% effort into each and every photo. 36MP have the ability to reveal every single flaw: The slightest camera shake is recorded & exposed. Minimal focus deviations that a small camera would not register, kill an otherwise solid photo. In other words: this big data camera requires big skills. And here is something else: The damn camera won’t help you create awesome photos. No, you still need to learn the basics such as composition, proper lighting etc.. That’s the hard stuff. But let me tell you this: If you know the basics, this big data camera certainly does some magic for you.burj khalifa

Big data – what’s next

Ok. That was my big data rant. I love data and analytics. No doubt – there is a tremendous amount of value we can gain from those new data sources. But let’s not forget that we need to learn the basics first. A Formula 1 driver learned his skills on the cart track. At the same time, there is a lot of information hidden in our ‘small data’ sources such as ERP, CRMs and historians. Let’s take a step back and put things into perspective. Big data is important but not THAT important.

With that: Thank your for following this blog. Happy holidays and see you next year!

Christoph

Related Posts:

Visual Analytics – The new frontier? (Guest Post)

WHAT IS VISUAL ANALYTICS – BY DR JOERN KOHLHAMMER

Massive sets of data are collected and stored in many areas today. As the volumes of data available to business people or scientists increase, it becomes harder and harder to use the data effectively. Keeping up to date with the flood of data using standard tools for data management and analysis is far from easy. The field of visual analytics tries to provide people with better and more effective ways to understand and analyze these massive data sets, while helping them to follow up on their findings immediately, in real-time. Visual analytics integrates the analytic capabilities of the computer and the abilities of the human. This means, the human is empowered to take control of the analytical process; he or she is not just the final stage of a reporting process. Visual analytics sheds light on unexpected and hidden insights, which may lead to innovation and increase profits. For example, many key performance indicators are simply calculated using statistical models. But the true relations between data, models and business objectives often remain unclear. If visualization is included as an integral part of the analysis process then comprehension of the models as well as of the data is increased. Errors in the basic assumptions of the models can be recognized early on and newly discovered dependencies in the data can lead to new and possibly better reporting indices. Continue reading

Related Posts: