Right this very minute that you are surfing the web, you are creating data. Checking your facebook page, updating your twitter, even reading your favourite blog – your steps are tracked creating line after line of code. Where does all this data go? The answer is that we are not quite sure yet, but there are many ideas around at the moment putting Big data on the buzzword map. Big data might in the future help catch criminals or fight disease. In the field of business intelligence, it could enable CEOs to gain insights into complex data environments that were previously unheard of.

Now we have more possibilities than ever for collecting data. In fact, 90% of all data that exists today has been collected in the last 2 years. The Internet, GPS satellites, sensors and CCTV cameras are creating thousands of megabytes every second. Megabytes might be an underestimation. While most people have heard of terabytes by now, fewer are familiar with terms like zettabytes and yottabytes. Even if you look up how much data that is the dimensions are beyond the imagination of most human minds.

The aim of the new search for complex data is to understand human behaviour and human communication as it happens on the internet. Business intelligence solutions can benefit from data inputs that mirror their customer’s behaviour. Big data is aiming at entering data into business intelligence services in real time, setting it into context with previously acquired data on the fly. Just like scientist are dreaming of predicting earthquakes by on-the-go-tracking large amounts of seismic activity and health officials hope to predict the outbreaks of diseases by real-time monitoring equal data amounts, CEOs are looking to unearth similar trends in business that were inaccessible before.

But with Big data also comes big challenges. Problems in storage, searching through and most of all analysing big data are obvious. Big data is largely unstructured data, which makes analysing tools central to any analysing effort. Visualization has proven a powerful tool and has rendered interesting and sometimes even artistic results.

Microsoft’s Bing, still a minor player in the field of search engines (and ridiculed for copying Google technology), could make it big as a big data collector. Bing was fed with technology designed in Microsoft’s Probase research project as well as technology acquired when the company purchased Powerset. Probase is an innovative data collector that does not regard web content as facts, but as assumptions (something everybody should do). Powerset is a search engine that is designed around semantic intelligence principles. Equipped with these technologies, Microsoft is hoping to turn from a software developer to a future Data Broker.

When Twitter announced on Tuesday that it had acquired BackType, a big data analytics platform, it became clear that the big data land grab had officially begun. We don’t know what the future holds – just one thing – it’s gonna be big!