Thursday, September 18, 2014

The Big Data Revolution

Heard of this term Big Data? 

This is a term that surfaced few years ago but has been well-embraced and adopted by researchers and organizations all around the world. The rapid adoption of smartphones, billions of posts made on social-networking sites such as Facebook and Twitter, tracking of anything and everything we do online, from purchasing a book online to visiting someone's profile on Facebook, all of it is adding to the "digital tsunami" [1] known as Big Data. Technically, it is used to describe data that is not only voluminous but also comes in all different formats (as an instance - no specific format on the posts you make on Facebook or consider the raw data coming from sensors) and at high speed (sensors monitoring a room temperature every second or so). 

One can certainly see how this deluge of information is benefiting our quality of lives. We can track the status and progress of any flight on the click of a button,  observe the traffic patterns to identify the fastest route to the destination, determine weather ahead of time for planning our trips. With the e-commerce websites,with every purchase or even browsing their catalog provides you with recommendations. The advent of websites such as Coursera has enabled open access to education worldwide. Algorithms have been developed that can mine this plethora of information and draw conclusions before a human mind can even detect patterns or hidden information in the data. We can keep going on with this list and it is all great. With so much data out there, we are literally drowned in an ocean of information these days. 

But the question that we then need to ask ourselves is: Are we paying a cost for having all this more information at our fingertips? What are we losing or trading-off and what can we do about it?

As the author of the article titled, The second age of reason [1] from Time magazine quotes, "Our brainspace seems to be invaded by cyberspace". We tend to interact more with people whom we possibly just met once but connected through facebook than with real friends. Our work, play, rest all include connecting to internet. With so many websites offering advice, be it medical advice or immigration or any other issue for that matter, while they can help you in figuring out answers to your questions, the amount of information out there and the worst-case scenarios they bring to your attention, can also create even more confusion, creating mountain out of a molehill. Kids these days start tapping on iPad or iPhones much before they start going to school and parents feel proud to call that as smart. 

Furthermore, similar to the era when computers arrived and started gaining adoption in more and more industries, the technology created a stir among the middle-class sector. With the fast computing power, they also led to loss of jobs that were earlier done manually. Thinking on similar lines, who would need travel agents when websites such as Kayak or Priceline are out there. 

I agree with the author when he remarks that all this new information is valuable and needed. But I also think that we need to acknowledge the benefits of some of the older way of doing things. Instilling the skill of reading maps rather than blindly depending on apps? Lets not favor shortcuts and an easy way out in all cases. In my opinion, a better strategy would be to pause and reflect. Before picking Big Data and technological solutions for every situation we encounter, lets just pause for a minute and think about what and how we would have seen the same situation without the tools we have in hand today. 

Just as a revolution always creates "collateral damage" [1], the Big Data paradigm also possesses its pros and cons. What matters is how we put it into use.

The floor is yours now. Feel free to share your thoughts. 

- Dippy

[1] Michael Grunwald, "The Second Age of Reason", Time Magazine, September 8/Sept. 15, 2014.

Post a Comment