Data for Breakfast, Lunch, Dinner, Desert and in between Snacks

Data for Breakfast, Lunch, Dinner, Desert

and in between Snacks

Yes, it is time to disclose my secrets. I had Data for Breakfast, Lunch, Dinner, Desert and in between Snacks already for few months, while I ate and drank data with great appetite for the past years.


I am a Big Data eating monster! So are all of you and your friends and the companies you work.


All of us also generate a lot of data every hour and day, month after month. Your home will generate data. Your smart phone and other mobile devices generate data, as you walk, drive in a car and even more when you walk in a mall or shopping center. These devices have embedded "spy" (actually sensors)  in them to report about all your personal activity data.


Ubiquitous wireless sensor networks in cities, suburbs, farms and even forests collect and report a lot of data. All type of machines, in your house, factory and all types of environments generate what is called "machine generated data"



Data has become a factor of production, almost on par with labor and capital. The companies and investors are placing their bets on big data and analytics. There  are many  different measures of  this phenomenon. 


IDC  predicts  that  the  digital  universe  will  be  44  times bigger in 2020  than it was in 2009,  totaling a staggering 35 zettabytes.


EMC  reports  that  the number of customers  storing  a  petabyte  or  more  of  data  will  grow  from 1,000 (reached in 2010) to 100,000 before the end of the decade, a hundred fold increase in 10 years. By 2012 it expects  that some customers will be storing exabytes (1,000 petabytes) of information. In 2010 
Gartner reported that enterprise data growth will be 650 percent over the next five years, and that 80 percent of that will be unstructured.


Ocean size data is useful, if the companies can make use of it by business intelligence analysis to predict and not just analyze. Competitiveness for enterprises will be pivoted on the big data analysis and prediction. Accuracy of the prediction is very vital for differential positioning.


Big army of Data Scientists and BI experts will be in big demand. 


According to Times of India, India itself will require a minimum of 1,00,000 data scientists in the next couple of years, in addition to scores of data managers and data analysts, to support the fast emerging big data space. 

According to a McKinsey study, the US alone would face a deficit of 1,92,000 data scientists against its requirement of 4,90,000 by 2018. The demand has arisen because global enterprises today use only 5% of the data they create and store. The potential in this field opens up newer business avenues and revenue steams for enterprises, more so for customer-facing businesses and governments.

But the issue is, data scientists are not an easy talent to get. They are a difficult combination of mathematicians, statisticians, analysts and technologists. Though India doesn't have enough of this talent at the moment, tech firms like EMC, Oracle, IBM, under their academic alliance programs, are already working or planning to work with universities in India and overseas to help introduce full-length electives or crash courses on various facets of Big Data. Training outfits like NIIT, Aptech, etc, too are exploring the space. 

Companies in European countries, Asian Countries and the Americas will all not only generate Zetta bytes of data, but also need big data professionals to  process and analyze them. 

What happens when the baby boom generation all become seniors? They create lots of big data, but the job and responsibility of big data analysis is left to the baby bust generation. Thee earth becomes a BIG data factory, when the big data clusters in the big data centers become the BIG BRIAN of the globe!