Monday, April 16, 2012

Where and How is Big Data generated anyway?


Where and How is Big Data generated anyway?

Every human being, every animal, every automobile, every house, every machine, every commercial and industrial building generate lots of data routinely. The total world population is 7 billion, who live in almost 2 billion abodes consisting of multiple rooms, working, eating, and doing a variety of daily activities in not only their residences, but also work place, community places like schools, restaurants and shops producing lots of data. Similar logic applies for machines and automobiles etc.

Each data is made up of a variety of parameters, many interrelated. The big data ecosystem has to collect, store, analyze, trend plotted and decision made, thus creating distributed centers of "world intelligence". We should use labels like  "world wide databases" World wide intelligence" and "Global Collective Decision Making". This also means that, to collect so much data from all parts of the world, we require "Big Data Rates" flowing through "Big Data Pipes", "Big Data Bandwidth" and "Distributed Big Data Centers". The analytic engines which do "Big Data Science" need to be very fast and perform very complicated algorithms, with some algorithmic engines continuously learning, evolving and morphing.

There will be a lot of data produced in many fields. The examples are given below, only to be indicative and not exhaustive, to give most glaring areas:

Wireless Sensor Networks: The world will be embedded with ever increasing number of sensors of all possible type, in the urban, suburban, rural, remote and even in forests, mountains \, rivers and oceans. The sensors will collect data to include video and photo, temperature, humidity, pressure, flow, GPS position, speed, acceleration, sound, etc. on a continuous basis resulting in large volumes of data, that need to be collected, validated, processed, put in the right data base format, analyzed, trend to be determined, and ultimately decisions to be made. All this, not as individual sensor data, small areas, not offline, but many times real time based on combined and interrelated information at a holistic level. This surely requires a big data paradigm, the right big data analysis tools, all of this with scalability.

No comments:

Post a Comment