Original Post here
It’s an exciting time to be in technology. As IT professionals, we get to say we are the generation that witnessed how Big Data transformed the world. we saw how virtualization revolutionized IT and gave birth to the cloud. But more importantly, we get to say that we are the generation that witnessed how technology helped us feed the world quicker, warm the world quicker, cure the world and maybe even heal the world.
You see, everything we do on a daily basis as IT professionals means something if we can put it in context. They say every action generates a reaction, an event could lead to a sequence of events or could have a trickle effect, but there is so much data out there, so much information and our brain can only process small nuggets, can only address information from a very narrow perspective.
Bottom line is, we are slow, and most research takes years. It takes years because it is necessary to correlate data and make sense of it. So, what if there was a way to correlate data and all sorts of information in real time? If that were possible, would you not be able to make better decisions for your business? Would that not enable you to find cures for disease faster? Or maybe even avert epidemics altogether? Maybe even avert wars? Stop terrorist attacks? The possibilities are limitless. They say society is knowledge, and knowledge is power. Well, Big Data promises to deliver information of all sorts, correlate it and analyze it so that you can make a better, more informed decision in real time.
Enough philosophy — let’s look at an example. What if the shortage in certain types of food, which is high in certain vitamins was responsible for people getting sick? Getting the flu? This type of data research and analysis would take years to gather, to understand and to conclude. The data would be so old and after-the-fact that it would only be useful in university studies and research papers. So, what if that information was available in real time? What can government do with it? How about they alert the communities mostly affected, suggest certain vitamin intakes or perhaps the consumption of other foods high in that vitamin. What about pharmaceutical companies? They could use that data to develop a cure to a flu variation that has not spread yet. But you will say that is all for large enterprise. I say Big Data is for all sizes of enterprise. If you owned a pharmacy, would this information help you? Absolutely it would. Based on Big Data analysis, you might stock more of a certain type of medicine
That’s just one example of many on how Big Data can impact and better our lives, but processing that large amount of data requires a platform capable of handling that amount of data, computing it fast enough and is capable of scaling as fast as we generate data. Today, there are many platforms capable of doing that. Hadoop is one such platform, capable of ingesting structured and unstructured data, Hadoop is quickly becoming the platform of choice and has garnered support from the three largest database developers: Microsoft, Oracle and IBM.
The power of Hadoop is that you can deploy it on standard x86 computers and you can scale it by adding more nodes, a true implementation of grid computing. As a result, Big Data can be leveraged by large and small enterprises. As the data grows and your ability to manage it becomes more challenging, there is always the cloud, ready and able to address these concerns, able to scale, absorbing large quantities of data and unlimited computational resources.
Make no mistake, Big Data is here and enterprises that appreciate it, leverage it, will reap the benefits and grow substantially. Big Data for me is the first time I can literally appreciate the technology beyond the geeky aspect of engineering or putting it all together, and being able to know that our efforts make a difference in the world we live in.
