Changing the World for the Better with Big Data, The Cloud

Original Post here

It’s an exciting time to be in technology. As IT professionals, we get to say we are the generation that witnessed how Big Data transformed the world. we saw how virtualization revolutionized IT and gave birth to the cloud. But more importantly, we get to say that we are the generation that witnessed how technology helped us feed the world quicker, warm the world quicker, cure the world and maybe even heal the world.

You see, everything we do on a daily basis as IT professionals means something if we can put it in context. They say every action generates a reaction, an event could lead to a sequence of events or could have a trickle effect, but there is so much data out there, so much information and our brain can only process small nuggets, can only address information from a very narrow perspective.

Bottom line is, we are slow, and most research takes years. It takes years because it is necessary to correlate data and make sense of it. So, what if there was a way to correlate data and all sorts of information in real time? If that were possible, would you not be able to make better decisions for your business? Would that not enable you to find cures for disease faster? Or maybe even avert epidemics altogether? Maybe even avert wars? Stop terrorist attacks? The possibilities are limitless. They say society is knowledge, and knowledge is power. Well, Big Data promises to deliver information of all sorts, correlate it and analyze it so that you can make a better, more informed decision in real time.

Enough philosophy — let’s look at an example. What if the shortage in certain types of food, which is high in certain vitamins was responsible for people getting sick? Getting the flu? This type of data research and analysis would take years to gather, to understand and to conclude. The data would be so old and after-the-fact that it would only be useful in university studies and research papers. So, what if that information was available in real time? What can government do with it? How about they alert the communities mostly affected, suggest certain vitamin intakes or perhaps the consumption of other foods high in that vitamin. What about pharmaceutical companies? They could use that data to develop a cure to a flu variation that has not spread yet. But you will say that is all for large enterprise. I say Big Data is for all sizes of enterprise. If you owned a pharmacy, would this information help you? Absolutely it would. Based on Big Data analysis, you might stock more of a certain type of medicine

That’s just one example of many on how Big Data can impact and better our lives, but processing that large amount of data requires a platform capable of handling that amount of data, computing it fast enough and is capable of scaling as fast as we generate data. Today, there are many platforms capable of doing that. Hadoop is one such platform, capable of ingesting structured and unstructured data, Hadoop is quickly becoming the platform of choice and has garnered support from the three largest database developers: Microsoft, Oracle and IBM.

The power of Hadoop is that you can deploy it on standard x86 computers and you can scale it by adding more nodes, a true implementation of grid computing. As a result, Big Data can be leveraged by large and small enterprises. As the data grows and your ability to manage it becomes more challenging, there is always the cloud, ready and able to address these concerns, able to scale, absorbing large quantities of data and unlimited computational resources.

Make no mistake, Big Data is here and enterprises that appreciate it, leverage it, will reap the benefits and grow substantially. Big Data for me is the first time I can literally appreciate the technology beyond the geeky aspect of engineering or putting it all together, and being able to know that our efforts make a difference in the world we live in.

My freeVPS..


Today i signed up for amazons free usage tier for its AWS (Amazon Web Service)
Whats more am leveraging this sign-up for bitnami (unfortunately i missed recent 3 months free VPS off from bitnami) but never-mind, still i own AWS instance for 1 year. Also am hooking up rightscale free edition.

As a new AWS user, i will be able to take advantage of the following services for a year for free

  • “750 hours of Amazon EC2 Linux Micro Instance usage (613 MB of memory and 32-bit and 64-bit platform support) – enough hours to run continuously each month”
  • “750 hours of an Elastic Load Balancer plus 15 GB data processing”
  • “10 GB of Amazon Elastic Block Storage, plus 1 million I/Os, 1 GB of snapshot storage, 10,000 snapshot Get Requests and 1,000 snapshot Put Requests”
  • “5 GB of Amazon S3 storage, 20,000 Get Requests, and 2,000 Put Requests”
  • “30 GB per of internet data transfer (15 GB of data transfer “in” and 15 GB of data transfer “out” across all services except Amazon CloudFront)”
  • “25 Amazon SimpleDB Machine Hours and 1 GB of Storage”
  • “100,000 Requests of Amazon Simple Queue Service”
  • “100,000 Requests, 100,000 HTTP notifications and 1,000 email notifications for Amazon Simple Notification Service”

Amazon Web Services already has far reaching influence. Its suite of products is deeply integrated into everything from web services like Tumblr to products from software companies like AutoDesk.

America’s largest online retailer is best known as, well, a retailer, but a free AWS tier signals an increased interest in web technologies. It’s an obvious statement, but by hooking developers early, Amazon is expanding its footprint on the web. Free Amazon Web Services will create a lower barrier of entry for many budding ideas that could eventually lead to fully realized companies.

What’s not so obvious is that this step is very much in the direction of expanding Amazon’s footprint as a technology company and not just a retailer.

Habits are Good (sometimes..)

People say that habits are good and as well bad.

GOOD if its like you are regular in workout, getting up early in the morning etc..
Bad like back biting, smoking etc..

What are the Good & Bad habits related to computing / IT / Security….etc..?
Well that’s out of scope of this blog to discuss everything the good and bad habits of IT and its related stuff..

Am going to tell you that how GOOD is the practice to take backup and how BAD if you forget it.. and specially how GOOD if its DROPBOX (recently came across, registered and now using it)

Yesterday morning @ my office i downloaded a GOOD WhitePaper (pdf) on Guide to IT Policies and Procedures (i needed them badly to enhance, amend our own) and saved straight away in “My Dropbox” folder (GOOD) and inspite of making a note in my Outlook tasks to print it to take home to have a glance at it, i forgot (BAD) due to hectic schedule (GOD! It happens with sysadmins only).

Back at my home sweet home (where my beautiful wife, two sons naughty & stress buster stays) after dinner before going to bed my usual habit (don’t know whether good or bad) to glance through my corporate mailbox and other things that interest me, a small notification in the system tray grabbed my attention on my incredible personal HP DV6188ea eNotebook.

The notification “Dropbox 0.6.570 All Files up to date”, aahhh… i recalled i had put one pdf file and lo its there is my notebook “My Dropbox” folder though i forgot the hard copy of the file, i managed to glance through the soft copy (something is better than nothing).

My GOOD habit developed recently to save critical, important & always needed files in “My Dropbox” folder helped me forget the concern of forgetting (BAD habit) FILES.

Five cloud computing myths exploded


Cloud computing is one of the most overhyped phenomena to have hit the IT industry in a long time. It is a business model that definitely has its advantages. The trouble is vendors of all sizes and stripes are so desperate for a piece of the cloud action, they are willing to blur distinctions and fudge  definitions for their own ends.

read more