Kevin Gamache on Google +1

Thursday, August 20, 2009

Is there such a thing as too much data?

Sometimes I like to consider the unanswerable questions. The question of "Is there such a thing as too much data?" may seem like a very simple question to answer but I would like to lay out the basis for why I believe this is more a conundrum than it seems.

1) It is all about perspective.
If you were to show up for class in Elementary School and the teacher began running through the "basics" of nuclear physics and there was a test at the end of the class do you think you would be able to pass that class? Can a client who has grown up in the traditional media "age" be given a log file with all the data he/she would need to show how his/her customers interact with their new website and be able to tell what they need to do?

Are these two cases similar?

I believe that they are. In each case the individuals are being given data to which they may or may not have a frame of reference for. Each having their own set of alien terminology with a mix of words that they may have heard before but could quite possible have a very different meaning in each context. Although the minds of each are in a different state of development, each has been trained in some specific tasks. Eventually with enough training each of these sets of people will be able to analyze and make sense of what they see before them.

2) Do we really need it?
Even though as I am writing this I am think of digital media, this really applies to all forms of analysis. Do we really need all the data that we have in our reach? As a society? Yes, we probably do need all the data that we can get. As individual people and companies? Probably not, but being able to pick and choose the data that will tell us what we need to know has started becoming a crucial factor in business and personal decision making. We now have match making services that can give you match making factors to see who you would like to date. We also have free tools that go beyond telling us how many people visited our sites (think of those cheesy visitor counters we used to put on the bottom of our websites). There really is now no way to limit the amount of data available because if limiting begins a group of people or companies will begin to cry foul. They are now reliant on that data to make "informed" decisions. Once again though just like in the first bullet point giving this information to the wrong group too early can lead to false positives and inaccurate decision making criteria.

4) Why is there so much data?
There is so much data because it is now inexpensive to capitalize on the World Wide Web's global community. Research can be done quickly and with a large sample size not previously available. The tools to analyze this data are also coming on strong as well with global open-source collaborations that are added-to and streamlined to perform the tasks that an analyst would have spent weeks or months creating with archaic tools like Excel. Don't get me wrong Excel has grown along with these online tools to maintain its needed status, but compare the first version of Excel with its current version and you will see the addition of many new analytical features that have grown with the times.

4) What happens if the data goes away?
Just like the show on the History Channel Life After People what will the business and personal worlds look like after the data is gone. We will actually have to talk to people face to face to figure out if a new restaurant good. We will have to rely on local data or expensive nationwide data. Without the expansive growth of the Internet this data may not have become the necessity it is today. We would have to rely on the traveling Minstrel to regale the stories of yonder countries and the trophies that they bring.

So for those of you who are allergic to data. BEWARE...your symptoms are going to get worse..a lot worse. Signing off for now! Kevin

No comments:

Contact Me