A Data Driven Society – Part 2

Last post I listed a bunch of articles discussing just a small fraction of the amount of information being collected on us and some of the issues society is raising about it. There is actually a whole industry out there of companies that buy and sell and package our data. 10 years ago this was called data mining but today it’s called either big data or data analytics. Different name, same idea.

Three things have changed from 10 years ago.

1. The quantity of data has vastly increased. As more and more of our lives move online we are leaving an increasingly large data footprint. Much of this data is now automatically generated and sits in a server waiting to be analyzed. With the dawning of the “cloud” our data is not our own anymore. It used to be you would store your files on your PC or Mac but today much of that information is stored online. Storage technology has gotten very cheap.

2. The ability to process the data has vastly increased. Moore’s Law has not stopped. Computers today are hundreds of times faster than they were 10 years ago. This permits more data to be processed but more importantly it allows new forms of analysis. We can now program a computer to watch video and images and pick out faces. This then becomes a new data point. As the article said a person in London is on a camera 300 times a day. This didn’t used to matter because most likely nobody was watching, and even if they were they didn’t know who you were. Today the computer is watching and while nobody might care right now, it is keeping a database of where you were. Pretty soon it will also understand what you are doing. Imagine using this network of cameras to watch what kind of food you buy, what your emotional state is based on your facial expressions, and who you hang out with. All of this will soon be possible. Thus, more data is created from existing data sources.

3. Who is collecting the data is also changing. It used to be that surveillance was the domain of governments only but today surveillance technology is getting cheap. We are also hooking various sensors to our smart phones, thus generating new data, some of it automatically. For example, you may have heard of Vicky Davis who goes by the twitter name of @coolcatteacher (if you aren’t following her and are a teacher, you should definitely check her out). Anyway a few years ago I was watching my Twitter stream and saw her check in via Foursquare. That is an app where you “check in” at a particular physical location you visit. If you have the most check ins over a certain period of time you become the mayor of that space. The tweet I saw was announcing that she was now the mayor of the Denver Convention Center… bathroom. After I stopped laughing hysterically I tweeted her back and asked he if she was proud of that. Turns out she had no idea it even happened! Her phone checked in automatically and also automatically tweeted out the mayorship. There are now gadgets you can wear that will monitor your heart rate, activity levels and so on and store it on your phone.

So who gets that data? Who wants it besides you and what will it tell them? Tomorrow I’ll explore some of the implications of big data and tie it back to education.

 

Here’s a video showing a whole year’s worth of Foursquare checkins for New York and Tokyo.