Big Data is Great, but What You Need is $mart Data
Michael Wu, Ph.D. is Lithium's Principal Scientist of Analytics, digging into the complex dynamics of social interaction and group behavior in online communities and social networks.
Michael was voted a 2010 Influential Leader by CRM Magazine for his work on predictive social analytics and its application to Social CRM.He's a regular blogger on the Lithosphere's Building Community blog and previously wrote in the Analytic Science blog. You can follow him on Twitter or Google+.
A little more than a week ago, I co-presented a session with Mike Fauscette from IDC on “Big Data” at our annual Lithium Network Conference (LiNC). Mike kicked off the session by laying out a very nice framework for understanding data systems. These systems are evolving from a system of record for transactions (system of transactions) to a system of analytics for decision support (system of decisions). But we also need a system of relationships to provide the context in order to make the decisions relevant.
Then I took a swipe at big data—because big data is hyped, and it’s overrated. Although data is the enabler for many interesting analytics and it’s a fertile ground for valuable information and insights, it takes quite a bit of work to extract the information from the data and discover the insights.
As an illustration, I thought it would be nice to show you an example of what big data might look like and contrast that to what I call $mart Data. So I’ve created an infographic that illustrates the difference between the two.
In page 1, you will see the raw, unprocessed, big data that I retrieved from twitter’s search API. That is what big data looks like (or a tiny snippet of it of it).
Now, ready for the surprise? In terms of information, page 2 contains exactly the same information as page 1, on the data on page 2 has been analyzed. Everything you see on page 2 (the $mart data), is derived from the big data on page 1. I simply presented the data in a different way—more relevant and actionable. Moreover, it’s presented in an intuitive way. Once you read the headline, you can pretty quickly understand what the data shows.
As I described at LiNC, $mart data is useful and digestible:
- Useful = relevant and actionable
- Digestible = intuitive and interactive
Too bad, this infographic is an image, and it’s static! Ideally, you should be able to interact with $mart data and get a deeper understanding and more personal appreciation of what the data is trying to tell you.
In a later blog post, I will describe in greater detail precisely what each of those terms means. But for now, I just want to show you and hope you can get a sense of the difference between Big Data and $mart Data. Can you see the difference? Although big data is a key enabler, $mart data is really what you want at the end of the day. Because it is the $mart data (not the big data) that is going to help you make smarter decisions.
A Confession from a Scientist
Finally, I have a confession to make. I know it’s been a while since I blogged. And there are several really good reasons contributed to this.
1. Earlier this year, I been looped into the product and engineering organization and become much more involved with product design as well as the backend architecture for our data and analytic system. Rather than sitting quietly in the corner and think, play with data, write, etc, I often get dragged into endless number of mind numbing meeting, which eats up most of my week. Although the actual number of meeting hour isn’t much compare to most managers, the interruption is definitely a hindrance to the more creative part of my work
2. I’ve just been on the road a lot. If you got a recent out-of-office auto-reply from me, you will get a sense of how bad my schedule is recently
2012-03-07 CSI -Search & Content Findability, Daly City, CA
2012-03-11 – 03-13 Deloitte -On Social Insight+Data, Westlake, TX
2012-03-14 – 03-16 Partner -Science of Gamification, WWW
2012-03-20 – 03-22 CSI -Swarming Reputation Model, Reston, VA
2012-03-29 – 03-30 SocialTech -Gamification for B2B, Seattle, WA
2012-04-14 TEDxSJCA -Pay It Forward, San Jose, CA
2012-04-17 – 04-20 Rotman -Exec sCRM Lecture Sereis, Toronto, Canada
2012-04-23 – 04-25 SugarCon -Science of Relationship, SF, CA
2012-05-01 VatorTV -Gamification for Entrepreneur, Berkeley, CA
2012-05-02 – 05-04 LiNC -Big Data Big Deal, SF, CA
2012-05-16 Stanford -Mobile Health, Stanford, CA
2012-05-17 MMSS -SoLoMoCo Context Inference, SF, CA
2012-05-20 – 05-21 Partner -Community CoCreation, Boston, MA
2012-05-30 – 06-03 IRF -Gamify Travel & Hospitality, San Antonio, TX
2012-06-04 – 06-09 Lithium -French SOS Launch, Paris, France
2012-06-17 – 06-21 e2.0 -Collaboration Experiment, Boston, MA
2012-06-20 – 06/22 MAE -Scaling Social CX, Shanghai, China
2012-06-27 – 06-30 Yumemi -Mobile Gamification, Tokyo, Japan
2012-06-30 – 07-08 PTO -Vacation
Again, this may not look so bad if you are a sales, or biz dev, but I’m a scientist who works with the product and engineering organizations. And I do have responsibilities there too.
3. Lastly, due to LiNC (which took place little more than a week ago), Lithium.com as well as the Lithosphere had face lifts. Consequently, my blog disappeared and its content has been shuffled around. And, we were all busy preparing for LiNC
So, I hope you can understand why my digital footprints seem to have disappeared from social media all of a sudden. I barely have time to eat and sleep, so I hope you will forgive me for not tweeting and writing as much as I used to. I hope I can fix this problem soon. And I am going to need all the help and support I can get from all of you.
Alright, thank you and I will see you again soon with more $mart data blogs. Stay tuned!
You must be a registered user to add a comment here. If you've already registered, please log in. If you haven't registered yet, please register and log in.