How do we use metrics, and how do they matter
Louis: nuance: some marketing/corporate person wants you to measure your success
prove that you are performing, show that your work is successful
"give me a break"
value, who's an influencer, ...
Rachel: community site "" (now data driven), without content except for "download"
Metrics to help make decisions, to make communities better, stronger
Example Metrix (Web-Analytics):
- source of traffic
- time on site
- keyword times
- paths through the site
- popular content
Commits ("what people do")
- how much is a commit "worth" ? "how much work is happening?"
- file touches, lines, commits , code authorship
Metrics only useful in context. You have to explain them.
"Provide as many you can and explain"
Obamba Reelection campaign as a big example of use of metrics / big data
Have to be careful and accureate in terms of the questions you're putting out and how you represent them.
Explains reasons behind numbers, and reasons, causes for changes.
Ross: "Metrics are another sense that you have".
Jen: "Use the metrics to tell a story"
Awareness, inbound marketing, Marketing goals, increase engagement, specific goals for people on the community
Telling stories about the success / effects of those goals. Metrics should map to the goals.
etsy talk: every last element on their website is A/B tested, continuous integration system with feature flags
Zohar: community health dashboard - tracks everything imaginable, don't care about any of the metrics
only uses the metrics when you need them
have a solid strategy that you understand, have the metrics back the strategy
John: Metrics tell you where there is a fire, when they change dramatically you're able to see the high level overview
Jason (opensource.com): Daily reports, Weekly (shared), Monthly: new followers, comments on blogs, downloads, ...
analysing impact/success of talks: pageviews, comments, video traffic
See example weekly report below - report
Jen: "use listia?" to track detailed video
Jason: metrics decide feature decisions, make pivots, run experiments, constantly evolve / prototype
- Metrics are important
- Narratives are important
- Act on the metrics (immediate basis, respond)
Jason: set goals for new things, if goals are not met scrap it
Kyle: measure how successful actions are, measure response, usefulness of technical content, tweet,publish times, gauge what community said, also less concrete things (touchy, feely, style of writing
Bryan: Metrics are only good for snapshots in time
Zohar: ask the community, use PRs, ask "why", "what is important for you"
Ward: Metrics for everything, "you can use metrics for everything that you want to be supported", constantly challenge everyone others metrics, metrics for decisions - purchase new hardward
Metrics for community "Qlikview dashboard", every transaction, create competition between business units
Louis: two models
- shapre community according to strategy (manipulate)
- metrics provide insights, cater to community
- monitor community's health, heal it
Luke: Targets for metrics, but longterm, but too late after a time, Pivot vs. hitting a target
"what are the right metrics"
how to I work with it, what do I measure, which measure to choose, what do they mean, do people understand the metric, why do choose it
how well anaylse the metrics-data
"it is your job to give your boss the right metrics"
Ross: a metric that everyone wants and no one can get, e.g. in open-source: usage, in the business world it is "licenses"
Metrics backfire, if used to prove the wrong thing
Bryan: SMART, specific metrics, you can track but you cannot really measure
Jason: Leading indiciator (target), track (lagging indicator), cannot directly influence lagging indicator, the measures are helping to understand the impact and make decisions
Metrics that you can actually influence.
Zohar: you need to know what you want to a achieve
Ryan: things that you cannot measure ....
Metrics are not the right way to influence,
- metrics: downloads vs. usage
- people ignore explanations, they just take the numbers
Asheesh Ubuntu: track first contributions, assign mentor on that
regular meetings, you want to take control of this discussion, you don't want a marketing person take control of it
Tools? Template narratives:
- Google Analytics, Conversions, Google Dashboard, Kissmetrics
- R, Gnuplot
Metrics help you to see if something that you worked on actually worked.
Example metrics report
- Open source downloads, Summer reading, and open source project management are just a few topics that were popular last week.
And now, your weekly stats...Here is your weekly snapshot from opensource.com. Check out some of the metrics that our team tracks. And thank you for sharing your favorite content on Hackernews, Reddit, Twitter, identi.ca, Google+, Facebook, Slashdot and many other places!
== From 7/6 - 7/12 ==
Page Views: 34,131 (previous week 37,891)
New registered users: 24 (previous week 27)
Comments: 34 (previous week 21)
Twitter: 12,440 (+110)
Facebook: 8,321 (+48)
Google +1's: 951 (+13)
Top 5 posts (page views)
- Open source downloads are an endangered species (1,465)
- Open source summer reading list (1,088)
- Four tips to transition your open source project into a viable business (709)
- Three open source school management software programs for teachers and student (696)
- Open source project management on the rise (627)