Monthly Archives: March 2016

Gaming Metrics

I talked about “hacking” academic performance metrics at the “Gaming Metrics” workshop on “Innovation & Surveillance in Academic Misconduct” workshop on February 4, 2016 at UC Davis, organized by the “Innovating Communication in Scholarship Project” and supported by the Center for Science and Innovation Studies. It was a blast. I talked about Goodhart’s Law and was making the case that metrics used to set performance standards are doomed to fail because, as Goodhart argued, people change their behavior when metrics are used as public standards. As Lucas argued, contra macro-econometric models serving as policy guides, this means that the underlying causal structure changes because of this kind of use of metrics. My talk was about taking Goodhart’s Law “meta” — to argue that we should explicitly and transparently try to hack metrics in order to learn about social dynamics in the age of digital measurement and have some fun doing it.

One of the terrific things that happened during another conference on a related topic is that while sitting next to Jonathan Eisen, I pitched the idea, which he loved, and he bought  the domain: viXraoib (the inverse of bioarXiv) so we can use it to set up a journal to hack metrics and write about it. So, it’s put up or shut up time.

Advertisements