<div dir="ltr">On Dec 17, 2014, at 9:54 AM, [deleted] wrote on another list:<div class="gmail_quote"><div style="word-wrap:break-word"><div><blockquote type="cite"><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><i>Those that advocate metrics have never, to at least my satisfaction, answered the argument that accuracy in the past does not mean effectiveness in the future, once the game has changed.</i></blockquote></div></blockquote>I recommend Bradley on metaphysics and Hume on <a href="http://plato.stanford.edu/entries/induction-problem/" target="_blank">induction</a>:</div><div><br></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_quote"><div style="word-wrap:break-word"><div><span style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif;font-size:13px;background-color:rgb(255,255,255)">"</span><em style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif;font-size:13px"><a href="https://www.goodreads.com/quotes/1369088-the-man-who-is-ready-to-prove-that-metaphysical-knowledge" style="color:rgb(0,51,102)" target="_blank">The man who is ready to prove that metaphysical knowledge is wholly impossible… is a brother metaphysician with a rival theory</a></em><span style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif;font-size:13px;background-color:rgb(255,255,255)">” Bradley, F. H. (1893) </span><em style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif;font-size:13px">Appearance and Reality</em></div></div></div></blockquote><div class="gmail_quote"><div style="word-wrap:break-word"><div><br></div><div>One could have asked the same question about apples continuing to fall down in future, rather than up (as Hume notes).</div><div><br></div><div>Yes, single metrics can be abused, but not only can abuses be named and shamed when detected, but it becomes harder to abuse metrics when they are part of a multiple, inter-correlated vector, with disciplinary profiles of their normal interactions: someone dispatching a robot to download his papers would quickly be caught out when the usual correlation between downloads and later citations fails to appear. Add more variables and it gets even harder,</div><div><blockquote type="cite"><div style="word-wrap:break-word"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><i>Even if one was able to define a set of metrics that perfectly matches REF2014. The announcement that these metric would be used in REF2020 would immediately invalidate there use.</i></blockquote></div></blockquote>In a weighted vector of multiple metrics like the sample I had listed, it’s no use to a researcher if told in advance that for REF2020 the metric equation will be the following, with the following weights for their particular discipline:</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div>w1(pubcount) + w2(JIF) + w3(cites) +w4(art-age) + w5(art-growth) w6(hits) +w7(cite-peak-latency) + w8(hit-peak-latency) +w9(citedecay) +w10(hitdecay) + w11(hub-score) + w12(authority+score) + w13(h-index) + w14(prior-funding) +w15(bookcites) + w16(student-counts) + w17(co-cites + w18(co-hits) + w19(co-authors) + w20(endogamy) + w21(exogamy) + w22(co-text) + w23(tweets) + w24(tags), +w25(comments) + w26(acad-likes) etc. etc.</div></div></blockquote><div><div><br></div><div>The potential list could be much longer, and the weights can be positive or negative, and varying by discipline.</div><div><br></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div><span style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif;font-size:13px;background-color:rgb(255,255,255)">"</span><em style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif;font-size:13px"><a href="https://www.goodreads.com/quotes/1369088-the-man-who-is-ready-to-prove-that-metaphysical-knowledge" style="color:rgb(0,51,102)" target="_blank">The man who is ready to prove that metric knowledge is wholly impossible… is a brother metrician with rival m</a>etrics</em><span style="background-color:rgb(255,255,255)"><font color="#333333" face="verdana, arial, helvetica, sans-serif"><i>…</i>”</font></span></div></div></blockquote><div><blockquote type="cite"><div style="word-wrap:break-word"><div><blockquote type="cite"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 17, 2014 at 2:26 PM, Stevan Harnad <span dir="ltr"><<a href="mailto:harnad@ecs.soton.ac.uk" target="_blank">harnad@ecs.soton.ac.uk</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="word-wrap:break-word"><span style="font-size:13px;color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif">Steven Hill of HEFCE has posted “an overview of the work HEFCE are currently commissioning which they are hoping will build a robust evidence base for research assessment” in LSE Impact Blog 12(17) 2014 entitled </span><a href="http://blogs.lse.ac.uk/impactofsocialsciences/2014/12/17/time-for-reflection/" style="font-size:13px;font-family:verdana,arial,helvetica,sans-serif" target="_blank">Time for REFlection: HEFCE look ahead to provide rounded evaluation of the REF</a><div dir="ltr" style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px"><div><span style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif"><br></span></div><div><span style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif">Let me add a suggestion, updated for REF2014, that I have made before (unheeded):</span><br style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif"><br style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif"><span style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif">Scientometric predictors of research performance need to be validated by showing that they have a high correlation with the external criterion they are trying to predict. The UK Research Excellence Framework (REF) -- together with the growing movement toward making the full-texts of research articles freely available on the web -- offer a unique opportunity to test and validate a wealth of old and new scientometric predictors, through multiple regression analysis: Publications, journal impact factors, citations, co-citations, citation chronometrics (age, growth, latency to peak, decay rate), hub/authority scores, h-index, prior funding, student counts, co-authorship scores, endogamy/exogamy, textual proximity, download/co-downloads and their chronometrics, tweets, tags, etc.) can all be tested and validated jointly, discipline by discipline, against their REF panel rankings in REF2014. The weights of each predictor can be calibrated to maximize the joint correlation with the rankings. Open Access Scientometrics will provide powerful new means of navigating, evaluating, predicting and analyzing the growing Open Access database, as well as powerful incentives for making it grow faster.</span><br style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif"><br style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif"><span style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif">Harnad, S. (2009) </span><a href="http://eprints.ecs.soton.ac.uk/17142/" style="color:rgb(0,51,102);font-family:verdana,arial,helvetica,sans-serif" target="_blank">Open Access Scientometrics and the UK Research Assessment Exercise</a><span style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif">. </span><em style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif">Scientometrics</em><span style="color:rgb(51,51,51);font-family:verdana,arial,helvetica,sans-serif"> 79 (1) </span><span style="font-family:'Times New Roman';font-size:14px">Also in </span><i style="font-family:'Times New Roman';font-size:14px">Proceedings of 11th Annual Meeting of the International Society for Scientometrics and Informetrics</i><span style="font-family:'Times New Roman';font-size:14px"> 11(1), pp. 27-33, Madrid, Spain. Torres-Salinas, D. and Moed, H. F., Eds. (2007) </span></div><div><span style="font-family:'Times New Roman';font-size:14px"><br></span></div><div><div style="font-family:verdana,arial,helvetica,sans-serif;color:rgb(51,51,51)"><center>See also:<br><a href="http://openaccess.eprints.org/index.php?/archives/1136-The-Only-Substitute-for-Metrics-is-Better-Metrics.html" style="color:rgb(0,51,102)" target="_blank">The Only Substitute for Metrics is Better Metrics</a> (2014)<br>and<br><a href="http://openaccess.eprints.org/index.php?/archives/479-On-Metrics-and-Metaphysics.html" style="color:rgb(0,51,102)" target="_blank">On Metrics and Metaphysics</a> (2008)</center></div></div></div></div></blockquote></div></div>
</blockquote></div><br></div></blockquote></div><br></div></div></div>