Media Releases

Negative citations important to scientific progress, should be tracked, says new study

February 23, 2016

Toron­to, ON — Hav­ing your work men­tioned in some­body else’s research is an impor­tant way for schol­ars to build their aca­d­e­m­ic rep­u­ta­tions. Cit­ing oth­ers’ work is also impor­tant to the cred­i­bil­i­ty of new research.

But what hap­pens if a researcher makes a cita­tion in order to point out flaws or weak­ness­es in a pre­vi­ous study?

Neg­a­tive cita­tions are not nec­es­sar­i­ly a bad thing, says Nico­la Lacetera, an asso­ciate pro­fes­sor of strate­gic man­age­ment at the Uni­ver­si­ty of Toron­to Mis­sis­sauga who is also cross appoint­ed to U of T’s Rot­man School of Man­age­ment. Track­ing those cita­tions can reveal where there is par­tic­u­lar “vital­i­ty” in a research area, espe­cial­ly when there is con­tro­ver­sy among sci­en­tists active in it.

“On the one hand, it might be that a par­tic­u­lar study is just of poor qual­i­ty and reli­a­bil­i­ty,” says Prof. Lacetera, whose find­ings have been pub­lished in the Pro­ceed­ings of the Nation­al Acad­e­my of Sci­ences. “On the oth­er hand, maybe it is good sci­ence that can be improved and so crit­i­cisms are more con­struc­tive. That’s the way sci­ence evolves and becomes bet­ter.”

The effect and fre­quen­cy of cita­tions have been stud­ied before but rarely has research looked at them in detail or attempt­ed to clas­si­fy them.

In this study, advance­ments in the com­put­er sci­ence field of nat­ur­al lan­guage pro­cess­ing allowed the researchers to tar­get neg­a­tive cita­tions in more than 15,000 arti­cles pub­lished over a decade in the Jour­nal of Immunol­o­gy, includ­ing the inci­dence of those cita­tions and details about their pat­terns.

The researchers, includ­ing Prof. Chris­t­ian Catal­i­ni of MIT’s Sloan School of Man­age­ment and Prof. Alexan­der Oet­tl of Geor­gia Tech, who are both grad­u­ates of the Rot­man PhD pro­gram, found that neg­a­tive­ly-cit­ed stud­ies are in the minor­i­ty, are usu­al­ly high qual­i­ty — cit­ed both pos­i­tive­ly and neg­a­tive­ly — and receive more atten­tion than the aver­age, show­ing only a slight drop-off lat­er in the life of the paper.

Their neg­a­tive cita­tions tend to come in the first few years after the orig­i­nal study has been pub­lished. They are often from oth­er sci­en­tists work­ing in a sim­i­lar research area who may even have worked with the cit­ed author on oth­er stud­ies. How­ev­er they don’t tend to come from col­leagues who may be in the same build­ing or down the hall, sug­gest­ing that per­haps those peo­ple share their crit­i­cisms infor­mal­ly with the orig­i­nal author rather than risk­ing the social awk­ward­ness of putting their cri­tique in print.

Besides reveal­ing pat­terns behind knowl­edge pro­duc­tion, the method the researchers devel­oped for their study holds promise for fur­ther explo­ration into the ways sci­en­tists build on each oth­er’s work and the improve­ment of sci­en­tif­ic paper search algo­rithms. Mean­while, the study should be con­so­la­tion to those researchers tar­get­ed for crit­i­cism — it’s the stuff great sci­ence is some­times built on, says Prof. Lacetera.

“No paper is per­fect, includ­ing ours,” he says.

For the lat­est think­ing on busi­ness, man­age­ment and eco­nom­ics from the Rot­man School of Man­age­ment, vis­it:

The Rot­man School of Man­age­ment is locat­ed in the heart of Canada’s com­mer­cial and cul­tur­al cap­i­tal and is part of the Uni­ver­si­ty of Toron­to, one of the world’s top 20 research uni­ver­si­ties. The Rot­man School fos­ters a new way to think that enables our grad­u­ates to tack­le today’s glob­al busi­ness chal­lenges.  For more infor­ma­tion, vis­it


For more infor­ma­tion:

Ken McGuf­fin
Man­ag­er, Media Rela­tions
Rot­man School of Man­age­ment
Uni­ver­si­ty of Toron­to
Voice 416.946.3818
Fol­low Rot­man on Twit­ter @rotmanschool
Watch Rot­man on You Tube