Google Scholar as replacement for systematic literature searches: good relative recall and precision are not enough
| Titel: | Google Scholar as replacement for systematic literature searches: good relative recall and precision are not enough |
|---|---|
| Verfasser: | ; ; |
| Körperschaft: |
|
| Veröffentlicht: | Freiburg, 2013 |
| Umfang: | Online-Ressource |
| Format: | E-Book |
| Sprache: | Englisch |
X
alg: 41642571 001A $06000:08-09-17 001B $01999:31-05-25 $t01:03:24.000 001D $06000:08-09-17 001U $0utf8 001X $00 002@ $0Oax 002C $aText $btxt $2rdacontent 002D $aComputermedien $bc $2rdamedia 002E $aOnline-Ressource $bcr $2rdacarrier 003@ $0416425712 004P $010.1186/1471-2288-13-131 004U $0urn:nbn:de:bsz:25-freidok-124742 $S0 006G $01138941018 006U $017,O09 $f20170828 007A $01138941018 $aDNB 009Q $S0 $uhttp://nbn-resolving.de/urn:nbn:de:bsz:25-freidok-124742 $xR 009Q $S0 $uhttp://d-nb.info/1138941018/34 $xN 009Q $qzip $S0 $uhttps://freidok.uni-freiburg.de/data/12474 $xH 010@ $aeng 010E $erda 011@ $a2013 $n2013 021A $aGoogle Scholar as replacement for systematic literature searches: good relative recall and precision are not enough 028A $BVerfasser $4aut $71070918385 $8Boeker, Martin [Tp3] $9358892368 028C $BVerfasser $4aut $7111242738 $8Vach, Werner [Tp1] $9145792730 028C $BVerfasser $#Motschall, E. $4aut $71103495550 $8Motschall, Edith [Tp3] $9381557324 029F $BMitwirkender $RAlbert-Ludwigs-Universität Freiburg $#Medizinische Fakultät Freiburg im Breisgau $#Medicinische Fakultät Freiburg im Breisgau $#University of Freiburg, Faculty of Medicine $4ctb $72120526-7 $8Albert-Ludwigs-Universität Freiburg$bMedizinische Fakultät [Tb1] $9043645283 029F $BMitwirkender $RAlbert-Ludwigs-Universität Freiburg $RUniversitätsklinikum Freiburg $#Department für Medizinische Biometrie und Medizinische Informatik $#Medical Center - University of Freiburg, Center for Medical Biometry and Medical Informatics $#Center for Medical Biometry and Medical Informatics $4ctb $71078057273 $8Universitätsklinikum Freiburg$bDepartment für Medizinische Biometrie und Medizinische Informatik [Tb1] $936571884X 033A $pFreiburg $02024338-8 034D $aOnline-Ressource 037A $aBMC Medical Research Methodology. 13 (2013), 131, DOI 10.1186/1471-2288-13-131, issn: 1471-2288 037A $aIN COPYRIGHT http://rightsstatements.org/page/InC/1.0 rs 045E $c020 047I $aAbstract: Background
Recent research indicates a high recall in Google Scholar searches for systematic reviews. These reports raised high expectations of Google Scholar as a unified and easy to use search interface. However, studies on the coverage of Google Scholar rarely used the search interface in a realistic approach but instead merely checked for the existence of gold standard references. In addition, the severe limitations of the Google Search interface must be taken into consideration when comparing with professional literature retrieval tools.
The objectives of this work are to measure the relative recall and precision of searches with Google Scholar under conditions which are derived from structured search procedures conventional in scientific literature retrieval; and to provide an overview of current advantages and disadvantages of the Google Scholar search interface in scientific literature retrieval.
Methods
General and MEDLINE-specific search strategies were retrieved from 14 Cochrane systematic reviews. Cochrane systematic review search strategies were translated to Google Scholar search expression as good as possible under consideration of the original search semantics. The references of the included studies from the Cochrane reviews were checked for their inclusion in the result sets of the Google Scholar searches. Relative recall and precision were calculated.
Results
We investigated Cochrane reviews with a number of included references between 11 and 70 with a total of 396 references. The Google Scholar searches resulted in sets between 4,320 and 67,800 and a total of 291,190 hits. The relative recall of the Google Scholar searches had a minimum of 76.2% and a maximum of 100% (7 searches). The precision of the Google Scholar searches had a minimum of 0.05% and a maximum of 0.92%. The overall relative recall for all searches was 92.9%, the overall precision was 0.13%.
Conclusion
The reported relative recall must be interpreted with care. It is a quality indicator of Google Scholar confined to an experimental setting which is unavailable in systematic retrieval due to the severe limitations of the Google Scholar search interface. Currently, Google Scholar does not provide necessary elements for systematic scientific literature retrieval such as tools for incremental query optimization, export of a large number of references, a visual search builder or a history function. Google Scholar is not ready as a professional searching tool for tasks where structured retrieval methodology is necessary lok: 41642571 3 exp: 41642571 3 1 #EPN 201B/01 $008-09-17 $t23:48:14.741 201U/01 $0utf8 203@/01 $01007118814 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 5 exp: 41642571 5 1 #EPN 201B/01 $008-09-17 $t23:48:14.744 201U/01 $0utf8 203@/01 $01007118822 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 8 exp: 41642571 8 1 #EPN 201B/01 $008-09-17 $t23:48:14.747 201U/01 $0utf8 203@/01 $01007118830 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 10 exp: 41642571 10 1 #EPN 201B/01 $008-09-17 $t23:48:14.751 201U/01 $0utf8 203@/01 $01007118849 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 11 exp: 41642571 11 1 #EPN 201B/01 $008-09-17 $t23:48:14.763 201U/01 $0utf8 203@/01 $01007118857 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 13 exp: 41642571 13 1 #EPN 201B/01 $008-09-17 $t23:48:14.773 201U/01 $0utf8 203@/01 $01007118865 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 20 exp: 41642571 20 1 #EPN 201B/01 $008-09-17 $t23:48:14.777 201U/01 $0utf8 203@/01 $01007118873 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 21 exp: 41642571 21 1 #EPN 201B/01 $008-09-17 $t23:48:14.781 201U/01 $0utf8 203@/01 $01007118881 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 23 exp: 41642571 23 1 #EPN 201B/01 $008-09-17 $t23:48:14.785 201U/01 $0utf8 203@/01 $0100711889X 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 24 exp: 41642571 24 1 #EPN 201B/01 $008-09-17 $t23:48:14.789 201U/01 $0utf8 203@/01 $01007118903 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 25 exp: 41642571 25 1 #EPN 201B/01 $008-09-17 $t23:48:14.796 201U/01 $0utf8 203@/01 $01007118911 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 36 exp: 41642571 36 1 #EPN 201B/01 $013-10-24 $t03:00:41.147 201C/01 $013-10-24 201U/01 $0utf8 203@/01 $01277822646 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a13-10-24 $bl lok: 41642571 49 exp: 41642571 49 1 #EPN 201B/01 $008-09-17 $t23:48:14.799 201U/01 $0utf8 203@/01 $0100711892X 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 51 exp: 41642571 51 1 #EPN 201B/01 $008-09-17 $t23:48:14.806 201U/01 $0utf8 203@/01 $01007118946 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 59 exp: 41642571 59 1 #EPN 201B/01 $008-09-17 $t23:48:14.817 201U/01 $0utf8 203@/01 $01007118962 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 74 exp: 41642571 74 1 #EPN 201B/01 $013-10-24 $t03:00:41.157 201C/01 $013-10-24 201U/01 $0utf8 203@/01 $01277822654 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a13-10-24 $bl lok: 41642571 76 exp: 41642571 76 1 #EPN 201B/01 $008-09-17 $t23:48:14.821 201U/01 $0utf8 203@/01 $01007118970 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 107 exp: 41642571 107 1 #EPN 201B/01 $022-05-20 $t16:50:06.850 201C/01 $022-05-20 201U/01 $0utf8 203@/01 $01107601851 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a22-05-20 $bl lok: 41642571 108 exp: 41642571 108 1 #EPN 201B/01 $022-05-20 $t16:50:06.853 201C/01 $022-05-20 201U/01 $0utf8 203@/01 $0110760186X 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a22-05-20 $bl lok: 41642571 204 exp: 41642571 204 1 #EPN 201B/01 $008-09-17 $t23:48:14.824 201U/01 $0utf8 203@/01 $01007118989 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl lok: 41642571 205 exp: 41642571 205 1 #EPN 201B/01 $008-09-17 $t23:48:14.871 201U/01 $0utf8 203@/01 $01007118997 204U/01 $0urn:nbn:de:bsz:25-freidok-124742 $S0 208@/01 $a08-09-17 $bl
| LEADER | 00000cam a22000002c 4500 | ||
|---|---|---|---|
| 001 | 416425712 | ||
| 003 | DE-603 | ||
| 005 | 20250531010324.0 | ||
| 007 | cr|||||||||||| | ||
| 008 | 170908s2013 xx |||| o| u00||u|eng | ||
| 015 | |a 17,O09 |2 dnb | ||
| 016 | 7 | |2 DE-101 |a 1138941018 | |
| 024 | 7 | |a 10.1186/1471-2288-13-131 |2 doi | |
| 024 | 7 | |a urn:nbn:de:bsz:25-freidok-124742 |2 urn | |
| 035 | |a (DE-599)DNB1138941018 | ||
| 040 | |a DE-603 |b ger |c DE-603 |d DE-603 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 020 |q DE-101 |2 sdnb | ||
| 100 | 1 | |a Boeker, Martin |e Verfasser |4 aut |0 (DE-603)358892368 |0 (DE-588)1070918385 |2 gnd | |
| 245 | 0 | 0 | |a Google Scholar as replacement for systematic literature searches: good relative recall and precision are not enough |
| 264 | 1 | |a Freiburg |c 2013 | |
| 300 | |a Online-Ressource | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a BMC Medical Research Methodology. 13 (2013), 131, DOI 10.1186/1471-2288-13-131, issn: 1471-2288 | ||
| 500 | |a IN COPYRIGHT http://rightsstatements.org/page/InC/1.0 rs | ||
| 520 | |a Abstract: Background<br><br>Recent research indicates a high recall in Google Scholar searches for systematic reviews. These reports raised high expectations of Google Scholar as a unified and easy to use search interface. However, studies on the coverage of Google Scholar rarely used the search interface in a realistic approach but instead merely checked for the existence of gold standard references. In addition, the severe limitations of the Google Search interface must be taken into consideration when comparing with professional literature retrieval tools.<br><br>The objectives of this work are to measure the relative recall and precision of searches with Google Scholar under conditions which are derived from structured search procedures conventional in scientific literature retrieval; and to provide an overview of current advantages and disadvantages of the Google Scholar search interface in scientific literature retrieval.<br><br>Methods<br><br>General and MEDLINE-specific search strategies were retrieved from 14 Cochrane systematic reviews. Cochrane systematic review search strategies were translated to Google Scholar search expression as good as possible under consideration of the original search semantics. The references of the included studies from the Cochrane reviews were checked for their inclusion in the result sets of the Google Scholar searches. Relative recall and precision were calculated.<br><br>Results<br><br>We investigated Cochrane reviews with a number of included references between 11 and 70 with a total of 396 references. The Google Scholar searches resulted in sets between 4,320 and 67,800 and a total of 291,190 hits. The relative recall of the Google Scholar searches had a minimum of 76.2% and a maximum of 100% (7 searches). The precision of the Google Scholar searches had a minimum of 0.05% and a maximum of 0.92%. The overall relative recall for all searches was 92.9%, the overall precision was 0.13%.<br><br>Conclusion<br><br>The reported relative recall must be interpreted with care. It is a quality indicator of Google Scholar confined to an experimental setting which is unavailable in systematic retrieval due to the severe limitations of the Google Scholar search interface. Currently, Google Scholar does not provide necessary elements for systematic scientific literature retrieval such as tools for incremental query optimization, export of a large number of references, a visual search builder or a history function. Google Scholar is not ready as a professional searching tool for tasks where structured retrieval methodology is necessary | ||
| 700 | 1 | |a Vach, Werner |e Verfasser |4 aut |0 (DE-603)145792730 |0 (DE-588)111242738 |2 gnd | |
| 700 | 1 | |a Motschall, Edith |e Verfasser |4 aut |0 (DE-603)381557324 |0 (DE-588)1103495550 |2 gnd | |
| 710 | 2 | |a Albert-Ludwigs-Universität Freiburg |b Medizinische Fakultät |e Mitwirkender |4 ctb |0 (DE-603)043645283 |0 (DE-588)2120526-7 |2 gnd | |
| 710 | 2 | |a Universitätsklinikum Freiburg |b Department für Medizinische Biometrie und Medizinische Informatik |e Mitwirkender |4 ctb |0 (DE-603)36571884X |0 (DE-588)1078057273 |2 gnd | |
| 856 | |u http://nbn-resolving.de/urn:nbn:de:bsz:25-freidok-124742 |x Resolving-System |z kostenfrei | ||
| 856 | |u http://d-nb.info/1138941018/34 |x Langzeitarchivierung Nationalbibliothek |z kostenfrei | ||
| 856 | |q zip |u https://freidok.uni-freiburg.de/data/12474 |x Verlag |z kostenfrei | ||
| 924 | 1 | |9 603 |a (DE-603)1007118814 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118822 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118830 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118849 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118857 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118865 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118873 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118881 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)100711889X |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118903 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118911 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1277822646 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)100711892X |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118946 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118962 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1277822654 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118970 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1107601851 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)110760186X |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118989 |b DE-603 |c HES |d d | |
| 924 | 1 | |9 603 |a (DE-603)1007118997 |b DE-603 |c HES |d d | |
