Ignoring Non-English-Language Studies May Bias Ecological Meta-Analyses
Konno, K., Akasaka, M., Koshida, C., Katayama, N., Osada, N., Spake, R. and Amano, T. (2020) Ecology and Evolution.
By reanalysing existing meta-analyses including both English- and Japanese-language studies, we show that effect sizes differ significantly between English- and Japanese-language studies, causing considerable changes in overall mean effect sizes and even their direction when Japanese-language studies are excluded (see figure below). The differences in effect sizes are likely attributable to systematic differences in reported statistical results (language bias in statistical results – see diagram below) as well as study characteristics, particularly taxa and ecosystems (language bias in study characteristics), between English- and Japanese-language studies. This finding has a broad, yet simple implication: future meta-analyses—particularly those conducted at global extents or in regions where English is not widely spoken—should actively search for relevant non-English-language studies, and if appropriate, include them.