Knowledge level of effect size statistics, con fi dence intervals and meta-analysis in Spanish academic psychologists

  1. Laura Badenes-Ribera 1
  2. Dolores Frias-Navarro 1
  3. Marcos Pascual-Soler 2
  4. Héctor Monterde-i-Bort 1
  1. 1 University of Valencia (España)
  2. 2 ESIC Business & Marketing School (Valencia, España)
Revista:
Psicothema

ISSN: 0214-9915

Año de publicación: 2016

Volumen: 28

Número: 4

Páginas: 448-456

Tipo: Artículo

DIALNET GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: Psicothema

Resumen

Antecedentes: el movimiento de la reforma estadística y la Asociación Americana de Psicología (APA) defienden el uso de estimadores del tamaño del efecto y sus intervalos de confianza, así como la interpretación de la significación clínica de los hallazgos. Método: se realizó una encuesta a psicólogos académicos sobre su conducta en el diseño y realización de estudios. La muestra estuvo compuesta de 472 participantes (45,8% hombres). La media en años como académico fue 13,56 (DT= 9,27). Resultados: el uso de estadísticos del tamaño del efecto se está generalizando, también la consideración de los estudios meta-analíticos. Sin embargo, persisten prácticas estadísticas inadecuadas. Se mantiene un modelo tradicional de comportamiento metodológico basado en las pruebas de significación estadística, predominio de la d de Cohen, y del R2/η2 no ajustado que no son inmunes a la existencia de outliers y violaciones de las asunciones y un escaso uso de los intervalos de confianza de los estadísticos del tamaño del efecto. Conclusiones: se concluye con recomendaciones para la mejora de la práctica estadística.

Referencias bibliográficas

  • Algina, J., Keselman, H. J., & Penfield, R. D. (2005). An alternative to Cohen’s standardized mean difference effect size: A robust parameter and confidence interval in the two independent groups case. Psychological Methods, 10, 317-328.
  • Allison, D. B., Brown, A. W., George, B. J., & Kaiser, K. A. (2016). Reproducibility: A tragedy of errors. Nature, 530, 27-29.
  • American Psychological Association (2001). Publication Manual of the American Psychological Association (5th ed.). Washington, DC: American Psychological Association.
  • American Psychological Association (2010). Publication Manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.
  • Anzures-Cabrera, J., & Higgins, J. P. T. (2010). Graphical displays for meta-analysis: An overview with suggestions for practice. Research Synthesis Methods, 1, 66-80.
  • Arnau, J., Bendayan, R., Blanca, M. J., & Bono, R. (2013). The effect of skewness and kurtosis on the robustness of linear mixed models. Behavior Research Methods, 45, 873-879.
  • Badenes-Ribera, L., Frias-Navarro, D., Monterde-i-Bort, H., & PascualSoler, M. (2015). Interpretation of the p value. A national survey study in academic psychologists from Spain. Psicothema, 27, 292-295.
  • Badenes-Ribera, L., Frias-Navarro, D., Monterde-i-Bort, H., & PascualSoler, M. (2013, February). Informar e interpretar el tamaño del efecto en Psicología y Educación [Reporting and interpreting the effect size in Psychology and Education]. Paper presented at the XIV Congreso Virtual de Psiquiatria.com. Interpsiquis, Palma de Malloca, Spain.
  • Balluerka, N., Gómez, J., & Hidalgo, D. (2005). The controversy over null hypothesis significance testing revisited. Methodology, 1, 55-70.
  • Balluerka, N., Vergara, A. I., & Arnau, J. (2009). Calculating the main alternatives to Null Hypothesis Significance testing in between subject experimental designs. Psicothema, 21, 141-151.
  • Beyth-Maron, R., Fidler, F., & Cumming, G. (2008). Statistical cognition: Towards evidence-based practice in statistics and statistics education. Statistics Education Research Journal, 7, 20-39.
  • Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. (2009). Introduction to Meta-analysis. Chichester: Wiley.
  • Botella, J., & Sánchez-Meca, J. (2015). Meta-análisis en ciencias sociales y de la salud [Meta-analysis in social and health sciences]. Madrid, Spain: Sintesis.
  • Campitelli, G. (2015). Answering research questions without calculating the mean. Frontiers in Psychology, 6, 1379-1381.
  • Caperos, J. M., & Pardo, A. (2013). Consistency errors in p-values reported in Spanish psychology journals. Psicothema, 25, 408-414.
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
  • Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York, NY: Routledge
  • Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25, 7-29.
  • Cumming, G., Fidler, F., Kalinowski, P, & Lai, J. (2012). The statistical recommendations of the American Psychological Association publication manual: Effect sizes, confidence intervals, and metaanalysis. Australian Journal of Psychology, 64, 138-146.
  • Cumming, G., Fidler, F., Leonard, M., Kalinowski, P, Christiansen, A., Kleining, A., Lo, L, McMenamin & Wilson, S. (2007). Statistical reform in Psychology: Is anything changing? Psychological Science, 18, 230-232.
  • Earp, B. D., & Trafi mow, D. (2015). Replication, falsifications, and the crisis of confidence in social psychology. Frontiers in Psychology, 6, 621
  • Ellis, P. D. (2010). The essential guide to effect size. Statistical power, meta-analysis, and the interpretation of research results. New York, NY: Cambridge.
  • Erceg-Hurn, D. M., & Mirosevich, V. M. (2008). Modern robust statistical methods: An easy way to maximize the accuracy and power of your research. American Psychologist, 63(7), 591-601.
  • Frias-Navarro, D. (2011). Reforma estadística. Tamaño del efecto. En D. Frias-Navarro (Ed.), Técnica estadística y diseño de investigación [Statistical technique and research design]. Valencia, Spain: Palmero Ediciones.
  • Frias-Navarro, D., Monterde-i-Bort, H., Pascual-Soler, M., Pascual-Llobell, J., & Badenes-Ribera, L. (2012, July). Improving statistical practice in clinical production: A case Psicothema. Poster presented at the V European Congress of Methodology, Santiago de Compostela, Spain.
  • Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology, General, 141, 2-18.
  • García, J., Ortega, E., & De la Fuente, L. (2008). Tamaño del efecto en las revistas de Psicología indizadas en Redalyc [Effect size in psychology journals indexed in redalyc]. Informes Psicológicos, 10, 173-188.
  • Grissom, R. J., & Kim, J. J. (2001). Review of assumptions and problems in the appropriate conceptualization of effect size. Psychological Methods, 6, 135-146.
  • Grissom, R. J., & Kim, J. J. (2012). Effect sizes for research. New York, USA: Routledge
  • Henson, R. K. (2006). Effect-size measures and meta-analytic thinking in counseling psychology research. Counseling Psychologist, 34, 601-629.
  • Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8): e124.
  • Kepes, S., Banks, G. C., & Oh, I.-S. (2014). Avoiding bias in publication bias research: The value of “null” findings. Journal of Business and Psychology, 29, 183-203.
  • Keselman, H. J., Algina, J., Lix, L. M., Wilcox, R. R., & Deerin, K. N. (2008). A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes. Psychological Methods, 13, 110-129.
  • Kirk, R. E. (2001). Promoting good statistical practices: Some suggestions. Educational and Psychological Measurement, 61, 213-218.
  • Kline, R. E. (2011). Principles and practice of structural equations modelling (3nd ed.). Guilford Press, London.
  • Kline, R. B. (2013). Beyond significance testing: Statistic reform in the behavioral sciences. Washington, DC: American Psychological Association.
  • Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 1-11.
  • Monterde-i-Bort, H., Frias-Navarro, D., & Pascual-Llobell, J. (2010). Uses and abuses of statistical significance tests and other statistical resources: A comparative study. European Journal of Psychology of Education, 25, 429-447.
  • Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301.
  • Nuzzo, R. (2014). Statistical errors: P values, the “gold standard” of statistical validity, are not as reliable as many scientists assume. Nature, 130, 150-152.
  • Palmer, A., & Sesé, A. (2013). Recommendations for the use of statistics in clinical and health psychology. Clínica y Salud, 24, 47-54.
  • Peng, C.-Y J., Chen, L.-T, Chiang, H., & Chiang, Y. (2013). The impact of APA and AERA guidelines on effect size reporting. Educational Psychology Review, 25, 157-209.
  • Peng, C.-Y. J., & Chen, L.-T. (2014). Beyond Cohen’s d: Alternative effect size measures for between subject designs. The Journal of Experimental Education, 82, 22-50.
  • Rosnow, R. L., & Rosenthal, R. (2009). Effect sizes: Why, when, and how to use them. Journal of Psychology, 217, 6-14.
  • Sánchez-Meca, J., & Botella, J. (2010). Revisiones sistemáticas y metaanálisis: herramientas para la práctica profesional [Systematic reviews and meta-analyzes: Tools for professional practice]. Papeles del Psicólogo, 31, 7-17.
  • Sánchez-Meca, J., & Marín-Martínez, F. (2010). Meta-analysis in psychological research. International Journal of Psychological Research, 3, 150-162.
  • Sesé, A., & Palmer, A. (2012). El uso de la estadística en psicología clínica y de la salud a revisión [The current use of statistics in clinical and health psychology under review]. Clínica y Salud, 23, 97-108.
  • Spellman, B. A. (2015). A Short (Personal) Future History of Revolution 2.0. Perspectives on Psychological Science, 10, 886-899.
  • Sun, S., Pan, W., & Wang, L. L. (2010). A comprehensive review of effect size reporting and interpreting practices in academic journals in education and psychology. Journal of Educational Psychology, 10, 989-1004.
  • Thompson, B. (2002). What future quantitative social science research could look like: Confidence intervals for effect size. Educational Researcher, 31, 25-32.
  • Thompson, B. (2007). Effect sizes, confi dence intervals, and confidence intervals for effect sizes. Psychology in the Schools, 44, 423-432.
  • Vacha-Haase, T. (2001) Statistical significance should not be considered one of life’s guarantees: Effect sizes are needed. Educational and Psychological Measurement, 61, 219-224.
  • Wang, Z., & Thompson, B. (2007). Is the Pearson r2 biased, and if so, what is the best correction formula? Journal of Experimental Education, 75, 109-125.
  • Wilcox, R. R., & Keselman, H. J. (2003). Modern robust data analysis methods: Measures of central tendency. Psychological Methods, 8, 254-274.
  • Wilkinson, L., & the Task Force on Statistical Inference (1999). Statistical methods in psychology journals: Guidelines and explanations. The American Psychologist, 54, 594-604.
Fuente de los datos: Dialnet