Protocolcomparing advantages and disadvantages of Rating Scales, Behavior Observation Scales and Paired Comparison Scales for behavior assessment of competencies in workers. A systematic literature review

  1. Marin-Garcia, Juan A.
  2. Ramirez Bayarri, Lucia
  3. Atares Huerta, Lorena
Journal:
WPOM

ISSN: 1989-9068

Year of publication: 2015

Volume: 6

Issue: 2

Pages: 49-63

Type: Article

DOI: 10.4995/WPOM.V6I2.4032 DIALNET GOOGLE SCHOLAR lock_openDialnet editor

More publications in: WPOM

Sustainable development goals

Abstract

Protocolo: comparación de las ventajas e inconvenientes de las “Rating Scales”, “Behavior Observation Scales”, y “ Paired Comparison Scale” para la evaluación basada en comportamientos, de las competencias de los trabajadores de empresas. Una revisión sistemática de la literatura. This is the protocol for a review and there is no abstract. The objectives are as follows: Identify the characteristics of each of the types of scale and how they differ from each other. Estimate the extent to which they are used by organizations in the assessment of skills of their employees. Summarize the advantages and disadvantages of each. Propose which of them would be more appropriate for assessing the competence of innovation in people (performance evaluation, promotion of workers, recruitment, etc.) and the mode of administration (self, peer, expert assessment).---------------------Resumen Por tratarse de un protocolo para una revisión sistemática no existe un resumen propiamente dicho. En esta revisión nos planteamos los siguientes objetivos: identificar qué características tiene cada uno de los tipos de escala y cómo se diferencian unas de otras. Es-timar el grado en que son usadas, unas u otras, por las organizaciones en la evaluación de competencias de sus empleados. Resumir las ventajas e inconvenientes que presentan cada una de ellas en general y a la hora de evaluar la competencia transversal de innovación en las personas. Proponer cuál de ellas sería más adecuada, atendiendo a los ob-jetivos específicos de evaluación de la competencia de innovación en las personas (evaluación del desempeño, promoción de trabajadores, selección de personal, etc.) y el modo de administración (self, peer, expert assessment).Palabras clave: Protocolo; Revisión Sitemática de literatura; Rating Scales; Behavior Observation Scales; Paired Comparison Scales; evaluación comportamiento.

Bibliographic References

  • Baartman, L. K. J., Bastiaens, T. J., Kirschner, P. A., & van der Vleuten, C. P. M. (2006). The wheel of competency assessment: Presenting quality criteria for competency assessment programs. Studies in Educational Evaluation, 32(2), 153-170. doi:http://dx.doi.org/10.1016/j.stueduc.2006.04.006
  • Boyatzis, R. E. (2008). Competencies in the 21st century. Journal of management development, 27(1), 5-12. doi:doi:10.1108/02621710810840730
  • Bretz, R. D., Milkovich, G. T., & Read, W. (1992). THE CURRENT STATE OF PERFORMANCE-APPRAISAL RESEARCH AND PRACTICE - CONCERNS, DIRECTIONS, AND IMPLICATIONS. Journal of Management, 18(2), 321-352. doi:10.1177/014920639201800206
  • Bryman, A., & Bell, E. (2011). Business Research Methods. USA: Oxford University Press.
  • Charmaz, K. (2006). Constructing grounded theory. A practical guide through qualitative analysis. London: SAGE.
  • Dolan, S. L., Valle Cabrera, R., Jackson, S. E., & Schuler, R. S. (2007). La gestión de los recursos humanos. Cómo atraer, retener y desarrollar con éxito el capital humano en tiempos de transformación. Madrid: Mcgraw-Hill.
  • Dowdy, E., Twyford, J., & Sharkey, J. D. (2013). Methods of Assessing Behavior: Observations and Rating Scales. In D. H. Saklofske, V. L. Schwean, & C. R. Reynolds (Eds.), The Oxford Handbook of Child Psychological Assessment (pp. 623-650): Oxford University Press.
  • DuVernet, A. M., Dierdorff, E. C., & Wilson, M. A. (2015). Exploring Factors That Influence Work Analysis Data: A Meta-Analysis of Design Choices, Purposes, and Organizational Context. Journal of Applied Psychology. doi:10.1037/a0039084
  • Friese, S. (2012). Qualitative Data Analysis with ATLAS.ti. London: SAGE Publications Ltd.
  • Groth-Marnat, G. (2003). Handbook of psichological assessment. Hoboken, New Jersey: John Wiley & Sons, Inc.
  • Hartig, J. (2008). Psychometric models for the assessment of competencies. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts (pp. 69-90). Guttingen: Hogrefe & Huber Publishers.
  • Hatzinger, R., & Dittrich, R. (2012). Prefmod: An R package for modeling preferences based on paired comparisons, rankings, or ratings. Journal of Statistical Software, 48(10). Retrieved from http://www.scopus.com/inward/record.url?eid=2-s2.0-84863314424&partnerID=40&md5=fe41f891ec6e3f17584078fe2d73b5ef
  • Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1, 77-89.
  • Heidemeier, H., & Moser, K. (2009). Self-Other Agreement in Job Performance Ratings: A Meta-Analytic Test of a Process Model. Journal of Applied Psychology, 94(2), 353-370. doi:10.1037/0021-9010.94.2.353
  • Hoyt, W. T., & Kerns, M. D. (1999). Magnitude and moderators of bias in observer ratings: A meta-analysis. Psychological Methods, 4(4), 403-424. doi:10.1037//1082-989x.4.4.403
  • Lenburg, C. (1999). The Framework, Concepts and Methods of the Competency Outcomes and Performance Assessment (COPA) Model Online Journal of Issues in Nursing, 4(2).
  • Lohmann, A., & Prumper, J. (2006). Questionnaire for direct participation in the office (FdP-B) - results concerning its reliability and validity. Zeitschrift fur Arbeits-und Organisationspsychologie, 50(3), 119-134. Retrieved from ://000239034600001
  • Marin-Garcia, J. A., Aragonés Belgran, P., & Melón, G. (2014). Intra-rater and inter-rater consistency of pair wise comparison in evaluating the innovation competency for university students. Working Papers on Operations Management, 5(2), 24-46. doi:http://dx.doi.org/10.4995/wpom.v5i2.3220
  • Marin-Garcia, J. A., Garcia-Sabater, J. J., Maheut, J., Valero-Herrero, M., & Andres-Romano, C. (2012). Gestión de recursos humanos para ingenieros de la rama industrial. Harlow: Pearson Education.
  • Marin-Garcia, J. A., Perez-Peñalver, M. J., & Watts, F. (2013). How to assess innovation competence in services: The case of university students. Direccion y Organizacion(50), 48-62. Retrieved from http://www.revistadyo.com/index.php/dyo/article/viewFile/431/451
  • Marin-Garcia, J. A., Ramirez Bayarri, L., & Andreu Andres, M. A. (2015). Comparación de los métodos de escalas y frecuencia de comportamiento para valorar la competencia de innovación. El punto de vista de alumnos y profesor en el caso de una asignatura de máster. Paper presented at the Congreso In-Red 2015-Universitat Politècnica de València.
  • Marin-Garcia, J. A., & Santandreu-Mascarell, C. (2015). What do we know about rubrics used in higher education? Intangible Capital, 11(1), 118-145. doi:http://dx.doi.org/10.3926/ic.
  • Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The, P. G. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med, 6(7), e1000097. doi:10.1371/journal.pmed.1000097
  • Moore, D. R., Cheng, M. I., & Dainty, A. R. J. (2002). Competence, competency and competencies: performance assessment in organisations. Work Study, 51(6), 314-319. doi:doi:10.1108/00438020210441876
  • Rowe, C. (1995). Clarifying the use of competence and competency models in recruitment, assessment and staff development. Industrial and Commercial Training, 27(11), 12-17. doi:doi:10.1108/00197859510100257
  • Saaty, T. L. (1980). The Analytic Hierarchy Process. New York: McGraw-Hill.
  • Saaty, T. L. (1996). Decision Making with Dependence and Feedback: The Analytic Network Process. Pittsburgh, PA.: RWS Publication.
  • Tziner, A., & Kopelman, R. E. (2002). Is there a preferred performance rating format? A non-psychometric perspective. Applied Psychology, 51(3), 479-503. doi:10.1111/1464-0597.00104
  • Voskuijl, O. F., & Van Sliedregt, T. (2002). Determinants of interrater reliability of job analysis: A meta-analysis. European Journal of Psychological Assessment, 18(1), 52-62. doi:10.1027//1015-5759.18.1.52
Data source: Dialnet