Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems

Henrique Bond; Marja Rudenhed; Eva Bergquist; Anna-Karin Andershed; Henrik Andershed

doi:doi:10.11648/j.ajap.20130202.11

| Peer-Reviewed

Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems

Henrique Bond, Marja Rudenhed, Eva Bergquist, Anna-Karin Andershed, Henrik Andershed

Published in American Journal of Applied Psychology (Volume 2, Issue 2)

Received: 7 March 2013 Accepted: Published: 10 March 2013

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

Behavioral problems in childhood have been associated with conduct problems later in life. Thus, it is essen-tial that youths with or at risk for conduct problems receive the help they need on time. Therefore, youth with or at risk for conduct problem must receive effective risk-need assessments and intervention plans regardless the person who conducts the assessment. ESTER-assessment is a structured, computer-aided, risk-need instrument developed for assessing youth (0-18) with or at risk for conduct problems. It uses a five-step response scale to assess 19 research-based risk and protective factors and the present study tests the inter-rater reliability of these 19 factors. This was done by comparing the assessments conducted by two independent raters who assessed the file information of 30 girls (mean age = 16.9) who had been incarcerated due to psychosocial problems, criminality and/or drug abuse. Results showed fair to good agreement for the majority of the factors via intra-class correlations and percentage agreement varied on the 19 factors from 24.1 to 80.8 % for exact agreement and from 72.2 to 96.7 % for exact agreement or difference by one step on the response scale. We conclude that it is possible to gain acceptable to excellent inter-rater reliability in assessing risk and protective factors via ESTER-assessment.

Published in	American Journal of Applied Psychology (Volume 2, Issue 2)
DOI	10.11648/j.ajap.20130202.11
Page(s)	16-21
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2024. Published by Science Publishing Group

Keywords

Risk-Need Assessment, Conduct Problems, Antisocial Behavior, Youth, Inter-Rater Reliability

References

[1]	Frick, P. J., & Viding, E. (2009). Antisocial Behavior from a Developmental psychopathology Perspective. Development and Psychopathology, 21, 1111-1131.
[2]	Krohn, M. D., Thornberry, T. P., Rivera, C., & Le Blanc, M. (2001). Later delinquency careers. In R. Loeber & D. P. Far-rington (Eds.), Child delinquents (pp. 67-94). Thousand Oaks, CA: Sage.
[3]	Snyder, H. N. (2001). Epidemiology of Official Offending. In R. Loeber & D. P. Farrington (Red.), Child delinquents (pp. 25-46). Thousand Oaks, CA: Sage.
[4]	Moffitt, T. E., & Scott, S. (2008). Conduct disorders of childhood and adolescence. In M. Rutter, D. Bishop, D. Pine, S. Scott, J. Stevenson, E. Taylor, & A. Thapar (Eds.), Rut-ter´s Child and Adolescent Psychiatry, (5th ed., pp. 543-564). Oxford: Blackwell Publishing.
[5]	Odgers, C., Caspi, A., Broadbent, J. M., Dickson, N., Han-cox, R. J., & Harrington, H. (2007). Prediction of Differential Adult Health Burden by Conduct Problem Subtypes in Males. Archives of General Psychiatry, 64, 476-484.
[6]	Andershed, H., & Andershed, A-K. (2010). Risk-need as-sessment for youth with or at risk for conduct problems: In-troducing the assessment system ESTER. Procedia Social and Behavioral Journal, 5, 377-383.
[7]	Andrews, D. A., Bonta, J., & Wormith, J. S. (2006). The Recent Past and Near Future of Risk and/or Need Assess-ment. Crime & Delinquency, 52, 7-27.
[8]	Dowden, C., & Andrews, D. A. (2003). Does family inter-vention work for delinquents? Results of a meta-analysis. Canadian Journal of Criminology and Criminal Justice, 45, 327-342.
[9]	Andrews, D. A., Bonta, J., & Hoge, R. D. (1990). Classifica-tion for effective rehabilitation- Rediscovering Psychology. Criminal Justice and Behavior, 17, 19-52.
[10]	Andrews, D. A., & Bonta, J. (2010). Rehabilitating criminal justice policy and practice. Psychology, Public Policy, and Law, 16, 39-55.
[11]	Borum, R., Bartel, P., & Forth, A. (2002). Manual for the Structured Assessment for Violence Risk in Youth (SAVRY), Consultation edition, Version 1. Tampa: University of South Florida.
[12]	Augimeri, L. K., Webster, C. D., Koegl, C. J., & Levene, K. S. (1998). Early Assessment Risk List for Boys: EARL-20B. Version 1: Consultation edition. Toronto, Canada: Earlscourt Child and Family Centre.
[13]	Levene, K. S., Augimeri, L. K., Pepler, D. J., Walsh, M. M., Webster, C. D., Koegl C. J. (2001). Early Assessment Risk List for Girls (EARL-21G). Version 1 - Consultation Edition. Toronto, Canada: Earlscourt Child and Family Centre.
[14]	Vincent, G. M., Guy, L. S., Fusco, S. L., & Gerhenson, B. G. (2011). Field Reliability of the SAVRY with Juvenile Proba-tion Officers: Implications for Training. Law and Human Behavior.doi: 10.1007/s10979-011-9284-2.
[15]	Hrynkiw-Augimeri, L. K. (2005). Aggressive and antisocial young children: Risk prediction, assessment and management utilizing the Early Assessment Risk List for Boys (EARL-20B). Unpublished doctoral dissertation, Ontario Institute for Studies in Education, University of Toronto, Ontario, Canada.
[16]	Catchpole, R., & Gretton, H. (2003). The predictive validity of risk assessment with violent young offenders: A 1-year examination of criminal outcome. Criminal Justice & Beha-vior, 30, 688-708.
[17]	Dolan, M. C., & Rennie, C. E. (2008). The Structured As-sessment of Violence Risk in Youth as a Predictor of Reci-divism in a United Kingdom Cohort of Adolescent Offenders With Conduct Disorder. Psychological Assessment, 20, 35-46.
[18]	Enebrink, P., Långström, N., Hultén, A., & Gumpert, C. H. (2006). Swedish Validation of the Early Assessment Risk List for Boys (EARL 20B), a decision aid for use with child-ren presenting with conduct-disrordered behaviour. Nordic Journal of Psychiatry, 60, 438-446.
[19]	Lodewijks, H. P. B., Doreleijers, T. A. H., de Ruiter, C., & Borum, R. (2008). Predictive validity of the Structured As-sessment of Violence Risk in Youth (SAVRY) during resi-dential treatment. International Journal of Law and Pyschiatry, 31, 263-271.
[20]	Meyers, J., & Schmidt, F. (2008). Predictive Validity of the Structured Assessment for Violence Risk in Youth (SAVRY) with Juvenile Offenders. Criminal Justice and Behavior 35, 696-709.
[21]	Viljoen. J. L., Scalora, M., Cuadra, L., Bader, S., Chávez, V., Ullman, D., Lawrence, L. (2008). Assessing risk for violence in adolescents who have sexually offended. A Comparison of the J-SOAP-II, J-SORRAT-II, and SAVRY. Criminal Justice and Behavior, 35, 5-23.
[22]	Hrynkiw-Augimeri, L. K. (1998). Assessing risk for violence in boys: A preliminary risk assessment study using the Early Assessment Risk List for Boys (EARL-20B). Unpublished master’s thesis, Ontario Institute for Studies in Education, University of Toronto, Ontario, Canada.
[23]	Levene, K. S., Walsh, M. M., Augimeri, L. K., & Pepler, D. J. (2004). Linking Identification and Treatment of Early Risk Factors for Female Delinquency. In M. M, Moretti, C. L., Odgers, & M. A., Jackson (Eds.), Girls and Agression: Contibuting Factors and Intervention Principles (pp. 147-163). New York: Kluwer Academic/Plenum Publishers.
[24]	Andershed, H., Fredriksson, J., Engelholm, K., Ahlberg, R., Berggren, S., & Andershed, A-K. (2010). Initial test of the new risk-need assessment instrument for youths with or at risk for conduct problems: ESTER-assessment. Procedia Social and Behavioral Sciences, 5, 488-492.
[25]	Friedman, A. S., & Utada, A. (1989). A method for diagnos-ing and planning the treatment of adolescent drug abusers: The Adolescent Drug Abuse Diagnosis (ADAD) instrument. Journal of Drug Education, 19, 285-312.
[26]	Wechsler, D. (2003). The Wechsler Intelligence Scale for Children (4rd ed.). San Antonio, TX: The Psychological Corporation.
[27]	Cicchetti, D. V. (1994). Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment instruments in Psychology. Psychological Assessment, 6, 284-290.
[28]	Svensson, E. (2001). Guidelines to Statistical Evaluation of Data from Rating Scales and Questionnaires. Journal of Re-habilitation Medicine, 33, 47-48.

Cite This Article

Plain Text BibTeX RIS

APA Style

Henrique Bond, Marja Rudenhed, Eva Bergquist, Anna-Karin Andershed, Henrik Andershed. (2013). Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems. American Journal of Applied Psychology, 2(2), 16-21. https://doi.org/10.11648/j.ajap.20130202.11

Copy | Download

ACS Style

Henrique Bond; Marja Rudenhed; Eva Bergquist; Anna-Karin Andershed; Henrik Andershed. Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems. Am. J. Appl. Psychol. 2013, 2(2), 16-21. doi: 10.11648/j.ajap.20130202.11

Copy | Download

AMA Style

Henrique Bond, Marja Rudenhed, Eva Bergquist, Anna-Karin Andershed, Henrik Andershed. Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems. Am J Appl Psychol. 2013;2(2):16-21. doi: 10.11648/j.ajap.20130202.11

Copy | Download

@article{10.11648/j.ajap.20130202.11,
  author = {Henrique Bond and Marja Rudenhed and Eva Bergquist and Anna-Karin Andershed and Henrik Andershed},
  title = {Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems},
  journal = {American Journal of Applied Psychology},
  volume = {2},
  number = {2},
  pages = {16-21},
  doi = {10.11648/j.ajap.20130202.11},
  url = {https://doi.org/10.11648/j.ajap.20130202.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajap.20130202.11},
  abstract = {Behavioral problems in childhood have been associated with conduct problems later in life. Thus, it is essen-tial that youths with or at risk for conduct problems receive the help they need on time. Therefore, youth with or at risk for conduct problem must receive effective risk-need assessments and intervention plans regardless the person who conducts the assessment. ESTER-assessment is a structured, computer-aided, risk-need instrument developed for assessing youth (0-18) with or at risk for conduct problems. It uses a five-step response scale to assess 19 research-based risk and protective factors and the present study tests the inter-rater reliability of these 19 factors. This was done by comparing the assessments conducted by two independent raters who assessed the file information of 30 girls (mean age = 16.9) who had been incarcerated due to psychosocial problems, criminality and/or drug abuse. Results showed fair to good agreement for the majority of the factors via intra-class correlations and percentage agreement varied on the 19 factors from 24.1 to 80.8 % for exact agreement and from 72.2 to 96.7 % for exact agreement or difference by one step on the response scale. We conclude that it is possible to gain acceptable to excellent inter-rater reliability in assessing risk and protective factors via ESTER-assessment.},
 year = {2013}
}

Copy | Download

TY  - JOUR
T1  - Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems
AU  - Henrique Bond
AU  - Marja Rudenhed
AU  - Eva Bergquist
AU  - Anna-Karin Andershed
AU  - Henrik Andershed
Y1  - 2013/03/10
PY  - 2013
N1  - https://doi.org/10.11648/j.ajap.20130202.11
DO  - 10.11648/j.ajap.20130202.11
T2  - American Journal of Applied Psychology
JF  - American Journal of Applied Psychology
JO  - American Journal of Applied Psychology
SP  - 16
EP  - 21
PB  - Science Publishing Group
SN  - 2328-5672
UR  - https://doi.org/10.11648/j.ajap.20130202.11
AB  - Behavioral problems in childhood have been associated with conduct problems later in life. Thus, it is essen-tial that youths with or at risk for conduct problems receive the help they need on time. Therefore, youth with or at risk for conduct problem must receive effective risk-need assessments and intervention plans regardless the person who conducts the assessment. ESTER-assessment is a structured, computer-aided, risk-need instrument developed for assessing youth (0-18) with or at risk for conduct problems. It uses a five-step response scale to assess 19 research-based risk and protective factors and the present study tests the inter-rater reliability of these 19 factors. This was done by comparing the assessments conducted by two independent raters who assessed the file information of 30 girls (mean age = 16.9) who had been incarcerated due to psychosocial problems, criminality and/or drug abuse. Results showed fair to good agreement for the majority of the factors via intra-class correlations and percentage agreement varied on the 19 factors from 24.1 to 80.8 % for exact agreement and from 72.2 to 96.7 % for exact agreement or difference by one step on the response scale. We conclude that it is possible to gain acceptable to excellent inter-rater reliability in assessing risk and protective factors via ESTER-assessment.
VL  - 2
IS  - 2
ER  -

Copy | Download

Author Information

Henrique Bond

School of Law, Psychology, and Social Work, ?rebro University, SE-701 82, Sweden
Marja Rudenhed

School of Law, Psychology, and Social Work, ?rebro University, SE-701 82, Sweden
Eva Bergquist

School of Law, Psychology, and Social Work, ?rebro University, SE-701 82, Sweden
Anna-Karin Andershed

School of Law, Psychology, and Social Work, ?rebro University, SE-701 82, Sweden
Henrik Andershed

School of Law, Psychology, and Social Work, ?rebro University, SE-701 82, Sweden

Download PDF

Sections

Plain Text BibTeX RIS

APA Style

Henrique Bond, Marja Rudenhed, Eva Bergquist, Anna-Karin Andershed, Henrik Andershed. (2013). Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems. American Journal of Applied Psychology, 2(2), 16-21. https://doi.org/10.11648/j.ajap.20130202.11

Copy | Download

ACS Style

Henrique Bond; Marja Rudenhed; Eva Bergquist; Anna-Karin Andershed; Henrik Andershed. Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems. Am. J. Appl. Psychol. 2013, 2(2), 16-21. doi: 10.11648/j.ajap.20130202.11

Copy | Download

AMA Style

Henrique Bond, Marja Rudenhed, Eva Bergquist, Anna-Karin Andershed, Henrik Andershed. Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems. Am J Appl Psychol. 2013;2(2):16-21. doi: 10.11648/j.ajap.20130202.11

Copy | Download

@article{10.11648/j.ajap.20130202.11,
  author = {Henrique Bond and Marja Rudenhed and Eva Bergquist and Anna-Karin Andershed and Henrik Andershed},
  title = {Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems},
  journal = {American Journal of Applied Psychology},
  volume = {2},
  number = {2},
  pages = {16-21},
  doi = {10.11648/j.ajap.20130202.11},
  url = {https://doi.org/10.11648/j.ajap.20130202.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajap.20130202.11},
  abstract = {Behavioral problems in childhood have been associated with conduct problems later in life. Thus, it is essen-tial that youths with or at risk for conduct problems receive the help they need on time. Therefore, youth with or at risk for conduct problem must receive effective risk-need assessments and intervention plans regardless the person who conducts the assessment. ESTER-assessment is a structured, computer-aided, risk-need instrument developed for assessing youth (0-18) with or at risk for conduct problems. It uses a five-step response scale to assess 19 research-based risk and protective factors and the present study tests the inter-rater reliability of these 19 factors. This was done by comparing the assessments conducted by two independent raters who assessed the file information of 30 girls (mean age = 16.9) who had been incarcerated due to psychosocial problems, criminality and/or drug abuse. Results showed fair to good agreement for the majority of the factors via intra-class correlations and percentage agreement varied on the 19 factors from 24.1 to 80.8 % for exact agreement and from 72.2 to 96.7 % for exact agreement or difference by one step on the response scale. We conclude that it is possible to gain acceptable to excellent inter-rater reliability in assessing risk and protective factors via ESTER-assessment.},
 year = {2013}
}

Copy | Download

TY  - JOUR
T1  - Further Testing of the Inter-rater Reliability of ESTER-assessment – a Risk-need Assessment Instrument for Youths with or at Risk for Conduct Problems
AU  - Henrique Bond
AU  - Marja Rudenhed
AU  - Eva Bergquist
AU  - Anna-Karin Andershed
AU  - Henrik Andershed
Y1  - 2013/03/10
PY  - 2013
N1  - https://doi.org/10.11648/j.ajap.20130202.11
DO  - 10.11648/j.ajap.20130202.11
T2  - American Journal of Applied Psychology
JF  - American Journal of Applied Psychology
JO  - American Journal of Applied Psychology
SP  - 16
EP  - 21
PB  - Science Publishing Group
SN  - 2328-5672
UR  - https://doi.org/10.11648/j.ajap.20130202.11
AB  - Behavioral problems in childhood have been associated with conduct problems later in life. Thus, it is essen-tial that youths with or at risk for conduct problems receive the help they need on time. Therefore, youth with or at risk for conduct problem must receive effective risk-need assessments and intervention plans regardless the person who conducts the assessment. ESTER-assessment is a structured, computer-aided, risk-need instrument developed for assessing youth (0-18) with or at risk for conduct problems. It uses a five-step response scale to assess 19 research-based risk and protective factors and the present study tests the inter-rater reliability of these 19 factors. This was done by comparing the assessments conducted by two independent raters who assessed the file information of 30 girls (mean age = 16.9) who had been incarcerated due to psychosocial problems, criminality and/or drug abuse. Results showed fair to good agreement for the majority of the factors via intra-class correlations and percentage agreement varied on the 19 factors from 24.1 to 80.8 % for exact agreement and from 72.2 to 96.7 % for exact agreement or difference by one step on the response scale. We conclude that it is possible to gain acceptable to excellent inter-rater reliability in assessing risk and protective factors via ESTER-assessment.
VL  - 2
IS  - 2
ER  -

Copy | Download