Skip to main content

Advertisement

SpringerLink
Log in

Closing the Loop: Assessing the Effectiveness of Psychiatric Competency Measures

  • Original Article
  • Published:
Academic Psychiatry Aims and scope Submit manuscript

Abstract

Background: During the past several years there has been a growing emphasis on competency assessment in graduate education in all specialties, including psychiatry. Methods of assessment are continuously being devised and refined. Objective: To outline principles and methods of assessing the validity of competency measures and their relevance to trainees. Methods: The authors reviewed the literature relevant to competency assessment and quality improvement in the fields of elementary and secondary education, adult learning, undergraduate medical education, and postgraduate medical training. Three means were used to evaluate the assessment program: 1) The authors surveyed graduates and credentialing agencies for feedback on the relevance of the measures they previously developed for current and envisioned future practice of psychiatry; 2) They measured the completion rates of competency measures by residents in the first 3 years of a program, measuring general psychiatric competencies; and 3) They surveyed residents and faculty regarding satisfaction with the existing process. Results: Despite small numbers, the results indicated that measures devised in 1998 were relevant to the practice of psychiatry in 2001 and envisioned by practitioners for the subsequent 5 to 10 years. Resident completion rates of competency requirements indicated that the structure of the measures was appropriately paced to parallel the learning expectations of the training program. Participant satisfaction with the process of developing and implementing the measures was high. Conclusions: While devising and implementing competency assessments is critical, the process should not stop with the application of the measures. Ongoing evaluation of these measures for their continued suitability is essential. Training programs can readily determine the effectiveness of competency programs and improve them as necessary.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wolf, RM: Evaluation in Education: Foundations of Competency Assessment and Program Review (3rd ed.). New York, Westport CT, London, Prager, 1990

    Google Scholar 

  2. Monjan S, Gassner SM: Critical Issues in Competency Based Education. New York, Oxford, Toronto, Sydney, Frankfurt, Paris, Pergamon Press, 1979, pp 1–35

    Google Scholar 

  3. Norris N: Understanding Educational Evaluation. New York, St. Martin’s Press, 1990, pp 97–109

    Google Scholar 

  4. Hibbard KM, van Wagenen L, Lewbel S, Waterbury-Wyatt S, et al.: A Teacher’s Guide to Performance-Based Learning and Assessment. Alexandria, VA, Association for Supervision and Curriculum Development, 1996, pp 33–46

    Google Scholar 

  5. Saylor JH: TQM Field Manual. New York, McGraw Hill, 1992, pp 99–105

    Google Scholar 

  6. Weaver CN: TQM: A Step-by-Step Guide to Implementation. Washington DC, ASQC Quality Press, 1991, pp 137–156

    Google Scholar 

  7. Brainard EZ: A Hands-On Guide to School Program Evaluation. Bloomington, IN, Phi Delta Kappa Foundation, 1996 pp 9–55

    Google Scholar 

  8. Marshall J: Assessement during postgraduate training. In JS Gonella, M Hojat, JB Erdmannn, JJ Veloski (eds), Assessment Measures in Medical School, Residency and Practice. New York, Springer Publishing Co., 1993, pp 47–54

    Google Scholar 

  9. Hojat M, Gonnella JS, Veloski JJ, Erdmann, JB: Is the glass half full or half empty? A reexamination of the associations between assessment measures during medical school and clinical competence after graduation. In JS Gonella, M Hojat, JB Erdmannn, JJ Veloski (eds), Assessment Measures in Medical School, Residency and Practice. New York, Springer Publishing Co., 1993, pp 137–152

    Google Scholar 

  10. Bienenfeld D, Klykylo W, Knapp V: Process and product: development of competency-based measures for psychiatryresidency. Acad Psych 2000; 24: 68–76

    Article  Google Scholar 

  11. Best Hospitals: PSYCHIATRY http://www.usnews.com/usnews/nycu/health/hosptl/specpsyc.htm, 2001

Download references

Author information

Authors and Affiliations

  1. Department of Psychiatry, Wright State University, Box 927, Dayton, Ohio, 45401-0927, USA

    David Bienenfeld M.D. (Professor, Vice Chairman, and Director of Residency Training)

  2. Division of Child and Adolescent Psychiatry, Wright State University, Dayton, Ohio, USA

    William Klykylo M.D. (Director)

Authors
  1. David Bienenfeld M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William Klykylo M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Douglas Lehrer M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Bienenfeld M.D..

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bienenfeld, D., Klykylo, W. & Lehrer, D. Closing the Loop: Assessing the Effectiveness of Psychiatric Competency Measures. Acad Psychiatry 27, 131–135 (2003). https://doi.org/10.1176/appi.ap.27.3.131

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1176/appi.ap.27.3.131

Keywords

Search

Navigation