Drop in performance of candidates taking a postgraduate medical examination between 1997-2001, UK

Main Category: Medical Students / Training
Article Date: 20 Jul 2005 - 21:00 PDT

email to a friend   printer friendly   opinions  

The performance of doctors taking a postgraduate medical exam declined between 1997 and 2001, reveals a study carried out by the Royal Colleges of Physicians and published today in the Open Access journal BMC Medicine.

After a steady increase between 1985 and 1997, the overall performance of postgraduate students taking the Part 1 examination of the Membership of the Royal Colleges of Physicians of the United Kingdom, MRCP (UK), which is a part of higher specialist training, showed a decline from 1997 until 2001. Performance on specific exam questions that were repeated across the years was 14.1% lower in 2001 than in 1996.

"There has been a decline in the performance of graduates from UK medical schools taking the MRCP(UK) Part 1 examination. The reasons for this are not clear, but the finding has implications for medical education, and further studies are needed of performance in other postgraduate and undergraduate examinations" write Professor Chris McManus from University College London, UK, and colleagues from the Royal Colleges of Physicians, the authors of the study.

"The role of postgraduate medical examinations is to set standards of practice and thus to assure the public and the medical profession that doctors have the knowledge and expertise required to diagnose and treat patients, and to progress in their medical careers", explain the authors.

McManus et al assessed the performance of students on 'marker questions' - questions that were repeated in different MRCP (UK) examinations held between 1985 and 2001, and which allow performance to be compared from one year to another. Overall, the ability of examination candidates increased from 1985 to 1997, after which it declined sharply: on a graphical representation of the results, the curve has a clear 'dog-leg' shape, which rises until 1997 and then drops sharply in the subsequent years.

In the second part of the study, the researchers compared the performance in an examination that took place in 1996 and another one that was held in 2001. The results show that fewer candidates who passed the examination in 2001 gave correct answers than had equivalents in 1996 and the candidates' mean score decreased from 69.4 in 1996 to 59.6 in 2001.

"The 'dog-leg' in performance [in] 1997 was not an artefact of changed Examination Regulations, mix of UK and overseas candidates, or time from qualification until taking the Examination", write the authors. "Study 2 confirmed that performance in 2001 was significantly worse than in 1996, that the poorer performance was found in graduates of UK medical schools, and that candidates passing the Examination in 2001 performed less well than those passing in 1996."

Reliance in the past on 'norm-referencing' for setting the pass mark, when a fixed proportion of candidates was allowed to pass at each examination, meant that candidates passing the examination at the end of this period had a somewhat lower standard. Norm referencing was abandoned by the MRCP(UK) in 2002 in favour of 'criterion-referencing', in which examiners determine the pass mark by assessing the desirable knowledge based on individual questions. Thus the standard required to pass all components of the examination is now fixed and independent of the standard of the candidates sitting at that diet.

BMC Medicine (http://www.biomedcentral.com/bmcmed) is published by BioMed Central (http://www.biomedcentral.com), an independent online publishing house committed to providing Open Access to peer-reviewed biological and medical research. This commitment is based on the view that immediate free access to research and the ability to freely archive and reuse published information is essential to the rapid and efficient communication of science.

BioMed Central currently publishes over 140 journals across biology and medicine. In addition to open-access original research, BioMed Central also publishes reviews, commentaries and other non-original-research content. Depending on the policies of the individual journal, this content may be open access or provided only to subscribers.

biomedcentral.com/1741-7015/3/13/abstract

• Additional
• References
• Citations