Statistical methodology for the concurrent assessment of interrater and intrarater reliability: using goniometric measurements as an example

HOME

HELP

FEEDBACK

SUBSCRIPTIONS

Article

Statistical methodology for the concurrent assessment of interrater and intrarater reliability: using goniometric measurements as an example

M Eliasziw, SL Young, MG Woodbury, and K Fryday-Field

Department of Epidemiology and Biostatistics, University of Western Ontario, London, Canada.

Statistical methodology for the concurrent assessment of interrater and intrarater reliability is presented. Application of the methodology is illustrated with an example of one therapist using two goniometers repeatedly to measure knee joint angles. Methods for estimating the coefficients, testing hypotheses, constructing confidence intervals, and computing sample size requirements are provided. In addition, the calculation and clinical interpretation of the standard error of measurement (SEM) are discussed. It is recommended that (1) when both interrater and intrarater reliability are being assessed, a repeated-measures design be used to take advantage of the increased precision gained by using all observations in the statistical analysis, and (2) appropriate statistical tests, confidence intervals, and SEMs always be used in conjunction with the estimated reliability coefficients.

This article has been cited by other articles:

V. Puetz, P.N. Sylaja, S. B. Coutts, M. D. Hill, I. Dzialowski, P. Mueller, U. Becker, G. Urban, C. O'Reilly, P. A. Barber, et al.
Extent of Hypoattenuation on CT Angiography Source Images Predicts Functional Outcome in Patients With Basilar Artery Occlusion
Stroke, September 1, 2008; 39(9): 2485 - 2490.
[Abstract] [Full Text] [PDF]

K. C. Y. Sie, J. R. Starr, D. C. Bloom, M. Cunningham, L. M. de Serres, A. F. Drake, R. G. Elluru, J. Haddad Jr, C. Hartnick, C. MacArthur, et al.
Multicenter Interrater and Intrarater Reliability in the Endoscopic Evaluation of Velopharyngeal Insufficiency
Arch Otolaryngol Head Neck Surg, July 1, 2008; 134(7): 757 - 763.
[Abstract] [Full Text] [PDF]

J. H.F. Rudd, K. S. Myers, S. Bansilal, J. Machac, C. A. Pinto, C. Tong, A. Rafique, R. Hargeaves, M. Farkouh, V. Fuster, et al.
Atherosclerosis Inflammation Imaging with 18F-FDG PET: Carotid, Iliac, and Femoral Uptake Reproducibility, Quantification Methods, and Recommendations
J. Nucl. Med., June 1, 2008; 49(6): 871 - 878.
[Abstract] [Full Text] [PDF]

A. Lyshchik, S. B. Hobbs, A. C. Fleischer, D. Khabele, D.-S. Son, J. C. Gore, and R. R. Price
Ovarian Volume Measurements in Mice With High-Resolution Ultrasonography
J. Ultrasound Med., October 1, 2007; 26(10): 1419 - 1425.
[Abstract] [Full Text] [PDF]

J. H.F. Rudd, K. S. Myers, S. Bansilal, J. Machac, A. Rafique, M. Farkouh, V. Fuster, and Z. A. Fayad
18Fluorodeoxyglucose Positron Emission Tomography Imaging of Atherosclerotic Plaque Inflammation Is Highly Reproducible: Implications for Atherosclerosis Therapy Trials
J. Am. Coll. Cardiol., August 28, 2007; 50(9): 892 - 896.
[Abstract] [Full Text] [PDF]

H. Van Waelvelde, W. Peersman, M. Lenoir, and B. C. Smits Engelsman
The reliability of the Movement Assessment Battery for Children for preschool children with mild to moderate motor impairment
Clinical Rehabilitation, May 1, 2007; 21(5): 465 - 470.
[Abstract] [PDF]

L N Brown, L M Metz, and M Eliasziw
Identifying reliable change in tactile temporal thresholds in multiple sclerosis: test-retest reliability
Multiple Sclerosis, September 1, 2006; 12(5): 573 - 577.
[Abstract] [PDF]

P. J. Yoon, J. R. Starr, J. A. Perkins, D. Bloom, and K. C. Y. Sie
Interrater and intrarater reliability in the evaluation of velopharyngeal insufficiency within a single institution.
Arch Otolaryngol Head Neck Surg, September 1, 2006; 132(9): 947 - 951.
[Abstract] [Full Text] [PDF]

P A Barber, M D Hill, M Eliasziw, A M Demchuk, J H W Pexman, M E Hudon, A Tomanek, R Frayne, A M Buchan, and for the ASPEC Study Group
Imaging of the brain in acute ischaemic stroke: comparison of computed tomography and magnetic resonance diffusion-weighted imaging
J. Neurol. Neurosurg. Psychiatry, November 1, 2005; 76(11): 1528 - 1533.
[Abstract] [Full Text] [PDF]

X. Wei, C. R. Guttmann, S. K Warfield, M. Eliasziw, and J R. Mitchell
Has your patient's multiple sclerosis lesion burden or brain atrophy actually changed?
Multiple Sclerosis, August 1, 2004; 10(4): 402 - 406.
[Abstract] [PDF]

A. Landry, J. D. Spence, and A. Fenster
Measurement of Carotid Plaque Volume by 3-Dimensional Ultrasound
Stroke, April 1, 2004; 35(4): 864 - 869.
[Abstract] [Full Text] [PDF]

V M Pomeroy, S H Chambers, G Giakas, and M Bland
Reliability of measurement of tempo-spatial parameters of gait after stroke using GaitMat II
Clinical Rehabilitation, February 1, 2004; 18(2): 222 - 227.
[Abstract] [PDF]

S. B. Coutts, J. E. Simon, A. I. Tomanek, P. A. Barber, J. Chan, M. E. Hudon, J. R. Mitchell, R. Frayne, M. Eliasziw, A. M. Buchan, et al.
Reliability of Assessing Percentage of Diffusion-Perfusion Mismatch
Stroke, July 1, 2003; 34(7): 1681 - 1683.
[Abstract] [Full Text] [PDF]

V M Pomeroy, A Pramanik, L Sykes, J Richards, and E Hill
Agreement between physiotherapists on quality of movement rated via videotape
Clinical Rehabilitation, March 1, 2003; 17(3): 264 - 272.
[Abstract] [PDF]

B. Van Gheluwe, K. A. Kirby, P. Roosen, and R. D. Phillips
Reliability and Accuracy of Biomechanical Measurements of the Lower Extremities
J Am Podiatr Med Assoc, June 1, 2002; 92(6): 317 - 326.
[Abstract] [Full Text] [PDF]

J. M Fritz and J. J Irrgang
A Comparison of a Modified Oswestry Low Back Pain Disability Questionnaire and the Quebec Back Pain Disability Scale
Physical Therapy, February 1, 2001; 81(2): 776 - 788.
[Abstract] [Full Text] [PDF]

A. D. HOUGH, A. P. MOORE, and M. P. JONES
Peripheral Nerve Motion Measurement with Spectral Doppler Sonography: A Reliability Study
J Hand Surg Eur Vol., December 1, 2000; 25(6): 585 - 589.
[Abstract] [PDF]

V. M Pomeroy, C. Frames, E. B Faragher, A. Hesketh, E. Hill, P. Watson, and C. J Main
Reliability of a measure of post-stroke shoulder pain in patients with and without aphasia and/or unilateral spatial neglect
Clinical Rehabilitation, June 1, 2000; 14(6): 584 - 591.
[Abstract] [PDF]

K. Jordan, K. Dziedzic, P. W. Jones, B. N. Ong, and P. T. Dawes
The reliability of the three-dimensional FASTRAK measurement system in measuring cervical spine and shoulder range of motion in healthy subjects
Rheumatology, April 1, 2000; 39(4): 382 - 388.
[Abstract] [Full Text] [PDF]

C. Leemrijse, O. G Meijer, A. Vermeer, B. Lambregts, and H. J Ader
Detecting individual change in children with mild to moderate motor impairment: the standard error of measurement of the Movement ABC
Clinical Rehabilitation, May 1, 1999; 13(5): 420 - 429.
[Abstract] [PDF]

				K. C. Y. Sie, J. R. Starr, D. C. Bloom, M. Cunningham, L. M. de Serres, A. F. Drake, R. G. Elluru, J. Haddad Jr, C. Hartnick, C. MacArthur, et al. Multicenter Interrater and Intrarater Reliability in the Endoscopic Evaluation of Velopharyngeal Insufficiency Arch Otolaryngol Head Neck Surg, July 1, 2008; 134(7): 757 - 763. [Abstract] [Full Text] [PDF]

				J. H.F. Rudd, K. S. Myers, S. Bansilal, J. Machac, C. A. Pinto, C. Tong, A. Rafique, R. Hargeaves, M. Farkouh, V. Fuster, et al. Atherosclerosis Inflammation Imaging with 18F-FDG PET: Carotid, Iliac, and Femoral Uptake Reproducibility, Quantification Methods, and Recommendations J. Nucl. Med., June 1, 2008; 49(6): 871 - 878. [Abstract] [Full Text] [PDF]

				A. Lyshchik, S. B. Hobbs, A. C. Fleischer, D. Khabele, D.-S. Son, J. C. Gore, and R. R. Price Ovarian Volume Measurements in Mice With High-Resolution Ultrasonography J. Ultrasound Med., October 1, 2007; 26(10): 1419 - 1425. [Abstract] [Full Text] [PDF]

				H. Van Waelvelde, W. Peersman, M. Lenoir, and B. C. Smits Engelsman The reliability of the Movement Assessment Battery for Children for preschool children with mild to moderate motor impairment Clinical Rehabilitation, May 1, 2007; 21(5): 465 - 470. [Abstract] [PDF]

				L N Brown, L M Metz, and M Eliasziw Identifying reliable change in tactile temporal thresholds in multiple sclerosis: test-retest reliability Multiple Sclerosis, September 1, 2006; 12(5): 573 - 577. [Abstract] [PDF]

				P. J. Yoon, J. R. Starr, J. A. Perkins, D. Bloom, and K. C. Y. Sie Interrater and intrarater reliability in the evaluation of velopharyngeal insufficiency within a single institution. Arch Otolaryngol Head Neck Surg, September 1, 2006; 132(9): 947 - 951. [Abstract] [Full Text] [PDF]

				P A Barber, M D Hill, M Eliasziw, A M Demchuk, J H W Pexman, M E Hudon, A Tomanek, R Frayne, A M Buchan, and for the ASPEC Study Group Imaging of the brain in acute ischaemic stroke: comparison of computed tomography and magnetic resonance diffusion-weighted imaging J. Neurol. Neurosurg. Psychiatry, November 1, 2005; 76(11): 1528 - 1533. [Abstract] [Full Text] [PDF]

				X. Wei, C. R. Guttmann, S. K Warfield, M. Eliasziw, and J R. Mitchell Has your patient's multiple sclerosis lesion burden or brain atrophy actually changed? Multiple Sclerosis, August 1, 2004; 10(4): 402 - 406. [Abstract] [PDF]

				A. Landry, J. D. Spence, and A. Fenster Measurement of Carotid Plaque Volume by 3-Dimensional Ultrasound Stroke, April 1, 2004; 35(4): 864 - 869. [Abstract] [Full Text] [PDF]

				V M Pomeroy, S H Chambers, G Giakas, and M Bland Reliability of measurement of tempo-spatial parameters of gait after stroke using GaitMat II Clinical Rehabilitation, February 1, 2004; 18(2): 222 - 227. [Abstract] [PDF]

				S. B. Coutts, J. E. Simon, A. I. Tomanek, P. A. Barber, J. Chan, M. E. Hudon, J. R. Mitchell, R. Frayne, M. Eliasziw, A. M. Buchan, et al. Reliability of Assessing Percentage of Diffusion-Perfusion Mismatch Stroke, July 1, 2003; 34(7): 1681 - 1683. [Abstract] [Full Text] [PDF]

				V M Pomeroy, A Pramanik, L Sykes, J Richards, and E Hill Agreement between physiotherapists on quality of movement rated via videotape Clinical Rehabilitation, March 1, 2003; 17(3): 264 - 272. [Abstract] [PDF]

				B. Van Gheluwe, K. A. Kirby, P. Roosen, and R. D. Phillips Reliability and Accuracy of Biomechanical Measurements of the Lower Extremities J Am Podiatr Med Assoc, June 1, 2002; 92(6): 317 - 326. [Abstract] [Full Text] [PDF]

				J. M Fritz and J. J Irrgang A Comparison of a Modified Oswestry Low Back Pain Disability Questionnaire and the Quebec Back Pain Disability Scale Physical Therapy, February 1, 2001; 81(2): 776 - 788. [Abstract] [Full Text] [PDF]

				A. D. HOUGH, A. P. MOORE, and M. P. JONES Peripheral Nerve Motion Measurement with Spectral Doppler Sonography: A Reliability Study J Hand Surg Eur Vol., December 1, 2000; 25(6): 585 - 589. [Abstract] [PDF]

				V. M Pomeroy, C. Frames, E. B Faragher, A. Hesketh, E. Hill, P. Watson, and C. J Main Reliability of a measure of post-stroke shoulder pain in patients with and without aphasia and/or unilateral spatial neglect Clinical Rehabilitation, June 1, 2000; 14(6): 584 - 591. [Abstract] [PDF]

				K. Jordan, K. Dziedzic, P. W. Jones, B. N. Ong, and P. T. Dawes The reliability of the three-dimensional FASTRAK measurement system in measuring cervical spine and shoulder range of motion in healthy subjects Rheumatology, April 1, 2000; 39(4): 382 - 388. [Abstract] [Full Text] [PDF]

				C. Leemrijse, O. G Meijer, A. Vermeer, B. Lambregts, and H. J Ader Detecting individual change in children with mild to moderate motor impairment: the standard error of measurement of the Movement ABC Clinical Rehabilitation, May 1, 1999; 13(5): 420 - 429. [Abstract] [PDF]