|
|
||||||||
Research Reports |
MS Goldstein, EdD, is Director, Research Services, Division of Practice and Research, American Physical Therapy Association, 1111 N Fairfax St, Alexandria, VA 22314 (USA) (marcgoldstein{at}apta.org). Address all correspondence to Dr Goldstein
SD Elliott, PhD, is Director of Analytic Support, Division of Practice and Research, American Physical Therapy Association
AA Guccione, PT, PhD, FAPTA, is Senior Vice-President, Division of Practice and Research, American Physical Therapy Association
Submitted December 1, 1999;
Accepted May 10, 2000
| Abstract |
|---|
Key Words: Patient satisfaction Physical therapy Reliability Validity
| Introduction |
|---|
|
|
|---|
| Satisfaction and Quality of Care |
|---|
|
|
|---|
Job satisfaction literature from the field of organizational psychology can be an important source of hypothesis generation for research on patient satisfaction. For example, equity theory9 posits that a person's degree of satisfaction with his or her salary is contingent on the expectation that the outcome, as measured by salary, is either greater or less than the person's perceived work input. Linder-Pelz10 has applied these theories of satisfaction in the workplace to satisfaction with the provision of health care. When patients' expectations of care are exceeded, their level of satisfaction is high. Likewise, if expectations of care exceed actual delivery, dissatisfaction will result. Satisfaction, therefore, is always relative to the patient's expectations. Satisfaction changes when the patient's expectations or standards of comparison change, even though the object of comparison (the actual health care received) may stay constant. Thus, satisfaction measures, although they may be objective (ie, have reliability), are actually reflecting subjective phenomena and are quite distinct from other types of evaluation of the provision of care.
In addition to the importance to the clinician of a patient's level of satisfaction with care as part of the patient-therapist relationship, maintaining a high level of patient satisfaction may also have an economic impact on the clinician. Patients who are satisfied with the services they have received are more likely to remain loyal to the provider (ie, the therapist). Even if a "successful" physical therapist may never again see a patient who has been rehabilitated to an optimal functional level or changes in insurance or personal considerations preclude returning to a particular physical therapist, levels of satisfaction, in our view, may contribute to more informal word-of-mouth communication with other people who may become patients. Dissatisfaction with a particular provider broadly voiced in the community in which the therapist practices may cause a potential patient to seek another physical therapist for intervention as the need arises.
| Benefits of Patient Satisfaction Surveys |
|---|
|
|
|---|
| Domains of Satisfaction |
|---|
|
|
|---|
|
| Administration of Surveys |
|---|
|
|
|---|
The number of patients surveyed and the manner in which patients are surveyed differ among providers as well.13 Within health care overall, we believe it is as likely that all discharged patients will be surveyed as it is that a random sample of discharged patients will be surveyed. However, among physical therapists, it appears that it is much less likely that all patients seen by a physical therapist will be surveyed.
Finally, numerous methods are used to distribute patient satisfaction survey instruments. According to Nelson,13 the standard method is to provide the survey instruments to patients at a facility on conclusion of services and request them to return the instruments by mail. A slightly less common method is to mail survey instruments to patients. Administering surveys by telephone is a technique not as widely used, and face-to-face interviews are used least often. Regardless of sampling frame, Nelson13 has indicated that approximately 60% of surveys do not require respondents to provide their names. This feature of the survey maintains confidentiality and increases the likelihood that respondents will be objective. In addition, it has the benefit of increasing response rate.
| Measurement Issues |
|---|
|
|
|---|
To address some of these problems, in particular threats to reliability and validity, patient satisfaction instruments should undergo evaluation to establish their psychometric properties. Such evaluation, however, cannot eliminate the conceptual argument against the use of patient satisfaction measures to reflect quality of care. Often, however, these studies have not been undertaken. A notable exception to the dearth of such studies is the work of Ware and colleagues.11,12,1719 Over an approximately 10-year period, these researchers developed and refined the Patient Satisfaction Questionnaire and the Group Health Association of America's Consumer Satisfaction Survey.12,15 These 2 research efforts, the former funded through a grant from the Agency for Health Care Policy and Research (now called the Agency for Healthcare Research and Quality) and the latter developed at the request of the Group Health Association of America, represent 2 of the most rigorous efforts to develop instruments for measuring patient satisfaction. These survey instruments, however, deal with consumer reactions to the provision of health (ie, medical) care. They do not deal specifically with the provision of physical therapy services. Adapting them to assess satisfaction with the provision of physical therapy would necessitate the measurement of reliability and validity of a new instrument.
Physical therapists have been confronted with the dilemma of either adopting instruments developed for other purposes or using instruments for physical therapists that have had a less than rigorous psychometric evaluation. As far as we can determine, survey instruments currently in use have been developed within a single location and generally have not been subjected to psychometric evaluations. Conversely, instruments that have undergone rigorous evaluation have not been developed to answer questions about patient satisfaction with intervention by physical therapists.
The purpose of this study was to develop an instrument to assess patient satisfaction with the provision of physical therapy services. The instrument was field tested across a number of practice settings to determine its psychometric properties. The availability of an instrument specific to physical therapy would allow therapists to be less reliant on instruments developed primarily for the assessment of other types of health care delivery, and the instrument could be used to better assess satisfaction with the provision of physical therapy services. The development process that we used included an evaluation of the psychometric properties of the instrument. One goal of this endeavor was to advance the assessment of patient satisfaction by physical therapists beyond the use of instruments that have passed only the test of face validity.
| Methods |
|---|
|
|
|---|
The Compendium was compiled by the American Physical Therapy Association (APTA) in 1995 from responses to a call for patient satisfaction instruments that was published in PT Bulletin and included in a letter to all members of APTA's Private Practice Section. Approximately 75 instruments were received in response to this call. The Compendium includes 36 of these instruments. The instruments were selected for inclusion in the Compendium on the basis of their overall utility and coverage of the domains of patient satisfaction. These instruments were in use by physical therapists in a variety of settings, including acute care hospitals, subacute rehabilitation hospitals, and private outpatient offices, at the time they were submitted to APTA.
From the instruments in the Compendium and other sources, 20 items were generated for the instrument that we used in our study. The items were selected to represent the domains postulated by Nelson13 as well as additional domains that we deemed relevant to patient satisfaction at the time the study was undertaken. The goal was to maintain a reasonable number of questions that would represent aspects of the delivery of care that would affect patient satisfaction. Table 2 contains a listing of domains and a categorization of items by domain as determined by the authors. As illustrated in the table, a total of 11 domains are listed. Some of these domains represent Nelson's domains, and some represent new or additional domains of patient satisfaction. Three such domains not covered in Nelson's list are (1) cost (domain 4), (2) convenience of appointment time (domain 6), and (3) ease of scheduling an appointment (domain 7).
|
Survey Administration
Data were collected at 12 practice settings that were part of a large physical therapy practice. These settings represented diverse locations, including a hospital-based outpatient clinic and a private physical therapy office. Thus, patients and clients were not limited to those from a private outpatient office. A total of 35 physical therapists were employed across all practice locations. The average daily number of patients seen was 366, with a range of 4 patients at the smallest location to 80 patients at the largest location. Orthopedic conditions were those typically seen most often. More specifically, patients were treated for back, knee, shoulder, elbow, and hand problems. However, patients were not restricted to those with orthopedic conditions. One of the clinics provided services to patients with stroke. Another clinic was devoted solely to providing services for women's health issues, including fibromyalgia and pelvic dysfunction.
Patients were asked to complete the instrument upon leaving a facility. Patients were assured that their responses were voluntary and would remain confidential. To ensure confidentiality, patients completed the instrument in the waiting room outside the view of the clinic therapists and other staff, and they were instructed to immediately place it in a locked box. This data collection effort was typical of this practice. Patient satisfaction data had been collected routinely, using a different instrument than the one being developed, for 2 years prior to this data collection effort. Data were collected over a period of 1 month. Instruments were then batched and returned to the researchers to begin processing and analysis. Responses were entered into a data file, and data analysis was accomplished using the SPSS/PC+ statistical package.20,*
Analysis Plan
Descriptive statistics and estimates of reliability and validity were computed for the instrument. Reliability was assessed using the Cronbach alpha coefficient, which measures the internal consistency of measurements obtained with an instrument. The internal consistency, or homogeneity, is a measure of the extent to which items assess the same characteristic.21 This type of reliability analysis was chosen because it has been used to assess the reliability of measurements obtained with other patient satisfaction instruments.22,23 It does not, however, address whether a measure is stable over time.
For example, we strongly believe that test-retest reliability would likely be influenced by factors most related to satisfaction. Responses to the second administration could be influenced by the manner in which respondents completed the instrument the first time. Furthermore, as time passes, respondents may experience "selective forgetting," causing them to perceive aspects of an intervention differently as more time elapses between the intervention and completing the instrument. Another method not selected was split-halves reliability. This technique correlates scores from one half of an instrument with scores from the other half. The Cronbach alpha coefficient is very similar to the split-halves reliability method, except that the split-halves reliability method considers only one split, whereas the Cronbach alpha coefficient accounts for all possible splits.24
Validity was assessed in 3 ways. First, an assessment of validity was made in terms of content (ie, content validity). This type of validity deals with the extent to which an instrument reflects the meaningful elements of the content without extraneous elements.25 There is no quantitative index available for this type of validity. Content validity is often judged simply by comparing the content of an instrument with the domains that are intended as the areas to be measured, and sometimes it is judged by seeking expert opinion.
The second assessment of validity was formulated as an assessment of concurrent validity. Concurrent validity is a measure of the degree to which an instrument correlates with a criterion variable that is available at the time the instrument is administered.24 This criterion variable is known to be a valid measure of the construct under investigation. Thus, if the instrument has a high correlation with the criterion variable, then the instrument is known also to be a valid measure of the construct. The criterion variable used for this assessment was overall satisfaction (domain 11 in Tab. 2). The items in this domain represent the most broad aspect of satisfaction, and, therefore, all other domains of satisfaction should correlate with domain 11.
The third type of validity that was assessed was construct validity, which refers to the degree to which an instrument accurately measures the construct under investigation. Several approaches are utilized to assess construct validity. In an approach to assessing construct validity, multitrait-multimethod matrixes are evaluated for the consistency of inter-item correlations. If items measuring different constructs or traits correlate highly within methods of measurement and to a lesser extent across methods of measurement, then the instrument is consistent with the hypothesized constructs; thus, it has construct validity. The evaluation of construct validity used in our study reflects the use of the multitraitmultimethod matrix. An inter-item correlation matrix was evaluated using the domains listed in Table 2 as a model for the patterns of correlation.
In addition, a factor analysis was performed to evaluate the factor structure relative to the domains presented in Table 2 and relative to the domains presented by Nelson.13 One aspect of factor analysis is the estimation of communalities for each item of an instrument. The communalities of an item reflect the extent to which it shares a common variance with other items. In the case of this instrument, communalities can be interpreted as measures of the same underlying factor, patient satisfaction.
| Results |
|---|
|
|
|---|
Psychometric Characteristics of the Instrument
The Cronbach alpha coefficient computed for the instrument was .99. Table 3 contains item analysis results and the Cronbach alpha coefficients that would be generated if each item were to be deleted from the instrument. These results illustrate the fact that question 25 (ie, "If I had to, I would pay for these physical therapy services myself") does not measure the underlying construct of the survey to the same extent as the other questions. The correlation between question 25 and the total for the questionnaire was only .59, whereas the next lowest correlation of a single question with the total was .80.
|
To assess concurrent validity, 3 of the 20 items were removed so that they could be used as criterion measures. The 3 items that were removed were: (1) "Would recommend to family and friends" (question 22), (2) "Would return to this facility for physical therapy in the future" (question 23), and (3) "Overall satisfaction with the physical therapy experience" (question 26). A case can be made that these items have face validity as the best overall indicators of satisfaction. The first and second items are the best behavioral indicators of satisfaction. The third item asks directly about overall satisfaction. The remaining 17 items were then used to form a summary score for the remainder of the survey instrument. This summary score was correlated with each of the criterion variables. The obtained correlations were r =.95 (P<.01) for question 22 and the summary score, r =.96 (P<.01) for question 23 and the summary score, and r =.96 (P<.01) for question 26 and the summary score. These correlations indicate a high level of agreement between the summary score and each of the criterion variables.
The multitrait-multimethod matrix for the assessment of construct validity is presented in Table 4. The questions in the matrix are ordered according to the domains listed in Table 2. That is, all of the questions from domain 1 are grouped together and appear first. Likewise, all of the items from domain 2 are grouped together and appear second, and so on. The correlations were consistent with the notion that patient satisfaction has an underlying dimension to which all of the domains are highly related. This is evident, in our view, because of the high correlations in all positions of the matrix. However, there is also evidence of distinct domains. As noted, an instrument that yields measurements that have construct validity has higher correlations for items measuring the same domain than for items measuring different domains. The upper left portion of Table 4 contains the correlations among items from domain 1. These correlations averaged around .9. The correlations in the upper middle and right-hand portions of the table represent the relationships among items from domain 1 and the other domains. With a few exceptions, these correlations, although still positive, were lower than the correlations within items from domain 1. This result is consistent with construct validity. The same pattern of correlations is evident for domains 7 and 11 as well. It should be noted that this type of evaluation is not possible for domains with only one item.
|
|
|
| Discussion |
|---|
|
|
|---|
The discovery of only one factor and a high coefficient for reliability may be a cause for concern. Several researchers8,10,13,14 have hypothesized that patient satisfaction is a multidimensional phenomenon. However, there is no consensus on this point. Other researchers15 have argued that there is little data regarding the type of information that contributes most to patients' satisfaction with the provision of health care. Perhaps, patient satisfaction is best described as a unidimensional construct, or as a construct that has one predominant dimension. Ratings of overall satisfaction may not be influenced to a great extent by experiences with ancillary aspects of care such as courtesy of support staff or the availability of parking.
The notion of a predominant dimension of satisfaction is consistent with our data on concurrent validity. In our analyses, the 3 measures of overall satisfaction were shown to be correlated highly with a composite score that was based on questions from all domains of satisfaction. Likewise, the analysis of construct validity via the multitrait-multimethod matrix and the factor analysis indicate that a single dimension accounts for most of the variance in the data.
Limitations
The generalizability of our results is limited by the fact that a sample of convenience (289 subjects selected from a network of 12 clinics) was used. Convenience samples are characterized by the inclusion of individuals who are conveniently available to participate in a study. These individuals may or may not be representative of the larger population. In our investigation, all of the study participants were patients from a single clinical network. Although the operation is large, and has several sites, each with multiple physical therapists, the fact that only one organization was used makes the sample homogeneous with respect to geographical region. More positively, however, it should be noted that the case mix with respect to presenting condition and treatment appears to be representative of the types of conditions for which physical therapists provide intervention.28,29
The fact that the organization we studied is large and growing is also supportive of courteous, the inference that it is a successful practice. The growth is evidence from the inclusion of 3 new settings as part of the practice within the past year. A successful practice in a competitive market must do a good job with respect to patient satisfaction, with all other factors held constant. Therefore, a broad range of ratings for satisfaction is not available on which to establish and evaluate the psycho-metric properties of the instrument. That is, the variability of the responses is limited because all of the ratings were highly positive. Although this narrow dispersion still permits the identification of questions that are not consistent with patient satisfaction, the results are limited because the properties of the instrument under less favorable conditions are unknown. Unfortunately, a recurring problem in this type of research is the fact that practices that anticipate lower patient satisfaction are less likely to participate in such studies.
Another possible limitation of the study concerns the evaluation of validity. Although the instrument has been shown to exhibit some content validity, concurrent validity, and construct validity, the degree of predictive validity was not assessed. Predictive validity is the degree to which an instrument correlates with an actual outcome. For example, the predictive validity of measurements obtained with a test for selecting fighter pilots might be evaluated by comparing test predictions with ratings from flight instructors after completion of flight school. In the case of patient satisfaction, outcome measures might be whether a patient returns to a facility or whether the patient recommends the facility to other people. No outcomes were measured in our study, and the predictive validity of measurements obtained with the instrument remains unknown.
| Summary and Conclusions |
|---|
|
|
|---|
The instrument that we studied was shown to yield reliable measurements, and the measurements appear to have some content, concurrent, and construct validity. The results from both the reliability analysis and the factor analysis suggest that a single dimension underlies patient satisfaction. This dimension appears to be manifested in all of the questions, but it is perhaps represented best in the questions pertaining to overall satisfaction. The analysis also revealed that only one question (ie, "If I had to, I would pay for these physical therapy services myself") was less correlated with the other questions on the instrument. Although this question addresses an important issue, that is, the cost equity of the provision of physical therapy services, it appears to be less related to a measure of patient satisfaction. Therefore, we do not recommend its inclusion in a composite measure of patient satisfaction.
Most instruments of patient satisfaction for physical therapists appear to have been evaluated only informally and with respect to content validity. Very few of the instruments contained in the Compendium were assessed for reliability or validity. Those that did undergo testing were analyzed within settings even more limited than the one in our study. Our instrument was subjected to some psychometric evaluation, and, although we believe that this evaluation is a step in the right direction, additional research assessing the instrument across a larger number of offices, clinics, or departments is warranted. This research will be essential to establish the utility of the instrument across the spectrum of quality in the provision of physical therapy services.
| Appendix |
|---|
|
|
|---|
|
| Footnotes |
|---|
* SPSS Inc, 444 N Michigan Ave, Chicago, IL 60611. ![]()
| References |
|---|
|
|
|---|
Related Article
This article has been cited by other articles:
![]() |
S. C. Slade, E. Molloy, and J. L. Keating `Listen to me, tell me': a qualitative study of partnership in care for people with non-specific chronic low back pain Clinical Rehabilitation, March 1, 2009; 23(3): 270 - 280. [Abstract] [PDF] |
||||
![]() |
P. F Beattie, R. M Nelson, and A. Lis Spanish-Language Version of the MedRisk Instrument for Measuring Patient Satisfaction With Physical Therapy Care (MRPS): Preliminary Validation Physical Therapy, June 1, 2007; 87(6): 793 - 800. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. Beattie, M. Dowda, C. Turner, L. Michener, and R. Nelson Longitudinal Continuity of Care Is Associated With High Patient Satisfaction With Physical Therapy Physical Therapy, October 1, 2005; 85(10): 1046 - 1052. [Abstract] [Full Text] [PDF] |
||||
![]() |
B Richardson, L Shepstone, F Poland, M Mugford, B Finlayson, and N Clemence Randomised controlled trial and cost consequences study comparing initial physiotherapy assessment and management with routine practice for selected patients in an accident and emergency department of an acute hospital Emerg. Med. J., February 1, 2005; 22(2): 87 - 92. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. Monnin and T. V Perneger Scale to Measure Patient Satisfaction With Physical Therapy Physical Therapy, July 1, 2002; 82(7): 682 - 691. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. F Beattie, M. B. Pinto, M. K Nelson, and R. Nelson Patient Satisfaction With Outpatient Physical Therapy: Instrument Validation Physical Therapy, June 1, 2002; 82(6): 557 - 565. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |