| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Research Reports |
KW Hayes, PhD, PT, is Assistant Professor of Physical Therapy and Curriculum Coordinator, Programs in Physical Therapy, Northwestern University Medical School, 645 N Michigan Ave, Suite 1100, Chicago, IL 60611 (USA) (k-hayes{at}nwu.edu).
G Huber, PT, is Instructor in Physical Therapy, Programs in Physical Therapy, Northwestern University Medical School
J Rogers, PT, is Assistant Professor in Physical Therapy and Academic Coordinator of Clinical Education, Programs in Physical Therapy, Northwestern University Medical School
B Sanders, PT, is Instructor in Physical Therapy, Programs in Physical Therapy, Northwestern University Medical School
Address all correspondence to Dr Hayes
Submitted October 10, 1997;
Accepted March 8, 1999
 | Abstract |
|---|
=.60–.75). Results. Behaviors in 3 categories emerged as red flags for CIs: 1 cognitive category—inadequate knowledge and psychomotor skill (43% of 134 behaviors)—and 2 noncognitive categories—unprofessional behavior (29.1%) and poor communication (27.6%). The CIs noticed and valued noncognitive behaviors but addressed cognitive behaviors more often with students. Students who did not receive feedback about their performance were unlikely to change their behavior. The CIs used cognitive behaviors often as reasons to recommend negative outcomes. Conclusion and Discussion. Clinical instructors need to identify unacceptable cognitive and noncognitive behaviors as early as possible in clinical experiences. Evidence suggests that they should discuss their concerns with students and expect students to change.
Key Words: Clinical education • Competence • Physical therapy
| Introduction |
|---|
 TopAbstract Introduction MethodResultsDiscussionConclusionsReferences |
|---|
We believe that faculty have been reluctant to use noncognitive factors, such as interpersonal skills, to make judgments about students' competence because of the paucity of data establishing that these factors are necessary elements of clinical competence. Duke6 suggested that nurses often feel that such judgments are subjective and open to challenge. When legal challenges have been initiated by students in response to dismissal for academic and clinical deficiencies, however, the court system has generally supported the academic institution.7 In spite of the reluctance of academic faculty to view noncognitive qualities as essential to clinical competence, clinicians identify qualities that are noncognitive in nature as necessary elements for clinical success. For example, at a meeting of the Illinois Consortium for Clinical Education, participants listed as components of competence the abilities to demonstrate professional advocacy, educate family and caregivers, facilitate patient adherence and responsibility, demonstrate empathy, receive and use feedback, discern critical cues, include the patient in the design of goals, demonstrate time management, delegate appropriately, demonstrate a holistic approach, and include prevention and wellness in the management plan.
May and colleagues,5 in a study of physical therapist education, addressed the issue of evaluation of noncognitive behaviors using a model developed at Alverno College in Wisconsin. This model holds that students must be able to demonstrate abilities, such as communication and problem solving, that are general in nature rather than specific to a discipline. During the 1970s, faculty at Alverno College identified a set of these abilities and criteria for their measurement. At Alverno College, assessment is ongoing, and students use feedback from the assessments to further their ability development.8 May and colleagues5 involved both academic faculty and CIs to develop a list of 10 generic abilities that include many noncognitive abilities. They developed criteria for each of these abilities and use them to evaluate students both in the classroom and in the clinic.5
In professions other than physical therapy, noncognitive factors are considered to be important.2,3,9 In a critical incident study of pediatric residents, Altmaier and colleagues2 found that only 29% of the incidents were related to cognitive factors. The remainder of the incidents were related to communication, professional behavior, and interpersonal skills.2 In addition, Hojat and co-workers3 found that the interpersonal skills of first-year medical residents contributed more to their being offered further residency training than their data gathering and processing skills. Smith and Kendall9 identified 6 dimensions of high-quality nursing performance: knowledge and judgment, conscientiousness, skill in human relationships, organizational ability, objectivity, and observational ability. The majority of these dimensions are noncognitive in nature.
In physical therapy, investigators have attempted to identify predictors of clinical performance, both during the admissions process and in professional education. Some investigators10,11 have found no relationship between clinical performance and the student's home state, possession of a baccalaureate degree, completion of prerequisite courses, applicant's statement, applicant's background, or licensure scores. Other investigators10–14 have found equivocal relationships between clinical performance and preadmission interviews, overall grade point average, science grade point average, letters of reference, and interviews. There may be weak relationships between clinical performance and personality inventories11 and between clinical performance and scores on aptitude tests.12 Searches for predictors once students are enrolled in a professional curriculum have shown weak relationships between clinical performance and achievement in behavioral science and physical therapy courses.15
Few predictors have emerged from this body of knowledge, and none are strongly related to clinical performance. Predictive relationships may be hard to identify because most of the predictor and outcome variables have limited variability among the scores. Nonetheless, most of the studies have focused on cognitive predictors, yet many of the components of clinical competence appear to be noncognitive behaviors and personal qualities.2,5,9 The emphasis on cognitive factors at the expense of the noncognitive factors may explain an inability to predict clinical performance. Although many lists of components have been compiled that contain both cognitive and noncognitive items,16–18 these lists have not been studied to identify the importance of each component in identifying students who may not be competent. Understanding the role of both cognitive and noncognitive predictors of successful clinical performance will allow the profession to know more about the behaviors and characteristics of physical therapist students that can alert CIs to unsafe and ineffective clinical performance.
This study was a qualitative study designed to identify behaviors and characteristics of professional physical therapist students that can alert CIs to clinical performance they deem to be unsafe and ineffective. The conceptual framework underlying the study was based loosely on the taxonomy of learning objectives developed by Bloom1 and on our observations with students. We proposed that inadequacies in the cognitive domains of knowledge and psychomotor skill as well as inadequate unidentified noncognitive factors can lead to unsafe or ineffective practice (Fig. 1). We attempted to examine the relative frequency of occurrence of these 3 dimensions and to determine the nature of the noncognitive factors.
|
| Method |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionsReferences |
|---|
Critical incidents should be carefully defined to include the situation (including place, people, conditions, and activities), the relevance to the activity, and the extent of the effect of the incident on the activity. According to Flanagan,19 the accuracy of the descriptions of the incidents may be assumed if the detail is rich.
Once a large sample of behavioral descriptions has been collected, the behaviors are then categorized by the investigators, either based on a theoretical framework or inductively, if no framework exists. The categories are checked for their validity and reliability by having them reviewed by others.19
Subjects
Eligible subjects were CIs who were physical therapists working in approximately 250 facilities with clinical education contracts with any of the professional physical therapist education programs in Illinois in 1995. Any physical therapist from one of these facilities would be considered qualified if he or she had worked with or had the opportunity to observe at least one physical therapist student who had problems functioning effectively in the clinic. The students with whom they had worked could be from any education program in the country, not just those in Illinois. Clinical instructors were recruited through advertisements placed in professional publications, announcements at clinical education meetings, and requests by academic coordinators of clinical education (ACCEs) from Northwestern University at facilities during clinical contacts. Although the sample was one of convenience, efforts were made to establish geographic diversity within the sample.
Procedure
We defined provision of physical therapy services as independent patient evaluation, treatment planning and implementation, and interactions with patients, families, and other health care providers by students. This definition predated the publication of the Guide to Physical Therapist Practice,21 but we believe it is consistent with the intent of the definitions used in that publication. The specific problem in the study was unsafe and ineffective delivery of physical therapy services by students.
In the interest of gathering rich data, group and individual interviews were conducted to gather information about the critical incidents from CIs. Most group interviews were face-to-face; one group interview was conducted by conference call. Two individual interviews were conducted when a group was not available.
Group interviews were limited to no more than 4 participants to enable the group to remain cohesive and complete its task in a reasonable period of time.20 Interviews were conducted at clinical facilities in combination with on-site clinical meetings, at a Chicago-Area Clinical Educators Forum meeting, at a meeting of the Illinois Consortium for Clinical Education, and at a Combined Sections Meeting of the American Physical Therapy Association (APTA). Some groups included participants from several facilities, and other groups were composed of participants from a single facility. All participants were assured of anonymity and protection of their rights as subjects and gave informed consent granting permission to record and use their information. Participants were asked to complete a questionnaire requesting the information shown in Table 1.
|
The facilitators informed the participants regarding the sponsorship of the study, the purpose of the study, and the definitions of the components of a critical incident. Critical incidents included, whenever possible, the information listed in Table 1. In addition to that information, participants were asked to provide a code name for the student (ie, a first name that was assumed to be fictitious). These code names allowed us to pair the questionnaire information with the subsequent interview transcripts. During the discussion, the student was referred to only by this code name. Neither the other participants nor the investigators knew the identity of the students or the professional programs in which they were enrolled. In all subsequent communications, each student was identified by a numeric code only.
Participants were encouraged to recall both incidents that occurred within the past year and incidents that may have happened in the past. Both extreme instances of unsafe and ineffective behavior and those that deviated only slightly from usual behavior were collected, because a large collection of relatively minor incidents might cause just as much concern as fewer, more serious incidents.
At the beginning of each interview, the facilitator read a script that had been pilot tested to check that participants were clear about the task and that the directions elicited the type of responses desired. Participants then had an opportunity to ask questions of the facilitator. Participants were given time to describe their incidents in writing.20 After the participants had written their information, the facilitator allowed each participant to fully discuss one student's performance. During this initial round, the facilitator's role was to probe and encourage elaboration to assist the participants in generating the needed specificity and clarity. Facilitators used a checklist to be certain that all elements had been requested. In group interviews, after each participant described one student's performance, an invitation was extended for additional student behavioral descriptions. Group interviews ran no more than 2 hours and were audiotaped. Following each interview, a typist transcribed the audiotapes verbatim, resulting in 141 pages of transcript.
Data Analysis
From the questionnaires, the transcripts, and the classification process, one investigator coded the participants, students, and incidents and entered the data into a database for later analysis. The numerical code indicated the number of the participant first, then the number of the student specific to that participant, and finally the number of the incident specific to that student. For example, the first incident reported about the 27th CI's second student would be coded 27.2.1.
While the interviews were proceeding, the members of the team of investigators independently reviewed the initial transcripts and identified the information listed in Table 2. A first classification was performed independently by each investigator; that is, each team member reviewed the incidents from the initial transcripts and identified and named categories of her choosing related to reasons for unsafe and ineffective delivery of physical therapy services. Knowledge and skills and some noncognitive factors were expected based on the initial framework, but the exact nature of those factors emerged from the data. The investigative team initially identified 61 behaviors and discussed the behaviors, the generated categories, and their relationship with the original categories in the initial framework. Pairs of team members then independently classified the 61 behaviors as a check on whether categories were mutually exclusive. The original 3 categories from the initial framework were revised by combining inadequate knowledge and inadequate psychomotor skill because they were not mutually exclusive and by identifying 2 noncognitive categories (ie, poor communication and unprofessional behavior).
|
To check reliability, all investigators classified the behaviors from the remaining transcripts (73 behaviors) into the 3 categories prior to discussion. Pair-wise kappa coefficients were calculated to determine agreement among investigators.23 The coefficients ranged from .60 to .75 for the 73 additional behaviors. All 4 investigators demonstrated perfect agreement on 42 behaviors (58%), and 3 investigators agreed on 19 more behaviors (an additional 26%). The categories assigned for these behaviors were accepted. For the other 12 behaviors, the group discussed each behavior in the context of the transcript and reached consensus for the category. As an additional reliability check, a clinician who had not participated in the study was given a sample of 24% of the behaviors (n=32) and transcripts for classification. The kappa coefficient between the reviewer and the team's classification was .67, with perfect agreement on 78% of the behaviors. Interviewing continued until we were satisfied that the categories were exhaustive in addition to being mutually exclusive.20 Following the completion of the reliability check, each investigator examined the incidents from one of the categories to sort the incidents in that category into subcategories of behaviors.
Although this study was not intended to be quantitative, certain statistical analyses were performed to describe the respondents and summarize detected patterns. Descriptive statistics (means, standard deviations, frequencies) were computed to characterize both the CIs and the students about whom they reported. Where appropriate, these characteristics were compared with the 1994 and 1996 APTA membership profiles and the 1995 profile of graduates of accredited programs.24–26 Patterns that emerged from the data were analyzed using chi-square analyses for goodness of fit and odds ratios.
| Results |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionsReferences |
|---|
|
Behaviors
A total of 134 behaviors were identified by the CIs. These incidents occurred between 1971 and 1994, with 73% of the incidents occurring during 1993 and 1994. Interviews were conducted from March 1995 through March 1996, so most incidents had occurred within 1 to 3 years of the report. Behaviors fell into 3 categories: inadequate knowledge and skills, poor communication, and unprofessional behavior. Inadequate knowledge and skills was the largest category, with 43.3% (n=58) of the behaviors. We identified 5 subcategories: inability to recall information, inability to perform a skill, inefficiency in task completion, poor problem solving, and unsafe judgments or actions (Tab. 4). Thirty-seven of the behaviors fell into the poor communication category and involved primarily inappropriate nonverbal behavior, inappropriate interactions with patients and colleagues, and inappropriate response to feedback (Tab. 5). Thirty-nine behaviors were classified into the unprofessional behavior category. These behaviors were related to problems with stress management, work ethic, accepting responsibility, commitment to learning, recognition of limits, and common sense (Tab. 6).
|
|
|
|
This study was not designed to analyze instructor behavior; nonetheless, several situations were reported in which a problem was related to CI-student interactions rather than student behavior. For example, one participant reported a situation in which a CI had corrected the student in front of a patient. The participant indicated that the incident "just blew away everybody, I mean, upset the student. The patient got upset as a result, and the patient's confidence in both the therapist and the student was probably damaged to an extent.... It wasn't something that was dangerous to the patient. It could have been done later on" (13.2.1). In another situation, a participant reported that a student refused to offer any ideas of what to do with a patient. When the student was invited to explain, he reported that he "felt like his opinions were not respected and that when he offered changes in the patient's program, they would be ignored" (3.2.1).
Relationships Among Student Behavior, Clinical Instructor Feedback, Student Behavioral Change, and Outcome of the Clinical Experience
Although this study was designed primarily to investigate the behaviors that concerned CIs, the CIs were asked about how they responded to the behaviors and whether a student's behavior continued or changed. There was an association between the category of behavior and whether the CI provided feedback to the student (
2=12.4, df=2, P=.002). Students having difficulty with behaviors in the inadequate knowledge and skills category were much more likely to receive feedback from the CI than were students displaying behaviors in the unprofessional behavior and poor communication categories (Fig. 2). When the 2 noncognitive categories were combined and an odds ratio was calculated, students with cognitive difficulties were 4.75 times more likely to receive feedback.
|
2=4.15, df=1, P=.04). Students who did not receive feedback were unlikely to change the behavior, but students who received feedback demonstrated almost equal tendencies to change or not change the behavior (Fig. 2). Whether students changed the behavior was unrelated to the category of behavior. This pattern was apparent in all 3 categories. Participants were also asked about the outcome of the students' clinical experiences, and the incident reports included that information to the best of the CIs' knowledge. We used the CI reports of the outcomes in all of our analyses and did not confirm them with the education programs because, in the interest of confidentiality, we did not have the information to identify students or programs. Given these limitations, the relationships among category of behavior, feedback, change in student behavior, and the outcome of the clinical experience were explored.
Based on the descriptions from the CIs, students could have had a positive or negative experience in clinical education. Positive outcomes were defined as successful completion of the clinical experience or continued clinical education as anticipated. The negative outcomes CIs reported included failure to complete the clinical experience at a particular facility, failure of the clinical experience (in the words of the CI), repeat of the experience at another facility, or dismissal from the professional program.
Two results were notable. First, change in behavior was strongly associated with a positive outcome (2=39.70, df=1, P<.000001). Of the 55 behaviors that were changed by the students, all were associated with a positive outcome. Of the 79 behaviors that did not change, half of them were associated with a positive outcome and half were associated with a negative outcome.
Second, outcome was also associated with the category of behavior. Even though noncognitive behaviors comprised 56.7% of the behaviors described, they were not as likely to be associated with a negative outcome as the behaviors in the inadequate knowledge and skills category (2=10.95, df=1, P=.0009). We interpreted this association to mean that CIs consider noncognitive behaviors to be important, but they are more inclined to use deficits in cognitive abilities to recommend a negative outcome.
Feedback given by the clinical instructor.
In 97 of the 134 incidents, the student was given some type of feedback about the behavior. Of these, 45 behaviors changed and 52 behaviors did not change (Fig. 2).
All changed behaviors were associated with a positive outcome. In the inadequate knowledge and skills category, change occurred for 20 of 51 behaviors about which the CIs provided feedback (Fig. 2). The most common behaviors that prompted instructor intervention in this category related to the student's inability to recall information or perform skills such as manual muscle testing (1.1.1, 2.1.5, 21.1.7) or measuring the length of a residual limb (1.1.2). Equally common were unsafe judgments or actions such as leaving a patient unattended in a precarious position (29.1.2) or transferring a patient who had had a cerebrovascular accident from the wrong side (21.1.5).
In the poor communication category, 12 of 21 behaviors changed with CI feedback (Fig. 2). The most common subcategory of behavior was inappropriate interactions with patients and colleagues, including such behaviors as calling a patient by an inappropriate name ("honey") (17.1.1) or speaking overly loudly (30.1.8). The second most common category was inappropriate nonverbal behavior. For example, a CI reported that a student "was just very quiet, downcast eyes...not authoritative." Later, she reported that "using a student journal, he could see how he had become a little bit more verbal in terms of some of his expressions to patients" (13.1.1).
In the unprofessional behavior category, 13 of 24 behaviors changed in response to feedback (Fig. 2). Most common were personal behaviors such as being inappropriately intimate with a male friend on the mat in the gym in full view of patients (27.1.1). These behaviors were followed by poor work ethic such as discussing personal financial business on the telephone during working hours (33.2.1) and arriving late to work (14.1.2).
Of the behaviors that did not change in response to feedback, about half were associated with a positive outcome and about half were associated with a negative outcome (Fig. 2). All of the uncorrected behaviors that were associated with a negative outcome involved 8 students.
Of the 31 behaviors in the inadequate knowledge and skills category that did not change with feedback, one third were associated with positive outcomes in the clinical experience (Fig. 2). These behaviors most often related to inability to perform a skill such as spelling (18.1.3), note writing (18.1.2), or manual muscle testing (10.1.1). Second most common were safety issues such as removing a knee immobilizer from a patient with a fracture (26.1.1). The 20 cognitive behaviors that were associated with a negative outcome most commonly included problem-solving difficulties such as not being able to figure out why a patient could not get out of bed (25.2.1) and safety problems such as forgetting hip precautions following surgery (22.1.4) and not knowing cardiac rehabilitation precautions (23.1.3).
In the poor communication category, 10 behaviors did not change with CI intervention, and 8 of those behaviors were associated with positive outcomes (Fig. 2). The most common behaviors in this category were those reflecting inappropriate interactions with patients and colleagues (eg, ignoring patients' questions and concerns) (25.1.4). The other behaviors were divided between inappropriate nonverbal behaviors such as not making eye contact (7.1.1) and inappropriate responses to correction such as crying in response to feedback from the CI (30.1.3). The 2 behaviors that led to a negative outcome were chattering with a distractible patient who had had a cerebrovascular accident (23.1.4) and telling a patient what to do without ever touching the patient (23.2.2).
Eleven behaviors in the unprofessional behavior category did not change with feedback, and 6 of those behaviors were associated with a positive outcome (Fig. 2). Nearly all of these behaviors involved not accepting responsibility, such as not getting an orthosis repaired for a patient (26.1.2) or refusing to perform a transfer (30.1.2). The 5 behaviors that were associated with a negative outcome primarily involved poor work ethic such as being late to meetings (4.1.1).
Based on these data, change in behavior following feedback always led to a positive outcome, but failure to change behavior following information about the behavior did not necessarily predict a negative outcome. For the noncognitive behaviors, positive outcomes occurred twice as often even if the student failed to change behavior following feedback.
Feedback not given by the clinical instructor.
In some instances, the CIs did not give specific feedback about particular behaviors. In the absence of feedback, some behaviors changed anyway and other behaviors did not change (Fig. 2). One of the behaviors that changed without feedback was in the inadequate knowledge and skills category, 5 behaviors were in the poor communication category, and 4 behaviors were in the unprofessional behavior category. In all cases, changed behavior was associated with a positive outcome for the clinical experience (Fig. 2). The inadequate knowledge and skills behavior was an efficiency problem (ie, not managing a full caseload) (21.1.8). The poor communication behaviors were mostly in interactions, such as being rude to the staff (21.1.2). The unprofessional behaviors all were related to poor stress management (eg, "getting red" and shaking during an evaluation) (29.1.3).
About half of incidents in which the CI gave no feedback and behavior did not change were associated with a positive outcome, and the other half were associated with a negative outcome. Those behaviors that were associated with a negative outcome involved 6 students. Six of the 27 behaviors were in the inadequate knowledge and skills category, and all 6 behaviors were associated with a negative outcome for the experience (Fig. 2). These behaviors were mostly related to poor problem solving (eg, reading exercises from a prepared sheet because the student could not figure out what exercises to do [9.1.1]) and inefficiency (eg, not being able to carry an appropriately sized caseload [25.2.3]).
Ten of the incidents that had no intervention and demonstrated no change were in the poor communication category; 6 of those behaviors were associated with positive outcomes (Fig. 2). Behaviors that were associated with a positive outcome were split between inappropriate interactions and inappropriate responses to feedback. They included mumbling during conversations (7.1.2) and a student turning his back on the CI during feedback (26.1.5). Those behaviors that were associated with a negative outcome primarily included inappropriate interactions such as screaming at a patient (23.3.2).
Eleven of the 27 incidents were in the unprofessional behavior category. Eight of the behaviors in this category were associated with a positive outcome (Fig. 2). Most were personal behaviors such as flirting with the nonprofessional staff (19.1.2) or behaviors related to work ethic such as not studying because commitments to friends took precedence (2.1.2) or using a CI's name as a reference without permission (19.1.4). The 3 behaviors that were associated with a negative outcome were related to accepting responsibility such as being unprepared for staffings (4.1.2).
Based on these data, we believe that when there is no discussion about a behavior related to knowledge and skills and the behavior does not change, a negative outcome is likely. For noncognitive behaviors, the outcome of the experience is twice as likely to be positive than negative.
Students with negative outcomes.
A total of 40 behaviors were associated with 8 students who had negative outcomes. The CIs made an effort to correct 27 of these behaviors, but they made no effort to correct the other 13 behaviors. The nature of these problems was discussed in the previous sections. Regardless of whether feedback was given, none of the behaviors changed. For 7 of the 8 students, inadequate knowledge and skills played a prominent role in their outcome, either solely or in combination with behaviors in other categories. Four of the 8 students were older than 25 years, 4 students were male, 2 students were nonwhite, and 5 students were enrolled in baccalaureate programs.
Three of the 8 students who the CIs reported had negative outcomes were dismissed from their professional programs. All 3 students had multiple, persistent problems, despite many educational interventions from their CIs in cooperation with the ACCEs. Two of the 3 students were in post-baccalaureate programs, 2 students were older than 25 years, and 2 students were nonwhite. Inadequate knowledge and skills was identified as the major reason for dismissal for 2 of the students, and poor communication in combination with unprofessional behavior was the reason for dismissal of the third student.
| Discussion |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionsReferences |
|---|
Students
There were more male students than expected based on the profile of student enrollment in professional physical therapist programs at the time of most of the incidents.26 The data in this study cannot explain the high proportion of male students. One possibility might be related to the practice settings represented in this study. Based on unpublished data, the areas of clinical practice with the largest concentration of 1994 male graduates were orthopedic and sports practices (Jody Gandy, personal communication, 1996). This type of practice is usually an outpatient facility. Most of the incidents in this study took place in inpatient and rehabilitation facilities. The incidents involving only 5 of 40 students took place in outpatient facilities. If a student's interest in a particular type of clinical practice were mismatched with the clinical assignment, inappropriate behaviors could result.
Other investigators have detected sex differences in various phenomena in physical therapy. Stith et al28 found that male physical therapist students are less satisfied with their clinical experiences than female students. We do not know whether dissatisfaction might be reflected in cognitive and noncognitive problems.
Learning preferences have been shown to differ between the sexes.29 A mismatch between student learning style and CI teaching style might lead to behavioral problems. Because most CIs are women, more men might be expected to experience mismatches. Further study is necessary to determine whether student or CI sex factors into clinical performance evaluation.
The number of nonwhite students among those with negative outcomes, including dismissal, also seemed higher than might be expected (2 of 8 students with negative outcomes and 2 of 3 students dismissed from their professional programs were nonwhite). The possibility of covert bias in clinical evaluation was raised in reports by Haskins and colleagues.30,31 Given the small numbers in our study and because we did not look for the presence of bias, we believe that any conclusions about bias would be inappropriate. In light of these reports, further investigation into the clinical experience outcomes of students of different racial backgrounds is merited.
Behaviors
The categories of behaviors generated in this study of factors that related to inadequate clinical performance were inadequate knowledge and skills, poor communication, and unprofessional behavior. These categories of behaviors are remarkably similar to those proposed by other authors.2,5,22 Communication, professionalism, commitment to learning, clinical judgment, communicating medical information, recognition of limits, interpersonal skills, and dealing with emergency situations have been supported as relevant to clinical competence, in addition to several cognitive items. All 10 of the generic abilities proposed by May and colleagues5 are included in the 3 categories generated in this study. Commitment to learning, professionalism, responsibility, and stress management are included in our unprofessional behavior category. Interpersonal skills, communication skills, and use of constructive feedback appear in our poor communication category. Effective use of time and resources, problem solving, and critical thinking are represented in our inadequate knowledge and skills category. The data from this study strongly support these generic abilities.
All of these previous investigations included noncognitive elements of clinical competence. In this study, noncognitive behaviors (poor communication and unprofessional behavior) comprised 56.7% of the behaviors that alerted CIs to unsafe and ineffective performance, yet they accounted for only 35.0% of behaviors that were associated with a negative outcome. We believe that this element of competence is valued by clinicians but infrequently used as a reason for recommending that a student's learning experiences be extended or terminated.
The prevalence of noncognitive factors in assessing clinical competence was observed by Cross and Hicks32 in their study of clinical educators' constructs of "good" and "bad" students. Eight constructs emerged in their study; those given the greatest weight and that were most discriminating between types of students were commitment, communication, and general disposition. These clinical educators weighted noncognitive factors as more important than cognitive factors in assessing student performance. There was no discussion of how these constructs related to student outcomes.32
The reasons why noncognitive behaviors are often considered in assessing student performance but not as determinants of student outcome are unclear. Some insight is provided by the CI who suggested that it is difficult for her to equate inappropriate noncognitive behavior and the ability to deliver safe and effective patient care (participant 23). In these days of litigation, perhaps CIs are concerned about being able to measure and document noncognitive behaviors. Incorrect information, skills performed in a manner that jeopardize a patient, or errors in decision making are easy to document and relate directly to patient outcome. It may be difficult, however, to document a negative effect on a patient outcome dependent on the student's communication and professionalism problems.
The presumed subjectivity of evaluation of noncognitive behaviors in decisions of student outcome was investigated by Duke6 in a study of clinical teachers in nursing. In her qualitative study, nurses felt a lack of confidence in their observations and decision-making abilities. When student nurses had difficulty in their psychomotor skills, the teaching nurses were able to correct them because they felt they had rational, objective evidence. Inevitably, when students had problems in values, attitudes, or caring, the clinical teachers passed the students, feeling that their judgments were subjective and not to be trusted. Each of the nurses in Duke's study indicated that her "gut reaction" was correct and had misgivings about passing the student.
Our study, along with the studies of medical residents by Altmaier et al2 and Hojat et al3 and of nurses by Duke,6 suggests that noncognitive factors should be given more attention by educators in the health care professions both during admissions and during the academic and clinical phases of professional education. The behaviors described in this report might be useful for identifying screening mechanisms to be used as part of admissions procedures, designing appropriate criteria for performance assessments in the academic setting, establishing policies for reviewing student performance based on their overall records, and developing more appropriate tools for assessing student performance in the clinic. The recently developed Physical Therapist Clinical Performance Instrument, for example, is supposed to address both cognitive and noncognitive skills18 but has yet to be validated through peer-reviewed publications.
Clinical Instructor-Student Relationships
The relationship between the CI and the student can have an impact on student behavior. Mismatches of various kinds could contribute to less-than-optimal student behavior. On occasion, personality conflicts between the CI and the student occur, leading to communication problems and unacceptable behaviors. Many of the clinical teachers in Duke's study referred to personality conflicts as a reason for student behavior, and the nurses assumed responsibility for the poor performance.6
Differences between students and CIs in the need to include and be included by others, the need to exert and desire control, and the need to express and receive affection can account for performance problems.33 When CIs' and students' needs do not match, the result can be communication problems, a perception that a student is disinterested, or the inability of the CI to provide constructive feedback about performance. Although we did not measure CIs' and students' needs in our study, mismatches in them could explain some of the problems encountered during clinical education.
Relationships Among Student Behavior, Clinical Instructor Feedback, Student Behavioral Change, and Outcome of the Clinical Experience
Feedback given by the clinical instructors.
In all categories, if students were given feedback about their behaviors and they corrected a behavior, a positive outcome occurred. Because most of the behaviors in the inadequate knowledge and skills category that changed with feedback related to inability to recall or safety, we believe that CIs felt these behaviors were correctable. In both noncognitive categories, the behaviors also appeared to be clear-cut actions toward which a CI could easily direct intervention. In the poor communication category, the behaviors included several instances of CIs drawing attention to nonverbal communication problems. Students may not be aware of such behaviors but can correct them once they are aware of them.
There are several reasons that behavior might not change with feedback, and the outcome of the experience may depend on the reason. When the outcome is positive, it is possible that the student was not in the final planned clinical experience. If a student demonstrates unacceptable behaviors, the CI may give feedback, and if the student's behaviors do not change, the CI may expect that deficiencies could be corrected in subsequent clinical experiences. Clinical instructors did not always indicate the clinical experience during which the incidents took place, so we do not know whether timing of the experience is a reasonable explanation.
Progression of students with uncorrected performance may also have occurred if CIs had difficulty recommending negative outcomes. Clinical instructors may feel that some behaviors are personally unacceptable but that they do not affect students' competence. As a result, although CIs might not ignore the behaviors, they might not penalize the student for them.
A CI may have worked very hard with a student and may have recommended a positive outcome whether the student's behavior changed or not. As one CI told us, "You don't want them not to finish. You want to kind of coach them through" (participant 23).
Several of the behaviors that were considered unacceptable, did not change, and still led to a positive outcome were related to 2 students who had disabilities. One disability was discovered as a result of the problems in the clinic; the other disability was known prior to the experience. Even though many of the behaviors were unrelated to the disability, it is possible that the CI or the school was concerned about showing bias if a negative outcome were recommended.
Even if a CI recommended a negative outcome, the professional program might not have agreed with the recommendation. For example, a CI stated, "We took [the problem] to the school administration, and there it was treated more as a lapse in the student's overall performance. We felt strongly that the student should be censured or even denied completion of the clinical affiliation. The long-range result was that the student was censured and counseled by the school. He did not have to repeat the affiliation, and he did graduate with his class" (6.1.1).
The behaviors that did not change in spite of feedback and were associated with negative outcomes were often serious problems related to problem solving and safety. They were also associated with multiple other behaviors and did not stand in isolation. In some cases, the collection of behaviors for an individual student indicated that the student had difficulty in being willing or able to change.
Feedback not given by the clinical instructors.
Most of the transcripts of interviews indicated whether a particular behavior was discussed with the student and the nature of the feedback offered by the CI. On occasion, however, the CI indicated that a specific behavior was noted but said nothing about whether the behavior had been discussed with the student. We interpreted these statements to mean that CIs noted the behaviors but did not discuss them specifically with students. This interpretation, however, may be erroneous.
We observed that most of the behaviors that changed even when the CI did not give specific feedback appeared to be related to the student being nervous and may have self-corrected over time during the clinical experience. Some behaviors appeared to have been single occurrences, so they would not have recurred.
If no feedback is given, students may have no opportunity to change their behavior. In about half of these incidents, students had positive outcomes in spite of no feedback and no change in behavior. There were several apparent reasons for this result. Some students had multiple behaviors for the CI to address, and CIs may have been selective about the behaviors on which they focused, overlooking other behaviors. Some CIs did not address certain behavioral problems because they felt uncertain or uncomfortable with them. For example, one CI talked of a student who neglected her patients because she spent her time flirting with the male staff. The CI said, "I didn't really know what to do, so ... I ignored it" (19.1.1). On occasion, the behavior that was reported was discovered only after the student had left the facility. For example, a student used the CI's name as a reference without her permission after she had graduated (19.1.4).
In about half of these situations in which no feedback was given and no change in behavior was observed, student behaviors were associated with negative outcomes. The behaviors in this group seemed to be less serious than the behaviors of the same students that the CIs chose to address. It is possible that the students demonstrated so many difficulties that the CIs were selective in the behaviors for which they chose to intervene. In addition, some of these deficits could have led to negative outcomes if the education program evaluated the student's performance lower than the CI did.
Limitations
Outpatient facilities may have been underrepresented in our study even though invitations to participate were issued to CIs at outpatient clinics. The data do not explain this apparent underrepresentation. Perhaps physical therapists in outpatient facilities find it difficult to be away from the clinic in order to participate in a study such as this one. Alternatively, perhaps fewer students have difficulties in outpatient clinical experiences.
The information from this study is admittedly limited by having no information from the students described or their ACCEs and no opportunity to follow up on the careers of those who graduated. Such a study would be valuable, though difficult to perform without breaching the confidentiality of the students.
| Conclusions |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionsReferences |
|---|
| Acknowledgments |
|---|
| Footnotes |
|---|
This study was approved by the Institutional Review Board of Northwestern University.
This study was supported in part by a grant from the Chicago Area Clinical Educators Forum.
Parts of this article were presented at the Combined Sections Meeting of the American Physical Therapy Association, February 14, 1997, Dallas, Tex, and at the Scientific Meeting and Exposition of the American Physical Therapy Association, May 31, 1997, San Diego, Calif.
| References |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionsReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  D. U Jette, A. Bertoni, R. Coots, H. Johnson, C. McLaughlin, and C. Weisbach Clinical Instructors' Perceptions of Behaviors That Comprise Entry-Level Clinical Performance in Physical Therapist Students: A Qualitative Study Physical Therapy, July 1, 2007; 87(7): 833 - 843. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. U Jette and L. G Portney Construct Validation of a Model for Professional Behavior in Physical Therapist Students Physical Therapy, May 1, 2003; 83(5): 432 - 443. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Strohschein, P. Hagler, and L. May Assessing the Need for Change in Clinical Education Practices Physical Therapy, February 1, 2002; 82(2): 160 - 172. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||
