Instead, say: “… wears a sweater that hampers his movements; that reveals parts of his body.” For example, “We can see Jean’s navel when he lifts his arms up.”
UNIT 4
Interpreting the information
Orientation to Unit 4
In this unit, you are invited to reflect on various aspects that will lead to a sound interpretation of the facts you have collected. We will see how this step evolves toward a judgement about your trainee’s performance.
Interpretation
Introduction to interpreting information
In Unit 3, we covered several means of gathering facts on a trainee’s performance. The next step consists in interpreting these facts in order to make a judgement about the trainee’s competencies. Interpreting information is closely linked to the quality and diversity of the information gathered during the previous step.
In order to conduct a sound analysis of the information observed, the supervisor must consider various aspects, such has:
The steps involved in making a judgement
Cognitive biases
Ethical notions
Interpreting with the purpose of making a judgement
Interpretation is the step that precedes the judgement made about the trainee’s performance.
info_outline Instructions:
- Click on the boxes for more details.
- Click on the button “Close” or on the X to close the window.
Interpretation
Facts
Judgement
Download the printable version: Interpreting with the purpose of making a judgement (.pdf, 326 Ko).
Although these notions may be well understood, when the time comes to interpret information, some challenges do arise!
assignment Activity: Observation or unfounded opinion
info_outline Instructions:
- For each of the statements, identify whether it is an observation or an unfounded opinion.
- Correct the statements formulated as an unfounded opinion so they become observable.
Note: Your answers will not be saved.
As you probably can tell by completing this activity, it is not that easy to formulate observations as concrete facts.
When reformulating a statement by specifying the facts that you observed, read or hear it becomes an observation.
In reviewing the information collected (interpretation step), it is important that you only refer to observations that are adequately formulated to make a judgement that is as objective as possible.
Being objective: a challenge in itself!
Interpreting information related to your trainee’s knowledge is a step that poses challenges, especially considering the throng of information that needs to be analyzed.
Let’s examine the following situation, to gain a better understanding of the pitfalls to avoid when interpreting information.
assignment Activity: Mylène, the supervisor, interprets information
info_outline Instructions:
- Watch the following video by clicking on the button. A new window will open.
- To close it, click on the area outside the video.
- Click on
to turn on the "Subtitles/closed captions (c)" and select your language in the "Settings" 
. - To answer the question below, think about aspects of the situation that will influence Mylène’s interpretation of the information collected. Your answer will not be saved.
- See the feedback.
Feedback
Here, we can see that Mylène makes a judgement that is not based on facts. In addition, the interpretation of observations is influenced by her personal characteristics. But then again, how is it possible to disregard these?
Making an objective judgement: is it possible?
Objectivity in assessment is an issue that needs thorough reflection. How can a supervisor avoid a judgement that is too subjective? Total objectivity is impossible, because you cannot erase your values and emotions. However, accepting that part of a professional judgement is subjective does not invalidate it.
In fact, being aware of the subjective aspect of your judgement will incite you to analyze and clarify your judgement even further during an assessment, so you can be as objective as possible. In this regard, strategies suggested in the previous steps of planning and collecting information still remain foundational. Your judgement will be more objective if you:
- Carry out ongoing assessments
- Record relevant information on the trainee’s behaviours as they occur, etc.
As you can see, making a judgement on a trainee’s performance requires a structured conduct and in-depth reflection. The ongoing assessment process depends on it, as does the trainee’s learnings. Interpretation forms the basis for judgement, and ultimately the grade that will be awarded to the trainee. Let’s look at other aspects that could influence the supervisor’s judgement during an assessment.
Being aware of our own cognitive biases
As we just saw, even though we may have the best intentions in the world, interpreting information and the ensuing judgement involve some degree of subjectivity. Some factors can skew an assessment when one person assesses another person. They are called “cognitive biases”. In most cases, the individual is not conscious of what is happening. However, being aware of our own cognitive biases is the best way to counter them.
In addition, it was established that individuals who participated in training on the matter are more skilled in mitigating their cognitive biases.
Cognitive biases are classified in different ways. We suggest four categories, addressing the most common cognitive biases in supervisors when assessing their trainees.
| Category | Related to judgement | Related to memory | Related to the assessor’s tendencies | Related to circumstances |
|---|---|---|---|---|
| Definition | Some pieces of information are valued at the expense of others, which alters the supervisor’s ability to make decisions. | Vivid or repetitive memories are more easily remembered, while others are eliminated, which influences the supervisor’s choices. | The supervisor has some personal inclinations (regarding assessment, culture, gender, etc.), which influence his decisions and behaviour. | The context in which a situation occurs influences the supervisor’s assessment of said situation. |
| Cognitive biases |
|
|
|
|
References
In the previous section, you explored general strategies that promote objectivity. Following are definitions of several biases, as well as specific strategies that will help you mitigate your cognitive biases.
info_outline Instructions: Click on each circle to know more about different biases, including their definition, examples and strategies.
1. The halo effect (or generalization)
Tendency to make an overall judgement on performance, based on one or two aspects, either because you find them more significant, or because they are more noticeable. As a result, the trainee is underestimated or overestimated on the basis of a small number of aspects that do not represent his true performance.
For example:
Your trainee’s overall performance is satisfactory, but he does not master one of the interventions. You give him a C.
To counter this bias:
Consult all of your sources to fully review the observed facts.
2. The hawk-dove effect
Tendency to show too much severity or leniency. Unlike other cognitive biases, the hawk-dove effect often occurs consciously.
The “hawkish” supervisor pays much more attention to the trainee’s mistakes than to his successes and has higher standards than average. As a result, the trainee is underestimated.
Conversely, the supervisor said to be a “dove” is too lenient toward the trainee, because he is reluctant to give a negative assessment. As a result, the trainee is overestimated.
For example:
- You rate your trainee’s performance as “adequate”, since he made a few mistakes, even though he is generally very good.
- Your trainee made several mistakes, but he is only completing his second placement, so you let it go and do not mention it in his assessment.
To counter this bias:
Pay special attention to the trainee’s strengths, as well as his challenges before taking a stand.
3. The central tendency error
Tendency to keep the trainee’s assessment at an average level even when the performance level varies, either to satisfy everyone or because you lack experience in assessment.
For example:
You give a 7.5 on 10 for various criteria assessed. As a result, your trainee will not ask too many questions.
To counter this bias:
- Use multiple choice nominal scales, without specific grades (e.g. very satisfactory, satisfactory, unsatisfactory, very unsatisfactory).
- Use descriptive scales listing expected behaviours
4. The anchoring effect
Tendency to assess the trainee on the basis of the initial information (first impression) available to you. You then focus on the information that is likely to confirm your initial judgement and ignore information that contradicts this initial judgement.
For example:
On his first day, your trainee did not conform to the dress code. You gave him a poor grade regarding respect for the institution’s labour standards.
To counter this bias:
Introduce other assessment sources, such as self-assessment or 360° assessment.
5. The recency effect
Tendency to assess the trainee according to the most recent information you have. As a result, a recent event, whether positive or negative, takes disproportionate importance in your mind, thus concealing facts that occurred prior to this event.
For example:
Your trainee is usually able to complete files. However, the file he submitted this week was not really thorough. You give him a C for clinical documentation.
To counter this bias:
Review your notes to read and remember all of the facts.
6. The contrast effect
Tendency to assess the trainee against a standard model (another trainee seen previously or yourself) rather than against predetermined criteria.
As a result, you assess performance based on the gap between what the trainee accomplished and what the standard model would have done.
For example:
- You rate the trainee’s performance as adequate, because he does possess the competencies that you had when you were a trainee.
- When a strong trainee follows a weaker trainee, the strong trainee may be perceived as even stronger than he really is. The reverse is also true.
To counter this bias:
Avoid comparing trainees’ performance to one another or comparing their performance to your own.
7. The contamination effect
Tendency to be influenced by the trainee’s past assessments (good or bad) during the current assessment. For example, you may have heard about this trainee from a colleague who supervised him previously.
For example:
Eric, your trainee, was under the supervision of your colleague Antoine in his previous placement. Antoine mentioned that Eric had trouble taking blood samples. When assessing the trainee, you note that this intervention “needs improvement”.
To counter this bias:
Put on “blinders” when you hear about your trainee’s past performance in previous placements.
8. The mirror-imaging effect (or similarity to the assessor error)
Tendency to overestimate the performance of a trainee with whom you have affinities (shares your values or personality traits).
For example:
Your trainee is shy with patients and it hinders his way of communicating. Being shy yourself, you understand he is doing his best and you give him a good grade for communication.
To counter this bias:
Introduce other assessment sources, such as self-assessment and 360° assessment.
9. The intercultural bias
Tendency to apply your cultural expectations when assessing someone whose beliefs and expectations are fundamentally different.
Intercultural biases can be reflected in how words are interpreted (when French is not the trainee’s first language), in the importance attached to success and how feedback is provided to the trainee (some cultural groups will expect the supervisor to start with strengths, while others emphasize challenges).
For example:
Your trainee is of Asian origin and French is his third language. When you assessed his clinical reasoning, his answers were incomplete. You gave him an “unsatisfactory” rating for this aspect, without knowing whether he simply had not understood the questions you asked him.
To counter this bias:
- Enroll in training on cultural diversity.
- Develop cultural sensitivity.
10. The attribution effect
Tendency to state that a trainee shows poor performance, while it actually results from the context (using obsolete equipment, lack of resources, etc.).
For example:
Most of the equipment in your clinic is obsolete. Your trainee does not master its use since he has never seen it in his courses. As a result, he makes several mistakes during his interventions. You note that he does not have the required competencies for his level.
To counter this bias:
- When assessing performance, take into account the context of the situation
- Put yourself in your trainee’s shoes
11. The social desirability bias
Tendency to overestimate the trainee in order to portray yourself positively. You want to be well recognized and repress negative comments, keeping only those that will please your trainee.
Unlike other cognitive biases, the social desirability bias often occurs consciously.
For example:
You feel that your trainee is distant, that he does not appreciate you very much. You do not want him to despise you, thus you avoid giving him feedback or poor grades.
To counter this bias:
- Determine your level of social desirability by using a scale, such as the abridged version of the Balanced Inventory of Desirable Responding.
- Take time to reflect on the impact of your level of social desirability on your interpretation.
12. The “order” effect
Tendency to overestimate or underestimate your trainee’s assignments according to their position in the stack of copies.
This effect is only present in some professions, when a professional supervises a group of trainees and corrects their written assignments.
For example:
- First, you correct Stephane’s work and it is of very high quality. Then you correct Emilie’s work, which is clearly inferior to Stephane’s paper. Thus, you are more critical in your rating.
- You first correct Marc’s paper, which does not meet expectations at all. You then review Elise’s work, which is of higher quality and seems exceptional against Marc’s assignment. Thus, you are more indulgent in your rating.
To counter this bias:
Avoid comparing trainees’ paper to one another.
Download the printable version: Being aware of our own cognitive biases (.pdf, 277 Ko).
assignment Activity: Which cognitive bias is it?
info_outline Instructions:
- Watch the following simulations by clicking on the button. A new window will open.
- To close it, click on the area outside the video.
- Click on to turn on the "Subtitles/closed captions (c)" and select your language in the "Settings" .
- Select the cognitive bias represented in the video.
- Feedback will appear automatically.
Note: Some biases are not covered.
Ethics in assessment, it’s fairer!
In performance assessment, the ethical aspect is a concept that is not very widely studied in the education field. However, this dimension transpires throughout the assessment process.
Indeed, ethics comes into play in various areas of an assessment: the significance of failure, affective issues, bargaining a grade, the confidential or public nature of an academic result, justice, the value of self-assessment, objectivity, etc..
In addition, the application of ethical principles is at the core of a fair and quality assessment.
As professionals, we know that we must demonstrate ethical behaviours, because a code of conduct (or code of ethics) guides our actions. In supervision, supervisors must behave ethically at all times. However, the assessment process must rely on specific ethical principles, particularly when making a judgement. Yet, many supervisors doubt their own judgement, because they know that part of it is arbitrary.
assignment Activity: Behaviours: ethical or not?
info_outline Instructions:
- Watch the following video by clicking on the button. A new window will open.
- To close it, click on the area outside the video.
- Click on to turn on the "Subtitles/closed captions (c)" and select your language in the "Settings" .
- Answer the question.
- Read the feedback.
Feedback
You probably mentioned the fact that Patricia’s approach lacks rigour, but also that she fails in her duty to ensure confidentiality and transparency.
A supervisor that has limited knowledge in assessment may unwillingly adopt a less rigorous ethical approach. However, we have already seen quite a few strategies that foster a rigorous assessment process. These strategies will allow you to support your trainee from an ethical point of view.
For example, we suggested that you:
- Plan the assessment process (Unit 2)
- Frequently review assessment tools, criteria, indicators, the frequency of assessments and assessment forms (Unit 3)
- Review assessment methods, strategies, etc. on a regular basis (Unit 3)
- Recognize that assessments are partly subjective (be mindful of biases!)
However, we have scarcely touched on two other ethical principles shown in the video, that we also need to consider:
- Respecting confidentiality
- Demonstrating transparency
Confidentiality
What is the supervisor’s duty of confidentiality when he is completing the judgement step?
It implies that the supervisor is committed to taking actions that will conform to guidelines that are usually established in that regard by the institution.
In the video, Patricia talks about her trainee’s performance with her colleague Veronique, who clearly does not need to be apprised of the situation. In addition, the discussion takes place in a public area.
In order to respect confidentiality in assessment, the supervisor can, among other things:
- Collect only the information that is relevant to the placement
- Share the information he collected and his judgement about the trainee only with people who are justified to know this information
- Protect assessment documents so they are not freely available
Transparency
How does the principle of transparency come into play when it is time to make a judgement?
If the supervisor demonstrates transparency, it means that the trainee knows exactly what to expect from his assessment. He knows which tools will be used to assess his competencies and the objectives or criteria that form the basis for his assessment.
In the video, Patricia does not seem to have taken time to clearly explain the assessment process to her trainee; she even hopes she won’t surprise him when she shares her judgement.
In fact, the principle of respect for transparency implies that the trainee will not be surprised with new elements regarding his assessment. The supervisor will have communicated clearly and in advance the details of the assessment process.
In this regard, the contract discussed in Unit 2 is a good way of demonstrating transparency.
Key concepts
In order to adequately interpret the information you collected and to make a fair judgement, it is important to remember that:
- Judgement consists in drawing a conclusion or stating an opinion about the acquired competencies; it must be based on measurable facts.
- To maintain objectivity in his judgement, the supervisor must accept the existence of subjectivity, while being mindful of his personal filters and avoiding the selection of facts and generalizations.
- Being aware of one’s own cognitive biases is the best way to counter them.
- Ethical notions of thoroughness, transparency and confidentiality must be at the forefront of an objective judgement.
