Double illusion of transparency
In educational or explanatory settings, the double illusion of transparency is a version of the illusion of transparency. It refers to a situation where the teacher (the person who is doing the teaching or explaining) , as well as the learner (the person who is doing the listening or understanding), both falsely believe that that the material (that is being explained) is being understood as originally intended. Explicitly:
- The teacher believes that the learner is understanding the material correctly.
- The learner believes that he/she is understanding the material being taught correctly.
- In fact, both sides are communicating in a manner where each is affirming to the other person that the process of explaining and understanding is proceeding smoothly.
- However, the learner is not understanding the material correctly in the manner intended by the teacher.
Factors that promote the double illusion of transparency
Questions that can be answered without understanding
Often, the teacher asks the learner questions, ostensibly to test whether the other person is following. Poorly designed questions can lead to false confidence. Some examples of poorly designed questions are:
- Questions that can easily be answered through a recall, based on short term memory, of what the other person said very recently. For instance, a sentence like "Rooks are more valuable than bishops in chess" followed by the question "who is more valuable in chess: rooks or bishops?" can be answered by invoking short term memory and elementary sentence-parsing skills, without necessarily having an understanding of what the terms rook, valuable, chess, and bishop actually mean.
- Questions that test other forms of knowledge. For instance, a teacher solving an arithmetic problem may need to compute the sum of 14 and 11 in one step, and writes "14 + 11" at the appropriate place in the step. The teacher asks the learner what the next step is. The learner sees the 14 + 11, and correctly replaces it by 25. The learner has correctly carried out an addition, but this does not mean that the learner actually understands or remembers the algorithm in use or that the learner can execute it correctly.
- Questions that can be answered by random guesses or by reading contextual cues: For instance, yes/no questions can be answered with 50% likelihood by random guessing. If the teacher reveals some cues, for instance, with the choice of wording or intonation, the learner may, consciously or subconsciously, use those cues to get the correct answer.
- Questions that can be answered through the repetition of magic phrases.
All of the above question types are great for maintaining engagement (in large classroom settings, they are ideal for quick cold calling). They may also serve the purpose that as the learner answers the questions, his/her understanding improves in the process of doing so. The danger arises when correct answers to the questions are used diagnostically to reflect what is going on.
Instruction that fails to consider potential misconceptions and implicitly or explicitly rule them out
This typically happens when a given instruction is capable of being interpreted in an alternative way by somebody, either due to immediate misunderstanding, or due to misremembering later on. This typically happens more if the instructor does not promote deep learning.
Superficially perceptive questions by the learner
Learners may sometimes ask questions that are superficially perceptive, but that do not actually require them to have understood material. Usually, these questions draw on understanding of the general modes of representation used and not on what they represent in the specific situation. The teacher should not view these as evidence either for or against an understanding of the material.
Here are examples:
- The teacher draws the graph of a function that looks sort of like a straight line. The learner asks "is this a linear function?" This demonstrates a good understanding of graphs and functions, but the teacher should not misconstrue it as evidence that the learner has understood anything about what the graph is supposed to represent.
- The teacher is explaining social and economic life in a past historic era, and discussing a particular aspect of the social system. The learner asks, "What effect did this have on gender relations?" The question may seem perceptive, but a learner could ask that question without any understanding of the specifics of the social system.
Ways to avoid the double illusion of transparency
Start with a presumption of learning failure
The presumption of learning failure is the idea that learning should by default be assumed to not have occurred, and any claim to the contrary bears the burden of proof.
Ask diagnostic questions
The goal here is to choose questions that can only be answered through an understanding of the underlying concept. Some possibilities include:
- For "worked problem" contexts, ask the learner to work out the entire problem first, and check the work. The conjunction and ordering of multiple steps gives more of an opportunity to dissect potential misunderstandings.
- Ask factual or objective questions that are (i) separated in time from the time that the relevant knowledge is provided, (ii) framed in a way that requires the learner to reframe the knowledge in a manner that can easily be done only if the learner understands the material well.
- Ask for full sentence explanations that go somewhat beyond the statement just said. For instance, a statement such as "generally, rooks are more valuable than bishops in chess" can be followed by a two-part question: "who is more valuable in chess: the rook or the bishop?" and "why do you think this is so?" or "what are some exceptions to this rule?" Even if the learner stumbles over getting the correct answer to the second question, the extent to which the learner has a reasonable thought process about it reveals the learner's degree of understanding of the answer to the first question.
- In large classroom settings, ask harder questions and generate an interactive dynamic through polling, or sequential cold calling, or some other mechanism that gets people pondering the question.
Encourage deep listening on the part of learners
Some strategies here are:
- Promise the learner that what's being done will answer a specific question, so that the learner is listening and processing each piece of information to search for the answers to those questions. In classroom settings, one method is to circulate a class quiz in advance of the beginning of the class (perhaps during the previous class) with the promise that material covered in the class will help with the questions on that quiz, but without specifying, during the class, what material corresponds to what quiz question.
- If the learner asks a question requesting clarification, it may occasionally be best to not immediately provide that clarification. Rather, mention instances in the past or future that relate to an answer.
- Double Illusion of Transparency by Eliezer Yudkowsky