Difference between revisions of "Double illusion of transparency"

From Learning
Jump to: navigation, search
Line 7: Line 7:
 
* In fact, both sides are communicating in a manner where each is affirming to the other person that the process of explaining and understanding is proceeding smoothly.
 
* In fact, both sides are communicating in a manner where each is affirming to the other person that the process of explaining and understanding is proceeding smoothly.
 
* However, the person listening is not really understanding what the other person is trying to explain.
 
* However, the person listening is not really understanding what the other person is trying to explain.
 +
 +
==Factors that promote the double illusion of transparency==
 +
 +
===Questions that can be answered without understanding===
 +
 +
Often, the person explaining asks the other person questions, ostensibly to test whether the other person is following. Poorly designed questions can lead to false confidence. Some examples of poorly designed questions are:
 +
 +
* Questions that can easily be answered through a recall, based on [[short term memory]], of what the other person said ''very recently''. For instance, a sentence like "Rooks are more valuable than bishops in chess" followed by the question "who is more valuable in chess: rooks or bishops?" can be answered by invoking short term memory and elementary sentence-parsing skills, without necessarily having an understanding of what the terms ''rook'', ''valuable'', ''chess'', and ''bishop'' actually mean.
 +
* Questions that test other forms of knowledge. For instance, an instructor solving an arithmetic problem may need to compute the sum of 14 and 11 in one step, and writes "14 + 11" at the appropriate place in the step. The instructor asks a student what the next step is. The student sees the 14 + 11, and correctly replaces it by 25. The student has correctly carried out an addition, but this does not mean that the student actually understands or remembers the algorithm in use or that the student can execute it correctly.
 +
* Questions that can be answered by random guesses or by reading contextual cues: For instance, yes/no questions can be answered with 50% likelihood by random guessing. If the person explaining reveals some cues, for instance, with the choice of wording or intonation, the other person may, consciously or subconsciously, use those cues to get the correct answer.
 +
 +
All of the above question types are great for maintaining engagement (in large classroom settings, they are ideal for quick [[cold calling]]). They may also serve the purpose that as the students or listeners answer the questions, their understanding improves in the process of doing so. The danger arises when correct answers to the questions are used ''diagnostically'' to reflect what is going on.
 +
 +
===Instruction that fails to consider potential misconceptions and implicitly or explicitly rule them out===
 +
 +
This typically happens when a given instruction is capable of being interpreted in an alternative way by somebody, either due to immediate misunderstanding, or due to misremembering later on. This typically happens more if the instructor does not promote [[deep learning]].
 +
 +
==Ways to avoid the double illusion of transparency==
 +
 +
===Ask diagnostic questions===
 +
 +
The goal here is to choose questions that can ''only'' be answered through an understanding of the underlying concept. Some possibilities include:
 +
 +
* For "worked problem" contexts, ask the listener or student to work out the entire problem first, and check the work. The conjunction and ordering of multiple steps gives more of an opportunity to dissect potential misunderstandings.
 +
* Ask factual or objective questions that are (i) separated in time from the time that the relevant knowledge is provided, (ii) framed in a way that requires the listener to reframe the knowledge in a manner that can easily be done only if the listener understands the material well.
 +
* Ask for full sentence explanations that go somewhat beyond the statement just said. For instance, a statement such as "generally, rooks are more valuable than bishops in chess" can be followed by a two-part question: "who is more valuable in chess: the rook or the bishop?" and "why do you think this is so?" or "what are some exceptions to this rule?" Even if the listener stumbles over getting the ''correct'' answer to the second question, the extent to which the listener has a reasonable thought process about it reveals the listener's degree of understanding of the answer to the first question.
 +
* In large classroom settings, ask harder questions and generate an interactive dynamic through polling, or sequential cold calling, or some other mechanism that gets people pondering the question.
 +
 +
===Refrain from answering requests for clarification immediately===
 +
 +
In some cases, if the listener asks a question requesting clarification, it's best to ''not'' immediately provide that clarification. Rather, mention instances in the past or future that relate to an answer.
  
 
==External links==
 
==External links==
  
 
* [http://lesswrong.com/lw/ki/double_illusion_of_transparency/ Double Illusion of Transparency] by Eliezer Yudkowsky
 
* [http://lesswrong.com/lw/ki/double_illusion_of_transparency/ Double Illusion of Transparency] by Eliezer Yudkowsky

Revision as of 05:20, 6 September 2013

Definition

In educational or explanatory settings, the double illusion of transparency is a version of the illusion of transparency. It refers to a situation where the person who is doing the teaching or explaining, as well as the person who is doing the listening or understanding, both falsely believe that that explanation is being understood as originally intended. Explicitly:

  • The person explaining believes that the other person is understanding it correctly.
  • The person listening believes that he/she is understanding the other person correctly.
  • In fact, both sides are communicating in a manner where each is affirming to the other person that the process of explaining and understanding is proceeding smoothly.
  • However, the person listening is not really understanding what the other person is trying to explain.

Factors that promote the double illusion of transparency

Questions that can be answered without understanding

Often, the person explaining asks the other person questions, ostensibly to test whether the other person is following. Poorly designed questions can lead to false confidence. Some examples of poorly designed questions are:

  • Questions that can easily be answered through a recall, based on short term memory, of what the other person said very recently. For instance, a sentence like "Rooks are more valuable than bishops in chess" followed by the question "who is more valuable in chess: rooks or bishops?" can be answered by invoking short term memory and elementary sentence-parsing skills, without necessarily having an understanding of what the terms rook, valuable, chess, and bishop actually mean.
  • Questions that test other forms of knowledge. For instance, an instructor solving an arithmetic problem may need to compute the sum of 14 and 11 in one step, and writes "14 + 11" at the appropriate place in the step. The instructor asks a student what the next step is. The student sees the 14 + 11, and correctly replaces it by 25. The student has correctly carried out an addition, but this does not mean that the student actually understands or remembers the algorithm in use or that the student can execute it correctly.
  • Questions that can be answered by random guesses or by reading contextual cues: For instance, yes/no questions can be answered with 50% likelihood by random guessing. If the person explaining reveals some cues, for instance, with the choice of wording or intonation, the other person may, consciously or subconsciously, use those cues to get the correct answer.

All of the above question types are great for maintaining engagement (in large classroom settings, they are ideal for quick cold calling). They may also serve the purpose that as the students or listeners answer the questions, their understanding improves in the process of doing so. The danger arises when correct answers to the questions are used diagnostically to reflect what is going on.

Instruction that fails to consider potential misconceptions and implicitly or explicitly rule them out

This typically happens when a given instruction is capable of being interpreted in an alternative way by somebody, either due to immediate misunderstanding, or due to misremembering later on. This typically happens more if the instructor does not promote deep learning.

Ways to avoid the double illusion of transparency

Ask diagnostic questions

The goal here is to choose questions that can only be answered through an understanding of the underlying concept. Some possibilities include:

  • For "worked problem" contexts, ask the listener or student to work out the entire problem first, and check the work. The conjunction and ordering of multiple steps gives more of an opportunity to dissect potential misunderstandings.
  • Ask factual or objective questions that are (i) separated in time from the time that the relevant knowledge is provided, (ii) framed in a way that requires the listener to reframe the knowledge in a manner that can easily be done only if the listener understands the material well.
  • Ask for full sentence explanations that go somewhat beyond the statement just said. For instance, a statement such as "generally, rooks are more valuable than bishops in chess" can be followed by a two-part question: "who is more valuable in chess: the rook or the bishop?" and "why do you think this is so?" or "what are some exceptions to this rule?" Even if the listener stumbles over getting the correct answer to the second question, the extent to which the listener has a reasonable thought process about it reveals the listener's degree of understanding of the answer to the first question.
  • In large classroom settings, ask harder questions and generate an interactive dynamic through polling, or sequential cold calling, or some other mechanism that gets people pondering the question.

Refrain from answering requests for clarification immediately

In some cases, if the listener asks a question requesting clarification, it's best to not immediately provide that clarification. Rather, mention instances in the past or future that relate to an answer.

External links