Cold calling

This article is about a format used within an exposition for the explainer to get feedback from the learner(s), and/or for learner(s) to self-assess.
View list of in-exposition feedback formats | View summary table of learning techniques (not limited to in-exposition feedback formats)

Definition

Cold calling refers to a protocol used in a class (generally, as part of in-class explainer exposition) where the explainer calls on individual learners to answer questions posed by the explainer on a regular basis. The key feature of cold call is the unpredictability about who will get called for a particular question, or equivalently, the unpredictability for each individual about when he or she will get called.

Cold calling (or rather, whether and how it is used) is an important element of classroom instruction as performance.

Learning contexts

Significance of class setting

The nature of cold calling -- the uncertainty of who will be called when -- makes it suitable only for a class setting, and more specifically, for the in-class explainer exposition period, where the explainer is controlling the visual and auditory cues for the learners.

In particular, it relies on each of the three defining ingredients of the class, namely:

Feature of class	How it's important to cold calling	What we would do instead if this was missing
multiple learners	Each learner has uncertainty about whether and when they will be called upon; a high overall cold calling frequency can be supported with each learner being called upon relatively rarely.	Cold calling with just one learner may better be framed as an interactive prompt. The dynamics are a little different because of greater call load per capita, or reduced call frequency.
synchronicity	The synchronous nature of the class setting is important because learners are being asked in real time, and not only that, get to know in real time how other learners are understanding the material.	Without synchronicity, it is better to implement a quiet prompt or interactive prompt.
explainer authority	The explainer's ability to call on individual learners -- without the voluntary initiative of the learners in that moment -- relies on explainer authority.	Reduced explainer authority would push in favor of voluntary participation. Related methods Related methods in class Polling (including clicker polling) Group calling Voluntary participation Quiet prompt (this can also be used outside class) Related methods outside class Quiet prompt Interactive prompt Advantages Cold calling is advantageous primarily because it dramatically increases the extent to which learners are required (or at any rate nudged) to paying attention and actively thinking about the material being covered in class. With a method such as voluntary participation, only the learners who choose to participate and think they have a shot at getting the answers will participate. On the other hand, with cold calling, all learners, knowing that they can be called upon, are listening closely and working out the answers. Cold calling might also help break the ice in the classroom and make learners more comfortable interacting with each other and the explainer. If executed well, people can feel more confident. Cold calling is a great way for everybody to get to know each other, connecting name, face, voice, and some features of their answers. Explainers have a constant and regularly updated idea of how well the class is understanding specific ideas, and is therefore less likely to succumb to the illusion of transparency. Learners have a constant and regularly updated idea of how well they and their peers are understanding specific material, and are therefore less likely to succumb to the double illusion of transparency. Note however, that cold calling using questions that are not of diagnostic use could cement, rather than shatter, the double illusion of transparency. Disadvantages If poorly executed, cold calling could generate resentment among learners, leading to learners missing class, or adopting an adversarial attitude to the explainer. Cold calling could be construed as an abuse of the explainer's authority or power over the learners. Cold calling could disorient explainers and learners if it shatters the illusion of transparency, causing a lecture to go off-plan. Cold calling could waste a lot of time. If learners sense that poor performance on cold calling will cause the explainer to go slower and cover less material, they are incentivized to perform poorly. Worst practices and pitfalls Cold calling is generally viewed negatively due to some perceived connotations. However, most of the negative connotations are generally avoidable. Some pitfalls to avoid with cold calling are: Do not be judgmental or negative at learners for passing on cold call questions, or for answering them incorrectly. If learner performance on a particular cold call question is cause for concern (e.g., you feel the learner doesn't understand a basic prerequisite for the course), then talk to the learner privately after class in a non-threatening manner. Do not spend too much time on a particular learner (with all others' eyes on that learner), even in an apparently positive manner, because some learners may react negatively but hide it. Do not force learners to participate against their will. Offer learners reasonable opt-out opportunities, either at the beginning of class or at the begining of the course (note that in settings where you want to teach classroom discipline in addition to the subject matter, you might disallow opting out; see no opt out). If overall performance on cold call questions is bad, do not blame the learners at large or otherwise lose your cool. Point out that the cold calling has helped you discover an area of learner weakness, and cooperatively attempt to find ways to remedy the problems so that you can proceed. Take part of the blame ("I didn't explain this clearly enough" or "I'll try to think of a better way of explaining it") to make the cooperative aspect more salient than any confrontational connotations learners might read. Do not use cold calling as a gotcha to catch learners who you think were not paying attention. This doesn't mean you shouldn't cold call learners who you think weren't paying attention, but you shouldn't do so with the explicit purpose of shaming them publicly. If they admit that they weren't paying attention, then be gracious about it (for instance, repeat the question, or let them pass on it, promising to come back to them soon). Parameters relevant to individual cold call instances The implementation of cold calling can be tweaked in a number of ways. Some of these are described below. Reveal point The explainer can choose different possibilities in time for when to reveal who is being called to answer the question. Below are some possibilities: Reveal point Advantage Disadvantage Before stating the question to the class The individual learner can listen clearly to the question and need not ask the explainer to repeat It forgoes one of the main benefits of cold calling: all learners are trying to think of the answer to every question. Instead, since the name of the person answering is specified in advance, other learners may reduce their level of attention. Within a few seconds of stating the question to the class All learners have had time to hear and process the question Individual learners may not have been paying concentrated attention since they didn't know for sure that it's a question they might answer. Thus, the learner who is asked the question may ask the explainer to repeat it. Give a chunk of time to everybody to work out the answer (this may involve flipping through recent or earlier notes, or doing calculations, or taking time to formulate thoughts), and then reveal who'll be called All learners have had time to hear and process the question and attempt an answer. Thus, they can grade themselves by proxy upon hearing the correct answer. The technique can also be combined with in-class desk work either before or after the cold call. This takes more time, and learners may slack off in that timespan of a few seconds if the material seems too easy or too difficult. Dealing with non-response The explainer can choose different possibilities for what to do if the learner says "pass" or "I have no idea" or "not sure" or something equally non-informative. The strategies include: Strategy Advantage Disadvantage Press the learner to try more (this may be accompanied with allotting a bit of extra time, providing a hint or cue, or modifying the question to make it somewhat easier) sends learners a message that partial progress is possible avoids sending the message that learners can easily pass in some cases, the learner just isn't equipped to answer the question, and pressing the learner can waste time and generate resentment. Deprives other learners of opportunity to answer, and therefore deprives them of incentives to work towards an answer. Cold call another learner results in more efficient reaching of the answer sends message to learners that they should keep thinking about how to solve the question even if somebody else is being asked the question. sends message to learners that they can easily "pass" on questions. Switch to seeking voluntary participation results in more efficient reaching of the answer undermines part of the psychological pressure of cold calling (and the incentives it generates for paying attention), to a greater extent than the two preceding methods Learners may not be willing to volunteer answers Return to either revealing the answer or re-teaching a relevant portion that makes the answer easier to obtain efficient use of time if a fixed amount of material needs to be covered undermines the psychological pressure of cold calling and the incentives it generates for paying attention. Dealing with responses Learners may give answers that are partly or wholly correct. They may give partial answers with promise. There are multiple strategies for dealing with the variety of answers possible. Strategy Advantage Disadvantage Press the learner more, allowing the learner to completely arrive at the answer. sends a "can-do" message encourages partial progress deprives other learners of the incentive to work out the answer may take too much time Do not immediately reveal your opinion on the response, but treat it as provisionally correct and work through the consequences. In case it is wrong, you should arrange the "work through the consequences" part in a manner that the contradiction emerges soon enough for the learner to realize it on his or her own. gives the learner (and others) the opportunity to debug and determine the point of failure may take more time people may get confused about what parts are correct and what parts are the consequences of the incorrect statement. The confusion may be temporary, but it may also result in later confusion if people note down the incorrect step. This can be remedied by clearly marking the incorrect part (if writing it on the board) and reminding learners to update their notes to reflect that that part was incorrect. Do not immediately reveal your opinion on the response (i.e., whether it was correct, complete, etc.). Instead, seek answers from other learners (using additional rounds of cold calling, polling, or voluntary participation). all learners continue thinking about the question, and also learn to evaluate the plausibility of existing answers takes more time, particularly if the original answer is correct Reveal minimal information (such as "right" or "wrong") and, if wrong, solicit more responses through cold calling, polling, or voluntary participation all learners continue thinking about the problem, but the possibility space narrows somewhat, allowing learners to discard some conjectures and hone in on others Takes more time compared to just revealing the answer destroys the suspense compared to soliciting multiple answers before revealing "right" or "wrong" Reveal the answer immediately takes less time reduces opportunities for learners to think about and arrive at the answer Learner and question selection parameters Matching learners with questions One of the harder problems in effective cold calling is to determine how learners and questions should be matched. There are two broad strategies: Strategy Advantage Disadvantage Match by difficulty, i.e., the distribution of questions a learner sees is adjusted to the competency demonstrated by the learner. This is done adaptively, rather than based on a static assessment of the learner. learners get questions they can reasonably attempt, but that are not too easy for them learners quickly "place" themselves based on the difficulty of the questions they are getting. Learners getting easy questions correctly infer that the explainer has a low opinion of their competency, and this may turn into a self-fulfilling prophecy (the pygmalion effect in reverse). Give similar questions to everybody fair and easy to implement learners may feel embarrassed at frequently getting questions they are stuck at, and others may be bored at getting questions that are too easy. Frequency of calling for individual learners Should all learners be called equally? There are two broad strategies: Strategy Advantage Disadvantage adjust the degree of cold calling to the degree of enthusiasm displayed by the learner makes for a more harmonious interaction. Some learners are okay with being cold called occasionally, but not frequently. Others love to be cold called. learners who are cold called infrequently lose part of the benefits of "always being on one's toes" that are conferred by cold calling. cold call everybody equally fair and equitable some learners may resent being cold called (one solution is to cold call everybody equally except learners who explicitly opt out; however, this fails to address the needs of learners who would like to be cold called but only occasionally). Allow for opting in or out Explainers may give learners the option of opting out of being cold called, either during a given lecture (for instance, if the learner is feeling sick or stressed out) or for the entire course. Some explainers may make cold calling an "opt-in" feature. However, this runs the risk of very few people choosing to opt in and the system thereby not being too different from voluntary participation. Difficulty distribution of questions The questions used for cold calling can range based on parameters such as: Competency level needed to answer. Time needed to answer. How far back in time the tested ideas were covered (easy means things that were either covered very recently or that are part of the learner's firm knowledge background; hardest is the stuff that was done a few days ago but that the learner has not yet mastered). Participation grade Some explainers may make performance on cold calling as part of a participation grade. Other explainers may choose to not use performance on cold calling as an input to determining the grade at all. Atypical forms of cold calling Cold calling about self-evaluation The questions or comments? question, instead of being addressed to the class at large, may be used as a cold calling question. Similar cold call questions include: Do you feel like you understood this? Do you think you will be able to solve this type of question? Such self-evaluation questions might on occasion be followed by demands for further explanation in the case that the learner says that he/she has understood the material, and on occasion be followed by a repetition of the material for the whole class in the case that the learner confesses confusion. This form of cold calling works best if it is clear to the learner that the explainer genuinely wants to know and is not merely looking for a pre-determined answer. However, the notorious unreliability of self-evaluation, as well as the double illusion of transparency, mean that mere subjective self-evaluation should not be weighted too highly. Cold calling about other forms of subjective experience This may include questions like: How does the topic compare in difficulty to topic X? Do you like this? Such questions are best not overdone, but they can be useful as an occasional complement to the rest of the class material. Investment and effort needed Effort on the teacher's part While some preparation on the teacher's part can make cold calling more effective, the bare minimum of preparation needed for cold calling to be effective is quite low. It is preferable that the teacher either know the names and faces of all learners, or have access to a printed photo roster that he or she can scan quickly. Pointing to learners for each cold call instance can be clumsy. However, this is not a dealbreaker, and teachers who want to create a precedent of using cold calling right from the beginning should not wait to master the names and faces of all learners before beginning. Eye contact and pointing might be necessary in the beginning. Having a sense of the cognitive profiles of learners can be helpful if matching learners to questions based on the level of difficulty. However, this again is not a necessity. Some teachers keep track (using tally marks or other means) of how much they are cold calling each learner. This can help them figure out if they are cold calling learners equally. However, this again is optional and may not be suited to all circumstances. Some teachers plan beforehand who they will cold call when (i.e., they include cold calling in their class screenplay), but this is not necessary and its advantages over making cold calling decisions in real time is unclear. In practice Identification as a technique used by good teachers In his book Teach Like A Champion Doug Lemov identifies cold calling as one of the 49 techniques used by champion teachers. Lemov's book is targeted at school level teachers. Effect on voluntary participation There are conflicting a priori theoretical intuitions regarding the effect of cold calling on voluntary participation. On the one hand, by breaking the ice, cold calling may encourage voluntary participation. On the other hand, cold calling may discourage voluntary participation by substituting away from it. A study claims to find a positive relationship between cold calling and voluntary participation over time.