Concept inventory

From Learning
Revision as of 23:59, 8 September 2013 by Vipul (talk | contribs) (Design)
Jump to: navigation, search

Definition

A concept inventory is a criterion-referenced assessment designed to evaluate whether a student has a specified level of mastery of a specific set of concepts. Most concept inventories use multiple choice questions, but concept inventories using essay responses and oral interviews have also been designed.

Design

Low loading on non-conceptual aspects

Since concept inventories are focused on testing for an understanding of the concepts, they are generally designed to have low loading on other skills that most tests load on. For instance, concept inventory scores should be as unaffected as possible by:

  • Fluid intelligence, including working memory, visual memory, spatial manipulation, etc.
  • Problem-solving practice

A marker of a good concept inventory question is the gap in answering ability and speed between an expert in the field and somebody who has recently learned and extensively practiced the material. The expert could even be "handicapped" -- for instance, tested at a time when he or she is distracted, or denied access to paper and pencil, or given less time. The stronger the gap in favor of the expert, the more likely it is that the test is a test mainly of concepts rather than of fluid intelligence or problem-solving aspects.

Using free responses to design the test items

The design of multiple choice questions on concept inventories is a subject of extensive investigation. Generally, the questions are presented to a few students who are asked to respond in the form of free responses. These responses are then subject to analysis to find the most common misconceptions, which are then used to construct the "distractor" options. Note that this particular character of concept inventories has two advantages:

  • The high quality of the distractors makes the score relatively immune to test-taking strategies specific to MCQs.
  • The distractors are connected to specific misconceptions, and therefore, the actual options chosen by test-takers can be used to obtain insight into their specific pattern of misconceptions.

Security of questions

Due to the large amount of research that goes into the design of individual questions, the questions are generally not available online and are made available only to select instructors upon request for administration to their classes. However, sample questions similar to the concept inventory questions are sometimes made available in discussions of the concept inventories.

Utility

Diagnostic utility

At the individual level, concept inventories are generally intended to be used only for diagnostic, rather than evaluative or grading, purposes. Note that the use of distractors to relate with specific misconceptions means that the responses can be used in a manner that goes beyond just looking at whether a person got a question right: even the specific wrong answer reveals information about the mental model used.

Utility for evaluating teaching methods

PLACEHOLDER FOR INFORMATION TO BE FILLED IN: [SHOW MORE]