Teacher-made tests
Deficiencies of teacher-made tests
Testing, Assessment, and Grading
[References] [Course Home] [Home]
Testing and Assessment in College Classes - Overview
Measurement Issues in Testing and Grading
Testing and Assessment in College Classes - Overview
This outline is based on Chapter 4 of Mehrens and Lehmann, Measurement and Evaluation, 1984, 3rd Ed. Holt.
|
|
Teacher-made tests |
|
|
Deficiencies of teacher-made tests |
- ambiguous questions
- excessive wording
- lack of appropriate emphasis
- use of inappropriate item formats
|
|
Classification of tests |
- by item format
- by stimulus material
- by purpose
|
|
Planning Test |
- see Table 4-1 (p. 64)
- what is to be tested?
- specify course objectives
- define objectives
- create table of specifications (p. 67-72)
- relating test items to objectives
- selecting item formats
|
|
Differences between essay and objective formats |
- freedom of response vs. restricted response
- fewer questions requiring more time vs. more questions each requiring little time
- thinking and writing vs. reading and thinking
- quality depends on the grader vs. quality depends on the writer (constructor)
- easy to prepare, difficult to grade vs. difficult to prepare, easy to grade
- individualism on the part of writer and test taker vs. creativity on the part of the writer
- less explicit criteria vs. explicit criteria
- bluffing vs. guessing
- distribution of scores may be variable based on reader of tests vs. distribution of scores depends on the test
|
|
Factors to consider in selecting an item format |
- purpose of test
- time
- numbers tested
- physical facilities
- your skill
|
|
Additional details |
- test length
- item difficulty
- when to test
|
|
Open and closed book exams |
|
|
Considerations in writing test items |
- define objectives
- prepare test blueprint
- well-defined questions
- avoid excessive verbiage
- test on what students should know
- avoid race/gender bias
- write each item on separate index card
- prepare scoring key
- prepare more items than you need
- prepare items when you are developing teaching unit
|
|
What does it take to be a good item writer? |
- knowledge
- knowing students
- skill in verbal expression
- knowing different formats
- perseverance
- creativity
Measurement Issues in Testing and Grading [Top]
|
|
Introduction |
- nature and uses of educational measurement
- dimensions of a measurement instrument: relevance, valence, efficiency, objectivity, specificity, difficulty, discrimination, reliability, fairness, and speed
|
|
Specifying learning outcomes |
- types and sources of objectives
- taxonomy of objectives
|
|
Test Development, Administration, and Scoring |
- table of specifications
- test administration
- scoring
- domain of sampling theory
|
|
Test Construction |
- short answer, essay, and multiple choice
- strengths and weaknesses of each type of test, including reliability and validity issues
|
|
Describing Test Performance |
- frequency distributions
- percentile ranks and percentiles
- measures of central tendency and dispersion
- standard scores: z scores
|
|
Validity and Reliability |
- measurement error
- defining/assessing validity and reliability
- factors influencing validity and reliability
|
|
Item Analysis |
- item difficulty and discrimination
- item-total scores
- distracter effectiveness
- item analysis and reliability/validity
|
|
Grading |
- growth versus status grading
- minimum passing scores
- the flaws in adding raw scores from different exams
- use of standard scores
- standard scores and regression toward the mean, with a suggested correction
- what method of grading should you use?
|
|
More on Grading |
- the normal curve
- stanine distribution
- putting it all together
This outline is based on Chapter 7 of Mehrens and Lehmann, Measurement and Evaluation, 1984, 3rd Ed. Holt.
Multiple Choice Format
|
|
Advantages of Multiple Choice Items |
- versatile
- quick and easy scoring
- efficient (many questions in short amount of time)
- difficulty can be controlled with relative ease
- smaller susceptibility to guessing compared with true-false items
- can provide diagnostic information
- greater reliability than true-false
- better liked by students than true-false
- of objective-type questions, students are most likely with multiple choice respond in the same way, even when the information is presented in a different form.
|
|
Disadvantages of Multiple Choice Items |
- difficult to write
- tend to ask factual questions
- student time to respond is longest of all objective-type formats
- students who are not "test-wise" perform more poorly compared to those who are "test-wise"
|
|
Writing Multiple Choice Items (quoted from pages 156-163) |
- the essence of the problem should be in the stem
- avoid repetition of words in options
- avoid superfluous wording
- when the incomplete statement format is used, the options should come at the end of the statement
- arrange the alternatives as simply as possible
- avoid highly technical distracters
- avoid using true-false distracters—it affects reliability
- all responses should be plausible and homogeneous
- avoid making the correct answer consistently longer than the incorrect one
- avoid giving irrelevant clues to the correct answer
- consider providing "I don’t know" option (for formative evaluations)
- provide only one correct or best answer per item
- avoid using "all of the above"
- use the "none of the above" option sparingly, if at all.
- use three to five options
- avoid overlapping options
- to measure the higher mental processes, cast the item in a novel situation
- use the multiple-choice item where most appropriate
This outline has been adapted from Payne’s The Specification and Measurement of Learning Objectives, Blaisdell Publishing, 1968.
Constructing and Scoring Essay Tests
|
|
Special problems |
- reader reliability
- instrument reliability
- instrument validity
|
|
General Types of Essay Tests |
- extended vs. restricted response
- content vs. expression
|
|
Special Types of Questions |
- recall
- understanding
- application
- judgement
|
|
Scoring |
- "Ideal" Observer
- anonymous scoring/score one item at a time
|
|
Scoring methods |
- rating methods
- analytical methods
|
|
Policy to Prevent Cheating |
- lessen emphasis on grades by having multiple opportunities to demonstrate competence
- make grading policies and expectations clear
- ask class to make a commitment to the honor system
|
|
Exam Questions |
- use alternate forms of the tests
- change exam questions often
- keep exams and rosters in locked file drawers or other such venue
|
|
Administration |
- stay in the room
- have students sit in alternate seats
- supply scratch paper
- ask students with "wandering eyes" to more to another seat
- spend time in the back of the room
- make sure handing in exams is orderly
- administer the test in smaller groups, if possible
|
|
Scoring of exams |
- clearly mark wrong answers
- return exams in person
|
|
Excuses |
- make an attempt to distinguish between legitimate and fraudulent excuses
- make policies regarding exams clear
- don’t become cynical
|
|
When cheating happens |
- check with institutional policy on cheating and follow the procedure
- if you suspect, ask students to sit in another seat to finish their exam