|
|
Websites | |||
|
|
||||
|
Quiz Feedback
(1) While you are online:
(2) Between one and three days later:
(3) Between one week and one month later:
(4) Between one month and six months later:
Questions and Feedback:
1. An instructional design team wants to create a coherent set of performance objectives to help it design and develop a new e-learning course on how to build a website using Microsoft FrontPage. Three instructional designers develop different formats for these instructional objectives. Which format (as exemplified in the examples below) will produce the best course?
Question 1 Feedback Performance objectives that most clearly specify the desired learner behavior will produce the best instructional design. Performance objectives should differ from learning objectives. Both types of objectives help guide human behavior, as all goals do, but it’s best to separate performance objectives from learning objectives on the basis of the different audiences that each is intended to serve. Performance objectives should be designed to influence the behavior of instructional designers and developers. Learning objectives should be designed to influence the behavior of learners. The objectives in the question above are performance objectives because the instructional designers are using them to design a course. Performance objectives should include (1) the situation in which the learners will perform the learned actions, (2) a description of the actions that should be undertaken, and (3) criteria for successful implementation of the actions. By clearly outlining the performance desired and providing guidelines for success, performance objectives enable instructional designers to hold themselves accountable. Choice C is the best choice because it meets these criteria.
2. Imagine that you are preparing for your company's senior-consultant qualifying exam, a full day of exercises plus a paper-and-pencil test of your knowledge of network systems. The test will be given in the meeting room of the San Francisco office. Which preparation method will enable you to pass the test with the best score?
Question 2 Feedback When the learning and performance contexts are similar, more information will be retrieved from memory. When we learn, we don’t just take in the topic information; we also absorb the sights, smells, and sounds of the environmental context. The topic information thereby becomes integrated with the contextual information. Thus, if you return to the same place or a place with similar environmental cues, you’re likely to be reminded of the topic information because of its association with the background information. Researchers have found that when people learn in one room, they’ll do better on a test if they return to that room rather than to a different room. Researchers have also found that performing underwater is buoyed if one learned underwater as opposed to learning on land, and vice versa; that performing while listening to Mozart is improved if one learned with Mozart as opposed to learning with jazz, and vice versa; and that performing while smelling peppermint is improved by learning in the presence of peppermint rather than osmanthus, and vice versa. In the question above, it is best for you to learn in the San Francisco office, because when you take the exam, the context will remind you of the information you studied.
3. Intrude Industries, Inc., is developing a multimedia course for its investigator trainees. It wants each course segment to take half an hour, and it wants the course to run for a total of 3 to 5 hours. Intrude’s learning architects expect trainees to work with the course for half an hour each morning and half an hour each evening for 3 to 5 days. Several designs have been suggested. Which one will produce the best long-term retention of the key learning points?
Question 3 Feedback Repetition of critical learning material improves long-term retention of that material. Repetition enhances learning and long-term retention for several reasons. First, it enables learners to add to their learning, providing them with information they might not have fully incorporated into memory on previous learning attempts. Second, it helps learners strengthen their previous knowledge. Third, it allows learners an opportunity to get feedback on their previous learning attempts. Choice A is best because it provides the most repetition. Note that learners may learn more slowly in this situation because they will receive only 20 minutes of new material in a session—but their long-term memory will be significantly improved. Repetition has often been given a bad rap because it has become associated with rote learning and drill-and-practice. Unfortunately, this oversimplification causes many of us to avoid repetition. The key to useful repetition is to make it interesting, relevant, and practical for the learners. Creativity is needed, but the learning results can be dramatic.
4. NASA is preparing a space-shuttle crew to perform a complex repair of the space-station antennae. The launch is scheduled for the first week in October. The two spacewalkers need to be able to perform the task in zero gravity and so must train in a zero-gravity environment. Because rich businessmen have reserved most of the time in the zero-gravity simulator, the astronauts have only 40 hours to practice for their mission. They need to practice 8 hours per session, so they’ll have 5 practice sessions. Which schedule of training will enable them to do the best job possible?
Question 4 Feedback The practice schedule that is spaced more widely over time will best enable learners to retrieve information from memory and hence will prompt them to perform at their best. This “spacing effect” is one of the most replicated findings in the research. Spacing provides several advantages. First, it helps learners overcome fatigue. Second, it enables learners to bring a new mindset to the learning task, which promotes the development of different retrieval routes through memory to the targeted information. Third, spacing generally provides multiple contexts in which learners can learn. Whether the contexts are internal (mental, emotional, or cognitive states) or external to the learner (sights, sounds, smells, etc.), having different contexts promotes the development of additional memory-retrieval routes. Researchers have found that long-term retention is almost always improved with more spacing of learning events rather than less spacing, but there are a few caveats. First, the spacing must include some repetition of information. It’s the spacing of repetitions that improves the learning, not the spacing of unrelated topics or courses. Second, the learner must process the repetitions. Third, when the information will be used immediately after the last learning session, spacing doesn’t seem to provide much benefit. So, if the zero-gravity simulator had been available in mid-September, the astronauts may have benefited from repetitions that weren’t spaced as widely. However, if the launch had been delayed, their learning would have deteriorated more rapidly than if it had been spaced. In the question above, the astronauts would be better off spacing their learning as widely as possible. Therefore, Choice C is best.
5. At the Buzzy Watch Company, a group of trainers has designed a new leadership course entitled Championing Change. They hope to present learning objectives to the learners but can’t decide which type will work best. Which type of learning objective (as exemplified in the examples below), when presented to participants at the beginning of a lesson, will generate the best learning performance?
Question 5 Feedback Learning objectives are most effective in facilitating learning when they are specific and include information that is related to the point to be learned. Learning objectives should differ from performance objectives. Learning objectives should be designed to influence the behavior of learners, whereas performance objectives should be designed to influence the behavior of instructional designers. Learning objectives should be very specific, not the typical global statements we tend to dangle in front of learners. Research has shown that only specific objectives have an impact. Learning objectives work because they focus learner attention on some aspects of the learning material and because they prepare our cognitive space to integrate the new information into memory. If learning objectives are too general, they cannot serve either function. They cannot focus learner attention and they cannot set up memory stores specific enough to integrate information. In the question above, Choice B is best because it provides lots of specific information. Choice C doesn’t add any useful information for the learners, although such a criterion may be useful to instructional designers in a performance objective. Choice D probably has too much information; although the information would be valuable in the learning material, it adds little value as a learning objective.
6. The PlayDoh Learning Company has developed its multimedia courseware to work as a modular system. Each lesson contains (a) a presentation of the learning material, (b) exercises, and (c) testing. Typically, their courses utilize 40% presentation, 40% exercises, and 20% testing, but they’re reconsidering this ratio. What ratio would create the best long-term performance for their learners?
Question 6 Feedback An instructional sequence that includes more testing will usually produce better performance than one that includes less testing, up to the point where there isn’t enough background information to provide the learners with a sense of what they’re supposed to learn. Testing provides memory-retrieval practice—probably the most powerful form of memory enhancement. Retrieval practice refers to the process of searching our memories for information. When we are asked a question and we respond with an answer, we have essentially retrieved the answer from our memory stores. Testing is very powerful because it fundamentally provides learners with practice on the type of behavior they’ll have to use when they are asked to perform on the job. When we do anything—when we perform—we are continuously retrieving information from our long-term memory stores and bringing it into our working memories or transferring it into bodily action. To have the proper impact, testing must be designed correctly. Too often, we test learners on information that is easy to test, yet we fail to test them on information they need to know. Let us use this test as an example. We’ve tried to develop questions that will benefit our target audience: trainers, instructional designers, performance consultants, and others in the learning-and-performance field. If a test question focuses on information learners can use to improve their practices, then the question is valuable—the current question is a case in point. Alternatively, we could have asked the following question: “Which of the following people advocated a behavioristic testing methodology? (Gagne, Skinner, Mager, Schank).” This question is easy to develop, but doesn’t help the target audience create better instruction. In the official question above, Choice C, which advocates 60% testing, is likely to produce the best learning-and-performance outcomes. An important note. The “exercises” in the question above are also likely to produce retrieval-practice effects, just as testing does. This is a key point because it’s not testing per se that creates the beneficial effects; it’s the retrieval practice. Thus, we instructional designers aren’t restricted to traditional forms of testing to create this powerful memory strengthener. We can also use exercises, case studies, simulations, role-plays, and on-the-job mentoring.
7. SDI, Inc., a maker of anti-ballistic missiles, is teaching a four-day course on the effects of acceleration on chemical reactions. Every half-day, a computer-based quiz is given on the material—one at 11:30 AM and the other at 4:30 PM. Feedback can be delivered to the learners at any time. Which schedule of feedback is likely to promote the greatest long-term retention of the material?
Question 7 Feedback Delayed feedback is generally more effective in producing long-term learning than is immediate feedback. It was once believed that immediate feedback was essential for learning. This belief was largely a relic of the notion that learners’ answers had to be reinforced to be remembered. We now know that this is not true. Researchers have found that a delay of about 24 hours generally produces the best long-term retention, although delays of 5 minutes are better than delays of 1 minute and delays of half a day are better than delays of one hour. Delays longer than 1 or 2 days have been found to be counterproductive. Delayed feedback produces benefits because it forces learners to rethink the issue. When feedback is immediate, learners can use the same memory-access routes, they can process the information quickly and casually, and they can suffer from fatigue so that their learning efforts lack effect. Delayed feedback often utilizes different background contexts, prompts learners to reprocess the information in a fresh manner, and enables learners to bring energy and attention to the task. There is also some evidence to suggest that providing sleep between learning efforts benefits long-term memory retention. In the question above, Choice C, utilizing feedback after 1 day, is likely to produce the best long-term retention and hence performance.
8. Some guy calls you out of the blue and wants to know which learning strategy will be more effective. He offers to pay you $100 for the answer. You need the money, so you agree to help. It turns out that he’s a motivational speaker and he wants to become a trainer. He’s a very entertaining guy, he tells you, but he’s worried that he might tell too many stories for a training audience. He’s also concerned about holding people’s interest for longer than an hour. Although his previous experience has involved 60 and 90-minute sessions, his training classes will be 4 hours long, they’ll be taught in a rather austere training room, and learners will be provided with three 15-minute breaks. Obviously, he has to be somewhat energizing for a training audience, but he needs specific answers. Which style should he use to create the best long-term learning for his trainees?
Question 8 Feedback Learners are often distracted enough by interesting irrelevant information that they learn less when this type of information is inserted into instruction. Trainers, textbook authors, and other educators often include entertaining yet irrelevant information in their learning presentations. Unfortunately, these interventions may actually cause learners to learn less of the main points of the learning material. Researchers have labeled this interesting yet irrelevant information “seductive details” and have demonstrated that it can harm learning. In the question above, the motivational speaker was right to worry about his entertaining yarns. They could become the most salient part of his training efforts and draw his learners’ attention away from his main themes. Alternatively, if his stories are illustrative of the main points, they may actually aid the learning process. The best answer above is Choice D. Although Choice C, a combination of entertaining and illustrative stories, seems attractive, those entertaining stories may still have a debilitating effect. Of course, entertaining stories that illustrate the key points can be quite effective because they focus learners’ attention on important information. There is one big caveat to this principle. Research has been done only on relatively short time-periods. It remains to be seen whether interesting yet irrelevant information will prove beneficial if learners are more fatigued or if they have to pay attention for longer periods of time. It seems reasonable that, after a couple of hours of instruction, learners who receive interesting yet irrelevant information may be able to re-energize themselves to pay attention to further learning materials. Certainly, it’s not recommended that trainers become automatons. It is still a truism that people are more likely to pay attention to another person if that person appears competent, respectful, and likable.
9. The Make-‘Em-Shine Car Wash Company is developing a video-based course for its car washers. The company’s president is especially concerned about how well the cars are waxed, so the course includes prequestions on Waxing but not on any other topics. No feedback is given on the prequestions. What is the likely result of these prequestions?
Question 9 Feedback Prequestions function as learning objectives. Learning objectives guide attention toward some aspects of the learning material and away from other aspects of the learning material. Learning objectives work by focusing learner attention and by creating memory stores in which new information can be integrated into memory. Learners cannot maintain optimal attention for long periods—their attention wanders. Learning objectives and prequestions can help them focus their attention toward the most important learning points. Unfortunately, they also make it less likely that learners will focus their attention on material that was not targeted by learning objectives. In the question above, the information on Waxing will become more salient as the information on Soaping, Drying, and Polishing wanes. Therefore, Choice F is the correct choice.
10. A group of performance consultants is designing a training course for electrical engineers. After months of intense study, they determine that the engineers will need to use the training-course skills in 148 different situations. They attempt to categorize the situations into a neat array of 5 to 9 categories, but they can’t agree on the categories. How should they design the course?
Question 10 Feedback Learners who learn information in many different contexts are less dependent on one particular set of cues to aid their memory retrieval and thus are able to perform in a wider variety of situations. When people perform their jobs (or do anything) they are constantly bombarded by cues in their surroundings that clue them into the appropriate behaviors. For example, when we listen to someone tell us about his or her day, we notice facial expressions, tone of voice, level of energy, etc., and we tailor our response to all of these cues. When we learn in a training course, the content gets linked in our memory to the context of the situation. If we are clear about what the workplace-performance situation is going to be like, we can tailor our training to mimic that situation. In this way, when learners later enter their performance situation, the cues will help them to retrieve the information that they learned. If we don’t know the performance situation, it is usually best to provide learners with a diverse set of learning situations, so they will have many memory-access routes to the learned information. For example, if learners learn computer skills only on an Apple computer and only using Word-Perfect, they are more likely to have difficulty working on a PC using Microsoft Word, Adobe Illustrator, or various other hardware/software configurations than someone who had a more diverse training environment. In the question above, because the engineers have to perform under so many diverse conditions, D is the best choice as it provides the most varied learning contexts.
11. Shark Learning Company recently merged with Anchovy Limited and thus acquired their e-learning products. Shark’s instructional designers are in the process of evaluating Anchovy’s courses with an eye toward improving them. Anchovy’s courses use well-designed multiple-choice tests, but they only provide feedback on incorrect answers. Each question has five multiple-choice answer alternatives. Typically, test-takers score about 60 percent correct the first time they take the test and 80 percent on a second test given a week later. Shark must decide whether to add a mechanism to provide feedback on correct answers. The costs would be substantial, so they want to be sure that the feedback would produce significantly more learning. What is the likely result if they add the feedback on correct answers?
Question 11 Feedback Because feedback works by correcting errors, it is most important for incorrect answers. Feedback has very little effect on most correct answers. However, for correct answers that learners obtained by guessing, feedback produces a corrective effect just as if the answer was incorrect. In this question, learners got 60’s on the first test. They knew 50 of the answers and guessed correctly on 10 without really knowing the answer. How do we know this? Well, since it is a multiple-choice test with 5 answer choices per question, learners will on average get 1 in 5 answers correct just by guessing. On average, they got 40 incorrect, which should represent 4 of 5 answers that they guessed on. This leaves 10 questions on which they guessed correctly. So, of the 60 they got correct, they didn’t know 10 of them. Thus, if Shark decides to give feedback on correct answers, they’ll only have a significant impact on 10 questions. The feedback, if it had a complete effect, would increase learners’ scores to 90 percent. However, as the previous results show, learners in this case typically only have 50% success when given feedback (when they got feedback on 40 questions, they improved on only 20). So the likely result of providing feedback on the correct answers is to improve the scores on the second test from 80 to 85. Although this improvement may be important for Shark and its learners, 5 percentage points doesn’t sound like much if the costs of providing the feedback are high. Note that giving feedback on questions for which learners know the answer has been shown to have little positive benefit. In fact, sometimes it has been found to annoy learners. The reason for the lack of benefits is simple. If learners know the answer and then get it correct on a test, they have had at least several opportunities to strengthen the knowledge. The first test provides them with practice in retrieving the information from memory, which is the same kind of processing they’ll have to do to perform well on the second test. Thus, extra feedback has very little benefit because learners are already prepared to access the information from memory. One final note. Although Shark may not decide to make the costly changes, most of our learning designs will benefit if tests provide feedback on incorrect answers and either feedback on correct answers (if it doesn’t annoy learners) or a choice of feedback on correct answers. Giving learners a choice of receiving feedback on correct answers will allow them to choose questions for which they want more information.
12. Learners attending an intensive 5-day workshop are given evening reading assignments to augment their classroom learning. The reading assignments are well written and explore the same learning points discussed in the day’s lectures and discussions. For those who do the reading assignments, what is the likely result?
Question 12 Feedback When presented with well-organized learning materials, people tend to focus initially on the most important points. Learners don’t always make the best learning decisions, but when it comes to sifting through well-organized learning materials they tend to work toward understanding the most important information first. Only later, after they fully understand the key points, do they work on the less significant points. Learners can act strategically when given well-organized learning materials such as lectures, articles, or some form of multimedia presentation, because well-organized materials provide clues to what’s most important. Paragraphs begin with topic sentences. Speakers emphasize or repeat key points. The connotations of words and phrases point toward the importance of some material. When material is not well organized, learners spend their first review of that material trying to make sense of its structure, and when they have difficulty they often latch on to easily digested points—points that are quickly grasped though they may be trivial and less important. In the question above, learners who read the assigned articles will learn the key points first and foremost. The reading assignments provide the learners with a repetition of the day’s lecture and discussions, so the reading gives them further opportunities to grasp the main points and build a rich and meaningful network of information about the topic in memory. If the learners read the articles two or three times, they’ll begin to learn more of the least important points—but only after they’ve fully grasped the main themes of the learning materials.
13. A Human Performance Technologist is called into the Gee-Whiz Sales Company and is asked to design a methodology that will help new sales recruits remember what to say in overcoming sales resistance. Twenty customer objections are highlighted, put in a conversational format, and a script is written for each one that illustrates how to overcome the objection. At the beginning of the training, the objections and the responses will be presented together, but as the training proceeds the objections will be presented and the sales trainees will be asked to respond with the appropriate words and phrases. The President of Gee-Whiz believes in immediate repetitions and can’t be persuaded against using this strategy. How should the Human Performance Technologist design the first part of the training?
Question 13 Feedback When learning material is repeated immediately, it will be better remembered if the words are slightly altered rather than if they are repeated verbatim. Although it is generally best to repeat learning points at spaced intervals, if this is impossible or impractical slight variations in the learning material will often ameliorate the problem. Spacing provides multiple retrieval routes because it usually delivers learning material in different mental and physical contexts. Varying the learning materials can provide similar benefits. Variation is a desirable learning principle in general. Variation enables learners to overcome problems with contextual cuing. When learners learn something in one context, they often become unable to retrieve the information in other contexts. When we can’t remember neighbors’ names when we see them downtown, this is a context problem. We come to rely on neighborhood cues (neighbors’ houses, our driveway, etc.) to recall names, but when we see our neighbors somewhere else we don’t have those cues to help us. It’s the same on the job. If we learn Microsoft Word with full menus, and then receive the new version with partial menus, we probably won’t be able to remember the easiest way to do things—because the cues that we use to remind us of what to do are hidden. Variation also provides different yet similar memory-retrieval routes, introducing redundancy into the memory system. This allows us to deal with our memory failures. When we have a failure using one retrieval route, we can rely on a redundant route. In the question above, the variations in wording and delivery will help the learners to create multiple retrieval routes to the memory nodes for the appropriate sales responses. Choice B will work best.
14. Fantabulush, an online wine retailer, is developing an e-learning course on how to choose wine to complement different types of food. The company calls in an instructional design guru to design the course. The guru develops a series of tests to keep learners’ interest and to provide interactivity. One question asks the learners to name the region in France famous for red wines. Another asks the learners the advantages and disadvantages of a dry grape-growing season. Still another question asks the learners to select the foods that have a high percentage of complex carbohydrates. The tests will be given once, with feedback provided immediately after each question if learners request it. Unfortunately, the marketing folks for Fantabulush don’t like testing of any kind—something to do with bad high-school experiences. The instructional guru recommends that they develop two tests, one with questions and one without, and see which one works best. The marketing folks agree. Which group will be more likely to reach the learning goals?
Question 14 Feedback Testing must be directly relevant to be useful in producing performance that matters. The moral of this story is: don’t trust gurus. Don’t even trust people who tell you not to trust gurus. On a more serious note, it’s critically important that testing—in fact, all forms of retrieval practice—focus on meaningful information and provide practical, relevant tasks for learners to think about and work on. The questions developed by the guru don’t help learners meet the instructional goals of the course. They do give retrieval practice, but not on the right material. Fantabulush wanted a course that prepared learners to choose appropriate food-and-wine combinations. Knowing the renowned of the Bordeaux region isn’t helpful. Knowing that a dry season produces better-tasting grapes doesn’t help either. Even the question on complex carbohydrates is probably tangential. It is better to ask learners directly about what they need to know; for example, present them with a list of foods and ask them to choose appropriate wines or to list the characteristics of the food that are important to consider. In the above question, because the online learning questions are not relevant to the instructional goals of the course, using them will make little difference. Choice C is the best choice. Note that retrieval practice is powerful because it helps to prevent forgetting. Presumably, Fantabulush wants learners to have access to their newly acquired knowledge for a long time. Asking them relevant questions would be a good place to start. Also note that the notion of interactivity is not as important as we tend to think. Interactivity by itself can be good, but it can be bad also. The key is to create interactivity that is relevant to learning—that helps learners create or strengthen appropriate memory stores.
15. An instructional designer develops the same course using two different media. One version delivers the instruction via video in a classroom seminar and the other uses an e-learning video-based course. Both versions are well designed and are intended to prepare the learners equally, utilizing the same learning methods with only minor differences. Which course will produce the best long-term retention and performance?
Question 15 Feedback If the learning methods are the same, two courses will produce the same results regardless of the media utilized. This does not mean that different media can’t produce different learning results. It’s very clear from both common sense and research that multimedia animations provide some learning advantages over straight text. However, if text presented on a computer screen is compared to text presented on paper (assuming equal visibility), we will find no differences in learning. The statement “a cudgel is a stick-like club” would have the same learning effect on paper as it does on a screen. It’s not the media that matters; it’s the learning method.
The reason for this is straightforward. Learning is created when a learner attends to learning material and cognitively processes it in a manner that gives rise to learning. When the learning method prompts this type of processing, a person’s memory stores are changed. It doesn’t matter what medium was used to produce these changes. What’s important is that the learning method facilitated the changes. In the question above, the instructional designer made sure that the learning methods were similar, so the learning will be similar too. Choice C is best.
You have now completed the Quiz Questions Feedback. Click here to return to Home Page. To see how others have scored on the quiz, click here.
|
||||
|
|
||||
|
Reproduction and/or distribution of any material from any Work-Learning Research web page is a violation of moral and legal principles and is strictly prohibited without prior written permission from Work-Learning Research.
© Copyright 2001-2008, Work-Learning
Research, Inc.
send comments to
webmeister |
||||
60percent.gif){kind=logo}