1. Describe evaluation techniques for all five elements of the ADDIE Model
ANALYSIS PHASE--Address these questions and use responses to evaluate this phase
* Is this an issue or problem that ca be completely fixed by training alone?
* Is this an issue or problem that can be improved by a training intervention?
* Have you gathered all the data concerning: population, subject matter, organizational goals, learner goals/needs, resources, constraints?
* Have you reviewed your analysis results with stakeholders, target population sample, other teachers/designers?
* Have you compared your findings against other internal/external benchmarks?
* Have you double-checked all the questions above?
DESIGN PHASE
* Review all the design plan elements by the SME's and at least one other designer/teacher
* review all objectives and evaluation tasks by the SMEs and at least one other designer/teacher
* review all draft participant AND facilitator materials
* review all draft media
* review everything by the decision maker (who would that be for my unit?)
* sign-off on everything
Evaluating Design Elements-(-Kirkpatrick level 2)
Objectives--these need to be evaluated by scrutinizing the 4 elements:
Audience, behavior, condition, and degree
DEVELOPMENT PHASE
Issues arising in this phase include: segment timing, problems in materials, lack of clarity, missing the target population --PILOT testing can help find these issues.
IMPLEMENTATION PHASE
Use of smile sheets, need to cover aspects of the interaction between the product and user of product
EVALUATION PHASE
Project-end reviews--looks at how well the process worked for delivering the project
Grants-need to evaluate because money was given
2. How would you evaluate the evaluation process used during the 5 elements of the ISD process?
KIRKPATRICK Level 3 Behavior--Post-Training Evaluation
Did the training stick? Use surveys and observation.
Did the student meet the training objectives? Evaluate learning
Did the students' performance has improved? Measure behavior
Why Students may lose the ability to meet objectives after the unit
Students Never Learned the Skill or Concept Was it my teaching? the unit? What caused students not to learn materials in the first place? What went wrong? Analysis phase--didn't look at my learners closely enough. Design--did I miss key steps in designing the unit? Development--did I use interesting lessons? Implementation phase--what could I have changed? NEED to be evaluating throughout the process of unit, so I don't get at the end of the unit and go--OMG my students didn't learn anything after 2-3 weeks of instruction!!!
Skill or Concept was Never Retained--too much content in too short of time or lack of any supportive materials or methods after the unit OR content had no meaning or importance to students This is a big concern for me because there is so much to cover and so little time. (design phase) How do you motivate those struggling learners? (analysis phase) How do you make it "real" or worth learning? (design & development phase)
Skill or Concept was Never Used After the Course--Don't use will lose! How do I spiral and interconnect concepts to ensure students use the new material and not lose it?
Level 4: RESULTS
What are the bottom-line results? CRT results? (Isn't this what the district and state are after?)
What was accomplished? Did my students learn anything? Can they use this knowledge be applied to different tasks? Can they use these skills 1 week from now? 1 month? next year?
3. Is there ever a case where you think a formal evaluation is not necessary?
I really don't think formal evaluation can be used on EVERYTHING that is taught. There are concepts that I teach that I do not expect mastery, such as in science and social studies, PE, health. Maybe it is because i don't have solid thematic units at this time. If I went through the ADDIE process for those subjects, the objectives and lessons would be better designed and developed which would allow for formal evaluation. I think formal evaluation should definitely be used for the "big rock" concepts that the common core expects students to master.
References
Hodell, Chuck, (2000). ISD from the Ground Up, e-book, [kindle edition] ASTD, Retrieved from Amazon.
ANALYSIS PHASE--Address these questions and use responses to evaluate this phase
* Is this an issue or problem that ca be completely fixed by training alone?
* Is this an issue or problem that can be improved by a training intervention?
* Have you gathered all the data concerning: population, subject matter, organizational goals, learner goals/needs, resources, constraints?
* Have you reviewed your analysis results with stakeholders, target population sample, other teachers/designers?
* Have you compared your findings against other internal/external benchmarks?
* Have you double-checked all the questions above?
DESIGN PHASE
* Review all the design plan elements by the SME's and at least one other designer/teacher
* review all objectives and evaluation tasks by the SMEs and at least one other designer/teacher
* review all draft participant AND facilitator materials
* review all draft media
* review everything by the decision maker (who would that be for my unit?)
* sign-off on everything
Evaluating Design Elements-(-Kirkpatrick level 2)
Objectives--these need to be evaluated by scrutinizing the 4 elements:
Audience, behavior, condition, and degree
DEVELOPMENT PHASE
Issues arising in this phase include: segment timing, problems in materials, lack of clarity, missing the target population --PILOT testing can help find these issues.
IMPLEMENTATION PHASE
Use of smile sheets, need to cover aspects of the interaction between the product and user of product
EVALUATION PHASE
Project-end reviews--looks at how well the process worked for delivering the project
Grants-need to evaluate because money was given
2. How would you evaluate the evaluation process used during the 5 elements of the ISD process?
KIRKPATRICK Level 3 Behavior--Post-Training Evaluation
Did the training stick? Use surveys and observation.
Did the student meet the training objectives? Evaluate learning
Did the students' performance has improved? Measure behavior
Why Students may lose the ability to meet objectives after the unit
Students Never Learned the Skill or Concept Was it my teaching? the unit? What caused students not to learn materials in the first place? What went wrong? Analysis phase--didn't look at my learners closely enough. Design--did I miss key steps in designing the unit? Development--did I use interesting lessons? Implementation phase--what could I have changed? NEED to be evaluating throughout the process of unit, so I don't get at the end of the unit and go--OMG my students didn't learn anything after 2-3 weeks of instruction!!!
Skill or Concept was Never Retained--too much content in too short of time or lack of any supportive materials or methods after the unit OR content had no meaning or importance to students This is a big concern for me because there is so much to cover and so little time. (design phase) How do you motivate those struggling learners? (analysis phase) How do you make it "real" or worth learning? (design & development phase)
Skill or Concept was Never Used After the Course--Don't use will lose! How do I spiral and interconnect concepts to ensure students use the new material and not lose it?
Level 4: RESULTS
What are the bottom-line results? CRT results? (Isn't this what the district and state are after?)
What was accomplished? Did my students learn anything? Can they use this knowledge be applied to different tasks? Can they use these skills 1 week from now? 1 month? next year?
3. Is there ever a case where you think a formal evaluation is not necessary?
I really don't think formal evaluation can be used on EVERYTHING that is taught. There are concepts that I teach that I do not expect mastery, such as in science and social studies, PE, health. Maybe it is because i don't have solid thematic units at this time. If I went through the ADDIE process for those subjects, the objectives and lessons would be better designed and developed which would allow for formal evaluation. I think formal evaluation should definitely be used for the "big rock" concepts that the common core expects students to master.
References
Hodell, Chuck, (2000). ISD from the Ground Up, e-book, [kindle edition] ASTD, Retrieved from Amazon.