Doug Trovinger

Course Blueprint American College of Education Course Blueprint Official Course Number: DL5803 Official Course Title: Evaluation and Assessment of Instructional Design Official Course Description: Assessment and evaluation of instructional activities will be explored within various experiences, including evaluation of course design as well as assessment of participant learning. Approaches to determining evaluation options for both levels are explored using evidence based assessment methods. Students will explore data gathering and analysis, as well as application of findings for continuous improvement of learning activities. Course Author Course Author Credential Course Guide’s Name Course Guide’s Credential Doug Trovinger Ed.S. Doug Trovinger Ed.S. Course Objectives 1. Explain various faucets of the research process is and what is comprised of it when executing instructional design principles. 2. Understand various ethical concerns that can arise from research obtainment and how it can affect instructional design. 3. Describe various qualitative methods that could be executed in creating effective course designs. 4. Describe various quantitative methods that could be used when developing instructional materials. 5. Understand that mixed method studies which employ qualitative and quantitative methods can be used together in determining improvements of learning activities. 6. Create measurement tools that can be applied in an instructional design environment both academically and professionally. 7. Analyze and review results from measurements created to determine potential learning outcomes and future improvements. 8. Engage as a reflective student and collaborator to the course and how skills learned can be applied to a design of instructional materials. Author Bio Doug Trovinger, Ed.S. serves as an instructional designer for the American College of Education. He currently holds a Specialist in Education from the University of Tennessee – Knoxville. He has taught courses at Tennessee Wesleyan College which specialized in statistics and other related business disciplines. He has also been an author and coauthor of several articles that are focused in instructional technology and online learning. Outside of the academic realm, he has numerous years of experience specializing in training, learning development, and employee coaching. Getting to Know You Discussion Prompt Welcome to Evaluation and Assessment of Instructional Design. To start, the first task at hand would introduce yourself to other classmates that would be in the course. Within this introduction, we would like to know a bit about your work experience and ways that you have analyzed others in one way or another. Do you like to analyze data in order to come up with strategies or improvements for future instructional materials? Share your answers with others that would have similar passions and response to at least two (2) other’s responses in the course. • What types of measurement do you prefer to use in instructional design? • What are some of the things that excite you about measuring instructional design successfulness? What concerns may you have as well? INTRO PARAGRAPH Measuring the effectiveness of learning can be a daunting task. There are dozens upon dozens of ways that this crucial piece of instructional design can be achieved. The purpose of the course is to familiarize oneself with a number of the methods that can be used to compile, receive, and disseminate data from a course. To get to that though, one must first get a strong understanding of the fundamentals of research as well as ethical concerns that may potentially occur. From there, a number of qualitative, quantitative, and mixed method options will be introduced that can be used in the measurement of items used in instructional design. Upon grasping this knowledge, this will lead one to applying them and reviewing results gathered from the measurements. Module 1 Educational Research Fundamentals & Ethics Module 2 Qualitative Methods of Data Collection Module 3 Quantitative Methods of Data Collection Module Introduction Educational research in the eyes of many can be a boring and tedious task to take on. Contrary to the point, it is one of the most important pieces of a successful learning outcome and effective instructional design. This introductory module focuses on the fundamentals of research and how it applies to instructional design (Michael, 2002; Nunan, 2000). Once established, a review of various ethical practices application to instructional design will be presented here as well. When it comes to qualitative methods used for data collection, one can easily confuse it with its counterpart quantitative research methods (Osler & Mansaray, 2004; Marshall & Rossman, 1999). While most prefer the use of quantitative methods for research, the use of ones described in this module are just as important. In some cases, they may be the only option available for data collection, Thus, the second module of the course looks at what qualitative methods comprise of and how they can be applied effectively in instructional design. Have you ever made a purchase from your local store and saw a request for feedback at the bottom of your register receipt? Or have you seen a popup on your favorite website (educational or otherwise) asking you to take a survey (Kirklees, 2015; Hampden-Thompson & Sundaram, 2013)? Both are prime examples of quantitative research methods. This module will review various ways that one can collect data in an efficient manner from the use of quantitative methods. Additionally, a couple of brief examples will be provided as a guiding tool for students. Module 4 Designing Effective Instructional Design Research Tools Up to this point, students have learned some of the fundamentals of research; the ethics behind performing studies, and ways to gather data. Although these combined can help one become a better researcher, there are best practices that can be applied to create even more effective measurement tools (Domun & Bahadur, 2014; Labor, 2014; and Stes, 2012). This module will focus on those as well as give learners a chance to create a couple of simple measurement tools that will be used for the final module of the course. Course Objectives • • • • Title of Module • 1. Explain various faucets of the research process is and what is comprised of it when executing instructional design principles. 2. Understand various ethical concerns that can arise from research obtainment and how it can affect instructional design. • 1. Explain various faucets of the research process is and what is comprised of it when executing instructional design principles. 3. Describe various qualitative methods that could be executed in creating effective course designs. • 1. Explain various faucets of the research process is and what is comprised of it when executing instructional design principles. 4. Describe various quantitative methods that could be used when developing instructional materials. • • 2. Understand various ethical concerns that can arise from research obtainment and how it can affect instructional design. 5. Understand that mixed method studies which employ qualitative and quantitative methods can be used together in determining improvements of learning activities. 6. Create measurement tools that can be applied in Module 5 Data Dissemination Best Practices & Personal Reflections Having the opportunity to create measurement tools is an enriching and rewarding exercise. Similar to riding a bike or learning how to drive a car with a manual transmission, these skills will be perfected in due time. The purpose of this module is to review knowledge received from the exercises done in the previous module and determine the effectiveness of the data received. Suggestions on how to improve results for better participation rates will be discussed as part of the module’s objectives here (Afifi, 2014; Richardson, 2014; and Magnussen, 2014). • 7. Analyze and review results from measurements created to determine potential learning outcomes and future improvements. • 8. Engage as a reflective student and collaborator to the course and how skills learned can be applied to a design of instructional materials. • Module Objectives • • • • Understand the definition of research and the concept of different types of observations. Explain the difference between the scientific method and scientific process and how they apply to educational research. Discuss common ethical issues that can arise in research studies. Discuss ways to prevent/combat possible ethical issues and provide examples of data collections gone wrong. • • • • Journal articles and books Research Methods In Education – Louis Cohen, Lawrence Manion, & Keith Morrison Cohen, L. Manion, L. & Morrison, K. (2007). Research Methods in Education. Briefly introduce the most common methods of research that can be executed by an instructional designer/ technologist (qualitative, quantitative, and mixed methods). Explain in brief the purpose of qualitative research and why it is important to use in instructional design research. Explain what some of the most commonly-used qualitative methods could be used in instructional design practices. Provide an opportunity for students to discuss methods learned and researched that are qualitative-based and why they would choose mentioned method(s). • • • • Explain in brief the purpose of quantitative research and why it is important to use in instructional design research. Discuss the concept of hypothesis testing and why it is important in quantitative research (as well as qualitative). Evaluate different examples of quantitative research that were found in materials provided. Provide an opportunity for students to comment on their reasoning why to use quantitative research methods in addition to qualitative methods. Conducting Research in The Classroom – Addrej A. Cirocki Using Graphs To Illustrate Quantitative Data – Centers of Disease Control (C.D.C.) Cirocki, A. A. (2013). Conducting research in the classroom. Modern English Teacher, 22(2), 63-69. C.D.C. (2008, July). Using Graphs To Illustrate Quantitative Data. Retrieved from • • • • an instructional design environment both academically and professionally. 8. Engage as a reflective student and collaborator to the course and how skills learned can be applied to a design of instructional materials. Describe best practices that should be used when developing research studies that are qualitative and quantitative in nature. Discuss methods that instructional designers / technologists may want to use to obtain the richest data sets. Incorporate in ethical considerations as part of the measurement development process. Create a sample set of measurements that could be used in data collection processes. Development of a SelfAssessment Tool and Investigating its Effectiveness for ELearning – Manisha Domun & Goonesh K. Bahadur • • • • Evaluate data collection methods that were provided by others in the course against a checklist provided. Assess methods that were used as a whole and provide an opportunity for students to comment on ones that should be used that were not provided. Explain how research accumulated in a study can affect the instructional design of a course. Reflect on how the use of various data collection methods will make instructional design an easier process in the future. Principles in Designing E-Course In Light of Learning Theories – Muhammed K. Afifi & Saad S. Alamri Afifi, M, M. & Alamri, S. S. (2014). Effective Principles Abingdon, Oxon, UK. 15-19. • How does the six (6) scientific process differ from the more commonlyused scientific method applied by authors mentioned in the text? • Explain your opinion on the statement authors made that the scientific method is not useable in human research studies. Rationale: This is a good piece to use to assist in the differentiation of the scientific method and process commonly used in research. Introduction to Primary Research: Observations, Surveys, and Interviews – Dana Lynn Driscoll Driscoll, D. L. (2011). Introduction to Primary Research: Observations, Surveys, and Interviews. Writing Spaces: Readings on Writing, 2, 153-154. Note: This would be referenced throughout the course but be a required reading in Module 1 only. • What is the concept of primary research and how could it apply to instructional design? • Reviewing the first graphic in the article, how could that be similar to different to the ADDIE approach of instructional design? Explain. Rationale: This article is a basis for a common research method that could Note: This would be referenced throughout the course but be a required reading in Module 2 only. • What are two (2) qualitative and quantitative options to conduct research mentioned in the article? • Name the five (5) points of observation made by the author and explain how they could be applied to instructional design. Rationale: Though this article highlights information in Modules 2 and 3, it is an important starter article to move the course along. Using Qualitative Research To Assess Teaching and Learning in TechnologyInfused TILE Classrooms – Sam Van Horne, Ceclia Titiek Murniati, Kem Saichaie, Maggie Jessie, Jean C. Florman, & Beth F. Ingram Horne, S., Murniati, C. T., Saichaie, K., Jesse, M., Florman, J. C., & Ingram, B. F. (2014). Using Qualitative Research to Assess Teaching and Learning in Technology-Infused TILE Classrooms. New Directions For Teaching & Learning,-), 17-26. doi:10.1002/tl.20082 • What were the most impactful qualitative research methods provided in the article in http://www.cdc.gov/healthyyou th/evaluation/pdf/brief12.pdf. • Which of the graphical data displays shown could be best applied in educational research studies and why? • What is the main difference between a continuous and categorical data set? Rationale: This brief article helps reinforce some key principles of qualitative research. Design and Implementation of a Simulation-Based Learning System for International Trade – Luo Guo-Heng, Liu Eric ZhiFeng, Kuo Hung-Wei, & Yuan Shyan-Ming Guo-Heng, L., Eric ZhiFeng, L., Hung-Wei, K., & ShyanMing, Y. (2014). Design and Implementation of a Simulation-Based Learning System for International Trade. International Review Of Research In Open & Distance Learning, 15(1), 203226. • What is a survey usability scale (SUS) and how effective could it be in instructional design? • Provide two (2) to three (3) ways that the brief article could be effective in the design of instructional materials and courses. EBSCO Rationale: The article, though a bit long, is a good Domun, M., & Bahadur, G. K. (2014). Design and Development of a SelfAssessment Tool and Investigating its Effectiveness for ELearning. European Journal Of Open, Distance & ELearning, 17(1), 1-25. doi:10.2478/eurodl-. • What are at least four (4) of the six (6) purposes of designing effective self-learning environments? • Do you think that the use of box plots are a good way to represent data in educational research and/or in a course? Explain. EBSCO Rationale: The article provides some justification on some methods that could be used in qualitative, quantitative, and/or mixed methods research. It also provides a glimpse of an online course. High Impact Leadership – John Hattie Hattie, J. (2015). HighImpact Leadership. Educational Leadership, 72(5), 36-40. • When developing courses, which of the seven (7) major mind frames provided could be most aligned with instructional course in Designing E-Course In Light of Learning Theories. Turkish Online Journal of Distance Education (TOJDE), 15(1), 128-142. • What are at least four (4) benefits of online learning that could be applied to an educational research study? • Provide at least two (2) ways that the authors show that instructional design research can be improved from their conclusions. EBSCO Rationale: This article is important in helping make conclusions on data and recommendations for future research initiatives. Color and Contrast in ELearning Design: A Review of the Literature and Recommendations for Instructional Designers and Web Developers – Rick T. Richardson, Tara L. Drexler, & Donna M. Delparte Richardson, R. T., Drexler, T. L., & Delparte, D. M. (2014). Color and Contrast in E-Learning Design: A Review of the Literature and Recommendations for Instructional Designers and Web Developers. Journal Of be applied to instructional design and research methods. Philosophies Guiding Research. Europe, Alzheimer. (2009, August 21). Philosophies Guiding Research. Retrieved from http://www.alzheimereurope.org/Research/Underst anding-dementiaresearch/Types-ofresearch/Philosophiesguiding-research • Using information provided, what is an experiment that could be done when designing a course? • What are the four (4) main characteristics of good (scientific) research studies? Provide a brief example of each. EBSCO Rationale: Though brief in nature, the article is good for background material on the module and understanding of education research fundamentals. Inquiry & Scientific Model – Robert S. Michael Michael, Robert S. (2002). Inquiry & Scientific Model. Retrieved from http://www.indiana.edu/~educ y520/sec5982/week_1/inquiry _sci_method02.pdf. • Explain whether or not the three (3) approaches your opinion? Why? Should courses have a component to develop the faculty or trainers that would facilitate it? Explain your reasoning. EBSCO Rationale: The article has a lot of good points on faculty development, its application to instructional design, and the concept of a TILE classroom format. • Designing Qualitative Research, 3e. – Catherine Marshall & Gretchen B. Rossman Marshall, C., & Rossman, G. B. (1999). Designing Qualitative Research, 3e. Thousand Oaks, CA: Sage. 21-54. • What are the three (3) most critical parts of a research proposal? • Which faucets of research are the most applicable in instructional design? Explain your choices. EBSCO Rationale: Good article presented here that highlights importances of educational research and the common base of research proposals. These can easily be applied to instructional designs. Model for Determining Teaching Efficacy Through The Use of Qualitative Single Subject Design, Student Learning physical representation of simulation-based training. This is good for materials designed in the private sector. This can apply to research methods as well as items provided in instructional materials for academia. Developing Quantitative Research Skills and Conceptualising an Integrated Approach to Teaching Research Methods to Education Students – Gillian Hampden-Thompson & Vanita Sundaram Hampden-Thompson, G., & Sundaram, V. (2013). Developing Quantitative Research Skills and Conceptualising an Integrated Approach to Teaching Research Methods to Education Students. AISHEJ: The All Ireland Journal Of Teaching & Learning In Higher Education, 5(3),-. • What is the main purpose of the study and how could it apply to instructional design? • How could the results concluded from the study be applied to instructional design? EBSCO Rationale: The article helps bridge a gap that is commonly seen in quantitative research methods and academia in general. This is also helpful design? Is it possible that leaders could influence the way educational research is conducted for instructional course designs? Why or why not? EBSCO Rationale: The article provides some information which could impact the effectiveness of instructional design. • Changing Assessment Practices of Teaching Candidates and Variables that Facilitate that Change - Ute Kaden & Phillip P. Patterson Kaden, U. u., & Patterson, P. P. (2014). Changing Assessment Practices of Teaching Candidates and Variables that Facilitate that Change. Action In Teacher Education, 36(5/6), 406-420. doi:10.1080/- .977700 • What is the concept of assessment literacy and how could it be applied to instructional design principles? • Provide at least one benefit and drawback of providing data packets to observers prior to a study being conducted. EBSCO Rationale: The article provides some helpful hints on creating an Online Learning & Teaching, 10(4), 657-670. • How could color research studies apply to instructional design methods in the future? • What is Section 508 of the Americans with Disabilities Act (ADA)? EBSCO Rationale: The article brings in an issue of compliance which can be very helpful in providing potential research from a number of different points of view. Data Mining for Education Decision Support: A Review – Suhirman, Jasni Mohammed Zain, & Tutut Herawan Suhirman, Zain, J. M., & Herawan, T. (2014). Data Mining for Education Decision Support: A Review. International Journal Of Emerging Technologies In Learning, 9(6), 4-19. doi:10.3991/ijet.v9i6.3950 • What is the concept of data mining and how could it apply to instructional design? • How is data mining similar and dissimilar to educational research. EBSCO Rationale: The article on to research provided in the article could be used in instructional design and why or why not. • Provide a brief example of the main research approaches detailed and documented in the article. Rationale: Research methods provided here are good to apply to instructional design and digital learning. Research Methods – David Nunan Nunan, D. (2000). Research methods. Routledge Encyclopedia of Language Teaching & Learning, 515-520. • What are the four (4) common categories of research mentioned in the text? • What are the key differences between the postpostivist and social constructivist paradigms and how could they apply to designing courses? EBSCO Rationale: Application of common research methods in educational research as well as tie-in to instructional design as a whole. What is Ethics In Research & Why Is It Important? – David B. Resnik Resnik, D. B. (2011, May 1). What is Ethics In Research & Why Is It Important? Retrieved Outcomes, and Associative Statistics – James Edward Osler & Mahmud Mansaray Osler, J. E., & Mansaray, M. (2014). A MODEL FOR DETERMINING TEACHING EFFICACY THROUGH THE USE OF QUALITATIVE SINGLE SUBJECT DESIGN, STUDENT LEARNING OUTCOMES AND ASSOCIATIVE STATISTICS. Journal On School Educational Technology, 10(1), 22-35. • What is the concept of single-subject research studies and how could it be applied to instructional design products? • How could the relationship between trainers/teachers and students be impacted by a single-subject research project? EBSCO Rationale: The single subject research design is a critical one to apply to instructional design because most learning that is done online is commonly alone. Can be applied to teams or mass learning groups as well. The Evolution of a Teacher Inquiry Culture – Carol R. Rinke & Divonna M. Stebick Rinke, C. C., & Stebick, D. M. (2013). "Not Just Learning About It But Actually Doing It": The Evolution of a Teacher Inquiry Culture. in getting feedback for instructional designs. Research and Consultation Guidelines – Kirklees (UK) Government Council Kirklees. (2015). Research and Consultation Guidelines. Retrieved from https://www.kirklees.gov.uk/co mmunity/yoursay/Questionnair es.pdf. • What are at least two (2) or three (3) reasons why questionnaires are important to use in gathering data for research projects and in essence instructional designs? • Name a few advantages of disadvantages of the article and how they relate to instructional designs. Rationale: The article is very interesting as instructional design usually has strict requirements that must be followed closely. The article mimics that to a point and is good to apply here. Doing Quantitative Research in Education with SPSS – Daniel Muijs Muijs, D. (2011). Doing Quantitative Research in Education with SPSS, 2e. Thousand Oaks, CA: Sage. 110. • What are four (4) reasons to use quantitative research methods in effective research study. It also can show one critical flaw (referencing data packets). Best Practices in Instructional Design for Web-Based Training – United States Department of Labor (LearningLink) Labor, United States Department of (2011). Best Practices in Instructional Design for Web-Based Training. Retrieved from http://www.dol.gov/oasam/l earninglink/2011BestPractic es.pdf. • When reviewing pages 31-33, what is the four (4) step process Kirkpatrick created for learning evaluations? • How could learner engagement practices provided be applied to educational research methods? Rationale: The guide provided is a good tool to have for instructional design as well as effective ways to engage learners with quality content. Combining Qualitative and Quantitative Approaches: Some Arguments for Mixed Methods Research – Thorleif Lund Lund, T. T. (2012). Combining Qualitative and Quantitative Approaches: data mining is a bit dense but provides good information on a more technical side of instructional design and educational research methods. from http://www.niehs.nih.gov/rese arch/resources/bioethics/whati s/ • What are the most common norms of ethical research principles provided? • Name two (2) reasons that researchers behave unethically and provide one (1) additional one that could be derived from the article. EBSCO Rationale: Important application of ethical considerations to research. Critical to the success of instructional design and tailoring to all learners. Five principles for research ethics Smith, D. (2003). Five principles for research ethics. American Psychological Association, 34(1), 56-61. • According to the American Psychological Association (APA), what are the five (5) principles of research ethics? • Name at least three (3) or more privacy concerns that could come up in research studies. EBSCO Rationale: Incorporation of the APA standards of ethical research as most people tend to not know about them and how they could be beneficial to research study creation and execution. Action In Teacher Education, 35(1), 7284. doi:10.1080/- • Referencing a teacherinquiry culture, what are two (2) or more qualitative methods that could apply from information provided in the article? • Is it possible that students AND facilitators contribute data to make more effective instructional materials in the future? Provide data from the article to prove your point. EBSCO Rationale: Interesting article which shows different qualitative methods being used but also incorporation of all into the design process (potentially). Qualitative and Quantitate Analysis – Ruth A. Wienclaw Wienclaw, R. A. (2009). Quantitative and Qualitative Analysis. Research Starters Sociology (Online Edition). Note: This article would be a required reading in both Modules 2 and 3 of course. • What are some common differences between qualitative and quantitative research methods that the authors provide? • Define the paradigms of qualitative research provided in the article. educational research studies? • Define a hypothesis in a sentence or two and how it is important in a research project. Rationale: Although this was from a book on a commonly-used statistical software for quantitative research, it is very rich in information with respect to the fundamentals of research (mostly quantitative though). Adding A Web-Based Perspective to the SelfAssessment of Knowledge: Compelling Reasons to Utilize Affective Measures of Learning – Jennifer C. Richardson, Yukiko Maeda, & Karen Swan Richardson, J. C., Maeda, Y., & Swan, K. (2010). Adding a Web-Based Perspective to the Self-Assessment of Knowledge: Compelling Reasons to Utilize Affective Measures of Learning. Academy Of Management Learning & Education, 9(2), 329-334. doi:10.5465/AMLE- • Is it possible that media can affect the results derived from quantitative research studies? Explain. • Please provide feedback on whether or not the study reviewed was effective or not? How could it apply to Some Arguments for Mixed Methods Research. Scandinavian Journal Of Educational Research, 56(2), 155165. doi:10.1080/- .568674 • What are some reasons one should execute a mixed methods research study? • Given the information in the article, what are some advantages of mixed method research and how could they be applied to instructional design? EBSCO Rationale: This article is a bridging article showing there is a third school of thought for educational research. It also helps with best practices for educational research. Bridging The QualitativeQuantitative Divide: Guidelines for Conducting Mixed Methods Research in Information Systems – Viswanath Venkatesh, Susan A. Brown, & Hillol Baia Venkatesh, V., Brown, S. A., & Bala, H. (2013). BRIDGING THE QUALITATIVEQUANTITATIVE DIVIDE: GUIDELINES FOR CONDUCTING MIXED METHODS RESEARCH IN INFORMATION SYSTEMS. How We Know What We Know": A Systematic Comparison of Research Methods Employed in Higher Education Journals,- v. 2006—2010 – Ryan S. Wells, Ethan A. Kolek, Elizabeth A. Williams, & Daniel B. Saunders Wells, R. R., Kolek, E. A., Williams, E. A., & Saunders, D. B. (2015). "How We Know What We Know": A Systematic Comparison of Research Methods Employed in Higher Education Journals,- v-. Journal Of Higher Education, 86(2), 171-195. • Do you think that most research studies are statistically-dependent as commented in the article. Explain. • How could concerns for professionals in academia and their lack of scholarly work be similar to what could be happening in instructional design? EBSCO Rationale: This is an interesting article for discussion and impact on good research methods for instructional designers when creating materials. What Teachers Need to Know About Teaching Methods – Peter S. Westwood EBSCO Rationale: Definitional differences between qualitative and quantitative research as well as paradigm explanations. Struggling With Theory? A Qualitative Investigation of Conceptual Tool Use In Instructional Design – Stephen C. Yanchar, Joseph B. South, David D. Williams, Stephanie Allen, & Brent G. Wilson Yanchar, S. C., South, J. B., Williams, D. D., Allen, S., & Wilson, B. G. (2010). Struggling with Theory? A Qualitative Investigation of Conceptual Tool Use in Instructional Design. Educational Technology Research And Development, 58(1), 39-60. doi: 10.1007/s- • How could theories similar to ones provided in the article could in general be incorporated in instructional design techniques? • Using data collected in the article, what, if anything, could be applied to instructional design techniques you could use in the future? EBSCO Rationale: The article is paramount to the general explanation of qualitative research. Though not shown above, questions about multiple interview segments will be addressed instructional design methods if possible? EBSCO Rationale: The measurement of learning is critical to update and modify instructional designs to appease to the majority of learners at the time of publication and in the near future after that. This article is a good application of that in the eyes of the author. Instructional Development for Teachers in Higher Education: Effects on Students’ Learning Outcomes – Ana Stes, Sven De Maeyer, David Gijbels, & Peter Van Petegem Stes, A. A., De Maeyer, S., Gijbels, D., & Van Petegem, P. (2012). Instructional development for teachers in higher education: effects on students' learning outcomes. Teaching In Higher Education, 17(3), 295-308. doi:10.1080/- Note: This article would be a required reading in both Modules 3 and 4 of course. • What is a pretest and posttest and how could they apply to instructional design materials? • Name at least three (3) takeaways that could be concluded from the article presented. EBSCO Rationale: The use of methods provided in the MIS Quarterly, 37(1), 21-54. • What are the four (4) most common types of mixed methods research? • Define the concept of validity and ones provided in the article EBSCO Rationale: The article is important to apply information needed for validity of research projects as well as types of mixed methods research studies. Lecture Capture: An Effective Tool for Universal Instructional Design? – Susan Watt, Susan Vajoczki, & Geraldine Voros Watt, S., Vajoczki, S., Voros, G., Vine, M. M., Fenton, N., & Tarkowski, J. (2014). Lecture Capture: An Effective Tool for Universal Instructional Design?. Canadian Journal Of Higher Education, 44(2), 1-29. • What were the three (3) technologies provided that were used to capture lectures? • How could lecture capturing be used as an education research method as well as help facilitate instructional design materials in the future? EBSCO Rationale: Lecture Presentations Westwood, P. S. (2008). What Teachers Need to Know About Teaching Methods. Camberwell, Vic: ACER Press, 82-88. • Applying your knowledge of instructional design techniques, how do the impressions provided change the way you feel courses are designed? • Which two (2) methods provided are ones that the authors hoped to see be used more often in the future? EBSCO Rationale: The article is important in potentially shaping the future of instructional design. In the opinion of the course designer, this is an article that can assist in additional research being done in the area. in lecture. text are important to define the purpose of quantitative research methods as well as commonly-used approaches. capturing is a newer technique in instructional design and should be mentioned as a best practice to accommodate learners of all ages. Part 1 – Course Introduction, Definitions of Research and Different Types of Observations American College of Education • This presentation provides a brief overview of the course. Additionally, this part of the first module gives an overview of research. Part 1 – Most Common Methods of Research Used by Instructional Designers/ Technologists American College of Education • This presentation provides a brief overview of the three (3) main methods used in research as a whole. This is built on future modules and portions of those as provided. Part 1 – Quantitative Research Fundamentals American College of Education • This presentation is an introductory review of quantitative research methods. Advantages and disadvantages are also provided for one to review as a starting point with this type of research option. Part 1 – Best Practices for Research Studies American College of Education • This presentation provides learners an overview of the concept of mixed research methods as a starting point. After that, a comprehensive process to effectively collect data is provided for learners to use in any type of study. Part 2 – Differences between Scientific Method & Process and Relation to Educational Research American College of Education • This presentation gives Part 2 – Qualitative Research Fundamentals American College of Education • This presentation is an Part 2 – Hypothesis Testing American College of Education • This presentation provides one with one of the most important core Part 2 – Optimizing Data Sets & Collections American College of Education Part 1 – Evaluation of Data Collection Methods Used American College of Education • Knowing the different types of research methods now can help bring an idea or topic alive. This presentation outlines some ways to successfully measure data that was collected in a study – regardless of method used. learners a chance to review the differences between the scientific method and process. Additionally, application to educational research and instructional design are provided as they are empirical to the success of these disciplines with respect to these methods. Part 3 – Ethical Issues in Educational Research Studies American College of Education • This presentation provides learners with a review of ethics and the purpose of it in educational research. Additionally, students get a brief understanding of some examples of educational research gone afoul. Part 4 – Prevention of Ethical Issues and Examples of Wrongdoing American College of Education • This presentation builds on the previous one with examples of studies that went wrong. This par also provides opportunities for ways to limit potential harm and exposure via proper ethical behaviors. introductory review of qualitative research methods. Different options to conduct research are provided for learners to review in this part of the module. Advantages and disadvantages are also provided for one to review as a starting point with this type of research option. Part 3 – Commonly-Used Qualitative Research Methods American College of Education • This presentation is a review of some of the most common method that are applied to qualitative research. The core of this part of the module is the composition of a qualitative research study. Part 4 – Discussion of Qualitative Research Methods Learned & Researched American College of Education • This presentation provides learners with three (3) different research studies that apply the use of qualitative research methods. These are applicable to educational research as well as instructional design concepts of quantitative research – hypothesis testing. Errors that could occur are included in this part of the module amongst other things related to them. Part 3 – Commonly-Used Quantitative Research Methods American College of Education • Research tools that one can use to create effective quantitative studies are the focus of this presentation. Included in this presentation are some of the more common tools that are used in this type of research. Part 4 – Discussion of Quantitative Research Methods Learned & Researched American College of Education • This presentation provides learners with three (3) different research studies that apply the use of quantitative research methods. These are applicable to educational research as well as instructional design practices. • This presentation provides is a bit different than the first part which gives several methods that can be used in any type of research project to effective collect data. A review of advantages and disadvantages for each method are provided here as well which tie into these methods. Part 3 – Ethical Considerations to Data Collections American College of Education • This presentation brings back the concept of ethics research in a different point of view when compared to the first module. Here, a brief historical timeline of ethics are provided which are used nowadays for research projects. Part 4 – Creating Effective Measuring Instruments American College of Education • This presentation provides recommendations for creating effective measuring instruments for research studies. Additionally, a research proposal framework is provided which can be adjusted Part 2 – Procedural Review of Data Collection Method Process American College of Education • This presentation provides another option for a research report that could be used in academia or professionally. An important topic of dangers that could occur when failing to pay attention to a research study is the next topic up for bids. A third topic that is also just as important to present here is the sharing of data and resources. Part 3 – Application to Instructional Design American College of Education • This presentation provides learners a bridge to instructional design and how it can be applied to items learned in this course. Benefits of doing research studies for the betterment of instructional design (ID) are also emphasized in this presentation. Part 4 – Self-Reflection & to meet one’s needs (based on the circumstance). practices. Discussion Prompt and Questions (18 points each; unless noted otherwise) Notate Peer Review, Signature Assessments, and DLCs Research is a core component of any discovery and obtainment of information to forward one’s knowledge. What one may not know though is that research is a lot more complex than one thinks. Hence, the study and understanding of it is paramount to successfully further advancements in instructional design. Qualitative research is one faucet of research that is commonly ignored because it can be more time consuming than quantitative research studies. What one doesn’t know though is that they are just as impactful and if not more at times. Questions Quantitative research methods are the most likely source of data in a research study. They’re easy to create, promote, administer, and decipher. However, some people shy away from these because of the mathematics and statistical analyses that may be required to come up with a conclusion. • Questions Questions • • • In your mind, what is the purpose of research and how does it benefit instructional design? In past experiences, have you found researching a specific topic easy or difficult at times? Why? Have you ever faced an ethical dilemma when conducting a research study? If so, what happened? • • Without reading any articles assigned, what is your view of qualitative research? Why do you think people shy away from qualitative studies? What are reasons that you would or would not use this method to gather data to improve instructional designs? Now that we have a better understanding of the two (2) most common research methods, a third one is now introduced and incorporated into some best practices that can be used in research. Keeping these in mind will ensure that the study will have minimal disruptions and be done properly. Questions • • Using knowledge from previous courses, which quantitative method would you like to use more than others? Why? Do you think that quantitative research studies are effective when creating improved instructional design instruments? Explain. • • • Do you think that the mixed method research study is effective to use in instructional design? Why or why not? Is it possible to have a research study with too many questions? Explain your answer for or against the point. Outside of ones presented, can you think of any other practices that researchers could Future Applications American College of Education • This presentation is the final piece of the module and course which gives an opportunity for students to reflect and share thoughts on others’ work. Also, a predictive future of ID is provided for one to share and comment on. Data mining, although commonly used in information system development, is quickly becoming an important talking point in instructional design and education as a whole. Data that is collected must be reviewed which can be daunting considering the quantities that one may have at their disposal. Combine these with standards that may be required by an institution or firm and one may have a challenging task on their hands to improve instructional materials. Questions • • Do you think that data mining is effective or ineffective when creating courses? Why or why not? Where would be some places would you look for additional employ to have solid research studies? Assignments (30 points each; unless noted otherwise) Notate Peer Review, Signature Assessments Analysis/Application Review of Ethics in Educational Research and Instructional Design In this assignment, you will review the concept of ethics in research and provide your opinion to how important or unimportant it is in the realm of research Please understand that the assignment that is asked of here should be done ONLY by yourself. This is a learning opportunity in research fundamentals. Within a three (3) to five (5) page paper, please answer the following questions. Graphical references, provided they are relevant, are welcome to be included as part of the submission. • What is your opinion on research ethics and why? • What research did you find to support your argument for or against ethics (please provide at least two (2) or more references in APA (6th edition) references to support your opinion and reason)? • Compare your responses Analysis/Application Qualitative Research Methods Overview Analysis/Application Quantitative Research Methods Overview To complete the assignments in this course and in the advanced course, you will decide on a specific group of learners to focus and plan for. You may choose from a list of possible learners including: military, healthcare, business, early childhood, K-12, or students in higher education. This will need to be the basis of assignments going forward as this group will become your population sample. Similar to the previous module, this one focused on the more commonly-used research method – quantitative research. It is more common to see quantitative research methods in educational research than qualitative research studies. In Module 2, we began the conversation on various ways to receive data from populations. Specifically, this module focused on qualitative research methods. It is important to have a good understanding as to what is being reviewed here with respect to the concept of this type of research. Using knowledge that was gathered from the readings and the lecture presentations, write at least a three (3) page paper, and please answer the Using knowledge that was gathered from the readings and the lecture presentations, write at least a three (3) page paper, and please answer the following questions below. Graphical references, provided they are relevant, are welcome to be included as part of the submission. • What are some reasons that quantitative research methods are more attractive than mixed method studies or qualitative research studies? • Which of the methods would you be most likely to use for your study and how could it be applied? • Do you feel that the concern of information Application Application of Research Methods & Creation of Sample Measurements This module brings together the two (2) most common research methods into a third one that is becoming more popular and more common – mixed research methods. In addition to that, this module is one which gives students a chance to apply their skills by creating simple measuring tools. Part 1: Video Presentation In this assignment, you will create a video or screencast of the methods that you chose for your research project. Using one of the many video software tools we have discussed, either free or paid, you will create a short video, providing a link to it in your assignment written out as a full URL. This video will focus on methods that you created to apply to the project at hand. This doesn’t have to be tips and strategies for improving instructional materials? • If you’ve done research studies in the past, where would you want to make improvements of them and why? Application Final Submission & Presentation to Group After receiving feedback from classmates in the Module 5 Discussion, consider the feedback and revise as needed. Submit your final video and paper. Reflection Educational Research & Instructional Design Reflection In this Reflection, think about what you have learned, and self-evaluate your current knowledge and skill level and relate your learning to one of the college dispositions by responding to the following questions in a four-page paper: • What aspects of this course were most beneficial to you? • How can you apply the knowledge and skills you have acquired in this course and the tools presented, including the checklists, in your personal and • with questions asked and discussed with others in the course that may align with your opinion or completely polarize it. What areas of ethics do you feel that should be looked at further that isn’t and why? following questions below. Graphical references, provided they are relevant, are welcome to be included as part of the submission. • Which of the methods that were presented in the module are ones that persons conducting research would be more likely to use and why? • Is this method more effective or less effective than the others? Explain your answer as part of the paper response. • Which methods would you prefer after briefly describing your sample population and your research topic? Note that it must be relevant to instructional design (ID) in order to be effective here. • What were some other research methods that were qualitative in nature that you would want to use and why? Provide at least two (2) references that are in APA (6th edition) references to support your viewpoints. Please review responses and comments provided by others to dynamically shape your paper for this module in a better light. You must respond to at least one (1) other person in the class to determine commonalities or polarizations in methods used. • overload could be a reality when population samples are too small or big? Explain why. Other than ones mentioned for the course, are there other quantitative methods that could be effective in educational research and instructional design? Provide at least two (2) references that are in APA 6.0 format to back up your claim. Please review responses and comments provided by others to dynamically shape your paper for this module in a better light. You must respond to at least one (1) other person in the class to determine commonalities or polarizations in methods used. anything of great length but a couple of minutes of discussion of what you chose. Additionally, you should be able to provide a glimpse of a couple of questions that you may use for each method. If they are online, you must provide the URL’s for the links to said methods. • • Depending on your level of skill, you may edit the video to create a more concise, high-quality learning experience. • Part 2: Video Summary Students will need to create at least one (1) qualitative and quantitative research method that they would use in a ‘normal’ research study. The study should not be carried out at this time since it is more of a demonstration of knowledge of materials presented. If applicable, one can include a mixed methods study as an added bonus. For each method in a three (3) page paper minimum, one should provide rationale why these were their final choices for the research study they could do in the future. Literature from the online library should be included to provide a sufficient backing of the use of said methods. Briefly summarize your professional life? How could your course or module be revamped, extended, and changed to make it more effective or engaging? In this course, you read, created, and may have made mistakes. Explain, and/or show what you have learned in this course. Relate your learning to at least one of the program outcomes. What goals do you have now as a result of the assignments and readings in this course? You may present this reflection in any way you please. The format you choose to present this information should reflect your current skill level, and push it beyond what you can easily create. Use APA (6th edition) references to support your viewpoints. audience and purpose of the study. Then answer these questions: • What aspects of the research method process were most challenging for you? • What did this assignment contribute to your knowledge of educational research & instructional design, and how you should determine your instructional strategy? • Use at a minimum of three (3) reference from this course or outside to support your viewpoint. • Submit your draft video and paper. Use APA (6th edition) references to support your viewpoints.