Gilbert C. Deniega

THEORY AND GAMES: DEVELOPING A METHOD FOR GAMIFYING HIGHER EDUCATION E. Rivera, C. Garden Edinburgh Napier University (UNITED KINGDOM) Abstract Gamification, the use of game attributes in non-game context, is used in the private sector as well as education[1]. Research into Gamified Learning (GL) often focuses on case studies of impact or value [2], and can suffer from a lack of parsimony. The results are frameworks for gamified learning with limited or unclear practical applications and even less research on how to “gamify”. However, consistent among this research are intersections with research in Student Engagement (SE). This presentation describes how a study on gamification’s impact on SE produced an unintentional by-product - a way to create GL without extensive knowledge of gamification, and potentially serving research with a testable technique for “gamifying” in Higher Education (HE). Existing means for observing GL were insufficient for our study, so we sought to create one via a synthesis of frameworks. First, Bedwell’s taxonomy of “game attributes” (GA’s) isolates and describes the basic parts of any game [3]. This helps to identify the presence of GA’s in learning. Next, the use of those GA’s must conform to a definition of GL that’s rigorous enough to be measured. Landers “Theory of Gamified Learning” (ToGL), contextualises GL as an intervention that identifies, extracts, and embeds GA’s into learning [4] where GA’s, instructional content, and students’ “behaviours and attitudes” (B/A’s) work together to affect learning. Encouragingly, much of the individual B/A’s described in the ToGL are SE concepts. If SE can stand in for B/A’s in the ToGL, then SE research may hold the key gamification. For this, we turned to Ella Kahu’s consolidation of multiple SE frameworks in HE [5]. This breaks SE down into measurable variable-states ideal for Landers’ B/A’s, completing our framework for observing GL’s impact on SE. However, in the process, Kahu’s work also implies that relationships between GA’s and SE variable-states are explainable by many learning theories. In which case, almost any learning theory could be used to select GA’s to target SE variable-states, using our newly completed observational framework. Rather than observing GL, you could create it. The result is a step-by-step methodology for gamifying formative assessments. Here, Kahu’s SE framework is mapped onto a player’s engagement (PE) with a game, and a learning theory of their choice explains how it’s GA’s affect (PE), and thus SE. This allows for the identification, extraction, and embedding of game attributes, targeting SE for learning. The methodology is already in test phases with staff members at Edinburgh Napier University. 1 INTRODUCTION Recently, The United Kingdom Higher Education sector has received pressure from the government[6] to enact policies and practices which encourage students to weigh their potential experience with the cost of education. However, enabling students to distinguish between universities via experience requires that universities provide some type of metric with which student experience could be measured. The impetus to better understand the student experience, not simply to measure it, but also to improve it is not new. In fact, it is one of the reasons why UK Universities have begun subscribing to the methodology of Transforming the Experience of Students Through Assessment (TESTA) [7]. TESTA has provided universities with an understanding of how assessments can affect the way students perceive their education as a whole, and what they take away from the experience. In that regard, one of the major conclusions of TESTA’s first few years is the value of formative assessment. Formative Assessment is an assessment that is not meant to evaluate the student, but rather evaluate the student’s learning, and through feedback provide the student with a tailored comprehensive picture of how to achieve their learning goals prior to a final evaluation[8]. In essence, to participate in a formative assessment is to participate in a developmental act of teaching and learning. However, lecturers often encounter problems with formative assessment. Attendance can be poor, participation can be minimal, and students often do not take advantage of the feedback that is offered. Unfortunately, this lack of engagement with formative assessment is actually in keeping with its theoretical foundations [9], where Formative Assessment prioritises learning above performance, feedback over marks, and choice over mandatory participation. However, evidence shows that when students are given a choice between marks or feedback, they will prioritise marks, and even ignore feedback in the presence of marks[10]. This might lead some to conclude that formative assessment causes or suffers from poor student engagement, however the theory that supports formative assessment practice in UK Higher Education suggests that engagement plays a major role in what makes formative assessment so effective[11]. Given the challenges to engagement in formative education, we began investigating gamification. Gamification, as defined by Deterding, is the use of game elements in a non-game context, affects how people participate in an experience[12]. Our study seeks to understand how gamification impacts student engagement with formative assessment. In order to explore the effect of gamification on student engagement, we required a means of observing and evaluating gamification as it acts in a formative assessment or learning scenario, and a means of observing and measuring levels of student engagement in relation to that gamified learning. However, gamification research is limited despite its rapid proliferation [2]. Dicheva et al’s mapping of gamification research confirmed the conclusions of our own literature review, that gamification research in education is largely anecdotal or statistical in nature, rarely including the application of any theoretical framework that grounds their definition of gamification, how it affects its outcomes, or how the outcomes qualify as engagement. This leads us to the conclusion that there is no existing framework to directly support our study. The most applicable exceptions are works that pertain to aspects of our study, but not the whole. First, Bedwell’s Taxonomy of Game Attributres [13] attempts to clarify, at least in part, what Deterding means by ‘game elements’. These attributes describe nineteen fundamental components used by any game, as well as providing context for their use. This work was further developed to demonstrate a link between specific Game Attributes and Training outcomes [14]. These Training Outcomes are grounded in the pedagogical theoretical framework of Kraiger at al [15]. Secondly, Lander’s Theory of Gamified learning offers a framework for understanding the mechanics of how gamification affects learning. This framework is an abstraction, distilled from previous gamification research, which anatomises a gamified learning scenario into four interrelated components. These are the instructional content, the game characteristics, the learning outcomes, and perhaps most importantly, the behaviours and attitudes of the student. The behaviours and attitudes are a significant addition to research in gamified learning, because they pertain to a key principle of Landers’ framework, that gamification does not affect learning but rather affects a student’s behaviour and attitudes, which then in turn affect learning. However, neither of these works relate explicitly to Student engagement. Given that there is no single framework that could allow us to observe gamification’s effect on engagement with formative assessment, we chose to design one. This framework would have to negotiate multiple theories, observing gamified learning as defined by Landers, focusing on how the use of Bedwell’s Game Attributes affected engagement, within a formative assessment as it is defined for use in our study the likes of Jessop, Black, and Williams [11], [16]and practiced in universities like Edinburgh Napier. However, doing this would also require a definition for student engagement which aligns with the other frameworks. One rigorousness enough to support observation and measurement. The process of combining these theories resulted in a framework for observing gamification and engagement in formative assessment, with implications for further development that might allow practitioners to employ gamification (even those unfamiliar with it) to purposefully and systematically target student engagement in the design of teaching and learning. 2 METHODOLOGY Our methodology involved identifying key concepts in gamification and formative assessment theory, accounting for varying shifts in language and terminology as well as shared foundations in previous research between the two areas, to search for overlapping concepts that might also exist in engagement theory. We sought to determine an approach to engagement that would not contradict our definitions for gamification, gamified learning, game attributes, and formative assessment, and would potentially aid in their consolidation. This began with an attempt to consolidate our existing theories, where ever possible. Beginning this process required an understanding of the potential scope of the outcome. The idea was to create an observational framework which could be repeated by other researchers, and was not limited in its use to our purposes or our setting. We began consolidation with Lander’s Theory of Gamified Learning(TOGL). Gamified learning is not gamification. Rather, it is a highly contextualised version of gamification. It is also not a basis for the practice of gamified learning, but merely a means of understanding it. This means that it is simple and abstract enough to apply to any gamified learning scenario, but still complex enough to describe conceptual relationships within that scenario. TOGL would provide a template to be expanded upon or further defined through the consolidation of our other theories through a close analysis of Lander’s work. This involved isolating each component in the theory of gamified learning and finding related concepts and functions in formative assessment and other parts of gamification. LEARNING OUTCOMES Starting with the most obvious relationship, learning outcomes are the final component to gamified learning. Learning outcomes, as defined by Taxonomy [17], describe what the student should be able to do at the conclusion of the learning experience. Bloom’s taxonomy is formally used in the design of learning outcomes throughout universities and the UK. While this taxonomy is often restricted to the cognitive levels of learning in Higher Education teaching practice, there is nothing about the TOGL that is exclusionary to those levels. At this stage in development, learning outcomes as featured in TOGL are not a point of consolidation but rather an indicator that this theory can be used to describe gamified learning as employed by a variety of learning theories and teaching methodologies. BEHAVIOURS AND ATTITUDES From Learning Outcomes, the closest component in the TOGL is ‘behaviour and attitude’. Landers never goes as far as defining behaviour/attitude, but he is clear about its place in gamified learning and even places it on par with an undefined use of the word engagement, stating ‘In contrast, gamification practitioners do not generally seek to influence learning directly; instead, the goal of gamification is to alter a contextual learner behavior or attitude (e.g., engagement)’. He goes on to draw numerous examples from education research that describe ‘cognitive effort’, ‘active participation’, ‘engagement in school work’, and ‘note taking and reflection’ and their effects on the students learning and overall benefit. The implication here is that Behaviour/Attitude is the primary object of gamification’s impact and is indicative of the actions and state of mind that, when viewed in the context of education, directly affect the achievement of learning. Unfortunately, though Lander’s seems content to equate Behaviour/Attitude with Engagement, his description of the term is not consistent enough to use a framework for engagement. Therefore, we’re remanded to noting Lander’s characterisation of behaviour/attitude and comparing it where possible to similar ideas in formative assessment. The pedagogic research that informs formative assessment is characterized by the incorporation of the cognitive acts of learning, as well as “self-attribution” and “readiness to learn”, an approach to learning which is informed by psychology and to a certain extent sociology [18]. Like Landers’ Behavior/Attitude, these are part of the psychological, sociological, and behavioral aspects of feedback that lecturers must understand in order to evaluate what a student needs to enable their own movement into a more advanced stage of learnedness, often referred to as the proximal zone of development [9], [19]. Therefore, in the context of our work, Behaviors/Attitudes are the cognitive, psycho-social, or behavioral ways in which students ‘engage’ with a gamified formative assessment and its feedback. Using these concepts, we reviewed research into the theory of Student engagement, seeking a framework which employs these concepts in analogous ways. Ella Kahu’s Framework for Student Engagement proved to be the prime candidate for our purposes. Kahu’s Engagement Framework consolidates a number of approaches to the subject throughout the Higher Education sector, including Behavioural, Psychological, Socio-cultural, and Holistic. Like Landers, Kahu causally anatomises her subject into constituent parts, those being the psycho-social and institutional factors which she refers to as Antecedents (the influences on a student engagement), the State of Engagement (the mitigating factor), and the consequences of engagement which include but are not limited to learning (the outcomes of engagement). The state of engagement, as she describes it, is a multi-dimensional set of variables that exist across cognitive (thinking), affective (emotional), and behavioral levels. This aligns with our employed theory of formative assessment and applies to Landers’ examples of Behavior/Attitudes. Moreover, in the construction of her framework, Kahu emphasizes the importance of distinguishing the state of engagement from its antecedents and consequences. With a framework for student engagement sufficient to our purposes, we can confidently substitute the mitigating component of Kahu’s framework, ‘student engagement’, for Lander’s behavior/attitudes and without concern that it overlaps or blurs into other components in the theory of gamified learning. GAME CHARACTERISTICS AND INSTRUCTIONAL CONTENT Proceeding behavior/attitudes in the TOGL are the components of game characteristics and instructional content. Both components feed directly in behavior/attitudes, but unlike game characteristics, instructional content has an additional link directly to learning outcomes. The link between instructional content and learning outcomes is a key aspect to Landers’ gamified learning, supporting his principle that gamification must be a transformative intervention, acting on a preexisting learning design which must effectively support learning without gamification. According to Landers, ‘Critical to the success of any gami- fication effort is that the instructional content in place is already effective. The goal of gamification cannot be to replace instruction, but instead to improve it. If the instructional content does not already help students learn, gamification of that content cannot itself cause learning.’ Much as he does with his behavior/attitudes component, Landers doesn’t provide a specific definition for game characteristics, but does consider Bedwell’s game attributes a sufficient example. Landers even goes so far as to incorporate game attributes into his definition for gamified learning, proposing ‘the gamification of learning can be best scientifically defined as the implementation of Bedwell and colleagues’ learning-related game attributes outside the context of a game. More specifically, in the context of learning, video game elements in Deterding and colleagues’ definition should refer to the game attribute categories described by Bedwell and colleagues. Based upon this contention, gamification of learning is defined as the use of game elements, including action language, assessment, conflict/challenge, control, environment, game fiction, human interaction, immersion, and rules/goals, to facilitate learning and related outcomes.’ While the taxonomy of game attributes can stand in for Landers’ game characteristics, its relationship to engagement is less straight forward than that of behavior/attitudes. The TOGL suggests that game attributes should be chosen based on the behavior/attitudes that a practitioner wishes to effect but is unclear about the means by which a practitioner makes that determination. Similarly, Kahu asserts that antecedents influence engagement, drawing on the various consolidated approaches to explain the different means by which different antecedents work together to influence engagement. However, despite the similarities between game attributes and engagement antecedents, or the comparative depth with which Kahu’s antecedents are constructed, this is not enough to determine where Kahu’s antecedents’s sit in the TOGL with respect to behavior/antecedents. However, listed amongst the antecedents in Kahu’s framework, are teaching, curriculum, and assessment, all of which are comparable to Landers’ component of instructional content. Given the nature of gamified learning as an intervention, it stands to reason that if engagement antecedents contain instructional content, and instructional content must be in place and effective prior to gamification, then it stands to reason that engagement antecedents should precede game attributes in the TOGL. Finally, by analyzing the component of instructional content to understand its role in our consolidated framework, we came across one of the first significant findings of this study. Beginning purely as a matter of linguistics, the idea of using substituting an assessment as instructional content seemed counter intuitive with regards to their role in obtaining and conveying information. Instruction delivers information, while assessment merely gathers it. Strictly speaking, assessments do not instruct. Delving into the TOGL even further, Landers only describes instructional content by what it does and where it comes from, implying that it is any material which is provided by a practitioner of teaching and learning, which serves as the basis for the achievement of learning outcomes and has a direct relationship to learning outcomes, stating ‘making the relationship between Instructional Content and Learning Outcomes stronger). Landers even cites his own experiment to draw a direct parallel between instructional content and formative assessment, ‘completion of the gamified practice tests was itself intended to increase learning... by gamifying the practice tests, the researchers hoped to encourage completion of more practice tests.’ Landers, however, does not state how completing the tests supports learning, but does describe how an ‘additional target behavior’ of engaging in repeated practice tests caused students to increase what Kahu’s Student Engagement refers to as ‘time and effort’ as a means of improving their own performance. Landers’ example may position assessment as instructional content; however it is the students’ performance and ‘additional target behavior’ that provides the student with information they use to improve their learning. What Landers is describing is analogous to a formative assessment wherein where learning takes not through the act of taking the assessment, but through the receipt and use of feedback. If gamified learning involves the application of game attributes to instructional content, the implication for gamifed formative assessments is that the intervention of gamification cannot be applied to any part of the formative assessment design, but rather must be applied the process of feedback or specifically targeted to affect the behavior/attitude with regard to feedback. This is the final component in our consolidated framework for understanding the impact of gamification on formative assessment. 3 RESULTS Our consolidated Framework for measuring gamification’s impact on student engagement with formative assessment (Fig.1) combines the major components of theories from gamification, gamified learning, formative assessment, and engagement, and preserves the original relationships between that link constituent components in their native theories. Working in concert with this framework, we employ a modified version of TESTA [16] to isolate the game attributes used in students’ gamified formative assessment and observe how each impacts on the affective, cognitive and behavioral aspects of a student’s engagement via their changing perceptions of their own engagement antecedents. Figure 1. The Gamification for Student Engagement Framework Relationship between elements of Landers’ Theory of Gamified Learning ([20], square), Kahu’s Framework for Student Engagement ([21], circle) and Bedwell’s Taxonomy of Game Attributes ([22], diamond). Assessment feedback is an antecedent of engagement, which can be likened to the instructional content of a game (A). The student/ player engages with the assessment/ content through behaviours and attitudes (C) to achieve the learning outcome, a consequence of engagement (B). Game characteristics/ attributes (D) influence the behaviour/ attitudes of the student/ player and hence the state of engagement, which is also influenced by the antecedents of engagement (E). Kahu proposes that engagement is circular, where increased engagement leads to increased outcomes of engagement such as academic performance, and in turn affects the antecedents of engagement, for example, the motivation to sustain academic performance (B, E). (PUTTING THEORY INTO PRACTICE: GAMIFICATION FOR STUDENT ENGAGEMENT,E.S. Rivera, C. L. P. Garden, Edinburgh Napier University, EduLearn2018) However, as our study developed and fellow teaching and learning practitioners became familiar with it, we perceived a need to provide lecturers with an accessible method for gamifying as none appeared available. Indeed throughout our research, we found no concise ‘how to’ for the process of gamifying let alone gamifying learning [2], [4], [23]. In response to this, and with a need to ensure the study had enough participants with gamified formative assessments to observe, we increasingly turned to our own framework to offer propositions for how one might gamify learning, ultimately developing a step-by-step process for the gamification of formative assessments, specifically targeting student engagement. This method for gamifying is currently being tested at our university in the form of instructional workshops. 4 CONCLUSIONS While method for the gamification of learning is still being refined, we encourage practitioners to test the framework, interrogate it, and further develop it into a robust and scalable tool for use across the sector. This is because the constituent theories used to create our framework are themselves broadly applicable, and its development and use has implications for how teaching and learning maintains preserves its purpose in the sectors response to market forces and changes in the political landscape. As universities become more competitive around student experience, the pressure to stand out could motivate institutions to quickly enact policies based on the latest movements, trends, or developments in education, of which gamification is one. These policies can put pressure on teaching and learning practitioners to engage in innovations they may believe are not support learning, innovations they do not fully understand, or innovations that are difficult or time consuming to implement given their current workload. Our intent is to not only provide those practitioners with a means of measuring gamification’s impact, but to also assist them in implementing gamification according to a process which is theoretically sound, accountable, and achievable in the face of limited time and resource, all while purposefully and systematically seeking to improve the way students engage with their learning. Political aspects. Future of HE. No more buzzwords. Accessible rigor, quality. ACKNOWLEDGEMENTS The authors wish to thank the Edinburgh Napier University Teaching Fellows Development Grant for supporting this work, Edinburgh Napier PhD student Kimberly Wilder for her consultation on TESTA, and Dr Samantha Campbell Casey, Dr Kevin Chalmers and Dr Bridget Hanna for co-supervision of ER’s PhD. REFERENCES References [Arial, 10-point, left alignment, upper and lower case] should be cited according to the Bibliography and Citation Style https://iated.org/citation_guide [1] S. Deterding, “Gamification: Designing for Motivation,” Interactions, vol. 19, no. 4, p. 14, 2012. [2] D. Dicheva, C. Dichev, G. Agre, and G. Angelova, “Gamification in Education: A Systematic Mapping Study,” Educ. Technol. Soc., vol. 18, no. 3, pp. 75–88, 2015. [3] K. A. Wilson et al., “Relationships Between Game Attributes and Learning Outcomes Review and Research Proposals,” Simul. Gaming, vol. 40, no. 2, pp. 217–266, 2009. [4] R. N. Landers, “Developing a Theory of Gamified Learning: Linking Serious Games and Gamification of Learning,” Simul. Gaming, vol. 45, no. 6, pp. 752–768, 2014. [5] E. R. Kahu, “Framing student engagement in higher education,” Stud. High. Educ., vol. 38, no. 5, pp. 758–773, 2013. [6] BIS, “Success as a Knowledge Economy: Teaching Excellence, Social Mobility and Student Choice,” 2016. [7] T. Jessop, Y. El Hakim, and G. Gibbs, “The whole is greater than the sum of its parts: a largescale study of students’ learning in response to different programme assessment patterns,” Assess. Eval. High. Educ., vol. 39, no. 1, pp. 73–88, 2013. [8] P. Black, C. Harrison, C. Lee, B. Marshall, and D. Wiliam, “The Nature and Value of Formative Assessment for Learning,” Improv. Sch., vol. 6, no. 3, pp. 1–17, 2003. [9] M. Yorke, “Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice,” High. Educ., vol. 45, pp. 477–501, 2003. [10] C. S. Dweck, “Self-theories: Their role in motivation, personality, and development.,” Essays Soc. Psychol., p. 214, 1999. [11] P. Black and D. Wiliam, “Developing the theory of formative assessment,” 2009. [12] S. Deterding, M. Sicart, L. Nacke, K. O’Hara, and D. Dixon, “Gamification. using game-design elements in non-gaming contexts,” Proc. 2011 Annu. Conf. Ext. Abstr. Hum. factors Comput. Syst. - CHI EA ’11, p. 2425, 2011. [13] E. Salas, W. L. Bedwell, D. Pavlas, K. Heyne, and E. H. Lazzara, “Toward a Taxonomy Linking Game Attributes to Learning: An Empirical Study,” Simul. Gaming, vol. 43, no. 6, pp. 729–760, 2012. [14] K. a. Wilson et al., “Relationships Between Game Attributes and Learning Outcomes: Review and Research Proposals,” Simul. Gaming, vol. 40, no. 2, pp. 217–266, 2009. [15] K. Kraiger, J. K. Ford, and E. Salas, “Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation.,” J. Appl. Psychol., vol. 78, no. 2, pp. 311–328, 1993. [16] T. Jessop, Y. El Hakim, and G. Gibbs, “Assessment & Evaluation in Higher Education The whole is greater than the sum of its parts: a large-scale study of students’ learning in response to different programme assessment patterns The whole is greater than the sum of its parts: a large-scale study of students’ learning in response to different programme assessment patterns,” 2017. [17] D. R. Krathwohl, “Revised Blooms Taxonomy,” Theory into Practice, 2002. [Online]. Available: http://www.unco.edu/cetl/sir/stating_outcome/documents/Krathwohl.pdf. [18] P. Black and D. Wiliam, Assessment and Classroom Learning, vol. 5, no. 1. 1998. [19] L. Vygotsky, “Interaction Between Learning and Development,” Mind Soc., pp. 79–91, 1978. [20] R. N. Landers, “Developing a Theory of Gamified Learning,” Simul. Gaming, vol. 45, no. 6, pp. 752–768, Dec. 2014. [21] E. R. Kahu, “Framing student engagement in higher education,” Stud. High. Educ., 2013. [22] E. Salas, W. L. Bedwell, D. Pavlas, K. Heyne, and E. H. Lazzara, “Toward a Taxonomy Linking Game Attributes to Learning: An Empirical Study,” Simul. Gaming, vol. 43, no. 6, pp. 729–760, 2012. [23] S. De, S. Borges, V. H. S. Durelli, H. M. Reis, and S. Isotani, “A Systematic Mapping on Gamification Applied to Education,” 2014.