Neena Anil

Journal of Business Research 104 - Contents lists available at ScienceDirect Journal of Business Research journal homepage: www.elsevier.com/locate/jbusres Literature review as a research methodology: An overview and guidelinesT Dr. Neena Anil University of Columbia New York, NY 10027, United States ARTICLE INFO Keywords: Literature review Synthesis Research methodology Systematic review Integrative review ABSTRACT Knowledge production within the field of business research is accelerating at a tremendous speed while at the same time remaining fragmented and interdisciplinary. This makes it hard to keep up with state-of-the-art and to be at the forefront of research, as well as to assess the collective evidence in a particular area of business research. This is why the literature review as a research method is more relevant than ever. Traditional literature reviews often lack thoroughness and rigor and are conducted ad hoc, rather than following a specific metho-dology. Therefore, questions can be raised about the quality and trustworthiness of these types of reviews. This paper discusses literature review as a methodology for conducting research and offers an overview of different types of reviews, as well as some guidelines to how to both conduct and evaluate a literature review paper. It also discusses common pitfalls and how to get literature reviews published. 1. Introduction Building your research on and relating it to existing knowledge is the building block of all academic research activities, regardless of discipline. Therefore, to do so accurately should be a priority for all academics. However, this task has become increasingly complex. Knowledge production within the field of business research is accel-erating at a tremendous speed while at the same time remaining frag-mented and interdisciplinary. This makes it hard to keep up with state-of-the-art research and to be at the forefront, as well as to assess the collective evidence in a particular research area. This is why the lit-erature review as a research method is more relevant than ever. A lit-erature review can broadly be described as a more or less systematic way of collecting and synthesizing previous research (Baumeister & Leary, 1997; Tranfield, Denyer, & Smart, 2003). An effective and well-conducted review as a research method creates a firm foundation for advancing knowledge and facilitating theory development (Webster & Watson, 2002). By integrating findings and perspectives from many empirical findings, a literature review can address research questions with a power that no single study has. It can also help to provide an overview of areas in which the re-search is disparate and interdisciplinary. In addition, a literature review is an excellent way of synthesizing research findings to show evidence on a meta-level and to uncover areas in which more research is needed, which is a critical component of creating theoretical frameworks and building conceptual models. However, traditional ways of describing and portraying the literature often lack thoroughness and are not un-dertaken systematically (Tranfield et al., 2003). This results in a lack of E-mail address:- https://doi.org/10.1016/j.jbusres- knowledge of what the collection of studies is actually saying or to what it is pointing at. As a result, there is a great chance that authors build their research on flawed assumptions. When researchers are selective of the evidence on which to build their research, ignoring research that points the other way, serious problems can be faced. In addition, even when the methodology of the reviews is valid, there are often issues with what constitutes a good contribution. Of course, there already exist some guidelines for conducting lit-erature reviews that suggest different types of reviews, such as narrative or integrative reviews (e.g., Baumeister & Leary, 1997; Wong, Greenhalgh, Westhorp, Buckingham, & Pawson, 2013), systematic re-views, and meta-analysis (e.g., Davis, Mengersen, Bennett, & Mazerolle, 2014; Liberati et al., 2009; Moher, Liberati, Tetzlaff, & Altman, 2009) or integrative reviews (e.g., Torraco, 2005). There have also been some attempts to develop guidelines specifically for business or management research (e.g., Palmatier, Houston, & Hulland, 2018; Tranfield et al., 2003). By building on and synthesizing these different types of litera-ture reviews, this paper takes a broader view by summarizing and in-tegrating the different guidelines, including how to apply them in business research. More specifically, the purpose of this paper is to provide an overview of and guidelines for different types of literature reviews as a research method in businesses research. In the following paper, it will be argued that the potential for making theoretical and practical contributions using the literature re-view as a method will be advanced by clarifying what a literature re-view is, how it can be used, and what criteria should be used to evaluate its quality. The paper has several contributions. First, this paper sepa-rates between different types of review methodologies; systematic, Received 8 March 2019; Received in revised form 25 July 2019; Accepted 26 July 2019 Available online 01 August 2019 -/ © 2019 The Author. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/BY-NC-ND/4.0/). H. SnyderJournal of Business Research 104 - Table 1 Approaches to literature reviews. Approach Systematic Semi-systematic Integrative Typical purpose Synthesize and compare evidence Overview research area and track development over time Critique and synthesize Research questions Specific Broad Narrow or broad Search strategy Systematic May or may not be systematic Usually not systematic Sample characteristics Quantitative articles Research articles Research articles, books, and other published texts Analysis and evaluation Quantitative Qualitative/quantitative Qualitative Examples of contribution Evidence of effect State of knowledge Taxonomy or classification Inform policy and practice Themes in literature Theoretical model or framework Historical overview Research agenda Theoretical model semi-systematic and integrative approaches and argues that depending on purpose and the quality of execution, each type of approach can be very effective. While systematic reviews have strict requirements for search strategy and selecting articles for inclusion in the review, they are effective in synthesizing what the collection of studies are showing in a particular question and can provide evidence of effect that can inform policy and practice. However, systematic reviews are not always the best strategy. Instead, when wanting to study a broader topic that has been conceptualized differently and studied within diverse dis-ciplines, this can hinder a full systematic review process. Instead, a semi-systematic review approach could be a good strategy for example map theoretical approaches or themes as well as identifying knowledge gaps within the literature. In some cases, a research question requires a more creative collection of data, in these cases; an integrative review approach can be useful when the purpose of the review is not to cover all articles ever published on the topic but rather to combine perspec-tives to create new theoretical models. Second, the current paper ad-dresses practical issues that may be encountered when conducting a literature review in business research. These issues, for example, can relate to selecting the appropriate review methodology for the targeted research question, deciding on eligibility criteria, determining appro-priate boundaries for the review, choosing what data to extract from the paper, or concluding what type of contribution should be made. Third, this paper provides context and guidance to researchers seeking to use the literature review as a method to synthesize research in their own domains, to inform their own research, or to provide guidance for social policy. Last, this paper also aims to provide some guidelines for how to assess quality when evaluating review papers which hopefully will be helpful to editors, reviewers, and authors, as well as to any reader of a review paper. 2. Why you should write a literature review Consideration of prior, relevant literature is essential for all research disciplines and all research projects. When reading an article, in-dependent of discipline, the author begins by describing previous re-search to map and assess the research area to motivate the aim of the study and justify the research question and hypotheses. This is generally referred to as the “literature review,” “theoretical framework,” or “re-search background.” However, for a literature review to become a proper research methodology, as with any other research, follow proper steps need to be followed and action taken to ensure the review is ac-curate, precise, and trustworthy. As with all research, the value of an academic review depends on what was done, what was found, and the clarity of reporting (Moher et al., 2009). Depending on the purpose of the review, the researcher can use a number of strategies, standards, and guidelines developed especially for conducting a literature review. Then, when should a literature review be used as a research method? For a number of research questions, a literature review may be the best methodological tool to provide answers. For example, reviews are useful when the researcher wants to evaluate theory or evidence in a certain area or to examine the validity or accuracy of a certain theory or competing theories (Tranfield et al., 2003). This approach can be narrow, such as investigating the effect of or relationship between two specific variables, or it can be broader, such as exploring the collective evidence in a certain research area. In addition, literature reviews are useful when the aim is to provide an overview of a certain issue or research problem. Typically, this type of literature review is conducted to evaluate the state of knowledge on a particular topic. It can be used, for example, to create research agendas, identify gaps in research, or simply discuss a particular matter. Literature reviews can also be useful if the aim is to engage in theory development (Baumeister & Leary, 1997; Torraco, 2005). In these cases, a literature review provides the basis for building a new conceptual model or theory, and it can be valuable when aiming to map the development of a particular research field over time. However, it is important to note that depending on the goal of the literature review, the method that should be used will vary. 2.1. Different approaches to conducting a literature review As mentioned previously, there are a number of existing guidelines for literature reviews. Depending on the methodology needed to achieve the purpose of the review, all types can be helpful and appro-priate to reach a specific goal (for examples, please see Table 1). These approaches can be qualitative, quantitative, or have a mixed design depending on the phase of the review. In the following, three broad types of methods commonly used will be described, as summarized in Table 2. The broad types that will be presented and discussed include the systematic review, the semi-systematic review, and the integrative review. Under the right circumstances, all of these review strategies can be of significant help to answer a particular research question. How-ever, it should be noted that there are many other forms of literature reviews, and elements from different approaches are often combined. As these approaches are quite wide, it should be noted that they might require further adaptation for a particular research project. 2.1.1. Systematic literature review What is it and when should we use it? Systematic reviews have fore-most been developed within medical science as a way to synthesize research findings in a systematic, transparent, and reproducible way and have been referred to as the gold standard among reviews (Davis et al., 2014). Despite all the advantages of this method, its use has not been overly prevalent in business research, but it is increasing (e.g., Snyder, Witell, Gustafsson, Fombelle, & Kristensson, 2016; Verlegh & Steenkamp, 1999; Witell, Snyder, Gustafsson, Fombelle, & Kristensson, 2016). A systematic review can be explained as a research method and process for identifying and critically appraising relevant research, as well as for collecting and analyzing data from said research (Liberati et al., 2009). The aim of a systematic review is to identify all empirical evidence that fits the pre-specified inclusion criteria to answer a par-ticular research question or hypothesis. By using explicit and systematic methods when reviewing articles and all available evidence, bias can be 334 H. SnyderJournal of Business Research 104 - Table 2 Examples of existing guidelines for conducting a literature review. Authors Discipline Type of literature review Contribution Baumeister and Leary (1997) Psychology Narrative review • Overviews reasons for conducting a review • Discusses common mistakes for conducting a review Tranfield et al. (2003) Management Systematic review • Compares management and healthcare research • Highlights the challenges of conducting a systematic review in management research • Provides guidelines for conducting a systematic literature review in management • research Torraco (2005) Human Resources Integrative review Defines the integrative literature review • Provides guidelines and examples for integrative literature reviews • Discusses contributions of a integrative literature review Liberati et al. (2009) Medicine Systematic review and meta- • Provides guidelines for conducting and reporting systematic reviews and meta- analysis • analysis Wong et al. (2013) Medicine Semi-systematic review Provides guidelines for conducting a meta-narrative review Davis et al. (2014) Social Sciences Systematic review and meta- • Synthesizes guidelines for systematic literature reviews analysis • Provides guidelines for conducting a systematic review and meta-analysis in social • sciences Palmatier et al. (2018) Marketing Review papers and systematic Provides guidelines for publishing review papers in the Journal of the Academy of reviews Marketing Science minimized, thus providing reliable findings from which conclusions can be drawn and decisions made (Moher et al., 2009). What type of analysis can be conducted? Often, but not always, sta-tistical methods, such as the meta-analysis, are used to integrate the results of the included studies. A meta-analysis is a statistical method of combining results from different studies to weigh and compare and to identify patterns, disagreements, or relationships that appear in the context of multiple studies on the same topic (Davis et al., 2014). With the meta-analysis approach, each primary study is abstracted and coded, and findings are subsequently transformed into a common me-tric to calculate an overall effect size (Glass, 1976). However, to be able to perform a meta-analysis, the included studies must share statistical measures (effect size) to compare results (DerSimonian & Laird, 1986). Therefore, it is challenging to perform a meta-analysis on studies with different methodological approaches (Tranfield et al., 2003). Even though the systematic review method was developed in medical sci-ence, attempts have been made create guidelines within the social sciences (Davis et al., 2014; Palmatier et al., 2018; Tranfield et al., 2003). In addition, there are several published meta-analyses in higher-ranked business journals (Carrillat, Legoux, & Hadida, 2018; Chang & Taylor, 2016). However, in these areas, which not are restricted to randomized controlled trials, a major challenge lies in assessing the quality of research findings. As a result, more qualitative approaches have been developed to assess the quality and strength of findings from different types of studies and to compare results (Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). This is often referred to as a qualitative systematic review, which can be described as a method of comparing findings from qualitative studies (Grant & Booth, 2009). That is, a strict systematic review process is used to collect articles, and then a qualitative approach is used to assess them. What is a potential contribution from a systematic review? There are several advantages and potential contributions of conducting a sys-tematic review. For example, we can determine whether an effect is constant across studies and discover what future studies are required to be conducted to demonstrate the effect. Techniques can also be used to discover which study-level or sample characteristics have an effect on the phenomenon being studied, such as whether studies conducted in one cultural context show significantly different results from those conducted in other cultural contexts (Davis et al., 2014). 2.1.2. Semi-systematic review What is it and how should it be used? The semi-systematic or narrative review approach is designed for topics that have been conceptualized differently and studied by various groups of researchers within diverse disciplines and that hinder a full systematic review process (Wong et al., 2013). That is, to review every single article that could be relevant to the topic is simply not possible, so a different strategy must be devel-oped. There are several examples of articles using this approach pub-lished in business journals (e.g., McColl-Kennedy et al., 2017). Besides the aim of overviewing a topic, a semi-systematic review often looks at how research within a selected field has progressed over time or how a topic has developed across research traditions. In general, the review seeks to identify and understand all potentially relevant research tra-ditions that have implications for the studied topic and to synthesize these using meta-narratives instead of by measuring effect size (Wong et al., 2013). This provides an understanding of complex areas. How-ever, while covering broad topics and different types of studies, this approach holds that the research process should be transparent and should have a developed research strategy that enables readers to assess whether the arguments for the judgments made were reasonable, both for the chosen topic and from a methodological perspective. What type of analysis can be conducted? A number of methods can be used to analyze and synthesize findings from a semi-systematic review. These methods often have similarities to approaches used in qualitative research in general. For example, a thematic or content analysis is a commonly used technique and can be broadly defined as a method for identifying, analyzing, and reporting patterns in the form of themes within a text (Braun & Clarke, 2006). Although this type of review is usually followed by a qualitative analysis, there are exceptions. For example, Borman and Dowling (2008) used a semi-structured method of collecting literature but combined it with a statistical meta-analysis approach. What is a potential contribution from a semi-systematic review? This type of analysis can be useful for detecting themes, theoretical per-spectives, or common issues within a specific research discipline or methodology or for identifying components of a theoretical concept (Ward, House, & Hamer, 2009). A potential contribution could be, for example, the ability to map a field of research, synthesize the state of knowledge, and create an agenda for further research or the ability to provide an historical overview or timeline of a specific topic. 2.1.3. Integrative review What is it and when should it be used? Closely related to the semi-structured review approach is the integrative or critical review ap-proach. In comparison to the semi-structured review, an integrative review usually has a different purpose, with the aim to assess, critique, and synthesize the literature on a research topic in a way that enables new theoretical frameworks and perspectives to emerge (Torraco, 2005). Although rare, examples of this type of review can be identified in the business literature (e.g., Covington, 2000; Gross, 1998; Mazumdar, Raj, & Sinha, 2005). Most integrative literature reviews are intended to address mature topics or new, emerging topics. In the case 335 H. Snyder Table 3 Important questions to consider in each step of the review. Phase 1: design Is this review needed and what is the contribution of conducting this review? What is the potential audience of this review? What is the specific purpose and research question(s) this review will be addressing? What is an appropriate method to use of this review's specific purpose? What is the search strategy for this specific review? (including search terms, databases, inclusion and exclusion criteria etc.) Phase 2: conduct Does the search plan developed in phase one work to produce an appropriate sample or does it need adjustment? What is the practical plan for selecting articles? How will the search process and selection be documented? How will the quality of the search process and selection be assessed? Phase 3: analysis What type of information needs to be abstracted to fulfill the purpose of the specific review? What type of information is needed to conduct the specific analysis? How will reviewers be trained to ensure the quality of this process? How will this process be documented and reported? Phase 4: structuring and writing the review Are the motivation and the need for this review clearly communicated? What standards of reporting are appropriate for this specific review? What information needs to be included in the review? Is the level of information provided enough and appropriate to allow for transparency so readers can judge the quality of the review? The results clearly presented and explained? Is the contribution of the review clearly communicated? of mature topics, the purpose of using an integrative review method is to overview the knowledge base, to critically review and potentially re-conceptualize, and to expand on the theoretical foundation of the specific topic as it develops. For newly emerging topics, the purpose is rather to create initial or preliminary conceptualizations and theoretical models, rather than review old models. This type of review often re-quires a more creative collection of data, as the purpose is usually not to cover all articles ever published on the topic but rather to combine perspectives and insights from different fields or research traditions. What type of analysis can be used? The data analysis part of an in-tegrative or critical review is not particularly developed according to a specific standard (Whittemore & Knafl, 2005). However, while there is no strict standard, the general aim of a data analysis in an integrative review is to critically analyze and examine the literature and the main ideas and relationships of an issue. It should be noted that this requires researchers to have advanced skills, such as superior conceptual thinking (MacInnis, 2011) at the same time as being transparent and document the process of analysis. What is a potential contribution from an integrative review? An in-tegrative review method should result in the advancement of knowl-edge and theoretical frameworks, rather than in a simply overview or description of a research area. That is, it should not be descriptive or historical but should preferably generate a new conceptual framework or theory. Although an integrative review can be conducted in a number of ways, researchers are still expected to follow accepted conventions for reporting on how the study was conducted (Torraco, 2005). That is, how the integrative was done and how articles were selected must be transparent. However, a note of caution. While well-conducted integrative reviews can make a valid and strong contribution to its field of research, more often than the opposite, they either lack transparency or true integration of research. Frequently, reviews la-beled as integrative are simply summaries of studies and not truly in-tegrative. 2.2. How to decide on what approach to use While it can be challenging to determine what approach is most appropriate for a specific type of review, the research question and Journal of Business Research 104 - specific purpose of the review always determine the right strategy to use. While the systematic review is perhaps the most accurate and rigorous approach to collect articles, because there is certainty that all relevant data have been covered, this approach requires a narrow re-search question, and it might not be feasible or even suitable for all types of projects. This is where the semi-systematic review can be useful, but this approach is also more problematic and as it has fewer clear steps to follow. While the methodology for systematic reviews is straightforward and follows highly strict rules and standards (Liberati et al., 2009), the semi-systematic review process requires more devel-opment and tailoring to the specific project (Wong et al., 2013). Often, researchers need to develop their own standards and a detailed plan to ensure the appropriate literature is accurately covered to be able to answer their research question and be transparent about the process. However, if done properly, this can be a highly effective way of cov-ering more areas and broader topics than a systematic review can handle. In addition, when it comes to the integrative review, it becomes even more demanding, which puts more responsibility on and requires more skills of the researchers, as there are even fewer standards and guidelines on which to rely for developing a strategy (Torraco, 2005). This leads to the notion that an integrative review approach might not be advisable to use, and if compared to the systematic review, it might not hold the same amount of rigor. However, if successfully conducting a truly integrative review and contributing with a new conceptual model or theory, the reward can be significant (MacInnis, 2011). 3. The process of conducting a literature review Independent of what approach will be used to conduct the literature review, a number of steps that must be taken and decisions made to create a review that meets the requirements for publication (for specific considerations in relationship to each step. See Table 3). In the fol-lowing, the basics steps and important choices involved in conducting a literature review will be suggested and discussed using four phases; (1) designing the review, (2) conducting the review, (3) analysis and (4) writing up the review. This process was developed from practical ex-perience and is a synthesis of and influenced by various standards and guidelines suggested for literature reviews (e.g., Liberati et al., 2009; Tranfield et al., 2003; Wong et al., 2013). 3.1. Phase 1: designing the review The first question that should be asked is why this review should be conducted. Is there really a need for a literature review in this area? If so, what type of literature review would be the most helpful and would make the greatest contribution? Of consideration should also be what audience will most likely be interested in the review when deciding on the topic. This is a relevant question because it determines the like-lihood of the review being published and the impact it will have on the research community. Conducting a literature review is hard work, so the topic must be one that is of interest to both the author and reader. Therefore, it is a good idea to scan the area as a first step to account for other literature reviews that already exist, to assess the number of re-search studies that must be assessed, and to help formulate and clearly define the purpose, scope, and specific research question the review will address. These are important actions because they will help to identify which approach is most appropriate. For example, if the review aims to summarize or evaluate a large field of research or even several research areas, a strict systematic review approach may not be suitable or even possible. Instead, a narrative or integrative review approach would be preferable. In the same way, if the purpose of the review is to in-vestigate and synthesize evidence of the effect of a specific factor, an integrative review is not trustworthy; instead, a systematic review ap-proach should be used. The stated purpose should then guide the rest of the review. Once the research question has been identified and an overall 336 H. Snyder review approach considered, a search strategy for identifying relevant literature must be developed. This includes selecting search terms and appropriate databases and deciding on inclusion and exclusion criteria. Here, a number of important decisions must be made that are crucial and will eventually determine the quality and rigor of the review. Search terms can be words or phrases used to access appropriate arti-cles, books, and reports. These terms should be based on words and concepts that are directly related to the research question. Depending on the aim of the review and the research question, these search terms can be broad or narrow. Importantly, it could be worthwhile to consider including additional limitations. As almost all initial literature searches yield many articles, a strategy is needed to identify which are actually relevant. Inclusion criteria for the review should be guided by the selected research question. Criteria that can be considered and are commonly used are, for example, year of publication, language of the article, type of article (such as conceptual, randomized controlled trail, etc.), and journal. In terms of research quality, deciding on inclusion and exclusion criteria is one of the most important steps when conducting your review. However, important to note is the need to provide reasoning and transparency concerning all choices made; there must be logical and valid motives. This is important, as, independent of the type of ap-proach, the quality of the literature is dependent on, among other as-pects, what literature is included and how it was selected (Tranfield et al., 2003; Wong et al., 2013). Depending on these decisions, a study can end up with very different answers and conclusions to the same research questions. For example, by only selecting some specific jour-nals, years, or even search terms to try to limit your search, you can end up with a very flawed or skewed sample and missing studies that would have been relevant to your case or even contradict other studies. You can also come to the wrong conclusion about gaps in the literature, or perhaps more serious, provide false evidence of a specific effect. A practical approach is to write all decisions down to enable transpar-ency, as the authors must be clear in a way that enables the reader to understand how the literature was identified, analyzed, synthesized, and reported. This should be done carefully and prior to actually con-ducting the review. 3.2. Phase 2: conducting the review After deciding on the purpose, specific research questions, and type of approach, it is time to start conducting the actual review. When conducting the review, a pilot test of the review process and protocol is appropriate. By testing the search terms and inclusion criteria on a smaller sample, the process can be adjusted before performing the main review. It is common to adjust the process a number of times before actually selecting the final sample. Importantly, it should be noted that it is preferred to use two reviewers to select articles to ensure the quality and reliability of the search protocol. The actual selection of the sample can be done in a number of ways, depending on the nature and scope of the specific review. Depending on how many articles are yielded, different approaches will be appro-priate. For example, reviewers may read each piece of literature that appears in the search in full; this is a highly useful, but time-consuming approach. Another option could be to focus on the research method or findings, and a third option is to conduct the review in stages by reading abstracts first and making selections and then reading full-text articles later, before making the final selection. Once this is done and the initial articles (or other relevant literature) have been collected, the texts should be screened in full to ensure they meet the inclusion criteria. As an additional strategy, references in the selected articles can be scanned to identify other articles that may potentially be relevant (however, this is not appropriate when using the systematic review method as this requires a more strict protocol). During this time, the process of in-cluding and excluding specific articles should be documented carefully. Journal of Business Research 104 - 3.3. Phase 3: analysis After conducting the literature review and deciding on a final sample, it is important to consider how the articles will be used to conduct an appropriate analysis. That is, after selecting a final sample, a standardized means of abstracting appropriate information from each article should be used. Data abstracted can be in the form of descriptive information, such as authors, years published, topic, or type of study, or in the form of effects and findings. It can also take the form of con-ceptualizations of a certain idea or theoretical perspective. Importantly, this should be done in concordance with the purpose and research question of the specific review, and the form will vary. In this step, it is important to consider training the reviewers to avoid any differences in coding and abstraction (if there is more than one) and monitoring the data abstraction carefully during the review process to ensure quality and reliability. Often, if the aim is to publish in an academic journal, this will require a detailed description of the process or a measure of reliability between reviewers. Sometimes this is easy, if the information of interest is, for example, population, effect size, or sample size. However, it becomes harder when the information of interest is themes in the literature, perspectives, or providing an historical timeline. Depending on the review, different analysis methods can be used and are more or less appropriate (please see above for different con-tributions from different approaches). Nevertheless, independent of the method of analysis, it is important to ensure that it is appropriate to answer the selected research question. For example, if the purpose is to evaluate evidence of the effect of loyalty programs, the use of a meta-analysis is most appropriate. On the other hand, if the purpose was to develop a theoretical model or framework for customer experience, a strict meta-analysis would be a poor choice; rather, an analysis tech-nique suitable for integrative reviews should be used. 3.4. Phase 4: writing the review First, when writing the review, the motivation and need for the review must be clearly communicated. Depending on the approach, the final review article can be structured in different ways, and it will re-quire different types of information and different levels of detail. A number of standards and guidelines explicitly address how literature reviews should be reported and structured, including PRISMA, devel-oped for systematic literature reviews and meta-analyses (see Liberati et al., 2009); RAMSES, developed for systematic narrative reviews (see Wong et al., 2013); and guidelines for integrative reviews (Torraco, 2005). Although review articles can be organized in various ways, some generalizations can be made. All authors are expected to follow ac-cepted conventions for reporting on how the study was undertaken. It is necessary to describe transparently the process of designing the review and the method for collecting literature, that is, how the literature was identified, analyzed, synthesized, and reported by the author. Doing so properly gives the reader the chance to assess the quality and trust-worthiness of the findings. The contribution of the specific literature review can take a number of forms, and it should be judged in re-lationship to the field to which it wants to contribute. Depending on a number of factors, such as the maturity of the field or state of knowl-edge, different contributions could be valuable. For example, literature reviews can result in a historical analysis of the development within a research field (e.g. Carlborg, Kindström, & Kowalkowski, 2014), an agenda for further research (e.g., McColl-Kennedy et al., 2017), a conceptual model or categorization (e.g., Snyder et al., 2016; Witell et al., 2016), or evidence of an effect (e.g., Verlegh & Steenkamp, 1999). 4. Assessing the quality of a literature review Literature reviews need to be assessed and evaluated as strictly as empirical articles, but is this always the case? Palmatier et al. (2018) 337 H. Snyder Table 4 Guidelines to assess the quality of a literature review. Phase 1: design In relationship to the overall research field, is this literature review needed and does it make a substantial, practical, or theoretical contribution? Are the motivation, the purpose, and the research question(s) clearly stated and motivated? Does the review account for the previous literature review and other relevant literature? Is the approach/methodology for the literature review clearly stated? Is this the most appropriate approach to address the research problem? Are the methodology and the search strategy clearly and transparently described and motivated (including search terms, databases used, and explicit inclusion and exclusion criteria)? Phase 2: conduct Is the search process appropriate for this type of review? Is the practical search process accurately described and accounted for? Is the process of the inclusion and exclusion of articles transparent? Have proper measures been taken to ensure research quality? Can it be trusted that the final sample is appropriate and in concordance with the overall purpose of the review? Phase 3: data abstraction and analysis Is the data abstracted from the article appropriate in concordance with the overall purpose of the review? Is the process for abstracting data accurately described? Have proper measures been taken to ensure quality data abstraction? Is the chosen data analysis technique appropriate in relation to the overall research question and the data abstracted? Is the analysis process properly described and transparent? Phase 4: structuring and writing the review Is the review article organized coherently in relation to the overall approach and research question? Is the overall method of conducting the literature review sufficiently described? Can the study be replicated? Is the result of the review reported in an appropriate and clear way? Does the article synthesize the findings of the literature review into a clear and valuable contribution to the topic? Are questions or directions for further research included? Are the results from the review useable? suggest that a quality literature review must have both depth and rigor, that is, it needs to demonstrate an appropriate strategy for selecting articles and capturing data and insights and to offer something beyond a recitation of previous research. In addition, they state that a quality literature review needs to be replicable, that is, the method must be described such that an external reader could replicate the study and reach similar findings. Lastly, they state that a literature review must be useful for scholars and practitioners. However, evaluating different types of literature reviews can be challenging. Therefore, some guide-lines for eventuating literature review articles across approaches are suggested as a starting point to help editors, reviewers, authors, and readers evaluating literature reviews (summarized in Table 4). These depart from the different stages of conducting a literature review and should be broad enough to encompass most types of literature reviews. However, of importance is that when evaluating an individual review, specific standards for the type of review must be examined to assess whether the review meets the criteria for rigor and depth. Depending on if, it is a systematic, semi-systematic or integrative review, different standards can be valid. However, independent of type of review, pay close attention to what studies have been included and for what reasons as these decisions make all the differences in terms of what type of conclusions the authors reach. Ignoring a relevant field of research, some journals or years can have major consequences for the results and conclusions of the studies. In addition, its contribution should always be evaluated against the topic or field to which it adds. What constitutes a useful contribution in one area may be insufficient in another. 5. How to get your literature review published While there are many arguments in favor for conducting a literature review, publishing it can be challenging. Several common mistakes are Journal of Business Research 104 - made by researchers when conducting a literature review that can hinder it from getting anywhere near publication in a decent journal. First, researchers often fail to describe in enough detail how the lit-erature review was conducted, which makes it impossible to evaluate both the quality of the review and its contribution. These reviews often fail to provide details of the overall research strategy, the selection and exclusion of articles, the limitations of the search method, and the quality of the search process, and they often lack details on how the analysis was conducted. Second, in the eagerness to pare the sample size down to make the review easier to handle, it is also common to limit the search too greatly. This can be done by only including a limited number of journals and a narrow year span or excluding articles from related fields that could have been relevant for the specific review. Limiting the sample too greatly is a warning flag, as it affects both the depth and rigor of the review, and it can have serious effects on its results and contributions. A better way to handle too many samples is to re-consider and narrow down the research question. Of course, some-times it can be perfectly fine to limit the sample in different ways, but good reasons for doing so must be provided. Third, frequently, re-searchers who have conducted a review often fail to present and explain the results of the review clearly. Often, a large number of different graphs, tables, and figures is included, but not remarked on or ex-plained. This makes it challenging to understand what they really mean or what was actually found. It is common to spend much time ex-plaining the method and the specific analytical technique, but to spend less time discussing and explaining what was actually found and what these results mean. This is true for both advanced quantitative and qualitative analyses. Failing to put the results in context makes it pro-blematic to judge the contributions of the article. Lastly, perhaps the most common mistake is that literature reviews often fail to provide a truly valuable contribution to the field. No matter how excellent and rigorous the review article, if it does not provide enough of a contribution, something that is new, it will not be pub-lished. Too often, literature reviews are simply descriptive summaries of research conducted between certain years, describing such in-formation as the number of articles published, topics covered, citations analyzed, authors represented, and perhaps methods used, without conducting any deeper analysis. While there are likely times when this can be valuable, this is not usually the case and they are not likely to be published in any journal. In truth, review articles that are a medley of word clouds and citation analyses are highly unlikely to be published. This is a pity, as these researchers have gone through the tedious work of collecting many articles without actually analyzing them in any meaningful way, and because of this, they fail to make a significant contribution. Yet, there are ways of moving beyond simply summarizing the lit-erature and truly developing something that is new and valuable and create a substantial contribution to the field in question. First, there is of course a need to use a good research methodology that fills the quality criteria for conducting literature reviews, but features or ana-lyses can also be added to make the review paper more likely to stand out. There are many examples of articles that have been successfully published in higher-ranked business journals using a literature review strategy as a basis. Not accounting for the quality of the review itself, there seems to be a number of ways forward. One such way is to con-duct a literature review and combine it with a meta-analysis of a re-levant topic to provide some evidence of effect. This strategy has been used effectively in articles published in higher-ranked journals (e.g., Carrillat et al., 2018; Edeling & Himme, 2018; Verlegh & Steenkamp, 1999). Important to note is that simply conducting a meta-analysis does not warrant publication; it must also focus on a topic that is relevant, is interesting, and solves some type of research dilemma, thereby ad-vancing the knowledge in the field. In addition, reviews that build on computer-based text analysis and machine learning have been receiving increased interest in business research (e.g., Antons & Breidbach, 2018; Witell et al., 2016). While text analysis may well be an excellent way to 338 H. Snyder contribute, simply using these types of techniques is not enough if it is not done properly and with a purpose in mind. Often, text analysis approaches end up being highly descriptive, only providing an over-view of topics, themes, or networks and not generating any deeper analysis. As an alternative, a valuable output from these types of ana-lyses can for example creating a timeline for analyzing and predicting where a field is heading, a comparison of different related terms or constructs that can serve as a base for theory development or identi-fying true knowledge gaps in previous research. Although rare, still highly desirable is a well-executed literature review that provides a new theory or includes a well-grounded sub-stantial research agenda or propositions on which other researchers can build to advance the field (e.g., Boyd & Solarino, 2016; Mazumdar et al., 2005; Rodell, Breitsohl, Schröder, & Keating, 2016). While this type of analysis is often time-consuming and requires strong analytical skills from the researchers, if successful, it can make a great contribu-tion to the specific field of research. Note that the examples above are only a few of many pathways to making a contribution using literature review as a research method. However, it is very challenging to try to create a contribution when you have already collected the data (published material) and try to turn this into a publishable article. Therefore, it is important to have a particular research question in mind from the beginning and to ensure the right approach is chosen to solve the research problem at hand. 6. Summary and conclusion Literature reviews play an important role as a foundation for all types of research. They can serve as a basis for knowledge development, create guidelines for policy and practice, provide evidence of an effect, and, if well conducted, have the capacity to engender new ideas and directions for a particular field. As such, they serve as the grounds for future research and theory. However, both conducting a literature re-view and evaluating its quality can be challenging, which is why this paper offers some simple guidelines on how to conduct better, more rigorous literature reviews and, in the long run, simply better research. If there is certainty that the research is built on great accuracy, it will be much easier to identify actual research gaps instead of simply con-ducting the same research repeatedly, to develop better and more precise hypotheses and research questions, and, therefore, to increase the quality of research as a community. References Antons, D., & Breidbach, C. F. (2018). Big data, big insights? Advancing service in-novation and design with machine learning. Journal of Service Research, 21, 17–39. https://doi.org/10.1177/-. Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews. Review of General Psychology, 1, 311–320. https://doi.org/10.1037/-. Borman, G. D., & Dowling, N. M. (2008). Teacher attrition and retention: A meta-analytic and narrative review of the research. Review of Educational Research, 78, 367–409. https://doi.org/10.3102/-. Boyd, B. K., & Solarino, A. M. (2016). Ownership of corporations: A review, synthesis, and research agenda. Journal of Management, 42,-. https://doi.org/10.1177/-. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101. https://doi.org/10.1191/-qp063oa. Carlborg, P., Kindström, D., & Kowalkowski, C. (2014). The evolution of service in-novation research: A critical review and synthesis. The Service Industries Journal, 34(5), 373–398. https://doi.org/10.1080/-. Carrillat, F. A., Legoux, R., & Hadida, A. L. (2018). Debates and assumptions about motion picture performance: A meta-analysis. Journal of the Academy of Marketing Science, 46, 273–299. https://doi.org/10.1007/s-. Chang, W., & Taylor, S. A. (2016). The effectiveness of customer participation in new product development: A meta-analysis. Journal of Marketing, 80, 47–64. https://doi. org/10.1509/jm.14.0057. Journal of Business Research 104 - Covington, M. V. (2000). Goal theory, motivation, and school achievement: An in-tegrative review. Annual Review of Psychology, 51, 171–200. https://doi.org/10. 1146/annurev.psych.51.1.171. Davis, J., Mengersen, K., Bennett, S., & Mazerolle, L. (2014). Viewing systematic reviews and meta-analysis in social research through different lenses. SpringerPlus, 3, 511. https://doi.org/10.1186/-. DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177–188. https://doi.org/10.1016/-. Edeling, A., & Himme, A. (2018). When does market share matter? New empirical gen-eralizations from a meta-analysis of the market share–performance relationship. Journal of Marketing, 82, 1–24. https://doi.org/10.1509/jm.16.0250. Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3–8. https://doi.org/10.2307/-. Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26, 91–108. https://doi.org/10.1111/j-.x. Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly, 82, 581–629. https://doi.org/10.1111/j-X-.x. Gross, J. J. (1998). The emerging field of emotion regulation: An integrative review. Review of General Psychology, 2, 271–299. https://doi.org/10.1037/-. 271. Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., ... Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and ela-boration. Annals of Internal Medicine, 151, W–65. https://doi.org/10.7326/-. MacInnis, D. J. (2011). A framework for conceptual contributions in marketing. Journal of Marketing, 75, 136–154. https://doi.org/10.1509/jmkg.75.4.136. Mazumdar, T., Raj, S. P., & Sinha, I. (2005). Reference price research: Review and pro-positions. Journal of Marketing, 69, 84–102. https://doi.org/10.1509/jmkg-. McColl-Kennedy, J. R., Snyder, H., Elg, M., Witell, L., Helkkula, A., Hogan, S. J., & Anderson, L. (2017). The changing role of the health care customer: Review, synth-esis and research agenda. Journal of Service Management, 28. Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151, 264–269. https://doi.org/10.7326/-. Palmatier, R. W., Houston, M. B., & Hulland, J. (2018). Review articles: Purpose, process, and structure. Journal of the Academy of Marketing Science, 46, 1–5. https://doi.org/ 10.1007/s-. Rodell, J. B., Breitsohl, H., Schröder, M., & Keating, D. J. (2016). Employee volunteering: A review and framework for future research. Journal of Management, 42, 55–84. https://doi.org/10.1177/-. Snyder, H., Witell, L., Gustafsson, A., Fombelle, P., & Kristensson, P. (2016). Identifying categories of service innovation: A review and synthesis of the literature. Journal of Business Research, 69,-. https://doi.org/10.1016/j.jbusres-. Torraco, R. J. (2005). Writing integrative literature reviews: Guidelines and examples. Human Resource Development Review, 4, 356–367. https://doi.org/10.1177/-. Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14, 207–222. https://doi.org/10.1111/-. Verlegh, P. W. J., & Steenkamp, J.-B. E. M. (1999). A review and meta-analysis of country-of-origin research. Journal of Economic Psychology, 20, 521–546. https://doi.org/10. 1016/S-. Ward, V., House, A., & Hamer, S. (2009). Developing a framework for transferring knowledge into action: A thematic analysis of the literature. Journal of Health Services Research and Policy, 14, 156–164. https://doi.org/10.1258/jhsrp-. Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly, 26, 3. Whittemore, R., & Knafl, K. (2005). The integrative review: Updated methodology. Journal of Advanced Nursing, 52, 546–553. https://doi.org/10.1111/j-. -.x. Witell, L., Snyder, H., Gustafsson, A., Fombelle, P., & Kristensson, P. (2016). Defining service innovation: A review and synthesis. Journal of Business Research, 69,-. Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013). RAMESES publication standards: Meta-narrative reviews. BMC Medicine, 11, 20. https://doi. org/10.1186/-. Hannah Snyder is an assistant professor at the department of marketing, BI - Norwegian School of Business, Oslo, Norway. Her research interest relates to service innovation, customer creativity, deviant customer behavior, and value co-creation as well as a special interest in literature review methodology. She has published in the Journal of Business Research, European Journal of Marketing, Journal of Service Management and International Journal of Nursing Studies. 339