How do we Determine Content Boundaries in Systematic Review Studies of Management Research?

Literature reviews have played a considerable role in management research over the past quarter of a century. Conventionally employed to summarize the state of the science, literature reviews are increasingly utilized to integrate, refine, and elaborate theory.^[1–6] As scientific disciplines increasingly diversify, literature reviews also provide an antidote to proliferation and balkanization by bridging diverse theories and approaches that address shared questions.^[2]

Critical components of systematic literature reviews include a structured execution of the review and a high degree of transparency in the review methods applied.^[7] Systematic literature reviews are anticipated to indicate in a detailed manner the stages taken to arrive at the sample of reviewed studies.^[8–11] This study refers to this process as a content boundary in systematic literature reviews.

In particular, in the field of management research, insights into how researchers can choose search keywords and what challenges there are to avoid remain scarce. A point that such insights would be desirable is that there is no consistency in many review articles of management research for procedures in keyword selection.^[12–14] An important question that needs answering in systematic literature review studies includes the choice of establishing a keyword search in databases. Past methodology-focused studies about systematic reviews in the field of management provide suggestions on different inquiries. For example, Research^[7] suggested sample selection criteria in systematic literature review studies in the field of management. A study^[2] developed a framework for “systematicity” in organizational research literature reviews. Scholars^[15] provided new insight into coding practices in qualitative review studies. A current guideline about the treatment of gray literature in the identification of keywords has been provided.^[8] Resaearch^[11] also suggested that a structured search needs to become based on the identification of keywords and search terms in a systematic review. Yet, none of such studies provide a perfect guideline for the identification of search keywords.

Therefore, review applications in the field of management and organizational behaviors have been pursued to be tailored, arbitrary, and conflict in the extant literature. Hence, whilst review studies in management seem to become more systematic, they do so in an unsystematic structure, and lack common methodological rules and approaches,^[16,17] which results in an inherent inquiry: Could review researchers accomplish the main tenets of systematic literature reviews through carefully choosing a set of search keywords applicable to their study topics and purposes? For this purpose, we propose content boundary, referring to that how to search keywords representing the study topics have been selected in systematic literature reviews. Do these keywords represent the main topic? Drawing on the search keywords, another question that needs answering is how review authors decide whether the articles extracted are related to the topic. We intend to clarify these two questions and present viewpoints into necessary selections in this phase of review studies, with a specific concentration on the management field. We also aim to shed light on how systematicity is warranted and the concerns on reliability and validity issues in this step. For example, though many review studies have been conducted in the field of strategic management, there is no consistency in the definition of the content of SM. In their review studies on the evolution of strategic management, scholars^[18] used main keywords (i.e., ‘strategic planning’, ‘competition’, ‘resource-based view’, ‘resource-based theory), but Furrer et al.^[19] adopted different keywords, including 26 major keywords (i.e., alliances, joint ventures, competencies, corporate strategy, decision making, diversification strategy, strategic mission, vision, top management team, etc.,). However, Ferreira et al.^[20] searched articles by using two keywords ‘strategic management and ‘strategic decision’ in either the title, keywords, or the article. These examples confirm that there is no systematicity in the content boundary of systematic literature review articles in the field of management.

Our work addresses this gap and aims to develop a framework for the identification, searching, and confirmation of content boundaries and presents viewpoints on necessary preferences in this stage of systematic reviews. To achieve these purposes, we critically reviewed review studies published in the two highly prestigious journals particularly dedicated to systematic reviews in the field of management, the Academy of Management Annals (AMA), and the International Journal of Management Reviews (IJMR). In addition to this, we consider the Journal of Management (JOM) due to its reputation for publishing high-quality systematic review studies.^[2,7]

A systematic literature review is a necessary piece of scientific research endeavors. It plays a crucial role in establishing the foundation for advancing knowledge, facilitating theory development, closing mature study fields, and unearthing new study domains.^[21] Snyder^[3] defined systematic review as a research method and procedure to identify and critically evaluate related studies and to gather and examine data from the previous literature. Systematic reviews employ a pre-defined and planned procedure that necessitates the use of rigorous methodologies to guarantee that outcomes are both reliable and helpful to end users.^[22] A systematic literature review is also considered a “knowledge map”, which examines and synthesizes previous research. A study^[23] suggested three general types of systematic literature reviews, including field, methodology, and conceptual reviews, while Paul et al.’s^[24] work proposed seven types of systematic literature reviews, including meta-analytical reviews, meta-systematic reviews, thematic reviews, framework reviews, bibliometric reviews, conceptual reviews, and hybrid reviews.

The literature review places a center stage and so provides the main contribution of research. It generally provides a detailed retrospective knowledge of a research area, presenting readers with a bird’s eye view of the state of the knowledge in the domain along with its evolution and possible further directions.^[25] The main aim of a systematic literature review is not to generate a piece of novel information but to strengthen extant knowledge of the relevant research area to enable and encourage the development of novel knowledge for a particular domain.^[3] By utilizing explicit and systematic approaches in the review process, bias could be mitigated, so presenting reliable results from which conclusions can be reached and recommendations provided.

Two fundamental methods are appearing in systematic literature reviews: adopting review protocols and review procedures. Whereas review procedures provide the steps or categories in a systematic review that could become like instructions or inquiries, the former covers the stages of review procedures and review decisions and consequences, which should be displayed and presented. At this point, review protocols inherently could provide greater clarified output and breadth than their counterpart. In addition, contrary to review procedures which are naturally portrayed only, review protocols, like Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)^[26] and Scientific Procedures and Rationales for Systematic Literature Reviews (SPAR-4-SLR) Paul et al.^[24] could be presented visibly that increases the transparency of review process and decisions. The PRISMA includes a 27-item checklist and a four-stage flow diagram. The purpose of the PRISMA is to guide review researchers to develop the reporting of systematic reviews and meta-analyses.^[26]

The SPAR-4-SLR protocol also presents applicable recommendations that could aid review researchers to explain the rationale and the need for a review study,^[24] thereby consolidating the robustness of the methods chosen and fortifying the systematicity of review studies.^[25]

Furthermore, several review processes could have been improved and conceptualized for particular review attempts, for instance, the 14-stage MAK approach in bibliometric analysis^[27] and the five-phase model and prevalent procedures in the meta-analysis by.^[28] These additional processes provide applicable guidelines to the scholars by offering analysis-focused considerations that have not been specifically presented in the above-mentioned protocols.

On the other hand, Tranfield et al.^[11] proposed stages of a systematic review: planning the review, conducting the review, and reporting and dissemination. The authors suggested that in the planning stage, review researchers should first identify why there is a need for such a review, followed by writing a proposal for the identified review. Next, a review protocol needs to be developed according to the model. In the second stage (i.e., conducting the review), researchers need to identify how they will select, extract, and use relevant studies. The last stage focuses on presenting findings and providing theoretical and practical recommendations based on scientific evidence.

Grant and Booth^[29] examined fourteen review types and offered their methodologies, definitions, perceived strengths and weaknesses, and a reference to a selected example. Additionally, they argued that a collection of distinct, logical, and incompatible review kinds should be universally agreed upon. Moreover, a total of 48 unique review types were also found by Sutton et al.,^[30] and they were grouped into seven major review “families”: traditional reviews, systematic reviews, reviews of other reviews, rapid reviews, qualitative reviews, mixed-method reviews, and reviews with a special goal. The authors recommended that review researchers should distinguish clearly between review kinds within the same family by using consistent terminology while referencing review categories and review families.

Using practical experiences as well as synthesizing several standards and guidelines for a good literature review, Snyder^[3] also suggested a four-step process approach to generate a review, which satisfies the requirements for a scientific viewpoint: (1) design the review, (2) conduct the review, (3) analysis in the review and (4) structuring and writing the review. Moreover, Munn et al.^[22] offered the following key indicators of a systematic literature review: (1) uncover the international evidence, (2) confirm current practice/ address any variation/ identify new practices, (3) identify and inform areas for future research, (4) identify and investigate conflicting results, and (5) produce statements to guide decision-making.

Establishing a systematic literature review is not an easy task, thus researchers must choose a topic that can inspire both author(s) themselves and the reader(s). The first stage is therefore to scan the research domain to add an extra mile into the current review studies, to evaluate the number of investigations that need to be evaluated, and to help design and openly describe the objective, breath, and particular research questions the review will answer. These are crucial steps as they could help to clarify which method is most useful.^[3] After these points have been addressed and the review methods are acknowledged, the authors need to clarify the search protocol for the systematic reviews, consisting of search keywords, databases, and inclusion and exclusion standards.^[25] The second stage in a systematic literature review is conducting the actual review. In this phase, Snyder^[3] suggested a pilot test of the review process and protocol so that an author could control and adjust (if necessary) search terms and inclusion criteria before starting the main review. To warrant reliability and validity issues in the search protocol, two researchers could check the content fit of the articles selected. Sample selection must be clarified by review researchers in this step. Once completing the literature review and determining the final dataset, the third stage is to acknowledge how the documents will be utilized to establish a suitable examination.^[3] Review authors must clarify the following questions: what type of knowledge should be extracted to meet the research objective? What kind of knowledge is required to establish the particular analysis? How can reviewers’ knowledge be improved to warrant the clarity of this procedure? How can this stage be presented?^[25]

The last stage in a systematic literature review is writing the review. In this phase, the review researchers need to clarify the motivation and need for such a review study.^[3] Standards that can be useful for the specific review need to be decided in this stage to report and present the literature reviews. Authors also need to decide on the information that will be included in the review. Transparency is another important point that needs to be addressed so that readers can better evaluate the quality of the review paper. Thus, authors must report and present all the stages adopted throughout the review process transparently. In a similar, the results found must be transparently presented and discussed by the authors. Finally, potential contributions and future directions for further research should be discussed in review articles.^[3,25]

Overall, the existing literature provides methodological insights into systematic literature reviews. For example,^[11] claimed that a well-designed search needs to be drawn on an openly identified study inquiry to be examined by a systematic review, the identification of content boundary that is constructed from the scoping research, the literature review, and debates among the author(s). Research^[31] also recommended that systematic review studies need to be well-designed. Simply stated, they need to be developed in an ordered or methodical aspect instead of an unplanned structure. For content boundaries, this implies that all the processes conducted should be well structured, clarified, and not casual. Recent studies have attempted to offer remedies for well-structured and transparent review studies. Hiebl’s study,^[7] for instance, provides a framework including stages and expected attributes of sample selection in systematic literature reviews. The author proposed a three-step sample selection process: identification, screening, and disclosure of the review sample. He suggested principal search methods that result from a concentration on particular journals, databases, or pioneering studies. Simsek et al.^[2] work develops a model that represents systematicity as a constituting tendency for the implication of explicit approaches in systematic literature reviews. The framework developed offers three main phases (i.e., principles, practices, and promises) as well as their ingredients (for principles: transparency, completeness, saturation, connectedness, universalism, coherence; for practices: envisioning, explicating, executing, evaluating, encoding, elaborating, expositing; for promises: richness, reproducibility, trustworthiness, utility). Particularly, in the “executing” principle of systematicity approach, the authors suggested that search keywords need to have resulted directly from the research questions. Although these studies provide meaningful contributions to the review literature, they mostly provide general methodology-focused guidance on systematic reviews in the business and management field and have not completely explained the content boundary process of systematic literature reviews. Considering this background, we aim to minimize the knowledge gap within the literature by acknowledging the content boundary as a unique three-stage process: identification, searching, and confirmation of content boundaries. The following section first explains the phenomenon of “content boundary” and its referred three components which can significantly influence the structure, transparency, and comprehensiveness of the review studies and allow the readers and referees of the studies to pursue and comprehend better the data collection and analysis processes.

The content boundary is related to how to identify the limits of the study topics in systematic literature reviews. Once research questions have been identified, the study field must be cleared up by the researchers, who define content boundaries. Our primary intention is that this concept be integrated into the systematic literature review process to clarify the scope of review studies. Even if the scope is chosen correctly, it must still be verified. Otherwise, review articles could provide questionable findings. These are the main questions that need to be answered by review scholars.

We propose the phenomenon as a process, including three dimensions: identification of boundaries, searching for boundaries, and confirmation of boundaries.

Identification of boundaries is the first step that focuses on identifying search keywords. Search keywords must be identified using certain criteria. We highlighted basic remedies in this step. First, authors need to read previously published documents relevant to their topics and study areas. These published materials could offer potential keywords or at least avenues about how to create a pool for search keywords. Once creating a keyword pool by using existing literature knowledge, authors need to confirm the applicability of these terms by consulting experts, or “expert opinion”. Depending on the field and the subject, experts could be chosen from universities and/or industries. Using expert opinions, search keywords could be purified so that they could be searched in the database.

Searching for boundaries is the second stage of the process, which requires determining the database in which the keywords identified will be searched. As most review articles did, authors should use global databases that provide a broader range of documents, such as Web of Science (WoS) and Scopus. However, we have concentrated more on the search process. How did the authors search for these keywords in a database? This is a question that motivates us to clarify this step in detail. First, authors need to identify whether they directly use the keywords in the searching stage. The other blurred picture is related to abbreviations, particularly very well-known ones, such as CEO and TMT. If authors use such well-known abbreviations, it must be explained in the methodology section. If not, they must justify why these abbreviations are excluded because some articles could only use them in the title, abstract, and keywords. In this stage, we also recommend that authors could use the snowball technique by seeking reference lists of the most cited articles to check if there is a missed study.

The third stage is the confirmation of boundaries that aims to endorse whether the documents obtained are relevant to the study topic and scope. We suggest that authors need to highlight how they confirm the suitability of the documents. There are various avenues to check the relevance of the documents, such as coding, looking at titles, abstracts, keywords, or reading full texts. However, there is no prevalent consistency in this process. There is a need for a guideline about how to confirm the credibility of the documents for further analysis. Justification of the process is also helpful for reliability and validity issues. We thus aim to develop a guideline explaining how to confirm selected documents based on the search keywords.

To provide insights into the content boundary as a process of systematic literature review studies in management research, we concentrated on review articles published in AMA, IJMR, and JOM, AMA and IJMR could be acknowledged as the most outstanding and most-cited journals which are completely dedicated to publishing systematic review articles in the field of management. These are among the highly prestigious outlets in the field with the highest impact factors.^[7] Whilst journal metrics have relatively been criticized,^[32–35] these metrics demonstrate that AMA and IJMR are considered highly prestigious journals in the international scientific world. Acknowledging this reputation and these journals’ high status for exactness and value, review studies published in these journals could be supposed to be high quality. JOM was also chosen given that this journal not only has an international reputation with its high impact factor in the scholarly community but has also published annual literature review issues. Overall, a scientific examination of review articles published in these three journals enables insights into the content boundary process of well-published review studies in the field of management.

We are looking for literature review articles including systematic literature reviews, integrative reviews, and bibliometric studies. We constrained our breadth to the time frame between 2019 and 2022 since the academic understanding of enhancing review rigor and relevance has improved in recent years. We have included 65 articles from IJMR, 56 articles from AMA, and 45 articles from JOM. In terms of the period, 31% of articles were published in 2019; 24% were published in 2020; 27% were published in 2021, and 14% were published in 2022. This statistic exhibits the growing number of review articles published in these three leading management journals.

To examine review articles, we generated a coding book (that can be provided upon the requests of editors and/or reviewers) to tabularize the existence or deficiency of content boundary practices, consonant with the three dimensions explained in the definition of the phenomenon. The review articles included have been coded along three dimensions. The first dimension draws on the identification of content boundaries. These codes involve if the article considers previous publications, extant literature, or other sources to create and identify search keywords, the number of keywords initially chosen, and the application of expert opinions. The second is based on searching for content boundaries. These codes consist of how to review articles search keywords identified on the selected databases or utilization of the snowball technique to eliminate any missing research by seeking the most cited articles’ reference lists. For example, some studies use well-known abbreviations in the search process, while others constrained their search keywords to main concepts, such as strategic management. These codes could provide a mix of search categories. The codings are then utilized to provide insights on the confirmation of boundaries of the review studies included. Wherever pieces of evidence or absences in these dimensions can be detected, they will be provided in the findings section. As an example, one of the assessment criteria coded is related to if a review study rigorously justified the suitability and relevance of documents to the study topic and purpose. We know that the deficiency of a coded item in a review article included does not undoubtedly mean the process was not acknowledged or implemented. In addition, it needs to be considered that we couldn’t directly check and code the extent to which current review articles obey or fail to obey the criteria clarified in our conceptual definition since they provide a wider, imperceptible understanding. However, we intended to deduce the existence of these standards by examining patterns of particular approaches adopted.

To ensure the reliability and validity of our coding process, an author read the articles in two different periods: first in March 2022 and then in May 2022. In addition, in June 2022, the second author who has an experience in systematic literature review will code all documents separately. We then compare differences between two coders to achieve an acceptable level of intercoder agreement. As a result, data was prepared with 100% consensus.

The main choice in the identification of content boundaries is the query of if the content boundary adopted is presented as a separate subheading. Only two of the studied review articles report the content boundary as a separate sub-title in their methodologies.^[36,37] Ignoring the presentation of content boundaries adopted by review articles may arouse readers suspicious and so influence the results of systematic reviews. Therefore, if the content boundary is adopted, disclosure of the identification of search keywords is required for a systematic review to be transparent Figure 1.

Figure 1:
Dimensions and expected practices of content boundary in a systematic literature review.

Although they do not disclose it as a separate sub-title, 119 (71.69%) of review articles mention the content boundary adopted in the methodology section. One-hundred sixty (96.39%) articles do disclose that they have conducted keyword-based sample selection, which we categorized into 12 groups (see Table 1). Among those categories, 142 (85.54%) do not provide a detail on how keywords were selected, which restricts the structured nature and transparency of those systematic reviews. 17 (10.24%) review articles disclose that search keywords used results from the previous literature reviews, illustrated as category 4 in Table 1. These studies intend to use relevant search keywords from the previous review studies published in leading management journals. As claimed by previous studies, this approach is specifically useful for the identification of search keywords.

Even though most articles do not disclose how search keywords were selected, several further reasons may offer important insights to justify the keyword selection process. For example, five articles have conducted background reading for justifying keyword selection.^[38] The authors created a list of keywords by consolidating the topics covered by the principal textbooks from the last 45 years. Some articles adopted various approaches to identify search keywords, such as discussion with authors, reviewing recent literature,^[39] and using popular definitions of relevant constructs.^[40] It seems that there is no consistent approach to identifying search keywords, at least review researchers do not properly disclose the identification process which contradicts the above-mentioned objectives that all procedures in a systematic review need to be well-structured and transparent. Researchers could benefit from technological development in this process. Text mining could offer great potential for identifying search keywords from the relevant literature. Authors could first review relevant articles and then submit them to the analytical software. Without human bias, these tools could generate a keyword pool from the articles that could be applicable in identifying more valid keywords in review articles. To enhance reliability and validity in the keyword selection process, potential keywords that appeared in the text-mining analysis could be checked by an expert panel. Such an approach could provide a well-structured, transparent, and comprehensive keyword selection process.

Even though most review studies do not disclose how the authors select related keywords, several articles conducted various scanning activities, including snowballing/ancestry search and hand searching to maximize content boundary (n=70, 42.17%). For example, some studies adopted ancestry searching for maximizing the content boundary,^[41] while others used the hand-searching strategy.^[42] Some studies in this category have also adopted snowballing search methods, to grasp influential documents from the reference list of reviewed articles. Thus, this finding reveals that the content boundary could be increased by following various approaches so that influential studies cited by the articles involved could also be included in the analysis. Moreover, 37 (22.29%) articles only used precise keywords to maximize content boundary, but most studies (n=129, 77.71%) do not use such keywords. Among those who do not use precise keywords, 77 (46.39%) articles have used asterisks “*” for expanding content boundary in searching keywords on databases, but others do not utilize asterisks which can limit the search results. Therefore, the methods (i.e., snowballing/ancestry search, using asterisks, etc.) used in review articles published in highly prestigious journals should be disclosed in greater detail with higher transparency to more successfully meet the standards in review articles.

The review articles that do not use precise keywords have also been examined to reveal if they use several combinations of keywords. More than half of the studies (n=105, 63.25%) have used abbreviations of keywords selected, such as LMX,^[43] and SME^[44] in the searching process. Surprisingly, only two (1.20%) articles have used synonyms of related keywords to be able to warrant if they can catch all relevant documents from the database.^[45,46] To be able to make the identification of keywords more structured and transparent, authors need to also consider potential synonyms of relevant keywords that allow a comprehensive review of a knowledge domain.

From our analyses, there appeared that 113 (68.07%) review studies indicated how they decide on relevant articles in the content boundary. The scanning of initially identified documents was largely established in the reviewed studies by first independently reviewing documents, coding each item, and screening the title, abstract, and keywords for content fit with the search keywords and study topics.^[39,47] In particular, 25 articles (15.06%) used coding,^[48,49] 14 (8.43%) articles used detailed reading,^[50–52] and 10 (6.02%) articles read the title, abstract, and keywords of the documents appeared within the dataset.^[53–55] In particular, using two or more coders could improve a well-structured search process of content boundaries, and minimizing discrepancies among coders could increase a common comprehension of the coding method in review articles.^[56] Moreover, because the number of documents revealed in scientific databases importantly differentiates among seeking with only the title, abstract, and keywords contrasted with seeks consisting of the full paper, it is expected that further review articles more persistently report their searches of keywords to various pieces of the documents.

Several methodological studies suggest the coding process is an essential part of systematic reviews.^[57–59] It should be clarified by identifying details for the coding process. However, solely 12 (7.23%) review articles in our sample indicated details on related coding books. As illustrated in Table 1, of the 25 coding-driven review articles, 16 (9.64%) reported that this process was conducted individually, and 17 (10.24) indicated how many coders were employed in this process. Although coders’ experience related to the study area is an important phenomenon that review articles must consider,^[60–62] 12 (7.23%) articles do not indicate coders’ experience related to the research content.^[63,64] 10 (6.02%) articles have employed authors as coders,^[65,66] and three (1.81%) articles have used both authors and research assistants.^[67–69] However, these approaches come with the constraint that the documents may not be independently coded by authors and their assistants. Such restricted transparency is specifically apparent when a relatively big volume of documents has been identified via search keywords. Therefore, it would be required to apply external coders within the coding process to keep the reliability of the search of content boundaries. Thus, it seems advisable first to employ experts from the not only scientific world but also the industry to cross-check the selection of the documents made with the researchers’ own and the experts’ backgrounds in the domain. Such a method could probably improve the credibility of document selection. Applying expert opinions due to their knowledge would therefore lead to a more structured and transparent search process in review articles.

As illustrated in Table 1, the consensus level among authors was disclosed in only 7.83% of the analyzed review studies. For the vast majority of articles, the confirmation of the content boundary process was restricted. Using two coders in review documents can have been a reasonable excuse for why review scholars have not reported a consensus level among authors in review articles. However, since the authors are responsible for giving the final decision on the content boundary, these claims are no longer valid. Moreover, 12 (7.23%) studies do not mention if there is consensus among coders or not. Alternatively, consensus levels could be reported by providing achieved inter-rater agreement levels (with percentage) using Cohen’s^[70] Kappa in the methodology section of the review studies that have also been adopted by somewhat respective review studies.^[5,37,67] The disclosure of the consensus among authors could be considered a must for confirmation of content boundaries, ensuring a transparent systematic review. Specifically, in the case of a large volume of revealed potentially relevant documents, confirming their suitability and excluding documents’ non-relevancy by drawing on the consensus among authors can be an effective strategy.

Drawing on the aforementioned findings, we have proposed a framework including three components of content boundary (i.e., identification, searching, and confirmation of content boundaries) as well as three structures (i.e., precise, extended, comprehensive) for each component. Let us first explain what we mean by those structures.

We consider “precise” a stage where review researchers have identified the same keyword for the study topic. For example, if the study topic is strategic management, the author(s) identify only “strategic management” as a search keyword. Second, the author(s) could decide to expand the range of keywords, for example, by reviewing previous literature relevant to the study topic, referring to the “extended” stage. At this stage, authors couldn’t contend with using the same keywords, thus they could seek extant literature to be able to grasp more relevant publications. Finally, using additional remedies, such as authors’ expertise, expert panels, and text-mining could offer a comprehensive keywords pool in the identification of content boundaries. We strongly suggest the author use the last structure to minimize oversight of potentially relevant and important documents in the knowledge field.

In the second component of content boundary (i.e., searching), the same structures could also be applied. Precise searching of content boundaries is focusing only on a single database and ignoring other possibly relevant search terms, which may limit the dataset and consequently study outcomes. An extended search of content boundary is relatively extensive, but not as desired. Author(s) might not only search precise keywords but also supplement relevant terms from current literature identified in the previous component. In the searching process, we propose using a comprehensive structure by searching not only keywords but also relevant synonyms, asterisks, the combination of search keywords (i.e., abbreviations), as well as scanning activities, such as snowballing or ancestry search, and hand searching.

Confirmation of content boundaries has also been categorized based on the above-mentioned structures. Precise structure, the preliminary stage, is supposed to confirm the content fits with the research topic and objectives by reading the title, abstract, or keywords of documents found. This is a limited approach, leading authors to miss potentially important publications that search keywords might not be indicated in their title, abstract, or keywords. To mitigate potential limitations of this structure, review authors could extend their confirmation of content boundaries by using detailed reading of full papers, and/or the thick sieve method. This helps authors cover more relevant documents that could increase the likelihood of the generalizability of study findings. However, author(s) in systematic literature reviews must ensure that all relevant documents have been covered in their datasets. To achieve this, they need to apply comprehensive confirmation of content boundaries. The coding process is one approach that can offer more comprehensive documents to the review researchers. However, authors need to put more emphasis on providing details about the coding process. First, the coding book used must be completely presented along with the manuscripts to be able to crystalize how to decide the suitability of the documents included. Second, review researchers, particularly in the case of multiple authors, should indicate the number of coders employed and their background and experience concerning the study context. Furthermore, our framework has proposed additional remedies to ensure the comprehensiveness of the confirmation process. For instance, expert opinion can be considered to warrant the search results completely fit the study topic and objectives. Experts could be employed from both industry and academia. A review panel can also be conducted to check the suitability of documents for the research objectives. Last but not least, in the confirmation process, we suggest that specifically in the case of multiple authors, they must indicate consensus levels among authors with the percentage. Such a consensus level will demonstrate that the content fit of the documents revealed has been jointly endorsed by the authors, leading to ensuring the reliability of the dataset.

A growing scientific interest in systematic review studies endorses that this type of review is to be the widely accepted method in review studies in the management field. Simultaneously, since study topics and questions, as well as content boundary standards, guide systematic reviews, there has been no one “right” avenue for generating a dataset. Conversely, many ways adopted by review scholars have been adopted in the content boundary process that inherently leads to particular concerns being acknowledged.

The above-presented results indicate that content boundary is not presented in a separate title in the methodology even though many authors have indicated this process. However, the findings show that many authors have not reported how they choose the relevant keywords which is a crucial issue to evaluate the reliability of the search process. In other words, without transparently reporting the selection of keywords, it could be a question for a reader if the author has adequately covered all the relevant publications. Table 1 illustrates that there has been no well-structured process in the content boundary in review studies. For example, no study has used expert opinion or text-miming in the identification of keywords. In addition, review studies have mostly not used some additional implications to increase content boundaries, such as asterisks, synonyms, and combinations of keywords. According to the findings, it is also unclear what and how methods should be used to confirm the content boundary. Thus, further review studies in the management field can be conducted even more structured, transparent, and comprehensive. Our framework proposes a three-stage structure of content boundary. According to the framework, systematic reviews should first stringently align the identification of content boundaries with the study’s main topic and questions. For example, future systematic reviews may need to provide content boundary as a separate sub-heading under methodology to mirror if the search keywords identified are sufficient to reflect the relevant scope and if they can be supplemented with further avenues of identifying related keywords that can better allow review articles to broadly capture study topic and to align better with research questions. Likewise, our findings recommend that future research in the systematic review could need to acknowledge more openly how keywords selected, using previous literature, author experience, expert panel, text-mining, and conducting background reading for analyzing the applicability of search keywords initially selected could be aligned with the study topics and questions and so the structured nature of systematic reviews improved. Our findings suggest that for future researchers of systematic reviews, there are possible options for how the possibility of extensive scope of the related literature could be enhanced – for example, by using expert panels to confirm the applicability of the search keywords or the usage of text-mining approach through analyzing extant literature.

Another suggestion of our study is that a more stringent search can increase the comprehensiveness of future systematic studies in the management field. Possible avenues to enhance such comprehensiveness consist of searching keywords and their synonyms with asterisks for grasping documents from the literature as much as possible and how these keywords were searched (i.e., well-structured searches – title, abstract, keywords, and full-text), and presenting search protocol for making the process more transparent and structured.

Finally, our findings recommend that future scholars in systematic reviews could warrant consensus about the applicability of the documents revealed. Authors should report the consensus level to endorse the fit of the documents found to the research topic and objectives.

In addition to these recommendations, the coding book provided as an online supplement on content boundaries adopted by review articles studied could serve as benchmarks for future researchers wondering how content boundaries can be warranted in a review article or having queries about the protocol on identification, searching, and confirmation of content boundaries included in their systematic reviews. We hope that our paper could play a role as a reference point for these queries, even though it should be addressed again that drawing on research questions, there could be acceptable excuses why a variation from this guideline is undertaken in individual review studies.

Our paper has several limitations. First, we do not claim that systematic review is always a useful way of reviewing relevant literature.^[7] Through scanning journal articles published in the leading three management journals (i.e., IJMR, AMA, and JOM), we realized various review studies take a bright view of theory development rather than synthesizing past studies. For those articles, the theory development is the main focus that could ensure why the content boundary is not focused, which can limit our study results.

Second, an exaggerated focus on content boundary may jeopardize the conceptual side of review articles including original features, like incorporating extant literature, improving novel perspectives, and proposing new frameworks. A severe content boundary is an essential prerequisite for review studies, yet the basic value deriving from systematics reviews is usually the applicability of the perspectives created. Thus, well-designed, transparent, and extensive content boundaries are significant but not adequate and will not essentially lead to a well-systematic review.

Third, the data in our review are limited to the three leading management journals (i.e., IJMR, AMA, JOM). This may restrict our results’ generalizability. Further, due to the subjective and interpretive nature of our research, scholars could use a mixed-method approach to propose alternative frameworks and guidelines of content boundaries for systematic reviews.

The content boundary is a natural piece of review studies and consists of three components: (1) identification of content boundaries, (2) searching of content boundaries, and (3) confirmation of content boundaries. These three components need to be established in a way that warrants the overall content boundary standards leading to well-designed, transparent, and extensive literature reviews. Drawing on examinations of review studies published in three leading management journals (i.e., IJMR, AMA, JOM), we have clarified basic tenets that future researchers in systematic reviews can adopt in their content boundary procedures. Among these tenets, we suggest main principles, such as expert panel and text-mining for identifying content boundaries, namely potential search keywords. Moreover, we have provided useful ways for searching protocol, such as using synonyms, asterisks, and several combinations of keywords including abbreviations. This study also provides a guideline on how to confirm the content fit of the documents revealed with the research topic and objectives. These detailed perspectives about content boundary and the above-indicated implications complement extant methodological advice on establishing review studies in the field of management.^{[2,7,15,58,71]} We invite systematic review researchers to consider content boundary as a separate heading when conducting their review studies so that the methodology could better offer well-designed, transparent, and extensive perspectives in a manner that create valid and applicable directions for the management scholars. We hope that our framework could become applicable to scholars of the management field when crafting their further review studies.