Bmc Medical Research Methodology Open Access a Multidisciplinary Systematic Literature Review on Frailty: Overview of the Methodology Used by the Canadian Initiative on Frailty and Aging

Background: Over the past two decades, there has been a substantial growth in the body of literature on frailty in older persons. However, there is no consensus on its definition or the criteria used to identify frailty. In response to this lack of consensus, the Canadian Initiative on Frailty and Aging carried out a set of systematic reviews of the literature in ten areas of frailty research: biological basis; social basis; prevalence; risk factors; impact; identification; prevention and management; environment and technology; health services; health and social policy. This paper describes the methodology that was developed for the systematic reviews.


Background
While researchers, policy makers, administrators and health care providers generally agree that frailty can have an important impact on affected individuals, their families (particularly those involved in care-giving), the health care system and society as a whole [1], the concept of frailty remains controversial. Despite a substantial increase in the number of articles published on frailty over the past twenty-five years, there is no consensus on its definition or on what criteria should be used to identify older individuals with frailty [2]. Numerous models, definitions, criteria and approaches for its study have been advocated [3][4][5][6][7]. Nevertheless, few publications have systematically reviewed the quality of the evidence on frailty [8].
The Canadian Initiative on Frailty and Aging (CIFA) is an international partnership of researchers and health care providers formed in 2002, with the objectives of seeking to improve our understanding of the causes, trajectory and implications of frailty to thereby improve the lives of older persons at risk of frailty or with frailty by disseminating knowledge on its prevention, detection and treatment as well as the cost-effective organisation of services for those with frailty http://www.frail-fragile.ca/. To this end, the members of CIFA agreed that an essential early task for the group was to summarise the state of research on frailty in older persons in order to develop a working framework and identify research priorities for a program of enquiry.
As a first step, a qualitative review of existing models, definitions, and criteria for frailty was conducted and has been published elsewhere [2]. This work highlighted the need for an integrated interdisciplinary research approach and set the stage for a series of systematic literature reviews on particular aspects of frailty. The goal of these reviews was to collate, critically review and synthesise the current evidence on frailty across disciplines and to identify gaps in existing research.
Ten distinct reviews of pre-specified areas were conducted. tic criteria to identify frailty?; prevention and management of frailty -can interventions aimed at the general population prevent frailty and/or its consequences?; use of environmental adaptations -what technological interventions have been demonstrated to increase quality of life and safety for frail older adults?; organisation of services -what are the integrated models of care delivery for the frail elderly?; and, health and social policy implications -what are the key policy issues in regard to care delivery and/or funding for the frail elderly?. Each area review was led by two to three CIFA investigators (referred to as Question Leaders) -see Table 1 for the list of Question Leaders and detailed research questions.
The initial steps of the review process were completed between 2002 and 2005. The steps included all activities related to the literature searches, the article selection and the quality assessment. The results of the specific reviews are being published separately. Apart from the substantive results, an important part of the review process was the development of a methodology that would facilitate the conduct of systematic reviews of literature addressing multidisciplinary research questions and the summary of the literature on a concept lacking a widely accepted definition. This paper presents the methodology used for these systematic reviews and discusses some of the challenges faced as a result of the inconsistencies in the definition for frailty and the heterogeneity of research methods across the disciplines included.

Central Coordination Group
The Central Coordination Group (CCG) was based in Montreal, Quebec and was composed of CIFA investigators, with expertise in frailty and/or in research methodology, as well as staff hired for specific duties. The role of the CCG was to develop the methods that would be used for the systematic reviews and to assist Question Leaders in applying the methodology. CCG staff executed literature searches, retrieved and blinded abstracts and full articles. The quality assessment of articles was also conducted by the CCG if the Question Leaders did not have the local resources required to conduct these activities.
The systematic reviews were conducted by a multidisciplinary team of investigators with expertise in geriatrics, gerontology, epidemiology, sociology, biology, health services research, occupational therapy and rehabilitation engineering. During the review process, the CCG staff organised face-to-face and teleconference meetings, bringing together the Question Leaders for discussions on the review process and sharing of preliminary results. The primary objective of these meetings was to ensure consistency in the theoretical and methodological approach used for the reviews. The general process for the methodology is outlined in figure 1.

Core Keywords for the Literature Search
As there is no widely accepted definition of frailty, the CIFA review strategies were inclusive and attempted to retrieve and examine all the literature relevant to the concept of frailty, rather than being limited to any single definition or model. Since most of the current definitions of frailty emphasize vulnerability to some adverse outcomes, CIFA reviews included literature on vulnerability [1]. Despite a growing consensus that frailty and disability are distinct concepts, several authors have used the terms synonymously [9]. In order not to miss this literature, the CIFA reviews also included papers on disability. Finally, since frailty has sometimes been described as the "flipside" of successful aging [2,10], this construct was also felt to be relevant to the reviews.
To ensure that all of these concepts were included within the CIFA reviews, the following general frailty-related terms were used as core keywords for all of the literature searches -'aged' combined with any of the following terms: 'frail', 'frailty', 'vulnerable', 'vulnerability', 'successful aging', 'healthy aging', 'disability', or 'disabled persons'. For each Question, these core keywords were combined with review-specific keywords.

Review Process
In the first step of the review, the Question Leaders refined the research questions to be addressed through their systematic review. They then identified review-specific keywords for the literature search, and decided upon the inclusion/exclusion criteria to be used in the selection of articles. The literature searches were conducted using MedLine and AgeLine, using the core frailty-related keywords combined with review-specific keywords. The searches were limited to publications in English or French, original research, and human studies. In accordance with the timing of the CIFA initiative, initial searches were conducted in the summer and fall of 2003 for papers published between 1997 and 2003. In the winter of 2005, the searches were updated to include literature published until the end of December 2004. All searches were overseen by a librarian with training and experience in the health sciences.
Following the literature search, the Question Leaders were sent the abstracts of the papers identified by the question specific search. The names of the authors and journal were removed from the abstracts. The two Question Leaders then independently reviewed each abstract for relevance to their assigned area. For abstracts that were selected as relevant by at least one of the Question Leaders, the full articles were retrieved. The Question Leaders then read through the full article, while remaining blinded for the author(s), abstract, journal, sponsors (if any) and acknowledgement (if any). They selected those studies that met the pre-specified inclusion criteria. Disagreements between Question Leaders were resolved through discussion until a consensus was reached. The references in each of the selected articles were then "pearled" (i.e., reviewed in an effort to find relevant papers not identified in the original searches) [11]. All identified relevant studies then underwent quality assessment and data abstraction. Pertinent editorials and review articles found during the literature search were retained as background papers. Figure 2 presents, as an illustration, the article selection flowchart for the review on the prevalence of frailty.

Quality Assessment and Data Abstraction
All studies included in the review underwent quality assessment to evaluate how well the study design and study conduct reduced the potential for bias [12]. In addition to addressing issues specific to the study design (i.e. cohort; cross-sectional; randomised controlled trial), the quality assessments incorporated the unique characteristics specific to the review question. For the most part, we were unable to find quality assessment and data abstraction tools that fully met the requirements of our review [13]. For this reason, we adapted existing tools or designed new ones [14,15]. Based on discussions between the methodologists and Question Leaders, we developed the items that would be included in the tools for each of the Questions. The scale and format of the tools were then tested and modified based on feedback from the Quality Assessors. Depending upon the study design, our quality assessment examined between eleven and twenty-four items concerning specific methodological features of the study [see Additional file 2].  Canadian Initiative on Frailty and Aging: general process for the systematic review of literature on frailty  * Articles were blinded for author(s), abstract, journal, sponsors and acknowledgements.
Step 3. Blinded articles* of abstracts selected in Step 2 are retrieved by the CCG and read by the 2 QLs for inclusion.
In the case of disagreement between the two readers, a consensus opinion is reached. QLs pearl the reference lists of selected articles to identify additional potentially relevant articles. Articles are classified as editorials or review articles, original studies to be included in the quality assessment process, and those to be excluded.
Step 4. Pearled articles* from Step 3, above, are read by the 2 QLs for their inclusion as review articles, editorials, background articles or as original studies to be forwarded for quality assessment.
Step 5. Quality assessment and data abstraction is done for the blinded articles* of all selected original studies.
CCG staff could execute the search and provide the blinded abstracts to the QLs.
CCG staff could provide the articles retrieved from blinded abstracts to the QLs.
CCG staff could provide the pearled articles to the QLs.
CCG staff could carry out the quality assessment and data abstraction.
nal validity, external validity, or reliability). For every tool, each item as well as the overall quality was rated on a scale of one to four (with four representing the highest quality). A global subjective ranking was also assigned by the Quality Assessors, to reflect the overall quality of the article considering the individual items that were rated. The assessors also indicated whether they felt the study was relevant to the systematic review. The data abstraction form used in the reviews was specific to the study design used, and included one section on the design features and another on results that were pertinent for the review.
The quality assessments and data abstractions were conducted by graduate students in epidemiology or a related discipline. Prior to being hired, the assessors were pre-tested to ensure that they had sufficient knowledge of epidemiological concepts. Once hired, they received a full day of training on the application of the CIFA quality assessment and data abstraction forms. A total of 16 assessors were involved in the quality assessments conducted by the CCG. Two assessors were randomly assigned to each article reviewed. They completed the assessment forms independently and then met to arrive at a consensus rating of individual items as well as on the overall methodological quality of the article. Data abstraction was completed by one of the two assessors.

Classification
The authors of the included studies differed widely on what they meant by terms like frailty, disability, and vul-Triage of articles for the systematic literature review on the prevalence of frailty nerability. For example, some defined frailty as being present when there were limitations in activities of daily living [16,17] or when participants were institutionalised [18,19]. Others required that distinct signs and symptoms be present in order to identify the presence of frailty [20].
After reviewing the first set of articles retrieved in our literature searches, we found that most author definitions for frailty fell into one of five broad categories: vulnerabilityphysiological syndrome; vulnerability -complex syndrome; vulnerability -geriatric syndrome; disability; and, setting/source of recruitment. To more clearly describe the studies included and permit subgroup analyses, we classified all of the studies reviewed based on the definition of frailty offered by the authors in the methods section of their article. Articles that did not fit into one of these categories were classified as "other" (see Table 2 for the definitions of the categories used).
Two of the authors (H.B. and D.H.), both geriatricianresearchers, classified each article selected for the detailed review. For the first one hundred articles, the classifications were done independently followed by a consensus discussion in the event of discrepant classification. As the agreement was high, for the remaining articles only one of the two (H.B. or D.H.) carried out the classification. In the event of uncertainty, there was consultation that resulted in consensus. Table 3 presents the final number of articles retained for review in each of the systematic reviews conducted, as well as the quality assessments and classification. The reviews on Health Services and Health and Social Policy followed a methodology that did not involve the same type of quality assessment and classification as the others, and so they have not been included in Table 3.

Grades of Evidence
To summarise both the quality of the evidence and the direction of the findings, the Question Leaders used the categories presented in Table 4, assigning number and letter codes to the quality of the methodology and direction of results of each article respectively. For example, a study that was found to be of very good quality and presented clear evidence of a negative association was assigned a Grade of Evidence of 4B.

Results
We conducted a series of systematic literature reviews that involved a range of disciplines on a concept that does not have a universally accepted definition. This endeavour posed several conceptual and methodological challenges.
By conducting a broad literature search, we retrieved a heterogeneous body of literature, and as a result it was not always appropriate or possible to compare results across studies. Grouping the studies according to the definition of frailty provided by the study author was a useful way of dealing with this heterogeneity. This approach allowed the reviewers to compare study results within a given definition of frailty. Nevertheless, a few Question Leaders chose to narrow down the final review to one or two definitions of frailty in order to make the process more manageable. For example for the review of risk factors for

Physiological syndrome
Operational criteria whereby vulnerability is based on physiological factors and/or physical functional impairments (performance measures); multiple systems have to be considered for a study to be classified as using a physiological syndrome definition for frailty.

Complex syndrome
Operational criteria whereby vulnerability is based on some combination of physiological and/or functional impairment (physical performance measures) with age, cognitive, psychological (e.g., depressive symptoms), sensory and/or social factors,

Geriatric syndrome
Operational criteria whereby vulnerability is based on some combination of chronic disease, the geriatric giants (such as falls, incontinence), functional impairment, disability, cognition, health service utilisation, and/or mortality risk

Disability
ADL/IADL measures with or without functional impairment 3. Setting or source of recruitment Criteria refer to the setting or the source of recruitment e.g., all participants in a nursing home or all participants eligible for geriatric assessment

Other
The definition used does not fit into any of the above-mentioned categories frailty, dealing with the numerous risk factors identified for the various different definitions of frailty created an unwieldy matrix of results that could not be summarized in a single review. For this reason the Question Leaders chose to limit themselves to one of the possible definitions for frailty. For other Questions, the use of a broad search contributed to noteworthy findings. For example, in the review of literature on the prevalence of frailty, there was marked variation in reported point-estimates, much of which was likely attributable to differences in the definitions, criteria and tools used to identify frailty. This underscored the observation that researchers are not speaking a common language when discussing frailty, and whichever definition one chooses to apply will have important effects on the findings [21].
The CIFA reviews examined research from different disciplines, requiring the review methodology to accommo- * These articles were rated on a scale from 1-3 (poor, fair, good). Articles that were rated "poor" are presented in the 1-2 category and articles rated as "fair" or "good" are presented in the 3-4 category. † 11 of these articles had study designs that could not be evaluated with any of the CIFA quality assessment tools. date differences in disciplinary conceptions and research practices [22]. At the outset, certain methodological guidelines were deemed necessary to ensure the validity and reliability of all the reviews. Elements such as formulating a clear research question, predetermining the article inclusion criteria, having two independent reviewers selecting the articles, and blinding at all stages of article selection and quality assessment were crucial in ensuring the reproducibility of the review. In an attempt to be as comprehensive as possible in the reviews, literature searches were conducted in two different electronic databases and the reviewers pearled the references of selected articles to identify additional articles that may have been missed by the electronic searches. A librarian was involved in reviewing all the electronic search strategies and proposing complementary steps when the initial searches did not yield appropriate results. The CCG proposed limits in the timeframe and languages of the literature to be reviewed. These were selected to match the resources available. Similar limits were implemented across all the reviews in order to achieve some consistency.
On the other hand, tools for quality assessment and data abstraction as well as the presentation of results had to be developed to cater to the individual reviews and the type of study designs that were being reviewed. Nevertheless, a number of common guidelines were proposed. For example, it was decided that the quality assessment of the overall study would result in a subjective score. A contentious issue in quality assessment is the method of arriving at a score for the overall quality of the study. Some advocate a sum of all the individual components assessed. We felt that would not accurately reflect the overall quality of a study since all the items could not be weighted equally. In certain cases, a single flaw could be fatal to the validity of a study, and in other cases multiple minor flaws did not necessarily compromise the overall methodological quality of the study. Another guideline proposed by the CCG was that reviewers determine and present grades of evidence for their findings. The latter were useful for review questions that addressed association between risk factors and frailty or biological markers and frailty, but were not applicable to questions on the clinical identification of frailty or even the prevalence of frailty.
In some instances, the Question Leaders modified aspects of the proposed methodology to allow them to meet specific objectives. For example, some included articles that were published outside the proposed timeframe in order to incorporate seminal papers that were judged to still shape thinking about frailty in their field. While consistency in methodology was clearly desirable, it was important to permit flexibility in the application of the proposed methodology to capture the essence of frailty research within the given fields of research. All modifica-tions to methodology were discussed and approved by the CIFA Steering Committee and CCG. Our main objective was to ensure that the methodology for each review could be clearly described (and replicated) and that the methodological choices were well aligned with the research questions.
Due to the complexity of the topic and process, as well as the number of investigators involved, the reviews took longer than expected. The development and fine-tuning of the methodology required lengthy discussions and consultations spanning several months. The quality assessment of an article for instance generally took between thirty minutes and two and a half hours per assessor, with an additional fifteen to thirty minutes for the consensus between assessors. Given the large number of articles assessed, this phase also involved a substantial amount of time. Further, since the quality assessors were not experts in the content areas, the Question Leaders sometimes deemed it necessary to modify the assessments, which led to some duplication of the work. Because the initial review process took so long, the reviews were behind schedule, and all the literature searches had to be updated by a year.

Discussion
The most important conceptual challenge in conducting these systematic reviews was to determine what would qualify as being part of the literature on frailty. Some authors have treated frailty as synonymous with vulnerability, disability, or dependence, whereas others have attempted to describe frailty as a distinct concept [2,9]. The diversity in models is an indication of the isolation of researchers working on frailty [2]. Though this raised challenges, we elected to be inclusive in our literature review. We did not begin by debating the merits of the competing viewpoints and adopting a specific model and/or definition of frailty [23,24]. By being as inclusive as possible in our selection of literature, we felt that we could contribute to a more comprehensive understanding of the concept of frailty.
The methodological challenges faced in conducting these reviews arose from our goal of trying to establish and adhere to a consistent methodology for reviewing literature from diverse fields of research. While there is no standardised methodology for systematic literature reviews [25], the methodology described was developed by the CCG in order to maintain consistency and a high standard in the systematic reviews of frailty. Several groups have provided guidelines for conducting systematic reviews. Most of these are limited to systematic reviews and meta-analyses of randomised clinical trials (RCTs). However, the systematic review of research using other study designs is increasingly common and recog-nised as necessary for answering many research questions that are not amenable to RCTs. The CIFA reviews included cross-sectional, longitudinal and qualitative studies and required the development of a methodological approach that would be applicable to all of these.
One of our interests in conducting these reviews was to determine whether the approaches to frailty in the different fields are complementary and whether their findings converge. We elected to be inclusive in our literature review as the interest in frailty research spans a number of fields as diverse as geriatrics, gerontology, biology of aging, sociology, rehabilitation science, and health policy. Each has their own approach to the conceptualisation of frailty and addresses different research questions. Our intention was to foster a dialogue between researchers from different fields in order to stimulate more interdisciplinary collaborative work on frailty thereby establishing new avenues for research on the topic [26]. From this perspective the CIFA review process was extremely successful.
The literature searches for all reviews yielded extensive material on disability, but very limited results on frailty itself. Within the literature on frailty, there was substantial variation in the definition and operationalisation of frailty, even within a single area of research. Evidently, differences in the definitions had important effects when investigating such questions as the prevalence, risk factors, interventions, or impact of frailty. The results of the reviews suggest that there is a need and potential for evidence-based convergence in the definition of frailty across the various fields of research. The CIFA reviews have resulted in manuscripts in each of the ten areas. At the time of submission of the current paper, these manuscripts have either been submitted for publication or are in preparation for submission.
The results of the systematic reviews across the various disciplines point to exciting new horizons in the area of frailty research. Future work has the potential of furthering our understanding of the aging process and offers the hope that we can identify vulnerable older adults [1]. Despite the debate on the exact nature of frailty, there is no disagreement on its catastrophic impact on older individuals, their families (particularly those involved in providing support to the older individual), and on society as a whole. Ultimately, to be useful frailty research must lead to the identification of effective health promotion, prevention, treatment, rehabilitation, and care interventions. To this end, future research in frailty needs to consider existing knowledge across disciplines in order to work towards the development of a comprehensive framework of frailty that is relevant to clinicians, researchers and frail individuals.
By conducting a systematic literature review on frailty using a meticulous and explicit process that involved many different fields of research, CIFA has made an important contribution to frailty research. The results of the systematic reviews will enable us to establish the status of current knowledge, make well-founded suggestions for future research, and promote collaboration between disciplines.
Conducting systematic literature reviews in health science involving multiple disciplines is a first step towards more interdisciplinary collaboration as well as a more integrated understanding of health. However, such an endeavour is labour-intensive and requires a team with a high level of methodological and substantive expertise. This initiative also highlighted the need for developments in the area of multidisciplinary systematic reviews.