- Open Access
- Open Peer Review
Why sex and gender matter in implementation research
BMC Medical Research Methodologyvolume 16, Article number: 145 (2016)
There has been a recent swell in activity by health research funding organizations and science journal editors to increase uptake of sex and gender considerations in study design, conduct and reporting in order to ensure that research results apply to everyone. However, examination of the implementation research literature reveals that attention to sex and gender has not yet infiltrated research methods in this field.
The rationale for routinely considering sex and gender in implementation research is multifold. Sex and gender are important in decision-making, communication, stakeholder engagement and preferences for the uptake of interventions. Gender roles, gender identity, gender relations, and institutionalized gender influence the way in which an implementation strategy works, for whom, under what circumstances and why. There is emerging evidence that programme theories may operate differently within and across sexes, genders and other intersectional characteristics under various circumstances. Furthermore, without proper study, implementation strategies may inadvertently exploit or ignore, rather than transform thinking about sex and gender-related factors. Techniques are described for measuring and analyzing sex and gender in implementation research using both quantitative and qualitative methods.
The present paper describes the application of methods for integrating sex and gender in implementation research. Consistently asking critical questions about sex and gender will likely lead to the discovery of positive outcomes, as well as unintended consequences. The result has potential to strengthen both the practice and science of implementation, improve health outcomes and reduce gender inequities.
Efforts to integrate sex and gender throughout all phases of the health research cycle have been rising sharply over the past two decades [1–4]. Since 2010, the Canadian Institutes of Health Research has been requiring researchers to indicate whether their research protocol accounts for sex or gender, using the term “sex” to refer to the biological attributes that distinguish male from female, and the term “gender” to refer to men and women’s socially constructed roles, identities and behaviors [5, 6]. As of 2016, the U.S. National Institutes of Health Research asks applicants to explain how they plan to factor consideration of sex as a biological variable into their research design and analysis [7, 8]. The Gender Advisory Group to the European Framework Program for Research and Innovation also mandates the Gender Dimension across all sectors . Journal editors are encouraged to increase accountability around sex and gender reporting requirements, by using the Sex and Gender Equity in Research (SAGER) guidelines [10, 11]. These events beg the question: how have research methods in implementation science addressed sex and gender? For the purpose of this article, we will use the term implementation research and practice (IRP) to include knowledge translation, implementation research and practice.
The opening argument for this debate article is that to date, despite the evidence on the impact of sex and gender on health, research methods in the field of implementation have neglected sex and gender considerations. An analysis of selected literature in IRP supports this proposition. For example, a review of the tables of contents and indexes of three popular implementation science texts [12–14] reveals that none devote a chapter to the role of sex and gender in implementation science. Only one includes gender in the index , which refers to a chapter in the text with a few lines describing how many sexually transmitted infection interventions targeting racial/ethnic minorities are gender specific and how the strategies to reach men and women may differ .
Searching the top 10 articles of 2015 as reported by Implementation Science (see Additional file 1: Appendix 1) for the words sex or gender shows that only one makes a minor mention of gender as it relates to controlling for ‘clinician gender’ in a modeling exercise . Sex and gender also do not appear to play a prominent role in implementation theories. For example, in Nilsen’s review  of implementation theories, models and frameworks, only 2 make minor references to gender [18, 19]. One simply includes ‘gender’ as one of the barriers to optimal clinical practice under the category “health care professional/physician barriers”  and the other includes a footnote about a study they were citing, that “a fourth factor, gender of participants, was also related to program outcomes but was not included in their subsequent analysis” . Furthermore, neither of the germinal papers on the Theoretical Domains Framework [20, 21], a widely used and influential framework  that guides assessment of barriers to implementation, makes any reference to sex or gender. There is a domain in the framework that focuses on ‘social/professional role and identity’, which could capture elements of sex and gender, but usually tends to be limited to assessing professional roles and almost never identity, let alone sex or gender identity.
Turning to the knowledge synthesis literature on implementation, the Cochrane Collaboration’s Methods Equity group  is active in increasing awareness of the need and methods for sex and gender analysis in systematic reviews [23–25]. Both the Effective Practice and Organisation of Care (EPOC) and Consumers and Communication review groups have official guidance in their resources for authors on equity [26, 27]. A recent assessment of a sample of systematic reviews from these two groups however, reveals limited consideration of sex and gender in the written report. For example, of 12 EPOC and seven Consumer and Communication reviews published between July 2014 and May 2015, none addressed sex or gender in the analysis or implications sections of the report (Personal Communication, Jennifer Petkovic, Peter Tugwell and Vivian Welch on behalf of the Campbell and Cochrane Equity Methods Group, April 13, 2016). It is possible that the review authors did consider sex and gender in their analyses and determined it was unimportant. However, they failed to report this.
Little research has been undertaken or reported to inform how sex and gender impact IRP, as evidenced by this analysis of key texts, well-used conceptual models, and Cochrane reviews on implementation strategies. The objective of this paper is to describe the rationale for why and how sex and gender should be considered in IRP.
What is sex? What is gender?
A first step for understanding how to integrate sex and gender in IRP involves operationalizing the two terms, and recognizing different components of gender. The term sex refers to a biological construct, whereby an individual is defined as being male or female according to genetics, anatomy and physiology [6, 7, 11, 28–32]. Researchers should use the term sex when describing the number of male or female patients or committee members, or when stratifying outcomes by male versus female participants or health care providers. It is more appropriate to say what the distribution by sex of a sample or target audience is, than to use the term ‘gender distribution.’ This is because gender is a multifaceted and fluid construct, influenced in a temporal manner by social and cultural contexts and environments to create gender norms [6, 7, 11, 28, 30–35]. Gender norms influence commonly accepted ways of how people behave, how they perceive themselves and each other, how they act and interact, and the distribution of power and resources in society [6, 28, 31–35]. Gender can be structured by, and operating within ethnicity, indigenous status, social status, sexuality, geography, socioeconomic status, education, age, disability/ability, migration status, and religion, requiring an intersectional approach to implementing practices, programs and policies [36, 37]. The acronym “PROGRESS” can be used to remember these variables: place of residence, race/ethnicity/culture/language, occupation, gender/sex, religion, education, socioeconomic status, and social capital . Researchers often understand gender as a function of gender roles (e.g. child care, housework), gender identity (e.g. personality traits such as being sensitive to the needs of others or having leadership abilities), gender relations (e.g. social support), and institutionalized gender (e.g. career opportunities, personal income, educational background) [6, 28, 34]. Gender as a broad term can also refer to the expressions and identities of girls, women, boys, men, and gender diverse people [39, 40]. For this reason, definitions of sex and gender are evolving as science changes, and it remains challenging to easily separate the biological from the social. Sex and gender are often interrelated, interactive and potentially inseparable [6, 11]. Given the epistemology of knowledge, and the social nature of implementation and behavior change, the effect of gender and other identity factors, either alone or in combination, can serve as barriers or enablers to the outcome or impact of IRP interventions.
Measuring and understanding sex and gender
Collecting and analyzing data on sex in IRP is relatively simple if using typical male and female categories. Sex can be self-reported, designated by an examination of external genitalia, or genetically determined based on an XX, XY or intersex genotype . Data on sex-related factors can include measuring sex hormones, body and organ size, metabolism, or fat tissue distribution . Gender is more complex, and can be operationalized along four different constructs: gender roles, gender identity, gender relations and institutionalized gender [6, 28, 31, 32]. Table 1 defines these four constructs, gives examples of key questions that can be asked of each in IRP, and lists measures and methods for use in IRP research [6, 28, 31, 32, 42–44].
Traditionally, individuals are asked to categorize their sex as male or female and many assumptions, often based in gender and not biology, are made on the basis of their responses. Researchers are now rethinking this approach to be more inclusive of gender identity and expression . A two-step approach to measuring sex and gender identity could first ask individuals to indicate their sex assigned at birth (male/female), and then ask the same individuals how they currently self-identify, which could include male, female, trans male/trans man, trans female/trans woman, gender queer/gender non-conforming; and provide space to self identify as another option not provided . Similarly, participants could also be given the option to disclose sexual orientation and whether they consider themselves part of the lesbian, gay, bisexual or transgender (LGBT) community.
The scales in Table 1 list measures that can be used to quantify different dimensions of gender. Researchers can also create gender scales using gender-related variables of relevance to their particular research topic [45, 46]. Pelletier et al. created a composite gender score using 7 characteristics: 1) status on primary earner; 2) personal income; 3) number of hours per week spent doing housework; 4) status of primary person responsible for doing housework; 5) level of stress at home; 6) masculine traits; and 7) feminine traits. They were able to demonstrate that gender, independent of sex, predicts poor outcome after acute coronary syndrome, pointing to new areas of intervention .
Qualitative methods are also useful for the collection of data on specific dimensions of gender. Case studies, ethnography, narrative and descriptive qualitative approaches can provide evidence and contextualized insight across a range of participants’ personal characteristics, including those of sex and gender. Qualitative methods can also be used to explore concepts of institutionalized gender, and to gain a more in-depth understanding of gender as a barrier or enabler to the use of implementation interventions, the uptake of the evidence-informed clinical interventions or program and the outcomes of implementation efforts. A number of texts, casebooks, examples and online courses are available that provide guidance on how to conduct sex and gender science using commonly employed quantitative and qualitative methods [6, 32, 42, 43].
The case for considering sex and gender in implementation research methods
Emerging evidence suggests that sex and gender are important in decision-making, stakeholder engagement, communication and preferences for the uptake of interventions. Furthermore, when gender norms, identities and relations are ignored, unintended consequences may occur. The following five scenarios give examples of when and why sex and gender should be measured and considered in implementation research:
When the implementation of an intervention requires decision-making on the part of individuals or organizations. Decision-making is a critical component of behavior change interventions, and plays a key role in the uptake of new organizational practices and programs . Research from the fields of business and management offer insights for IRP on important sex and gender factors related to decision-making [48–50]. Qualitative research conducted by Deloitte Consulting with 18 large business organizations suggests that female executives have a tendency to be more attuned to micro-level signaling during meetings, and may favour discovery options and iterative thinking during decision-making processes . Male executives tend to end a conversation once they connect with a good idea or solution. Their female counterparts are inclined to be more inquisitive, wanting to hear everyone’s thoughts before deciding, and taking more time to find the ideal solution. Different leadership traits among male and female leaders can therefore influence the outcome of decision-making processes [49, 50].
When sex and gender dynamics may play a role in stakeholder engagement and conflict resolution. A survey queried reasoning methods among 624 corporate board directors, of whom 75 % were male and 25 % female . Female directors scored significantly higher scores on the complex moral reasoning dimension, which implies attending more to relationships and to the challenge of balancing multiple stakeholders’ interests. Females may also engage in more collaboration and consensus building, not only to make sound decisions but also to elicit common support for a course of action [49, 50]. The outcome of an implementation intervention may therefore depend on the sex and gender dynamics in each particular context.
When communication strategies are being tested, as sex and gender may be differentially responsive to the choice of language used, the strength of persuasion of the communication strategy, and the way promotional information is processed. This is why sex and gender form the basis for market segmentation in the fields of marketing and consumer behaviour, where subtle changes in language and emotional appeal can have a differential effect on men and women’s attitudes towards the brand advertised and purchase intentions . The way messages and interventions are primed or packaged to reflect gender norms or stereotypes may also influence the outcomes of health promotion interventions. For instance, priming individuals to the perception that women eat healthier foods than men leads both male and female study participants to prefer healthy foods, whereas priming masculinity results in unhealthy food preferences . When the packaging and healthiness of the food are gender congruent (i.e., feminine packaging for a healthy food, masculine packaging for an unhealthy food) both male and female participants rate the product as more attractive, report that they would be more likely to purchase it, and even rate it as tasting better compared to when the product is stereotype incongruent.
When negative or harmful gender stereotypes may impede the uptake and outcomes of an IRP initiative . A realist review of the implementation of school-based interventions to prevent domestic abuse for children and young people reported that lesbian, gay, bisexual and transgender youth felt excluded from the programmes, as the content did not address gender identity or sexual orientation in high-risk populations . Similarly, data suggest that masculine norms around emotional control and self-reliance are associated with recurrent non-suicidal self-injury . Stigma related to healthcare seeking for male depression and suicide [57, 58], may explain why women are more likely to benefit from psychosocial treatment for the prevention of suicide and suicidal ideation compared to men . Some studies purport that gender bias in prescription patterns among health care providers results in more women receiving treatment with antidepressants for mental health  and pain symptoms, but only among female clinicians . Men, on the other hand, may be preferentially managed with orthopaedic surgery to manage knee arthritis .
When gendered power relations may inadvertently skew the uptake of information focused on women’s health needs, such as maternal and child health, sexual and reproductive health, or family planning . This occurs in cultures and settings where male partners and head of households play a large role in female’s health-seeking behaviour due to their authority and decision-making role. For example, the introduction of health programs enhanced by mobile phone technologies overall fosters women’s empowerment in low-income countries . However, in some cases these programs exacerbated gender relations and gender inequalities, such as when women were pressured to give the phones provided by the program to their husband if he did not already own a phone, or when conflicts about phone use led to cases of spousal abuse.
The World Health Organization outlines a spectrum of gender-responsive programs, illustrating the progression from the exploitative use of gender stereotypes in IRP messaging, through to accommodation and ultimate transformation to gender equity (Fig. 1) [65, 66]. Making active choices reflecting content, messaging and decision-making processes during the implementation of an intervention can have a critical impact on gender equity for women and men. Gender transformative approaches are preferred as they anticipate unintended barriers and consequences and address the causes of gender-based health inequities where they exist . For example, during implementation of a tobacco control program, investigators can decide to use motivational recruitment techniques that appeal to a person’s health and self-respect, as opposed to messaging that invokes and reinforces stereotypical gendered norms of sexual attractiveness, beauty claims or images based on masculinity or femininity . Recent guidance based on qualitative research suggests de-linking messages for men and for women when promoting tobacco reduction during pregnancy and post partum, since the uptake of the intervention can be hindered by negative couple dynamics if the partners have different smoking behaviours or attitudes about smoking during this period [68–70]. Another transformative approach to encourage uptake of smoking cessation interventions would be to focus on a wider range of non-stereotypical gendered roles that include fathering for men and work for women as potential motivators.
Sex and gender can be therefore be pivotal at multiple points along the IRP process, from the content and messaging surrounding the intervention, to decision making around the uptake and unintended consequences of an intervention. Asking sex and gender questions can also elucidate enablers and barriers to the adoption of complex behavioral interventions. For example, examining the outcome of implementing a multidisciplinary cardiac rehabilitation program merits asking whether women or men have less time to devote to recovery and prevention activities due to gender-based expectations regarding their responsibilities at home. The potential advantages of including sex and gender in the study of other complex behavioral interventions (e.g., hand hygiene, reducing clinician's opioid prescribing, reducing falls in hospital, increasing vaccination rates, and obesity prevention) require further investigation. Measuring the way sex and gender influences these interventions may help elucidate potential mechanisms and contexts behind the success or failure of various IRP efforts, as shown in the examples above on tobacco cessation, healthy eating, depression and suicide, pain, heart disease and domestic violence.
Some research questions to drive the selection of methods
Researchers can start by asking a series of questions about how sex and gender can have an impact on their implementation initiative in order to determine the best way to measure and analyze the effect of sex and gender. First, how might sex or gender affect decision-making and stakeholder engagement, or facilitate or impede the uptake of evidence-informed practice, programs, policies? Second, how might sex based characteristics or prevailing gender norms or gender roles serve as barriers or enablers to the uptake of evidence-informed practices, programs, policies? Third, when and how should the communication strategy, wording or messaging be tailored across sex, gender or other identity characteristics? Fourth, when using participatory/collaborative or integrated knowledge translation research approaches, could the sex and gender of the researchers and knowledge users matter, and if so, how? Similarly, how might gender relations as a function of dyads or interpersonal dynamics within an organization, community, workplace or institution influence the outcome of the intervention? And finally, should the research protocol consider examining whether there are unintended impacts of implementation that exacerbate or diminish sex, gender or other diversity-related inequities? Table 2 lists a series of questions for researchers to consider when designing their IRP research. Additional opportunities for integrating sex and gender in IRP as relates to models of health systems research have been reviewed in detail elsewhere .
Another way to study issues of sex and gender in IRP resides in the realist approach which attempts to answer the question, “What works, for whom, in what circumstances, and why?”  This is accomplished through the identification and examination of underlying generative mechanisms or program theories associated with the implementation intervention or program, the conditions or contexts under which the mechanisms operate, and the pattern of outcomes produced. Realist evaluators may wish to examine sex and gender through the lens of this Context-Mechanism-Outcomes configuration for the evaluation of new initiatives, programs and scale-up [71, 72].
Through this lens, context can be defined as the particular sub-groups for whom the outcomes were successful, the gender relations between the stakeholders, the sex and/or gender of the individuals who implemented the intervention, and the institutional, socio-economic, cultural and political conditions. Mechanism refers to the explanation of how a particular program’s resources work to change the reasoning and responses of participants to bring about the adoption of the clinical intervention or program that results in both intended and unintended outcomes. Outcomes are the impacts of the intervention. Some questions of how sex and gender considerations can align with the Context-Mechanism-Outcomes configuration are: How do gender roles, gender identity, gender relations, and institutionalized gender influence the way in which an implementation strategy works, for whom, under what circumstances and why? Or, how do program theories operate/work within and across sexes, genders and other diversity characteristics, in what circumstances and why? Finally, research results should be disaggregated and reported by sex or gender groups . It is important to report whether there are similar effects or differences.
When critically appraising the publications of implementation research, reviewers should increasingly ask whether the reports consider sex and gender during a study’s life cycle, and if so, how? Table 3 provides a beginning list of questions that can be asked of implementation research to help the reader assess whether sex and gender have been adequately considered, and the extent to which this may have influenced the study findings and conclusions.
This paper argues that sex and gender should always be considered in implementation research. Considering sex and gender should be an essential component of IRP. Failing to integrate sex and gender may neglect an important determinant of knowledge use, reducing the effectiveness of implementation interventions, inadvertently reinforcing sex neutral claims and negative gender stereotypes, and possibly creating or increasing gender and health inequities in care and health outcomes. Only by consistently investigating sex and gender in a critical and reflective manner that addresses underlying gender inequities, will the field of IRP reach its full potential for meeting the requirements of scientific rigour, excellence and maximal impact.
Effective Practice and Organisation of Care
Research and practice
Mazure CM, Jones DP. Twenty years and still counting: including women as participants and studying sex and gender in biomedical research. BMC Women’s Health. 2015;15:94.
Clayton JA, Collins FS. Policy: NIH to balance sex in cell and animal studies. Nature. 2014;509:7500.
Gates MF. Putting women and girls at the center of development. Science. 2014;345:6202.
Klein SL, Schiebinger L, Stefanick ML, Cahill L, Danska J, de Vries GJ, Kibbe MR, McCarthy MM, Mogil JS, Woodruff TK, Zucker I. Opinion: sex inclusion in basic research drives discovery. Proc Natl Acad Sci U S A. 2015;112:17.
Johnson J, Sharman Z, Vissandjée B, Stewart DE. Does a change in health research funding policy related to the integration of sex and gender have an impact? PLoS One. 2014;9(6):e99900.
Canadian Insitute of Gender and Health. What a difference sex and gender make: a gender, sex and health research casebook. 2012. Available at: http://www.cihr-irsc.gc.ca/e/documents/What_a_Difference_Sex_and_Gender_Make-en.pdf. Accessed 14 April 2016.
Clayton JA. Studying both sexes: a guiding principle for biomedicine. FASEB J. 2015;30:519–24.
Tannenbaum C, Schwarz JM, Clayton JA, de Vries GJ, Sullivan C. Evaluating sex as a biological variable in preclinical research: the devil in the details. Biol Sex Differ. 2016;7:13.
Pollitzer E, Buitendijk S, Hermann C, Muhlenbruch B, Schiebinger L. Integrating Gender in Horizon 2020. Pen Sci Technol. 2015;15:1–3. https://s3-eu-west-1.amazonaws.com/data.epws.org/DOCUMENTS/WEBSITE2016/PRESS/ST15+E+Pollitzer+et+al+1022+Atl_CH.pdf. Accessed 25 Oct 2016.
Johnson JL, Beaudet A. Sex and gender reporting in health research: why Canada should be a leader. Can J Public Health. 2012;104:e80–1.
Heidari S, Babor TF, De Castro P, Tort S, Curno M. Sex and gender equity in research: rationale for the SAGER guidelines and recommended use. Res Integr Peer Rev. 2016;1(1):1–9.
Straus SE, Tetroe J, Graham ID, editors. Knowledge translation in health care: moving from evidence to practice. 2nd Edition. West Sussex: Wiley-Blackwell; 2013. p. 424.
Grol R, Wensing M, Eccles M, Davis D. Improving patient care: the implementation of change in health care, 2nd Edition. West Sussex: Wiley-Blackwell; 2013. p. 392.
Brownson RC, Colditz GA, Proctor EK. Dissemination and implementation research in health: Translating science to practice. New York: Oxford University Press; 2012. p. 528.
Yancey AT, Gleen BA, Bell-Lewis L, Ford CL. Dissemination and implementation research in populatons with health disaparities. In: Brownson RC, Colditz GA, Proctor EK. Dissemination and implementation research in health: Translating science to practice. New York: Oxford University Press; 2012. p. 528.
Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implement Sci. 2015;10:11.
Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53.
Cochrane LJ, Olson CA, Murray S, Dupuis M, Tooman T, Hayes S. Gaps between knowing and doing: understanding and assessing the barriers to optimal health care. J Contin Educ Health Prof. 2007;27:94–102.
Durlak JA, Dupre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41:327–50.
Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37.
Francis JJ, O’Connor D, Curran J. Theories of behaviour change synthesized into a set of theoretical groupings: introducing a thematic series on the theoretical domains framework. Implement Sci. 2012;7:35.
Sarmast H, Mosavianpour M, Collet J-P, Kissoon N. TDF (Theoretical Domain Framework): how inclusive are TDF domains and constructs compared to other tools for assessing barriers to change? BMC Health Services Res. 2014;14 Suppl 2:81.
The Cochrane Collaboration’s Methods Equity group. Available at: http://methods.cochrane.org/equity/igh-cochrane-corner. Accessed 14 April 2016.
Runnels V, Tudiver S, Doull M, Boscoe M. The challenges of including sex/gender analysis in systematic reviews: a qualitative survey. Syst Rev. 2014;3:33.
Doull M, Runnels V, Tudiver S, Boscoe M. Appraising the evidence: applying sex and gender based analysis (SGBA) to Cochrane systematic reviews on cardiovascular diseases. J Women’s Health. 2010;19:997–1003.
Effective Practice and Organisation of Care (EPOC). Equity considerations in EPOC reviews. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2013. Available at: http://epoc.cochrane.org/sites/epoc.cochrane.org/files/uploads/18%20Equity%20considerations%20in%20EPOC%20reviews%202013%2008%2012_1.pdf. Accessed 14 April 2016.
Ryan R. Cochrane Consumers and Communication Review Group. ‘Cochrane Consumers and Communication Review Group reviews: how to consider equity issues’. 2013. Available at: http://cccrg.cochrane.org/sites/cccrg.cochrane.org/files/uploads/Equity.pdf Accessed 14 April 2016.
Johnson JL, Greaves L, Repta R. Better science with sex and gender: a primer for health research. Vancouver: Women’s Health Research Network; 2007. Available at: http://bccewh.bc.ca/wp-content/uploads/2012/05/2007_BetterSciencewithSexandGenderPrimerforHealthResearch.pdf. Accessed 14 April 2016.
Wizeman TM, Pardue M. Institute of Medicine. Exploring the biological contributions to human health: Does sex matter? Washington, DC: The National Academies Press; 2001. Available at https://www.nap.edu/read/10028/chapter/1. Accessed 14 April 2016.
Fishman JR, Wick JG, Koenig BA. The use of sex and gender to define and characterize meaningful differences between men and women: a report of the task force on the NIH women’s health research agenda for the 21st Century. Nat Inst Health Office Res Women’s Health. 1999;1:15–9.
Canadian Institutes of Health Research (CIHR). Gender, sex and health research guide: a tool for CIHR applicants; 2010. Available at http://www.cihr-irsc.gc.ca/e/32019.html. Accessed 14 April 2016.
Schiebinger L, Stefanick ML. Gender matters in biological research and medical practice. J Am Coll Cardiol. 2016;67(2):136–8.
Morgan R, George A, Ssali S, Hawkins K, Molyneux S, Theobald S. How to do (or not to do)… gender analysis in health systems research. Health Policy Plan. 2016;31(8):1069–78. doi:10.1093/heapol/czw037.
Rossi AM, Pilote L. Let’s Talk About Sex…and Gender! Circ Cardiovasc Qual Outcomes. 2016;9(2 Suppl 1):S100–1.
Gender. Gendered Innovations in Science, Health & Medicine, Engineering, and Environment. Available at: https://genderedinnovations.stanford.edu/terms/gender.html Accessed 14 April 2016.
Bauer GR. Incorporating intersectionality theory into population health research methodology: challenges and the potential to advance health equity. Soc Sci Med. 2014;110:10–7.
Bowleg L. The problem with the phrase women and minorities: intersectionality-an important theoretical framework for public health. Am J Public Health. 2012;102:1267–73.
O’Neil J, Tabish H, Welch V, Petticrew M, Pottie L, Clarke M, Evans T, Pardo Pardo J, Waters E, White H, Tugwell P. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol. 2014;67:56–64.
Westbrook L, Saperstein A. New categories are not enough: rethinking the measurement of sex and gender in social surveys. Gender Soc. 2015;29:534–60.
Scheim AI, Bauer GR. Sex and gender diversity among transgender persons in Ontario, Canada: results from a respondent-driven sampling survey. J Sex Res. 2015;52:1–14.
Einstein G. Measuring Biological Sex. In Oliffe J, Greaves L, editors. Designing and Conducting gender sex and health research. Washington D.C.: SAGE; 2012. pp. 85-101.
Canadian Institutes of Health Research, Institute of Gender and Health. Online Training Modules: Integrating Sex & Gender in Health Research. Available at http://www.cihr-irsc.gc.ca/igh-competency.html. Accessed 2 Dec 2015.
Oliffe J, Greaves L. (eds). Designing and Conducting Gender Sex and Health Research. Washington D.C.: SAGE; 2012. p. 280.
Pelletier R, Khan NA, Cox J, Daskalopoulou SS, Eisenberg MJ, Bacon SL, Lavoie KL, Daskupta K, Rabi D, Humphries KH, Norris CM, Thanassoulis G, Behlouli H, Pilote L, GENESIS-PRAXY Investigators. Sex versus gender-related characteristics: which predicts outcome after acute coronary syndrome in the young? J Am Coll Cardiol. 2016;67:127–35.
Smith PM, Koehoorn M. Measuring gender when you don’t have a gender measure: constructing a gender index using survey data. Int J Equity Health. 2016;15:82.
Pelletier R, Ditto B, Pilote L. A composite measure of gender and its association with risk factors in patients with premature acute coronary syndrome. Psychosom Med. 2015;77:517–26.
Pawson R. The science of evaluation: a realist manifesto. Washington D.C.: SAGE; 2013. p. 240
Benko C, Pelster B. Gender. How Women Decide. Harvard Business Review 2013 September. https://hbr.org/2013/09/how-women-decide. Accessed 29 May 2016.
Eagly AH, Johannesen-Schmidt MC, van Engen ML. Transformational, transactional, and laissez-faire leadership styles: a meta-analysis comparing women and men. Psychol Bull. 2003;129:569–91.
Cuadrado I, García-Ael C, Molero F. Gender-typing of leadership: evaluations of real and ideal managers. Scand J Psychol. 2015;56:236–44.
Bart C, McQueen G. Why women make better directors. Int J Business Govern Ethics. 2013;8:93–9.
Banks IB, Dens N, De Pelsmacker P. The influence of gender on the effectiveness of probability markers in advertising. Int J Advertising. 2016;35:682–705.
Zhu LL, Brescoli VL, Newman GE, Uhlmann EL. Macho nachos: The implicit effects of gendered food packaging on preferences for healthy and unhealthy foods. Soc Psychol. 2015;46:182–96.
Brown CS, Stone EA. Gender stereotypes and discrimination: how sexism impacts development. Adv Child Dev Behav. 2016;50:105–33.
Stanley N, Ellis J, Farrelly N, Hollinghurst S, Downe S. Preventing domestic abuse for children and young people: a review of school-based interventions. Child Youth Serv Rev. 2015;59:120–31.
Green JD, Kearns JC, Ledoux AM, Addis ME, Marx BP. The Association Between Masculinity and Nonsuicidal Self-Injury. Am J Mens Health. 2015. [Epub ahead of print]
Creighton GM, Oliffe JL, Lohan M, Ogrodniczuk JS, Palm E. “Things I did not know”: Retrospectives on a Canadian rural male youth suicide using an instrumental photovoice case study. Health (London). 2016. [Epub ahead of print]
Oliffe JL, Ogrodniczuk JS, Gordon SJ, Creighton G, Kelly MT, Black N, Mackenzie C. Stigma in male depression and suicide: a Canadian sex comparison study. Community Ment Health J. 2016;52:302–10.
Krysinska K, Batterham PJ, Christensen H. Differences in the effectiveness of psychosocial interventions for suicidal ideation and behaviour in women and men: a systematic review of randomised controlled trials. Arch Suicide Res. 2016;16:1–21.
Tannenbaum C, Lexchin J, Tamblyn R, Romans S. Indicators for measuring mental health: towards better surveillance. Health Policy. 2009;5:e177–86.
Hirsh AT, Hollingshead NA, Matthias MS, Bair MJ, Kroenke K. The influence of patient sex, provider sex, and sexist attitudes on pain treatment decisions. J Pain. 2014;15:551–9.
Fraenkel L, Suter L, Weis L, Hawker GA. Variability in recommendations for total knee arthroplasty among rheumatologists and orthopedic surgeons. J Rheumatol. 2014;41:47–52.
Kululanga L, Sundby J, Malata A, Chirwa E. Striving to promote male involvement in maternal health care in rural and urban settings in Malawi - a qualitative study. Reprod Health. 2011;8:36.
Jennings L, Gagliardi L. Influence of health interventions on gender relations in developing countries: a systematic review. Int J Equity Health. 2013;12:85.
WHO Gender Responsive Assessment Scale: criteria for assessing programmes and policies. In: WHO Gender Mainstreaming Manual for Health Managers: a practical approach. 2015. Available at: http://www.who.int/gender/mainstreaming/GMH_Participant_GenderAssessmentScale.pdf. Accessed 17 Sept 2015.
Greaves L, Pederson A, Poole N (Eds). Making It Better: Gender Transformative Health Promotion. Canadian Scholar’s Press/Women’s Press. 2014. Available at http://promotinghealthinwomen.ca/book/. Accessed 14 April 2016.
Greaves L. Can tobacco control be transformative? Reducing gender inequity and tobacco use among vulnerable populations. Int J Environ Res Public Health. 2014;11:792–803.
Bottorff JL, Kalaw C, Johnson JL, Stewart M, Greaves L, Carey J. Couple dynamics during women’s tobacco reduction in pregnancy and postpartum. Nicotine Tob Res. 2006;4:499–509.
Greaves L, Kalaw C, Bottorff J. Case studies of power and control related to tobacco use during pregnancy. Women’s Health Issues. 2007;17:325–32.
Bottorff JL, Carey J, Poole N, Greaves L, Urquhart C. Couples and Smoking: What you need to know when you are pregnant. 2008. Available from: http://facet.ubc.ca/wp-content/files/Couples-and-Smoking.pdf. Jointly published by the British Columbia Centre of Excellence for Women’s Health and the Centre for Healthy Living and Chronic Disease Prevention, University of British Columbia Okanagan.
Dalkin SM, Greenhalgh J, Jones D, Cunningham B, Lhussier M. What’s in a mechanism? Development of a key concept in realist evaluation. Implement Sci. 2015;10:49.
Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review – a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10:21–34.
O’Neil JM, Helms BJ, Gable RK, David L, Wrightman LS. Gender-role conflict scale: College men’s fear of femininity. Sex Roles. 1986;14:335–50.
Bem SL. On the utility of alternative procedures for assessing psychological androgyny. J Consult Clin Psychol. 1977;45:196–205.
Spence JT, Helmreich R, Stapp J. Ratings of self and peers on sex role attributes and their relation to self-esteem and conceptions of masculinity and femininity. J Pers Soc Psychol. 1975;32:29–39.
Mahalik JR, et al. Development of the conformity to masculine norms inventory. Psychol Men Masc. 2003;4:3–25.
Sörlin A, Lindholm L, Ng N, Öhman A. Gender equality in couples and self-rated health-A survey study evaluating measurements of gender equality and its impact on health. Int J Equity Health. 2011;10:37–47.
We express gratitude to Jo Rycroft-Malone, Professor of Implementation & Health Services Research, Bangor University, United Kingdom, for providing direction to get out of the “realist swamp.” We also thank Krystle van Hoof, Assistant Director of the CIHR Institute of Gender and Health for her thoughtful comments, and Justin Presseau, Scientist at the Ottawa Hospital Research Institute for insightful discussions about sex and gender in implementation research and the Theoretical Domains Framework. IDG is a recipient of Canadian Institutes of Health Research, Foundation Grant Scheme, Inaugural competition, FDN #143237.
Institute of Gender and Health, Canadian Institutes of Health Research.
Availability of data and materials
All data generated or analysed during this study are included in this published article.
CT, LG and IDG conceived and drafted the article. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
All authors consented.
Ethics approval and consent to participate
Appendix 1. List of the top implementation science papers published in 2015. (DOCX 19 kb)