Creating groups with similar expected behavioural response in randomized controlled trials: a fuzzy cognitive map approach
© Giabbanelli and Crutzen; licensee BioMed Central Ltd. 2014
Received: 20 March 2014
Accepted: 8 December 2014
Published: 12 December 2014
Controlling bias is key to successful randomized controlled trials for behaviour change. Bias can be generated at multiple points during a study, for example, when participants are allocated to different groups. Several methods of allocations exist to randomly distribute participants over the groups such that their prognostic factors (e.g., socio-demographic variables) are similar, in an effort to keep participants’ outcomes comparable at baseline. Since it is challenging to create such groups when all prognostic factors are taken together, these factors are often balanced in isolation or only the ones deemed most relevant are balanced. However, the complex interactions among prognostic factors may lead to a poor estimate of behaviour, causing unbalanced groups at baseline, which may introduce accidental bias.
We present a novel computational approach for allocating participants to different groups. Our approach automatically uses participants’ experiences to model (the interactions among) their prognostic factors and infer how their behaviour is expected to change under a given intervention. Participants are then allocated based on their inferred behaviour rather than on selected prognostic factors.
In order to assess the potential of our approach, we collected two datasets regarding the behaviour of participants (n = 430 and n = 187). The potential of the approach on larger sample sizes was examined using synthetic data. All three datasets highlighted that our approach could lead to groups with similar expected behavioural changes.
The computational approach proposed here can complement existing statistical approaches when behaviours involve numerous complex relationships, and quantitative data is not readily available to model these relationships. The software implementing our approach and commonly used alternatives is provided at no charge to assist practitioners in the design of their own studies and to compare participants' allocations.
KeywordsAllocation method Artificial intelligence Computational model Randomization
Randomized Controlled Trials (RCTs) are a powerful approach to conduct quantitative and comparative controlled experiments. However, many sources of bias can negatively affect the quality of the experiments. While carefully controlling bias is always required, the difficulty of doing so varies depending on the area. For example, RCTs for health behaviour change are more prone to bias than pharmaceutical trials, and reducing bias may also be more difficult for the former than latter condition [1, 2]. A statistical introduction to sources of bias can be found in Matthews , while an overview for a broader audience is provided in Jadad and Enkin .
Two techniques that take into account prognostic factors are stratification and minimization. The former applies randomization for every combination of participant prognostics (e.g., to guarantee the same number of patients across groups for each age category and ethnicity), which may not be feasible in small trials and/or when the number of such combinations is very large. The latter handles this problem by ensuring a balance only in the individual prognostics as participants enter the trial (e.g., to guarantee the same number of patients across groups for each age category or ethnicity). Both techniques assume that the mechanisms by which the prognostics contribute to the trial outcome are unknown, thus they aim at controlling the distributions of prognostic factors and expect this to carry onto the distribution of outcomes. In this paper, we take a different approach to the allocation phase of RCTs for behaviour change by focusing on the distribution of outcomes. As will be illustrated throughout the paper, the advantages of our approach over the aforementioned ones are twofold. First, our approach accounts for the many complex interactions among prognostic factors, since they could balance each other out. Second, practitioners do not need to make any prior assumption as to which prognostic factor(s) are relevant. Practically, the free and open-source software that we provide allows practitioners to simply list the prognostic factors and behavioural outcome of the intervention, automatically generating questionnaires and allocating participants into groups upon completion of the questionnaires.
Our allocation method uses novel computational tools to infer a participant’s outcome from the prognostic factors, and uses that inferred outcome to allocate the participant. This does not mean that the relationships among prognostic factors and participants’ outcome should be specified before conducting the RCT. Intuitively, imagine that a black box takes as input a participant’s prognostic factors and outputs the participant’s expected outcome. To ensure that groups are comparable at baseline, it suffices to assign participants such that the distribution of expected outcomes among groups are similar. Practically, the black box explicitly structures how the prognostic factors contribute to the outcome. For example, an intervention may aim at improving exercise, with weight as measured outcome. Based on the participants’ age, gender and socio-economic status, the black box simulates how an intervention on exercise may impact their weight. Accordingly, participants are assigned into groups such that the groups have a similar distribution of participant’s simulated impact. These groups might have a different demographic make-up, since they are equalized in terms of their expected trial outcome rather than on prognostic factors. This is more flexible than stratification in small trials (e.g., n < 30) since combinations of prognostic factors may yield the same expected outcome and may thus not have to be kept equal among groups. It is also less simplified than minimization, since it accounts for the interactions of prognostic factors.
We first provide a technical background on the computational solutions to inferring one’s outcome. In particular, we review how participants’ knowledge can be used to deal with vague or conflicting relationships such as those occuring among prognostic factors, and we summarize how the uncertainty of participants’ responses can be incorported into models. Then, we introduce our proposed solution to allocating participants to groups such that groups have a similar distribution of expected response to the intervention. This solution is exemplified via three cases studies, and differences with existing alternatives are also highlighted. We also evaluate our solution in the presence of missing data, for sequential trials, and with the addition of randomness. In order to support practitioners in putting our solution to practice and generate more case studies, we also provide free software that covers all necessary steps, from setting-up an intervention to collecting participants’ questionnaires and allocating them. Finally, we discuss the role that our approach could play in conducting RCTs, and we analyze its current limitations.
Technical solutions to infer behavioural outcomes
Computational approaches typically aim at solving a problem by creating simulations that repeatedly apply a set of rules governing the behaviour of individuals or groups. Therefore, they differ from statistical approaches such as regression models. Computational models of human behaviour have been created in a variety of fields to explain and structure the relationships among prognostic factors and outcome at the individual-level. For example, in criminology, a model of how individuals navigate an urban space was created to explain the locations of crimes (outcome) given offenders’ home locations and the locations of major venues such as shopping malls [8, 9]. Similarly in health psychology, a model of how individuals interact was developed to explain binge drinking (outcome) given demographic information and drinking motives . Numerous methods can explicitly model the mechanisms that link prognostic factors to outcome. In the following paragraph, we will divide these methods as either data-driven or expert-driven. The former (i.e., data-driven) can automatically create a model from the data, but cannot straightforwardly test what an intervention will lead to, since all mechanisms are derived from past data. The latter (i.e., expert-driven) is not automatically built from data but instead involves experts who directly articulate a set of assumptions, which can easily be altered to test what-if scenarios such as the expected consequences if an intervention was to be put to practice.
Creating groups with similar expected baseline behavioural outcome in a suitable way for RCTs requires several changes from the aforementioned approaches. First, the procedure must be able to structure the relationships among prognostic factors and behavioural outcome in a fully automatic manner (as for classifiers built using data mining – in line with Figure 2a) since it should be applicable to any trial without requiring human expertise during the randomization procedure. Second, the resulting structure must be easily amenable to changes (as for man-made models – in line with Figure 2b) such that it is possible to assess the expected effect of an intervention. We present an approach that satisfies both requirements (Figure 2c). Its main practical difference compared to the previous two solutions is that all participants must first fill a baseline questionnaire that directly surveys them about the relationships within the prognostic factors themselves and the outcome. This questionnaire can be automatically designed once the prognostic factors and outcome have been specified. Our approach uses neither classifiers nor models based on pre-specified hypotheses: instead, it relies on Fuzzy Cognitive Maps (FCMs), which are artificial intelligence tools used to represent human knowledge. The next section briefly situates FCMs among the techniques used to model human knowledge, and then FCMs are formally specified.
Modelling human knowledge
Individuals who have managed a condition (e.g., obesity) or performed a certain health behaviour (e.g., smoking) over a long time have gained knowledge about its underlying mechanisms. Historically, individuals have rarely been prompted to directly share this knowledge. Instead, they are more commonly asked about contributing factors in isolation, and associations are inferred statistically. For example, a person may be asked during screening to estimate his/her level of depression and physical activity, and regressions can be used to assess the mechanisms that may link the two factors. However, it is less common to directly ask the person to estimate the impact of depression on physical activity.
This tendency mostly owes to the assumption that individuals cannot accurately reflect on what shapes a condition. This assumption was widely made in older theories, such as the Freudian theory from the late 19th century. Freud proposed a topographical model that separated the mind into conscious, preconscious, and unconscious. Much of the behaviour was supposed to be determined by unconscious thoughts , as the mind would actively prevent unconscious traumatic events from reaching consciousness . However, the many psychological systems developed in the last decades have challenged this assumption of behaviours as mostly unconscious, by stating that individuals are to a large extent aware of the mechanisms surrounding their actions. Consequently, theories have been proposed regarding how individuals can reach an unambiguous understanding of such mechanisms . Empirical evidence has also confirmed that individuals can carefully depict how factors influence one another based on complex internal schematics . Several tools have been developed in the last decade, due to the growing importance of patient centered care and the realization that the insight of individuals about the mechanisms can be valuable for behaviour change. In the United Kingdom, a pack of cards named Agenda Cards was developed. Each card contains a statement about diabetes and individuals pick cards to indicate which mechanisms and factors are at work for them. These cards have been “very well accepted by people with diabetes and health professionals” . Cards were later developed for obesity in Canada and included statements such as “I do not feel confident in my ability to resist overeating when I am nervous, depressed, or angry” .
Participants’ generated-evidence about (causes of) their behaviour has also been used to create models, motivated by the fact that models of complex social phenomena should be built using all evidence available . To build models by prompting participants to share their knowledge, we are asking them to consciously recall facts. This is called declarative memory, and it is part of the long-term memory together with procedural memory (e.g., skills). Within declarative memory, model building taps into the semantic memory, which is the conceptualization of the world. That is, participants are not asked to share one specific experience (which may be be representative of their experience as a whole): rather, they are prompted to share the concepts that they have derived from their experiences.
When participants are asked to create models, they might “(un)consciously reduce complexity in order to prevent information overload and to reduce mental effort” . Abundant research has demonstrated that such simplifications occur regardless of expertise  or training in model building . Therefore, several approaches have been developed to circumvent this limitation.
The simplest approach is to get the knowledge of a group rather than a single participant. However, experiments have showed that groups show the same biases as individuals and do not result in higher quality decisions . This has motivated the design of mixed methods approaches that use quantitative tools to aggregate the participants’ qualitative answers on questionnaires. In the 1960s, the Delphi method was used to created an aggregate group response by using the median of the individual responses; experiments showed that this leads to more accurate responses than the natural process of group decision-making . In the 1970s, Roberts modelled interactions if they were endorsed by 6 out of 7 respondents, and gave them a directionality if it was agreed to by at least 60% of the respondents  pp. 142–179.
Three issues remain in aggregating the knowledge of several participants. First, solving the conflicts among participants using a majority vote results in losing some of the individual nuances. Second, relationships vary in weight and capturing this can help focus on the most important drivers of a system. However, the use of linguistic terms to assess the strength of causations (e.g., weak, strong) produces the vagueness inherent to human language. Third, there is uncertainty in human knowledge and the participants themselves may want to express the extent to which they are confident regarding their knowledge of specific relationships.
FCMs provide the mathematical tools to address these three limitations. FCMs aggregate participants’ experiences using Fuzzy Set Theory (FST), which is precisely “designed to mathematically represent uncertainty and vagueness, and to provide formalized tools for dealing with imprecision in real-world problems” . Here, we are giving particular attention to the two mechanisms through which the uncertainty and vagueness associated with participants’ responses is incorporated in FCMs.
First, participants share their experience via linguistic terms. For example, they may say that weight-based discrimination has had a “strong” impact on their stress level. Perceptions of what constitute a “strong” impact differs among individuals. Rather than associating a term to a specific value, Fuzzy Set Theory associates it to a range (through a fuzzy membership function). For example, on a scale from 1 to 5, “strong” may be matched with a normal distribution from 3 to 5 and peaking at 4. Ranges can overlap, which accounts for the possibility that the reality described by a term (e.g., “strong”) partly includes that of another (e.g., “very strong”). The equations of fuzzy membership functions can be found for instance in .
Second, the linguistic terms chosen by the participants are aggregated through rules. For example, if 14 out of 42 participants said that a relationship is “strong” while the remaining 28 see it as “very strong” then the first term is associated with a confidence factor of 14/42 while the second is associated with 28/42. The confidence factor represents the uncertainty, and it is central to deriving a numerical value that summarizes the overall experience of the participants. The equations for that derivation are provided for example in [8, 23].
The mathematics of fuzzy cognitive maps
Asking participants about their experiences for each relationship and aggregating their answers using FST results in a Fuzzy Cognitive Map. Informally, an FCM uses FST to articulate the relationships among concepts, which can be used to represent prognostic factors. An FCM is graphically depicted as a set of nodes (standing for concepts), linked by directed edges (standing for causality) whose thickness (weight) is computed using FST.
Sigmoid functions such as the hyperbolic tanget are suitable for complex problems .
Allocation of all participants to groups after they all provided baseline data
The FCM has k prognostic factors, denoted f1, …, f k and all linked.
The FCM has one intervention linked to the k prognostic factors.
The Fuzzy Cognitive Map is built from all participants data and denoted FCM(f1, …, f k )
For each participant i
The participant’s expected change is computed by Ei = FCM(f1, …, fk)
Participants are sorted in ascending order of Ei
Assume that participants must be allocated to n groups. For each participant in order:
Allocate participant i to group (i modulo n)
A questionnaire is automatically generated to ask all participants to evaluate the strength of each possible relationships3 (i.e., the weight of each edge) (line 3, Figure 3a). While a regression approach may only include the relationships from prognostic factors to the behavioural outcome, allowing these relationships to run both ways as well as having interactions between prognostic factors is necessary for the presence of loops, which are typically found in complex problems. Upon completion of all the questionnaires, the value of each edge is automatically computed by applying Fuzzy Set Theory on the participants’ answers (Figure 3b). The FCM has then been built.Once the FCM is built, it is used to infer each participant’s behavioural outcome with and without the intervention (line 4), using each participant’s value for the prognostic factors as an initial value for the corresponding node in the FCM (line 5, Figure 3c). As the FCM inference mechanism has loops in its structure, the values of concepts will change until the behavioural outcome stabilizes. Note that because the behavioural outcome is initially unknown, the FCM is used to assess the relative difference in outcome.
After line 4-a, we have an overall distribution of how much participants would change under the intervention. The goal is then to take equal sizes samples of that distribution such that the samples’ distributions are similar. In other words, we want to allocate participants to each of the groups such that the groups have a similar expected baseline behavioural outcome. To do this, participants are first sorted by their simulated change, from smaller to larger (line 5). Then, the first participant is assigned to the first group, the second to the second group, and so on until all groups received a participant. The allocation then cycles back to the first group, and the cycle repeats until all participants have been allocated (line 6-6a).
In this section, we contrast the performances of our methods with commonly used alternatives on three case studies. For two of these case studies, we obtained data from human subjects. The data collection was performed in accordance with the Declaration of Helsinki, and it was approved by the ethics committee of Simon Fraser University under the studies 2012 s0725 and 2013 s0494. Informed consent to participate in the study was obtained from all participants. For the last case study, we create a dataset from population-level distributions in order to assess the performance of our proposed methods in the case of large sample sizes.
Case study 1: designing a RCT for eating disorders
Case study 2: designing a RCT regarding exercise
To ensure that questionnaires are filled in a realistic manner, answers are randomly generated by drawing on real-world data. Note that drawing on real-world data differs from the previous case study: here the answers from participants are generated from a probability distribution calibrated from large samples, whereas in the previous case study each answer was either directly provided by a participant or calculated from what was provided. The age reported is drawn from Statistics Canada  using the adult Canadian population as of July 1st 2010, while income uses the 2009 data from Statistics Canada. The answers to five other prognostic factors are also drawn from large Canadian datasets and are separated by age category. We used the National Population Health Survey  (NPHS, n = 14,500) for depression and stress, the Canadian Community Health Survey [36, 37] (CCHS) for antidepressants (n = 36,984) and physical health (n = 131,486), and the Prince Edward Island Nutrition Survey  (PEINS) for obesity (n = 1,995). The distribution of exercise in the population relies on the observation that most individuals are sedentary , which is approximated using an Inverse Gaussian Distribution. An individual’s food intake is set to be slightly above exercise, to account for an environment that promotes eating over exercising. As in the previous study , we consider that among Canadian adults, fatness is often perceived as negative and there often is a belief in personal responsibility; thus, both factors are assigned high values. As in the previous case study, we set weight discrimination to linearly depend on obesity. Simulations previously concluded that results obtained by the relationships in Figure 6 were almost indistuinguishable whether weight discrimination was set to depend linearly on weight or when individuals start being discriminated only if very obese . Details of the methodology regarding prognostic factors can be found elsewhere . The strength of the relationships in Figure 6 are drawn from our pilot study in which a panel of participants evaluated each relationship . In other words, the answers of a given virtual participant are set to the ones provided by a previous real participant chosen at random from our panel of respondents. Software to create virtual participants based on the above methodology is distributed with the simulation software.
Case study 3: designing a RCT regarding unhealthy eating
Sample questions used for the relationships depicted in Figure 8
Statements in the online survey (from “never” to “always”)
Sad →+ Unhealthy eating
When I’m sad, I eat more than I should.
Body shape →+ Stress
I feel stressed because of my body shape.
Pain →- Exercise
When I’m in pain, I avoid exercise.
Exercise →+ Pain
Exercise is painful.
Weight discrimination →+ Stress
Other people’s negative comments or attitudes about my weight make me stressed.
Medications →+ Hunger
I take medications that make me feel hungry.
Impact of missing data
Missing answers on factors are a problem for the methods that allocate participants based on their specific values (e.g., stratification but not randomization). For example, if participants are stratified based on their age category, then a person whose age category was not provided cannot be assigned to a group. Imputation is a typical way to address missing data, by replacing by an estimate. Using imputation, our method can thus operate in the presence of missing data. As for methods such as stratification, the allocation would be affected by the way in which the estimate for missing data is computed. This is illustrated in Figure 10c, where a synthetic population of 5000 individuals was generated using the method of the 2nd case study, and the growing percentage of missing answers (x-axis) was replaced using different imputation strategies (average, fitting a normal distribution, replacing by a constant). The quality of the allocation (y-axis) worsens but remaining better than commonly used methods, for up to 8% of missing data. As noted in  p. 280, “missing variables are often from severely ill persons on the skewed end of the distribution” thus nonparametric imputation methods could provide more robust results than regressions or maximum likelihood approaches. We would recommend using a nonparametric method such as the Kernel Density Estimation together with fast evaluation algorithms (e.g., the Fast Gauss Transform ) such that missing values can be replaced by good estimates with little computational power even for large trials with numerous factors.
Fixed sample design versus sequential design
Our proposed method is based on a fixed sample design: the sample size is calculated at the beginning of the trial, all participants complete the baseline questionnaire, and they are then allocated to groups. This is sufficient for many behavioural trials as well as drug trials where immediate randomisation is required. However, many trials fail to reach their planned size within the expected timeline and nonetheless require immediate randomisation. Thus, several sequences are sometimes needed  pp. 65–66. Our method can be straightforwardly adapted to work with several waves of recruitments. Assume that a target sample of n participants has to be recruited over s sequences of allocation with equal sizes, that is, participants are recruited in batches of and immediately allocated to groups. Instead of applying our method to n participants, it can be applied to them batch by batch. Practically, the FCM can be built on participants 1 to b who are then allocated (1st wave); next, the FCM is built on participants 1 to 2 × b (combining questionnaires from the 1st and 2nd waves) who are then allocated, etc. It should be noted that this is an adaptive design, since the ways in which participants in new batches are allocated is based on a data-driven adjustment based on data accumulated so far. This is similar to the covariate-adjusted response-adaptive (CARA) designs investigated by Li-Xin Zhang and van der Laan, in which subjects who join a trial are allocated based on cumulative information from previous subjects, adjusted according to the individual’s information [43, 44].
Impact of randomization
Allocation based on optimization is not randomization since the result is entirely deterministic. The deterministic method that we present still addresses accidental bias as well as selection bias, since the complexity of the protocol carried out by a computer prevents an observer from inferring the group to which a participant would be allocated from the participant’s baseline characteristics. However, randomization provides a basis for the statistical analysis of the trial results . Consequently, optimization-based approaches such as minimization are not randomized in their pure form but often include some randomization in practice. In this section, we show that randomness can straightforwardly be included in our protocol, and we evaluate its impact on the quality of the allocations.
While the approach to randomization above is relatively simple to implement, more constrained forms of randomization may be preferrable. Indeed, randomizing the whole sequence may result for example in swapping an individual having a very low expected behavioural response with one having a very high expected behavioural response, which negatively affects the balance of groups. Consequently, swapping individuals that are closer in the allocation sequence may have a lower impact. To favour this, the probability to swap may be made proportional to the distance between individuals in the sequence. If one wishes to fully prevent individuals far apart from swapping, then we would recommend binning the continuous behavioural response estimated by the system. Instead of following the order of participants, the final allocation can thus follow the order of the bins and pick participants within each bin uniformly at random. Such approaches to transforming the data are further discussed in , and their consequences on the balance of the groups would have to be investigated in future research.
Bias is a key issue in RCTs, and particularly so for interventions regarding behaviour change. In this paper, we focused on limiting bias coming from allocations, that is, ensuring that participants are allocated to groups which are similar in terms of behavioural response to the intervention. While several methods are readily available, they are typically used to balance prognostic factors in isolation. In other words, the determinants of behaviour are independently balanced, under the assumption that it would lead to balanced behaviours at baseline as well. We took a different approach by focusing on the interactions between prognostic factors via a novel computational method that aims at balancing a simulated behaviour rather than isolated prognostic factors. This method using Fuzzy Cognitive Maps, which have successfully been used in aspects of health where the accuracy of the result could be easily measured and was critical, such as radiotherapy  or brain tumor characterization . They have also been used in health behaviour, for example in relation to obesity  or diabetes .
The potential of our method was assessed through three case studies, using real-world data as well as generated synthetic data from large samples. Results suggest that our method can create groups with more similar behavioural responses than other commonly used approaches. Indeed, other approaches exhibited larger differences between the inferred behaviours of the allocation groups.
To assess the potential of our method, we developed a free and open-source software (CAPT: Computational Allocation of Participants in Trials). Software can be downloaded with no registration needed from http://rctsoft.free.fr, or http://www.crutzen.net/capt, where a detailed tutorial is also provided in order to support practitioners in integrating our solution to the design of their trials and, subsequently, the allocation of participants. The software is stand-alone and has an easy-to-use Graphical User Interface (GUI), so no other programs or writing of syntaxes are needed. Two independent parts are provided in order to provide flexibility to practitioners: the design part and the allocation part.
Precisely assessing the quality of our approach requires individual-level data in which participants report on relationships and have a known trial outcome. However, because full disclosure is not yet a common practice [51, 52], such data is extremely scarce. One reason is that sharing trial data remains overwhelmingly a matter of choice: for example, less than one out of five authors of a clinical trial(s) were required by their funder to deposit trial data in a repository and, even when asked to do so, only 57% reported doing it . Furthermore, while many studies highlight the benefits of directly asking participants to share their experiences in order to inform the design of interventions [54, 55], they typically focus on what drives behaviour (i.e., prognostic factors) rather than how (i.e., underlying mechanisms). That is, etiological frameworks do suggest many of the prognostic factors that could be considered when designing a trial, but they rarely operationalize the mechanisms linking prognostic factors to each other and the behavioural outcome. Therefore, the real-world case study used to compare our allocation method to others had to be based on one of the few studies that provided individual-level data and directly asked participants on the perceived impact of prognostic factors . Consequently, we were able to contrast the results of different allocation methods, but individual-level clinical data directly speaking to mechanisms is necessary to adequately assess a given allocation method by itself. In addition, our assessment did not aim to compare our protocol to an exhaustive set of allocation methods. Given the large number of methods available, we focused on the most commonly used ones. A more extensive assessment would have to take into accounts randomization methods such as the maximal procedure [56, 57].
The protocol introduced here can automatically allocate participants into groups once they have provided their completed questionnaires, as illustrated by the free open-source software developed to provide an easy-to-use environment such that practitioners can integrate the protocol to their trials. Two advantages to a fully automatic allocation are as follows. First, there is no risk to obtain sub-optimal allocations by focusing on prognostic factors that turn out not to be as decisive as expected, which can happen when using stratification as exemplified in our case studies. Second, automatically using complex artificial intelligence techniques makes it particularly difficult for staff to know which group a participant will be allocated to, which is a valuable feature given that blinding is difficult trials for behaviour change . One drawback of operating in a fully-automatic mode is that participants are asked to evaluate every possible relationship, and the number of such relationships is proportionate to the square of the number of prognostic factors. When many prognostic factors are considered, operating in a fully-automatic mode can thus contribute to a high quality trial providing the evidence needed to improve healthcare, but at the same time it would increase the burden on participants by generating a larger questionnaire (i.e., not only assessing prognostic factors and outcomes, but also the relationships between them). This may lead an increase in missing data, which our methods can already address but at the expense of lower allocation qualities; this points to the need for further research in handling missing data. The burden placed on participants may also be deemed excessive in some settings. Our software provides a trade-off to practitioners at the trial design stage by allowing them to remove the relationships deemed less relevant for the sake of parsimony, thereby providing input on the logic model behind the intervention . It should be noted that altering the logic model always runs the risk of resulting in a “wrong” model. A model that is “wrong” by assessing inexisting relationships (e.g., impact of age on the weather) would not be a problem in our approach, as participants would be expected to eliminate that relationship. However, a model that is “wrong” by removing important relationships would decrease the quality of the relationships. Based on our simulations for missing data on relationships, we would expect that, as a model increasingly removes important relationships, its performances align with those of stratification.
Future software versions may allow practitioners to directly provide evidence (e.g., from previous studies or literature reviews) regarding select relationships rather than having to systematically query participants. While our method was motivated by trials for behaviour change, enabling practitioners to directly enter equations (e.g., relationships between body tissues ) would be an important step toward supporting the application of our method in the clinical setting. Future versions may also offer practitioners the possibility of creating different categories of relationships, which could automatically translate to different phrasing for the questions provided to participants. Many such categories have already been proposed to structure the contributors of behaviour. In Malle’s framework building on folk theory of behavior , relationships are classified as reason explanations (e.g., when the environment drives toward an action whose outcome was not desired) or causal explanations (e.g., when the participant has the intention of performing an action that leads to a desired outcome). In the theory of action identification , relationships may be seen through a hierarchy where the lowest level dictates simple motor actions (e.g., moving a finger) while the highest level shapes the broader understanding (e.g., the beliefs surrounding a behaviour and its consequences). Categorizing each relationship also offers additional information that can then be leveraged when applying computational methods to infer the participants’ reaction to an intervention. For example, relationships that are higher in the aforementioned hierarchy are also deemed more stables, and they could be attached a larger confidence than relationships regarding lower levels. However, further research is needed to integrate confidence levels into the development of the Fuzzy Cognitive Maps used here. While we developed a method and compared it to commonly used alternatives through multiple case studies, we also believe that the next step of this research should assess how to best guide the analysis of trial data generated by our protocol. Specifically, allocation methods that attempt to improve balance can negatively affect the type I error depending on how the allocation method is accounted for in the analysis. Hagino and colleagues showed that an unadjusted analysis is conservative when stratified randomization or minimization is used, but an analysis adjusting for the allocation factors as covariates does not affect type I error . While we would expect an analysis that ignores our allocation method to be conservative and lose some power, developing specific mechanisms to account for the baseline covariate balancing (resulting from our allocation method) would be critical to “facilitate acceptance of trial results and minimize potential for controversial interpretations” .
The protocol that we introduced provided individual-level allocation once all individuals filled the questionnaires, using deterministic artificial intelligence methods. An adaptation was also evaluated to support the sequential enrollment of participants by batches, rather than aiming at reaching the desired sample size before allocating participants. A new version could be developed to satisfy the needs of other possible types of allocations. Specifically, an intervention may be directed at a group rather than a person, or participants may influence each other. In this case, Cluster Randomized Controlled Trials could be used and allocations should take place at the group-level rather than the individual-level. A straigthforward way to using our protocol in this context would be to have all participants still complete the questionnaires, but instead of submitting individual answers to our software, their answers would first be aggregated using Fuzzy Logic to represent the group, and then submitted. However, further simulation research is needed to explore how the aggregation of individual answers into a group answer affects the allocations.
A new protocol was presented to limit the allocation bias of RCTs, focusing on, but not limited to, interventions on behaviour change. The protocol relies on artificial intelligence techniques that focus on relationships among prognostic factors, thereby providing an alternative to commonly used techniques centred on the prognostic factors themselves.
1 When an outcome is given, this task is known as supervised learning. When there is no outcome, a typical task of unsupervised learning is to cluster individuals rather than classifying them, because the outcome is unknown.
2 Synthetic data refers to data that was not obtained by direct measurement. In our case, data is generated by a computer in order to observe the reaction of our solution to criteria for which measured data is not available. This process is particularly used to test clinical trials  and computational solutions .
3 If there are k prognostic factors and one behavioural outcome, then the baseline questionnaire has to assess each of the k(k+1) relationships. This may not be scalable for large k due to the resulting size of the questionnaire, and it is possible to remove relationships as summarized in the discussion.
4 A sample invitation to participate is provided in http://web.archive.org/web/20131218220754/ and http://blogs.plos.org/obesitypanacea/2013/12/02/participants-needed-an-online-survey/
PJG would like to thank Simon Fraser University for providing financial support via the Graduate Student Research Award and the President’s Phd Scholarship. RC would like to thank the Netherlands Organisation for Scientific Research – Division for the Social Sciences (NWO-MaGW) for funding through the Innovational Research Incentives Scheme Veni. Both authors thank Vijay K. Mago and Francine Schneider for providing insightful feedback.
- Boutron I, Guittet L, Estellat C, Moher D, Hrobjartsson A, Ravaud P: Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med. 2007, 4 (2): 270-280.View ArticleGoogle Scholar
- Crocetti MT, Amin DD, Scherer R: Assessment of risk of bias among pediatric randomized controlled trials. Pediatrics. 2010, 126 (2): 298-305. 10.1542/peds.2009-3121.View ArticlePubMedGoogle Scholar
- Matthews JNS: Chapter 2: Bias. Introduction to Randomized Controlled Clinical Trials. 2006, Chapman & Hall/CRC: Texts in Statistical Science Series, 15-22. 2View ArticleGoogle Scholar
- Jadad AR, Enkin MW: Chapter 3: Bias in Randomized Controlled Trials. Randomized Controlled Trials. 2007, Blackwell Publishing, 29-47. 2View ArticleGoogle Scholar
- Senn S: Seven myths of randomisation in clinical trials. Stat Med. 2013, 32 (9): 1439-1450. 10.1002/sim.5713.View ArticlePubMedGoogle Scholar
- Bonfill X, Rigau D, Jauregui-Abrisqueta ML, Barrera Chacon JM, Salvador de la Barrera S, Aleman-Sanchez CM, Bea-Munoz M, Morelada Perez S, Borau Duran A, Ramon Espinosa Quiros J, Ledesma Romano L, Esteban Fuertes M, Araya I, Martinez-Zapata MJ: A randomized controlled trial to assess the efficacy and cost-effectiveness of urinary catheters with silver allow coating in spinal cord injured patients: trial protocol. BMC Urol. 2013, 13: 38-10.1186/1471-2490-13-38.View ArticlePubMedPubMed CentralGoogle Scholar
- Langera D, Burtin C, Schepers L, Ivanova A, Verleden G, Decramer M, Troosters T, Gosselink R: Exercise training after lung transplantation improves participation in daily activity: a randomized controlled trial. Am J Transplant. 2012, 12 (6): 1584-1592. 10.1111/j.1600-6143.2012.04000.x.View ArticleGoogle Scholar
- Mago VK, Frank R, Reid A, Dabbaghian V: The strongest does not attract all but it does attract the most – evaluating the criminal attractiveness of shopping malls using fuzzy logic. Expert Syst. 2014, 31 (2): 121-135. 10.1111/exsy.12015.View ArticleGoogle Scholar
- Iwanski N, Frank R, Dabbaghian V, Reid A, Brantingham P: Proceedings of the Intelligence and Security Informatics Conference (EISIC). Analyzing an offender’s Journey to Crime: A Criminal Movement Model (CriMM). 2011, 70-77.Google Scholar
- Giabbanelli PJ, Crutzen R: An agent-based social network model of binge drinking among dutch adults. J Artif Soc Soc Simulat. 2013, 16 (2): 10-Google Scholar
- Crutzen R, Giabbanelli PJ: Using classifiers to identify binge drinkers based on drinking motives. Subst Use Misuse. 2014, 49 (1–2): 110-115.View ArticleGoogle Scholar
- Gross R: Psychology: The Science of Mind and Behaviour. 2012, Hodder Education, 6Google Scholar
- Carlson NR, Heth DS, Miller HL, Donahoe JW, Buskist W, Martin GN: Psychology: The Science of Behavior. 2006, Allyn & Bacon, 6Google Scholar
- Vallacher RR, Wegner DM: What do people think they’re doing? Action identification and human behavior. Psychol Rev. 1987, 94 (1): 3-15.View ArticleGoogle Scholar
- Parrott RL, Silk KJ, Condit C: Diversity in lay perceptions of the sources of human traits: genes, environments, and personal behaviors. Soc Sci Med. 2003, 56: 1099-1109. 10.1016/S0277-9536(02)00106-5.View ArticlePubMedGoogle Scholar
- Pennington J: Unusual partnerships and new ways of working in diabetes. J Diabetes Nurs. 2006, 10 (7): 261-264.Google Scholar
- Matteson CL, Merth TDN, Finegood DT: ISRN Obesity. Health Communication Cards as a Tool for Behaviour Change. 2014, 579083-Google Scholar
- Edmonds B, Moss S: From KISS to KIDS – an ‘anti-simplistic’ modelling approach. Lect Notes Comput Sci. 2005, 3415: 130-144. 10.1007/978-3-540-32243-6_11.View ArticleGoogle Scholar
- Vennix JAM: Group Model Building. 1996, John Wiley and SonsGoogle Scholar
- Axelrod R: Structure of decision: the cognitive maps of political elites. 1974, Princeton University PressGoogle Scholar
- Dalkey NC: The Delphi Method: An Experimental Study of Group Opinion. 1969, RAND RM-58888-PRGoogle Scholar
- Li Z: Fuzzy Chaotic Systems. 2006, SpringerGoogle Scholar
- Mamdani EH: Applications of fuzzy logic to approximate reasoning using linguistic synthesis. IEEE Trans Comput. 1977, 26 (12): 1182-1191.View ArticleGoogle Scholar
- Giabbanelli PJ, Torsney-Weir T, Mago VK: A fuzzy cognitive map of the psychosocial determinants of obesity. Appl Soft Comput. 2012, 12 (12): 3711-3724. 10.1016/j.asoc.2012.02.006.View ArticleGoogle Scholar
- Pratt SF, Giabbanelli PJ, Jackson P, Mago VK: Proceedings of the IEEE International conference on Intelligence and Security Informatics. Rebel with Many Causes: A Computational Model of Insurgency. 2012, 90-95.Google Scholar
- Voinov A, Bousquet F: Modelling with stakeholders. Environ Model Softw. 2010, 25: 1268-1281. 10.1016/j.envsoft.2010.03.007.View ArticleGoogle Scholar
- Korb KB, Nicholson AE: Bayesian Artificial Intelligence. 2004, CRC PressGoogle Scholar
- Stylios CD, Groumpos PP: Modeling fuzzy cognitive maps. IEEE Trans Syst Man Cybern Syst Hum. 2004, 34: 155-162. 10.1109/TSMCA.2003.818878.View ArticleGoogle Scholar
- Stach W, Kurgan L, Pedrycz W, Reformat M: Genetic learning of fuzzy cognitive maps. Fuzzy Sets Syst. 2005, 153 (3): 371-401. 10.1016/j.fss.2005.01.009.View ArticleGoogle Scholar
- Tsadiras AK: Comparing the inference capabilities of binary, trivalent and sigmoid fuzzy cognitive maps. Inf Sci. 2008, 178 (20): 3880-3894. 10.1016/j.ins.2008.05.015.View ArticleGoogle Scholar
- Deck P, Giabbanelli PJ, Finegood DT: Exploring the heterogeneity of factors associated with weight management in young adults. Can J Diabetes. 2013, 37 (S2): S269-S270.View ArticleGoogle Scholar
- National Institutes of Health: Clinical Guidelines on the Identification, Evaluation, and Treatment of Overweight and Obesity in Adults. 1998, NIH Publication, 98-4083.Google Scholar
- Giabbanelli PJ, Jackson P, Finegood DT: Modelling the joint effect of social determinants and peers on obesity. Theories and Simulation of Complex Social Systems, Intelligence Systems Reference Library. 2014, 145-160.View ArticleGoogle Scholar
- Statistics Canada: Annual Demographic Estimates: Canada, Provinces and Territories. 2011, Catalogue 91-215-WXEGoogle Scholar
- Stephens T, Dulberg C, Joubert N: La santé mentale de la population canadienne: une analyse exhaustive. Maladies chroniques au Canada. 1999, 20: 118-126.Google Scholar
- Statistics Canada: Health indicator profile, two year period estimates, by age group and sex, Canada, provinces, territories, health regions (2011 boundaries) and peer groups. 2011, CANSIM, Table 105–0502Google Scholar
- Beck CA, Patten SB, Williams JVA, Wang JL, Currie SR, Maxwell CJ, El-Guebaly N: Antidepressant utilization in Canada. Soc Psychiatry Pshyciatr Epidemiol. 2005, 40: 799-807. 10.1007/s00127-005-0968-0.View ArticleGoogle Scholar
- MacLellan DL, Taylor JP, Van Til L, Sweet L: Measured weights in PEI adults reveal higher than expected obesity rates. Revue canadienne de santé publique. 2004, 95 (3): 174-178.PubMedGoogle Scholar
- Booth FW, Chakravarthy MV: Cost and consequences of sedentary living: new battleground for an old enemy. President’s Council on Physical Fitness and Sports Research Digest. 2002, 3 (16):Google Scholar
- Crawley J, Pauler Ankerst D: Handbook of statistics in clinical oncology. 2006, Boca Raton: Taylor & Francis, 2Google Scholar
- Elgammal A, Duraiswami R, Davis LS: Efficient kernel density estimation using the fast gauss transform with applications to color modeling and tracking. IEEE Trans Pattern Anal Mach Intell. 2003, 25 (11): 1499-1504. 10.1109/TPAMI.2003.1240123.View ArticleGoogle Scholar
- Prescott RJ, Counsell CE, Gillespie WJ, Grant AM, Russell IT, Kiauka S, Colthart IR, Ross S, Shepherd SM, Russell D: Factors that limit the quality, number and progress of randomised controlled trials. Health Technol Assess. 1999, 3 (20):Google Scholar
- Ma W, Hu F, Zhang L: Testing hypotheses of covariate-adaptive randomized clinical trials. J Am Stat Assoc. 2014, In printGoogle Scholar
- Chambaz A, van der Laan MJ, Zheng W: U.C. Berkeley Division of Biostatistics Working Paper Series. Targeted Covariate-Adjusted Response-Adaptive LASSO-Based Randomized Controlled Trials. 2014, working paper 323Google Scholar
- Berger VW, Bears JD: When can a clinical trial be called ‘randomized’?. Vaccine. 2003, 21: 468-472. 10.1016/S0264-410X(02)00475-9.View ArticlePubMedGoogle Scholar
- Kotsiantis S, Kanellopoulos D: Discretization techniques: a recent survey. GESTS International Transactions on Computer Science and Engineering. 2006, 32 (1): 47-58.Google Scholar
- Stylios CD, Georgeopoulos PC, Groumpos PP: 2nd international conference in Fuzzy Logic and Technology. Decision Support System for Radiotherapy Based on Fuzzy Cognitive Maps. 2001Google Scholar
- Papageorgiou E, Spyridonos P, Glotsos D, Stylios C, Groumpos P, Nikiforidis G: Brain tumor characterization using the soft computing technique of fuzzy cognitive maps. Appl Soft Comput. 2008, 8: 820-828. 10.1016/j.asoc.2007.06.006.View ArticleGoogle Scholar
- Giles BG, Findlay CS, Haas G, LaFrance B, Laughing W, Pembleton S: Integrating convenetional science and aboriginal perspectives on diabetes using fuzzy cognitive maps. Soc Sci Med. 2007, 64: 562-576. 10.1016/j.socscimed.2006.09.007.View ArticlePubMedGoogle Scholar
- Bartholomew LK, Parcel GS, Kok G, Gottlieb NH, Fernández ME: Planning Health Promotion Programs: An Intervention Mapping Approach. 2011, San Francisco, CA: Jossey-BassGoogle Scholar
- Crutzen R, Peters G-JY, Abraham C: What about trialists sharing other study materials?. BMJ. 2012, 345: e8352-10.1136/bmj.e8352.View ArticlePubMedGoogle Scholar
- Peters G-JY, Abraham C, Crutzen R: Full disclosure: doing behavioural science necessitates sharing. The European Health Psychologist. 2012, 14: 77-84.Google Scholar
- Rathi V, Dzara K, Gross CP, Hrynaszkiewicz I, Joffe S, Krumholz HM, Strait KM, Ross JS: Sharing of clinical trial data among trialists: a cross sectional survey. BMJ. 2012, 345: e7570-10.1136/bmj.e7570.View ArticlePubMedPubMed CentralGoogle Scholar
- Whitby M, McLaws ML, Ross MW: Why healthcare workers don’t wash their hands: a behavioural explanation. Infect Control Hosp Epidemiol. 2006, 27 (5): 484-492. 10.1086/503335.View ArticlePubMedGoogle Scholar
- Penza-Clyve SM, Mansell C, McQuaid EL: Why don’t children take their asthma medications? A qualitative analysis of children’s perspectives on adherence. J Asthma. 2004, 41 (2): 189-197. 10.1081/JAS-120026076.View ArticlePubMedGoogle Scholar
- Berger VW, Ivanova A, Deloria-Knoll M: Minimizing predictability while retaining balance through the use of less restrictive randomization procedures. Stat Med. 2003, 22 (19): 3017-3028. 10.1002/sim.1538.View ArticlePubMedGoogle Scholar
- Berger VW: Selection bias and covariate imbalances in randomized clinical trials. 2005, Chichester: John Wiley & SonsView ArticleGoogle Scholar
- Giabbanelli PJ, Alimadad A, Dabbaghian V, Finegood DT: Modeling the influence of social networks and environment on energy balance and obesity. Journal of Computational Science. 2012, 3: 17-27. 10.1016/j.jocs.2012.01.004.View ArticleGoogle Scholar
- Malle BF: How people explain behavior: a new theoretical framework. Pers Soc Psychol Rev. 1999, 3 (1): 23-48. 10.1207/s15327957pspr0301_2.View ArticlePubMedGoogle Scholar
- Hagino A, Hamada C, Yoshimura I, Ohashi Y, Sakamoto J, Nakazato H: Statistical comparison of random allocation methods in cancer clinical trials. Control Clin Trials. 2004, 25 (6): 572-584. 10.1016/j.cct.2004.08.004.View ArticlePubMedGoogle Scholar
- Zhao W, Hill MD, Palesch Y: Statistical Methods in Medical Research. Minimal Sufficient Balance – a new Strategy to Balance Baseline Covariates and Preserve Randomness of Treatment Allocation. 2012Google Scholar
- Bonate PL: Clinical trial simulation in drug development. Pharm Res. 2000, 17: 252-256. 10.1023/A:1007548719885.View ArticlePubMedGoogle Scholar
- Dentler K, ten Teije A, Cornet R, de Keizer N: Towards the automated calculation of clinical quality indicators. Knowledge representation for health-care, Lecture Notes in Computer Science. 2012, 51-64. 6924View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/14/130/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.