Prediction of two month modified Rankin Scale with an ordinal prediction model in patients with aneurysmal subarachnoid haemorrhage
© Risselada et al; licensee BioMed Central Ltd. 2010
Received: 16 May 2010
Accepted: 29 September 2010
Published: 29 September 2010
Aneurysmal subarachnoid haemorrhage (aSAH) is a devastating event with a frequently disabling outcome. Our aim was to develop a prognostic model to predict an ordinal clinical outcome at two months in patients with aSAH.
We studied patients enrolled in the International Subarachnoid Aneurysm Trial (ISAT), a randomized multicentre trial to compare coiling and clipping in aSAH patients.
Several models were explored to estimate a patient's outcome according to the modified Rankin Scale (mRS) at two months after aSAH. Our final model was validated internally with bootstrapping techniques.
The study population comprised of 2,128 patients of whom 159 patients died within 2 months (8%). Multivariable proportional odds analysis identified World Federation of Neurosurgical Societies (WFNS) grade as the most important predictor, followed by age, sex, lumen size of the aneurysm, Fisher grade, vasospasm on angiography, and treatment modality. The model discriminated moderately between those with poor and good mRS scores (c statistic = 0.65), with minor optimism according to bootstrap re-sampling (optimism corrected c statistic = 0.64).
We presented a calibrated and internally validated ordinal prognostic model to predict two month mRS in aSAH patients who survived the early stage up till a treatment decision. Although generalizability of the model is limited due to the selected population in which it was developed, this model could eventually be used to support clinical decision making after external validation.
International Standard Randomised Controlled Trial, Number ISRCTN49866681
Prediction research typically aims to predict outcome of individual patients after the onset of a certain disease, using prognostic models. These models, preferably based on data directly available at hospital admission, are essential to support clinical decision making, and to facilitate reliable comparison of outcomes between different patient series and variation in results over time. Furthermore, prognostic models have an important role in randomized controlled trials (RCT), for stratification  and statistical analyses that explicitly consider prognostic information, such as covariate adjustment [2, 3], and may provide realistic and evidence-based expectations to relatives.
The majority of published prognostic models predicts a binary outcome, such as case-fatality using binary logistic regression [4–7]. Also, outcomes at ordinal scales are often considered as a dichotomized variable. However, there are several objections against collapsing an ordinal outcome scale into a binary one. First, the cut off for dichotomisation is arbitrary and may vary over studies in a single medical field [4, 5, 7]. Secondly, from a statistical perspective dichotomisation is a waste of information and reduces statistical power for the analysis of treatment effects or other covariates of interest [8, 9]. Furthermore, from a clinical point of view dichotomisation may lead to less useful models. For example, for a patient with a minor stroke a model predicting survival versus mortality is of limited value since the risk is low, while a prediction of complete recovery versus some remaining symptoms may be very useful. For a patient with a severe stroke, this will be the other way around.
An alternative for dichotomisation is application of a statistical approach that uses the full ordinal outcome scale. This leads to efficient use of the data and clinically relevant predictions. Several of these approaches for modelling ordinal response variables have been proposed, including proportional odds (PO) logistic regression, multinomial (or polytomous) logistic regression, or simple linear regression . Each of these methods has its pros and cons.
Definition of the modified Rankin scale
No symptoms at all
No significant disability despite symptoms; able to carry out all usual duties and activities
Slight disability; unable to carry out all previous activities, but able to look after own affairs without assistance
Moderate disability; requiring some help, but able to walk without assistance
Moderately severe disability; unable to walk without assistance and unable to attend to own bodily needs without assistance
Severe disability; bedridden, incontinent and requiring constant nursing care and attention
Data were collected prospectively by the Medical Research Council funded International Subarachnoid Aneurysm Trial (ISAT) (International Standard Randomised Controlled Trial, Number ISRCTN49866681). All centres obtained local ethics or institutional review board consent before enrolling patients (see Appendix 1). Able patients provided written informed consent. However, some ethics committees allowed assent from relatives to enable patients who could not give their own written consent to be enrolled in the trial. Full details of ISAT are available elsewhere . The aim of the trial was to determine whether treatment using endovascular coiling reduced the risk of patients being dependent or dead at one year by 25 percent (as defined by modified Rankin Scale grade 3-6) when compared with neurosurgical treatment (clipping) for that cohort of patients.
Predictors and outcome
We considered all patient characteristics that could be obtained easily and reliably within the first hours after hospital admission and that were also present in the ISAT database. These included age, gender, previous occurrence of SAH, CT scan Fisher grading, World Federation of Neurosurgical Societies (WFNS) grading, number of intracranial aneurysms, location of the aneurysm, maximum lumen size of the aneurysm, vasospasm on angiography, and intended treatment at randomization. Fisher grading of blood visible on a plain CT scan runs from grade 1 ("no blood visible") up to grade 4 ("intraventricular or intraparenchymal blood"). WFNS scale runs from grade 1 ("Glasgow Coma Scale (GCS) 15 and no motor deficit") to grade 5 ("GCS 3-6 with or without motor deficit"). One additional grade was created in ISAT for those in whom WFNS could not be assessed; 'grade 6'. The number of aneurysms was dichotomized into one or more than one intracranial aneurysms. Four aneurysm locations were distinguished: Anterior Cerebral Artery (ACA), Internal Carotid Artery (ICA), Middle Cerebral Artery (MCA), and Posterior Circulation (PC). The maximum lumen size of the aneurysm was expressed in millimetres. Vasospasm was examined on angiography and dichotomized into 'absent' or 'present'. Treatment was either neurosurgical clipping or endovascular coiling; we used treatment as allocated by the randomization procedure. The outcome measure in our study was the modified Rankin Scale (mRS) at two months (table 1) .
We started the development of the model discarding patients without information on outcome. The few missing values in predictors were imputed by means of single imputation (SI, in R language: aregImpute, n.impute = 1, type = 'pmm').
A simple approach to analyze an ordinal outcome, such as the mRS, is to dichotomize the outcome variable by one of several possible cut off points, e.g. 01 vs. 23456 , 012 vs. 3456 [4, 15], 0123 vs. 456 , and 012345 vs. 6 (case-fatality) . We applied binary logistic regression to develop models for these dichotomized responses. Next, we addressed the two main aspects of our ordinal outcome; the fact that it contains order and separate categories. A simple solution for modelling order, while neglecting the categorised nature of our outcome, is to apply linear regression using ordinary least squares. For the opposite - modelling categories, while neglecting order - we used multinomial regression. A more sophisticated approach is to use a proportional odds (PO) model. Such a model takes both order and separate categories into account. The PO logistic model is a rather straightforward extension of binary logistic regression . A common set of regression coefficients is assumed across all levels of the outcome, and intercepts are estimated for each level. The advantage of the PO model is its parsimony in dealing with an ordered outcome. The price we pay is the assumption of proportionality of the odds. This assumption is equivalent to saying that any cut-point on the outcome scale would lead to the same (binary) logistic regression coefficient .
We inspected proportionality by studying the univariate odds ratios for each cut off for each predictor. We plotted the score residuals of binary logistic models for each potential predictor separately. The trend of the score components against the levels of the outcome scale should be flat if the proportional odds assumption holds . When the PO assumption is not fulfilled for all potential predictors, we could also investigate a further alternative model: the partial PO model .
The association between predictors and outcome is expressed as odds ratios (OR). Predictors have statistically significant effects when the 95% confidence interval does not include the value one.
A multivariable PO model was developed containing predictors that met Akaike's Information Criterion (AIC) in a backward stepwise procedure . AIC compares models based on how well they fit the data, but penalizes for the complexity of the model. AIC requires that the increase in model χ2 when entering a new predictor has to be larger than two times the degrees of freedom: χ2 >2 df. When considering a predictor with 1 df, such as gender, this implies that χ2 has to exceed 2, equivalent to p < 0.157. When considering a predictor with 2 df, χ2 >4, or p < 0.135; and in case of 4 df, χ2 >8, or p < 0.092 .
The performance of the final PO model was assessed with respect to calibration and discrimination. Calibration is the ability of the model to produce unbiased estimates of the probability of the outcome. Calibration was tested with a goodness of fit test, which assesses agreement between predicted and observed risks over the full range of predicted probabilities. Discrimination is the model's ability to separate patients with different outcomes. To quantify the discrimination, we used the c statistic. A model with a c statistic of 0.5 has no discriminative power at all, for example a coin flip. A c statistic of 1.0 reflects perfect discrimination.
The performance of a prediction model is generally worse in new patients then initially expected. This 'optimism' of the original model can be studied with internal validation techniques . Internal validity of the models was assessed with standard bootstrapping procedures. Bootstrapping involves drawing samples of patients with replacement from the development population. Each sample can be considered as if one is repeating the data collection with the same number of patients and under identical circumstances as the original. Regression models were estimated in 300 bootstrap samples. Each of these 300 models was evaluated on the original sample. The average difference in the c statistic was determined to indicate the optimism in the initially estimated discriminative ability . A shrinkage factor was estimated from the bootstrap validation procedure and we shrunk the regression coefficients to provide better predictions for future patients .
All statistical analyses were performed using R software, version 2.8.1 (R Foundation for Statistical Computing, Vienna, Austria).
A total of 2,143 patients were recruited to the ISAT trial by 43 neurosurgical centres, mainly in Europe. We excluded 15 patients with missing information on the two month mRS. Fisher grade of 14 patients was not available and in one patient no information on vasospasm was available. We statistically imputed these missing values, leaving 2,128 patients for analysis, of whom 347 were in mRS grade 0 (16%), 583 in mRS grade 1 (27%), 528 in mRS grade 2 (25%), 296 in mRS grade 3 (14%), 80 in mRS grade 4 (4%), 135 in mRS grade 5 (6%) at the two month assessment, and of whom 159 (8%) died before the two month assessment.
Univariate associations for different cut offs for mRS (odds ratios), the univariate PO estimate (odds ratios), and linear regression (linear regression coefficient)
age [10 years]
(not assessable) 6
number of IA
Final PO and linear models - shrunken estimates of OR and regression coefficients.
age [10 years]
(not assessable) 6
We developed and validated a prognostic proportional odds model to predict the risk of two month modified Rankin Scale in individual patients after aneurysmal subarachnoid haemorrhage. Predictions were based on characteristics that were collected in a large clinical trial and that are regularly readily available on admission to a neurological or neurosurgical unit. The c statistic was modest, indicating a mediocre ability to predict clinical outcome at the two month assessment.
The dependence of our proportional odds model on the assumption of proportionality should not be overstressed. The potential inaccuracy caused by mild violation of the PO assumption is likely less severe than would be the case in arbitrary dichotomisation of an ordinal outcome. Dichotomisation involves more loss of information . Probably one would prefer a "wrong, but useful" model, despite possibly violating some underlying model assumptions . Moreover, the PO model predicts the probability of being in each mRS level for each individual patient. This makes the model useful for all patients, regardless of severity.
Besides the PO model we explored several other models. The ordinary least squares model seemed to perform quite well (see table 3). Although the categorical nature of the outcome variable is neglected, the model seems to perform reasonable and may yield estimates of regression coefficients that are quite similar to the PO model. This model might suffice to gain insight in which predictors play an important role in this clinical question. On the contrary, the multinomial model and - to a lesser extent - the partial PO models suffer from highly limited interpretability and therefore usability. A plethora of coefficients is produced by these models. If one is very specifically interested in one outcome grade, the model might be of some use. In most cases, we consider a more pragmatic approach however preferable. There are many more potentially useful modelling techniques for ordinal outcomes [22, 23]. One such technique is the continuation ratio (CR) model, which has been said to be likely to fit ordinal responses when subjects have to 'pass through' one category to get to the next. For a worked example see a tutorial by Harrell et al. .
Several limitations of our analyses should be acknowledged. This study used data from one large trial on a selected population of patients in equipoise regarding treatment with either endovascular coiling or neurosurgical clipping, which limits generalizability. Nonetheless, according to a recently published paper, the ISAT population proved to be quite similar to the population admitted with an aSAH to neurosurgical units in the United Kingdom . The model may perform well in the development sample, but poorly when applied to other groups of patients, for example, a less strictly selected one. Validation of a prognostic model in independent patient series is considered an essential next step . However since large samples of systematically collected data on aSAH are sparse, assessment of external validity is difficult. For now the generalizability and overall validity of our model remains to be established. This will be a topic of future research.
Although our model represents knowledge obtained from 2,128 SAH patients, predictions for individual aSAH patients are always subject to uncertainty. The model makes certain structural assumptions and statistical interaction terms were not included. Hence, it is possible that specific patterns of risk factors are inadequately reflected in the model predictions. Therefore, predictions should be regarded with care and not directly be applied for treatment limiting decisions.
The modest performance of the presented model might potentially be improved by including neuro-imaging biomarkers other than lumen size, location, Fisher grade on plain CT scan, and vasospasm on angiography. Biomarkers regarding anatomy and morphology might be considered, as well as aneurysm characteristics obtained from three and four dimensional angiography [26, 27]. Performance may also be improved by inclusion of subsequent information obtained after admission, including temporal course, neuro-imaging at later time points, eventual rebleeding of the aneurysm, delayed ischemic deficit, and other parameters such as hydrocephalus. The objective of the present study, however, was to investigate prognostic models that predict two month mRS with predictors available at admission.
For scientific purposes, we chose to present the final ordinal model as a formula. To increase usability of the model in clinical practice, it could eventually also be presented as a score chart, giving points for the presence of each predictor. The predicted probabilities for each mRS level corresponding to a certain score can subsequently be read from a score plot. Another possibility is to present the model in an Excel sheet or e.g. as a PDA application.
We presented a calibrated and internally validated ordinal prognostic model for predicting two month outcome after aSAH. Although generalizability of the model is limited due to the selected population in which it was developed, this model could eventually be used to support clinical decision making after external validation in a clinical setting.
International Subarachnoid Aneurysm Trial, ISRCTN49866681
Ethics/IRB committees for the contributing centres for ISAT
Oxford (Centre 01) - Local Oxford Research Ethics Committee: COREC 94.039
Edinburgh (Centre 02) - LREC Western General, Lothian Health Board: 1702/93/5/42
Nottingham (Centre 04) - Queens Medical Centre LREC: IMH/LE July 95
Cardiff (Centre 05) - LREC Heath Park, Cardiff: 96/1470
Atkinson Morley (Centre 06) - LREC St George's Healthcare Trust: 96/33/7
Newcastle (Centre 07) - Newcastle & North Tyneside Health Authority: 96/173
Bristol (Centre 08) - Frenchay Healthcare Trust Ethics Committee: 96/70
Hurstwood Park (Centre 11) - East Unit REC, Mid Sussex NHS Trust: 04/97
Manchester RI (Centre 12) - Manchester Health Authority REC (Central): CM/97/068
Manchester, Salford (Centre 13) - Salford & Trafford REC: 97/074
King's, London (Centre 14) - King's College Hospital REC: 1997/0198
Birmingham (Centre 18)- Queen Elizabeth Hospital REC: 1998
Plymouth (Centre 21) - Plymouth REC, South West Devon Health Authority: 1127
Glasgow (Centre 22) - Southern General Hospital REC: EC/99/S/9
Queens Square (Centre 23) - NHNN Joint Medical Ethics: 98/N109
Liverpool (Centre 29) - South Sefton REC: EC.70.99.M
Cambridge (Centre 30)- Cambridge REC: 2000
Sheffield (Centre 32) - South Sheffield REC: SS99-164
Hull (Centre 33)- LREC Hull Royal Infirmary: 2000
Belfast (Centre 34) - Queens University Belfast REC: 140-00-MREC-98-5-73
Royal London (Centre 41) - East London & City HA, ELCHA REC: P/00/252/M
Oldchurch (Centre 42) - LREC (Barking & Havering): 64 (SC)
Freiburg (Centre 10) - Albert Ludwigs Universität Freiburg: 207/96
Mainz (Centre 20) - Universitatsklinik Mainz: Aug 98
Dresden (Centre 24) - Ethik Kommission Universitatsklinik Carl Gustav Carus: EK10199
Würzburg (Centre 27) - Ethik Kommission der Medizinischen Fakultät der Universität Würzburg: 53/99
Augsburg (Centre 28) - Krankenhaus Zweckverband Augsburg: Jan/99
Hamburg (Centre 44) - Ethik Kommission der arztekammer Hamburg: M-247/2000
Lausanne (Centre 25) - Commission d'Ethique de la Recherche de la Faculté de Medicine: F/15/99
Paris (Centre 35) Amiens (Centre 36) Lille (Centre 37) Montpellier (Centre 39)
Rennes (Centre 40) Sainte Anne (Centre 45) - Comité Consultatif de protection des personnes dans la recherche biomedicale éthique: 17 May 2000
Gothenburg (Centre 09) - Gothenburg University Hospital: 285-94
Uppsala (Centre 19) - Uppsala Universitet: 97446
Turku (Centre 16) - Joint Commission on Ethics of Turku University Central Hospital: 8/1997
Odense (Centre 26) - Scientific Ethics Committee of Vejle & Funen: 19990050
Prague (Centre 46) - Charles University Etika Komise: Dec 2001
Montreal (Centre 15) - CHUM Pavillion Notre Dame Ethics: 09/1997
Toronto (Centre 17) - Toronto Hospital Medical Research Directorate: 97/HO62
Baltimore (Centre 31) - Joint Committee on Clinical Investigation: 99-06-18-08
Perth (Centre 03) - Royal Perth Hospital Ethics Committee: 05/06/004/F57
Details of the Prognostic Model
The probability of an outcome (mRS level) within two months is calculated according to the logistic formula: 1/(1 + exp-LP). The linear predictor (LP) takes the form of LP = intercept + regression coefficients × predictor values.
LP for mRS = 0.250 [mRS ≥ 1] - 1.15 [mRS ≥ 2] - 2.25 [mRS ≥ 3] - 3.08 [mRS≥4] - 3.39 [mRS ≥ 5] - 4.11 [mRS ≥ 6] + 0.164 × age - 0.255 × [male] - 0.00445 × [Fisher grade II] + 0.245 × [Fisher grade III] + 0.404 × [Fisher grade IV] + 0.555 × [WFNS grade 2] + 0.889 × [WFNS grade 3] + 1.80 × [WFNS grade 4] + 2.07 × [WFNS grade 5] + 2.04 × [WFNS grade 6, 'not assessable'] + 0.0404 × lumen size of aneurysm + 0.307 × [vasospasm present]- 0.367 [coil].
Coding of the predictors was as follows: age in decades, lumen size in millimetres; all other predictors, 1 if true and 0 if false.
This study was performed within the scope of the @neurIST-project ('@neurIST, Integrated biomedical informatics for the management of cerebral aneurysms', http://www.aneurist.org), funded by the European Commission 6th Framework Programme. The International Subarachnoid Aneurysm Trial was supported by grants from: the Medical Research Council, UK; and Programme Hospitalier de Recherche Clinique 1998 of the French Ministry of Health (AOM 98150). It was sponsored by Assistance Publique, HÔpitaux de Paris (AP-HP); the Canadian Institutes of Health Research; and the Stroke Association of the UK for the Neuropsychological assessments.
- Machado SG, Murray GD, Teasdale GM: Evaluation of designs for clinical trials of neuroprotective agents in head injury. European Brain Injury Consortium. Journal of neurotrauma. 1999, 16 (12): 1131-8. 10.1089/neu.1999.16.1131.View ArticlePubMedGoogle Scholar
- Hernandez AV, Steyerberg EW, Butcher I, Mushkudiani N, Taylor GS, Murray GD, et al: Adjustment for strong predictors of outcome in traumatic brain injury trials: 25% reduction in sample size requirements in the IMPACT study. Journal of neurotrauma. 2006, 23 (9): 1295-303. 10.1089/neu.2006.23.1295.View ArticlePubMedGoogle Scholar
- Hernandez AV, Steyerberg EW, Taylor GS, Marmarou A, Habbema JD, Maas AI: Subgroup analysis and covariate adjustment in randomized clinical trials of traumatic brain injury: a systematic review. Neurosurgery. 2005, 57 (6): 1244-53. 10.1227/01.NEU.0000186039.57548.96. discussion-53View ArticlePubMedGoogle Scholar
- Eriksson M, Norrving B, Terent A, Stegmayr B: Functional outcome 3 months after stroke predicts long-term survival. Cerebrovascular diseases (Basel, Switzerland). 2008, 25 (5): 423-9. 10.1159/000121343.View ArticleGoogle Scholar
- Hacke W, Kaste M, Fieschi C, von Kummer R, Davalos A, Meier D, et al: Randomised double-blind placebo-controlled trial of thrombolytic therapy with intravenous alteplase in acute ischaemic stroke (ECASS II). Second European-Australasian Acute Stroke Study Investigators. Lancet. 1998, 352 (9136): 1245-51. 10.1016/S0140-6736(98)08020-9.View ArticlePubMedGoogle Scholar
- Risselada R, Lingsma HF, Bauer-Mehren A, Friedrich CM, Molyneux AJ, Kerr RS, et al: Prediction of 60 day case-fatality after aneurysmal subarachnoid haemorrhage: results from the International Subarachnoid Aneurysm Trial (ISAT). European journal of epidemiology. 2010, 261-6. 10.1007/s10654-010-9432-x. 4Google Scholar
- Starke RM, Komotar RJ, Otten ML, Schmidt JM, Fernandez LD, Rincon F, et al: Predicting long-term outcome in poor grade aneurysmal subarachnoid haemorrhage patients utilising the Glasgow Coma Scale. J Clin Neurosci. 2009, 26-31. 10.1016/j.jocn.2008.02.010. 1Google Scholar
- Roozenbeek B, Maas AI, Lingsma HF, Butcher I, Lu J, Marmarou A, et al: Baseline characteristics and statistical power in randomized controlled trials: selection, prognostic targeting, or covariate adjustment? Critical care medicine. 2009, 2683-90. --- Either ISSN or Journal title must be supplied.. 10Google Scholar
- Royston P, Altman DG, Sauerbrei W: Dichotomizing continuous predictors in multiple regression: a bad idea. Statistics in medicine. 2006, 25 (1): 127-41. 10.1002/sim.2331.View ArticlePubMedGoogle Scholar
- Steyerberg EW: Clinical Prediction Models. A Practical Approach to Development, Validation, and Updating. 2009, New York: SpringerGoogle Scholar
- Hop JW, Rinkel GJ, Algra A, van Gijn J: Case-fatality rates and functional outcome after subarachnoid hemorrhage: a systematic review. Stroke; a journal of cerebral circulation. 1997, 28 (3): 660-4.View ArticlePubMedGoogle Scholar
- van Gijn J, Kerr RS, Rinkel GJ: Subarachnoid haemorrhage. Lancet. 2007, 369 (9558): 306-18. 10.1016/S0140-6736(07)60153-6.View ArticlePubMedGoogle Scholar
- Hop JW, Rinkel GJ, Algra A, van Gijn J: Changes in functional outcome and quality of life in patients and caregivers after aneurysmal subarachnoid hemorrhage. Journal of neurosurgery. 2001, 95 (6): 957-63. 10.3171/jns.2001.95.6.0957.View ArticlePubMedGoogle Scholar
- Bonita R, Beaglehole R: Recovery of motor function after stroke. Stroke; a journal of cerebral circulation. 1988, 19 (12): 1497-500.View ArticlePubMedGoogle Scholar
- Molyneux A, Kerr R, Stratton I, Sandercock P, Clarke M, Shrimpton J, et al: International Subarachnoid Aneurysm Trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised trial. Lancet. 2002, 360 (9342): 1267-74. 10.1016/S0140-6736(02)11314-6.View ArticlePubMedGoogle Scholar
- McCullagh P: Regression models for ordinal data. J R Statis Soc Series B. 1980, 42: 109-42.Google Scholar
- Harrell FE, Margolis PA, Gove S, Mason KE, Mulholland EK, Lehmann D, et al: Development of a clinical prediction model for an ordinal outcome: the World Health Organization Multicentre Study of Clinical Signs and Etiological agents of Pneumonia, Sepsis and Meningitis in Young Infants. WHO/ARI Young Infant Multicentre Study Group. Statistics in medicine. 1998, 17 (8): 909-44.PubMedGoogle Scholar
- Peterson BL, Harrell FE: Partial proportional odds models for ordinal response variables. Appl Stat. 1990, 39: 205-17. 10.2307/2347760.View ArticleGoogle Scholar
- Ambler G, Brady AR, Royston P: Simplifying a prognostic model: a simulation study based on clinical data. Statistics in medicine. 2002, 21 (24): 3803-22. 10.1002/sim.1422.View ArticlePubMedGoogle Scholar
- Senn S, Julious S: Measurement in clinical trials: a neglected issue for statisticians? Statistics in medicine. 2009, 28 (26): 3189-209. --- Either ISSN or Journal title must be supplied..Google Scholar
- Box GEP: Robustness in the strategy of scientific model building. 1979, New York: Academic PressView ArticleGoogle Scholar
- Ananth CV, Kleinbaum DG: Regression models for ordinal responses: a review of methods and applications. International journal of epidemiology. 1997, 26 (6): 1323-33. 10.1093/ije/26.6.1323.View ArticlePubMedGoogle Scholar
- Yee TW: The VGAM Package for Categorical Data Analysis. Journal of Statistical Software. 2010, 2010 (10): 32-Google Scholar
- Langham J, Reeves BC, Lindsay KW, van der Meulen JH, Kirkpatrick PJ, Gholkar AR, et al: Variation in outcome after subarachnoid hemorrhage: a study of neurosurgical units in UK and Ireland. Stroke; a journal of cerebral circulation. 2009, 40 (1): 111-8.View ArticlePubMedGoogle Scholar
- Justice AC, Covinsky KE, Berlin JA: Assessing the generalizability of prognostic information. Annals of internal medicine. 1999, 130 (6): 515-24.View ArticlePubMedGoogle Scholar
- Hayakawa M, Katada K, Anno H, Imizu S, Hayashi J, Irie K, et al: CT angiography with electrocardiographically gated reconstruction for visualizing pulsation of intracranial aneurysms: identification of aneurysmal protuberance presumably associated with wall thinning. Ajnr. 2005, 26 (6): 1366-9.PubMedGoogle Scholar
- Zhang C, Villa-Uriol MC, De Craene M, Pozo J, Frangi A: Morphodynamic Analysis of Cerebral Aneurysm Pulsation From Time-Resolved Rotational Angiography. IEEE transactions on medical imaging. 2009Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/10/86/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.