A randomised trial was conducted as part of the third and final pilot survey for Wave 1 of the Medicine in Australia: Balancing Employment and Life (MABEL) longitudinal cohort/panel study of the dynamics of the medical labour market in Australia, focusing on workforce participation and its determinants among Australian doctors . The first wave of data collection, establishing the baseline cohort for the study, was undertaken in 2008.
The questionnaire included eight sections: job satisfaction, attitudes to work and intentions to quit or change hours worked; a discrete choice experiment (DCE) examining preferences and trade-offs for different types of jobs; characteristics of work setting (public/private, hospital, private practice); workload (hours worked, on-call arrangements, number of patients seen, fees charged); finances (income, income sources, superannuation); geographic location; demographics (including specialty, qualifications, residency); and family circumstances (partner and children). There were four versions of the survey, each differing slightly in order to tailor them to the type of doctor: GPs, specialists, specialists-in-training, and hospital non-specialists. Although survey length also matters for response rates, the context of the survey and research questions being tested required a long questionnaire, to ensure that sufficient data were collected to adequately test study hypotheses . The length ranged from 58 questions in an eight-page booklet (for specialists-in-training), to 87 questions in a 13-page booklet (for specialists). In all modes, doctors in remote and rural areas (defined using the Rural, Remote and Metropolitan Area (RRMA) classification to doctors in RRMA 6 (Remote centre with population > 5,000) and RRMA7 (Other remote centre with population < 5,000)), mainly GPs, were given a cheque for $100 that was enclosed with the invite letter to recognise both their importance from a policy perspective, and the significant time pressures on these doctors. The purpose was to draw meaningful inferences about recruitment and retention in rural and remote areas. Pre-paid monetary incentives, not conditional on response, have been shown to double response rates . The survey described in this paper was also the main pilot survey for the main wave of MABEL, so it was important to pilot the administration of these incentives. However, they did not influence the outcome of this trial as randomisation ensured approximately equal numbers of cheques going out in each arm of the trial.
The process of logging in and completing the survey online was kept as simple as possible. Users were directed to the main web page (http://www.mabel.org.au), where they clicked on the 'Login' link, which directed them to a login page where they entered their username and password. They were then directed to the first page of the survey. Respondents could save their responses and logout, and then login again to complete the survey, and they could skip questions. Once logged in, the padlock icon was visible, indicating that the website was secure.
The primary outcomes of interest in the trial were response rates, survey response bias with respect to age, gender, doctor type and geographic location, and item-response (the percentage of completed items). A three-arm parallel trial design was used with equal randomisation across arms. The sample size for the trial was calculated to detect a difference of 5% in the response rate at the 95% level of statistical significance, and with a power of 80%. This indicated that a sample of 900 doctors in each arm of the trial would be required, 2,700 doctors in total. This represented just under 5% (2700/54,160 = 0.04985) of all doctors undertaking clinical practice on the Australian Medical Publishing Company's (AMPCo) Medical Directory, which includes all doctors in all States and Territories of Australia and formed our sampling frame. This national database is used extensively for mailing purposes (e.g. the Medical Journal of Australia). The Directory is updated regularly using a number of sources. AMPCo receives 58,000 updates to doctors' details per year, through biannual telephone surveys, and checks medical registration board lists, Australian Medical Association membership lists and Medical Journal of Australia subscription lists to maintain accuracy. The directory contains a number of key characteristics that can be used for checking the representativeness of the sample and to adjust for any response bias in sample weighting. These characteristics include age, gender, location, and job description (used to group doctors into the four types).
A 4.9% stratified random sample of doctors was therefore taken, with stratification by four doctor types (general practitioners (GPs), specialists, doctors enrolled in a specialist training program, and non-specialist hospital doctors (including interns and salaried medical officers)), and six rural/remoteness categories (Rural, Remote and Metropolitan Area (RRMA) classification). This produced a list of 2,702 doctors. Doctors in this sample were then randomly allocated to a response mode by AS using random numbers generated in STATA. The AMPCo unique identifiers for each of the three groups were sent to AMPCo who conducted the mailing of invitation letters, and survey materials were mailed in late February 2008. Survey invitation letters indicated the University of Melbourne and Monash University as responsible for the survey. AMPCo also provided individual-level data on the population of doctors so we could examine response bias. Doctors were aware of which mode they had been allocated to on receipt of the invitation letter. The survey manager (AL) recorded responses and organised data entry and was blinded to group allocation. SH analysed the data and was not blinded to group allocation. Analysis was by 'intention to treat'.
Analysis included comparisons of response rates; estimation of means and proportions of respondents by age, gender, doctor type, and geographic location compared to the doctor population; logistic regression of response bias; and comparisons of the proportion of missing values (item non-response). The statistical significance of the differences between the response rates of the three response modes was analysed using a probit model with response (0/1) as the dependent variable and two dummy variables for response mode, with online as the reference category. The difference between the sequential and simultaneous modes was tested using the restriction that their coefficients be equal. Although respondents were randomly allocated across modes, it is still important to test whether any specific/particular respondent characteristics influenced the response rate. The probit model therefore included age, gender, doctor type and geographic location. Survey response bias was examined using a multinomial logit model of respondents (= 1) and the total population of doctors (= 0), with age, gender, geographic location and doctor type as independent variables. For item non-response, a comparison of the proportion of completed items was supplemented using generalised linear models that controlled for differences due to age, gender, geographic location and doctor type . Analysis of geographic location was based on the Australian Standard Geographic Classification (AGSC) Accessibility and Remoteness Index for Australia (ARIA).
The economic evaluation compared the costs of consumables (rental of AMPCo list, printing of surveys, letters, fax forms, further information fliers and reply-paid envelopes, mail-house processing costs, postage and data entry) across each mode of survey administration. The costs of researcher and staff time were the same for each mode, as each mode required the development of both the paper and web survey, and time liaising with AMPCo and the printers. These costs were therefore not included in the comparison of costs between modes. The expected costs for the main wave 1 survey were estimated based on sending out a survey to all doctors on the AMPCo database (n = 54,168) and using the response rates from the randomised trial to estimate the number of respondents. Data on the three primary outcome measures are presented alongside data on costs.