Results (Table 2) show that even when the CATI response rate is calculated using the number of letters sent as the denominator, a probable underestimate given that some letters would be sent to ineligible households, it is higher at 30% (all households) than that obtained for similar water usage studies conducted by Sydney Water Corporation (SWC) in the same period (Sydney Water Corporation, personal communication). In a 2005 residential survey performed by SWC where response rates were calculated using the number of eligible households contacted by telephone as the denominator, 1600 households (or 18%) responded fully out of 9729 potential contacts drawn from the EWP, of which 778 were ineligible. This compares with a response rate of 39% (all households) obtained for this study when response rates were calculated in exactly the same way. In another SWC study, where the internal client database was used and letters were sent to householders offering them an incentive to undertake interviews in their home, the response rate was 24%. In this latter case, householders were required to contact the water authority by telephone if they wished to participate in the survey (Sydney Water Corporation, personal communication). Taken together these comparisons show that irrespective of the way in which the CATI response rate is calculated, response rates in this study are between 25% – 116% higher than those obtained for water authority initiated surveys. This suggests that the combination of strategies used in this study to maximise telephone survey response rates was effective and that these strategies should continue to be used.
A limitation of the study is that the effectiveness of each of the individual strategies employed could not be assessed. It is therefore possible that some of the strategies may have prevented the response rate from being even higher. We regard this as unlikely based on documented enhanced telephone response rates of a personally addressed introductory letter in advance of a telephone approach [1–4] and the short time (less than 3 weeks) between the letter and telephone contact. In addition, all comments about the letter if made by the householder on contact were positive. Also, contact rates increased progressively with the number of call attempts and the final telephone contact rate was high at 84%. Furthermore, results of the pilot study showed that rate of telephone contact was highest for the 6–9 pm time frame when the majority of telephone contact attempts were made. Despite recent lobbying for, and support of, "Do not call" registers in Australia, giving a strong indication of attitudes to unwanted phone calls, the high response rate relative to water authority studies also suggests that the establishment of University credentials and significance of the survey at the start of the telephone interview assisted to maximise response rates. Training of interviewers and establishment of credentials and survey significance are also practices documented as improving response rates [5].
The high telephone contact rate (84%) and low rate of invalid telephone numbers (4%) validate the accuracy of the EWP. However, it is the decreasing coverage of the EWP that is of concern for future studies. Although it is difficult to obtain information from telecommunication service providers about the rate of telephone listing in directories, it is important that such rates are monitored so that recruitment strategies can be adjusted accordingly. The 45% of households without a listed telephone number in this study compares with a reported estimate of 13% in 1996 [7], demonstrating the impact of changing patterns of telephone ownership and the increasing availability of new technology.
When the percentage of target households with listed telephone numbers (57%) is combined with an estimated coverage rate of the filtered Australian Electoral Commission (AEC) database (13,940 out of 16,000 target households), the overall coverage of the combined AEC and EWP sampling frames for this study approximates at 50%. Despite this selection of only one half of the target households using the combined AEC and EWP sampling frame, we believe the strategy employed nonetheless maximised the number of completed CATIs compared with alternatives. In the absence of name and address details it is not possible to use the EWP to locate target households nor is there an ability to send a primer letter about the study in advance of a telephone approach. AEC records and client databases are alternative means of obtaining name and address information. However, use of water authority client records by independent researchers is provisional upon privacy provisions to householders being met and ethics approval being given for the water authority to provide this information. Random digit dialling, another possible alternative to select households for telephone surveys does not permit the sending of a primer letter and household selection is problematic. For example, the 16,000 target households in this study were covered by four telephone exchanges and at least 11 non-sequential sets of telephone numbers. In addition, use of telephone numbers for other telecommunication options (e.g. internet or fax) makes it more difficult to select a household from a random digit sampling frame [8]. The use of AEC records alone to administer a postal survey was not contemplated because of the number of survey questions and low participation rates of postal surveys.
The ability to select an average of 55% of households for telephone interview based on their listing in the EWP is a limitation of the selection and recruitment strategy employed leading to questions about the representativeness of the sample households. While it is possible that the water-using behaviour of listed and non-listed households in the EWP does vary, we consider that EWP listing is unlikely to be a primary determinant in the volume of water used by households. The relative uniformity in household characteristics that drive water usage such as garden area, garden age and household size in the entire target survey area independent of EWP listing supports this assumption. However, water usage by households within each of the target and control areas may not be homogenous. Those households that completed the CATI may have different water usage to those refusing to complete the CATI based on a greater commitment to water recycling schemes or water conservation. This bias is unavoidable and applies to all surveys whether administered by post, personal interview or the Internet.
A possible emerging alternative to telephone surveys is the use of the Internet. However, despite increasing access of the Australian population to the Internet at home (60% in 2006) [9], a relatively small proportion of the population have registered in on-line databases. Such databases thus currently provide a smaller available sample for survey purposes than do telephone databases. The utility of Internet databases to produce a large enough sample is further eroded when 'niche' households in a small number of suburbs are to be selected as in this study. It is not until the diminishment in telephone directory listings, coupled with increasing resistance to unwanted phone calls, reaches a point of 'cross over' with an increase in Internet access and database registration, that on-line surveys, rather than telephone surveys, will become the preferred survey method. In the interim period prior to this 'cross over' it is important that telephone surveys are supported by good strategies to maximise survey participation. Continued monitoring of both the rate of household listings in telephone directories and registration in on-line databases is also required so this point of 'cross over' is known.