ASC Conference, Imperial College London, 26th April 2000
Automatically Better?: The Impact of Automation on the Survey Process
The general theme of the conference, as elaborated on by Laurance Gerrard of TRBI Ltd in the first paper, was whether automation leads inevitably to improvement, and what problems can emerge as users move from traditional to computerized methods of survey research. In a general discussion of reasons to resist automation, 3 key points emerged:
Two papers stood out as particularly useful from the point of view of my own project, and they neatly encapsulated a number of problems faced by such projects. Joanne Lamb and Joan Fairgrieve from the CES at Edinburgh University spoke about a project called IQML. This is an EU-funded Fifth Framework collaborative project which is aiming to create software to generate metadata. Such metadata, it was argued, needs to be accessible since people increasingly want to reuse datasets, and also want this data via the Web. The speakers summed up some of the important requirements for such a project:
However, the project is still at a very early stage, having started in February 2000, with first outputs expected in July 2000, and the whole project running for three years.
In contrast, Neil Walker from the MRC Biostatistics Unit in Cambridge was able to present a working automatic tool for generating questionnaire documentation, called qgen (questionnaire generator). This uses the Data Archive's approved markup language for documentation - the Data Documentation Initiative's (DDI) DTD (Document Type Definition), version 1.0 of which was published in March. Whilst the output from qgen is impressive, it has only been used on small, one-off questionnaires, and did not provide any means of enabling users to update ongoing questionnaires on a frequent basis (such as will be essential for the FRS). In addition, the software will only be developed for the Linux environment.
Relevant websites
Data Documentation Initiative: http://www.icpsr.umich.edu/DDI/codebook.html
qgen information: http://www.mrc-bsu.cam.ac.uk/qgen/
IQML project information: http://www.cordis.lu/ist/projects/99-10338.htm
The other papers covered a range of issues related to automation of the survey process. Sally Gale of the ONS described the development of a system for managing information about field researchers, in order to acquire better information on cost, efficiency and quality of the research team. Phil Hearn of MRD Consultants discussed the usefulness of Microsoft products, in particular Excel, in reducing some of the more mundane tasks carried out by skilled staff. Stephen Jenkins of Mercator Software borrowed the OO paradigm from software development to discuss ways in which components of survey design can be reused across the survey process. Most fascinating in the exhibition was a palmtop computer running a Windows based survey questionnaire, although the software can currently only cope with very small surveys of up to 50 questions.
Although it was interesting to hear some of the broader issues discussed, without doubt the most useful aspects of the conference were, obviously, those related to automating documentation. It was helpful to see two different approaches to similar issues contrasted, and see that usable standards are now emerging. The general point that a non-expert user can be left more confused by an automated system was well-recognized.