’Demonstration projects’ are used to find out whether a proposed policy or practice can effectively address an identified problem. They form part of what Karl Popper termed ’piecemeal social engineering’ (Popper, 1945, 1957). The idea is that before embarking on widespread and expensive changes, organizations-in Popper’s case governments- should try out innovations and examine whether they work. Popper also hoped that piecemeal social engineering could be a useful tool for social scientists to test their hypotheses (Popper, 1957).

The issue of replication is crucial for demonstration projects. A demonstration project can only usefully inform policy and practice if its results are replicable. To the social scientist the replication of findings can be an important indicator as to their generalizability. Moreover, where there are scientific disputes about issues, the route to resolution is often seen to lie in replications of contentious studies. This article begins by highlighting a number of theoretical and practical problems in relation to the nature, conduct and evaluation of replications, and then goes on to suggest how they might be resolved. The argument is developed through detailed discussion of a highly influential British burglary prevention project, and efforts to replicate it within a national crime prevention program. The demonstration project in question is called ’The Kirkholt Burglary Prevention Project’, and the national program, ’The Safer Cities Programme’. The first phase of the Safer Cities Programme was run by the Home Office between 1988 and 1995. It operated in 20 cities, each of which had £250,000 per annum to spend on efforts to reduce local crime problems. Sections 2 and 3 of the article discuss Kirkholt and some candidate replications of it within Safer Cities. Referring to the examples outlined, Section 4 considers critically various ways of construing replication, and advocates a scientific realist approach. Section 5 concludes with general comments on the conduct of evaluations of demonstration projects and of their replications.

The recorded crime rates for the years before and after the implementation of the Kirkholt Burglary Prevention Project indicated dramatic and sustained falls in burglary. Using March to February figures, in 1986/7-the year before the project-there were 526 burglaries. The corresponding figures for succeeding years are given in the 1990 report of the project. There were 223 burglaries in 1987/8, 167 in 1988/9 and 132 in 1989/90 (Forrester et al., 1990). These represent a fall from approximately 25 to 6 percent per annum of the 2280 households on the estate. Moreover, the probability of reburglary amongst those already victimized was reduced from four times the expected rate to zero in the first 7 months of the project. Clearly practice and policy interest in replication follows in this case from the apparent success of Kirkholt.

Controversy, however, has surrounded interpretation of the ’success’ of the scheme. It has been argued that other estate improvements undertaken by the local authority could have been responsible (Safe Neighbourhoods Unit, 1993). Farrington (1992) has attempted independently to referee between the two interpretations of the burglary reduction by reanalysing data from Kirkholt itself. Replications clearly offer another and classic way of arbitration. Moreover, unlike the Safe Neighbourhoods Unit and Farrington discussions, which are concerned with internal validity (the nature of the causal relationships within the project), replications attend also to the issue of external validity (the reproducibility of these causal relationships elsewhere). It is the latter which is crucial for practitioners, policy-makers and social scientists. It was decided, therefore, to look at replications of the Kirkholt project within the Safer Cities Programme. The first task was clearly to identify a sample of replications. Difficulties in doing so turned out to be not so much technical as conceptual-they begin to highlight methodological issues.