The Evaluation of Community Economic Development Initiatives

Community economic development (CED) initiatives expanded rapidly in urban and regional policy in the 1990s. Traditional evaluation methodology has, however, proved to be extremely difficult to apply effectively to CED. This paper examines existing monitoring and evaluation procedures for CED, the problems faced in applying traditional methodology and possible ways forward to ensure that more effective monitoring and evaluation can be undertaken. Evidence is drawn from four detailed case studies of urban CED projects funded as part of the 1994-96 Yorkshire and the Humber Objective 2 Structural Funds programme. These case studies were deliberately designed to experiment with alternative quantitative and qualitative evaluation methods.


Introduction
This paper examines the suitability of the traditional evaluation methodology for community economic development (CED) initiatives. As a vehicle for examining the issues, evidence is drawn from detailed case studies of four CED projects-an environmental project, a childcare scheme, a development trust and a credit union. These projects were undertaken as part the 1994-96 Yorkshire and Humber Objective 2 Structural Funds programme, spending on which continued until the end of 1998. The case studies were deliberately included within the formal ex post evaluation as a means of experimenting with new approaches to CED evaluation (Government Of ce for Yorkshire and the Humber, 1999a). All four projects were urban projects, as was typical of most CED projects in the programme. This re ects the urban concentration of socially excluded communities in the UK (Green, 1997).
The paper begins (section 2) with a brief description of the 1994-96 Yorkshire and Humber Objective 2 programme and the position of CED within it. This is followed in section 3 by a review of current evaluation H. W. Armstrong and B. Kehrer are in the Department of Geography, University of Shef eld, Winter Street, Shef eld, S10 2TN, UK. Fax: 0114 279 7912. E-mail: H.Armstrong@shef eld.ac.u k and ggp97bk@shef eld.ac.uk . P. Wells is in the Policy Research Institute, Leeds Metropolita n University, Bronte Hall, Beckett Park Campus, Leeds, LS6 3QS, UK. Fax: 0113 283 1748. E-mail: pri@lum.ac.uk . A. M. Wood is in the Department of Geography, University of Oklahoma, 100 East Boyd Street, Norman, OK 73019 1007, USA. Fax: 405 325 6090. E-mail: amwood@ou.edu . The case studies on which this paper is based were conducte d as part of a Department for the Environment, Transport and the Regions funded evaluation of the 1994-99 Yorkshire and Humber Objective 2 programme. The paper has been publishe d with their agreement. The wider evaluation research team comprised Ekos Ltd, Milburn Trinnaman La Court (MTL) and researchers drawn from Hull University (Department of Geography), Leeds Metropolita n University (Policy Research Institute), Shef eld Hallam University (Policy Research Centre) and the University of Shef eld. The authors would like to acknowledg e the assistance of Janet Sharpe (Unit Manager, Capital Resource Management Team, Shef eld City Council), Irene Hudson (Holmewood Consortium) and the other project managers of all four case-study projects, together with Isobel Mills (Government Of ce for Yorkshire and the Humber) and other members of the evaluatio n Steering Group. All opinion s expressed in the paper are, however, solely those of the authors, as is responsibilit y for any errors or misunderstandings . methodology. Section 4 presents the results of the four case studies. The paper concludes with suggestions for the future monitoring and evaluation of CED.

CED in the 1994-96 Yorkshire and Humber Objective 2 Programme
The European Union (EU) has been involved in regional policy initiatives in the Yorkshire and Humber region of the UK since the origins of the European Regional Development Fund (ERDF) in 1975. Large-scale EU activity in the region, as elsewhere in the UK, really only began in earnest with the major reforms of the ERDF and other Structural Funds phased in between 1989and 1993. The 1989 programme, as with earlier EU interventions in regional policy in the region, tended to focus on traditional types of regional policy initiative (for example, support for infrastructure projects, business support measures for small and medium-sized rms, measures to assist in introducing new technology).
CED initiatives are a radical new type of policy initiative. They have their origins in urban policy in the UK in the late 1980s and early 1990s. They represent an attempt to draw local communities much more closely into policies designed to regenerate economic activity in their neighbourhoods . CED was, however, introduced to the Structural Funds only in late 1994. Since then, it has rapidly expanded, mirroring a similar trend in UK domestic government policies (for example, New Deal for the Communities, and recent rounds of the Single Regeneration Budget-SRB).
The initial impetus for CED in the Structural Funds came from debates on social and economic exclusion in the Delors White Paper (Commission of the European Communities, 1993). These were reinforced by the Essen and Madrid Presidencies which made job creation the paramount priority of the Structural Funds. The importance of employment initiatives was enhanced by a report on employment in 'third-sector' activities (European Commission, 1995) and by the 1997 Amsterdam Treaty.
The UK regions have had to cope with a steep learning curve for CED. Guidelines were, however, eventually provided by the European Commission (1996). These identi ed two elements of CED. First, it should be a process, "the end point of which is sustainable development and reconversion in spatially targeted areas" (European Commission, 1996, p. 19). Above all, this process is based on capacity building which aims to release "forces for reconversion and development at the local level and facilitate the wider mobilisation of social capital" (p. 19). The second part of CED is community linking: the process by which communities become reinserted back into the mainstream economy. In common with CED in the UK, the Commission's model envisages an areabased approach in which policy initiatives release latent community energies through a combination of partnerships and the local 'ownership' of the strategy.
A serious problem with all CED is that any given designated area may contain several social, ethnic or demographic communities. Thus, one challenge for CED is to mobilise different interest-groups to tackle the problems within their area, whilst recognising that different communities will rarely be wholly enclosed by geographical boundaries. Identity is not necessarily a precursor for de ning a community in this context. Rather, 'identity' is something which could be developed by the process of CED itself.
It should be noted that the Structural Funds have deliberately been focused on a narrower range of types of help for socially excluded communities than other schemes such as SRB. The Structural Funds are concentrated on initiatives having the potential to generate economic bene ts in the form of jobs, new enterprises, etc. The Commission's goals are therefore more explicitly economic in nature (Lloyd and Ramsden, 1998).
The Yorkshire and Humber region is a large one (with a population of 5.4 million persons) in the north-east of England. It con-tains a diverse mixture of urban, rural, industrial and tourist sub-regions. The Objective 2 area of the region during the 1994-99 programme period comprised virtually the whole of South Yorkshire (including the city of Shef eld), together with large parts of West Yorkshire (although excluding its largest city, Leeds) and Humberside. The 1994-96 Yorkshire and Humber programme had 6 Priorities and 23 Measures (Government Of ce for Yorkshire and the Humber, 1995). CED was a free-standing Priority (Priority 6) which was allocated 17.4 per cent of the budget. Table 1 gives the amount of money made available for CED, together with the sums eventually committed to projects. In addition to £41 million of European Regional Development Fund (ERDF) and European Social Fund (ESF) money, an extra £65 million of matching funding was eventually found (almost all from the public sector, principally local authorities). Other British Objective 2 regions have also witnessed large and expanding CED Priorities (Government Of ce for Yorkshire and the Humber, 1999b, Table 6.2).
As Table 1 shows, CED spending was spread between four separate Measures. Measure 6.20 (Training and employment) was an ESF-funded Measure focusing on training and skills acquisition. Measure 6.21 (Support for community based economic projects) was the largest of the 4 Measures (both in terms of spending and the number of individual projects-111). Measure 6.21 also contained most of the types of CED of interest here, in the sense of posing new challenges for evaluation. Despite its name, in practice only a small part of Measure 6.21 help went directly to genuinely community businesses (Armstrong et al., 1999). Measures 6.22 (Targeted environmental improvements) and 6.23 (Access to work through improved public transport) were dominated by a few large infrastructure projects.
The designated CED areas were nominated by partners (particularly the local authorities) on the basis of a variety of indicators of exclusion and deprivation. The resulting areas encompassed 35 per cent of the Objective 2 region's population.
The 1994-96 programme sought to develop a more comprehensive set of monitoring indicators than in earlier programmes, together with signi cant improvements in evaluation methods. The Single Programming Document also incorporated a more extensive ex ante evaluation than had been the case prior to 1994. Subsequently, in 1997, the Programme Monitoring Committee (PMC) commissioned an unusually large interim evaluation (or 'thematic review '-Armstrong et al., 1998; Government Of ce for Yorkshire and the Humber, 1997).

Current Evaluation Methodology
The European Commission has developed a recommended system of evaluation for the Structural Funds (MEANS, 1995a(MEANS, , 1995b. This, like the UK government's domestic equivalent (HM Treasury, 1995), represents a distillation of 30 years of regional and urban policy evaluation research. The fundamental principles of regional policy evaluation command a wide consensus and, as a result, the Treasury and MEANS systems are extremely similar. While monitoring and evaluation procedures have been signi cantly strengthened in the 2000-06 programme period (European Commission, 1997), they remain rmly based on traditional evaluation principles.
The MEANS approach recommends a combination of 'top-down' and 'bottom-up' evaluation techniques (for cross-checking purposes). 'Top-down' methods analyse secondary data sources, while 'bottom-up' methods collect and analyse primary data through sample surveys of bene ciary businesses and individuals. 'Top-down' techniques seek to estimate 'global' impacts (for example, aggregate GDP increases, or employment), whereas the 'bottom-up' techniques proceed by identifying microeconomic impacts on individuals and rms. The 'micro' impacts are then aggregated to give global effects.
The origins of traditional evaluation methodology lie in 'top-down' techniques. Early regression-based methods (Moore and Rhodes, 1973) were quickly extended from job creation to other important impacts and from time-series to spatial cross-sectional data-sets (see Armstrong and Taylor, 2000, for a survey). Regression analysis, in more sophisticated forms, continues to be used (Williamson, 1998;Wren and Taylor, 1999). However, 'top-down' evaluations now also comprise sophisticated alternatives to regression analysis. Macroeconomic models estimate an array of impacts other than employment and also enable the system-wide effects of a policy to be traced. Unfortunately, they are also extremely demanding of data. It is for this reason that most Objective 2 regions use less sophisticated regional econometric models and input-output models. Even these are unavailable in the English regions.
In the English regions, 'top-down' evaluation is reduced to using very simple methods. Most regions have used baseline analysis as the main type of 'top-down' evaluation. This approach involves collecting detailed statistics on a range of economic and social indicators for the start year of the programme (hence 'baseline'). Evaluation then tracks changes from these baselines over the programme period in order to check whether improvements have occurred.
Baseline analysis is very useful in its own right and led to signi cant improvements in the analysis of economic trends within the Objective 2 areas in the 1990s. Baseline indicators can be disaggregated to the level of the sub-regions and local areas within the Objective 2 region. The region's performance can also be pro tably compared with other UK Objective 2 areas (Government Of ce for Yorkshire and the Humber, 1999b). Seasoned students of evaluation will, however, recognise how weak a technique baseline analysis is. The economic performance of a region is affected by many things, of which the Structural Funds represent only one. Many other things can change, such as international trading conditions or sudden sectoral shifts (for example, the recent rundown of the coal industry in South Yorkshire). Baseline analysis is fundamentally incapable of estimating the 'counterfactual': what would have happened to the region in the absence of the Structural Funds.
It should come as no surprise, therefore, to nd that 'bottom-up' methods have dominated recent Structural Funds evaluations. This is also true for evaluations of domestic regional and urban policy initiatives (see, for example, PA Cambridge Economic Consultants, 1993). 'Bottom-up' surveys seek information directly from the bene ciaries and contain four types of question: (1) Deadweight. Would the additional directly created jobs (or other forms of economic activity) have occurred irrespective of the Structural Funds assistance? (2) Displacement. Are the jobs created at the expense of jobs in competitor rms elsewhere in the Objective 2 area? (3) Supply chain effects. Do the expanding businesses buy signi cant amounts of their inputs from other local rms, creating knock-on bene ts elsewhere in the Objective 2 area? (4) Multiplier effects. To what extent do the extra jobs (via the wages paid) trigger multiplier bene ts elsewhere in the region?
While these four differing effects of the Structural Funds help have been described in terms of help for businesses (rather than trainees) and in respect of employment, the principles they embody are relevant to every other type of impact, whether economic, social or environmental in nature. The net impact is obtained by beginning with the gross impact, subtracting the deadweight and displacement effects, and adding back the supply chain and multiplier effects. Net impact is the key measure of success in traditional evaluation methodology since it incorporates both the counterfactual and system-wide impacts of a programme.
The MEANS methodology recommends that each Objective 2 region should conduct its own 'bottom-up' evaluation. This is because the crucial coef cients for deadweight, displacement, supply chain effects and multipliers can vary from one region to another.
The Commission has, however, sought to distil benchmark coef cients by trawling previous evaluations of the Structural Funds (Ekos, 1998). These benchmark values, such as the typical cost-per-job for a CED project, are valuable in their own right as summary measures of how effective the Structural Funds have been. They are also invaluable in ex ante evaluations where the necessary empirical evidence of outcomes does not, of course, exist. Benchmarking is not, however, a substitute in interim and ex post evaluations for the direct collection of data from bene ciary rms and individuals.
Perhaps the most important role of all for 'benchmark' estimates is that they permit comparisons of the relative effectiveness of CED to be made with other types of initiative (for example, inward investment, traditional business support schemes). Indeed, where they are in a suitable form (such as cost per net new job), they can also be compared with non-EU policies such as SRB or Regional Selective Assistance. This is important since it ensures that a further fundamental principle of evaluation can be attained: the estimation of the opportunity cost of a policy. The Structural Funds are nite-spending on one project is at the cost of spending elsewhere.
Application of traditional 'bottom-up' survey-based methods to the ERDF Measures in Priority 6 of the 1994-96 Yorkshire and the Humber programme gave estimates of only 1000-1600 gross new jobs and a mere 500-1100 net new jobs (Government Of ce for Yorkshire and the Humber, 1999a). Traditional methodology therefore appears at rst sight to produce meaningless results for CED. Such a conclusion would, however, be as unfair as taking the opposite stance: that they 'prove' how inadequate CED is. Neither is correct. The truth is much simpler: the newness of CED in the 1994-96 Structural Funds programmes meant that too little time had elapsed by the time of the evaluations for employment bene ts to accumulate.
Another fallacious argument is that the traditional evaluation methodology cannot be applied to CED because the methodology relies on quanti able indicators such as employment. This criticism rests on a misunderstanding of the nature of traditional evaluation methodology. Evaluation has never had a problem with qualitative indicators. Better examples of the genre have always incorporated qualitative measures (see, for example, King, 1990). The focus on quanti able measures in the past has arisen simply because the goals of regional policy have been almost wholly economic in nature and therefore amenable to quanti cation. The issue of the choice of appropriate indicators for CED (whether quantitative or qualitative) is, however, an important one and is carefully considered in this paper. The four case studies were deliberately used to experiment with a variety of modern methods of qualitative analysis.
Nor should we fall into the trap of thinking that the fundamental principles of evaluation are inappropriate for CED because they are somehow solely 'economic' in nature. It is certainly true that supply chains and multipliers are concepts relevant only to economic impacts. The same is not, however, true for deadweight, displacement and opportunity cost. These apply equally to social, environmental and political impacts of CED. Take, for example, capacity building. In evaluating a CED project it is surely important to seek answers to the following types of question. Would local community groups and networks have evolved anyway, irrespective of the CED project? Have the highly localised community networks stimulated by CED led to a diminution of family-centred links or contacts across the wider city? Has the CED project absorbed funds which might better have been spent elsewhere (on more traditional types of initiative), or perhaps even just handed over as a cash transfer to local residents? These are, respectively, deadweight, displacement and opportunity cost questions.
If arguments that traditional evaluation is solely quantitative and economic are incorrect, it is pertinent to ask in what ways CED poses new challenges for evaluation. A priori reasoning suggests that four characteristics of CED appear to pose unusually severe challenges: (1) CED has multiple objectives. These include social as well as the more traditional economic objectives.
(2) Multiple target-groups of bene ciaries exist. This is because CED areas typically contain different socially excluded groups (for example, single parents, ethnic minorities, the long-term unemployed).
(3) The bene ts of CED have unusually long lead times, particularly during the capacity building phase. The pressing need to develop suitable indicators of capacity building has attracted considerable research attention and is an issue examined in detail in the next section of this paper. Even more serious a problem for evaluation, however, where assessing the ultimate outputs and outcomes of CED must be the prime goal, are the exceptionally long lead times experienced by many types of CED. (4) CED areas typically suffer from 'multiple deprivation' (i.e. complex different barriers to social and economic inclusion). Hence, they attract the attention of many different public policies: there is complex interlocking of initiatives. This makes it hard to disentangle the impacts of the different policy interventions-a problem also known as the 'multiple counting of bene ts'.
It is interesting to note that the presence of multiple objectives (challenge (1) above) along with multiple bene ciary groups (challenge (2) above) means that single measures of impact (such as jobs) or deprivation indices (combining several measures) are unlikely to be satisfactory. The former cannot capture the multifaceted nature of CED objectives. The latter would require a system for weighting different indicators, something not possible without an accepted social welfare function, itself impracticable since different groups in the local communi-ties would attach different values to different outcomes.
In the next section, each of the four challenges identi ed above is systematically examined.

The Four Case Studies
In order to obtain maximum bene t from the case studies, attention was deliberately focused on those types of CED most likely to pose the greatest challenges for traditional evaluation methods. As noted earlier, the key challenges arise in projects with multiple objectives, extensive capacity building, long lead times before signi cant bene ts are observed, multiple funding streams and those closely intertwined with other regeneration policies such as the SRB. Case study projects were therefore selected on the basis of these characteristics. In addition, the selected casestudy projects had to meet the following requirements: (1) The project managers and local partnership had to be willing to participate. (2) Each project had to be reasonably representative of CED projects in the programme; they should not be prominent ' agship' projects. (3) The selected projects should cover, as far as was possible, a wide range of different types of CED (such as micro-nance, intermediate labour market schemes).
Three of the four case studies subsequently chosen were funded under Measure 6.21 and the fourth under Measure 6.22. Direct employment creation projects, such as subsidies for rms, were deliberately excluded on the grounds that their immediate economic bene ts are amenable to traditional evaluation techniques.

Case Study 1: NWICA Environmental Improvements, Shef eld
This project was located in the north-west inner-city area (NWICA) of Shef eld and was undertaken by Shef eld City Council.
Shef eld is located in the South Yorkshire sub-region of the Objective 2 area. NWICA lies within a belt of socially deprived localities to the North West of the city centre. The project involved a series of environmental improvements on council-owned land around and between housing. Seven targets were set for monitoring purposes. These proved to be typical of other CED projects in that they comprised a mixture of activity indicators (for example, 15 trainees; the establishment of a local community group), and outputs (for example, 15 jobs). As is common with CED, the majority of the target indicators measure activities (i.e. inputs and processes) rather than nal outputs such as jobs. Across the CED Priority as a whole, fewer than one in four targets were output indicators, the remainder being activity measures (Government Of ce for Yorkshire and the Humber, 1999a, Table  6.4). It is interesting to note that the list of targets contained qualitative indicators (such as 'improving' the community) as well as quantitative measures.
The case study comprised an analysis of the project's monitored targets, semistructured interviews with those involved in the delivery of the project, a review of supporting documentation (including local UR-BAN and SRB programmes), two focus groups involving local residents (designed to examine the process of community involvement in the area, as well as obtaining views on possible new monitoring methods) and a series of pilot surveys of local residents, businesses and property agents (estate agents, housing action trusts and the council lettings department). Formal questionnaires and semi-structured interviews were conducted with target bene ciary groups (residents and local businesses), using 'community interviewing' with local residents being trained to carry out the surveys. This method is beginning to be widely used in this type of research. It tends to give more accurate responses from local residents than is obtained by using 'outsider' professional interviewers.
Taking in turn each of the four challenges identi ed earlier: (1) Multiple objectives. The project at rst sight has a refreshingly simple environmental objective: to improve the green environment of areas close to social housing (i.e. built environment) being simultaneously renovated. The green environment objective was re ected in some of the targets (for example, hectares of land 'improved'). However, since the project also had the aim of "adding to the groundswell of activity in the area" (Business Plan), there were clearly other wider objectives too. These were not only economic (as shown by a job creation target), but social too (as shown by a crime reduction target). Multiple social, environmental and economic objectives therefore did exist. A problem with this project, and virtually all other CED projects, is that these wider objectives were not explicitly stated. This is a serious problem for evaluation since if the nal objectives are not clearly stated, progress towards them cannot be properly assessed.
(2) Multiple bene ciaries. The case study was able to identify no fewer than six target bene ciary groups: local residents, existing local businesses, visitors to the area, property agents (both public and private), community groups, and new rms entering the area. Most of these were surveyed, new rms and visitors being the only exceptions. The long list of target bene ciary groups is typical of CED. Moreover, NWICA is known to be typical of many inner-city localities in that it contains a mixture of socially excluded groups (JISER, 1997). Hence there are also separate sub-groups within the 'local residents' category.
It is not clear which bene ciary group forms the principal target of the project. In common with many CED projects, it is assumed that the bene ts will accrue to all of the groups simultaneously. In practice, the fact that the money is targeted at green areas close to the residential blocks, and the involvement of people in the local community group established by the capacity-building stage, strongly implies that local residents are the principal initial target. Other groups, however, are clearly intended to be longerterm bene ciaries.
(3) Capacity building and lead times. Capacity building is part of the process of creating 'social capital'. To economists, capital is an input, irrespective of whether one is considering physical capital, human capital or social capital. By contrast, many proponents of CED appear to consider social capital to be an output-a bene t in its own right for local residents. This difference of opinion is likely to become an important one in the future. In the event of CED initiatives failing to create many traditional economic bene ts via community linkage (such as permanent jobs), it is likely that proponents will argue that the creation of social capital will have been worthwhile in its own right.
Given its importance in CED, it is clear that new methods of measuring 'community capacity' are needed. Fortunately, this issue is now the subject of a lot of research. This has led locally to indicators of capacity building which quantify network ability (the degree to which local community groups come together to develop and deliver regeneration programmes), the extent of volunteering by local community members, the degree of democratic participation and representation within local organisations and the degree of involvement of local people in prior monitoring, representation and feedback (JISER, 1997).
Our own case study identi ed three further dimensions of capacity building: (1) the consultation process; (2) community-strengthening effects; (3) the role of community groups.
Taking each of these in turn, the effects of the project on the consultation process were sought using a combination of qualitative methods (i.e. surveys, interviews and a focus group) designed to obtain the opinions of local residents. These suggested that in analysing capacity building it is important to distinguish between awareness-raising, consultation prior to the project, and consultation during and after the project. Awareness-raising in NWICA had generally been more successful than subsequent consultation phases.
Turning to community-strengthening effects, previous research had successfully measured participation rates in ve types of network: family/community, employment, private service, public service and voluntary service (JISER, 1997). This work suggested that local networks of friends were as important as family links in NWICA. Moreover, unemployment tends to lead to exclusion from many networks.
In our own case-study work, again using a range of different qualitative methods, four further sets of questions were asked to ascertain community-strengthening impacts.
(1) Has the project had signi cant effects on the interaction between individuals? Responses suggested that activities other than the project had a more signi cant impact.
(2) Have the environmental improvements had an effect on how safe the residents feel? This question re ected one of the key target indicators of the project. The majority of responses were positive. (3) Has the project changed interaction with other residents? The majority of residents did not feel that the project had stimulated links between residents which were subsequently maintained after the initial consultation process. (4) Finally, residents were asked whether, even if their own involvement in the community had not increased, the involvement of others in the community had been. Respondents were aware of increased activity of residents associations and community groups. However, they felt that the actual project itself had only a minimal effect.
Finally, there is the issue of the role of community groups. NWICW has an Open Spaces Group of residents and a Ponderosa Environment Group (Ponderosa is an area of open space within NWICA). Both groups contain representation from an array of community groups, interest-groups, local agen-cies and the council. However, it was evident that the structure of groups and their relationships (for example, with Shef eld City Council) had changed rapidly over the previous ve years. These changes had been the result of many factors; some were due to the proactive role of key community activists, some were due to the council and others were due to requirements of European and SRB funding. The focus groups and supporting interviews highlighted the dif culties in unravelling the impact of an individual project on community-level partnerships. Given that the case-study project did not have a speci c remit to alter community-level partnerships, it is dif cult to assess its impact in this area. Responses from the Open Spaces focus group suggested that the project's effect on partnership activity was either neutral or negative. Focus group work also showed that the local community group established speci cally for the NWICA project (and one of the formal targets of the project) had subsequently been disbanded, with members being subsumed into new groups subsequently formed.
The case study showed that capacity building in a CED project is amenable to systematic monitoring and evaluation. 'Community capacity' is, however, a complex concept, with a number of different dimensions to it. Moreover, only some of these dimensions are amenable to quanti cation. Our case study did, however, show that modern qualitative approaches can be combined with quantitative indicators to allow evaluation to proceed. The result, however, is more complex and costly monitoring and evaluation, particularly as a combination of different qualitative methods give the best results.
There remains the issue of lead times for capacity building. This issue is crucial if CED is ever to be properly evaluated. In each of the case studies, a determined effort was made to try to establish orders of magnitude of lead times. For environmental projects such as NWICA, the case study found that the lead times depend crucially upon which market the project is seeking to produce community linkage into. The NWICA project offers possibilities for linkage to three different markets.
1. The property market. By improving the green environment of the area, increased demand for council, housing action trust (HAT) and private residential property, together with commercial property (retail, industrial and leisure sector), should occur as the image of the area improves. Surveys were conducted with local property agents (estate agents, HATs and the council lettings department) and residents. The aim of these was to identify the relative importance of environmental improvements to the regeneration of the area (for instance, in comparison with issues such as crime) and the lead times for the environmental improvements to affect the property market. Most residents felt that the improvements had contributed to the area feeling safer and encouraged residents to stay in the area. These are, of course, immediate bene ts with short lead times.
Gauging how long a lead time there will be before the project has an impact on property prices in the private-sector property market is more dif cult, and is hindered by the small size of the local private property sector (70 per cent of residential property is social housing). As an experiment, interviews were conducted with local estate agents and other property market businesses in Shef eld. Only a small proportion of the total turnover of estate agents in Shef eld is in NWICA. For instance, one estate agent estimated that only 0.3 per cent of the rm's annual turnover was in NWICA. However, drawing on experience in both NWICA and other similar areas in Shef eld, the estate agents suggested that lead times of around 20 years would be the appropriate time-scale for environmental impacts to be re ected in the local property market. This then is the kind of order of magnitude of lead time one can expect for targeted environmental improvements in CED areas.
2. The labour market. The project was designed to have both direct bene ts (i.e. training, work experience and temporary employment) and indirect effects (i.e. increasing the attractiveness of the area for rms). The key impact of the NWICA project on the labour market is through its 15 trainees. A pilot survey was conducted of the trainees who were employed by Shef eld Works Department for the duration of the scheme. This sought information on the intermediate labour market (ILM) within which the trainees found themselves. Information was sought on the trainees' experience of the project (type of of training, work experience, etc.), what each trainee was doing immediately prior to the scheme, the level of satisfaction with the scheme, quali cations gained, trainee activities since leaving the project and various personal details (such as gender, ethnic group). Additional questions were included to ascertain the number and spells of vocational training, employment and unemployment the trainee had faced, since the project nished, in the past ve years and over their working life.
The questions asked are typical of ESF and other training policy evaluations. They re ect the need to track the effects of ILM schemes over time via longitudinal analysis. Former trainees can be dif cult to track. This problem can be addressed by requiring project sponsors to maintain details on former trainees, something which is vital if ILM components of CED are ever to be properly evaluated. Our own survey suggested that the lead times for ILM are likely to be much shorter than for property market impactsthree years or less for the NWICA project.

Final product markets.
The project may also improve the competitiveness of rms in the area, hence impacting on nal products markets. In order to explore this issue, business surveys were piloted on two groups of bene ciaries: manufacturing businesses within the small (and diminishing) NWICA industrial area; and businesses in the retail area. There was a clear divide in responses.
Manufacturing rms had few views on the environmental improvements and most were not even aware that they had taken place. This group, however, held very strong views (some positive, some negative) on the effects of other local EU-supported projects (especially Shef eld's Supertram and other local built environment improvements). Retail businesses had a far greater awareness of the changes which had occurred. From the surveys, it was apparent that the environmental improvements undertaken will have only long-term effects on the competitiveness of local businesses (again up to 20 years). Most rms also thought that the bene ts of environmental improvements would only accrue if integrated with the wider economic and social regeneration of the area.
(4) Interlocking initiatives. NWICA is eligible not only for ERDF help, but also UR-BAN and SRB funding. Moreover, parts of the area have rms eligible for EU and national industrial policy incentives. A wide array of projects has been implemented in the area, drawing on many different funding streams. Across Priority 6 as a whole, it was found that CED projects in the 1994-96 programme had accessed no fewer than 11 major types of external funding (for example, Lottery, charities, etc.) from over 50 different sponsor organisations (Government Of ce for Yorkshire and the Humber, 1999a). These greatly exceed gures for numbers of funding streams and sponsors engaged in other more traditional types of regional policy such as business support measures. Initiatives in NWICA at the time of the case-study project included the remodelling of the housing stock, together with projects designed to increase social solidarity, build community capacity and develop employment opportunities. The case-study project itself was part of a wider environmental initiative (SRB and local authority funded), focused on the manufacturing and recreational parts of NWICA. Hence, although the project itself involved only three funding streams (ERDF, local authority and SRB), it was part of a closely interlocked group of other initiatives.

Case Study 2: Holmewood Kidzone (Work and Training through Childcare), Bradford
This was one of a series of initiatives delivered by a local partnership, the Holmewood Consortium. Holmewood is a large peripheral housing estate in south Bradford. Bradford is part of the West Yorkshire conurbation (which also includes the main city of Leeds). It combines inner city socially excluded communities with peripheral disadvantaged housing estates. The project provided an array of types of out-of-school support for children and parents, including collecting children from school, providing care and activities during holidays and signposting residents to other community-based initiatives leading to training and employment opportunities. The project's targets were a typical CED mix of activity measures (such as before-and after-school places for children) and output indicators (for example, permanent jobs for parents). More, however, were quantitative targets than is usual for CED.
The case study comprised an analysis of monitored targets of the project, semistructured interviews with the project manager and others closely involved (such as the local college and TEC), a review of supporting documentation (for example, the project's business plan), a review of the ndings of an in-house 'capacity-building' research project and a Bradford-wide capacity-building evaluation, a focus group convened with the members of the Kidzone management committee, and series of pilot surveys of local residents and businesses on the Holmewood estate (the target bene ciaries). The surveys were again conducted by trained community residents.
(1) Multiple objectives. The project deliberately focused on an economic bene t-helping parents (especially lone parents) to nd employment. Where possible, staff to run Kidzone were recruited locally. The ERDF support was utilised by Kidzone to extend existing services (themselves funded from a variety of sources).
Although its primary objective was the straightforward economic one of assisting lone parents in nding training and employment, Kidzone also aimed to provide activities for children which would 'keep them off the streets' and reduce the risks of them becoming involved in crime. Provision of high-quality care was also seen as providing parents with peace of mind which would not exist otherwise. There were, therefore, secondary objectives of a social nature. Multiple objectives clearly therefore did exist.
(2) Multiple bene ciary groups. Given the stated objectives of the project, it is clear that unemployed lone parents were the principal target-group. However, the project in practice had an array of other potential bene ciaries. The case study was able to identify these as local residents, community groups, existing local businesses and new rms entering the area. Most of these groups were subject to pilot surveys as part of the case study.
(3) Capacity building and lead times. Considerable efforts have been made to build community capacity on Holmewood and the ERDF project was part of a wider capacitybuilding initiative encompassing all of Bradford's CED areas. As part of this initiative, a full-time 'community capacity builder' had already been employed on Holmewood.
The experience of the capacity builder in running the project highlighted a key difference between the focus of the Holmewood project and NWICA, the previous case study. NWICA's capacity building focused on individual participation and the strengthening of community partnerships. Holmewood's capacity builder focused on how capacity building can help individuals to overcome barriers to their re-entry into the labour market. These barriers may be individual (for example, a lack of self-con dence) or organisational (for example, poor links between the different parts of the Holmewood Consortium). Unlike NWICA, core community organisational capacity in the form of the Holmewood Consortium was already in place. For the Kidzone project, therefore, capacity building was mainly concerned with enabling residents to nd employment or take up training opportunities (i.e. personal capacity building). The Kidzone project is therefore much closer to the nal community-linking phase of CED than the NWICA project.
The more advanced nature of capacity building in the Kidzone project meant that the Holmewood Consortium had already built up an unusually deep understanding of how best to measure capacity building. In addition to the aspects of capacity building stressed in the NWICA study (i.e. consultation process, community-strengthening and the role of community groups), two further characteristics are seen as crucial to success (1) Awareness and use of community resources (including Kidzone, the consortium's Edwards Rainbow Centre, job search and the family services unit).

(2) Interest in courses and training. The
Holmewood research discovered that more than half of respondents were interested in taking courses for the following reasons: personal interest, improved job prospects, to gain additional quali cations, social reasons and con dence building These appear to be important measures of how personal capacity has grown.
As with the other case studies, project managers, the focus group and the surveyed residents were asked to identify the likely lead times required for initiatives to have an impact. Typical responses were that two years was a suf cient length of time for most of the intermediate impacts of the project (for example, for provision of childcare places). Successfully overcoming labour market exclusion (i.e. permanent jobs) and crime would take longer it was thought, but within a 10-year horizon.
(4) Interlocking initiatives. The Kidzone project turned out to be an excellent example of just how complex the interrelationships between a CED project and other initiatives can be. The Holmewood Consortium is a mature organisation. It has an array of initiatives funded from different sources. The project managers of Kidzone were therefore well placed to exploit strong links with other activities designed to increase the quality of life on the Holmewood estate.
In summary, the Kidzone project exhibited multiple economic and social objectives and also had multiple bene ciary groups. In practice, however, the key goal was an economic one (successfully linking residents into the formal labour market) and lone parents were the principal target-group. In other respects, Kidzone is more typical of wider CED projects in that capacity building was important and the ERDF project was interwoven with many other initiatives.

Case Study 3: Heeley Development Trust, Shef eld
This project was built on an earlier round of ERDF support provided to Heeley City Farm, an initiative located within the innercity area of Shef eld. The project sought to place the organisation on a rmer footing by establishing a Trust with an independent nancial and legal status, and by building up its activities to take the capacity-building process to its next logical stage. The project, in stark contrast to the Bradford Kidzone, was almost wholly concerned with capacity building. It is therefore an excellent example of CED activity in which capacity is being built from a low base and in which there are at present few community linkage effects yielding economic bene ts.
Four targets were set for the project. One was an output indicator (to create two jobs with the Trust), another was an activity target (to 'assist' six SMEs) and two were measures of capacity building (to establish the Trust and develop a long-term strategy). The targets are very revealing since they are typical of the kinds of target currently being set for capacity-building CED projects. They show how dif cult it is to de ne quanti able targets for capacity building.
With capacity-building projects, the ability to sustain the process is crucial and is often hampered by the short time-horizons of most of the funding bodies (including the Structural Funds). The Trust has therefore sought to turn itself into a self-sustaining organisation supported by involving itself in an array of different public-sector and private-sector activities. The Trust's role is to act as a catalyst to community-based organisations that would in turn support residents and local businesses.
Research for the case study comprised an analysis of monitored targets, semistructured interviews with the project manager and others closely involved in the project, a review of the supporting documentation (for example, business plans, action plans and previous research) and pilot surveys of the main target bene ciary groups.
(1) Multiple objectives. The project had the following stated objectives (1) To work towards the reduction of local unemployment, particularly youth unemployment, through the development and establishment of community enterprises and the provision of training and work experience.
(2) To work towards improving the environment of the local area, through the development of further funding bids. (3) To create an information and resource agency for the area with the possibility of this eventually forming a separate community enterprise. (4) To co-ordinate capacity building in the area through consultation with various groups to identify need and develop common solutions to problems and to disseminate ideas and expertise.
These show clearly that there are multiple objectives present: economic, environmental and social. Unlike the NWICA and Kidzone projects, however, the focus of the project itself is almost entirely on capacity building. Very little attempt has been made to specify the objectives (for example, reduced unemployment) expected once the community-linking phase is reached. It is only through the rst objective (help for new local enterprises) that immediate community linkage to the economy is expected.
(2) Multiple bene ciary groups. Identifying the target bene ciaries of capacity-building organisations such as the Trust is a complex task given its interest in almost all aspects of a community's activities. Interviews with the Trust's managers, however, showed that the primary aim of the Trust was to stimulate the activities of a lower tier of other community organisations-it is these therefore which are the direct bene ciaries. However, the Trust's ultimate aim is clearly to help residents and businesses. Hence, three different sets of target bene ciary groups can be identi ed, between them encompassing virtually the whole community. This is a wider array of target-groups than in any of the other casestudy projects.
(3) Capacity building and lead times. As noted earlier, the project was designed to provide almost all of its support for the capacity-building phase of CED, primarily for its establishment and identi cation of possible future initiatives for the Trust. It would therefore be inappropriate to expect that the Trust will have a substantial impact on community organisations, residents or businesses over the time-period of the project (20 months). The Trust is the product of earlier rounds of capacity building. Through the interaction of individuals, established community groups, schools, local businesses and outside agencies, the Trust is now in a position to enable a process whereby local people are involved in regeneration. However, interviews with the Trust managers showed that the real aim of the Trust in the long term was to achieve economic viability and sustainability, both for residents and for the Trust itself.
The case-study research discovered that local awareness of the Trust was high, primarily because of its newsletter, the New Heeley Voice. The consultation process is therefore well under way. Prior research on community capacity in the Heeley area has shown that resident participation in the SRB area of which Heeley is part was low (34.8 per cent of residents being involved in one or more voluntary groups against gures for NWICA and the UK of 42.1 per cent and 54.3 per cent respectively). This suggests that the Trust still has more communitystrengthening work to do.
It is, however, via community groups that the Trust has had most success. The Trust provides support for an array of community organisations. Comparing the Trust's activities with the wider research literature on development trusts suggests that the Heeley Trust has less direct control over activities which help residents than trusts elsewhere. This is an interesting nding and is perhaps re ected in the wide-ranging initiatives in which the Trust's bene ciary organisations are involved, and the wide range of organisations af liated to the Trust.
A European Commission study identi es three aspects of capacity building which are useful for assessing the success of organisations (such as the Heeley Trust) which seek to strengthen community groups (Ekos, 1998, Appendix C) (1) Function. What is it that assisted community groups are enabled to do through Structural Funds intervention? (2) Sustainability. To what extent has assistance improved the sustainability of groups in terms of the tenure of their existence and their need for on-going public-sector support?
To what extent have groups been assisted to increase their penetration of their target constituencies (i.e. bene ciaries)?
The Ekos study suggests that a useful starting-point for gauging the impacts on the function of capacity building is the extent to which support enables community-based groups to extend their role and play a greater part in regeneration and development. The suggested starting-point is a roles model developed by the Community Development Foundation which posits a continuum in terms of the functions that groups can adopt (see Figure 1). This model suggests that evaluation should attempt to gauge the distance travelled by those groups supported. This will not be appropriate to all organisations, but is relevant to most and can be adapted to t most circumstances. It is also true that many organisations will not aim to become a 'deliverer of economic services'. However, the rationale behind CED is that a critical mass of organisations should be engaged either as the deliverer of economic services or as a long-term partner service by the end of a project. From interviews with the project managers, a survey of assisted organisations and analysis of the content of the Trust's action plan, it appears that the Trust has established the capacity to act as a long-term partner service. Moreover, the organisational, communications and nancial capacity to design, negotiate and implement a CED action plan, would suggest that the Trust has more than ful lled the Commission's criterion of having built capacity (European Commission, 1996). Our survey of bene ciary organisations also suggests that these have progressed to become the 'source of project ideas and focus of activity' suggested by the roles model ( Figure 1). All of the organisations surveyed in the case study stated that they would not have undertaken these activities without the support of the Trust.
Turning to the issue of sustainability, one measure proposed by the Ekos (1998) study is the period of tenure of the assisted groups. The organisations surveyed in our case study were already in existence prior to the establishment of Trust. However, the organisations did stress that the support provided by the Trust had enabled them either to provide training to residents or had provided them with business advice. The organisations were still in existence at the end of the programme. The Ekos study suggests that a good measure of sustainability is a simple count of the groups established/assisted by a project which remain in existence at the end of the programme. However, this clearly has limitations and does not provide an indication of long-term sustainability. A more suitable measure would be to gauge whether organisations supported are capable of continuing their operations without public support.
Finally, there is the issue of coverage: the ability of the supported organisations to target their key bene ciary groups. The casestudy research suggested here that there is wide variation in the degree of coverage from organisation to organisation. Development trusts in other communities typically have both social and economic functions, of which some may have commercial viability. This is certainly true of the Heeley Trust and most of the other organisations it itself supports. The case-study research did show, however, that once the target bene ciaries have been identi ed (whether residents, businesses or both), evaluating coverage should not be too dif cult a task. It should be possible for most CED projects to identify and count the number of residents and businesses of different types involved. More importantly still, it should be possible to calculate the proportion of residents who are involved in community economic regeneration groups. This should allow close monitoring of the penetration of community activity.
(4) Interlocking initiatives. Development trusts and the various other community organisations they support typically access a wide array of different funding streams, and Heeley is no exception to this. Moreover, the CED area is eligible for a range of other types of national, urban and regional policy scheme. In the case of Heeley, it is SRB which has provided the most visible help in recent years, but there have been many other types of intervention too. The case study showed clearly that this type of CED project has many more interlocked initiatives than other types of CED.

Case Study 4: Dewsbury Credit Union
Credit unions and other micro credit initiatives operate in intermediate credit markets, from which mainstream banking services have withdrawn or which they have always ignored. Although intermediate credit markets do not necessarily have a geographical dimension, the existence of concentrations of nancial exclusion (or 'red-lining') suggests that they at least have some geographical elements.
Why banks withdraw services from intermediate credit markets may be largely explained by the existence of one or all of the characteristics of risk aversion, information asymmetry, transactions costs in serving low-income communities and prejudice (Dymski, 1995).
Credit unions attempt to overcome some of the problems of transactions costs and information asymmetries. For instance, credit unions, by being locally based, know more about the people they lend to and the likelihood that they will repay. They are also in a better position to monitor repayments.
In the context of Structural Funds, credit unions and other micro credit institutions appear to be a hybrid of initiatives which combat poverty and those which focus on economic regeneration and jobs. At present, support is overwhelmingly skewed towards the former. Evaluation is therefore unlikely to nd evidence of much direct job creation.
The aim of the case-study project was to create a single credit union for the whole of Dewsbury in West Yorkshire. Dewsbury is a town within the West Yorkshire conurbation and lies immediately to the south of Leeds, the principal city in the sub-region. It is one of the smaller towns in the area, and comprises a diverse mixture of areas of social exclusion and more prosperous suburbs. The target population included both residents and employees commuting into Dewsbury. The project had a series of novel features intended to build on previous experience with credit unions to ensure long term success. In particular, four distinct phases were envisaged (1) Initial awareness-raising by actively promoting the bene ts of a credit union in the area.
(2) The establishment of a study group drawn from a core group of individuals who expressed an interest in being actively involved from the start. (3) Capacity building using the study group as the start-point (for example, training the study group via an accredited Open College Network course in skills relevant to credit union developmentespecially book keeping and credit union management). The core members of the study group were eventually to become the management committee. (4) Registering and establishing the credit union itself. At the time of the case study, the credit union was in the process of being registered with the Registrar of Friendly Societies.
The case-study project was sponsored by Kirklees Metropolitan Council and was run by full-time project managers who had extensive prior experience of credit unions. ERDF support was provided for pilot work, the training and establishment of the credit union and the refurbishment of premises. Matching funding was provided by SRB. To ensure suf cient membership (to maximise economies of scale and reduce risk), the credit union was made available to the whole of Dewsbury (i.e. both CED and non-CED areas alike). This is an interesting feature since most previous credit unions have been con ned to residents within the socially excluded area. The project had ve main target indicators for monitoring purposes, only two of which were output measures (jobs). As in the previous projects, the case-study research comprised an analysis of target indicators, semi-structured interviews with the project managers, a review of the supporting documentation (the business plan, previous national research on credit unions and the literature on nancial exclusion) and a focus group involving members of the management committee of the credit union. Focus groups and surveys of local citizens were not conducted in this case study since at the time of the research the credit union had not been set up, the project having only just reached the end of its capacity-building phase. However, semi-structured interviews with local nancial institutions were used to examine the role of banks in the regeneration of Dewsbury and views of professional local bankers were sought on nancial exclusion and credit unions.
(1) Multiple objectives. The project appears at rst sight to have a very straightforward economic objective: to rectify capital market failure. This is re ected in the stated longterm aims of the project which include generating more widespread access to nancial services as an alternative to mainstream high-interest, low-convenience nancial institutions, and retaining more money within the local economy (i.e. 'internal blocking' of the local multiplier). The objectives were to be achieved by extending core credit union services (by providing a cheque service) to include bill-paying services, payment of insurance premiums and credit links to other community-based businesses.
While it is true that credit unions do have more focused economic objectives than most CED projects, it is important not to take too simplistic a view of this in the evaluation process. The manager interviews and the focus group both highlighted the fact that the credit union is expected eventually to have a wide array of direct and indirect effects on residents, including community capacity building. Multiple objectives are therefore again present.
(2) Multiple bene ciaries. The challenge to evaluation posed by the presence of multiple bene ciaries is less serious with credit unions than with other CED projects. The target bene ciary group is residents and one can also safely assume that this means all residents in the area. However, the Dewsbury case study does highlight an inherent dif culty with CED micro-credit initiatives. Previous experience with credit unions has shown that if they are con ned solely to the residents of a single social exclusion area they are more likely to fail (or else stabilise at a very low level of membership). The Dewsbury project therefore wisely sought a wider membership from the start. The implication of this for evaluation, however, is that a careful check needs to be kept of how far the scheme brings bene ts to the original target area of social exclusion. The hijacking of social policy initiatives by the af uent middle class has been a perennial problem in Britain, a problem CED too may experience.
Credit unions must also be aware that, while the target bene ciaries comprise the full population of the CED area, different sub-groups within the area may require speci c targeting within the scheme. For example, the Koran contains strict usury laws forbidding the receipt of interest on savings. Issues such as this have implications for evaluation, particularly the need to present results disaggregated by ethnicity (and perhaps also age and gender).
(3) Capacity building and lead times. At the time of the research, it was impossible to gauge whether the credit union had led to any capacity-building effects over and above those required to select the study group and to raise awareness. This is because the actual credit union had not yet been created. However, much of the literature on credit unions highlights that most succeed only when key capacities of both the core group and the wider membership are built (see New Economics Foundation, 1997, for a review). Work by Birmingham Credit Union Development Agency suggests the following as minimum criteria for a mature credit union (1) An active membership of over 250.
Dewsbury Credit Union aimed for 300 members in its rst year and an eventual target of 1000 members.
(2) Assets of over £400 000 with at least £240 000 out as loans (a 60 per cent ratio). The Dewsbury Credit Union aimed for an asset base of £330 000 but with a higher assets-loan ratio of 75 per cent. This would produce a surplus suf cient to allow the employment of a full-time manager.
The case-study research suggested the fol-lowing additional criteria for capacity building via a credit union (1) A core group of volunteers. Perhaps numbering at least 20 persons, who will eventually form the management board and develop an appropriate array of skills ( nancial, marketing, promotional, organisational).
(2) A high-quality service. In the early stages of credit union development, it is essential that the core services (savings and loans) be delivered without fault. Without this, trust will be quickly lost.
(3) Membership. Credit unions need to ensure that membership is a balanced one, primarily by geographical location, and also to ensure that services are open to as wide a population as possible. However, the focus group and the wider literature counsel that detailed information on members should not be collected as this may create barriers to key customer groups. (4) Links to other initiatives. If credit unions are to contribute actively to the wider community economic regeneration agenda, they need to engage with their local communities. This is likely to be realised through a number of roles such as awareness-raising and promotion through other CED initiatives, interlocking membership with other community partnerships, using community facilities as collection-points for interest and loan repayments, and using community partnerships to provide a vehicle through which the credit union services are made more responsive to the members.
The evidence from the case study also suggests that it is the consultation and community-strengthening aspects which are of particular importance. Once established, a credit union can play a role in strengthening wider community groups, but this is less important for such a focused initiative as a credit union than it is for other types of CED.
(4) Interlocking initiatives. This is much less severe a challenge with credit unions than with other types of CED. The Dewsbury Credit Union project did have SRB support in addition to ERDF. The local authority was also actively involved. However, evaluating the success or otherwise of a single credit union rarely involves disentangling the effects of more than two or three partners.

Conclusion: A Way Forward?
This paper has argued that the fundamental principles of traditional evaluation methodology are as relevant for CED as for other types of regional and urban policy. Indeed, it is particularly vital with such an untried type of policy as CED to estimate the counterfactual (particularly deadweight and displacement effects), and its effectiveness compared with the traditional types of assistance with which CED must compete for scarce resources (i.e. its opportunity cost). Unfortunately, in the 1994-99 Structural Funds programmes, as with the UK's domestic CED policies, very little progress has been made in extending traditional evaluation principles to non-economic impacts of CED (i.e. social, political and environmental). In addition, there has been a tendency to treat CED evaluation differently from other types of policy. A good example of this is the way in which displacement effects on jobs are calculated. Displacement for CED projects is typically calculated at the level of the individual tiny local CED area, whereas for other policies (for example, support for SMEs), displacement is estimated at the level of the whole Objective 2 region. The bigger the geographical area, the bigger is displacement. Current evaluation practice is therefore preventing comparison of the effectiveness of CED relative to other traditional initiatives.
The case studies have clearly shown that all four characteristics of CED which pose the greatest challenges for evaluators seeking to apply traditional evaluation methods (i.e. multiple objectives, multiple bene ciary groups, capacity building and interlocking initiatives) are present in each of the projects studied. The fact that all four characteristics are present even in projects with super cially of provision places simple objectives (such as a credit union) is a signi cant nding. The four sets of characteristics are by no means unique to CED and have long been present in traditional regional policy. What is new, however, is the degree to which they are present in CED. Is there a way forward for evaluation in the 2000-06 Structural Funds programming period? The simplest way forward would be to soldier on with current methods. This is super cially attractive. Many of the problems with evaluation in the 1994-99 programming period will melt away as the necessary hard empirical evidence on CED at long last begins to accumulate.
There are, however, three reasons to reject the 'no change' option. First, the evidence available even by 2006 will still fall short of many of the lead times for bene ts identi ed in our case studies (for instance, NWICA has lead times in excess of 20 years). Moreover, such a long run of evidence will only be available for those CED projects 'rolled forward' into the 2000-06 programme. Secondly, if one accepts the (still untested) opinion that capacity building confers utility in its own right, then monitoring and evaluation need to be extended to incorporate greatly improved measures of capacity building. Thirdly, to concentrate, as at present, on direct economic bene ts (such as jobs) is at variance with the underlying philosophy of CED. Proponents of CED have made radical claims: that many different economic, social and environmental objectives can be attained simultaneously by a single initiative, and that all target bene ciary groups can bene t simultaneously (i.e. pure pareto gains). Far better, surely, to use evaluation rigorously to test these assertions than to assume that they are true.
Since the simple option is not available, what light do the case studies throw on a way forward? In our opinion, four steps are possible.
(1) More Comprehensive Monitoring and Evaluation CED evaluation in the future will inevitably be more costly and time-consuming if it is to be effective. This is fully recognised by the European Commission. Advances in the MEANS system, such as transversal evaluation (European Commission, 1999a), more advanced evaluation techniques (European Commission, 1999b) and improved sets of indicators (European Commission, 1999c), are well suited for future evaluation. Our case studies, however, show that CED projects unusually long lists of both objectives and target bene ciary groups. The inescapable logic of this is that extremely large matrices of impacts will need to be estimated for CED (comprising all combinations of objectives and target-groups). Moreover, there will be a different matrix of objectives and target-groups for each project. Table 2, for example, sets out a fuller list of the indicators appropriate for each of the four case-study projects. This kind of pragmatic project-by-project set of indicators will be vital in future CED evaluation exercises.
Our case studies therefore show just how formidable a task 'bigger and wider' evaluation for CED is going to be. Much more effort will need to be put into the initial clari cation of objectives and target bene ciary groups. In addition, the overwhelming evidence of previous (traditional) evaluation research is that policies almost never attain multiple objectives simultaneously or generate pure pareto gains. Trade-offs are much more likely. The almost complete lack of trade-off analysis for CED means that much original research still needs to be done. Who is lling the jobs created in the CED area (locals or in-commuters, the unemployed or those previously in jobs)? Is the social networking encouraged by CED enhancing or disrupting existing (perhaps family) networks? Are some groups bene ting from the project at the cost of others? These are the kinds of issue for which hard evidence must be sought.
The need for 'bigger and wider' evaluation is not the fault of traditional evaluation methodology. It arises directly from the more ambitious objectives and community-wide 'reach' of CED. The expansion of monitor-ing and evaluation will place heavy bureaucratic burdens on partner organisations and project managers. Bureaucratic overload is already a serious problem in the Structural Funds programmes (Government Of ce for Yorkshire and the Humber, 1997). One possible escape route from this dilemma could be to focus on only one or two key objectives and target-groups in the CED area. However, even if the key objectives and targets could be clari ed at the outset, this approach would run counter to the all-embracing nature of CED. It also runs counter to the Commission's attempts to widen evaluation in all Priorities to incorporate horizontal aims such as equal opportunities, environmental impact and social inclusion. A second escape route would be to focus only on the economic impacts of CED, which are more tractable. This too, however, clearly runs counter to the philosophy of CED. There therefore seems to be no escape from 'bigger and wider' evaluation.

(2) Combining Quantitative and Qualitative Methods
The case studies have revealed that CED projects are best monitored and evaluated using a combination of quantitative and qualitative methods. This is particularly true of the capacity-building phase. Moreover, what works best is a combination of different qualitative methods (focus groups, community interviewing, etc.), a range of different indicators (of consultation, communitystrengthening, etc.) and the disentangling of personal and community capacity building. The particular combination of quantitative and qualitative methods will vary from project to project. (Further details of the methods found successful for the four case-study projects discussed in this paper can be found in Government Of ce for Yorkshire and the Humber, 1999a, Annexes.)

(3) Lead Times for Capacity Building
This is a greatly neglected area. Future evaluation will need to develop much better initial clari cation of expected lead times, as has been undertaken for each of the four case studies reported here, and to ensure that these are monitored and subsequently evaluated. The case studies reveal that lead times differ hugely between and within CED projects. The real problem of long lead times in CED is a pragmatic one: hard evidence of the nal community-linking phase has not had time to accumulate. Evaluators in the 2000-06 programming period will be fortunate in having much more data available on this issue. There is a real opportunity here to make progress.

(4) Interlocking Initiatives and 'Synergy'
The problem faced by CED with interlocking initiatives is a triple-barrelled one: many different partner organisations, funding streams and projects are jumbled together in the same small area. This situation raises two fundamental questions: whether it is worth trying to unpick this tangled web and, if so, how to do it. Current evaluation practice does indeed try to unpick the web since it is individual CED projects which are evaluated. Our case studies show that all four projects exhibit complex interlocking effects. Our own proposal for a way forward in this matter is a two-fold one.
First, many CED projects (perhaps most) are relatively free-standing in terms of their impact on the local area. The Dewsbury Credit Union and Holmewood Kidzone projects are good examples. They can probably be safely evaluated as free-standing projects. There is still the task of dividing up bene ts between the EU and matching funding providers, but a simple rule of thumb such as a division in proportion to nancial contribution would probably suf ce.
Secondly, projects such as NWICA and the Heeley Development Trust, however, are inextricably intertwined with other projects and partners. The presence of genuine synergy in these cases makes the division of bene ts a pointless exercise. What then can be done? A two-stage procedure may offer a way forward. First, with an area-based policy such as CED, the only effective way to discover if synergy is occurring for the cluster of interrelated projects and partners is to conduct 'top-down' analysis over a reasonable time-period. This was conspicuously lacking in the 1994-99 programmes, mainly because CED areas are so small that key secondary data series are lacking. It is vital that suitable area-level data series be developed for the CED localities. The 2001 population census will greatly assist this process, but much more primary data collection is inevitable if progress is to be made. The extension of 'top-down' evaluation to CED will still not enable the contribution of individual projects to be evaluated, but will allow the extent of synergy-or lack of it!-to be judged and also to restore cross-checking to the evaluation. Secondly, 'bigger and wider' evaluation offers the chance to obtain more disaggregated evaluation results (by objective and target-groups). A better picture of the distribution of bene ts locally is valuable in its own right (for example, for exposing trade-off situations). However, a further bene t is that disaggregated results can shed more light on precisely which project or partner organisation has had the most effect. A project such as NWICA, for example, should impact much more on certain types of properties and resident groups than others. Disaggregated results will help to reveal whether this has in fact been the case. There will, of course, be limits to how far this process can be taken. After all, if genuine synergy does exist, it is illogical to try to separate individual contributions: the whole is greater than the sum of the individual parts! In practice, however, we may well nd that once top-down analysis comes into play, genuine cases of synergy are much thinner on the ground than many proponents of CED currently argue.