Abstract

Scriven (1994) states that some existing procedures for the summative ex-ante evaluation of the effects of strategic development programs are erroneous. This paper studies the problem using a standard matrix approach (Leopold, 1971; Ekins and Medhurst, 2003) to the evaluation of the development programme for the Slovene region of Pomurje (RDPP) as a case study. It is assumed that summation of diverse policy impacts is a complex task, since it is possible to make conflicting assessments due to: (i) incommensurable scopes of operation (I-scopes: economic, social, human, ecological) and concurrently (ii) incommensurable scales with which they are assessed (I-scales: micro, meso, macro). Three matrix assessment approaches are compared that apply different summative techniques in synthesising detailed expert assessments of individual policy measure's impacts. The first, Leopold matrix is detailed and conceived on the micro-scale and does not permit summation of individual impacts into aggregate impact to avoid mixing incomensurabilities. Ekins and Medhurst (2003) proposed vertically and horizontally reduced matrix where columns are condensed on the four main I-scopes only; then they allowed for vertical aggregation of all policy impacts represented in rows of their matrix on each particular I-scope. However, this macro-summation approach is unappropriate, as the effects of individual policy measures on each I-scope are not homogenous (Rotmans, 2006), and therefore can only be aggregated by source and area of impact, i.e. partially. Their summative procedure has been thus demoted into a square Leontief matrix (1970) of I-scopes that presents the RDPP in the meso-perspective. This introduces I-scales to the evaluation. From a meso-matrix view I-scopes are evident at the micro-scale as conditionally substitutive, incommensurable at the meso-scale, while complementary at the macro-scale. The proposed summative procedure retains three types of development conflict intact for study: exclusion at the macro-scale, legitimate conflicts at the meso-scale, and transaction conflicts at the micro-scale. The paper concludes that an appropriate summative evaluation of strategic policy intervention is a condition for correctly diagnosing the nature of development conflicts, which is a condition not only for politically relevant but equally for independent and neutral strategic policy evaluation.