Assessment of policies and detection of unintended effects: Guiding principles for the consideration of methods and tools in policy-packaging

Abstract

Single policies or entire policy packages are often assessed using different methods aiming at a quantification of effects as well as the detection of undesired outcomes. The knowledge of potential impacts is essential to take informed policy actions. Hence, there is a constant need for efficient assessment approaches to support policy decision-making. A broad range of such assessment methods is used in policymaking. Some of them are using quantitative data; others are characterized by qualitative information, observations or opinions. Practical experiences with transport policy prove that these methods all have their pros and cons, but none of them are able to detect the full range of effects. This leads to important questions this article deals with, such as what are the strengths and limitations of the different tools and methods for assessing impacts, and how should different approaches be integrated into the policymaking processes? We analyze the ability of assessment methods to detect different kinds of intended and unintended effects, and introduce the concepts of structurally open (mainly qualitative) and structurally closed (mainly quantitative) methods. It is argued that these concepts support making the pros and cons of the tools and methods more explicit and, thus, allow integrating the different tools and methods into the process of policy packaging. Based on a policy package example, we provide practical recommendations on how to integrate different assessment methods adequately and show that both quantitative and qualitative tools should be used in different phases of the process. The main recommendation is to alternate the application of assessment methods with structurally open methods used in the beginning and the end of the policymaking process and applying structurally closed methods in between.