{"id":477,"date":"2017-11-30T22:16:31","date_gmt":"2017-11-30T22:16:31","guid":{"rendered":"http:\/\/eulawenforcement.com\/?p=477"},"modified":"2019-03-24T18:31:04","modified_gmt":"2019-03-24T18:31:04","slug":"fit-for-purpose-scrutinizing-the-eus-system-for-ex-post-evaluation","status":"publish","type":"post","link":"https:\/\/eulawenforcement.com\/?p=477","title":{"rendered":"Fit for purpose? Scrutinizing the EU\u2019s system for ex-post evaluation"},"content":{"rendered":"

Based on Ellen Mastenbroek\u2019s inaugural lecture \u2018More than the sum of its parts\u2019, held at Radboud University, 15 September 2017 and two related articles (Van Voorst and Mastenbroek, 2017; Mastenbroek et al, 2016).<\/em><\/span><\/p>\n

In this contribution, we critically assess the EU\u2019s system for ex-post evaluation, using fresh empirical data. We find that the Commission has not met its repeated ambition to produce systematic and high-quality ex-post evaluation of all important EU legislation. Both the coverage and quality of ex-post evaluations have been variant. This lack of systematic high-quality evaluation is problematic, as such evaluations can fulfill important functions in the EU policy cycle: learning, accountabilit<\/span>y and improved enforcement. It also <\/span>bring strategic risks for the Commission, primarily reduced credibility.<\/span><\/p>\n

The importance of ex-post evaluation in the EU<\/strong><\/p>\n

\"\"<\/a><\/p>\n

The European Union (EU) primarily relies on legislation to reach policy aims. Much of this legislation is established for functional reasons: the desire to solve interdependent societal problems such as air pollution or terrorism. At the same time, the EU is heavily criticized. Some critics target the EU\u2019s limited democratic character. Others criticize the system\u2019s output: EU legislation. While some find EU laws too modest and ineffective, others argue them to be invasive, cumbersome and restrictive. The EU\u2019s regulatory burden was one of the key themes in the British BREXIT campaign, the British telegraph arguing that BREXIT offers a \u2018golden chance to throw EU regulations on a bonfire\u2019<\/a>.
\nThe picture of a UK suffocated by EU red tape may be somewhat of an exaggeration, given the fact that it is still comparatively lightly regulated<\/a>. Yet, there is no smoke without fire: EU legislation incurs both benefits and costs on member states\u2019 authorities and target groups. They must devote time and capacity to speedy and correct implementation of EU rules, and they see their policy discretion reduced. A case in point are national parliaments. According to Vivienne Schmidt these have become arenas for \u2018politics without policy<\/a>\u2019.
\nAs many EU laws are a mixed blessing, a crucial question is whether they succeed in solving the societal problems they seek to solve. This implies a strong case for careful scrutiny of the practical implementation and effects of EU legislation. Evaluation reports may enable such scrutiny, and thus enable policy learning and accountability, two key tasks in any representative democracy. Additionally, by uncovering deficiencies in national implementation, they may form a basis for the enforcement of EU law, so as to improve implementation and effectiveness.<\/span><\/p>\n

The Commission\u2019s ambitions<\/strong><\/p>\n

The European Commission recognizes the importance of strengthening the \u2018evidence-base\u2019 under EU legislation. Since 2001 this is evident from its \u2018better regulation\u2019<\/a> program. Central to this program is the Commission\u2019s desire for effectiveness: laws that really solve the problems that citizens and politicians want to be solved. In the words of Commission president Juncker: \u2018Only by focusing our resources on the issues where Europe can (\u2026) deliver results (\u2026) will we be able to make Europe a better place\u2019<\/a>.
\nThis ambition to produce effective legislation has primarily materialized in the first part of the EU\u2019s regulatory cycle: policy development. As the flagship of Better Regulation, the Commission has developed an extensive system of ex ante evaluation (Radaelli and Meuwese, 2010): evaluation before legislation is enacted. Accordingly, most legislative proposals are assessed on their expected environmental, social, and economic impacts<\/a>.<\/p>\n

From ambition to practice<\/strong><\/p>\n

Since 2000, the Commission has repeatedly stated its ambition to produce systematic, high-quality ex-post evaluations of EU laws. Despite these clear statements, actual practice has lagged behind. Firstly, only 42%<\/a> of all important legislation enacted in the period 2000-2004 has been followed by an evaluation report. Thus, more than half of the EU\u2019s major directives and regulations enacted in this period still have not been evaluated. Secondly, the quality of evaluations varied greatly. Crucially, 43 percent<\/a> of the ex-post evaluations scored insufficiently on a set of rather basic methodological criteria, such as replicability and robust methodology. In addition, a mere 9% of the reports indicated serious involvement of stakeholders in the evaluation process. Finally, about half of the evaluations identified concerned effects. The other reports concerned only procedural matters like transposition, implementation or enforcement of the rules by national authorities.
\nIn sum, there is a gap between the ambitions and practice of ex-post evaluation in the European Union. This is problematic with an eye on the important democratic functions evaluations may fulfill: learning, accountability, and enforcement. Also, patchy evaluation brings a strategic risk. According to the OECD (2015)<\/a>, an evaluation system is only credible if its procedures for initiating evaluations are systematic and transparent. Absent or low-quality evaluations could thus reduce the credibility of the Commission. According to Poptcheva<\/a>, the European Parliament already tends to disregard the Commission\u2019s evaluations because of their variant quality.<\/p>\n

Explaining the gap<\/strong><\/p>\n

In a recent article<\/a>, we set out to uncover the reasons for the variant initiation of ex-post legislative evaluations by the European Commission. In doing so, we distinguished between two political motives. The first political motive was the enforcement of EU legislation in the member states. We expected that legislation running a higher risk of non-compliance is relatively likely to be evaluated. We thus expected directives to be evaluated more often than regulations, and evaluation to depend on the extent of legislative complexity and politicization of decision making. The second political motive was the strategic protection of competences, which is in line with the image of the Commission as a political actor seeking to expand its powers. We expected evaluations to be more likely if the relevant legislation is harder to change, which is the case under EP involvement and unanimity rather than the consultation procedure and qualified majority voting. To this, we added two control variables: the presence of an evaluation clause in the legislation, and the evaluation capacity of the DG responsible for the evaluation.
\nWe tested these hypotheses using a dataset developed for this purpose, containing all major legislation from 2000 to 2004 and all ex-post legislative evaluations conducted by the Commission from 2000 through 2014. The findings of our study were mostly in line with the enforcement perspective: directives indeed are more likely to be evaluated ex-post than regulations. Legislative complexity also increases the chances of evaluation, while politicization does not have a significant effect. We did not find evidence for the second political perspective: the strategic protection of competences. The presence of evaluation clauses and a DG\u2019s evaluation capacity, finally, produce further explanations of the variance in the initiation of ex-post legislative evaluations. In sum, enforcement concerns combined with evaluation clauses and evaluation capacity explain the Commission\u2019s variant ex-post evaluation practice.<\/p>\n

Conclusion and reflection<\/strong><\/p>\n

To conclude, the Commission has not fully lived up to its own ambitions to step up ex-post evaluation of major EU legislation, despite repeated pledges to do so. These deficiencies are problematic with an eye on EU representative democracy, given the fact that evaluations can provide an important basis for policy learning, accountability, and enforcement. They also may compromise the Commission\u2019s credibility.
\nThis having been said, one may wonder whether the glass is half-empty or half-full. Compared to most member states, the Commission is a frontrunner when it comes to ex-post legislative evaluation. Second, this analysis offers a snapshot only. Given the fact that the Commission has revamped its evaluation ambitions under Commission president Juncker<\/a>, it may have stepped up its evaluation practice in the meantime, making it \u2018fit for purpose\u2019 after all.<\/p>\n","protected":false},"excerpt":{"rendered":"

Based on Ellen Mastenbroek\u2019s inaugural lecture \u2018More than the sum of its parts\u2019, held at Radboud University, 15 September 2017 and two related articles (Van Voorst and Mastenbroek, 2017; Mastenbroek et al, 2016). In this contribution, we critically assess the EU\u2019s system for ex-post evaluation, using fresh empirical data. We find that the Commission has … Continue reading “Fit for purpose? Scrutinizing the EU\u2019s system for ex-post evaluation”<\/span><\/a><\/p>\n","protected":false},"author":57,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10],"tags":[],"class_list":["post-477","post","type-post","status-publish","format-standard","hentry","category-general"],"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/eulawenforcement.com\/index.php?rest_route=\/wp\/v2\/posts\/477","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/eulawenforcement.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/eulawenforcement.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/eulawenforcement.com\/index.php?rest_route=\/wp\/v2\/users\/57"}],"replies":[{"embeddable":true,"href":"https:\/\/eulawenforcement.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=477"}],"version-history":[{"count":13,"href":"https:\/\/eulawenforcement.com\/index.php?rest_route=\/wp\/v2\/posts\/477\/revisions"}],"predecessor-version":[{"id":1377,"href":"https:\/\/eulawenforcement.com\/index.php?rest_route=\/wp\/v2\/posts\/477\/revisions\/1377"}],"wp:attachment":[{"href":"https:\/\/eulawenforcement.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=477"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/eulawenforcement.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=477"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/eulawenforcement.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=477"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}