Regulatory cost-benefit analysis: Getting one you can use
Principal Consultant Jo Smith explains the role of cost-benefit analysis in developing regulatory proposals. She describes some of the common challenges and pitfalls, and ways of overcoming them.
In a fast-paced policy environment, carving out time and resources for a cost-benefit analysis of regulatory proposals can seem overwhelming. A well-executed CBA should add value to your policy process, but too often it can feel like a compliance burden that you don’t have the time or headspace for.
In our work at MartinJenkins, we often get brought in to do regulatory CBAs, and we’ve encountered some common problems and challenges. We’ve also found some things that can help you and your team produce a good-quality CBA, and in turn get your RIA (regulatory impact analysis) over the line.
A CBA HELPS YOU CHOOSE THE BEST OPTION
A high-quality RIA is essential for any proposal to change or create regulations. It helps ensure that the regulatory response is proportional to the problem, will have benefits that outweigh the costs, and won’t have unintended effects.
The government’s requirements for regulatory impact analysis are set out in a Cabinet Office circular and explained in Treasury guidance. An important part of a RIA is identifying the full range of feasible options, and then assessing the costs, benefits, and risks of each option.
This is where regulatory cost-benefit analysis comes in. A CBA can provide insights to inform the design of regulatory proposals, and help you choose the best option.
But where to start?
HOW IT WORKS IN THEORY
The CBA process involves identifying and describing the full range of impacts, both positive (benefits) and negative (costs). These benefits and costs may be economic, financial, social, environmental, or cultural.
It’s also important to think about who will be affected – for example, businesses, consumers, households, employees, not-for-profit organisations, particular population groups, and local or central government agencies.
The next task is to understand the size of the impacts, and if possible, to quantify and then monetise them. Converting impacts into a common unit (dollars) allows options to be compared. There are tools to help do this, like Treasury’s CBAx, which reflects the Living Standards Framework.
THE REALITY IS NOT SO STRAIGHTFORWARD
While the theory sounds straightforward, putting it into practice is usually anything but. Agencies doing CBAs for regulatory proposals usually come up against some of the following obstacles.
Tight deadlines
Regulatory proposals are often being developed within short timeframes. You may be facing deadlines to get proposals through Cabinet, publish consultation documents, and engage with the relevant sector. This means you need your CBA experts to work with you at pace, so the results can feed into these other documents.
Regulatory options that aren’t fully formed
We often find that the options evolve through the process of preparing consultation documents and briefings. This can make it hard to pin down assumptions for the CBA modelling.
It also works the other way – the results of the cost-benefit analysis can inform the development of the policy.
Limited resources to dedicate to the CBA
Policy teams are often under pressure and lack the time and resources to engage with the people doing the CBA. While outsourcing the CBA takes some of the pressure off, your team will still need to be engaged to test assumptions and scenarios, and sense-check the findings.
Patchy data
In our experience, policy agencies often over-estimate how useful the available data will be for the cost-benefit analysis. Many sectors are data-poor, and the available data may be of varying quality. This means the CBA experts will need to spend extra time researching, and developing assumptions and proxies to help quantify the impacts of the proposals.
Impacts that can’t be quantified
Often there are categories of benefits and costs that can’t be quantified, much less monetised. In these cases, the CBA experts will need to develop qualitative assessments of the nature and size of the impacts.
These qualitative assessments sit alongside the quantitative analysis. There are techniques, like multi-criteria analysis, for bringing both analyses together, to provide overall assessments of the options. This often entails a degree of professional judgment.
TIPS FOR GETTING THE BEST OUT OF YOUR CBA
You can’t ignore those challenges, but there are some ways you can manage them, to help get the best out of your cost-benefit analysis.
Be upfront about the challenges
It’s not your fault that the sector you’re working with has limited data, or that you’ve been given ambitious timeframes. Being upfront with your CBA experts about the challenges you’re facing will help them be prepared, and help them to not under-estimate the resources and effort needed to get the job done.
Have a dedicated project manager
Consider appointing a dedicated project manager, for example from your Project Management Office, to work with your CBA experts and take care of things like project reporting and invoicing.
This can help take the pressure off your policy team, freeing them up to focus on providing subject-matter expertise for the analysis.
Be prepared to work iteratively
Your CBA experts are probably not subject-matter experts, and they’ll be looking to your policy team to help test assumptions and sense-check results.
If you think the assumptions look wonky, ask for them to be adjusted or for sensitivity analysis, to see how it changes the results. Don’t accept “black box” models that aren’t transparent about their assumptions, caveats, and limitations.
Treat your CBA experts as part of the team
Working collaboratively with your CBA experts will help you understand what’s going on “under the hood” with the CBA modelling, and what’s driving the results (why Option A comes out better than Option B).
This will mean you can confidently integrate the cost-benefit analysis into your regulatory impact analysis, and explain the results to your decision makers and affected parties.
Getting a CBA you can understand and use
Attending to those areas I’ve just listed will help in producing a good-quality cost-benefit analysis that you understand and can use to inform your regulatory policy development.
This will support the ultimate objective of your regulatory impact analysis, which is to develop proportionate, cost-effective regulation to achieve benefits for and reduce harms to New Zealanders.