2. Overview of appraisal &
3. Justifying action
4. Setting objectives
5. Appraising the options
6. Developing & implementing the
Annex 1: Government intervention
Annex 2: Valuing non-market impacts
[Annex 3: Land & buildings]
Annex 4: Risk & uncertainty
Annex 5: Distributional impacts
Annex 6: Discount rate
has, for many years, provided guidance to other
public sector bodies on how proposals should be
appraised, before significant funds are committed
and how past and present activities should be
evaluated. Decisions taken at the appraisal stage
affect the whole lifecycle of new policies,
programmes and projects. Similarly, the proper
evaluation of previous initiatives is essential in
avoiding past mistakes and to enable us to learn from
experience. The Green Book therefore constitutes
binding guidance for departments and executive
INTRODUCTION AND BACKGROUND
1.1 All new
policies, programmes and projects, whether revenue,
capital or regulatory, should be subject to
comprehensive but proportionate assessment, wherever
it is practicable, so as best to promote the public
1.2 The guidance
emphasises the need to take account of the wider
social costs and benefits of proposals, and the need
to ensure the proper use of public resources.
1.3 This is
other possible approaches which may achieve
feasible, attributing monetary values to all
impacts of any proposed policy, project and
an assessment of the costs and benefits for
The Green Book
describes how the economic, financial, social and
environmental assessments of a policy, programme or
project should be combined.
ACTIVITIES COVERED BY THE GREEN BOOK
- Policy and
programme development: Decisions on ... the
extent of regulation.
of regulations: Decisions, for example, on
standards for health and safety, environment
quality, sustainability, or to balance the
costs and benefits of regulatory standards
and how they can be implemented.
- At the
start ... to any analysis used to support a
government decision to adopt a new policy, or
to initiate, renew, expand or re-orientate
programmes or projects, which would result in
measurable benefits and/or costs to the
public. This is the appraisal part of the
- And at the
finish ... to retrospective analysis of a
policy, programme or project at its
completion, conclusion or revision. This is
the evaluation part of the process.
OVERVIEW OF APPRAISAL AND EVALUATION
AND EVALUATION CYCLE
and evaluation often form stages of a broad policy
cycle that some departments and agencies formalise in
the acronym ROAMEF (Rationale, Objectives, Appraisal,
Monitoring, Evaluation and Feedback).
THE ROLE OF
should provide an assessment of whether a proposal is
worthwhile, and clearly communicate conclusions and
recommendations. The essential technique is option
appraisal, whereby government intervention is
validated, objectives are set, and options are
created and reviewed, by analysing their costs and
benefits. Within this framework, cost-benefit
analysis is recommended, as contrasted with
cost-effectiveness analysis below, with supplementary
techniques to be used for weighing up those costs and
benefits that remain unvalued.
which quantifies in monetary terms as many of the
costs and benefits of a proposal as feasible,
including items for which the market does not
provide a satisfactory measure of economic value.
that compares the costs of alternative ways of
producing the same or similar outputs.
APPRAISAL AND EVALUATION
are often iterated a number of times before their
proposals are implemented in full. Therefore the
stages set out below may be repeated, and they may
not always be followed sequentially. In particular,
as options are developed, it will usually be
important to review more than once the impact of
risks, uncertainties and inherent biases. This helps
to avoid spurious accuracy, and to provide a
reasonable understanding of whether, in the light of
changing circumstances, the proposal is likely to
remain good value for money.
2.6 The first
step is to carry out an overview to ensure that two
pre-requisites are met: firstly, that there is a
clearly identified need; and secondly, that any
proposed intervention is likely to be worth the cost.
This overview must include an analysis of the
negative consequences of intervention, as well as the
results of not intervening, both of which must be
outweighed to justify action.
2.7 The second
step is to set out clearly the desired outcomes and
objectives of an intervention in order to identify
the full range of options that may be available to
deliver them. Targets should be set to help progress
towards meeting objectives.
2.8 The third
step is to carry out an option appraisal. This is
often the most significant part of the analysis.
Initially a wide range of options should be created
and reviewed. This helps to set the parameters of an
appropriate solution. A shortlist may then be created
to keep the process manageable, by applying the
techniques summarised below to high level estimates
or summary data. The do minimum option
should always be carried forward in the shortlist, to
act as a check against more interventionist action.
2.9 Each option
is then appraised by establishing a Base Case
.This is the best estimate of its costs and
benefits. These estimates can then be adjusted by
considering different scenarios, or the options
sensitivity to changes can be modelled by changing
key variables. More fully, the appraisal may develop
and value the costs of each option.
and value the benefits of each option.
required, adjust the valued costs and
impacts (the effects of proposals on
different sections of society);
- Adjust for
the timing of the incidence of costs and
benefits by discounting them, to obtain their
necessary, adjust for material differences in
tax between options.
- Adjust for
risk and optimism to provide the Base Case,
and consider the impacts of changes in key
variables and of different future scenarios
on the Base Case.
unvalued impacts (both costs and benefits),
using weighting and scoring techniques if
Developing and implementing a solution
option appraisal, decision criteria and judgement
should be used to select the best option or options,
which should then be refined into a solution.
Consultation is important at this stage, regardless
of whether it has taken place earlier.
2.11 Issues that
may have a material impact on the successful
implementation of proposals must be considered during
the appraisal stage, before significant funds are
committed. This is to ensure that the outcome
envisaged in the appraisal is close to what
is similar in technique to appraisal, although it
obviously uses historic (actual or estimated) rather
than forecast data, and takes place after the event.
Its main purpose is to ensure that lessons are widely
learned, communicated and applied when assessing new
 The term
Base Case is sometimes used to refer to
the do minimum option, but it is not used
in this way in the Green Book.
ultimate outcome of any appraisal is a decision
whether or not to proceed with a proposal or a
particular option. As these decisions will often have
far reaching consequences, the presentation of the
conclusions and recommendations to decision makers
and key stakeholders can be as important as the
analysis itself. In all cases, transparency is vital.
Presentations and reports should be clear, logical,
well founded, and geared towards helping the decision
should provide sufficient evidence to support their
conclusions and recommendations. They should provide
an easy audit trail for the reader to check
calculations, supporting evidence and assumptions.
Major costs and benefits should be described, and the
values attached to each clearly shown rather than
netted off in the presentation of the analysis. This
should help to ensure that decision makers understand
the assumptions underlying the conclusions of the
analysis, and the recommendations put forward.
Appraisal reports should contain sufficient
information to support the conduct of any later
APPRAISALS AND EVALUATIONS
an assessment can be resource-intensive. Appraisals
and evaluations should therefore be carried out
collaboratively wherever possible between
stakeholders, but lead responsibilities need to be
well defined, and accountability for accuracy and
thoroughness clearly understood.
frameworks below are particularly relevant to
appraisals and evaluations:
- The OGC
Gateway Review (mainly for programmes and
Regulatory Impact Assessment (mainly for
policies involving regulatory impacts); and
- The Centre
for Management and Policy Studies (CMPS)
Government Commerce Gateway Process
2.21 Gateway is
a review process for civil procurement projects
conducted by the Office of Government Commerce. It
examines policies and projects at critical stages in
their lifecycle to provide assurance that they can
progress successfully to the next stage. Compliance
with the Green Book is incorporated into the first
and second gateways. Detailed information is
available from the OGC website.
regulatory impact assessment (RIA) is a policy tool
that assesses the impact, in terms of costs, benefits
and risks of any proposed regulation that could
affect businesses, charities or the voluntary sector.
It is Government policy that all government
departments and agencies where they exercise
statutory powers and make rules with general effect
on others must produce an RIA. They should also
produce an RIA for proposed European legislation that
will have an effect on businesses, the public sector,
charities or the voluntary sector in the UK.
the trigger for producing an RIA is that the proposal
could affect businesses, charities or the voluntary
sector, the RIA itself should cover the full range of
economic, social and environmental effects, in line
with the Green Book methodology.
The CMPS Policy
2.24 The CMPS
Policy Hub aims to improve policy making and
delivery, by providing:
access to resources and activities from the
UK and abroad that help formulate, develop
and evaluate policy more efficiently and
examples of improved policy making and
- Tools to
help break down organisational and
geographical barriers, and improve
collaborative working within and beyond
- A platform
for promoting the highest standards of
research and evaluation.
TO APPRAISAL AND EVALUATION
2.25 There is a
wide range of generic issues that may need to be
considered as part of any assessment. The following
list should be checked for relevance to options under
appraisal, and used for later evaluations:
impact new proposals can be said to
have strategic impacts on organisations if
they significantly affect the whole or major
part of an organisation over the medium to
long term. Proposals should therefore be
considered in terms of their potential scale
of impact, and how they fit in with the
strategy of the organisation(s) they affect.
rationale proposals need to be
underpinned by sound economic analysis, which
should be provided by a cost benefit analysis
in an option appraisal. See Chapter 5 in
arrangements and affordability
proposals need to be affordable, and an
affordable financial plan needs to be
developed. See Chapter 6.
all proposals should be assessed for
their achievability, and recognised programme
and project management arrangements set up as
necessary. See Chapter 6.
impact as discussed previously, the
impacts of new proposals on businesses,
voluntary sector and charities should be
assessed. See Chapter 2.
consideration should be given to
legislation specific to the case in hand, as
well as statutes that affect many proposals,
such as the Human Rights Act, or the Data
Protection and Freedom of Information Acts.
management and control The information
requirements of proposals, including the data
needed for later evaluation, and the
supporting IT that may be required. Further
guidance is available from the OGC .
impacts The effects on the environment
should be considered, including air and water
quality, land use, noise pollution, and waste
production, recycling and disposal. Further
guidance is available from ODPM, Defra and
Impacts on various groups in society
should be considered as part of an appraisal.
Chapter 5 describes how distributional
impacts should be brought into the appraisal
the impacts of proposals on health
should be considered, and evaluation made of
the impact on health of poverty, deprivation
and unemployment, as well as poor housing or
workplace conditions. The Department of
Health can provide further advice12, or can
be accessed via the policy hub .
- Health and
safety the health and safety of people
at work or arising from work activity may
need to be safeguarded. Obviously this is of
particular concern in construction. The
Health and Safety Commission can provide
further advice .
focus Assessments may need to involve
consideration of the cost and quality of
goods and services, as well as access to,
choice of, and information about them .
Union It will often be important to
take account of proposals and activities in
other European Union countries, as well as
specific legislation and regulations. State
aid rules are particularly important to
consider, as these prescribe the extent to
which government can intervene .
underlying rationale is usually founded either in
market failure or where there are clear government
distributional objectives that need to be met. Market
failure refers to where the market has not and cannot
of itself be expected to deliver an efficient
outcome; the intervention that is contemplated will
seek to redress this. Distributional objectives are
self-explanatory and are based on equity
intervention can incur costs and create economic
distortions. These must be taken into account to
determine whether intervention is warranted. For
example, a regulation may be successful in addressing
a particular market failure, but might also involve
other costs that mean that overall it is not
OUTCOMES, OUTPUTS, AND TARGETS
should be stated so that it is clear what proposals
are intended to achieve. Objectives may be expressed
in general terms so that the range of options to meet
them can be considered. The objectives of individual
proposals should be consistent with statements of
government policy, departmental or agency objectives,
departmental Public Service Agreements (PSAs), and
wider macro-economic objectives.
4.3 There is
usually a hierarchy of outcomes, outputs, and targets
that should be clearly set out in an appraisal.
Outcomes are the eventual benefits to society that
proposals are intended to achieve. Often, objectives
will be expressed in terms of the outcomes that are
desired. But outcomes sometimes cannot be directly
measured, in which case it will often be appropriate
to specify outputs, as intermediate steps along the
way. Outputs are the results of activities that can
be clearly stated or measured and which relate in
some way to the outcomes desired.
4.4 Targets can
be used to help progress in terms of producing
outputs, delivering outcomes, and meeting objectives.
Targets should be SMART;
BOX 5: SETTING
OBJECTIVES AND TARGETS
questions may help to set suitable objectives and
- What are we
trying to achieve? What are our objectives?
What would constitute a successful outcome or
set of outcomes?
similar objectives been set in other contexts
that could be adapted?
- Are our
objectives consistent with strategic aims and
objectives as set out, for example, in the
departments Public Service Agreements
- Are our
objectives defined to reflect outcomes (e.g.,
improved health, crime reduction or enhanced
sustainable economic growth,) rather than the
outputs (e.g. operations, prosecutions or job
placements), which will be the focus of
- How might
our objectives and outcomes be measured?
- Are our
objectives defined in such a way that
progress toward meeting them can be
factors are critical to success?
- What SMART
targets can we then set? What targets do we
need to meet?
APPRAISING THE OPTIONS
5.3 This step
involves preparing a list of the range of actions
which government could possibly take to achieve the
identified objectives. The list should include an
option where government takes the minimum amount of
action necessary (the do minimum option),
so that the reasons for more interventionist actions
can be judged.
5.4 The range of
options depends on the nature of the objectives. For
a major programme, a wide range should be considered
before short-listing for detailed appraisal. Both new
and current policies, programmes and projects should
be included as options. At the early stages, it is
usually important to consult widely, either formally
or informally, as this is often the best way of
creating an appropriate set of options.
BOX 8: CREATING
range of options can be challenging. The following
actions are suggested:
the full range of issues likely to affect the
the full range of policy instruments or
projects that may be used to meet the
objectives. This may span different sorts or
scales of intervention; regulatory (or
deregulatory) solutions may be compared with
self-regulation, spending or tax options.
- Develop and
consider radical options. These options may
not become part of the formal appraisal but
can be helpful to test the parameters of
feasible solutions. Well-run brainstorming
sessions can help to generate such a range of
COSTS AND BENEFITS OF OPTIONS
5.8 The relevant
costs and benefits to government and society of all
options should be valued, and the net benefits or
costs calculated. The decision maker can then compare
the results between options to help select the best.
social and environmental costs and benefits for which
there is no market price also need to be brought into
any assessment. They will often be more difficult to
assess but are often important and should not be
ignored simply because they cannot easily be costed.
Annex 2 provides more information on how to take into
account the wider impacts of proposals.
should be expressed in terms of relevant opportunity
costs. It is important to explore what opportunities
may exist. An example of an opportunity is to use
land in a different, more valuable, way than in its
substantial proposals, the relevant costs are likely
to equate to the full economic cost of providing the
associated goods and services, and for these
proposals, the full economic cost should be
calculated, net of any expected revenues, for each
option. The full cost includes direct and indirect
costs, and attributable overheads.
value of benefits
5.24 The purpose
of valuing benefits is to consider whether an
options benefits are worth its costs, and to
allow alternative options to be systematically
compared in terms of their net benefits or net costs.
The general rule is that benefits should be valued
unless it is clearly not practicable to do so. Even
if it is not feasible or practicable to value all the
benefits of a proposal, it is important to consider
valuing the differences between options.
principle, appraisals should take account of all
benefits to the UK. This means that as well as taking
into account the direct effects of interventions, the
wider effects on other areas of the economy should
also be considered. These effects should be analysed
carefully as there may be associated indirect costs,
such as environmental costs, which would also need to
be included in an appraisal. In all cases, these
wider effects should be clearly described and
and benefits where there is no market value
appraisals will identify some costs and benefits for
which there is no readily available market data. In
these cases, a range of techniques can be applied to
elicit values, even though they may in some cases be
subjective. There will be some impacts, such as
environmental, social or health impacts, which have
no market price, but are still important enough to
VALUES OF COSTS AND BENEFITS
5.33 It is
important that the distributional implications of
each option are considered during appraisal. This
type of analysis enhances the understanding of the
fairness of proposals, their social impacts and their
5.34 The impact
of a policy, programme or project on an
individuals well-being will vary according to
his or her income; the rationale being that an extra
pound will give more benefit to a person who is
deprived than to someone who is well off. In
economics, this concept is known as the
diminishing marginal utility of additional
distributional issues may also arise, and should be
considered during appraisal. A proposal may have
differing impacts according to age, gender, ethnic
group, health, skill, or location. These effects
should be explicitly stated and quantified wherever
feasible. For example, the costs and benefits of a
proposal might be broken down according to the ethnic
group they accrue to, providing appraisers with a
basis for comparison and analysis.
appraisers decide not to adjust explicitly for
distributional impacts, they must provide a
justification for this decision. This judgement
should be informed by the following considerations:
significance of the impact of distributional
analysis to the proposal under consideration;
- The ease
with which distributional impacts can be
- The scale
of the impact associated with a particular
project or proposal.
DIFFERENCES IN TAX BETWEEN OPTIONS
adjustment of market prices for taxes in appraisal is
appropriate where it may make a material difference
to the decision. In practice, it is relatively rare
that adjustments for taxation are required, because
similar tax regimes usually apply to different
options. It can also be difficult in practice to
estimate costs net of tax. However, where the tax
regimes applying to different options vary
substantially, this should not be allowed to distort
option choice. In such cases it is important to
adjust for any differences between options in the
incidence of tax arising from different contractual
arrangements, such as in-house supply versus buying
in, or lease versus purchase.
BIAS AND RISKS
5.61 There is a
demonstrated, systematic, tendency for project
appraisers to be overly optimistic. This is a
worldwide phenomenon that affects both the private
and public sectors . Many project parameters are
affected by optimism appraisers tend to
overstate benefits, and understate timings and costs,
both capital and operational.
5.62 To redress
this tendency, appraisers should make explicit
adjustments for this bias. These will take the form
of increasing estimates of the costs and decreasing,
and delaying the receipt of, estimated benefits.
Sensitivity analysis should be used to test
assumptions about operating costs and expected
analysis is fundamental to appraisal. It is used to
test the vulnerability of options to unavoidable
future uncertainties. Spurious accuracy should be
avoided, and it is essential to consider how
conclusions may alter, given the likely range of
values that key variables may take. Therefore, the
need for sensitivity analysis should always be
considered, and, in practice, dispensed with only in
calculation of switching values shows by how much a
variable would have to fall (if it is a benefit) or
rise (if it is a cost) to make it not worth
undertaking an option. This should be considered a
crucial input into the decision as to whether a
proposal should proceed. It therefore needs to be a
prominent part of an appraisal.
5.71 Examples of
variables that are likely to be both inherently
uncertain and fundamental to an appraisal are the
growth of real wages, forecast revenues, demand,
prices, and assumptions about the transfer of risks.
A prior analysis of costs into fixed, step, variable,
and semi variable categories can help in
understanding the sensitivity of the total costs of
are also useful in considering how options may be
affected by future uncertainty. Scenarios should be
chosen to draw attention to the major technical,
economic and political uncertainties upon which the
success of a proposal depends. Considering scenarios
needs to be proportionate. It may take the form of
asking simple what if questions for
small and medium sized projects, but extend to
creating detailed models of future states of the
world for major policies and large programmes. The
expected NPV can be calculated for each scenario. It
may also be helpful to undertake some sensitivity
analysis within a scenario.
UNVALUED COSTS AND BENEFITS
5.76 Costs and
benefits that have not been valued should also be
appraised; they should not be ignored simply because
they cannot easily be valued. All costs and benefits
must therefore be clearly described in an appraisal,
and should be quantified where this is possible and
may need to be undertaken to determine the best unit
of measurement. Alternative non-monetary measures
might be considered most appropriate (See Box 17).
5.78 The most
common technique used to compare both unvalued costs
and benefits is weighting and scoring (sometimes
called multi-criteria analysis). The basic approach
to weighting and scoring involves assigning weights
to criteria, and then scoring options in terms of how
well they perform against those weighted criteria.
The weighted scores are then summed, and these sums
can be used to rank options. An even simpler method
is to list the required performance criteria
(sometimes called critical success
factors), and assess options in terms of
whether they meet them or not.
practice, the weight to give to factors that are
thought to be important by key players cannot be
decided by experts. They inevitably
incorporate the judgements of stakeholders and
decision makers. The risk that they are weighted
towards acceptance of more expensive solutions by
those who would enjoy the potential benefits should
be tempered by at least one stakeholder representing
the opportunities that an expensive solution would be
foregone elsewhere. There are other pitfalls to avoid
in carrying out this type of analysis, and reference
should be made to guidance on multi-criteria
DEVELOPING AND IMPLEMENTING THE SOLUTION
the identification and description of all costs,
benefits and risks, their valuation where feasible,
and their testing through sensitivity and scenario
analysis, the best option should be selected.
Transparency is important at this stage, so that it
is clear on what basis decisions are taken.
6.2 Once an
option has been selected, it will need to be refined
into a solution. Consultation is important at this
stage. Further consideration will need to be given to
the implementation of the proposal, including the
involvement of the private sector, procurement
options and processes, and the programme and project
management arrangements that may be required.
6.3 If a full
cost benefit analysis has been undertaken, the best
option is likely to be the one with the highest risk
adjusted net present value. To the extent that all
costs, benefits and risks have been robustly valued,
this guideline can be applied with more certainty. In
cost effectiveness analysis, the option with the
lowest net present cost should be the best, again
assuming that the cost estimates are as accurate and
reliable as possible.
decision criteria can be used to help select options
where risk is an important consideration. The
maximin-return option is the most
important to consider. It is the most risk averse
option, as it is the option that provides the least
bad outcome if the worst possible conditions prevail.
BOX 20: EXAMPLE
services are being considered, which are mutually
exclusive. Their NPVs under different market
conditions are shown below:
||Low demand (£)
||Expected value (£)
||High demand (£)
criteria points to Service A, as it provides the
highest value in the worst market conditions.
6.6 In practice,
other factors will also affect the selection of the
best option, in particular the consideration of
unvalued costs and benefits. Weighting and scoring
techniques are useful in comparing different options
in terms of the same criteria. However, as scores are
not expressed in monetary terms, judgement is then
required to compare the results of weighting and
scoring with the cost benefit or cost effectiveness
analysis. The two analyses should complement each
other, and may indicate that further analysis is
required before a decision can be reached. Annex 2
provides further information on how weighting and
scoring can be brought into the decision making
process. Fully involving stakeholders is very
important in making judgements between monetised and
6.7 There is
always a value imputed by decisions to proceed, and
this value should always be clearly identified and
funding, and cashflows
affordability of options should always be considered
when developing and selecting options. In addition to
the analysis of economic costs and benefits,
appraisals usually need three major financial
statements, at least for the lead options:
- A budget
statement. This should be based on resource
accounting and budgeting (RAB) principles,
and show the resource costs over the lifetime
of the proposal. For strategic initiatives,
the budget will often comprise the forecast
RAB financial statements of a whole
organisation over a number of years.
- A cashflow
statement. This should show the additional
cash that will be spent on the lead option if
it goes ahead.
- A funding
statement. This should show which internal
departments, partners and external
organisations would provide the resources
(and in some cases cash) required.
6.12 The best
option is likely to require further refinement before
a solution emerges. Options are rarely completely
mutually exclusive, so it is useful to review the
other options to see if their good parts can be
grafted onto the leading option.
Consultation with external experts and with those
affected is very important at this stage, whether or
not formal or informal consultation has taken place
Consultation on projects will usually be on one or
two lead proposals; whereas consultation on policy
and programme proposals that have more widespread
effects should usually be undertaken both earlier,
and on a wide range of options and alternatives.
6.15 Analysis of
who is affected by a proposal, undertaken as part of
the appraisal, may be very useful in determining who
should be consulted, and also in considering the
details of implementation. Attention should be drawn
to the key assumptions, options and implementation
issues. Consultation exercises should be drawn up in
line with the following best practice guidelines :
- Use the
most appropriate approach. Written
consultation may not the best way to canvass
views on a policy or project option. Methods
include meetings with interested parties and
should be easy to respond to (e.g., by
- Check if
statutory obligations apply.
sufficient time; consultation should be built
into the planning process at the start.
- Be clear
about who is being consulted, about what, in
what time-scale, for what purpose.
joining up with other consultations, for
instance in other government departments.
documents should be clear, concise and
- Ensure that
the process reaches the target audience.
- Ensure that
people are told the results, and the reasons
for decisions taken.
Implementation plans should be sufficiently complete
to enable decisions to be taken on whether or not to
proceed. So that evaluations can be completed
satisfactorily later on, it is important that during
implementation, performance is tracked and measured,
and data captured for later analysis.
Economically justifiable and financially affordable
proposals are of no value if realistically they
cannot be implemented. The implementation of
proposals must be considered as part of the appraisal
process, enough to ensure at least that proposals are
viable, risks are manageable, and that benefits can
be realised, before significant funds are committed.
These aspects of appraisal develop iteratively as
with the analysis of costs and benefits.
management and measurement
management concerns tracking the success of a policy,
programme or project in achieving its objectives and
in securing the expected benefits. For appraisal and
evaluation purposes, it involves the systematic
collection of data relating to the financial
management and outcomes of the policy, programme or
project during implementation.
performance measurement and monitoring means tracking
all categories of benefit and ensuring that:
have defined target benefits and outputs;
of the delivery of benefits remains with the
- Outputs of
a project or policy remain consistent with
changing government objectives;
- Targets and
achieved benefits are measured, reported and
- Costs are
closely monitored and managed; and,
costs and benefits are frequently reviewed.
monitoring system should establish:
management data is actually measuring what it
purports to measure; and,
- Put in
place sufficient controls to ensure that the
data is accurate.
financial reporting on policies, programmes and
projects should be performed. Reports may be
integrated into the normal financial reporting cycle
of an organisation, issued separately, or possibly
combined with the reporting of progress against plan,
benefits, and risks.
realisation management is the identification of
potential benefits, their planning, modelling and
tracking, the assignment of responsibilities and
authorities and their actual realisation. In many
cases, benefits realisation management should be
carried out as a duty separate from day to day
fall into four main categories, which are described
BOX 24: BENEFIT
||Operating cost reduction,
||Number of customer
complaints, reduction in road accidents,
percentage of government departments on-line
||Staff skills, staff morale
||Improved standards of
6.35 It is also
useful to identify financial savings that release
cash for other uses.
7.1 When any
policy, programme or project is completed or has
advanced to a pre-determined degree, it should
undergo a comprehensive evaluation. Major or on-going
programmes, involving a series of smaller capital
projects, must also be subject to ex post
examines the outturn of a policy, programme or
project against what was expected, and is designed to
ensure that the lessons learned are fed back into the
decision-making process. This ensures government
action is continually refined to reflect what best
achieves objectives and promotes the public interest.
comprises a robust analysis, conducted in the same
manner as an economic appraisal, and to which almost
identical procedures apply. It focuses on conducting
a cost benefit analysis, in the knowledge of what
actually occurred rather than what is forecast to
evaluation itself would normally follow this
exactly what is to be evaluated and how past
outturns can be measured.
alternative states of the world and/or
alternative management decisions as
- Compare the
outturn with the target outturn, and with the
effects of the chosen alternative states of
the world and/or management decisions.
- Present the
results and recommendations.
and use the results and recommendations.
requires management initiative (sometimes political
commitment) and intensive monitoring. The
thoroughness of an evaluation should depend upon the
scale of the impact of a policy, programme or
project, and to some extent on the level of public
interest. There may be a high level of media interest
around a project which has required a significant
degree of expenditure, or one which is highly
complex, novel, or represents a pilot for future
large scale programmes. Evaluation reports should be
widely disseminated and published, where appropriate,
to contribute to the knowledge base upon which future
decisions will be taken.
is to be evaluated
7.6 The activity
to be evaluated needs to be clearly specified. The
evaluation might be of a project, programme or
policy, particular aspects of the activity, or of key
common issues affecting a number of activities. It
might also be a pilot designed especially for
outcomes and outputs should be defined and quantified
as precisely as possible for use in step three below.
It is important to distinguish between the objectives
and outcomes, and the outputs and targets.
availability of output and performance measures and
targets, and other monitoring data, and how they
relate to the objectives should be reviewed. If this
information is inadequate, consideration should be
given to the collection of additional data, although
ideally, data needs would have been considered at the
outset of the project.
States / Management Decisions
definition of exactly what needs to be compared with
what needs to be clearly stated. The outturn of any
complex activity will never be exactly as projected
in advance. However, the reasons for the outturn
being better or worse than expected may be
attributable to the state of the world,
or to actions of the responsible body. These might
include the management of the project, forecasting
assumptions, or the inherent design of the policy.
Outturn with Targets
discussed earlier, the technical methodologies used
for appraisal and evaluation are similar. Each should
identify and measure, where possible, both the direct
and indirect benefits of the policy, programme or
project. The main difference is that evaluation tends
to be based on actual data, and appraisal on
forecasts and projections.
evaluation should include the following:
assessment, quantified where possible, of
comparison with the target outturn; and
comparative assessment of one or more
counterfactuals (i.e. alternative outturns
given different states of the world, or
different management decisions).
possible the comparative assessment should include a
control group, to whom the activity was
7.13 It is usual
to take as a benchmark for comparison, what would
have happened if the activity under consideration had
not been implemented. It is also useful to consider
the consequences of implementing one or more of the
alternatives considered during appraisal.
Occasionally it may be appropriate to consider an
option that was not originally appraised, as long as
it was feasible at the time of implementation.
evaluation should assess the success of the project,
programme or policy in achieving its objectives, and
also how this achievement has contributed to the
wider outcomes. If the objectives were not achieved,
the evaluation should establish why that was the
results and recommendations
7.15 The results
of an evaluation should summarise:
- Why the
outturn differed from that foreseen in the
effective the activity was in achieving its
objectives, and why;
- The cost
effectiveness of the activity; and
- What the
results imply for future management or policy
7.16 The results
obtained should generally lead to recommendations for
the future. These may include, for example, changes
in procurement practice, delivery, or the
continuation, modification, or replacement of a
results and recommendations
7.17 The results
and recommendations from evaluation should feed into
future decision making. The methods used to achieve
this will generally require senior management
endorsement. Efforts should be made to disseminate
the results widely, and, for this purpose, it may be
helpful to use summaries of the main points, and
reports which synthesise the results from a number of
evaluations with common features.
reports and the research that informs them should be
placed in the public domain unless there are good
reasons, in terms of security or commercial
confidentiality, for not doing so.
Appraisal and Evaluation
7.19 Box 25 sets
out the differences between undertaking an assessment
at the outset, in support of government intervention
appraisal and undertaking an assessment
to evaluate how successful such action has been
COMPARISON OF APPRAISAL AND EVALUATION
||Ex ante assessment of
whether action is worthwhile and impacts
||Ex post assessment of
whether action was worthwhile and impacts
|Use of Output
policy and programme design
(a) future procurement, project management,
(b) wider policy debate, and
(c) future programme management.
||Projects, policies and
||Projects, policies and
||Always prior to
||. During implementation
. After implementation
||Historic and current,
estimated and actual.
Estimates of counterfactuals
||Comparison of options
against do nothing option
assessment of risk
|Comparison of results
against do nothing option.
Comparison of actual outturns against target
outturns/ alternative outturns.
Assessment of risks that did or did not
cash flow analysis.
Other statistical analysis.
cash flow analysis.
Other statistical analysis e.g.:
analysis of performance indicators.
|Comparison of NPV, NPC for
Non quantifiable factors may be included if
|Consideration of whether
correct criteria were used
|Public Accounts Committee
(PAC), NAO, HMT, OGC Gateways 0, 1.
||PAC, NAO, HMT, OGC Gateway