Effectiveness of Benefits

Management Frameworks
for monitoring and controlling
public sector projects in the
United Kingdom
Ahmed AbuElmaati, Trym Sørensen Bernløv

Department of Business Administration

Master's Program in Management
Master's Thesis in Business Administration I, 15 Credits, Autumn 2020
Supervisor: Dr Medhanie Gaim
Effectiveness of Benefits
Management Frameworks
in monitoring and controlling public
sectors projects in the United
Kingdom

Ahmed M AbuElmaati Omar, and Trym Sørensen Bernløv

Department of Business Administration

Master's Program in Management
Master's Thesis in Business Administration I, 15 Credits, Autumn 2020
Supervisor: Dr Medhanie Gaim
Abstract
Purpose – This research aims to explore the effectiveness of utilising Benefits
Realisation Management (BRM) as part of comprehensive success measures,
emphasising the stage in-between appraisal and evaluation of projects in the UK public
sector.

Design/methodology/approach – The study is constructed as a qualitative case study.

Semi-structured interviews are used as part of the inductive, exploratory approach to
achieve the study's objectives. It employs an approach based on grounded theory for its
analysis.

Findings – This paper suggests that Benefits Realisation Management is not used
effectively in the UK public sector during projects lifetime to control and monitor
projects and ensure their success. The current reviews of projects and programmes,
through their execution, may not be sufficient.

Research limitations/implications – This study offers contributions to the project

success literature and benefits management literature by adding empirically supported
insights about BM utilisation during project reviews. The research may be limited
primarily by the research method – predominantly the snow-balling data collection. The
assumptions made about the UK public sector may limit the broader generalisation of
the findings.

Practical implications – This research may be used to advise the practising managers
of the need to maintain benefits orientation after appraisal throughout a project's
lifetime and after delivery. Project governance structures are advised to update and
improve their current project review practices. The study additionally identifies possible
obstacles to the process and biases.

Originality/value – This paper attempts to fill a literature gap by providing empirical

results that explore the success definition and measures and the effectiveness of BRM
during project execution and gate reviews.

Keywords: Benefits Management; Project Success; Project Performance; Performance

Measurement; Public Sector.

Paper type: Research Thesis

II
Acknowledgement
The authors would like to take this opportunity to express their sincerest appreciation to
those who have made the writing of this thesis and completion of the Master's
Programme in Strategic Project Management (European) a possibility.

First and foremost, we would like to thank our supervisor at Umeå University, Dr
Medhanie Gaim, for his guidance and feedback throughout the writing of this thesis.
Also, we would like to recognise our supervisor at Heriot-Watt University, Mr Bryan
Rodgers, who has been of immense help and support throughout the completion of our
Master's Programme – both whilst studying in Edinburgh and in Umeå. We would also
like to thank Dr Tomas Blomquist for making the transition from Heriot-Watt to our
final semester in Umeå easier.

Secondly, we would like to extend our deepest gratitude to the participants of this study
for sharing their opinions and experiences, whose contributions made the completion of
this research a reality.

Finally, we would like to thank our families, friends, and close ones for their continued
support and encouragements from the very start of our journey pursuing this higher
educational degree until the end.

Ahmed Magdi AbuElmaati Trym Sørensen Bernløv

Cairo, 8th of March 2021 Bergen, 8th of March 2021

III
Table of Contents
1. Introduction ......................................................................................................... 1
1.1 Background .............................................................................................. 1
1.2 Research Gap ........................................................................................... 3
1.3 Research Motivation ................................................................................ 4
1.4 Research Question and Objectives ........................................................... 5
1.5 Unit of Analysis ....................................................................................... 6
1.6 Practical and Theoretical Contributions ................................................... 7
1.7 Relevant Concepts .................................................................................... 7
1.8 Outline of the Research Disposition ........................................................ 9

2. Theoretical Frame of Reference ........................................................................ 11

2.1 Overview ................................................................................................ 11
2.2 The Link Between Projects and Strategy ............................................... 12
2.3 Project Selection and Portfolio Management ......................................... 13
2.4 Project Success ....................................................................................... 14
2.5 Value and Benefits ................................................................................. 16
2.6 Benefits Realisation Management.......................................................... 18
2.6.1 Development of BRM ............................................................... 18
2.6.2 The Managerial Process ............................................................. 19
2.6.3 Benefits Management Challenges ............................................. 21
2.7 Benefits in the Context of UK Government Guidelines ........................ 22
2.8 Summary ................................................................................................ 24

3. Research Methodology ...................................................................................... 26

3.1 Research Philosophy .............................................................................. 26
3.1.1 Ontological Stance ..................................................................... 27
3.1.2 Epistemological Stance .............................................................. 28
3.1.3 Axiological Stance ..................................................................... 28
3.1.4 Philosophical Paradigm ............................................................. 29
3.1.5 Research Approach .................................................................... 29
3.2 Research Design ..................................................................................... 30
3.2.1 Methodological Choice.............................................................. 30
3.2.2 Research Strategy ...................................................................... 32
3.2.3 Time Horizon ............................................................................. 32

IV
 3.3 Data Collection and Analysis ................................................................. 33
3.3.1 Data Collection Methods ........................................................... 33
3.3.2 Interview Protocol ..................................................................... 34
3.3.3 Sample Selection ....................................................................... 34
3.3.4 Data Collection Process ............................................................. 35
3.3.5 Data Analysis Technique ........................................................... 36
3.4 Quality Criteria....................................................................................... 37
3.4.1 Reliability (Dependability) ........................................................ 38
3.4.2 Internal Validity (Credibility) .................................................... 39
3.4.3 External Validity (Transferability) ............................................ 39
3.5 Ethical Considerations ........................................................................... 40
3.6 Methodological Statement ..................................................................... 41

4. Empirical Data and Findings ............................................................................. 42

4.1 Empirical Data ....................................................................................... 42
4.1.1 Data Sources .............................................................................. 42
4.1.2 Analysis ..................................................................................... 44
4.2 Findings .................................................................................................. 45
4.2.1 Project Management and Benefits Management Frameworks and
Governance ......................................................................................... 47
4.2.2 On Success: Its Definition, Measures, and Factors ................... 50
4.2.3 BRM Effectiveness .................................................................... 54
4.2.4 BRM Effectiveness during Implementation .............................. 56
4.2.5 Negative Influences and Obstacles on BRM Effectiveness ...... 58

5. Discussion.......................................................................................................... 61

6. Conclusion ......................................................................................................... 65
6.1 Closing Remarks .................................................................................... 65
6.2 Theoretical Implications......................................................................... 65
6.3 Managerial Implications......................................................................... 66
6.4 Future Research Agenda ........................................................................ 67

References ............................................................................................................. 68

Appendix 1: Benefits Map..................................................................................... 74

Appendix 2: Participation Introduction Letter....................................................... 75

Appendix 3: Interview Protocols ........................................................................... 76

V
 Version 1: ..................................................................................................... 76
Version 2: ..................................................................................................... 78
Version 3: ..................................................................................................... 81

Appendix 4: Code Book ........................................................................................ 84

Appendix 5: Extended Gioia Presentation and Codes Map ................................. 89

List of Figures
Figure 1. Concepts related to benefits management (Authors). ....................................... 8
Figure 2. Inverted pyramid illustration of relevant theoretical subjects (Authors). ....... 11
Figure 3. Using projects to add value to the business (Authors, modelled after Serra &
Kunc, 2015, p. 55). ......................................................................................................... 12
Figure 4. Breakdown of success factors as identified by the authors (Authors). ........... 16
Figure 5. Benefits Lifecycle as proposed by Minney et al. (2019, p. 6). ....................... 18
Figure 6. Classification of social costs and benefits. Model extracted from The Green
Book (HM Treasury, 2020, p. 23). ................................................................................. 18
Figure 7. The research onion, as proposed by Saunders et al. (2019, p. 130) (Authors).26
Figure 8. The research onion, our methodological selections (Authors)........................ 41
Figure 9 Data Structure presentation; inspired by Gioia method (Authors)................... 46
Figure 10. Iron Star, 6-point criteria for success (Authors)............................................ 52
Figure 11. BRM effectiveness findings mind map (Authors). ....................................... 54
Figure 12. BRM effectiveness is determined by the interaction of several dimensions
(Authors)......................................................................................................................... 61
Figure 13 Benefits mapping; adopted from: (Serra, 2017, p.96-101; Serra and Kunc,
2015, p.56; Ward and Daniel, 2012, p. 118) .................................................................. 74
Figure 14 First and second order codes map (extracted for Nivio12 project) ............... 92

List of Tables
Table 1. Definitions of relevant terms. ............................................................................. 8
Table 2. BRM concepts in practice, adapted from Serra & Kunc (2015, p. 57) and
Breese et al. (2015, p. 1447). .......................................................................................... 20
Table 3. HM Treasury 5 business cases; adapted from HM Treasury (2018a; 2018b). . 23
Table 4. Key Elements of Benefits Realisation assurance reviews; adapted from (IPA,
2016). .............................................................................................................................. 23
Table 5. Quality Criteria. ................................................................................................ 38
Table 6. List of interviews. ............................................................................................. 42
Table 7. List of participants. ........................................................................................... 42
Table 8. Summary of initial codes categorises arranged alphabetically......................... 44
Table 9. Implementation gap between the theoretical definition of success and practice.
........................................................................................................................................ 52
Table 10. Negative influences and obstacles against BRM effectiveness. ..................... 58
Table 11 Extended presentation inspired by Gioia method ............................................ 89

VI
List of Abbreviations
ALB Arm’s-length Body
APM Association for Project Management
BAU Business-as-usual
BM Benefits Management
BRM Benefits Realisation Management
IPA Infrastructure and Projects Authority
MSP Managing Successful Programmes
NHS National Health Services
OGC Office of Government Commerce
PM Project Management/Project Manager
PMBOK Project Management Body of Knowledge
PMI Project Management Institute
PMP Project Management Professional – a qualification by the PMI
PRINCE2 Projects in Controlled Environment 2
QALY Quality-Adjusted Life-Year
UKPLC United Kingdom Public Limited Company – an expression used to refer
to the wider UK public, especially the commercial environment and their
interests

VII
1. Introduction
1.1 Background
Business strategies often imply organisational change or process improvement. Projects,
programmes, and portfolios are believed to be the vessel to achieve these strategic
objectives through benefits realisation (PMI, 2017, p. 8-15; APM, 2019, p. 25).
Consequently, an increasing number of organisations employ project-oriented
management for both increasing external performance and internal client satisfaction
(Aalto, 2000, p. 1). Similarly, in the public sector context, government policies are
typically materialised through projects and project management structures (IPA, 2017,
p. 7). Thus, it is crucial to employ sound and effective project, programme, and
portfolio management techniques to ensure the successful implementation of business
strategies (Aalto, 2000, p. 1; Serra & Kunc, 2015, p. 53).

Benefits realisation is described as the pillar that business cases are built on and
ultimately the rationalisation for executing a particular project (APM, 2019; IPA, 2017;
PMI, 2017, p. 33,75; Farbey et al., 1999). It is a vital measure of an initiative's
perceived and actual success. Success can be determined by the benefits realised from a
project in addition to its ability to relieve dis-benefits (negative benefits). Thus, benefits
realisation is an important concept to understand in business value creation.

However, benefits management practice adoption is still limited in both project

management and general management. Breese et al. (2015, p. 1439) have reported the
BRM adoption is a potential gap in the literature. Furthermore, according to the latest
edition Pulse of the Profession report, only 60% of the organisation apply benefits
realisation management techniques (PMI, 2020, p. 10). The percentage listed under
"strategy alignment and business value" is an alarming indicator. According to the same
report, benefits realisation maturity has also been reported as the least mature
performance measure (PMI, 2020, p. 15). Despite BRM being a vital concept, it might
be under-utilised in practice. Changing this is believed to positively contribute to higher
projects' success rate (Badewi, 2016; Breese et al., 2015, p. 1438; Serra & Kunc, 2015,
p. 64). Therefore, there is a ground for further research and investigation to understand
the exact situation and the reasons that led to it.

The BRM concept initially appeared to address failure in information technology

projects, as those projects were significantly different from mainstream projects like
construction, engineering or product development projects (Breese et al., 2015, p. 1440-
1442). IT projects typically had an internal client, whereas the mainstreams' clients were
typically external. The expected value and motivation of IT projects are also entirely
different from the predominated commercial mainstream projects. Therefore, new
techniques and practices to assess the project's value, like Benefits Realisation
Management (BRM), had to be developed and implemented.

The public sector has been chosen to understand benefits management's concept and
practices for various reasons. Historically, the governmental and public sector context
was the incubator in which the BRM concept initially appeared, developed, and matured
(Breese et al., 2015, p. 1442). According to Breese et al. (2015, p. 1442-1444), the

1
governments' influence– along with professional bodies – was significant in developing
BRM guidelines in the establishing phase. The spread of the benefits management
practices and guidelines occurred as the public sector employed it as a tool to improve
the procurement process through gateway reviews (Breese et al., 2015, p. 1442).
Government literature forms an integral part of the literature about BRM (Breese et al.,
2016) – for example, through the Infrastructure and Projects Authority (IPA) Guide
(2017) and the Green Book (HM Treasury, 2020).

The unique settings, structure and complexity of the public sector differentiate its
management from the private sector. In the private sector, the commercial value and
future financial returns from activities dominate projects' valuation. On the other hand,
in the public and third sector projects, the social benefits are not a mere secondary
consideration but rather a central one. Additionally, the public sector finance and
governance are more involved in the form of extensive monitoring and reporting. For
instance, in specific major projects, governance can reach a high level as direct
parliamentary reporting. Therefore, the public sector's organisational structure,
consisting of several government layers, has a developed complexity and oversight – for
example, through inter-agency and intra-agency monitoring and reporting, and the
organisations' need to align their practices with rigorous governmental guidelines and
mandates (Williams et al., 2020, p. 645).

Whilst the UK's government structure is not the focus of this study, it is beneficial to
establish a background to understand the political influence on the project's selection
process and execution. When considering the public sector in the United Kingdom, the
different government levels, or governmental structures, must also be considered.
Hence, this research needs to highlight the central government influence through
organisations like Her Majesty's Treasury and the IPA (Infrastructure and Projects
Authority), both of which have authored benefits management guidelines (HM
Treasury, 2020; IPA, 2017). However, most projects in the public sector are executed
by the local government and Arm's-length Bodies. Arm's-length Bodies (ALB) in the
United Kingdom is a term that refers to a variety of public bodies that are independent
of a ministry, yet publicly funded to provide government services (Institute for
Government, 2015, p. 90). These can be divided into four categories depending on how
close their ties are to a ministerial department, ranging from non-ministerial
departments to executive agencies, non-departmental public bodies, and public
corporations (Institute for Government, 2015, p. 90). Therefore, the variety of ALB's is
vast, with examples spanning from HM Revenue & Customs to Public Health England,
Transport for London (TfL), and Channel 4 (Institute for Government, 2015, p. 90).
What makes these organisations unique is the influence exerted on them by both central
government departments (for example, Department for Transport in the case of
Transport for London) and the local government (for example, city councils) where they
and their direct stakeholders geographically reside. Furthermore, there might be
individual differences between the four nations (England, Scotland, Wales, and
Northern Ireland) that the authors consider out of this thesis's scope.

2
1.2 Research Gap
As a starting point for the literature gap, in the International Journal of Project
Management call for papers, the editor (Zwikael, 2014, p. 543) suggests the following
research questions:

"How can project benefit management enhance the achievement of organisational

strategic objectives?[...]
What is the relationship between project efficiency and effectiveness?[...]
What are the implications of benefit management research on project governance, the
concept of project success, and the project management tool kit?"

Based on these suggestions, it can be deduced that there is a literature gap regarding
benefits management itself and its relationship to project management. These questions
were taken into consideration when developing this thesis as possible areas of
exploration and research, and later when scooping its focus.

As aforementioned, according to PMI (2020, p. 15), benefits realisation management is

the least mature performance measure. This fact indicates barriers to further widespread
adoption and effectiveness. Notwithstanding benefits realisation management being a
vital concept, it is under-utilised in practice with only 25% of organisational adoption
(Ward et al., 2007). These elements call for research and investigation to understand the
exact situation and the reasons that led to it.

From an academic perspective, benefits management is thought to be an under-

researched area of project management compared to other aspects of classical project
management such as time management and risk management (Serra & Kunc, 2015;
Breese et al., 2015, p. 1439; Ika, 2009). BRM is frequently mentioned as a subject for
future research (Thorp, 2003; Ward et al., 2007). Breese et al. (2015, p. 1439) suggest
that the current literature is mainly 'how to guides' like Bradley (2006) and Thorp
(2003) or 'analysis of processes and practices' like Breese (2012), Serra & Kunc (2015),
and Ward et al. (2007).

However, the current research suggests that BRM combined with high project
performance is essential to ensure project success and subsequential value creation
(Badewi, 2016, p. 761; Serra & Kunc, 2015, p. 64). Badewi (2016) has concluded that
the combination of project management and benefits management is valuable for a
higher rate of project success. However, there is a need for empirical data to support
this. Williams et al. (2020) attempted to address this research gap. Using a cross-
national study on the public sector, they found that a shift occurs after a project's
appraisal where the focus on the project management performance comes at the expense
of benefits and benefits management practices. In turn, they (Williams et al., 2020, p.
653) suggest the need for more study of how exactly projects can be managed better,
further exploring BRM effectiveness. Therefore, it is evident that the area has
considerable potential for further development and research. Hence, this study will
attempt to build on the previous findings of both Badewi (2016) and Williams et al.
(2020) to contribute to the literature on BRM practices, specifically during a project's
lifetime.

As a final remark, benefits management has also started to receive more traction and
attention in the past years (Breese, 2012, p. 341), emphasising its relevance in

3
contemporary research. However, there is still a lack of research in many related areas
like monitoring tools for owners and steering committees (Zwikael, 2014, p. 543).

1.3 Research Motivation

The prevalence of such a clear literature gap in BRM and the scientific traction for more
research on the topic makes it encouraging to pursue the subject. However, this traction
is not sufficient for justifying the author's selection. The author's robust research
motivation is rather formed by combination factors. These factors can be traced to the
importance of public sector projects, the importance of BRM in value management and
investment success, and the link between BRM development and the public sector.
These motivational factors will be further elaborated on through this section.

Public sector projects are vital due to the immense value of investments made in them.
Simultaneously, the need to translate these investments into economic, financial, social,
and environmental values is pressing. Hence, the importance of BRM as an investment
success measure and a managerial practice. From a financial or an economic
perspective, governmental spending is a significant part of a nation's GDP, with vast
macroeconomic and social development implications. According to the Scottish
Parliament Briefing (Hudson & Thom, 2019, p. 1), the Scottish government had spent
over £11.1 billion on infrastructure projects alone since 2007, with an estimated £3.7
billion under construction in 2020. The Scottish government plans to further increase
these figures by an additional 1% of the GDP to reach the level of 3.6%, raising the
annual infrastructure investments to £6.7 billion by 2025-2026 (Infrastructure
Commission for Scotland, 2020, p. 14). Overall, HM's Treasury figures in 2019/2020
put the UK government spending at an amount equivalent to over 35% of the national
GDP or over £880 Billion (Statista, 2020, p. 2-3). Therefore, due to the substantial size
of the sector spending, it is crucial to achieve the best accumulative value and ensure
that the spending of taxpayer’s money is well optimised.

Therefore, considering this increasing capital investment and the size of vast executive
bodies that spend on behalf of the government, it is crucial to establish practical
measures and criteria for investment success in creating value. These measures need to
encompass the entire project lifecycle, starting from selection and appraisal until after
delivery. These measures need to encompass the entire project life cycle, starting from
selection and appraisal until after delivery. Consequently, it is essential to ensure
effective portfolio management to implement governance policies and achieve strategic
objectives. This study will implement the rationale that this can be achieved through a
sound balance of projects portfolios, established through effective benefits realisation
management as the variety of different economic, social, and political dimensions in
question increases the need for such a robust and practical framework.

On the other hand, despite benefits management being a vital thirty years old concept, it
is still encompassed in ambiguity that may form an obstacle for wide-spread adoption
(Breese et al., 2015, p. 1438-1439). This is especially prominent with the presence of
similar terms like value and value management (Breese et al., 2015, p. 1449; Breese et
al., 2016; Laursen & Svejvig, 2016, p. 736). As elaborated above in the research gap,
there is significant room for a more profound and further understanding of the practice,
value, adoption rate, maturity, challenges, and enablers.

4
Additionally, the combination of BRM in the public sector was chosen due to its
maturity in term of project management and benefits management practices (Williams
et al., 2020, p. 646). Benefits management is thought to be both mature and prevalent in
this sector. Historically, the concept has its roots in the public sector – specifically
information technology and systems (IT/IS) projects (Breese et al., 2015, p. 1440-1442).
Furthermore, the presence of the embossed practices and frameworks such as IPA and
HM's Treasury guidelines makes the system interesting for analysis to understand the
gap between the prescribed framework and actual practice, shedding light on the
perceived effectiveness and drawbacks of such frameworks (Williams et al., 2020).

Finally, the research is also motivated by the expected managerial and theoretical
implications. The authors see merit in the understanding of the actual BRM practices
and the obstacles that practitioners may be facing. Therefore, this thesis’s potential
usefulness for managers and academics alike is strengthening the motivations. The
expected implications will be further discussed in detail later in this chapter.

1.4 Research Question and Objectives

This thesis aims to explore the use of benefits management practices as part of the
definition of UK public sector projects' success. The study's aim is further focused on
BRM involvement during the specific phase of project implementation, which occurs
after appraisal and before delivery.

Therefore, the study will contribute to both practitioners and academics knowledge
through researching the answer to the following research question:

How effectively is benefits realisation management (BRM) used

in the UK's public sector during projects execution to ensure
success?
In order to achieve the study’s research objective and answer the research question, the
exploration process has to follow a few main stages or milestones conceptually.

The research's first milestone is to establish a sound theoretical foundation through an

extensive literature review. It is elemental for the study to form a good understanding of
the literature to differentiate whether the issues encountered are a matter of lack of
literature and guidelines or a practical implementation deficiency. This literature review
will include a comprehensive review of previous academic research on the subject and
look into practitioners and governmental guidelines.

The second milestone is achieved through examining and creating an empirical

understanding of benefits management practices' effectiveness in general. The purpose
is to either confirm or question the results of Williams et al. (2020), who found a lack of
benefits management frameworks' utilisation after the appraisal and approval stages of
the business case study phase.

Based on the understanding and rationalisation established through the first two stages,
a foundation is laid to achieve the thesis's primary objective. The main objective – to
explore benefits management utilisation as a control and monitoring practice that

5
ensures success during the project lifetime – is realised by collecting and analysing
empirical data. As a result, our knowledge on this phase of projects success measures
will be further expanded.

The last stage of this thesis is to attempt to form an understanding deeper than reporting
observations. This step attempts to answer questions like “why, or why not, is benefits
management effectively utilised as a success measure?” Additionally, through this step,
the authors will attempt to identify the enablers and barriers for benefits management in
general and as a tool to review the progress of projects.

To conclude this section about the thesis objectives, the authors find it necessary to
clarify the study's exact scope and its delimitations. Additionally, also clarify few
assumptions or specific use of the terms that might be a potential source of confusion.
First, by "effectiveness" of benefits management practices, the authors generally refer to
how effective BRM is utilised. The thesis focuses on the use of BRM and whether it is
in its best form. The merits of the BRM and whether it is a valuable practice are out of
this study's scope. The study is rationalised by assuming that it is a valuable practice
based on others like Badewi (2016).

Furthermore, few terms are used loosely as the authors believed that a clear discern will
not reflect the work's quality. Most importantly, the terms like implementation phase
and executions phase are treated as synonyms. They are also used to refer to the specific
stage of a project cycle and the whole project lifetime in some situations. A
knowledgeable reader will be able to make the distinction through context. Therefore,
the authors opted not to dwell on the issue when the main purpose is to illustrate the
BRM role in control processes. However, it is crucial to distinguish that the thesis
focuses on the stage between appraisal and evaluation. Finally, the term project
management is sometimes used to describe the practices and the processes that can also
be applied to programmes or even portfolios management.

1.5 Unit of Analysis

This study has selected UK public sector projects as the unit of analysis. Through
interviewing public sector project managers and benefits managers, a case study was
formulated to gain insight into the effectiveness of benefits management utilisation to
monitor and control the execution of strategic projects in portfolios.

As identified by literature on the topic (Badewi, 2016; Breese et al., 2015; Breese et al.,
2016; Serra & Kunc, 2015; Williams et al., 2020), despite the belief in benefits
management vitality, the topic is still not sufficiently researched and is lacking in terms
of unifying theories and universal frameworks. The area of applying benefits
management concept during a project's life cycle that follows the initial approval of the
business cases is, as mentioned previously, specifically under-represented in the
literature and actual practice (Williams et al., 2020). According to Williams et al.
(2020), the area, in particular, was identified in need of further research and exploration.

6
1.6 Practical and Theoretical Contributions
As a result of the lacking maturity of benefits management that has previously been
mentioned, the authors are hoping to contribute to improving the practice of BRM by
highlighting potential barriers currently opposing its implementation during the
execution phase of projects, particularly in the UK public sector. These problems might
not only materialise as a consequence of benefits management ambiguity but also from
the process of benefits management itself in today's form. Furthermore, the researchers
hope to uncover practical, underlying issues in implementing benefits management as a
managerial tool in the UK public sector and possible contributing factors. Finally, the
authors hope to contribute to practitioners' recommendations to overcome these
challenges.

On the theoretical level, the authors hope to contribute with knowledge that can uncover
current trends and concerns in benefits management implementation during the
execution phase of projects. Establishing connections between factors that contributes to
– or potentially impair – the benefits management process is also desired. The
researchers also hope to promote future standardisation to lessen the ambiguity
surrounding the subject, potentially promoting it as a useful managerial tool.
Additionally, the authors hope that this research can fill identified knowledge gaps in
understanding BRM after the appraisal phase, with particular reference to the work of
Williams et al. (2020). Finally, it is intended to identify previously uncovered areas and
make suggestions for further research. Hereby, the authors are hoping for this thesis to
act as a catalyst for additional benefits management development and encouraging other
academics to continue filling the knowledge gaps.

1.7 Relevant Concepts

To fully understand benefits management, the authors have identified several related
concepts to it, which will be briefly defined. The links between certain concepts and
benefits management frameworks will be further clarified and discussed in the literature
review section of this paper when relevant.

Benefits realisation is strongly related to an organisation's strategy and its strategic

objectives as one of its essential processes is benefits mapping (Minney et al., 2019, p.
1). Benefits mapping is the illustration of how measured benefits from project outcome
are linked to the strategic objectives of the organisation (Minney et al., 2019, p. 37). In
other words, benefits are the rationalisation of a project and the way to achieve a
strategy. Therefore, it is essential to understand the concepts of 'project success' and
how they relate to benefits realisation. In addition to knowledge management,
governance, portfolio management, and programme management are near related
concepts to BRM (Breese et al., 2015, p. 1449). Key concepts are illustrated in Figure 1.

7
 Strategy and
Portfolio
Managment

Stakeholder Benefits Project

Management Managment Success

Value
Creation

Figure 1. Concepts related to benefits management (Authors).

An assortment of terms that are frequently used while discussing benefits management
are listed in Table 1 below. The purpose of this table is to provide a short summarisation
to start the conversation on the topic. By themselves, they do not provide a
comprehensive definition but rather serve as an identification of key terms. Whereas
providing a complete definition of all the terms is essentially a significant issue in
understanding and effectively utilising benefits management, such definitions are
beyond the scope of this work. Therefore, the authors rely on the definition of
recognised industry partitioners.

Table 1. Definitions of relevant terms.

Term Definition Reference

Agile "A family of development methodologies where APM
requirements and solutions are developed iteratively and (2019, p.
incrementally throughout the life cycle." 209)
Gateway "Project gateways correspond to go/no-go decisions that are Morris &
typically situated at significant milestones and concern Pinto (2007,
major project deliverables or phases." p. 131)
Gateway Review "Gateway review processes subject projects and Turner
programmes to predetermined review and approvals and (2007, p.
provide executive owners with a mechanism for oversight, 700)
monitoring and control."
Governance "The framework of authority and accountability that defines APM
and controls the outputs, outcomes and benefits from (2019, p.
projects, programmes and portfolios. The mechanism 212)
whereby the investing organisation exerts financial and
technical control over the deployment of the work and the
realisation of value."
Key Performance "A measure that demonstrates whether a company or IPA (2017,
Indicators organisation is achieving key business goals." p. 53)

8
 Operations "The on-going operational environment." IPA (2017,
Management and BAU may be used in the context of PM to refer to the p. 52)
Business as Usual permeant organisation in contrast to temporary (transit)
(BAU) projects.
Portfolio "A collection of projects and/or programmes used to APM
structure and manage investments at an organisational or (2019, p.
functional level to optimise strategic benefits or operational 214)
efficiency."
Programme "A unique, transient strategic endeavour undertaken to APM
achieve beneficial change and incorporating a group of (2019, p.
related projects and business-as-usual (steady-state) 214)
activities."
Project "A unique, transient endeavour undertaken to bring about APM
change and to achieve planned objectives." (2019, p.
214)
Project "The application of processes, methods, knowledge, skills APM
Management and experience to achieve specific objectives for change." (2019, p.
214)
Sponsor "A critical role as part of the governance board of any Minney et
project, programme or portfolio. The sponsor is accountable al. (2019, p.
for ensuring that the work is governed effectively and 69)
delivers the objectives that meet identified needs."
Stakeholder "Individuals or groups who have an interest or role in the Minney et
project, programme or portfolio, or are impacted by it." al. (2019, p.
69)
Strategic "These express the planned objectives of the organisation – Minney et
Objectives what they want to achieve in the future; the vision for the al. (2019, p.
company." 69)
Strategy "An approach created to achieve a long-term aim, can exist IPA (2017,
at different levels within the organisation." p. 55)

1.8 Outline of the Research Disposition

This paper will continue with the literature review in the following chapter, where a
comprehensive assessment of previous theory will take place before dissecting the
benefits management concept from a contemporary theoretical point of view. The
literature review chapter will also link benefits management to other managerial
concepts and practices, like project success and portfolio management. The literature
review chapter will similarly investigate the governmental guidelines and attempt to
illustrate the formal perspective on the topic. This formal perspective is essential in
comparison to the actual practices that the thesis aims to investigate.

The third chapter of this study will be dedicated to the research methodology. It will
explain the researcher's choice of a semi-structured interview to conduct a qualitative
case-study. The chapter will elaborate on the philosophical and practical implications
leading to that choice, justifying the authors' selections. It will also account for the
research procedures and protocols for academic verification and correctness.

The fourth chapter will include a presentation of the author's selection of presentation, a
summary of the study's empirical findings, and a showcase of the results of the analysis.

9
It will attempt to categorise those findings in a logical and structured way. The data will
be presented along with the trends and dimensions identified.

The findings will be followed by a discussion chapter, in which the results and their
implications will be thoroughly examined. Here, the authors will link the theoretical
frame of reference with the findings, after which the authors answer the research
question.

The final chapter of this paper will be the conclusion, summarising the work in this
paper. Possible theoretical and practical implications will be highlighted before
suggesting areas for further research.

10
2. Theoretical Frame of Reference
According to Hart (1998, p. 1, 13), conducting a literature review is both fundamental to
understanding the topic being studied and also an integral part of any research process.
It acts as a written account on the exploration of a field of research, establishing the
academic understanding that later allows for discussion of the findings (Hart, 1998, p.
15). In this research, the literature review will be conducted as part of the theoretical
frame of reference. Thus, understanding the literature is determinantal to defining key
terminology like that of benefits, value, benefits management, and benefits management
frameworks. The theoretical frame of reference aims to establish an understanding and
baseline for further discussion of what benefits realisation management, success, and its
implementation to the project life cycle entails. The answer to this is crucial to
achieving the research objectives and answering the research question. For practical
purposes, the theoretical framework will start from the broader topics of project success
and portfolio management, then funnelling down before eventually discussing benefits
management's specifics during project implementation. It can, therefore, be visualised
as an inverted pyramid, as depicted in Figure 2.

Figure 2. Inverted pyramid illustration of relevant theoretical subjects (Authors).

The authors would like to point out that they have opted to restrict the search of
literature to that of peer-reviewed articles, government and industry guidelines, and
published books to ensure the relevance and quality of the literature being studied.

2.1 Overview
Benefits realisation is thought to be closely linked to value management and that the
creation of value results from realising specific benefits. The benefit is not the outcome
or the output of a project, but it is an outcome that results from the delivery of project
output and perceived as an advantage by one or more of the stakeholders (IPA, 2017, p.
51 ). Some authors (Williams et al., 2020) use outcome and benefit interchangeable,
which might spark confusion that the authors have elected to avoid. For this thesis,
project output, project outcome, and benefits will be considered three distinct aspects, as
illustrated in Figure 3, following the suit of Serra & Kunc (2015, p. 55). To further

11
clarify the difference, consider the example of a project to deliver a payroll system. The
project output would be the new payroll software. The outcome would be an increase in
productivity due to this software. The benefits are savings in the overhead costs by a
specific value, thus freeing capital.

curent project project desired

benefits
value output outcome value

Figure 3. Using projects to add value to the business (Authors, modelled after Serra & Kunc, 2015, p. 55).

Breese et al. (2015, p. 1441) report that benefits management as a managerial concept is
still understood and translated differently, with no encompassing unified framework,
which hampers its broader effective adaption. The overall literature on BRM is not as
developed as the literature on other aspects of project management (Breese et al., 2015,
p. 1439). The source of this ambiguity may be as fundamental as defining terms like
benefit and value (Breese et al., 2015, p. 1449). The terms of benefits management
(BM), benefits realisation management (BRM), and benefits realisation (BR) can be
used as synonyms and are used interchangeably by different authors (Badewi, 2016, p.
762; Breese et al., 2016, p. 2). This thesis's authors will similarly use the terms BRM
and BM interchangeable as they refer to the same management concept. However,
benefits realisation is sometimes used to refer to a specific phase of the wider life
process of BRM (Breese et al., 2016, p. 2).

Project success is dependent on the perspective and perception of the evaluator (Ika,
2009). Therefore, understanding the relationship between benefits management and
stakeholder's management is essential. What one stakeholder define as a benefit might
be described as a dis-benefit by another. Knowledge management is key to projects
success and project management maturity (Todorović et al., 2015, p. 772). Knowledge
management is related to BRM, like other management techniques; as the process is
reviewed and learned, lessons are transferred to other organisation's endeavours.

2.2 The Link Between Projects and Strategy

Projects are often defined as temporary endeavours that exclusively set out to produce a
predefined output (Laursen & Svejvig, 2016, p. 736). However, this is an ageing
perception from a time when projects were dominantly implemented for product or
service developments resulting from market demand or customer request (Laursen &
Svejvig, 2016, p. 736). In the contemporary age of project management, several other
sources of motivation can be identified. Social needs, environmental concerns,
technological advancement, and forecasted problems are all examples of considerations
leading to project initiation today (Meredith et al., 2018, p. 1). Common for these is how
they link to the strategic intent of the parent organisation. Hence, a projects' purpose is
to harvest opportunities that align with overall organisational strategic goals (PMI,
2017, p. 546; Turner, 2007, p. 1). Nevertheless, ensuring strategic alignment between
projects and organisational goals has proven to be difficult (Aalto, 2000, p. 33). Many
reasons for this have been cited in the literature; however, most commonly mentioned is
the sole focus of project managers to successfully complete their projects according to
process measures (Aalto, 2000, p. 33). Nonetheless, ensuring strategic alignment falls
outside of the expectations set for project managers (Meredith et al., 2018, p. 12). A

12
critical part of this alignment process has been cited by many scholars to necessitate
upstream activities such as a distinct project selection practise (Haniff & Fernie, 2008,
p. 6). Thus, another entity linking the projects themselves with the organisation as a
whole must take the place of an intermediary, such as portfolios or programmes.

2.3 Project Selection and Portfolio Management

Organisations – both public, private, and non-profit alike – are all constrained by
limited resources. Whilst they have their differences, for instance, depending on the
industry, they all have an assortment of objectives to accomplish in order to fulfil
stakeholder or shareholder needs, justifying their existence and continued support.
Furthermore, these contemporary organisations incorporate a high degree of projects
into their daily operations to achieve their objectives. In order to optimise their
portfolios and achieve the right mix of projects, certain methodologies need to be
utilised.

This selection process varies from organisation to organisation, typically starting with
analysing net-present value (NPV) or internal rates of return (IRR) (Ben-Horin & Kroll,
2017, p. 108). Nevertheless, no organisation can invest in all available projects that are
modelled to provide a profit. As a result of resource scarcity, some projects are
prioritised at the expense of others. These can emerge as unsystematic bottlenecks like
financial limitations, manufacturing and procurement lead-times, and limited
availability of skilled professionals, or systematic bottlenecks such as restrictions
imposed by trade agreements (Larson & Grey, 2018, p. 33). It therefore quickly
becomes apparent that other aspects than those of pure financial character must be
considered.

The selection of projects is typically the responsibility of portfolio managers to ensure

effective resource allocation. Benaija & Kjiri (2015, p. 134) emphasises the competing
nature of projects in a portfolio environment. They define a portfolio as "a collection of
single projects and programmes that are carried out under a single sponsorship and
typically compete for scarce resources" (Benaija & Kjiri, 2015, p. 134). Here, Benaija &
Kjiri (2015, p. 134) highlights the fact that managing constraints is an essential part of
portfolio management. Portfolio management is therefore particularly emphasised
during the selection of projects, where it is deemed as an essential tool to ensure
organisational success (Aalto, 2000, p. 8). Aalto (2000, p. 7) provides three reasons for
this. Firstly, the implementation of portfolio management is necessary to ensure that the
organisations' efforts are funnelled into the appropriate projects. Secondly, projects are
seen as the most suitable venue to realise the organisational strategy. Finally, as
resource scarcity is an essential organisational concern, portfolio management provides
a tool to ensure rightful allocation. Furthermore, Lopes & Flavell (1998, p. 224)
mentions synergistic effects as an important aspect of selecting projects. This synergy
effect is thought to ensure that the combination of benefits between projects does not
merely overlap but rather reinforces one another. However, throughout the process,
difficulties in the selection of projects can emerge, such as too many varying goals,
qualitative goals, risk ambiguity, project interlinkages, and the vast number of portfolios
administered (Ghasemzadeh & Archer, 2000, p. 73).

APM (2019, p. 214), on the other hand, emphasises the importance of strategic intent in
the managing of portfolios. As a sum of the above, the process of portfolio management

13
can therefore be said to involve dimensions like value maximisation, risk minimisation,
and strategic alignment (Benaija & Kjiri, 2015, p. 134). Strategy is cited to be
paramount to the success of portfolio management (Aalto, 2000, p. 33). However,
according to Aalto (2000, p. 31), one of the biggest obstacles of effective portfolio
management is the inclusion of projects that does not align with organisational strategy.
According to Archer & Ghasemzadeh (1999, p. 208), a strategic direction must be
defined for a firm before project selection, emphasising the importance of overall
organisational strategic clarity. The strategic goals of an organisation must be clear as
strategic implications resulting from project selection can be great (Archer &
Ghasemzadeh, 1999, p. 208). During the business case of a project, its contributions
towards achieving an organisational strategic objective and its strategic alignment must
therefore be proven (Archer & Ghasemzadeh, 1999, p. 212). Consequently, for the
purpose of this thesis, the authors have selected to focus on the strategic alignment
dimension.

However, traditional portfolio management rarely extends from the project selection
phase. This makes it apparent that other, more robust and specific managerial tools are
needed to ensure that project selection translates to organisational success.

2.4 Project Success

One of the most crucial considerations for organisations to assess a project's
effectiveness is by measuring project success and failure. Project failure is most
commonly correlated with project size, duration, and complexity (Jenner, 2015, p. 7).
Public sector projects often involve a mixture of these factors to some degree, implying
that they indeed should be subject to failure. Furthermore, it is commonly agreed upon
the perception that generally, 50-70% of projects and programmes fail (Jenner, 2015, p.
6). Projects are the most successful driver of change (McElroy, 1996, p. 325), of which
70% fails (Nohria & Beer, 2000, p. 133). Naturally, one would therefore expect many
projects and programmes not to succeed. However, measuring success is not an exact
science, as many attributes and variables come into play in such an undertaking. These
factors are often subject to strategic goals that directly depend upon the motivation
behind project execution, which is outlined in the initial business case. Project success
factors are commonly divided into two main categories: those that relate to project
management performance and those related to organisational success criteria, where the
organisational criteria can be further divided into two measuring processes: appraisal
and evaluation (Serra & Kunc, 2015, p. 54).

The difficulty with measuring project success is the lack of consensus on its definition
(Serra & Kunc, 2015, p. 53; Jha & Iyer, 2007, p. 527). PMI (2017, p. 34) emphasises
that different stakeholders will have differing views on which factors contribute to
success and what project success will entail. Consequently, some stakeholders might
view a project as a success, whereas others may find it lacking in output (APM, 2019, p.
154), highlighting the perceptive nature and relativity of project success. However, there
is a long tradition of attempting to standardise the subject.

Project success has conventionally been linked to the project process; the assessment
has typically been evaluated by applying the iron triangle criteria consisting of project
delivery time, budget, and quality (Badewi, 2016, p. 761; PMI, 2019, p. 34). By this
metric, success is achieved by not exceeding the deadline, adhering to budgetary

14
constraints, and meeting the output's required quality. Thus, from a project management
point of view, effectively managing the iron triangle is a central tool in ensuring project
success (Pollack et al., 2018, p. 527). It is of such importance that some researchers
have cited it to be significant enough that misinterpretation or misunderstanding of it
can lead to project failure, despite the project potentially being managed effectively by
any other metric (Mokoena et al., 2013, p. 813). Therefore, the effective management of
the iron triangle must be seen as essential for project success (Pollack et al., 2018, p.
527). However, successfully accomplishing this is a balancing act as the three factors
are interrelated and consequently involves trade-offs (van Wyngaard et al., 2012, p.
1991). Thus, the increase in stakeholder needs for one constraint must impact the
realisation of another. In the same line, some authors have expressed concern that no
more than two factors can be emphasised by consequence of project constraints, leading
to the classic expression of "better, faster, or cheaper? Pick two" (van Wyngaard et al.,
2012, p. 1993). Something which further complicates the iron triangle is the increasing
lack of consensus on exactly which three factors should be included. Some industry
guides claim that the iron triangle comprises of the fundamental constraints of time,
budget, and quality as discussed above (APM, 2019, p. 217). However, as previously
stated, stakeholders will have subjective views on success. Some authors argue that this
also extends to the matter of quality, leading to a diffuse definition by the nature of
different stakeholder perceptions (Chan & Chan, 2004, p. 213), which complicates the
inclusion of quality as a metric. Thus, some argue that scope supersedes quality (van
Wyngaard et al., 2012, p. 1991).

Nevertheless, recent developments in the field have pushed for the inclusion of
additional factors in the iron triangle. Some of these are safety (Toor & Ogunlana, 2010,
p. 230), access (Daniel, 2019, p. 199), and efficiency (Williams et al., 2020, p. 645), to
name a few. It is therefore evident that researchers express a need for a more
comprehensive and situational tool than the iron triangle offers for measuring project
success.

In recent decades, however, the focus on success has evolved to embrace a variety of
other factors. Specific aspects, such as social and environmental concerns (Ebbesen &
Hope, 2013, p. 7), among others, have become increasingly important to describe
success in an organisational context. Project learning can also be added under this
general umbrella, which has typically become a tell-tale sign of the maturity of project
management environments (Todorović et al., 2015, p. 772). Common for all these is
their strategic and business development implications. More specifically, it can be
explained as an increasing focus on including broader organisational factors like
stakeholder satisfaction and the achievement of strategic objectives, which has
progressed to become an integral part of the success requirements (Badewi, 2016, p.
761). All of this constitutes a broader view on project success than that of the process-
oriented one, a perspective on success as a fundamental part of organisational
development. This perspective is commonly referred to as benefits management. The
relationship between project process success and organisational success is shown in
Figure 4.

15
 Project Success

Project
Management Organisational Success
Success

Benefits and
Budget Time Quality
Value

Appraisal Evaluation
Relevance and
(Before Project (After Project
Expectations
Selection) Closure)

Figure 4. Breakdown of success factors as identified by the authors (Authors).

Whilst project managers seek to create specified outputs, benefits often fall out of their
scope and reach (Mossalam & Arafa, 2014, p. 305). It is therefore often seen as the
responsibility of another entity, typically that on a higher level such as portfolio
management (Mossalam & Arafa, 2014, p. 306). This is further reinforced by the
literature clarifying which factors are to be managed by project managers and those that
should primarily be managed at portfolio, programme, or business strategic level (APM,
2019, p. 15). Here, it becomes evident that benefits management should be initiated and
monitored at the highest strategic level, all the way from business case to after the
project closure (APM, 2019, p. 30-31).

The existence of two very different discussions is therefore evident: that of satisfactory
project management performance and that of success as a strategic and development
initiative. Whereas the discussion on project performance has been covered above, a
discussion on benefits and value will follow.

2.5 Value and Benefits

A benefit is a positively perceived result or outcome created by a project (Laursen &
Svejvig, 2016, p. 737). This description is in line with both Ward & Daniel (2012, p.
70), who defines benefits as "an advantage on behalf of a particular stakeholder or
group of stakeholders", and Bradley (2006, p. 18), who defines it as "an outcome of
change that is perceived as positive by stakeholders".

Value is the relationship between benefit and cost, in which it is proportional to benefit
and inversely proportional to cost (Breese et al., 2016, p. 2; Laursen & Svejvig, 2016, p.
737). Like benefits, value is subjective to the stakeholder's perspective (Laursen &
Svejvig, 2016, p. 737). Some authors have used value as an equivalent to benefits in
referring to benefits contributing to organisational strategy, or simply as a synonym
thereof. However, this interchangeable use is opposed by the authors and must be
avoided to ensure clarity and the best benefits optimisation and maximisation (Breese et
al., 2016, p. 3-4).

Benefits are, by definition, subject to perception. Consequently, they can be both

tangible and intangible (Minney et al., 2019, p. 3). Tangible benefits can be easily

16
quantified, such as the reduction of costs associated with automation of process (Serra,
2017, p. 106). A potential intangible benefit from the same scenario will be error
reduction, leading to improved regulatory compliance (Melton et al., 2008, p. 78).
However, in the latter example, not only is the benefit intangible but also extremely
difficult to measure. Therefore, benefits can also be divided into measurable or un-
measurable benefits (PMI, 2017, p. 7). Benefits can thus take many shapes and forms
(Minney et al., 2019, p. 4). However, other authors reported that the best practice is to
consider benefits as measurable change only (Breese et al., 2016, p. 16-18; Williams et
al., 2020, p. 645). In the context of the UK public sector, the definition of the UK
Cabinet Office disregards the non-measurable benefits, defining it as:

"The measurable improvement resulting from an outcome perceived as an advantage by

one or more stakeholders, which contributes towards one or more organisational
objectives." (IPA, 2017, p. 51)

This definition, adopted by the Infrastructure and Projects Authority for their Guide for
Effective Benefits Management, forms a robust foundation for the management process.

Nonetheless, it also highlights the likely obstacles regarding harder-to-measure

intangible benefits. Tangible financial benefits are relatively easy to measure (Minney et
al., 2019, p. 21). However, there is a need for non-financial benefits to be converted into
a quantifiable financial benefit which might be challenging (HM Treasury, 2020, p. 51;
Minney et al., 2019, p. 21). Thus, the Cabinet Office has created proxy metrics like
statistical life years (SLY) to measure the impact of risks to the length of life, quality-
adjusted life years (QALY) for the purpose of measuring the benefit of health outcomes,
and value of a prevented fatality (VPF) to measure changes in fatality risk (HM
Treasury, 2020, p. 62). Whilst all these metrics are closely linked to public health, there
are frameworks in place to measure an assortment of other benefits like reduced travel
time or those of environmental or recreational concern (HM Treasury, 2020, p. 75-89).
This has made it possible to incorporate formerly unmeasurable benefits in a benefits
management environment. It therefore becomes evident that all benefits still need to be
considered, even if they are non-quantifiable or non-financial (HM Treasury, 2020, p.
65).

Most of the emphasis is placed on identifying and forecasting benefits during the
business case, with less emphasis placed on the implementation and post-delivery
phase. This leads to the revealing of benefits as part of a planned process. Nonetheless,
new and undiscovered benefits might not become evident until after the delivery of a
project (Minney et al., 2019, p. 4). These benefits can emerge in a variety of unforeseen
ways, typically as a result of the dynamic environment of projects. Because of this,
Minney et al. (2019, p. 6) propose a classification system and process of benefits
depending on their importance and time of discovery. Firstly, Y-list (why-list) benefits
are those that shape the need for a project, whether it is by chasing opportunities or
solving challenges. Thereafter, A-list benefits are identified as part of the business case
development, which is additional and harder to identify than the Y-list ones. The
benefits might have changed during the later stages of the project, leading to re-
assessments. Here, the problems encountered can be a source to multiply benefits,
which coins the term X-list. Finally, benefits that emerge unplanned after project
delivery are impossible to avoid. These benefits are important to manage to maximise
benefits and minimise dis-benefits. These final benefits constitute the B-list. Minney et

17
al. (2019, p. 6) therefore suggest that the benefits lifecycle can be illustrated as depicted
in Figure 5.

Figure 5. Benefits Lifecycle as proposed by Minney et al. (2019, p. 6).

It is widely accepted that whereas a benefits' positive improvement results from a new
capability or a change that is typically provided by a project, it is not the project or the
change itself (APM, 2019; IPA, 2017, p. 51; Minney et al., 2019, p. 43). Despite the
robust classification system of Minney et al. presented above, providing a logical and
process-based foundation, other sources cite competing classification systems. An
example of this is the Green Book, which instead focuses on the source of the benefit,
further dividing it corresponding to its properties. These different types and categories
of benefits are illustrated in the model in Figure 6, extracted from the HM Treasury
guide the Green Book.

Figure 6. Classification of social costs and benefits. Model extracted from The Green Book (HM Treasury, 2020, p.
23).

2.6 Benefits Realisation Management

2.6.1 Development of BRM

To further understand the concept and BRM processes, it may be beneficial to first
examine it from a historical perspective. This examination adds context to the process

18
development and will help create an understanding of why it has developed into its
current form.

The starting point of BRM was in the information technology sector, where it was
developed to facilitate the management's decision-making process (Badewi, 2016, p.
762; Breese et al., 2015, p. 1440). Here, the concept first appeared in the early 1990s to
address failure in information and information technology (ICT) projects as these
projects boomed (Breese et al., 2015, p. 1440). This appearance is described as the
pioneering stage, constituting the first of the four stages (Breese et al., 2015, p. 1440).
This stage was led by business-oriented universities, like Cranfield School of
Management and consultants (Breese et al., 2015, p. 1440). For instance, Farbey et al.
(1999) worked with NHS Wales with a focus on information systems and technology
projects.

During the second wave, in the late 1990s and early, the charge of BRM development
was led by governments and regulators, most notably the CCTA's "Managing
Successful Programmes" (MSP) and the HM Treasury Green Book (Breese et al., 2015,
p. 1442). Both guides are still highly influential in the UK public sector's benefits
management (Williams et al., 2020, p. 657-658). Through this stage, BRM became an
integral part of the gateway review process to improve procurement in the public sector
(Breese et al., 2015, p. 1442). This stage has also witnessed the interest and the uptake
from the professional bodies like PMI, APM, the Australian Institute of Project
Management, and International Project Management Association (Breese et al., 2015, p.
1442). In the next stage, by the mid and late 2000s, maturity models for BRM
developed in addition to best practice models. During this time, dedicated special
interest groups and networks started appearing (Breese et al., 2015, p. 1443). The
current fourth stage, a decade-old stage, is characterised by the specialise accreditation
programme (Breese et al., 2015, p. 1444).

2.6.2 The Managerial Process

Benefits management is commonly defined as the managerial process of identifying,
defining, planning, tracking, and realising benefits. (APM, 2019, p. 209; Minney et al.,
2019; Williams et al., 2020, p. 644). IPA (2017, p. 51) defines benefits management as
"the process of organising and managing investments in change and their measurable
improvements". BRM generally assumes the organisation's technical ability is sufficient
to deliver the output and carry any necessary changes to its requirements (Ward &
Daniel, 2012, p. 68).

Ward & Daniel (2012) structure the process of benefits management in a five-step
recurring model consisting of identifying, planning, executing, reviewing, and
evaluating and establishing the potential for further benefits. It is widely agreed upon
that BRM is a process that spans over the whole life cycle of an initiative or investments
(Breese, 2012, p. 342). An addition perspective to the topic comes from the Cranfield
Method, one of the foundational methods of BRM. The Cranfield process model argues
that benefits management is a continuous process, and it should not be imposed via
single projects (Badewi, 2016, p. 763).

Shortly summarised, BRM is generally an organisational and managerial cycle that

starts with identifying benefits before setting plans of how they can be realised. Firstly,

19
one of the focused questions is how the benefits realisation is to be measured. After this,
the assignment of responsibilities and time plans are directed. Then next part of the
cycle is executing it by means of measuring, tracking and realisation. It is then a good
practice to review the process before repeating it. The BRM process is to be
implemented as part of an organisation wide approach, where the holistic and the
strategic view is viewed as essential.

Fundamental to identifying benefits is the strategic analysis. Here, benefits are

attempted identified to bridge a value gap (Serra & Kunc, 2015, p. 55; Ward & Daniel,
2012, p. 68). The identified benefits are used for carrying cost-benefit analysis (Farbey
et al., 1999, p. 1441). This forms the business case used for project appraisal and
selections (APM, 2019; IPA, 2017; Farbey et al., 1999). The relevance of the benefits
and the expected achievement from the appraisal phase will then be used to form the
success criteria. A project is to be measured against these criteria periodically up to the
final evaluation (Serra & Kunc, 2015, p. 54). Therefore, in relation to benefits
identification and planning, 'benefits mapping' is often involved.

The benefits map illustrates the links between projects, benefits, and the strategic
objective (Minney et al., 2019, p. 1). The benefits map is often referred to as an inter-
dependency map. The inter-dependency on a vertical level shows output (which enables
change), outcome, intermediate benefit, and benefit and strategic objectives. On a
horizontal level, it may show how different projects or programmes contribute to
different benefits. A benefits plan is another vital process outcome that is a record of
each benefit, its metrics, realisation timeline, and ownership. It is essential for tracking
purposes as well as the business case. An example of a Benefits map is illustrated in
Appendix 1 (page 74); this example might help depict the process and its usefulness to
management.

Whilst benefits tracking is the process of ensuring the metrics are approaching the
planned threshold (Melton et al., 2008, p. 9). The realisation is sometimes referred to as
'benefits harvesting' or 'benefits delivery', when the broader BRM component is
considered (Breese et al., 2016, p. 3). This stage entails preparing the BAU to harvest
the benefits from the project. It would typically be after delivery in waterfall projects. It
is pivotal to ensure that tracking is continued after project completion and is integrated
into the business performance system (Melton et al., 2008).

To further elaborate on BRM and its framework, the most critical concepts in BRM
practices are outlined by Serra & Kunc (2015, p. 57) and categorised into four main
groups by Breese et al. (2015, p. 1447) as demonstrated in Table 2 below. In this
approach, BRM is structured in concepts and groups of concepts instead of distinct
processes; this might be beneficial to understand BRM from a different perspective.

Table 2. BRM concepts in practice, adapted from Serra & Kunc (2015, p. 57) and Breese et al. (2015, p. 1447).

Group/ Stage of the cycle Concepts/ Highlights

Identification
- The expected outcome clearly defined
- Value to the organisation by the project outcome
measurable

20
 - Strategic objectives to be achieved through the
support of the project outcome are clearly
defined
- A business case was approved at the beginning
of the project describing all outputs, outcomes,
and benefits
Review
- Project outputs and outcomes are frequently
reviewed and realigned to the current
expectations
- Project reviews are frequently communicated to
the stakeholders, and their need is frequently
reassessed
- Project outcomes adhere to the expected
outcomes planned in the business case
Realisation
- The project scope includes activities aiming to
ensure the integration of project outputs to the
regular business routine
- The organisation monitors project outcomes
after project closure in ordered to ensure the
achievement of all the benefits planned
Strategy
- BM strategy defines the standard procedures for
the whole organisation
- BM strategy defines the stand procedure for the
project under analysis

Benefits management is a process that should continue through the life cycle of the
project, herein its closure, final evaluation, and likely well after its closure (APM, 2019;
IPA, 2017; Farbey et al., 1999). According to Williams et al. (2020), the benefits
management frameworks are mature and strongly present in the 'onset' stage, the
appraisal or business case stage. Here, there is a clear presence of the benefits
identification and forecasting practices before the project approval and commencement
(Williams et al., 2020). However, according to their findings, these practices deteriorate
in the subsequent stages (Williams et al., 2020). This thesis's primary focus will be the
execution phase, where the relevance of the benefits and expectation of the benefits
realisation must be checked and confirmed. This phase is vital to confirm a project's
continual validity and vitality on an individual portfolio's overall performance.

2.6.3 Benefits Management Challenges

There are several challenges identified by the literature, ranging from the initial
identification phase all the way until the evaluation phase. The Infrastructure and
Project Authority (2017) mentions some key challenges and suggests ways to mitigate
them. Optimism bias is, like in most business cases, cited to be a big challenge in
estimating the benefits and cost. Optimism bias "is the appraiser tendency to be over-
optimistic about key parameter like cost, duration and benefits delivery" (HM Treasury,
2020, p. 107). The result if the over-optimism bias is over-estimations that will be in

21
favour of proceeding with the project, regardless of how objectively beneficial it might
actually be. Other challenges include the perception of BM being a bureaucratic and
time-consuming process, not engaging stakeholders, and difficulties in measuring
realised benefits either due to the timescale of their realisation or inadequacy in the
measurement metrics and process (IPA, 2017). It has been advised that a comprehensive
integration of benefits management in project management and business-as-usual will
positively mitigate these obstacles (IPA, 2017). Meanwhile, Breese (2012) suggests that
other challenges might be understood through the relationship between benefits
management and organisational culture and the modern paradigm of management's
assumptions.

The use of BM frameworks can be enhanced when the organisation ensures the
presence of certain enablers. According to Williams et al. (2020, p. 651), five factors
were reported as enablers in the public sector; those are:
1. Maintaining a best practice database
2. A benefits management-oriented organisational culture
3. Leadership support and commitment to the process
4. Engaging the stakeholders
5. Clear accountability and responsibility

In general, effective BRM can be ensured through sound management practices that
guarantee projects success in general. In order to facilitate project success, organisations
must ensure their ability to clearly focus on an assortment of factors like good
governance and leadership, clear definition of requirement and strategic vision, and
effective communication (Williams et al., 2020, p. 651).

2.7 Benefits in the Context of UK Government Guidelines

Benefits realisation management is considered a standard practice in the UK (Badewi,
2016, p. 763). This is particularly evident in the emphasis of the concepts presented by
the IPA guide (2017), the supplementary guide for review teams (IPA, 2016) and HM
Treasury Guidance (2018a; 2018b; 2020). For the appraisal and the evaluation, the
authors would like to highlight the Treasury's “five business cases framework”
summarised in Table 3 below. Moreover, considering the reviews (also referred to as
gate reviews or gates), the IPA gates guidance is summarised in Table 4. Additionally,
the UK Environment Agency and UK Highways Agency, among others, have their own
BRM frameworks (Minney et al., 2019, p. 71).

Considering the benefits map, also called the benefits dependency network, the IPA
(2016, p. 30) suggest a 5-level structure to connect the project output to the strategic
objective. Those five levels are: 1) Project output 2) Enabling changes 3) Intermediate
benefits 4) End benefits, and 5) Strategic benefits. However, the guidance leaves it to
individual project teams to adopt the standard or methodology for benefits mapping that
is deemed the most appropriate for their project (IPA, 2016, p. 30).

22
Table 3. HM Treasury 5 business cases; adapted from HM Treasury (2018a; 2018b).

Business Case Brief Description

Strategic Case Making the strategic case of the project in terms of its alignment
with current organisational and departmental strategies, whilst also
taking into account different governmental policies and targets.
Economic Making a case for the project or programme from the economic
Case perspective and ensuring that it is a valuable solution to the public.
Commercial This case primary deals with supply and procurement of aspects of
Case the option, including market conditions and viability of agreements
between the public sector and the services providers.
Financial Case This case demonstrates that the proposed case is “affordable and
fundable”; it also looks to the support of stakeholders and the
customers from that perspective. Typically contains the impact on
balance sheet and income and expenditure accounts.
Management “The purpose of the management dimension of the business case is
Case to demonstrate that robust arrangements are in place for the
delivery, monitoring and evaluation of the scheme, including
feedback into the organisation’s strategic planning cycle” (HM
Treasury 2018a, p. 10).
Contains, for example, governance arrangements, BRM plans and
register, risk plan and register.

Table 4. Key Elements of Benefits Realisation assurance reviews; adapted from (IPA, 2016).

Review Key Elements

Gate 0 - This review is a programme level review that ensures the
(Strategic link between a project's and the organisation's (department)
Assessment) overall strategic goals and may include elements from other
gate reviews.
Project - Occurs at an early stage in the life cycle of a project, when
Validation the BRM process/cycle may not yet be mature.
Review (PVR) - Is concerned with the definition of success.
- This review is also elemental in linking the strategy to
individual projects. At this review, the project strategy and
strategic objective are to be clear. Thus, the definition of
success of the project is agreed upon between all
stakeholders.
Gate 1 - Requires a mature definition of success and benefits
(Business identification. Linked to the Strategic Outline Case; (see
Justification) Table 3 above) and (HM Treasury 2018a; 2018b).
- The key question here is “has the benefits from the project
been identified, categorised and mapped?”
- Prioritisation of the benefits is also reviewed in addition to
the project scope alignment with benefits and eventually
strategic objectives.
- Also, the reviewer must ensure that dis-benefits are
considered through this gate.

23
 Gate 2 - The conversion of the most important benefits into value
(Delivery (i.e., economic justification).
Strategy) - Key documents for this gate (from BRM perspective) are:
Strategic Case, Economic Case, Financial Case, Benefits
Map, and Risk Register.
Gate 3 - This review is related to the full business case; the maturity
(Investment of tracking is ensured (i.e., metrics and mechanisms for
Decision) tracking; priorities for realisation and responsibilities)
- In addition to the documents in gate 2, relevant parts of the
Management Case and performance management plans in
contracts are of interest for this review.
Gate 4 - This review is carried out alongside the delivery, as the
(Readiness for project output transitions to BAU.
Delivery) - Benefits realisation cycle matures; it is also the focal point
of the review.
- The review gate is to ensure that necessary changes in BAU
and operations have been identified, along with mechanism
and responsibilities.
Gate 5 - Evaluation and performance reviews relative to the business
(Operation case.
and Benefits - Ensuring that benefits are embedded in the business.
Realisation) - The complete BRM cycle is expected to be mature by this
gate.
Project - This is not typically a BRM review, but the guidance refers
Assessment to it as “deep-dives into a specific project issue”.
Review (PAR) - It is included as the IPA believe that such issues are often
due to ‘immature approaches to benefits realisation’.”

2.8 Summary
In summary, the theoretical framework used in this thesis is based on an understanding
of BRM as the connecting link between strategy, portfolio and programme
management, and project success. “Benefits realisation management is the glue that
binds together all the other management techniques” (Breese, 2012, p. 341).
Organisations, in all shapes and forms, are implementing their strategies through
portfolios. Portfolios typically target a specific strategic objective or group of related
objectives. Therefore, the whole portfolio components, herein projects and programmes,
must align to achieve these strategic objectives. The projects and programmes deemed
not to add to the achievement of strategic objectives are to be disregarded. Benefits
realisation management practices and frameworks are thought to be useful for such
alignment (Minney et al., 2019, p. 3). Benefits mapping is often mentioned in this
context (Minney et al., 2019, p. 1).

If the projects achieve their planned benefits, they are considered successful. The
combination of BRM success criteria and PM success criteria combined ensures a
higher probability of success than using these practices alone (Badewi, 2016).

24
BRM is defined as the process of planning and organising benefits. It is, though, a
cyclic process that matures over the lifetime of a project. According to different authors,
the stages of the cycle may differ slightly. However, in addition to process review, it
typically includes identifying, planning, measuring and evaluating, and realising
benefits (IPA, 2016, p. 6).

25
3. Research Methodology
The research methodology is the chosen practice based on philosophical beliefs
concerning exactly how the data of a research project is to be collected and analysed for
the research question to be answered (Fisher, 2010, p. 61; Maylor & Blackmon, 2005, p.
155). This chapter will discuss the various approaches the researchers have considered.
This includes their strengths and weaknesses, as well as justification for the choices
being made.

The reason why such a discussion is critical is due to the fact that researchers are
continuously affected by inherent biases and individual rationale (Saunders et al., 2019,
p. 29). This means that without due attention to these phenomena, the research may
become distracted, which increases the risk of the research evolving into a disorganised
project, causing issues with less reliability and validity.

The authors have chosen to follow the research onion by Saunders et al. (2019, p. 130)
to illustrate their methodological approach and choices. The research onion, illustrated
in Figure 7, compares the exploration and selection of a research methodology as the
act of "peeling an onion" whereby the methodology is exposed layer after layer.
Through the remainder of this chapter, the authors will uncover the underlying
perceptions and consideration to demonstrate the research methodology.

Figure 7. The research onion, as proposed by Saunders et al. (2019, p. 130) (Authors).

3.1 Research Philosophy

Research philosophy refers to the different assumptions and beliefs associated with the
development of knowledge (Saunders et al., 2019, p. 159). More precisely, these
perceptions form the basis of how the research is conducted by highlighting the
assumptions that shape how one sees the world and, in doing this, it also shapes each
aspect of the research project (Saunders et al., 2019, p. 159). Hence, the authors are
required to reflect on their assumptions and beliefs in a critical way, similar to how one
would be critical to the assumptions and beliefs of others (Saunders et al., 2019, p. 159).

26
This section will attempt to clarify the authors' philosophical position by discussing the
ontological, epistemological, and axiological considerations of this thesis. While
carrying out any research project, it is essential to base it on a sound philosophical
foundation and a clear understanding of ontology, epistemology and axiology (Maylor
& Blackmon, 2005, p. 154). These considerations form what is usually referred to as
paradigms (Burrell & Morgan, 1993, p. 24), and are fundamental, rationalising the
practical methodology, which is explained in the subsequent section. Lastly, the
research approach is also touched upon in this sub-chapter.

3.1.1 Ontological Stance

Ontology is defined as the nature of the subject and its being (Gill & Johnson, 2002, p.
228; Hammersley, 2013, p. 21). As such, it encompasses reality and specifies whether
the world is experienced and perceived in an objective or subjective manner (Saunders
et al., 2019, p. 133). This perception is typically further reinforced through culture and
language.

In this sense, an objective stance would demand that the individuals in general, and
researchers specifically, looks at reality as made up of measurable objects and, most
importantly, exist independent of the individuals' perceptions of them (O'Gorman &
MacIntosh, 2014, p. 56). As an extension to this, it allows for precise conclusions based
on reliable measurements, hereby permitting individuals to claim objective and general
truths. Objectivism is more common in natural or experimental science research than
social sciences, where the independence of experiment results and findings from
researchers' biases or perceptions is believed to be possible and achievable. Objectivism
is also strongly related to positivism in social science (Saunders et al., 2009, p. 113).

On the contrary, a subjective stance deems that perceptions construct reality and that the
interaction between living subjects continuously alter it (Saunders et al., 2009, p. 111).
A direct consequence of this is the acceptance of different individual realities, whereby
two individuals can experience an architectural piece at different levels of appreciation,
or a fixed statement can evoke separate feelings in different individuals. The authors are
more inclined towards adopting such a stance in this research project.

An example that easily shows the difference between an objectivist and subjectivist
position is a hypothetical study of countries' happiness. An objectivist approach allows
for the research to be based upon assumptions that happiness is linked to attributes like
wealth, education, health, and literacy, which neglects that individuals may value these
aspects to a varying degree or not at all. An example of this would be connecting the
Human Development Index to happiness by basing it upon the assumption that
happiness is a product of a nations' development. A subjectivist approach would be a
questionnaire where the respondents themselves define what brings happiness to their
lives and their individual level of it. This is conducted yearly through the World
Happiness Report.

Owing to the main considerations aforementioned, the authors chose a subjectivist

stance as the initial starting point of the methodological approach. This comes from the
fact that the subject of benefits management can be described as a managerial practice
(APM, 2019; IPA, 2017; Minney et al., 2019). Therefore, it is a subjective matter, being
subject to the practice and practitioners. Furthermore, the understanding of BRM and

27
management is, in general, shaped by the evaluators' culture, experience, and cognitive
processes. This is further reinforced by the fact that business management itself is
subjective to the culture, cognitive process, and experience of people participating in it
and their self-awareness. Finally, Maylor & Blackmon (2005, p. 156) suggests this as
the most appropriate for management research. Because of this, the authors have
considered the ontological stance of subjectivity through social constructivism to be the
most appropriate choice.

3.1.2 Epistemological Stance

Epistemology is the study of knowledge, what is worth knowing about the subject, and
how it can be known (Gill & Johnson, 2002, p. 226; Hammersley, 2013, p. 21).
Fundamentally, it therefore also constitutes what is acceptable knowledge (Saunders et
al., 2019, p. 144).
The primary concern is whether the same principles used in natural science can be
applied in studying social science with its intangible social reality. This makes for two
opposite approaches, namely a positivist or interpretive one.

Despite positivism being associated with natural science, some advocate that social
science can benefit from it as well (Hammersley, 2013, p. 25). This approach focuses on
facts and investigates for causality that can be extended to create fundamental laws
(O'Gorman & MacIntosh, 2014, p. 60). In the search for these facts, researchers need to
formulate and test hypotheses and, by doing this, try to find the essence of the research
question to limit a phenomenon to its simplest elements. By taking this approach,
researchers inherently assume that individuals are acting rationally and, therefore, that
their actions can be explained through reason.

On the other hand, the interpretivist approach does not view individuals as rational
actors. Rather, the presence of varying meaning based on social and behavioural
differences is emphasised in an attempt to understand exactly what is happening
(Hammersley, 2013, p. 27). Contrary to the hypothesis-driven research of the positivist
approach, ideas are developed through data induction, and the focus is on
acknowledging the existence of different views on a phenomenon (O'Gorman &
MacIntosh, 2014, p. 60).

This thesis is based on an interpretivist epistemological approach, largely owing to the

fact that the aim of the thesis is to understand a socially constructed framework and the
interactions herein. In the circumstance of this thesis, the key to fully acknowledging
this is understanding the subjective perspectives and experiences of those practising
benefits management. Therefore, the authors will not utilise questionnaires or surveys;
rather, the understanding of links and meaning is emphasised through qualitative
practices as explained in detail in 3.2.1 Methodological Choice. Besides, both Saunders
et al. (2009, p. 116) and Maylor & Blackmon (2005, p. 157) suggests that the ideal
approach whilst researching the field of business and management is that of an
interpretivist one.

3.1.3 Axiological Stance

Axiology refers to the philosophy of value and ethics, which inevitably leads to biases.
It is distinct from research ethics which will be discussed in specific in 3.5 Ethical

28
Considerations. Largely, it revolves around the possible preconceptions a researcher
might hold and how detached from possible experiences the researcher places
themselves (O’Gorman & MacIntosh, 2014, p. 70; Saunders et al., 2019, p. 143). If
researchers do not sufficiently reflect on their axiological perceptions, it may undermine
the research's outcomes and quality. Consequently, it is vital to consider the axiology to
ensure the credibility of the work.

In the case of this research project, the authors have thoroughly considered their
axiological standing. It is evident in its nature that the topic itself was selected owing to
personal interests and curiosity. Nevertheless, the authors did not possess sufficient
knowledge about the subject prior to this thesis to effectively form strong biases.
Despite this, the research question was discussed in detail, which ended with the
expectation that benefits management is not adequately considered through the life
cycle of a project in a portfolio. A great deal of effort has been put into mitigating this,
not least during interviews where this possibly could have steered the conversation in an
unwanted direction by influencing the respondents and creating expectations for their
responses.

3.1.4 Philosophical Paradigm

The three different philosophical assumptions discussed above are sometimes mutually
exclusive or inclusive. For example, a positivistic epistemological approach is typically
aligned with an objective ontology (O'Gorman & MacIntosh, 2014, p. 59). As part of
this, drawing connections between philosophical standings concerning reality and the
development of valid knowledge is required by researchers. This connection between
the philosophical areas, and the sum of them, is what creates what is known as a
research paradigm. Usually, five major ones are identified: positivism, critical realism,
interpretivism, postmodernism, and pragmatism.

Parts of our philosophical approach aligns with multiple paradigms. For instance, both
an interpretivist and postmodern approach – and partly also a pragmatic one – fits our
proclaimed subjectivist ontology. However, whereas an interpretivist methodology
argues that reality is socially constructed through culture and language, allowing for
multiple meanings, a postmodern one focuses on the varying levels of power relations
where interpretations are dominated and silenced by others. Because of this, an
interpretivist philosophical paradigm has been found to be the most appropriate. The
challenge presented by this paradigm is that it demands the researchers to understand
the world from the participant's point of view (Saunders et al., 2019, p. 149). However,
it is viewed as the most appropriate perspective in business and management research as
these situations typically involve complex and unique context's (Saunders et al., 2019,
p. 149).

This stance typically involves inductive reasoning, qualitative methodology, small

sample sizes, and in-depth investigations (Saunders et al., 2019, p. 145).

3.1.5 Research Approach

One of the most pressing issues when developing a research model is whether it should
be based upon inductive or deductive reasoning (Cooper & Schindler, 2014, p. 64),

29
where the selection thereof largely depends on the reliance upon theory and/or
experience, or empirical data.

Cooper & Schindler (2014, p. 67) argue that in the case of deduction, a conclusion must
follow from previously given reasoning, which by themselves represents proof and
implies the conclusion. This makes for a powerful bond between reason and conclusion,
where logical justification stems from the proven linkage between premises. An
example of this is the thought logic where premise A says that Jacob loves nature;
premise B says that people who love nature is conscientious, whereby the conclusion
must be that Jacob is conscientious. If Jacob is believed to be conscientious, it is proven
as a sound deduction (Cooper & Schindler, 2014, p. 67).

In contrast, using inductive reasoning does not start from premises but arrive at them as
a product of the reasoning process (Gill & Johnson, 2002, p. 40). While using induction,
the researcher observes the subject of the phenomenon of interest, after which an
attempt to recognise patterns are made to arrive at a conclusion (Cooper & Schindler,
2014, p. 68).

Most of the work done in the field of business realisation management is empirical
research with few papers attempting to formulate a concept or theory (Laursen &
Svejvig, 2016, p. 739). Hence, it is believed that the area lacks an encompassing
framework for benefits management (Breese et al., 2015, p. 1439), with a general lack
of prior premises being made. As a result, deductive reasoning was rejected, whereas
inductive reasoning was considered appropriate to arrive at a conclusion about the
effectiveness of benefits management.

3.2 Research Design

According to Saunders et al. (2019, p. 173), the research design is an encompassing plan
surrounding how the research question is to be answered. Therefore, clear objectives
shaped by the research question must be created, methodological choices need to be
made, which sources the collection of data is based on must be contemplated, how data
is to be collected and analysed needs to be reflected upon, and quality criteria must be
considered. Whilst the research objectives and research question are discussed in 1.4
Research Question and Objectives, the rest will be discussed in the following sections
of this chapter.

3.2.1 Methodological Choice

The methodological choices can be divided into three distinct categories: qualitative,
quantitative, or mixed-method design (Saunders et al., 2019, p. 174). The biggest
differentiator between qualitative and quantitative design is that whilst the former
generates non-numerical data, and the latter produces numerical data (Saunders et al.,
2019, p. 174). However, often times, the research design incorporates parts of both
qualitative and quantitative methodology, called mixed methods design (Saunders et al.,
2019, p. 175). The selection of methodology is generally closely linked to the research
philosophy, which has been previously discussed (Saunders et al., 2019, p. 174).

30
Quantitative methodology is designed to test the relationship between variables to
explain correlation or causality. To achieve this, the data collection method is highly
structured and results in numerical data, whereby the analysis of the standardised data is
conducted through the application of diagrams and statistics (Saunders et al., 2019, p.
178).

Qualitative methodology, on the other hand, is designed to examine meanings and

relationships (Dey, 1993, p. 11). Thus, information is conveyed in any other form than
numbers is considered qualitative (Dey, 1993, p. 13). This is generally accomplished by
emphasising meanings expressed through words by the means of unstructured or semi-
structured data collection, after which the analysis of the non-standardised data requires
classification before being conceptualised (Saunders et al., 2019, p. 180).

Finally, mixed methodology integrates part of both quantitative and qualitative design to
varying degrees. This approach is often applied to provide boundaries to the research's
scope or to test a theoretical proposition before further qualitative or quantitative
research to develop a richer understanding (Saunders et al., 2019, p. 181).

For the purpose of this thesis, a quantitative approach was deemed unfit since it, by
nature, tries to examine relationships between variables – variables which we do not yet
know of because of the deficiency of encompassing frameworks allowing for robust
premises and hypothesis to be tested. Additionally, a mixed methodology was
considered to be inappropriate considering the scope of the research and limitations
such as time. Thus, a qualitative methodological approach was selected for this
research, the selection of which was also deemed the most appropriate when taking an
interpretivist philosophical standing (Saunders et al., 2019, p. 179; Hammersley, 2013,
p. 21).

The research's purpose is another important aspect to consider when constructing the
research design. The purpose directly shapes the selection process of the design, which
can result in a descriptive, evaluative, explanatory, or exploratory study – or a
combination of these (Saunders et al., 2019, p. 186). Descriptive research seeks to
establish accurate portrayals of events or situations. Because of the very nature of such
an approach, having a clear picture of the studied phenomenon is essential, which
makes it unfit for this research project. An evaluative study emphasises the extent of
which something functions as intended and is often implemented when assessing the
effectiveness of process, initiative, or strategy. Accordingly, the approach is typically
chosen when producing local knowledge (Reinking & Alvermann, 2005, p. 143) and is
unsuitable for gaining the insight required to answer the research questions. Explanatory
research, however, seek to demonstrate causal relationships among variables (Saunders
et al., 2019, p. 188). This requires a sound knowledge base and deductive reasoning;
however, no such basis exist for the subject of this thesis. Finally, exploratory research
focuses on gaining insights and understanding on a topic where prior knowledge is
inadequate (Saunders et al., 2019, p. 187). Seeing that this research started without a
conclusive hypothesis, an exploratory design has been selected. This is suitable for the
inductive reasoning previously discussed and supports the open-ended objectives of the
study.

31
3.2.2 Research Strategy
The research strategy refers to the plan researchers create to answer the research
question (Saunders et al., 2019, p. 189). The selection of research strategy is closely tied
to prior choices made in regard to research philosophy, research approach, and research
design; however, limiting factors such as existing knowledge, available time, and
resource scarcity also contributes to the selection of strategy (Saunders et al., 2019, p.
190). According to Saunders et al. (2019, p. 190), the different strategies are: action
research, case study, documentary research, ethnography, experimental, grounded
theory, narrative inquiry, or survey. The strategies are not mutually exclusive, rather a
combination of different strategies is sometimes employed to effectively answer the
research question (Saunders et al., 2019, p. 190).

For the purpose of this thesis, a case study strategy has been employed. According to
Yin (2018), a case study can be defined as "an in-depth inquiry into a topic or
phenomenon within its real-life setting" (Quoted in Saunders et al., 2019, p. 196) . Case
studies are extensively used in business and management (Maylor & Blackmon, 2005,
p. 242) and are suitable for exploratory methodology (Maylor & Blackmon, 2005, p.
243). According to Eisenhardt & Graebner (2007, p. 25), a case study is the best choice
to build a theory on by utilising qualitative data and inductive research. Additionally, it
is suitable when studying under-researched areas that complicate creating a clear and
robust hypothesis. Finally, case studies are regarded to produce the most interesting
findings and theory leading to the greatest impact (Eisenhardt & Graebner, 2007, p. 25).

There are two distinct options regarding case studies which strongly depends on the
research question (Yin, 2014, p. 31). Either that of a single-case study or a multiple-case
study can be selected, each with its own advantages and disadvantages. As implied by
their names, a single-case study strictly investigates one unit of analysis (Maylor &
Blackmon, 2005, p. 245), whereas the scope of a multiple-case study involves
investigating numerous cases. Because of this, during the selection process, practical
variables like accessibility and resource availability needs to be considered. Moreover,
whereas a singular case study usually excels in instances where a unique or extreme
case is studied, multiple-case studies have a wider scope that leads to the creation of
more robust and generalisable findings (Saunders et al., 2019, p. 198).

Because of the favourable properties of more robust and generalisable findings, a

multiple-case study in specific has been chosen for this study. Maylor & Blackmon
(2005, p. 246) consider between two and eight cases with an adequate depth to be
appropriate for such a study. Because of this, the author's set out to identify no less than
five adequate cases. This allows for a wider context to be established.

3.2.3 Time Horizon

The time horizon of the research refers to whether it is intended to be a single
"snapshot" at a specific time or several "snapshots" over a period of time (Saunders et
al., 2019, p. 212). A single "snapshot" is referred to as a cross-sectional time horizon,
whereas a series of "snapshots" is referred to as longitudinal (Saunders et al., 2019, p.
212). When implementing a longitudinal horizon, the phenomena researched is studied
for long-term effects, which implies that time acts as a parameter with an effect on the
results.

32
When it comes to the subject of this study, the author's consider time to be an irrelevant
parameter. For this exact reason, a longitudinal time horizon has been deemed as
inappropriate. Moreover, as industry guides and government guidelines are subject to
continuous change and refinement, the interest of the author's lie in exploring the
effectiveness of benefits management in a contemporary setting. Also, case studies
based on interviews are typically cross-sectional (Saunders et al., 2019, p. 212). Finally,
the research objectives can be fulfilled in their entirety by applying a cross-sectional
time horizon which, given the subject to time constraints, makes a longitudinal time
horizon redundant.

3.3 Data Collection and Analysis

3.3.1 Data Collection Methods

Primary data is data that has been generated specifically for the purpose of the research
project (Maylor & Blackmon, 2005, p. 172). This can stem from a variety of original
sources, like interview transcripts, observations, or surveys. The selection of a specific
primary data source depends on an assortment of factors, such as the philosophical
orientation, methodological selections, and the purpose of the research (O’Gorman &
MacIntosh, 2014, p. 78). O’Gorman & MacIntosh (2014, p. 78) advocates that primary
qualitative data is favourable in a study that aims to understand an underlying process or
phenomena, given that it is empirically oriented and based on an interpretivist or social
constructionist paradigm. The authors share this sentiment. Therefore, the primary data
of this case study was collected by conducting interviews. Bryman & Bell (2011, p.
467) suggests that for qualitative purposes, only unstructured or semi-structured
interviews can be selected. In the case of multiple-case studies, some structure is needed
to ensure cross-case comparability (Bryman & Bell, 2011, p. 473). The selection of
semi-structured interviews for primary data is therefore made to ensure consistency
between the research philosophy, research design, and the exploratory nature of the
research project. Moreover, it allows the flexibility for participants to add relevant
points, the researchers to add probing questions, and a generally natural flow of the
discussion (Bryman & Bell, 2011, p. 467). Semi-structured interviews enable the
research to be conducted with the right balance between the structure based on literature
and exploration (O'Gorman & MacIntosh, 2014, p. 78; Bryman & Bell, 2011, p. 467).

Secondary data, on the other hand, is data that originates from other sources and is
readily available to both collect and analyse (Maylor & Blackmon, 2005, p. 172). This
data can originate from a variety of second-hand sources such as company reports,
publicly available documents, or statistical databases. While a research project can
comprise solely of secondary data, it can also provide new insights when
complementing primary data (O’Gorman & MacIntosh, 2014, p. 79). Therefore, to
complement the primary data sources outlined above, this study will also consider a
robust assortment of secondary data. This secondary data set will comprise of both
government guidelines, industry guides, peer-reviewed research papers, and government
statistics. Furthermore, this secondary data can be utilised to either support or contradict
the findings of this study, leading to more robust and reliable findings (O’Gorman &
MacIntosh, 2014, p. 79).

33
3.3.2 Interview Protocol
Questions for the interviews were prepared in advance, which is pivotal in ensuring the
inclusion of all critical points. Furthermore, it is considered beneficial as a clear idea of
the analysis was established before the interviews (Bryman & Bell, 2011, p. 472). To
create an interview guide, Bryman & Bell (2011, p. 475) suggests creating a list of
topics arranged by bullet points. However, the authors deemed it necessary to create a
more detailed list of questions due to the understanding of the subject and the research
question. The prepared interview questions were formulated using the existing literature
that is part of the theoretical frame of reference and the authors' understanding of the
issue. Throughout this paper, these questions will be referred to as the interview
protocol. As the researchers, the emphasis was placed on acknowledging that the
understanding of the topics developed throughout the interviews. Therefore, the
protocol was revised to reflect such development. For example, redundant questions that
were shown to not directly contribute to answering the research question were removed
or implied in other questions. This revision had the effect of better utilising the time
reserved for interviews and provided a more natural flow of the questions. In total, three
interview protocols were produced, all of which are found in Appendix 3: Interview
Protocols. The practice of producing these revised interview protocols was deemed
essential to ensure that the interviews would effectively contribute to achieving the
research aims and objectives.

3.3.3 Sample Selection

One attribute that is often considered important when conducting research is to what
extent the results can be generalised. For the findings to be generalisable, sampling
techniques that ensure a representative sample must be utilised to lower the sampling
error (Bryman & Bell, 2011, p. 176-178). However, research limitations like time and
cost often place restrictions on the possibility of a truly representative sample (Bryman
& Bell, 2011, p. 188). This means that after a certain point, the marginal increase in
precision falls below what is acceptable considering the linear cost of resources. Since
Maylor & Blackmon (2005, p. 246) suggests that eight cases are more than sufficient for
a multiple-case study, the authors have selected to settle for seven.

During the research, it was deemed vital to seek different perspectives from various
participants for data verification through sources triangulation (O'Gorman & MacIntosh,
2014, p. 89). Initially, the plan was to fulfil this by interviewing multiple individuals in
the same organisation at various levels to ensure good inter-organisational sampling.
However, for practical reasons, this was not possible. Instead, the authors settled on two
different sampling methods.

Initially, LinkedIn was used to identify and invite prospective participants based on the
author's mutual connections. Here, the target of the initial invitations was project or
programme managers and benefits managers. Simultaneously, the researchers reached
out to people in the industry by announcing the research project on the LinkedIn group
"Managing Benefits - Community of Interest". Since this group must be considered an
interest group whose sole focus is on benefits management, it must also be regarded that
the people part of it is particularly interested and knowledgeable on the subject, forming
a somewhat homogenous group. The authors utilised these venues by virtue of
accessibility. It must therefore be regarded as a form of convenience sampling (Bryman

34
& Bell, 2011, p. 190). Convenience sampling is prominently used in business and
management (Bryman & Bell, 2011, p. 191), however, it is usually considered inferior
to alternative sampling techniques as it might not create a representative group (Bryman
& Bell, 2011, p. 190). Nevertheless, since the purpose of this study and its research
question necessitates the inclusion of an otherwise homogeneous sample, it was deemed
satisfactory by the authors. Additionally, time and resource limitations of this study
make other sampling methods problematic.

After having connected with people over LinkedIn, the authors shifted their sampling
focus to rely on snowballing sampling for the recruitment of further participants. This
approach relies on the authors first making initial contact with individuals of a group to
then establish contact with others (Bryman & Bell, 2011, p. 192). Snowballing sampling
is generally considered to have the same drawback as convenience sampling (Bryman &
Bell, 2011, p. 192). Still, the snowballing method is common for this type of research in
which the target subjects are difficult to reach, considering the nature of their job.
Snowballing has been cited as an acceptable way to overcome this difficulty (Williams
et al., 2020, p. 647). The authors implemented this method by encouraging participants
found through convenience sampling to introduce them to colleagues and connections
that may be of interest. Therefore, following each interview, new parties were invited.
The invitation to join this study was also forwarded to "Public Sector Benefits
Management Forum (PSBMF)". Appendix 2: Participation Introduction Letter (page 75)
includes the letter sent with all invitations clarifying the study's purpose and topics to be
discussed. Several members from the forum volunteered to participate in the study.

3.3.4 Data Collection Process

The interviews were conducted employing video conferencing for convenience. Video
interviews are an acceptable practice and are thought to be an ideal compromised
between face-to-face interviews and phone interviews due to it allowing for more
interviews being conducted within a given timeframe and over a larger geographical
area (Cooper & Schindler, 2014, p. 153). Video interviews provide convenience for
both parties with the ability to retain some non-verbal cues (Edwards & Holland, 2013;
Gilbert, 2008; O'Gorman & MacIntosh, 2014, p. 120).

The data collection was conducted in three main phases:

1. Pre-interview phase. This was a planning phase that comprised of two distinct
parts: preparing the interview questions and selecting the sample. The result of
this phase was the first interview protocol and a selection of potential
participants to whom we reached out to.
2. Pilot interview. In this phase, a pilot interview was conducted to test and refine
the questions in the interview protocol. The interview exposed the authors to the
interview process with the presence of an interviewee and allowed for the further
refinement of the interview protocol. This revision resulted in the creation of
Version 2 of the interview protocol.
3. Primary interviews. After having refined the interview process and protocol, the
study proceeded with the remaining interviews. Each participant was
interviewed once for approximately an hour, although some interviews
occasionally lasted for around 90 minutes. During this process, the interview
guide was revised a final time, although the revision was minor and primarily for
the sake of the authors.

35
Prior to all the interviews, an invitation letter was sent to the participants. This letter
(Appendix 2: Participation Introduction Letter) was important for ethical and academic
consideration. Importantly, it also aided to ensure participants were set and clear about
the discussion, and it served as a mental preparation tool. All interviews were recorded
with participants consent. As the final stage of the data collection, all interviews were
transcribed and shared with the relevant interviewee. Artificial intelligence software
(Otter.ai) was utilised to provide quick and accurate transcription of the interviews. The
transcripts were then revised manually by the authors to ensure that it was free from
machine errors. The combination of automated and manual transcription allowed more
efficient use of the researchers’ time. The final sharing of transcripts was conducted as a
possible venue for verifying the data collected (O'Gorman & MacIntosh, 2014, p. 122).

3.3.5 Data Analysis Technique

Qualitative data often leaves researchers with difficulty in finding an analytical path to
follow through the richness of data (Bryman & Bell, 2011, p. 571). This difficulty is
oftentimes made even harder by the lack of well-established rules of quantitative
analysis in comparison to that of qualitative data (Bryman & Bell, 2011, p. 571). To
make sense of the broad and rich data collected, the authors elected to base their
analysis on grounded theory.

Typically, an overlap between the data collection and analysis is to be expected

(O'Gorman & MacIntosh, 2014, p. 88; Gioia et al., 2013, p 20). It is also suggested that
the data collection and the analysis might be performed as part of a cyclic process,
where continuous feedback and reflection from the analysis is used to improve and
develop the data collection (Saunders et al., 2019, p. 640). This approach was deemed
useful and was subsequently utilised in this study, implying the need for using priori
codification which is codification prepared in advance of the interview (O'Gorman &
MacIntosh, 2014, p. 140). This overlap between collection and analysis creates the
foundation of grounded theory, which is often distinguished by utilising data to create
theory through an iterative approach (Saunders et al., 2019, p. 668). Thus, the data
collection, analysis, and theory development are interrelated (Bryman & Bell, 2011, p.
576). As discussed in the previous section, the analysis of this research was that of an
iterative one. This is one of the reasons for selecting this analysis technique, as it
allowed for the analysis to commence immediately after initial data collection (Bryman
& Bell, 2011, p. 574).

Whilst the process might appear rather intricate, owing to its assortment of techniques
and phases, it essentially focuses on revealing similarities (Saunders et al., 2019, p.
669). Grounded theory allows for the evolution of knowledge during the various phases.
While grounded theory is seeing great use in contemporary research, it is not without its
criticism. For instance, the time required for transcription, the risk of losing context and
flow, and the subtle ambiguities between techniques have all been cited as some of its
weaknesses (Bryman & Bell, 2011, p. 583). However, others give it merit for capturing
complexity, creating a better understanding of practices, allowing for new perspectives
on well-established fields, and generating theory (Locke, 2001, p. 95-98). Gioia et al.
(2013, p. 22) elaborate on this, emphasising that grounded theory displays the dynamic
relationships between the concepts that explain or describe the phenomenon. These
constructive attributes further form the basis of the authors’ selection of grounded

36
theory as benefits management is practised in complex environments with a lack of
current research.

Locke (2001, p. 46-58) presents the phases of grounded theory as stages, describing
them as following: Stage 1 is conducted by comparing the incidents that are applicable
to each theory, often by means of naming, coding, memoing, and comparing (Locke,
2001, p. 46-51). Stage 2 integrates the findings from the previous stage in categories
according to their attributes, thus establishing similarities, parallels, and
interrelationships (Locke, 2001, p. 51-52). Stage 3 proceeds by developing a theory
based on the categorisations and their affiliations, with an emphasis on scoping the
theory’s limitations (Locke, 2001, p. 52-54). Stage 4 is the final phase, wherein the
research is composed for publication by integrating the memos previously produced into
the theoretical framework to substantiate it (Locke, 2001, p. 54). According to Bryman
& Bell (2011, p. 578), these stages create a variety of products, which can be classified
as follows: concepts, categories, properties, hypothesis, and theory.

Regardless of approach, typically, the data analysis starts with first transcribing the
interviews, which is then followed by codifying these transcripts. Codification is the
analytical process of breaking the textual parts, then identifying and refining the
concepts underlined in them (O'Gorman & MacIntosh, 2014, p. 140). Coding is
regarded as a fundamental part of any qualitative analysis, especially that of grounded
theory (Bryman & Bell, 2011, p. 584), and will be implemented as a part of the analysis.
Emphasis on several aspects has been suggested to ensure that the codes derived are
both consistent and accurate. For instance, Bryman & Bell (2011, p. 586) suggests
familiarising oneself with the data before coding by reviewing it several times, as well
as reviewing codes to ensure consistency, something which was considered throughout.
The principal data analysis was conducted in the form of posteriori codification of the
responses, performed after the interview. For this study, after the data collection, it is
expected for codes that the literature review may not have predicted to emerge.

Thus, an approached inspired by grounded theory will be utilised in the analysis. It

starts with the priori codification, followed by posteriori codification and analysis to
draw similarities revealing categories. The analysis will then reveal the relationships
between the categories to form theories that will be refined before inducing a
conclusion. Through this approach, it is expected that both codes and data will funnel
down as relations are identified to narrower themes and, eventually, an induced theory.

3.4 Quality Criteria

A well-considered research design is important in order to lessen the possibility of
drawing the wrong conclusions. Several different factors that will affect this possibility
is collectively summarised under the term research quality (Saunders et al., 2019, p.
213). The relevance and role of some quality criteria in qualitative research are disputed
(Saunders et al., 2019, p. 213), but their inclusion has been discussed where relevant.
Measures to ensure the quality of the research are summarised in Table 5 below and
further elaborated on in detail thereafter.

37
Table 5. Quality Criteria.

Quality Criteria Challenge Solution

Dependability Ensuring consistency - Mitigate sources of participant
and replicability if a error (e.g., ensuring anonymity).
new study were to be - Mitigate sources of authors’ bias
conducted. (e.g., two interviewers).
- Recording and documenting the
research process.
- Continuously confirming and
cross-referencing between the
authors.
- Restricted sources to those of
recognised character.
Credibility Ensuring the - Sampling selection was
trustworthiness of the emphasised.
data. - Confirming clarity and accuracy
in the data and analysis with
participants.
- Emphasising link between
literature, interview guide,
discussion, and any combination
thereof.
Transferability Ensuring that the - Involving a diverse sampling.
findings can be - Including a more-than-adequate
generalised. number of participants.
- Explicitly documenting the
procedures and limitations to
allow third-parties to
subjectively judge the
transferability.

3.4.1 Reliability (Dependability)

Reliability focuses on whether or not the findings of a research project is consistent and
replicable if the research project was to be reproduced (Saunders et al., 2019, p. 213).
While it is impossible to ensure that the results of a study are truly reliable, the
emphasis is placed on the possibility of getting wrong answers (Saunders et al., 2019, p.
213).

This quality criterion is heavily emphasised in quantitative research and, because of its
ties to natural science, has increasingly become more relevant in qualitative research
that takes a positivist stance (Golafshani, 2003, p. 597). However, part of it is otherwise
difficult to apply during qualitative research. Specifically, ensuring a high level of
external reliability by applying traditional logic would be next to impossible considering
the chosen research design since different researchers seldom would end up with the
same findings. Despite this, Golafshani (2003, p. 601) proposes that reliability in
qualitative research can be evaluated by examination of trustworthiness and
dependability. To accomplish this, several regular sources of error and bias (Saunders et

38
al., 2019, p. 214), both on the part of the researcher and participant, has been considered
and mitigated.

First and foremost, to increase the participants' reliability, the authors have decided to
conduct the interviews at times chosen by the participants and at their leisurely pace.
Furthermore, the possibility of incorrect response has been minimised by carefully
considering questions and responses not to lead the conversation. All participants were
also ensured anonymity to ensure that no employer retribution would be possible.

Secondly, the researchers' reliability was ensured by, most importantly, the recording of
interviews for later review and transcription, as well as the appearance of both
researchers during all interviews. Furthermore, an independent review of the data was
completed to confirm that both researchers agreed about the data and its analysis. To
alleviate any biases, the interviews were conducted in a semi-structured manner with the
interviewees to some degree being allowed to steer the conversation and tone. Finally,
the participants were offered the opportunity to add anything of their liking to the
discussion at the end of the interviews. This open end was provided to allow
participants to add perspectives or suggest topics the authors had not covered during the
conversation.

Finally, to increase reliability, the selection of literature has been restricted to well-
known and recognised authors. However, the scarcity of literature on the specific topic
of this study may have a negative impact as it complicated the literature search and by
extension the literature review. Despite this, the emphasis was placed on the search for
peer-reviewed articles from highly ranked journals. In cases where there was a lack of
such articles, publications of renowned authors or government-issued guides and
industry best-practice guidelines were utilised.

3.4.2 Internal Validity (Credibility)

Internal validity, also referred to as credibility, is the extent to which the findings are a
function of the intervention being researched and not flaws in the research design
(Saunders et al., 2019, p. 215).

For the research to exert high credibility, emphasis was placed on the sourcing of
information, whereby selection of participants became crucial. All interviewees are
practicing in the field of benefits management and confirmed their inclusion and
experience in the topic researched. Moreover, the data collected is clear and consistent,
and was shared with the participants to ensure the credibility of its interpretation.
Finally, the content of the interview guide has a clear link to the theoretical frame of
reference, as does the discussion.

3.4.3 External Validity (Transferability)

The external validity, referred to as transferability, is concerned with whether or not the
findings can be generalised (Saunders et al., 2019, p. 216). Golafshani (2003, p. 603)
suggests that the ability of research to be generalised is a staple to its validity and
trustworthiness. Furthermore, Gioia et al. (2013, p. 24) emphasises that there is little
benefit of research lacking the ability to be generalised. It is thus an important
consideration to make.

39
To increase the transferability of this study, a diverse group of participants was selected
where no two respondents worked in the same company, organisation, or government
body. In addition, the number of respondents should increase the generalisable nature of
the findings. Maylor & Blackmon (2005, p. 246) suggests that between two and eight
cases will provide an adequate selection for our study. The authors therefore consider
this research’s inclusion of seven participants to increase transferability. Because of the
aforementioned, the data collected represents benefits management practitioners at large
in the UK. However, despite the authors being confident that the results are transferable
in a domestic setting, it is unclear if the findings of this study can be transferred to apply
to foreign practitioners. This is especially clear since different countries often apply
local benefits management procedures to their practices.

3.5 Ethical Considerations

Ensuring moral and ethical conduct is a significant part of any research project and
should be considered throughout (Saunders et al., 2019, p. 252). It is especially
important when conducting qualitative research as it typically leads to a wider range of
ethical concerns compared to qualitative research (Saunders et al., 2019, p. 281). The
authors have chosen to go by the wider definition of ethics presented by Bell & Bryman
(2007, p. 64), which emphasises ethical consideration to every aspect of management
research, thus ensuring that ethical considerations accompany the entire research project
throughout its duration. Saunders et al. (2019, p. 253) argue that there are two
predominant philosophical positions regarding research ethics based on two different
theories in moral philosophy, which is deontology and teleology. Whereas a
deontological view founded on abiding to rules that guides the researchers conduct,
thereby strictly opposing any disregard of them, a teleological view allows for the
researcher to act in contempt of ethical rules provided the act of conduct can be justified
based on its consequences (Saunders et al., 2019, p. 253). For the purpose of this
research, and in line with proper research ethics, the authors have decided to base their
conduct on a deontological view. The rules that constitute our ethical conduct will be
established by the ethical guidelines for thesis work at Umeå University, as described in
the Thesis Manual (Umeå University, 2018, p. 6). The considerations that have followed
this decision is explained below.

Throughout this research, the primary ethical concern was ensuring the security of the
data collected and preserving the privacy of the participants. Therefore, regulations
concerning data collection and handling have been considered, which includes ensuring
compliance to GDPR (Regulation 2016/679), which replaced the older
Personuppgiftslagen (SFS 1998:204) in 2018. As part of this, all information gathered
was secured and stored in a safe manner, and an encrypted platform was selected for the
video conference. Furthermore, the confidentiality of participants' privacy and identity
is an imperative matter. Because of this, the authors have had to ensure a balance
between achieving the research objectives and their ethical obligations. As part of this,
the names of the participants and specific details about their organisation has been
redacted and anonymised. However, to increase context and provide relevant
discussion, some attributes have been disclosed as it is relevant to the scope of the
study. The attributes disclosed includes the participant role and organisational and
projects characteristics. These attributes have not been considered as personal data in
accordance with GDPR Article 4 (Regulation 2016/679).

40
To ensure approval from participants, informed consent was collected before the
interviews. Interview transcripts were finalised following the interviews. The transcripts
were shared with the relevant participant to allow the opportunity to reflect on the
interview and ultimately also to retract or modify part of their contribution.
Furthermore, the option to completely withdraw consent was presented. Finally, the
outcomes of the research were shared for verification. Overall, the researchers have
emphasised the implementation of an overt approach, were clearly stating the purpose
of the study, its process, and what observations are made has been vital. This is assumed
to have a positive impact on the reliability of the research.

The importance of proper referencing has been considered to ensure that original
authors are credited where their work has been cited or otherwise influenced the
research.

As a closing remark, it is important to note that no conflict of interest is present.

3.6 Methodological Statement

By summarising what has been discussed in this chapter, the methodological selections
can be visualised in the research onion mentioned at the beginning, as depicted in
Figure 8.

Figure 8. The research onion, our methodological selections (Authors).

As shown, the research will be based on an interpretivist philosophical paradigm, as this

will ensure adherence to the subjectivist and interpretive nature of our research.
Furthermore, owing to the lack of prior research and literature on the subject at hand,
the authors have chosen an inductive approach to theory development. This is applied
through a mono-method qualitative study which takes the form of a cross-sectional case
study design. Finally, the data collection will be accomplished through semi-structured
interviews as a mean to allow interviewees to share their personal perspectives whilst
still somewhat adhering to an interview guide. After this, the data will be analysed by
first transcribing the interviews, then applying iterative coding before links, similarities,
and common themes will be identified.

41
4. Empirical Data and Findings
4.1 Empirical Data

4.1.1 Data Sources

As elaborated in the methodology section, this thesis's authors adopted subjectivism and
social construction as the methodological approach. Subsequentially, this reflects the
selection of the qualitative approach. Therefore, the authors feel that it is crucial to
elaborate on the data sources and provide the reader with the context that strongly
contributed to forming the research conclusion. The sources of the data are showcased
in Table 6. The table also includes the relevant version of the interview protocol in
addition to the necessary contextual information.

Table 6. List of interviews.

Interview Participant Interview Protocol Interview Interview

Number Identifier Date Version Location and Duration
(dd.mm.yyyy) Mode (hh:mm:ss)
1 P1 12.12.2020 1 Zoom, video 00:38:27
2 P2 14.12.2020 2 Zoom, audio* 01:06:38
3 P3 15.12.2020 2 Zoom, audio* 00:50:42
4 P4 18.12.2020 2 Zoom, video** 00:50:02
5 P5 21.12.2020 3 Zoom, video 00:50:34
6 P6 22.12.2020 3 Zoom, video 01:08:43
7 P7 23.12.2020 3 Zoom, video 01:30:57
Total time 06:57:05
* interviewers had their cameras on
**partially in audio-only due to technical difficulties

It is also vital for confirmability and contextual purposes to highlight the interviewee’s
experience and position. The position showed in Table 7 is the main capacity of the
interviewee at the time the interview was conducted, although most participants have
provided insights to the UK public sector beyond their current role and organisation. It
is worth noting that it was a vital part of the interviews to ask the participants about
their background, role, experience, and the nature of the projects they are involved in
during daily operations. It was important for researchers to understand their perceptions.
Most crucial was understanding how their perception of BRM was formed as it is
essential for this research's quality. Information about the participants is summarised in
Table 7.

Table 7. List of participants.

Participant Education and Experience Types of

Role
Identifier Certifications in Years Projects
M.Sc. & IT systems
P1 PM Less than 5
PRINCE2 improvement
Chartered BRM
P2 10-15 Various
Accountant Consultant

42
 Business and
IST
P3 DBA Director Over 20
improvement
projects
Various Benefits
Various projects
P4 professional Manager and Over 20
on contract base
qualifications Subject Expert
M.Sc. & several PMO Manager Various projects
10-15 (in the
P5 professional and BRM (diverse
UK)
qualifications Subject Expert portfolio)
Infrastructure,
transport, and
Benefits and
P6 M.Sc. Over 15 business
Value Lead
improvement
projects
M.Sc. & several Benefits
P7 professional Realisation Over 15 IT projects
qualifications Manager

The first interview was conducted with a professional project manager and the second
was with a freelance benefits realisation consultant. Both interviews served as a means
to refine the interview protocol and the question asked by the interviewers. In a sense,
these interviews can be considered as a pilot. However, the authors have decided to
include the data from these interviews in the data set and their analysis. The reason for
this inclusion is that the research’s conclusion is drawn based on the reoccurrence of
patterns and is not dependent on a single piece of information or an isolated statement.
Moreover, comparing the first version of the protocol and the second, the refinements
can be considered minor. The interview protocols and their iterations can be found in
Appendix 3: Interview Protocols.

Except for the second interview, all participants have supported their opinions regarding
their specific organisation or project. Our second interviewee was obliged to preserve
the confidently of their clients. Despite this fact, it was confirmed that their experience
is relevant with several insights to the UK public sector benefits management.

Concerning interview 3 to 7, the participants were all senior managers. The participants'
roles varied between project management office leaders, benefits realisation mangers,
performance, and planning. These roles substantiate that they are typically in charge of
managing several projects at any given time. More importantly, due to the nature of
their roles, they are closely involved in the selection, appraisal, and outcome integration
into business-as-usual and the realisation of benefits. Therefore, it provides an
opportunity for more in-depth insight into BRM processes and the obstacles it faces,
especially while controlling and monitoring on-going projects.

A final important point to highlight is that most managers interviewed exhibited a long
and established career in the public sector. Therefore, in addition to their current
organisation, they provided examples from their previous employers (also public sector
organisations). Furthermore, several interviewees typically consulted other
organisations on the matter at some point in their career. Three of the interviews are also
experts who have published contributions on the subject. Therefore, the data set can be

43
considered high quality, relaying insights from diverse public sector organisations and
projects.

4.1.2 Analysis
As outlined in 3.3.4 Data Collection Process and 3.3.5 Data Analysis Technique, the
analysis of the data was carried out in three main phases. The first phase was concurrent
with the data collection. In this phase, the researchers collected notes and interview
summaries. The notes generally correspond to the question in the interview protocol.
These was divided into three main categories according to their subject: PM and BRM
guidelines, benefits realisation and the success definition (general overviews), and BRM
during gateways or control and monitoring (specific focus) – in addition to the notes
about interviewee background which is summarised in the above section of this chapter.

The second phase of the analysis comprised of developing a codebook based on the
researchers' notes from the data collection. A summarised version of the codebook is
presented in Table 8; a strong link to the aforementioned three categories can still be
seen. The extended version of the codes (Appendix 4: Code Book) was processed in
NVivo, a software package designed to aid in qualitative data analysis.

Table 8. Summary of initial codes categorises arranged alphabetically.

Code/ Description
Group of Codes
Benefits and their What are the benefits to be realised in the organisation, their
Categorisation categorisation, and other related issues?
To which guideline does the public sector organisation refer?

Biases Biases involved in BRM.

For example, the illusion of control, optimism bias, etc.

BRM Guide The application of governmental, practitioners, and/or

academic guidelines or frameworks for BRM.
For example, APM, PRINCE2, MSP, the Green Book,
Cranfield Method, etc.

BRM Process Statements about the BRM process effectiveness, mainly at

Effectiveness which stages it is effective and ineffective.
Statements about possible ways to improve the process.

Gateway Review Statements related to the effectiveness of the gate-reviews.

Process Does it lead to a change in direction or termination of the
projects? Is it a performance-oriented or benefits oriented
process? Challenges related to gate-reviews?

Organisational The organisation’s alignment between the set

Alignment guidelines/procedures and the practice.

PM Guide Refers to issues related to the project management body of

knowledge, guidelines, or methodology in use.

44
 For example, Agile, APM or PMBOK, PRINCE2, MSP, etc.

Project Termination Statements about termination as an indicator of BRM and

control monitoring effectiveness. If the projects are identified
as not realising benefits planned or is otherwise unsuccessful,
are they being terminated? How effective are organisations in
terminating projects? What are the obstacles to termination in
non-beneficial or underperforming projects?

Success Factors and Statements related to the definition and the measure of
Measurement success in the organisation.
Related guidelines to the measure of success.

The NVivo software package was then utilised to code each relevant statement from the
interviews' transcripts to a corresponding code. Simultaneously, the addition of new
codes and expansion of the codebook was performed when deemed appropriate before a
final round of code revision. During the final step of the analysis, the codes were
grouped and arranged in higher-order themes and patterns. The result of this process is
the findings as presented in the next subchapter.

4.2 Findings
To present the findings of the thesis, the authors will be applying the Gioia method. The
Gioia method is a common approach to present the findings of qualitative interpretive
research, especially research that is based on interviews as the source of data (Reay et
al., 2019, p. 202-203). The Gioia method was selected as it allows for a familiar and
easy to follow presentation of the analytical process (Reay et al., 2019, p. 205).
Compared to other approaches to present findings, the Gioia approach is the most
suitable for data collected through interviews (Reay et al., 2019, p. 205). Moreover, the
lack of eventual sequence or any influence of time on the data makes it difficult for the
authors to present the findings in the form of a story; instead, short snippets from the
interview transcripts will be used.

The Gioia approach is based on the utilisation of different order codes, typically three,
with the first order being the closest to the data and participants statements (Reay et al.,
2019, p. 206). Here, no attempts should be made at creating categories. Instead, codes
should adhere to the informant terms, making the participants the knowledge agents
(Gioia et al., 2013, p. 20). This often leads to an overwhelming amount of first-order
codes which merely resembles concepts. As the analysis progresses, both differences
and similarities alike emerge, permitting the analysts to label and structure the codes
(Gioia et al., 2013, p. 20). This allows the analysts to become knowledgeable agents and
creates the foundation for the second-order themes (Gioia et al., 2013, p. 20).
Subsequently, similarities between the second-order themes are drawn to create
aggregated dimensions. These final, higher-order codes are the result of thematic
aggregation, representing the theoretical development (Reay et al., 2019, p. 206).

The data structure (Figure 9) inspired by the Gioia method summarises the findings of
this thesis. Additionally, a more extended version is presented in Table 11 (Appendix
5); in this version, the first-order concepts are presented as the codes used in the
analysis.

45
Figure 9 Data Structure presentation; inspired by Gioia method (Authors)

46
4.2.1 Project Management and Benefits Management Frameworks and
Governance
The first theme of findings was concerning the project management and benefits
management environment in the public sector organisations, especially with a focus on
the frameworks in use. The authors were interested in understanding what sort of formal
guidance, terminology, and language are used in the public sector organisations to
govern project management and benefits realisations management. These findings of the
project’s governance environment are essential to provide context to the research
question's answer. It is also essential to understand the governance of the project,
programmes, and portfolios, in addition to it being a part of understanding the
participant’s background understanding of the project management and BRM.

In response to questions like:

- Regarding language and terminology, which framework or methodology do you
apply or refer to?
- Based on your experience, which qualification do you hold and recommend for
project management or benefits management?

The authors found that all participants could confirm that particular project management
and benefits management guides had been implemented in their organisation to aid
project and portfolio management. An assortment of such guides was cited by the
interviewees, including the Green Book, IPA, PRINCE2, and APM. Therefore, the
authors argue that there is evidence of different guidelines being implemented in the UK
public sector.

However, with more close examination regarding project management, the authors find
that PRINCE2 is the primary qualification sought-after in the public sector. This is
because it has the most influence on the guidance and governance of projects. PRINCE2
is followed by MSP, which is mentioned by several participants. One of the managers
(P3) expressed this explicitly, explaining:

P3: "I would say that the reality is we use a combination of different methodologies. But
the language that we tend to use when we talk about projects and programmes is
definitely sat within PRINCE2 and MSP language."

P3: “I would say that, that's been true of all the organisations that I've worked in where
there has been a combination of methodologies that have been used. But the language
tends to be in PRINCE2 and MSP terms."

Additionally, quoting another highly experienced manager and expert in the area:

P5: “When you're in England in general, I think most organisations […] stick to the
PRINCE2 language, and that's what I tried a little bit. So what I do is, most of the
language I refer back [to and,] the terminology I refer back to is the PRINCE2
definitions, and I think it makes sense because it's the country’s […] framework, [it]
was developed by the government and […] most the UK public sector organisations

47
 they use PRINCE2 and that's why I adopt this language. Because most people here
they're qualified PRINCE2, most project managers.”

Despite that, there is a pattern that suggests the inefficacy and inadequacy associated
with PRINCE2. This is especially true with managers working with information
technology and systems improvement projects and programmes. This environment is
predominately agile. The waterfall orientation of the PRINCE2 framework is especially
problematic, as described by two of the managers.

P1: "So, it [PRINCE2] gives you just a status […] I don't think you get something extra.
Especially because you might be working in projects that are more agile. And then
PRINCE2, the standard one the waterfall, it won't really help apply the techniques
they're asking, or at least not all of them."

P7: "We tend not to use PRINCE2. […], within the projects, or rather, within the
products, we don't use PRINCE2, we use our child, we use a form of Agile […] that's
taught by GDS, the Government Digital Services."

The authors concluded that the framework's waterfall orientation is problematic at a

portfolio level as a framework for governing agile projects. The waterfall approach
depends on clear starting and delivery milestones for the clearly defined requirements
and scope. On the other hand, the Agile approach works in cycles (or sprints) to deliver
a project with evolving and changing requirements. This different approach to project
management might even be described as contradicting. Therefore, Agile project
management and waterfall governance (or portfolio management) is a potential source
of inefficiency and conflict in the control processes and governance.

On the other hand, the limited scope of the PRINCE2 qualification was criticised,
suggesting that more holistic guides might be beneficial. This criticism comes in line
with few organisations starting to move away from PRINCE2 and adopting the more
comprehensive guides like APM Body of Knowledge and the association's
qualifications.

P5: “What I recommend to people is to take the APM qualifications, […] because I
believe that they give you a broader view of things, and PRINCE2 can be used as an
entry path to take the APM certification, so you can take PRINCE2 first and then take
the APM qualifications. But I always recommend people, if people don't have any
qualifications, I always recommend them to take the APM qualification straightaway,
you know, because they cover things in a more like, holistic way, I think.”

The PMI qualification was also praised but less prevalent in the UK than in North America and
other parts of the world. Cross-national comparison is beyond this work's scope. Despite this,
there seemed to be awareness of this fact among managers.

P1: “I was thinking for PMI. But I realised it's not really been asked [for].”

P6: “PMI is starting to come into UK a bit. Obviously, they're a US institute, and they
started there in the US, but they are slowly increasing their coverage, especially with
central government. So you'll find a few PMI organisations as well, I think it'll be

48
 unlikely you find somebody that is 100%, one of those, and they will all tend to borrow
different bits and create their own methodologies.”

Furthermore, we find a clear presence of the mixing and matching of different

guidelines. It is expected that the process and practices are adopted and altered to fit the
project management conditions. Project managers leverage their experience to “beg,
borrow, or steal” to find the best practice to fit different projects' conditions and
circumstances. Then again, expressed by one of the senior managers when asked about
the languages and terminologies in their organisation:

P6: “We've [used to] beg, borrow and steal […] probably from lots of other
organisations and lots of other frameworks.”

Then on another occasion, they add:

P6: “Well, I would be surprised if you found everybody following exactly the same
thing. They will, in spirit, follow the same thing and then adapt it to what suits their
circumstances.”

This sentiment is shared across different organisations, sectors, and project types across
the public sector, highlighted by the following two quotes:

P3: “We use the language of PRINCE2 and MSP, and we use a mix of methodologies.”

P5: “I've been working for more than ten different organisations in my career, and all
of them had their own project frameworks that were designed based on their specific
needs, their specific project type.”

When examining the benefits management guidelines specifically, the authors found
that the influence of PRINCE2 and MSP guidance is still present. However, it was
observed that PRINCE2 and MSP presence complement the Green Book and IPA
guide’s strong presence and dominance on BRM guidelines. The authors have also
noted that organisations modify, tailor, and adapt the methods from different guides to
suit their specific needs. It must also be acknowledged that this observation was noted
in a non-central government public sector organisation, Arm’s Length Bodies. Most of
the interviewees' experiences are relevant to ALB. ALB’s may not have an obligation to
follow central government-specific guides, although they tend to align themselves with
their regulator or parent central government department, for example, NHS and the
Department of Health and Social Care, or Transport for London and the Department of
Transport.

P2: “In the public sector […] the Green Book is the Holy Bible…”

P2: “…the Green Book is number one, and Managing Successful Projects is another.
So that's […] the OGC, OGC guideline.”

P7: “When you look at a standard job description for anyone in project program,
portfolio, or benefits management, it will not say APM, PMP, or PMQ or any of the
others, for benefits, it will say the managing benefits, qualification for programs, it'll
say MSP and it might have PRINCE2 or equivalent.”

49
 P7: “We follow the Green Book, the IPA Guide, […] Best Practice Guide for benefits
management […] we use the categories in there. So […] it's a mash up, I hated it.”

Then they added:

P7: “…And as far as benefits is concerned, yes, we refer to the IPA guide and so on.
But ultimately, certainly, our techniques are essentially the managing benefits textbook
and all of our guys follow that, because we have a community of practice, as do all our
professions, where we standardise the way that we work. So, we'll use for instance at
least some version of the modified results chain, which is a start-up benefits mapping
that came out of the Fujitsu results chain. And it's a hybrid between that and
Cranfield.”

The findings in this area are essential to understand the governance’s influence on BRM
effectiveness. It is connected and overlapping with the findings under that theme,
especially when it comes to an understanding of the definition of benefits. However,
before moving on to examine benefits management effectiveness more closely, another
contextual introduction is essential. This introduction serves to understand the definition
of success and what roles benefits realisation plays in it.

4.2.2 On Success: Its Definition, Measures, and Factors

The definition and metrics of success have been recognised as one of the discussion's
elemental axis. It was frequently raised as a conceptual line of reference to explain
benefits and benefits management matters. About 30% of the statements codified in this
study were related to success measure. This high frequency of codifying (reference)
might indicate the qualitative importance of the concept as a glue that links the project
management performance and benefits management. It is a crucial concept to
understand the managers' perception of benefits management and its importance. Hence,
understand the effectiveness of the process and factors affecting it. This thesis focuses
on control and monitoring during implementation; therefore, it is essential to understand
whether benefits are part of the baseline for control and monitoring.

When it comes to success measures, the authors will present their findings by breaking
the concept into three distinct aspects: the theoretical definition, the actual practice, and
the gap between theory and practice. The theoretical definition can be further divided
into the managers' understanding and beliefs and the guidelines' definition and
measures. For the latter, referring to theoretical guides, the authors have already covered
in the previous section of the findings. This section will focus on the beliefs and the gap
between theory and practice.

The first theme is the managers' beliefs and understanding of success. The data suggest
a general state of awareness of the difference between project management performance
or, as referred to in another part of the literature, project management success (Williams
et al., 2020) and the achievement of the strategic objectives and benefits realisation. For
instance, one of the managers in charge of a project management office has expressed
this saying:

P5: “If you are assessing the performance of the project manager in [a] British
management team, the Iron Triangle is probably fine. But if you are assessing the

50
 success of the project, in enabling the realisation of benefits, in delivering value to the
business, then the Iron Triangle is not enough. But what I see is many organisations are
wanting to assess the return on investments, and assess the performance of the project
manager but by the return on investment delivered by the project. And that's something
that's unfair, you know, because in many cases, the return on investment will happen a
long time after the end of the project.”

The data also suggests that the managers believe that project management high
performance does not always translate to a successful project or initiative if benefits yet
to be realised. Therefore, there is a certain tolerance for project management
professionals to accept delays in schedule and variations in the budget if this translates
to either benefits realisation, an increase in the quality of the delivered outcome, or an
increase in the customer (or user) satisfaction. Most interviewees recognise the
importance of enabling benefits realisation management through the definition of
success and found it as necessary and vital as incorporating the 'triple constraint' or 'iron
triangle'. It has been found that an extended measure of success is the more prevalent
sentiment.

P3: “What we strive for, within the programmes of change that we put in place, we
don't always get this, I should say, but this is the intention. And what we strive for in
terms of success is to ensure that the benefits that are identified at the beginning are
managed and tracked and well defined throughout projects and programme delivery
and that, this is the critical bit for me, they are part of the business change
programme.”

P3: “Where I have a problem is if you don't do the benefits realisation and you run
over, whether it's time cost, money, whatever it may be. If there is an overrun, and you
haven't done the benefits realisation either, then it's a double failure for me.”

P6: “So what makes a project successful? Yes, we could give you lots of reasons why
our projects are successful. But do we define it as you meet this criteria, therefore, your
project is a success? No, we probably don't. We probably use exactly the same as you
will have heard from other people if you've done this kind of work. So the typical Iron
Triangle of time, cost quality. You want things to be on time where possible, you want
them to cost what you said they were going to cost, and you want them to be meeting
user expectations, which is fine. And I would always obviously add benefits to that, that
says you actually deliver the benefits that you expected to get from the project. And
there's probably lots of other ways we can describe success as well.”

P7: “…how they [the organisation] define success, and they will say that benefits are
one of the six factors of control. So going back to the PRINCE2 time, cost, quality,
scope, benefit, [and] risk, which is also what's taught in management of portfolios.”

The Iron Triangle is still a primary measure of management performance. However,

other aspects were recognised, like stakeholder satisfactions. When it comes to a
comprehensive definition, the authors would like to borrow the expression of ‘Iron Star’
as a definition of success, in which benefits realisation and risk management are added
to the triple constrain.

51
 P7: “I'm a big fan of the iron star, not the
triangle, that this, it needs to be both triangles...”

P7: “I don't agree with the culture of focusing on

cost and quality and scope to the exclusion of risk
and benefit management. And our formal
definition would say that benefits is, you know,
they would say, cost, quality, and benefits. They
don't even think about the other three, which also
annoys me.”

Additionally, the discussion of success is not

complete without examining themes regarding the
Figure 10. Iron Star, 6-point criteria for
gap between theory and practice. Defining and success (Authors).
ensuring a project's success is still a challenging
project management area despite the clear understanding of benefits realisation
importance when it comes to the projects’ success. This comes in conjunction with the
difficulties of differentiation between project management performance and the
achievement of strategic objectives. The managers interviewed still reported difficulties
in their organisations for stating a clear definition of success. In other cases, difficulty in
adhering to the organisation's definition or definition of a successful project was
mentioned. The authors have also noted that definitions might change situationally,
adapting to the project at hand. The issues pertaining to the implementation of a success
definition are exhibited in

Table 9 below.

Table 9. Implementation gap between the theoretical definition of success and practice.

First Order Code Supporting Extracts from the data

The definition of success is P1: “Not aware to be honest, because like, with my
sometimes unclear or teammates, it’s very specific to what we are doing.” they
contradicting definitions replied when asked about the definition of success in their
organisation.

P5: “So it's based on someone's perspective, so what

success is for me, may be something different for you, and
may be something different to another person. So I think
this is the first thing that's important about project
success. Project success, it's about different
perspectives…”

P6: “So what makes a project successful? Yes, we could

give you lots of reasons why our projects are successful.
But do we define as you meet this criteria, therefore, your
project is a success? No, we probably don't…”

P7: “It’s like, what are the tolerances? I don't know. We

haven't had any sets. And those can't be set by the project,
they need to be set by the organisation, by the governance
structure.” - referring to tolerances in cost, time, quality

52
 and other parameters used to define success.

It is difficult to adhere to P3: “What we strive for, within the programmes of

organisational definitions in change that we put in place, we don't always get this, I
practice should say, but this is the intention. And what we strive
for in terms of success.”
The definition of success P3: “I think it's relative to what the particular
changes situationally programme of change is…”

P4: “That depends how constrained you are. If you have

a constrained budget, it may be more important that you
stick to the budget. And as soon as it becomes evident that
you're going to go over budget, you stop the project,
because that's your main constraint. If you have unlimited
funds, and you will you need to achieve a particular
objective for any cost, then if for instance it's a
compliance or regulatory issue, you might need to keep
on spending until you meet the objective. So I think it's
very, it' a sort of it depends question. I think it depends
what the value is that you're looking to get out of the
endeavour.”

P6: “We need to get to an agreement and we need to

make sure that everyone remembers that agreement all
the way through. Because what I saw happening a few
times is people agree on success right here at the
beginning, across the budget life cycle they forget what
they agreed on.”

53
4.2.3 BRM Effectiveness
Through this section, the authors will be examining overarching themes uncovered
through the interviews about the BRM effectiveness in the UK public sector. Then, they
will funnel to more findings specific to the process effectiveness during the
implementation of the projects in the next section. Both sections are outlined in Figure
11.

Figure 11. BRM effectiveness findings mind map (Authors).

The first theme found revolves around the definition of benefits and their measure. It
should be noted that this theme also overlaps with the frameworks and governance
concept discussed above, in addition to the previously discussed influence of the
Greenbook and IPA guide. The importance of societal benefits – benefits realised by the
whole UK general population – has been emphasised. These are often referred to as
benefits realised by the UK public.

P2: “… in huge organisations like the health sector, defence, [...] transport definitely,
[…] the benefits are beyond the organisation. And it's not just improving the
organisation, but the effects of society. So, carbon emissions are a good example of
them. Things that affect the GDP...”

P7: “then you've got societal or public benefit, […] it can be defined in cash terms. So,
it has a cash or a monetary proxy value or actual value, but the beneficiary is someone
other than the public, than the Treasury, which is either citizens or UKPLC, …”

54
Despite the importance and the fundamental roles of such benefits in the public sector, it
was found that they are problematic to deal with from a BRM perspective. Public
benefits are difficult and expensive to track and measure. Two primary sources of
difficulties related to these benefits are their nature – often being non-financial – and the
long time-horizon they are realised over. For non-financial benefits, QALY (Quality-
Adjusted Life-Year) or proxy measures are often needed.

P2: “QALY, which is quality adjusted life years. There's a fantastic formula that people
in the health sector use but also, in things like defence and transport for valuing the
number of years of benefit by either proving the technology or processes. So it's a […],
it's a financial benefit that relates to society.”

P7: “that's where the value of 60,000 pounds per QALY comes from […] it is ultimately
putting a price on life on quality of life. […] I've nearly managed to persuade them [the
organisation] on several occasions [to use it]. And then my funding has been cut,
because it requires a very large before and after data set, […] and that's an expensive
thing to do, even if you're using someone else's primary source. […] so they have a data
set, they were already bearing the cost of collecting that data, you'd still need to employ
an economist to analyse that. So yeah, it's not a cheap thing to do.”

P6: “...some of the benefits can take a long time to be realised. And I think that is a bit
of an issue that says, okay, in order to truly see the return on your investment, you need
to keep measuring for 10 years, 20 years, the asset life of some of our bridge[s] is 100
years, asset life never met the benefits for 100 years. It's just not cost effective…”

The participant has added that their organisation limits tracking benefits to generally 36
months after its infrastructure projects delivery to overcome this issue. In this particular
type of projects, transport infrastructure, several factors interact, and the longer the time
horizon, the harder an effect or benefit can be tracked to a particular project – at least
with sufficient economic certainty. This has been identified as a suggested improvement
in the benefits of realisation management.

On the other hand, the authors have identified a significant issue regarding the
organisational alignment to BRM. It has been found that organisations tend to define the
solution before the problem. This is especially prevalent in situations where there is a
strong influence to implement a specific project. Hence the process becomes
maximising benefits, eliminating disbenefits, and reducing cost. This comes at the
expense of systematic selection from a range of alternatives. Detailed findings of the
influences and obstacles will be elaborated on later in this chapter.

P1: “…the most important thing is just getting the project right from the very
beginning.”

P2: “And that's why a lot of projects fail, because they try and do the business case
before the benefits rather than vice versa...”

P7: “In reality, the organisation for a long time has had an attitude of ‘benefits is what
we need to do in order to get business case past so we can get on with the important bit
of building the thing’. Not even using the thing, let alone using the thing in order to
realise benefits…”

55
Despite the above issue regarding organisational orientation and selection, BRM has
been reported to have a significantly superior condition at the front-end of projects
compared to the rest of the lifecycle. The front-end of the project, when benefits are
identified and planned, is thought to be executed more effectively than tracking and
measuring or the back-end of the project. The data suggest that BRM practices are
inadequate once a project is appraised. This links strongly to issues related to the
definition of benefits previously mentioned and as expressed in the below extracts:

P4: “The public sector is a lot better at doing the work at the front end of the project
lifecycle to, to find and identify benefits and quantify them for business cases. But I still
don't think they're very good at tracking and reporting benefits. And I think the number
of post implementation reviews that include a full accounting of how many benefits have
been realised is very, very small”

P5: “[…], they just, well, sign off benefits to get the investment approved. And then as
the project’s progressing, people just forget. […] that's one of the challenges I face
every time [I] implement project [management] or I reshape the project management
process of an organisation, because we need to educate people to keep tracking
benefits, keep reporting progress. But you're right, most organisations don't do it. And
it's a matter of maturity”

P5: “I cannot say that my current organisation is very good. And it's something that I'm
working on, it is work in progress. But this becomes more difficult. But it's something
that is important. But you need to make sure that you have the metrics in place, and you
need to have the process in place to track benefits after the project closure.”

P6: “I think benefits realisation is the forgotten bit of benefits management that most
organisations are pretty good at identification.”

4.2.4 BRM Effectiveness during Implementation

As demonstrated above, benefits are carefully considered during the selection and
planning phase of a project. However, the responses on how well-implemented benefits
management is during the projects' execution phase are less positive. This study was
carried to understand the effectiveness of BRM during the phase between appraisal and
delivery. The main question the authors were trying to answer was if benefits realisation
is an important aspect in the definition of success and whether this reflects
proportionally on its utilisation during monitoring and controlling projects or not.
Therefore, two indicators were chosen to answer this question: gateway reviews and
project termination through gateway reviews.

Gateway Reviews
The authors chose the gate-way reviews, particularly gate reviews after the appraisal, as
an indicator of the effectiveness of BRM for project monitoring and success assurance.
The governmental guidelines mandate many projects in the public sector to go through
gate reviews. It has been found that benefits are considered during these reviews.
However, it has also been found to be an unsatisfactory consideration, with the process
often showing signs of being flawed and inadequate.

56
 P3: “There's definitely a review, it's very formal. In fact, there's more than one review
so that there'll be the internal review, and there's an external review that occurs from
independent organisations. So generally, sort of the Big Four accountancy firms will be
involved in in doing their own independent review as well. And then there is a final
review by the politicians with that particular project with the kind of the leaders of the
[organisation name] and [organisation name] improvement will also look at the
outcomes from those gateways and make their own assumptions. So, for that particular
project, there's […] a lot of scrutiny that happens at the gateway points, and that
considers those changes that you've described, in terms of the where we currently are.”

P5: “Because benefits are part of the whole process, you start the project with benefits
in mind. And then you might manage the project across the entire lifecycle with benefits
in mind. And that's why not only in the [organisation name], but in most of the
organisations I worked in before, every project status report has a section on benefits.
And we ask a question, I used to expect them to realise the same benefits. Is anything
happening that may affect the benefits realisation that was expected at the beginning?
Has anything changed that may affect the benefits realisation? So that's in all projects
that I've been managing in my portfolios. This is the type of question that we ask in each
review cycle in its reporting cycle.”

The data suggest that during the execution phase, benefits management inclusion is not
often prioritised in gate reviews for various reasons.

P6: “So one of the problems with benefits or gates is, we tend to be very product
focused.”

P7: “So as a gateway reviewer, my experience of it is that the ... the perception from
many of the organisations that received gateway reviews or have had them conducted
on their projects, is that benefits are dealt with in gate zero and gate five, and not in one
through four, or is it two through four? I can't remember exactly.”

P7: “So a gateway team is usually three to five people, a team leader and additional to
them up to 1-4 members. So you've only got four slots. And most of them, all of them
that I have worked with, they have never called on a benefits person to fill one of those
rare, four slots. And that's the maximum for a big project.”

Project Termination
Additionally, to further deepen the understanding of the gate-reviews effectiveness,
questions about the gate reviews' outcome were asked. It was believed that the useful
gate review would result in the proceeding of the successful projects and the termination
or redirection of less successful initiatives. The termination was chosen as an indicator
because it is a severe but desirable reaction to limit losses. It has been found that
projects are rarely terminated, although gate reviews often lead to holding or redirection
of the less successful projects.

P6: “I think having a good gate process allows you the mechanisms to stop projects.
And obviously the earlier the better because of the amount of money that you're
spending and the amount of effort that you put into them.” … “as I say the gates are
definitely a good mechanism to allow you to stop a project if you want to. But it's
unusual. It's an unusual process to stop projects, once they've they built a momentum
and they've got going.”

57
 P7: “I certainly definitely hear of projects that have changed direction or improved
governance processes or have significantly altered something about a project as a
result of a gateway review. And I have also worked on a couple that have, as a result of
their gateway review, change direction. And what I don't see, in gateway review or not,
I don't see our organisation killing enough projects.”

4.2.5 Negative Influences and Obstacles on BRM Effectiveness

Finally, it was essential to understand the cause of the current state of BRM. Therefore,
the interviewees were asked for the reasons that might contribute to this in-adequacy.
The authors have concluded that several obstacles could serve as barriers to project
termination. Examples of these originate from a variety of sources, and the most
prominent were political influence, government financing structure, press or public
opinion, sunk-cost, and self-serving bias. These obstacles are summarized with
reference in data Table 10 below. The effects of these negative influences are not
limited to a specific phase but can undermine the whole management process and the
realisation of benefits.

Table 10. Negative influences and obstacles against BRM effectiveness.

Obstacle Data Extract

Fear of P4: “I think one of the underlying issues in the public sector is a chronic fear of
Accountability accountability by civil servants. They, they have a sort of charter as employees
that they must take personal responsibility for their actions. And because of that,
they have an enduring fear of accountability. So any decisions they make in their
tenure, they can be held responsible for even after they've left their post. So this,
this creates a fear, culture, fear of accountability, culture and fear of
responsibility culture…”
P7: “The second is the culture. Again, I like to say accountability, but it is a
culture that does not recognise stopping as a project as success.”
Governance - P7: “So you've got Agile in the products, you've then got a very waterfall
Top governance process. So our business case structure, for instance, does not work
Management well with the idea of phase release the funding by stages.”
Influence
P7: “What are the tolerances? I don't know. We haven't had any sets. And those
can't be set by the project, they need to be set by the organisation, by the
governance structure. Because it's the governance structure telling you, as long
as you're within these points, we're fine about it. And you only have to tell us
about it. If you go outside of those lines.”
Government P6: “funding uncertainty, which is probably a bigger issue. So, so you will
Finance always get changing stakeholders’ perception of what's important, and whether
Structure some parties will support a project and some parties won't kind of stuff, you'll
always get there. I think the bit that makes it very difficult is, in order to manage
a big or any kind of project, you need to know [you have] the money to start it
and you need to know you’ve got the money to finish it”
P7: “We spent some money, we discovered that it wasn't going to work, and
therefore we stopped spending money. Surely that is a success? Surely that is a
good thing. But no, it's not seen as that. One of the difficulties you have in the
public sector is we're about the only sector of the economy that sees and
understand it as a bad thing. It's like, congratulations – you always should be
congratulations – you managed to achieve the same level of benefit that we

58
 wanted you to, for 20% less cost, we should be throwing a party. Oh no, you've
not spent the money. And the problem with that is the structure of government
finance. That means that we are awarded, we’re not awarded a recurring budget
in real terms. I mean, they call them recurring budgets, but effectively, they're
not. So if you don't spend the money, then next year, they will knock that much
more money off your budget. But next year, you might have needed it, or for the
next project you might have needed it. So they don't celebrate the fact that you
achieve the same for less money.”
Political P2: “But Boris has said, ‘No, we are going to sign off on this project’. Because
Influence this is good from a societal benefits aspect, it's going to improve the
transportation in the northern England, it's going to get people moving in, people
are going to buy houses near the railway line, we're going to reduce the weight of
the traffic on the on the system. And people … it's going to improve the economy
societal benefits.” - Referring to the issues related to the High-Speed Train 2
Project (HS2 project).
P3: “But I think unfortunately, as is often the case with very significant
programmes of work like this, that have a kind of a national context as well, the
politicians will always have their influence. It will always be the overriding factor
in the decision making. So with the best will in the world in delivering this, if
there is a political benefit to be gained by making a decision in one way or
another, or by influencing in one way or another, then that will always be the
number one factor. And that's not true of this project. That's true of any project.
That's, that's significant in that way, I think, definitely in the public sector,
because the politicians have the ultimate power in in the public sector in terms of
what is happening on the ground.”
Press or P7: “The press is the final thing I need to talk about. So the I talked about the
Public Daily Mail effect” … “It's a case of […] the Daily Mail or an equivalent
Opinion organisation – equivalent media outlet – spinning anything to make it look bad.
Influence And so that's … all that is absolutely political agendas, but it's not just political
agendas amongst politicians. It's political agendas amongst the free press. And
I'm all for free press, don't get me wrong. But we have to find a way for the press
to actually view it positively when we kill a project, and to celebrate it.”
Self-serving P7: “Personal accountability. So because there's rapid turnover in a lot of posts,
Bias particularly the more senior levels. It's about it being seen that to kill a project
can be career limiting.”
Senior P4: “So I think lack of strong sponsorship is an endemic problem.”
Sponsors or
Stakeholders’ P6: ”I think there are lots of projects that keep going because there is certain
Influence political supporters or senior stakeholders – doesn't have to be politics, it can be
senior stakeholders that are behind the project – want to see it succeed. So they
keep pushing, even though the project might not be the right thing to do, or be the
priority against other projects and programmes.”
Sunk Cost P2: “Let's use the money that we use for those smaller projects and focus on the
80/20 rule, and focus on getting it [completed] because we have already spent a
few million in the last 10 years. So we can't just write it off and say ‘Okay, let's go
back to the drawing board’.”

59
P2: “Absolutely. reprioritise. Not kill a project, that is quite rare, actually.
Because like I say, you've spent a few million pounds getting to this stage.”
P4: “And quite often it's to do with this attitude to sunk costs, you know, we spent
all this money so we should keep going. And I think it's a perverse view myself.
It's a bit like being a gambling addict, isn't it, you know, you just spend more and
more money in the hope that you might eventually make up your losses. I just
don't think it's a good attitude to have. But it is an attitude that does prevail that,
you know, we've sunk all this money, so we should keep going.”
P6: “I think there's a case that says you don't stop projects because of the huge
sunk costs that are involved. So you've spent so much and got so far that you're
never going to stop them because it just wouldn't be economically viable, because
you've already committed loads of costs.”

60
5. Discussion

Guidance
and
Goverance

Success BRM Process

Definiton Implementation

Figure 12. BRM effectiveness is determined by the interaction of several dimensions (Authors).

This thesis's findings resulted in seven main conceptual themes about 1. frameworks
and governance, 2. success definition, 3. BRM effectiveness in general and 4. during
implementation in particular, in addition to 5. gateways reviews and 6. termination as
indicators of the process effectiveness and finally, an overview of 7. possible barriers
facing BRM. Despite presenting these as isolated topics, the authors theorise that they
strongly related and overlapping. Whilst the governance structure and choice of
frameworks influences, or even determines, the organisational definition of benefits and
success, it also influences the implementation of BRM. Therefore, the issue must be
understood as an interconnected cycle, as demonstrated in Figure 12; the funnelling
illustrated in Figure 2 is also applicable as the processes and concepts are built over
each other, with BRM during implementation being the top of the pyramid.

First, concerning project management and benefits management frameworks and

governance. Evidence suggests that IPA (2017) guidance and the Green Book (HM
Treasury, 2020) are the most influential on the practice. The findings in that area are
consistent with the findings of Williams et al. (2020). They (Williams et al., 2020, p.
657) reported that organisations are also influenced by the MSP suite, which is also a
theme that has been identified through this study. PRINCE2 was the most influential
with regard to project management, even in Agile environments. In contrast to Williams
et al. (2020, p. 649), regarding the Agile environments, our data pointed out that the
practices and governance in place are more suited to waterfall projects with no effective
accommodation to Agile. The authors were not able to gather enough evidence to
suggest that there is enough accommodation made for Agile projects in the UK public
sector.

Furthermore, Williams et al. (2020 p. 657) have reported that benefits identification (or
definition) is made in accordance with the five-case model by HM Treasury. The results
of this study are in support of these findings. However, data suggest that the definitions
and terminologies are not clearly defined or strictly adhering to one specific source.
‘Borrowing, begging, and stealing’ different aspects from different frameworks seem to

61
be common in several organisations in an attempt to find the most suitable fit for their
organisation and specific project’s conditions.

Similarly, in regard to the definition of success, like many authors suggest, there are no
consistent definition of success. It has been found common in this sample that no clear
definition is typically adopted and implemented through the organisation. Rather,
success is defined in a subjective and relative manner. However, there has been a
general trend of accepting that success should be defined beyond the iron triangle, as
promoted by several authors (Badewi, 2016, p. 762; Zwikael & Smyrk, 2012, p. 17) and
practitioners (APM, 2019; PMI, 2017). The interviewees tended to agree that projects
can in cases be delayed or over budget but still successful, given that the discrepancies
from the plan are within certain tolerances that do not compromise benefits realisation.
Tolerance definitions is a matter that is related to the governance structure of the
initiative, for example the portfolio manager or the steering committee. The importance
of the benefits-cost ratio is emphasised in regard to the erosion of benefits. Additionally,
it requires mature benefits tracking and portfolio management.

The previous two themes were essential to arrive at an understanding of the general
management environment that enables or obstruct the effectiveness of BRM. As
discussed in the theoretical framework chapter, the definition of what substitutes a
benefit and success varies greatly according to the application of a certain guide and the
understanding of the organisation. Therefore, organisations utilising different guides
have contrasting definitions of what constitutes both a benefit and success. This may
ultimately lead to confusion and misalignment, with disappointed sponsors but satisfied
managers – indicating the ineffective application of benefits realisation management.
Therefore, it is essential to establish clear definitions and consistent (and concise)
language.

Previous studies (Williams et al., 2020, p. 650) about the effectiveness of BRM
frameworks application were inconclusive of the frameworks' usefulness. This study
also shares this sentiment. Nevertheless, more importantly, the findings of this study
suggest that BRM is not sufficiently mature, with the exception of the front-end
practices necessary to the approval of the business case. Despite the front-end stages
being the most mature and effective part of BRM, challenges were still reported. The
most notable of these challenges are quantifying and measuring benefits. Apart from
transport and infrastructure, organisations reported, for instance, difficulties using
QALY to measure benefits realised. The findings suggest that all stages of monitoring
and managing after the appraisal are challenging and in-adequate from a benefits
management perspective. Additionally, it was reported that projects lacked benefits
orientation as management's focus is often on building and delivering the product—this
is thought to be undermining the effectiveness of BRM.

The previous findings are essential in formulating the answer to the research question,
which aimed to understand the BRM effectiveness during the project's lifetime, with a
particular focus on the implementation phase (of the project). As suggested by the
findings, the authors concluded that benefits realisation management is not effectively
applied and utilised in UK public sector projects, neither to ensure success or monitor
and control programmes and portfolios' performance.

62
In addition to the immaturity of BRM after appraisal discussed above, gate-reviews
were used as an indicator of benefits realisation management effectiveness. The gate-
review as a process was reviewed, with additional attention given to the outcome of
gate-reviews. The rationalisation behind this was that an effective gate-review process
would result in the termination of un-healthy projects. Un-healthy projects here refer to
projects that will not deliver an output that will result in value creation to the
organisation or the public, as the benefits of such projects are compromised or
insufficient to justify project continuation. Gate-reviews, after appraisal, seem to lack an
orientation to benefits, with benefits coming second to other performance measures.
Although this is inconsistent with the interviewees understanding of success. This
inconsistency points to a cultural issue as the practice is not based on sound
rationalisation linked to BRM, in addition to a lack of robust frameworks to rectify this.

Furthermore, the findings suggest that there are not enough projects being terminated.
This fact can be used as an indicator of the ineffectiveness of the gate-reviews. Further
investigation has revealed that this can mainly be attributed to the biases in the process,
such as self-serving bias and sunk cost fallacy. In addition to this, the fear of
accountability, the influence of senior stakeholders and politicians, and the influence of
press and public opinion has been found to have detrimental effects, frequently
swinging the pendulum away from project termination. Addressing these issues will
require a more holistic approach than BRM can provide with its link to organisational
cultures and human behaviour.

It was rationalised that gate-reviews and terminations are an appropriate indicator of

benefits management's utilisation during the project implementation. However, this
research did not consider other indicators and the potential effectiveness of these
indicators. Different rationalisation may be an area for future research to add a different
perspective to the findings.

Throughout this research, some assumptions were deemed necessary to make. Most of
these assumptions were made due to practical reasons and the limited scope of the
study. It is vital to reflect on these assumptions carefully as part of the social
constructivist approach of this research. Other researchers may rely on other
assumptions or considerations depending on their subjective view. Hence, this might
have led to a different understanding and interpretation of the data.

The first of these assumptions is considering the UK public sector as a homogenous

entity and that conclusions built on understanding a part of it can be generalised for the
sector as a whole. Therefore, the difference between defence, law enforcement,
education, health care, infrastructure, and transport was disregarded. However, this
assumption is a limitation of the study, as most of the data collected were from health
care, transportation, and infrastructure organisations. Despite that, with the exception of
military projects, most types of projects in the public sector were included. For example,
the data included a reference to infrastructure, transportation, business improvement,
and IT improvement projects. Therefore, the findings still have merits for creditability
and generalisation. Another similar assumption is that the data was mainly collected
from ALB organisations, as it was not possible to include the central government or
local government in the sample. The perspective of the findings being ALB
organisations must be considered when generalising the implications. Both assumptions

63
were necessary due to the sampling method and the difficulty in reaching and
interviewing a wide range of participants.

This study's limitations are due to the inherent limitations of the methodology applied,
similar to previous work, for example Williams et al. (2020). First, the sample size of
seven interviews is regarded as acceptable in general, as justified in previous sections
(3.3.3 Sample Selection), especially considering the study's limited scope and schedule.
Despite this, a larger sample is desirable to verify and confirm the trends uncovered.
The authors believe that a larger sample can lead to better generalisation and the overall
higher quality of research. In addition, the sampling method might have some
limitations, possibly affecting the results. As convenience and snowball sampling were
used, the sample focuses on highly qualified and knowledgeable professionals – in some
cases subject matter experts – with a good understanding of the theoretical and practical
issues related to benefits realisation management. An outcome of this might be
subjecting the results to confirmation bias. This effect is amplified by the participants all
being part of the mutual professional spheres.

A significant challenge to this work, and academic work in general, is expanding the
discussion sphere to include more views from the industry and the general public. This
was indeed a significant issue in front of this research. The researchers had difficulties
connecting and reaching out to diverse groups of project management professionals.
Due to snowballing as a sampling method, most of the participants are already
interested in benefits management with high qualifications in project management and
benefits management. Furthermore, a good portion of the data collected was collected
from what can be considered subject matter experts in benefits management. The prior
knowledge and expertise about benefits management are not considered a limiting
factor as the data are approached critically. The primary issue is in the self-fulfilling
prophecy or confirmation bias. The researchers and the participants shared knowledge
of the same literature and issues pointed in it.

Furthermore, similar limitations were reported by Williams et al. (2020) due to bias in
recruiting 'enthusiasts' of benefits management. Like our subjects, their subjects often
shared mutual connections, social and professional networks, lack of educational
diversity, and were highly educated and qualified BRM experts. This issue in recruiting
a diverse sample may lead to possible confirmation bias, which would invalidate a
quantitative research’s results. However, in qualitative research, this is identified as a
limitation that will not compromise the research and findings' integrity (Williams et al.,
2020).

64
6. Conclusion
6.1 Closing Remarks
Benefits are measurable and positively perceived changes that follow a project. BRM,
the process of managing and organising benefits realisation, is a vital complementary
practice to ensure the validity of investments in different projects. The BRM concept's
importance increases with the growing adoption of projects to achieve strategic
organisational objectives. It comes as a part of the need for more rigorous practices to
measure and ensure projects' performance and success. Therefore, this thesis intended to
explore the effectiveness and use of BRM practices in the UK public sector projects
environment. A further focus on benefits orientation during the project lifetime,
specifically after appraisal and before evaluation, was set. The UK public sector was
chosen due to its complexity which led to the development and maturity of BRM
practices.

The topic was approached through an exploratory, inductive method. Hence, the data
necessary was collected by utilising semi-structured interviews. A total of seven
interviews were conducted with managers, experts, and consultants working in the
public sector. The interviewees' combined experience in various projects and
organisation covered almost the entire sector, resulted in high-quality data and a broad
understanding of the topic.

Through the analysis of the data, it has been found that the practice of the PM and BRM
is greatly based on the guidance of PRINCE2, MSP, the Green Book, and the IPA
guide. However, the general sentiment is that BRM is only effective and developed in
the front-end of a project. The governance of projects tends to lose their benefits
orientation after the appraisal phase. The data suggest that the BRM comes in lower
priority after the project performance metrics, hence, undermining the effectiveness of
the practice. BRM review processes during the lifetime can be an area of further
improvement in many public sector organisations. Additionally, this deficiency is
closely related to the continuation of ‘pet projects’ and sunk-cost fallacy in many
organisations, which is undermining portfolios and investment optimisation.

To further improve the success of projects and enhance benefits realisation, it is

suggested to, amongst others, a) take a critical approach towards sunk cost fallacy,
which might be a significant obstacle b) improve stakeholder management as senior
stakeholders and sponsors play a predominant role in the public sector, and c) ensure
that the governance and the organisations' leadership will support BRM processes
throughout the life cycle.

6.2 Theoretical Implications

The authors believe that this thesis has theoretical implications that is valuable to the
academic management community. First, the study was rationalised by the existence of
a research gap and the importance of addressing this gap. Subsequentially, this study
aimed to narrow the research gap as mentioned earlier that was described as related to
BRM frameworks application effectiveness and role in monitoring and controlling.
Therefore, it contributes through providing new empirical data focused on the utilisation

65
of BRM as part of the monitoring and the controlling processes through projects and
programmes lifetime, especially the execution phase. These empirical data have
confirmed previous authors' results (Williams et al., 2020) and built on them to
understand the BRM practices during projects implementation further.

As an explorative and inductive study, the thesis provides through its findings’ new
theorisation about the utilisation effectiveness of BRM. It theorises that BRM effective
utilisation is not adequate in the projects’ lifetime phases after the appraisal and before
evaluation. This is an important outcome as it provides academics of an understanding
of the observed utilisation of BRM. In addition to the likely underlying issues, for
example issues related to biases, external and internal influences, and outcome-oriented
culture. This is a vital implication to further understand the practice and develop it to
improve public sector organisation performance or organisational success. This is also
helpful in understanding the effectiveness of BRM in general as these implications can
also be generalised to other non-commercial projects outside the public sector. This
theorisation can additionally provide sufficient starting grounds for further research in
BRM. These starting grounds are in the form of testable hypothesises.

In conclusion, this thesis contributes to project management and portfolio management

literature in general and the literature on projects and organisational success measures.
Additionally, it provides new empirical data from field research and theorisation by
analysing these data.

6.3 Managerial Implications

This thesis's managerial implications are addressed to different managerial levels. It is
thought to be of interest to policymakers, project governance structures (senior
management), portfolio managers and project management office leaders, benefits
managers, and project managers who might find it of interest.

On the highest level, policymakers and senior stakeholders must be aware of their
influence on the BRM process and the fact that specific policies are hard to translate to
successful projects regardless of the project management performance. In other words,
the tension between policy and project delivery must be avoided by employing an
objective BRM practice. BRM is one of the most effective ways to connect the strategy
with projects through objective, measurable parameters. Therefore, it is vital to ensure
its promotion in different organisations and take steps to align the work culture with
better benefits orientation.

Similarly, governance structures – the highest managerial level – are responsible for
supporting and promoting rigorous objective BRM, ensuring benefits-oriented culture
and better benefits realisation. Governance, along with portfolio managers, benefits
managers, and other concerned managers, must outline a clear and comprehensive
definition of success. This definition must include benefits realisation – in addition to
appropriate tolerances – in benefits and performance metrics that fit each portfolio’s and
project’s condition. Furthermore, and more importantly, it is crucial to ensure an
adequate number of reviews during project implementation that assesses the benefits
realisation and the BRM process. These reviews before delivery are vital, as it is often

66
too late and non-beneficial to investigate the benefits realised after the output is
delivered, and the investment is fulfilled.

On the programme level, programme managers are responsible for accessing and
tracking the component of the programme with BRM tools. Along with gate reviews or
reviews in general, it may be useful to identify any elements that may impact benefits
realisation.

Finally, different management levels must be aware and cautious of 'pet projects' and
the sunk cost fallacy. Structures, procedures, and culture for termination of pet projects
and avoiding sunk cost must be promoted. Strict objective measures must be applied to
projects, such as their potential benefits realisation and strategic fit; whereby they will
proceed only if the result is positive. Terminating a project must be discussed openly as
a valid alternative, especially in projects without a physical output like organisational
change and IT improvement project.

6.4 Future Research Agenda

The findings of this thesis serve as a basis for future research. More focused research on
specific organisations, such as the NHS, is necessary to better understand BRM's
effectiveness during the projects’ lifetime. Additionally, structured cross-organisation
and cross-sector comparisons as part of a macro-study are necessary for highlighting the
exact differences between different organisations, sectors, and government bodies – for
example, ALB’s compared to the central government.

In future research, it is suggested to utilise quantitative methods to confirm this

exploratory study’s results and the literature it was based on (for example, Williams et
al., 2020). This study's findings can be used as hypothesises for a future project with a
different methodological approach. It may test the following findings as a hypothesis:
- BRM is not sufficient after the appraisal stage, during the project lifetime, and
after delivery.
- Public sector organisations lack benefits orientation during gate-reviews; the
concept is not given enough priority.
- The underperforming project's termination rate is low – an indicator of broader
governance and portfolio management issues.

The authors also encourage future research to understand the impact of behavioural
psychology on managers' and stakeholders’ perceptions and decisions in the context of
BRM effectiveness. For example, a deeper understanding of the extent and exact
impacts of self-serving bias and optimism bias is a vital and exciting future research
topic. The impact of sunk cost fallacy in public projects is also identified as an area for a
further research agenda. Although the sunk cost fallacy may have been previously cited
as a barrier to project success, a further understanding is still required on why the issue
is persisting. Finally, it has been found that the influence of policies and policymaking
is significantly impacting the selection, continuation, or termination of public sector
projects. Therefore, it is a critical perspective to consider as a focus in future studies
about projects success and BRM.

67
References
Aalto, T. (2000). Strategies and methods for project portfolio management. In: TU-
22.451 Seminar in Project Management. Helsinki, Finland, March 7.

APM (2019). APM Body of Knowledge. 7th edition. Buckinghamshire: Association for
Project Management.

Archer, N. P., & Ghasemzadeh, F. (1999). An integrated framework for project

portfolio selection. International Journal of Project Management, 17 (4), 207-216.
doi:10.1016/S0263-7863(98)00032-5

Badewi, A. (2016). The impact of project management (PM) and benefits management
(BM) practices on project success: Towards developing a project benefits governance
framework. International Journal of Project Management, 34 (4), 761-778.
doi:10.1016/j.ijproman.2015.05.005

Bell, E., & Bryman, A. (2007). The ethics of management research: an exploratory
content analysis. British journal of management, 18 (1), 63-77. doi: 10.1111/j.1467-
8551.2006.00487.x

Ben-Horin, M., & Kroll, Y. (2017). A simple intuitive NPV-IRR consistent ranking.
The Quarterly Review of Economics and Finance, 66, 108-114.
doi:10.1016/j.qref.2017.01.004

Benaija, K., & Kjiri, L. (2014). Project portfolio selection: Multi-criteria analysis and
interactions between projects. International Journal of Computer Science Issues, 11 (6),
134.

Bradley, G. (2006). Benefit Realisation Management: A practical guide to achieving

benefits through change. 1st edition. Aldershot: Gower.

Breese, R. (2012). Benefits realisation management: Panacea or false dawn?

International Journal of Project Management, 30 (3), 341-351.
doi:10.1016/j.ijproman.2011.08.007

Breese, R., Jenner, S., Serra, C., & Thorp, J. (2015). Benefits management: Lost or
found in translation. International Journal of Project Management, 33 (7), 1438-1451.
doi:10.1016/j.ijproman.2015.06.004

Breese, R., Jenner, S., Serra, C., Thorp, J., & Badewi, A. (2016). A unified view of
benefits management/benefits realization management to be integrated into PMI
standards. [Unpublished report]. Sheffield: Sheffield Hallam University. Available at:
http://shura.shu.ac.uk/15299/ [Retrieved: 2021-02-16].

Bryman, A., & Bell, E. (2011). Business Research Methods. 3rd edition. Oxford: Oxford
University Press.

Burrell, G., & Morgan, G. (1993). Sociological paradigms and organisational analysis:
Elements of the sociology of corporate life. 1st edition. Farnham: Ashgate.

68
Chan, A. P., & Chan, A. P. (2004). Key performance indicators for measuring
construction success. Benchmarking: An International Journal, 11 (2), 203-221.
doi:10.1108/14635770410532624

Cooper, D., & Schindler, P. (2014). Business Research Methods. 12th edition. New
York: McGraw-Hill/Irwin.

Daniel, J. S. (2019). Open Universities: Old concepts and contemporary challenges.

International Review of Research in Open and Distributed Learning, 20 (4), 195-211.
doi:10.19173/irrodl.v20i3.4035

Dey, I. (1993). Qualitative data analysis: A user-friendly guide for social scientists. 1st
edition. London: Routledge.

Ebbesen, J. B., & Hope, A. (2013). Re-imagining the iron triangle: embedding
sustainability into project constraints. PM World Journal, 2 (3).

Edwards, R., & Holland, J. (2013). What is Qualitative Interviewing? 1st edition.
London: Bloomsbury Academic.

Eisenhardt, K., & Graebner, M. (2007). Theory Building from Cases: Opportunities and
Challenges. The Academy of Management Journal, 50 (1), 25-32.
doi:10.5465/AMJ.2007.24160888

Farbey, B., Land, F., & Targett, D. (1999). Moving IS evaluation forward: Learning
themes and research issues. The Journal of Strategic Information Systems, 8 (2), 189-
207. doi:10.1016/S0963-8687(99)00021-9

Fisher, C. (2010). Researching and Writing a Dissertation. 3rd edition. Harlow: Pearson.

Ghasemzadeh, F., & Archer, N. P. (2000). Project portfolio selection through decision
support. Decision support systems, 29 (1), 73-88. doi:10.1016/S0167-9236(00)00065-8

Gilbert, N. (2008). Researching Social Life. 3rd edition. London: Sage.

Gill, J., & Johnson, P. (2002). Research Methods for Managers. 3rd edition. London:
Sage.

Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2013). Seeking qualitative rigor in
inductive research: Notes on the Gioia methodology. Organizational research methods,
16 (1), 15-31. doi:10.1177/1094428112452151

Golafshani, N. (2003). Understanding reliability and validity in qualitative

research. Qualitative Report, 8 (4), 597. doi:10.46743/2160-3715/2003.1870

Hammersley, M. (2013). What is Qualitative Research? 1st edition. London:

Bloomsbury Academic.

Hart, C. (1998). Doing a Literature Review. 1st edition. London: Sage.

HM Treasury (2018a). Guide to Developing the Programme Business Case

[Government Guidance]. HM Treasury and Welsh Government. Available via:

69
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment
_data/file/749085/Programme_Business_Case_2018.pdf

HM Treasury (2018b). Guide to Developing the Project Business Case. [Government

Guidance]. HM Treasury and Welsh Government. Available via:
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment
_data/file/749086/Project_Business_Case_2018.pdf

HM Treasury (2020). The Green Book, Central Government Guidance on Appraisal

and Evaluation. [Government Guidance]. HM Treasury and Government Finance
Function.

Hudson, N., & Thom, I. (2019). SPICe Briefing: Scottish Government infrastructure
investment. [Parliament Report]. The Scottish Parliament. Available via: https://sp-bpr-
en-prod-cdnep.azureedge.net/published/2019/1/15/Scottish-Government-infrastructure-
investment/SB%2019-02.pdf

Ika, L. (2009). Project success as a topic in project management journals. Project

Management Journal, 40 (4), 6-19. doi:10.1002/pmj.20137

Infrastructure Commission for Scotland (2020). Phase 1: Key findings report.

[electronic]. Available via: https://infrastructurecommission.scot/page/key-findings-
report [Retrieved January 21, 2021].

Institute for Government (2015). Arm’s-length bodies. [Annual Report]. Institute for
Government. Available via: https://www.instituteforgovernment.org.uk/albs

IPA (2016). Guidance for Departments and review teams: Assurance of Benefits
Realisation in Major Projects: Supplementary guidance. [Government Guidance].
Infrastructure and Projects Authority, and Cabinet Office. Available via:
https://www.gov.uk/government/publications/assurance-of-benefits-realisation-in-
major-projects

IPA (2017). Guide for Effective Benefits Management in Major Projects. [Government
Guidance]. Infrastructure and Projects Authority, and Cabinet Office.

Jenner, S. (2015). Why do projects ‘fail’ and more to the point what can we do about it?
The case for disciplined, ‘fast and frugal’ decision-making. PM World Journal, 45 (2),
6-19.

Jha, K. N., & Iyer, K. C. (2007). Commitment, coordination, competence and the iron
triangle. International Journal of Project Management, 25 (5), 527-540.
doi:10.1016/j.ijproman.2006.11.009

Larson, E. W, & Gray, C. F. (2018). Project Management: The Managerial Process. 7th
edition. New York: McGraw Hill Education.

Laursen, M., & Svejvig, P. (2016). Taking stock of project value creation: A structured
literature review with future directions for research and practice. International Journal
of Project Management, 34 (4), 736-747. doi:10.1016/j.ijproman.2015.06.007

Locke, K. (2001). Grounded Theory in Management Research. London: Sage.

70
Maylor, H., & Blackmon, K. (2005). Researching Business and Management. 1st
edition. Hampshire: Palgrave Macmillan.

McElroy, W. (1996). Implementing strategic change through projects. International

Journal of Project Management, 14(6), 325-329. doi:10.1016/0263-7863(95)00060-7

Meredith, J. R., Shafer, S. M., & Mantel Jr., S. J. (2018). Project Management: A
Managerial Approach. 10th edition. Hoboken: Wiley.

Minney, H., Parris, S., & APM Benefits and Value SIG (2019). A Guide to Using a
Benefits Management Framework. 1st edition. Buckinghamshire: Association for Project
Management.

Mokoena, T. S., Pretorius, J. H. C., & Van Wyngaard, C. J. (2013). Triple constraint
considerations in the management of construction projects. In: 2013 IEEE International
Conference on Industrial Engineering and Engineering Management. Bangkok,
Thailand, December 10-13. doi:10.1109/IEEM.2013.6962524

Haniff, A., & Fernie, S. (2008). Projects: Where Strategies Collide. In: CIB
Wo65/WO55 Transformation through construction. Dubai, UAE, November 15-18.

Melton, T., Iles-Smith, P., & Yates, J. (2008). Project Benefits Management: Linking
your Project to the Business. Oxford: Butterworth-Heinemann.

Morris, P., & Pinto, J. (2007). The Wiley Guide to Project, Program & Portfolio
Management. 1st edition. Hoboken: John Wiley & Sons.

Mossalam, A., & Arafa, M. (2016). The role of project manager in benefits realization
management as a project constraint/driver. HBRC Journal, 12 (3), 305-315.
doi:10.1016/j.hbrcj.2014.12.008

Nohria, N., & Beer, M. (2000). Cracking the Code of Change. Harvard Business
Review, 78 (3), 133-141.

O'Gorman, K., & MacIntosh, R. (2015). Research Methods for Business &
Management. 2nd edition. Oxford: Goodfellow Publishing.

PMI (2017). A Guide to the Project Management Body of Knowledge. 6th edition.
Newton Square: Project Management Institute.

PMI (2020). Pulse of the profession. [electronic]. Available via: https://www.pmi.org/-

/media/pmi/documents/public/pdf/learning/thought-leadership/pulse/pmi-pulse-2020-
appendix.pdf?v=f3ef13d4-2187-4818-a80e-dd8045157d97 [Retrieved February 2,
2021].

Pollack, J., Helm, J., & Adler, D. (2018). What is the Iron Triangle, and how has it
changed?. International Journal of Managing Projects in Business, 11 (2), 527-547.
doi:10.1108/IJMPB-09-2017-0107

Reay, T., Zafar, A., Monteiro, P., & Glaser, V. (2019). Presenting Findings from
Qualitative Research: “One Size Does Not Fit All!”. In: Zilber, T., Amis, J. & Mair, J,
eds. The Production of Managerial Knowledge and Organizational Theory: New

71
Approaches to Writing, Producing and Consuming Theory Research in the Sociology of
Organizations. 59th volume. Emerald Publishing. 201-216.

Regulation 2016/679. On the protection of natural persons with regard to the

processing of personal data and on the free movement of such data, and repealing
Directive 95/46/EC (General Data Protection Regulation). European Parliament,
Council of the European Union. http://data.europa.eu/eli/reg/2016/679/2016-05-04

Reinking, D., & Alvermann D. (2005). What are evaluation studies, and should they be
published in RRQ? Reading Research Quarterly, 40 (2), 142-146.
doi:10.1598/RRQ.40.2.1

Saunders, M., Lewis, P., & Thornhill, A. (2009). Research methods for business
students. 5th edition. Harlow: Pearson.

Saunders, M., Lewis, P., & Thornhill, A. (2019). Research methods for business
students. 8th edition. Harlow: Pearson.

Serra, C., & Kunc, M. (2015). Benefits Realisation Management and its influence on
project success and on the execution of business strategies. International Journal of
Project Management, 33 (1), 53-66. doi:10.1016/j.ijproman.2014.03.011

SFS 1998:204. Personuppgiftslagen. Stockholm: Justitiedepartementet (Department of

Justice).

Statista (2020). Government spending in the UK. [statistics report]. Statista.

https://www.statista.com/study/37726/uk-government-spending/ [Retrieved: 2020-12-
07].

Thorp, J. (2003). The Information Paradox. 2nd edition. Toronto: Fujitsu Consulting

Todorović, M., Dejan, P., Mihić, M., Obradović, V., & Bushuyev, S. (2015). Project
success analysis framework: A knowledge-based approach in project management.
International Journal of Project Management, 33 (4), 772-783.
doi:10.1016/j.ijproman.2014.10.009

Toor, S., & Ogunlana, S. O. (2010). Beyond the ‘iron triangle’: Stakeholder perception
of key performance indicators (KPIs) for large-scale public sector development projects.
International Journal of Project Management, 28 (3), 228-236.
doi:10.1016/j.ijproman.2009.05.005

Turner, J. (2007). Gower Handbook of Project Management. 4th edition. Hampshire:

Gower Publishing.

Umeå University (2018). Thesis Writing in Business Administration. [Thesis Manual].

Umeå: Umeå School of Business, Economics and Administration.

Van Wyngaard, C. J., Pretorius J. H. C., & Pretorius L. (2012). Theory of the triple
constraint – A conceptual review. In: International Conference on Industrial
Engineering and Engineering Management. Hong Kong, Hong Kong, December 10-13.
IEEE. doi:10.1109/IEEM.2012.6838095.

72
Ward, J., & Daniel, E. (2012). Benefits management: how to increase the business value
of your IT projects. 2nd edition. West Sussex: Wiley & Sons.

Ward, J., De Hertogh, S., & Viaene, S. (2007). Managing benefits from IS/IT
investments: An empirical investigation into current practice. In: 40th Annual Hawaii
International Conference on System Sciences. (pp. 206-215). Waikoloa, HI, USA,
January 3-6. IEEE. doi:10.1109/HICSS.2007.330

Williams, T., Vo, H., Bourne, M., Bourne, P., Cooke-Davies, T., Kirkham, R.,
Masterton, G., Quattrone, P., & Valette, J. (2020). A cross-national comparison of
public project benefits management practices – the effectiveness of benefits
management frameworks in application. Production Planning & Control, 31 (8), 644-
659. doi:10.1080/09537287.2019.1668980

Yin, R. (2014). Case Study Research: Design and Methods. 5th edition. Thousand Oaks:
Sage.

Zwikael, O., & Smyrk, J. (2012). A General Framework for Gauging the Performance
of Initiatives to Enhance Organizational Value. British Journal of Management, 23, 6-
22. doi:10.1111/j.1467-8551.2012.00823.x

Zwikael, O. (2014). Call for papers—Special Issue on “Project Benefit Management”.

International Journal of Project Management, 32 (4), 543.
doi:10.1016/j.ijproman.2014.02.001

73
Appendix 1: Benefits Map

Figure 13 Benefits mapping; adopted from: (Serra, 2017, p.96-101; Serra and Kunc, 2015, p.56; Ward and Daniel,
2012, p. 118)

74
Appendix 2: Participation Introduction Letter
Subject: Participation in Study about Benefits Management

Dear Sir/Madam,

Thank you for your interest in the study. The following is as a short introduction of us and the
research project we are developing, as well as what your participation will involve.

This research project is conducted by Ahmed AbuElmaati and Trym Bernløv under the supervision of
Dr Gaim at Umeå University. It is completed as an important part of our master’s dissertation, which
is the final module of the MSPME a joint programme between Umeå University in Sweden and
Heriot-Watt University in Scotland. In this study, we are aiming to explore benefits management
practices in the UK public sector. By benefits management, we mean the discipline of identification,
defining, planning, tracking and realisation of benefits to achieve an organisation's strategic
objectives. Benefits being all positively perceived results of a change or a project. In this thesis, we
aim to focus on the framework as a practice to monitor and control projects success.

Your participation in this study will be through a 30-45 minutes virtual (video) interview. The agenda
will be to generally discuss project management and benefits management, and your experiences
and perspectives on the topic. The discussion will be semi-structured, allowing room to
accommodate your unique expertise and experience. However, as a general pointer, we are
planning to touch upon the following topics during the meeting:

• Benefits management in public sector projects

• Benefits management and project success
• How benefits management is applied to monitor and control projects
• Benefits management challenges and possibilities

Your privacy and confidentiality are important to us throughout the completion of the study. Upon
your consent, the interview session will be recorded and transcribed. This is exclusively for analytical
processing, and the data storage and sharing will comply with both the UK and European data
protection laws. Only the researches and examiners will have access to recordings and data derived
from them. In the published thesis, all references to the interview will be anonymised by retracting
names and any potentially identifying attributes. We ensure that your contribution and identity will
be entirely confidential.

You can expect a copy of the interview transcript once it is ready. In addition, access to the complete
results and study will be provided after its publication. We are both morally and legally obliged to
fulfil any of your requests to withdraw, modify, or add to your contribution if given ample time
before publication. The expected publication date is January 2021.

We would like to personally thank you for your time and contribution, and for making this study a
reality. Your help to complete this study is much appreciated.

Yours sincerely,

Ahmed AbuElmaati, Trym Bernløv,

ahmedmagdi@aucegypt.edu tb66@hw.ac.uk

75
Appendix 3: Interview Protocols
Version 1:
Introduction questions:
The purpose of these questions is to understand more about the interviewee's
experiences, education, and expertise. This part of the interview will be informal. It is a
personal and professional introduction.
- Introduce one another to ensure the name is on the record
- What is your formal job title and role?
- What is your formal education and training, and do you have any other
professional qualifications? Qualifications (APM / PMI / PRINCE2 / …others)?
- Formal Benefits Management training?
- Please, describe the typical project that you are involved in?
OR:
- Can you tell us more about your most recent projects?
- Who are your typical stakeholders? How diverse would you describe the
stakeholders you typically manage? (clients, SRO, …etc.,)
Project success
- How is project success defined in your organisation?
- Do you personally agree with the organisation's definition?
- Can a project be delayed or over-time and still successful? How?
Iron triangle
- Do you think a project on time, within budget, and with quality to specification
qualifies as a successful project?
- Can a project be over time and out of budget and still be considered successful?
General benefits management awareness
- When we mention the term "benefits management", what comes to mind?
What does that term mean from your perspective and experience?
Follow up:
o How do you define benefits?
o How do you define benefits management?
Ask for a contextual example
- What are the specific benefits you try to achieve through your projects?
Follow up:
- Do you believe benefits management is something that project managers have
to concern themselves with or is it a tool for the strategic level?
- Who, from your perspective, is responsible for applying the benefits
management process?
- Can you elaborate on each party responsibility in the process?

Benefits management utilisation in the organisation

- Does your organisation use benefits management? If so, why?
- How do you evaluate the application and effectiveness of benefits management
in your organisation?

- What procedures and guidelines govern the process in your organisation?

76
Follow up:
- Do you refer to a specific body of knowledge/governmental guidelines? - (in-
house own guidelines / treasury Green book/ IPA / APM/PMI)

Project Success and Benefits management

- Is "benefits management" currently used in your gateway reviews?

- How BRM is a part of the process, then?
If NO:
Why not? Do you think it will be useful if they are?
How to achieve that? What are the obstacles/challenges?
If YES:
Is it a sound practice?
How to improve/encourage it?
Does it ensure more project success?
- What do you think is the relation between benefits management and project
success?
- Do you think it is a good practice to measure the benefits realisation after the
project delivery?

Optional Questions (if time / need allows)

- How do you think the benefits management process can be improved in your
organisation (or in your clients') to ensure a higher rate of project success?
- Do you believe the frame-work can contribute to better fulfilment of strategic
objectives and policies?

77
Version 2:

Interview Protocol
General Guidelines to consider:
• Priority is for section 1 and 2. Important and must ask questions (point to touch
upon) are bolded
• The interviews are to be conducted through “Zoom”, a video-conferencing
platform
• Cameras of the researches to be on all time to encourage participants to do the
same
• After an informal introduction and greeting, the interview is to be recorded;
before recording, the interviewee shall be asked for consent
• It is advisable to reconfirm the consent and name on record
Section 1 | Introduction and Background
Researchers to introduced themselves, the research project and confirm the
participants consent to record.
It is essential to introduce the benefits management and terms related to that are
going to be used to form a common ground of understanding.

Introduction questions:
The purpose of these questions is to understand more about the interviewee's
experiences, education, and expertise. This part of the interview will be informal. It is a
personal and professional introduction.
• Introduce one another to ensure the name is on the record
• What is your formal job title and role?
• What is your formal education and training, and do you have any
recognised qualifications?
Qualifications (APM / PMI / PRINCE2 / …others)?
Formal Benefits Management training?
• Can you give us an example of one of your projects and how they are
unique? What makes it so?
OR:
Can you tell us more about your most recent projects?
• Who are your typical stakeholders? How diverse would you describe the
stakeholders you typically manage?
(clients, SRO, …etc.,)

Section 2 | Project success and benefits management

Project success
- How is project success defined in your organisation?

78
 - What do you think about that definition? What is your view about project
success definition in your organisation?
- Can you give us examples you think are success according to you but not
necessarily in the way your organisation defines success (or vis-versa)?

Iron triangle
- What are your views about the iron-triangle?
- Do you think a project on time, within budget, and with quality to specification
qualifies as a successful project?
- [Important] Can a project be delayed or over-budget and still be successful?
How? Can you give us examples?
- [Important] How the success criteria be further improved?

Project Success and Benefits management

- Can you tell us about gate-reviews? What are the typical KPI checked?
- How "benefits management" currently used in your gateway reviews?
- How BRM is a part of the process then?
- If NO:
o Why not? Do you think it will be useful if they are?
o How to achieve that? What are the obstacles/challenges?
- If YES:
o Is it a sound practice?
o How to improve/encourage it?
o Does ensure more project success?
- What do you think is the relation between benefits management and project
success?
- How do you evaluate measuring benefits realisation after the project delivery?

Section 3 |
Benefits Management in general:
General benefits management awareness

Ask for a contextual example

What are the specific benefits you try to achieve through your projects?

Follow up:
- Do you believe benefits management is something that project managers have
to concern themselves with or is it a tool for the strategic level?
- Who, from your perspective, is responsible for applying the benefits
management process?
- Can you elaborate on each party responsibility in the process?

Benefits management utilisation in the organisation

- Does your organisation use benefits management? If so, why?

79
 - How do you evaluate the application and effectiveness of benefits management
in your organisation?
- What procedures and guidelines govern the process in your organisation?
Follow up:
- Do you refer to a specific body of knowledge/governmental guidelines? - (in-
house own guidelines / treasury Green book/ IPA / APM/PMI)

Optional Questions (if time / need allows)

- How do you think the benefits management process can be improved in your
organisation (or in your clients') to ensure a higher rate of project success?
- Do you believe the frame-work can contribute to better fulfilment of strategic
objectives and policies?

80
Version 3:

Interview Protocol
General Guidelines to consider:
• Priority is for section 1 and 2. Important and must ask questions (point to touch
upon) are bolded
• The interviews are to be conducted through “Zoom”, a video-conferencing
platform
• Cameras of the researches to be on all time to encourage participants to do the
same
• After an informal introduction and greeting, the interview is to be recorded;
before recording, the interviewee shall be asked for consent
• It is advisable to reconfirm the consent and name on record

Section 1 | Introduction and Background

Researchers to introduce themselves, the research project and confirm the
participant’s consent to record.
It is essential to introduce the benefits management and terms related to that are
going to be used to form a common ground of understanding.

Introduction questions:
The purpose of these questions is to understand more about the interviewee's
experiences, education, and expertise. This part of the interview will be informal. It is a
personal and professional introduction.
• Introduce one another to ensure the name is on the record
• What is your formal job title and role? (Make this relate more to current
work, expand on LinkedIn profile)
• What is your formal education and training, and do you have any
recognised qualifications?
Qualifications (APM / PMI / PRINCE2 / …others)?
Formal Benefits Management training?
Which one do you recommend, from your experience?
• Can you give us an example of one of your projects and how they are
unique? What makes it so?
OR:
Can you tell us more about your most recent projects?

• Who are your typical stakeholders? How diverse would you describe the
stakeholders you typically manage?
(clients, SRO, …etc.,)
• What procedures and guidelines govern the process in your
organisation?
Follow up:

81
 - Do you refer to a specific body of knowledge/governmental
guidelines? - (in-house own guidelines / treasury Green book/
IPA / APM/PMI
- Where do the terms or the terminology come from in your
organisation?

Section 2 | Project success and benefits management

Project success
- How is project success defined in your organisation?
- What do you think about that definition?
- Can you give us examples you think are success according to you but not
necessarily in the way your organisation defines success (or vis-versa)?
- [Optional] How do you think the benefits management process can be improved
in your organisation (or in your clients') to ensure a higher rate of project
success?
Iron triangle
- What are your views about the iron-triangle?
- Do you think a project on time, within budget, and with quality to specification
qualifies as a successful project?
- [Important] Can a project be delayed or over-budget and still be successful?
How? Can you give us examples
- [Important] How the success criteria are further improved?
Project Success and Benefits management
- Can you tell us about gate-reviews? What are the typical KPI checked?
- How "benefits management" currently used in your gateway reviews?
- How BRM is a part of the process, then?
If NO:
o Why not? Do you think it will be useful if they are?
o How to achieve that? What are the obstacles/challenges?
If YES:
o Is it a sound practice?
o How to improve/encourage it?
o Does it ensure more project success?

- What do you think is the relation between benefits management and project
success?
- How do you evaluate measuring benefits realisation after the project delivery?
- How do you evaluate the application and effectiveness of benefits management
in your organisation?

Section 3 | Benefits Management in general

General benefits management awareness

Ask for a contextual example

What are the specific benefits you try to achieve through your projects?

82
Follow up:
Do you believe benefits management is something that project managers have to
concern themselves with or is it a tool for the strategic level?
Who, from your perspective, is responsible for applying the benefits management
process?
Can you elaborate on each party responsibility in the process?

Optional Questions (if time / need allows)

Do you believe the frame-work can contribute to better fulfilment of strategic
objectives and policies?

83
Appendix 4: Code Book
Code/Theme Description

❖ Benefits and their What are the benefits to be realised in the

categorisation organisation, their categorisation, and other related
issues?
To which guideline do public sector organisation refer
to?

▪ APM Benefits categorisation and definition according to or

with reference to the APM guidelines

• Partial - With partial reference

reference
• Theoretical - With full theoretical reference to the
Reference guidelines

▪ Green Book Benefits categorisation and definition according to or

with reference to HM Treasury’s the Green Book

▪ IPA Benefits categorisation and definition according to or

with reference to IPA guidelines

▪ Mix and Match Benefits categorisation and definition is a mix and

match between several guidelines

▪ MSP Benefits categorisation and definition according to or

with reference to MSP

▪ PRINCE2 Benefits categorisation and definition according to or

with reference to PRINCE2 guidelines

▪ QALY Quality Adjusted Life Year is used as a measure of

projects benefits

▪ UKPLC Benefits realised are benefits to the UK public or

UKPLC

❖ Biases Biases involved in BRM

▪ Illusion of
control

▪ Optimism

▪ Self-serving

84
Code/Theme Description

❖ BRM guide The application of governmental, practitioners, and/or

academic guidelines or frameworks for BRM

▪ APM

▪ Green Book

▪ IPA

▪ Mix and Match

▪ MSP

▪ PRINCE2

❖ BRM Process
Effectiveness

▪ Adequate Front- The BRM is effective at the front end of the project
end (appraisal, selection, initiation, and planning)

▪ Adequate post- The BRM is effective after the project delivery and
delivery during its integration in Business-as-Usual

▪ Challenging after BRM is generally challenging after the project

initiation initiation

▪ Inadequate BRM is inadequate and ineffective during the whole

during project lifetime of the project (including the front-end)
lifetime

❖ Definition changes The organisation changes the definition according to

situationally different project parameters

❖ Gateway review process

▪ APM The organisation(s) follow the guidelines prescribed

by the APM

▪ Does NOT lead The gateway reviews are ineffective in terminating

to termination projects that are not successful

▪ Green Book Refers to HM Treasury’s the Green Book’s definitions

and guiding for the gate review process

▪ IPA Refers to the IPA definitions and guiding for the gate
review process

85
Code/Theme Description

▪ Lack of benefits Poor utilisation of BRM in the gate reviews

focus

▪ Lacking gate- General lack of gateways or deficiency in the gateway

way Reviews reviews process

▪ Mix and Match A mix and match between different guidelines for the
gate-reviews

▪ Proprietary The organisation has its own guidelines, not just mix
guidelines and match between existing governance guides

• Floating gates The organisation utilises the concept of a floating

gateways (with variable time or milestone)

▪ PRINCE2 The organisation refers to the guides prescribe by

PRINCE2

❖ Organisational alignment The alignment in the organisation between the

prescribed guidelines and procedures and the practice

▪ +ve alignment to the organisation is positively aligned and consistent in

definition of its definition of success
BRM

▪ +ve alignment to the organisation is positively aligned and consistent in

definition of its definition of success
success

▪ -ve alignment to the organisation is negatively aligned (misaligned)

definition of and/or inconsistent in its definition of BRM
BRM

▪ -ve alignment to the organisation negatively aligned (misaligned) and

definition of inconsistent in its definition of success
success

❖ PM Guide Refers to issues related to the project management

body of knowledge, guidelines, or methodology in
use;

▪ Agile Generally, projects utilise an agile methodology

▪ APM For PM, organisation(s) refer to methodologies and

terminologies from the APM body of knowledge

86
Code/Theme Description

▪ Mix and Match The organisation mix and match from multiple guides,
bodies of knowledge and methodologies

▪ MSP Organisation(s) refer to MSP for project management

guidance and terminology

▪ PRINCE2 Organisation(s) refer to PRINCE2 for project

management guidance and terminology

❖ Project Termination If the projects are identified as not able to realise

benefits planned or unsuccessful, are they terminated?
How effective are organisations in this process?

▪ Obstacles Obstacles in front of on-going project termination

• Fear of
accountability

• Government
finance
structure

• Political
influence

• Press or
public
opinion
influence

• Self-serving
Bias

• Senior
sponsors or
stakeholders
influence

• Sunk cost

▪ Termination is The termination of the on-going projects is generally

Challenging challenging

▪ Termination is It is rare to terminate un-successful projects

rare

87
Code/Theme Description

❖ Success Factors and Date (statements) regarding the definition of success

Measurement

▪ APM Organisation refers to the definitions of APM

▪ Clear Definition Organisations have a clear and consistent definition of

success

▪ Extended Extended measure definition of success [iron triangle

Measure + benefits + risk + other];

▪ Greenbook Refers to HM Treasury’s 'the Green Book' definition

and measure of success

▪ IPA Refers to IPA definition and measuring of success

▪ Iron Triangle The use of the Iron Triangle (Time, budget and
Dominates quality) dominates the definition of success over the
use of benefits realised

▪ Mix and Match Organisation mix and match between several

guidelines or metrics for defining success according to
their projects’ circumstances

▪ MSP Refers to MSP definitions and measures of success

▪ Partial Iron The use of the Iron Triangle (Time, budget and
Triangle quality), however, is flawed or incomplete,
emphasises one or two aspects over the others.
Can be with or without consideration of benefits
realisation

▪ PRINCE2 Refers to HM Treasury’s the Green Book’s definition

and measure of success

▪ Benefits The realisation of the planned benefits is the measure

Realisation of the success of projects (with or without the
conjunction of other measures)

▪ Unclear or Organisation(s) definition and measure of success are

contradicting unclear or contradicting with practice
definition

88
Appendix 5: Extended Gioia Presentation and Codes Map
Table 11 Extended presentation inspired by Gioia method

Aggerate Dimensions Second-Order Themes First Order Codes

Project management and benefits management environment and governance
PM Guides
• Mixing and Matching
• PRINCE2
• MSP
• APM
• Others
BRM Guides
• PRINCE2
• HM Treasury’s the Green Book
• IPA Guide
Projects Governance
• Organisation Structure
• Organisation Culture
Success: its definition, measures, and factors
Project Management Performance
• Iron Triangle Dominates
Extended Measure of Success
• Iron Star
• Benefits Supersedes the Iron Triangle
The Gap Between Theory and Practice

89
Aggerate Dimensions Second-Order Themes First Order Codes
• Unclear or Contradicting Definitions
• Difficult to Adhere to Organisational Definition in Practice
• Definition Changes Situationally
BRM process and effectiveness
Benefits Definition and Guidance
• Formal BRM Guides
• Mixing and Matching Between Different Guidelines
• QALY is Used to Measure Benefits
• Benefits Realised for the UK public and UKPLC
• BRM by BAU, Not Project Team
Biases Revolving the BRM Process
• Illusion of Control
• Optimism
• Self-serving
BRM Process Effectiveness with Regard to Project Lifecycle
• Adequate Front-end
• Inadequate During the Project Lifetime
• Inadequate Post-delivery
Suggested Improvement to BRM
Organisational alignment
• Positive Alignment to the Definition of BRM
• Negative Alignment to the Definition of BRM
Solution Before Problem
Planning Benefits is Key to Ensure Success
BRM during project lifetime
Gateway Reviews as an Indicator

90
Aggerate Dimensions Second-Order Themes First Order Codes
• Gateway Review Process
• Lead to HOLD
• Does NOT lead to termination
• IPA
• Lack of Benefits Focus
• Lacking Gateway Reviews
• Mix and Match
• Organisation has its Own Guidelines or Modifications
Termination as an Indicator for the Effectiveness of BRM
• Better Hold than Continue or Terminate
• Termination is Challenging and Rare
Obstacles
• Fear of Accountability
• Governance - Top Management
• Government Finance Structure
• Political Influence
• Press or Public Opinion Influence
• Self-serving Bias
• Senior Sponsors or Stakeholders’ Influence
• Sunk Cost

91
Figure 14 First and second order codes map (extracted for Nivio12 project)

92
Business Administration SE-901 87 Umeå www.usbe.umu.se