Received: 6 May 2015 Revised: 29 June 2016 Accepted: 19 September 2016

DOI: 10.1002/smr.1834

SPECIAL ISSUE PAPER

An empirical study of portfolio management and Kanban in

agile and lean software companies
Muhammad Ovais Ahmad | Lucy Ellen Lwakatare | Pasi Kuvaja | Markku Oivo |

Jouni Markkula

M3S Research Unit, Faculty of Information

Technology and Electrical Engineering, Abstract
University of Oulu, Oulu, Finland Traditional portfolio management tools and methods are challenging for software companies that
Correspondence use agile and lean approaches, because of the complex pre‐planning activities required. In this
Muhammad Ovais Ahmad, University of Oulu, paper, traditional portfolio management tools and methods that have been conventionally used
Oulu, Finland.
Email: [email protected]
in industry and presented in literature are reviewed. Findings concerning their use in agile and
lean software companies are investigated, on the basis of the literature review and semi‐
structured interviews conducted in 7 agile and lean software companies in Finland. The findings
Funding information
DIMECC Need for Speed program, Tekes, indicate that the studied companies do not use traditional portfolio management tools and
Nokia foundation and Ella and Georg methods in their entirety. The software companies adapted suitable components from traditional
Ehrnrooth foundation grant.
portfolio management tools and methods into their practices. They also apply new tools and
methods to manage their portfolios of offerings, to cope with the dynamics required by agile
and lean methods. The companies' emphasis is on using tools to facilitate immediate feedback
from customers, to assess the value of offerings in their portfolio. The results showed a
growing interest in using Kanban at the portfolio level to provide a visual overview of
offerings.

KEY W ORDS

agile, Kanban, lean, portfolio, software product management

1 | I N T RO D U CT I O N companies must incorporate frequently changing customer require-

ments during product and service development, without compromising
Portfolio management has been in existence for several decades since on quality. However, the adoption of agile and lean approaches in the
its origins from portfolio theory in finance in 1950s.1 Companies use software industry has brought about challenges in the management of
portfolio management to ensure effective use of limited resources products and services in companies' portfolios.4–6 Kalliney4 reports on
and to achieve balance of resources in terms of value maximization, the lack of awareness of the impacts of agile and lean approaches to
strategic alignment, risk level, and number of ongoing products and portfolio management when a company begins to adopt them. Over
projects. According to Cooper et al,2 portfolio management “is a time, as agile and lean approaches become assimilated in the company,
dynamic decision making process whereby a business's list of active “portfolio problems,” such as difficulties in implementing company's
new products and R&D projects is constantly up‐dated and revised.” vision and managing cross‐team risks and dependencies, become more
The process involves the evaluation, selection, and prioritization of visible.4 Additionally, as software companies often embark on
new projects at the same time as existing projects are reevaluated concurrent development of multiple offerings that exhibit high
for resources allocation and reallocation to active projects. In the con- interdependency across different software products,7 the process of
text of this study, portfolio management looks at a combination of new managing interdependencies and synergies becomes more complex
and existing products and services that are collectively grouped to as the number of shared software components increases and develop-
ensure their effective management, which is aligned with strategic ment projects expand. The latter according to Kalliney4 goes beyond
business objectives crucial for creating value. simply having teams and managers attend each other's sprint reviews.
Today, a growing number of software companies have adopted One possible approach, especially prominent in project‐based organi-
agile and lean approaches3 to be successful in global markets; software zations (PBOs), is to use project portfolio management for selecting,

J Softw Evol Proc. 2017;29:e1834. wileyonlinelibrary.com/journal/smr Copyright © 2016 John Wiley & Sons, Ltd. 1 of 16
https://doi.org/10.1002/smr.1834
2 of 16 AHMAD ET AL.

prioritizing, and evaluating the development of multiple offerings that proposed frameworks, techniques, and methods4,5,14,15 for portfolio
takes into consideration synergies and interdependencies among pro- management lack empirical evidence for their use.7,12,13,19,20
8
jects. Successful implementation of portfolio management in a com- The aim of this study is to gain insights of what traditional portfo-
pany helps in the alignment of initiatives with the company strategy. lio management tools, and methods are applicable in portfolio manage-
Further, portfolio management allows the use of reliable metrics to ment of agile and lean software companies. The study addresses the
effectively interpret market demands and customer requests, rather following research questions through review of traditional portfolio
than managers merely using ad hoc feedback mechanisms to get input management literature and interviews with managerial level practi-
from sales and customer representatives.4 Using portfolio manage- tioners from 7 software companies.
ment can therefore make companies more alert to the offerings in their
company portfolio. • RQ1. What are the main traditional portfolio management
The tools and methods used in portfolio management have been methods and tools reported in the literature?
well established in the literature on R&D management,9 information • RQ2. What tools and methods are used to manage portfolios in
systems (IS),1 and project management. However, evidence of their lean and agile software companies?
application in the context of agile and lean software development
• RQ3. How does portfolio management in agile and lean software
literature is scarce.5,10–12 In the last decade, there has been growing
companies differ from traditional portfolio management?
interest in portfolio management in agile and lean software develop-
• RQ4. What is the role and potential for the use of Kanban in port-
ment.3,4,7 Adopting traditional portfolio management tools and
folio management for agile and lean software companies?
methods in agile and lean software companies poses challenges
4,6
because of the dynamic nature of development methods. The latter
The paper is structured as follows: Section 2 provides an overview
leads to a situation where 2 different approaches that are contradicting
of the related work on portfolio management in agile and lean software
are used in combination (ie, agile and lean approaches are used in com-
bination with traditional portfolio management). Traditional portfolio companies. Section 3 describes the research methodology. Section 4

management does not take into consideration the operability of agile presents the results from the literature review. Section 5 presents
results from case companies. Sections 6 and 7 discuss the comparison
and lean operations because it often involves a centralized decision‐
making process with detailed annual plans giving an impression of of portfolio management tools and methods in literature and company

waterfall milestones. On the other hand, the nontraditional portfolio practice along with Kanban use in portfolio management. Section 8 dis-
cusses validity and reliability of the study, and finally Section 9
management that is used in agile and lean software companies tends
presents the discussion, conclusion, and directions for future work.
to focus on decentralized decision making with light weighted planning
of product and service development activities.13 Various approaches
have been proposed to help scale agile methods for portfolio manage-
ment.4,5,13–15 However, despite these various approaches, software 2 | P O R T FO L I O M A N A G E M E N T I N A G I L E
companies are still facing various problems in portfolio management, A N D L E A N SO F T W A R E C O M P A N I E S
such as lack of visibility for offerings about to enter the development
pipeline, and their effective prioritization because some of the offer- Software companies are often described as PBOs, because they use
ings maybe induced “under the hood” by some managers.5 To address projects as a means to develop products and services.8,21 The shift to
13
some of these problems, Leffingwell points out that by using Kanban a more strategic perspective in choosing between projects and tackling
for portfolio management, companies can make their business initia- the challenge between “doing projects right” and “doing the right pro-
tives and offerings significantly more visible to relevant stakeholders. jects” has led to the implementation of portfolio management in many
However, there is a lack of empirical evaluation of this suggestion PBOs.8,22,23 The literature on portfolio management contains a signif-
and more importantly studies that evaluate combinatory existence of icant contribution from research in R&D management, IS, and project
traditional portfolio management tools and methods and agile and lean management.1,9,24 In IS, portfolio theory was introduced by McFarlan
approaches in software companies. in 1981, where he proposed that management use a risk‐based
According to leading agile and lean experts, such as Anderson and approach to selection and management of IT project portfolios.1 The
Arne,16 Leffingwell,13 and Shalloway,17 Kanban holds the promise of focus of portfolio management in IS has mostly been on operational
being one of the easiest, most visual portfolio management tools. investments rather than on products and services developed for cus-
Kanban entered into software development in 2004, when David tomers.1 Numerous research contributions have been made to portfo-
Anderson introduced it in practice while assisting a software develop- lio management in the area of R&D management and product
ment team at Microsoft.18 Kanban in Japanese means “visual card.” In management, specifically in new product development,2,9,25–27 but
software development, a Kanban board shows the current status of all very often these studies have not paid specific attention to the nature
tasks. Kanban has 5 core principles: (1) visualize workflow; (2) limit of the products and services making up the portfolios, such as soft-
work in progress (WIP); (3) measure and manage flow; (4) make process ware. The lack of such specific focus on software products led to
policies explicit; and (5) use models to recognize improvement and findings such as inapplicability of the proposed portfolio management
18
opportunities. The biggest benefit of using Kanban is visualization, tools and methods in software projects.24 On the other hand, over
which provides a concise overview of all of the company's offerings, the past few years, there has been an increasing interest and research
along with information about each one. However, the existing conducted on portfolio management specific to software products and
AHMAD ET AL. 3 of 16

services, because of the unique characteristics of software products He divides portfolio management into project, resource, and asset
such as ease of making changes after release.4,5,7,11 In software prod- portfolio management.33 Krebs suggests to use dashboard for
uct management, portfolio management is identified as an important assessing the overall situation of all projects, while for individual pro-
component of product management, along with others such as product jects quality and team morale are the key metrics. Krebs also presents
road‐mapping, requirements management, and release planning.28,29 project portfolio management challenges, including too many active
Portfolio management involves a dynamic decision‐making pro- projects and an incorrect mix of projects in the portfolio.33
cess for new and existing products, as well as lifecycle management With respect to the 4 main goals of portfolio management identi-
of products on the basis of market trends, product strategies, and so fied by Cooper et al,30 agile and lean software companies maximize
on.2 The process is significant for ensuring that the value of invest- their portfolio through continuous formulation, estimation, and priori-
ments is maximized, limited resources are distributed effectively, and tization of items in the portfolio backlog.13 Agile and lean approaches
that investments are aligned with overall corporate objectives and emphasize the ability to add or remove items even at the last minute
goals.2 To maximize the value of investments under conditions of high and provide the possibility of evaluating the business potential of an
risk, the portfolio concept advocates selecting and constructing a port- offering on a smaller scale.4 According to Bhattacharya et al,34 in a
folio of items (such as products) that collectively yield high expected highly dynamic business environment, product definition is resolved
returns at low risk.1 Several research contributions describe how port- through “frequent, repeated interactions with customers, and [by]
folio management process ought to be implemented in practice. The using a flexible development process.” Agile methods have often
2,25,26,30
most well known of these include the work of Cooper et al addressed the latter. To ensure company vision and strategic alignment
and the guidelines provided by the Project Management Institute.31 in portfolio management, software companies that are using agile
Successful implementation of portfolio management means achieving approaches tend to emphasize the business value of software features.
the following 4 main goals2,25,26,30: (1) maximizing the financial value For instance, Ultimate Software12 decided to use a standard numeric
of the portfolio, (2) linking the portfolio to strategy, (3) balancing the measurement where members assign values and rankings to each high
portfolio on relevant dimensions, and (4) ensuring that the total num- feature, on the basis of specific criteria such as the feature's impor-
ber of ongoing offerings is feasible. To achieve these goals and allow tance to the market.12 Similarly, the technologies division of the New
for rational decisions, various methods and tools are used to provide York Stock Exchange11 used a portfolio management technique to
decision makers with information regarding the optimal set of alterna- assess each feature in their backlog regarding its relative business
tives to select from, which is complemented by tacit knowledge for value to customers, using a high‐medium‐low scale. A relative cost
assessment of other contextual factors that cannot be computed.1 for each feature was also calculated.
Adopting traditional portfolio management is challenging for agile Leffingwell13 describes, in his scaled agile framework, several
and lean software companies.4–6,12 In contrast with the iterative practices to implement agile methods at enterprise scale. He divides
nature of agile methods, traditional portfolio management constitutes his framework into portfolio, program, and team levels. At the portfolio
a linear process, which also views development as a separate phase level, the company's executives maintain the portfolio vision, allocate
to identifying, prioritizing, allocating, balancing, and reviewing the resources to value streams through investment themes, and define
5
offerings in a portfolio. In that sense, the frequent reevaluation of and prioritize their portfolio backlog, the highest‐level mechanism
offerings during development challenges the traditional linear process and artifact holding business, and technology development initiatives.
of portfolio management practice.20 Moreover, agile and lean According to Abrantes and Figueiredo,7 it is important to enable a cen-
approaches help to tackle uncertainty in software development tralized view, giving visibility to all ongoing and planned work given
through incremental delivery of product features, such as delivering a limited resources. Such visibility makes it possible to ensure that com-
minimum viable product to customers. panies do not underuse or overuse their resources. Lean approaches
To tackle tensions between traditional portfolio management and such as Kanban have enabled companies such as YLE to bring visibility
agile and lean approaches, scholars12,13,20 have proposed agile portfo- and awareness to all ongoing development activities, facilitating ease
lio management frameworks. According to Anderson,16 empirical eval- of management of project dependencies and pipeline.10 According to
uation of agile methods in portfolios and the implementation of the Leffingwell,13 Kanban is very effective for such purposes. Kanban
32
proposed frameworks are scattered. Hodgkins and Hohmann report helps in giving visibility to the work process, limiting WIP, communicat-
that Scrum backlogs were insufficient for addressing business needs ing priorities, and highlighting bottlenecks.18,35–37 The motivation
and introduced roadmaps as a filter to aid backlog prioritization and behind visualization and setting limitations on WIP is to identify the
to communicate strategic intent and business opportunities between constraints of the process and to focus on one item at a time. This
product managers and the technical team. Vähäniitty12 discusses agile results in a constant flow of work items being released to cus-
product and portfolio management in the context of small software tomers.35–38 Multiple “swim lanes” on the Kanban board help a com-
organizations and proposes a framework that shows 3 key processes pany to visually present, organize, and track down offerings in its
that should connect business and development decision making such portfolio, along with a visually depicted value stream for all offerings.
as product road‐mapping, release planning, and different levels of port- According to Anderson and Roock,16 Kanban is particularly beneficial
12
folio management. This framework offers transparency for business for large companies because changes are made in small increments,
priorities while enabling the just‐in‐time elaboration required by agile and because of its similarities with lean approaches. In a nutshell,
software development.12 On the basis of agile principles, Krebs33 pro- Kanban aids portfolio management, brings visibility to the entire port-
poses a dynamically managed portfolio with flexible financial models. folio of offerings, speeds up implementation, and helps decision
4 of 16 AHMAD ET AL.

making while handling multiple offerings with a given set of formulation for each database, while snowballing does not explicitly
resources.16,17,39 Furthermore, Leffingwell13 describes the following require searching in more than one database. They add that
5 Portfolio Kanban states that a new offering passes through before snowballing is more understandable and easy to follow, particularly
it can be implemented or rejected: for novice researchers, whereas systematic literature review provides
more guidelines, which novice researchers might find confusing rather
1. The capture state, where all new ideas are welcome. than helpful.41

2. The review state, where preliminary estimates of opportunity, Webster and Watson23 recommended using both backward
effort, and cost of delay traditionally take place. snowballing (working from the reference lists) and forward snowballing
(finding citations to the papers). In a study conducted by Jalali and
3. The analysis state, where more thorough work is done to establish
Wohlin,41 on comparing the results of database search reviews with
viability, measurable benefit, development and deployment
backward snowballing reviews, the findings were that the actual
impact, and potential availability of resources. In this stage, a light-
results are not highly dependent on the search strategy. In this study,
weight business case is developed to gain approval for the
after the first literature search, only backward snowballing used. In
offering.
each process, the first 2 researchers reviewed and selected papers
4. The portfolio backlog state, where the offerings have made it
together. Figure 1 shows our complete process of literature review.
through the Portfolio Kanban with “go” approval, and they await The first step of our search involved finding relevant papers in
the “pull” from the implementation units or teams.
Google Scholar using a selected general keywords (“portfolio manage-
5. The implementation state, where the approved offerings are allo- ment,” “product,” and “service”), published from 2000 to 2014.
cated to the available teams with the resources and skills neces- According to Wohlin,40 a good starting point may be identified by
sary for implementation. Offerings are subsequently tracked using the Google Scholar search engine, which helps to avoid any bias
using various portfolio metrics, such as the Portfolio Kanban toward researchers or any specific publisher. The generic and broad
board, Net Promoter Score, team‐ and program‐level self‐ keywords were selected on the basis of key terms that relate to our
assessment, employee survey, feature cycle time, number of research as expressed in the research questions. The key term “portfo-
releases per year, defect data, and support call volume, partner, lio management” was selected on the basis of our understanding of the
and vendor surveys. topic from most popular cited portfolio management studies to limit
studies to only those that focus on portfolio rather than projects or
programs, thus giving a narrower search of studies that fall under our
topic of interest. The use of “product” and “service” keywords were
3 | METHODOLOGY
aimed to exclude studies that focus on other types of portfolio assets
such as financial bonds without necessarily limiting our search to the
The study was performed in 2 phases. In the first phase, traditional
nature of the products and services such as software. While we do
portfolio management tools and methods were investigated in the lit-
not claim that this would result to the inclusion of all relevant studies,
erature using snowballing technique. In the second phase, interviews
the researcher saw that it is reasonable to use this combination would
were conducted with experts from agile and lean software companies.
give important studies that are also representative of traditional port-
The reasons for choosing these methods are explained in the subsec-
folio management studies. Moreover, as specified by Jalali and
tions below. Finally, we compared tools and methods presented in
Wohlin,41 the efficiency of snowballing might be higher when the key-
the traditional portfolio management literature with those used in
words for searching include general terms. From Google Scholar search
selected agile and lean software companies, according to 4 goals of
results, we selected the first 200 search results from which 19 papers
portfolio management.
were identified as relevant because their inclusion focused on those
that reported portfolio management of products and services. We then
3.1 | Literature review proceeded to call these relevant papers as “start set papers.” The start
A literature review was conducted first to identify portfolio manage- set papers spanned from several different publishers, authors and
ment tools and methods using a snowballing technique.40 across the years.
Snowballing—using the reference list of a paper and the citations to In the second step, we started backward snowballing by going
the paper to identify additional papers—was found to be a suitable sys- through the reference lists of our “start set papers.” For each paper
tematic approach for this exploratory research. According to Wohlin,40 of the start set papers, we read the publication title from the reference
using snowballing as a first search strategy may be a good alternative list and included relevant studies. This process resulted into the selec-
to the use of database searches. The motivation for choosing a tion of 81 papers. Two researchers read all titles and abstracts and
23
snowballing technique for the literature review was to collect all nec- checked the inclusion and exclusion criteria for each entry from the
essary relevant literature without performing a full, systematic litera- first and second step of the review. Finally, we identified 24 primary
ture review, as the topic of “portfolio management” is very broad and studies out of 281. We analyzed and classified the selected studies
established in various disciplines. Snowballing offers a good coverage according to type, tools, and methods used for portfolio management.
of the investigating area or topic, and it is less influenced by the The identified portfolio management tools and methods were then
amount of noise from digital libraries searches.41 Further, according categorized according to the 4 portfolio management goals identified
to Jalali and Wohlin,41 systematic literature reviews require separate by Cooper et al.30
AHMAD ET AL. 5 of 16

FIGURE 1 Stages of the study selection process

3.2 | Interviews We interviewed a total of 8 candidates (henceforth referred as key

informants) that were proposed by contacted company representa-
In the second phase of the study investigated the tools and methods tives from the N4S consortium. The key informants were higher man-
used by agile and lean software companies to manage their portfolios agement senior people, actively taking part in portfolio management
by conducting face‐to‐face semi‐structured interviews with key infor- activities in their companies. One key informant from each company
mants from the selected companies. In our previous studies3,35,36,38 was invited to take part in the study; all agreed to participate. In total,
these selected companies were identified as blending agile and lean there were 7 interviews with 1 interviewee from each company,
approaches (such as Scrumban) in various units of their respective except for 1 company that had 2 key informants. The 8 interviews
companies. This allowed us to collect rich data while keeping the flex- resulted in a total of 582 minutes of recorded material and 50 pages
ibility necessary for an exploratory study. According to Runeson and of interview transcripts. Although this sample cannot be representative
Höst,42 semi‐structured interviews allow for improvisation and explo- of the entire population of software companies, the interviews yielded
ration of the studied objects. Prior to contacting the key informants, rich data about portfolio management from 7 companies. Using the
we selected companies on the basis of convenience sampling. How- key informant technique assures that the interviewees are experts
ever, the companies also needed to be using agile and lean approaches. who participate in portfolio management activities within their
As a result, 7 software companies from the N4S Program were respected companies. Detailed descriptions of the interview partici-
selected for the study, as they were actively using agile and lean pants and their roles are shown in Table 1.
approaches.3,35,36,38 The N4S program was initiated in 2014, According to Runeson and Höst,42 researchers should formulate a
involving 14 large industrial organizations, 18 small and medium size research question that includes potentially important variables; how-
enterprises (SMEs), and 10 research institutes and universities. ever, they should avoid linking variables and theories as much as pos-
The key informant technique43 was adopted in selecting company sible. On the basis of the literature we had researched,4,5,7,12,20,44 we
representatives for the interviews, as it allowed us to focus the inter- created a protocol for semi‐structured interviews which covered the
views on highly knowledgeable individuals. The key informant 4 research themes: (1) tools and methods for portfolio management
technique is meant to identify key experts from whom rich and high‐ in general; (2) tools and methods for ensuring balance in the portfolio;
quality data may be acquired. The large multinational companies in (3) tools and methods for ensuring that the portfolio reflects business
N4S program share key characteristics, in that they are transforming strategy; and (4) tools and methods for ensuring a proper number of
their way of working on the basis of agile and lean approaches, and projects against organizational capacity. In addition, for each theme,
in the respondents' roles and work experience. We first contacted the role and potential use of Kanban was also explored, as software
company representatives from the N4S consortium and discussed companies are increasingly using Kanban for various organizational
the goals and objectives of the study. Thereafter, the contacted com- levels and teams.
pany representative identified and suggested suitable candidates to The interview guide was piloted with 1 expert from the software
be interviewed from their respective companies on the basis of the industry and 3 researchers. At least 2 researchers participated in all
person's work experience, knowledge of agile and lean approaches, interview sessions. At the start of each interview session, the objec-
and their participation in portfolio management related activities. tives of the study and definitions of key terminologies were presented
6 of 16 AHMAD ET AL.

TABLE 1 Companies and roles of interviewed participants

No. of Employees in the
Company ID Primary Domain Company Roles of Interviewees (Years in Company)

A Computer security >900 Director of external R&D collaboration (8 y.)

Senior manager for quality of operations (4 y.)
B Software and systems (specializing >1000 Head of quality and environment, and strategy process
in embedded systems) facilitator (12 y)
C IT services and consultancy >75 CEO and co‐founder (13 y)
D Telecom network >45 000 Business improvement manager (>20 y)
E IT and product engineering services >14 000 Senior solutions consultant (>9 y)
F Telecom network >25 000 Internal coach for agile and lean organizational
transformation (>20 y)
G Services and consultancy >250 Agile, lean and Kanban consultant coach (2.5 y)

to the interviewees. All interviews were audio‐recorded and notes TABLE 2 Primary studies and publication forums
were also taken. The audio recordings were transcribed and sent back
Research
to all interviewees to review and validate their responses. Publication Forum Publication Strategy
Interview transcripts were analyzed using template analysis International Journal of Quality and [S9] Case study
developed by King45 with Nvivo, a qualitative data analysis software. Reliability Management [S10] Survey
[S17] Case study
Template analysis is a systematic technique for categorizing qualitative [E4] Case study
data thematically. Template analysis was chosen because it allows a Research Technology Management [S4] Case study
priori themes to be used to help develop an initial version of the coding [S5] Case study
[S6] Case study
template. An initial interview template was constructed using the 4
[S7] Survey
previously mentioned portfolio management goals. Interview data
Journal of Product Innovation [S1] Survey
were then mapped onto the initial template, modifying it further until Management [E3] Case study
all relevant data were coded. The process was concluded by applying Management Science [S2] Literature review
the final version of the template to the data as a whole, and a priori [E1] Literature review

themes were redefined or discarded if they did not prove helpful in IEEE Transaction on Engineering [S8] Case study
Management
capturing key details from the data.
Portland International Center for [S11] Literature
Management Review
of Engineering and Technology
4 | R E S U L T S F RO M TH E LI T E R A T U R E Proceedings

REVIEW Australasian Conference on [S12] Literature review

Information Systems
International Journal of Project [S13] Literature review
In the literature review 24 relevant papers were identified. The papers Management
discussed portfolio management methods and tools. Nineteen primary Technological Forecasting and [S15] Case study
studies (S1‐S19) were selected from the Google Scholar search and 5 Social Change
(E1‐E5) from backward snowballing (see Appendix A). The topic of International Journal of [S16] Literature review
Management Reviews
portfolio management was found in a wide variety of journals and con-
Computers and Industrial [S18] Survey
ferences, as seen in Table 2 below. These studies discuss various
Engineering
methods and tools for managing portfolios of offerings.
Technovation, Elsevier Science [S14] Case study
Portfolio management practices have a strong base in R&D
Journal of Marketing Research [E5] Theoretical
management and in managing (PBOs) [S10]. From this perspective, modeling
our research focuses on the tools and methods used to evaluate and Decision Support Systems [S19] Case study
manage new and existing products and services, including R&D pro- Operation Management [E2] Theoretical
jects. The results from the literature review showed that a variety of framework

portfolio management tools and methods have been proposed in tradi- R&D Management [S3] Multiple‐case
study
tional portfolio management literature for selection of offerings and
resource allocation to new and existing offerings.
To answer RQ1, the results of the literature analysis are synthe-
4.1 | Maximizing the value of the portfolio
sized according to 4 main goals of portfolio management: to maximize
the value of the portfolio; to ensure a balance among portfolio initia- Maximizing the value of the portfolio means allocating resources to
tives; to achieve strategic alignment of the portfolio; and to pick the offerings according to business objectives, to increase the expected
right number of projects. Table 3 presents a summary of the identified, return on investment. Such business objectives could be profitability,
selected, and analyzed studies. strategy, and acceptable risk. To evaluate portfolio for value
AHMAD ET AL. 7 of 16

TABLE 3 Traditional portfolio management tools and methods in literature

Main Goal of Portfolio
Management Traditional Portfolio Management Methods Literature

Maximize the value Financial models Real option analysis (ROA), payback period (PBP), [S19] [S1‐S8] [E1‐E4]
of the portfolio productivity index (PI), decision tree analysis, [S10‐S13] [S15‐S17]
net present value (NPV), return on investment
(ROI), internal rate of return (IRR), discounted
cash flow (DCF), options pricing theory (OPT),
earned value analysis (EVA), and expected
commercial value (ECV)
Multi‐criteria decision Fuzzy logic, delphi model, market potential, scoring [S1] [S3‐S8] [E2‐E5]
models models and checklists, and analytical hierarchy [S10‐S13] [S15‐ S19]
process (AHP)
Ensure balance among Mapping and matrix Bubble diagram, portfolio matrix, McKinsey matrix, [S1] [E3] [S3] [S4] [S6‐S14]
portfolio offerings approaches and Boston Consulting Group matrix [S18] [S19]
Achieve strategic alignment Strategic approaches Strategic buckets and maps, and product‐technology [E4] [S1‐S7] [S9‐S11]
of the portfolio roadmaps, Balanced scorecard [S15] [S19]
Pick the right number Capacity analysis Pipeline management, resource capacity analysis, [S1‐S7] [S11]
of offerings and rank ordering projects

maximization, companies need to identify select right projects using risk versus low risk), and types of markets. Risk level assessment is
financial models and maintain and make sure the intended benefits will important when considering portfolio balance to avoid over‐
be realized [S1‐S8]. For this purpose, the portfolio is constantly evalu- commitment to high‐risk projects that may jeopardize the future of
ated (minimum twice a year), and actions are taken if offerings are not the company. To illustrate balance in a portfolio, various graphical
performing at the right level or not aligned to the business strategy. techniques are used, such as portfolio maps, bubble diagrams, and
Financial models are used to quantitatively evaluate the economic pie charts [S3, S7, S10, S11, E3]. These tools provide a graphical repre-
return on new project ideas, while taking into consideration the sentation of the current situation of the portfolio and the distribution
income flow and time dependency of an investment. Financial models of resources. Portfolio maps plot proposed offerings in 2 axes or other
such as ROI, NPV, PI, IRR, payback analysis, and Discounted Cash Flow matrices to facilitate graphical and visual display [S11, S12]. Commonly
have been popular portfolio management tools for value maximization used portfolio maps balance risk versus return and benefits to cus-
[S1‐S8, E1, E3, E5], while financial models such as options pricing have tomers versus competitive advantage [S11]; one example of a popular
no evidence of being applied in practice [S11]. Nonfinancial models portfolio map is the Boston Consulting Group matrix [S12]. These
such as analytical hierarchy process (AHP) and scoring models use graphical tools and methods help plan the distribution of resources
several criteria to decide on the inclusion of a project in a portfolio, with respect to the company's capacity. These tools also help to estab-
using both quantitative and qualitative techniques to assess individual lish an appropriate balance between the offerings in the portfolio. In a
projects and compare a number of projects with a particular dimension. balanced portfolio, the need to maximize value is high, and a balanced
These models first determine the weightings for different objectives of portfolio helps a company ensure its portfolio reflects its business
the company's business, then compute and compare the contributions strategy.
of the projects to the weighted objectives; AHP is difficult to use
because it involves a large number of comparisons, and sudden addi-
4.3 | Achieving the strategic alignment of the
tion or deletion of a project means starting again from the beginning.
This is not the case for scoring models, however, because they use a
portfolio
relatively small number of decision criteria (such as cost and probability Achieving the strategic alignment of the portfolio means that all the
of technical success) to determine the merits of a project. Checklists offerings should be consistently aligned with the company's strategy.
are similar to the criteria of scoring models, but the outcome of It is important that the company's business strategy takes into account
checklists is a “yes or no” answer to the questions, where each project both internal and external business factors, to build a strategic direc-
must achieve a certain number of “yes” answers to proceed or obtain tion focused on achieving competitive advantage, and to better deter-
funding [S3]. These financial and nonfinancial tools and methods mine and allocate funds across different types of projects [S3, E4].
should be used with care, because their analysis requires rather According to Cooper et al [S3], business strategy is the second most
detailed estimation and their uncertainty is usually high [S4, S6]. popular portfolio approach after financial methods. A number of
methods are presented in the literature to align offerings with the busi-
ness strategy. The most common approaches are “top down” and “bot-
4.2 | Ensuring a balance among portfolio offerings tom up.” In the “top down” approach, companies set aside funds for
Ensuring a balance among portfolio offerings is another important ele- different types of strategic priorities before project selection, using
ment in portfolio management, because companies typically have a methods such as strategic buckets and product roadmaps, whereas
variety of projects. The goal of the portfolio is to achieve a desired “bottom up” approach builds on the philosophy that if good decisions
balance of offerings by using a number of parameters, such as time are made for individual offerings, the portfolio will care of itself
(for example, long term versus short term), risk level (for example, high [S1‐S7, S9]. A strategic bucket equates implementation of strategy
8 of 16 AHMAD ET AL.

with allocation of resources to specific projects (or focus areas) that and F). In these companies, new offering ideas were formally presented
mirror the business strategy; it also improves the alignment of the as business plan to decision‐making bodies. In smaller companies (such
portfolio with the strategy [S11]. It is used to allocate funds in the as companies C and G) the process was less formal. The criteria used
overall budget to specific types of projects or focus areas of strategic for evaluating new ideas in decision gates became more “tight” as ideas
importance. A survey conducted by Barczak [S1] reported that the went forward in the large companies. New ideas for offerings were
strategic bucket is the most frequently used portfolio management mostly evaluated using a combination of financial models with scoring
method in companies. Other suggested methods and tools include models and checklists. The most common financial models were those
roadmaps [S1‐S7, S11, E4]. that analyzed and evaluated the costs and benefits of the new pro-
posed offerings. These methods included ROI in company A; NPV in
companies B, E, and G; EVA in companies D and G; and cash flow in
4.4 | Picking the right number of offerings
company C. However, there were some challenges with the use of
Picking the right number of offerings is essential for portfolio manage- financial models, and it was evident that all companies used them less
ment, owing to limited resources. Too many offerings competing for because of lack of data or were used because of long existing company
limited resources results in gridlock in the company's offerings pipe- practice. For example, in company A, cost‐benefit analyses were
line. To avoid such situations and to have the right number of offerings derived as rough estimates rather than actual financial metrics as
in the portfolio, various methods such as resource capacity analysis depicted in the following response:
and pipeline management may be used [S1‐S7]. Pipeline management
focuses on managing offerings by ensuring that resources are not Honestly, it's very seldom that you have actual financial

overstretched and that the development pipeline timing and flow of metrics created as like an exact number or something.

offerings are managed [S11]. The literature shows that the main bene- There is like a clear, okay, this business goal is going to

fits of portfolio management are as follows: (1) the portfolio of offer- increase our sales this much. Or for the technology

ings aligns with the company's strategy; (2) companies use resources business goal, if we do this improvement, it will result in

in a better way; (3) poorly performing or outdated offerings are termi- 15 percent cut in operational costs. So those [financial]

nated; and (4) key offerings are monitored constantly to take necessary numbers exist but exact sales numbers are sometimes

actions whenever needed. very difficult to put into this. Usually there is a kind of

In summary, to achieve the 4 goals of portfolio management, deci- rough idea of the business benefit.

sions on offerings in the portfolio are mainly on the basis of sophisti- Further, in the case of company B, financial models were used
cated quantitative modeling methods, which are derived from because of long‐term established company practice over several years:
financial and operations research. Most of these financial methods
are theoretically well‐justified, but have significant practical shortcom- We have certain business decision processes which we

ings. For example, the absence of accurate data regarding offerings, have used already some six, seven years. For product

especially new offerings, makes these methods unreliable. Interest- initiative we normally have understanding of the

ingly, financial methods are popular, and companies rely more on them customers or markets and competition. Then we have

when making decisions about an offering in the portfolio. In the litera- business management who is deciding the ownership.

ture, there is a consensus that there is no single method or tool We create a collaborated team, initiate financial

suitable for all situations [S10]. However, across the literature, there calculations, product or service concept and then assess

still remains a large gap in understanding what constitutes effective the capability for execution. In the financial calculation

portfolio decision making to achieve the above‐mentioned 4 goals. we have for example discounted value of the proposal,
for all development cost, possible markets and etc. It's
creating for us [company]that what is the point of
5 | R E S U L T S F R O M EM P I R I C A L S T U D Y [product initiative] and what is the calculation of the
payback for the investment.
Altogether, 8 interviews were conducted with 7 companies that
actively use agile and lean approaches. The 4 main goals of portfolio All companies make frequent use of checklists containing a list of

management defined by Cooper et al 30

were used as a framework to criteria that new ideas must meet. In company A, criteria such as lead

analyze the collected data from interviews. Table 4 provides an over- time, customer preferences, and industry standards were used to mea-

view of the common portfolio management tools and methods used sure and assess offerings at high levels:

by agile and lean software companies. We measure four things on a high level: lead time (how
fast things go through this flow), value throughput (how
5.1 | Maximizing the value of the portfolio much new business is generated through actualisation of
the roadmap), net promoter score (how well do
To maximize the value of the portfolio, companies used a variety of
customers like your offering) and work in progress count.
tools and methods to evaluate the business potential of all proposed
new offerings. The process of selecting and evaluating new offering Product lead time, customer preference, and competitive situation
ideas for inclusion in portfolio pass through several investment deci- were criteria evaluated by all companies. The companies preferred
sion gates particularly in large companies (such as companies A, B, D, short lead times, as this encouraged incremental delivery with lowered
AHMAD ET AL. 9 of 16

TABLE 4 Portfolio management tools and methods in companies

Company ID
Main Goal of Portfolio
Management Portfolio Management Methods A B C D E F G

Maximize the value of Cost‐benefit analysis X X X

the portfolio
DCF X
PBP X
EVA X X
NPV X X
Cash flow X
Scoring models, checklists, eg, lead time, X X X X X X X
value throughput
Product experimentation X X X
Ensure balance among Strategic buckets (key focus areas) X X X X X X
portfolio offerings
Business roadmap X X
Kanban X X X X X X X
Bubble diagram, burn down charts, and X X X
cumulative flow diagrams
Ad hoc process, matrix X
Achieve strategic alignment Strategic goals X X X X X X
of the portfolio
Product roadmap X X
Kanban X X X X X X
Key performance indicators (KPI) X X
PowerPoint and videos to communicate X
strategy
Pick the right number R&D capacity pipeline analysis X X X
of offerings Capacity or capability planning X X X X X

risk. Other practices such as product demonstrations (“demos”) were we thought might be a great market. So we actually
also increasingly seen to be important, as in the case of company B. created a service to support that market.
Demos help to quickly develop ideas into actual product features
It was therefore indicated that agile and lean software companies
before beginning product development efforts:
place more emphasis on nonfinancial models, such as lead time,
When we are starting something, a business plan passes customer satisfaction, and quality of offerings.
through business idea screening. The idea is grown up Interestingly, less mentioned financial methods are taken in
by creating a (Hackathon) concept; the purpose is to consideration in case of small services company “C”:
make a demo for the idea before it goes to the Cash flow situation is very important for us to understand
traditional business development cycle. because it shows us our profit level and how much effort
we can put on product development. For the new things
According to the interviewees, early product demonstrations, cus-
we see that what progress we have made, what we
tomer preferences, and satisfaction are crucial portfolio evaluation
have learned, the competition and most promising
criteria, both during selection and after delivery of an offering. This is
things we need to continue to develop. New features, is
because they serve as important feedback to develop and improve
it something that we want to stabilise the product, is it
the portfolio by directing software product development activities.
that we are looking for a pilot customer that would be
Companies A and D used Net Promoter Score and usability surveys,
able to test it in a real production. So if we have
respectively, to gather customer preference and satisfaction data.
something ready but we cannot find a pilot customer, it
Product experimental practices and good customer relationships help
can take one year to move on.
service companies (such as company G) to identify and determine
new services to be ramped up:

[for example] two years ago we ramped up a concept and

visual design service just because we found out that
5.2 | Ensuring a balance among portfolio offerings
customers keep asking for it and are buying it from According to the interviewees, ensuring a balanced portfolio is facili-
other people. Then there are highly experimental tated through inputs from the business strategy and roadmap. Prior
services like the [X service] for embedded software that to the selection of new ideas, resources were first distributed and
10 of 16 AHMAD ET AL.

allocated annually according to key focus areas (or business units), We have a business strategy and based on it, we do
which corresponded to the business strategy of the company. In busi- products or offerings’, and ‘we have a combination of
ness roadmaps key focus areas were observed to have great impor- some of the focus areas [strategic goals specify it]... Our
tance in companies developing embedded systems (companies A, B, offering is one of the first ones that make the strategy
and F). This is because high competition, an increased need for short real.
lead time, and operating in a business‐to‐business environment require
Along with the existing strategy, companies are constantly
that the customer be informed in advance about the features and
considering and incorporating future strategic directions into current
benefits of new product releases. This was achieved by using product
business opportunities, rather than merely doing the same things:
roadmaps that were shown to customers such as network operators.
As the interviewee from company F put it We are trying to make room for future strategic goals and
their fulfilment. At our company and I also believe this is
The competition in our business is quite tough. I believe
common for others also that it's easy to keep doing the
that each of the vendors is taking a high risk.
same thing that you have done before, and then in the
[Customers] are picking the company that shows the
end you will then have so much work to do just to keep
most attractive roadmap. The roadmap in [our
the machine running without thinking that, does it make
company] is in supported technologies. The [customer]
sense economically.
wants to understand when certain new releases are
coming out, and what are the main new functionalities To achieve strategic alignment of the portfolio, it was identified
or benefits for the [customer]. The main reason for that that companies need to effectively communicate their strategy across
is the roll‐out time for the [customer], when compared the entire company, as well as frequently reviewing the strategy, for
to easy software products. You just download [those]. example, on a quarterly or annual basis. Companies A and B reported
that they construct roadmaps from their strategic plan to visualize
To ensure balance among the portfolio offerings interviewee
what projects will be undertaken and what products will be delivered
express it as
in the near future. To achieve strategic alignment of the offerings,
we balance and visualise the stuff that we do [for example company B used Kanban as a tool for the execution of all offerings.
using] cumulative flow diagrams and Kanban. Kanban An interviewee explained it like this:
always finds the balance between demand and
Kanban, for us, is a tool for the execution [of business
capability. …. Kanban doesn't help in balancing the
strategy]. We use JIRA Kanban.
portfolio but it somehow signals that, e.g. the working
pipeline is full, and, in turn, helps to evaluate suitable In agile and lean software companies, techniques used to commu-
available options at that given time to take necessary nicate the strategy throughout the company included PowerPoint
actions. presentations, short videos, monthly meetings, magazines, and drawing
the big picture on the walls for each unit.

5.3 | Achieving strategic alignment of the portfolio

5.4 | Picking the right number of offerings
All interviewees mentioned that their development and delivery of
Software companies found it important to use tools and methods for
offerings were driven by the company's business strategy. The
estimating workload, rather than specifying a number of products
business strategy was identified as an important factor, which
and services. Tools and methods to estimate the workload for the
defines the main business areas into which offerings and invest-
development of proposed product or service features helped deter-
ment resources were streamlined. Firstly, business strategy facili-
mine their release dates, associated risks, and capacity planning. For
tates the initial allocation of resources according to prioritized
example, one interviewee said:
main business areas that realize the company's portfolio. This was
particularly evident in all companies, as an interviewee in company [There is] no limit on new initiatives; they are stored in
A elaborated: the [portfolio] backlog. We take everything that we find
to be reasonable and nice to have in the portfolio‐level
We have classified different business lines (each business
product backlog for these [existing] products. Then, we
line is responsible for their own portfolio management
get realistic about what we can do; we try to estimate
and R&D) in terms of their strategic importance. This
the effort required for new ideas. We then have a kind
means that the different business lines are given certain
of velocity for the team, and if it seems that there is a
weight in the company‐level investment. As a result
lot of stuff [to do], we prefer to get another team. So
each business line operates under their own investment
we don't want to say “No new ideas for now, because
which comes from the strategy.
our team capacity is used up for next year”. Instead, we
Secondly, new and existing offerings in the portfolio are stream- try to say “Please give us any new ideas you have”, and
lined and constantly reevaluated (minimum twice a year) on the basis then we try to negotiate for instance, it is possible to
of the company's business strategy: get another team.
AHMAD ET AL. 11 of 16

All software companies place emphasis on team effort planning, usually, scaling resources up or down is a very slow and,
which involves balancing the required effort with available resources sometimes, costly and painful process. So we try not to
during the development and delivery of offerings. This is because, in do that too rapidly.’ Another interviewee mentioned
service companies (companies C, E, and G), employees are working that: ‘Kanban brings more clarity to activities, helps in
on multiple projects at the same time; and product companies (compa- identifying high‐priority work and resource allocation,
nies A, B, D, and F) need to ensure that the company has enough helps management to select high‐priority activities and
resources to match portfolio needs at all levels. This is evident from to decide where to put more (or less) effort for the
the following extract from an interview: organisation to survive.

On the portfolio level, we break things down into the In summary, portfolio management in agile and lean software com-
product level and match them with our capabilities. This panies is a challenging undertaking. Some of the challenges mentioned
helps to plot which teams are involved in which by interviewees included the following. (1) It was difficult to determine
development [product project] and able to see which the long‐term and short‐term value of proposed initiatives. (2) Multiple
team will be free at which point of time, and we can offerings in the portfolio, as well as the increasing number of proposed
plan accordingly for the upcoming features. initiatives, made it difficult to prioritize and balance them against
available capacity. (3) Most importantly, balancing the time factor
Teams' feedback proved to be particularly important in the con-
was challenging for the companies, mainly because the portfolio was
text of agile software development, as it helps speed up product
constantly exposed to change and evolution. As such, there was a need
development activities by mitigating the challenges associated with
for dynamic tools and methods that would help to break down strate-
the process of adjusting resources. On a more abstract level, this also
gic decisions and goals and link them to the portfolio of products and
drives companies to seek and place emphasis on people with broad
services and daily tasks.
competencies to increase the company's ability to make quick changes
while maintaining the balance in portfolio:
6 | COMPARISON OF PORTFOLIO
We build an organisation where we can make very quick
MANAGEMENT TOOLS AND METHODS IN
moves and changes capability‐wise. Any portfolio
L I T E R A T U R E A N D CO M P A N Y P R A C T I C E
change triggers capability thinking [meaning] whether
we handle the change. We have teams and people that
Tools and methods presented in traditional portfolio management
are capable of doing a very broad scale of things.
literature were compared with those used in the selected agile and lean
Some of the challenges experienced by companies in the pro- software companies, according to our 4 goals of portfolio management.
cess of adjusting resources included the need to hire employees The 4 goals are used as guide to report portfolio management tools and
with new competencies for service companies (companies G and methods from the literature and software companies. Table 5 presents
C), and making requests to higher levels of the hierarchy. This latter a synthesized outline of findings and comparison of tools and methods
process can be slow and painful, as expressed by the interviewee in traditional portfolio management literature with the portfolio
from company E: management practice of agile and lean software companies.

TABLE 5 Comparison of portfolio management tools and methods in literature and company practice
Portfolio Methods and Tools
Reported Both in Portfolio Only Reported in Traditional
Main Goal of Portfolio Management Literature and Portfolio Management Only Reported in Software
Management Company Practice Literature Company Practice

Maximize the value Financial models: DCF, NPV, ROI, Financial models: PI, IRR, OPT, Multi‐criteria models: Net Promoter
of the portfolio and PBP EVA, Decision tree analysis, Score, lean metrics (lead time, value
ECV and ROA throughput), product demos, and
product experimentation
Multi‐criteria models: scoring models, cost‐ Multi‐criteria models: Fuzzy logic,
benefit analysis, market potential, and delphi model, analytical hierarchy
technical capability process (AHP)
Ensure balance among Strategic buckets (key focus areas) Portfolio matrix (such as McKinsey Cumulative flow diagrams, Kanban,
portfolio offerings and business and product matrix, and Boston Consulting business agilefant (items‐effort
roadmaps Group matrix) distribution), burn down charts,
and ad hoc process
Achieve strategic Strategic approaches (strategic Strategic alignment model and Key performance indicators (KPI),
alignment of the buckets and product and balanced scorecard Kanban, and short‐ and long‐term
portfolio technology roadmaps) customer relations and continuity
Pick the right number Capacity analysis (pipeline management) Resource capacity analysis and rank Work in progress count (Kanban
of offerings ordering projects principle) competency use and
performance
12 of 16 AHMAD ET AL.

Portfolios methods (such as financial models and checklists) used also reported that there are indications of an increasing tendency
by the agile and lean software companies to evaluate and maximize toward using Kanban in software companies.
the value of a portfolio were similar to those described in traditional In our earlier study38 with lean software companies, we found out
portfolio management literature. However, financial models were less that using Kanban in software development teams resulted in various
preferred by companies than checklists. These checklists consisted of benefits, like better visibility, improved transparency of work and com-
multiple criteria that were not formalized across all companies. Agile munication, and better control over workflow and WIP. In some more
and lean software companies prefer to use lead time, customer prefer- recent books,13,18,33,39 journals (for example, Cutter IT Journal),16 and
ence, and satisfaction, as well as other measures that solicit customer blogs for software practitioners, it is reported that Kanban is being
feedback. These techniques were also advocated by agile and lean adopted in a wide set of circumstances, from IT operations, develop-
methodologies. This is in contrast to portfolio management tools and ment and operations (DevOps), software development, and project
methods described in the literature, particularly the Cooper et al46 management through to portfolio management. This is further
study, which found that the predominant portfolio tools and methods explained by Alan Shalloway,17 who says that Kanban provides
were financial methods (for example, NPV and ROI) and the least visibility and helps management make appropriate decisions about
predominant were other methods such as customer appeal. tasks on the basis of business value.
The predominant portfolio management methods used by agile The companies in our study had recently started to use Kanban for
and lean software companies to ensure balance among portfolio portfolio management, but it was still in the early stages of adoption.
offerings were portfolio matrices. In traditional portfolio literature, The companies understood the Kanban method and used it at the
portfolio matrixes (such as Boston Consulting Group matrix and portfolio level. To identify high‐priority work and allocate resources
McKinsey matrix) have been emphasized, but these were not found at the portfolio level, one interviewee mentioned that Kanban “helps
to be used in the studied agile and lean software companies. in identifying and clarifying the most important (high priority) activi-
Additionally, for balancing the portfolio, the literature emphasized ties.” Such clarity helps management to decide what to do first and
using parameters such as various types of initiatives or risk levels. In how to adjust efforts for the organization to stay in the market. It also
contrast, agile and lean software companies used tools (such as helps with the management and the allocation of resources in the
Kanban and CFDs) that visualize the flow of items in product develop- selection and implementation process. For example, one interviewee
ment activities and signal where more attention is required. Kanban is mentioned that “with Kanban, it is easy to see the high priority work
also recommended in the literature, as it helps to balance offerings in against available resources.” So if a team is working on high‐priority
the portfolio.13 In companies that offered R&D services, matrices that tasks, the work becomes more visible at a higher level and might trig-
helped to visualize the allocation of employees to customer projects ger the reassignment of resources, depending on the context. The JIRA
according to demand and competencies were adopted. The latter Kanban board, 1 of the tools used, involves cumulative flow diagrams
was not mentioned in the portfolio management literature. Generally, and aids the visualization process. Kanban helps in finding a balance
portfolio mapping tools in traditional portfolio literature and in agile between demand and capability. As one of the interviewees observed:
and lean software companies were similar, as they all emphasized the
Kanban doesn't balance the portfolio, but it signals that
visualization aspect through the use of graphs or charts.
(for example, that the working pipeline is full), and, in
In portfolio management literature, strategic buckets are the pre-
turn, helps us to evaluate suitable available options at a
dominant method used for achieving strategic alignment of the portfo-
given time, and thus take the necessary actions.
lio. This is similar to what was observed in the agile and lean software
companies, although different names were used to represent the Kanban, at the portfolio level, was explained by companies B and F
method, such as key focus areas in which resources were allocated through the following scenario. A single business line has 3 Kanban
and prioritized annually according to the company's business objec- tiers: (1) business‐line Kanban, containing releases of all offerings; (2)
tives. Product and technology roadmaps were also used to ensure program‐level Kanban, containing the product features for a specific
alignment, similar to methods proposed in the literature. Neither the offering; and (3) team‐level Kanban, containing user stories for a spe-
literature nor the studied agile and lean companies explicitly described cific offering. The company's business goals were linked at tier (1) with
methods for picking the right number of offerings, other than using business‐line investments, with the help of Kanban, which made the
different means for analyzing capacity. investment visible for each business line's offerings. Similarly, at tiers
(2) and (3), Kanban was used to visualize the flow of features from pro-
gram level to team level. As a result, the workflow, along with the
7 | K A N B A N I N P O R T F O L I O M A NA G E M E N T investment on each offering, became clearly visible at each level; JIRA
was used for each Kanban tier, because it encodes and unifies tiny bits
In light of the interviews of the experts, this section answers RQ4. The and details of the actual processes.
use of Kanban for portfolio management seems not to have been
extensively reported in the scientific literature so far. Edmondson
and McManus47 explained that any area of research has 3 stages: 8 | VALIDITY
nascent, intermediate, and mature. A recent systematic literature
review on Kanban35 reported that research on Kanban in software In this study, threats to validity were considered by following the
engineering is currently nascent and requires further exploration. It is guidelines outlined by Runeson and Höst.42 Construct validity is
AHMAD ET AL. 13 of 16

concerned with obtaining the right measures for the concept being products through the quick delivery of functionality.20,34 This aspect
studied. A potential threat is the selection of interviewees with a view increases the probability of reducing the risk of uncertainty, and also
to obtaining appropriate data to answer the research questions. There- maximizes value for the customer and the company.
fore, the key informant technique was applied for selecting the pool of From the studied companies, we can identify and draw 3 impli-
interviewees. Construct validity is also threatened if interview ques- cations of portfolio management in agile and lean software develop-
tions are misunderstood or misinterpreted. Therefore, pre‐test ment. First, the selected agile and lean software companies actively
interviews were conducted. Additionally, at the beginning of each used tools and methods that helped facilitate customer feedback.
interview session, interviewees were introduced to key concepts and Moreover, agile and lean approaches in software development and
terminologies used in the study to avoid misinterpretation. delivery served to lower the level of investment risk for the compa-
Internal validity focuses on how to establish a causal relationship nies. This is because, instead of relying on financial models to
and is mainly used for explanatory and causal studies. As this study evaluate value maximization, companies relied on the immediate
was of an exploratory nature, internal validity was not considered. delivery of products to solicit customer feedback and quickly ascer-
External validity is the ability to generalize the findings to a spe- tain the value of products. Financial models, as portfolio manage-
cific context. The findings of our study give insight into the portfolio ment tools, require long periods to compute, and also to confirm
management tools and methods of 7 software companies from the their estimations after product delivery. In addition, incremental
N4S program. The participating companies differed in terms of size delivery and the ability to make changes at any time during product
and the nature of their offerings, but similar patterns were found in development further increased a company's ability to quickly respond
their portfolio management tools and methods. However, the findings to emerging opportunities.
should be considered with caution, because the companies in the Second, agile and lean approaches allowed companies to have
present study cannot be assumed to represent agile and lean software more frequent and decentralized decision‐making points, in contrast
companies in general. to traditional portfolio management processes, which are often plan‐
One threat to reliability is concerned with repetition or replication driven and centralized.20,34
that the same result would be found if study is replicated. There is Third, the traditional literature on portfolio management methods
always a risk that the outcome of the study is affected by the interpre- and tools does not take into consideration customer interaction to the
tation of the researcher. To mitigate this threat, the study was extent required in agile and lean software companies. Generally, these
designed so that data were collected from different sources, existing findings have important implications for traditional portfolio manage-
scientific literature, and various role interviewees from 7 different ment processes, which tend to incorporate pre‐planning activities on
companies. Further, in each phase of the research process, at least 2 a large scale with minimum flexibility, or which use complex methods
researchers were involved to reduce the risk of bias. and tools.20
One limitation of this study is the fairly small number of inter- For portfolio management, a variety of methods and tools are
viewees from the participating software companies. However, we adopted and adjusted according to the context. The findings high-
obtained rich, interesting, and valuable information from the selected light that agile and lean software companies rarely use financial
top‐level decision makers. Additionally, the responses from the key models for their offerings at the portfolio level. Portfolio manage-
informants in the different companies were quite similar, and not ment tools and methods need to be dynamic and adaptable to pro-
much new information was added in later interviews. This means that cess changes within the company. Agile, lean, and Kanban adoption
the information was saturating sufficiently, and so no more inter- in companies has changed the nature of collaboration, coordination,
viewees were needed. A second limitation is the use of a single search and communication in software development projects. This also illus-
engine (Google Scholar) in the literature review, which could lead to a trates a shift toward more collaborative and coordinated portfolio
risk of missing relevant studies. However, we saw that most of the management processes with greater transparency, trust, and
portfolio management tools and methods were already identified after frequent interaction.20
reading 20 out of the 200 papers. Hence, this threat is considered to For the studied agile and lean software companies, business
be under control. strategy played a significant role and was a crucial tool for managing
the portfolio. This puts emphasis on the means used to communicate
the business strategy clearly across the entire company and its active
9 | D I S C U S S I O N A N D CO N C L U SI O N renewal. For better portfolio management, the business strategy needs
to be actively reviewed and clearly communicated across the entire
The findings of this study revealed that agile and lean software compa- company. Additionally, the software companies interviewed use
nies used some of the traditional portfolio management tools and Kanban at the portfolio level, which brings more visibility to the
methods, but the emphasis was different from that described in the workflow and clarity with respect to high‐priority activities measured
portfolio management literature. The main distinguishing factors were against resources.
that agile and lean software companies used tools and methods that This study contributes to the body of knowledge on portfolio
emphasized getting immediate feedback from customers and management and agile and lean software product management by
immediate delivery of offerings to evaluate the value maximization of providing empirical evidence on tools and methods used to manage
the portfolio. Agile and lean companies strongly advocate frequent a portfolio of offerings. The findings highlight that agile and lean
interaction with customers, and gaining immediate feedback on software companies have a different emphasis than that described
14 of 16 AHMAD ET AL.

in the traditional portfolio management literature. The main differ- International Computer Software and Applications Conference.
ence found was that companies relied on and used tools and 2008:1383–1391.

methods that solicited customer feedback and engagement. Addi- 16. Anderson D, Roock A. An agile evolution: why Kanban is catching on in
Germany and around the world. Cutter IT J. 2011;24:6–17.
tionally, in agile and lean software companies, Kanban played a sig-
17. Shalloway A. The real differences between Kanban and Scrum, http://
nificant role in terms of limiting the number of offerings in the
www.netobjectives.com/blogs/real‐differences‐between‐kanban‐and‐
portfolio and their strategic alignment. The companies that partici- scrum.
pated in this study varied in terms of size and nature of the offerings 18. Anderson D. Kanban. Blue Hole Press, sequim, Washington 2010.
being delivered. This suggests that further studies are required to 19. Carbonell‐Foulquié P, Munuera‐Alemán JL, Rodrı́guez‐Escudero AI.
investigate portfolio management in software companies that have Criteria employed for go/no‐go decisions when developing
similar characteristics. successful highly innovative products. Ind Mark Manag.
2004;33:307–316.
20. Stettina CJ, Hörz J. Agile portfolio management: an empirical perspec-
ACKNOWLEDGMENTS tive on the practice in use. Int J Proj Manag. 2015;33:140–152.

This research was performed within the DIMECC (Digital, Internet, 21. Maylor H. Project Management. New Jersey, USA: Prentice Hall; 2010.

Materials & Engineering Co‐Creation) Need for Speed program and 22. Linenberg Y, Stadlker Z, Arbuthnot S. Optimising organisational
performance by managing project benefits, Europe 2003.
partially funded by Tekes (the Finnish Funding Agency for Technology
23. Webster J, Watson R. Analyzing the past to prepare for the future:
and Innovation), Nokia foundation grant, and Ella and Georg Ehrnrooth
writing a literature review. Manag Inf Syst Q. 2002;26:13–19.
foundation grant. We would like to thank the participating companies.
24. Frey T, Buxmann P. IT project portfolio management—a structured lit-
erature review. European Conference on Information Systems. p. Paper
167 2012.
RE FE R ENC E S
25. Cooper R, Edgett S, Kleinschmidt E. Portfolio management for new
1. Dolci P, Maçada A. Portfolio theory: the contribution of markowitz's
product development: results of an industry practices study. R&D
theory to information system area. In: Information Systems Theory.
Manag. 2001b;31:361–380.
New York: Springer; 2012.
2. Cooper RG, Edgett SJ, Kleinschmidt EJ. Portfolio Management for New 26. Cooper R, Edgett S, Kleinschmidt E. Portfolio management: fundamen-
Products. Cambridge, Massachusetts: Perseus Publishing; 2001a. tal to new product success. In: Belliveau P, Griffin A, Somermeyer S,
eds. The PDMA ToolBook 1 for New Product Development. New York:
3. Rodríguez P, Markkula J, Oivo M, Turula K. Survey on agile and lean John Wiley & Sons; 2002:331–364.
usage in Finnish software industry. In Proceedings of the ACM‐IEEE Inter-
national symposium on Empirical software engineering and measurement. 27. Mikkola JH. Portfolio management of R&D projects: implications for
ACM Press, New York, USA; 2012:139–148. innovation management. Technovation. 2001;21:423–435.

4. Kalliney M. Transitioning from agile development to enterprise prod- 28. Maglyas A, Nikula U, Smolander K. What do we know about software
uct management agility. IEEE Agile conference. 2009:209–213. product management?—a systematic mapping study. 5th IEEE Interna-
tional Workshop on Software Product Management. 2011:26–35.
5. Rautiainen K, von Schantz J, Va J. Supporting scaling agile with portfo-
lio management: case paf.com. 44th IEEE Hawaii International 29. van de Weerd I, Brinkkemper S, Nieuwenhuis R, Versendaal J, Bijlsma L.
Conference on System Sciences. 2011:1–10. Towards a reference framework for software product management.
14th International Requirements Engineering Conference. 2006:319‐322.
6. Sweetman R, Conboy K. Exploring the tensions between software pro-
ject portfolio management and agile methods: a research in progress 30. Cooper RG, Edgett SJ, Kleinschmidt EJ. New problems, new solutions:
paper. 4th International Conference on Lean Enterprise Software and making portfolio management more effective. Res‐Technol Manag.
Systems .Springer Berlin Heidelberg. 2013:210–217. 2000;43:18–33.
7. Abrantes R, Figueiredo J. Feature based process framework to manage 31. PMI: The standard for portfolio management, 2008.
scope in dynamic NPD portfolios. Int J Proj Manag. 2014;32:874–884. 32. Hodgkins P, Hohmann L. Agile program management: lessons learned
8. Thiry M, Deguire M. Recent developments in project‐based organisa- from the VeriSign managed security services team. IEEE Agile confer-
tions. Int J Proj Manag. 2007;25:649–658. ence. 2007:194–199.
9. Killen CP, Hunt RA, Kleinschmidt EJ. Managing the new product devel- 33. Krebs J. Agile Portfolio Management. Washington: Microsoft Press;
opment project portfolio: a review of the literature and empirical 2009.
evidence. IEEE Portland International Conference on Management of
34. Bhattacharya S, Krishnan V, Mahajan V. Managing new product defini-
Engineering & Technology. 2007:1864–1874.
tion in highly dynamic environments. Manag Sci. 1998;44:50–64.
10. Laanti M, Kangas M. Is agile portfolio management following the prin-
35. Ahmad MO, Markkula J, Oivo M. Kanban in software development: a
ciples of large‐scale agile? Case study in Finnish Broadcasting
systematic literature review. IEEE 39th Euromicro Conference on
Company Yle. IEEE Agile Conference. 2015:92–96.
Software Engineering and Advanced Applications. 2013:9–16.
11. Roche GM. Agile portfolio management at NYSE. IEEE Agile Conference.
36. Ahmad MO, Kuvaja P, Oivo M, & Markkula J. Transition of soft-
2012:117–122.
ware maintenance teams from Scrum to Kanban. IEEE proceedings
12. Vähäniitty J. Towards agile product and portfolio management, http:// of 49th Hawaii International Conference on System Sciences.
urn.fi/URN:ISBN:978‐952‐60‐4506‐1, 2012. 2016a:5427‐5436.
13. Leffingwell D. Agile Software Requirements: Lean Requirements Practices 37. Ahmad MO, Markkula J, & Oivo M. Insights into the perceived benefits
for Teams, Programs, and the Enterprise. Massachusetts: Addison‐ of Kanban in software companies: practitioners' views. International
Wesley Professional; 2010. Conference on Agile Software Development. Springer International Pub-
14. Kettunen P, Laanti M. Combining agile software projects and large‐ lishing. 2016b:156‐168.
scale organizational agility. Softw Process: Improv and Pract. 38. Ahmad MO, Markkula J, Oivo M. Kuvaja P. Usage of Kanban in soft-
2008;13:183–193. ware companies—an empirical study on motivation, benefits and
15. Laanti M. Implementing program model with agile principles in a challenges. 9th International Conference on Software Engineering
large software development organization. 32nd Annual IEEE Advances 2014.
AHMAD ET AL. 15 of 16

39. Burrows M. Kanban from the Inside: Understand the Kanban Method, and portfolio management are the focus of this second in a three‐
Connect it to what you already Know, Introduce it with Impact. Sequim, part series. Research‐Technology Management, 2004, 47(3): 50.
Washington: Blue Hole Press; 2014.
40. Wohlin C. Guidelines for snowballing in systematic literature studies [S8] Dickinson, Michael W., Anna C. Thornton, and Stephen Graves.
and a replication in software engineering. Proceedings of the 18th Inter- Technology portfolio management: optimizing interdependent
national Conference on Evaluation and Assessment in Software projects over multiple time periods. IEEE Transactions on Engineer-
Engineering. ACM Press, New York, New York, USA 2014:1–10.
ing Management, 2001, 48(4): 518‐527.
41. Jalali S, Wohlin C. Systematic literature studies. Proceedings of the ACM‐
IEEE international symposium on Empirical software engineering and mea- [S9] Augusto Cauchick Miguel, Paulo. Portfolio management and
surement. p. 29. ACM Press, New York, USA; 2012. new product development implementation: a case study in a
42. Runeson P, Höst M. Guidelines for conducting and reporting case study manufacturing firm. International Journal of Quality & Reliability
research in software engineering. Empir Softw Eng. 2009.
Management, 2008, 25(1): 10‐23.
43. Kumar N, Stern LW, Anderson JC. Conducting interorganizational
research using key informants. Acad Manage J. 1993;36:1633–1651. [S10] Killen, Catherine P., Robert A. Hunt, and Elko J. Kleinschmidt.
44. Reyck BD, Grushka‐Cockayne Y, Lockett M, Calderini SR, Moura M, Project portfolio management for product innovation. Interna-
Sloper A. The impact of project portfolio management on information tional Journal of Quality & Reliability Management, 2008, 25(1):
technology projects. Int J Proj Manag. 2005;23:524–537. 24‐38.
45. King N. Template analysis. In: Symon G, Cassell C, eds. Qualitative
Methods and Analysis in Organizational Research: A Practical Guide. [S11] Killen, Catherine P., Robert A. Hunt, and Elko J. Kleinschmidt.
Thousand Oaks, CA: Sage Publications Ltd; 1998:118–134. Managing the new product development project portfolio: a
46. Cooper R, Edgett S, Kleinschmidt E. Portfolio management for new review of the literature and empirical evidence. In International
product development: results of an industry practices study. R&D Center for Management of Engineering and Technology Portland pro-
Manag. 2001c;31:361–380.
ceedings, IEEE, 2007, 1864‐1874.
47. Edmondson AC, McManus SE. Methodological fit in management field
research. Acad Manage Rev. 2007;32. [S12] Kohlborn, Thomas, Erwin Fielt, Axel Korthaus, and Michael
Rosemann. Towards a service portfolio management framework.
In Evolving Boundaries and New Frontiers: Defining the IS Disci-
pline: Proceedings of the 20th Australasian Conference on Informa-
How to cite this article: Ahmad MO, Lwakatare LE, Kuvaja P,
tion Systems, 2009, 861‐870.
Oivo M, Markkula J. An empirical study of portfolio manage-
ment and Kanban in agile and lean software companies. J Softw [S13] Meskendahl, Sascha. The influence of business strategy on

Evol Proc. 2017;29:e1834. https://doi.org/10.1002/smr.1834 project portfolio management and its success—a conceptual
framework. International Journal of Project Management, 2010,
28(8):807‐817
APPENDIX [S14] Mikkola, Juliana Hsuan. Portfolio management of R&D pro-
[S1] Barczak, Gloria, Abbie Griffin, and Kenneth B. Kahn. Perspec- jects: implications for innovation management. Technovation,
tive: trends and drivers of success in NPD practices: results of 2001, 21(7): 423‐435.
the 2003 PDMA best practices study. Journal of product innova- [S15] Oliveira, Maicon G., and Henrique Rozenfeld. Integrating tech-
tion management, 2009, 26(1): 3‐23. nology roadmapping and portfolio management at the front‐end of
[S2] Chao, Raul O., and Stylianos Kavadias. A theoretical framework new product development. Technological forecasting and social
for managing the new product development portfolio: when and change, 2010, 77(8): 1339‐1354.
how to use strategic buckets. Management Science, 2008, 54(5): [S16] Adams, Richard, John Bessant, and Robert Phelps. Innovation
907‐921. management measurement: a review. International Journal of Man-
[S3] Cooper, Robert, Scott Edgett, and Elko Kleinschmidt. Portfolio agement Reviews, 2006, 8(1): 21‐47.
management for new product development: results of an industry [S17] Christiansen, John K., and Claus Varnes. From models to
practices study. R&D Management, 2001, 31(4): 361‐380. practice: decision making at portfolio meetings. International
[S4] Cooper, Robert G., Scott J. Edgett, and Elko J. Kleinschmidt. Journal of Quality & Reliability Management, 2008, 25(1):
New problems, new solutions: making portfolio management 87‐101.
more effective. Research‐Technology Management, 2000, 43(2): [S18] Lee, Hakyeon, Chulhyun Kim, and Yongtae Park. Evaluation
18‐33. and management of new service concepts: an ANP‐based portfolio
[S5] Cooper, Robert G., and Scott J. Edgett. Maximizing productivity approach. Computers & Industrial Engineering, 2010, 58(4):
in product innovation. Research‐ Technology Management, 2008, 51(2): 535‐543.
47‐58. [S19] Lin, Chinho, and Ping‐Jung Hsieh. A fuzzy decision support
[S6] Cooper, Robert G., and Scott J. Edgett. Overcoming the crunch system for strategic portfolio management. Decision Support
in resources for new product development. Research‐Technology Systems, 2004, 38(3): 383‐398.
Management, 2003, 46(3): 48‐58. [E1] Loch, Christoph H., and Stylianos Kavadias. Dynamic portfolio
[S7] Cooper, Robert G., Scott J. Edgett, and Elko J. Kleinschmidt. selection of NPD programs using marginal returns. Management
Benchmarking best NPD practices‐II: strategy, resource allocation Science, 2002, 48(10): 1227‐1241.
16 of 16 AHMAD ET AL.

[E2] Gustafsson, Janne, and Ahti Salo. Contingent portfolio program- [E4] Lee, Sungjoo, Sungryong Kang, Euisuk Park, and Yongtae Park.
ming for the management of risky projects. Operations research, Applying technology road‐maps in project selection and planning.
2005, 53(6): 946‐956. International Journal of Quality & Reliability Management, 2008, 25(1):

[E3] Hart, Susan, Erik Jan Hultink, Nikolaos Tzokas, and Harry R. 39‐51.
Commandeur. Industrial companies' evaluation criteria in new [E5] Bordley, Robert. Determining the appropriate depth and
product development gates. Journal of Product Innovation breadth of a firm's product portfolio. Journal of Marketing Research,
Management, 2003, 20(1): 22‐36. 2003, 40(1): 39‐53.