Reliability Engineering and System Safety 231 (2023) 108988

Contents lists available at ScienceDirect

Reliability Engineering and System Safety

journal homepage: www.elsevier.com/locate/ress

Condition-based maintenance for a multi-component system in a dynamic

operating environment
Nan Zhang a , Yingjun Deng b , Bin Liu c ,∗, Jun Zhang d
a
School of Management and Economics, Beijing Institute of Technology, Beijing, China
b
Center of Applied Mathematics, Tianjin University, Tianjin, China
c
Business School, University of Strathclyde, Glasgow, UK
d
Advanced Research Institute of Multidisciplinary Science, Beijing Institute of Technology, Beijing, China

ARTICLE INFO ABSTRACT

Keywords: This paper develops a condition-based maintenance (CBM) model for a multi-component system operating
Dynamic environment under a dynamic environment. The degradation process of each component depends on both its intrinsic
Condition-based maintenance characteristic and the common operating environment. We model the environment evolution by a continuous-
Multi-component systems
time Markov process, given which, the degradation increment of each component is described by a Poisson
Markov decision process
distribution. System reliability is firstly obtained, followed by a CBM policy to sustain system operation and
ensure safety. In modelling the environmental effect on component degradation processes, two scenarios are
considered. The first scenario considers renewable environment evolution while the second scenario on non-
renewable environment evolution. The problem is casted into the Markov decision process (MDP) framework
where the total expected discounted cost in the long-run horizon is utilized as the optimization objective
to assess the policy. Structural properties of the optimal maintenance policy are investigated under mild
conditions, which are further embedded into the value iteration algorithm to reduce the computational burden
in calculating the maintenance cost. Applicability of the proposed model is illustrated through numerical
examples.

1. Introduction Relevant studies have addressed the above issues in the literature [3–
5]. However, existing works mainly focus on reliability assessment of
With the increasing demand on high quality and safety of sys- single-component systems. Models concerning multi-component sys-
tems/products, reliability evaluation and maintenance planning be- tems, especially multi-component degraded systems under dynamic
come increasingly more important in improving system safety and environment are rather limited. In recent decades, conditional-based
reducing operation budget in industrial field. Compared with the ideal maintenance of multi-component degraded systems has received in-
laboratory environment, in practice, a system usually operates in a creasing attention [6,7]. Most of the existing studies consider that
time-varying environment which exerts impacts on system functionality maintenance decisions are based on the system, 𝑖.𝑒., the states of
and performance. For instance, the lifetimes of components in an components. However, in practical operation, the health condition of a
aircraft are related to the atmospheric operating condition such as system depends not only on its manufacturing characteristics, but also
pressure, temperature, mechanical vibration [1]. One can imagine that
on its working environment. For instance, the degradation of blades
during flying and landing, the security requirements under the two
in offshore and onshore wind turbine are supposed to be different.
scenarios are different. In the oil and gas industry, the flows of gas,
Maintenance schedules should be adaptive to the working condition
oil and water contribute to the corrosion of throttle valves. In railway
of the system. To the best of the authors’ knowledge, little attention
transportation, locomotive wheels have to be reprofiled occasionally
has been paid to the influence of the environment on the maintenance
to reduce the roughness and imbalance among the wheels [2]. The
roughness and imbalance is associated with various environmental decision-making.
factors such as weather conditions, running speed, load, 𝑒𝑡𝑐. It is there- This paper will develop a generic model of degraded systems with
fore necessary to take the operating environment into consideration heterogeneous components under a dynamic working environment. We
in assessing system reliability and developing maintenance strategies. intend to derive some system reliability measures and present how to

∗ Corresponding author.
E-mail address: [email protected] (B. Liu).

https://doi.org/10.1016/j.ress.2022.108988
Received 18 October 2022; Received in revised form 16 November 2022; Accepted 19 November 2022
Available online 25 November 2022
0951-8320/© 2022 Elsevier Ltd. All rights reserved.
N. Zhang et al. Reliability Engineering and System Safety 231 (2023) 108988

Notation 𝑉𝜏 (𝒙, 𝑤) Value function when the environment is re-

newable, denoting minimum expected total
𝑁 Number of components in the system
discounted maintenance cost in the long-
𝛷 Set of components, 𝛷 = {1, 2, … , 𝑁} run horizon given the initial degradation
𝑆𝑙 Set of environment states, 𝑆𝑙 = {0, 1, … , 𝑚} state 𝒙 and environmental state 𝑤
𝐸1 ∖𝐸2 Set-theoretic difference of sets 𝐸1 and 𝐸2 𝑉̂𝜏 (𝒙, 𝑤) Value function when the environment
𝑊𝑡 Environment state at time 𝑡 is non-renewable, denoting minimum
𝐿𝑖 Failure state of component 𝑖, 𝑖 ∈ 𝛷 expected total discounted maintenance cost
𝒙 Vector of length 𝑁 representing the degra- in the long-run horizon given the initial
dation state of the system degradation state 𝒙 and environmental
𝑥𝑖 The 𝑖th element in 𝒙 denoting the degrada- state 𝑤
tion of component 𝑖
𝑺 Set of the system state
𝑆𝑔 Set of system degradation states under
component and their working environment state are fully and perfectly
which the system is functioning
observed. A condition-based maintenance (CBM) model is constructed
𝑆𝑔(1) Subset of 𝑆𝑔 under which all components
to prevent system failure: failed components are replaced, meanwhile,
are functioning
non-failed components can be preventively replaced, depending on
𝑆𝑔(2) Subset of 𝑆𝑔 under which failures exist at their degradation levels. In terms of the environmental impact, this
the component level, 𝑆𝑔 = 𝑆𝑔(1) ∪ 𝑆𝑔(2) study considers two scenarios: the first one is that when the system is
𝜆𝑖 (𝑡, 𝑤) Parameter of the Poisson distribution corre- renewed, the working environment can be restored to its initial friendly
sponding to component 𝑖 in environment 𝑤, state. As stated in [3], in the reliability analysis of the transmission
𝑖 ∈ 𝛷, 𝑤 ∈ 𝑆𝑙 system from a mining company, the wear metals iron was chosen as
𝒐𝑡 Realization of the system degradation at the covariate related to the system health. Whenever the system was
time 𝑡 replaced, this wear metals iron was removed and the environmental
𝑃𝑤𝑙 (𝑡, 𝒙) Probability that the degradation increment effect could be reset to the initial state. The other scenario is that the
is 𝒙 with environment state 𝑙 given the ini- environment is independent of maintenance actions, such as the phys-
tial brand-new state and environment state ical weather. The two scenarios will be referred to as renewable and
𝑙, 𝑤, 𝑙 ∈ 𝑆𝑙 . non-renewable environment in the following contexts We formulate
𝑷 (𝑡, 𝒙) Matrix form of 𝑃𝑤𝑙 (𝑡, 𝒙) this problem into a Markov decision process (MDP) [8,9] and the total
𝜦𝑖 (𝑡, 𝒙) Diagonal matrix with the (𝑤, 𝑤)th entry expected discounted cost is utilized to assess the policy. Specifically,
𝜆𝑖 (𝑡, 𝑤), 𝑖 ∈ 𝛷, 𝑤 ∈ 𝑆𝑙 under some mild conditions, structural properties of the value function
∑ and the maintenance policy are derived. The main contributions of this
𝜦(𝑡, 𝒙) Diagonal matrix, 𝜦(𝑡, 𝒙) = 𝑖∈𝛷 𝜦𝑖 (𝑡, 𝒙)
paper are summarized as follows.
I{⋅} Indicator function
𝑅(𝑡; 𝒗) System reliability function at 𝑡 given the • We investigate the reliability measures and a CBM policy for a sys-
initial brand new state and the environment tem with multiple heterogeneous degrading components where
state 𝒗 the system is generic and no specific system configuration is
𝑅(𝑡; 𝒙, 𝒗) System reliability function at 𝑡 given the assumed.
initial degradation state 𝒙 and the environ- • We discuss both renewable and non-renewable environment ef-
ment state 𝒗 fects on the maintenance strategy and the corresponding mainte-
𝑐𝐼 Inspection cost each time nance cost.
𝑐𝑠 Maintenance set-up cost each time • We provide exact system reliability measures and the impacts of
𝑐𝑝𝑖 Preventive replacement cost of component 𝑖 the environment on the system reliability and maintenance policy
𝑐𝑟𝑖 Corrective replacement cost of component 𝑖 are examined.
• Under mild conditions, the structural properties of the mainte-
𝑐𝑑 Penalty of the system down-time per unitary
nance policy and the optimal maintenance cost are derived.
time
𝑐𝑓 System replacement cost The rest of the paper is organized as follows. In Section 2, related lit-
𝛾 Parameter associated with the discounted erature review is given. In Section 3, we present the model assumptions,
factor including description of the dynamic environment, system configura-
𝐶𝑖𝑛𝑠 (𝜏) Total expected inspection cost in the long- tion and component characteristics. The system reliability and condi-
run horizon tional reliability function given the component internal degradation
𝜅𝜏 (⋅) Transition kernel and the dynamic working environment are presented. Section 4 focuses
𝐷𝜏 (𝒙, 𝑤) Expected downtime during time period 𝜏 on the development of maintenance policy, and formulates the problem
given the initial system degradation 𝒙 and into the MDP framework. Structural properties of the optimal mainte-
environmental state 𝑤 nance policy are investigated, which are further incorporated into a
value iteration algorithm for calculation purpose. Numerical illustra-
𝑈𝜏 (𝒙, 𝑤) Expected cost until the next inspection
tions are presented to show the applicability of the proposed model in
epoch given the initial system degradation
Section 5. Finally, our conclusions and future research directions are
𝒙 and environmental state 𝑤
provided in Section 6.

2. Literature review
make maintenance planning based on the observation: the state of sys-
tem and the state of environment. We consider the case that the system In the literature, most researchers take the model proposed in [10]
is periodically inspected, upon which, the degradation level of each as the first study concerning the environment impact on the system

2
N. Zhang et al. Reliability Engineering and System Safety 231 (2023) 108988

reliability where damages induced by random shocks depended on an overview of the existing literature, it can be concluded that not
environment conditions. Since then, a bunch of studies have been much attention has been paid to modelling the influence of the dynamic
developed [11–14]. To model the effect of the dynamic environment on working environment on multi-component systems. In addition, when
the system health condition, various approaches have been proposed. considering maintenance interventions, most of the existing works
The proportional hazards rate model (PHM) and accelerated failure assume that perfect maintenance can restore the system to the as-good-
time model (AFTM) are extensively utilized where the environment as-new state, meanwhile, the environment can be restored to its initial
effect is incorporated into the failure rate as a covariate [15–18]. state too. However, this assumption can be violated in some situations.
Markov chain has also received considerable attention in modelling For instance, when the dynamic environment is seasons or weathers, it
the variation of operating condition, among which, a continuous time is independent of maintenance actions and is non-adjustable. Both the
Markov chain (CTMC) is implemented to the scenarios that the transi- two scenarios should be taken into consideration.
tion epochs are arbitrary. Shen et al. [19] studied the availability and
imperfect maintenance of a degrading system under a dynamic environ- 3. Problem statement
ment described by a CTMC. Under the constraints of system availability
and operating times, the number of optimal imperfect actions were 3.1. System descriptions
determined by minimizing the long term expected cost rate. Zhang
et al. [5] incorporated the impact of the environmental influence on a In this study, we aim to develop a CBM model of a multi-component
degradation-threshold-shock model, where system reliability measures, system operating in dynamic environment modelled by CTMC Consider
expected total maintenance cost in the finite horizon and expected cost a system consisting of 𝑁 heterogeneous components. Denote the set
rate in the infinite horizon were derived. Discrete time Markov chain of components as 𝛷 = {1, 2, … , 𝑁}. At the start of operation, it is
(DTMC) is utilized regarding cyclical transitions of the environment assumed that all the components are in the as-good-as-new state. The
states. Zhao et al. [20] considered the conditional-based inspection and degradation level of component 𝑖 is categorized into 𝐿𝑖 + 1 different
replacement policy of a non-monotone deteriorating system. The ob- states, defined as 0, 1, … , 𝐿𝑖 , where state 0 represents the as-good-as-
servable system degradation was the difference between two competing new state and state 𝐿𝑖 indicates the failure state. It is assumed that each
processes that depended on an environmental covariate modelled by a component goes through a monotonic degradation process, where the
DTMC. degradation level is monotonically increasing if no maintenance action
In degradation modelling, for single-component systems, when the is executed. The degradation processes of components are assumed to
degradation state space is continuous, the Gamma process [21–24] and be independent of each other given the working environment, 𝑖.𝑒., no
Inverse Gaussian process [25,26] have been widely implemented to interaction among the components is considered. The degradation state
model the monotonic degradation processes. Diffusion processes like of the system is represented by 𝒙 = (𝑥1 , 𝑥2 , … , 𝑥𝑁 ) where 𝑥𝑖 is the 𝑖th
Wiener process, Ornstein–Uhlenbeck process are preferable in describ- entry of 𝒙.
ing non-monotonic degradations [27–31]. When the degradation state The system works in a dynamic environment which is modelled by a
is discrete, in general, most of the studies assume that the degrada- CTMC {𝑊𝑡 }𝑡≥0 with a finite state space 𝑆𝑙 = {0, 1, … , 𝑀}, infinitesimal
tion process exhibits Markov properties [32–35]. For example, Markov generator 𝑸 and transition probability 𝜋𝑖𝑗 (𝑡), 𝑖, 𝑗 ∈ 𝑆𝑙 . Under environ-
chain and compound Poisson process have been widely used to model ment 𝑤, the internal degradation increment of component 𝑗 follows a
the degradation process of discrete states [see 36]. In fact, both the Poisson distribution with parameter 𝜆𝑗 (𝑡, 𝑤) by time 𝑡, 𝑗 ∈ 𝛷, 𝑡 ≥ 0.
Gamma process and Inverse Gaussian process can be treated as limits Thus the degradation evolution of each component is characterized by
of compound Poisson process [21,25]. For degrading systems with mul- its internal degradation and the working environment. The system state
{ }
tiple components, aside from modelling each component, dependency space is defined as 𝑺 = (𝒙, 𝑤) ∶ 𝑥𝑖 ∈ {0, 1, … , 𝐿𝑖 }, 𝑤 ∈ 𝑆𝑙 where
is also an important characteristic which should be taken into account: 𝒙 = {𝑥1 , 𝑥2 , … , 𝑥𝑁 } represents the system degradation state and 𝑤
stochastic dependence, economic dependence, resource dependence, represents the environment state.
𝑒𝑡𝑐. are presented in detail in [8].
In recent decades, conditional-based maintenance of 3.2. Descriptions of the maintenance process
multi-component degraded systems has received increasing attention
due to advanced sensors and the corresponding available data [6, Suppose that the failures are non-self-announcing at both the system
29,37]. Poppe et al. [7] studied a hybrid CBM policy where two and component level. The system is periodically inspected with interval
degradation threshold of the monitored component were determined to 𝜏. The inspection is perfect upon which the environment state and
minimize the maintenance cost or the system downtime. Sun et al. [38] the state of each components can be fully observed. Upon inspection,
considered a 𝐾-out-of-𝑁 system where the degradation of each compo- the system is renewed to the as-good-as-new state if it fails. If the
nent was described by a Wiener process. They showed that the optimal system can still operate but failure exists at the component level, then
maintenance policy was a multi-dimensional control limit policy. Sim- the failed components are replaced, at the same time, each non-failed
ilarly, Liu et al. [9] studied the optimal inspection and replacement component can be preventively replaced, or remain in its current state.
policy of a two-component parallel system over a finite time horizon. Otherwise, if no component fails, then the decision-maker can choose
The system was modelled by a bivariate Gamma process. Several stud- to:
ies have considered multi-component systems working under dynamic
• do nothing and wait for the next decision epoch;
environment. For example, Zhang et al. [1] examined the reliability of
• preventively replace some degraded components.
a 𝑘-out-of-𝑛 system under a dynamic environment, where each compo-
nent possessed binary states. The authors investigated system reliability The inspection and replacement times are negligible. The cost items
measures such as the conditional reliability function, remaining useful involved in the maintenance policy include:
life and asymptotic availability. Shi et al. [39] developed a condition-
• Inspection cost: 𝑐𝐼 ,
based maintenance policy of a degrading system where degradation and
environmental observations were utilized to update the posterior distri- • Maintenance set-up cost: 𝑐𝑠 ,
butions of the failure model parameters. They reported that cost-saving • The preventive maintenance cost of component 𝑖: 𝑐𝑝𝑖 , 𝑖 ∈ 𝛷,
could be achieved when the environment information was incorporated • The corrective maintenance cost of component 𝑖: 𝑐𝑟𝑖 , 𝑐𝑝𝑖 ≤ 𝑐𝑟𝑖 ,
in the decision making. Peng et al. [40] presented a summary of 𝑖 ∈ 𝛷,
the state of the arts on the reliability modelling of complex systems • The system replacement cost: 𝑐𝑓 ,
with consideration of the effects of dynamic environments. Through • Downtime penalty per unitary time due to system failure: 𝑐𝑑 .

3
N. Zhang et al. Reliability Engineering and System Safety 231 (2023) 108988

We intend to determine the optimal maintenance strategy, 𝑖.𝑒., the The proof is omitted here as it is similar to the proof of Lemma 4.2.
inspection interval and the corresponding maintenance policy that It is worth mentioning that the exact expression of 𝑷 (𝑡, 𝒙) is in-
minimizes the long-run total discounted inspection and maintenance tractable due to the fact that 𝑸 and 𝜦(𝑡) are not commute in general.
cost. We will turn to numerical methods to solve the above differential
equations. In this work, we utilize the product-integration method to
4. Optimal maintenance policy solve these equations. Details are given in Appendix A.3. A special case
is that if the state of each component is binary, meaning that 𝐿𝑖 = 1,
This section develops a CBM policy for the system operating in ∀𝑖 ∈ 𝛷, then the problem and associated solutions can be found in [1].
dynamic environment, where the optimal policy is obtained in terms The system reliability can be obtained based on the above lem-
of the inspection interval and maintenance action for each system mas. Denote 𝑆𝑔 as the set of the system degradation states under
state. We first present the system reliability analysis, which is an which the system is in the functioning state. For instance, consider
essential part in measuring system failure probability and is also the
a two-component system with 𝐿1 = 1, and 𝐿2 = 2. If it is a se-
stepping-stone for the subsequent maintenance analysis.
ries system, then 𝑆𝑔 = {(0, 0), (0, 1)}. If it is a parallel system, then
𝑆𝑔 = {(0, 0), (0, 1), (0, 2), (1, 0), (1, 1)}. If the system works if and only if
4.1. Reliability analysis
component 1 works, then 𝑆𝑔 = {(0, 0), (0, 1), (0, 2)}.
The following theorem presents the system reliability and condi-
For a new system, denote 𝑃𝑤𝑙 (𝑡, 𝒙) as the probability that the system
state is (𝒙, 𝑙) by time 𝑡 given its initial state (𝟎, 𝑤) at time 0, 𝒐𝑡 as the tional reliability functions.
realization of the system degradation state at time 𝑡. Then 𝑃𝑤𝑙 (𝑡, 𝒙) can
be expressed as Theorem 4.1. Let 𝒗 = (𝑣1 , 𝑣2 , … , 𝑣𝑀 ) represent the initial environment
( ) state vector with 𝑣𝑖 representing the probability the initial environment state
𝑃𝑤𝑙 (𝑡, 𝒙) = 𝑃 𝒐𝑡 = (𝑥1 , 𝑥2 , … , 𝑥𝑁 ), 𝑊𝑡 = 𝑙 ∣ 𝒐0 = 𝟎, 𝑊0 = 𝑤 . (1) is 𝑖, 𝑖 ∈ 𝑆𝑙 . The system reliability function 𝑅(𝑡; 𝒗) can be given as
∑
Define 𝑷 (𝑡, 𝒙) as the matrix corresponding to 𝑃𝑤𝑙 (𝑡, 𝒙) which is a 𝑅(𝑡; 𝒗) = 𝒗 𝑷 (𝑡, 𝒚)𝒆. (6)
|𝑀 + 1| × |𝑀 + 1| matrix with the (𝑤, 𝑙)𝑡ℎ element as 𝑃𝑤𝑙 (𝑡, 𝒙). Then 𝒚∈𝑆𝑔
the expression of 𝑷 (𝑡, 𝒙) can be derived with the following lemma.
Given the initial degradation state level 𝒙 with 𝑥𝑖 < 𝐿𝑖 , ∀𝑖 ∈ 𝛷, the
Lemma 4.1. Let 𝜦(𝑡) be a diagonal matrix with the (𝑖, 𝑖)th element given as conditional reliability function 𝑅(𝑡; 𝒙) can be derived as follows.
∑ ∑
𝛬𝑖𝑖 (𝑡) = 𝑙∈𝛷 𝜆𝑙 (𝑡, 𝑖). When 𝒙 = 𝟎, meaning that no degradation increment 𝑅(𝑡; 𝒙, 𝒗) = 𝒗 𝑷 (𝑡, 𝒛 − 𝒙)𝒆, (7)
occurs to each component, 𝑷 (𝑡, 𝟎) satisfies: 𝒛∈𝑆𝑔
d𝑷 (𝑡, 𝟎) ( )
= 𝑷 (𝑡, 𝟎) 𝑸 − 𝜦(𝑡) , (2) where 𝒆 is a column vector where each entry is 1, 𝑷 (𝑡, 𝒚) has been presented
d𝑡 in Lemmas 4.1–4.3. 𝑷 (𝑡, 𝒛 − 𝒙) = 𝟎 if 𝑧𝑖 < 𝑥𝑖 for some 𝑖 ∈ 𝛷.
where 𝑸 is the infinitesimal generator of the CTMC. The initial value is
𝑷 (0, 𝟎) = 𝑰 which is an identity matrix. In particular, when 𝜦(𝑡) = 𝜦, 4.2. Maintenance analysis
meaning that the degradation increment of each component under given
environment follows a homogeneous Poisson process, then 𝑷 (𝑡, 𝟎) can be For simplicity in maintenance analysis, it is assumed that the Pois-
further expressed by son distribution parameter is independent of the time 𝑡, 𝑖.𝑒., for all
( ) 𝑖 ∈ 𝛷, 𝜆𝑖 (𝑡, 𝑤) = 𝜆𝑖 (𝑤), which is independent of 𝑡 and related only
𝑷 (𝑡, 𝟎) = exp (𝑸 − 𝜦)𝑡 . (3)
to the working environment in the following analysis. In this study,
The detailed proof is given in Appendix A.1. we consider two scenarios to model the effect of maintenance actions
To calculate 𝑷 (𝑡, 𝒙) in a general case, let us define 𝒙𝑖− = (𝑥1 , 𝑥2 , … , on the environment. The first one is that when the system is renewed,
𝑥𝑖−1 , 𝑥𝑖 − 1, 𝑥𝑖+1 , … , 𝑥𝑁 ) as the system degradation state where only the the working environment will also be restored to its initial friendly
degradation of the 𝑖th component is 1 state less compared with 𝒙, while state. The other scenario is that the environment is independent of
the other component state are identical as 𝒙. Then the following lemma maintenance actions, such as the physical weather. Both of the two
can be concluded.
cases are studies in the subsequent maintenance modelling. We utilize
the expected discounted total cost in the long-run horizon to assess
Lemma 4.2. For system state 𝒙 with 𝑥𝑖 < 𝐿𝑖 , ∀𝑖 ∈ 𝛷, 𝑷 (𝑡, 𝒙) satisfies
the maintenance performance. Define exp(−𝛾𝑡) as the discounted factor,
d𝑷 (𝑡, 𝒙) ( ) ∑ where 𝛾 ≥ 0. The long-run discount total inspection cost is given as:
= 𝑷 (𝑡, 𝒙) 𝑸 − 𝜦(𝑡) + 𝑷 (𝑡, 𝒙𝑖− )𝜦𝑖 (𝑡), (4)
d𝑡 𝑖∈𝛷 ∑
∞
𝑐𝐼
where 𝑸 is the infinitesimal generator of the CTMC. 𝜦𝑖 (𝑡) is a diagonal 𝐶𝑖𝑛𝑠 (𝜏) = 𝑐𝐼 exp(−𝛾𝑘𝜏) = . (8)
𝑘=0
1 − exp(−𝛾𝜏)
matrix with the (𝑗, 𝑗)th element given as 𝜆𝑖 (𝑡, 𝑗), 𝑖 ∈ 𝛷, 𝑗 ∈ 𝑆𝑙 , and
𝑷 (𝑡, 𝒙𝑖− ) = 𝟎 if 𝑥𝑖 = 0. The initial value is 𝑷 (0, 𝒙) = 𝟎. Since inspection is implemented at every 𝜏 intervals, the total dis-
counted cost can be obtained by summating the inspection component
The detailed proof is provided in Appendix A.2. We have studied
𝐶𝑖𝑛𝑠 (𝜏) and the maintenance cost at each inspection epoch. In the
the scenario that no failure at component level occurs. For the scenarios
following, considering renewable and non-renewable environmental
that failure occurs at the component level, the following lemma can be
affects upon maintenance respectively, we will formulate the mainte-
used to derive 𝑷 (𝑡, 𝒙).
nance problem into a Markov decision process (MDP) framework and
value iteration algorithm will be used to calculate the maintenance
Lemma 4.3. For a non-failed system state 𝒙, where failures occur at the
cost.
component level, i.e. 𝑥𝑖 = 𝐿𝑖 for some 𝑖 ∈ 𝛷, then 𝑷 (𝑡, 𝒙) can be expressed
as:
( ∑ ) ∑ 4.2.1. Maintenance policy for renewable environment state
d𝑷 (𝑡, 𝒙)
= 𝑷 (𝑡, 𝒙) 𝑸 − 𝜦𝑖 (𝑡) + 𝑷 (𝑡, 𝒙𝑖− )𝜦𝑖 (𝑡), (5) If the environment state is renewable, when the system is replaced
d𝑡 𝑖∈𝛷,𝑥 ≠𝐿
𝑖 𝑖 𝑖∈𝛷 to the as-good-as-new state, the environment is restored to its initial
where 𝑸 is the infinitesimal generator of the CTMC. 𝜦𝑖 (𝑡) is a diagonal friendly state. Define 𝑆𝑑 as the set of the system degradation vectors,
matrix with the (𝑗, 𝑗)th element given as 𝜆𝑖 (𝑡, 𝑗), 𝑖 ∈ 𝛷, 𝑗 ∈ 𝑆𝑙 , and 𝑆𝑔(1) as the set of the system degradation states where all compo-
𝑷 (𝑡, 𝒙𝑖− ) = 𝟎 if 𝑥𝑖 = 0. 𝒙𝑖− = (𝑥1 , 𝑥2 , … , 𝑥𝑖−1 , 𝑥𝑖 − 1, 𝑥𝑖+1 , … , 𝑥𝑁 ). The nents are in their non-failed states, and 𝑆𝑔(2) as the set of the system
initial value is 𝑷 (0, 𝒙) = 𝟎. degradation states where the system is functioning but failures at the

4
N. Zhang et al. Reliability Engineering and System Safety 231 (2023) 108988

component level exist. Then 𝑆𝑔 = 𝑆𝑔(1) ∪ 𝑆𝑔(2) represents the set of the 𝑈𝜏 (𝒙, 𝑤) can be derived as follows.
system degradation states denoting the system is functioning. 𝑆𝑑 ∖𝑆𝑔 ∑ ( )
𝑈𝜏 (𝒙, 𝑤) = 𝜅𝜏 (𝒙, 𝑤), (𝒚, 𝑙) 𝑉𝜏 (𝒚, 𝑙)
represents the system degradation states when the system fails.
𝑦∈𝑆𝑔
Denote 𝑉𝜏 (𝒙, 𝑤) as the expected total discounted maintenance cost
given the initial system degradation state 𝒙 and the environment state + (1 − 𝑅(𝜏, 𝒙, 𝑣(𝑤)))(𝑐𝑠 + 𝑐𝑓 + 𝑉𝜏 (𝟎, 0)), (13)
𝑤. Hence, the optimal inspection interval can be searched via
where 𝑆𝑔 is the set of the system degradation vectors indicating that the
𝜏 ∗ = arg min[𝐶𝑖𝑛𝑠 (𝜏) + 𝑉𝜏 (𝟎, 0)]. system is functioning, 𝜅𝜏 is the transition probability given by Eq. (12),
𝜏 𝑅 is the reliability function given in Theorem 4.1, 𝑐𝑠 is the maintenance
The Bellman equation of 𝑉𝜏 (𝒙, 𝑤) can be expressed as set-up cost, and 𝑐𝑓 is the system replacement cost.
Based on the above Bellman equation, some structural properties of
𝑉𝜏 (𝒙, 𝑤)
𝑉𝜏 can be obtained.
⎧
⎪𝑐 𝑠 + {
𝑐𝑓 + 𝑉𝜏 (𝟎, 0),
}
if 𝒙 ∈ 𝑆𝑑 ∖𝑆𝑔 ,
⎪ Lemma 4.4. 𝑉𝜏 (𝒙, 𝑤) is a non-decreasing function in 𝑥𝑖 , ∀𝑖 ∈ 𝛷, where
⎪min 𝑐𝑠 + 𝐶(𝒙, 𝒚) + 𝑉𝜏 (𝒚, 𝑤I{𝒚≠𝟎} ), ∀𝒚 ∈ 𝑀(𝒙) , if 𝒙 ∈ 𝑆𝑔(2) , 𝑥𝑖 is the 𝑖th element of 𝒙.
⎪ {
=⎨ −𝛾𝜏 The proof is provided in Appendix B.1.
⎪min 𝐷𝜏 (𝒙, 𝑤) + 𝑒 𝑈𝜏 (𝒙, 𝑤),
⎪ } Lemma 4.4 indicates that the expected total maintenance cost in the
⎪ 𝑐𝑠 + 𝐶(𝒙, 𝒚) + 𝑉𝜏 (𝒚, 𝑤I{𝒚≠𝟎} ), ∀𝒚 ∈ 𝑀(𝒙) , if 𝒙 ∈ 𝑆𝑔(1) , long-run horizon increases with the initial degradation states. With the
⎪
⎩ monotonicity property of the value function, we can have the following
(9) theorem that provides insights for the optimal maintenance actions.

where 𝑐𝐼 is the inspection cost, 𝑐𝑠 is the set-up cost, 𝑐𝑓 is the sys-

Theorem 4.2. For the system with state (𝒙, 𝑤) and inspection interval 𝜏,
tem replacement cost. 𝐶(𝒙, 𝒚) is the maintenance cost such that the
we have the following statements:
degradation vector transfers from 𝒙 to 𝒚, expressed as
∑ ∑ • Given (𝒙, 𝑤), if component 𝑖 is replaced in the optimal policy, then for
𝐶(𝒙, 𝒚) = 𝑐𝑟𝑖 + 𝑐𝑝𝑗 I{𝑦𝑗 =0} , (10)
systems with state (𝒙𝑖+ , 𝑤), component 𝑖 should also replaced.
𝑖∈𝑠𝑟 (𝒙) 𝑗∈𝛷∖𝑠𝑟 (𝒙)
• Given (𝒙, 𝑤), if component 𝑖 has not been replaced in the optimal
where 𝑦𝑗 is the 𝑗th element in 𝒚. 𝑐𝑝⋅ and 𝑐𝑟⋅ are respectively the pre- policy, then for systems with state (𝒙𝑖− , 𝑤), component 𝑖 should also
ventive and corrective maintenance cost of the considered component. remain unchanged.
I{⋅} is the indicator function. 𝑠𝑟 (𝒙) is the set of failed components
given the degradation observation 𝒙. 𝑀(𝒙) contains all the possible where 𝒙𝑖− = (𝑥1 , 𝑥2 , … , 𝑥𝑖−1 , 𝑥(−)
𝑖 , 𝑥𝑖+1 , … , 𝑥𝑁 ) is the system degradation
degradation states after the corresponding maintenance action given state where only the degradation of the 𝑖th component is less comparing to
the degradation vector 𝒙. 𝒙, and 𝒙𝑖+ = (𝑥1 , 𝑥2 , … , 𝑥𝑖−1 , 𝑥+
𝑖 , 𝑥𝑖+1 , … , 𝑥𝑁 ) is the system degradation
The Bellman equation can be interpreted as follows. Given the state where only the degradation of the 𝑖th component is larger comparing
degradation vector 𝒙, the state of each component and the operating to 𝒙.
environment can be perfectly revealed. The decision can thus be made The proof of Theorem 4.2 is provided in Appendix B.2. A special
based on 𝒙. If 𝒙 indicates that the system enters into the failed state, case is that when all the components follow an identical Poisson distri-
then it is renewed with cost 𝑐𝑓 after which the system returns to the bution for the internal degradation process, i.e. 𝜆𝑖 (𝑤) = 𝜆𝑗 (𝑤), ∀𝑖, 𝑗 ∈ 𝛷,
as-good-as-new state 𝟎 and the environment state is restored to the most 𝑤 ∈ 𝑆𝑙 , the optimal maintenance policy can be further simplified.
friendly scenario. If the system is functioning but failure occurs at the Obviously, when the component characteristics and the maintenance
component level, i.e. 𝑥 ∈ 𝑆𝑔(2) , then the failed ones should be replaced, unitary costs are identical to all the components, it is unnecessary
preventive maintenance may also be implemented upon the functioning to distinguish the components. In this case, if components are to be
components. When no component fails upon inspection, the decision- replaced, it is always optimal to replace the components with the
maker can choose to do the preventive maintenance at component level largest degradation states. In addition, the following corollary shows
or system level, or do nothing and wait for the next inspection epoch. the properties of the optimal maintenance policy.
Under the scenario that do nothing and wait for the next decision
epoch, the cost consists of the expected discounted downtime cost until Corollary 4.1. If all the components follow an identical Poisson distri-
the next inspection 𝐷𝜏 (𝒙, 𝑤) and the expected value function of the next bution for the internal degradation process, 𝑖.𝑒., 𝜆𝑖 (𝑤) = 𝜆𝑗 (𝑤), ∀𝑖, 𝑗 ∈ 𝛷,
inspection epoch 𝑈𝜏 (𝒙, 𝑤). The expected discounted downtime can be 𝑤 ∈ 𝑆𝑙 , and the maintenance costs are identical with 𝑐𝑝𝑖 = 𝑐𝑝 < 𝑐𝑟 = 𝑐𝑟𝑖
calculated as follows. for any 𝑖 ∈ 𝛷. Given the environment state 𝑤, the inspection interval 𝜏, and
𝜏
𝑒−𝛾𝜏 − 𝑒−𝛾𝑡 initial system degradation state 𝒙 with corresponding ordered degradation
𝐷𝜏 (𝒙, 𝑤) = 𝑐𝑑 d𝑅(𝑡; 𝒙, 𝒗(𝑤)) (11)
∫0 𝛾 ̃ where 𝑥̃ 𝑖 is the 𝑖th largest element in 𝒙, it follows
𝒙,
where 𝑐𝑑 is the downtime cost per unitary time. 𝑅(𝑡; 𝒙, 𝒗(𝑤)) is the con-
• There exist a boundary 𝑙1∗ (𝑥̃ 2 , 𝑥̃ 3 , … , 𝑥̃ 𝑛 ) between ‘‘Do nothing and
ditional reliability function which has been presented in Theorem 4.1.
wait for the next decision epoch’’ and ‘‘replace 1 component’’. When
We utilize 𝒗(𝑤) is the row vector that the 𝑤th element is 1 and others
𝑥̃ 1 < 𝑙1∗ (𝑥̃ 2 , 𝑥̃ 3 , … , 𝑥̃ 𝑛 ), ‘‘do nothing’’ is better, otherwise ‘‘replace 1
are 0, and 𝜏 is the inspection period.
component’’ is better.
To obtain 𝑈𝜏 (𝒙, 𝑤), the transition probabilities of the system states
( ) • For 𝑖 = 2, 3, … , 𝑛 − 1, there exist a boundary 𝑙𝑖 (𝑥̃ 𝑖+1 , 𝑥̃ 𝑖+2 , … , 𝑥̃ 𝑛 )
should be calculated first. Define 𝜅𝜏 (𝒙, 𝑤), (𝒚, 𝑙) as the probability that
between ‘‘replace 𝑖 − 1 components ’’ and ‘‘replace 𝑖 components’’.
the system degradation state transfers to 𝒚 from 𝒙 and the environment
When 𝑥̃ 𝑖 < 𝑙𝑖 (𝑥̃ 𝑖+1 , 𝑥̃ 𝑖+2 , … , 𝑥̃ 𝑛 ), ‘‘replace 𝑖 − 1 components’’ is better,
transfers from 𝑤 to 𝑙 between two consecutive inspection epoch when
otherwise, ‘‘replace 𝑖 components’’ is better.
the inspection interval is 𝜏. Then for 𝑦 ∈ 𝑆𝑔 , it follows
( ) ∑ Corollary 4.1 can be readily obtained based on Theorem 4.2, and
𝜅𝜏 (𝒙, 𝑤), (𝒚, 𝑙) = 𝒗(𝑤) 𝑷 (𝑡, 𝒛 − 𝒙)𝒆(𝑙), (12) therefore the detailed proof is omitted to avoid repetition. However,
𝒛∈𝑆𝑔
it is difficult to develop theoretical properties with respect to the
𝑷 can be derived from Lemmas 4.1–4.3. 𝒆(𝑙) is the column vector where environment state and the inspection interval. We intend to evaluate
the 𝑙th element is 1 and others are 0. Based on the transition kernel, the influence of the two factors numerically.

5
N. Zhang et al. Reliability Engineering and System Safety 231 (2023) 108988

Algorithm 1: The value iteration algorithm

Input : Component set 𝛷, failure threshold 𝐿𝑖 𝑖 ∈ 𝛷,
environment state set 𝑆𝑙 , infinitesimal generator 𝑸,
Poisson distribution parameters 𝜆𝑖 (𝑤), Sets 𝑺, 𝑆𝑑 , 𝑆𝑔 ,
𝑆𝑔(1) , 𝑆𝑔(2) , 𝑀(𝒙), unitary costs, inspection period 𝜏,
discount 𝛾, discretize step 𝛿, precision 𝜖.
Compute 𝑷 (𝑖𝛿, 𝒙) using Eqs. (A.7) and (A.9) and 𝐷𝜏 (𝒙, 𝑤);
Compute the transition probability 𝜅𝜏 in Eq. (12) and 𝐶(𝒙, 𝒚) in
Eq. (10);
Set 𝑉𝜏(0) (𝒙, 𝑤) = 0, ∀(𝒙, 𝑤) ∈ 𝑺;
while |𝑉𝜏(𝑚) (𝒙, 𝑤) − 𝑉𝜏(𝑚−1) (𝒙, 𝑤)| > 𝜖, for some (𝒙, 𝑤) ∈ 𝑆𝑑 , do
for (𝒙, 𝑤) ∈ 𝑆𝑑 , do
if (𝒙, 𝑤) ∈ 𝑆𝑑 ∖𝑆𝑔 , then
calculate 𝑉𝜏(𝑚) (𝒙, 𝑤) = 𝑐𝑠 + 𝑐𝑓 + 𝑉𝜏(𝑚−1) (𝟎, 0)
else if (𝒙, 𝑤) ∈ 𝑆𝑔(2) , then
𝑉𝜏(𝑚) (𝒙, 𝑤) =
{ }
Fig. 1. System reliability 𝑅(𝑡; 𝒗) with various initial environment states 𝒗. min 𝑐𝑠 + 𝐶(𝒙, 𝒚) + 𝑉𝜏(𝑚−1) (𝒚, 𝑤I{𝒚≠𝟎} ), ∀𝒚 ∈ 𝑀(𝒙)
else
∑
calculate 𝑈𝜏(𝑚) (𝒙, 𝑤) = 𝜅(𝒙, 𝑤, 𝒚, 𝑙)𝑉𝜏(𝑚−1) (𝒚, 𝑙);
(𝒚,𝑙)∈𝑺
4.2.2. Maintenance policy for non-renewable environment states
calculate
In this section, we consider the case that the environment state {
𝑉𝜏(𝑚) (𝒙, 𝑤) = min 𝐷𝜏 (𝒙, 𝑤) + 𝑒−𝛾𝜏 𝑈𝜏(𝑚) (𝒙, 𝑤), 𝑐𝑠 +
is non-renewable, meaning that if the system is in the failed states, }
then it is renewed to state 𝟎 as discussed in Section 4.2.1, while 𝐶(𝒙, 𝒚) + 𝑉𝜏(𝑚−1) (𝒚, 𝑤I{𝒚≠𝟎} ), ∀𝒚 ∈ 𝑀(𝒙)
the environment state remains unchanged. Other assumptions are as end
identical as those in Section 4.2.1. Likewise, the Bellman equation of end
the value function 𝑉̂𝜏 (𝒙, 𝑤) is given as follows. end
Output: For each (𝒙, 𝑤) ∈ 𝑆𝑑 , 𝑉𝜏 (𝒙, 𝑤) = 𝑉𝜏(𝑚) (𝒙, 𝑤), ∀(𝒙, 𝑤) ∈ 𝑆𝑑
𝑉̂𝜏 (𝒙, 𝑤)
and the corresponding maintenance policy.
⎧
𝑐𝑓 + 𝑉̂𝜏 (𝟎, 𝑤),
⎪𝑐 𝑠 + { if 𝒙 ∈ 𝑆𝑑 ∖𝑆𝑔 ,
}
⎪
⎪min 𝑐𝑠 + 𝐶(𝒙, 𝒚) + 𝑉̂𝜏 (𝒚, 𝑤), ∀𝒚 ∈ 𝑀(𝒙) , if 𝒙 ∈ 𝑆𝑔(2) ,
⎪ { the degradations of chock valves depend on the oil and gas flows in
=⎨ −𝛾𝜏
(14)
⎪min 𝐷𝜏 (𝒙, 𝑤) + 𝑒 𝑈𝜏 (𝒙, 𝑤), the subsea production system. The weather condition, pressure and
⎪ } humidity play important roles in aircraft security. This model can help
⎪ 𝑐𝑠 + 𝐶(𝒙, 𝒚) + 𝑉̂𝜏 (𝒚, 𝑤), ∀𝒚 ∈ 𝑀(𝒙) , if 𝒙 ∈ 𝑆𝑔(1) ,
⎪ the decision-maker to develop the inspection and maintenance policy,
⎩ and to estimate the maintenance cost which can help to control opera-
where 𝐶(𝒙, 𝒚) a has been given in Eq. (10), 𝑀(𝒙) contains all the tion and maintenance budget in practice. In this section, we present
possible degradation states after the corresponding maintenance action a numerical example to illustrate the applicability of the proposed
given the degradation vector 𝒙. 𝑠𝑟 (𝒙) is the set of failed components model. The following results are obtained by utilizing Matlab2018b
given the degradation observation 𝒙. It can be observed that the only on a Windows 8 Core 64-Bits operating system.
difference between Eqs. (9) and (14) lies in the effect of maintenance
actions on the environmental states. In Section 4.2.1, when the envi- 5.1. Reliability analysis
ronment is renewable, environment state is renewed to 𝟎 together with
renewal of the system states. Otherwise, the environment remain un- Consider a 3-component system in series that operates in a dynamic
changed. When the environment is non-renewable, environment state environment. Assume that the environment states can be classified into
is always remain unchanged whatever the maintenance actions on the there types, for instance, it can been classified as ‘‘friendly’’, ‘‘mild’’ and
‘‘severe’’ according to its impact on the system. The sojourn time in the
system (14).
friendly state is exponentially distributed with mean 1/3. Whenever
Similar to Lemma 4.4, it can be readily proved that 𝑉̂ (𝒙, 𝑤) is a
the environment change occurs, it will transfer to the ‘‘mild’’ state
non-decreasing function in 𝑥𝑖 when other elements in 𝒙 and 𝑤 remain
with probability 1/3 and to the ‘‘severe’’ state with probability 2/3.
unchanged. Unfortunately, it is difficult to obtain the monotonicity
Similarly, the transition properties of the environment with respect to
properties of the optimal maintenance policy due to the non-renewal
the other states can be defined. We have the following infinitesimal
environmental affect. The properties of the maintenance policies will be
generator to represent the transition properties.
examined numerically in the following section. To further illustrate the
applicability of the proposed model, A detailed numerical examples are ⎡−3 1 2⎤
presented in Section 5. The value iteration algorithm is implemented 𝑸=⎢1 −2 1 ⎥. (15)
⎢ ⎥
to calculate the value function 𝑉𝜏 where details are illustrated in ⎣1 3 −4⎦
Algorithm 1. Similarly, 𝑉̂𝜏 can be obtained numerically. Assume that under a harsher environment state, the failure rate of each
component becomes larger. The following matrix presents the failure
5. A numerical example rate of each component under each environment state:

So far, we have evaluated the system reliability and maintenance of ⎡0.6 0.7 0.8⎤
𝜦 = ⎢0.6 0.65 0.7⎥ , (16)
a degrading system under a dynamic environment, where the system ⎢ ⎥
consists of multiple heterogeneous components. For many mechanical ⎣0.7 0.8 0.9⎦
systems, they usually operate in a changing environment where the where the (𝑖, 𝑗)th entry is the parameter of the Poisson distribution that
system degradation is highly related to the environment. For instance, is utilized to describe the degradation increment of component 𝑗 under

6
N. Zhang et al. Reliability Engineering and System Safety 231 (2023) 108988

Fig. 2. System reliability 𝑅(𝑡; 𝒙, 𝒗) with various initial system degradation states 𝒙. Fig. 4. Total expected maintenance cost under various inspection intervals.

𝑐𝑝𝑖 = 1 + 𝑖, 𝑐𝑟𝑖 = 𝑐𝑝𝑖 + 2. The discretize step is 0.01 and the precision is
𝜖 = 10−6 .

5.2.1. Maintenance under renewable and non-renewable environment

Fig. 4 shows the variation of the total expected maintenance cost
under various inspection intervals when the environment is renewable.
It can be observed that the total expected cost is a unimodal function
with respect to the inspection interval in the considered region. The
optimum achieves at 𝜏 = 1 and the corresponding cost is 54.18.
Table 1 shows the variations of maintenance costs and the cor-
responding maintenance actions under renewable and non-renewable
environment given 𝜏 = 1. The value function 𝑉𝜏 corresponds to the
scenario with renewable environment and 𝑉̂𝜏 is associated with the non-
renewable environment states. ‘‘DN’’ means do nothing and wait for the
next decision epoch. ‘‘RE+component number’’ indicates the considered
components are replaced. It can be observed that given 𝑤, 𝑉𝜏 and
𝑉̂𝜏 increase with 𝑥𝑖 , which verifies the statement of the monotonic
property of the value function under the two scenarios. For each 𝒙,
Fig. 3. System reliability 𝑅(𝑡; 𝒗) with various failure thresholds 𝑳. with renewal environment states, it seems that the 𝑉𝜏 increases with
the environment state. Part of the reason might be that with a lower
environment state, the probability of having degradation increment
environment state 𝑖. The failure threshold is 𝐿𝑖 = 2, 𝑖 ∈ 𝛷. Given is smaller as shown in 𝜦 from Eq. (16). In addition, we can observe
the initial environment state, by utilizing Theorem 4.1, the system that ‘‘RE12’’ is the optimal maintenance policy for system with initial
reliability function can be obtained. Fig. 1 shows the variation of the degradation (2, 3, 0) and (2, 4, 0), which coincides with Theorem 4.2. For
system reliability function with various initial environment states. It each given (𝒙, 𝑤), 𝑉̂𝜏 (𝒙, 𝑤) is always larger than 𝑉𝜏 (𝒙, 𝑤). This is due to
can be observed that the reliability shows a decreasing tendency under the fact that the system benefits more from the replacement when the
a harsher environmental state. As shown in 𝜦, for each component, the environment is renewable.
parameter of the Poisson distribution increases with the environment In the subsequent analysis, we will focus on the scenario that the
state, meaning that it is more easier to enter into the deteriorating or environment is renewable, while the analysis for non-renewable envi-
failure state under a harsher initial environment. ronment can be conducted in a similar way. To further investigate the
Fig. 2 shows the variation of the system reliability function with impact of the environment, we consider the case that the components
various initial system degradation states given the initial environmental are homogeneous with 𝜦 given as
state 𝒗 = [1, 0, 0]. Fig. 3 shows the influence of the system reliability
⎡0.6 0.6 0.6⎤
with respect to the system failure threshold 𝑳. As expected, when the
𝜦 = ⎢0.7 0.7 0.7⎥ , (17)
initial degradation level is higher or the failure threshold is smaller, the ⎢ ⎥
⎣0.8 0.8 0.8⎦
reliability function becomes smaller as the system inclines to enter into
the failure state. while other parameters setting are remain unchanged. The optimal
inspection interval is 𝜏 ∗ = 1.1 and the associated total expected cost
5.2. Illustration for maintenance models is 50.73. Focusing on the two scenarios where 𝑥3 = 0 and 𝑥3 = 4,
the corresponding maintenance policies are presented in Table 2. The
Consider the system described in the above section for maintenance (𝑖, 𝑗)th entry represents the maintenance policy given 𝑥1 = 𝑖 − 1 and
modelling. To further illustrate the effectiveness of the proposed model, 𝑥2 = 𝑗 − 1. It can be observed that for each degradation state 𝒙,
the failure threshold of component 𝑖 is changed to 𝐿𝑖 = 5, 𝑖 ∈ {1, 2, 3}. components are inclined to be replaced under environment 𝑤 = 3 than
The parameter corresponding to the discount factor is 𝛾 = 0.1. Assume under environment 𝑤 = 1. An interesting observation is that it may
the unitary costs are 𝑐𝑠 = 1, 𝑐𝐼 = 1.1, 𝑐𝑑 = 10, 𝑐𝑓 = 30 for 𝑖 ∈ {1, 2, 3}, be optimal to do maintenance with a lower initial system degradation

7
N. Zhang et al. Reliability Engineering and System Safety 231 (2023) 108988

Table 1
Variations of value functions and corresponding maintenance policies with 𝑥 and 𝑤 under renewable and non-renewable
environments respectively.
Environment Degradation Value function Maintenance Value function Maintenance
state 𝑤 state 𝑥 𝑉𝜏 (𝑥, 𝑤) policy 𝑉̂𝜏 (𝑥, 𝑤) policy
0 (0, 0, 0) 42.6159 DN 44.0840 DN
(0, 3, 0) 46.6159 RE2 48.0839 RE2
(2, 3, 0) 48.6159 RE12 50.0839 RE12
(2, 4, 0) 48.6159 RE12 50.0839 RE12
(0, 2, 3) 50.4292 RE3 51.9016 RE3
(1, 3, 3) 51.6190 RE23 53.0966 RE23
(2, 3, 3) 52.6159 RE123 54.0839 RE123
1 (0, 0, 0) 44.0321 DN 45.5709 DN
(0, 3, 0) 48.0320 RE2 49.5708 RE2
(2, 3, 0) 50.0320 RE12 51.5708 RE12
(2, 4, 0) 50.0320 RE12 51.5708 RE12
(0, 2, 3) 51.7348 RE3 53.3133 RE3
(1, 3, 3) 52.9907 RE23 54.5453 RE23
(2, 3, 3) 54.0320 RE123 55.5708 RE123
2 (0, 0, 0) 45.4052 DN 47.1440 DN
(0, 3, 0) 49.4051 RE2 51.1439 RE2
(2, 3, 0) 51.4051 RE12 53.1439 RE12
(2, 4, 0) 51.4051 RE12 53.1439 RE12
(0, 2, 3) 52.9811 RE3 54.7935 RE3
(1, 3, 3) 54.3187 RE23 56.0795 RE23
(2, 3, 3) 55.4051 RE123 57.1439 RE123

Table 2
Variations of the expected discounted maintenance costs with 𝑥 and 𝑤.
𝑤=1 𝑤=3
⎡ DN DN DN RE2 RE2 ⎤ ⎡ DN DN DN RE2 RE2 ⎤
⎢ DN DN DN RE2 RE2 ⎥ ⎢ DN DN DN RE2 RE2 ⎥
⎢ ⎥ ⎢ ⎥
𝑥3 = 0 ⎢ DN DN DN RE12 RE12⎥ ⎢ DN DN DN RE12 RE12⎥
⎢RE1 RE1 RE12 RE12 RE12⎥ ⎢RE1 RE1 RE1 RE12 RE12⎥
⎢ ⎥ ⎢ ⎥
⎣RE1 RE1 RE12 RE12 RE12⎦ ⎣RE1 RE1 RE1 RE12 RE12⎦
⎡ RE3 RE3 RE23 RE23 RE23 ⎤ ⎡ RE3 RE3 RE3 RE23 RE23 ⎤
⎢ RE3 RE3 RE3 RE23 RE23 ⎥ ⎢ RE3 RE3 RE3 RE23 RE23 ⎥
⎢ ⎥ ⎢ ⎥
𝑥3 = 4 ⎢RE13 RE13 RE123 RE123 RE123⎥ ⎢RE13 RE13 RE13 RE123 RE123⎥
⎢RE13 RE13 RE123 RE123 RE123⎥ ⎢RE13 RE13 RE13 RE123 RE123⎥
⎢ ⎥ ⎢ ⎥
⎣RE13 RE13 RE123 RE123 RE123⎦ ⎣RE13 RE13 RE13 RE123 RE123⎦

but do nothing with a higher degradation. For instance, under 𝑤 = 3, Table 3

components 2 and 3 are replaced when the initial system degradation Optimal maintenance costs and the corresponding inspection intervals.

is (0, 2, 4), while only component 3 is replaced when the initial system Optimal inspection & cost 𝑋𝑝 = 1 𝑋𝑝 = 2 𝑋𝑝 = 3 𝑋𝑝 = 4
degradation is (1, 2, 4). Optimal inspection 𝜏1∗ 2.1 1.0 0.8 6.2
Optimal cost 51.95 50.99 58.03 64.87
5.2.2. Comparison with other maintenance policies
To further illustrate the advantages of the proposed maintenance
policy, we compare it with two commonly used maintenance policies.
Impact of components’ degradation. When 𝜦 is changed to We change 𝜦
The first one is the degradation-threshold policy, where a common
threshold governs the maintenance of all components: each component to
is preventively replaced if its deterioration level exceeds the threshold ⎡0.7 0.7 0.7⎤
𝑋𝑝 and the system is renewed upon failure. Table 3 shows the optimal ̃ = ⎢0.8
𝜦 0.8 0.8⎥ , (18)
⎢ ⎥
total expected cost and the associated inspection interval with different ⎣0.9 0.9 0.9⎦
maintenance thresholds. It is observed that 𝑋𝑝 = 2 is the optimal
the optimum cost arrives at 𝜏 ∗ = 0.8. and the associated total expected
threshold under which the optimal cost is 50.99, which is larger than
50.73 — the value obtained by our proposed model. cost is 60.72. It can be observed that when the failure probability is
The second one is based on the working state of each compo- higher, the maintenance cost is larger and the decision-maker prefers to
nent: upon inspection, replace the failed components and do nothing inspect the system more frequently, with a smaller inspection interval
to the non-failed components. Hence, no PM actions is considered. 𝜏.
Under this policy, the optimal inspection and maintenance cost is
Impact of environmental variations. To consider the impact of the work-
64.87 with inspection period 𝜏2∗ = 6.2, which is equal to the first
ing environment, let us consider the case where the infinitesimal gen-
degradation-threshold policy where the PM threshold is 4. Given the
initial environment 𝒗 = [1, 0, 0], the system reliabilities are 𝑅(𝜏 ∗ ; 𝒗) = 1, erator is given as
𝑅(𝜏2∗ ; 𝒗) = 0.14. It is seen that the system is highly likely to be main- ⎡ −3 0.5 2.5⎤
tained at failure under the second maintenance policy. Our proposed ̃ = ⎢0.5
𝑸 −2 1.5⎥ . (19)
model is more reliable and economical. ⎢ ⎥
⎣1 3 −4 ⎦

5.2.3. Sensitivity analysis The optimum cost arrives at 𝜏 ∗ = 1.1 and the associated total
In the section, sensitivity analysis is presented to investigate the expected cost is 51.62. which is larger than the value under 𝑸. This is
impact of parameter values on the optimal maintenance policy. ̃ when environment state transition occurs,
due to the fact that with 𝑸,

8
N. Zhang et al. Reliability Engineering and System Safety 231 (2023) 108988

Table 4 CRediT authorship contribution statement

Optimal maintenance costs and the corresponding inspection intervals with various 𝑐𝑝
and 𝑐𝑠 .
Cost units 𝑐𝑝 = [1.5, 2.5, 3.5] 𝑐𝑝 = [2.5, 3.5, 4.5] 𝑐𝑠 = 2 𝑐𝑠 = 0.5
Nan Zhang: Writing – original draft, Software, Methodology, In-
vestigation, Conceptualization. Yingjun Deng: Writing – original draft,
Optimal cost 48.38 52.95 54.25 48.83
Optimal inspection 1.1 1.2 1.3 1.1 Methodology, Conceptualization. Bin Liu: Writing – original draft, Su-
pervision, Methodology. Jun Zhang: Supervision, Methodology, Con-
ceptualization.
Table 5
Optimal maintenance costs and the corresponding inspection intervals with different
cost units. Declaration of competing interest
Cost units 𝑐𝐼 = 1.0 𝑐𝐼 = 1.1 𝑐𝐼 = 1.2 𝑐𝐼 = 1.3 𝑐𝑑 = 12 𝑐𝑓 = 32
Optimal cost 49.75 50.73 51.63 52.46 50.32 51.01 The authors declare that they have no known competing finan-
Optimal inspection 1.0 1.0 1.2 1.3 1.0 1.0 cial interests or personal relationships that could have appeared to
influence the work reported in this paper.

the environment is inclined to enter into a worse state, under which Data availability
the probability of having degradation increments is larger, thus a larger
cost is expected. Data will be made available on request.
Impact of cost parameters. In the part, we examine the influence of
the cost parameters on the optimal maintenance policy: the optimal Acknowledgements
inspection interval and maintenance cost. Tables 4 and 5 show the
variation of the optimal maintenance cost and the associated inspection This work is supported by the National Natural Science Foundation
interval with different cost parameters. The 𝑖th element in 𝒄 𝑝 is 𝑐𝑝𝑖 , of China (No. 71901026).
for 𝑖 = 1, 2, 3. As expected, the maintenance cost increases with the
cost parameters, and the optimal inspection interval increases with the
Appendix A. Proofs for system reliability
inspection cost.

6. Conclusions A.1. The proof of Lemma 4.1

In this study, we have developed a condition-based maintenance Proof. Define 𝒐𝑡 = (𝑥1 , 𝑥2 , … , 𝑥𝑁 ) as the realization of the system
model for a multi-unit system with heterogeneous components that degradation state at time 𝑡 where 𝑥𝑖 represents the degradation state of
operates in a dynamic environment. The environment evolution is component 𝑖, 𝑖 ∈ 𝛷, 𝑥𝑖 ∈ {0, 1, … , 𝐿𝑖 }. Let 𝑃𝑤𝑙 (𝑡+𝛥, 𝟎) be the probability
modelled by a continuous-time Markov chain. Given the environment that the system is in state 𝟎 and the environment state is 𝑙 by time 𝑡 + 𝛥
states, the degradation increment of each component is assumed to given the initial degradation state 𝟎 and the environment state 𝑤. Then
follow a specific Poisson distribution. The reliability and conditional we have that
reliability functions of the system are derived considering the inter- ∑
nal degradation process and the environmental effects. Two scenarios 𝑃𝑤𝑙 (𝑡 + 𝛥, 𝟎) = 𝑃𝑤𝑚 (𝑡, 𝟎)P(𝒐𝑡+𝛥 = 𝟎, 𝑊𝑡+𝛥 = 𝑙 ∣ 𝒐𝑡 = 𝟎, 𝑊𝑡 = 𝑚)
𝑚∈𝑆𝑙
considering whether the environment can be renewed or not are stud-
∑ ( ∑ 𝑡+𝛥 )
ied when developing maintenance policy. Structural properties of the = 𝑃𝑤𝑚 (𝑡, 𝟎)𝜋𝑚𝑙 (𝛥) exp − 𝜆𝑖 (𝜃, 𝑚)d𝜃 , (A.1)
maintenance policy given the inspection interval and the environment ∫𝑡
𝑚∈𝑆𝑙 𝑖∈𝛷
state are investigated in detail and further incorporated into the value
where 𝜋𝑚𝑙 (𝛥) is the transition probability from state 𝑚 to 𝑙 during time
iteration algorithm to solve the MDP model. We have also examined the
𝛥, which satisfies
influence of the working environment and other parameters settings on {
the optimal maintenance decisions. It has shown that under a harsher 1 + 𝑞𝑚𝑚 𝛥 if 𝑚 = 𝑙,
initial environment, the maintenance cost is expected to be larger. The 𝜋𝑚𝑙 (𝛥) ≈ (A.2)
𝑞𝑚𝑙 𝛥 if 𝑚 ≠ 𝑙.
proposed policy has also been compared with the classical CBM models
with constant preventive thresholds, under which each component is Hence, Eq. (A.1) can be further expressed as:
replaced whenever its degradation level exceeds a predetermined value. ( ∑ )( ∑ )
𝑃𝑤𝑙 (𝑡 + 𝛥, 𝟎) = 𝑃𝑤𝑙 (𝑡, 𝟎) + 𝑃𝑤𝑚 (𝑡, 𝟎)𝑞𝑚𝑙 𝛥 1 − 𝜆𝑖 (𝑡, 𝑚)𝛥 ,
It turns out that our policy is more cost-effective in presenting system 𝑚∈𝑆𝑙 𝑖∈𝛷
failures. This work could be further extended in the following three
aspects. which yields to
First, we have assumed that components are mutual independent d𝑃𝑤𝑙 (𝑡, 𝟎) ∑ ∑
= −𝑃𝑤𝑙 (𝑡, 𝟎) 𝜆𝑖 (𝑡, 𝑚) + 𝑃𝑤𝑚 (𝑡, 𝟎)𝑞𝑚𝑙 .
given the working environment. It would be more realistic to con- d𝑡 𝑖∈𝛷 𝑚∈𝑆 𝑙
sider dependencies among components such as the failure dependence,
structural dependence, spare limitations, 𝑒𝑡𝑐. Writing in the matrix form, we have
Secondly, we have assumed that both the environment state and the d𝑷 (𝑡 + 𝛥, 𝟎)
= 𝑷 (𝑡, 𝟎)(𝑸 − 𝜦(𝑡)), (A.3)
degradation state of each component are observable. When the system d𝑡
is partially observable, inspection and maintenance decisions based on where 𝑸 is the infinitesimal generator and 𝜦(𝑡) is a diagonal matrix
imperfect information is more challenging and interesting. ∑
with the (𝑗, 𝑗)𝑡ℎ element given by 𝛬𝑗𝑗 (𝑡) = 𝑖∈𝛷 𝜆𝑗 (𝑡, 𝑖), 𝑗 ∈ 𝑆𝑙 . In
Thirdly, we have supposed that the maintenance actions are per- general, the exact solution of Eq. (A.3) is intractable. A special case
fect at the component level, 𝑖.𝑒., the state of each component is re- is that when 𝜦(𝑡) is independent of 𝑡, i.e. 𝜦(𝑡) = 𝜦, meaning that
stored to the as-good-as-new state after maintenance. Imperfect mainte- the changement of the degradation under given environment state is
nance can be an interesting and challenging topic in decision-makings constant, then 𝑷 (𝑡, 𝟎) can be given by
for multi-component systems, at the cost of a heavier computational
( )
burden. 𝑷 (𝑡, 𝟎) = exp (𝑸 − 𝜦)𝑡 . ⋄

9
N. Zhang et al. Reliability Engineering and System Safety 231 (2023) 108988

A.2. The proof of Lemma 4.2 Proof. We will prove the monotonicity of 𝑉 by mathematical induc-
tion. At the first iteration, set 𝑉𝜏 (𝒙, 𝑤) = 0 for all system degradation
Proof. Let 𝑃𝑤𝑙 (𝑡 + 𝛥, 𝒙) be the probability that the system is in state 𝒙 state 𝒙 and environment state 𝑤.
and the environment state is 𝑙 by time 𝑡+𝛥 given the initial degradation Assume that 𝑉𝜏 (𝒙, 𝑤) is a non-decreasing function in 𝑥𝑖 at the 𝑛th
state 𝟎 and the environment state 𝑤. It is seen that iteration where 𝑥𝑖 is the degradation state of component 𝑖, 𝑖 ∈ 𝛷.
∑ ( ∑ 𝑡+𝛥 ) For simplicity, we drop off ‘‘𝜏’’ and use 𝑉 (𝑛) to represent 𝑉 at the
𝑃𝑤𝑙 (𝑡 + 𝛥, 𝒙) = 𝑃𝑤𝑚 (𝑡, 𝒙)𝜋𝑚𝑙 (𝛥) exp − 𝜆𝑖 (𝜃, 𝑚)d𝜃 , 𝑛th iteration. Then at the 𝑛 + 1th iteration, if 𝒙 ∉ 𝑆𝑔 , from Eq. (9),
∫𝑡
𝑚∈𝑆𝑙 𝑖∈𝛷
∑∑ 𝑉 (𝑛+1) (𝒙, 𝑤) = 𝑐𝑠 + 𝑐𝑓 + 𝑉 (𝑛) (𝟎, 0) which is constant and satisfy the non-
𝑖−
+ 𝑃𝑤𝑚 (𝑡, 𝒙 )𝜋𝑚𝑙 (𝛥)(1 − exp(−𝜆𝑖 (𝑡, 𝑚)𝛥)) decreasing property. If 𝒙 ∈ 𝑆𝑔(2) , for all 𝑦 ∈ 𝑀(𝒙), 𝐶(𝒙, 𝒚) + 𝑉 (𝑛) (𝒚, 𝑤)
𝑚∈𝑆𝑙 𝑖∈𝛷
either remains constant or is a non-decreasing function of 𝑥𝑖 , which
( ∑ 𝑡+𝛥 ) yields the non-decreasing property of 𝑉 (𝑛+1) (𝒙, 𝑤) due to that operator
× exp − 𝜆𝑗 (𝜃, 𝑚)d𝜃 ,
∫𝑡 ‘‘min’’ keeps the monotonicity.
𝑗∈𝛷∖𝑖
If 𝒙 ∈ 𝑆𝑔(1) , we need to prove the monotonicity of 𝐷(𝒙, 𝑤) and
which yields to
𝑈 (𝑛) (𝒙, 𝑤). According to the expression of the system reliability function
d𝑃𝑤𝑙 (𝑡, 𝒙) ∑ ∑ ∑ in Eq. (7), for initial system degradation states 𝒙 and 𝒙(𝑖−) , clearly
= −𝑃𝑤𝑙 (𝑡, 𝒙) 𝜆𝑖 (𝑡, 𝑙) + 𝑃𝑤𝑚 (𝑡, 𝒙)𝑞𝑚𝑙 + 𝑃𝑤𝑙 (𝑡, 𝒙𝑖− )𝜆𝑖 (𝑡, 𝑙).
d𝑡 that 𝑅(𝑡; 𝒙, 𝒗) ≤ 𝑅(𝑡; 𝒙(𝑖−) , 𝒗) for any initial environment state vector
𝑖∈𝛷 𝑚∈𝑆 𝑙 𝑖∈𝛷
𝒗. Looking back at Eq. (11), we can conclude that 𝐷(𝒙, 𝑤) is non-
(A.4) decreasing in 𝑥𝑖 , 𝑖 ∈ 𝛷. In addition, 𝑈 (𝑛) (𝒙, 𝑤) is the expectation
It can be expressed in the matrix form as follows: of the value function given the system degradation state 𝒙 and the
environment state 𝑤. Obvious that 𝒚 ∣ 𝒙 ≻𝑠𝑡 𝒚 ∣ 𝒙(𝑖−) and as 𝑉 (𝑛) is
d𝑷 (𝑡, 𝒙) ( ) ∑ ( ) ( )
= 𝑷 (𝑡, 𝒙) 𝑸 − 𝜦(𝑡) + 𝑷 (𝑡, 𝒙𝑖− )𝜦𝑖 (𝑡), (A.5) a non-decreasing function in 𝑥𝑖 , thus E 𝒚 ∣ 𝒙 ≥ E 𝒚 ∣ 𝒙(𝑖−) which
d𝑡 𝑖∈𝛷 means that 𝑈 (𝑛) (𝒙, 𝑤) is non-decreasing in 𝑥𝑖 . Thus we can conclude
where 𝑸 is the infinitesimal generator of the CTMC. 𝜦𝑖 (𝑡) is a diagonal that 𝑉 (𝑛+1) (𝒙, 𝑤) is non-decreasing in 𝑥𝑖 when 𝒙 ∈ 𝑆𝑔(1) . Hence, 𝑉 (𝒙, 𝑤)
matrix with the (𝑗, 𝑗)th element given as 𝜆𝑖 (𝑡, 𝑗), 𝑖 ∈ 𝛷, 𝑗 ∈ 𝑆𝑙 , and is non-decreasing in 𝑥𝑖 , 𝑖 ∈ 𝛷. ⋄
𝑷 (𝑡, 𝒙𝑖− ) = 𝟎 if 𝑥𝑖 = 0. The initial value is 𝑷 (𝑡, 𝒙) = 𝟎. ⋄
B.2. The proof of Theorem 4.2
A.3. Calculations of reliability functions
Proof. Define 𝑖 (𝒙, 𝑤) as the action set in which component 𝑖 is not
To calculate the system reliability, In this study, we will solve this replaced under each action. Define 𝑉𝜏 (𝑎; 𝒙, 𝑤) as the value function that
problem by the product-integration method [1]. To calculate 𝑷 (𝑡, 𝟎), as action 𝑎 is chosen in the decision-making given (𝒙, 𝑤). Let 𝑎𝑖 be the
optimal action given (𝒙, 𝑤) and assume that component 𝑖 is replaced
d𝑷 (𝑡, 𝟎) under 𝑎𝑖 , it means that ∀𝑏 ∈ 𝑖 (𝒙, 𝑤),
= 𝑷 (𝑡)(𝑸 − 𝜦(𝑡)), (A.6)
d𝑡
𝑉𝜏 (𝑎𝑖 ; 𝒙, 𝑤) ≤ 𝑉𝜏 (𝑏; 𝒙, 𝑤).
then 𝑷 (𝑡, 𝟎) can be numerically obtained as follows.
∑
𝑡∕𝛿 Given (𝒙(𝑖+) , 𝑤), it is obvious that 𝑖 (𝒙(𝑖+) , 𝑤) ⊆ 𝑖 (𝒙, 𝑤) and 𝑉𝜏 (𝑎𝑖 ;
( )
𝑷 (𝑡, 𝟎) = 𝑰 + 𝑴(𝑖𝛿)𝛿 . (A.7) 𝒙(𝑖+) , 𝑤) = 𝑉𝜏 (𝑎𝑖 ; 𝒙, 𝑤). Due to the monotonicity of 𝑉𝜏 , ∀𝑐 ∈𝑖 (𝒙(𝑖+) , 𝑤),
𝑖=0
𝑉𝜏 (𝑐; 𝒙, 𝑤) ≤ 𝑉𝜏 (𝑐; 𝒙(𝑖+) , 𝑤),
where 𝑴(𝑖𝛿) = (𝑸 − 𝜦(𝑖𝛿)), 𝑰 is the identity matrix, 𝛿 is the discretized
step size. which yields that 𝑉𝜏 (𝑎𝑖 ; 𝒙(𝑖+) , 𝑤) ≤ 𝑉𝜏 (𝑐; 𝒙(𝑖+) , 𝑤), indicating policy 𝑎𝑖
To calculate 𝑷 (𝑡, 𝒙) where 𝑥𝑖 < 𝐿𝑖 , ∀𝑖 ∈ 𝛷. As shown in Lemma 4.2, is better than policies where component 𝑖 is not replaced. The first
𝑷 (𝑡, 𝒙) satisfies statement in Theorem 4.2 is thus verified. Similarly, we can prove the
d𝑷 (𝑡, 𝒙) ( ) ∑ second statement, which is omitted here. ⋄
= 𝑷 (𝑡, 𝒙) 𝑸 − 𝜦(𝑡) + 𝑷 (𝑡, 𝒙𝑖− )𝜦𝑖 (𝑡), (A.8)
d𝑡 𝑖∈𝛷
References
Then 𝑷 (𝑡, 𝒙) can be obtained numerically as follows.
𝑡
𝑡∕𝛿 [1] Zhang Nan, Fouladirad Mitra, Barros Anne. Reliability-based measures and
𝑷 (𝑡, 𝒙) = 𝑵(𝜃)𝛱𝑖=𝑥∕𝛿 (𝑰 + 𝑴(𝑖𝛿)𝛿)d𝜃, (A.9) prognostic analysis of a 𝐾-out-of-𝑁 system in a random environment. European
∫0
∑ 𝑖−
J Oper Res 2019;272(3):1120–31.
where 𝑴(𝑖𝛿) = (𝑸 − 𝜦(𝑖𝛿)), 𝑵(𝜃) = 𝑖∈𝛷 𝑷 (𝑡, 𝒙 )𝜦𝑖 (𝜃), 𝛿 is the [2] Zheng Rui, Chen Bingkun, Gu Liudong. Condition-based maintenance with
discretized step size, 𝑰 is the identity matrix. dynamic thresholds for a system using the proportional hazards model. Reliab
Similarly, 𝑷 (𝑡, 𝒙) in Lemma 4.3 can be calculated. Hence the system Eng Syst Saf 2020;204:107123.
[3] Banjevic Dragan, Jardine AKS. Calculation of reliability function and re-
reliability function can be obtained.
maining useful life for a Markov failure time process. IMA J Manag Math
2006;17(2):115–30.
Appendix B. Proofs for maintenance modelling [4] Shen Jingyuan, Elwany Alaa, Cui Lirong. Reliability modeling for systems
degrading in 𝐾 cyclical regimes based on gamma processes. Proc Inst Mech
B.1. The proof of Lemma 4.4 Eng O 2018;232(6):754–65.
[5] Zhang Nan, Fouladirad Mitra, Barros Anne, Zhang Jun. Reliability and mainte-
nance analysis of a degradation-threshold-shock model for a system in a dynamic
Before the proof, we first introduce the definition of stochastic
environment. Appl Math Model 2021;91:549–62.
orders. [6] Zhu Qiushi, Peng Hao, van Houtum Geert-Jan. A condition-based maintenance
policy for multi-component systems with a high maintenance setup cost. Or
Definition. Random variable 𝑋1 is said to be stochastically larger Spectrum 2015;37(4):1007–35.
than 𝑋2 if P(𝑋1 > 𝑡) ≥ P(𝑋2 > 𝑡), written as 𝑋1 ≻𝑠𝑡 𝑋2 . [7] Poppe Joeri, Boute Robert N, Lambrecht Marc R. A hybrid condition-based main-
tenance policy for continuously monitored components with two degradation
A property of this stochastic order is that, if 𝑋1 ≻𝑠𝑡 𝑋2 , then
thresholds. European J Oper Res 2018;268(2):515–32.
for all increasing functions 𝑔, the corresponding expectations satisfy [8] Keizer Minou CA Olde, Flapper Simme Douwe P, Teunter Ruud H. Condition-
E(𝑔(𝑋1 )) ≥ E(𝑔(𝑋2 )). The above property will be utilized in the fol- based maintenance policies for systems with multiple dependent components: A
lowing proof. review. European J Oper Res 2017;261(2):405–20.

10
N. Zhang et al. Reliability Engineering and System Safety 231 (2023) 108988

[9] Liu Bin, Pandey Mahesh D, Wang Xiaolin, Zhao Xiujie. A finite-horizon condition- [25] Ye Zhi-Sheng, Chen Nan. The inverse Gaussian process as a degradation model.
based maintenance policy for a two-unit system with dependent degradation Technometrics 2014;56(3):302–11.
processes. European J Oper Res 2021;295(2):705–17. [26] Chen Nan, Ye Zhi-Sheng, Xiang Yisha, Zhang Linmiao. Condition-based main-
[10] Esary JD, Marshall AW. Shock models and wear processes. Ann Probab tenance using the inverse Gaussian degradation model. European J Oper Res
1973;627–49. 2015;243(1):190–9.
[11] Özekici Süleyman. Complex systems in random environments. In: Reliability and [27] Zhang Zhengxin, Si Xiaosheng, Hu Changhua, Lei Yaguo. Degradation data
maintenance of complex systems. Springer; 1996, p. 137–57. analysis and remaining useful life estimation: A review on Wiener-process-based
[12] Lawless Jerry, Crowder Martin. Covariates and random effects in a gamma methods. European J Oper Res 2018;271(3):775–96.
process model with application to degradation and failure. Lifetime Data Anal [28] Dong Qinglai, Cui Lirong, Si Shubin. Reliability and availability analysis of
2004;10(3):213–27. stochastic degradation systems based on bivariate Wiener processes. Appl Math
[13] Charki A, Barreau M, Guerin F, Voiculescu S. Reliability estimation in random en- Model 2020;79:414–33.
vironment: Different approaches. In: 2007 annual reliability and maintainability [29] Zhang Nan, Cai Kaiquan, Zhang Jun, Wang Tian. A condition-based mainte-
symposium. IEEE; 2007, p. 202–7. nance policy considering failure dependence and imperfect inspection for a
[14] Wu Shaomin, Scarf Philip. Decline and repair, and covariate effects. European J two-component system. Reliab Eng Syst Saf 2022;217:108069.
Oper Res 2015;244(1):219–26. [30] Deng Yingjun, Barros Anne, Grall Antoine. Degradation modeling based on
[15] Hu Jiawen, Chen Piao. Predictive maintenance of systems subject to hard failure a time-dependent ornstein-uhlenbeck process and residual useful lifetime
based on proportional hazards model. Reliab Eng Syst Saf 2020;196:106707. estimation. IEEE Trans Reliab 2015;65(1):126–40.
[16] Najafi Seyedvahid, Zheng Rui, Lee Chi-Guhn. An optimal opportunistic mainte- [31] Xiao Weilin, Zhang Weiguo, Xu Weidong. Parameter estimation for frac-
nance policy for a two-unit series system with general repair using proportional tional Ornstein–Uhlenbeck processes at discrete observation. Appl Math Model
hazards models. Reliab Eng Syst Saf 2021;215:107830. 2011;35(9):4196–207.
[17] Zio Enrico. Some challenges and opportunities in reliability engineering. IEEE [32] Makis Viliam. Multivariate Bayesian control chart. Oper Res 2008;56(2):487–96.
Trans Reliab 2016;65(4):1769–82. [33] Yin Zhijian, Makis Viliam. Economic and economic-statistical design of a mul-
[18] Kundu Pradeep, Darpe Ashish K, Kulkarni Makarand S. Weibull accelerated fail- tivariate Bayesian control chart for condition-based maintenance. IMA J Manag
ure time regression model for remaining useful life prediction of bearing working Math 2011;22(1):47–63.
under multiple operating conditions. Mech Syst Signal Process 2019;134:106302. [34] Pandey Mayank, Zuo Ming J, Moghaddass Ramin. Selective maintenance mod-
[19] Shen Jingyuan, Cui Lirong, Ma Yizhong. Availability and optimal maintenance eling for a multistate system with multistate components under imperfect
policy for systems degrading in dynamic environments. European J Oper Res maintenance. IIE Trans 2013;45(11):1221–34.
2019;276(1):133–43. [35] Ettaye Ghita, Barkany Abdellah El, Khalfi Ahmed El. The integration of main-
[20] Zhao Xuejing, Fouladirad Mitra, Bérenguer Christophe, Bordes Laurent. tenance plans and production scheduling for a degradable multi-state system: a
Condition-based inspection/replacement policies for non-monotone deteriorating literature review. Int J Prod Qual Manag 2016;19(1):74–97.
systems with environmental covariates. Reliab Eng Syst Saf 2010;95(8):921–34. [36] Zuckerman Dror. Inspection and replacement policies. J Appl Probab
[21] van Noortwijk Jan M. A survey of the application of gamma processes in 1980;17(1):168–77.
maintenance. Reliab Eng Syst Saf 2009;94(1):2–21. [37] Chen Yuan, Qiu Qingan, Zhao Xian. Condition-based opportunistic maintenance
[22] Wu Shaomin, Castro Inma T. Maintenance policy for a system with a policies with two-phase inspections for continuous-state systems. Reliab Eng Syst
weighted linear combination of degradation processes. European J Oper Res Saf 2022;228:108767.
2020;280(1):124–33. [38] Sun Qiuzhuang, Ye Zhi-Sheng, Chen Nan. Optimal inspection and replace-
[23] Zhao Xian, Sun Jinglei, Qiu Qingan, Chen Ke. Optimal inspection and mis- ment policies for multi-unit systems subject to degradation. IEEE Trans Reliab
sion abort policies for systems subject to degradation. European J Oper Res 2017;67(1):401–13.
2021;292(2):610–21. [39] Shi Yue, Zhu Weihang, Xiang Yisha, Feng Qianmei. Condition-based mainte-
[24] Fouladirad Mitra, Salem Marwa Belhaj, Deloux Estelle. Variance Gamma process nance optimization for multi-component systems subject to a system reliability
as degradation model for prognosis and imperfect maintenance of centrifugal requirement. Reliab Eng Syst Saf 2020;202:107042.
pumps. Reliab Eng Syst Saf 2022;108417. [40] Peng Weiwen, Hong Lanqing, Ye Zhisheng. Degradation-based reliability mod-
eling of complex systems in dynamic environments. In: Statistical modeling for
degradation data. Springer; 2017, p. 81–103.

11