Forootani, Ali and Zarch, Majid Ghaniee and Tipaldi, Massimo and Iervolino, Raffaele (2023) A stochastic dynamic programming approach for the machine replacement problem. Engineering Applications of Artificial Intelligence, 118. p. 105638. ISSN 0952-1976

Download (3MB)

Abstract

This paper addresses both the modeling and the resolution of the replacement problem for a population of machines. The main objective is the computation of a minimum cost replacement policy, which, based on the status of each machine, determines whether one or more machines have to be replaced over a given finite time horizon. The replacement problem of a set of machines can be regarded as a sequential decision-making problem under uncertainty. Thanks to this, we propose a novel formulation for such problems consisting of a composition of discrete-time multi-state Markov Decision Processes (MDPs), one for each specific machine. The underlying optimization problem is formulated as a stochastic Dynamic Programming (DP), and then solved by using the principles of the backward DP algorithm. Moreover, to deal with the curse of dimensionality due to the high-cardinality state–space of real-world/industrial applications, a new generalized multi-trajectory Least-Squares Temporal Difference (LSTD) based method is introduced. The resulting algorithm computes an approximate optimal cost function by: (i) running Monte Carlo simulations over different trajectories of a given length; (ii) embedding the policy improvement step within the recursive LSTD iterations; (iii) enforcing an off-policy mechanism to improve the LSTD exploration capabilities. A study on the convergence properties of the proposed approach is also provided. Several numerical examples are given to illustrate its effectiveness in terms of parametric sensitivity, computational burden, and performance of the computed policies compared with some heuristics defined in the literature.

Item Type:	Article
Keywords:	Machine replacement problem; Markov Decision Process; Dynamic Programming; Least-Squares Temporal Difference; Monte Carlo Simulations;
Academic Unit:	Faculty of Science and Engineering > Research Institutes > Hamilton Institute
Item ID:	18798
Identification Number:	https://doi.org/10.1016/j.engappai.2022.105638
Depositing User:	Ali Forootani
Date Deposited:	22 Aug 2024 13:03
Journal or Publication Title:	Engineering Applications of Artificial Intelligence
Publisher:	Elsevier
Refereed:	Yes
URI:	Publisher
Use Licence:	This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

Repository Staff Only(login required)

Item control page

Downloads

Origin of downloads