MURAL - Maynooth University Research Archive Library



    A stochastic dynamic programming approach for the machine replacement problem


    Forootani, Ali and Zarch, Majid Ghaniee and Tipaldi, Massimo and Iervolino, Raffaele (2023) A stochastic dynamic programming approach for the machine replacement problem. Engineering Applications of Artificial Intelligence, 118. p. 105638. ISSN 0952-1976

    [img] Download (3MB)


    Share your research

    Twitter Facebook LinkedIn GooglePlus Email more...



    Add this article to your Mendeley library


    Abstract

    This paper addresses both the modeling and the resolution of the replacement problem for a population of machines. The main objective is the computation of a minimum cost replacement policy, which, based on the status of each machine, determines whether one or more machines have to be replaced over a given finite time horizon. The replacement problem of a set of machines can be regarded as a sequential decision-making problem under uncertainty. Thanks to this, we propose a novel formulation for such problems consisting of a composition of discrete-time multi-state Markov Decision Processes (MDPs), one for each specific machine. The underlying optimization problem is formulated as a stochastic Dynamic Programming (DP), and then solved by using the principles of the backward DP algorithm. Moreover, to deal with the curse of dimensionality due to the high-cardinality state–space of real-world/industrial applications, a new generalized multi-trajectory Least-Squares Temporal Difference (LSTD) based method is introduced. The resulting algorithm computes an approximate optimal cost function by: (i) running Monte Carlo simulations over different trajectories of a given length; (ii) embedding the policy improvement step within the recursive LSTD iterations; (iii) enforcing an off-policy mechanism to improve the LSTD exploration capabilities. A study on the convergence properties of the proposed approach is also provided. Several numerical examples are given to illustrate its effectiveness in terms of parametric sensitivity, computational burden, and performance of the computed policies compared with some heuristics defined in the literature.

    Item Type: Article
    Keywords: Machine replacement problem; Markov Decision Process; Dynamic Programming; Least-Squares Temporal Difference; Monte Carlo Simulations;
    Academic Unit: Faculty of Science and Engineering > Research Institutes > Hamilton Institute
    Item ID: 18798
    Identification Number: https://doi.org/10.1016/j.engappai.2022.105638
    Depositing User: Ali Forootani
    Date Deposited: 22 Aug 2024 13:03
    Journal or Publication Title: Engineering Applications of Artificial Intelligence
    Publisher: Elsevier
    Refereed: Yes
    URI:
    Use Licence: This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

    Repository Staff Only(login required)

    View Item Item control page

    Downloads

    Downloads per month over past year

    Origin of downloads