A deep reinforcement learning framework for solving two-stage stochastic programs

Authors

Dogacan Yilmaz, Newark College of Engineering
Esra Büyüktahtakın, Virginia Tech College of Engineering

Document Type

Article

Publication Date

12-1-2024

Abstract

In this study, we present a deep reinforcement learning framework for solving scenario-based two-stage stochastic programming problems. Stochastic programs have numerous real-time applications, such as scheduling, disaster management, and route planning, yet they are computationally challenging to solve and require specially designed solution strategies such as hand-crafted heuristics. To the extent of our knowledge, this is the first study that decomposes two-stage stochastic programs with a multi-agent structure in a deep reinforcement learning algorithmic framework to solve them faster. Specifically, we propose a general two-stage deep reinforcement learning framework that can generate high-quality solutions within a fraction of a second, in which two different learning agents sequentially learn to solve each stage of the problem. The first-stage agent is trained with the feedback of the second-stage agent using a new policy gradient formulation since the decisions are interconnected through the stages. We demonstrate our framework through a general multi-dimensional stochastic knapsack problem. The results show that solution time can be reduced up to five orders of magnitude with sufficiently good optimality gaps of around 7%. Also, a decision-making agent can be trained with a few scenarios and can solve problems with many scenarios and achieve a significant reduction in solution times. Considering the vast state and action space of the problem of interest, the results show a promising direction for generating fast solutions for stochastic online optimization problems without expert knowledge.

Identifier

85160721399 (Scopus)

Publication Title

Optimization Letters

External Full Text Location

https://doi.org/10.1007/s11590-023-02009-5

e-ISSN

18624480

ISSN

18624472

First Page

1993

Last Page

2020

Issue

9

Volume

18

Grant

1554018

Fund Ref

Division of Mathematical Sciences

Recommended Citation

Yilmaz, Dogacan and Büyüktahtakın, Esra, "A deep reinforcement learning framework for solving two-stage stochastic programs" (2024). Faculty Publications. 79.
https://digitalcommons.njit.edu/fac_pubs/79