An Evolutionary Framework with Improved Variance-Stabilized Multi-Objective Proximal Policy Optimization and NSGA-II

Document Type

Conference Proceeding

Publication Date

1-1-2024

Abstract

Multi-objective optimization algorithms are essential for addressing real-world challenges characterized by conflicting objectives. Although conventional algorithms are effective in exploring solution spaces and generating non-dominated solutions, solution quality and dynamic adaptability of true Pareto fronts need to be improved. This work proposes a multi-objective algorithm that integrates Non-dominated sorting genetic algorithm II (NSGA-II) and Multi-Objective Reinforcement Learning (N-MORL). N-MORL consists of two parts including upstream and downstream components. In the upstream component, this work improves the Variance-stabilized Multi-objective Proximal Policy Optimization (VMPPO) for enhanced convergence stability by adjusting its iteration mechanism. Additionally, this work optimizes variance networks and action sampling to balance exploration and exploitation, which improves experience sampling efficiency. This work adopts high-quality solution sets yielded by MORL as the initial solution set for downstream NSGA-II, guiding the exploration space and increasing the solution number. High-quality initial solutions significantly accelerate the iterative convergence speed of N-MORL. N-MORL provides the quality and the number of solutions, better covering or approaching the true Pareto front. Experimental results with five benchmark multi-objective functions demonstrate that N-MORL outperforms the other three multi-objective evolutionary algorithms regarding high-quality solutions with the same iterations.

Identifier

85217859896 (Scopus)

ISBN

[9781665410205]

Publication Title

Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics

External Full Text Location

https://doi.org/10.1109/SMC54092.2024.10831366

ISSN

1062922X

First Page

3733

Last Page

3738

Grant

4232049

Fund Ref

Natural Science Foundation of Beijing Municipality

This document is currently not available here.

Share

COinS