Action evaluation hardware accelerator for next-generation real-time reinforcement learning in emerging IoT systems

Document Type

Conference Proceeding

Publication Date

7-1-2020

Abstract

Internet of Things (IoT) sensors often operate in unknown dynamic environments comprising latency-sensitive data sources, dynamic processing loads, and communication channels of unknown statistics. Such settings represent a natural application domain of reinforcement learning (RL), which enables computing and learning decision policies online, with no a priori knowledge. In our previous work, we introduced a post-decision state (PDS) based RL framework, which considerably accelerates the rate of learning an optimal decision policy. The present paper formulates an efficient hardware architecture for the action evaluation step, which is the most computationally-intensive step in the PDS based learning framework. By leveraging the unique characteristics of PDS learning, we optimize its state value expectation and known cost computational blocks, to speed-up the overall computation. Our experiments show that the optimized circuit is 49 times faster than its software implementation counterpart, and six times faster than a Q-learning hardware accelerator.

Identifier

85090412773 (Scopus)

ISBN

[9781728157757]

Publication Title

Proceedings of IEEE Computer Society Annual Symposium on VLSI Isvlsi

External Full Text Location

https://doi.org/10.1109/ISVLSI49217.2020.00084

e-ISSN

21593477

ISSN

21593469

First Page

428

Last Page

433

Volume

2020-July

Grant

1711335

Fund Ref

National Science Foundation

This document is currently not available here.

Share

COinS