摘 要：随着物联网的飞速发展，高速、海量的数据通信向服务质量保障机制提出了挑战。为了很好地满足用户对高效、低延迟路由的需求，文章结合软件定义网络（SDN）技术，提出一种软件定义物联网中基于深度强化学习的路由优化算法（RDIS）。RDIS 利用控制器收集网络信息，基于改进的深度确定性策略梯度算法，在经验回放池中根据重要性对样本采样，最终获得一条性能近乎最优的路径。仿真实验表明，相较于传统的路由算法，RDIS 在吞吐量和延迟方面具有更好的网络性能。
关键词：物联网；软件定义网络；路由优化；深度强化学习
DOI:10.19850/j.cnki.2096-4706.2022.08.043
中图分类号：TP393 文献标识码：A 文章编号：2096-4706（2022）08-0158-05
Routing Optimization Algorithm Based on Deep Reinforcement Learning in Software Defined Internet of Things
WANG Jian, ZHU Xiaojuan
(School of Computer Science and Engineering, Anhui University of Technology, Huainan 232001, China)
Abstract: With the rapid development of the Internet of Things, high-speed and massive data communication poses a challenge to the service quality assurance mechanism. In order to meet the needs of users for efficient and low-latency routing, this paper proposes a routing optimization algorithm based on deep reinforcement learning (RDIS) in the software-defined Internet of Things (SDN) technology. RDIS uses the controller to collect network information, and based on the improved deep deterministic policy gradient algorithm, samples are sampled according to importance in the experience playback pool, and finally a path with near-optimal performance is obtained. Simulation experiments show that, compared with traditional routing algorithms, RDIS has better network performance in terms of throughput and delay.
Keywords: Internet of Things; software defined network; routing optimization; Deep Reinforcement Learning
参考文献：
[1] 张顺淼，邹复民 . 软件定义网络研究综述 [J]. 计算机应用研究，2013，30（8）：2246-2251.
[2] LILLICRAP T P，HUNT J J，PRITZEL A，et al. Continuous control with deep reinforcement learning [J/OL].arXiv:1509.02971 [cs. LG].https://arxiv.org/abs/1509.02971v5.
[3] 汤季，谭小波，朱腾 .SNDQ-WISE：一种新的基于 SDWSN 的 QOS 路由协议 [J]. 沈阳理工大学学报，2019，38（4）：18-24+51.
[4] 晏勇 . 基于海豚回声定位算法的 SDWSN 路由算法 [J]. 西南师范大学学报（自然科学版，2021，46（8）：79-85.
[5] 赵志豪 . 软件定义无线传感网的节能算法研究 [D]. 北京：北京邮电大学，2020.
[6] 张芬 .基于软件定义的 WSNs 分簇路由算法研究 [D]. 兰州：西北师范大学，2021.
[7] ADIL M，KHAN R，ALI J. et al. An Energy Proficient Load Balancing Routing Scheme for Wireless Sensor Networks to Maximize Their Lifespan in an Operational Environment [J].IEEE Access，2020，8：99.
[8] DANIELA M，Casas-Velasco，RENDON O M C，et al. Intelligent Routing Based on Reinforcement Learning for SoftwareDefined Networking [J].IEEE Transactions on Network and Service Management，2021，18（1）：870-881.
[9] YUAN Z W，ZHOU P，WANG S S，et al. Research on Routing Optimization of SDN Network Using Reinforcement Learning Method [C]//2019 2nd International Conference on Safety Produce Informatization (IICSPI). 2019：442-445.
[10] 李荥，王芳，景栋盛，等 . 一种基于 Q 学习的无线传感网络路由方法 [J]. 计算技术与自动化，2017，36（2）：155-160.
[11] YU C H，LAN J L，GUO Z H，et al.DROM: Optimizing the Routing in Software-Defined Networks With Deep Reinforcement Learning [J].IEEE Access，2018，6：64533-64539.
[12] 周浩 . 基于深度强化学习的 SDN 智能路由优化研究 [D].武汉：湖北工业大学，2020.
[13] LILLICRAP T P，HUNT J J，PRITZEL A，et al. Continuous control with deep reinforcementlearning [J/OL]. arXiv:1509.02971 [cs.LG].https://arxiv.org/abs/1509.02971v5.
[14] GALLUCCIO L，MILARDO S，MORABITO G，et al. SDN-WISE: Design, prototyping and experimentation of a stateful SDN solution for WIrelessSEnsor networks [C]//IEEE Conference on Computer Communications (INFOCOM).Hong Kong：IEEE，2015：513-521.
[15] SHALABI L，SHAABAN Z. Normalization as a preprocessing engine for data mining and the approach of preference matrix [C]//2006 International Conference on Dependability of Computer Systems. Szklarska Poreba：IEEE，2006：207-214.
作者简介：王健（1997—），男，汉族，安徽合肥人，硕士研究生在读，研究方向：智能物联网；朱晓娟（1978—），女，汉族，安徽淮南人，副教授，博士，研究方向：智能物联网。