摘 要：针对网络拥塞控制问题，结合机器学习算法，提出了基于强化学习的拥塞窗口调整CWARL）策略。首先定义了部分网络知识来表示所感知到的网络拥塞程度，设计了动作集合以确定调整拥塞窗口的幅度，设计了兼顾吞吐量和丢包率的奖励函数。其次提出了基于 Q 学习的窗口调整策略，通过学习网络特征合理地调整拥塞窗口。最后使用实验评估 CWARL 策略，实验结果表明，提出的 CWARL 策略的综合性能优于所对比的拥塞控制策略。
关键词：强化学习；拥塞控制；窗口调整
DOI:10.19850/j.cnki.2096-4706.2022.08.024
中图分类号：TP391 文献标识码：A 文章编号：2096-4706（2022）08-0086-03
Research on Congestion Window Adjustment Strategy Based on Reinforcement Learning
ZHOU Ping
(Nanchang Vocational University, Nanchang 330599, China)
Abstract: Aiming at the problem of network congestion control, combined with machine learning algorithms, this paper proposes a strategy of Congestion Window Adjustment based on Reinforcement Learning (CWARL). This paper first defines some network knowledge to represent the perceived degree of network congestion, designs actions set to determine the magnitude of the adjustment congestion windows, and designs a reward function to juggle throughput and packet loss rate. Second, this paper proposes a window adjustment strategy based on Q-learning, adjusts reasonably the congestion window by learning network features. Finally, it uses experiments to evaluate the CWARL strategy, and the experimental results show that the overall performance of the proposed CWARL strategy is better than the contrasted congestion control strategy.
Keywords: reinforcement learning; congestion control; window adjustment
参考文献：
[1] MIŠIC J，ALI M Z，MIŠIC V B. Architecture for IoT domain with CoAP observe feature [J]. IEEE Internet of Things Journal， 2018，5（2）：1196-1205.
[2] XU Z，TANG J，YIN C，et al. Experience-driven congestion control：When multi-path TCP meets deep reinforcement learning [J]. IEEE Journal on Selected Areas in Communications，2019，37（6）： 1325-1336.
[3] POLESE M，CHIARIOTTI F，Bonetto E，et al. A survey on recent advances in transport layer protocols [J].IEEE Communications Surveys & Tutorials，2019，21（4）：3584-3608.
[4] TSIKNAS K G，AIDINIDIS P I，Zoiros K E. On the Fairness of DCTCP and CUBIC in Cloud Data Center Networks [C]//2021 10th International Conference on Modern Circuits and Systems Technologies （MOCAST）.Thessaloniki：IEEE，2021：1-4.
[5] XIAO Y，LIU J，WU J，et al. Leveraging deep reinforcement learning for traffic engineering：A survey [J].IEEE Communications Surveys & Tutorials，2021，23（4）：2064-2097.
[6] TANG F，MAO B，KAWAMOTO Y，et al. Survey on machine
learning for intelligent end-to-end communication toward 6G：From etwork access，routing to traffic control and streaming adaption [J].IEEE Communications Surveys & Tutorials，2021，23（3）：1578-1598.
作者简介：周萍（1980—），女，汉族，浙江江山人，讲师，硕士，研究方向：计算机及应用。