摘 要：为了降低分解型算法求解大规模问题的运行时间成本，结合分解型多目标进化算法 (MOEA/D) 和 Spark 分布式计算框架的特点，提出了一个主从分布式分解型多目标进化算法（MODEA/D-RDD）。在新的方案中每个 Map 保存且进化一个子问题，从而通过多个 Map 分布式计算提高效率。测试例上的实验结果表明，在求得解集质量不明显降低的前提下，全局种群进化方案能够有效缩短求解多目标问题的计算时间。
关键词：Spark 计算框架；多目标优化；MOEA/D 算法
DOI:10.19850/j.cnki.2096-4706.2021.22.020
中图法分类号 : TP391 文献标识码：A 文章编号：2096-4706（2021）22-0066-05
Implementation of Multi-objective Optimization Algorithm Based on Spark Computing Framework
HE Yuqi, LI Deyu
(Jinan University–University of Birmingham Joint Institute, Guangzhou 511443, China)
Abstract: In order to reduce the running time cost of decomposition algorithm for solving large-scale problems, a master-slave distributed multi-objective evolutionary algorithm (MODEA/D-RDD) is proposed based on the characteristics of the decomposition multiobjective evolutionary algorithm (MOEA/D) and Spark distributed computing framework. In the new scheme, each Map saves and evolves a sub problem, so as to improve the efficiency through multiple Map distributed computing. The Experimental results on test cases show that the global population evolution scheme can effectively shorten the computational time on solving multi-objective problems on the premise that the quality of the solution set is not significantly reduced.
Keywords: Spark computing framework; multi-objective optimization; MOEA/D algorithm
参考文献：
[1] FONSECA C M, FLEMING P J. An overview of evolutionary algorithms in multiobjective optimization [J].Evolutionary computation， 1995，3（1）：1-16.
[2] DEB K,ZOPE P,Jain A. Distributed computing of Pareto-optimal solutions with evolutionary algorithms [C]//Proceedings of the 2nd international conference on Evolutionary multi-criterion optimization.Springer-Verlag：534-549.
[3] ZAHARIE D, PETCU D, PANICA S. A Hierarchical Approach in Distributed Evolutionary Algorithms for Multiobjective Optimization [J].Lecture Notes in Computer Science，2008，4818：516-523.
[4] SADASIVAM G S，SELVARAJ D. A novel parallel hybrid PSO-GA using MapReduce to schedule jobs in Hadoop data grids [C]//2010 Second World Congress on Nature and Biologically Inspired Computing (NaBIC).Kitakyushu：IEEE，2010：377-382.
[5] DONKAL G, VERMA G K. Securing Big Data Ecosystem with NSGA-II and Gradient Boosted Trees Based NIDS Using Spark [C]//2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS).Madurai：IEEE.2018：146-151.
作者简介：何昱琪（2001—），女，汉族，浙江义乌人，本科在读，研究方向：数据处理、数据分析统计。李德禹（2001—）， 男，汉族，安徽阜阳人，本科在读，研究方向：算法设计与分析、数据处理。