摘 要：针对服装图像特征提取不全面和服装兼容性难以建模等问题，提出了一种基于通道注意力的多模态服装兼容性模型 ECA-RMCN。在特征提取网络 CNN 的残差模块上引入高效通道注意力模块 ECA-Net 来增强服装低级和高级等重要特征，抑制无效特征。采用组合损失函数处理服装正负样本不均衡的问题，达到更好的搭配效果。在公共的 Polyvore 数据集进行对比实验来验证模型有效性。实验结果表明，该算法对服装的兼容性预测和搭配性能优于其他方法，有很好的应用价值。
关键词：通道注意力；卷积神经网络；兼容性建模；组合损失函数
DOI:10.19850/j.cnki.2096-4706.2022.04.001
中图分类号：TP18 文献标识码：A 文章编号：2096-4706（2022）04-0001-07
Multimodal Clothing Compatibility Learning Based on Channel Attention
WEI Xiong1,2,3, YAN Kun1,2,3
(1.Textile and Clothing Intelligent Hubei Provincial Engineering Research Center, Wuhan 430200, China; 2.Hubei Provincial Garment Informatization Engineering Technology Research Center, Wuhan 430200, China; 3.School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan 430200, China)
Abstract: Aiming at the problems of incomplete feature extraction of clothing images and difficult modeling of clothing compatibility and so on, a multimodal clothing compatibility model ECA-RMCN based on channel attention is proposed. The highefficiency channel attention module ECA-Net is introduced on the residual module of the feature extraction network CNN to enhance important features such as low-level and high-level clothing, and suppress invalid features. The combined loss function is used to deal with the problems of unbalanced positive and negative samples of clothing to achieve better matching effect. Comparative experiments are performed on the public Polyvore dataset to verify the effectiveness of the model. The experimental results show that the algorithm is better than other methods in the compatibility prediction and matching performance of clothing, and it has good application value.
Keywords: channel attention; convolutional neural network; compatibility modeling; combined loss function
参考文献：
[1] 纪丹丹，戴宏钦 . 服装搭配方法研究综述 [J]. 现代丝绸科学与技术，2020，35（4）：31-35.
[2] VEIT A，KOVACS B，Bell S，et al.Learning visual clothing style with heterogeneous dyadic co-occurrences [C]//Proceedings of the IEEE International Conference on Computer Vision.2015：4642-4650.
[3] HE R，PACKER C，MCAULEY J.Learning compatibility across categories for heterogeneous item recommendation [C]//2016 IEEE 16th International Conference on Data Mining （ICDM）. Santiago：IEEE，2016：937-942.
[4] MCAULEY J，TARGETT C，SHI Q，et al.Image-based recommendations on styles and substitutes [C]//Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval.New York：Association for Computing Machinery，2015：43-52.
[5] HAN X，WU Z，JIANG Y G，et al.Learning fashion compatibility with bidirectional lstms [C]//Proceedings of the 25th ACM international conference on Multimedia.New York：Association for Computing Machinery，2017：1078-1086.
[6] CUI Z，LI Z，WU S，et al.Dressing as a whole：Outfit compatibility learning based on node-wise graph neural networks [C]//The World Wide Web Conference.2019：307-317.
[7] MA Y，YANG X，LIAO L，et al.Who，where，and what to wear? Extracting fashion knowledge from social media [C]//Proceedings of the 27th ACM International Conference on Multimedia.New York：Association for Computing Machinery，2017：2019：257-265.
[8] HSIAO W L，GRAUMAN K.ViBE：Dressing for diverse body shapes [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle：IEEE，2020：11059-11069.
[9] SONG X，HAN X，LI Y，et al.GP-BPR：Personalized compatibility modeling for clothing matching [C]//Proceedings of the 27th ACM International Conference on Multimedia.New York：
Association for Computing Machinery，2019：320-328.
[10] 朱丰，胡晓峰，吴琳，等 . 从态势认知走向态势智能认知 [J]. 系统仿真学报，2018，30（3）：761-771.
[11] 梁斌，刘全，徐进，等 . 基于多注意力卷积神经网络的特定目标情感分析 [J]. 计算机研究与发展，2017，54（8）：1724-1735.
[12] 耿磊，邱玲，吴骏，等 . 结合深度可分离卷积与通道加权的全卷积神经网络视网膜图像血管分割 [J]. 生物医学工程学杂志，2019，36（1）：107-115.
[13] WANG X，SUN Z，ZHANG W，et al.Matching user photos to online products with robust deep features [C]//Proceedings of the 2016 ACM on international conference on multimedia retrieval.New York： Association for Computing Machinery，2016：7-14.
[14] QING Y，LIU W.Hyperspectral Image Classification Based on Multi-Scale Residual Network with Attention Mechanism [J].Remote Sensing，2021，13（3）：335.
[15] ZHOU J，QIAN S，YAN Z，et al.ESA-Net：A Network with Efficient Spatial Attention for Smoky Vehicle Detection [C]//2021 IEEE International Instrumentation and Measurement Technology Conference （I2MTC）.IEEE，2021：1-6.
[16] YAMAGUCHI K，KIAPOUR M H，BERG T L.Paper doll parsing：Retrieving similar styles to parse clothing items [C]// Proceedings of the IEEE international conference on computer vision. IEEE，2013：3519-3526.
[17] YANG W，LUO P，LIN L.Clothing co-parsing by joint image segmentation and labeling [C]//Proceedings of the IEEE conference on computer vision and pattern recognition.IEEE，2014：3182-3189.
[18] WANG X，SUN Z，ZHANG W，et al.Matching user photos to online products with robust deep features [C]//Proceedings of the 2016 ACM on international conference on multimedia retrieval.New York：Association for Computing Machinery，2016：7-14.
[19] AL-HALAH Z，STIEFELHAGEN R，GRAUMAN K.Fashion forward：Forecasting visual style in fashion [C]//Proceedings of the IEEE international conference on computer vision.Venice：IEEE，2017：388-397.
[20] LEE H，SEOL J，LEE S.Style2vec：Representation learning for fashion items from style sets [J/OL].arXiv：1708.04014. （2017-08-14）.https：//arxiv.org/abs/1708.04014.
[21] HSIAO W L，GRAUMAN K.Learning the latent” look”：Unsupervised discovery of a style-coherent embedding from fashion images [C]//Proceedings of the IEEE International Conference on Computer Vision.Venice：IEEE，2017：4203-4212.
[22] VEIT A，KOVACS B，BELL S，et al.Learning visual clothing style with heterogeneous dyadic co-occurrences [C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago：IEEE，2015：4642-4650.
[23] LI X，WANG X，HE X，et al.Hierarchical fashion graph network for personalized outfit recommendation [C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York：Association for Computing Machinery，2020：159-168.
[24] SONG X，FENG F，LIU J，et al.Neurostylist：Neural compatibility modeling for clothing matching [C]//Proceedings of the 25th ACM international conference on Multimedia.New York：Association for Computing Machinery，2017：753-761.
[25] HAN X，SONG X，YAO Y，et al.Neural compatibility modeling with probabilistic knowledge distillation [J].IEEE Transactions on Image Processing，2019，29：871-882.
[26] WANG X，WU B，ZHONG Y.Outfit compatibility prediction and diagnosis with multi-layered comparison network [C]//Proceedings of the 27th ACM International Conference on Multimedia.2019：329-337.
[27] 常惠，饶志强，赵玉林，等 . 基于改进 U-Net 网络的隧道裂缝分割算法研究 [J]. 计算机工程与应用，2021，57（22）：215-222.
[28] WANG Q，WU B，ZHU P，et al.ECA-Net：efficient channel attention for deep convolutional neural networks [C]//CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle：IEEE，IEEE.2020.
[29] ZHANG Y，JIN R，ZHOU Z H.Understanding bag-ofwords model：a statistical framework [J].International Journal of Machine Learning and Cybernetics，2010，1（1-4）：43-52.
[30] LI Y，CAO L，ZHU J，et al.Mining fashion outfit composition using an end-to-end deep learning approach on set data [J]. IEEE Transactions on Multimedia，2017，19（8）：1946-1955.
[31] TANGSENG P，YAMAGUCHI K，O K ATA N I T.Recommending outfits from personal closet [C]//Proceedings of the IEEE International Conference on Computer Vision Workshops. Venice：IEEE，2017：2275-2279.
[32] VASILEVA M I，PLUMMER B A，DUSAD K， et al.Learning type-aware embeddings for fashion compatibility [C]// Proceedings of the European Conference on Computer Vision （ECCV）.Cham：Springer，2018：390-405.
[33] HAN X，WU Z，JIANG Y G，et al.Learning fashion compatibility with bidirectional lstms [C]//Proceedings of the 25th ACM international conference on Multimedia.New York：Association for Computing Machinery，2017：1078-1086.
[34] WANG X，GIRSHICK R，GUPTA A，et al.Non-local neural networks [C]//Proceedings of the IEEE conference on computer vision and pattern recognition.Salt Lake City：IEEE，2018：7794-7803.
[35] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need [J].NIPS’17：Proceedings of the 31st International Conference on Neural Information Processing Systems.New York：Association for Computing Machinery，2017：6000-6010.
作者简介：魏雄（1974—），男，汉族，湖北武汉人，副教授，CCF 会员，博士，研究方向：并行计算、纺织服装大数据等；闫坤（1997—），女，汉族，湖北黄冈人，CCF 会员，硕士在读，研究方向：图像处理。