2018-12-24 13:43:09 XiangJiaoJun_ 阅读数 1092
  • 大数据:深度学习项目实战-人脸检测视频教程

    购买课程后,请扫码入学习群,获取唐宇迪老师答疑 进军深度学习佳项目实战:人脸检测项目视频培训课程,从数据的收集以及预处理开始,一步步带着大家完成整个人脸检测的项目,其中涉及了如何使用深度学习框架Caffe完成整个项目的架构,对于每一个核心步骤详细演示流程和原理解读,在完成检测代码之后进行了详细评估分析,并给出一篇顶级会议论文作为学习参考,详细分析了针对人脸检测项目的优缺点和改进策略。

    16412 人正在学习 去看看 唐宇迪

  宣传一波个人博客 https://hongbb.top,大家有空去玩玩23333333

  “读万卷书,行万里路”,深度学习领域每时每刻都在萌生新的灵感和想法。要成为这方面的大牛,我想理论知识、代码功底都得多多锻炼。我们不仅仅要对某一个方向深入了解,更要对CV这个领域有一个全面的认识。所以,读paper肯定是不能少的啦,从ImageNet比赛,到目标检测、图像分割,都有许多许多优秀的论文。这篇博客整理出一些优秀深度学习论文,也是对自己学习过程的一些记录吧,不断地学习state-of-the-art论文中的最新思想,这样才能跟得上时代的步伐吧~

深度学习大爆发:ImageNet 挑战赛

  ImageNet 挑战赛属于深度学习最基础的任务:分类。从最早最早的LeNet,到后来的GoogleNet,再到现在的Shufflenet,涌现了一大批优秀的卷积神经网络框架。这些框架也被广泛用于目标检测等更复杂的深度学习任务中作为backbone,用来提取图像的特征。各种state-of-the-art的CNN框架,也是我们首要学习的知识。

  • (LeNet) Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, “Gradient-based learning applied to document recognition,” in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.[PDF] ,CNN的开山之作,也是手写体识别经典论文
  • (AlexNet) Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems. 2012 [PDF]ILSVRC-2012冠军,CNN历史上的转折,也是深度学习第一次在图像识别的任务上超过了SVM等传统的机器学习方法
  • (VGG) Simonyan, Karen, and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).[PDF] 使用了大量的重复卷积层,对后面的网络产生了重要影响
  • (GoogLeNet) Szegedy, Christian, et al. Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015. [PDF] 提出Inception模块,第一次在CNN中使用并行结构,后来的ResNet等都借鉴了该思想,CNN不再是一条路走到底的网络结构了
  • (InceptionV2、InceptionV3) Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the Inception Architecture for Computer Vision[J]. Computer Science, 2015:2818-2826.[PDF]由于BN(Batch Normalization)等提出,改进了原始GoogLeNet中的Inception模块
  • (ResNet) He, Kaiming, et al. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385 (2015).[PDF] 提出残差结构,解决了深度学习网络层数太深梯度消失等问题,ResNet当时的层数达到了101层。
  • (Xception) Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions[J]. arXiv preprint arXiv:1610.02357, 2016.[PDF]
  • (DenseNet) Huang G, Liu Z, Weinberger K Q, et al. Densely Connected Convolutional Networks[J]. 2016. [PDF] 将shortcut思想发挥到极致
  • (SeNet) Squeeze-and-Excitation Networks. [PDF] 主打融合通道间的信息(channel-wise),并且只增加微量计算
  • (MobileNet v1) Howard A G, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017. [PDF]
  • (Shufflenet) Zhang X, Zhou X, Lin M, et al. Shufflenet: An extremely efficient convolutional neural network for mobile devices[J]. [PDF] 使用shuffle操作来代替1x1卷积,实现通道信息融合,大大减小了参数量,主要面向一些计算能力不足的移动设备。
  • (capsules) Sabour S, Frosst N, Hinton G E. Dynamic routing between capsules[C][PDF]
  • (Partial Labels) Durand, Thibaut, Nazanin Mehrasa and Greg Mori. “Learning a Deep ConvNet for Multi-label Classification with Partial Labels.” CoRR abs/1902.09720 (2019): n. pag.[PDF] 多标签分类
  • (Res2Net) Gao, Shang-Hua, Ming-Ming Cheng, Kai Zhao, Xin-yu Zhang, Ming-Hsuan Yang and Philip H. S. Torr. “Res2Net: A New Multi-scale Backbone Architecture.” (2019). [PDF]
  • (Residual Attention Network) Wang, F., Jiang, M., Qian, C., Yang, S., Li, C.C., Zhang, H., Wang, X., & Tang, X. (2017). Residual Attention Network for Image Classification. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6450-6458.[PDF]
  • (Deformable CNN) Dai, Jifeng et al. “Deformable Convolutional Networks.” 2017 IEEE International Conference on Computer Vision (ICCV) (2017): 764-773.[PDF]
  • (GCNet) Cao, Yue et al. “GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond.” CoRR abs/1904.11492 (2019): n. pag.[PDF]
  • (NASNet) Zoph, B., Vasudevan, V., Shlens, J., & Le, Q.V. (2018). Learning Transferable Architectures for Scalable Image Recognition. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8697-8710.[PDF]

物体检测

  深度学习另外一个重要的任务就是物体检测,在1990年以前,典型的物体检测方法是基于 geometric representations,之后物体检测的方法像统计分类的方向发展(神经网络、SVM、Adaboost等)。
  2012年当深度神经网络(DCNN)在图像分类上取得了突破性进展时,这个巨大的成功也被用到了物体检测上。Girshick提出了里程碑式的物体检测模型Region based CNN(RCNN),在此之后物体检测领域飞速发展、并且提出了许多基于深度学习的方法,如YOLO、SSD等…

  • (R-CNN) Girshick, Ross, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition. 2014.[PDF] 里程碑式的物体检测框架,RCNN系列的开山鼻祖,后续深度学习的物体检测都借鉴了思想,不得不读的paper
  • (SPPNet) He K, Zhang X, Ren S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[C]//European Conference on Computer Vision. Springer International Publishing, 2014: 346-361.[PDF] 主要改进了R-CNN中计算过慢重复提取特征的问题
  • (Fast R-CNN) Girshick R. Fast r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision. 2015: 1440-1448.[PDF] RCNN系列的第二版,提出RoI Pooling,同时改进了 R-CNN 和 SPPNet,同时提高了速度和精度
  • (Faster R-CNN) Ren S, He K, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks[C]//Advances in neural information processing systems. 2015: 91-99.[PDF] R-CNN系列巅峰,提出了anchor、RPN等方法,广泛被后续网络采用。
  • (YOLO) Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 779-788.[PDF] One-Stage目标检测框架代表之一,速度非常快,不过精度不如R-CNN系列
  • (SSD) Liu W, Anguelov D, Erhan D, et al. SSD: Single shot multibox detector[C]//European Conference on Computer Vision. Springer International Publishing, 2016: 21-37.[PDF] One-Stage目标检测框架代表之二,提高速度的同时又不降低精度
  • (R-FCN) Li Y, He K, Sun J. R-fcn: Object detection via region-based fully convolutional networks[C]//Advances in Neural Information Processing Systems. 2016: 379-387.[PDF]
  • (DSSD) Fu, C., Liu, W., Ranga, A., Tyagi, A., & Berg, A.C. (2017). DSSD : Deconvolutional Single Shot Detector. CoRR, abs/1701.06659.[PDF] 和FPN的思想有类似,采用deconvolution,进行了特征融合,提高了SSD在小物体,重叠物体上的检测精度
  • (FPN) T. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan and S. Belongie, “Feature Pyramid Networks for Object Detection,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017 [PDF] 提出特征图金字塔,让卷积神经网络中深层中提取的语义信息融合到每一层的特征图中(特别是底层的高分辨率特征图也能获得高层的语义信息),提高特征图多尺度的表达,提高了一些小目标的识别精度,在图像分割和物体监测中都可用到。
  • (RetinaNet) Lin, T., Goyal, P., Girshick, R.B., He, K., & Dollár, P. (2017). Focal Loss for Dense Object Detection. 2017 IEEE International Conference on Computer Vision (ICCV), 2999-3007.[PDF] 提出了FocalLoss 解决物体检测中负类样本过多,类别不平衡的问题
  • (TDM) Shrivastava, Abhinav, Rahul Sukthankar, Jitendra Malik and Abhinav Gupta. “Beyond Skip Connections: Top-Down Modulation for Object Detection.” CoRR abs/1612.06851 (2016): n. pag.[PDF] 和FPN思想类似,不过文中提出的方法是一层一层的添加top-down模块
  • (YOLO-v2) Redmon, J., & Farhadi, A. (2017). YOLO9000: Better, Faster, Stronger. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6517-6525.[PDF] YOLO的改进版本
  • (SIN) Liu, Y., Wang, R., Shan, S., & Chen, X. (2018). Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships. CVPR.[PDF] 将RNN用于CV中,很新颖的网络结构
  • (STDN) Scale-Transferrable Object Detection Peng Zhou, Bingbing Ni, Cong Geng, Jianguo Hu, Yi Xu; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 528-537 [PDF] 主要是改进DSSD、FPN等结构中由于特征融合而引入额外参数,导致速度变慢问题。提出用DenseNet作为Backbone 从而在forward 时候进行特征融合,并提出了不带参数的scale-transform module ,保证精度的同时提高速度
  • (RefineDet) Shrivastava, Abhinav et al. “Training Region-Based Object Detectors with Online Hard Example Mining.” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016): 761-769.[PDF] 结合了one-stage 和 two-stage的优点
  • (MegDet) Peng, Chao, Tete Xiao, Zeming Li, Yuning Jiang, Xiangyu Zhang, Kai Jia, Gang Yu and Jian Sun. “MegDet: A Large Mini-Batch Object Detector.” CVPR (2018).[PDF] 关注于物体检测训练过程中batch size过小的问题,提出Cross-GPU Batch Normalization,训练时能达到256的batch size,coco2017数据集训练时间缩短到4小时。
  • (DA Faster R-CNN) Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, Luc Van Gool; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 3339-3348[PDF]主要是解决物体检测中domain shift问题,提出了domain adaptation component,能够训练出一个domain invariant的鲁棒网络,在大雨天,雾天等复杂场景也能达到很好的检测精度
  • (ExtremeNet) Bottom-up Object Detection by Grouping Extreme and Center Points,Xingyi Zhou, Jiacheng Zhuo, Philipp Krähenbühl,arXiv technical report (1901.08043) 将关键点检测的方法用在了物体检测上面
  • (RelationNet) Han Hu, Jiayuan Gu, Zheng Zhang, Jifeng Dai, Yichen Wei:Relation Networks for Object Detection. CVPR 2018: 3588-3597[PDF]
  • (YOLOv3) YOLOv3: An Incremental Improvement | Joseph Redmon, Ali Farhadi arXiv’18 [PDF]
  • (Cascade R-CNN) Zhaowei Cai, Nuno Vasconcelos:Cascade R-CNN: Delving Into High Quality Object Detection. CVPR 2018: 6154-6162[PDF]
  • (RFBNet) Liu, Songtao et al. “Receptive Field Block Net for Accurate and Fast Object Detection.” ECCV (2018).[PDF]
  • Zhong, Y., Wang, J., Peng, J., & Zhang, L. (2018). Anchor Box Optimization for Object Detection. CoRR, abs/1812.00469.[PDF]
  • (CornerNet) CornerNet: Detecting Objects as Paired Keypoints Hei Law, Jia Deng European Conference on Computer Vision (ECCV), 2018
  • (Grid R-CNN) Lu, X., Li, B., Yue, Y., Li, Q., & Yan, J. (2018). Grid R-CNN. CoRR, abs/1811.12030.[PDF]
  • (SNIPER) Singh, B., Najibi, M., & Davis, L.S. (2018). SNIPER: Efficient Multi-Scale Training. NeurIPS.[PDF]
  • (TridentNet) Li, Yanghao et al. “Scale-Aware Trident Networks for Object Detection.” CoRR abs/1901.01892 (2019): n. pag.[PDF]
  • (GIoU) Rezatofighi, Seyed Hamid et al. “Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression.” CoRR abs/1902.09630 (2019): n. pag.[PDF] 一种新型的IoU计算方法以及loss
  • (MetaAnchor) Yang, Tong et al. “MetaAnchor: Learning to Detect Objects with Customized Anchors.” NeurIPS (2018). [PDF]
  • (M2Det) Zhao, Q., Sheng, T., Wang, Y., Tang, Z., Chen, Y., Cai, L., & Ling, H. (2019). M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network. CoRR, abs/1811.04533.
  • Kong, T., Sun, F., Liu, H., Jiang, Y., & Shi, J. (2019). Consistent Optimization for Single-Shot Object Detection. CoRR, abs/1901.06563. [PDF] 针对物体检测one-stage中的一些改进方案
  • (RepMet) Schwartz, Eli et al. “RepMet: Representative-based metric learning for classification and one-shot object detection.” CoRR abs/1806.04728 (2018): n. pag.[PDF]
  • (Guided Anchor) Wang, Jiaqi et al. “Region Proposal by Guided Anchoring.” CoRR abs/1901.03278 (2019): n. pag. [PDF] 提出了一种新型的学习式的anchor生成方案
  • (FCOS) Tian, Z., Shen, C., Chen, H., & He, T. (2019). FCOS: Fully Convolutional One-Stage Object Detection. [PDF]
  • (KL-Loss) He, Y., Zhu, C., Wang, J., Savvides, M., & Zhang, X. (2018). Bounding Box Regression with Uncertainty for Accurate Object Detection. [PDF]
  • (ScrathDet) Zhu, Rui et al. “ScratchDet: Exploring to Train Single-Shot Object Detectors from Scratch.” CoRR abs/1810.08425 (2018): n. pag.[PDF]
  • Zhu, Chenchen et al. “Feature Selective Anchor-Free Module for Single-Shot Object Detection.” CoRR abs/1903.00621 (2019): n. pag.[PDF]
  • (FoveaBox) Kong, Tao & Sun, Fuchun & Liu, Huaping & Jiang, Yuning & Shi, Jianbo. (2019). FoveaBox: Beyond Anchor-based Object Detector. [PDF]
  • (Libra R-CNN) Pang, Jiangmiao, Kai Chen, Jianping Shi, Huajun Feng, Wanli Ouyang and Dahua Lin. “Libra R-CNN: Towards Balanced Learning for Object Detection.” (2019). [PDF]
  • (R-DAD) Bae, Seung-Hwan. “Object Detection based on Region Decomposition and Assembly.” CoRR abs/1901.08225 (2019): n. pag.[PDF]
  • Saito, Kuniaki et al. “Strong-Weak Distribution Alignment for Adaptive Object Detection.” CoRR abs/1812.04798 (2018): n. pag. [PDF]
  • (AP-Loss) Chen, K. , Li, J. , Lin, W. , See, J. , Wang, J. , & Duan, L. , et al. (2019). Towards accurate one-stage object detection with ap-loss. [PDF]
  • (RepPoints) Yang, Ze et al. “RepPoints: Point Set Representation for Object Detection.” CoRR abs/1904.11490 (2019): n. pag. [PDF]
  • (YOLOv3+) Derakhshani, M.M., Masoudnia, S., Shaker, A.H., Mersa, O., Sadeghi, M.A., Rastegari, M., & Araabi, B.N. (2019). Assisted Excitation of Activations: A Learning Technique to Improve Object Detectors.[PDF] 将分割等辅助信息引入优化物体检测的分支中,提高检测效果
  • (PASSD) Jang, H., Woo, S., Benz, P., Park, J., & Kweon, I.S. (2019). Propose-and-Attend Single Shot Detector.[PDF]
  • (DR-Loss) Qian, Q., Lei, C., Li, H., & Jin, R. (2019). DR Loss: Improving Object Detection by Distributional Ranking. ArXiv, abs/1907.10156.[PDF]
  • Chen, Joya et al. “Are Sampling Heuristics Necessary in Object Detectors?” (2019).[PDF]
  • (CBNet) Liu, Y., Wang, Y., Wang, S., Liang, T., Zhao, Q., Tang, Z., & Ling, H. (2019). CBNet: A Novel Composite Backbone Network Architecture for Object Detection.[PDF]

深度学习一些tricks以及CNN网络结构的改善

  • (BatchNorm) Ioffe, Sergey and Christian Szegedy. “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.” ICML (2015).[PDF]训练过程中的大杀器,可以加速模型收敛,并且训练过程数值更加稳定
  • Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, Mu Li:Bag of Tricks for Image Classification with Convolutional Neural Networks. CoRR abs/1812.01187 (2018) [PDF] 系统的介绍了许多训练CNN的trick
  • Zhang, Z., He, T., Zhang, H., Zhang, Z., Xie, J., & Li, M. (2019). Bag of Freebies for Training Object Detection Neural Networks. CoRR, abs/1902.04103.[PDF] 系统的介绍了目标检测过程中的许多tricks
  • (RePr) Prakash, A., Storer, J.A., Florêncio, D.A., & Zhang, C. (2018). RePr: Improved Training of Convolutional Filters. CoRR, abs/1811.07275.[PDF]
  • (WS) Weight Standardization Siyuan Qiao, Huiyu Wang, Chenxi Liu, Wei Shen, Alan Yuille [PDF]
  • Santurkar, S., Tsipras, D., Ilyas, A., & Madry, A. (2018). How Does Batch Normalization Help Optimization? NeurIPS.[PDF]
  • (DML) Zhang, Ying et al. “Deep Mutual Learning.” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018): 4320-4328. [PDF]
  • (Softer-NMS) He, Yihui et al. “Bounding Box Regression with Uncertainty for Accurate Object Detection.” (2019). [PDF]
  • (AutoAugment) Cubuk, E.D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q.V. (2018). AutoAugment: Learning Augmentation Policies from Data. CoRR, abs/1805.09501.[PDF]
  • Zoph, Barret et al. “Learning Data Augmentation Strategies for Object Detection.” ArXiv abs/1906.11172 (2019): n. pag. [PDF]
  • Zhang, Haichao and Jianyu Wang. “Towards Adversarially Robust Object Detection.” (2019). [PDF]
  • (AlignDet) Chen, Yuntao et al. “Revisiting Feature Alignment for One-stage Object Detection.” (2019).[PDF]
  • (Attention Normalization) Li, Xilai, Wei Sun and Tianfu Wu. “Attentive Normalization.” (2019).[PDF]
  • (IoU-Balanced Loss) IoU-balanced Loss Functions for Single-stage Object Detection

人脸检测

  • (SSH) Najibi, Mahyar et al. “SSH: Single Stage Headless Face Detector.” 2017 IEEE International Conference on Computer Vision (ICCV) (2017): 4885-4894. [PDF]
  • (S3S^3FD) Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., & Li, S.Z. (2017). S^3FD: Single Shot Scale-Invariant Face Detector. 2017 IEEE International Conference on Computer Vision (ICCV), 192-201. [PDF] 使用了锚框匹配策略和max-out增大了小尺寸人脸的召回率和假正例
  • (DSFD) Wang, Y., Wang, C., Tai, Y., Qian, J., Yang, J., Wang, C., Li, J., & Huang, F. (2018). DSFD: Dual Shot Face Detector. CoRR, abs/1810.10220. [PDF]
  • (PyramidBox++) Li, Zhihang et al. “PyramidBox++: High Performance Detector for Finding Tiny Face.” (2019). [PDF]
  • Zhang, F., Fan, X., Ai, G.P., Song, J., Qin, Y., & Wu, J. (2019). Accurate Face Detection for High Performance.[PDF]
  • (SRN) Chi, C., Zhang, S., Xing, J., Lei, Z., Li, S.Z., & Zou, X. (2019). Selective Refinement Network for High Performance Face Detection. CoRR, abs/1809.02693.[PDF]

姿态估计

  • (DeepPose) Toshev, A., & Szegedy, C. (2014). DeepPose: Human Pose Estimation via Deep Neural Networks. 2014 IEEE Conference on Computer Vision and Pattern Recognition, 1653-1660.[PDF]

图像分割

GAN生成对抗网络

  • (GAN) Goodfellow, I.J. (2016). NIPS 2016 Tutorial: Generative Adversarial Networks. CoRR, abs/1701.00160. [PDF] 入门级经典论文,里面详细介绍了GAN的数学原理
  • (ConditionalGAN) Mirza, M., & Osindero, S. (2014). Conditional Generative Adversarial Nets. CoRR, abs/1411.1784.[PDF]
  • (DCGAN) Radford, A., Metz, L., & Chintala, S. (2016). Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. CoRR, abs/1511.06434. [PDF]
  • (WGAN) Arjovsky, Martín, Soumith Chintala and Léon Bottou. “Wasserstein GAN.” CoRR abs/1701.07875 (2017): n. pag.[PDF]
  • (WGAN-GP) Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A.C. (2017). Improved Training of Wasserstein GANs. NIPS. [PDF]
  • (BiGAN) Donahue, J., Krähenbühl, P., & Darrell, T. (2017). Adversarial Feature Learning. CoRR, abs/1605.09782.[PDF]
  • (CycleGAN) Zhu, Jun-Yan et al. “Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks.” 2017 IEEE International Conference on Computer Vision (ICCV) (2017): 2242-2251.[PDF]
  • Liu, Ming-Yu et al. “Unsupervised Image-to-Image Translation Networks.” NIPS (2017).[PDF]
  • (PG-GAN) Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2018). Progressive Growing of GANs for Improved Quality, Stability, and Variation. CoRR, abs/1710.10196.[PDF]
  • (Sim-GAN) Shrivastava, Ashish et al. “Learning from Simulated and Unsupervised Images through Adversarial Training.” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017): 2242-2251. [PDF]
  • (Pixel-DA) Unsupervised Pixel–Level Domain Adaptation with Generative Adversarial Networks [PDF]
  • (Co-GAN) Liu, Ming-Yu and Oncel Tuzel. “Coupled Generative Adversarial Networks.” NIPS (2016).[PDF]
  • (DTN) Taigman, Yaniv et al. “Unsupervised Cross-Domain Image Generation.” CoRR abs/1611.02200 (2017): n. pag.[PDF]
  • (SN-GAN) Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y. (2018). Spectral Normalization for Generative Adversarial Networks. CoRR, abs/1802.05957.[PDF]
  • (Pizza-GAN) Papadopoulos, Dim P. et al. “How to make a pizza: Learning a compositional layer-based GAN model.” ArXiv abs/1906.02839 (2019): n. pag.[PDF]
  • (ENlightenGAN) Jiang, Yifan, Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang, Xiaohui Shen, Jianchao Yang, Pan Zhou and Zhangyang Wang. “EnlightenGAN: Deep Light Enhancement without Paired Supervision.” (2019).[PDF]

知识蒸馏

  • Hinton, Geoffrey E., Oriol Vinyals and Jeffrey Dean. “Distilling the Knowledge in a Neural Network.” CoRR abs/1503.02531 (2015): n. pag.[[PDF]](Hinton, Geoffrey E., Oriol Vinyals and Jeffrey Dean. “Distilling the Knowledge in a Neural Network.” CoRR abs/1503.02531 (2015): n. pag.)
  • (CCKD) Peng, Baoyun et al. “Correlation Congruence for Knowledge Distillation.” ArXiv abs/1904.01802 (2019): n. pag.[PDF]
  • Yuan, L., Tay, F.E., Li, G., Wang, T., & Feng, J. (2019). Revisit Knowledge Distillation: a Teacher-free Framework. ArXiv, abs/1909.11723.[PDF]
  • Tian, Y., Krishnan, D., & Isola, P. (2019). Contrastive Representation Distillation. ArXiv, abs/1910.10699.[PDF]
  • Liu, X., He, P., Chen, W., & Gao, J. (2019). Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding. ArXiv, abs/1904.09482.[PDF]
  • (BAM) Clark, K., Luong, M., Khandelwal, U., Manning, C.D., & Le, Q.V. (2019). BAM! Born-Again Multi-Task Networks for Natural Language Understanding. ACL.[PDF]

NLP

  • (ELMo) Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep Contextualized Word Representations. ArXiv, abs/1802.05365.[PDF]
  • (GPT) Radford, Alec. “Improving Language Understanding by Generative Pre-Training.” (2018).[PDF]
  • (Transformer) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., & Polosukhin, I. (2017). Attention Is All You Need. NIPS. [PDF]
  • (BERT) Devlin, J., Chang, M., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. ArXiv, abs/1810.04805.[PDF]
  • (Transformer-XL) Dai, Zihang et al. “Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context.” ArXiv abs/1901.02860 (2019): n. pag.[PDF]
  • (XLNet) Yang, Zhilin, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov and Quoc V. Le. “XLNet: Generalized Autoregressive Pretraining for Language Understanding.” ArXiv abs/1906.08237 (2019): n. pag. [PDF]
  • (Region Embedding) Johnson, Rie and Tong Zhang. “Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding.” Advances in neural information processing systems 28 (2015): 919-927 . [PDF]
  • (DPCNN) Johnson, R., & Zhang, T. (2017). Deep Pyramid Convolutional Neural Networks for Text Categorization. ACL.[PDF]
  • (BERT Augmentation) Wu, Xing et al. “Conditional BERT Contextual Augmentation.” ArXiv abs/1812.06705 (2018): n. pag.[PDF]
  • (Few-shot Learning Induction Net) Geng, R., Li, B., Li, Y., Ye, Y., Jian, P., & Sun, J. (2019). Few-Shot Text Classification with Induction Network. ArXiv, abs/1902.10482.[PDF]
  • Zhang, Ye and Byron C. Wallace. “A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification.” IJCNLP (2015).[PDF]
  • (Attention Survey) Chaudhari, S., Polatkan, G., Ramanath, R., & Mithal, V. (2019). An Attentive Survey of Attention Models. ArXiv, abs/1904.02874.[PDF]
  • (TB-CNN) Short Text Classification Improved by Feature Space Extension [PDF]
  • (QA-Net) Yu, A.W., Dohan, D., Luong, M., Zhao, R., Chen, K., Norouzi, M., & Le, Q.V. (2018). QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension. ArXiv, abs/1804.09541.[PDF]
  • (BIDAF) Seo, M.J., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016). Bidirectional Attention Flow for Machine Comprehension. ArXiv, abs/1611.01603.[PDF]
  • (CNN-Seq2Seq) Gehring, Jonas et al. “Convolutional Sequence to Sequence Learning.” ICML (2017).[PDF]
  • (PointNet) Vinyals, Oriol, Meire Fortunato, and Navdeep Jaitly. “Pointer networks.” Advances in Neural Information Processing Systems. 2015.[PDF] 可以很好地解决OOV ( out of vocabulary ) 问题
  • Liu, Shanshan et al. “Neural Machine Reading Comprehension: Methods and Trends.” ArXiv abs/1907.01118 (2019): n. pag.[PDF]
  • Hu, M., Wei, F., Peng, Y., Huang, Z., Lau, V.K., & Li, D. (2018). Read + Verify: Machine Reading Comprehension with Unanswerable Questions. ArXiv, abs/1808.05759.[PDF]
  • (U-Net) Sun, F., Li, L., Qiu, X., & Liu, Y.P. (2018). U-Net: Machine Reading Comprehension with Unanswerable Questions. ArXiv, abs/1810.06638.[PDF]
  • (MIX) Chen, H., Han, F.X., Niu, D., Liu, D., Lai, K., Wu, C., & Xu, Y. (2018). MIX: Multi-Channel Information Crossing for Text Matching. KDD.[PDF]
  • Rei, Marek and Anders Søgaard. “Jointly Learning to Label Sentences and Tokens.” AAAI (2018).[PDF]
  • Zhang, Xuchao, Fanglan Chen, Chang-Tien Lu and Naren Ramakrishnan. “Mitigating Uncertainty in Document Classification.” NAACL-HLT (2019).[PDF]
  • (SpanBERT) Joshi, M.S., Chen, D., Liu, Y., Weld, D.S., Zettlemoyer, L.S., & Levy, O. (2019). SpanBERT: Improving Pre-training by Representing and Predicting Spans. ArXiv, abs/1907.10529.[PDF]
  • (RoBERTa) Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M.S., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L.S., & Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv, abs/1907.11692.[PDF]
  • (MT-DNN) Liu, X., He, P., Chen, W., & Gao, J. (2019). Multi-Task Deep Neural Networks for Natural Language Understanding. ACL.[PDF]
  • (ERNIE) Zhang, Zhengyan et al. “ERNIE: Enhanced Language Representation with Informative Entities.” ACL (2019).[PDF]
  • (ERNIE2.0) Sun, Yu, Shuohuan Wang, Yukun Li, Shikun Feng, Hao Tian, Hua Wu and Haifeng Wang. “ERNIE 2.0: A Continual Pre-training Framework for Language Understanding.” ArXiv abs/1907.12412 (2019): n. pag.[PDF]
  • (RE2) Yang, R., Zhang, J., Gao, X., Ji, F., & Chen, H. (2019). Simple and Effective Text Matching with Richer Alignment Features. ACL.[PDF]
  • (TinyBERT) Jiao, Xiaoqi et al. “TinyBERT: Distilling BERT for Natural Language Understanding.” ArXiv abs/1909.10351 (2019): n. pag.[PDF]
  • (RCNN) Choi, Keunwoo et al. “Convolutional recurrent neural networks for music classification.” 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2016): 2392-2396.[PDF]
  • (GEAR) Zhou, J., Han, X., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2019). GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification. ACL.[PDF]
  • (CogNet) Ding, M., Zhou, C., Chen, Q., Yang, H., & Tang, J. (2019). Cognitive Graph for Multi-Hop Reading Comprehension at Scale. ACL.[PDF]
  • Tan, C., Wei, F., Wang, W., Lv, W., & Zhou, M. (2018). Multiway Attention Networks for Modeling Sentence Pairs. IJCAI.[PDF]
  • Deshmukh, Neil et al. “Semi-Supervised Natural Language Approach for Fine-Grained Classification of Medical Reports.” (2019).[PDF]
  • (ELECTRA) ELECTRA: PRE-TRAINING TEXT ENCODERS AS DISCRIMINATORS RATHER THAN GENERATORS [PDF]
  • (BART) Lewis, Mike, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov and Luke Zettlemoyer. “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension.” (2019).[PDF]
  • Nguyen, T.H., & Grishman, R. (2015). Relation Extraction: Perspective from Convolutional Neural Networks. VS@HLT-NAACL.[PDF]
  • Santos, Cícero Nogueira dos et al. “Classifying Relations by Ranking with Convolutional Neural Networks.” ACL (2015).[[PDF]](Santos, Cícero Nogueira dos et al. “Classifying Relations by Ranking with Convolutional Neural Networks.” ACL (2015).)
  • Wang, Linlin et al. “Relation Classification via Multi-Level Attention CNNs.” ACL (2016).[PDF]
  • Soares, Livio Baldini et al. “Matching the Blanks: Distributional Similarity for Relation Learning.” ACL (2019).[PDF]
  • Wu, Shanchan and Yifan He. “Enriching Pre-trained Language Model with Entity Information for Relation Classification.” CIKM (2019). [PDF]
  • Eberts, Markus and Adrian Ulges. “Span-based Joint Entity and Relation Extraction with Transformer Pre-training.” ArXiv abs/1909.07755 (2019): n. pag.[PDF]
  • Alt, Christoph et al. “Improving Relation Extraction by Pre-trained Language Representations.” ArXiv abs/1906.03088 (2019): n. pag.[PDF]
  • Xue, K., Zhou, Y., Ma, Z., Ruan, T., Zhang, H., & He, P. (2019). Fine-tuning BERT for Joint Entity and Relation Extraction in Chinese Medical Text. ArXiv, abs/1908.07721.[PDF]

(LSTM、RNN) 训练注意事项:

  1. 初始化begin state
  2. 梯度裁剪
  3. hidden state 截断时间反向传播 state.detach()

Ranking

  • (LambdaRank) Learning to Rank with Nonsmooth Cost Functions
  • (LTR) From RankNet to LambdaRank to LambdaMART: An Overview [PDF]
  • (LambdaLoss) The LambdaLoss Framework for Ranking Metric Optimization
  • Understanding the Behaviors of BERT in Ranking
  • PASSAGE RE-RANKING WITH BERT

知识图谱

  • Wang, Quan et al. “Knowledge Graph Embedding: A Survey of Approaches and Applications.” IEEE Transactions on Knowledge and Data Engineering 29 (2017): 2724-2743.[PDF]
  • (FastText) Joulin, A., Grave, E., Bojanowski, P., Nickel, M., & Mikolov, T. (2017). Fast Linear Model for Knowledge Graph Embeddings. ArXiv, abs/1710.10881.[PDF]
  • Moussallem, Diego et al. “Augmenting Neural Machine Translation with Knowledge Graphs.” ArXiv abs/1902.08816 (2019): n. pag.[PDF]
  • (ConvKB) Nguyen, Dai Quoc et al. “A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network.” NAACL-HLT (2017).[PDF]
  • Cheng, H., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G.S., Chai, W., Ispir, M., Anil, R., Haque, Z., Hong, L., Jain, V., Liu, X., & Shah, H. (2016). Wide & Deep Learning for Recommender Systems. DLRS@RecSys.[PDF]

图卷积神经网络

  • (DeepGCNs) Li, G., Muller, M., Thabet, A., & Ghanem, B. (2019). DeepGCNs: Can GCNs Go as Deep as CNNs?[PDF]
  • (SGN) Wu, Felix, Tianyi Zhang, Amauri H. de Souza, Christopher Fifty, Tao Yu and Kilian Q. Weinberger. “Simplifying Graph Convolutional Networks.” ICML (2019). [PDF]

参考博客:https://blog.csdn.net/qq_21190081/article/details/69564634

2019-10-02 09:07:10 qidailiming1994 阅读数 160
  • 大数据:深度学习项目实战-人脸检测视频教程

    购买课程后,请扫码入学习群,获取唐宇迪老师答疑 进军深度学习佳项目实战:人脸检测项目视频培训课程,从数据的收集以及预处理开始,一步步带着大家完成整个人脸检测的项目,其中涉及了如何使用深度学习框架Caffe完成整个项目的架构,对于每一个核心步骤详细演示流程和原理解读,在完成检测代码之后进行了详细评估分析,并给出一篇顶级会议论文作为学习参考,详细分析了针对人脸检测项目的优缺点和改进策略。

    16412 人正在学习 去看看 唐宇迪

搞深度学习如何找到需要的代码

此博客整理了查找所需要代码的最佳方法以及与如何找到paper相关联的网址,都是非常不错的网址。

  • 此网址集合了arXiv上最新的关于深度学习或者机器学习的研究论文并且关联了code,还可以直接看star 非常不错。
    Paperswithcode
  • 此网址可以利用扩展工具查找code
    research code
  • 计算机界的Google scholar ,非常好用,体验过你会爱上它,不仅包括源代码和复现代码,还可以垂直搜索到同领域中类似的文章。
    semanticscholar
  • 其他建议
    遇到很难找的文章源码可以尝试在论文作者首页查找,经常会有意外的发现。
2019-01-20 18:41:53 qq_40859461 阅读数 330
  • 大数据:深度学习项目实战-人脸检测视频教程

    购买课程后,请扫码入学习群,获取唐宇迪老师答疑 进军深度学习佳项目实战:人脸检测项目视频培训课程,从数据的收集以及预处理开始,一步步带着大家完成整个人脸检测的项目,其中涉及了如何使用深度学习框架Caffe完成整个项目的架构,对于每一个核心步骤详细演示流程和原理解读,在完成检测代码之后进行了详细评估分析,并给出一篇顶级会议论文作为学习参考,详细分析了针对人脸检测项目的优缺点和改进策略。

    16412 人正在学习 去看看 唐宇迪

含论文下载链接,部分含有代码链接,持续整理中…

经典论文

基础网络

Alexnet

ImageNet Classification with Deep Convolutional Neural Networks.pdf
深度学习兴起的引子

VGG

Very Deep Convolutional Networks for Large-Scale Image Recognition
使用 3*3 卷积减少参数量的深层网络

GoogleNet,Inception系列

Densenet

Densely Connected Convolutional Networks

參差系列

何恺明个人主页http://kaiminghe.com/

轻量级网络

LOSS FUNCTION

AM-Softmax

A-Softmax

L-Softmax

Object Detection

Semantic Segmentation

OTHERS

A guide to convolution arithmetic for deep learning

上采样的优点

Escaping From Saddle Points –Online Stochastic Gradient for Tensor Decomposition

batch的优点

Dynamic Curriculum Learning for Imbalanced Data Classification

商汤的不均衡样本分类文章

Data Distillation: Towards Omni-Supervised Learning

数据蒸馏,一种使用无标签数据训练的全方位学习方法,在Kaggle等大数据竞赛中非常有用

IQA

No-reference Image Quality Assessment 相关论文,包括人脸姿态估计

DeepLearning-500-questions

四川大学深度学习500问,包含了深度学习数学基础、经典框架、常见问题等
github

2018-10-16 16:11:14 ScorpC 阅读数 469
  • 大数据:深度学习项目实战-人脸检测视频教程

    购买课程后,请扫码入学习群,获取唐宇迪老师答疑 进军深度学习佳项目实战:人脸检测项目视频培训课程,从数据的收集以及预处理开始,一步步带着大家完成整个人脸检测的项目,其中涉及了如何使用深度学习框架Caffe完成整个项目的架构,对于每一个核心步骤详细演示流程和原理解读,在完成检测代码之后进行了详细评估分析,并给出一篇顶级会议论文作为学习参考,详细分析了针对人脸检测项目的优缺点和改进策略。

    16412 人正在学习 去看看 唐宇迪

感谢:SnailTyan

https://github.com/SnailTyan/deep-learning-papers-translation

这个Github库 “Deep Learning Papers Translation” 提供了经典深度学习论文的英文版、中文版,以及中英文对照版。尤其是后者,对于想快速准确学习的读者而言非常适合。

如下:(直接点击上述GitHub地址,及时更新)

Deep Learning Papers Translation(CV)

Image Classification

Object Detection

OCR

Mobile

  • MobileNetV2
    To be added.
没有更多推荐了,返回首页