精华内容
参与话题
问答
  • ResNet

    千次阅读 2018-08-13 23:41:59
    ###看文章的时候为什么优化残差更容易这个问题困扰了我很久,所以下面主要记录这个问题的理解以及resnet脉络的整理。 Motivation: 1.神经网络的结构并不是越深越好,随着网络的加深伴随着严重的梯度下降和梯度...

    ###看文章的时候为什么优化残差更容易这个问题困扰了我很久,所以下面主要记录这个问题的理解以及resnet脉络的整理。

    Motivation:

    1.神经网络的结构并不是越深越好,随着网络的加深伴随着严重的梯度下降和梯度爆炸的问题,目前batch normalization等方法可以解决梯度消失和梯度爆炸的问题。但是使用了各种normalization的方法也不能是深层网络的效果好于浅层网络。

    2.理论上,若A为浅层网络,B为深层网络,且B的浅层结构完全复制于A,后几层为线性层(identity mapping),那么B网络的效果应该是和A的相同的。但是实验发现,A网络的训练准确率反而比浅层网络要低,这说明在实际应用时,高层的这种线性关系很难学到,也就是出现了文中提到的degradation problem(退化)。训练集准确率下降的原因肯定不是过拟合,因为过拟合的话训练集的准确率应该很高。由此出发,我们将这种线性关系加到网络的学习中,最后学出来的网络效果应该大于等于浅层网络的效果,也可以认为,学习这种线性映射会更加容易。

    Residual learning:

    identity mapping:x->x

    residual mapping:F\left ( x \right )=H(x)-x

    H\left ( x \right ) :desired underlying mapping——期望拟合的特征图,一个building block要拟合的就是这个特征图,未使用残差网络时,F\left ( x \right ) 的目标是拟合H\left ( x \right )。使用参差网络后F\left ( x \right )的目标是拟合H\left ( x \right )-x,后者比前者更容易优化!

    解释:

    假设我们认为deeper层相较于shallower层的有一些层是冗余的,那我们的学习目标应该是使这些层变成线性层,这样才能不影响网络的性能。那么拟合的目标就变成了:H\left ( x \right )=x,即F(x)=0,这要比之前的F(x)=x更容易拟合!

     

     

    展开全文
  • Resnet

    万次阅读 多人点赞 2016-10-26 11:59:34
    再上一偏博文中我们说到越复杂的问题需要越深层的神经网络拟合,但是越深层的神经网络越难训练,原因可能是过拟合以及损失函数的局部最优解过多(鞍点过多?导致经过相同的epoch更深的网络的trainerror大于较浅的...

    再上一偏博文中我们说到越复杂的问题需要越深层的神经网络拟合,但是越深层的神经网络越难训练,原因可能是过拟合以及损失函数的局部最优解过多(鞍点过多?导致经过相同的epoch更深的网络的trainerror大于较浅的网络,因为过鞍点需要更多的epoch,鞍点附近梯度很小,相关文献The Loss Surfaces of Multilayer Networks)导致模型更容易收敛到局部最优解。上一篇博文讲到可以把复杂问题分解成几个较简单的问题,然后分别训练几个较简单的模型,最后讲几个模型的输出连接起来送入FC层分类。
    Resnet采用了另一种方法,也就是不分解原问题,而是把神经网络分解来降低拟合函数的复杂度,而损失函数是拟合函数的函数,所以降低拟合函数的复杂度也等同于降低了损失函数的复杂度。假设我们需要拟合的复杂函数为H(x),现在我们把H(x)分解为两个更简单的函数f(x)和g(x),即令H(x) = f(x) + g(x)。(Resnet不考虑过拟合)此处f(x)函数的结构和H(x)其实是一致的,都是几层卷积池化加上最后的fc层,但是因为有g(x)的加入使得f(x)的复杂度小于H(x).具体如下图所示:

    这里写图片描述

    由于我们并不知道什么样的g(x)最好,所以这里g(x)也需要训练,而一个三层的神经网络可以拟合连续闭区间上的任意函数,我们用一个三层的神经网络代替g(x),但是这样又会因为增加了额外的参数造成过拟合的风险,我们看到图中有两个三层的神经网络,我们可以把它们合并起来,所以一个更好的方法如下:
    这里写图片描述

    这样一来由于fc层的输入增加为了维持表达能力,fc层的隐层单元势必要增加,这样还是增加了参数,所以Resnet又做了调整:
    这里写图片描述

    展开全文
  • ResNet结构分析

    万次阅读 多人点赞 2018-11-08 11:38:06
    ResNet封装 在torchvision中封装了Resnet的源码,下面通过对ResNet源码的分析进一步了解ResNet网络结构,方便对ResNet结构进行修改,同时学习网络源码的组织方式,方便日后搭建自己的神经网络 源码解析 def resnet...

    源码

    ResNet封装 在torchvision中封装了Resnet的源码,下面通过对ResNet源码的分析进一步了解ResNet网络结构,方便对ResNet结构进行修改,同时学习网络源码的组织方式,方便日后搭建自己的神经网络

    在这里插入图片描述

    源码解析

    def resnet50(pretrained=False, **kwargs):
        """Constructs a ResNet-50 model.
        Args:
            pretrained (bool): If True, returns a model pre-trained on ImageNet
        """
        model = ResNet(Bottleneck, [3, 4, 6, 3], **kwargs)
        if pretrained:
            model.load_state_dict(model_zoo.load_url(model_urls['resnet50']))
        return model
    

    从源码的入口出发,通过 model = ResNet(Bottleneck, [3, 4, 6, 3], **kwargs) 构造网络结构,主要分成两个部分即 Bottleneck 和 [3,4,6,3] 由这两个参数共同决定了ResNet50的网络结构 ,当pretrained 为true时,为model加载imageNet中预训练的参数。
    这里涉及到了Bottleneck这个类,[3,4,6,3]对应于上图中ResNet50中 conv2_x中 有三个(1164,3364,11256)卷积层的堆叠 ,同理conv3_中有4个(11128,33128,11512)卷积层的堆叠,resnet将卷积层分为4个大层,[3,4,6,3]代表每一个大层中11,33,1*1 卷积层组合的重复次数 总共1(第一个卷积层)+1(第一个池化层)+(3+4+6+3)*3=50层

    这里涉及到一个Bottleneck类,可以把一个Bottleneck当成一个基础的block就是对应上图的(11,33,11)卷积核大小的卷积层的组合
    在这里插入图片描述
    解释一下为什么输入Bottleneck之前是56 * 56 * 64 的,因为ResNet接受的图像大小为224 * 224 经过第一层卷积层
    self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
    floor((224-7+2
    3)/2)+1=112
    经过第一层池化之后
    self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
    floor((112+2*1-3)/2)+1=56

    因此在输入到Bottleneck之前得到一个56(height)*56(weight)*64(channel)大小的feature map

    
    class Bottleneck(nn.Module):
        expansion = 4
    
        def __init__(self, inplanes, planes, stride=1, downsample=None):
            super(Bottleneck, self).__init__()
            self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)
            self.bn1 = nn.BatchNorm2d(planes)
            self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride,
                                   padding=1, bias=False)
            self.bn2 = nn.BatchNorm2d(planes)
            self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)
            self.bn3 = nn.BatchNorm2d(planes * self.expansion)
            self.relu = nn.ReLU(inplace=True)
            self.downsample = downsample
            self.stride = stride
    

    因为1 * 1卷积核不改变feature map的大小,3 * 3卷积核padding=1 也不改变输入feature map的大小,因此经过一个Bottleneck组成的卷积层组操作后feature map大小不会改变

    下面看一下Bottleneck的forward函数

    def forward(self, x):
            residual = x
            out = self.conv1(x)
            out = self.bn1(out)
            out = self.relu(out)
            out = self.conv2(out)
            out = self.bn2(out)
            out = self.relu(out)
            out = self.conv3(out)
            out = self.bn3(out)
            if self.downsample is not None:
                residual = self.downsample(x)
            out += residual
            out = self.relu(out)
            return out
    

    这里要留意一下downsample,因为feature map的大小不变 但是在经过Bottleneck 之后channel变成了原来的四倍,因此想要和原始的feature map相加 需要将原始的feature map 也变为原来的四倍 ,downsample 作用是residual+当前feature map时将维度统一

    接下来分析ResNet 类的具体构成

    class ResNet(nn.Module):
    
        def __init__(self, block, layers, num_classes=1000):
            self.inplanes = 64
            super(ResNet, self).__init__()
            self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3,
                                   bias=False)
            self.bn1 = nn.BatchNorm2d(64)
            self.relu = nn.ReLU(inplace=True)
            self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
            self.layer1 = self._make_layer(block, 64, layers[0])
            self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
            self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
            self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
            self.avgpool = nn.AvgPool2d(7, stride=1)
            self.fc = nn.Linear(512 * block.expansion, num_classes)
    
            for m in self.modules():
                if isinstance(m, nn.Conv2d):
                    nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
                elif isinstance(m, nn.BatchNorm2d):
                    nn.init.constant_(m.weight, 1)
                    nn.init.constant_(m.bias, 0)
    
        def _make_layer(self, block, planes, blocks, stride=1):
            downsample = None
            if stride != 1 or self.inplanes != planes * block.expansion:
                downsample = nn.Sequential(
                    nn.Conv2d(self.inplanes, planes * block.expansion,
                              kernel_size=1, stride=stride, bias=False),
                    nn.BatchNorm2d(planes * block.expansion),
                )
    
            layers = [ ]
            layers.append(block(self.inplanes, planes, stride, downsample))
            self.inplanes = planes * block.expansion
            for i in range(1, blocks):
                layers.append(block(self.inplanes, planes))
    
            return nn.Sequential(*layers)
    
    

    Resnet 也是由__init__ 和forward构成 ,为了方便分析 这里首先分析init函数,在init中最重要的是_make_layer 函数,以layer1为例
    block为Bottleneck,planes=64(即channel数目)blocks=3 ([3,4,6,3] 分别代表每一层的blocks数目)这里要注意layer1的stride为1 其他layer的stride为2
    对于layer1而言 inplanes=64 planes=64 block.expansion=4,因此需要经过downsample 才能够使得残差和经过该层的feature map能够相加,downsample即为右路部分

    在这里插入图片描述
    这样说其实还是不太清晰 ,最直观的方法就是分析每一层究竟发生了什么变化
    layer1:
    输入 :[batch_size,56,56,64]
    此时self.inplanes=64 planes * block.expansion=644不相等 (之所以要二者相等时在Bottleneck最后一个卷积层会将channel变为 planes乘block.expansion 如果inplanes(实际就是输入的channel)与之不相等 不可相加)
    构造右路downsample (1
    1卷积核的卷积层 扩展channel +BN层)

    构造layer1 3个Bottleneck中的第一个:

    主体分支
    在这里插入图片描述.
    downsample 分支
    在这里插入图片描述
    更新 inplanes=64*4

    构造layer1 3个Bottleneck中的第2个

    self.conv1 = nn.Conv2d(256, 64, kernel_size=1, bias=False)
    self.bn1 = nn.BatchNorm2d(64)
    self.conv2 = nn.Conv2d(64, 64, kernel_size=3, stride=stride,
    padding=1, bias=False)
    self.bn2 = nn.BatchNorm2d(64)
    self.conv3 = nn.Conv2d(64,256, kernel_size=1, bias=False)
    self.bn3 = nn.BatchNorm2d(planes * self.expansion)
    self.relu = nn.ReLU(inplace=True)
    self.downsample = None
    self.stride = 1

    构造layer1 3个Bottleneck中的第3个

    self.conv1 = nn.Conv2d(256, 64, kernel_size=1, bias=False)
    self.bn1 = nn.BatchNorm2d(64)
    self.conv2 = nn.Conv2d(64, 64, kernel_size=3, stride=stride,
    padding=1, bias=False)
    self.bn2 = nn.BatchNorm2d(64)
    self.conv3 = nn.Conv2d(64,256, kernel_size=1, bias=False)
    self.bn3 = nn.BatchNorm2d(planes * self.expansion)
    self.relu = nn.ReLU(inplace=True)
    self.downsample = None
    self.stride = 1

    构造layer2 4个Bottleneck中的第一个

    此时stride=2 self.inplanes=256 planes * block.expansion=128*4
    需要生成downsample层
    downsample = nn.Sequential(
    nn.Conv2d(256 ,512
    kernel_size=1, stride=2, bias=False),
    nn.BatchNorm2d(512),
    )

    生成第一个Bottleneck的主干
    self.conv1 = nn.Conv2d(256, 128, kernel_size=1, bias=False)
    self.bn1 = nn.BatchNorm2d(planes)
    self.conv2 = nn.Conv2d(128, 128, kernel_size=3, stride=2, //此时feature map 大小由56变成28
    padding=1, bias=False)
    self.bn2 = nn.BatchNorm2d(planes)
    self.conv3 = nn.Conv2d(128, 512, kernel_size=1, bias=False)
    self.bn3 = nn.BatchNorm2d(planes * self.expansion)
    self.relu = nn.ReLU(inplace=True)
    self.downsample = downsample
    self.stride = stride

    构造layer2 4个Bottleneck中的第2,3,4个

    更新inplanes=512
    self.conv1 = nn.Conv2d(512, 128, kernel_size=1, bias=False)
    self.bn1 = nn.BatchNorm2d(64)
    self.conv2 = nn.Conv2d(128, 128, kernel_size=3, stride=1,
    padding=1, bias=False)
    self.bn2 = nn.BatchNorm2d(64)
    self.conv3 = nn.Conv2d(128,256, kernel_size=1, bias=False)
    self.bn3 = nn.BatchNorm2d(planes * self.expansion)
    self.relu = nn.ReLU(inplace=True)
    self.downsample = None
    self.stride = 1

    构造layer3 6 个Bottleneck中的第1个

    对于之后的layer3和layer4 都同理
    第一层会将feature map缩小2倍
    同时生成一个11卷积核 inchannel =inplanes outchannel=4planes的支路用来进行对输入的feature mapdownsample操作
    生成一个11 33(stride=2) 1*1 的卷积层组 对featuremap进行卷积操作 并与支路相加

    (0): Bottleneck(
    (conv1): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (relu): ReLU(inplace)
    (downsample): Sequential(
    (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)
    (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
    )
    剩余的Bottleneck均生成如下结构;
    (1): Bottleneck(
    (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (relu): ReLU(inplace)
    )

    整体结构如下:

    (net): ResNet(
        (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace)
        (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
        (layer1): Sequential(
          (0): Bottleneck(
            (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
            (downsample): Sequential(
              (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            )
          )
          (1): Bottleneck(
            (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
          (2): Bottleneck(
            (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
        )
        (layer2): Sequential(
          (0): Bottleneck(
            (conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
            (downsample): Sequential(
              (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
              (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            )
          )
          (1): Bottleneck(
            (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
          (2): Bottleneck(
            (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
          (3): Bottleneck(
            (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
        )
        (layer3): Sequential(
          (0): Bottleneck(
            (conv1): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
            (downsample): Sequential(
              (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)
              (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            )
          )
          (1): Bottleneck(
            (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
          (2): Bottleneck(
            (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
          (3): Bottleneck(
            (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
          (4): Bottleneck(
            (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
          (5): Bottleneck(
            (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
        )
        (layer4): Sequential(
          (0): Bottleneck(
            (conv1): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
            (downsample): Sequential(
              (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)
              (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            )
          )
          (1): Bottleneck(
            (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
          (2): Bottleneck(
            (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (relu): ReLU(inplace)
          )
        )
        (avgpool): AvgPool2d(kernel_size=7, stride=1, padding=0)  
        (fc): Linear(in_features=2048, out_features=1000, bias=True)
      )
    )
    
    最后看一下Resnet的forward函数:
    
    
    def forward(self, x):
        x = self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)
        x = self.maxpool(x)
    
        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)
    
        x = self.avgpool(x)
        x = x.view(x.size(0), -1)
        x = self.fc(x)
    
        return x
    
       self.avgpool = nn.AvgPool2d(7, stride=1)
      x = self.avgpool(x)  因为最后feature map在输入为224时 经过layer4之后大小为7乘7   ,此时经过 nn.AvgPool2d(7, stride=1)大小变为1乘1  再经过全连接层时
    self.fc = nn.Linear(512 * block.expansion, num_classes) 前者是输出的所有channel数目 实际应该为channel * 1 * 1 后者为分类数
      x = x.view(x.size(0), -1) 将数据拉伸成batchsize * channel * 1 * 1    
      如果输入大小不为224  那么相应的可以修改AvgPool2d 或者在全连接层第一个参数中乘上最终的width 和height
    

    Ref:[https://blog.csdn.net/jiangpeng59/article/details/79609392](https://blog.csdn.net/jiangpeng59/article/details/79609392)

    展开全文
  • resnet

    2019-09-26 11:18:44
    Resnet到底在解决一个什么问题呢? - 薰风初入弦的回答 - 知乎 https://www.zhihu.com/question/64494691/answer/786270699

    Resnet到底在解决一个什么问题呢? - 薰风初入弦的回答 - 知乎 https://www.zhihu.com/question/64494691/answer/786270699

    展开全文

空空如也

1 2 3 4 5 ... 20
收藏数 11,047
精华内容 4,418
关键字:

resnet