精华内容
下载资源
问答
  • 自己搭建的网络运行报错,不知道原因,希望大家指点一下,代码如下: <p>def conv_block(inputs,   filters,   use_bias,  kernel_size,  padding= 'same',  ...
  • 神经网络②——python实现神经网络

    千次阅读 2019-07-03 18:05:09
    上一篇博客介绍了神经网络其实就是两步,第一步是前向传输,根据输入的特征值与权重矩阵以及激活函数,不断往前计算,得到输出层的值,第二步就是后向传播,根据残差的加权求和,不断往后,直到输入层,然后更新权重...

    神经网络系列目录
    神经网络①——神经网络原理介绍(BP算法)
    神经网络③——sklearn参数介绍及应用
    神经网络实战④——主播综合评分回归预测实战
    最小二乘法&梯度下降法区别及python实现

    上一篇博客介绍了神经网络其实就是两步,第一步是前向传输,根据输入的特征值与权重矩阵以及激活函数,不断往前计算,得到输出层的值,第二步就是后向传播,根据残差的加权求和,不断往后,直到输入层,然后更新权重,不断进行这两步,直接输出结果的误差在所要求范围内即可

    本篇博文通过python代码实现神经网络,采用sklearn自带的手写数字集,看分类情况如何在这里插入图片描述

    一、导入数据集并标准化
    数据集的加载非常简单,直接调用load_digits即可,常用到的数据有:
    digits.data:手写数字特征向量数据集,每一个元素都是一个64维的特征向量。
    digits.target:特征向量对应的标记,每一个元素都是自然是0-9的数字。
    digits.images:对应着data中的数据,每一个元素都是8*8的二维数组,其元素代表的是灰度值,转化为以为是便是特征向量

    # python实现神经网络
    from sklearn.model_selection import train_test_split
    import numpy as np
    from sklearn.preprocessing import StandardScaler
    from sklearn.datasets import load_digits
    import matplotlib.pyplot as plt
    import seaborn as sns
    
    #导入数据集(dataset自带的手写数字数据集)
    data = load_digits()
    print(data.images.shape)
    X = data.data
    Y = data.target
    print(X.shape,X[:2])
    print(Y.shape,Y[:2])
     stander = StandardScaler() # 特征值进行标准化,得到均值为0,方差为1的分布
     X = stander.fit_transform(X)
     print(np.unique(Y,return_counts=True))  # 输出每一种类别及类别的数量
    

    在这里插入图片描述在这里插入图片描述

    二、数据拆分

    将数据拆分成80%的训练集和20%的测试集

    X_train,X_test,Y_train,Y_test = train_test_split(X,Y,test_size=0.2)
    print(X_train.shape)
    print(Y_train.shape,Y_train)
    print(Y_test.shape)
    

    三、创建神经网络

    V = np.random.random((64, 100))*2-1 # 输入层到隐藏层的权重矩阵,范围0~1
    W = np.random.random((100,10))*2-1  # 隐藏层到输出层的权重矩阵,范围0~1
    
    # 激活函数sigmoid函数
    def sigmond(x):
        return 1/(1+ np.exp(-x))
    
    # sigmoid函数的导数
    def sigmondf(x):
        return x*(1-x)
    
    # 预测函数
    def predict(x):
        L1 = sigmond(np.dot(x, V))
        L2 = sigmond(np.dot(L1, W))
        return L2
    
    # 神经网络分类器
    def network(X, Y, steps=10000):
        global W,V  # 定义全局变量
        acc_list = []
        for lt in list(np.linspace(0.001,0.01,100)):
            acc_0 = 0
            for n in range(steps+1):
                i = np.random.randint(X.shape[0])
                X = X[i]  # 这是一个一维的矩阵,(64,)
                X = np.atleast_2d(X)  # atlease_xd变成几维,如果是2,则是2维,如果是3,就变成3维,[],[[]],[[[]]]
                L1 = sigmond(np.dot(X,V)) # 得到一个(1,100)的矩阵
                L2 = sigmond(np.dot(L1,W)) # 得到一个(1,10)的矩阵
                L2_delta = (Y[i] - L2)*sigmondf(L2)  # 输出节点的残差: -(输出值-样本值)* 激活函数导数 (1,10)
                L1_delta = np.dot(L2_delta,W.T)*sigmondf(L1)  # 隐藏层的残差:右侧输出层的残差加权求和*隐藏层的导数 (1,10)(10,100)=(1,100)
                W += lt*np.dot(L1.T,L2_delta) # 隐藏层更新权重:(100,1)*(1,10)=(100,10)
                V += lt*np.dot(X.T,L1_delta)  # 输入层更新权重:(64,1)*(1,100)=(64,100)
                if n%1000 == 0:  # 取余,如果迭代达到1000的倍数
                    predict_ = predict(X_test)
                    predictions = np.argmax(predict_, axis=1) # 每一行最大值的下标,对于多分类,输出值最大的即为分类结果
                    acc = np.mean(np.equal(predictions, Y_test)) # np.equal是判断是否相等,相等的变为True,不相等的就变为False,然后计算均值
                    if acc>acc_0:
                        acc_0 = acc
                    # print('第{}次'.format(n)+'      '+ '平均准确率:', acc)
            acc_list.append(acc_0)
        return acc_list
        
    acc_list = network(X_train,Y_train)
    plt.plot(np.linspace(0.001,0.1,100),acc_list)
    plt.show()
    

    在这里插入图片描述
    在这里插入图片描述
    在这里插入图片描述
    在这里插入图片描述
    跑出来准确率不高,于是不断替换学习率,发现最高的也仅能达到13%,尝试对初始权重进行调整,也没有太大的变化,不过本篇主要的目的就是想用python实现神经网络,对于分类的准确度倒在其次,后续打算会用sklearn库封装的神经网络试试看。

    展开全文
  • ResNet残差神经网络 正如之前文章所提到,普通的神经网络,如果超过很多层,比如超过25层之后,模型的accuracy将会下降,按理说,神经网络越深,模型的效果应该越好,但是现实却是相反的情况。这可能是因为梯度消失...

    这文章是理解ResNet神经网络,并且使用简单使用python实战ResNet的残差块以及其他部分。

    ResNet残差神经网络

    正如之前文章所提到,普通的神经网络,如果超过很多层,比如超过25层之后,模型的accuracy将会下降,按理说,神经网络越深,模型的效果应该越好,但是现实却是相反的情况。这可能是因为梯度消失问题所导致的。 于是有人提出残差块,在某一层中间插入多个残差块,便可以训练超过2000层的神经网络,而且效果越来越好。ResNet网络的提出,使得我们可以训练很深的神经网络,并取得不错的效果。

    2015年,ResNet首次被提出,把经过卷积之后的features map和input相加得到新的部分。便是残差块。左图是首次提出的残差块图。2015年的残差块的x是先跟weights相乘之后,再做BN归一化和relu激活。2016年 微软提出新的残差块,对输入input-x提前处理,经过BN归一化和relu 激活之后,再跟weight相乘。
    在这里插入图片描述
    现在大多人都用微软proposed后的残差块(pre-trained activation),即右图。我们的实战也会是用右图的残差块。

    使用python实现残差块

    def Unit(x,filters):
    	‘’‘
    	param x - 输入
    	param filters - 特征数量
    	’‘’
    	res = x
    	#第一层
        out = BatchNormalization()(x)  #先对x做BN归一化处理
        out = Activation("relu")(out)  #再激活x,然后做卷积
        out = Conv2D(filters=filters, kernel_size=[3, 3], strides=[1, 1], padding="same")(out)  
    	#第二层,跟第一层同理处理
        out = BatchNormalization()(out)  
        out = Activation("relu")(out)
        out = Conv2D(filters=filters, kernel_size=[3, 3], strides=[1, 1], padding="same")(out)
    	#最后跟res-即原来的input x相加
    	out = keras.layers.add([res,out])
        return out
    
    但如果我们想加入pooling,那么经过pool之后的x和res的维度就不一样,无法相加。所以我们需要对res也做维度处理。使得两者维度相同。

    加载packages

    import keras
    from keras.datasets import cifar10
    from keras.layers import Dense,Conv2D,MaxPooling2D,Flatten,AveragePooling2D,Dropout,BatchNormalization,Activation
    from keras.models import Model,Input
    from keras.optimizers import Adam
    from keras.callbacks import LearningRateScheduler
    from keras.callbacks import ModelCheckpoint
    from math import ceil
    import os
    from keras.preprocessing.image import ImageDataGenerator
    
    def Unit(x,filters,pool=False):
    	‘’‘
    	param x - 输入
    	param filters - 特征数量
    	param pool - False or True 表示是否要pooling
    	’‘’
        res = x
        if pool:
            x = MaxPooling2D(pool_size=(2, 2))(x)
            #困惑点-尽管padding = ‘same’,由于strides是(2,2),所以出来的宽度和长度是 W/strides
            res = Conv2D(filters=filters,kernel_size=[1,1],strides=(2,2),padding="same")(res)   
            #由于strides=2 所以形状缩小了一倍
        out = BatchNormalization()(x)
        out = Activation("relu")(out)
        out = Conv2D(filters=filters, kernel_size=[3, 3], strides=[1, 1], padding="same")(out)
    
        out = BatchNormalization()(out)
        out = Activation("relu")(out)
        out = Conv2D(filters=filters, kernel_size=[3, 3], strides=[1, 1], padding="same")(out)
    
        out = keras.layers.add([res,out])
        
        return out
    
    现在把完善好的残差块组合入模型里面

    手写一个小模型

    def MiniModel(input_shape):
    	‘’‘
    	param input_shape: 输入的shape
    	’‘’
    	images = Input(input_shape)  #表示接受的image的shape将会是input_shape
    	#<tf.Tensor 'input_2:0' shape=(None, 32, 32, 3) dtype=float32>
    	
    	#开始带入网络
    	#第一层卷积
    	net = Conv2D(filters=32, kernel_size=[3, 3], strides=[1, 1], padding="same")(images)
    	#然后放入残差块,stacking堆积三次
    	net = Unit(net,32)   #32代表filters数量
    	net = Unit(net,32)
    	net = Unit(net,32)
    	#再次堆积 但输入和输出的维度将会减少一倍when pool = True
    	net = Unit(net,64,pool=True)
    	net = Unit(net,64)
    	net = Unit(net,64)
    	#再次堆积
    	net = Unit(net,128,pool=True)
    	net = Unit(net,128)
    	net = Unit(net,128)
    	#再次堆积
    	net = Unit(net, 256,pool=True)
    	net = Unit(net, 256)
    	net = Unit(net, 256)
    	############# 残差块 堆积结束
    	
    	#下一层进行BN处理
    	net = BatchNormalization()(net)
    	#激活
    	net = Activation("relu")(net)
    	#随机丢掉25%的神经元
    	net = Dropout(0.25)(net)
    	#进行average pooling 把形状缩小为原来1/4
    	net = AveragePooling2D(pool_size=(4,4))(net)
    	############# 卷积结束
    	#现在开始铺平,并开始预测y 
    	#铺平卷积网络
    	net = Flatten()(net)
    	#用softmax预测 数量为10 
    	net = Dense(units=10,activation="softmax")(net)
    	############# 预测结束,把网络放入tf的Model里面
    	model = Model(inputs=images,outputs=net)
    	return model
    
    模型我们已经手写好了,现在开始我们要准备数据,然后开始进行训练。

    数据准备 – 使用cifar10数据集

    #(1)加载 cifar10 dataset
    (train_x, train_y) , (test_x, test_y) = cifar10.load_data()
    #(2)数据normalize化,并把数据类型改为float32
    train_x = train_x.astype('float32') / 255
    test_x = test_x.astype('float32') / 255
    #(3)减去平均值
    train_x = train_x - train_x.mean()
    test_x = test_x - test_x.mean()
    #(4)除 std值 
    train_x = train_x / train_x.std(axis=0)
    test_x = test_x / test_x.std(axis=0)
    #(5)ImageDataGenerator()是keras.preprocessing.image模块中的图片生成器,
    #同时也可以在batch中对数据进行增强,扩充数据集大小,这里我们是对data进行了水平翻转和角度转移
    datagen = ImageDataGenerator(rotation_range=10,
                                 width_shift_range=5. / 32,
                                 height_shift_range=5. / 32,
                                 horizontal_flip=True)
    #把train data放入图片生成器 ,只对train data进行生成更多的图片,对 test data 不做处理
    datagen.fit(train_x)  
    #########################对x的处理结束
    #现在对Y做处理,把Y变成vector形式
    #Encode the labels to vectors
    train_y = keras.utils.to_categorical(train_y,10)
    test_y = keras.utils.to_categorical(test_y,10)
    #即Y变成了N个[0., 0., 0., 0., 0., 0., 1., 0., 0., 0.]形式的vector               
    

    开始训练模型

    input_shape = (32,32,3)
    model = MiniModel(input_shape)
    #compile模型
    model.compile(optimizer=Adam(0.001),loss="categorical_crossentropy",metrics=["accuracy"])
    
    #batch_size确定每个小批次中的样品数量。它的最大值是所有样本的数量
    epochs = 50   #参数epoch决定网络中所有图像的训练次数
    steps_per_epoch = ceil(50000/128)  #由样本数量/batch size
    
    #放入训练过程的模型中
    model.fit_generator(datagen.flow(train_x, train_y, batch_size=128),
                        validation_data=[test_x,test_y],
                        epochs=epochs,steps_per_epoch=steps_per_epoch, verbose=1, workers=4)
    
    训练结束

    用test data评估

    accuracy = model.evaluate(x=test_x,y=test_y,batch_size=128)
    model.save("cifar10model.h5")
    
    展开全文
  • Matlab如何指定gpu运行代码用于光场重构的高维密集残差卷积神经网络 该项目是Tensorflow的实现 “用于光场重构的高维密集残差卷积神经网络”, IEEE模式分析和机器智能交易,南梦,海登·科赫。 所以,孙星,林德霖...
  • 本文来自于个人博客,这篇文章主要为大家详细介绍了python机器学习之神经网络的实现方法。神经网络在机器学习中有很大的应用,甚至涉及到方方面面。本文主要是简单介绍一下神经网络的基本理论概念和推算。同时也会...
  • 深度残差卷积神经网络作为交互式Web服务器上的性别分类器。 该网络是通过优化实现的 项目规格书 奥斯纳布吕克大学 2016/17冬季学期 课程:使用Tensorflow的人工神经网络简介 贡献者:Clemens Hutter,Michele ...
  • 残差值使用长短记忆神经网络(LSTM)预测 使用残差值的前70%作为测试集,使用后30%作为验证集。 train, test =c[0:int(len(c)*0.7)], c[int(len(c)*0.7):int(len(c))] def timeseries_to_supervised(data, lag=1):...

    本实验使用环境为Anaconda3 Jupyter,调用Sklearn包,调用keras包,请提前准备好。

    1.引入一些常见包

    主要有keras包、numpy包、metrics包、pandas包等。

    import csv
    import numpy as np
    import time
    from sklearn.preprocessing import StandardScaler
    from sklearn.model_selection import train_test_split
    from sklearn.ensemble import RandomForestClassifier
    from sklearn.metrics import accuracy_score
    from sklearn.metrics import confusion_matrix
    from sklearn.metrics import classification_report
    from sklearn.metrics import explained_variance_score
    from sklearn import metrics
    from sklearn.svm import SVR
    import matplotlib.pyplot as plt  
    from pandas import DataFrame
    from pandas import Series
    from pandas import concat
    from pandas import read_csv
    from pandas import datetime
    from sklearn.metrics import mean_squared_error
    from sklearn.preprocessing import MinMaxScaler
    from keras.models import Sequential
    from keras.layers import Dense
    from keras.layers import LSTM
    from math import sqrt
    from matplotlib import pyplot
    import numpy
    

    2.引入数据

    其中data为全部数据、traffic_feature为特征集、traffic_target目标集

    data=[]
    traffic_feature=[]
    traffic_target=[]
    
    csv_file = csv.reader(open('GoodData.csv'))
    for content in csv_file:
        content=list(map(float,content))
        if len(content)!=0:
            data.append(content)
            traffic_feature.append(content[0:4])
            traffic_target.append(content[-1])
            
    traffic_feature=np.array(traffic_feature)
    traffic_target=np.array(traffic_target)
    data=np.array(data)
    

    对数据进行分割,本文使用70%为分割点。

    
    feature_train=traffic_feature[0:int(len(traffic_feature)*0.7)]
    feature_test=traffic_feature[int(len(traffic_feature)*0.7):int(len(traffic_feature))]
    target_train=traffic_target[0:int(len(traffic_target)*0.7)]
    target_test =traffic_target[int(len(traffic_target)*0.7):int(len(traffic_target))]
    

    对后30%的目标值继续分割,分割点仍然为70%,预留做对照。

    target_test_last=target_test[int(len(target_test)*0.7):int(len(target_test))]
    

    3.数据标准化

    使用StandardScaler()方法将特征数据标准化归一化。

    scaler = StandardScaler() # 标准化转换
    scaler.fit(traffic_feature)  # 训练标准化对象
    traffic_feature= scaler.transform(traffic_feature)   # 转换数据集
    

    4.使用ELM算法预测

    class HiddenLayer:
        def __init__(self, x, num):  # x:输入矩阵   num:隐含层神经元个数
            row = x.shape[0]
            columns = x.shape[1]
            rnd = np.random.RandomState(9999)
            self.w = rnd.uniform(-1, 1, (columns, num))  #
            self.b = np.zeros([row, num], dtype=float)  # 随机设定隐含层神经元阈值,即bi的值
            for i in range(num):
                rand_b = rnd.uniform(-0.4, 0.4)  # 随机产生-0.4 到 0.4 之间的数
                for j in range(row):
                    self.b[j, i] = rand_b  # 设定输入层与隐含层的连接权值
            self.h = self.sigmoid(np.dot(x, self.w) + self.b)  # 计算隐含层输出矩阵H
            self.H_ = np.linalg.pinv(self.h)  # 获取H的逆矩阵
            # print(self.H_.shape)
     
        # 定义激活函数g(x) ,需为无限可微函数
        def sigmoid(self, x):
            print(x)
            return 1.0 / (1 + np.exp(-x))
     
        '''  若进行分析的训练数据为回归问题,则使用该方式 ,计算隐含层输出权值,即β '''
     
        def regressor_train(self, T):
            C = 2
            I = len(T)
            sub_former = np.dot(np.transpose(self.h), self.h) + I / C
            all_m = np.dot(np.linalg.pinv(sub_former), np.transpose(self.h))
            T = T.reshape(-1, 1)
            self.beta = np.dot(all_m, T)
            return self.beta
     
        """
               计算隐含层输出权值,即β 
        """
     
        def classifisor_train(self, T):
            en_one = OneHotEncoder()
            # print(T)
            T = en_one.fit_transform(T.reshape(-1, 1)).toarray()  # 独热编码之后一定要用toarray()转换成正常的数组
            # print(T)
            C = 3
            I = len(T)
            sub_former = np.dot(np.transpose(self.h), self.h) + I / C
            all_m = np.dot(np.linalg.pinv(sub_former), np.transpose(self.h))
            self.beta = np.dot(all_m, T)
            return self.beta
     
        def regressor_test(self, test_x):
            b_row = test_x.shape[0]
            h = self.sigmoid(np.dot(test_x, self.w) + self.b[:b_row, :])
            result = np.dot(h, self.beta)
            return result
     
        def classifisor_test(self, test_x):
            b_row = test_x.shape[0]
            h = self.sigmoid(np.dot(test_x, self.w) + self.b[:b_row, :])
            result = np.dot(h, self.beta)
            result = [item.tolist().index(max(item.tolist())) for item in result]
            return result
    

    跑起来~
    设置神经元个数为8,可以自行调优。

    import matplotlib.pyplot as plt
    from sklearn.metrics import explained_variance_score
    a = HiddenLayer(feature_train,8)
    a.regressor_train(target_train)
    result = a.regressor_test(feature_test)
    plt.plot(result)#测试数组
    plt.plot(target_test)#测试数组
    plt.legend(['ELM','TRUE'])
    fig = plt.gcf()
    fig.set_size_inches(18.5, 10.5)
    plt.title("ELM")  # 标题
    plt.show()
    print("EVS:",explained_variance_score(target_test,result))
    

    结果如下:
    EVS等于0.8705
    在这里插入图片描述

    5.使用真实值减预测值,获的ELM的残差

    a=[]#真实值
    for i in target_test:
        a.append(i)
    b=[]#预测值
    for i in result:
        b.append(i[0])
    
    c=[]#残差值
    num=[]
    for inx,i in enumerate(a):
        c.append(b[inx]-i)
        num.append(inx)
    plt.plot(c)#残差
    fig = plt.gcf()
    fig.set_size_inches(18.5,5)
    plt.xlim(0,1560)
    plt.title("Residual Signal")  # 标题
    plt.show()
    

    残差值变化如下:
    在这里插入图片描述
    将预测的后30%截取,预留做对照。

    result_last=b[int(len(b)*0.7):int(len(b))]
    

    6.对残差值使用长短记忆神经网络(LSTM)预测

    使用残差值的前70%作为测试集,使用后30%作为验证集。

    train, test =c[0:int(len(c)*0.7)], c[int(len(c)*0.7):int(len(c))]
    
    def timeseries_to_supervised(data, lag=1):
      df = DataFrame(data)
      columns = [df.shift(i) for i in range(1, lag+1)]
      columns.append(df)
      df = concat(columns, axis=1)
      df.fillna(0, inplace=True)
      return df
    def fit_lstm(train, batch_size, nb_epoch, neurons):
      X, y = train[:, 0:-1], train[:, -1]
      X = X.reshape(X.shape[0], 1, X.shape[1])
      model = Sequential()
      model.add(LSTM(neurons, batch_input_shape=(batch_size, X.shape[1], X.shape[2]), stateful=True))
      model.add(Dense(1))
      model.compile(loss='mean_squared_error', optimizer='adam')
      for i in range(nb_epoch):
          model.fit(X, y, epochs=1, batch_size=batch_size, verbose=0, shuffle=False)
          model.reset_states()
      return model
    # make a one-step forecast
    def forecast_lstm(model, batch_size, X):
      X = X.reshape(1, 1, len(X))
      yhat = model.predict(X, batch_size=batch_size)
      return yhat[0,0]
    
    c2d=[]
    for i in c:
        c2d.append([i,i])
    scaler = StandardScaler() # 标准化转换
    scaler.fit(c2d)  # 训练标准化对象
    supervised= scaler.transform(c2d)   # 转换数据集
    c1d=[]
    for j in supervised:
        c1d.append(j[0])
    supervised = timeseries_to_supervised(c1d, 1)
    train_scaled, test_scaled  =supervised[0:int(len(supervised)*0.7)], supervised[int(len(supervised)*0.7):int(len(supervised))]
    
    train_scaled=np.array(train_scaled)
    test_scaled=np.array(test_scaled)
    
    print("开始")
    # fit the model
    lstm_model = fit_lstm(train_scaled, 1, 30, 4)
    # forecast the entire training dataset to build up state for forecasting
    train_reshaped = train_scaled[:, 0].reshape(len(train_scaled), 1, 1)
    lstm_model.predict(train_reshaped, batch_size=1)
    
    # walk-forward validation on the test data
    predictions = list()
    for i in range(len(test_scaled)):
      # make one-step forecast
      X, y = test_scaled[i, 0:-1], test_scaled[i, -1]
      yhat = forecast_lstm(lstm_model, 1, X)
      # store forecast
      predictions.append(yhat)
    print("结束")
    

    经过数据格式改变后,残差预测效果如下。

    predictions2d=[]
    for i in predictions:
        predictions2d.append([i,i])
    predictions2d
    predictions2d=scaler.inverse_transform(predictions2d)
    predictions1d=[]
    for j in predictions2d:
        predictions1d.append(j[0])
    
    # report performance
    rmse = sqrt(mean_squared_error(test, predictions1d))
    print('RMSE: %.3f' % rmse)
    # line plot of observed vs predicted
    fig = pyplot.gcf()
    fig.set_size_inches(18.5, 10.5)
    pyplot.plot(test)
    pyplot.plot(predictions1d)
    pyplot.show()
    

    在这里插入图片描述

    7.ELM与ELM-LSTM预测结果进行对比

    ELM:

    print("mse:",metrics.mean_squared_error(target_test_last,result_last)) 
    print("R2:",metrics.r2_score(target_test_last,result_last))
    

    在这里插入图片描述
    ELM-LSTM:

    test1=np.array(test)
    predictions1d1=np.array(predictions1d)
    result1=result_last-test1+predictions1d1
    print("mse:",metrics.mean_squared_error(target_test_last,result1)) 
    print("R2:",metrics.r2_score(target_test_last,result1))
    

    在这里插入图片描述
    COMPARE:

    x=range(1089,1556,1)
    plt.plot(x,target_test_last,marker=',')
    plt.plot(x,result_last,marker='+')
    plt.plot(x,result1,marker='x')
    plt.legend(['True','ELM','ELM_LSTM'])
    fig = plt.gcf()
    fig.set_size_inches(18.5, 10.5)
    plt.title("COMPARE")  # 标题
    plt.show()
    

    结果如下:
    由上可知ELM-LSTM比ELM的R方要高、MSE要低。且下图显示ELM-LSTM比ELM更贴近真实值。
    在这里插入图片描述

    提供思路

    由于数据并非我本人所有,所以我不能提供给大家。数据格式如下:
    在这里插入图片描述

    展开全文
  • Keras 是一个用 Python 编写的高级神经网络 API,它能够以 TensorFlow, CNTK, 或者 Theano 作为后端运行。 Keras 的开发重点是支持快速的实验。能够以最小的时延把你的想法转换为实验结果,是做好研究的关键。 ...

    测试题:参考博文

    笔记:04.卷积神经网络 W2.深度卷积网络:实例探究

    作业1:Keras教程

    Keras 是一个用 Python 编写的高级神经网络 API,它能够以 TensorFlow, CNTK, 或者 Theano 作为后端运行。
    Keras 的开发重点是支持快速的实验。能够以最小的时延把你的想法转换为实验结果,是做好研究的关键。

    Keras 是更高级的框架,对普通模型来说很友好,但是要实现更复杂的模型需要 TensorFlow 等低级的框架

    • 导入一些包
    import numpy as np
    from keras import layers
    from keras.layers import Input, Dense, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D
    from keras.layers import AveragePooling2D, MaxPooling2D, Dropout, GlobalMaxPooling2D, GlobalAveragePooling2D
    from keras.models import Model
    from keras.preprocessing import image
    from keras.utils import layer_utils
    from keras.utils.data_utils import get_file
    from keras.applications.imagenet_utils import preprocess_input
    import pydot
    from IPython.display import SVG
    from keras.utils.vis_utils import model_to_dot
    from keras.utils import plot_model
    from kt_utils import *
    
    import keras.backend as K
    K.set_image_data_format('channels_last')
    import matplotlib.pyplot as plt
    from matplotlib.pyplot import imshow
    
    %matplotlib inline
    

    1. 快乐的房子

    问题背景:快乐的房子的门口的摄像头会识别你的表情是否是 Happy 的,是 Happy 的,门才会打开,哈哈!

    我们要建模自动识别表情是否快乐!

    • 归一化图片数据,了解数据维度
    X_train_orig, Y_train_orig, X_test_orig, Y_test_orig, classes = load_dataset()
    
    # Normalize image vectors
    X_train = X_train_orig/255.
    X_test = X_test_orig/255.
    
    # Reshape
    Y_train = Y_train_orig.T
    Y_test = Y_test_orig.T
    
    print ("number of training examples = " + str(X_train.shape[0]))
    print ("number of test examples = " + str(X_test.shape[0]))
    print ("X_train shape: " + str(X_train.shape))
    print ("Y_train shape: " + str(Y_train.shape))
    print ("X_test shape: " + str(X_test.shape))
    print ("Y_test shape: " + str(Y_test.shape))
    

    输出:

    number of training examples = 600
    number of test examples = 150
    X_train shape: (600, 64, 64, 3)
    Y_train shape: (600, 1)
    X_test shape: (150, 64, 64, 3)
    Y_test shape: (150, 1)
    

    600个训练样本,150个测试样本,图片维度 64*64*3 = 12288

    2. 用Keras建模

    Keras 可以快速建模,且模型效果不错

    举个例子:

    def model(input_shape):
        # 定义输入的 placeholder 作为 tensor with shape input_shape. 
        # 想象这是你的图片输入
        X_input = Input(input_shape)
    
        # Zero-Padding: pads the border of X_input with zeroes
        X = ZeroPadding2D((3, 3))(X_input)
    
        # CONV -> BN -> RELU Block applied to X
        X = Conv2D(32, (7, 7), strides = (1, 1), name = 'conv0')(X)
        X = BatchNormalization(axis = 3, name = 'bn0')(X)
        X = Activation('relu')(X)
    
        # MAXPOOL
        X = MaxPooling2D((2, 2), name='max_pool')(X)
    
        # FLATTEN X (means convert it to a vector) + FULLYCONNECTED
        X = Flatten()(X)
        X = Dense(1, activation='sigmoid', name='fc')(X)
    
        # Create model. This creates your Keras model instance, 
        # you'll use this instance to train/test the model.
        model = Model(inputs = X_input, outputs = X, name='HappyModel')
    
        return model
    

    本次作业很open,可以自由搭建模型,修改超参数,请注意各层之间的维度匹配

    Keras Model 类参考链接

    • 定义模型
    # GRADED FUNCTION: HappyModel
    
    def HappyModel(input_shape):
        """
        Implementation of the HappyModel.
        
        Arguments:
        input_shape -- shape of the images of the dataset
    
        Returns:
        model -- a Model() instance in Keras
        """
        
        ### START CODE HERE ###
        # Feel free to use the suggested outline in the text above to get started, and run through the whole
        # exercise (including the later portions of this notebook) once. The come back also try out other
        # network architectures as well. 
        X_input = Input(input_shape)
        
        X = ZeroPadding2D((3,3))(X_input)
        
        X = Conv2D(32,(7,7), strides = (1,1), name='conv0')(X)
        X = BatchNormalization(axis = 3, name='bn0')(X)
        X = Activation('relu')(X)
        
        X = MaxPooling2D((2,2), name='max_pool')(X)
        
        X = Flatten()(X)
        X = Dense(1, activation='sigmoid', name='fc')(X)
        
        model = Model(inputs = X_input, outputs = X, name='HappyModel')
        
        ### END CODE HERE ###
        
        return model
    
    • 创建模型实例
    happyModel = HappyModel(X_train[0].shape)
    
    • 配置训练模型
    import keras
    opt = keras.optimizers.Adam(learning_rate=0.01)
    happyModel.compile(optimizer=opt, 
    		loss=keras.losses.BinaryCrossentropy(),metrics=['acc'])
    
    • 训练 并 存储返回的训练过程数据用于可视化
    history = happyModel.fit(x=X_train, y=Y_train, 
    		validation_split=0.25, batch_size=32, epochs=30)
    
    • 绘制训练过程
    # 绘制训练 & 验证的准确率值
    plt.plot(history.history['acc'])
    plt.plot(history.history['val_acc'])
    plt.title('Model accuracy')
    plt.ylabel('Accuracy')
    plt.xlabel('Epoch')
    plt.legend(['Train', 'Test'], loc='upper left')
    plt.show()
    
    # 绘制训练 & 验证的损失值
    plt.plot(history.history['loss'])
    plt.plot(history.history['val_loss'])
    plt.title('Model loss')
    plt.ylabel('Loss')
    plt.xlabel('Epoch')
    plt.legend(['Train', 'Test'], loc='upper left')
    plt.show()
    
    。。。省略
    Epoch 29/30
    15/15 [==============================] 
    - 2s 148ms/step - loss: 0.1504 - acc: 0.9733 - val_loss: 0.1518 - val_acc: 0.9600
    Epoch 30/30
    15/15 [==============================] 
    - 2s 147ms/step - loss: 0.1160 - acc: 0.9711 - val_loss: 0.2242 - val_acc: 0.9333
    

    准确率
    损失

    • 测试模型效果
    ### START CODE HERE ### (1 line)
    from keras import metrics
    preds = happyModel.evaluate(X_test, Y_test, batch_size=32, verbose=1, sample_weight=None)
    ### END CODE HERE ###
    print(preds)
    
    print ("Loss = " + str(preds[0]))
    print ("Test Accuracy = " + str(preds[1]))
    

    输出:

    5/5 [==============================] - 0s 20ms/step - loss: 0.2842 - acc: 0.9400
    [0.28415805101394653, 0.9399999976158142]
    Loss = 0.28415805101394653
    Test Accuracy = 0.9399999976158142
    

    3. 用你的图片测试

    ### START CODE HERE ###
    img_path = 'images/1.jpg'
    ### END CODE HERE ###
    img = image.load_img(img_path, target_size=(64, 64))
    imshow(img)
    
    x = image.img_to_array(img)
    x = np.expand_dims(x, axis=0)
    x = preprocess_input(x)
    
    print(happyModel.predict(x))
    

    预测正确 1笑脸

    4. 一些有用的Keras函数

    • happyModel.summary() 模型的结构,参数等信息
    happyModel.summary()
    
    Model: "HappyModel"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #   
    =================================================================
    input_1 (InputLayer)         [(None, 64, 64, 3)]       0         
    _________________________________________________________________
    zero_padding2d (ZeroPadding2 (None, 70, 70, 3)         0         
    _________________________________________________________________
    conv0 (Conv2D)               (None, 64, 64, 32)        4736      
    _________________________________________________________________
    bn0 (BatchNormalization)     (None, 64, 64, 32)        128       
    _________________________________________________________________
    activation (Activation)      (None, 64, 64, 32)        0         
    _________________________________________________________________
    max_pool (MaxPooling2D)      (None, 32, 32, 32)        0         
    _________________________________________________________________
    flatten (Flatten)            (None, 32768)             0         
    _________________________________________________________________
    fc (Dense)                   (None, 1)                 32769     
    =================================================================
    Total params: 37,633
    Trainable params: 37,569
    Non-trainable params: 64
    _________________________________________________________________
    
    • plot_model() 把模型结构保存成图片

    模型结构

    作业2:残差网络 Residual Networks

    使用残差网络能够训练更深的神经网络,普通的深层神经网络是很难训练的。

    • 导入包
    import numpy as np
    from keras import layers
    from keras.layers import Input, Add, Dense, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D, AveragePooling2D, MaxPooling2D, GlobalMaxPooling2D
    from keras.models import Model, load_model
    from keras.preprocessing import image
    from keras.utils import layer_utils
    from keras.utils.data_utils import get_file
    from keras.applications.imagenet_utils import preprocess_input
    import pydot
    from IPython.display import SVG
    from keras.utils.vis_utils import model_to_dot
    from keras.utils import plot_model
    from resnets_utils import *
    from keras.initializers import glorot_uniform
    import scipy.misc
    from matplotlib.pyplot import imshow
    %matplotlib inline
    
    import keras.backend as K
    K.set_image_data_format('channels_last')
    K.set_learning_phase(1)
    

    1. 深层神经网络的问题

    深层网络优点:

    • 可以表示复杂的函数
    • 可以学习很多不同层次的特征(低层次,高层次)

    缺点:

    • 梯度消失/爆炸,梯度变的非常小或者非常大


    随着迭代次数增加,浅层的梯度很快的就降到 0

    2. 建立残差网络

    通过跳跃的连接,允许梯度直接反向传到浅层

    • 跳跃连接使得模块更容易学习恒等函数
    • 残差模块不会损害训练效果

    残差网络有两种类型的模块,主要取决于输入输出的维度是否一样

    2.1 identity恒等模块

    结构1
    结构2
    下面我们要实现:跳过3个隐藏层的结构,其稍微更强大一些

    convolution2d 参考:https://keras.io/api/layers/convolution_layers/convolution2d/

    batch_normalization 参考:
    https://keras.io/api/layers/normalization_layers/batch_normalization/

    add 参考:
    https://keras.io/api/layers/merging_layers/add/

    # GRADED FUNCTION: identity_block
    
    def identity_block(X, f, filters, stage, block):
        """
        Implementation of the identity block as defined in Figure 3
        
        Arguments:
        X -- input tensor of shape (m, n_H_prev, n_W_prev, n_C_prev)
        f -- integer, specifying the shape of the middle CONV's window for the main path
        filters -- python list of integers, defining the number of filters in the CONV layers of the main path
        stage -- integer, used to name the layers, depending on their position in the network
        block -- string/character, used to name the layers, depending on their position in the network
        
        Returns:
        X -- output of the identity block, tensor of shape (n_H, n_W, n_C)
        """
        
        # defining name basis
        conv_name_base = 'res' + str(stage) + block + '_branch'
        bn_name_base = 'bn' + str(stage) + block + '_branch'
        
        # Retrieve Filters
        F1, F2, F3 = filters
        
        # Save the input value. You'll need this later to add back to the main path. 
        X_shortcut = X
        
        # First component of main path
        X = Conv2D(filters = F1, kernel_size = (1, 1), strides = (1,1), padding = 'valid', name = conv_name_base + '2a', kernel_initializer = glorot_uniform(seed=0))(X)
        X = BatchNormalization(axis = 3, name = bn_name_base + '2a')(X)
        X = Activation('relu')(X)
        
        ### START CODE HERE ###
        
        # Second component of main path (≈3 lines)
        X = Conv2D(filters = F2, kernel_size=(f, f), strides = (1, 1), padding='same', name=conv_name_base+'2b', kernel_initializer=glorot_uniform(seed=0))(X)
        X = BatchNormalization(axis=3, name=bn_name_base+'2b')(X)
        X = Activation('relu')(X)
        
        # Third component of main path (≈2 lines)
        X = Conv2D(filters=F3, kernel_size=(1,1), strides=(1,1), padding='valid', name=conv_name_base+'2c', kernel_initializer=glorot_uniform(seed=0))(X)
        X = BatchNormalization(axis=3, name=bn_name_base+'2c')(X)
    
        # Final step: Add shortcut value to main path, and pass it through a RELU activation (≈2 lines)
        X = Add()([X_shortcut, X])
        X = Activation('relu')(X)
        
        ### END CODE HERE ###
        
        return X
    

    测试代码:

    # import tensorflow.compat.v1 as tf
    # tf.disable_v2_behavior()
    
    tf.reset_default_graph()
    
    with tf.Session() as test:
        np.random.seed(1)
        A_prev = tf.placeholder("float", [3, 4, 4, 6])
        X = np.random.randn(3, 4, 4, 6)
        A = identity_block(A_prev, f = 2, filters = [2, 4, 6], stage = 1, block = 'a')
        test.run(tf.global_variables_initializer())
        out = test.run([A], feed_dict={A_prev: X, K.learning_phase(): 0})
        print("out = " + str(out[0][1][1][0]))
    

    输出:

    out = [0.19716819 0.         1.3561226  2.1713073  0.         1.3324987 ]
    

    2.2 卷积模块

    该模块可以适用于:输入输出维度不匹配的情况

    其 跳跃连接上有一个 CONV2D 卷积层,它没有使用非线性激活函数,作用是改变输入的维度,使后面的加法维度匹配

    # GRADED FUNCTION: convolutional_block
    
    def convolutional_block(X, f, filters, stage, block, s = 2):
        """
        Implementation of the convolutional block as defined in Figure 4
        
        Arguments:
        X -- input tensor of shape (m, n_H_prev, n_W_prev, n_C_prev)
        f -- integer, specifying the shape of the middle CONV's window for the main path
        filters -- python list of integers, defining the number of filters in the CONV layers of the main path
        stage -- integer, used to name the layers, depending on their position in the network
        block -- string/character, used to name the layers, depending on their position in the network
        s -- Integer, specifying the stride to be used
        
        Returns:
        X -- output of the convolutional block, tensor of shape (n_H, n_W, n_C)
        """
        
        # defining name basis
        conv_name_base = 'res' + str(stage) + block + '_branch'
        bn_name_base = 'bn' + str(stage) + block + '_branch'
        
        # Retrieve Filters
        F1, F2, F3 = filters
        
        # Save the input value
        X_shortcut = X
    
    
        ##### MAIN PATH #####
        # First component of main path 
        X = Conv2D(F1, (1, 1), strides = (s,s), padding = 'valid', name = conv_name_base + '2a', kernel_initializer = glorot_uniform(seed=0))(X)
        X = BatchNormalization(axis = 3, name = bn_name_base + '2a')(X)
        X = Activation('relu')(X)
        
        ### START CODE HERE ###
    
        # Second component of main path (≈3 lines)
        X = Conv2D(F2, (f,f), strides=(1,1),padding='same',name=conv_name_base+'2b',kernel_initializer=glorot_uniform(seed=0))(X)
        X = BatchNormalization(axis=3, name=bn_name_base+'2b')(X)
        X = Activation('relu')(X)
    
        # Third component of main path (≈2 lines)
        X = Conv2D(F3,(1,1), strides=(1,1),padding='valid',name=conv_name_base+'2c',kernel_initializer=glorot_uniform(seed=0))(X)
        X = BatchNormalization(axis=3, name=bn_name_base+'2c')(X)
    
        ##### SHORTCUT PATH #### (≈2 lines)
        X_shortcut = Conv2D(F3, (1,1), strides=(s,s),padding='valid',name=conv_name_base+'1',kernel_initializer=glorot_uniform(seed=0))(X_shortcut)
        X_shortcut = BatchNormalization(axis=3, name=bn_name_base+'1')(X_shortcut)
    
        # Final step: Add shortcut value to main path, and pass it through a RELU activation (≈2 lines)
        X = Add()([X, X_shortcut])
        X = Activation('relu')(X)
        
        ### END CODE HERE ###
        
        return X
    

    测试:

    tf.reset_default_graph()
    
    with tf.Session() as test:
        np.random.seed(1)
        A_prev = tf.placeholder("float", [3, 4, 4, 6])
        X = np.random.randn(3, 4, 4, 6)
        A = convolutional_block(A_prev, f = 2, filters = [2, 4, 6], stage = 1, block = 'a')
        test.run(tf.global_variables_initializer())
        out = test.run([A], feed_dict={A_prev: X, K.learning_phase(): 0})
        print("out = " + str(out[0][1][1][0]))
    

    输出:

    out = [0.09018463 1.2348979  0.46822023 0.03671762 0.         0.65516603]
    

    3. 建立你的第一个残差网络(50层)

    模型结构

    ID(Identity)恒等模块,ID BLOCK x3 表示恒等模块3次

    pooling 参考 https://keras.io/zh/layers/pooling/

    # GRADED FUNCTION: ResNet50
    
    def ResNet50(input_shape = (64, 64, 3), classes = 6):
        """
        Implementation of the popular ResNet50 the following architecture:
        CONV2D -> BATCHNORM -> RELU -> MAXPOOL -> CONVBLOCK -> IDBLOCK*2 -> CONVBLOCK -> IDBLOCK*3
        -> CONVBLOCK -> IDBLOCK*5 -> CONVBLOCK -> IDBLOCK*2 -> AVGPOOL -> TOPLAYER
    
        Arguments:
        input_shape -- shape of the images of the dataset
        classes -- integer, number of classes
    
        Returns:
        model -- a Model() instance in Keras
        """
        
        # Define the input as a tensor with shape input_shape
        X_input = Input(input_shape)
    
        
        # Zero-Padding
        X = ZeroPadding2D((3, 3))(X_input)
        
        # Stage 1
        X = Conv2D(64, (7, 7), strides = (2, 2), name = 'conv1', kernel_initializer = glorot_uniform(seed=0))(X)
        X = BatchNormalization(axis = 3, name = 'bn_conv1')(X)
        X = Activation('relu')(X)
        X = MaxPooling2D((3, 3), strides=(2, 2))(X)
    
        # Stage 2
        X = convolutional_block(X, f = 3, filters = [64, 64, 256], stage = 2, block='a', s = 1)
        X = identity_block(X, 3, [64, 64, 256], stage=2, block='b')
        X = identity_block(X, 3, [64, 64, 256], stage=2, block='c')
    
        ### START CODE HERE ###
    
        # Stage 3 (≈4 lines)
        X = convolutional_block(X, f=3, filters=[128,128,512], stage=3, block='a', s=2)
        X = identity_block(X, 3, [128,128,512],stage=3, block='b')
        X = identity_block(X, 3, [128,128,512],stage=3, block='c')
        X = identity_block(X, 3, [128,128,512],stage=3, block='d')
    
        # Stage 4 (≈6 lines)
        X = convolutional_block(X, f=3, filters=[256,256,1024], stage=4, block='a', s=2)
        X = identity_block(X, 3, [256,256,1024],stage=4, block='b')
        X = identity_block(X, 3, [256,256,1024],stage=4, block='c')
        X = identity_block(X, 3, [256,256,1024],stage=4, block='d')
        X = identity_block(X, 3, [256,256,1024],stage=4, block='e')
        X = identity_block(X, 3, [256,256,1024],stage=4, block='f')
    
        # Stage 5 (≈3 lines)
        X = convolutional_block(X, f=3, filters=[512,512,2048], stage=5, block='a', s=2)
        X = identity_block(X, 3, [512,512,2048], stage=5, block='b')
        X = identity_block(X, 3, [512,512,2048], stage=5, block='c')
    
        # AVGPOOL (≈1 line). Use "X = AveragePooling2D(...)(X)"
        X = AveragePooling2D(pool_size=(2,2))(X)
        
        ### END CODE HERE ###
    
        # output layer
        X = Flatten()(X)
        X = Dense(classes, activation='softmax', name='fc' + str(classes), kernel_initializer = glorot_uniform(seed=0))(X)
        
        
        # Create model
        model = Model(inputs = X_input, outputs = X, name='ResNet50')
    
        return model
    
    • 建立模型
    model = ResNet50(input_shape = (64, 64, 3), classes = 6)
    
    • 配置模型
    model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
    
    • 数据导入 + one_hot 编码
      手势数字数据集
    X_train_orig, Y_train_orig, X_test_orig, Y_test_orig, classes = load_dataset()
    
    # Normalize image vectors
    X_train = X_train_orig/255.
    X_test = X_test_orig/255.
    
    # Convert training and test labels to one hot matrices
    Y_train = convert_to_one_hot(Y_train_orig, 6).T
    Y_test = convert_to_one_hot(Y_test_orig, 6).T
    
    print ("number of training examples = " + str(X_train.shape[0]))
    print ("number of test examples = " + str(X_test.shape[0]))
    print ("X_train shape: " + str(X_train.shape))
    print ("Y_train shape: " + str(Y_train.shape))
    print ("X_test shape: " + str(X_test.shape))
    print ("Y_test shape: " + str(Y_test.shape))
    

    输出:

    number of training examples = 1080
    number of test examples = 120
    X_train shape: (1080, 64, 64, 3)
    Y_train shape: (1080, 6)
    X_test shape: (120, 64, 64, 3)
    Y_test shape: (120, 6)
    
    • 训练(迭代两次测试下)
    model.fit(X_train, Y_train, epochs = 2, batch_size = 32)
    

    输出:(损失在下降,准确率在上升)

    Epoch 1/2
    1080/1080 [==============================] 
    - 208s 192ms/step - loss: 2.6086 - acc: 0.3037
    Epoch 2/2
    1080/1080 [==============================] 
    - 193s 178ms/step - loss: 2.2677 - acc: 0.3972
    
    • 测试
    preds = model.evaluate(X_test, Y_test)
    print ("Loss = " + str(preds[0]))
    print ("Test Accuracy = " + str(preds[1]))
    

    输出:(准确率 19%)

    120/120 [==============================] - 5s 38ms/step
    Loss = 12.753657023111979
    Test Accuracy = 0.19166666467984517
    

    该模型训练2次效果很差,训练更多次效果才会好(时间比较久)

    老师直接给出了训练好的模型

    model = load_model('ResNet50.h5') 
    
    preds = model.evaluate(X_test, Y_test)
    print ("Loss = " + str(preds[0]))
    print ("Test Accuracy = " + str(preds[1]))
    
    Loss = 0.5301782568295796
    Test Accuracy = 0.8666667
    

    老师给的 ResNets 残差网络 预测准确率为 86.7%
    前次作业中 TF 3层网络模型的预测准确率为 72.5%

    4. 用自己的照片测试

    import imageio
    
    img_path = 'images/my_image.jpg'
    img = image.load_img(img_path, target_size=(64, 64))
    x = image.img_to_array(img)
    x = np.expand_dims(x, axis=0)
    x = preprocess_input(x)
    print('Input image shape:', x.shape)
    my_image = imageio.imread(img_path)
    imshow(my_image)
    print("class prediction vector [p(0), p(1), p(2), p(3), p(4), p(5)] = ")
    print(model.predict(x))
    

    输出:

    Input image shape: (1, 64, 64, 3)
    class prediction vector [p(0), p(1), p(2), p(3), p(4), p(5)] = 
    [[1. 0. 0. 0. 0. 0.]]
    

    • 模型结构总结
    model.summary()
    
    Model: "ResNet50"
    __________________________________________________________________________________________________
    Layer (type)                    Output Shape         Param #     Connected to                     
    ==================================================================================================
    input_1 (InputLayer)            [(None, 64, 64, 3)]  0                                            
    __________________________________________________________________________________________________
    zero_padding2d_1 (ZeroPadding2D (None, 70, 70, 3)    0           input_1[0][0]                    
    __________________________________________________________________________________________________
    conv1 (Conv2D)                  (None, 32, 32, 64)   9472        zero_padding2d_1[0][0]           
    __________________________________________________________________________________________________
    bn_conv1 (BatchNormalization)   (None, 32, 32, 64)   256         conv1[0][0]                      
    __________________________________________________________________________________________________
    activation_4 (Activation)       (None, 32, 32, 64)   0           bn_conv1[0][0]                   
    __________________________________________________________________________________________________
    max_pooling2d_1 (MaxPooling2D)  (None, 15, 15, 64)   0           activation_4[0][0]               
    __________________________________________________________________________________________________
    res2a_branch2a (Conv2D)         (None, 15, 15, 64)   4160        max_pooling2d_1[0][0]            
    ________________________________________________________________________
    
    省略省略省略省略
    省略省略省略省略
    
    add_16 (Add)                    (None, 2, 2, 2048)   0           bn5b_branch2c[0][0]              
                                                                     activation_46[0][0]              
    __________________________________________________________________________________________________
    activation_49 (Activation)      (None, 2, 2, 2048)   0           add_16[0][0]                     
    __________________________________________________________________________________________________
    res5c_branch2a (Conv2D)         (None, 2, 2, 512)    1049088     activation_49[0][0]              
    __________________________________________________________________________________________________
    bn5c_branch2a (BatchNormalizati (None, 2, 2, 512)    2048        res5c_branch2a[0][0]             
    __________________________________________________________________________________________________
    activation_50 (Activation)      (None, 2, 2, 512)    0           bn5c_branch2a[0][0]              
    __________________________________________________________________________________________________
    res5c_branch2b (Conv2D)         (None, 2, 2, 512)    2359808     activation_50[0][0]              
    __________________________________________________________________________________________________
    bn5c_branch2b (BatchNormalizati (None, 2, 2, 512)    2048        res5c_branch2b[0][0]             
    __________________________________________________________________________________________________
    activation_51 (Activation)      (None, 2, 2, 512)    0           bn5c_branch2b[0][0]              
    __________________________________________________________________________________________________
    res5c_branch2c (Conv2D)         (None, 2, 2, 2048)   1050624     activation_51[0][0]              
    __________________________________________________________________________________________________
    bn5c_branch2c (BatchNormalizati (None, 2, 2, 2048)   8192        res5c_branch2c[0][0]             
    __________________________________________________________________________________________________
    add_17 (Add)                    (None, 2, 2, 2048)   0           bn5c_branch2c[0][0]              
                                                                     activation_49[0][0]              
    __________________________________________________________________________________________________
    activation_52 (Activation)      (None, 2, 2, 2048)   0           add_17[0][0]                     
    __________________________________________________________________________________________________
    avg_pool (AveragePooling2D)     (None, 1, 1, 2048)   0           activation_52[0][0]              
    __________________________________________________________________________________________________
    flatten_1 (Flatten)             (None, 2048)         0           avg_pool[0][0]                   
    __________________________________________________________________________________________________
    fc6 (Dense)                     (None, 6)            12294       flatten_1[0][0]                  
    ==================================================================================================
    Total params: 23,600,006
    Trainable params: 23,546,886
    Non-trainable params: 53,120
    
    • 绘制模型结构图
    plot_model(model, to_file='model.png')
    SVG(model_to_dot(model).create(prog='dot', format='svg'))
    

    图片很长,只截取部分
    模型结构图
    参考论文


    我的CSDN博客地址 https://michael.blog.csdn.net/

    长按或扫码关注我的公众号(Michael阿明),一起加油、一起学习进步!
    Michael阿明

    展开全文
  • snnTorch是一个Python软件包,用于使用尖峰神经网络执行基于梯度的学习。 它无需重新设计轮子,而是位于PyTorch的顶部,并利用其GPU加速的张量计算。 预先设计的尖峰神经元模型无缝集成在PyTorch框架内,可以视为...
  • https://blog.csdn.net/diamonjoy_zone/article/details/70904212环境:Win8.1 TensorFlow1.0.1软件:Anaconda3 (集成Python3及开发环境)TensorFlow安装:pip install tensorflow (CPU版) pip install tensorflow-...
  • 神经网络在机器学习中有很大的应用,甚至涉及到方方面面。本文主要是简单介绍一下神经网络的基本理论概念和推算。同时也会介绍一下神经网络在数据分类方面的应用。 首先,当我们建立一个回归和分类模型的时候,无论...

空空如也

空空如也

1 2 3 4 5
收藏数 86
精华内容 34
关键字:

python残差神经网络

python 订阅