精华内容
下载资源
问答
  • Dropout in Keras

    2019-10-08 09:48:31
  • Keras dropout

    千次阅读 2018-11-17 16:23:56
    之前一直困惑,keras在测试的时候到底模型dropout没,经过到处查阅,可以确定的是,kerasdropout仅用于训练集。

    之前一直困惑,keras在测试的时候到底模型dropout没,经过到处查阅,可以确定的是,keras的dropout仅用于训练集。

    展开全文
  • kerasdropout

    万次阅读 2018-12-31 19:41:31
    1,在input第一层之间加droppout 2.在hiddenlayer中加 W_constraint=maxnorm(3)是原始论文里面建议的认为系数最好不要超过3 import numpy import pandas from keras.models import ...from keras.layers...

    1,在input第一层之间加droppout

    2.在hiddenlayer中加

    W_constraint=maxnorm(3)是原始论文里面建议的认为系数最好不要超过3

    import numpy
    import pandas
    from keras.models import Sequential
    from keras.layers import Dense
    from keras.layers import Dropout
    from keras.wrappers.scikit_learn import KerasClassifier
    from keras.constraints import maxnorm
    from keras.optimizers import SGD
    from sklearn.cross_validation import cross_val_score
    from sklearn.preprocessing import LabelEncoder
    from sklearn.cross_validation import StratifiedKFold
    from sklearn.preprocessing import StandardScaler
    from sklearn.grid_search import GridSearchCV
    from sklearn.pipeline import Pipeline
    # fix random seed for reproducibility
    seed = 7
    numpy.random.seed(seed)
    # load dataset
    dataframe = pandas.read_csv("../data/sonar.csv", header=None)
    dataset = dataframe.values
    # split into input (X) and output (Y) variables
    X = dataset[:,0:60].astype(float)
    Y = dataset[:,60]
    # encode class values as integers
    encoder = LabelEncoder()
    encoder.fit(Y)
    encoded_Y = encoder.transform(Y)
    
    
    # baseline
    def create_baseline():
    	# create model
    	model = Sequential()
    	model.add(Dense(60, input_dim=60, init='normal', activation='relu'))
    	model.add(Dense(30, init='normal', activation='relu'))
    	model.add(Dense(1, init='normal', activation='sigmoid'))
    	# Compile model
    	sgd = SGD(lr=0.01, momentum=0.8, decay=0.0, nesterov=False)
    	model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy'])
    	return model
    
    numpy.random.seed(seed)
    estimators = []
    estimators.append(('standardize', StandardScaler()))
    estimators.append(('mlp', KerasClassifier(build_fn=create_baseline, nb_epoch=300, batch_size=16, verbose=0)))
    pipeline = Pipeline(estimators)
    kfold = StratifiedKFold(y=encoded_Y, n_folds=10, shuffle=True, random_state=seed)
    results = cross_val_score(pipeline, X, encoded_Y, cv=kfold)
    print("Accuracy: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))
    # Accuracy: 82.68% (3.90%)
    
    
    # dropout in the input layer with weight constraint
    def create_model1():
    	# create model
    	model = Sequential()
    	model.add(Dropout(0.2, input_shape=(60,)))
    	model.add(Dense(60, init='normal', activation='relu', W_constraint=maxnorm(3)))
    	model.add(Dense(30, init='normal', activation='relu', W_constraint=maxnorm(3)))
    	model.add(Dense(1, init='normal', activation='sigmoid'))
    	# Compile model
    	sgd = SGD(lr=0.1, momentum=0.9, decay=0.0, nesterov=False)
    	model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy'])
    	return model
    
    numpy.random.seed(seed)
    estimators = []
    estimators.append(('standardize', StandardScaler()))
    estimators.append(('mlp', KerasClassifier(build_fn=create_model1, nb_epoch=300, batch_size=16, verbose=0)))
    pipeline = Pipeline(estimators)
    kfold = StratifiedKFold(y=encoded_Y, n_folds=10, shuffle=True, random_state=seed)
    results = cross_val_score(pipeline, X, encoded_Y, cv=kfold)
    print("Accuracy: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))
    # Accuracy: 86.04% (6.33%)
    
    
    # dropout in hidden layers with weight constraint
    def create_model2():
    	# create model
    	model = Sequential()
    	model.add(Dense(60, input_dim=60, init='normal', activation='relu', W_constraint=maxnorm(3)))
    	model.add(Dropout(0.2))
    	model.add(Dense(30, init='normal', activation='relu', W_constraint=maxnorm(3)))
    	model.add(Dropout(0.2))
    	model.add(Dense(1, init='normal', activation='sigmoid'))
    	# Compile model
    	sgd = SGD(lr=0.1, momentum=0.9, decay=0.0, nesterov=False)
    	model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy'])
    	return model
    
    numpy.random.seed(seed)
    estimators = []
    estimators.append(('standardize', StandardScaler()))
    estimators.append(('mlp', KerasClassifier(build_fn=create_model2, nb_epoch=300, batch_size=16, verbose=0)))
    pipeline = Pipeline(estimators)
    kfold = StratifiedKFold(y=encoded_Y, n_folds=10, shuffle=True, random_state=seed)
    results = cross_val_score(pipeline, X, encoded_Y, cv=kfold)
    print("Accuracy: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))
    # Accuracy: 82.16% (6.16%)
    
    

    几个建议

    1.dropout在0.2~0.5之间,一般0.2就挺好的

    2.input也可以用,不止是hiddenlayer

    3. Use a larger network. You are likely to get better performance when dropout is used on a larger network, giving the model more of an opportunity to learn independent representations.
    4. Use a large learning rate with decay and a large momentum. Increase your learning rate by a factor of 10 to 100 and use a high momentum value of 0.9 or 0.99.
    5. Constrain the size of network weights. A large learning rate can result in very large network weights. Imposing a constraint on the size of network weights such as max-norm regularization with a size of 4 or 5 has been shown to improve results 

     

    展开全文
  • tf.keras.layers.Dropout( rate, noise_shape=None, seed=None, **kwargs ) 官网地址:https://tensorflow.google.cn/api_docs/python/tf/keras/layers/Dropout 作用 防止过拟合,提高模型的泛化能力。 参数 rate...

    函数原型

    tf.keras.layers.Dropout(
        rate, noise_shape=None, seed=None, **kwargs
    )
    

    官网地址:https://tensorflow.google.cn/api_docs/python/tf/keras/layers/Dropout

    作用

    防止过拟合,提高模型的泛化能力。

    参数

    • rate:0~1之间的小数。让神经元以一定的概率rate停止工作,提高模型的泛化能力。
    • noise_shape:1D张量类型,int32表示将与输入相乘的二进制丢失掩码的形状;例如,如果您的输入具有形状(batch_size, timesteps, features),并且你希望所有时间步长的丢失掩码相同,则可以使用noise_shape=[batch_size, 1, features],就是哪一个是1,那么就在哪一维度按照相同的方式dropout,如果没有1就是普通的。
      这个参数有篇文章我觉得解释的很好,dropout中的noise_shape参数的作用
    • seed:随机种子
    展开全文
  • kerasdropout使用

    万次阅读 2019-06-12 16:07:41
    model.add(Dropout(0.02)) #!/usr/bin/env python # -*- coding:utf-8 -*- # Author: Jia ShiLin import numpy as np import pandas as pd from matplotlib import pyplot as plt from keras.models import Se....
  • TensorFlow keras dropout

    2020-02-03 19:07:00
    model = keras.Sequential([ keras.layers.Flatten(input_shape=(28, 28)), # 将输入数据的形状进行修改成神经网络要求的数据形状 keras.layers.Dense(128, activation=tf.nn.relu), # 定义隐藏层,128个神经元.....
  • 参考博客:keras在测试时,dropout rate不需要改变成1.0
  • import keras from keras.datasets import mnist from keras.utils import np_utils from keras.layers import ...from keras.layers import Dropout from keras.regularizers import l2#正则化 from keras.models...
  • keras基础--5.Dropout

    2021-05-12 17:19:36
    import numpy as np from keras.datasets import mnist ...from keras.layers import Dense,Dropout from keras.optimizers import SGD # 载入数据 (x_train,y_train),(x_test,y_test) = mnist.load_data() #
  • import numpy as np from keras.datasets import mnist from keras.utils import np_utils from keras.models import ...from keras.layers import Dense,Dropout from keras.optimizers import SGD #load...
  • keras LSTM中间的dropout

    千次阅读 2017-07-17 14:58:00
    TM有三个 model.add(LSTM(100, ...第一个dropout是x和hidden之间的dropout,第二个是hidden-hidden之间的dropout 在tensorflow里面有 第三个是层-层之间的dropout model.add(Embedding(top_words, embeddi...
  • keras学习-SpatialDropout1D层

    千次阅读 2019-04-25 21:32:58
    keras.layers.core.SpatialDropout1D(p) SpatialDropout1D与Dropout的作用类似,但它断开的是整个1D特征图,而不是单个神经元。如果一张特征图的相邻像素之间有很强的相关性(通常发生在低层的卷积层中),那么...
  • 一.引言 Dropout 层用于解决过拟合的问题,当原始样本偏少而深度模型参数偏多时,模型偏向于学习一些极端的特征从而发生过拟合,在xun'li'na'yang
  • keras 关于 dropout 的一点讨论

    千次阅读 2018-03-26 15:08:22
    dropout in training and testing #5357 wenouyang commented on 11 Feb 2017In this link, devinplatt gives the following way to include dropout in training,model = Sequential() model.add(Dropout(0.5,...
  • 基于keras的CNN的蒙特卡罗dropout 在利用keras或者tensorflow.keras写CNN时,一般会利用dropout层来防止CNN的过拟合。此时,除了一般如loss和val_accuracy之外,基于蒙特卡罗方法dropout的模型不确定性(uncertainty...
  • 4、 Dropout 正则是通过在代价函数后面加上正则项来防止模型过拟合。在神经网络中,有一种方法是通过修改自身结构来实现的,叫做Dropout。这是对网络训练的一种技巧,在训练过程中随机删除一些隐藏层的神经元,同时...
  • 1、dropout from sklearn import datasets import numpy as np from keras.models import ...from keras.layers import Dropout from keras.layers import Dense from keras.constraints import maxnorm fr...
  • Keras源代码阅读-Dropout

    千次阅读 2016-04-26 17:00:20
    Dropout层源代码dropout层在layer下的core.py中class Dropout(Layer): '''Applies Dropout to the input. Dropout consists in randomly setting a fraction `p` of input units to 0 at each update during ...
  • 第16章 Keras使用Dropout正则化防止过拟合 Dropout虽然简单,但可以有效防止过拟合。本章关于如何在Keras中使用Dropout。本章包括: dropout的原理 dropout的使用 在隐层上使用dropout 我们开始吧。 16.1 Dropout...
  • tf.keras学习之layers.Dropout

    千次阅读 2020-05-14 14:56:34
    tf.keras.layers.Dropout( rate, noise_shape=None, seed=None, **kwargs ) 参数 rate:在0到1之间浮动。要降低的输入单位的分数。 noise_shape:1D张量类型,int32表示将与输入相乘的二进制丢失掩码的形状;...
  • 转: https://zhuanlan.zhihu.com/p/37918855
  • 如何在tf.keras.layers中添加Dropout

    千次阅读 2020-05-12 10:44:44
    在tf.keras高级API中添加Dropout层的方式与早期的Tensorflow+keras的方式有所不同。典型的案例如下: dropout_model = tf.keras.Sequential([ tf.keras.layers.Dense(512, activation='elu', input_shape=...
  • 在这篇文章中,你将发现Dropout正则化技术,以及如何使用Keras将其应用于Python中的模型。 看完这篇文章后,你会知道: Dropout正则化的原理。 如何在输入层上使用Dropout。 如何在隐藏的层上使用Dropout。 ...
  • 主要介绍了浅谈kerasDropout在预测过程中是否仍要起作用,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧
  • <div><p>my trained model has dropout layer ,and it gives out different servables every time. <p>how can I solve the problem?</p><p>该提问来源于开源项目:tensorflow/serving</p></div>
  • seid=17003307842787199553笔记使用dropout是要改善过拟合,将训练和测试的准确率差距变小训练集,测试集结果相比差距较大时,过拟合状态使用dropout后,每一周期准确率可能不高反而最后一步提升很快,这是训练的...
  • 通过实验分析kerasDropout在训练阶段和测试阶段的使用情况。 结论: Keras使用的 Inverted Dropout,因此测试时不需要修改 Dropout中的参数(rate)。 1. Dropout 的实现方式 Dropout的实现方式有两种。 ...

空空如也

空空如也

1 2 3 4 5 ... 20
收藏数 1,294
精华内容 517
关键字:

dropoutkeras