精华内容
下载资源
问答
  • def cross_entropy_error(y, t): if y.ndim == 1: t = t.reshape(1, t.size) y = y.reshape(1, y.size) # 监督数据是one-hot-vector的情况下,转换为正确解标签的索引 if t.size == y.size: t = t.argmax...
    def cross_entropy_error(y, t):
        if y.ndim == 1:
            t = t.reshape(1, t.size)
            y = y.reshape(1, y.size)
            
        # 监督数据是one-hot-vector的情况下,转换为正确解标签的索引
        if t.size == y.size:
            t = t.argmax(axis=1)
                 
        batch_size = y.shape[0]
        return -np.sum(np.log(y[np.arange(batch_size), t] + 1e-7)) / batch_size

    看到这段代码可能不熟悉python的同学可能会犯迷糊,这里我谈一下我的理解。

    1.为什么有以下代码?

    if y.ndim == 1: t = t.reshape(1, t.size) y = y.reshape(1, y.size)

    答:因为现在是对min_batch个数据进行操作,通常数据是大于一个的。此时batch_size等于数据个数。但如果只有一个数据呢?此时如果不加上述代码,batch_size就会等于一维数组的元素个数,产生错误,所以加上 上述代码。

    2.t=t.argmax(axis=1)是什么意思?

    答:当t是一维数组,也就是只有一个数据时,t返回最大数的下标。

           当t是二维数组时,axis等于1,以数组形式返回每一行最大数的下标。

    3.np.sum(np.log(y[np.arange(batch_size), t] + 1e-7))是什么意思?

        3.1 首先要知道np.arange(batch_size), t] 是什么意思?

               它返回的是一个一维数组,数组中的元素是每个数据中最大的元素。

        3.2 那么这段代码的意思就是,把每个数据中最大的元素的对数值进行求和。

             

    展开全文
  • pfl_mpc.layers.softmax_with_cross_entropy(logits=fc_out, AttributeError: 'module' object has no attribute 'softmax_with_cross_entropy' Traceback (most recent call last): File &#...
  • 交叉熵cross_entropy的python实现

    千次阅读 2019-09-18 11:03:34
    代码实现 import math # Write a function that takes as input two lists Y, P,...# and returns the float corresponding to their cross-entropy. def cross_entropy(Y, P): sum=0.0 for x in map(lambda y,p:(...

    代码实现

    import math
    # Write a function that takes as input two lists Y, P,
    # and returns the float corresponding to their cross-entropy.
    def cross_entropy(Y, P):
        sum=0.0
        for x in map(lambda y,p:(1-y)*math.log(1-p)+y*math.log(p),Y,P):
            sum+=x
        return -sum/len(Y)
    if __name__ == "__main__":
        # 测试数据
        # Y表示是否为蓝色
        Y=[1,1,0,0]
        # P表示蓝色的概率
        P=[0.6,0.2,0.9,0.2]
        print(cross_entropy(Y,P))
    

    相关知识

    关于交叉熵

    图片
    图片

    关于map()函数

    >>>def square(x) :            # 计算平方数
    ...     return x ** 2
    ... 
    >>> map(square, [1,2,3,4,5])   # 计算列表各个元素的平方
    [1, 4, 9, 16, 25]
    >>> map(lambda x: x ** 2, [1, 2, 3, 4, 5])  # 使用 lambda 匿名函数
    [1, 4, 9, 16, 25]
     
    # 提供了两个列表,对相同位置的列表数据进行相加
    >>> map(lambda x, y: x + y, [1, 3, 5, 7, 9], [2, 4, 6, 8, 10])
    [3, 7, 11, 15, 19]
    

    关于map()更多相关知识

    关于log()函数

    import math   # 导入 math 模块
     
    print "math.log(100.12) : ", math.log(100.12)
    print "math.log(100.72) : ", math.log(100.72)
    print "math.log(119L) : ", math.log(119L)
    print "math.log(math.pi) : ", math.log(math.pi)
    # 设置底数
    print "math.log(10,2) : ", math.log(10,2)
    

    关于log()更多相关知识

    展开全文
  • 飞桨中softmax_with_cross_entropy的用法 本文只作为笔记用。 环境:paddle 1.8.5 一直以来不太清楚这个如何调用,只知道例程里是这样用,不知所以然。 下面记录一下它的用法。 源码如下: def softmax_with_cross_...

    飞桨中softmax_with_cross_entropy的用法


    本文只作为笔记用。
    环境:paddle 1.8.5

    一直以来不太清楚这个如何调用,只知道例程里是这样用,不知所以然。
    下面记录一下它的用法。
    源码如下:

    def softmax_with_cross_entropy(logits,
                                   label,
                                   soft_label=False,
                                   ignore_index=kIgnoreIndex,
                                   numeric_stable_mode=True,
                                   return_softmax=False,
                                   axis=-1):
        """
        This operator implements the cross entropy loss function with softmax. This function 
        combines the calculation of the softmax operation and the cross entropy loss function 
        to provide a more numerically stable gradient.
    
        Because this operator performs a softmax on logits internally, it expects
        unscaled logits. This operator should not be used with the output of
        softmax operator since that would produce incorrect results.
    
        When the attribute :attr:`soft_label` is set :attr:`False`, this operators 
        expects mutually exclusive hard labels, each sample in a batch is in exactly 
        one class with a probability of 1.0. Each sample in the batch will have a 
        single label.
    
        The equation is as follows:
    
        1) Hard label (one-hot label, so every sample has exactly one class)
    
        .. math::
    
            loss_j =  -\\text{logits}_{label_j} +
            \\log\\left(\\sum_{i=0}^{K}\\exp(\\text{logits}_i)\\right), j = 1,..., K
    
        2) Soft label (each sample can have a distribution over all classes)
    
        .. math::
    
            loss_j =  -\\sum_{i=0}^{K}\\text{label}_i
            \\left(\\text{logits}_i - \\log\\left(\\sum_{i=0}^{K}
            \\exp(\\text{logits}_i)\\right)\\right), j = 1,...,K
    
        3) If :attr:`numeric_stable_mode` is :attr:`True`, softmax is calculated first by:
    
        .. math::
    
            max_j &= \\max_{i=0}^{K}{\\text{logits}_i}
    
            log\\_max\\_sum_j &= \\log\\sum_{i=0}^{K}\\exp(logits_i - max_j)
    
            softmax_j &= \\exp(logits_j - max_j - {log\\_max\\_sum}_j)
    
        and then cross entropy loss is calculated by softmax and label.
    
        Args:
            logits (Variable): A multi-dimension ``Tensor`` , and the data type is float32 or float64. The input tensor of unscaled log probabilities.
            label (Variable): The ground truth  ``Tensor`` , data type is the same
                as the ``logits`` . If :attr:`soft_label` is set to :attr:`True`, 
                Label is a ``Tensor``  in the same shape with :attr:`logits`. 
                If :attr:`soft_label` is set to :attr:`True`, Label is a ``Tensor`` 
                in the same shape with :attr:`logits` expect shape in dimension :attr:`axis` as 1.
            soft_label (bool, optional): A flag to indicate whether to interpretant the given
                labels as soft labels. Default False.
            ignore_index (int, optional): Specifies a target value that is ignored and does
                                          not contribute to the input gradient. Only valid
                                          if :attr:`soft_label` is set to :attr:`False`. 
                                          Default: kIgnoreIndex(-100).
            numeric_stable_mode (bool, optional): A flag to indicate whether to use a more
                                                  numerically stable algorithm. Only valid
                                                  when :attr:`soft_label` is :attr:`False` 
                                                  and GPU is used. When :attr:`soft_label` 
                                                  is :attr:`True` or CPU is used, the 
                                                  algorithm is always numerically stable.
                                                  Note that the speed may be slower when use
                                                  stable algorithm. Default: True.
            return_softmax (bool, optional): A flag indicating whether to return the softmax
                                             along with the cross entropy loss. Default: False.
            axis (int, optional): The index of dimension to perform softmax calculations. It 
                                  should be in range :math:`[-1, rank - 1]`, while :math:`rank`
                                  is the rank of input :attr:`logits`. Default: -1.
    
        Returns:
            ``Variable`` or Tuple of two ``Variable`` : Return the cross entropy loss if \
                                                        `return_softmax` is False, otherwise the tuple \
                                                        (loss, softmax), softmax is in the same shape \
                                                        with input logits and cross entropy loss is in \
                                                        the same shape with input logits except shape \
                                                        in dimension :attr:`axis` as 1.
    
        Examples:
            .. code-block:: python
    
                import paddle.fluid as fluid
    
                data = fluid.data(name='data', shape=[-1, 128], dtype='float32')
                label = fluid.data(name='label', shape=[-1, 1], dtype='int64')
                fc = fluid.layers.fc(input=data, size=100)
                out = fluid.layers.softmax_with_cross_entropy(
                    logits=fc, label=label)
        """
        if in_dygraph_mode():
            softmax, loss = core.ops.softmax_with_cross_entropy(
                logits, label, 'soft_label', soft_label, 'ignore_index',
                ignore_index, 'numeric_stable_mode', numeric_stable_mode, 'axis',
                axis)
            if not return_softmax:
                return loss
            else:
                return loss, softmax
    
        attrs = {
            'soft_label': soft_label,
            'ignore_index': ignore_index,
            'numeric_stable_mode': numeric_stable_mode,
            'axis': axis
        }
        helper = LayerHelper('softmax_with_cross_entropy', **locals())
        softmax = helper.create_variable_for_type_inference(dtype=logits.dtype)
        loss = helper.create_variable_for_type_inference(dtype=logits.dtype)
        helper.append_op(
            type='softmax_with_cross_entropy',
            inputs={'Logits': logits,
                    'Label': label},
            outputs={'Softmax': softmax,
                     'Loss': loss},
            attrs=attrs)
    
        if return_softmax:
            return loss, softmax
    
        return loss
    

    一开始我以为,它会使用两个相同的array丢进去就OK了。
    但是以下做法是错误的。

    logit_y = np.array([[1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11]]).astype(np.float32)
    output_y = np.array([[0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0]]).astype(np.float32) # ng
    logit_y = fluid.dygraph.to_variable(logit_y)
    output_y = fluid.dygraph.to_variable(output_y)
    print(logit_y.shape, output_y.shape)
    print(logit_y.numpy(), output_y.numpy())
    loss = fluid.layers.softmax_with_cross_entropy(logit_y, output_y)
    print(loss.numpy()[0])
    

    以下做法是错误的。

    logit_y = np.array([[1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11]]).astype(np.float32)
    output_y = np.array([[0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0]]).astype(np.float32) # ng
    logit_y = fluid.dygraph.to_variable(logit_y)
    output_y = fluid.dygraph.to_variable(output_y)
    print(logit_y.shape, output_y.shape)
    print(logit_y.numpy(), output_y.numpy())
    loss = fluid.layers.softmax_with_cross_entropy(logit_y, output_y)
    print(loss.numpy()[0])
    

    错误信息:

    EnforceNotMet                             Traceback (most recent call last)
    <ipython-input-46-4960b363e8fb> in <module>
          7 print(logit_y.shape, output_y.shape)
          8 print(logit_y.numpy()[0], output_y.numpy()[0])
    ----> 9 loss = fluid.layers.softmax_with_cross_entropy(logit_y, output_y)
         10 print(loss.numpy()[0])
    
    d:\programdata\anaconda3\envs\parl\lib\site-packages\paddle\fluid\layers\loss.py in softmax_with_cross_entropy(logits, label, soft_label, ignore_index, numeric_stable_mode, return_softmax, axis)
       1254             logits, label, 'soft_label', soft_label, 'ignore_index',
       1255             ignore_index, 'numeric_stable_mode', numeric_stable_mode, 'axis',
    -> 1256             axis)
       1257         if not return_softmax:
       1258             return loss
    
    EnforceNotMet: 
    
    --------------------------------------------
    C++ Call Stacks (More useful to developers):
    --------------------------------------------
    Windows not support stack backtrace yet.
    
    ----------------------
    Error Message Summary:
    ----------------------
    InvalidArgumentError: If Attr(soft_label) == false, the axis dimension of Input(Label) should be 1.
      [Hint: Expected labels_dims[axis] == 1UL, but received labels_dims[axis]:4 != 1UL:1.] at (D:\1.8.5\paddle\paddle\fluid\operators\softmax_with_cross_entropy_op.cc:174)
      [operator < softmax_with_cross_entropy > error]
    
    

    在paddle里,它的softmax_with_cross_entropy的第一个参数输入的是对应你多种类别分别的概率,第二个参数是类别。
    以下是正确的结果的code。

    logit_y = np.array([[1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11]]).astype(np.float32)
    output_y = np.array([[0], [0], [0], [0]]).astype(np.int64)  # ok
    logit_y = fluid.dygraph.to_variable(logit_y)
    output_y = fluid.dygraph.to_variable(output_y)
    print(logit_y.shape, output_y.shape)
    print(logit_y.numpy(), output_y.numpy())
    loss = fluid.layers.softmax_with_cross_entropy(logit_y, output_y)
    print(loss.numpy()[0])
    # logit_y
    # [[1.23 2.33 3.33 2.11]
    #  [1.23 2.33 3.33 2.11]
    #  [1.23 2.33 3.33 2.11]
    #  [1.23 2.33 3.33 2.11]]
    # 结果
    # [[2.6797354]
    #  [2.6797354]
    #  [2.6797354]
    #  [2.6797354]]
    

    或者

    logit_y = np.array([[1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11], [1.23, 2.33, 3.33, 2.11]]).astype(np.float32)
    output_y = np.array([[0], [0], [0], [0]]).astype(np.int64)  # ok
    logit_y = fluid.dygraph.to_variable(logit_y)
    output_y = fluid.dygraph.to_variable(output_y)
    logit_y = fluid.layers.softmax(logit_y)
    print(logit_y.shape, output_y.shape)
    print(logit_y.numpy(), output_y.numpy()[0])
    loss = fluid.layers.softmax_with_cross_entropy(logit_y, output_y)
    print(loss.numpy())
    # logit_y
    # [[0.0685813  0.2060296  0.5600465  0.16534261]
    #  [0.0685813  0.2060296  0.5600465  0.16534261]
    #  [0.0685813  0.2060296  0.5600465  0.16534261]
    #  [0.0685813  0.2060296  0.5600465  0.16534261]]
    # 结果
    # [[1.5858928]
    #  [1.5858928]
    #  [1.5858928]
    #  [1.5858928]]
    
    展开全文
  • F.binary_cross_entropy_with_logits

    千次阅读 2020-11-01 11:52:38
    torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum) Error1: RuntimeError: the derivative for 'weight' is not implemented binary_cross_entropy_with_...

    torch.nn.functional.binary_cross_entropy_with_logits(inputtargetweight=Nonesize_average=Nonereduce=Nonereduction='mean'pos_weight=None)

     

    Error1: 

    RuntimeError: the derivative for 'weight' is not implemented

    binary_cross_entropy_with_logits doesn't support back-propagating through the weights attribute.

    If you don't need the derivative weights then you can use weights.detach()/weights.data instead of weights.

    If you need the derivative, then you'll having to implement binary_cross_entropy_with_logits yourself.

     

    Error2: 

    RuntimeError: result type Float can't be cast to the desired output type Long

    The error points to the target. Convert it to a torch.cuda.FloatTensor using: 

    target = target.float()

     

    Error3:

    RuntimeError: weight tensor should be defined either for all or no classes at /opt/conda/conda-bld/pytorch_1595629408163/work/aten/src/THCUNN/generic/SpatialClassNLLCriterion.cu:27

    Check the number of classes and classes_weights.

    展开全文
  • fluid.layers.softmax_with_cross_entropy 作用:OP实现了softmax交叉熵损失函数。该函数会将softmax操作、交叉熵损失函数的计算过程进行合并,从而提供了数值上更稳定的梯度值。 链接:pp飞桨API说明 复现问题 ...
  • tf.nn.sigmoid_cross_entropy_with_logits()

    万次阅读 多人点赞 2018-08-03 20:59:34
    在深度学习的编程题里面...tf.nn.sigmoid_cross_entropy_with_logits(_sentinel=None,,labels=None,logits=None,name=None) 对于给定的logits计算sigmoid的交叉熵。 衡量的是分类任务中的概率误差,他也是试用于每...
  • loss_func = F.cross_entropy xb = torch.randn(2,784) yb = torch.tensor([1,1]) print ("model(xb):",model(xb)) print ("yb:",yb) loss = loss_func(model(xb),yb) print ("loss:",loss) 这里要注意的是yb是...
  • tf.nn.weighted_cross_entropy_with_logits函数 tf.nn.weighted_cross_entropy_with_logits( targets, logits, pos_weight, name=None ) 定义在:tensorflow/python/ops/nn_impl.py。 计算加权交叉熵。 ...
  • torch.nn.functional.binary_cross_entropy_with_logits(input, target, weight=None, size_average=None, reduce=None, reduction=‘mean’, pos_weight=None) 参数说明 input神经网络预测结果(未经过sigmoid),...
  • wrapped_fn().__wrapped__(*args, **kwds) File "F:\Anaconda3\lib\site-packages\tensorflow\python\framework\func_graph.py", line 973, in wrapper raise e.ag_error_metadata.to_exception(e) TypeError: in ...
  • 1.tf.nn.softmax_cross_entropy_with_logits作用 这里先给出两个函数: tf.nn.softmax_cross_entropy_with_logits(_sentinel=None, # pylint: disable=invalid-name labels=None, logits...
  • tf.nn.sigmoid_cross_entropy_with_logits(_sentinel=None,,labels=None,logits=None,name=None) logits和labels必须有相同的类型和大小 参数: _sentinel:内部的并不使用 labels:和logits的shape和type一样 ...
  • sigmoid_cross_entropy_with_logits 可以衡量已标注过数据预测的正确度。比如一个(x,y)数据项,x表示输入取值为实数,y表示标注取值0或1。根据x算出一个概率p=1/(1+e^(-x)) 。 可以看到,当x=0时p=0.5 ,标注为...
  • print(sess.run(tf.nn.sigmoid_cross_entropy_with_logits(labels=labels,logits=logits))) label是one-hot的形式,有三行,表示有三个instance,1对应的位置表示每个instance真实的类别,即分别为类1,类2,类3...
  • 运行一个程序时提示出错如下:Traceback (most recent call last): File "/MNIST/softmax.py", line 12, in cross_entropy2=tf.reduce_sum(tf.nn.softmax_cross_entropy_with_logits(logits, y_))#dont forget tf....
  • 1、tf.Session函数 ...2、tf.losses.sparse_softmax_cross_entropy函数 这个函数做的事情: 预测值y_->softmax,取概率最大 实际值y->onehot 计算loss = ylogy_ tf.losses.sparse_softmax_cros...
  • 而label:[0,0,1] 所以交叉熵较小,但是第二个数据和label的差异明显就比较大,所以对应它的交叉熵就比较大,所以直接用logtis数据带入到softmax_crossentropy里面其实是正确的结果! 而下面就是将softmax的结果...
  • torch.nn.BCELoss()=F.binary_cross_entropy()

    千次阅读 2019-05-27 16:58:28
    loss = nn.CrossEntropyLoss() l=loss(fc_out,lable) print(l) # tensor(0.8327) # 将上面的函数编写出来 sigmoid_fc=1/(1+torch.exp(-fc_out)) def binary_cross_entropyloss(prob,target,reduce=True): loss = ...
  • tf.nn.sparse_softmax_cross_entropy_with_logits

    万次阅读 2017-09-15 14:34:35
    tf.nn.sparse_softmax_cross_entropy_with_logits(logits, labels, name=None) 计算logits 和 labels 之间的稀疏softmax 交叉熵 度量在离散分类任务中的错误率,这些类之间是相互排斥的(每个输入只能对应唯一...
  • weighted_cross_entropy_with_logits原创文章,请勿转载!!!weighted_cross_entropy_with_logits(targets, logits, pos_weight, name=None):此函数功能以及计算方式基本与tf_nn_sigmoid_cross_entropy_with_logits...
  • sparse_softmax_cross_entropy_with_logits实际上是softmax和cross_entropy(交叉熵)两个函数的组合一、softmax核心是做归一化而softmax把一个k维的real value向量(a1,a2,a3,a4….)映射成一个(b1,b2,b3,b4….)...
  • tf.reduce_mean()函数使用方法为: 因为交叉熵一般会与softmax回归一起使用,所以Tensorflow对这两个功能进行了同一封装,并提供了tf.nn.softmax_cross_entropy_with_logits()函数。比如可以直接通过下面的代码实现...
  • softmax cross_entropy

    2018-11-10 11:10:43
    softmax函数作用 交叉熵代价函数的优点 一般的代价函数比如二次代价函数,对参数w或者b求偏导之后,结果和激活函数的导数成正比,也就是说当激活函数导数很小的时候(随机初始化导致激活函数输入小,从而对应导数大)...
  • tf.nn.sigmoid_cross_entropy_with_logits

    千次阅读 2018-02-04 17:49:57
    tf.nn.sigmoid_cross_entropy_with_logits sigmoid_cross_entropy_with_logits(  _sentinel=None,  labels=None,  logits=None,  name=None ) Defined in tensorflow/python/ops/nn_impl.py.
  • cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=tf.argmax(y_, 1), logits=y)#labels=tf.argmax(y_, 1), logits=y #这里tf.argmax(y_,1)表示在“行”这个维度上张量最大元素的索引号 ...
  • class torch.nn.CrossEntropyLoss(weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='elementwise_mean') https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss 在...
  • 首先把Tensorflow英文API搬...tf.nn.softmax_cross_entropy_with_logits(_sentinel=None, labels=None, logits=None, dim=-1, name=None) Computes softmax cross entropy between logits and labels. Measure
  • cross_entropy target参数只需要标签即可, 不需要传one-hot向量 代码试验 传入one-hot向量报错 import torch import torch.nn.functional as F a = torch.Tensor([[0, 0, 1], [1, 0, 0]]).long() b = torch.Tensor(...

空空如也

空空如也

1 2 3 4 5 ... 20
收藏数 11,634
精华内容 4,653
关键字:

cross_entropy_error