精华内容
下载资源
问答
  • 2020-04-11 01:08:39

    TF:tensorflow框架中常用函数介绍—tf.Variable()和tf.get_variable()用法及其区别

     

     

    目录

    tensorflow框架

    tensorflow.Variable()函数

    tensorflow.get_variable()函数


     

    tensorflow框架

    tf.Variable()和tf.get_variable()在创建变量的过程基本一样。它们之间最大的区别在于指定变量名称的参数。

    • tf.Variable(),变量名称name是一个可选的参数。
    • tf.get_variable(),变量名称是一个必填的参数。

     

    tensorflow.Variable()函数


    @tf_export("Variable")
    class Variable(checkpointable.CheckpointableBase):
      """See the @{$variables$Variables How To} for a high level overview.

      A variable maintains state in the graph across calls to `run()`. You add a  variable to the graph by constructing an instance of the class `Variable`.

      The `Variable()` constructor requires an initial value for the variable, which can be a `Tensor` of any type and shape. The initial value defines the  type and shape of the variable. After construction, the type and shape of
      the variable are fixed. The value can be changed using one of the assign  methods.

      If you want to change the shape of a variable later you have to use an  `assign` Op with `validate_shape=False`.

      Just like any `Tensor`, variables created with `Variable()` can be used as inputs for other Ops in the graph. Additionally, all the operators overloaded for the `Tensor` class are carried over to variables, so you can
      also add nodes to the graph by just doing arithmetic on variables.

      ```python
      import tensorflow as tf

      # Create a variable.
      w = tf.Variable(<initial-value>, name=<optional-name>)

      # Use the variable in the graph like any Tensor.
      y = tf.matmul(w, ...another variable or tensor...)

      # The overloaded operators are available too.
      z = tf.sigmoid(w + y)

      # Assign a new value to the variable with `assign()` or a related method.
      w.assign(w + 1.0)
      w.assign_add(1.0)

    @tf_export(“变量”)

    类变量(checkpointable.CheckpointableBase):

    查看@{$variables$ variables How To}获取高级概述。

    一个变量在调用“run()”时维护图中的状态。通过构造类“variable”的一个实例,可以将一个变量添加到图形中。

    ‘Variable()’构造函数需要一个变量的初值,它可以是任何类型和形状的‘张量’。初始值定义变量的类型和形状。施工后,的类型和形状

    变量是固定的。可以使用指定方法之一更改值。

    如果以后要更改变量的形状,必须使用' assign ' Op和' validate_shape=False '。

    与任何“张量”一样,用“Variable()”创建的变量可以用作图中其他操作的输入。此外,“张量”类的所有运算符都重载了,因此可以转移到变量中

    还可以通过对变量进行运算将节点添加到图中。

    ”“python

    导入tensorflow作为tf

    创建一个变量。

    w =特遣部队。变量(name = <可选名称> <初值>)

    像使用任何张量一样使用图中的变量。

    y =特遣部队。matmul (w,…另一个变量或张量……)

    重载的操作符也是可用的。

    z =特遣部队。乙状结肠(w + y)

    用' Assign() '或相关方法为变量赋值。

    w。分配(w + 1.0)

    w.assign_add (1.0)

    ' ' '

    When you launch the graph, variables have to be explicitly initialized before you can run Ops that use their value. You can initialize a variable by running its *initializer op*, restoring the variable from a save file, or simply running an `assign` Op that assigns a value to the variable. In fact,  the variable *initializer op* is just an `assign` Op that assigns the variable's initial value to the variable itself.

      ```python
      # Launch the graph in a session.
      with tf.Session() as sess:
          # Run the variable initializer.
          sess.run(w.initializer)
          # ...you now can run ops that use the value of 'w'...
      ```

      The most common initialization pattern is to use the convenience function global_variables_initializer()` to add an Op to the graph that initializes  all the variables. You then run that Op after launching the graph.

      ```python
      # Add an Op to initialize global variables.
      init_op = tf.global_variables_initializer()

      # Launch the graph in a session.
      with tf.Session() as sess:
          # Run the Op that initializes global variables.
          sess.run(init_op)
          # ...you can now run any Op that uses variable values...
      ```

      If you need to create a variable with an initial value dependent on another variable, use the other variable's `initialized_value()`. This ensures that variables are initialized in the right order. All variables are automatically collected in the graph where they are created. By default, the constructor adds the new variable to the graph  collection `GraphKeys.GLOBAL_VARIABLES`. The convenience function

      `global_variables()` returns the contents of that collection.

      When building a machine learning model it is often convenient to distinguish  between variables holding the trainable model parameters and other variables  such as a `global step` variable used to count training steps. To make this  easier, the variable constructor supports a `trainable=<bool>` parameter. If `True`, the new variable is also added to the graph collection `GraphKeys.TRAINABLE_VARIABLES`. The convenience function `trainable_variables()` returns the contents of this collection. The various `Optimizer` classes use this collection as the default list of  variables to optimize.

      WARNING: tf.Variable objects have a non-intuitive memory model. A Variable is represented internally as a mutable Tensor which can non-deterministically alias other Tensors in a graph. The set of operations which consume a Variable  and can lead to aliasing is undetermined and can change across TensorFlow versions. Avoid writing code which relies on the value of a Variable either  changing or not changing as other operations happen. For example, using Variable objects or simple functions thereof as predicates in a `tf.cond` is  dangerous and error-prone:

      ```
      v = tf.Variable(True)
      tf.cond(v, lambda: v.assign(False), my_false_fn)  # Note: this is broken.
      ```

      Here replacing tf.Variable with tf.contrib.eager.Variable will fix any nondeterminism issues.

      To use the replacement for variables which does not have these issues:

      * Replace `tf.Variable` with `tf.contrib.eager.Variable`;
      * Call `tf.get_variable_scope().set_use_resource(True)` inside a  `tf.variable_scope` before the `tf.get_variable()` call.

      @compatibility(eager)
      `tf.Variable` is not compatible with eager execution.  Use  `tf.contrib.eager.Variable` instead which is compatible with both eager  execution and graph construction.  See [the TensorFlow Eager Execution  guide](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/eager/python/g3doc/guide.md#variables-and-optimizers)
      for details on how variables work in eager execution.
      @end_compatibility
      """

    启动图形时,必须显式初始化变量,然后才能运行使用其值的操作。您可以通过运行它的*initializer op*来初始化一个变量,也可以从保存文件中恢复这个变量,或者简单地运行一个' assign ' op来为这个变量赋值。实际上,变量*初始化器op*只是一个' assign ' op,它将变量的初始值赋给变量本身。

    ”“python
    在会话中启动图形。
    session()作为sess:
    #运行变量初始化器。
    sess.run (w.initializer)
    #……现在可以运行使用'w'值的ops…
    ' ' '

    最常见的初始化模式是使用方便的函数global_variables_initializer() '将Op添加到初始化所有变量的图中。然后在启动图形之后运行该Op。

    ”“python
    #添加一个Op来初始化全局变量。
    init_op = tf.global_variables_initializer ()

    在会话中启动图形。
    session()作为sess:
    运行初始化全局变量的Op。
    sess.run (init_op)
    #……您现在可以运行任何使用变量值的Op…
    ' ' '

    如果需要创建一个初始值依赖于另一个变量的变量,请使用另一个变量的' initialized_value() '。这样可以确保以正确的顺序初始化变量。所有变量都自动收集到创建它们的图中。默认情况下,构造函数将新变量添加到图形集合“GraphKeys.GLOBAL_VARIABLES”中。方便的功能

    ' global_variables() '返回该集合的内容。

    在构建机器学习模型时,通常可以方便地区分包含可训练模型参数的变量和其他变量,如用于计算训练步骤的“全局步骤”变量。为了简化这一点,变量构造函数支持一个' trainable=<bool> '参数。</bool>如果为True,则新变量也将添加到图形集合“GraphKeys.TRAINABLE_VARIABLES”中。便利函数' trainable_variables() '返回这个集合的内容。各种“优化器”类使用这个集合作为要优化的默认变量列表。

    警告:tf。变量对象有一个不直观的内存模型。一个变量在内部被表示为一个可变张量,它可以不确定性地混叠一个图中的其他张量。使用变量并可能导致别名的操作集是未确定的,可以跨TensorFlow版本更改。避免编写依赖于变量值的代码,这些变量值随着其他操作的发生而改变或不改变。例如,在“tf”中使用变量对象或其简单函数作为谓词。cond’是危险的,容易出错的:

    ' ' '
    v = tf.Variable(真正的)
    特遣部队。cond(v, lambda: v.assign(False), my_false_fn) #注意:这个坏了。
    ' ' '

    这里替换特遣部队。与tf.contrib.eager变量。变量将修复任何非决定论的问题。

    使用替换变量不存在以下问题:

    *取代“特遣部队。变量与“tf.contrib.eager.Variable”;
    *在一个tf中调用' tf.get_variable_scope().set_use_resource(True) '。在调用tf.get_variable()之前调用variable_scope。

    @compatibility(渴望)
    “特遣部队。变量'与立即执行不兼容。使用“tf.contrib.eager。变量',它与立即执行和图形构造都兼容。参见[TensorFlow Eager执行指南](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/eager/python/g3doc/guide.md#变量和优化器)
    有关变量在立即执行中如何工作的详细信息。
    @end_compatibility
    ”“”

      Args:
     initial_value: A `Tensor`, or Python object convertible to a `Tensor`,   which is the initial value for the Variable. The initial value must have  a shape specified unless `validate_shape` is set to False. Can also be a callable with no argument that returns the initial value when called. In  that case, `dtype` must be specified. (Note that initializer functions from init_ops.py must first be bound to a shape before being used here.)
          trainable: If `True`, the default, also adds the variable to the graph collection `GraphKeys.TRAINABLE_VARIABLES`. This collection is used as the default list of variables to use by the `Optimizer` classes. collections: List of graph collections keys. The new variable is added to these collections. Defaults to `[GraphKeys.GLOBAL_VARIABLES]`.
          validate_shape: If `False`, allows the variable to be initialized with a value of unknown shape. If `True`, the default, the shape of initial_value` must be known. caching_device: Optional device string describing where the Variable  should be cached for reading.  Defaults to the Variable's device.   If not `None`, caches on another device.  Typical use is to cache on the device where the Ops using the Variable reside, to deduplicate  copying through `Switch` and other conditional statements.
          name: Optional name for the variable. Defaults to `'Variable'` and gets uniquified automatically.
          variable_def: `VariableDef` protocol buffer. If not `None`, recreates the Variable object with its contents, referencing the variable's nodes
            in the graph, which must already exist. The graph is not changed. `variable_def` and the other arguments are mutually exclusive.
          dtype: If set, initial_value will be converted to the given type.  If `None`, either the datatype will be kept (if `initial_value` is  a Tensor), or `convert_to_tensor` will decide.
          expected_shape: A TensorShape. If set, initial_value is expected  to have this shape.
          import_scope: Optional `string`. Name scope to add to the   `Variable.` Only used when initializing from protocol buffer.
          constraint: An optional projection function to be applied to the variable after being updated by an `Optimizer` (e.g. used to implement norm constraints or value constraints for layer weights). The function must  take as input the unprojected Tensor representing the value of the   variable and return the Tensor for the projected value   (which must have the same shape). Constraints are not safe to  use when doing asynchronous distributed training.

        Raises:
          ValueError: If both `variable_def` and initial_value are specified.
          ValueError: If the initial value is not specified, or does not have a shape and `validate_shape` is `True`.
          RuntimeError: If eager execution is enabled.

        @compatibility(eager)
        `tf.Variable` is not compatible with eager execution.  Use
        `tfe.Variable` instead which is compatible with both eager execution
        and graph construction.  See [the TensorFlow Eager Execution
        guide](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/eager/python/g3doc/guide.md#variables-and-optimizers)
        for details on how variables work in eager execution.
        @end_compatibility

    参数:
    initial_value:一个“张量”,或者Python对象可转换成一个“张量”,它是变量的初始值。除非将“validate_shape”设置为False,否则必须指定初始值的形状。也可以是可调用的,没有参数,调用时返回初始值。在这种情况下,必须指定' dtype '。(注意,在这里使用初始化器函数之前,init_ops.py必须先绑定到一个形状上。)
    可训练的:如果“True”是默认值,那么也会将变量添加到图形集合“GraphKeys.TRAINABLE_VARIABLES”中。此集合用作“优化器”类使用的默认变量列表。集合:图形集合键的列表。新变量被添加到这些集合中。默认为“[GraphKeys.GLOBAL_VARIABLES]”。
    validate_shape:如果为“False”,则允许使用未知形状的值初始化变量。如果' True '是默认值,则必须知道initial_value '的形状。caching_device:可选的设备字符串,用于描述变量应该被缓存到什么地方以便读取。变量设备的默认值。如果不是“None”,则缓存到另一个设备上。典型的用法是在使用变量驻留的操作系统所在的设备上进行缓存,通过“Switch”和其他条件语句进行重复复制。
    name:变量的可选名称。默认值为“变量”,并自动uniquified。
    variable_def: ' VariableDef '协议缓冲区。如果不是“None”,则使用其内容重新创建变量对象,并引用变量的节点
    在图中,它必须已经存在。图形没有改变。' variable_def '和其他参数是互斥的。
    如果设置了,initial_value将转换为给定的类型。如果‘None’,那么数据类型将被保留(如果‘initial_value’是一个张量),或者‘convert_to_张量’将决定。
    expected_shape: TensorShape。如果设置了,initial_value将具有此形状。
    import_scope:可选“字符串”。将作用域命名为“变量”。仅在从协议缓冲区初始化时使用。
    约束:一个可选的投影函数,在被“优化器”更新后应用到变量上(例如,用于实现规范约束或层权重的值约束)。函数必须将表示变量值的未投影张量作为输入,并返回投影值的张量(其形状必须相同)。在进行异步分布式培训时使用约束是不安全的。

    提出了:
    ValueError:如果同时指定了' variable_def '和initial_value。
    ValueError:如果没有指定初始值,或者没有形状,并且‘validate_shape’为‘True’。
    RuntimeError:如果启用了立即执行。

    @compatibility(渴望)
    “特遣部队。变量'与立即执行不兼容。使用
    tfe。变量',而不是与两个立即执行兼容
    和图施工。参见[TensorFlow立即执行]
    指南](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/eager/python/g3doc/guide.md # variables-and-optimizers)
    有关变量在立即执行中如何工作的详细信息。
    @end_compatibility

    
    @tf_export("Variable")
    class Variable(checkpointable.CheckpointableBase):
      """See the @{$variables$Variables How To} for a high level overview.
    
      A variable maintains state in the graph across calls to `run()`. You add a
      variable to the graph by constructing an instance of the class `Variable`.
    
      The `Variable()` constructor requires an initial value for the variable,
      which can be a `Tensor` of any type and shape. The initial value defines the
      type and shape of the variable. After construction, the type and shape of
      the variable are fixed. The value can be changed using one of the assign
      methods.
    
      If you want to change the shape of a variable later you have to use an
      `assign` Op with `validate_shape=False`.
    
      Just like any `Tensor`, variables created with `Variable()` can be used as
      inputs for other Ops in the graph. Additionally, all the operators
      overloaded for the `Tensor` class are carried over to variables, so you can
      also add nodes to the graph by just doing arithmetic on variables.
    
      ```python
      import tensorflow as tf
    
      # Create a variable.
      w = tf.Variable(<initial-value>, name=<optional-name>)
    
      # Use the variable in the graph like any Tensor.
      y = tf.matmul(w, ...another variable or tensor...)
    
      # The overloaded operators are available too.
      z = tf.sigmoid(w + y)
    
      # Assign a new value to the variable with `assign()` or a related method.
      w.assign(w + 1.0)
      w.assign_add(1.0)
      ```
    
      When you launch the graph, variables have to be explicitly initialized before
      you can run Ops that use their value. You can initialize a variable by
      running its *initializer op*, restoring the variable from a save file, or
      simply running an `assign` Op that assigns a value to the variable. In fact,
      the variable *initializer op* is just an `assign` Op that assigns the
      variable's initial value to the variable itself.
    
      ```python
      # Launch the graph in a session.
      with tf.Session() as sess:
          # Run the variable initializer.
          sess.run(w.initializer)
          # ...you now can run ops that use the value of 'w'...
      ```
    
      The most common initialization pattern is to use the convenience function
      `global_variables_initializer()` to add an Op to the graph that initializes
      all the variables. You then run that Op after launching the graph.
    
      ```python
      # Add an Op to initialize global variables.
      init_op = tf.global_variables_initializer()
    
      # Launch the graph in a session.
      with tf.Session() as sess:
          # Run the Op that initializes global variables.
          sess.run(init_op)
          # ...you can now run any Op that uses variable values...
      ```
    
      If you need to create a variable with an initial value dependent on another
      variable, use the other variable's `initialized_value()`. This ensures that
      variables are initialized in the right order.
    
      All variables are automatically collected in the graph where they are
      created. By default, the constructor adds the new variable to the graph
      collection `GraphKeys.GLOBAL_VARIABLES`. The convenience function
      `global_variables()` returns the contents of that collection.
    
      When building a machine learning model it is often convenient to distinguish
      between variables holding the trainable model parameters and other variables
      such as a `global step` variable used to count training steps. To make this
      easier, the variable constructor supports a `trainable=<bool>` parameter. If
      `True`, the new variable is also added to the graph collection
      `GraphKeys.TRAINABLE_VARIABLES`. The convenience function
      `trainable_variables()` returns the contents of this collection. The
      various `Optimizer` classes use this collection as the default list of
      variables to optimize.
    
      WARNING: tf.Variable objects have a non-intuitive memory model. A Variable is
      represented internally as a mutable Tensor which can non-deterministically
      alias other Tensors in a graph. The set of operations which consume a Variable
      and can lead to aliasing is undetermined and can change across TensorFlow
      versions. Avoid writing code which relies on the value of a Variable either
      changing or not changing as other operations happen. For example, using
      Variable objects or simple functions thereof as predicates in a `tf.cond` is
      dangerous and error-prone:
    
      ```
      v = tf.Variable(True)
      tf.cond(v, lambda: v.assign(False), my_false_fn)  # Note: this is broken.
      ```
    
      Here replacing tf.Variable with tf.contrib.eager.Variable will fix any
      nondeterminism issues.
    
      To use the replacement for variables which does
      not have these issues:
    
      * Replace `tf.Variable` with `tf.contrib.eager.Variable`;
      * Call `tf.get_variable_scope().set_use_resource(True)` inside a
        `tf.variable_scope` before the `tf.get_variable()` call.
    
      @compatibility(eager)
      `tf.Variable` is not compatible with eager execution.  Use
      `tf.contrib.eager.Variable` instead which is compatible with both eager
      execution and graph construction.  See [the TensorFlow Eager Execution
      guide](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/eager/python/g3doc/guide.md#variables-and-optimizers)
      for details on how variables work in eager execution.
      @end_compatibility
      """
    
      def __init__(self,
                   initial_value=None,
                   trainable=True,
                   collections=None,
                   validate_shape=True,
                   caching_device=None,
                   name=None,
                   variable_def=None,
                   dtype=None,
                   expected_shape=None,
                   import_scope=None,
                   constraint=None):
        """Creates a new variable with value `initial_value`.
    
        The new variable is added to the graph collections listed in `collections`,
        which defaults to `[GraphKeys.GLOBAL_VARIABLES]`.
    
        If `trainable` is `True` the variable is also added to the graph collection
        `GraphKeys.TRAINABLE_VARIABLES`.
    
        This constructor creates both a `variable` Op and an `assign` Op to set the
        variable to its initial value.
    
        Args:
          initial_value: A `Tensor`, or Python object convertible to a `Tensor`,
            which is the initial value for the Variable. The initial value must have
            a shape specified unless `validate_shape` is set to False. Can also be a
            callable with no argument that returns the initial value when called. In
            that case, `dtype` must be specified. (Note that initializer functions
            from init_ops.py must first be bound to a shape before being used here.)
          trainable: If `True`, the default, also adds the variable to the graph
            collection `GraphKeys.TRAINABLE_VARIABLES`. This collection is used as
            the default list of variables to use by the `Optimizer` classes.
          collections: List of graph collections keys. The new variable is added to
            these collections. Defaults to `[GraphKeys.GLOBAL_VARIABLES]`.
          validate_shape: If `False`, allows the variable to be initialized with a
            value of unknown shape. If `True`, the default, the shape of
            `initial_value` must be known.
          caching_device: Optional device string describing where the Variable
            should be cached for reading.  Defaults to the Variable's device.
            If not `None`, caches on another device.  Typical use is to cache
            on the device where the Ops using the Variable reside, to deduplicate
            copying through `Switch` and other conditional statements.
          name: Optional name for the variable. Defaults to `'Variable'` and gets
            uniquified automatically.
          variable_def: `VariableDef` protocol buffer. If not `None`, recreates
            the Variable object with its contents, referencing the variable's nodes
            in the graph, which must already exist. The graph is not changed.
            `variable_def` and the other arguments are mutually exclusive.
          dtype: If set, initial_value will be converted to the given type.
            If `None`, either the datatype will be kept (if `initial_value` is
            a Tensor), or `convert_to_tensor` will decide.
          expected_shape: A TensorShape. If set, initial_value is expected
            to have this shape.
          import_scope: Optional `string`. Name scope to add to the
            `Variable.` Only used when initializing from protocol buffer.
          constraint: An optional projection function to be applied to the variable
            after being updated by an `Optimizer` (e.g. used to implement norm
            constraints or value constraints for layer weights). The function must
            take as input the unprojected Tensor representing the value of the
            variable and return the Tensor for the projected value
            (which must have the same shape). Constraints are not safe to
            use when doing asynchronous distributed training.
    
        Raises:
          ValueError: If both `variable_def` and initial_value are specified.
          ValueError: If the initial value is not specified, or does not have a
            shape and `validate_shape` is `True`.
          RuntimeError: If eager execution is enabled.
    
        @compatibility(eager)
        `tf.Variable` is not compatible with eager execution.  Use
        `tfe.Variable` instead which is compatible with both eager execution
        and graph construction.  See [the TensorFlow Eager Execution
        guide](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/eager/python/g3doc/guide.md#variables-and-optimizers)
        for details on how variables work in eager execution.
        @end_compatibility

     

     

    tensorflow.get_variable()函数

    # The argument list for get_variable must match arguments to get_local_variable.
    # So, if you are updating the arguments, also update arguments to
    # get_local_variable below.
    @tf_export("get_variable")
    def get_variable(name,
                     shape=None,
                     dtype=None,
                     initializer=None,
                     regularizer=None,
                     trainable=None,
                     collections=None,
                     caching_device=None,
                     partitioner=None,
                     validate_shape=True,
                     use_resource=None,
                     custom_getter=None,
                     constraint=None,
                     synchronization=VariableSynchronization.AUTO,
                     aggregation=VariableAggregation.NONE):
      return get_variable_scope().get_variable(
          _get_default_variable_store(),
          name,
          shape=shape,
          dtype=dtype,
          initializer=initializer,
          regularizer=regularizer,
          trainable=trainable,
          collections=collections,
          caching_device=caching_device,
          partitioner=partitioner,
          validate_shape=validate_shape,
          use_resource=use_resource,
          custom_getter=custom_getter,
          constraint=constraint,
          synchronization=synchronization,
          aggregation=aggregation)

     

     

     

     

     

     

     

     

     

     

     

     

     

    更多相关内容
  • cvx凸优化工具 自己写的demo 用于设置变量
  • 查看 tensorflow api manual 时,看到关于 variable.read_value() 的注解如图: 那么在 tensorflow 中,variable的值 与 variable.read_value()的值 到底有何区别? 实验代码 # coding=utf-8 import tensorflow as...
  • Pytorch之Variable的用法

    2020-12-23 06:08:34
    1.简介 torch.autograd.Variable是Autograd的核心类,它封装了Tensor,并整合了反向传播的相关实现 Variable和tensor的区别...比如grad,梯度variable.grad是d(y)/d(variable)保存的是变量y对variable变量的梯度值,如
  • 基于源码修改JDBC8驱动连接Mycat1.6报错 Unknown system variable 'query_cache_size' ,配置好mycat相应配置直接bin目录启动即可
  • 我们通过tf.Variable构造一个variable添加进图中,Variable()构造函数需要变量的初始值(是一个任意类型、任意形状的tensor),这个初始值指定variable的类型和形状。通过Variable()构造函数后,此variable的类型和...
  • Jira6不支持mysql5.7,需要此jar,具体就是报错 Unknown system variable 'storage_engine' 的解决
  • tf.Variable() vs tf.get_variable() tf.name_scope() vs tf.variable_scope() 1. 基础功能 1.1 tf.Variable() 是一个类,tensorflow.python.ops.variables.Variable,其构造函数原型为 def __init__( self, ...

    tf.Variable() vs tf.get_variable()
    tf.name_scope() vs tf.variable_scope()

    1. 基础功能

    1.1 tf.Variable()

    是一个类,tensorflow.python.ops.variables.Variable,其构造函数原型为

    def __init__(
        self,
        initial_value: Any = None,
        trainable: bool = True,
        collections: Any = None,
        validate_shape: bool = True,
        caching_device: Any = None,
        name: Any = None,
        variable_def: Any = None,
        dtype: Any = None,
        expected_shape: Any = None,
        import_scope: Any = None,
        constraint: Any = None
    ) -> Any
    

    创建一个初始值为 initial_value 的变量。
    Params:(仅关注几个常用的)

    • initial_value – 变量初始值,是一个 Tensor 或者 可转换为 Tensor 的 python 对象;
    • trainable – 可否训练;
    • collections – 一个 List[graph collections keys],变量将会加入这些 collections,列表默认为 [GraphKeys.GLOBAL_VARIABLES]
    • nameOptional(可选) 变量的名字,默认为 'Variable',如果已有同名变量,则新变量 name 为 'Variable_num',其中 num 递增;
    • constraint – 一个可选的投影函数,在被 Optimizer(例如,用于实现层权重的范数约束或值约束)更新之后应用于该变量。该函数必须将代表变量值的未投影张量作为输入,并返回投影值的张量(它必须具有相同的形状)。进行异步分布式训练时,约束条件不安全。

    例子:

    a = tf.Variable(
        initial_value=[1.0, 2.0],
        trainable=True,
        name='va',
        dtype=tf.float32
    )
    print(a)  # <tf.Variable 'va:0' shape=(2,) dtype=float32_ref>
    

    1.2 tf.get_variable()

    函数,原型为:

    def get_variable(
        name: Any,
        shape: Any = None,
        dtype: Any = None,
        initializer: Any = None,
        regularizer: Any = None,
        trainable: Any = None,
        collections: Any = None,
        caching_device: Any = None,
        partitioner: Any = None,
        validate_shape: bool = True,
        use_resource: Any = None,
        custom_getter: Any = None,
        constraint: Any = None,
        synchronization: VariableSynchronization = VariableSynchronization.AUTO,
        aggregation: VariableAggregation = VariableAggregation.NONE
    ) -> Any
    

    通过变量名来获取或创建一个变量。
    Params:(仅关注几个常用的)

    • name必填 变量的名字,如果已有同名变量,则报错(variable_scope(reuse=True) 除外);
    • shape – 变量形状;
    • dtype – 数据类型;本来猜测 initializer 中的 dtype 可能只决定随机值,最终类型由此参数决定,initializer 中的 dtype 似乎并不起作用
    • initializer – 初始化器,如 tf.tf.random_uniform_initializer(minval=1, maxval=10)
    • regularizer – 可能是传入一个正则化器,如 `tf.contrib.layers.l2_regularizer(0.1)';
    • synchronization – 以后再说(应该挺有用的);

    例子:

    a = tf.get_variable(
    	name='v',
    	shape=[1, 2],
    	dtype=tf.float64,
    	initializer=tf.random_uniform_initializer(
            minval=1,
            maxval=10,
            dtype=tf.int32
        )  # initializer 里的 dtype 没什么卵用
    )
    print(a)  # `<tf.Variable 'v:0' shape=(1, 2) dtype=float64_ref>`
    with tf.Session() as sess:
    	sess.run(tf.global_variables_initializer())
    	print(sess.run(a))  # [[3.98646081 3.30710938]]
    

    1.3 tf.name_scope()

    def __init__(
        self,
        name: Any,
        default_name: Any = None,
        values: Any = None
    ) -> None
    

    初始化上下文管理器。
    Params:

    • name – 传递给 OP 函数的 name 参数;
    • default_name – 如果 name 参数为 None 则使用默认的名称;
    • values – 要传递给 op 函数的张量参数列表(不清楚);

    这篇博文 讲得不错。

    例子:

    with tf.name_scope(name='ns'):
    	a = tf.Variable(0.0)
    	b = tf.Variable(0.0)
    	c = a + b
    print(a)  # <tf.Variable 'ns/Variable:0' shape=() dtype=float32_ref>  
    print(b)  # <tf.Variable 'ns/Variable_1:0' shape=() dtype=float32_ref>
    print(c)  # Tensor("ns/add:0", shape=(), dtype=float32)
    

    1.4 tf.variable_scope()

    def __init__(
        self,
        name_or_scope: Any,
        default_name: Any = None,
        values: Any = None,
        initializer: Any = None,
        regularizer: Any = None,
        caching_device: Any = None,
        partitioner: Any = None,
        custom_getter: Any = None,
        reuse: Any = None,
        dtype: Any = None,
        use_resource: Any = None,
        constraint: Any = None,
        auxiliary_name_scope: bool = True
    ) -> Any
    

    初始化上下文管理器。
    Params:

    • name_or_scopestring or VariableScope: 要打开的 scope;
    • default_name – 如果参数 name_or_scopeNone, 则启用默认名;
    • values – 要传递给 op 函数的张量参数列表(不清楚);
    • reuseTrue, None, or tf.AUTO_REUSE
      • 如果为 True,该 scope 及其 sub-scope(reuse=None)开启 reuse 模式,get_variable(name='v') 将获取已有的 name == v 的变量;
      • 如果为 tf.AUTO_REUSEget_variable(name='v') 创建新变量 if v 不存在 esle 获取已有变量 v
      • 如果为 None,则继承其 parent-scope 的 reuse flag;

    Returns:
    A scope that can be captured and reused. 可以作为参数传递给 tf.variable_scope()

    例子

    with tf.variable_scope('vs') as scope:
    	a = tf.get_variable(name='a', shape=[2, 3], initializer=tf.random_uniform_initializer(0.0, 1.0))
    print(a)  # <tf.Variable 'vs/a:0' shape=(2, 3) dtype=float32_ref>
    with tf.variable_scope(scope, reuse=True):  # 需要指明 reuse
    	b = tf.get_variable(name='a')
    print(a is b)  # True
    

    2. 四者之间的关系

    2.1 tf.Variable() 和 tf.get_variable()

    # 两个定义是等价的
    v = tf.Variable(tf.constant(1.0, shape=[1]), name='v')
    v = tf.get_variable(name='v', shape=[1], initializer=tf.constant_initializer(1.0))
    print(v)  # <tf.Variable 'v:0' shape=(1,) dtype=float32_ref>
    
    • 最大的区别在于tf.Variable()name 是可选的;而 tf.get_variable()name 是必填的;
    • 当已存在同名变量时:tf.Variable() 会在变量名后加 _n 其中 n 递增;而 tf.get_variable() 会报错(最后一个例子特殊,如果已存在 tf.Variable() 创建的同名变量,则遵循 +1 规则);

    例子

    a = tf.Variable(tf.constant(1.0, shape=[1]), name='v')
    b = tf.Variable(tf.constant(1.0, shape=[1]), name='v')
    print(a)  # <tf.Variable 'v:0' shape=(1,) dtype=float32_ref>
    print(b)  # <tf.Variable 'v_1:0' shape=(1,) dtype=float32_ref>
    
    a = tf.get_variable(name='v', shape=[1], initializer=tf.constant_initializer(1.0))
    b = tf.get_variable(name='v', shape=[1], initializer=tf.constant_initializer(1.0))
    # ValueError: Variable v already exists, disallowed.
    
    a = tf.Variable(1.0, name='v')
    b = tf.get_variable(name='v', shape=[1], initializer=tf.constant_initializer(1.0))
    c = tf.Variable(1.0, name='v1')
    d = tf.get_variable(name='v1', shape=[1], initializer=tf.constant_initializer(1.0))
    print(a)  # <tf.Variable 'v:0' shape=() dtype=float32_ref>
    print(b)  # <tf.Variable 'v_1:0' shape=(1,) dtype=float32_ref>
    print(c)  # <tf.Variable 'v1:0' shape=() dtype=float32_ref>
    print(d)  # <tf.Variable 'v1_1:0' shape=(1,) dtype=float32_ref>
    

    只有在 tf.get_variable() & tf.get_variable() 的情况下重名才报错,其他情况遵循 +1 规则。

    2.2 tf.Variable() 和 tf.name_scope()

    with tf.name_scope('ns'):
    	a = tf.Variable(tf.constant(1.0, shape=[1]), name='v')
    b = tf.Variable(tf.constant(1.0, shape=[1]), name='v')
    print(a)  # <tf.Variable 'ns/v:0' shape=(1,) dtype=float32_ref>
    print(b)  # <tf.Variable 'v:0' shape=(1,) dtype=float32_ref>
    

    2.3 tf.Variable() 和 tf.variable_scope()

    with tf.variable_scope('vs'):
    	a = tf.Variable(tf.constant(1.0, shape=[1]), name='v')
    b = tf.Variable(tf.constant(1.0, shape=[1]), name='v')
    print(a)  # <tf.Variable 'vs/v:0' shape=(1,) dtype=float32_ref>
    print(b)  # <tf.Variable 'v:0' shape=(1,) dtype=float32_ref>
    

    此时,tf.variable_scope() == tf.name_scope()

    2.4 tf.get_variable() 和 tf.name_scope()

    with tf.name_scope('ns'):
    	a = tf.get_variable(name='v', shape=[1], initializer=tf.constant_initializer(1.0))
    # b = tf.get_variable(name='v', shape=[1], initializer=tf.constant_initializer(1.0))
    print(a)  # <tf.Variable 'v:0' shape=(1,) dtype=float32_ref>
    # print(b)  # Err
    

    tf.name_scope()tf.get_variable() 不起作用。

    2.5 tf.get_variable() 和 tf.variable_scope()

    with tf.variable_scope(
    		'vs',
    		initializer=tf.constant_initializer(1.0),
    		regularizer=tf.contrib.layers.l2_regularizer(0.01)
    ):
    	a = tf.Variable(0.0, name='v')
    	b = tf.get_variable(name='v', shape=[1])
    
    print(a)  # <tf.Variable 'vs/v:0' shape=() dtype=float32_ref>
    print(b)  # <tf.Variable 'vs/v_1:0' shape=(1,) dtype=float32_ref>
    
    with tf.Session() as sess:
    	sess.run(tf.global_variables_initializer())
    	print(sess.run(a))  # 0.0
    	print(sess.run(b))  # [1.]
    
    • tf.get_variable()tf.variable_scope() 是更强大的结合,可以设置 scope 下变量的默认属性。但对 tf.Variable() 来说,没有这些功能。
      图1
    图 1 正则化器 regularizer 的输入只有 v_1

    图 1 说明 a 的初始化是自己的 initial_value,regularizer 也未作用到 a 上。而 b 得到了这两个默认属性。

    • reuse 功能给了编程更大的灵活性
    def forward(input_tensor):
    	with tf.variable_scope('vs', reuse=tf.AUTO_REUSE, initializer=tf.constant_initializer(3.0)):
    		w = tf.get_variable(name='w', shape=[1])
    	return input_tensor * w
    
    
    a = tf.get_variable(name='input', shape=[1], initializer=tf.constant_initializer(2.0))
    b = forward(a)
    c = forward(b)
    
    with tf.Session() as sess:
    	sess.run(tf.global_variables_initializer())
    	print(sess.run(a))  # [2.]
    	print(sess.run(b))  # [6.]
    	print(sess.run(c))  # [18.]
    
    在这里插入图片描述在这里插入图片描述
    get_variavble()Variavble()
    图2 tf.variable_scope() reuse 的作用

    可以看到,wtf.get_variable() 时得到了 reuse,非常方便。而 wtf.Variable() 则不 reuse。

    结论

    tf.get_variable()tf.variable_scope() 一起可以发挥强大灵活的作用。而 tf.Variable() 只把 tf.variable_scope() 当作简化的 tf.name_scope() 使用。

    2.6 tf.name_scope() 和 tf.variable_scope()

    • 2.22.3 可以看到,tf.name_scope()tf.variable_scope() 下,tf.Variable() 的名字都会加上前缀 scope
      2.4 说明tf.name_scope()tf.get_variable() 不起作用;
    • tf.name_scope() 更像是 tf.variable_scope() 的阉割版,从形参上看,前者较为简单,被后者完全覆盖;tf.variable_scope() 的形参中存在大量针对变量的参数(几乎是 name_scope + get_variable 参数的合集),如 initializerregularizer 等,可以指定 scope 内创建的变量的默认属性;
    • 重名+1:与 tf.Variable() 一样,空间重名 +1,(tf.variable_scope() 不使用 tf.get_variable() 时)
    with tf.name_scope('s'):
    	a = tf.Variable(0.0, name='v')
    	c = tf.get_variable(name='v', shape=[1], initializer=tf.constant_initializer(1.0))
    with tf.name_scope('s'):
    	b = tf.Variable(0.0, name='v')
    print(a)  # <tf.Variable 's/v:0' shape=() dtype=float32_ref>
    print(b)  # <tf.Variable 's_1/v:0' shape=() dtype=float32_ref>
    print(c)  # <tf.Variable 'v:0' shape=(1,) dtype=float32_ref>
    
    with tf.variable_scope('s'):
    	a = tf.Variable(0.0, name='v')
    	c = tf.get_variable(name='v', shape=[1], initializer=tf.constant_initializer(1.0))
    with tf.variable_scope('s'):
    	b = tf.Variable(0.0, name='v')
    	# d = tf.get_variable(name='v', shape=[1], initializer=tf.constant_initializer(1.0))
    print(a)  # <tf.Variable 's_2/v:0' shape=() dtype=float32_ref>
    print(b)  # <tf.Variable 's_3/v:0' shape=() dtype=float32_ref>
    print(c)  # <tf.Variable 's/v_1:0' shape=(1,) dtype=float32_ref>,变量重名 +1
    # print(d)  # Err,对于 b = Variable() 而言,s+1,对于 d = tf.get_variable() 而言,s 内变量重名报错
    

    Note1 可以看到,当用于 tf.Variable() 时,tf.variable_scope()tf.name_scope() 没什么区别,重名的话都会使空间名自动 +1,即使混用,也一样;

    Note2 发现一个很有意思的现象:

    在第三个 scope 中,a 和 c 竟然不是同一个 scope,{a:s_2, c:s},可见,当 tf.variable_scope() 用于 tf.get_variable(),空间会另起炉灶(这个新炉灶不会+1)。
    且新炉灶的第一个 tf.get_variable() 前有 tf.Variable() 创建的重名变量的话,则命名重复 +1。
    所以,tf.variable_scope() 下, tf.get_variable()tf.Variable() 混用时,实际上是各自独立的

    总结

    当使用 tf.Variable() 时,tf.name_scope()tf.variable_scope() 没什么区别;
    当使用 tf.get_variable() 时,tf.name_scope() 不起作用。
    tf.variable_scope() 下, tf.get_variable()tf.Variable() 混用时,实际上是各自独立的

    展开全文
  • 1 Variable类型与自动微分模块概述 1.1 Variable类型 Variable是由Autograd模块对张量进行进一步封装实现的,具有自动求导的功能 1.2 Autograd模块(自动求导模块) Autograd模块:在神经网络的反向传播中,...

    1 Variable类型与自动微分模块概述

    1.1 Variable类型

    Variable是由Autograd模块对张量进行进一步封装实现的,具有自动求导的功能

    1.2 Autograd模块(自动求导模块)

    Autograd模块:在神经网络的反向传播中,基于正向计算的结果进行微分计算,从而实现对于网络权重的更新与迭代,提供了张量的自动求微分功能,可以通过代码来实现对反向过程的控制,使得权重参数朝着目标结果进行更新与发展。

    2 Variable类型与自动微分模块实战

    2.1 Variable类型对象与张量对象之间的转化

    2.1.1 代码实现

    import torch
    from torch.autograd import Variable
    
    a = torch.FloatTensor([4]) #创建张量
    print(Variable(a)) # 将张量转化为Variable对象
    # 输出 tensor([4.])
    print(Variable(a,requires_grad=True)) # requires_grad允许自动求导
    # 输出 tensor([4.], requires_grad=True)
    print(a.data) #将Variable对象转化为张量
    # 输出 tensor([4.])

    2.1.2 注意

    import torch
    from torch.autograd import Variable
    
    ### 使用requires_grad时,要求张量的值必须为浮点型
    x = torch.tensor([1],requires_grad=True) #报错 
    
    x = torch.tensor([1.],requires_grad=True) #正确写法
    

    2.2 torch.no_grad()

    2.2.1 概述

    torch.no_grad():使Variable类型变量的requires_grad失效

    torch.enable_grad():使Variable类型变量的requires_grad有效

    2.2.2 使用torch.no_grad()配合with语句限制requires_grad的作用域

    import torch
    from torch.autograd import Variable
    
    x = torch.ones(2,2,requires_grad=True) # 定义一个需要梯度计算的Variable类型对象
    with torch.no_grad():
        y = x * 2
    print(y.requires_grad) # 输出 False
    

    2.2.3 使用装饰器@实现

    import torch
    from torch.autograd import Variable
    
    ### 在神经网络中将网络模型进行封装,使用装饰器方便实现开发的便捷性
    
    x = torch.ones(2,2,requires_grad=True) # 定义一个需要梯度计算的Variable类型对象
    @torch.no_grad()
    def doubler(x):
        return x * 2
    z = doubler(x)
    print(z.requires_grad) # 输出 False

    2.3 函数enable_grad()与no_grad()的嵌套使用

    2.3.1 enable_grad()配合with语句限制requires_grad的作用域

    import torch
    x = torch.ones(2,2,requires_grad=True) # 定义一个需要梯度计算的Variable类型对象
    with torch.no_grad():
        with torch.enable_grad():
            y = x * 2
            print(y.requires_grad) # True
        print(y.requires_grad) # True
    print(y.requires_grad) # True

    2.3.2 使用enable_grad装饰器

    import torch
    x = torch.ones(2,2,requires_grad=True) # 定义一个需要梯度计算的Variable类型对象
    @torch.enable_grad()
    def doubler(x): #封装到函数中
        return x * 2
    with torch.no_grad(): #使得计算梯度失效
        z = doubler(x)
    print(z.requires_grad) #True

    2.3.3 作用在没有requires_grad的Variable类型变量上将会失效,不能使其重新获得计算梯度的属性

    import torch
    x = torch.ones(2,2) # 定义一个不需要梯度计算的Variable类型对象
    with torch.enable_grad():
        y = x * 2
    print(y.requires_grad) # False

    2.3 set_grad_enabled()实现统一管理梯度计算

    import torch
    x = torch.ones(2,2,requires_grad=True) # 定义一个需要梯度计算的Variable类型对象
    torch.set_grad_enabled(False) # 统一关闭梯度计算
    y = x * 2
    print(y.requires_grad) # False
    torch.set_grad_enabled(True) # 统一开启梯度计算
    y = x * 2
    print(y.requires_grad) # True

    2.4 Variable类型对象的grad_fn属性

    2.4.1 grad_fn属性概述

    Variable类型对象在经过前向传播后,将会增加一个grad_fn属性,该属性随着backward()方法进行自动的梯度计算。没有经过计算的Variable类型对象是没有这个属性的,在requires_grad=False的情况下,无论如何计算他都不会有grad_fn属性。

    2.4.2 grad_fn属性代码实现

    import torch
    from torch.autograd import Variable
    
    x = Variable(torch.ones(2,2),requires_grad=True)
    print(x)
    # 输出 tensor([[1., 1.],[1., 1.]], requires_grad=True)
    print(x.grad_fn)
    # 输出 None
    
    m = x + 2 # 经过正向计算,获得grad_fn属性
    print(m.grad_fn)
    # 输出 <AddBackward0 object at 0x0000024E1AA14D00>
    print(m.grad_fn(x)) #对x变量进行求梯度计算
    # 输出 (tensor([[1., 1.],[1., 1.]], requires_grad=True), None)
    
    x2 = torch.ones(2,2) # 创建一个不需要梯度计算的张量
    m = x2 + 2
    print(m.grad_fn)
    # 输出 None

    2.5 Variable类型对象的is_leaf函数

    2.5.1 is_leaf()概述

    1、定义Variable类型对象时,若将requires_grad设为True,则将该Variable类型对象称为种子节点,其 is_leaf的属性为True。

    2、若Variable类型对象不是通过自定义生成的,而是通过其他张量计算所得时,则不是叶子节点,则该该Variable类型对象称为叶子节点,其 is_leaf的属性为False。

    3、Pytorch会记录每个张量的由来,由此来在内存中行程树状结构实现反向链式计算,叶子节点主要在求导过程为递归循环提供信号指示,当反向链式计算遇到叶子节点则终止递归循环。

    2.5.2 is_leaf()代码

    import torch
    from torch.autograd import Variable
    
    x = Variable(torch.ones(2,2),requires_grad=True)
    print(x.is_leaf) # True
    m = x + 2
    print(m.is_leaf) # False

    2.6 backward()实现自动求导

    2.6.1 backward()概述

    backward():必须在当前变量内容是标量的情况下使用,否则会报错。

    2.6.3 自动求导的作用

    从复杂的神经网络中,自动将每一层中的每个参数的梯度计算出来,实现训练过程中的反向传播。

    2.6.2 代码

    import torch
    from torch.autograd import Variable
    
    ### y = (x + 2)/4
    x = Variable(torch.ones(2,2),requires_grad=True)
    m = x + 2
    f = m.mean() #得到一个标量
    f.backward() # 自动求导
    print(f) #输出 tensor(3., grad_fn=<MeanBackward0>)
    print(x.grad) # 输出 tensor([[0.2500, 0.2500],[0.2500, 0.2500]])

    2.7 detach()将Variable类型对象分离成种子节点

    2.7.1 将需要求梯度的Variable类型对象转化为Numpy对象

    import torch
    from torch.autograd import Variable
    
    ### 如果被分离的Variable对象的volatile属性为True,那么被分离出的volatile属性也为True
    ### 被返回的Variable对象和被分离的Variable对象指向同一个张量,并且永远不会需要梯度
    x = Variable(torch.ones(2,2),requires_grad=True)
    # x.numpy() # 报错Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.
    x1 = x.detach().numpy()
    print(x1)# 输出 [[1.,1.],[1.,1.]]

    2.7.2 实现对网络中的部分参数求梯度

     2.8 volatile属性

    早期代码中可以通过设置Variable类型对象的volatile属性为True的方法来实现停止梯度更新。

    展开全文
  • Reusing模式会被子vs继承 ...print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse) with tf.variable_scope('ss'): # ss是默认vs的子vs,故虽然没有使用reuse=True,wit...

    一、VariableScope的reuse模式的设置

    1.1节

    1、tf.get_variable_scope()可以获得当前的变量域,还可以通过其.name.reuse来查看其名称和当前是否为reuse模式。

    2、变量域有name,变量也有name。默认变量作用域的name为空白字符串。

    3、在变量域内命名的变量的name全称为:“变量域的name+变量定义时传入的name”(就像一个文件有名字作为标识符,但是在前面加上绝对路径就是它在整个文件系统中的全局标识符)。

    这三点贯穿本文,如果不太清楚,可以直接看后面的多个例子,会不断地体现在代码中。

    1.2节

    with tf.variable_scope()可以打开一个变量域,有两个关键参数。name_or_scope参数可以是字符串或tf.VariableScope对象,reuse参数为布尔值,传入True表示设置该变量域为reuse模式。

    还有一种方法可以将变量域设置为reuse模式,即使用VariableScope对象的reuse_variables()方法,例如tf.get_variable_scope().reuse_variables()可以将当前变量域设置为reuse模式。

    with tf.variable_scope('vs1'):
        tf.get_variable_scope().reuse_variables()
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    
    with tf.variable_scope('vs2',reuse=True):
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    
    '''
    "vs1" True
    "vs2" True
    '''
    

    1.3节

    对某变量域设置reuse模式,则reuse模式会被变量域的子域继承

    # 注意,默认变量域的名称为空白字符串
    tf.get_variable_scope().reuse_variables() # 将默认变量域设置为reuse模式
    print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse) # 为了显示空白字符串,在名称两边加上双引号
    
    with tf.variable_scope('vs'): 
    	# vs是默认变量域的子域,故虽然没有明确设置vs的模式,但其也更改成了reuse模式
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    
    '''
    
    输出为:
    "" True
    "vs" True
    
    '''
    

    1.4节

    每次在with块中设置变量域的模式,退出with块就会失效(恢复回原来的模式)。

    with tf.variable_scope('vs1'):
        tf.get_variable_scope().reuse_variables()
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    
    with tf.variable_scope('vs1'):
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    
    
    '''
    
    输出为:
    "vs1" True
    "vs1" False
    
    '''
    

    1.5节

    可以使用一个变量域(tf.VariableScope对象)的引用来打开它,这样可以不用准确的记住其name的字符串。下面的例子来自tensorflow官网

    with tf.variable_scope("model") as scope:
      output1 = my_image_filter(input1)
    with tf.variable_scope(scope, reuse=True):
      output2 = my_image_filter(input2)
    

    tf.VariableScope对象作为with tf.variable_scope( name_or_scope ):的参数时,该with语句块的模式是该scope对应的模式。(下面的代码同时也展现了前面所说的“继承”和“失效”的现象。)

    with tf.variable_scope('vs1'):
        tf.get_variable_scope().reuse_variables()
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
        with tf.variable_scope('vs2') as scope: # vs2(全称是vs1/vs2)将会继承vs1的reuse模式
            print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    
    # 重新用with打开vs1和vs2,他们的reuse模式不受之前with块中的设置的影响
    with tf.variable_scope('vs1'): 
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
        with tf.variable_scope('vs2'):
            print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    
    # tf.variable_scope也可以传入tf.VariableScope类型的变量,此处的scope是第4行with语句中定义的
    with tf.variable_scope(scope): 
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    # 第二个with中, vs1/vs2的reuse模式为False,即前面所说的退出with块之后reuse模式的设置“失效”
    # 但是第三个with中,vs1/vs2的reuse却为True,这是因为当`name_or_scope`参数是tf.VariableScope对象时,
    # 其打开的变量域的reuse模式由这个参数scope决定。
    # 此处的`scope`在第4行定义,“继承”vs1的reuse,且之后没有改变,所以第三个with打开的就是reuse=True
    
    '''
    
    输出为:
    "vs1" True
    "vs1/vs2" True
    "vs1" False
    "vs1/vs2" False
    "vs1/vs2" True
    
    '''
    

    二、reuse模式对tf.Variable() 的影响

    tf.Variable() 只有新建变量的功能,一个变量域是否为reuse模式不影响tf.Variable()的作用。如果该变量域中已经有同名的变量,则新建的变量会被重命名,加上数字后缀以区分。

    print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    v1=tf.Variable(tf.constant(1),name='v')                                         
    v2=tf.Variable(tf.constant(1),name='v')
    
    tf.get_variable_scope().reuse_variables()
    print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    
    v3=tf.Variable(tf.constant(1),name='v') # 在reuse模式下使用tf.Variable(),仍然会新建,故v3名称为v_2
    
    print(v1.name)
    print(v2.name)
    print(v3.name)
    
    '''
    
    输出为:
    "" False
    "" True
    v:0
    v_1:0
    v_2:0
    
    '''
    

    三、reuse模式对tf.get_variable()的影响

    reuse模式会对tf.get_variable()的实际效果有决定作用。

    3.1节

    在non-reuse模式下,tf.get_variable()作用为新建变量(设为v)。若变量域内已经有同名变量(设为w),则分两种情况:

    1. w是之前通过tf.Variable()创建的,则v将被重命名,即加上数字后缀。
    2. w是之前通过tf.get_variable()创建的,则不允许新建同名变量v
    with tf.variable_scope('vs'):
    	# 打印当前scope的名称和是否为reuse模式
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse) 
        v1=tf.Variable(tf.constant(1),name='v')
        print(v1.name) # 前缀为scope的名称加一个反斜线,即“vs/”,故全称为“vs/v:0”,“冒号0”的解释见后文。
        v2=tf.get_variable('v',shape=()) 
        print(v2.name) # 已经有名为v的变量,故v2的name会在v后面加上数字后缀(从1开始)
        v3=tf.get_variable('v',shape=()) # 已经有名为v且由tf.get_variable创建的变量,故v3的创建抛出异常
        print(v3.name)
    

    输出为:(题外话,“:0” 指的是该变量是创建它的operation的第一个输出,见 这个链接

    "vs" False
    vs/v:0
    vs/v_1:0
    --------------------------------------------------------------------------
    
    ValueError                               Traceback (most recent call last)
    
    <ipython-input-2-e0b97b39994d> in <module>()
          5     v2=tf.get_variable('v',shape=())
          6     print(v2.name)
    ----> 7     v3=tf.get_variable('v',shape=())
          8     print(v3.name)
          9
          
    <省略部分输出>
    ValueError: Variable vs/v already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope?
    

    3.2节

    在reuse模式下,tf.get_variable()作用为重用(reuse)变量。注意只能重用之前在本变量域创建的、且使用tf.get_variable()创建的变量,即不能在本变量域中重用其他变量域中创建的变量,也不能重用那些使用tf.Variable()创建的变量。

    1.重用变量

    with tf.variable_scope('vs'):
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
        v=tf.get_variable('v',shape=())
        print(v.name)
    
    with tf.variable_scope('vs',reuse=True):
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
        reused_v=tf.get_variable('v',shape=()) # reused_v就是之前的v,他们是共享内存的变量
        print(reused_v.name)
    
    '''
    
    输出为:
    "vs" False
    vs/v:0
    "vs" True
    vs/v:0
    
    '''
    

    2.不能重用其他变量域中命名的变量(相当于你在A文件夹新建了v.txt,但是不能到B文件夹里面找v.txt)。

    # 在vs变量域新建v,尝试到vs1中重用变量
    with tf.variable_scope('vs'):
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
        v=tf.get_variable('v',shape=())
        # v=tf.Variable(tf.constant(1),name='v')
        print(v.name)
    
    with tf.variable_scope('vs1',reuse=True):
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
        # 下一行会报错,因为vs1这个变量域并没有用get_variable()创建过名为v的变量
        reused_v=tf.get_variable('v',shape=()) 
        print(reused_v.name)
    
    '''
    报错:
    ValueError: Variable vs1/v does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?
    '''
    

    3.只能重用那些使用tf.get_variable()创建的变量,而不能重用那些使用tf.Variable()创建的变量。

    with tf.variable_scope('vs'):
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
        v=tf.Variable(tf.constant(1),name='v')
        print(v.name)
    
    with tf.variable_scope('vs',reuse=True):
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
        reused_v=tf.get_variable('v',shape=())
        print(reused_v.name)
    

    输出为:

    "vs" False
    vs/v:0
    "vs" True
    --------------------------------------------------------------------------
    
    ValueError                               Traceback (most recent call last)
    
    <ipython-input-2-63ddfa598083> in <module>()
          6 with tf.variable_scope('vs',reuse=True):
          7     print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
    ----> 8     reused_v=tf.get_variable('v',shape=())
          9     print(reused_v.name)
         10
        
    <省略部分输出>
    
    ValueError: Variable vs/v does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?
    

    附加1:tf.name_scope()与tf.variable_scope()的区别

    tf.name_scope()tf.variable_scope()的功能很像,这里也顺便探讨一下他们的区别,以助于加深对两个方法的理解。

    本小节参考自 这篇知乎文章

    with tf.name_scope('ns'):
        print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
        v1=tf.get_variable('v',shape=())
        v2=tf.Variable(tf.constant(1),name='v')
        print(v1.name)
        print(v2.name)
        with tf.variable_scope('vs'):
            print('"'+tf.get_variable_scope().name+'"', tf.get_variable_scope().reuse)
            v3=tf.get_variable('v',shape=())
            v4=tf.Variable(tf.constant(1),name='v')
            print(v3.name)
            print(v4.name)
    
    with tf.variable_scope('vs'):
        with tf.name_scope('ns'):
            v5=tf.Variable(tf.constant(1),name='v')
            print(v5.name)
            v6=tf.get_variable('v',shape=()) # 这里将会抛出异常
            print(v6.name)
    

    输出如下,解释见对应的注释:

    "" False   # 1.with打开NameScope并不影响所在的VariableScope
    v:0        # 2.NameScope对于以tf.get_variable()新建的变量的命名不会有影响
    ns/v:0     # 3.对于以tf.Variable()方式新建的变量的命名,会加上NameScope的名字作为前缀
    "vs" False # 4.印证了第1点
    vs/v:0     # 5.印证了第2点
    ns/vs/v:0  # 6.对于被多层NameScope和VariableScope包围的、以tf.Variable()新建的变量,其命名以嵌套顺序来确定前缀
    vs/ns/v:0  # 7.印证了第6点
    # 下面的异常是由v6=tf.get_variable('v',shape=())导致的
    # 因为tf.get_variable()获得的变量的命名不受NameScope影响,所以这里其实对应了3.1节第2点的情况
    # 即在相同的VariableScope中使用tf.get_variable()定义了重名的变量
    Traceback (most recent call last):
      File "scope.py", line 79, in <module>
        v6=tf.get_variable('v',shape=())
      File "C:\Users\pyxies\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1203, in get_variable
        constraint=constraint)
      File "C:\Users\pyxies\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1092, in get_variable
        constraint=constraint)
      File "C:\Users\pyxies\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 425, in get_variable
        constraint=constraint)
      File "C:\Users\pyxies\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 394, in _true_getter
        use_resource=use_resource, constraint=constraint)
      File "C:\Users\pyxies\Anaconda3\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 742, in _get_single_variable
        name, "".join(traceback.format_list(tb))))
    ValueError: Variable vs/v already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope?
    

    附加2:单机多GPU下的变量共享/复用

    见 https://blog.csdn.net/xpy870663266/article/details/99330338

    展开全文
  • UEFI下Variable的实现

    千次阅读 2019-02-01 16:20:14
    这部分功能在标准的uefi中都是使用Variable 这套机制实现的.熟悉uefi的人都知道Variable这套机制在uefi中使用非常的广泛.现在我们来说说Variable是如何实现的. 首先:Variable在uefi的公共源码上是有相应的代码...
  • Pytorch中的Variable

    千次阅读 2022-04-10 17:06:10
    Pytorch中的Variable pytorch两个基本对象:Tensor(张量)和Variable(变量) 其中,tensor不能反向传播,variable可以反向传播。 Varibale包含三个属性: ●data: 存储了Tensor, 是本体的数据 ●grad: 保存了data的...
  • 关于variable的理解

    千次阅读 2021-11-20 11:18:32
    引用莫烦大大的话来说,tensor是一个鸡蛋,而variable相当于一个篮子,把tensor装起来 其中variable有三个参数: data:存储了Tensor,是本体的数据 grad:保存了data的梯度,本事是个Variable而非Tensor,与data...
  • PyTorch中的Variable变量详解

    千次阅读 2020-12-03 04:35:24
    一、了解Variable顾名思义,Variable就是 变量 的意思。实质上也就是可以变化的量,区别于int变量,它是一种可以变化的变量,这正好就符合了反向传播,参数更新的属性。具体来说,在pytorch中的Variable就是一个存放...
  • tf.Variable()函数

    千次阅读 2021-01-28 15:14:02
    tf.Variable(initializer,name),参数initializer是初始化参数,name是可自定义的变量名称,用法如下: import tensorflow as tf v1=tf.Variable(tf.random_normal(shape=[4,3],mean=0,stddev=1),name='v1') v2=tf...
  • 浅谈Pytorch中的Variable的使用方法

    千次阅读 多人点赞 2020-06-19 17:51:10
    浅谈Pytorch中的Variable的使用方法Variable的基本概念Variable的自动求导PyTorch中Variable的使用方法获取Variable里面的数据完整教学实例 Variable的基本概念 autograd.Variable 是包的核心类。它包装了张量,并且...
  • Tensorflow的Variable和get_variable

    千次阅读 2018-07-11 21:00:47
    tf创建变量主要有两种方式:tf.Variable()(V大写)、tf.get_variable()。 1、tf.Variable() (1)每次调用得到的都是不同的变量,即使使用了相同的变量名,在底层实现的时候还是会为变量创建不同的别名var1 = tf....
  • Tensorflow:variable变量和变量空间

    千次阅读 2019-08-01 21:13:12
    name_scope: 为了更好地管理变量的命名空间而提出的。比如在 tensorboard 中,... variable_scope: 大部分情况下,跟 tf.get_variable() 配合使用,实现变量共享的功能。with tf.variable_scope('scopename', reu...
  • tf.get_variable()和tf.Variable()的区别(最清晰的解释)

    千次阅读 多人点赞 2019-04-27 14:27:01
    ==== 1、tf.Variable()
  • 示例代码如下,首先我们在模块的配置阶段添加这个变量,这里可以在ngx_sean_block里调用sean_add_indexed_variable,变量名sean_key和上面的set指令的变量名保持一致。得到的变量索引保存在sean_cf->sean_key_index...
  • Variable Variable 和 Tensor 自动求导 从反向传播中排除子图 注册钩子 自定义Function 这个是0.3的版本,之后修改。 Autograd: 自动微分 autograd 包是 PyTorch 中神经网络的核心, 它可以为基于 tensor 的的...
  • PyTorch中Variable变量与torch.autograd.Variable

    万次阅读 多人点赞 2020-04-13 22:20:22
    一、了解Variable 顾名思义,Variable就是 变量 的意思。实质上也就是可以变化的量,区别于int变量,它是一种可以变化的变量,这正好就符合了反向传播,参数更新的属性。 具体来说,在pytorch中的Variable就是一个...
  • Pytorch的Variable详解

    万次阅读 多人点赞 2020-08-20 20:57:17
    pytorch两个基本对象:Tensor(张量)和Variable(变量) 其中,tensor不能反向传播,variable可以反向传播。 tensor的算术运算和选取操作与numpy一样,一次你numpy相似的运算操作都可以迁移过来。 Variable ...
  • torch的Variable

    千次阅读 2020-10-29 19:56:37
    Variable数据格式及使用意义 使用这种数据相当于将数据加入到一个节点,这种数据有grad属性,而torch数据没有该属性 import torch import numpy as np from torch.autograd import Variable tensor = torch....
  • ____tz_zs学习笔记 tf.Variable 官网api:https://www.tensorflow.org/api_docs/python/tf/Variable def __init__(self, initial_value=None, trainable=True, colle
  • 条件变量std::condition_variable的作用是阻塞线程,然后等待通知将其唤醒。我们可以通过某个函数判断是否符合某种条件来决定是阻塞线程等待通知还是唤醒线程,由此实现线程间的同步。所以简单来说condition_...
  • input=tf.variable(tf.random_normal([2,3,3,5])) 报错内容为: Traceback (most recent call last): File "E:/HelloWorld/AI/CV/project/exam.py", line 28, in input=tf.variable(tf.random_normal([2,3,...
  • tensorflow tf.Variable()和tf.get_variable()详解

    万次阅读 多人点赞 2018-07-26 22:29:53
    一、tf.Variable() (1)参数说明 tf.Variable是一个Variable类。通过变量维持图graph的状态,以便在sess.run()中执行;可以用Variable类创建一个实例在图中增加变量; Args参数说明: initial_value:Tensor或可...
  • 定义的量没使用 1.删除。 2.骗过编译器。(不懂格式,回来补充) int D; (void)i;

空空如也

空空如也

1 2 3 4 5 ... 20
收藏数 809,986
精华内容 323,994
关键字:

variable