tf.variable vs tf.constant in tensorflow

QBoard » Artificial Intelligence & ML » AI and ML - Tensorflow » tf.variable vs tf.constant in tensorflow

User Dashboard

tf.variable vs tf.constant in tensorflow

Back To Topics

Tags : None

Vaibhav Mali

259
I ran the following code to
```
W = tf.Variable(tf.zeros([1, 3]), dtype=tf.float32, name="W")
 B = tf.constant([[1, 2, 3]], dtype=tf.float32, name="B")
 act = tf.add(W, B)
 init = tf.global_variables_initializer()
 with tf.Session() as sess:
     sess.run(init)
     sess.run(act)
     writer = tf.summary.FileWriter("./graphs", sess.graph)
 writer.close()
```
And verified it with tensorboard:

What confuses me is the read operation and the operation prior to that which is denoted as (W). Constant B is directed straight to the Add operation while tf.variable has all these operation nodes inside. Here are my questions:

What is (W) operation? Constant B is a regular circle which denotes a constant. Oval shaped nodes denote Operation node. (W) doesn't seem like any operation yet it is denoted with the same oval shaped node? What is that node's job?

Add node explicitly reads (W) node with a read operation as opposed to constant node B. Why is read necessary for variable nodes?
This post was edited by Vaibhav Mali at July 21, 2021 2:07 PM IST
July 20, 2021 2:10 PM IST

0
Viaan Prakash

461

W operation is tf.Variable you declared here: W = tf.Variable(tf.zeros([1, 3]), dtype=tf.float32, name="W"). Behind the scenes it does many operations (like W.initializer - your init op, W.value() your read op, W.assign() to assign the value to itself, and may be more). Also you see your initial value zeros.

All of this is internal to the tf.Variable and you should not worry about it. This is why all of this was folded (abstracted away) from you behind the big red border.

July 22, 2021 12:39 PM IST

0
Samar Patil

346 3

For lack of a link to any intermediate-level documentation, this is my pragmatic, conceptual model of tensorflow variables.

The following, from https://www.tensorflow.org/programmers_guide/graphs#visualizing_your_graph seems to at least imply an answer to your question.

"Executing v = tf.Variable(0) adds to the graph a tf.Operation that will store a writeable tensor value that persists between tf.Session.run calls. The tf.Variable object wraps this operation, and can be used like a tensor, which will read the current value of the stored value. The tf.Variable object also has methods such as assign and assign_add that create tf.Operation objects that, when executed, update the stored value."

And this from https://www.tensorflow.org/programmers_guide/variables

style="font-family: arial, helvetica, sans-serif; font-size: 10pt;">"Internally, a tf.Variable stores a persistent tensor. Specific ops allow you to read and modify the values of this tensor. "

And this from http://www.goldsborough.me/tensorflow/ml/ai/python/2017/06/28/20-21-45-a_sweeping_tour_of_tensorflow/

style="font-family: arial, helvetica, sans-serif; font-size: 10pt;">"variables are in-memory buffers containing tensors."

Note that the lines between nodes of a graph MUST be tensors. tf.constant(...) returns a (instance of class) Tensor. However, tf.Variable(...) returns not a Tensor instance, but an instance of a class Variable

x = tf.Variable()

print(type(x)) # <class 'tensorflow.python.ops.variables.Variable'>

y = tf.constant()

print(type(y)) # <class 'tensorflow.python.framework.ops.Tensor'>

To use a tf variable in an operation (whose arguments must be tensors), its value must first be "transformed" into a tensor, and the "read" operation returns the "hidden" tensor that the variable represents. I believe the value is returned as a tf.constant (which is a tensor).

Note the capital V in tf.Variable(...), and the small c in tf.constant(..). A tf.Variable(...) returns an instance of a tf.Variable class, so tf.Variable(...) instantiates a class instance, and read() is a (visualization of a) method on this class which returns a value. When a value is assigned to a variable, it modifies this "hidden" tensor.

On the other hand, at least conceptually, tf.constant(...) is a factory function which returns an instance of class Tensor.

It would be nice to have a link to some intermediate-level documentation about this.

July 24, 2021 2:57 PM IST

0

Member Sign In

Member Sign In

Create Account

tf.variable vs tf.constant in tensorflow

Connect With Us