1
1
# Variables: Creation, Initialization, Saving, and Loading <a class =" md-anchor " id =" AUTOGENERATED-variables--creation--initialization--saving--and-loading " ></a >
2
2
3
- When you train a model, you use [ Variables ] ( ../../api_docs/python/state_ops.md )
3
+ When you train a model, you use [ variables ] ( ../../api_docs/python/state_ops.md )
4
4
to hold and update parameters. Variables are in-memory buffers containing
5
- tensors. They need to be explicitly initialized and can be saved to disk during
5
+ tensors. They must be explicitly initialized and can be saved to disk during
6
6
and after training. You can later restore saved values to exercise or analyse
7
7
the model.
8
8
9
9
This document references the following TensorFlow classes. Follow the links to
10
10
their reference manual for a complete description of their API:
11
11
12
- * The ` Variable ` class [ tf.Variable] ( ../../api_docs/python/state_ops.md#Variable ) .
13
- * The ` Saver ` class [ tf.train.Saver] ( ../../api_docs/python/state_ops.md#Saver ) .
12
+ * The [ ` tf.Variable ` ] ( ../../api_docs/python/state_ops.md#Variable ) class .
13
+ * The [ ` tf.train.Saver ` ] ( ../../api_docs/python/state_ops.md#Saver ) class .
14
14
15
15
16
16
## Creation <a class =" md-anchor " id =" AUTOGENERATED-creation " ></a >
17
17
18
18
When you create a [ Variable] ( ../../api_docs/python/state_ops.md ) you pass a
19
19
` Tensor ` as its initial value to the ` Variable() ` constructor. TensorFlow
20
- provides a collection of Ops that produce tensors often used for initialization
20
+ provides a collection of ops that produce tensors often used for initialization
21
21
from [ constants or random values] ( ../../api_docs/python/constant_op.md ) .
22
22
23
- Note that all these Ops require you to specify the shape of the tensors. That
23
+ Note that all these ops require you to specify the shape of the tensors. That
24
24
shape automatically becomes the shape of the variable. Variables generally
25
25
have a fixed shape, but TensorFlow provides advanced mechanisms to reshape
26
26
variables.
@@ -32,28 +32,28 @@ weights = tf.Variable(tf.random_normal([784, 200], stddev=0.35),
32
32
biases = tf.Variable(tf.zeros([200 ]), name = " biases" )
33
33
```
34
34
35
- Calling ` tf.Variable() ` adds a few Ops to the graph:
35
+ Calling ` tf.Variable() ` adds several ops to the graph:
36
36
37
- * A ` variable ` Op that holds the variable value.
38
- * An initializer Op that sets the variable to its initial value. This is
39
- actually a ` tf.assign ` Op .
40
- * The Ops for the initial value, such as the ` zeros ` Op for the ` biases `
37
+ * A ` variable ` op that holds the variable value.
38
+ * An initializer op that sets the variable to its initial value. This is
39
+ actually a ` tf.assign ` op .
40
+ * The ops for the initial value, such as the ` zeros ` op for the ` biases `
41
41
variable in the example are also added to the graph.
42
42
43
43
The value returned by ` tf.Variable() ` value is an instance of the Python class
44
44
` tf.Variable ` .
45
45
46
46
## Initialization <a class =" md-anchor " id =" AUTOGENERATED-initialization " ></a >
47
47
48
- Variable initializers must be run explicitly before other Ops in your model can
49
- be run. The easiest way to do that is to add an Op that runs all the variable
50
- initializers, and run that Op before using the model.
48
+ Variable initializers must be run explicitly before other ops in your model can
49
+ be run. The easiest way to do that is to add an op that runs all the variable
50
+ initializers, and run that op before using the model.
51
51
52
52
You can alternatively restore variable values from a checkpoint file, see
53
53
below.
54
54
55
- Use ` tf.initialize_all_variables() ` to add an Op to run variable initializers.
56
- Only run that Op after you have fully constructed your model and launched it in
55
+ Use ` tf.initialize_all_variables() ` to add an op to run variable initializers.
56
+ Only run that op after you have fully constructed your model and launched it in
57
57
a session.
58
58
59
59
``` python
@@ -62,13 +62,13 @@ weights = tf.Variable(tf.random_normal([784, 200], stddev=0.35),
62
62
name = " weights" )
63
63
biases = tf.Variable(tf.zeros([200 ]), name = " biases" )
64
64
...
65
- # Add an Op to initialize the variables.
65
+ # Add an op to initialize the variables.
66
66
init_op = tf.initialize_all_variables()
67
67
68
68
# Later, when launching the model
69
69
with tf.Session() as sess:
70
70
# Run the init operation.
71
- sess.Run (init_op)
71
+ sess.run (init_op)
72
72
...
73
73
# Use the model
74
74
...
@@ -77,7 +77,7 @@ with tf.Session() as sess:
77
77
### Initialization from another Variable <a class =" md-anchor " id =" AUTOGENERATED-initialization-from-another-variable " ></a >
78
78
79
79
You sometimes need to initialize a variable from the initial value of another
80
- variable. As the Op added by ` tf.initialize_all_variables() ` initializes all
80
+ variable. As the op added by ` tf.initialize_all_variables() ` initializes all
81
81
variables in parallel you have to be careful when this is needed.
82
82
83
83
To initialize a new variable from the value of another variable use the other
@@ -98,27 +98,29 @@ w_twice = tf.Variable(weights.initialized_value() * 0.2, name="w_twice")
98
98
99
99
### Custom Initialization <a class =" md-anchor " id =" AUTOGENERATED-custom-initialization " ></a >
100
100
101
- The convenience function ` tf.initialize_all_variables() ` adds an Op to
101
+ The convenience function ` tf.initialize_all_variables() ` adds an op to
102
102
initialize * all variables* in the model. You can also pass it an explicit list
103
103
of variables to initialize. See the
104
104
[ Variables Documentation] ( ../../api_docs/python/state_ops.md ) for more options,
105
105
including checking if variables are initialized.
106
106
107
107
## Saving and Restoring <a class =" md-anchor " id =" AUTOGENERATED-saving-and-restoring " ></a >
108
108
109
- The easiest way to save and restore a model is to use a ` tf.train.Saver `
110
- object. The constructor adds ` save ` and ` restore ` Ops to the graph for all, or
111
- a specified list, of variables. The saver object provides methods to run these
112
- Ops, specifying paths for the checkpoint files to write to or read from.
109
+ The easiest way to save and restore a model is to use a ` tf.train.Saver ` object.
110
+ The constructor adds ` save ` and ` restore ` ops to the graph for all, or a
111
+ specified list, of the variables in the graph. The saver object provides
112
+ methods to run these ops, specifying paths for the checkpoint files to write to
113
+ or read from.
113
114
114
115
### Checkpoint Files <a class =" md-anchor " id =" AUTOGENERATED-checkpoint-files " ></a >
115
116
116
- Variables are saved in binary files that, roughly, contains a map from variable
117
- names to tensors .
117
+ Variables are saved in binary files that, roughly, contain a map from variable
118
+ names to tensor values .
118
119
119
- When you create a ` Saver ` object, you can optionally chose names for the
120
- variables in the checkpoint files. By default, it uses the names passed to the
121
- ` tf.Variable() ` call.
120
+ When you create a ` Saver ` object, you can optionally choose names for the
121
+ variables in the checkpoint files. By default, it uses the value of the
122
+ [ ` Variable.name ` ] ( ../../api_docs/python/state_ops.md#Variable.name ) property for
123
+ each variable.
122
124
123
125
### Saving Variables <a class =" md-anchor " id =" AUTOGENERATED-saving-variables " ></a >
124
126
@@ -130,20 +132,20 @@ the model.
130
132
v1 = tf.Variable(... , name = " v1" )
131
133
v2 = tf.Variable(... , name = " v2" )
132
134
...
133
- # Add an Op to initialize the variables.
135
+ # Add an op to initialize the variables.
134
136
init_op = tf.initialize_all_variables()
135
137
136
- # Add Ops to save and restore all the variables.
138
+ # Add ops to save and restore all the variables.
137
139
saver = tf.train.Saver()
138
140
139
141
# Later, launch the model, initialize the variables, do some work, save the
140
142
# variables to disk.
141
143
with tf.Session() as sess:
142
- sess.Run (init_op)
144
+ sess.run (init_op)
143
145
# Do some work with the model.
144
146
..
145
147
# Save the variables to disk.
146
- save_path = saver.Save (sess, " /tmp/model.ckpt" )
148
+ save_path = saver.save (sess, " /tmp/model.ckpt" )
147
149
print " Model saved in file: " , save_path
148
150
```
149
151
@@ -157,23 +159,23 @@ restore variables from a file you do not have to initialize them beforehand.
157
159
v1 = tf.Variable(... , name = " v1" )
158
160
v2 = tf.Variable(... , name = " v2" )
159
161
...
160
- # Add Ops to save and restore all the variables.
162
+ # Add ops to save and restore all the variables.
161
163
saver = tf.train.Saver()
162
164
163
165
# Later, launch the model, use the saver to restore variables from disk, and
164
166
# do some work with the model.
165
167
with tf.Session() as sess:
166
168
# Restore variables from disk.
167
- saver.Restore (sess, " /tmp/model.ckpt" )
169
+ saver.restore (sess, " /tmp/model.ckpt" )
168
170
print " Model restored."
169
171
# Do some work with the model
170
172
...
171
173
```
172
174
173
- ### Chosing which Variables to Save and Restore <a class =" md-anchor " id =" AUTOGENERATED-chosing -which-variables-to-save-and-restore " ></a >
175
+ ### Choosing which Variables to Save and Restore <a class =" md-anchor " id =" AUTOGENERATED-choosing -which-variables-to-save-and-restore " ></a >
174
176
175
- If you do not pass any argument to ` tf.train.Saver() ` the saver
176
- handles all variables . Each one of them is saved under the name that was
177
+ If you do not pass any argument to ` tf.train.Saver() ` the saver handles all
178
+ variables in the graph . Each one of them is saved under the name that was
177
179
passed when the variable was created.
178
180
179
181
It is sometimes useful to explicitly specify names for variables in the
@@ -196,10 +198,10 @@ Notes:
196
198
* You can create as many saver objects as you want if you need to save and
197
199
restore different subsets of the model variables. The same variable can be
198
200
listed in multiple saver objects, its value is only changed when the saver
199
- ` Restore ()` method is run.
201
+ ` restore ()` method is run.
200
202
201
203
* If you only restore a subset of the model variables at the start
202
- of a session, you have to run an initialize Op for the other variables. See
204
+ of a session, you have to run an initialize op for the other variables. See
203
205
[ ` tf.initialize_variables() ` ] ( ../../api_docs/python/state_ops.md#initialize_variables )
204
206
for more information.
205
207
@@ -208,7 +210,7 @@ Notes:
208
210
v1 = tf.Variable(... , name = " v1" )
209
211
v2 = tf.Variable(... , name = " v2" )
210
212
...
211
- # Add Ops to save and restore only 'v2' using the name "my_v2"
213
+ # Add ops to save and restore only 'v2' using the name "my_v2"
212
214
saver = tf.train.Saver({" my_v2" : v2})
213
215
# Use the saver object normally after that.
214
216
...
0 commit comments