TF. Repeat and stack operations of slim

Reference resources   TF.Slim Of repeat and stack operation - cloud + Community - Tencent cloud

One 、 routine

When building the network ,TF-Slim Provide repeat and stack, Allow users to repeat the same operation , Convenient network construction , for example ：

net = ...
net = slim.conv2d(net, 256, [3, 3], scope='conv3_1')
net = slim.conv2d(net, 256, [3, 3], scope='conv3_2')
net = slim.conv2d(net, 256, [3, 3], scope='conv3_3')
net = slim.max_pool2d(net, [2, 2], scope='pool2')

Common practice ： Available cycles reduce work

net = ...
for i in range(3):
   net = slim.conv2d(net, 256, [3, 3], scope='conv3_%d' % (i+1))
   net = slim.max_pool2d(net, [2, 2], scope='pool2')

Two 、TF-Slim Medium repeat operation

Use TF-Slim Medium repeat Operation overrides the definition above ：

net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3], scope='conv3')
net = slim.max_pool2d(net, [2, 2], scope='pool2')

slim.repeat Will automatically give each convolution layer scopes Name it ’conv3/conv3_1’, ’conv3/conv3_2’ and ’conv3/conv3_3’

3、 ... and 、TF-Slim Medium stack operation

TF-Slim Of slim.stack Operation allows the user to call the same operation repeatedly with different parameters slim.stack Also create a new one for each created operation tf.variable_scope

# Verbose way: Primitive operation 
x = slim.fully_connected(x, 32, scope='fc/fc_1')
x = slim.fully_connected(x, 64, scope='fc/fc_2')
x = slim.fully_connected(x, 128, scope='fc/fc_3')
     
# Equivalent , TF-Slim way using slim.stack: Equivalent operation 
slim.stack(x, slim.fully_connected , [32, 64, 128], scope='fc')
     
# slim.stack  Called  slim.fully_connected  Three times 

notice repeat and stack Differences in details . Check the documents

Example , Definition vgg

def vgg16(inputs):
   with slim.arg_scope([slim.conv2d, slim.fully_connected],
   activation_fn=tf.nn.relu,
   weights_initializer=tf.truncated_normal_initializer(0.0, 0.01),
   weights_regularizer=slim.l2_regularizer(0.0005)):  # with under 
            net = slim.repeat(inputs, 2, slim.conv2d, 64, [3, 3], scope='conv1')
            net = slim.max_pool2d(net, [2, 2], scope='pool1')
            net = slim.repeat(net, 2, slim.conv2d, 128, [3, 3], scope='conv2')
            net = slim.max_pool2d(net, [2, 2], scope='pool2')
            net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3], scope='conv3')
            net = slim.max_pool2d(net, [2, 2], scope='pool3')
            net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv4')
            net = slim.max_pool2d(net, [2, 2], scope='pool4')
            net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv5')
            net = slim.max_pool2d(net, [2, 2], scope='pool5')
            net = slim.fully_connected(net, 4096, scope='fc6')
            net = slim.dropout(net, 0.5, scope='dropout6')
            net = slim.fully_connected(net, 4096, scope='fc7')
            net = slim.dropout(net, 0.5, scope='dropout7')
            net = slim.fully_connected(net, 1000, activation_fn=None, scope='fc8')
return net

be based on slim Of vgg frame ：

import tensorflow as tf
import tensorflow.contrib.slim.nets as nets
     
slim = tf.contrib.slim
vgg = nets.vgg
     
...
     
train_log_dir = ...
if not tf.gfile.Exists(train_log_dir):
        tf.gfile.MakeDirs(train_log_dir)
     
with tf.Graph().as_default():
        # Set up the data loading:
        images, labels = ...
     
        # Define the model:
        predictions = vgg.vgg_16(images, is_training=True)
     
        # Specify the loss function:
        slim.losses.softmax_cross_entropy(predictions , labels)
     
        total_loss = slim.losses.get_total_loss()
        tf.summary.scalar('losses/total_loss', total_loss)
     
        # Specify the optimization scheme:
        optimizer = tf.train.GradientDescentOptimizer(learning_rate=.001)
     
        # create_train_op that ensures that when we evaluate it to get the loss,
        # the update_ops are done and the gradient updates are computed.
        train_tensor = slim.learning.create_train_op(total_loss , optimizer)
     
        # Actually runs training.
        slim.learning.train(train_tensor , train_log_dir)