资讯专栏INFORMATION COLUMN

tensorflow学习笔记3——MNIST应用篇

baishancloud / 1149人阅读

摘要:的卷积神经网络应用卷积神经网络的概念卷积神经网络是一种前馈神经网络,它的人工神经元可以响应一部分覆盖范围内的周围单元,对于大型图像处理有出色表现。

MNIST的卷积神经网络应用 卷积神经网络的概念

卷积神经网络(Convolutional Neural Network,CNN)是一种前馈神经网络,它的人工神经元可以响应一部分覆盖范围内的周围单元,对于大型图像处理有出色表现。[2] 它包括卷积层(convolutional layer)和池化层(pooling layer)。

使用卷积神经网络来训练MNIST数据集
    import tensorflow as tf
    import numpy as np
    from tensorflow.examples.tutorials.mnist import input_data
    mnist = input_data.read_data_sets("MNIST_data", one_hot=True)

    trX, trY, teX, teY = mnist.train.images, mnist.train.labels,         mnist.test.images, mnist.test.labels

    trX = trX.reshape(-1, 28, 28, 1)#28*28*1 input image
    teX = teX.reshape(-1, 28, 28, 1)

    X = tf.placeholder("float", [None, 28, 28, 1])
    Y = tf.placeholder("float", [None, 10])
    conv_dropout  = tf.placeholder("float")
    dense_dropout = tf.placeholder("float")
    w1 = tf.Variable(tf.radom_normal([3, 3, 1, 32], stddev=0.01))
    w2 = tf.Variable(tf.radom_normal([3, 3, 32, 64], stddev=0.01))
    w3 = tf.Variable(tf.radom_normal([3, 3, 64, 128], stddev=0.01))
    w4 = tf.Variable(tf.radom_normal([4*4*128, 1024], stddev=0.01))
    wo = tf.Variable(tf.random_normal([1024, 10], stddev=0.01))

    #卷积和池化、dropout
    def conv_and_pool(x, w, step, dropout):
        x = tf.nn.relu(tf.nn.conv2d(x, w, strides=[1,1,1,1], padding="SAME"))
        x = tf.nn.max_pool(x, ksize=[1, step, step, 1], strides=[1, step, step, 1], padding="SAME")
        x = tf.nn.dropout(dropout)
        return x
    #构建模型
    def conv_model(x, w1, w2, w3, w4, wo, dropout, dense_do):
        x = conv_and_pool(x, w1, 2, 0.5)#第一层卷积 
        x = conv_and_pool(x, w2, 2, 0.5)#第二层卷积 
        x = conv_and_pool(x, w3, 2, 0.5)#第三层卷积 

        x = tf.nn.relu(tf.nn.matmul(x, w4))#全连接
        x = tf.nn.dropout(x, dense_do)#dropout,防止过拟合

        x = tf.nn.relu(tf.nn.matmul(x, wo))#输出预测分类
        return x;

    py_x = conv_model(X, w1, w2, w3, w4, wo, conv_dropout, dense_dropout)

    cost =     tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=py_x, labels=Y))
    train_op = tf.train.RMSPropOptimizer(0.001, 0.9).minize(cost)
    predict_op = tf.argmax(py_x, 1)

    batch_size = 128
    test_size = 256

    #训练模型和评估模型
    with tf.Sesseion() as sess:
        tf.global_variables_initializer().run()

        for i in range(100):
            training_batch = zip(range(0, len(trX), batch_size),         range(batch_size, len(trX)+1, batch_size))
        for start, end in training_batch:
            sess.run(train_op, feed_dict={X:trX[start:end], Y:trY[start:end], conv_dropout:0.8, dense_dropout:0.5})
        
    test_indices = np.arange(len(txX))
    np.random.shuffle(test_indices)
    test_indices = test_indices[0:test_size]
    print(i, np.mean(np.ragmax(teY[test_indices], axis=1) == sess.run(predict_op, feed_dict={X:teX[test_indices], conv_dropout:1.0, dense_dropout:1.0})))
输出结果:

0.179688
0.453125
0.671875
0.773438
0.765625
0.789062
0.804688
0.84375
0.796875
0.828125
...
0.953125
0.921875
0.945312
0.9375
0.914062
0.929688
0.953125
0.9375

MNIST的循环神经网络应用 循环神经网络的概念(RNN,又称为递归神经网络)

在传统的神经网络模型中,是从输入层到隐含层再到输出层,层与层之间是全连接的,每层之间的节点是无连接的。但是这种普通的神经网络对于很多问题却无能无力。例如,你要预测句子的下一个单词是什么,一般需要用到前面的单词,因为一个句子中前后单词并不是独立的。RNN(Recurrent Neuron Network)是一种对序列数据建模的神经网络,即一个序列当前的输出与前面的输出也有关。具体的表现形式为网络会对前面的信息进行记忆并应用于当前输出的计算中,即隐藏层之间的节点不再无连接而是有连接的,并且隐藏层的输入不仅包括输入层的输出还包括上一时刻隐藏层的输出。
RNN在自然语言处理领域的以下几个方向应用得非常成功:

机器翻译;

语音识别;

图像描述生成(把RNN和CNN结合,根据图像的特征生成描述)

语言模型与文本生成,即利用生成的模型预测下一个单词的可能性.

使用循环神经网络(RNN)训练MNIST数据集
    import tensorflow as tf
    from tensorflow.examples.tutorials.mnist import input_data
    from tensorflow.contrib import rnn
    tf.set_random_seed(1)

    mnist = input_data.read_data_sets("/tmp/data", one_hot=True)
    optimize_op = 0.01
    train_count = 100000
    batch_size  = 128

    #
    n_inputs = 28
    n_steps = 28
    n_hidden_units = 128
    n_classes = 10

    x = tf.placeholder(tf.float32, [None, 28, 28])
    y = tf.placeholder(tf.float32, [None, 10])

    weights = {
        "in": tf.Variable(tf.random_normal([28, 128])),
        "out": tf.Variable(tf.random_normal([128, 10])),
    }

    baises = {
        "in": tf.Variable(tf.constant(0.1, shape=[128, ])),
        "out": tf.Variable(tf.constant(0.1, shape=[10, ])),
    }

    def RNN(X, weights, baises):
        #Xtransform to [128*28, 28]
        X = tf.reshape(X, [-1, 28])
        X_in = tf.matmul(X, weights["in"]) + baises["in"]
        #[128*28, 128]->vonvert[128, 28, 128]
        X_in = tf.reshape(X_in, [-1, 28, 128])
        lstm_cell = tf.contrib.rnn.BasicLSTMCell(n_hidden_units, forget_bias=1.0, state_is_tuple=True)
        init_state = lstm_cell.zero_state(batch_size, dtype=tf.float32)
        #dynamic_rnn
        #outputs, final_state = rnn.static_rnn(lstm_cell, X_in, initial_state=init_state)
        outputs, final_state = tf.nn.dynamic_rnn(lstm_cell, X_in,         initial_state=init_state, time_major=False)
        results = tf.matmul(final_state[1], weights["out"]) + baises["out"]
        return results;

    pred = RNN(x, weights, baises)
    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))
    train_op = tf.train.AdamOptimizer(optimize_op).minimize(cost)

    correct_pred = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
    accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
    init = tf.global_variables_initializer()
    with tf.Session() as sess:
        sess.run(init)
        step = 0
        while step * batch_size < train_count:
            batch_xs, batch_ys = mnist.train.next_batch(batch_size)
            batch_xs = batch_xs.reshape([batch_size, 28, 28])
            sess.run([train_op], feed_dict={
             x: batch_xs,
             y: batch_ys,
             })
        
            if step % 20 == 0:
               print(sess.run(accuracy, feed_dict={x:batch_xs, y:batch_ys,}))
            step += 1
输出结果:

0.179688
0.453125
0.671875
0.773438
...
0.9375
0.914062
0.929688
0.953125
0.9375

MNIST的自编码网络实现应用 自编码网络的概念

自编码器是神经网络的一种,是一种无监督学习方法,使用了反向传播算法,目标是使输出=输入。 自编码器内部有隐藏层 ,可以产生编码表示输入。自编码器主要作用在于通过复现输出而捕捉可以代表输入的重要因素,利用中间隐层对输入的压缩表达,达到像PCA那样的找到原始信息主成分的效果。

使用自编码网络编码MNIST
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
from tensorflow.contrib import rnn
import matplotlib.pyplot as plt
import numpy as np
tf.set_random_seed(1)

mnist = input_data.read_data_sets("/tmp/data", one_hot=True)
learning_rate = 0.01
training_epochs = 20
batch_size  = 256#batch size for once training
display_step = 1

examples_to_show = 10#images to show in view

n_hidden_1 = 256#first hidden layer feature count
n_hidden_2 = 128#second hidden layer feature count
n_input = 784 #input data count


X = tf.placeholder("float", [None, n_input])#input image data


weights = {
     "encoder_h1": tf.Variable(tf.random_normal([n_input, n_hidden_1])),
     "encoder_h2": tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),
     "decoder_h1": tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])),
     "decoder_h2": tf.Variable(tf.random_normal([n_hidden_1, n_input])),
 }
biases = {
     "encoder_b1": tf.Variable(tf.random_normal([n_hidden_1])),
     "encoder_b2": tf.Variable(tf.random_normal([n_hidden_2])),
     "decoder_b1": tf.Variable(tf.random_normal([n_hidden_1])),
     "decoder_b2": tf.Variable(tf.random_normal([n_input])),
 }

def encoder(x):
    layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights["encoder_h1"]), biases["encoder_b1"]))
    layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1,         weights["encoder_h2"]), biases["encoder_b2"]))
    return layer_2

def decoder(x):
    layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(x, weights["decoder_h1"]), biases["decoder_b1"]))
    layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, weights["decoder_h2"]), biases["decoder_b2"]))
    return layer_2

encoder_op = encoder(X)#encoder image data
decoder_op = decoder(encoder_op)#decoder image data

y_pred = decoder_op#prediction image data
y_true = X
cost = tf.reduce_mean(tf.pow(y_pred - y_true, 2))
optimizer = tf.train.RMSPropOptimizer(learning_rate).minimize(cost)
init = tf.global_variables_initializer()
with tf.Session() as sess:

   sess.run(init)
   total_batch = int(mnist.train.num_examples/batch_size)
   for epoch in range(training_epochs):
      for i in range(total_batch):
          batch_xs, batch_ys = mnist.train.next_batch(batch_size)
          _, c = sess.run([optimizer, cost], feed_dict={X:batch_xs})
      if epoch %display_step == 0:
         print("Epoch:", "%04d" % (epoch+1), "cost=", "{:.9f}".format(c))
  print ("Optimization Finished!")
  encode_decode = sess.run(y_pred, feed_dict={X: 
  mnist.test.images[:examples_to_show]})
  
  f, a = plt.subplots(2, 10, figsize=(10, 2))#绘图比较原始图片和编码网络重建结果
  print ("after plt.subplots")
  for i in range(examples_to_show):
      a[0][i].imshow(np.reshape(mnist.test.images[i], (28, 28)))#测试集
      a[1][i].imshow(np.reshape(encode_decode[i], (28, 28)))#重建结果
   f.show()
   plt.draw()
   
输出结果:

文章版权归作者所有,未经允许请勿转载,若此文章存在违规行为,您可以联系管理员删除。

转载请注明本文地址:https://www.ucloud.cn/yun/41320.html

相关文章

  • TensorFlow学习笔记(6):TensorBoard之Embeddings

    摘要:前言本文基于官网的写成。是自带的一个可视化工具,是其中的一个功能,用于在二维或三维空间对高维数据进行探索。本文使用数据讲解的使用方法。 前言 本文基于TensorFlow官网的How-Tos写成。 TensorBoard是TensorFlow自带的一个可视化工具,Embeddings是其中的一个功能,用于在二维或三维空间对高维数据进行探索。 An embedding is a map ...

    hover_lew 评论0 收藏0
  • 深度学习

    摘要:深度学习在过去的几年里取得了许多惊人的成果,均与息息相关。机器学习进阶笔记之一安装与入门是基于进行研发的第二代人工智能学习系统,被广泛用于语音识别或图像识别等多项机器深度学习领域。零基础入门深度学习长短时记忆网络。 多图|入门必看:万字长文带你轻松了解LSTM全貌 作者 | Edwin Chen编译 | AI100第一次接触长短期记忆神经网络(LSTM)时,我惊呆了。原来,LSTM是神...

    Vultr 评论0 收藏0

发表评论

0条评论

最新活动
阅读需要支付1元查看
<