# 一天学会TensorFlow

Learn Tensorflow in 1 day

In this tutorial, you are able to learn Tensorflow in 1 day, practically. For theory, I suggest read this book.

a. Useful TensorFlow operators
b. Tensorflow Neural Networks
c. TensorFlow Deep Learning
d. TensorFlow Convolutional Neural Networks (CNN) for image recognition
e. TensorFlow Recurrent Neural Networks (RNN) for text analysis
f. Keras

Useful TensorFlow operators

Setup environment

 1 `pip install tensorflow`

Install Jupyter via PIP.

 1 `pip install jupyter`

Create a new folder called tensorflow-tutorial and cd into that folder via terminal. Run jupyter notebook command.

Useful TensorFlow operators

The official documentation carefully lays out all available math ops: https://www.tensorflow.org/api_docs/Python/math_ops.html.

Some specific examples of commonly used operators include:

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 `tf.add(x, y) ` `Add two tensors of the same type, x + y` `tf.sub(x, y) ` `Subtract tensors of the same type, x — y` `tf.mul(x, y) ` `Multiply two tensors element-wise` `tf.pow(x, y) ` `Take the element-wise power of x to y` `tf.exp(x) ` `Equivalent to pow(e, x), where e is Euler’s number (2.718…)` `tf.sqrt(x) ` `Equivalent to pow(x, 0.5)` `tf.div(x, y) ` `Take the element-wise division of x and y` `tf.truediv(x, y) ` `Same as tf.div, except casts the arguments as a float` `tf.floordiv(x, y) ` `Same as truediv, except rounds down the final answer into an integer` `tf.mod(x, y) ` `Takes the element-wise remainder from division`

Create a new Jupyter notebook with python 2.7 kernel. Name it as TensorFlow operators. Lets write a small program to add two numbers.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 `# import tensorflow` `import` `tensorflow as tf` `# build computational graph` `a ``=` `tf.placeholder(tf.int16)` `b ``=` `tf.placeholder(tf.int16)` `addition ``=` `tf.add(a, b)` `# initialize variables` `init ``=` `tf.global_variables_initializer()` `# create session and run the graph` `with tf.Session() as sess:` `    ``sess.run(init)` `    ``print` `"Addition: %i"` `%` `sess.run(addition, feed_dict``=``{a: ``2``, b: ``3``})` `# close session` `sess.close()`

Exercise: Try all these operations and check the output. tf.add(x, y), tf.sub(x, y), tf.mul(x, y), tf.pow(x, y), tf.sqrt(x), tf.div(x, y) & tf.mod(x, y).

Tensorflow Neural Networks

Create a new Jupyter notebook with python 2.7 kernel. Name it as TensorFlow Neural Networks. Let’s import all the required modules.

 1 2 3 4 5 6 7 8 `%``pylab inline` `import` `os` `import` `numpy as np` `import` `pandas as pd` `from` `scipy.misc ``import` `imread` `from` `sklearn.metrics ``import` `accuracy_score` `import` `tensorflow as tf`

Set a seed value, so that we can control our models randomness

 1 2 3 `# To stop potential randomness` `seed ``=` `128` `rng ``=` `np.random.RandomState(seed)`

Set directory paths

 1 2 3 4 5 6 7 8 `root_dir ``=` `os.path.abspath(``'../'``)` `data_dir ``=` `os.path.join(root_dir, ``'tensorflow-tutorial/data'``)` `sub_dir ``=` `os.path.join(root_dir, ``'tensorflow-tutorial/sub'``)` `# check for existence` `os.path.exists(root_dir)` `os.path.exists(data_dir)` `os.path.exists(sub_dir)`

Read the datasets. These are in .csv formats, and have a filename along with the appropriate labels

 1 2 3 4 `train ``=` `pd.read_csv(os.path.join(data_dir, ``'Train'``, ``'train.csv'``))` `test ``=` `pd.read_csv(os.path.join(data_dir, ``'Test.csv'``))` `train.head()`

 1 2 3 4 5 6 7 8 `img_name ``=` `rng.choice(train.filename)` `filepath ``=` `os.path.join(data_dir, ``'Train'``, ``'Images'``, ``'train'``, img_name)` `img ``=` `imread(filepath, flatten``=``True``)` `pylab.imshow(img, cmap``=``'gray'``)` `pylab.axis(``'off'``)` `pylab.show()`

Show image in numpy array format

 1 `img`

Store all our images as numpy arrays

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 `temp ``=` `[]` `for` `img_name ``in` `train.filename:` `    ``image_path ``=` `os.path.join(data_dir, ``'Train'``, ``'Images'``, ``'train'``, img_name)` `    ``img ``=` `imread(image_path, flatten``=``True``)` `    ``img ``=` `img.astype(``'float32'``)` `    ``temp.append(img)` `    ` `train_x ``=` `np.stack(temp)` `temp ``=` `[]` `for` `img_name ``in` `test.filename:` `    ``image_path ``=` `os.path.join(data_dir, ``'Train'``, ``'Images'``, ``'test'``, img_name)` `    ``img ``=` `imread(image_path, flatten``=``True``)` `    ``img ``=` `img.astype(``'float32'``)` `    ``temp.append(img)` `test_x ``=` `np.stack(temp)`

Split size of 70:30 for train set vs validation set

 1 2 3 4 `split_size ``=` `int``(train_x.shape[``0``]``*``0.7``)` `train_x, val_x ``=` `train_x[:split_size], train_x[split_size:]` `train_y, val_y ``=` `train.label.values[:split_size], train.label.values[split_size:]`

Define some helper functions

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 `def` `dense_to_one_hot(labels_dense, num_classes``=``10``):` `    ``"""Convert class labels from scalars to one-hot vectors"""` `    ``num_labels ``=` `labels_dense.shape[``0``]` `    ``index_offset ``=` `np.arange(num_labels) ``*` `num_classes` `    ``labels_one_hot ``=` `np.zeros((num_labels, num_classes))` `    ``labels_one_hot.flat[index_offset ``+` `labels_dense.ravel()] ``=` `1` `    ` `    ``return` `labels_one_hot` `def` `preproc(unclean_batch_x):` `    ``"""Convert values to range 0-1"""` `    ``temp_batch ``=` `unclean_batch_x ``/` `unclean_batch_x.``max``()` `    ` `    ``return` `temp_batch` `def` `batch_creator(batch_size, dataset_length, dataset_name):` `    ``"""Create batch with random samples and return appropriate format"""` `    ``batch_mask ``=` `rng.choice(dataset_length, batch_size)` `    ` `    ``batch_x ``=` `eval``(dataset_name ``+` `'_x'``)[[batch_mask]].reshape(``-``1``, input_num_units)` `    ``batch_x ``=` `preproc(batch_x)` `    ` `    ``if` `dataset_name ``=``=` `'train'``:` `        ``batch_y ``=` `eval``(dataset_name).ix[batch_mask, ``'label'``].values` `        ``batch_y ``=` `dense_to_one_hot(batch_y)` `        ` `    ``return` `batch_x, batch_y`

Define a neural network architecture with 3 layers; input, hidden and output. The number of neurons in input and output are fixed, as the input is our 28 x 28 image and the output is a 10 x 1 vector representing the class. We take 500 neurons in the hidden layer. This number can vary according to your need. We also assign values to remaining variables.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 `### set all variables` `# number of neurons in each layer` `input_num_units ``=` `28``*``28` `hidden_num_units ``=` `500` `output_num_units ``=` `10` `# define placeholders` `x ``=` `tf.placeholder(tf.float32, [``None``, input_num_units])` `y ``=` `tf.placeholder(tf.float32, [``None``, output_num_units])` `# set remaining variables` `epochs ``=` `5` `batch_size ``=` `128` `learning_rate ``=` `0.01` `### define weights and biases of the neural network (refer this article if you don't understand the terminologies)` `weights ``=` `{` `    ``'hidden'``: tf.Variable(tf.random_normal([input_num_units, hidden_num_units], seed``=``seed)),` `    ``'output'``: tf.Variable(tf.random_normal([hidden_num_units, output_num_units], seed``=``seed))` `}` `biases ``=` `{` `    ``'hidden'``: tf.Variable(tf.random_normal([hidden_num_units], seed``=``seed)),` `    ``'output'``: tf.Variable(tf.random_normal([output_num_units], seed``=``seed))` `}`

Create neural networks computational graph

 1 2 3 4 `hidden_layer ``=` `tf.add(tf.matmul(x, weights[``'hidden'``]), biases[``'hidden'``])` `hidden_layer ``=` `tf.nn.relu(hidden_layer)` `output_layer ``=` `tf.matmul(hidden_layer, weights[``'output'``]) ``+` `biases[``'output'``]`

Define cost of our neural network

 1 `cost ``=` `tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits ``=` `output_layer, labels``=``y))`

 1 `optimizer ``=` `tf.train.AdamOptimizer(learning_rate``=``learning_rate).minimize(cost)`

Initialize all the variables

 1 `init ``=` `tf.global_variables_initializer()`

Create a session, and run the neural network in the session. Then validate the models accuracy on validation set that just created

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 `with tf.Session() as sess:` `    ``# create initialized variables` `    ``sess.run(init)` `    ` `    ``### for each epoch, do:` `    ``###   for each batch, do:` `    ``###     create pre-processed batch` `    ``###     run optimizer by feeding batch` `    ``###     find cost and reiterate to minimize` `    ` `    ``for` `epoch ``in` `range``(epochs):` `        ``avg_cost ``=` `0` `        ``total_batch ``=` `int``(train.shape[``0``]``/``batch_size)` `        ``for` `i ``in` `range``(total_batch):` `            ``batch_x, batch_y ``=` `batch_creator(batch_size, train_x.shape[``0``], ``'train'``)` `            ``_, c ``=` `sess.run([optimizer, cost], feed_dict ``=` `{x: batch_x, y: batch_y})` `            ` `            ``avg_cost ``+``=` `c ``/` `total_batch` `            ` `        ``print` `"Epoch:"``, (epoch``+``1``), ``"cost ="``, ``"{:.5f}"``.``format``(avg_cost)` `    ` `    ``print` `"\nTraining complete!"` `    ` `    ` `    ``# find predictions on val set` `    ``pred_temp ``=` `tf.equal(tf.argmax(output_layer, ``1``), tf.argmax(y, ``1``))` `    ``accuracy ``=` `tf.reduce_mean(tf.cast(pred_temp, ``"float"``))` `    ``print` `"Validation Accuracy:"``, accuracy.``eval``({x: val_x.reshape(``-``1``, input_num_units), y: dense_to_one_hot(val_y)})` `    ` `    ``predict ``=` `tf.argmax(output_layer, ``1``)` `    ``pred ``=` `predict.``eval``({x: test_x.reshape(``-``1``, input_num_units)})`

Test the model and visualize its predictions

 1 2 3 4 5 6 7 8 9 10 11 `img_name ``=` `rng.choice(test.filename)` `filepath ``=` `os.path.join(data_dir, ``'Train'``, ``'Images'``, ``'test'``, img_name)` `img ``=` `imread(filepath, flatten``=``True``)` `test_index ``=` `int``(img_name.split(``'.'``)[``0``]) ``-` `49000` `print` `"Prediction is: "``, pred[test_index]` `pylab.imshow(img, cmap``=``'gray'``)` `pylab.axis(``'off'``)` `pylab.show()`

TensorFlow Deep Learning

Create a new Jupyter notebook with python 2.7 kernel. Name it as TensorFlow Deep Learning. Let’s import all the required modules.

 1 2 3 4 `import` `tensorflow as tf` `import` `tempfile` `import` `pandas as pd` `import` `urllib`

Define Base Feature Columns that will be the building blocks used by both the wide part and the deep part of the model.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 `tf.logging.set_verbosity(tf.logging.ERROR)` `# Categorical base columns.` `gender ``=` `tf.contrib.layers.sparse_column_with_keys(column_name``=``"gender"``, keys``=``[``"Female"``, ``"Male"``])` `race ``=` `tf.contrib.layers.sparse_column_with_keys(column_name``=``"race"``, keys``=``[` `  ``"Amer-Indian-Eskimo"``, ``"Asian-Pac-Islander"``, ``"Black"``, ``"Other"``, ``"White"``])` `education ``=` `tf.contrib.layers.sparse_column_with_hash_bucket(``"education"``, hash_bucket_size``=``1000``)` `relationship ``=` `tf.contrib.layers.sparse_column_with_hash_bucket(``"relationship"``, hash_bucket_size``=``100``)` `workclass ``=` `tf.contrib.layers.sparse_column_with_hash_bucket(``"workclass"``, hash_bucket_size``=``100``)` `occupation ``=` `tf.contrib.layers.sparse_column_with_hash_bucket(``"occupation"``, hash_bucket_size``=``1000``)` `native_country ``=` `tf.contrib.layers.sparse_column_with_hash_bucket(``"native_country"``, hash_bucket_size``=``1000``)` `# Continuous base columns.` `age ``=` `tf.contrib.layers.real_valued_column(``"age"``)` `age_buckets ``=` `tf.contrib.layers.bucketized_column(age, boundaries``=``[``18``, ``25``, ``30``, ``35``, ``40``, ``45``, ``50``, ``55``, ``60``, ``65``])` `education_num ``=` `tf.contrib.layers.real_valued_column(``"education_num"``)` `capital_gain ``=` `tf.contrib.layers.real_valued_column(``"capital_gain"``)` `capital_loss ``=` `tf.contrib.layers.real_valued_column(``"capital_loss"``)` `hours_per_week ``=` `tf.contrib.layers.real_valued_column(``"hours_per_week"``)`

The wide model is a linear model with a wide set of sparse and crossed feature columns:

 1 2 3 4 5 6 `wide_columns ``=` `[` `  ``gender, native_country, education, occupation, workclass, relationship, age_buckets,` `  ``tf.contrib.layers.crossed_column([education, occupation], hash_bucket_size``=``int``(``1e4``)),` `  ``tf.contrib.layers.crossed_column([native_country, occupation], hash_bucket_size``=``int``(``1e4``)),` `  ``tf.contrib.layers.crossed_column([age_buckets, education, occupation], hash_bucket_size``=``int``(``1e6``))` `]`

The Deep Model: Neural Network with Embeddings

 1 2 3 4 5 6 7 8 9 `deep_columns ``=` `[` `  ``tf.contrib.layers.embedding_column(workclass, dimension``=``8``),` `  ``tf.contrib.layers.embedding_column(education, dimension``=``8``),` `  ``tf.contrib.layers.embedding_column(gender, dimension``=``8``),` `  ``tf.contrib.layers.embedding_column(relationship, dimension``=``8``),` `  ``tf.contrib.layers.embedding_column(native_country, dimension``=``8``),` `  ``tf.contrib.layers.embedding_column(occupation, dimension``=``8``),` `  ``age, education_num, capital_gain, capital_loss, hours_per_week` `]`

Combining Wide and Deep Models into one

 1 2 3 4 5 6 7 `model_dir ``=` `tempfile.mkdtemp()` `m ``=` `tf.contrib.learn.DNNLinearCombinedClassifier(` `    ``fix_global_step_increment_bug``=``True``,` `    ``model_dir``=``model_dir,` `    ``linear_feature_columns``=``wide_columns,` `    ``dnn_feature_columns``=``deep_columns,` `    ``dnn_hidden_units``=``[``100``, ``50``])`

Process input data

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 `# Define the column names for the data sets.` `COLUMNS ``=` `[``"age"``, ``"workclass"``, ``"fnlwgt"``, ``"education"``, ``"education_num"``,` `  ``"marital_status"``, ``"occupation"``, ``"relationship"``, ``"race"``, ``"gender"``,` `  ``"capital_gain"``, ``"capital_loss"``, ``"hours_per_week"``, ``"native_country"``, ``"income_bracket"``]` `LABEL_COLUMN ``=` `'label'` `CATEGORICAL_COLUMNS ``=` `[``"workclass"``, ``"education"``, ``"marital_status"``, ``"occupation"``,` `                       ``"relationship"``, ``"race"``, ``"gender"``, ``"native_country"``]` `CONTINUOUS_COLUMNS ``=` `[``"age"``, ``"education_num"``, ``"capital_gain"``, ``"capital_loss"``,` `                      ``"hours_per_week"``]` `# Download the training and test data to temporary files.` `# Alternatively, you can download them yourself and change train_file and` `# test_file to your own paths.` `train_file ``=` `tempfile.NamedTemporaryFile()` `test_file ``=` `tempfile.NamedTemporaryFile()` `urllib.urlretrieve(``"http://mlr.cs.umass.edu/ml/machine-learning-databases/adult/adult.data"``, train_file.name)` `urllib.urlretrieve(``"http://mlr.cs.umass.edu/ml/machine-learning-databases/adult/adult.test"``, test_file.name)` `# Read the training and test data sets into Pandas dataframe.` `df_train ``=` `pd.read_csv(train_file, names``=``COLUMNS, skipinitialspace``=``True``)` `df_test ``=` `pd.read_csv(test_file, names``=``COLUMNS, skipinitialspace``=``True``, skiprows``=``1``)` `df_train[LABEL_COLUMN] ``=` `(df_train[``'income_bracket'``].``apply``(``lambda` `x: ``'>50K'` `in` `x)).astype(``int``)` `df_test[LABEL_COLUMN] ``=` `(df_test[``'income_bracket'``].``apply``(``lambda` `x: ``'>50K'` `in` `x)).astype(``int``)` `def` `input_fn(df):` `  ``# Creates a dictionary mapping from each continuous feature column name (k) to` `  ``# the values of that column stored in a constant Tensor.` `  ``continuous_cols ``=` `{k: tf.constant(df[k].values)` `                     ``for` `k ``in` `CONTINUOUS_COLUMNS}` `  ``# Creates a dictionary mapping from each categorical feature column name (k)` `  ``# to the values of that column stored in a tf.SparseTensor.` `  ``categorical_cols ``=` `{k: tf.SparseTensor(` `      ``indices``=``[[i, ``0``] ``for` `i ``in` `range``(df[k].size)],` `      ``values``=``df[k].values,` `      ``dense_shape``=``[df[k].size, ``1``])` `                      ``for` `k ``in` `CATEGORICAL_COLUMNS}` `  ``# Merges the two dictionaries into one.` `  ``feature_cols ``=` `dict``(continuous_cols.items() ``+` `categorical_cols.items())` `  ``# Converts the label column into a constant Tensor.` `  ``label ``=` `tf.constant(df[LABEL_COLUMN].values)` `  ``# Returns the feature columns and the label.` `  ``return` `feature_cols, label` `def` `train_input_fn():` `  ``return` `input_fn(df_train)` `def` `eval_input_fn():` `  ``return` `input_fn(df_test)`

Training and evaluating The Model

 1 2 3 4 `m.fit(input_fn``=``train_input_fn, steps``=``200``)` `results ``=` `m.evaluate(input_fn``=``eval_input_fn, steps``=``1``)` `for` `key ``in` `sorted``(results):` `    ``print``(``"%s: %s"` `%` `(key, results[key]))`

TensorFlow Convolutional Neural Networks (CNN)

Create a new Jupyter notebook with python 2.7 kernel. Name it as TensorFlow CNN. In this tutorial we will train a simple classifier to classify images of birds. Open your Chrome browser and install Fatkun Batch Download Image. Google this keyword malabar pied hornbill. Select Images and click Fatkun Batch Download Image icon on the right top. Select This tab and new windows will appear.

Unselect which images that not related to malabar pied hornbill bird category then click Save Image. Make sure minimum images that need to be train is 75. Wait until all images finish download. Copy all the images and place it into <your_working_space> > tf_files > birds > images > Malabar Pied Hornbill. Repeat the same steps over and over again for these categories.

 1 2 3 4 5 6 7 8 9 10 11 `sacred kingfisher` `pied kingfisher` `common hoopoe` `layard s parakeet` `owl` `sparrow` `brahminy kite` `sparrowhawk` `wallcreeper` `bornean ground cuckoo` `blue crowned hanging parrot`

Download retrain script (https://raw.githubusercontent.com/datomnurdin/tensorflow-python/master/retrain.py) to the current directory (<your_working_space>) . Go to terminal/command line and cd to <your_working_space> directory. Run this command to retrain all the images. It takes around 30 minutesto finish.

 1 2 3 4 5 6 `python retrain.py ` `--bottleneck_dir=tf_files/bottlenecks ` `--model_dir=tf_files/inception ` `--output_graph=tf_files/retrained_graph.pb ` `--output_labels=tf_files/retrained_labels.txt ` `--image_dir <``your_absolute_path``>/<``your_working_space``>/tf_files/birds`

Create a prediction script and load generated model into it.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 `import` `tensorflow as tf` `import` `sys` `# change this as you see fit` `image_path ``=` `sys.argv[``1``]` `# Read in the image_data` `image_data ``=` `tf.gfile.FastGFile(image_path, ``'rb'``).read()` `# Loads label file, strips off carriage return` `label_lines ``=` `[line.rstrip() ``for` `line ` `                   ``in` `tf.gfile.GFile(``"tf_files/retrained_labels.txt"``)]` `# Unpersists graph from file` `with tf.gfile.FastGFile(``"tf_files/retrained_graph.pb"``, ``'rb'``) as f:` `    ``graph_def ``=` `tf.GraphDef()` `    ``graph_def.ParseFromString(f.read())` `    ``_ ``=` `tf.import_graph_def(graph_def, name``=``'')` `with tf.Session() as sess:` `    ``# Feed the image_data as input to the graph and get first prediction` `    ``softmax_tensor ``=` `sess.graph.get_tensor_by_name(``'final_result:0'``)` `    ` `    ``predictions ``=` `sess.run(softmax_tensor, \` `             ``{``'DecodeJpeg/contents:0'``: image_data})` `    ` `    ``# Sort to show labels of first prediction in order of confidence` `    ``top_k ``=` `predictions[``0``].argsort()[``-``len``(predictions[``0``]):][::``-``1``]` `    ` `    ``for` `node_id ``in` `top_k:` `        ``human_string ``=` `label_lines[node_id]` `        ``score ``=` `predictions[``0``][node_id]` `        ``print``(``'%s (score = %.5f)'` `%` `(human_string, score))`

Predict the image using terminal/command line.

 1 `python detect.py test_image.png`

TensorFlow Recurrent Neural Networks (RNN) for text analysis

Create a new Jupyter notebook with python 2.7 kernel. Name it as TensorFlow RNN – model. In this tutorial we will train chat conversation. There are two phases in this tutorial, training the modelling and test the chat response. Import libraries and modules.

 1 2 3 4 5 6 7 8 9 10 `# things we need for NLP` `import` `nltk` `from` `nltk.stem.lancaster ``import` `LancasterStemmer` `stemmer ``=` `LancasterStemmer()` `# things we need for Tensorflow` `import` `numpy as np` `import` `tflearn` `import` `tensorflow as tf` `import` `random`

Import chat-bot intents file.

 1 2 3 `import` `json` `with ``open``(``'data/intents.json'``) as json_data:` `    ``intents ``=` `json.load(json_data)`

Start organize our documents, words and classification classes.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 `words ``=` `[]` `classes ``=` `[]` `documents ``=` `[]` `ignore_words ``=` `[``'?'``]` `# loop through each sentence in our intents patterns` `for` `intent ``in` `intents[``'intents'``]:` `    ``for` `pattern ``in` `intent[``'patterns'``]:` `        ``# tokenize each word in the sentence` `        ``w ``=` `nltk.word_tokenize(pattern)` `        ``# add to our words list` `        ``words.extend(w)` `        ``# add to documents in our corpus` `        ``documents.append((w, intent[``'tag'``]))` `        ``# add to our classes list` `        ``if` `intent[``'tag'``] ``not` `in` `classes:` `            ``classes.append(intent[``'tag'``])` `# stem and lower each word and remove duplicates` `words ``=` `[stemmer.stem(w.lower()) ``for` `w ``in` `words ``if` `w ``not` `in` `ignore_words]` `words ``=` `sorted``(``list``(``set``(words)))` `# remove duplicates` `classes ``=` `sorted``(``list``(``set``(classes)))` `print` `(``len``(documents), ``"documents"``)` `print` `(``len``(classes), ``"classes"``, classes)` `print` `(``len``(words), ``"unique stemmed words"``, words)`

Create training data.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 `training ``=` `[]` `output ``=` `[]` `# create an empty array for our output` `output_empty ``=` `[``0``] ``*` `len``(classes)` `# training set, bag of words for each sentence` `for` `doc ``in` `documents:` `    ``# initialize our bag of words` `    ``bag ``=` `[]` `    ``# list of tokenized words for the pattern` `    ``pattern_words ``=` `doc[``0``]` `    ``# stem each word` `    ``pattern_words ``=` `[stemmer.stem(word.lower()) ``for` `word ``in` `pattern_words]` `    ``# create our bag of words array` `    ``for` `w ``in` `words:` `        ``bag.append(``1``) ``if` `w ``in` `pattern_words ``else` `bag.append(``0``)` `    ``# output is a '0' for each tag and '1' for current tag` `    ``output_row ``=` `list``(output_empty)` `    ``output_row[classes.index(doc[``1``])] ``=` `1` `    ``training.append([bag, output_row])` `# shuffle our features and turn into np.array` `random.shuffle(training)` `training ``=` `np.array(training)` `# create train and test lists` `train_x ``=` `list``(training[:,``0``])` `train_y ``=` `list``(training[:,``1``])`

Build a model.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 `# reset underlying graph data` `tf.reset_default_graph()` `# Build neural network` `net ``=` `tflearn.input_data(shape``=``[``None``, ``len``(train_x[``0``])])` `net ``=` `tflearn.fully_connected(net, ``8``)` `net ``=` `tflearn.fully_connected(net, ``8``)` `net ``=` `tflearn.fully_connected(net, ``len``(train_y[``0``]), activation``=``'softmax'``)` `net ``=` `tflearn.regression(net)` `# Define model and setup tensorboard` `model ``=` `tflearn.DNN(net, tensorboard_dir``=``'data/tflearn_logs'``)` `# Start training (apply gradient descent algorithm)` `model.fit(train_x, train_y, n_epoch``=``1000``, batch_size``=``8``, show_metric``=``True``)` `model.save(``'data/model.tflearn'``)`
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 `def` `clean_up_sentence(sentence):` `    ``# tokenize the pattern` `    ``sentence_words ``=` `nltk.word_tokenize(sentence)` `    ``# stem each word` `    ``sentence_words ``=` `[stemmer.stem(word.lower()) ``for` `word ``in` `sentence_words]` `    ``return` `sentence_words` `# return bag of words array: 0 or 1 for each word in the bag that exists in the sentence` `def` `bow(sentence, words, show_details``=``False``):` `    ``# tokenize the pattern` `    ``sentence_words ``=` `clean_up_sentence(sentence)` `    ``# bag of words` `    ``bag ``=` `[``0``]``*``len``(words)  ` `    ``for` `s ``in` `sentence_words:` `        ``for` `i,w ``in` `enumerate``(words):` `            ``if` `w ``=``=` `s: ` `                ``bag[i] ``=` `1` `                ``if` `show_details:` `                    ``print` `(``"found in bag: %s"` `%` `w)` `    ``return``(np.array(bag))`
 1 2 3 `p ``=` `bow(``"What is Soding?"``, words)` `print` `(p)` `print` `(classes)`

Show prediction model.

 1 `print``(model.predict([p]))`

Save all of our data structures

 1 2 `import` `pickle` `pickle.dump( {``'words'``:words, ``'classes'``:classes, ``'train_x'``:train_x, ``'train_y'``:train_y}, ``open``( ``"data/training_data"``, ``"wb"` `) )`

Build a chatbot framework.

Create a new Jupyter notebook with python 2.7 kernel. Name it as TensorFlow RNN – response. Import libraries and modules.

 1 2 3 4 5 6 7 8 9 10 `# things we need for NLP` `import` `nltk` `from` `nltk.stem.lancaster ``import` `LancasterStemmer` `stemmer ``=` `LancasterStemmer()` `# things we need for Tensorflow` `import` `numpy as np` `import` `tflearn` `import` `tensorflow as tf` `import` `random`

Restore all of our data structures and import chat-bot intents file.

 1 2 3 4 5 6 7 8 9 10 `import` `pickle` `data ``=` `pickle.load( ``open``( ``"data/training_data"``, ``"rb"` `) )` `words ``=` `data[``'words'``]` `classes ``=` `data[``'classes'``]` `train_x ``=` `data[``'train_x'``]` `train_y ``=` `data[``'train_y'``]` `import` `json` `with ``open``(``'data/intents.json'``) as json_data:` `    ``intents ``=` `json.load(json_data)`

Build neural network.

 1 2 3 4 5 6 7 8 `net ``=` `tflearn.input_data(shape``=``[``None``, ``len``(train_x[``0``])])` `net ``=` `tflearn.fully_connected(net, ``8``)` `net ``=` `tflearn.fully_connected(net, ``8``)` `net ``=` `tflearn.fully_connected(net, ``len``(train_y[``0``]), activation``=``'softmax'``)` `net ``=` `tflearn.regression(net)` `# Define model and setup tensorboard` `model ``=` `tflearn.DNN(net, tensorboard_dir``=``'data/tflearn_logs'``)`
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 `def` `clean_up_sentence(sentence):` `    ``# tokenize the pattern` `    ``sentence_words ``=` `nltk.word_tokenize(sentence)` `    ``# stem each word` `    ``sentence_words ``=` `[stemmer.stem(word.lower()) ``for` `word ``in` `sentence_words]` `    ``return` `sentence_words` `# return bag of words array: 0 or 1 for each word in the bag that exists in the sentence` `def` `bow(sentence, words, show_details``=``False``):` `    ``# tokenize the pattern` `    ``sentence_words ``=` `clean_up_sentence(sentence)` `    ``# bag of words` `    ``bag ``=` `[``0``]``*``len``(words)  ` `    ``for` `s ``in` `sentence_words:` `        ``for` `i,w ``in` `enumerate``(words):` `            ``if` `w ``=``=` `s: ` `                ``bag[i] ``=` `1` `                ``if` `show_details:` `                    ``print` `(``"found in bag: %s"` `%` `w)` `    ``return``(np.array(bag))`
 1 2 3 `p ``=` `bow(``"What is Soding?"``, words)` `print` `(p)` `print` `(classes)`

 1 `model.load(``'data/model.tflearn'``)`

Create a data structure to hold user context.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 `context ``=` `{}` `ERROR_THRESHOLD ``=` `0.25` `def` `classify(sentence):` `    ``# generate probabilities from the model` `    ``results ``=` `model.predict([bow(sentence, words)])[``0``]` `    ``# filter out predictions below a threshold` `    ``results ``=` `[[i,r] ``for` `i,r ``in` `enumerate``(results) ``if` `r>ERROR_THRESHOLD]` `    ``# sort by strength of probability` `    ``results.sort(key``=``lambda` `x: x[``1``], reverse``=``True``)` `    ``return_list ``=` `[]` `    ``for` `r ``in` `results:` `        ``return_list.append((classes[r[``0``]], r[``1``]))` `    ``# return tuple of intent and probability` `    ``return` `return_list` `def` `response(sentence, userID``=``'123'``, show_details``=``False``):` `    ``results ``=` `classify(sentence)` `    ``# if we have a classification then find the matching intent tag` `    ``if` `results:` `        ``# loop as long as there are matches to process` `        ``while` `results:` `            ``for` `i ``in` `intents[``'intents'``]:` `                ``# find a tag matching the first result` `                ``if` `i[``'tag'``] ``=``=` `results[``0``][``0``]:` `                    ``# set context for this intent if necessary` `                    ``if` `'context_set'` `in` `i:` `                        ``if` `show_details: ``print` `(``'context:'``, i[``'context_set'``])` `                        ``context[userID] ``=` `i[``'context_set'``]` `                    ``# check if this intent is contextual and applies to this user's conversation` `                    ``if` `not` `'context_filter'` `in` `i ``or` `\` `                        ``(userID ``in` `context ``and` `'context_filter'` `in` `i ``and` `i[``'context_filter'``] ``=``=` `context[userID]):` `                        ``if` `show_details: ``print` `(``'tag:'``, i[``'tag'``])` `                        ``# a random response from the intent` `                        ``print` `(random.choice(i[``'responses'``]))` `            ``results.pop(``0``)`
 1 `classify(``'What is Soding?'``)`
 1 `response(``'What is Soding?'``)`
 1 `response(``'Where are your office?'``)`
 1 `response(``'How can I apply a job?'``)`
 1 `response(``'What sectors or industries do you recruit for?'``)`

Neural Network on Keras

Create a new Jupyter notebook with python 2.7 kernel. Name it as TensorFlow Keras. Let’s import all the required modules. Import libraries and modules.

 1 2 3 4 5 6 7 8 9 `%``pylab inline` `import` `numpy as np` `np.random.seed(``123``)  ``# for reproducibility` `from` `keras.models ``import` `Sequential` `from` `keras.layers ``import` `Dense, Dropout, Activation, Flatten` `from` `keras.layers ``import` `Convolution2D, MaxPooling2D` `from` `keras.utils ``import` `np_utils`

Populating the interactive namespace from numpy and matplotlib. Load image data from MNIST.

 1 2 3 4 `from` `keras.datasets ``import` `mnist` ` ` `# Load pre-shuffled MNIST data into train and test sets` `(X_train, y_train), (X_test, y_test) ``=` `mnist.load_data()`

Shape of the dataset

 1 `print` `X_train.shape`

60,000 samples in the training set, and the images are 28 pixels x 28 pixels each. Plotting the first sample in matplotlib

 1 2 `from` `matplotlib ``import` `pyplot as plt` `plt.imshow(X_train[``0``])`

Preprocess input data for Keras. When using the Theano backend, you must explicitly declare a dimension for the depth of the input image. For example, a full-color image with all 3 RGB channels will have a depth of 3. Our MNIST images only have a depth of 1, but we must explicitly declare that. In other words, we want to transform our dataset from having shape (n, width, height) to (n, depth, width, height).

 1 2 `X_train ``=` `X_train.reshape(X_train.shape[``0``], ``1``, ``28``, ``28``)` `X_test ``=` `X_test.reshape(X_test.shape[``0``], ``1``, ``28``, ``28``)`

Show X_train’s dimensions again

 1 `print` `X_train.shape`

The final preprocessing step for the input data is to convert our data type to float32 and normalize our data values to the range [0, 1].

 1 2 3 4 `X_train ``=` `X_train.astype(``'float32'``)` `X_test ``=` `X_test.astype(``'float32'``)` `X_train ``/``=` `255` `X_test ``/``=` `255`

Preprocess class labels for Keras. Check shape of our class label data.

 1 `print` `y_train.shape`

Convert 1-dimensional class arrays to 10-dimensional class matrices

 1 2 `Y_train ``=` `np_utils.to_categorical(y_train, ``10``)` `Y_test ``=` `np_utils.to_categorical(y_test, ``10``)`

Check shape of our class label data again

 1 `print` `Y_train.shape`

Define model architecture. Declaring a sequential model format

 1 `model ``=` `Sequential()`

Declare the input layer which is (1, 28, 28) that corresponds to the (depth, width, height) of each digit image

 1 `model.add(Convolution2D(``32``, (``3``, ``3``), activation``=``'relu'``, input_shape``=``(``1``,``28``,``28``), data_format``=``'channels_first'``))`

Shape of the current model output

 1 `print` `model.output_shape`

Add more layers to our model. For Dense layers, the first parameter is the output size of the layer. Keras automatically handles the connections between layers.

Note that the final layer has an output size of 10, corresponding to the 10 classes of digits.

Also note that the weights from the Convolution layers must be flattened (made 1-dimensional) before passing them to the fully connected Dense layer.

 1 2 3 4 5 6 7 8 `model.add(Convolution2D(``32``, (``3``, ``3``), activation``=``'relu'``))` `model.add(MaxPooling2D(pool_size``=``(``2``,``2``)))` `model.add(Dropout(``0.25``))` `model.add(Flatten())` `model.add(Dense(``128``, activation``=``'relu'``))` `model.add(Dropout(``0.5``))` `model.add(Dense(``10``, activation``=``'softmax'``))`

Compile model. Declare the loss function and the optimizer (SGD, Adam, etc.).

 1 2 3 `model.``compile``(loss``=``'categorical_crossentropy'``,` `              ``optimizer``=``'adam'``,` `              ``metrics``=``[``'accuracy'``])`

Fit model on training data. Declare the batch size and number of epochs to train for, then pass in training data

 1 `model.fit(X_train, Y_train, batch_size``=``32``, epochs``=``10``, verbose``=``1``)`

Evaluate model on test data.

 1 `score ``=` `model.evaluate(X_test, Y_test, verbose``=``0``)`

Predict the image

 1 `Y_test ``=` `model.predict_classes(X_test, verbose``=``2``)`

If you are a tech company looking for evaluation platform, you can register here and we will be in touch.

If you resonated with this tutorial, please subscribe to my personal email subscription. You will get a weekly update from me about coding, recruitment & Artificial Intelligence.