|
1 | 1 |
|
2 | | -# Transfer learning |
| 2 | +This directory contains two examples of transfer learning using the "Inception V3" image classification model. |
3 | 3 |
|
4 | | - - [Introduction](#introduction) |
5 | | - - [1. Take a look at the the Inception v3 model](#1-take-a-look-at-the-the-inception-v3-model) |
6 | | - - [Data sets](#data-sets) |
7 | | - - [The "hugs/no-hugs" data set](#the-hugsno-hugs-data-set) |
8 | | - - [(Or, you can use the Flowers data set if you want)](#or-you-can-use-the-flowers-data-set-if-you-want) |
9 | | - - [Pre-generated 'bottleneck' values for both example datasets](#pre-generated-bottleneck-values-for-both-example-datasets) |
10 | | - - [2. Run a training session and use the model for prediction](#2-run-a-training-session-and-use-the-model-for-prediction) |
11 | | - - [Train the model](#train-the-model) |
12 | | - - [Do prediction using your learned model in an ipython notebook](#do-prediction-using-your-learned-model-in-an-ipython-notebook) |
13 | | - - [3. A Custom Esimator for the transfer learning model](#3-a-custom-esimator-for-the-transfer-learning-model) |
14 | | - - [Named Scopes and TensorBoard Summary information](#named-scopes-and-tensorboard-summary-information) |
15 | | - - [4. Exercise: Building the Custom Estimator's model graph](#4-exercise-building-the-custom-estimators-model-graph) |
| 4 | +The [cloudml](cloudml) example shows how to use [Cloud Dataflow](https://cloud.google.com/dataflow/) ([Apache |
| 5 | +Beam](https://beam.apache.org/)) to do image preprocessing, then train and serve your model on Cloud ML. It supports |
| 6 | +distributed training on Cloud ML. |
| 7 | +It is based on the example [here](https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/flowers), with |
| 8 | +some additional modifications to make it easy to use other image sets, and a prediction web server that demos how to |
| 9 | +use the Cloud ML API for prediction once your trained model is serving. |
16 | 10 |
|
| 11 | +The [TF_Estimator](TF_Estimator) example takes a similar approach, but is not packaged to run on Cloud ML. It also |
| 12 | +shows an example of using a custom [`Estimator`](https://www.tensorflow.org/api_docs/python/contrib.learn/estimators). |
17 | 13 |
|
18 | | -## Introduction |
19 | | - |
20 | | -This lab shows how we can use an existing model to do *transfer learning* -- effectively bootstrapping an existing model to reduce the effort needed to learn something new. |
21 | | - |
22 | | -Specifically, we will take an 'Inception' v3 architecture model trained on ImageNet images, and using its penultimate "bottleneck" layer, train a new top layer that can recognize other classes of images. |
23 | | -We'll see that our new top layer does not need to be very complex, and that we don't need to do much training of this new model, to get good results for our new image classifications. |
24 | | - |
25 | | -The core of the approach is the same as that used in [this TensorFlow example](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining), but here we will use a custom [Estimator](https://www.tensorflow.org/versions/r0.11/api_docs/python/contrib.learn.html#estimators) (and train on a different set of photos). |
26 | | - |
27 | | -## 1. Take a look at the the Inception v3 model |
28 | | - |
29 | | -We can use the `view_inception_model.ipynb` Jupyter notebook to take a look at the structure of the Inception model before we start working with it. |
30 | | - |
31 | | -First, download the inception model from: |
32 | | -http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz , extract it, |
33 | | -and copy the model file `classify_image_graph_def.pb` into `/tmp/imagenet` (you may need to first create the directory). This is where our python scripts will look for it, so we're saving a later download by putting it in the same place. |
34 | | - |
35 | | -Then, start a jupyter server in this directory. For convenience, run it in a new terminal window. (Don't forget to activate your virtual environment first as necessary). |
36 | | - |
37 | | -```sh |
38 | | -$ jupyter notebook |
39 | | -``` |
40 | | - |
41 | | -Load and run the `view_inception_model.ipynb` notebook. Poke around the model graph a bit. |
42 | | - |
43 | | -<a href="https://storage.googleapis.com/oscon-tf-workshop-materials/images/incpv3.png" target="_blank"><img src="https://storage.googleapis.com/oscon-tf-workshop-materials/images/incpv3.png" width="500"/></a> |
44 | | - |
45 | | -See if you can find the 'DecodeJpeg/contents:0' and 'pool_3/_reshape:0' nodes-- these will be our input and 'bottleneck' nodes, respectively, for the transfer learning. |
46 | | - |
47 | | -<a href="https://storage.googleapis.com/oscon-tf-workshop-materials/images/incpv3_pool_3_reshape.png" target="_blank"><img src="https://storage.googleapis.com/oscon-tf-workshop-materials/images/incpv3_pool_3_reshape.png" width="500"/></a> |
48 | | - |
49 | | -Note: If you should want to write the model graph to a text file to browse it that way, you can use |
50 | | -the `tf.train.write_graph()` method. See [`mnist_hidden.py`](../mnist_series/the_hard_way/mnist_hidden.py) for |
51 | | -a (commented-out) example of how to call it. |
52 | | - |
53 | | -## Data sets |
54 | | - |
55 | | -We've provided training images for you, but if you want to play around further, you can use any image datasets you like. The training script simply assumes you have a top-level directory containing class-named subdirectories, each containing images for that class. It then infers the classes to be learned from the directory structure. |
56 | | - |
57 | | -### The "hugs/no-hugs" data set |
58 | | - |
59 | | -For this exercise, we'll use a training set of images that have been sorted into two categories -- whether or not one would want to hug the object in the photo. |
60 | | -(Thanks to Julia Ferraioli for this dataset). |
61 | | - |
62 | | -This dataset does not have a large number of images, but as we will see, prediction on new images still works surprisingly well. This shows the power of 'bootstrapping' the pre-trained Inception model. |
63 | | - |
64 | | - |
65 | | -```sh |
66 | | -$ curl -O https://storage.googleapis.com/oscon-tf-workshop-materials/transfer_learning/hugs_photos.zip |
67 | | -$ unzip hugs_photos.zip |
68 | | -``` |
69 | | - |
70 | | -### (Or, you can use the Flowers data set if you want) |
71 | | - |
72 | | -If you want to do flower classification instead, as with the original tutorial, you can find the data here: |
73 | | - |
74 | | -```sh |
75 | | -$ curl -O http://download.tensorflow.org/example_images/flower_photos.tgz |
76 | | -$ tar xzf flower_photos.tgz |
77 | | -``` |
78 | | - |
79 | | - |
80 | | -### Pre-generated 'bottleneck' values for both example datasets |
81 | | - |
82 | | -When you run the transfer learning training, you'll first need to generate "bottleneck values" for the images, using the Inception v3 model. (We'll take a look at how that works). |
83 | | -If this process is too time-consuming for the workshop context, you can download the pre-calculated bottleneck files for both the data sets above: |
84 | | - |
85 | | -- https://storage.googleapis.com/oscon-tf-workshop-materials/transfer_learning/bottlenecks_hugs.zip |
86 | | -- https://storage.googleapis.com/oscon-tf-workshop-materials/transfer_learning/bottlenecks_flowers.zip |
87 | | - |
88 | | -## 2. Run a training session and use the model for prediction |
89 | | - |
90 | | -Let's start by training our new model and using the results to make predictions. |
91 | | - |
92 | | -### Train the model |
93 | | - |
94 | | -```sh |
95 | | -$ python transfer_learning.py --image_dir=hugs_photos --bottleneck_dir=bottlenecks_hugs |
96 | | -``` |
97 | | - |
98 | | -**Note the name of the model directory** that is created by the script. |
99 | | - |
100 | | -### Do prediction using your learned model in an ipython notebook |
101 | | - |
102 | | -Start up a jupyter server in this directory as necessary. Select the `transfer_learning_prediction.ipynb` notebook in the listing that comes up. |
103 | | - |
104 | | -Find this line: |
105 | | -``` |
106 | | -MODEL_DIR = '/tmp/tfmodels/img_classify/your-model-dir' |
107 | | -``` |
108 | | - |
109 | | -and edit it to point to the model directory used for your training run. |
110 | | - |
111 | | -Then, run the notebook. |
112 | | -You should see some predictions made for the images in the `prediction_images` directory! |
113 | | - |
114 | | -If you like, you can try adding additional images to that directory, and rerunning the last part of the notebook to find out whether they're more huggable than not. |
115 | | - |
116 | | -## 3. A Custom Esimator for the transfer learning model |
117 | | - |
118 | | -Before we jump into the coding part of the lab, we'll take a look at `transfer_learning_skeleton.py`. |
119 | | -It has the scaffolding in place for building a custom Estimator to do the transfer learning. |
120 | | -We'll look at how the `fit()`, `evaluate()`, and `predict()` methods are being used. |
121 | | - |
122 | | -We'll also take a look at how the Inception model is being loaded and accessed. |
123 | | - |
124 | | -## Named Scopes and TensorBoard Summary information |
125 | | - |
126 | | -Note that this code includes some examples of use of `tf.name_scope()` when defining nodes, particularly |
127 | | -in the `add_final_training_ops()` function. You'll be able to spot these scope names when you look at the model graph in TensorBoard. |
128 | | -We saw use of `tf.name_scope` earlier in ['mnist_hidden.py'](../mnist_series/the_hard_way/mnist_hidden.py) as well. |
129 | | - |
130 | | -The code in `add_final_training_ops()` also includes some examples of defining summary information for TensorBoard (we saw a simple example of doing this in ['mnist_hidden.py'](../mnist_series/the_hard_way/mnist_hidden.py) also). |
131 | | - |
132 | | -However, here, as we're wrapping things in an Estimator, we don't need to an an explicit `tf.merge_summary` op-- it will do that for us. |
133 | | - |
134 | | - |
135 | | -## 4. Exercise: Building the Custom Estimator's model graph |
136 | | - |
137 | | -Start with [`transfer_learning_skeleton.py`](transfer_learning.py), and complete the `_make_model` |
138 | | -function definition. This function builds the model graph for the custom estimator. |
139 | | - |
140 | | -As noted above, the Inception model graph is doing the heavy lifting here. We will just train a new |
141 | | -top layer to identify our new classes: that is, we will just add a new softmax and fully-connected |
142 | | -layer. The input to this layer is the generated "bottleneck" values. The `add_final_training_ops` |
143 | | -function defines this layer, then defines the loss function and the training op. |
144 | | - |
145 | | -Then, the `add_evaluation_step` function adds an op to evaluate the accuracy of the results. Add |
146 | | -'loss' and 'accuracy' metrics to the prediction_dict, as per the `METRICS` dict below |
147 | | -`make_model_fn` in the code, which we will then pass to the Estimator's `evaluate()` method. |
148 | | - |
149 | | -Then, add support for generating prediction value(s). |
150 | | -See if you can figure out how to derive the index of the highest-value the entry in the result |
151 | | -vector, and store that value at the `"index"` key in the `prediction_dict`. As a hint, take a look |
152 | | -at the ops used in `add_evaluation_step()`. |
153 | | - |
154 | | -As shown in the skeleton of `_make_model`, be sure to return the prediction dict, the loss, and the |
155 | | -training op. This info sets up the Estimator to handle calls to its `fit()`, `evaluate()`, and |
156 | | -`predict()` methods. |
157 | | - |
158 | | - |
159 | | -If you get stuck, you can take a peek at `transfer_learning.py`, but try not to do that too soon. |
| 14 | +The list of image sources for the images used in the "hugs/no-hugs" training is here: |
| 15 | +https://storage.googleapis.com/oscon-tf-workshop-materials/transfer_learning/hugs_photos_sources.csv |
0 commit comments