Chapter 19: Training and Deploying TensorFlow Models at Scale
A Review of Hands-On Machine Learning with Scikit-Learn, Keras & Tensorflow by Aurélien Géron
Summary
So, you’ve used the techniques we have learned throughout the first 18 chapters and you have created a sick Machine Learning Model that will change the world. But, how do you take it from a model on your machine, or array of machines, and put it out into the world so that other people can use it? Well this chapter provides a brief overview of all of that. It is essentially a primer on MLops which by itself is a whole career. But, in this chapter you learn about on-prem, cloud, and mobile development as well as different architectures to optimize your training process.
TensorFlow Models
Saving TensorFlow models is relatively easy with using built-in TensorFlow functions. The easiest way to accomplish this is to run:
tf.saved_model.save(<model name>, <model path>)And then reopen in keras through:
model = keras.models.load_model(<path to saved model>)Now that your model has been saved through TensorFlow you can run it through a TensorFlow Docker container or upload it to the cloud where you can host it without using local resources.
With how quickly the cloud moves I suggest relying on official documentation for this upload rather than blog posts which can quickly become outdated. For uploading to GCP I suggest using this guide. This is a quick and straightforward guide to using GCP’s AI Platform and Cloud Storage to create simple cloud based resources. More complex examples involving barebones or hosted Kubernetes will be more difficult and are outside of the scope of this chapter.
GPUs and TPUs for Speed
GPUs and TPUs are processors that are far more optimized for training Machine Learning Models than CPUs. While I am not 100% sure how they work I know that they distribute the computations into smaller chunks and processes than CPUs (which can only run one process per core with a typical max of about 16 cores). If you aren’t willing to spend a few thousand dollars during normal times and many more during the current chip shortage I recommend you use Kaggle, Google Colabs, or a cloud provider like Azure, GCP, or AWS to gain access to cheap or free GPUs. Free services like Kaggle and Google Colabs will limit how much you can do with their GPUs (and even TPUs on Colabs) but they should be able to hold you over for a while before you’re willing to invest in your own GPUs.
Mobile ML and TFLite
Machine Learning on your mobile or embedded device? How is that possible after we talked about how we need to use specialized processors just to train and store our data efficiently. Well, you can use TFLite. TFLite is a mobile friendly and lightweight version of TensorFlow that will allow you to deploy models onto smaller machines. To do this, TFLite will lighten your regular TensorFlow models and reduce the amount of compute and ram that is necessary to run your model. TFLite is a whole topic on its own and I need to learn far more about it!
Multiple Devices
So, you now have a complex and difficult to train Deep Learning Model using TensorFlow that would take your laptop three weeks to train on max cpu and gpu utilization. So, what should you do?
One way to solve this would be to look for stronger processors like a state of the art GPU rather than your built in one (if you even have a built in GPU). In general one really strong GPU is better than using two medium strength GPUs. Or if this isn’t a viable option you can look to the cloud to gain computing power for this project specifically.
But, what if just maximizing your GPU is not enough? Then you should probably look to parallel computing with either multiple machines or multiple GPUs on one machine. To do this you will have to implement some form of model or data parallelism so that the multiple machines that you are using will work well with each other. This is another topic that you could (and many have) write entire books or take entire classes on and go a bit beyond the scope of this project. So, up until the time where this is needed I am going to focus on the resources I have and building better ML Models.
*Any reference to GPUs in this section can also substitute TPUs.
My Thoughts
This final chapter was actually more in my wheelhouse than most of the other chapters coming from a devops background. Although this was very much a primer on how to deploy apps in production I think that it gave a good overall approach to multiple methods. While I am primarily an Azure user (with light AWS and GCP experience) I thought it was good to see what it would be like to deploy to GCP and some of the advantages of deploying and training in the cloud. The one thing I wish that this chapter got into a little more is how TPUs work because even though I’ve heard a decent amount about them I don’t really know how they differ from cpus and gpus. A nice chart describing the difference between the three types of procuessors would have been helpful for my understanding but it is something I will do some learning about on my own. If you’re curious like I am here are a few starter resources:
This was the last chapter of Hands On Machine Learning and I’m honestly excited to be finished. The last 19 days have been a bit of a slog at times but I am happy to have gotten through the entire book and have improved my knowledge of Machine Learning. I definitely know a lot more than when I have started and I think that this book has set me up to continue learning! I am going to do a more in-depth review at a later date (I want a day off haha) and will talk more about this book. I will bring this up again in that review but I do NOT recommend that you do what I did with this book. If you are a strong beginner or light intermediate like me I recommend that you work your way through this book slower than I did and complete every exercise as you read it. I believe this will be a much more beneficial strategy to your long-term learning.
Thanks for reading!
If you have any questions or feedback please reach out to me on twitter @wtothdev or leave a comment!
Additionally, I wanted to give a huge thanks to Aurélien Géron for writing such an excellent book. You can purchase said book here (non-affiliate).
Disclaimer: I don’t make any money from any of the services referenced and chose to read and review this book under my own free will.
