SE Radio 660: Pete Warden on TinyML

Pete Warden, CEO of Useful Sensors and a founding member of the TensorFlow team at Google, discusses TinyML, the technology enabling machine learning on low-power, small-footprint devices. This innovation opens up applications such as voice-controlled devices, offline translation tools, and smarter embedded systems, which are crucial for privacy and efficiency.

SE Radio host Kanchan Shringi speaks with Warden about challenges like model compression, deployment constraints, and privacy concerns. They also explore applications in agriculture, healthcare, and consumer electronics, and close with some practical advice from Pete for newcomers to TinyML development.

Brought to you by IEEE Computer Society and IEEE Software magazine.

Pete Warden, CEO of Useful Sensors and a founding member of the TensorFlow team at Google, discusses TinyML, the technology enabling machine learning on low-power, small-footprint devices. This innovation opens up applications such as voice-controlled devices, offline translation tools, and smarter embedded systems, which are crucial for privacy and efficiency.

SE Radio host Kanchan Shringi speaks with Warden about challenges like model compression, deployment constraints, and privacy concerns. They also explore applications in agriculture, healthcare, and consumer electronics, and close with some practical advice from Pete for newcomers to TinyML development.

Brought to you by IEEE Computer Society and IEEE Software magazine.

Show Notes

• TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers

• Blog post: Why the Future of Machine Learning is Tiny

• Harvard Courses

• Fundamentals Tinyml

• Applications Tinyml

• Deploying Tinyml

• YouTube: @PeteWarden

• Github: ee292d labs

Transcript

Transcript brought to you by IEEE Software magazine and IEEE Computer Society. This transcript was automatically generated. To suggest improvements in the text, please contact [email protected] and include the episode number.

Kanchan Shringi 00:00:19 Hello everyone, welcome to this episode of Software Engineering Radio. Our guest today is Pete Warden. Pete is CEO of useful sensors. Pete was the founding member of the TensorFlow team at Google and the tech lead for all non-cloud TensorFlow infrastructure at Google for seven years. Welcome Pete to the show. It’s really great to have you here. Is there anything you’d like to add to your bio before we get started?

Pete Warden 00:00:44 No, I think that’s great. Thanks so much.

Kanchan Shringi 00:00:46 So Pete, our goal today is to explore the reasons to and the challenges in deploying machine learning on low power, small footprint hardware. This is the essence of TinyML. A term I understand has been coined by you. So Pete, to start, what is TinyML? Why is it important?

Pete Warden 00:01:07 Yeah, and to start with, I should say that Evgeni Gousev of Qualcomm and me with the coiners of TinyML and actually started the TinyML conference series together. So I want to make sure he gets credit, but really the idea behind TinyML is that being able to run machine learning on tiny sheet embedded devices that are in everyday objects all around us, opens up a whole bunch of possibilities of doing things, especially around interacting with our environment in ways that we’ve never been able to before.

Kanchan Shringi 00:01:43 Can you give an example?

Pete Warden 00:01:45 So one of my things I really want to be able to do is just look at a table lamp and say ìonî and have the lamp come on. And that sounds like a straightforward task, but actually there’s a whole bunch of technological challenges in order to make that happen. And then on top of those, a whole bunch of economic and business challenges.

Kanchan Shringi 00:02:08 Before we get more into that, is TinyML related to IoT, the Internet of Things or is it completely unrelated?

Pete Warden 00:02:16 So this is actually one of my bug bears is I really don’t think that the internet of things has succeeded. And the definition that I’ve seen most often of the internet of things is the idea that by adding network connectivity to everyday objects, you are going to actually enable a whole bunch of really exciting new things in the same way that adding internet connectivity to laptops and cell phones created incredible innovation and change and growth. I think that the analogy does not hold for embedded systems and everyday objects for a whole bunch of reasons. And I think it kind of misled a lot of people to invest in adding connectivity and then just hoping wonderful things happen. TinyML from my perspective is a lot more about saying, hey, let’s leave connectivity until we need it, but let’s try and make these individual objects smarter on their own locally with no internet connection and let’s see how far we can go with that.

Kanchan Shringi 00:03:25 So in your example of being able to look at a lamp and that turning on it has got nothing to do with network connectivity?

Pete Warden 00:03:32 Yeah, and one of the key things with that is you can imagine buying a lamp, plugging it in and then immediately being able to talk to it and have it work just like you could in the old days with any other object you bought, you didn’t have to go through set up, you didn’t have to use a phone app, you didn’t have to have an account, you didn’t have to worry about firmware updates. It’s kind of the idea of going back to objects that just work out of the box is I think a big part of the promise.

Kanchan Shringi 00:04:05 But there is machine learning involved. How is this TinyML different from traditional machine learning?

Pete Warden 00:04:13 So I didn’t know much about embedded devices when I first joined Google. I sort of knew that they were this esoteric area, but I’d never had much contact with them. When I joined Google, one of the first people I met was Azi Alvarez and he explained that they were using 30 kilobyte sized models for the, I won’t say the full wake word for Google’s voice assistant because I don’t want everybody’s phones going off, but the detection of that wake word only ran in 30 kilobytes and I couldn’t understand why. And so that led me down this path of realizing that there are these chips that only cost 50 cents or 25 cents, but they use almost no power. So you can have them running for a very long time on a battery and they’re so cheap and low power means that they can be in almost everything you buy.

Pete Warden 00:05:11 So there were 40 billion of these being sold a year. So I grew fascinated by the constraints that were involved in taking what is traditionally you are looking at maybe hundreds of megabytes for machine learning model and trying to run it in something that may only have as much memory as like a Commodore 64 from the ë80s. And it turns out that machine learning actually scales down really, really well and you can do things not with the same accuracy but with enough accuracy to be useful even in these really cheap low power devices. Especially things around understanding what people are doing and understanding the environment around them.

Kanchan Shringi 00:05:56 So you compare the amount of memory with the very old hardware, how much is it actually, the memory?

Pete Warden 00:06:03 The smallest devices I’ve actually done work on have had 64 kilobytes of RAM and one of the design goals of the open-source software that I worked on at Google aimed at these kinds of devices, the whole of the software framework had to fit in less than 20 kilobytes. So we are really talking very much kind of 1980s scales of memory when we are looking at these very ubiquitous embedded devices.

Kanchan Shringi 00:06:35 So the devices that we’re looking at, you give an example of a lamp, what else are we talking, doorbells?

[...]

Original source