Isabella: Becoming useful

Previously on Dr. Lambda's blog:

In a previous post I presented my newest pet project: Isabella. Isabella is a voice controlled personal assistant, like Siri, Alexa, and others. We have decided to investigate how difficult it is to make such a program. In the last post we discussed the importance of shortening the distance between coding and testing.

Now, the continuation...

Hello, World!

So far all of the functionality we in Isabella (apart from the speech recognition), have been coded directly, Functions for getting the time, date, or day, are coded directly into her source code. And that is fine for some stuff; they are useful functions. However, if we really want her to grow fast, the next obvious step was to connect her to the rest of the world, by making calls to APIs.

There are so many APIs out there (APIs), with tons of functionality which can be integrated for free. I do listen to a lot of music, and it would be nice to integrate that somehow.

We do have an advantage over many of the other personal assistants like Siri, or Alexa; we have a very large screen at our disposal. There is a theater saying "if you bring it to the stage, use it!" the point being that if you have something you should aim to utilize it as much as possible. An example could be, if you bring a cane onto the stage, it can also be an umbrella, a pointing stick, a gun, and many other things. Therefore, as we have a large screen let's do something with it.

Youtube, and wildcards

Personally I spend a lot of time on youtube, listening to music, or re-watching classic videos. Therefore the first API I want to integrate is Youtube's. First we need to play Youtube videos. A quick Google search gives us the a link to Youtube API for embedding videos. This is perfect for what we need. The only slight problem was that we need the ID of the video we want to play.

So, we need some way to search. This took a bit more work, and we had to get the first API key for Isabella: Youtube Search API. However, once we figured out the call we wanted, combining the two APIs was very easy.

However, we still needed to extend our basic command format to support wildcards, so we could say "Play ___ from youtube", and the same command would work wether you said "ed sheeran", "adele", or whatever. This was not too difficult, but it is very powerful!

Finally, Isabella can do something that we are actually going to use!

Field Testing

As a test I challenged myself not to use youtube – manually – for an entire day. While working I constantly listen to music, and it worked great. It was actually easier to ask Isabella to lookup videos instead of: switching window (away from work), going to youtube, typing the name of the video I wanted, click the top result, and switch back to what I was doing. Just like I argued in the last post, not having to fully context switch was a noticeable improvement on my workflow.

This function alone, I think, is enough that I'm going to keep using her, for a long time.