File, New: Personal AI Assistant

With the launch of my latest iOS app out of the way, I’ve decided on an ambitious new side-project. Since I’m often guilty of starting things and then never seeing them through, I figured that launching this sub-blog might give me the motivation I need to stay focused.

The Elevator Pitch

A voice-activated AI assistant, with a visual avatar, built using C# running with the mono runtime on a Raspberry Pi.

Okay, so this will require a lot of technology that I’m not much familiar with. To break it down, I’m going to need to master at least the following:

  • Building console-first .NET executables with GUIs
  • Audio I/O hardware for a Raspberry Pi
  • Simple(ish) graphics and animation
  • Implementing local speech recognition (for voice activation)
  • Training an AI engine

I’ve put a lot on my plate.


Loosely borrowing from agile principles I’m planning on implementing this using small, achievable increments, or sprints. The way I envisage these are as follows:

Week 1:

Build a simple console-based chat bot, powered by a mildly-trained AI engine.

Week 2:

Train the engine to understand commands that I want my assistant to act upon.

Week 3:

Implement always-on speech recognition, to activate the chat features when a certain phrase is detected (think “Hey, Siri”).

Week 4:

Achieve a complete audio ‘conversation’ with my AI

Week 5:

Implement a simple graphical avatar (the ‘face’ of my AI) which reacts to commands with facial expressions

Week 6:

Deploy the project to a Raspberry Pi, and enjoy!

I hope that by setting these goals weekly will keep me focused on achieving this project in just 6 weeks. It may not happen that way, but that’s what I’m shooting for!

Chosen Technologies

For the AI engine, I have decided on using the power of Wit is a free bot engine which powers thousands of chat bots, and has good documentation and a large community of developers, all of which I expect to depend on throughout this project. I’m excited to get stuck in to training my AI in preparation for running my own virtual assistant.

As mentioned within my brief, I am attempting this using C#, which will run on my Raspberry Pi running Ubuntu Mate. The executable will run using the mono runtime. I’m aware that there are other languages which might be much better for the task, but I have chosen this approach for several reasons: Firstly, I am primarily a .NET developer by trade, working with Xamarin and ASP.NET, so I am most familiar within this realm. Also, assuming all goes well, I’ll be interested to know what other platforms I might be able to run this on, and with the versatility and range that C# provides thanks to mono, I won’t just be limited to a desktop application.

My decision for running this on the Pi is just a personal endeavour – I bought the Pi with no idea what I was going to use it for, and since this project holds the potential for home automation, what better IoT tool than a Raspberry Pi?

I don’t have time to reinvent the wheel, so I will be experimenting with some open source libraries. So far I’ve come across this great post on CodeProject which seems to provide an answer on how I could implement voice activation. I haven’t tried it out yet, but hopefully it’ll be a great springboard. I’ve also found several 3rd-party libraries for the API, and having looked at each of them I think I’ll be experimenting with this one, because it seems to work out-of-the-box with just a couple of lines of simple code.


Project H.E.N.R.Y

This project, and the AI, will need a name. Henceforth, I name thee HENRY. Not just because (like Siri or Cortana) it’s an easily distinguishable word for voice-activation, but also because it’s the name of the man who inspires me the most… My Father.

Get Involved

While this is a personal endeavour, I’m going to keep this project open source and available for anyone to borrow, copy, comment on or contribute to. Since I’ll be using other open source libraries to help me with this project I find it fitting to offer up this project to others, and I’m excited by the idea that others may wish to join in. Who knows, perhaps this might generate a small community! Link to the Github repo is here.

You can also follow along by following this blog or signing up for email updates – all links below.

…Off we go!