Pocketsphinx. Here is a demo https://www.youtube.com/watch?v=tfcme7maygw and a blog post about how to do it: https://scaryreasoner.wordpress.com/2016/05/14/speech-recogn... Code is in here (gpl2) https://github.com/smcameron/space-nerds-in-space In particular, look at snis_nl.h, snis_nl.c

The trick with pocketsphinx is to limit the vocabulary you want to recognize, and create a corpus of the types of things you want to be able to recognize and feed it through here: http://www.speech.cs.cmu.edu/tools/lmtool-new.html

If you try to use pocketsphinx to recognize arbitrary English (e.g. dictation) it's not going to work very well in my experience.

Two realistic options, one is pocketsphinx, the other Kaldi. When running on a Pi, pocketsphinx will be your only realistic option for realtime detection. You'll want to move to a RaspPi 3 as well, and you'll want to use a customized dictionary to try and get your recognition speed up. Lastly, there are several parameters you can tweak that'll affect recognition speed.

Raw processing power will be the bottleneck on a Raspberry Pi.