Speech Recognition on ODROID

[nshmyrev]

Hi there

I just got an ODROID C1 board, mostly using it with Kodi. In this thread I wanted to discuss speech recognition capabilities, microphones, microphone arrays and so on. Voice interfaces get more and more attention recently and believed to be one of the major shift in computer interfaces, many users get used to voice due to wide spread of smart homes etc. At the same time, machine learning algorithms are getting more and more complex and privacy concerns raise as long as data like voices are transferred to corporate endpoints.

One of my dreams are voice-managed Kodi smart TV, not there yet but getting there.

There are many open source speech recognition packages these days - DeepSpeech, Nvidia Nemo, Facebook Wav2letter, CMUSphinx. Some of them are mostly intended for research, some use outdated technology. There are many assistants too but sometimes it is hard to install them, sometimes they are not dealing with speech properly.

I'm working on open source speech recognition library Vosk https://github.com/alphacep/vosk-api:

• Vosk is an offline open source speech recognition toolkit. It enables speech recognition models for 17 languages and dialects - English, Indian English, German, French, Spanish, Portuguese, Chinese, Russian, Turkish, Vietnamese, Italian, Dutch, Catalan, Arabic, Greek, Farsi, Filipino. We might support Korean some time too.

• Vosk doesn't send the audio outside, all recognition happens on the devices

• Vosk models are small (50 Mb) but provide continuous large vocabulary transcription, zero-latency response with streaming API, reconfigurable vocabulary and speaker identification.

• Speech recognition bindings implemented for various programming languages like Python, Java, Node.JS, C#, C++ and others.

• Vosk supplies speech recognition for chatbots, smart home appliances, virtual assistants. It can also create subtitles for movies, transcription for lectures and interviews.

• Vosk scales from small devices like Raspberry Pi or Android smartphone to big clusters.

We have prebuilt packages for ARMv7 https://github.com/alphacep/vosk-api/re ... armv7l.whl, they should probably work fine on ODROID devices. Just install them with Pip.
We also have android demo https://github.com/alphacep/vosk-android-demo, it should probably work too in Android.

Please let me know if you tested any of the speech recognition software on ODROID, what device in particular, what are your thoughts and ideas. Hope to try and discuss some related topics on ODROID too - deepspeech, microphone arrays from respeaker and matrix, text-to-speech.

[L67GS]

First of all, welcome to the world of Odroid!
I've been working on voice recognition for my Candice Robot project:
viewtopic.php?f=116&t=39347

For hardware I chose a mini USB mic from Ameridroid because of the size:
https://ameridroid.com/products/usb-min ... 463f&_ss=r

The element was useable but less than sensitive so I replaced it with better ones, due to their impedance I installed two in parallel and I'm much happier with it now.

https://www.mouser.com/ProductDetail/CU ... pf9w%3D%3D

20201023_003858.jpg (1.12 MiB) Viewed 2271 times

I've been doing most of my testing and installation on a C1+, and then putting the eMMC module on Candice's C0 and running it in her case.

/////// Software ///////

1) Kaldi was a complete waste of my time, I never got any version of it running.

2) Pocketsphinx set up incredibly easy but I never worked toward training models because it takes forever to return a result.

3) Julius set up fairly simple and accuracy is pretty good out of the box, so I wrote a crude C++ function to return a string. As of now Julius is still my hopeful and I have most of a thorough dictionary assembled so I can start training a LM with HTK (HTK didn't want any part of running on a SBC for me so training will be on a big computer).

4) DeepSpeech looks promising, and assembling the Julius dictionary is beyond tedious so I'm currently trying to get Deepspeech up and running for tests to break up the monotony.

To summarize Julius and Pocketsphinx both would be adequate for keyword recognition, Julius is a better bet because of the speed. Neither of them really shines with recognition of continuous speech as strings are kind of limited in length. Pocketsphinx appears to have a handy .h file to use in C but I didn't play with it because of the slow thing. Julius has a limitation in lack of API but there is a server to interact with it, and I made it run on the C1+ so there may be a way to run that as a thread and control it from C.

Festival does the talking for Candice, and does it well though like everyone else I wish there were more natural voices that run on it.

I'm curious to try what you're working on and would love to play a part in bringing a good voice recognition to Odroid.
I've got a lead on DeepSpeech from Reddit to chase as well but I'll be looking more closely at your Vosk. I'm excited someone else is on this train, I've been a solo passenger for months now.

PS: where the hell did you get a C1?

[joerg]

@L67GS, I am with this idea of speech recognition long time now. I think I will jump on the train now .
My plan is to use a ReSpeaker Mic Array v2.0 for the microphone part. I have to order it and then the journey will start...

[L67GS]

@L67GS, I am with this idea of speech recognition long time now. I think I will jump on the train now .
My plan is to use a ReSpeaker Mic Array v2.0 for the microphone part. I have to order it and then the journey will start...

I'm thankful to hear this!
The respeaker array is nice, I considered that one, if you have better results with it I may pick one up as well but the current draw is 580ma!

I've been on Reddit in r/speechrecognition and got a lead on a couple of packages for ARM, both DeepSpeech and Vosk:
https://github.com/mozilla/DeepSpeech/r ... armv7l.whl

https://github.com/alphacep/vosk-api/re ... armv7l.whl

It's my belief that Kaldi is the best tool from what I've read and some videos but I never got past the dependencies.

[nshmyrev]

> First of all, welcome to the world of Odroid!

Thank you! It is actually an important area we want to support better.

> The element was useable but less than sensitive so I replaced it with better ones, due to their impedance I installed two in parallel and I'm much happier with it now.

It is ok for a start, in the future you would need something like Respeaker microphone array. There is an issue with respeaker though, their hardware is ok but software is not very functional, many complains about half-made software.

> Festival does the talking for Candice, and does it well though like everyone else I wish there were more natural voices that run on it.

There are newer higher quality synthesizers for mobile, like TensorflowTTS TensorSpeech, but it might need some porting . You can listen for samples here: https://drive.google.com/drive/folders/ ... DUAFXlQ_Zp

> PS: where the hell did you get a C1?

Got very old one from a friend, but, unfortunately, it looks it will be here not for a long. Probably I'll get newer Odroid board soon.

> It's my belief that Kaldi is the best tool from what I've read and some videos but I never got past the dependencies.

Vosk uses Kaldi and specifically simplifies the installation.

> My plan is to use a ReSpeaker Mic Array v2.0 for the microphone part. I have to order it and then the journey will start...

Honestly couple of people tried it and feedback is so-so, it does give you access to multiple streams for beamforming but you have to write the software yourself, only "direction of arrival" works, no much real beamforming too. Algorithms track only the single source, it doesn't work if you have multiple sources. And development is stalled on Github. I would wait a little bit and experiment with more straightforward microphone first.

[L67GS]

It is ok for a start, in the future you would need something like Respeaker microphone array. There is an issue with respeaker though, their hardware is ok but software is not very functional, many complains about half-made software.

What's the best microphone for an embedded project right now? Bearing in mind that current draw is a big thing to some of us.

[nshmyrev]

What's the best microphone for an embedded project right now? Bearing in mind that current draw is a big thing to some of us.

Honestly no good solution here. If we consider a practical situation where tv plays in the same room and two people speaking and make a command to a robot, you need many things.

In ideal case you want a microphone array like respeaker v1 and good algorithms for source tracking and direction of arrival estimation, but it requires computation resources on Odroid and very far from being easily implementable. In V2 they decided to put doa algorithms on the external chip (xmos) but that even complicated things since default algorithms on xmos are not very flexible and you can not really update them. So xmos itself is kind of useless. I would vote for https://wiki.seeedstudio.com/ReSpeaker_ ... pberry_Pi/ design, but no software still.

Commercial companies license https://www.iis.fraunhofer.de/en/ff/amm ... phone.html software, much more advanced.

Some more designs here: https://developer.amazon.com/en-US/alex ... s/dev-kits, most of them are pretty expensive.

For custom microphones frequency response at distance and angle needs serious testing. You can have issues at different angles.

You can try very directed microphone too if it fits your idea about robot, that will simplify things.

[nshmyrev]

And for simple usecase where you need just a few commands microphone is not that critical, even with a simple microphone it will work. But then the usefulness of the robot is pretty limited as well.

It is more for alexa-style usecase where you are going to ask complicated voice queries.

[L67GS]

I suppose it's more important first to get voice recognition working, then focus on either a software noise filtering solution or a fancier microphone.
I have considerable battery power but I'd prefer not to burn most of it on just the microphone.
So far this whole project is very efficient.

[joerg]

@nshmyrev
The ReSpeaker 4Mic Array for RaspberryPi looks good, but I want to use it with my C1 that has already nrf24L01 on the spi interface. As I understand this ReSpeaker needs the spi interface. I would need to make a stackable board to use them both. I want to place it in the living room, and yes there would be some noise from TV and it should recognize the spoken key words.
Btw. Thank you for all this information. I was about to buy the V2.0 and have it in the shopping car, but as it is ~60€, I must think well before to buy.

[L67GS]

I still think a cheap microphone is a better bet to get proof of concept. Then we're not battling a tricky hardware setup and software setup.
LM training happens in a quiet, controlled environment anyway. Fancy microphones are on my list, but for later.

[L67GS]

Set up vosk, portaudio, pyaudio!
Then I run a test from the /example folder:

Code: Select all

root@odroid:/vosk-api/python/example# python test_microphone.py                 
Traceback (most recent call last):                                              
  File "test_microphone.py", line 3, in <module>                                
    from vosk import Model, KaldiRecognizer                                     
ImportError: No module named vosk          

GRRRRR!!!

[nshmyrev]

Probably you need to try `python3 test_microphone.py`, default python points to python2.

Set up vosk, portaudio, pyaudio!
Then I run a test from the /example folder:

Code: Select all

root@odroid:/vosk-api/python/example# python test_microphone.py                 
Traceback (most recent call last):                                              
  File "test_microphone.py", line 3, in <module>                                
    from vosk import Model, KaldiRecognizer                                     
ImportError: No module named vosk          

GRRRRR!!!

[L67GS]

Probably you need to try `python3 test_microphone.py`, default python points to python2.

Set up vosk, portaudio, pyaudio!
Then I run a test from the /example folder:

Code: Select all

root@odroid:/vosk-api/python/example# python test_microphone.py                 
Traceback (most recent call last):                                              
  File "test_microphone.py", line 3, in <module>                                
    from vosk import Model, KaldiRecognizer                                     
ImportError: No module named vosk          

GRRRRR!!!

I'll have to try again tonight, I didn't even think of that, this last couple of months is all the experience with python I've ever had.

[nshmyrev]

Some more discussion on microphones on Rhasspy forum, some interesting ideas there:

https://community.rhasspy.org/t/best-us ... spy/663/31

[L67GS]

ImportError: libgfortran.so.3: cannot open shared object file: No such file or directory

libgfortran.so.3 is old news, we have version 5 on 20.04 and the new version of gcc compiler apparently doesn't support version 3.

python stuff is not fun, I hate it, I'm too tired after a 12 hour workday but I'll work on this a bit more tomorrow. I'm moments away from just being done with python programs, beginning to think I had the right Idea when I stopped wasting my time with kaldi the first time. Julius is likely the correct path for me, not as accurate but it works.

[nshmyrev]

Unfortunately python is the language of modern AI, you have to use it anyway. It has some drawbacks but in general this gfortran thing is not a python fault. It is dockcross which sucks

Let me look how to solve this coming days and I'll get back to you.

[nshmyrev]

And Ubuntu too, removing old libraries is not a great idea. So many complains about libgfortran.

[L67GS]

So we have the alias command for making custom command links of sorts, but I can't see that working with libraries like alias libgfortran.so.3= libgfortran.so.5.
Is there a way to redirect so when a program checks for or reaches for a library it finds the new version?

[L67GS]

I'm going to flash 18.04 minimal tonight and give this another go, a shame since everything else is working so well on 20.04.

[L67GS]

I just learned AMD64 Vosk installs libfortran within the pip install. So here's my though process:
Install an early gcc
do a temporary alias to make it = gcc
Pip3 install vosk

????????

[nshmyrev]

I just learned AMD64 Vosk installs libfortran within the pip install. So here's my though process:
Install an early gcc
do a temporary alias to make it = gcc
Pip3 install vosk

????????

You don't need gcc for pip. You can probably install aarch64 pip directly if you have 64-bit system.

18.04 should also work with libgfortran3 package.

[L67GS]

The last few days & nights have been hectic for me, tonight after work I have to run a service call after hours too but I'm going to try to get something done on this. Since Vosk setup it quick maybe I'll pop 18.04 onto an eMMC and try it.

[L67GS]

Progress, but not success!
I flashed 18.04 and did the following steps, vosk, portaudio, pyaudio, libfortran.so.3 all set up.
** My decision to include every step is not to insult any experienced user, but rather to help novice users that find this later **

Code: Select all

Vosk installation:
apt update 
apt-get install python3
apt-get install python3-pip
pip3 install https://github.com/alphacep/vosk-api/releases/download/0.3.15/vosk-0.3.15-cp36-cp36m-linux_armv7l.whl
 wget http://files.portaudio.com/archives/pa_stable_v190600_20161030.tgz
tar -xf pa_stable_v190600_20161030.tgz
rm pa_stable_v190600_20161030.tgz
cd portaudio
./configure
make
make install
pip install pyaudio
apt-get install git
 git clone https://github.com/alphacep/vosk-api
cd vosk-api/python/example
wget https://alphacephei.com/kaldi/models/vosk-model-small-en-us-0.15.zip
apt-get install libgfortran3

When I run python3 ./test_simple.py test.wav I get:

Code: Select all

root@odroid:/vosk-api/python/example# python3 ./test_simple.py test.wav         
Illegal instruction  

A quick search online gives me the github issues where this has been seen before, it looks like a fix was released for ARM6, but not ARM7.

https://github.com/alphacep/vosk-api/issues/48

Getting closer ....

[L67GS]

Changed gears and decided to try DeepSpeech on 18.04 since vosk is on pause:

Code: Select all

root@odroid:/# pip3 install https://github.com/mozilla/DeepSpeech/releases/download/v0.9.3/deepspeech-0.9.3-cp37-cp37m-linux_armv7l.whl
deepspeech-0.9.3-cp37-cp37m-linux_armv7l.whl is not a supported wheel on this platform.

[nshmyrev]

Code: Select all

root@odroid:/vosk-api/python/example# python3 ./test_simple.py test.wav         
Illegal instruction  

Looks strange. Can you figure out what instruction breaks things:

Code: Select all

gdb --args python3 ./test_simple.py test.wav

then type

Code: Select all

run

once it stops with SIGILL type

Code: Select all

disassemble

and provide the output

[L67GS]

Looks strange. Can you figure out what instruction breaks things:

Code: Select all

gdb --args python3 ./test_simple.py test.wav

then type

Code: Select all

run

once it stops with SIGILL type

Code: Select all

disassemble

and provide the output

Is this what you want? disassemble doesn't give anything but "No function contains program counter for selected frame."

Code: Select all

root@odroid:/vosk-api/python/example# gdb --args python3 ./test_simple.py test.wa
GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.                                                                          
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>                                              
This is free software: you are free to change and redistribute it.                                                         
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"                                                 
and "show warranty" for details.                                                                                           
This GDB was configured as "arm-linux-gnueabihf".                                                                          
Type "show configuration" for configuration details.                                                                       
For bug reporting instructions, please see:                                                                                
<http://www.gnu.org/software/gdb/bugs/>.                                                                                   
Find the GDB manual and other documentation resources online at:                                                           
<http://www.gnu.org/software/gdb/documentation/>.                                                                          
For help, type "help".                                                                                                     
Type "apropos word" to search for commands related to "word"...                                                            
Reading symbols from python3...(no debugging symbols found)...done.                                                        
(gdb) run                                                                                                                  
Starting program: /usr/bin/python3 ./test_simple.py test.wav                                                               
Cannot parse expression `.L1207 4@r4'.                                                                                     
warning: Probes-based dynamic linker interface failed.                                                                     
Reverting to original interface.                                                                                           
                                                                                                                           
[Thread debugging using libthread_db enabled]                                                                              
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".                                              
                                                                                                                           
Program received signal SIGILL, Illegal instruction.                                                                       
0xb6673098 in ?? ()                                                                                                        
   from /usr/local/lib/python3.6/dist-packages/vosk/_vosk.cpython-36m-arm-linux-gnueabihf.so                               
(gdb) disassemble                                                                                                          
No function contains program counter for selected frame.                                                                   
(gdb) 

[nshmyrev]

Hm, looks strange. How about

disassemble $pc

what does it output?

[L67GS]

Hm, looks strange. How about

disassemble $pc

what does it output?

Code: Select all

avot@odroid:/vosk-api/python/example# gdb --args python3 ./test_simple.py test.a
GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git                                
Copyright (C) 2018 Free Software Foundation, Inc.                               
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>   
This is free software: you are free to change and redistribute it.              
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"      
and "show warranty" for details.                                                
This GDB was configured as "arm-linux-gnueabihf".                               
Type "show configuration" for configuration details.                            
For bug reporting instructions, please see:                                     
<http://www.gnu.org/software/gdb/bugs/>.                                        
Find the GDB manual and other documentation resources online at:                
<http://www.gnu.org/software/gdb/documentation/>.                               
For help, type "help".                                                          
Type "apropos word" to search for commands related to "word"...                 
Reading symbols from python3...(no debugging symbols found)...done.             
(gdb) run                                                                       
Starting program: /usr/bin/python3 ./test_simple.py test.wav                    
Cannot parse expression `.L1207 4@r4'.                                          
warning: Probes-based dynamic linker interface failed.                          
Reverting to original interface.                                                
                                                                                
[Thread debugging using libthread_db enabled]                                   
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".   
                                                                                
Program received signal SIGILL, Illegal instruction.                            
0xb6673098 in ?? ()                                                             
   from /usr/local/lib/python3.6/dist-packages/vosk/_vosk.cpython-36m-arm-linuxo
(gdb)  disassemble $pc                                                          
No function contains specified address.

[nshmyrev]

x/i $pc

please

[L67GS]

x/i $pc

please

Code: Select all

(gdb) x/i $pc                                                                   
No registers.  

[nshmyrev]

x/i $pc

after you run the program like above, not just in plain gdb

[L67GS]

x/i $pc

after you run the program like above, not just in plain gdb

I did, sorry I didn't specify but I just posted that part

[L67GS]

I get it now, apologies:

Code: Select all

(gdb) disassemble x/i $pc                                                       
No symbol "x" in current context.

[L67GS]

I'm missing something, I uninstalled python 3,6, then installed Python 3.7, then installed pip3.

But I still get this:

Code: Select all

root@odroid:~# python3 --version
Python 3.6.9                                                                    

and of course DeepSpeech won't install with python 3.6:

Code: Select all

root@odroid:~# pip3 install https://github.com/mozilla/DeepSpeech/releases/downl
oad/v0.9.3/deepspeech-0.9.3-cp37-cp37m-linux_armv7l.whl                         
deepspeech-0.9.3-cp37-cp37m-linux_armv7l.whl is not a supported wheel on this pl
atform.       

What am I missing to get Ubuntu to report and use Python 3.7?
installed version is python 3.7.5-2~18.04

[L67GS]

I got it with alias, still a problem

Code: Select all

root@odroid:~# python3 --version                                                
Python 3.7.5                                                                    
root@odroid:~# pip3 install https://github.com/mozilla/DeepSpeech/releases/downl
oad/v0.9.3/deepspeech-0.9.3-cp37-cp37m-linux_armv7l.whl                         
deepspeech-0.9.3-cp37-cp37m-linux_armv7l.whl is not a supported wheel on this pl
atform.          

[nshmyrev]

@L67GS sorry, do you try this on C1 or on XU4? You have XU4 to try, right?

[L67GS]

@L67GS sorry, do you try this on C1 or on XU4? You have XU4 to try, right?

I'm on a C1+, I do have aa XU4 but it's on another project right now.

[L67GS]

@L67GS sorry, do you try this on C1 or on XU4? You have XU4 to try, right?

The C0, C1, and C1+ use the same OS image so your C1 will behave the same.

This is what I'm currently using:
https://odroid.in/ubuntu_18.04lts/C0_C1 ... 923.img.xz

Etcher:
https://www.balena.io/etcher/

I'd like to get speech recognition running on this platform, then work on the XU4 so the community has both.

[nshmyrev]

Any chance you can give an ssh access to any of this? Its possible to forward ssh port.

[L67GS]

Yes, that's how I access the Odroid

[L67GS]

Back to work on Julius assembling a good dictionary of words for the voca file. The file should be done today, then I'll introduce it one letter at a time to make it easier to chase errors.

[L67GS]

I got the dictionary finished last night! It started with 265,353 lines, a friend of mine does a lot of spreadsheet work so he stripped out a whole column for me, then I dropped the whole thing to lower case and thinned nonsense and odd words one at a time.
The new size is 28,943 words and I'm concerned it may return strings slowly with such a dictionary.
Tonight I'll start introducing it to Julius and I know some of the triphones will conflict so I'll just extract those words as they conflict and put them in a file to match supported triphones later, I'm expecting at least 2000 words to fault.
I'm expecting that step to take 2 evenings so Julius should be ready for tests with a comprehensive dictionary in a few days.

[L67GS]

Wow! I had 2 errors total and while they took some work to find they were quick fixes.
My whole dictionary is integrated now and returning results almost instantly despite the size of the dictionary (I was REALLY sweating that one).
I also found a .h file for juliuslib, the Julius server so there's clearly some more work to do but I'm going to write a parrot program to run on the C1+ and C0 (Candice robot) for further development.

[L67GS]

What are the chances of getting one of these little 2 mic arrays to work?
There's a Linux driver for rpi4.
https://www.seeedstudio.com/ReSpeaker-2 ... i-HAT.html

[joerg]

I have ordered the Respeaker 4 Mic for Rpi and I am waiting for it. But I am so overloaded with other work, I don't know when I can start with it.

[L67GS]

The PlayStation eye camera is listed in that link above and I read a comment about it being fairly small when disassembled. It's a 4 microphone array for $22 .USD.
It's a favorite of many bot builders.

Edit:
Oh my, that's very small once the warranty is properly voided:

https://www.behance.net/gallery/8955421 ... Eye-Camera

[nshmyrev]

Btw, we have just released Vosk 0.3.16 https://github.com/alphacep/vosk-api/re ... tag/0.3.16

No libgfortran dependency anymore, you can try it on any recent system.

It is also 30% faster.

For install just `pip3 install -U vosk`

[L67GS]

Btw, we have just released Vosk 0.3.16 https://github.com/alphacep/vosk-api/re ... tag/0.3.16

No libgfortran dependency anymore, you can try it on any recent system.

It is also 30% faster.

For install just `pip3 install -U vosk`

Hell yes, I'll try it! Tonight I'm pretty burned out from reading Julius docs so, tomorrow.

I read through those packages you suggested on Reddit too, very interesting.

[n2fan]

is it possible to use Odroid N2?

odroid@odroid:~/vosk$ uname -m
aarch64

trying to install:

odroid@odroid:~/vosk$ pip3 install vosk-0.3.16-cp38-cp38-linux_aarch64.whl
vosk-0.3.16-cp38-cp38-linux_aarch64.whl is not a supported wheel on this platform.

odroid@odroid:~/vosk$ pip3 install -U vosk
Collecting vosk
Could not find a version that satisfies the requirement vosk (from versions: )
No matching distribution found for vosk
odroid@odroid:~/vosk$

thanks!