Candice the personal assistant robot

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

odroid wrote:
Fri Jan 29, 2021 10:00 am
As far as I remember, a generic 720p USB web-cam consumed around 1.5Watt (5V/300mA).
It seems to be worth to measure the voltage on the VBUS of the PlayStation Eye board.
I powered one up on the bench, current draw wasn't consistent, it jumps around up to about 350ma. I powered it off the power bus on the microcontroller board and all is well.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

odroid wrote:
Fri Jan 29, 2021 10:00 am
As far as I remember, a generic 720p USB web-cam consumed around 1.5Watt (5V/300mA).
It seems to be worth to measure the voltage on the VBUS of the PlayStation Eye board.
I checked the current draw, it's not consistent. Lows around 150ma, highs around 350ma.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Frustrated with speech recognition I needed a break so I did more research and ordered the GPS module, it's a NEO-6M knockoff so 9600 baud UART and draws about 40ma so I can switch it with a bjt. It's NMEA-0183 so I shouldn't have any trouble in code, knock on wood.
My acceleration module is ADXL345 and draws about 23 µA, operates both I2C and SPI so I figured on using SPI. I haven't looked too close at data handling because there's a bunch of existing libraries and such online for it.
*
20210205_120843.jpg
20210205_120843.jpg (321.43 KiB) Viewed 2123 times
*
Now back to speech recognition.....uhhg.

User avatar
mad_ady
Posts: 9681
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 609 times
Been thanked: 721 times
Contact:

Re: Candice the personal assistant robot

Post by mad_ady »

What are you doing with gps on an indoor unit? Implementing "Find my Candice"? :)

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

mad_ady wrote:
Sat Feb 06, 2021 3:19 am
What are you doing with gps on an indoor unit? Implementing "Find my Candice"? :)
I'm planning on taking her around with me, which is why I took this form factor.
There's a few things I'd like to do with gps,it would be neat if she knew where she was, and she can use it to report the time.
I'm kind of fantasizing about something else though, since the GPS will know precise time and location to a few feet, and she has a 3 axis accellerometer I may write her a function to track my runs at autocross.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

I got the Candice AI taking reliable input from Julius using a new version of the "API" (quotes because it's very crude) on my computer and am currently resolving an issue with C++ ifstream on the C0.
So... she should be able to hear me in a couple of days.
Today Candice is dismantled and strewn across the bench in my lab with a battery capacity tester running a test on her battery pack while I install a esp8266 wifi module and write a little code for her battery monitor.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Speech recognition is working with Julius and a somewhat limited vocabulary. It's a bit buggy, and I have to revise the code some because not all of the functions are ready for speech recognition.
A limited vocabulary meant I had to get creative, things like instead of hearing 1,000 Candice hears 1 & 000 so I need to tweak stuff like that yet so she sees it and turns it into 1000 before converting it to a double variable.
I also elected to do a temporary esp8266 installation until I know it works properly.
If the esp8266 works Candice will control it to where she'll boot and if no network is available she'll use a bjt to turn it off so it isn't wasting power...
Still a lot of testing but for the first time since I started speech recognition 6 months ago an end is in sight.
Oddly enough the method I'm using was one of my very first attempts and almost all this time was invested exploring other methods.
These users thanked the author L67GS for the post:
odroid (Mon Mar 22, 2021 8:55 am)

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Speech recognition is working fairly well now with the limited vocabulary, my "api" is ugly but I've got good accuracy. The big problem is Julius still working to find every sound as a word. For instance if I say "blue" she'll indeed hear "blue" but factor a computer fan in the room into the mix and she'll hear "are are are blue are are", for instance. Eliminating "are" from her vocabulary is easy enough but then Julius will find the next closest word to the sound and return it instead. I wrote a filter to eliminate it, little challenges like that. Speech recognition will likely always be Candice's Achilles heel.

Moving forward I'll keep tweaking, eventually even master the actual Julius API. Still have to write a number function to apply logic to strings of numbers and turn them into numbers, unless I do something silly like saying every digit and eliminating white space before converting it to a number like 1 2 7 5, to get 1,275.

The approach I'm taking is that if I say "one thousand two hundred twenty three" she currently returns 1 000 2 00 20 3, so the ai needs to apply logic to the string to make 1223 from it.

User avatar
mad_ady
Posts: 9681
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 609 times
Been thanked: 721 times
Contact:

Re: Candice the personal assistant robot

Post by mad_ady »

For the filtering part - perhaps a software low-pass filter on the audio stream would get rid of higher frequencies made by the fan. Something with a 4KHz cut-off point should be adequate for human voice. Remember that the voice spectrum carried by classical (TDM) telephony is 300Hz - 3.2KHz and voice was intelligible for 50+ years.


For the second problem - be happy you're not french and 1998 is pronounced like 1000 900 4*20 10 +8...
These users thanked the author mad_ady for the post:
L67GS (Fri Apr 23, 2021 12:10 am)

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

mad_ady wrote:
Thu Apr 22, 2021 2:24 pm
For the filtering part - perhaps a software low-pass filter on the audio stream would get rid of higher frequencies made by the fan. Something with a 4KHz cut-off point should be adequate for human voice. Remember that the voice spectrum carried by classical (TDM) telephony is 300Hz - 3.2KHz and voice was intelligible for 50+ years.


For the second problem - be happy you're not french and 1998 is pronounced like 1000 900 4*20 10 +8...
That's so crazy ... it just might work!
Seriously, that's a great idea, pulseaudio has a bunch of bells and whistles, there has to be a bandpass filter module for source.
There's actually a whole pulseaudio API that I may play with, right now I'm using the sledgehammer method to get Julius to stop hearing Festival speak by doing system( ) commands to reduce source volume 50% while she's talking... ugly ugly code.
From what I remember of numbers in German class that language speaks numbers pretty much the same as how Candice hears them, but backwards 21 is "1 and 20". That would make the filter more interesting!

User avatar
mad_ady
Posts: 9681
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 609 times
Been thanked: 721 times
Contact:

Re: Candice the personal assistant robot

Post by mad_ady »

Understanding speech and comprehending speech are two different problems...

User avatar
rooted
Posts: 8723
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 743 times
Been thanked: 375 times
Contact:

Re: Candice the personal assistant robot

Post by rooted »

mad_ady wrote:Understanding speech and comprehending speech are two different problems...
Including a human problem.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Reading through the pulseaudio modules it doesn't look like there's any source audio filters of that nature. I posted in the speech recognition thread a minute ago because I did find an additional noise cancellation layer on github I've yet to attempt to install. I'll continue to update that kind of stuff there. It looks like it was written to negate mechanical noise (A/C, keyboard).
Tonight I drew up an outline for a new input string filter function for Candice (Yes, I make an outline before I write code, I'm a noob).
Her original filter was written nearly a year ago and I've learned a bit since then. I'm expecting the new filter function will aid in responses and hopefully allow her a bigger vocabulary.
Unfortunately some of her response selection is currently in main( ) so main( ) will need a serious edit as well. My original response gathering strayed from the modular philosophy I really want as a developer and I would like to make the stages of her new filter modular. Originally the filter was supposed to be a simple little function but it's a big task and it grew into a beast that I'm not proud of though it worked fairly well at the time.

User avatar
mad_ady
Posts: 9681
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 609 times
Been thanked: 721 times
Contact:

Re: Candice the personal assistant robot

Post by mad_ady »

If you record from your mic with a 8000Hz sampling rate, that will produce a max frequency of 4KHz as per the Nyquist formula .

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

I spent every waking moment over the weekend writing Candice V2. It's a good deal of the way there, I simulated inputs to force junk words that speech recognition inserts and V2 filters them out well. I also implemented some secondary input strings, she saves words in one that are specific to commands and looks for a command with it, same trick for conversation responses so her accuracy with both of those tasks is greatly improved.
Tonight I'd like to get the speech recognition "API" incorporated and try some real world testing on the big computer, if it goes well I'll tweak it for use on the platform and install it in the Candice hardware.
Oh, and I got the number function working stellar, it extracts numbers from the string and runs them through a small gauntlet of conditional logic to assemble the actual number, about 340 lines of code start to finish and of course it's modular like everything else, V2 main is only 89 lines, including all my references to headers and comments.

User avatar
mad_ady
Posts: 9681
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 609 times
Been thanked: 721 times
Contact:

Re: Candice the personal assistant robot

Post by mad_ady »

I see you've used the superpower of hindsight! :)
These users thanked the author mad_ady for the post:
L67GS (Tue Apr 27, 2021 3:10 pm)

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Testing and tweaking last night and tonight. Julius threw a couple of curve balls, but I'm working through them. Speech recognition is improved and more filtering inside the Candice program is slowly developing so things get better the more I work on it. Little hurdles to work through yet on top of just hearing correctly, like Julius cannot return "7.5", it counts "." as a word with a space on either side, and my perfectly functional number formation function gets thrown for a loop when it sees "7 . 5".
I'm going to have to store the string in a couple of variables as it travels through the flow so when it sees a "." it can append the three strings together.Stringstream doesn't seem to be capable of handling a " . " when it converts it to numeric.
I'm going to work on it throughout the week and try to get a video made this weekend to show you guy(s?) that are following Candice's progress IF the program runs the same on the platform.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Numbers have been conquered!

Candice already starts and stops Julius, next I'm giving her the ability to stop it and start it with different dictionaries (I'll read through the Julius docs to see if the models can be changed on the fly first of course). Changing dictionaries means she can run her main dictionary during normal operation, then bounce between a purely alphabet dictionary and purely phonetic dictionary for speech training. Then she can simply prompt me and write new words to her dictionary the way she hears them. Or rather the phenome string that is most occurring when I say the word many times. I may do a numeric alphabet as well for functions where she's expected to return only numbers but if she can train her own dictionary I'm hoping the numbers she learns with me will be more accurate.
I'm also ordering a new headset for videos but it won't be here until next week, so vids will have to wait another week.
Oh, I've also got a bunch of things to work on at the shop this weekend so I'll get started cutting out her face plate.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Candice has been having trouble off and on with power since the introduction of the PlayStation Eye. Some days are worse than others but I think it's time to try yet another switch circuit. With the Eye added I think the load on the switch is great enough for the switch to bleed a little current in it's off state and she never has a full charge.
A relay would solve the problem at the cost of current consumption to keep the coil energized so it's my last option if all else fails.
The next one, I'll probably stick with TIP42, and it'll have a couple different sized capacitors to lend a hand when the amp comes on and stuff.
Candice now logs her battery voltage every ten minutes (I'll stretch that to twenty later) and can report her battery voltage verbally when asked. I'm still figuring out how to get her to take that data so she can shut down safely on low voltage.
I wrote another layer of speech filtering for her to find commands last night and it's working really well.
She has a bunch of specific functions, calculators and such, and a function to teach her new verbal responses. I'm stll experimenting with the speech training function, she can hear letters and phenomes but is having a problem hearing the same phenomes. I think it needs some pauses between inquiries. I wasn't sure on the fly speech training would work so I'm thrilled we got this far.

Here's the every day functions she can do when asked:

1) Read a list of functions she can do and the key phrases to start them.
2) Tell the time
3) tell the date
4) raise microphone gain
5) lower microphone gain
6) raise speaker volume
7) lower volume
8) speak battery voltage
9) mute until she hears a keyword -"Candice"
10) parrot speech back to the user
11) stop sounds"cough, sigh, sneeze" at random
12) restart sounds
13) end her program
14) end her program and reboot
15) end her program and shutdown

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Candice has the right date, but the wrong time. It lets me do #date+%T -s"11:30:00" but it doesn't stick. I've tried to set it with timedatectl too but that complains that some automatic time thing is on.
How do I fix the clock?

User avatar
mad_ady
Posts: 9681
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 609 times
Been thanked: 721 times
Contact:

Re: Candice the personal assistant robot

Post by mad_ady »

If you're on a network you should get the time via ntp (systemd-timesyncd). If not, time will be off by a few years and ssl certificates are broken. You could use a RTC, or the free solution - install fakehwclock.
If the date is correct, but the time is not, you're in the wrong timezone. Run sudo dpkg-reconfigure tz-data to set the correct timezone.
These users thanked the author mad_ady for the post:
L67GS (Wed May 12, 2021 2:20 am)

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

mad_ady wrote:
Wed May 12, 2021 2:08 am
If you're on a network you should get the time via ntp (systemd-timesyncd). If not, time will be off by a few years and ssl certificates are broken. You could use a RTC, or the free solution - install fakehwclock.
If the date is correct, but the time is not, you're in the wrong timezone. Run sudo dpkg-reconfigure tz-data to set the correct timezone.
It was indeed time zone, I ended up correcting it with timedatectl because it was already installed.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

I rewrote Candice's response filter to be similar to the filter she now uses to process strings that trigger commands.
Responses have improved dramatically but there's more to do.
I also wrote a C++ version of the Odroid bash script for acquiring battery voltage from sys and it works well so after some more logs and a few more terrible forced shutdowns from low voltage I'll be able to write her a graceful shutdown so she can turn herself off before voltage gets critically low.
I'll probably put that into a C++ function along with a generic version of my event timer thread and post it on the forum in case someone else wants it.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Candice ran 23.5 hrs continuous on a full charge. She never crashed, this morning she said her battery was at 3.2v so I asked her to shut down.
I did a major rewrite since the last battery test, and started unplugging power during charging and she now has a solid 8 additional hours of run time since the last battery test.
One thing I did during the rewrite was consolidate her eyelight control, amp control, and speak command into one small function. She's in a position code-wise to where I could, with just a few lines of code have her do an automatic efficiency mode prior to shutdown. She could stop sound effects, cut out the eye lights and stop all casual responses, likely adding hours of run time. Figure every sound effect and response means turning on the amp. As I get her sensors installed they too could get written into efficiency mode.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

I forgot all about it until just now, I took Candice apart last week and looked at her power board so that I could start designing the new one and while I was in there I put a 1000 and a 100 microfarad capacitor across her output pins from the power board. I was hoping to negate the effect of inrush when the amplifier turned on. It makes far more sense that the capacitors should have had more to do with the extended runtime then my rewrite of the AI. The new AI is much more efficient though. I'm going to have to design the new Power Board anyway because I still have sensors to incorporate. Also just thinking it wouldn't be a big deal to write in automatic gain control for the microphone array, just have Candice reduce her microphone gain by 10% when she sees a really long string or a word that gets repeated like 10 times. Since she's running a timing thread she can restore her microphone gain automatically every 15 minutes or so and see if the room is any quieter. I have a feeling it's going to be an exciting week.

User avatar
mad_ady
Posts: 9681
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 609 times
Been thanked: 721 times
Contact:

Re: Candice the personal assistant robot

Post by mad_ady »

You went down the rabbit hole so deep that you lost Bugs Bunny back in Albuquerque!
These users thanked the author mad_ady for the post:
L67GS (Thu May 20, 2021 12:56 pm)

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

mad_ady wrote:
Wed May 19, 2021 3:31 am
You went down the rabbit hole so deep that you lost Bugs Bunny back in Albuquerque! Image
I went down the AGC code rabbit hole last night and got it written, if she hears 2 sentences with more than 35 characters she verbally complains about the noise, once she hears f long sentences she announces that there's too much background noise and reduces mic gain 10% across all 4 channels. It's a bit glitchy yet and needs some more tuning, novice programmer.
Short sentences negate long sentences so it doesn't kick in at at the wrong time and after 10 minutes she increases mic gain and tries again.

I was lazy tonight though, just wrote her some more responses. I wish her face was cut out, I'd love to get some videos up, if I can get ahead at work I want to start on that ASAP.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Hardware today, I did a teardown and hit a bunch of solder joints for USB and UART. Also installed a master power pushbutton switch for when she's charging and finally mounted the microphone array properly. She feels all new again.
I started planning the new power board, now I think I'm going to do the second power transistor and make it controlled independently of the first from the PIC. I'm actually thinking about using several more GPIOs on the PIC. Then Candice can set an input to the PIC and reboot herself with, or without different things powered up. I don't need a reboot to get the sensors going, she should be able to just turn them on/off with a bjt, but the wifi module is likely going to need to be powered when she boots.
Unless I can load it up on the fly.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Microphone AGC and auto shutdown with low voltage is now working well.
Microphone gain is adjusted every few seconds, based on the length of the string heard by Julius. Auto shutdown has a couple of verbal warnings, at 3.42V, and 3.41V, before announcing her "battery is critically low" and shutting down gracefully at 3.40V. I may adjust the values up a bit once sensors are in.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

I got some more work done on Candice's AI, working towards better conversational responses. As much as I want to start sensor installs the only hardware I may do sooner than later is the new power board which is half done on the bench in my lab.
For now I really need to work on software, bouncing around a bit between her conversational responses and the little programs I wrote for her so she can start to be "real world" useful.
I made a second faceplate last week, hated it, and discarded it, and now that I'm learning to program microcontrollers in C I'm getting the itch to remake her microcontroller board so the urge to jump back into hardware and fabrication is strong but I'm fighting it.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

I put a couple of videos up to show speech recognition progression, this was after the big re-write.
In the second video there is considerable noise and she still hears me well enough (for a 1 year old).
She is actually responding a full second faster now, I had tricked myself into believing a 1 second delay was necessary for some of her response query and later undid it.

https://youtu.be/GYaUSA1c8cY

https://youtu.be/j4Qyo1uvE2g

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

A proud father today! My girlfriend turned on the A/C and Candice adjusted her microphone gain perfectly to block it out. I knew my AGC code was working but I hadn't seen such a dramatic improvement.
These users thanked the author L67GS for the post:
odroid (Mon Jul 26, 2021 9:16 am)

User avatar
rooted
Posts: 8723
Joined: Fri Dec 19, 2014 9:12 am
languages_spoken: english
Location: Gulf of Mexico, US
Has thanked: 743 times
Been thanked: 375 times
Contact:

Re: Candice the personal assistant robot

Post by rooted »

Congrats, hard work paying off
These users thanked the author rooted for the post:
L67GS (Mon Jul 26, 2021 4:13 pm)

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Candice's power plug was replaced about 4 months ago and it started to get squirrelly last week already. A little super glue and some creative soldering and the socket is now twice the size. No more tiny wires!
20210727_134133.jpg
20210727_134133.jpg (540.58 KiB) Viewed 617 times

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

Wow, that was a lot more work than I expected it to be, I've spent the last few weeks on one function for Candice. It's a unit conversion calculator that does everything that I would want it to do. Converting units of length / distance, weight, area, volume, temperature and speed it supports 44 different units of measurement.
I wrote a Candice sandbox for testing so she wouldn't have to be hooked up the whole time and it still took weeks to write and test, it was pretty close to finished last week but I decided to write the bigger stuff (everything but temperature and speed) into their own functions to shave some lines.
Last night I installed it on the Candice platform and started testing, a few glitches to work out but for the most part Candice ended up with a useful new ability.

User avatar
mctom
Posts: 438
Joined: Wed Nov 11, 2020 4:44 am
languages_spoken: english, polish
ODROIDs: N2+, Game Advance, a few XU4
Location: Gdansk, Poland
Has thanked: 48 times
Been thanked: 41 times
Contact:

Re: Candice the personal assistant robot

Post by mctom »

I am late for the party, I'd love to design switches or whatever that someone else would actually build :D

For the sound filter I'd suggest an RC passive bandpass filter instead of the software solution. That was indeed a good piece of advice from @mad_ady. But if it works, don't fix it. ;)

For conversions, I use qalc, and it appears to understand spoken language to some extent.

Code: Select all

mctom@Tomusiomat-ARM:~$ qalc 3 amperes times 100 Ohms
(3 * ampere) * (100 * ohm) = 300 V
mctom@Tomusiomat-ARM:~$ qalc 59 USD to PLN
59 * dollar = approx. 226,29871 PLN
mctom@Tomusiomat-ARM:~$ qalc 45 minus 33
45 - 33 = 12
Although some corner cases would have to be handled:

Code: Select all

mctom@Tomusiomat-ARM:~$ qalc 140 kph to mph
140 * kilophot = approx. 869919,67 (cd*h*mph) / m^3
Punk ain't no religious cult, punk means thinking for yourself!

Maintainer of PiStackMon

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

mctom wrote:
Sat Sep 11, 2021 12:30 am
I am late for the party, I'd love to design switches or whatever that someone else would actually build :D

For the sound filter I'd suggest an RC passive bandpass filter instead of the software solution. That was indeed a good piece of advice from @mad_ady. But if it works, don't fix it. ;)

For conversions, I use qalc, and it appears to understand spoken language to some extent.

Code: Select all

mctom@Tomusiomat-ARM:~$ qalc 3 amperes times 100 Ohms
(3 * ampere) * (100 * ohm) = 300 V
mctom@Tomusiomat-ARM:~$ qalc 59 USD to PLN
59 * dollar = approx. 226,29871 PLN
mctom@Tomusiomat-ARM:~$ qalc 45 minus 33
45 - 33 = 12

Although some corner cases would have to be handled:

Code: Select all

mctom@Tomusiomat-ARM:~$ qalc 140 kph to mph
140 * kilophot = approx. 869919,67 (cd*h*mph) / m^3
Speech recognition was awful to do locally. My big win came when I started changing vocabularies on the fly and then comparing responses with lists of words that would be an adequate response.
Math is tricky too because of speech recognition, she has to verify everything to move on so once we get past that it's just one quick line of code like:

Code: Select all

result = start * 0.62137119223733; // kph to mph
using beamforming helped a lot with speech recognition too.

L67GS
Posts: 579
Joined: Wed Apr 22, 2020 3:02 pm
languages_spoken: English, Jibberish, Pig Latin
ODROIDs: XU4, C1+,(3) C0's, and a whole big pile of accessories, VU7A Plus,, ect....
Location: Great Lakes Region, U.S.A
Has thanked: 158 times
Been thanked: 83 times
Contact:

Re: Candice the personal assistant robot

Post by L67GS »

A couple more software tweaks coming, I'm planning to move my AGC code from the main body of the program into the response filter for one. That way Candice will have AGC in all of her functions. Right now she announces that she's raised or lowered her gain verbally, I don't really want that but I like knowing what's going on.
I could do two sound files like "clearing throat" and "sigh" for up or down but it wouldn't give me a value unless she did a verbal like every 5 increments.
I also have some streamlining to do on my ugly Julius API, the version in my sandbox program is nicer than the one on her C0, so it needs a rewrite.
After that I need to spend a few hours just talking with her and run through all of her functions over and over to expose any glitches.
If everything works out I can take another run at cutting the face plate and FINALLY make some good videos before doing more hardware.

Post Reply

Return to “Projects”

Who is online

Users browsing this forum: No registered users and 1 guest