New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bounty: Implement noise cancellation on RPi-3 based hardware devices (Mark 1 and Picroft) #1478
Comments
I'm going to take a look at this shortly |
Info that may be helpful re: OSS DSP: |
More helpful information: |
Some hopefully useful links about the pulse module: |
@KathyReid @penrods |
I believe it was tried a couple of years ago but the cpu strain was quite high. (This is what I've heard so no personal experience on the Pi). The pulse audio echo cancellation works great on my workstation so it'd be cool if it could work on the Pi as well. If it's too intensive on the hardware maybe there are tweaks that can be made. Give it a try, and see what the result is! |
I've worked on projects using a Broadcom chipset not unlike that of the BCM2837 (which is used in RPi3) and we'd seen good success using the Opus echo canceler. It does take CPU to do, but it wasn't particularly bad. Unfortunately, I don't have the necessary free time to contribute to the bounty hunt, but I thought perhaps suggesting this would help someone else. |
Thanks for your feedback, @roadriverrail - great suggestion! |
Potentially interesting: |
@forslund, When working on a workstation with the mycroft source, does pulse echo cancellation get loaded automatically or do we have to do that ourselves? Do USB speakerphone devices such as the Jabra 410 (popular in the forums) do echo cancellation? I'm using one with a RPi 3B+ and "Hey Mycroft, stop" seems to work. Although, I'm not sure if it works "well" at "normal volume". |
Currently, no distros load the pulse echo cancellation (that I know of). |
@tlc as @el-tocino states the echo cancellation isn't loaded by default. Loading it creates a virtual microphone that you need to set as default to use with mycroft. (basically selecting it in the pulse audio volume control) |
How are we all going with this one - any questions? Any information we could provide to help? |
Not my work, but just ran into it; https://github.com/voice-engine/ec Looks interesting and ticking the boxes. |
I have experimented with voice-engine/ec (which is basically a wrapper for speex) and PulseAudio's echo-cancel module (you have to install PA 7.1 from the Debian-Jessie-Backports for that) using algorithms "webrtc" and "speex" (adrian is not usable at all) but had no luck so far. I see mainly two reasons:
|
I'd be willing to consider a solution that requires a minor and cheap add-on or modification to the Mark 1, e.g. acoustic foam separating the mic and speaker or wire rerouting. But not board level changes. |
Beamforming based on the mic position plus a cheapo usb mic might be an option. One or two of these mini mics (search "overfly portable usb 2.0 mic") set in the ports combined with the audio from the existing mic run through a beamformer should be able to do aec and improve listening. I haven't tried it myself yet, alas. |
After some more experimenting I have a configuration with the PulseAudio echo-cancel module that works reasonably* with volume levels up to 5 (Mark-1's maximum is 11) within a distance of approx. 4 feet. There is some more room for tweaking parameters that might increase reliability. *depends on the music material, the more compressed (see "loudness war") the less reliable it works. |
I believe @forslund already did some work on the ducking part. Believe it is already in PR / Issue section somewhere. With you that AEC has to be combined with audio ducking. |
I used some door/window insulating foam (similar: https://www.homedepot.com/p/Frost-King-3-4-in-x-5-16-in-x-10-ft-Black-Rubber-Foam-Weatherseal-Tape-R534H/202262324) to make a barrier around the front of the mic between the face circuitboard and the faceplate. Secondarily to that, I covered the back of the speaker with foam as well. |
NOTE: This issue supercedes Issue #57
Problem statement
The current audio bus on the Mark 1 and Picroft images does not eliminate the speaker audio from the microphone. This leads to undesirable device behavior, most noticeably when an audio stream is playing and the user is unable to “barge in” easily with a
Hey Mycroft
.The device is aware of what audio is being output from the speaker. The essential idea desired is to subtract the speaker audio-out from the microphone audio-in using an appropriate approach - such as time-shifting the outbound audio and matching it to the audio in from the microphone.
Acceptance criteria
Useful information
Key technical contact - Steve Penrod (@penrods) (@steve-mycroft at https://chat.mycroft.ai)
Bounty
The Bounty for this feature request is $USD1000, as well as a free Mark 1 and a Gold Mycroft Challenge Coin.
The text was updated successfully, but these errors were encountered: