8000 [micro_wake_word] add new VPE features by kahrendt · Pull Request #8655 · esphome/esphome · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[micro_wake_word] add new VPE features #8655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 1, 2025
Merged

[micro_wake_word] add new VPE features #8655

merged 2 commits into from
May 1, 2025

Conversation

kahrendt
Copy link
Contributor
@kahrendt kahrendt commented Apr 30, 2025 •
edited
Loading

What does this implement/fix?

  • Runs mWW inferences in a task to avoid slowing down the main loop.
  • Individual models can be enabled or disabled with new yaml actions
    • A future PR will add support to the voice_assistant component so they can be enabled/disabled via HA (models marked internal will not be exposed to HA)
    • Model's enabled state is restored from flash
  • Adds new option for stop_after_detection. Defaults to true to mimic current behavior. Can be disabled if the device has duplex audio support (currently requires separate I2S busses) and the user wants to leave mWW always running
  • Bumps the esp-tflite-micro library to the latest version

This is a breaking change in that only the first model listed will be enabled on the first boot (currently all models would be enabled).

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Code quality improvements to existing code or addition of tests
  • Other

Related issue or feature (if applicable):

Pull request in esphome-docs with documentation (if applicable):

Test Environment

  • ESP32
  • ESP32 IDF
  • ESP8266
  • RP2040
  • BK72xx
  • RTL87xx

Example entry for config.yaml:

Silly example, you probably wouldn't want to enable/disable models in the wake_word_detected action.

# Example config.yaml

micro_wake_word:
  microphone:
    gain_factor: 4
  models:
    -
8000
 model: hey_jarvis
      id: hey_jarvis_model
    - model: okay_nabu
      id: okay_nabu_model
  on_wake_word_detected:
    - if:
        condition:
          - micro_wake_word.model_is_enabled: hey_jarvis_model
        then:
          - micro_wake_word.disable_model: hey_jarvis_model
        else:
          - micro_wake_word.enable_model: hey_jarvis_model

Checklist:

  • The code change is tested and works locally.
  • Tests have been added to verify that the new code works (under tests/ folder).

If user exposed functionality or configuration variables are added/changed:

@codecov-commenter
Copy link
codecov-commenter commented Apr 30, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 56.65%. Comparing base (4d8b5ed) to head (614a785).
Report is 2429 commits behind head on dev.

Additional details and impacted files 
@@            Coverage Diff             @@
##              dev    #8655      +/-   ##
==========================================
+ Coverage   53.70%   56.65%   +2.94%     
==========================================
  Files          50       50              
  Lines        9408     9907     +499     
  Branches     1654     1338     -316     
==========================================
+ Hits         5053     5613     +560     
+ Misses       4056     3945     -111     
- Partials      299      349      +50

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This was referenced Apr 30, 2025
Closed
Merged
@kahrendt kahrendt marked this pull request as ready for review April 30, 2025 16:10
@kahrendt kahrendt requested review from jesserockz and a team as code owners April 30, 2025 16:10
@probot-esphome
Copy link

Hey there @jesserockz, mind taking a look at this pull request as it has been labeled with an integration (micro_wake_word) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)

@kahrendt kahrendt mentioned this pull request Apr 30, 2025
Merged
14 tasks
jesserockz
jesserockz approved these changes May 1, 2025
View reviewed changes
Copy link
Member
@jesserockz jesserockz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor thing below I will fix in a new PR to not hold this one up

esphome/core/defines.h
@@ -79,6 +79,7 @@
#define USE_LVGL_TEXTAREA
#define USE_LVGL_TILEVIEW
#define USE_LVGL_TOUCHSCREEN
#define USE_MICRO_WAKE_WORD
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last time I checked I came after E 😉
But also, this should go inside the #ifdef USE_ESP32 block below

@jesserockz jesserockz merged commit cdc7750 into esphome:dev May 1, 2025
26 checks passed
@kahrendt kahrendt deleted the mww-add-vpe-features branch May 1, 2025 11:19
joshuaboniface added a commit to joshuaboniface/supersensor that referenced this pull request May 11, 2025
@joshuaboniface
Enable MicroWakeWord by default.
df4bd66 
This enables MWW by default. This has become more important since
HomeAssistant recently disabled offloaded WakeWord engines by default in
new Assist pipelines, ostensibly because "all" client devices now
support it.

Currently we are limited to just `hey_jarvis` until esphome/esphome#8655
lands in a release and enables dynamic configuration. This will be fully
implemented with the SuperSensor v2.0 coming later this month.
@jesserockz jesserockz mentioned this pull request May 13, 2025
Merged
@jesserockz jesserockz mentioned this pull request May 21, 2025
Merged
sa-crespo pushed a commit to sa-crespo/esphome that referenced this pull request May 26, 2025
@kahrendt @sa-crespo
[micro_wake_word] add new VPE features (esphome#8655)
ab74b14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jesserockz jesserockz jesserockz approved these changes

Assignees

@jesserockz jesserockz

Labels
by-code-owner core has-tests integration: micro_wake_word
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

micro_wake_word: Support "id" and "internal" for mww Allow micro_wake_word model to be set from a template select value
3 participants
@kahrendt @codecov-commenter @jesserockz
0