ReSpeaker Messenger: Sending and receiving Slack and Telegram messages using ReSpeaker Voice Interaction Board
My previous project using ReSpeaker was a Home Automation project to control different lights using your voice. This projects is about sending messages to Slack or Telegram using voice input and read out the message from the messenger. User can touch button 1 to send Slack message and touch button 7 to send Telegram message. For Telegram message, first you need to initiate the chat from Telegram client. This is because to send a message to Telegram you need the userid of the receiving person.
This project uses Slack, Telegram, MQTT, hook.io and ReSpeaker.
In case of Slack messenger, We set an Outgoing Webhook to a hook.io microservice written in Node.js. This Node.js script receives the text and publish MQTT message with a specified topic. The Python script running on ReSpeaker subscribe this topic and upon receiving this message, the text is converted to audio using Microsoft Cognitive Computing Text to Speech API. This is read out uinsg PyAudio.
In case of Telegram, the ReSpeaker python application use Telepot Python library to integrate Telegram API. Whenever a message is received the text is send to Microsoft Cognitive Service API and the receiving audio is played using PyAudio.
We have two applications running on the ReSpeaker, on Arduino Sketch and Python script. The Arduino Sketch receives touch event and send to the Python Script using Serial to start recording the voice input. When the start recording message is received, the script records the user's voice input and convert to text using Microsoft Cognitive Service Speech to Text API and send to the corresponding messenger depending on the touch button id. If the button touched is 1 then send to Slack messenger and if the touch button id is 7 then to Telegram messenger. Telepot library is used to send to Telegram messenger. For Slack, we have created a Slack Incoming Webhook. Below picture depicts working of the application.
As you know ReSpeaker has two UART connections, "serial" and "serial1". The serial is connected to the USB port. The serial1 can be used to communicate from Arduino to the Python script. The serial1 is connected to the UART_RXD2 and UART_TXD2 of MT7688. So you can just open the Serial port /dev/ttyS2 and read the incoming data. The following test script opens the /dev/ttyS2 port and reads strings (Thanks to Jerry of Seeedstudio for suggesting this).
import serial
import io
port = '/dev/ttyS2'
baud = 57600
ser = serial.Serial(port, baud, timeout=1)
sio = io.TextIOWrapper(io.BufferedRWPair(ser, ser))
if ser.isOpen():
print(ser.name + ' is open...')
while True:
serialdata = sio.readline()
print(serialdata)
Follow this document to get started with ReSpeaker
- Open Arduino IDE and load this file
- Connect your ReSpeaker to the network
- SSH to your ReSpeaker
- Create a folder to clone the project files
- Move to the folder and clone the repo https://github.com/krvarma/respeaker-messenger.git
- Got to
respeaker-messengerfolder - Create an account or log on to Microsoft Cognitive Service Speech API](https://www.microsoft.com/cognitive-services/en-us/speech-api)
- Get an API Key and replace the variable BING_KEY in the Python Script
Settings.py - Log on to Slack.com, go to Settings->Add and app or integrations
- Go to Manage->Custom Integrations->Incoming Webhook. Create Incoming Webhook and copy the URL
- Replace the variable SLACK_WEBHOOK in the Python Script
Settings.py - Go to Slack->Manage->Custom Integrations->Incoming Webhook and create Outgoing Webhook. Copy the Token
- Log on to hook.io and create a new hook
- Copy the contents of this file to the hook
- Replace the access_token
- Log on to Telegram.org and create a new bot following this instructions
- Copy the token and replace the variable TELEGRAM_KEY in the Python Script
Settings.py - Install required Python package paho-mqtt using pip install paho-mqtt
- Install required Python package paho-mqtt using pip install telepot
- Install required Python package monotonic using pip install monotonic
- Run
messenger.pyusing command line pythonmessenger.py - Open Telegram client application and send a simple 'hi' to any user, this will receive at the ReSpeaker end and the
useridwill be saved for sending future messages. This step is necessary because to send a message to Telegram, you need the userid of the user to whom you want to send the message. - Touch the button 1 to send to messages to Slack and touch button 7 to send messages to Telegram.
- You can also send messages from Slack to the specified channel using the specified trigger work, this will be send to ReSpeaker and it will read out the text
If everything goes well, you have a working ReSpeaker Messenger and you can send messages using your voice. The application will read out the messages send from Slack or Telegram.
Slack Messenger https://www.youtube.com/watch?v=Q7amXfVdt4k
Telegram Messenger https://www.youtube.com/watch?v=_FrRhzkRTGc