🏆 Join the TuyaOpen AI Hardware Contest! 100 days to build the future of AI devicesLearn More →
Skip to main content

Duo-Eyes Mood Robot

duo_eye_mood is an intelligent chatbot based on Tuya AI's open-source large language model. It captures voice input by using a microphone, processes it with speech recognition, and enables conversational interaction, responsive replies, and humorous banter. The chat content is displayed in real-time on the screen.

Features

  • Supports Tuya voice agent with four modes:

    • Hold-to-talk mode: Users press and hold a button to start voice input, and release to end.

    • Press-to-talk mode: The device enters standby mode upon power-on. A short button press switches it to the listening state for voice input and interaction with the agent.

    • Wake word mode: Users wake the device with a wake word for a single interaction, similar to talking to a smart speaker.

    • Free talk mode: After waking the device with a wake word, users can engage in continuous conversation. If no speech is detected within 30 seconds, the device returns to standby and requires rewaking for the next interaction.

  • Supports local interruption of ongoing chats via button operation.

  • Provides various status prompt tones, such as pairing, going offline, and wake-up.

  • Shows various eye GIFs to reflect the current emotion of the chatbot, such as angry, surprised, scared, confused, and other expressive eye animations, enhancing interactivity and fun.

    Supported emotions: Neutral, surprised, angry, crying, touched, sad, thinking, happy, rejecting, and disappointed.

  • Allows device volume adjustment on the app.

  • Enter pairing mode by power cycling the device three times consecutively.

  • Supports real-time switching of AI agent roles on the app.

Hardware support

BoardConfiguration filePID
T5AI_BOARD development board + dual-screen accessory
(two 1.28-inch SPI565 ST7735S round displays)		TUYA_T5AI_BOARD_EYES.config8ixyalzpn0jrun9y

File structure

Root directory

.
├── app_default.config    # Default application configuration file
├── CMakeLists.txt        # CMake build script
├── config                # Hardware configuration
├── include               # Header files
├── Kconfig               # Project configuration file
├── README.md             # Documentation in English
├── README_zh.md          # Documentation in Chinese
└── src                   # Source code

Hardware configuration

The files in the config directory are configuration files adapted for various development boards. If you have completed adaptation for a new board based on this project, you can add the corresponding configuration file in this directory.

├── config       
│   └── TUYA_T5AI_BOARD_EYES.config    # T5AI_BOARD development board + dual-screen accessory

Header files

├── include
│   ├── app_chat_bot.h       # Chatbot functionality
│   ├── app_display.h        # Display module
│   ├── reset_netcfg.h       # Power cycle reset functionality
│   └── tuya_config.h        # Device license configuration and related information

Source file

└── src
    ├── app_chat_bot.c      # Chatbot functionality implementation
    ├── display             # Display module
    ├── reset_netcfg.c      # Power cycle reset functionality
    └── tuya_main.c         # Application entry

Display module

├── display                 # Display module
│   ├── app_display.c       # Main control file of the display module
│   ├── CMakeLists.txt      # CMake configuration of the display module
│   ├── image               # Image resources
│   ├── tuya_lvgl.c         # LVGL adaptation file for the application
│   ├── tuya_lvgl.h         # LVGL adaptation header file
│   └── ui                  # UI implementation
│       ├── ui_display.h    # Common header file of the UI module
│   └── ui_eyes.c           # Dual-screen emotion display interface

Application configuration

  • Product ID (PID)

    MacroTypeDescription
    TUYA_PRODUCT_KEYStringThe product ID, which is bound to the Tuya AI agent.

  • Select chat mode

    MacroTypeDescription
    ENABLE_CHAT_MODE_KEY_PRESS_HOLD_SINGELBooleanHold-to-talk mode
    ENABLE_CHAT_MODE_KEY_TRIG_VAD_FREEBooleanPress-to-talk mode
    ENABLE_CHAT_MODE_ASR_WAKEUP_SINGELBooleanWake word mode
    ENABLE_CHAT_MODE_ASR_WAKEUP_FREEBooleanFree talk mode

  • Select wake word

    This configuration is only available when the chat mode is set to wake word mode and free talk mode.

    MacroTypeDescription
    ENABLE_WAKEUP_KEYWORD_NIHAO_TUYABooleanThe wake word is "Hey Tuya".
    ENABLE_WAKEUP_KEYWORD_NIHAO_XIAOZHIBooleanThe wake word is "Hey Xiaozhi".
    ENABLE_WAKEUP_KEYWORD_XIAOZHI_TONGXUEBooleanThe wake word is "Xiaozhi Tongxue".
    ENABLE_WAKEUP_KEYWORD_XIAOZHI_GUANJIABooleanThe wake word is "Xiaozhi Guanjia".

Process

This project depends on the application component ai_audio. This component implements various features, including audio capture, audio playback, and creating cloud-based AI sessions.

  • Hold-to-talk mode

  • Press-to-talk mode

  • Wake word mode

  • Free talk mode

Build project

  1. Install and configure the prerequisite environment. For more information, see Set up Environment.

  2. Change from the TuyaOpen root directory to the application directory.

    cd apps/tuya.ai/duo_eye_mood

  3. Select the specified development board configuration.

    tos.py config choice

  4. Build the project.

    tos.py build

Adapt to new development boards

Peripheral requirements

For a list of peripherals supported by TuyaOpen, refer to the Peripheral driver list. New development boards must include all required peripheral drivers listed.

PeripheralPeripheral componentDescriptionRequired or not
Audio driveraudio_codecsCapture PCM audio data and play MP3 audio (parameters: 16-bit depth, 16 KHz sample rate, mono).Yes
Display driverDisplayFeatures two 1.28-inch round screens, representing a pair of eyes. Both round screens are controlled by the same driver interface, meaning they display identical animations.Yes
Indicator driverLEDIndicates whether the device is currently in the listening state.No
Button driverButtonInterrupts the chat and starts a chat in press-to-talk mode.No

Adapt to development boards

For more information on how to add and adapt to a new development board, refer to Adapt to new board-level drivers (board).

You can register the peripheral driver hardware on the development board through board_register_hardware. The implementation of this interface is placed in the board-level folder.

In the target development board's folder (boards/target development environment/target development board, for example, boards/T5AI/TUYA_T5AI_BOARD), you can modify the driver's hardware configuration (such as changing the pins or chip type).