🏆 Join the TuyaOpen AI Hardware Contest! 100 days to build the future of AI devicesLearn More →
Skip to main content

Your Chat Bot

your_chat_bot is an open-source large-model intelligent chat bot based on tuya.ai. It collects voice through a microphone, performs speech recognition, enables conversation, interaction, and banter, and allows you to see real-time chat content on the screen.

Features​

  • Supports Tuya voice AI agent and four chat modes:
    • Long-press talk mode: The user must keep pressing the button to input voice; releasing the button ends the voice input.
    • Button talk mode: The device enters standby by default after powering on. The user short-presses the button to enter listening mode, then can input voice and interact with the AI agent.
    • Wakeup talk mode: The user must say the wake word before each conversation to wake up the device, then can have a single conversation, similar to a smart speaker.
    • Free talk mode: After waking up the device with the wake word, the user can have continuous conversations. If there is no conversation after wakeup, the device will return to standby after 30 seconds and needs to be woken up again for the next conversation.
  • Supports interrupting the conversation locally by pressing the button.
  • Supports various status prompt sounds, such as network configuration, offline, wakeup, etc.
  • Supports various displays and multiple UI styles:
    • WeChat style
    • Chat bot
    • Scrolling subtitles, suitable for small OLED screens
  • Supports adjusting device volume via App.
  • Power cycling the device 3 times in a row will enter network configuration mode.
  • Supports real-time switching of AI agent roles via App.

Hardware Support List​

BoardConfig FileUIPID
T5AI_BOARD dev board + 3.5" rgb565 LCD (touch, lcd:ili9488, tp:gt1151)TUYA_T5AI_BOARD_LCD_3.5.configWeChatalon7qgyjj8yus74
T5AI_EVB dev board (white box, built-in 2.4" spi565 st7789)TUYA_T5AI_EVB.configChat botr3ulobrs5nwreguj
T5AI_MOJI_1_28 dev board (built-in 1.28" spi565 gc9a01 round)T5AI_MOJI_1.28.configChat botkf8wauubey0doaiz
T5AI_MINI dev board + 1.3" spi565 st7789 LCDTUYA_T5AI_MINI_LCD_1.3.configChat bot8btswykdiium7t8k
DNESP32S3_BOX dev board (built-in 320*340 mcu8080 st7789 lcd)DNESP32S3_BOX.configChat botznw8prbujidtzavd
DNESP32S3 dev board (built-in 320*340 spi st7789 lcd)DNESP32S3.configWeChatscig7pauzzid3w4b
ESP32S3 Breadboard (built-in 128*32 i2c ssd1306 oled)ESP32S3_BREAD_COMPACT_WIFI.configScrollingj1y437proohznfbs
Waveshare ESP32S3 dev board (built-in 364*448 spi sh8601 lcd)WAVESHARE_ESP32S3_TOUCH_AMOLED_1_8.configWeChatlased5audtah8wcp
XINGXZHI_ESP32S3_CUBE dev board (built-in 128*64 i2c ssd1306 oled)XINGZHI_ESP32S3_CUBE_0_96OLED_WIFI.configScrollinguyidyzglm2m1bpcl

File Structure​

Project Root Directory​

.
├── app_default.config    # Default application config file
├── assets                # Static resources directory
├── CMakeLists.txt        # CMake build script
├── config                # Hardware config directory
├── include               # Header files directory
├── Kconfig               # Project config file
├── README_zh.md          # Chinese documentation
├── README.md             # English documentation
├── script                # Scripts directory
└── src                   # Source code directory

Assets Directory​

├── assets                 # Static resources directory
│   ├── lang_config.h      # Multi-language config header
│   └── zh-CN              # Chinese language pack
│       └── language.json  # Chinese text resources

Hardware Config Directory​

The config directory contains configuration files for various boards. If you adapt a new board based on this application, you can add the config file here.

├── config                 # Hardware config directory
│   ├── DNESP32S3_BOX.config
│   ├── DNESP32S3.config
│   ├── ESP32S3_BREAD_COMPACT_WIFI.config
│   ├── T5AI_MOJI_1.28.config
│   ├── TUYA_T5AI_BOARD_EYES.config
│   ├── TUYA_T5AI_BOARD_LCD_3.5.config
│   ├── TUYA_T5AI_EVB.config
│   ├── TUYA_T5AI_MINI_LCD_1.3.config
│   ├── WAVESHARE_ESP32S3_TOUCH_AMOLED_1_8.config
│   └── XINGZHI_ESP32S3_CUBE_0_96OLED_WIFI.config

Scripts Directory​

├── script                      # Scripts directory
│   └── gen_lang.py             # Multi-language resource generation script

Include Directory​

├── include
│   ├── app_chat_bot.h       # Chat bot feature header
│   ├── app_display.h        # Display module header
│   ├── app_system_info.h    # System info (network status, memory, etc.)
│   ├── reset_netcfg.h       # Power cycle reset feature header
│   └── tuya_config.h        # Device license and info config

Source Code Directory​

└── src
    ├── app_chat_bot.c      # Chat bot feature implementation
    ├── app_system_info.c   # System info features
    ├── display             # Display module
    ├── reset_netcfg.c      # Power cycle reset feature
    └── tuya_main.c         # Application entry

Display Module​

├── display                 # Display module
│   ├── app_display.c       # Main display control file
│   ├── CMakeLists.txt      # Display module CMake config
│   ├── font                # Font resources
│   ├── image               # Image resources
│   ├── Kconfig             # Display config options
│   ├── tuya_lvgl.c         # LVGL adaptation file
│   ├── tuya_lvgl.h         # LVGL adaptation header
│   └── ui                  # UI implementation
│       ├── ui_chatbot.c    # Chat bot UI
│       ├── ui_display.h    # UI module common header
│       ├── ui_oled.c       # OLED scrolling subtitle UI
│       └── ui_wechat.c     # WeChat style UI

Application Configuration​

  • Product PID

    MacroTypeDescription
    TUYA_PRODUCT_KEYStringProduct ID, bound to Tuya AI agent

  • Select Chat Mode

    MacroTypeDescription
    ENABLE_CHAT_MODE_KEY_PRESS_HOLD_SINGELBoolLong-press talk mode
    ENABLE_CHAT_MODE_KEY_TRIG_VAD_FREEBoolButton talk mode
    ENABLE_CHAT_MODE_ASR_WAKEUP_SINGELBoolWakeup talk mode
    ENABLE_CHAT_MODE_ASR_WAKEUP_FREEBoolFree talk mode

  • Select Wake Word

    This config only appears when selecting Wakeup Talk or Free Talk mode.

    MacroTypeDescription
    ENABLE_WAKEUP_KEYWORD_NIHAO_TUYABoolWake word is "Nihao Tuya"
    ENABLE_WAKEUP_KEYWORD_NIHAO_XIAOZHIBoolWake word is "Nihao Xiaozhi"
    ENABLE_WAKEUP_KEYWORD_XIAOZHI_TONGXUEBoolWake word is "Xiaozhi Tongxue"
    ENABLE_WAKEUP_KEYWORD_XIAOZHI_GUANJIABoolWake word is "Xiaozhi Guanjia"

Workflow​

This project depends on the ai_audio application component, which mainly implements audio collection, playback, cloud AI session creation, and other functions.

  • Long-press talk mode

  • Button talk mode

  • Wakeup talk mode

  • Free talk mode

Build Project​

  1. For environment setup and configuration, please refer to Quick Start.

  2. Change directory from the TuyaOpen root to the application directory

    cd apps/tuya.ai/your_chat_bot

  3. Select the corresponding dev board config

    tos.py config choice

  4. Build the project

    tos.py

Adapting New Boards​

Peripheral Requirements List​

For the list of peripherals supported by tuyaopen, please refer to Peripheral Driver List. Any new board must have the required peripheral drivers listed below.

PeripheralPeripheral ComponentDescriptionRequired
Audio Driveraudio_codecsCan collect audio data in PCM format. Can play voice in MP3 format, 16bit, 16KHz, mono.Yes
Display DriverdisplayLCD screen / OLED screenYes
LED DriverledIndicates whether the device is listening, etc.No
Button DriverbuttonInterrupts conversation, starts conversation in button talk mode.No

Board Adaptation​

tip

The unified entry for board peripheral driver registration is board_register_hardware, and the implementation of this interface is placed in the board-level folder.

If developers want to modify the hardware configuration of the driver (such as changing pins, chip types, etc.), please make changes in the target board's folder (boards/target environment/target board, such as boards/T5AI/TUYA_T5AI_BOARD).

Please refer to Adding New Board Support for the steps to add and adapt new boards.