[Open Source] Play with T5-Pocket: Your AI+IoT Smart Companion

T5-Pocket show

✨ T5-Pocket Latest Features

Based on the latest TuyaOpen firmware, T5-Pocket has implemented the following core features:

🎮 AI Virtual Pet System

Pet Animations: Rich animations including eating, bathing, sleeping, dancing, getting sick, using bathroom, etc.
State Management: Real-time changes in health, energy, cleanliness, and happiness
Emotion Expression: Happy, sad, sick and other emotions that switch automatically based on state
Interactive Operations: Feeding (burgers, pizza, apples, etc.), bathing, seeing doctor, sleeping, etc.
Smart Timer System: Automatically decreases stats every 20 minutes to simulate real pet needs
Data Persistence: Pet states automatically saved to KV storage, no data loss on power off
Cloud Sync: Real-time status reporting to Tuya Cloud, supports remote viewing and control

💬 Multimodal AI Conversation

4 Conversation Modes:
- Push-to-talk single conversation (press to talk, release to send)
- VAD free conversation (automatic voice activity detection)
- Wake word single conversation (like smart speakers)
- Wake word free conversation (continuous dialogue mode)
Emotion Recognition: AI recognizes user emotions (happy, angry, fearful, sad) and adjusts responses
Streaming Text Processing: Supports streaming reception and display of AI responses
Multi-language Support: Chinese and English voice recognition and synthesis

🖥️ UI Interface

Main Screen: Displays pet animations, status bar (Wi-Fi, battery, 4G), pet attributes
Menu System:
- Feeding menu (food selection)
- Bath menu
- Health menu (see doctor)
- Sleep menu
- Info menu (detailed pet attributes)
- Peripheral menu (gyroscope, I2C scan, WiFi scan, games)
Game Interface:
- Snake game (fully implemented)
- Dino runner game (fully implemented)
Tool Interface:
- RFID scanning interface
- I2C device scanning interface
- Temperature/humidity display
- Level meter interface
- AI log analysis interface
- E-book reader
- Keyboard input interface

🎵 Audio System

System Sounds:
- Network configuration sounds
- Network connection success/failure sounds
- Wake-up sounds
- Various conversation mode sounds
Pet Sound Effects:
- Eating sounds
- Bathing sounds
- Sick sounds
- And more
Volume Control: 0-100 level volume adjustment with cloud sync support

🔌 Extended Features

RFID Card Reader: Supports UID reading and data parsing, CRC16 verification
Thermal Printer: Supports text and image printing, UTF-8 to GBK encoding
AI Log Analysis: Real-time serial log capture with AI intelligent error analysis
BMI270 Gyroscope: Posture detection and step counting
I2C Device Scanning: Automatically scans and displays I2C device addresses

How to Play with T5-Pocket?

T5-Pocket is an open-source project based on TuyaOpen. Users can download source code or firmware from the TuyaOpen official website to implement custom features. The TuyaOpen team provides rich example projects that users can reference to develop their own projects.

Joystick

Overview

T5-Pocket is equipped with a dual-channel joystick and independent button, supporting x, y dual-channel 12-bit precision ADC sampling. It can function as a joystick, knob, or directional pad. The joystick enables more precise UI switching and function selection, along with many hidden gameplay features.

Use Cases

Game Controller: Like common game controllers, you can use the joystick on T5-Pocket to control game character movement. Classic games like Dino Runner and Snake can all be implemented.
Menu Navigation: Use the joystick for more precise UI page switching, including function icon selection, e-book page turning, and flexible page navigation controlled by the joystick.

Development

For joystick development, refer to the example project.

Gyroscope

Overview

T5-Pocket features a high-performance 6-axis inertial measurement unit (IMU) from Bosch Sensortec—BMI270. It integrates a 16-bit three-axis accelerometer and a 16-bit three-axis gyroscope in an extremely small package, achieving ultra-low power consumption with precise device posture sensing.

Use Cases

Activity Recognition/Step Counting: Utilize BMI270's built-in powerful algorithms to easily implement real-time monitoring of user steps and movement status.
Gesture Control and Level Meter: By calculating the quaternion data output from the IMU, you can obtain precise Euler angles of the device, enabling creative applications like gesture mice and level meters.
Action Recognition: Collect dynamic gesture data from the device, deploy a CNN model locally on T5, train and recognize specific actions such as "shake" or "flip," creating new interaction dimensions.
Balance Vehicle: Combined with expansion interfaces to drive motors, use the pitch angle data calculated from the IMU to build a mini balance vehicle.

Development

For BMI270 development, refer to the example project. The latest T5-Pocket firmware already supports level meter display.

Display

Overview

T5-Pocket features a 2.9-inch 384 × 168 resolution fully reflective low-power eye-care screen, developed based on LVGL (Light and Versatile Graphics Library), providing users with intuitive visual feedback and interactive interface.

Use Cases

AI Pet Animation: The display is the "face" of the AI virtual pet, showcasing various states and emotions through rich animations. Pets eat, bathe, sleep, and dance, with corresponding animation effects for each action, making the virtual pet more lively and adorable.
Information Dashboard: Use T5-Pocket as a desktop information center, displaying weather, time, to-do items, sensor data, and more in real-time. Switch between different information pages with the joystick to grasp important information at a glance.
Gaming Console: Using the display and joystick, develop various classic games such as Snake, Tetris, and Dino Runner. The monochrome screen brings the unique charm of retro gaming consoles.
E-Reader: The high-reflective screen features 0 blue light, making it eye-friendly for long-term use. Developers can read txt files from SD cards for handheld reading.

Development

The latest T5-Pocket firmware supports AI pet animations, information display, gaming console, e-reader, and many other functions. Developers can easily create their own UI pages following the UI Design Guide.

For display development, refer to Display Examples and LVGL Examples. LVGL provides rich documentation and examples for quick UI development.

SD Card

Overview

T5-Pocket supports external Micro SD cards, providing large-capacity external storage for audio files, log data, configuration files, image resources, and more.

Use Cases

Audio Player: Store music files on the SD card, and T5-Pocket can function as a portable music player. Supports multiple audio formats, with joystick control for play, pause, track skipping, and display of song information and playback progress.
Data Logger: Connect various sensors to record collected data in real-time to the SD card. Suitable for environmental monitoring, motion tracking, experimental data collection, and other scenarios. Data is stored in CSV or JSON format for easy subsequent analysis.
Offline Voice Library: Pre-store common voice prompts and TTS audio on the SD card to provide voice feedback in offline environments. Suitable for applications requiring offline operation.

Development

For SD card development, refer to the example project.

Microphone/Speaker

Overview

T5-Pocket has a built-in high-quality audio system, including 2 digital microphones and 1 speaker, which is the core hardware for AI voice interaction. Combined with TuyaOpen's audio processing capabilities, it can achieve professional-level voice recognition and synthesis.

Use Cases

Smart Voice Assistant: Interact with AI through voice to inquire about weather, set reminders, query information, control smart home devices, and more. Supports multi-turn conversations, with AI understanding context to provide smarter responses.
Voice Translator: Utilize AI's multilingual capabilities for real-time voice translation. Speak in Chinese and AI responds in English; speak in English and AI responds in Chinese. Suitable for foreign language learning or cross-language communication.
Story Machine: Tell stories, sing nursery rhymes, and recite poems for children. AI can generate personalized story content based on the child's age and interests. You can also record parents' voices, letting the pet tell stories in the parents' voice.
Voice Memo: Quickly record voice memos, with AI automatically converting to text and saving. Supports voice search to quickly find needed memo content.
Music Player: Play music files from SD card or stream online music. Supports playback control, volume adjustment, lyrics display, and other functions.

Development

For audio development, refer to the following examples:

Camera

Overview

T5-Pocket supports connecting camera modules through the DVP (Digital Video Port) interface, adding visual perception capabilities to the device. Combined with AI vision algorithms, it can implement face recognition, object detection, QR code scanning, and other functions. (AI video capabilities are currently being enhanced...)

Use Cases

AI Vision Pet: Pets can "see" the owner. When a face is detected, they'll actively greet. Can recognize the owner's expressions—if the owner smiles, the pet will be happy; if the owner frowns, the pet will ask with concern.
Smart Access Control: Implement access control through face recognition. When authorized personnel are recognized, automatically open the door and record entry/exit times. Can connect to electromagnetic locks, servos, and other actuators.
Object Recognition Assistant: Take photos of objects, and AI identifies the category and provides related information. For example, identify food and display nutritional information, identify plants and introduce care methods.
QR Code Scanner: Scan QR codes to obtain information for product traceability, electronic business cards, Wi-Fi configuration, and other scenarios. Scan results are displayed on screen or announced via voice.

Development

For camera development, refer to the example project. TuyaOpen provides complete camera drivers and image processing APIs.

Extended Devices

T5-Pocket provides rich expansion interfaces including I2C, SPI, UART, GPIO, PWM, Pogo Pin, etc. Developers can freely connect various peripherals and sensors to create personalized IoT applications.

RFID Card Reader

Overview

RFID (Radio Frequency Identification) technology is a contactless information transmission technology using radio waves, enabling automatic identification and data collection of target objects. T5-Pocket connects RFID reader modules through UART interface, supporting reading of various types of RFID tags.

Use Cases

Access Card Duplicator: Read access cards, elevator cards, and other unencrypted RFID card data, write data to blank cards to achieve card duplication. Suitable for backing up important access cards to avoid inconvenience from loss.
Virtual Pet Item System: Design physical item cards for AI virtual pets. Scan "food card" to feed pet, scan "toy card" to play with pet, scan "medicine card" to treat pet. Combine virtual and reality to enhance interactive experience.
Smart Attendance System: Employees or students swipe cards to check in, system automatically records attendance time and uploads to cloud. Can set late reminders, statistics on attendance rates, and other functions.
Item Management System: Attach RFID tags to items, scan tags to view item information, usage records, maintenance plans, etc. Suitable for tool management, asset inventory, warehouse management, and other scenarios.
Interactive Game Cards: Develop games based on RFID cards, such as card battles, role-playing, etc. Each card represents different characters, skills, or items. Scanning cards triggers corresponding game events.

Development

T5-Pocket provides complete RFID scanning example code, including data parsing, CRC verification, UI display, and other functions. Developers can quickly develop their own RFID applications based on this.

AI Thermal Printer

Overview

Thermal printers print text and images by heating the print head to contact special thermal paper (coating changes color when heated), without requiring ink cartridges. T5-Pocket connects thermal printer modules through UART interface, combining AI capabilities to achieve intelligent printing functions.

Use Cases

AI Story Printer: Ask AI "Tell me a story about courage," AI generates story content and automatically prints it. Children can collect printed stories to create their own storybooks.
Smart Recipe Printer: Ask AI "What should I cook for dinner tonight," AI recommends recipes based on season, nutrition, and taste, and prints detailed ingredient lists and cooking steps.
Math Problem Generator: Tell AI "Generate 20 second-grade addition and subtraction problems," AI generates problems and prints them with answers on the back. Helps children practice math while giving parents peace of mind.
Daily Quote Printer: Automatically print an inspirational quote or poem every morning, post it in a visible place to start a beautiful day.
AI Drawing Printer: Describe a scene to AI, such as "a cat sitting on the moon," AI generates an image and prints it. Although black and white, it's full of creativity and fun.
Shopping List Printer: Voice input items to buy, AI organizes them into a list and prints. When shopping at the supermarket, don't worry about forgetting what to buy.
Meeting Minutes Printer: After meetings, AI summarizes meeting content and prints minutes, including discussion points, decisions, and to-do tasks.

Development

T5-Pocket provides thermal printer example code, supporting text and image printing. Developers can refer to example code to implement their own printing applications.

AI Log Analysis Assistant

Overview

In embedded development, analyzing logs is an important means of locating problems, but facing massive logs, manual analysis is inefficient. T5-Pocket's AI log analysis function can automatically capture device logs, use AI for intelligent analysis, and quickly locate problem root causes.

Use Cases

Embedded Device Debugging Assistant: Connect the serial port of the embedded device under development to T5-Pocket. When the device runs, T5-Pocket monitors logs in real-time. Once an error occurs, immediately capture and analyze, providing possible causes and solutions.
Production Test Tool: On product production lines, connect T5-Pocket to devices under test, automatically run test programs and analyze logs. If anomalies are found, immediately alert and record, improving test efficiency and accuracy.
Remote Technical Support: When customer devices have problems, guide customers to connect devices to T5-Pocket, capture logs and upload to cloud. Technical support personnel remotely view AI analysis results to quickly locate problems without customers returning devices.
Learning Embedded Development: Beginners often encounter various errors when learning embedded development. T5-Pocket can serve as a learning assistant, analyzing error logs and providing easy-to-understand explanations to help understand problem essence.

Development

Based on T5-Pocket's AI log analysis example code. Developers can customize error keywords, analysis logic, display styles, etc.

Sensor Expansion

Overview

T5-Pocket supports connecting various sensors to digitize physical world information for environmental monitoring, motion tracking, health management, and other functions.

Supported Sensor Types

Environmental Sensors

Type	Example Sensors
Temperature/Humidity	SHT30, AHT20, DHT22
Barometric Pressure	BMP280, BMP180
Light	BH1750, TSL2561
Air Quality	SGP30, CCS811
PM2.5	PMS5003, GP2Y1010AU

Motion Sensors

Type	Example Sensors
Accelerometer / Gyroscope	BMI270 (onboard), MPU6050
Magnetometer	QMC5883L, HMC5883L
GPS Module	NEO-6M, NEO-M8N
Ultrasonic Distance	HC-SR04

Biometric Sensors

Type	Example Sensors
Heart Rate	MAX30102, MAX30100
Body Temperature	MLX90614, DS18B20
Blood Oxygen	MAX30102

Other Sensors

Type	Examples/Description
Soil Moisture	Capacitive, Resistive
Water Quality	TDS, pH
Gas Sensors	MQ Series (CO, methane, alcohol, etc.)
Infrared	Human presence, obstacle avoidance

Use Cases

Smart Environmental Monitoring Station: Connect temperature/humidity, barometric pressure, light, air quality, and other sensors to monitor indoor environment in real-time. Data is displayed on screen and uploaded to cloud. When environmental parameters are abnormal (such as high temperature, low humidity, poor air quality), provide voice reminders and improvement suggestions.
Fitness Health Tracker: Utilize onboard BMI270 accelerometer and gyroscope to implement step counting, motion recognition, posture detection, and other functions. Connect heart rate sensor to monitor heart rate changes during exercise. Data can sync to mobile app to generate exercise reports.
Smart Plant Care Assistant: Connect soil moisture and light sensors to monitor plant growth environment. Remind to water when soil is too dry, remind to move location when light is insufficient. AI can provide personalized care suggestions based on plant species.
Air Quality Monitor: Connect PM2.5, formaldehyde, CO2 sensors to monitor indoor air quality in real-time. Visualize data and generate Air Quality Index (AQI). When air quality is poor, remind to open windows for ventilation or turn on air purifier.
Smart Aquarium Manager: Connect water temperature, TDS, pH sensors to monitor aquarium water quality. When water quality parameters are abnormal, timely remind and provide treatment suggestions. Can also control heaters, filters, lights, and other equipment.

Development

TuyaOpen provides rich sensor drivers and example code:

Developers can refer to these examples to quickly integrate various sensors.

Conclusion

Whether you're an embedded developer, AI enthusiast, maker, or student, T5-Pocket can help you quickly realize your ideas and build your own smart devices.

Start your T5-Pocket journey now and explore the infinite possibilities of AI+IoT!

🚀 Go to Project Repository