mirror of https://github.com/78/xiaozhi-esp32.git synced 2026-01-13 16:57:17 +08:00

Build your own AI friend

Go to file

so95 06da25fd26 Some checks failed Build Boards / Determine variants to build (push) Has been cancelled Details Build Boards / Build ${{ matrix.name }} (push) Has been cancelled Details ESP-P4-Function-EV-Board add SD card, camera, and font initialization (#1403 ) * feat: add support for ESP32-P4-Function-EV-Board with configuration and display handling * detect wake word model from index.json * update wait time before entering wifi configure mode * feat: Enhance ESP32-P4 Function EV Board support with LCD and touch initialization * feat: Update ESP32-P4 Function EV Board configuration for improved touch and SD card support * feat: add touch I2C configuration and improve initialization structure * Remove ESP hosted configuration from defaults Removed ESP hosted configuration options. * chore: update documentation for improved clarity * refactor: remove obsolete files for ESP32-P4-Function-EV-Board and add updated configurations * refactor: reintroduce ESP32-P4-Function-EV-Board implementation with updated configurations * refactor: restore esp32_p4_function_ev_board dependency with updated version * Remove ESP-P4-Function-EV-Board configuration and update README with detailed features; add SD card, camera, and font initialization in esp-p4-function-ev-board.cc * Update main/boards/esp-p4-function-ev-board/esp-p4-function-ev-board.cc Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update main/boards/esp-p4-function-ev-board/esp-p4-function-ev-board.cc Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Enhance camera initialization in ESP-P4-Function-EV-Board; add fallback configuration for direct camera setup and update README for clarity * Update clock source for MIPI DSI configuration in ESP-P4-Function-EV-Board --------- Co-authored-by: n2flowjs-bot <n2flowjs@gmail.com> Co-authored-by: Terrence <terrence@tenclass.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>		2025-11-10 21:41:59 +08:00
.github	Refactor: Use esp_video component (#1245 )	2025-10-14 10:44:45 +08:00
docs	Update README (#1251 )	2025-10-02 03:29:16 +08:00
main	ESP-P4-Function-EV-Board add SD card, camera, and font initialization (#1403 )	2025-11-10 21:41:59 +08:00
partitions	feat: build default assets instead of downloading and v2 tables for esp-hi, echoear (#1203 )	2025-09-16 15:24:24 +08:00
scripts	Update component version (#1382 )	2025-11-04 18:40:54 +08:00
.gitignore	Add esp32c5 configuration (#1258 )	2025-10-05 09:59:28 +08:00
CMakeLists.txt	Bump to 2.0.4	2025-10-25 04:21:46 +08:00
LICENSE	Update LICENSE	2025-07-22 10:06:11 +08:00
README_en.md	Update README (#1251 )	2025-10-02 03:29:16 +08:00
README_ja.md	Update README (#1251 )	2025-10-02 03:29:16 +08:00
README.md	Update README (#1251 )	2025-10-02 03:29:16 +08:00
sdkconfig.defaults	Fix lichuang-dev camera (#1290 )	2025-10-14 20:44:44 +08:00
sdkconfig.defaults.esp32	Detect wake word model from index.json (#1211 )	2025-09-17 08:31:51 +08:00
sdkconfig.defaults.esp32c3	Switch to 2.0 branch (#1152 )	2025-09-04 15:41:28 +08:00
sdkconfig.defaults.esp32c5	Add esp32c5 configuration (#1258 )	2025-10-05 09:59:28 +08:00
sdkconfig.defaults.esp32c6	Switch to 2.0 branch (#1152 )	2025-09-04 15:41:28 +08:00
sdkconfig.defaults.esp32p4	feat: build default assets instead of downloading and v2 tables for esp-hi, echoear (#1203 )	2025-09-16 15:24:24 +08:00
sdkconfig.defaults.esp32s3	feat: build default assets instead of downloading and v2 tables for esp-hi, echoear (#1203 )	2025-09-16 15:24:24 +08:00

README_en.md

An MCP-based Chatbot

(English | 中文 | 日本語)

Introduction

👉 Human: Give AI a camera vs AI: Instantly finds out the owner hasn't washed hair for three days【bilibili】

👉 Handcraft your AI girlfriend, beginner's guide【bilibili】

As a voice interaction entry, the XiaoZhi AI chatbot leverages the AI capabilities of large models like Qwen / DeepSeek, and achieves multi-terminal control via the MCP protocol.

Version Notes

The current v2 version is incompatible with the v1 partition table, so it is not possible to upgrade from v1 to v2 via OTA. For partition table details, see partitions/v2/README.md.

All hardware running v1 can be upgraded to v2 by manually flashing the firmware.

The stable version of v1 is 1.9.2. You can switch to v1 by running git checkout v1. The v1 branch will be maintained until February 2026.

Features Implemented

Wi-Fi / ML307 Cat.1 4G
Offline voice wake-up ESP-SR
Supports two communication protocols (Websocket or MQTT+UDP)
Uses OPUS audio codec
Voice interaction based on streaming ASR + LLM + TTS architecture
Speaker recognition, identifies the current speaker 3D Speaker
OLED / LCD display, supports emoji display
Battery display and power management
Multi-language support (Chinese, English, Japanese)
Supports ESP32-C3, ESP32-S3, ESP32-P4 chip platforms
Device-side MCP for device control (Speaker, LED, Servo, GPIO, etc.)
Cloud-side MCP to extend large model capabilities (smart home control, PC desktop operation, knowledge search, email, etc.)
Customizable wake words, fonts, emojis, and chat backgrounds with online web-based editing (Custom Assets Generator)

Hardware

Breadboard DIY Practice

See the Feishu document tutorial:

👉 "XiaoZhi AI Chatbot Encyclopedia"

Breadboard demo:

Supports 70+ Open Source Hardware (Partial List)

Software

Firmware Flashing

For beginners, it is recommended to use the firmware that can be flashed without setting up a development environment.

The firmware connects to the official xiaozhi.me server by default. Personal users can register an account to use the Qwen real-time model for free.

👉 Beginner's Firmware Flashing Guide

Development Environment

Cursor or VSCode
Install ESP-IDF plugin, select SDK version 5.4 or above
Linux is better than Windows for faster compilation and fewer driver issues
This project uses Google C++ code style, please ensure compliance when submitting code

Developer Documentation

Custom Board Guide - Learn how to create custom boards for XiaoZhi AI
MCP Protocol IoT Control Usage - Learn how to control IoT devices via MCP protocol
MCP Protocol Interaction Flow - Device-side MCP protocol implementation
MQTT + UDP Hybrid Communication Protocol Document
A detailed WebSocket communication protocol document

Large Model Configuration

If you already have a XiaoZhi AI chatbot device and have connected to the official server, you can log in to the xiaozhi.me console for configuration.

👉 Backend Operation Video Tutorial (Old Interface)

For server deployment on personal computers, refer to the following open-source projects:

xinnan-tech/xiaozhi-esp32-server Python server
joey-zhou/xiaozhi-esp32-server-java Java server
AnimeAIChat/xiaozhi-server-go Golang server

Other client projects using the XiaoZhi communication protocol:

huangjunsen0406/py-xiaozhi Python client
TOM88812/xiaozhi-android-client Android client
100askTeam/xiaozhi-linux Linux client by 100ask
78/xiaozhi-sf32 Bluetooth chip firmware by Sichuan
QuecPython/solution-xiaozhiAI QuecPython firmware by Quectel

Custom Assets Tools:

78/xiaozhi-assets-generator Custom Assets Generator (Wake words, fonts, emojis, backgrounds)

About the Project

This is an open-source ESP32 project, released under the MIT license, allowing anyone to use it for free, including for commercial purposes.

We hope this project helps everyone understand AI hardware development and apply rapidly evolving large language models to real hardware devices.

If you have any ideas or suggestions, please feel free to raise Issues or join the QQ group: 1011329060