AI MUSIC VISUALIZE PLUGIN (Sold and under protection)
Side project sold to Beijing JinChang ShiJie Technology Co., Ltd.
The AI Music Visualize Plugin for Unreal Engine 5 is a powerful tool designed to seamlessly bridge audio and visual experiences. The plugin automatically analyzes scene elements suited for audio visualization, such as flame bursts, smoke triggers, and dynamic motion elements. Leveraging AI, it intelligently assigns up to 20 distinct audio channels while performing emotional analysis of music in real time to adapt lighting and scene colors dynamically.
The plugin enhances synchronization by analyzing global audio characteristics, breaking them down across time, space, and timbre dimensions to ensure real-time visual harmony. It processes art scenes within UE5, outputs data for integration via WebSocket, and synchronizes seamlessly with external software and hardware setups in environments like live electronic music festivals, nightclubs, and immersive audio-visual productions.
ROLE: Project Manager, Programmer | 2024. 5
TEAM: 10 Members
PLATFORM: External Software Call
DURATION: 7 Months
TOOLS: Unreal Engine 5
CONTRIBUTIONS TO THE PROJECT:
· Project Manager:
- Manage production over multi-timezone team (10 people) form 4 different timezones;
- Run daily communication with stakeholders, delivered client requirement to the team and negotiated when change is needed;
- Use Click Up and JIRA manage team;
- Manage workflows using the Sprint methodology, which included Sprint planning, Sprint backlogs, and clear goals for each cycle.
· Programmer:
- Audio Analysis: Performs foundational audio analysis using UE Blueprints, with Python integration enabling detailed multi-channel
breakdown;
- Music Emotion Recognition: Leverages external Python calls to identify the emotional tone of the music in real time;
- Seamless Integration: Incorporates a WebSocket plugin in Unreal Engine to connect and synchronize external data sources effortlessly.