In animation and VFX production, sound design often begins where the visual pipeline ends – but what if the pipeline itself could kickstart the audio process? Christoph Groß-Fengels built a set of AI-powered tools that do exactly that: they watch your edit, understand what’s on screen, and suggest sounds before a sound designer even opens a session.

In this 45-minute talk, Christoph demonstrates a custom Python-based workflow designed to slot into existing post-production pipelines. The system automatically detects every cut in a video, extracts key frames, and sends them to a vision AI (Google Vision API) for content analysis – identifying objects, environments, actions, and mood. Based on these visual tags, it searches a local sound effects library for matching audio and assembles a first-pass sound bed. On top of that, it generates a synchronized MIDI file aligned to every edit point, giving composers a rhythmic and structural blueprint that mirrors the visual rhythm of the piece.

For teams working on animation or VFX-heavy projects, this means the sound department gets a head start while picture is still being finalized. Instead of waiting for picture lock, the pipeline can generate preliminary sound passes that evolve alongside the visual edit – a workflow that mirrors how previs and techvis already function on the image side.

Christoph also covers supporting automation tools he developed: local speech recognition (Whisper) for metadata tagging of audio assets, API-driven loudness normalization to broadcast standards (EBU R128), and automated audio-video muxing – all eliminating repetitive manual steps familiar to anyone running a production pipeline.

Through live demos and real examples, the talk honestly examines where AI delivers genuine time savings, where it produces surprisingly useful creative suggestions – and where human ears, dramaturgical instinct, and artistic judgment remain irreplaceable. A talk for pipeline TDs, production supervisors, sound designers, and anyone curious about where AI fits into the audio side of their production workflow.