Real-time video avatars are getting cheap—but not yet emotionally smart

LemonSlice is building real-time, photorealistic avatars that can interact on video calls. Their approach uses a video diffusion model trained on high-quality audio data to capture emotional expressiveness. They achieve interactivity by using causal attention masks and single-step denoising, but face error accumulation

over long sessions—a problem they claim to have solved with a novel method, though details are undisclosed. Cost is a key differentiator: they claim per-call cost is comparable to pure voice models, despite the heavy pixel data. This opens consumer use cases like

language learning and AI sales calls. However, the model is not yet fully controllable—emotional reactions are still awkward, and they're building an 'emotional engine' to predict actions from audio and text input. The long-term bet is an end-to-end EQ model that takes user

video/audio and outputs avatar video/audio, with a separate IQ model for reasoning. This is a speculative path, but early papers show proof-of-concept. The practical signal for engineering teams is that real-time video generation is becoming cost-viable, but governance and orchestration challenges remain significant.