Multimodal perception system for real-time conversation

Show HN: Multimodal perception system for real-time conversation

Based, Multimodal perception system for real-time conversation works in show hn launch.

Profile

I work on real-time voice/video AI at Tavus and for the past few years, I’ve mostly focused on how machines respond in a conversation.One thing that’s always bothered me is that almost all conversational systems still reduce everything to transcripts, and throw away a ton of signals that need to be used downstream. Some existing emotion understanding models try to analyze and classify into small sets of arbitrary boxes, but they either aren’t fast / rich enough to do this with conviction in real-time.So I built a multimodal perception system which gives us a way to encode visual and audio conversational signals and have them translated into natural language by aligning a small LLM on these s

Within the show hn launch space, the company focuses on its core offering rather than a broad product line.

The company is listed at the Show HN launch stage of development.

Website: raven.tavuslabs.org