YouTube Video
Project Description
Digital Twin is a multimodal AI agent that preserves human identity through voice cloning and personality capture. Users complete a guided 20-question voice interview on iOS, which our backend orchestrates through multiple AI services: ElevenLabs Scribe transcribes responses in real-time, Instant Voice Clone captures their unique voice from recordings, and Conversational AI deploys an intelligent agent powered by their personality data stored in the Knowledge Base.
Core Functionality & Stability: Working prototype handles the complete pipeline: Clerk authentication → voice recording → real-time transcription → voice cloning → knowledge base population → agent deployment. All API endpoints tested and production-ready with async processing and error handling.
Technical Complexity: Orchestrates 6 APIs across 2 cloud services (ElevenLabs, Clerk) with multimodal processing: audio → text → voice synthesis → conversational AI.
Innovation: Unlike generic chatbots, we capture the essence of a person — their voice, stories, and values. 20 questions span 8 psychological categories designed to extract personality depth.
Real-World Impact: Preserves elderly relatives before cognitive decline, provides companionship for isolated seniors, creates lasting digital legacies.
Theme Alignment: Transforms voices (recordings) into cloned speech, leverages cloud AI services, orchestrates multiple tools into a cohesive conversational agent.
Tech Stack: Next.js 14, TypeScript, MySQL, SwiftUI, ElevenLabs (Speech-to-Text, IVC, TTS, Knowledge Base, ConvAI), Clerk Auth
Future: Visual avatar generation with Anam AI (backend API ready, frontend integration planned)
Git links
http://github.com/Anima-Felix/hackaton-backend
http://github.com/Anima-Felix/hackaton-frontend
Prior Work
We have a patent pending dimension adding time awareness to chatbots / llms, called SCN-LLM, added also as a trade secret to the main Anima Felix anxiety relief chatbot.