Open-Source AI-Enhanced WebRTC

High-performance native WebRTC implementation with integrated AI capabilities

Architecture Overview

Native implementation combining WebRTC, speech processing, and AI inference

Optimized for Apple Silicon

Native C++ implementation leveraging Apple Metal framework
Hardware-accelerated AI inference on M1/M2 chips
Efficient audio capture and processing pipeline

NVIDIA CUDA Support

Ubuntu Linux compatibility with NVIDIA GPUs
CUDA-accelerated AI model inference
Optimized for server-side deployment

Core Components

Audio Processing

• Custom audio device implementation
• Real-time playback capture
• Low-latency processing pipeline

AI Integration

• Whisper.cpp for transcription
• LLama.cpp for inference
• Hardware-accelerated processing

Speech Synthesis

• eSpeak-NG integration
• Real-time audio generation
• Customizable voice options

Getting Started

Clone the repository and follow the setup instructions for your platform:

git clone https://github.com/wilddolphon2022/webrtcsays.ai
cd webrtcsays.ai
{Follow README.md}

Check the documentation for detailed build instructions for Mac M1/M2 and Ubuntu + NVIDIA configurations.