Production AI and Private Infrastructure

RelayIQ - Real-Time Communications Intelligence

Private real-time communications intelligence combining WebRTC and custom SFU media infrastructure with local speech, language models, RAG, recording, composition, and central or edge inference.

ContextOPTIME-developed product family
PeriodPart of an approximately six-year product program
RelationshipAI and intelligence layer inside an OPTIME-developed product family
Team footprintApproximately 20 engineers contributed across the broader Relay product program.
Real-time communicationsPrivate AISpeech AIMedia systemsKnowledge systems

The system

RelayIQ is the intelligence layer of the OPTIME-developed Project Relay communications family. Because OPTIME engineers both the real-time communications path and the private AI processing path, speech, meeting, search, privacy, and workflow functions can operate directly alongside live media inside customer-controlled infrastructure.

Participants and devices connect through WebRTC and custom signaling to a custom selective forwarding unit. GStreamer, NVIDIA DeepStream, and nvinfer connect selected live media to local speech, video, and language-model processing rather than requiring every communication stream to be sent to unrelated public AI services.

The platform supports different deployment boundaries. NVIDIA L40S serves central private or on-premises environments and larger aggregate workloads. NVIDIA Jetson Thor T5000 supports distributed edge deployments, while L40S can also serve larger edge installations. Storage and retrieval can remain local or connect to customer-selected external pgvector and S3-compatible infrastructure.

Engineering relationship

RelayIQ is not a separate customer engagement or a second 20-engineer program. Approximately 20 engineers contributed across the broader Relay product family over roughly six years; those contributors were not assigned exclusively to RelayIQ.

OPTIME owns the communications, media, recording, composition, storage, and AI integration layers needed to make intelligence part of the platform rather than an external afterthought.

Project Relay demonstrates the multi-tenant communications platform. Project RelayIQ demonstrates the speech, language, retrieval, and communications-intelligence layer built on that same foundation.

Engineering constraints

  • Process speech and selected live media without separating AI latency from communications latency.
  • Preserve participant, tenant, meeting, recording, language, and permission context across the real-time path.
  • Support central and edge compute profiles without presenting one hardware topology as appropriate for every customer.
  • Keep recordings and participant tracks useful for later search, analysis, archival, and recomposition.
  • Allow local data control while supporting customer-selected external vector and object storage.
  • Perform live privacy checks as bounded system signals without claiming guaranteed detection or legal compliance.

What OPTIME engineered

  • Custom signaling and a custom SFU forming the real-time communications foundation.
  • WebRTC ingest into GStreamer, NVIDIA DeepStream, and nvinfer processing paths for local media and AI execution.
  • Parakeet-TDT as the primary local speech-recognition path, with optional Whisper for multilingual speech processing.
  • Llama 3.1 integration for meeting summaries, action-item extraction, communications context, and related intelligence workflows.
  • Live translation, TTS, semantic search, RAG, and privacy-checking integration inside the communications platform.
  • A recorder that preserves participant media as separate tracks for analysis, archival, and later composition.
  • A custom FFmpeg-library-based composer that combines multiple tracks into an output using a requested layout.
  • Local pgvector and storage plus optional customer-selected external pgvector and S3-compatible storage for files and recordings.

Architecture

  1. Participants and devices

    Users join calls, meetings, and communications workflows through the Relay application layer.

  2. WebRTC and custom signaling

    Custom signaling establishes sessions and connects participants to the platform’s custom SFU.

  3. Custom SFU

    The media layer routes live participant streams while preserving session and tenant context.

  4. GStreamer, DeepStream, and nvinfer

    Selected media is ingested for local speech, video, and inference processing.

  5. Parakeet-TDT or optional Whisper

    Primary local transcription uses Parakeet-TDT; Whisper supports optional multilingual speech recognition.

  6. Llama 3.1, TTS, and RAG

    Language, voice, retrieval, and workflow functions operate with meeting and communication context.

  7. Recorder and custom composer

    Separate participant tracks are retained and can be recombined by FFmpeg-library-based software into requested layouts.

  8. Local or selected external storage

    Local storage and pgvector can be complemented by external pgvector and S3-compatible stores chosen by the customer.

  9. Central or edge NVIDIA inference

    L40S supports central and higher-capacity deployments; Jetson Thor T5000 supports compact distributed edge operation.

Key engineering decisions

Own the media path and the AI path

The same platform controls WebRTC sessions, SFU routing, media processing, model execution, recording, and application context, reducing boundaries between communications and intelligence.

Support central and edge deployment explicitly

L40S targets central private infrastructure and larger aggregate workloads; Jetson Thor T5000 targets smaller distributed deployments, with L40S also available for higher-capacity edge sites.

Preserve participant tracks

Recording individual tracks retains evidence for speaker-level analysis, archival, quality workflows, and later recomposition instead of collapsing all media at capture time.

Make external storage optional

Local pgvector and file storage provide a customer-controlled baseline, while external pgvector and S3-compatible systems remain available when the chosen architecture requires them.

Platform evolution

The broader Relay program established multi-tenant communications, media, recording, composition, and storage foundations before and alongside the intelligence layer.

RelayIQ extends that foundation with representative capabilities including live transcription, multilingual speech processing, translation, meeting summaries, action items, semantic search, RAG, and live privacy checking. Additional workflows are intentionally omitted.

Current capability

The platform combines custom WebRTC/SFU infrastructure with local speech and language processing, RAG, recording, composition, storage, and customer-controlled deployment options.

Primary local speech recognition uses Parakeet-TDT, with optional Whisper for multilingual use. Llama 3.1 supports communications and meeting intelligence, while TTS provides voice output without naming an unverified public model.

Technology & Engineering Role

WebRTC
Real-time participant media ingest and communications transport.
Custom SFU
Tenant-aware routing of live participant streams.
Custom signaling
Session establishment and access to the custom media platform.
GStreamer
Real-time media pipeline construction and processing integration.
NVIDIA DeepStream
GPU media analytics and inference pipeline integration.
nvinfer
Local model execution inside DeepStream processing paths.
Parakeet-TDT
Primary local speech-recognition path.
Whisper
Optional multilingual speech-recognition path.
Llama 3.1
Meeting, language, summary, action-item, and communications-intelligence workflows.
TTS
Voice-output component without an unverified public model claim.
RAG and pgvector
Semantic retrieval across meeting and communications knowledge.
S3-compatible storage
Optional external storage for files and recordings.
FFmpeg libraries
Custom multi-track composition into requested output layouts.
NVIDIA L40S
Central private inference and higher-capacity edge deployments.
NVIDIA Jetson Thor T5000
Compact distributed edge inference.

Related engineering

Production AI Systems EngineeringReal-Time AI & WebRTC EngineeringTelecom & Real-Time Communications EngineeringPrivate AI Infrastructure & Inference OptimizationVideo, Audio, Broadcasting & Streaming EngineeringReal-Time Voice and Video AI AgentsAI-Powered Communications, UCaaS and Contact-Center PlatformsPrivate and On-Premises AI PlatformsAI Inference and ServingMedia and Computer VisionNetworking and InfrastructureMessaging and Distributed SystemsP2P + SFU platformRelay - Multi-Tenant Communications and Collaboration Platform

CONTACT US

Tell us about your project, and let’s create something together

Austin, Texas

Distributed engineering teams across North America, Europe, the Caucasus, and Latin America.

[email protected]

We use the information you submit to respond to your inquiry and process it through the service providers required to operate this form.