Target the existing product rather than replace it
The architecture respected the customer’s Intel Agilex device, RTL boundaries, ARM Linux environment, firmware, and media workflow.
Production AI and Private Infrastructure
CaptionEdge - FPGA-Executed Whisper for Real-Time Subtitles and Translation
Whisper Small and Large FP16 implemented for an existing Intel Agilex platform through custom RTL, a purpose-built model compiler, an ARM Linux driver, and firmware integration.
FPGASpeech AIReal-time mediaSubtitlesTranslation
The system
The customer already operated an FPGA-based media product built around an Intel Agilex-family device, ARM Linux, existing RTL, and an established firmware and media environment. OPTIME was engaged to add a substantial speech-AI capability to that product without replacing the hardware platform or taking ownership of the customer’s complete firmware stack.
The resulting feature executes Whisper Small and Whisper Large in FP16 through an architecture implemented specifically for the available FPGA fabric and DSP resources. Audio from the existing media path is prepared for inference, passed through the new execution layer, and returned to subtitle and translation workflows under the control of the host software.
This was not a conventional host application calling a general-purpose accelerator. The work required a compatible chain from model representation through compilation, RTL execution, host/device control, driver integration, and the customer’s existing firmware environment.
Engineering relationship
OPTIME delivered an independent AI feature inside an established customer product. The four-engineer program ran for two years and focused on the Whisper execution capability, its supporting compiler and driver, and integration with the existing platform.
OPTIME did not replace the customer’s FPGA product. It engineered a new execution capability specifically for that platform, including the Whisper model architecture in RTL, the compiler required to target it, the host driver, and integration into the existing firmware environment.
The completed implementation was delivered for integration into the customer’s existing deployed product family. OPTIME does not claim that firmware containing the new feature has been released to downstream customer devices.
Engineering constraints
- Fit Whisper Small and Large FP16 execution into the customer’s existing Intel Agilex accelerator environment and available DSP resources.
- Integrate with established RTL and firmware boundaries without redesigning the complete device.
- Maintain continuous media behavior, subtitle timing, and translation integration around a new inference path.
- Coordinate model compilation, host control, memory movement, driver behavior, and FPGA execution as one system.
- Preserve the customer’s board, firmware, media, update, and operational constraints.
- Validate a long-running real-time feature without publishing proprietary resource, timing, utilization, or compiler details.
What OPTIME engineered
- The Whisper Small and Whisper Large FP16 model architecture in RTL.
- An FPGA execution architecture using the Intel Agilex fabric and available DSP resources inside the customer’s existing accelerator design.
- A custom model compiler targeting the OPTIME RTL implementation rather than a generic accelerator interface.
- A custom ARM Linux driver for host/device control and runtime integration.
- C/C++ host integration connecting model control, the Linux environment, and the customer’s existing firmware.
- Integration with the customer’s existing RTL without claiming ownership of the complete FPGA or board architecture.
- Real-time subtitle and translation feature integration within the established media product.
- Profiling, functional validation, and integration support across the compiler, RTL, driver, and firmware boundaries.
Architecture
Whisper Small / Large FP16
The supported model variants form the source representation for the customer-specific execution path.
OPTIME model compiler
A custom compiler transforms model representation for the OPTIME RTL target without exposing its proprietary internals.
Whisper RTL architecture
OPTIME’s model architecture implements the required inference behavior in programmable logic.
Intel Agilex FPGA and DSP resources
Execution uses the existing device’s FPGA fabric and available DSP resources within the customer accelerator architecture.
Existing customer RTL integration
The new capability connects to established hardware functions without replacing the complete RTL platform.
Custom ARM Linux driver
The driver provides host/device control and connects the accelerator execution path to the Linux environment.
Existing firmware and media product
The customer’s firmware invokes and supervises the added feature within its established product behavior.
Subtitle and translation workflow
Recognition output is integrated into real-time timed-text and translation functions.
Key engineering decisions
Implement the model path in RTL
Whisper execution was engineered as a native capability for the available FPGA and DSP resources, not described merely as host-side FPGA assistance.
Build a matching compiler
A purpose-built compiler targets the OPTIME RTL architecture so model representation and hardware execution remain aligned.
Treat driver and firmware integration as part of inference
The usable feature required host control, a custom ARM Linux driver, existing RTL integration, and media-product behavior-not only the inference datapath.
Platform evolution
The completed feature extends an existing product family with a new AI execution path while preserving the customer’s established hardware and firmware foundation.
Verified capability
OPTIME completed and delivered the Whisper Small/Large FP16 implementation, custom model compiler, RTL architecture, ARM Linux driver, and integration support for the customer’s existing product.
The implementation was delivered for integration into an already deployed hardware family. Downstream firmware rollout and end-customer activation remain outside OPTIME’s verified visibility.
Verified result
Whisper inference itself executes through the FPGA implementation.
The completed implementation was delivered to the customer for integration into its existing product family.
Technology & Engineering Role
- Whisper Small
- Supported speech-recognition model variant for the FPGA execution path.
- Whisper Large
- Higher-capacity supported model variant targeting the same custom architecture.
- FP16
- Verified model precision used by the delivered implementation.
- Intel Agilex
- Existing customer FPGA family into which the new execution capability was integrated.
- FPGA
- Programmable execution fabric for the Whisper model architecture.
- RTL
- Hardware-level implementation of the model execution architecture.
- DSP resources
- Arithmetic resources used within the existing FPGA accelerator environment.
- Custom model compiler
- Transforms supported models for the OPTIME RTL target.
- ARM Linux
- Host operating environment controlling the accelerator feature.
- Custom Linux driver
- Host/device control and integration with the existing firmware environment.
- C/C++
- Host integration and driver-facing control software.