FastFrames-L: Apple's 14B Text-to-Video Model With Interactive Latency

Apple's FastFrames-L is a 14-billion parameter AI model that generates 480p video from text prompts at interactive latency under 36 milliseconds.

By Sci Herald Editorial Team ·

Released on Hugging Face by apple — view the original study

FastFrames-L: Apple's 14B Text-to-Video Model With Interactive Latency

TL;DR: Apple has released FastFrames-L, a 14-billion parameter AI model that generates 480p video from text descriptions. The model achieves what the researchers call "interactive latency," producing video frames in just 36 milliseconds.

Key Facts

FastFrames-L at a Glance

SpecificationDetail
PublisherApple
Model Size14 billion parameters
TaskText-to-video generation
Output Resolution480p (832 × 480)
Frame Rate16 fps
Licenseapple-amlr
Hugging Face Likes354

What's New

FastFrames-L represents Apple's entry into high-speed video generation AI. The model uses a two-step diffusion process that the researchers have optimized for speed without sacrificing quality. Unlike many video generation models that can take seconds to produce output, FastFrames-L achieves what the developers describe as "interactive latency" - the ability to generate video quickly enough for real-time applications.

The model is part of a pair released by Apple, with FastFrames-S (1.3B parameters) serving as a smaller, faster alternative. Both models use an autoregressive approach, meaning they generate video frames sequentially while maintaining consistency between frames.

Figure 2 : Inference pipeline and latency profiling for one autoregressive generation. State-of-the-art few-step autoregressive video diffusion models generate each new frame by first updating the KV cache using the previous frame’s denoised latent ( 𝒛 0 i − 1 {\bm{z}}^{i-1}_{0} ) through a DiT eva
Figure 2 : Inference pipeline and latency profiling for one autoregressive generation. State-of-the-art few-step autoregressive video diffusion models generate each new frame by first updating the KV cache using the previous frame’s denoised latent ( 𝒛 0 i − 1 {\bm{z}}^{i-1}_{0} ) through a DiT eva
Figure from the paper (arXiv:2611.23226, arXiv.org perpetual non-exclusive license)

Benchmarks

The model card provides detailed latency measurements for the smaller FastFrames-S variant running on a single GPU:

ComponentCallsLatency per callTotal latency
DiT evaluation, t=1000116.1 ms16.1 ms
DiT evaluation, t=833116.2 ms16.2 ms
KV refinement12.2 ms2.2 ms
Pixel decoder10.9 ms0.9 ms
Total35.4 ms (paper: 36 ms)

Scores as reported by apple on the model card; not independently verified.

FastFrames-L: benchmark scores reported on the model card, compared with the other models in the card's table.
FastFrames-L: benchmark scores reported on the model card, compared with the other models in the card's table.
Chart: Sci Herald, from scores reported on the model card

These measurements show how the model breaks down the video generation process into discrete components, with the diffusion transformer (DiT) evaluation representing the most computationally intensive step. The total latency of approximately 36 milliseconds means the model can theoretically generate about 28 frames per second - faster than standard video playback rates.

Figure 3 : Comparing pixel decoders. Our pixel decoder (right) renders thin structures and details like hair and raindrops while being two orders of magnitude faster than Wan’s VAE (left). A single linear projection (middle) cannot render sharp outputs, but is sufficient for recognizing the scene an
Figure 3 : Comparing pixel decoders. Our pixel decoder (right) renders thin structures and details like hair and raindrops while being two orders of magnitude faster than Wan’s VAE (left). A single linear projection (middle) cannot render sharp outputs, but is sufficient for recognizing the scene an
Figure from the paper (arXiv:2611.23226, arXiv.org perpetual non-exclusive license)

How to Run It

The model requires Linux x86-64 with an NVIDIA GPU and a CUDA 12.8-compatible driver. Installation uses the uv package manager:

``bash uv sync ``

After downloading the model weights and third-party components, users can generate video from text prompts using:

``bash uv run python scripts/generate.py --model ff-l --checkpoint-dir checkpoints \ --prompt "A corgi surfing a wave at sunset" --seed 0 --output outputs/corgi.mp4 ``

This command produces an 832 × 480 video with 81 frames at 16 frames per second. The repository also includes an interactive demo featuring LingBot, an optimized version that provides 16x faster performance than the original implementation and enables fluid camera-conditioned control.

Figure 4 : Qualitative comparisons. We compare FastFrames -S with Causal Forcing++ at 1.3B parameters and FastFrames -L with Krea-Realtime at 14B. Each example shows frames at 0 0 and 5 5 seconds. FastFrames -S achieves comparable visual sharpness to Causal Forcing++, such as the details in the scul
Figure 4 : Qualitative comparisons. We compare FastFrames -S with Causal Forcing++ at 1.3B parameters and FastFrames -L with Krea-Realtime at 14B. Each example shows frames at 0 0 and 5 5 seconds. FastFrames -S achieves comparable visual sharpness to Causal Forcing++, such as the details in the scul
Figure from the paper (arXiv:2611.23226, arXiv.org perpetual non-exclusive license)

License and Commercial Use

FastFrames-L is released under Apple's apple-amlr license, which is a proprietary license agreement. The licensing terms for third-party components vary: LingBot-World v2 code and weights follow a CC BY-NC-SA 4.0 license, while other FastFrames code uses the Apple Sample Code License. Developers should carefully review all license terms before using the model in commercial applications.

Limitations

The model card does not explicitly document limitations or potential failure modes. As with any generative AI system, users should evaluate the model's performance on their specific use cases and verify output quality and appropriateness for their applications. The researchers acknowledge building upon several existing works including Wan2.1, Self-Forcing, and Causal Forcing++, suggesting the model may share limitations common to these architectures.

Frequently Asked Questions

Is FastFrames-L free for commercial use?

The model uses Apple's proprietary apple-amlr license, which means commercial use requires reviewing and complying with Apple's license terms. Some components like LingBot use a CC BY-NC-SA 4.0 license that prohibits commercial use.

What hardware do I need to run FastFrames-L?

The model requires Linux x86-64 systems with NVIDIA GPUs and CUDA 12.8-compatible drivers. The benchmarking example for the smaller FastFrames-S model uses a single GPU, while FastFrames-L benchmarking uses eight GPUs.

Can I run an interactive demo with FastFrames-L?

Yes, the repository includes an interactive demo featuring LingBot that can be launched locally. The demo runs on localhost:8000 and provides camera-conditioned control for video generation.

What's the difference between FastFrames-S and FastFrames-L?

FastFrames-S is a 1.3-billion parameter model while FastFrames-L is a 14-billion parameter model. The larger model likely produces higher quality video but requires more computational resources to run.

Does FastFrames-L require an internet connection to run?

No, once downloaded and installed, the model runs locally on your hardware without requiring an ongoing internet connection for inference.

Source: apple/FastFrames-L on Hugging Face