Hardware Video Decoding and Encoding on Windows and Mac: D3D11, CUDA, Vulkan, OpenGL, and Metal Explained

Key takeaways

Hardware video decoding and encoding hand the heavy codec work to fixed-function engines on the GPU instead of the CPU. On Windows, apps reach those engines through D3D11VA, D3D12, NVIDIA's CUDA-based SDK, AMD AMF, Intel oneVPL, or Vulkan Video; on a Mac, through VideoToolbox. The result is lower CPU load, faster exports, and frames that can stay in GPU memory.

• GPUs carry dedicated video engines (NVIDIA NVDEC and NVENC, Intel Quick Sync, AMD VCN, Apple's media engine) that decode and encode separately from the graphics cores.
• Windows reaches them through several APIs: D3D11VA and D3D12 video, NVIDIA's CUDA-based SDK, AMD AMF, Intel oneVPL and Media SDK, and Vulkan Video. Macs use VideoToolbox.
• The real gain is often lower CPU load: in our test, hardware decoding of 4K H.264 used about 95% less CPU time than software decoding.
• Hardware encoding is fast but spends more bits for the same quality. Our GPU HEVC encode matched x265's VMAF score 3.3 times faster, using 15% more bits.
• A zero-copy pipeline keeps frames in GPU memory between decoding, processing, display, and encoding, with bridges between D3D11, CUDA, OpenGL, Vulkan, and Metal.

What happens to a frame

Every video app runs the same four stages: decode the compressed file into frames, process them (scaling, color, AI models), then display them on screen or encode them into a new file. Each stage can run on the CPU or the GPU, and each hand-off between the two costs time.

The frames are bigger than most people expect. A 4K frame in 10-bit 4:2:0 (the P010 format GPUs use) takes 3,840 × 2,160 × 1.5 samples × 2 bytes, about 24.9 MB. At 60 frames per second, that's 1.5 GB every second, in each direction, if frames bounce between system memory and GPU memory. A pipeline that decodes, processes, and encodes on the GPU without that round trip is called zero-copy, and it's where most of the speed in modern video software comes from.

The video engines inside your GPU

Video decoding and encoding don't run on the same cores that draw games or run AI models. GPUs carry separate fixed-function blocks for them:

NVIDIA states it plainly: its GPUs contain hardware decoders and encoders separate from the CUDA cores. Two consequences follow. First, hardware decoding and encoding barely slow down other GPU work, so an AI model can run while the video engine decodes the next frames. Second, what an engine can do is fixed in silicon: codec, bit depth, and chroma format support depend on the GPU generation. NVIDIA's support matrix, for example, lists 4:2:2 encoding as new with its Blackwell generation. Check your vendor's matrix before you assume a codec is accelerated.

The decode and encode APIs on Windows and Mac

One video engine can usually be reached through several APIs. Each has its own strengths:

Software decoding and encoding (libavcodec, x264, x265, SVT-AV1) remain the fallback for everything the hardware can't handle, and the reference for quality.

The five pipelines: D3D11, CUDA, OpenGL, Vulkan, and Metal

Decoding produces frames; a pipeline decides where they live and who works on them next. Each graphics or compute API is a different kind of pipeline:

• D3D11 is the Windows workhorse for video. Decoders write straight into D3D11 textures, the video processor converts and scales them, and the swap chain shows them. It's mature, works on every Windows GPU, and is the natural home for hardware-decoded frames.
• CUDA is NVIDIA's compute platform. Frames decoded by NVDEC can stay in CUDA memory for processing and go straight to NVENC. NVIDIA documents a CUDA–Direct3D 11 interoperability API, so CUDA results can become D3D11 textures without a copy through system memory.
• OpenGL is the most portable graphics API and still common in players and editors. On Windows it shares textures with D3D11 through interop extensions, NVIDIA's WGL_NV_DX_interop2 or the vendor-neutral GL_EXT_memory_object_win32, which imports D3D11 memory into OpenGL.
• Vulkan is the modern explicit API: graphics, compute, and, with Vulkan Video, decoding and encoding in one place, on any vendor. The VK_KHR_external_memory_win32 extension lets Vulkan share memory with D3D11 on Windows.
• Metal is Apple's GPU API for graphics and compute on Apple silicon. Apple describes the chip's unified memory as letting all of its parts access the same data without copying it between memory pools, so a frame decoded through VideoToolbox can be processed and shown without crossing a bus.

The hard part isn't any single API; it's the borders between them. A frame decoded with CUDA but displayed with OpenGL needs a bridge, and a bridge that falls back to system memory gives away the advantage of decoding on the GPU.

What our tests showed

We tested an ordinary laptop: an Intel Core i7-1360P with Intel Iris Xe integrated graphics, using FFmpeg 8.1. FFmpeg listed three hardware paths on it: D3D11VA, D3D12VA, and Vulkan.

Decoding 5 seconds of 4K H.264 (121 frames at 25 Mbps):

The CPU decoded H.264 quickly by spreading the work over many threads, but it used about 4.5 seconds of CPU time to do it. All three hardware paths finished about as fast while using roughly 95% less CPU time, which leaves the processor free for everything else in the pipeline.

Encoding 5 seconds of detailed 1080p footage at a 4 Mbps target:

The GPU's HEVC encoder matched x265's VMAF score 3.3 times faster, but needed about 15% more bits to do it; its H.264 encoder used 26% more bits than x264 and still scored lower. That's the usual trade: hardware encoders are tuned for speed and power, software encoders for quality per bit. On this driver, FFmpeg's Vulkan Video encoders didn't start, a reminder that support differs by GPU and driver. How we score video quality explains what VMAF measures.

How KwaFlux uses every path

KwaFlux doesn't bet on one API. Its media engine carries a plugin for each decode and encode path and bridges between the pipelines, so it can pick a working hardware route on each machine and keep frames on the GPU.

• Eight decode paths, tried in order. On Windows, the engine's configuration tries D3D11VA, then D3D12VA, NVIDIA NVDEC, AMD AMF, Intel oneVPL, Intel Media SDK, and Vulkan Video, and decodes in software only when none of them can take the file.
• Six encode paths. For encoding, it tries NVIDIA NVENC, AMD AMF, Intel oneVPL, Media SDK, and Vulkan Video before software. Its encoder table also includes Apple's VideoToolbox encoders.
• Four render backends. The picture is drawn with D3D11, OpenGL, Vulkan, or Metal. The default, AUTO, lets the engine choose, and a backend the platform doesn't support falls back to AUTO.
• Bridges that avoid system memory. NVDEC frames reach D3D11 through CUDA's D3D11 interop; D3D11 and OpenGL share textures through WGL_NV_DX_interop2 or GL_EXT_memory_object_win32; Vulkan shares memory with D3D11 through VK_KHR_external_memory_win32; D3D12 and D3D11 exchange frames through shared DXGI handles. When OpenGL has no efficient bridge on a machine, the player switches to D3D11 instead of copying every frame.
• One GPU per job. On laptops with two GPUs, the engine matches adapters by their LUID, so decoding, processing, and display don't bounce frames between the integrated and the discrete GPU.
• Recovery, not crashes. If Windows resets the graphics driver (TDR), the player rebuilds its D3D11 device. Every bridge has a CPU path, and an export whose GPU encoder can't start is encoded in software.
• AI on any recent GPU. On Windows, AI processing runs through ONNX Runtime with DirectML, which works on DirectX 12 GPUs from Intel, AMD, and NVIDIA. On macOS 13 or later, the GPU Acceleration Component runs supported features through Apple's Core ML. Which GPU you need and KwaFlux on Mac cover the hardware side.

Two settings let you steer it. Settings › Performance › Hardware Accelerated Decoding turns hardware decoding on or off (it applies after a restart), and AI Processor chooses Auto, GPU, or CPU for AI work. In the Converter, High-speed conversion › GPU acceleration encodes H.264 and HEVC on the graphics card. It's off by default: KwaFlux notes that it's faster but slightly lower in quality than software encoding at the same bitrate, which matches our test. Each file shows a GPU encoded badge, or Software encoded if the GPU encoder couldn't start.

How to check your own machine

List the hardware paths FFmpeg can use:

ffmpeg -hide_banner -hwaccels

Compare CPU time for software and hardware decoding. The -benchmark line reports user (utime), system (stime), and wall-clock (rtime) time:

ffmpeg -hide_banner -benchmark -i input.mp4 -an -f null -

ffmpeg -hide_banner -benchmark -hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -an -f null -

Encode HEVC on the GPU through D3D12 and compare it with x265 at the same target:

ffmpeg -hide_banner -loglevel error -y -init_hw_device d3d12va=gpu -filter_hw_device gpu -i input.mp4 -vf "format=nv12,hwupload" -c:v hevc_d3d12va -rc_mode VBR -b:v 4M -maxrate 6M -c:a copy gpu_hevc.mp4

ffmpeg -hide_banner -loglevel error -y -i input.mp4 -c:v libx265 -preset medium -b:v 4M -c:a copy cpu_hevc.mp4

Then score each against the source. Download vmaf_v0.6.1.json from the Netflix VMAF repository into the same folder first, because some FFmpeg builds don't include a built-in model:

ffmpeg -hide_banner -i gpu_hevc.mp4 -i input.mp4 -lavfi "[0:v][1:v]libvmaf=model=path=vmaf_v0.6.1.json" -f null -

If the hardware encode scores close to the software one at a similar bitrate, use it for drafts and quick shares; keep software encoding for archives. For codec choices themselves, see H.264 vs H.265 vs AV1.

Common problems

What this can't do

• Hardware decoding doesn't improve the picture. Decoders produce the same frames as software; only the work moves.
• Hardware encoding doesn't beat a slow software preset on quality per bit. It wins on speed and power.
• No GPU accelerates every codec. Support is fixed by the chip generation and the driver.
• Zero-copy needs matching hardware. When two APIs or two GPUs can't share memory, frames take the slower CPU route.

Final thoughts

Hardware decoding and encoding are about where the work happens: on fixed-function engines instead of the CPU, and ideally without frames leaving GPU memory. Turn on hardware decoding for smoother work, use GPU encoding when speed matters more than the last bit of efficiency, and measure both on your own machine.

KwaFlux decodes, processes, and encodes video on your own GPU, in the Converter and in every AI module. Media files are not uploaded for processing. It runs on Windows 10 (64-bit, version 1809 or later), Windows 11, and macOS 12 or newer on Apple Silicon. Importing files and exploring the app need no account. Signing in with an active paid plan is required only to export, convert, repair a file, or download a file from a link.