Neural networks already shape almost every video you watch, but at both ends of a standard codec, not instead of it. Here is where neural coding ships in 2026, what it costs in silicon, and who gets paid, with the full report and slides.
Abstract
Neural coding is already mainstream in 2026, but it works around standard codecs rather than replacing them. Networks shape the picture before encoding, in phone cameras and in Netflix's encoding pipeline, and repair it after decoding, in YouTube's Super Resolution, at Meta and in TV processors; the bitstream in between is still AV1, HEVC, AVC or VVC, decoded by fixed-function hardware. End-to-end neural codecs ship only in closed loops where one company controls both ends: Microsoft's MLVC in Teams calls, the tokenizers inside every voice and video model, neural textures in one game. Mass playback of neural bitstreams waits for H.267, due at the end of 2029, and a silicon cycle after it. The reason is in the chip: the only published neural video decoder design needs about 12 times the energy per pixel of a measured HEVC decoder, and a fast research codec needs around 22 TOPS for 1080p30. The money follows the standards and the platforms; the best-known neural-codec startups were acquired.
Why I looked at this
Every few months someone tells me that AI codecs are about to replace H.264, HEVC and AV1. Most of what I read about it either comes from research papers, where the compression records are, or from vendors. Neither says much about what actually ships, to whom, and at what cost.
I have many years of fabless chip R&D and management behind me, including full-custom ASIC datapath design and multimedia camera-chip projects such as video codec accelerators, so I looked at it the way I would review a chip: energy per pixel, memory traffic, what a standard lets you fix in silicon, and who pays. I read standards documents and meeting reports from JVET, MPEG, JPEG, AOMedia and the IETF, company engineering blogs, chip papers, patent-pool rate cards, export-control texts and the press. Every number below is as of November 12, 2026, and money is in US dollars. The full report, with all the sources, is at the end.
Results at a glance
Figure 1. Where neural coding ships in 2026. Rows are ordered by how many people each domain reaches.
| Question | Short answer |
|---|---|
| Where does neural coding ship at scale? | Around the codec: camera ISPs, encode-side downscaling, playback super-resolution. End to end only in closed loops: calls, game assets, generative-AI tokenizers |
| Will neural codecs replace AV1, HEVC and VVC in playback? | Not before H.267 (version 1 at the end of 2029) and a device cycle after it; mainstream in the mid-2030s by my estimate |
| What does it cost in silicon? | About 12 times the energy per pixel of a hardware HEVC decoder, plus memory traffic and a requirement for bit-exact decoding |
| Who gets paid? | Platforms keep the savings; IP vendors and patent pools are paid per design and per device; neural-codec startups were acquired |
| Where can a company play? | Neural pre- and post-processing, closed-loop products, codec-plus-NPU silicon, and the evaluation and standards work in between |
Three ways in
"Neural coding" covers three different things, and most arguments about it come from mixing them up. A network can work around a standard codec, processing pixels before encoding or after decoding while the bitstream stays standard. It can sit inside a standard, as a tool every decoder must run, which means it has to be specified exactly. Or it can replace the codec end to end, with a learned encoder and decoder that only work as a pair.
Figure 2. Three ways a neural network enters a codec. Most of what ships in 2026 is the first.
The first needs no one's permission. The second needs a standard. The third needs control of both ends. That is the order in which they reach people.
Playback and streaming: neural at both ends, standard in the middle
The big services finished one codec transition and are starting the next, and neither involves a neural decoder. Netflix reported in December 2025 that AV1 carried about 30% of its viewing; YouTube says more than 90% of its catalogue, weighted by watch time, is available in AV1. AV2 was finished on May 28, 2026 with about 30% better compression than AV1, and within three weeks Chips&Media, Allegro DVT and VeriSilicon had announced AV2 decoder customers. Brazil's TV 3.0 began commercial VVC broadcasting in June 2026.
The neural part sits at both ends. Since 2022 Netflix has used a small neural network instead of a conventional filter to downscale its masters, which needs no change on any device and was preferred by about 77% of viewers in Netflix's tests. YouTube upscales videos uploaded below 1080p with machine learning on TVs, Meta runs super-resolution on its servers and on phones, and Samsung's flagship TV processor advertises 768 upscaling networks.
What does not exist is a neural bitstream on mainstream screens, and the reason is time. A playback format must decode on every device that will ever play it. AV1 took nearly eight years from finished specification to 30% of Netflix viewing.
Figure 3. From a finished standard to mainstream screens takes most of a decade.
The standard that could carry neural tools to every screen is H.267, the working name for VVC's successor. JVET issued its call for proposals in July 2026; responses are evaluated in January 2027 and the first version is due at the end of 2029. The classical track already saves about 28% over VVC, while JVET's neural in-loop filters save 6–14% at a decoder cost of 10 thousand MACs per pixel and up. I expect a classical core with a few neural tools where the bits saved justify the cost.
Photography: neural inside the camera, classical on the web
Every photo from a recent flagship phone has passed through neural networks before it is saved. Qualcomm's Cognitive ISP has run real-time semantic segmentation since 2023, and at CES 2026 Chips&Media and Visionary.ai showed a full ISP that runs entirely on an NPU. Then the result is saved as JPEG or HEIF.
JPEG AI, the first learned image standard, was published in 2025, with gains of 12.5–27.9% over VVC intra coding at decoder costs of 7 to 216 thousand MACs per pixel. I found no camera, gallery app or browser that uses it by default. The format the web adopted in 2026 is classical: Chrome 155 turned on JPEG XL on October 6, with a fast, memory-safe decoder that runs on any CPU. Neural coding wins inside one company's pipeline; formats that must open everywhere stay classical.
Calls: where end-to-end neural codecs ship
Calls are a closed loop: both ends run the same app and can fall back to a standard codec mid-call. That is why the first end-to-end neural codecs at scale are there. Microsoft's AI speech codec Satin has run in all two-party Teams and Skype calls since 2021. In July 2026 Microsoft open-sourced MLVC, a neural video codec now rolling out in peer-to-peer Teams calls.
Figure 4. In calls, a neural codec beats hardware HEVC by 65–76%, on Microsoft's own test set.
The headline 88% is against H.264; against the hardware HEVC already in the same laptops it is 65–76%, still large. MLVC runs at 540p and 360p using at most half of the NPU, and it gave up some compression to decode identically on hardware Microsoft does not own.
Not everyone chose this route. Opus added deep learning in 2024 as optional, backward-compatible extras. Meta built MLow, a classical speech codec, because more than 20% of its calls run on old ARMv7 phones and tens of millions of WhatsApp calls a day run on phones over ten years old. Neural codecs ship first where the devices are new and one vendor controls both ends.
Generative AI: the largest deployment nobody calls a codec
Every voice and video model runs on a learned codec. Speech models read and write the tokens of neural audio codecs such as Kyutai's Mimi, at 12.5 frames a second and 1.1 kbps. Video generators work in the latent space of video autoencoders such as LTX-Video's, which compresses 1:192. None of these is a format anyone else decodes, but they prove that learned compression works at scale when one company owns the encoder, the decoder and the hardware. For anyone serving generative video, tokenizer efficiency is a direct cost line.
Games: neural reconstruction is mainstream, neural assets are starting
NVIDIA's DLSS renders fewer pixels and frames and lets a network rebuild the rest. Neural texture compression shipped in Assassin's Creed Mirage, used only on selected assets because of its runtime cost.
Figure 5. When one company controls the stack, neural reconstruction spreads fast.
Satellite, drone and robot links are the best technical fit for end-to-end neural codecs, but they are a small market and increasingly export-controlled on both sides: China's drone export rules from November 18, 2026 even name semantic image compression as a threshold.
The silicon view
This is where my own background matters most. A chip team asks four questions about any new codec: joules per pixel, DRAM traffic per pixel, gates, and whether every decoder in the world computes exactly the same bits. On all four, a neural decoder in 2026 is where hardware HEVC was before its first chips.
Figure 6. Energy per decoded pixel: fixed-function decoders are an order of magnitude ahead.
The only published neural video decoder chip design, NVCA, needs 14.6 nJ per pixel; MIT's HEVC decoder chip needs 1.19 with its DRAM and 0.31 for the core. At 1080p30 that is roughly 900 mW against 74 mW. Microsoft's fast research codec DCVC-RT needs, by my sum of its own per-module table, about 175 thousand MACs per decoded pixel, or about 22 TOPS at 1080p30: half of the 40 TOPS that defines an AI PC, for one stream.
Figure 7. A neural tool costs tens of thousands of MACs per pixel; a whole neural decoder costs hundreds.
Two more costs matter. Memory: in the MIT chip about three-quarters of the system energy goes to DRAM, and neural decoders carry large feature maps between frames. Exactness: entropy decoding fails if two devices compute a probability differently, which is why every serious neural codec now uses integer arithmetic and why MLVC trades compression for identical decoding.
Figure 8. The split that ships: fixed-function hardware for the bitstream, a neural engine for the pixels.
The silicon that sells pairs the two. Tencent has more than 100,000 of its in-house Canghai codec chips deployed, and the second generation adds H.266 and on-chip super-resolution; IP vendors sell codec blocks and video NPUs side by side. If I were planning a media SoC now: fixed-function decoders for every standard you must play, including AV2; an NPU sized for enhancement at display resolution, without an extra trip through DRAM; deterministic integer modes; and H.267 as a design start after 2029.
Who gets paid, and where a company can play
Figure 9. The money follows the standards and the platforms.
Platforms keep the savings and sometimes give the code away, as Microsoft did with MLVC. IP vendors license decoders for every new standard. Patent pools collect per device: for VVC, $0.50 per phone and $1.00–1.50 per 4K TV in the US and other Region 1 markets. WaveOne went to Apple, iSIZE to Sony and Deep Render to InterDigital. I found no stand-alone company earning its living by licensing an end-to-end neural video codec.
For a company, the open plays are neural preprocessing for video services, display-side enhancement for chip makers, closed-loop neural codecs for conferencing and private links, codec-plus-NPU silicon, and the evaluation and conformance tooling everyone shipping neural tools will need. Standards work on H.267 needs patience and a licensing strategy.
Claims to discount, and what to watch
Ask any neural-codec claim for four things: the anchor, the metric, the resolution and the decode cost. "88% savings" is against H.264 at 360p; "85–96% less video memory" is a demo scene, against 30% in the shipped game; "JPEG AI is a standard" has no product behind it yet.
The dates that matter: the H.267 evaluation in January 2027, YouTube's first AV2 streams in early 2027, H.267's first test model in the second half of 2027, whether MLVC spreads beyond Teams, the first product that uses JPEG AI by default, and NPU budgets in phones and PCs. Version 1 of H.267 at the end of 2029 starts the silicon cycle for neural tools in mainstream playback.
Download the full report and slides
The full report covers each domain in more detail, the silicon arithmetic, the standards and patents, a table of claims against evidence, and all the sources.
Open questions
A few questions here have no public answer yet, and they are the ones I find most interesting technically: how close a neural decoder can get to bit-exact integer arithmetic across different vendors' NPUs before it loses its compression advantage; what neural post-processing costs in DRAM traffic per pixel; at what cost per pixel a neural tool stops paying for itself in an H.267 decoder; which closed systems beyond calls can adopt end-to-end neural codecs first; and how to measure fidelity when a decoder generates detail rather than reconstructing it.
These are areas we can explore together in the years ahead, through technical exchange and technical collaboration: i@tanshuai.com.
About the author
Tan Shuai is a technologist and founder with more than twenty years of work across chips, hardware and system software, and 20 granted patents. He has many years of fabless chip R&D and management behind him, including full-custom ASIC datapath design and multimedia camera-chip projects such as video codec accelerators. His work has taken him through Silicon Valley, New York, Chicago, Taiwan, the Greater Bay Area (Shenzhen and Hong Kong), Vietnam and Malaysia. Today he works as an AI researcher and developer.
Technical discussion: i@tanshuai.com.