Fox Networks Group

My QA background goes back to console testing at Sony PlayStation and it shaped a conviction I still carry: quality is an engineering discipline, not a gate. On the Video Performance team at Fox, that discipline translates into building instrumentation so we see problems before viewers do.

Quality of Experience metrics — Stream Playback Initiation, Video Start Failure, Connection-Induced Rebuffer Ratio and abandonment — all surface in a shared Grafana dashboard, identical across our Apple TV, Roku and Fire TV apps. When the whole team reads the same dial, the conversation shifts from “is there a bug” to “where is the regression and which lever do we pull.” Observability first, opinions second. You keep streaming reliable at scale by giving the team visibility before the incident, not during it.

Solving the Hidden Challenges of Video Quality at Scale

Modern streaming runs across a stack where any layer can quietly degrade the experience: encoder, packager, CDN, ABR algorithm, DRM, player SDK or device firmware. The hard part is telling them apart.

A recent example involved Video Playback Failure spikes of 8 to 12 percent during UHD live events. The reflex is often to suspect a player regression. The data told a different story. The failures correlated with malformed fMP4 segments served from a specific CDN path, not anything the client app had shipped.

We isolated the cause because sessionlevel telemetry through Conviva, combined with controlled probes, pointed us at the bytes on the wire rather than the code in the app. Most “video quality” problems are observability problems first. You address them by investing in telemetry long before you need to debug an incident.

Balancing Innovation with Performance and Quality Standards

I lean on eval-driven development. Anything we ship touching a non-deterministic component — and AI tooling falls squarely there — gets a Promptfoo evaluation suite with measurable pass criteria before it merges. One recent integration went out with 88 assertions and a 100 percent pass rate as the gating bar.

The same philosophy applies to the test automation itself. A three-tier resolution pipeline in our internal AI helper library is designed so an AI-based fallback only fires when deterministic selectors fail and every fallback is logged for audit.

Innovation happens. It does not happen blindly. Treat AI components the way you treat any other production system, with SLOs, regression suites and rollback paths and adoption becomes durable instead of a series of demos.

Emerging Technologies Reshaping Video Engineering

Three major shifts are reshaping the future of video engineering and content delivery.

The first is agentic QA: small AI agents capable of driving an app, verifying state and reasoning about failures. They are moving from demo to dependable when paired with strong evaluation frameworks.

The second is the rise of multimodal vision models. Through an Appium-based engine for Android and Fire TV, these systems are finally becoming reliable enough to validate UI on platforms where traditional DOM inspection is not an option.

The third is the emergence of the Model Context Protocol as a standard integration layer for connecting AI systems with existing engineering tools without relying on bespoke glue code. We have been building internal MCP servers and a small library of slash commands so an on-call engineer can investigate a QoE alert conversationally, with the model pulling real session data rather than guessing.

Advice for Building Expertise and Leadership in Video Engineering

Three things matter most.

First, get close to the metrics your viewers actually feel: start time, rebuffer and failure rate. Those metrics matter at 8 p.m. on game night.

Second, build instead of only consuming. QA leaders who ship internal tools and small libraries accumulate leverage that pure script-writers never do.

Third, teach what you learn. Preparing material for initiatives like Engenious University and conferences such as Innovate QA forces a level of clarity that day-to-day work does not demand and almost every framework I now rely on was sharpened through the process of explaining it to others.