Real Time Augmented Reality Meets the NFL

In September, my beloved Atlanta Falcons played the Jacksonville Jaguars in an NFL game in London. If you don’t follow the NFL, you might think it unusual that they played in Europe, but actually they play a handful of games in Europe every year. The unusual part was that the game looked like the picture above. Not the traditional warriors of the gridiron that we’re accustomed to, but animated caricatures of football players that looked like kids action figures.

This was Toy Story Funday Football.

Through a collaboration between ESPN and Pixar (who are both owned by Disney) the game was shown like this, with each play broadcast in real time in the animated toy style. The technological foundations behind this broadcast have been nearly a decade in the making.

It all started in 2014 when the NFL started installing RFID tags in each players’ shoulder pads as well as key objects on the field (first down markers, end zone pylons, even the ball). This enabled tracking of positions of all players and objects during the game. Data generated by this provides a mountain of metrics about player performance: how fast they move on the field, how far they run during a typical game, and the impact generated by collisions.

All of that tracking data is great, and eventually the technology developed to the point that player locations could be tracked in real time. But that just gets us dots moving around the field. It doesn’t get you the real movement of the quarterback throwing a pass, players reaching for a first down, or defenders diving for a tackle. For this, Pixar had to combine the location data with optical tracking and limb tracking through infrared cameras to generate a full body animated representation of each players’ movements.

The uses of this technology expand far beyond sports. With the advancement and ubiquity of virtual reality, visualizing the positions and movements of objects in real time has numerous benefits for not only entertainment, but medical, military, space exploration, and a host of other fields. As VR headsets become cheaper and more common, this technology can create immersive experiences by putting users into the environments they’ve previously only been able to observe from a camera. And as for me, I can attest to the realism of the VR visualization: the Falcons lost to the Jaguars 23-7, which is perfect representation of how they usually play.