The Augmented Reality Of Live Sport

With Kevin Jenkins and Andrew Millar

Talk about being spoilt for choice. Over the last few weeks we’ve enjoyed watching the British and Irish Lions tour the country to take on the best of New Zealand. The titanic test series - even if it did all end up in a draw - was undoubtedly the biggest rugby event here since the 2011 Rugby World Cup final.

In parallel, Team New Zealand fought their way through the challenger’s field of the America’s Cup to take on current champions Oracle, where a dominant display saw them hold their nerve to win the Auld Mug 7 to 1.

Source: Wikipedia

Early morning and late nights aside, the contrast in spectator experience of these two events is fascinating.

Back in the day, watching the America’s Cup was a bit like racing snails. There was an eventual winner, but not a lot happened in any given five minutes. In fact the sailing was worse, because the camera was often back on shore.

Thirty odd years ago when New Zealand stunned the sailing world by becoming only the third country to win the Cup after taking it off Australia, we went on to stun the world again by introducing a whole new level of spectator experience. Ian Taylor’s Animation Research Ltd in Dunedin created wholly-original graphics, which not only made the action comprehensible to casual fans, but also a lot clearer to sailing nuts who wanted to know in detail what’s going on during a race.

Since then the distance between participants and spectators has become less and less. In addition to on-board cameras, these high-tech boats are covered in hundreds of sensors to provide real-time information to the crew, coaches and audience.

This tech extends to the crew themselves. America’s Cup sailors are fitted with biometric sensors to measure their fitness and psychological state and help optimize their conditioning. Spectators can even track heart rates of the five man crew throughout the race.

And our expectations continue to rise. An Auckland relative – part of the sailing tribe – grumbled when the coverage cut to wealthy supporters supping champagne on land, rather than the micro changes in weather and tactics in the race.

It’s hard to imagine a place further from the azure waters of Bermuda than a grey and frosty New Zealand morning, but thanks to the adoption of technology the physical distance between sailors and spectators is reduced, almost to feel like you’re part of the crew from the comfort of your couch.

Meanwhile, watching the first Lions-All Blacks test at Eden Park in Auckland, and the second test at Westpac Stadium in Wellington, pretty much felt the same as when the Lions last toured in 2005. Actually it pretty much felt the same as 1993.

Source: Wikimedia Commons

Sitting high up behind the posts at Eden Park watching replays at an acute angle on the big screen, while it rained, was still fun. The rugby was captivating and the crowd was in good spirits.

However, the distance between players and spectators hasn’t changed. Rugby matches have struggled in recent years to attract and retain crowds, and even die-hard rugby friends have started to concede that the rugby viewing experience might be better at home.

But what if we could combine the unbeatable atmosphere of a live game with the superior viewing experience afforded by technology at home, to reduce the distance between participant and viewer like never before? It’s easy – in a Pokémon GO world – to imagine a vastly different spectator experience.

Unsurprisingly, connectivity is the key to keeping audiences engaged, especially millennials. Levi Stadium – the state of the art stadium built for the San Francisco 49ers in 2014 and known as the “Field of Jeans” – is several generations ahead of anything in New Zealand.

It is the most digitally connected stadium in the world, with ethernet, cloud-based voice, Wi-Fi and broadcast services. It has over 1,500 Wi-Fi access points, which are designed to be 30 times faster than those at any other stadium.

But can technology also improve how we watch a game? Panasonic is planning Pokémon GO-style augmented reality for the stadium of the future. By covering the glass windows of a stadium box with a special film, the company claims it can project graphics, statistics, replays and live video into fans' field of view.

The projections are solid enough to be seen clearly, but transparent enough to not obscure the game. Team selections, player stats, live video feeds and replays of key moments like goals and fouls can all be projected onto the glass.

Back at Levi Stadium, Intel has installed 38 5K cameras, which together capture terabytes of data, with plans to support virtual reality for spectators to further reduce the gap between spectator and participant.

Technology has the potential to not only enhance the action on the field, it can also improve the stadium visiting experience of live sport.

The Barclays Centre – home of the Brooklyn Nets – is setting out to improve the spectator experience and entice people away from their wide screen TVs. Their plans include an in-house team app with seat upgrades featuring dynamic pricing (drops as the game goes on), and cameras that show various queues.

The Levi Stadium mobile app is designed to enhance every aspect of a fan’s stadium experience, from steering fans to their parking spots to identifying the least-crowded restrooms. From this app fans can order food and drink delivered directly to their seats or picked up at express windows, and watch instant replays from four camera angles.

Source: Wikimedia Commons

Why can’t we sit at Westpac Stadium and order a Garage Project IPA and some fresh pork dumplings from our seat? Why can’t we instantly call up information about the game when and how we want it?

Imagine if our stadia had all that? We’re starting to see the beginnings of this now. Launched for the Lion’s tour, fans attending the matches could use the All Black mobile app to replay tries, view highlights and watch alternative live footage from multiple different camera angles.

But if we take this concept further, what else would capture our attention? Imagine if the wearable sensors used by Team NZ for sailing could be adopted by the All Blacks. What if we could use an app to check the force Owen Franks brings to a scrum, or the distance of Beauden Barrett’s kick? How about instantly knowing how long Kane Williamson has been batting for, or “being inside his head” by seeing what he sees when he checks outfield placements and then readies to face the next ball?

Team NZ and Auckland have the perfect opportunity to stun the world yet again with a mind-blowing augmented reality regatta in four years’ time, and their track record on and off the water shows they will do just that. New Zealand will lead the world in spectator experience.

One day stadium sports in Aotearoa may get there too. If they get it right, the problem will be coping with demand as millennials – and post-millennials – flock back for big tribal, hyper-connected eventathons.