Ookla shares Speedtest data from CenturyLink Field, other stadiums

Ookla ad banner being flown over CenturyLink Field in Seattle. Credit: Ookla

Ookla ad banner being flown over CenturyLink Field in Seattle. Credit: Ookla

Anyone who follows Mobile Sports Report knows that I use the Speedtest app from Ookla to measure stadium network performance whenever I visit a sporting venue. While my one-man tests do show some measure of network power, I always dreamed of harnessing the results from many fans at the same game to see a better picture of the network performance.

Well, Speedtest’s creators think along the same lines, and conducted an experiment during an Aug. 25 Seattle Seahawks preseason game at CenturyLink Field in Seattle. You can read their very thorough post and neat results here, with some interesting twists — for instance, the cellular networks are way faster than the CenturyLink Wi-Fi, according to the Ookla results.

UPDATE: Ookla responded to our email and let us know that on Aug. 25, there were 252 Speedtests at CenturyLink Field, a great sampling to draw results from. Ookla also talked about tests from 12 different events at CenturyLink Field, and said in the email that across those events it saw 1,143 tests conducted.

Ookla also published some test result totals from other stadiums as well, including Levi’s Stadium, AT&T Stadium and Bank of America Stadium, but didn’t say when those tests were recorded, or how many tests were taken.

What we really like, however, is that Ookla’s tests show what our stadium tech report surveys have been showing — that overall, in-stadium network performance is steadily improving. Over time, more data like this can help dispel the still-lingering rumor that stadium networks don’t deliver good connectivity. Now if we could only get Ookla to partner with us to do league-wide or college-comparison speedtests… anyone ready for that idea?

NRG Stadium Wi-Fi ‘soft launches’ at Texans preseason game

Screen Shot 2016-08-26 at 10.35.17 AMThe Houston Texans gave their fans more to cheer about with a 16-9 preseason victory over the New Orleans Saints last Saturday, but what might have made a lot of people happy at NRG Stadium was the unofficial debut of the stadium’s new Wi-Fi network, which was available in a sort of “soft launch” mode.

We say “sort of,” because according to people who were at the game there was pretty heavy promotion of the new network’s availability, with bandwidth sponsor Comcast distributing flyers in seat cup-holders as well as making in-stadium announcements about the wireless connectivity. NRG Stadium had been one of the few NFL venues without Wi-Fi, but with the Super Bowl headed to Houston at the end of this season installing Wi-Fi became a priority.

Starting after the Final Four concluded this past spring, integrator 5 Bars and Wi-Fi gear provider Extreme Networks got busy, eventually installing approximately 1,250 Wi-Fi APs inside NRG Stadium. According to 5 Bars representatives, many of the APs in the seating bowl were installed under the seats, a deployment method that is becoming a trend in larger stadiums.

Though we don’t have any stats yet (since the network isn’t really “officially” launched) we did hear from network sources that there was a good uptake on the system, and we are looking forward to watching the Wi-Fi’s performance this season leading up to Super Bowl 51 in February. If any fans out there hit another Texans game anytime soon, send us a speedtest of the Wi-Fi!

Comcast bringing backbone bandwith to NRG Stadium Wi-Fi

Free Wi-Fi is something Houston Texans fans will be able to cheer about this season at NRG Stadium. Credit: HoustonTexans.com

Free Wi-Fi is something Houston Texans fans will be able to cheer about this season at NRG Stadium. Credit: HoustonTexans.com

With the new Wi-Fi network just getting ready to go live at Houston’s NRG Stadium, Comcast announced that it will provide backbone bandwidth to the network under a sponsorship deal, with two 100-gig pipes that will both be available to the stadium facility.

Michael Bybee, a Houston-based director of external affairs for Comcast, said the company has a large fiber network throughout the Houston area, and a longtime relationship with the NFL’s Texans, making the network sponsorship deal a great fit. Comcast supplies (or will soon supply) similar broadband bandwidth to a number of other large stadiums, including the San Francisco 49ers’ Levi’s Stadium, the Sacramento Kings’ new Golden 1 Center, the Atlanta Braves’ new SunTrust Park, the Denver Broncos’ Sports Authority Field, the Detroit Tigers’ Comerica Park, the Tennessee Titans’ Nissan Field and the Oakland A’s Oakland Coliseum, among others.

In Houston, where integrator 5 Bars is installing a new Wi-Fi network using Extreme Networks gear ahead of next February’s Super Bowl 51, Bybee said Comcast will have two 100-gigabit per second pipes coming into the stadium, where they will feed a main 10-Gbps circuit for the regular-season NFL Wi-Fi needs. Should the stadium require more circuits, Bybee said, the facility will be able to “seamlessly add additional circuits” given the surplus of bandwidth supplied by the 100-Gbps pipes.

Though there hasn’t yet been an official announcement of the network being ready for public use, sources tell us that fans at Saturday’s preseason game between the Texans and the New Orleans Saints should finally find fan-facing Wi-Fi (look for the xfinitywifi SSID), an amenity that had been missing at NRG Stadium in the past. Anyone who goes to the game — send us a speed test!

Wi-Fi Analytics: Taking the first steps

Wi-Fi antennas at Joe Louis Arena. Credit: Detroit Red Wings (click on any photo for a larger image)

Wi-Fi antennas at Joe Louis Arena. Credit: Detroit Red Wings (click on any photo for a larger image)

Even though the physical construction and deployment of a fan-facing Wi-Fi network seems like the biggest challenge facing a stadium’s information technology team, in reality everyone involved knows it’s just step one.

While turning on a live network is certainly a great accomplishment, once the data starts flowing the inevitable questions follow: Now that we have Wi-Fi, what do we do with it? And how do we find out who’s using it, why they are using it, and how can we use that information it to find out better ways to improve the fan experience while also improving our business?

Those “step two” questions can only be answered by analytics, the gathering of information about Wi-Fi network performance and user activity. And while almost every live network operator almost instantly uses performance numbers to help tune the system, plans to harvest and digest the more personalized information like end-user identification, application use and fan engagement are just getting started, even at the most technically advanced stadiums with Wi-Fi networks in place.

What follows here are some conversations with stadium tech professionals who are already running fan-facing Wi-Fi networks, exploring how they use Wi-Fi metrics and analytics to both enhance the game-day experience for fans while also building a base of information that can be used by both technical staffs and marketing organizations inside the team, school and venue organizations.

Even this small sample seems to suggest that while Wi-Fi networks may be somewhat pervasive in the larger stadiums across the country, the harvesting and processing of data generated by digital fan engagement is just getting started, with plenty of unanswered questions and experiments that have yet to bear significant fruit. Yet everyone we spoke with also had an unshakable confidence that getting metrics and analytics right was the key to wireless success over the long haul, and all are fully engaged in pursuing that goal. It may take longer than physical deployment, but the “step two” of learning from the networks is well underway.

Detroit Red Wings: Pushing past the initial learning curve

Editor’s note: This profile is from our most recent STADIUM TECH REPORT, the Q2 issue which contains a feature story on Wi-Fi analytics, and a sneak peek of the Minnesota Vikings’ new US Bank Stadium. DOWNLOAD YOUR FREE COPY today!

Now that the Wi-Fi network at Joe Louis Arena in Detroit is coming up on its second birthday, Tod Caflisch said network administrators can relax a bit on game nights. Early on, however, he remembers “babysitting” the network during games, watching live performance stats to make sure everything was working correctly.

Watching the live network performance statistics, Caflisch said, “I could tell if there were issues. If throughput looked a little flat, we might have to reboot a switch. It was important, because there was so much at stake.”

As former director of information technology for the NHL’s Detroit Red Wings (he recently left Detroit and is now with the Minnesota Vikings), Caflisch helped drive the deployment of an Extreme Networks Wi-Fi network at the “Joe.” Though Joe Louis Arena is only going to host games a little while longer — a new downtown arena is just around the corner — Caflisch said the team in Detroit is already heading down the learning curve of interpreting analytics, with big goals on the horizon.

Right now, some of the most interesting network statistics have to do with fan Wi-Fi usage, including total tonnage, which Caflisch said hit 14 terabytes of data for the Red Wings’ home games this past season. That number is one and a half times bigger per game than the first year the network was in place, he said.

Big spikes for a score

One of the more interesting results came when Caflisch mapped network data to game action, an exercise that showed that hockey games may have bigger data spikes and troughs than other sports.

“We saw that traffic spikes corresponded with scores, and we also had huge spikes during intermissions,” Caflisch said. “And there were huge craters during the periods of regular action.”

While Caflisch said “it was kind of cool” to watch the network action mapped to the game action, in the future he sees the ability for the Red Wings use such actionable moments to better engage fans.

“There’s got to be some kind of marketing potential” to connect with fans during a network-activity spike, Caflisch said. What that is, is still unknown. But using networks to more closely engage fans is a big part of the Red Wings’ road map, especially as Detroit builds out a “venue environment” around the new arena.

According to Caflisch, the team in Detroit is planning to build out a network surrounding the arena, in parking lots and public spaces, including lots of beacons for proximity engagement. Though DAS and Wi-Fi numbers can show where foot traffic goes in and around stadiums, the next level of analytics Caflisch sees as important is on fan spending behavior, on items like parking, concessions and in restaurants and bars near the arena. Future projects in Detroit, he said, might include beacon-generated discounts, like a free coffee at a nearby Tim Horton’s or a free beer at a nearby bar.

“The kinds of things you want to find out are what kind of money are fans spending, and how often do they buy,” Caflisch said. “Do they stick around after the game? Do they rush in at the start? That’s the kind of stuff you’re looking for.”

Of course to get some of that data Caflisch knows the team needs to convince fans to engage digitally, by downloading a team app and providing some information for identification. So far some efforts in that direction have been helpful in identifying fans not in the team’s ticketing database, especially fans coming across the border from Canada.

In Detroit, Caflisch said, the Wings are “now marketing to those people, trying to get them to more games for the same or less money.”

Baylor University: Enlisting fans to help pinpoint problems

When Baylor University built its new football mecca, McLane Stadium, the stadium technology department was often as nervous as a football team before a big game. Would the new fan-facing Wi-Fi work as planned? Would they be able to solve problems before they became big problems?

Baylor's McLane Stadium. Credit: Paul Kapustka, MSR

Baylor’s McLane Stadium. Credit: Paul Kapustka, MSR

“At the beginning, the questions we asked were along the lines of, ‘can we get through the day,’ ” said Pattie Orr, vice president for information technology and Dean of University Libraries and the public face of the McLane Stadium network. Now that the network team is a couple years into running stadium Wi-Fi, Orr can laugh a bit about the initial fears. But from the beginning, she said, analytics “were a big factor” in making sure the network was running right.

An Extreme Networks deployment, Baylor uses Extreme’s Purview analytics system, which Orr lauds for being “easy to use” and a “great console for real-time information during a game.”

Solving for 2.4 GHz and using fan input

Mostly that means watching the dashboards to see if any APs are causing any errors, something the network stats package can usually show clearly. One of the things the network crew learned quickly during the first season with Wi-Fi was that Baylor fans were using a lot more 2.4 GHz Wi-Fi devices than anyone had thought, meaning that there were more older phones in use that didn’t have the newer 5 GHz Wi-Fi chips.

“The first season we were about 50-50 between 2.4 GHz and 5 GHz, and that surprised us,” said Bob Hartland, director of IT servers and networking services at Baylor. “We had to prioritize for more 2.4 GHz.” This past season, Hartland said, the fan devices skewed closer to 60 percent using 5 GHz bands.

A Baylor "Wi-Fi Coach" helps a fan negotiate the network. Credit: Baylor University

A Baylor “Wi-Fi Coach” helps a fan negotiate the network. Credit: Baylor University

Baylor also added Wi-Fi to its basketball arena this past season, presenting a whole new set of problems, like devices trying to connect to APs across the smaller stadium. Though network analytics were a start, Baylor’s team found out that fan input could also help isolate where problems might be from a physical standpoint. Having a team of “network coaches” on hand also helped pinpoint the problems in a way that might be impossible just working from the network side of things.

This past year, Orr said Baylor added a feature to its stadium app to let fans “send a message to the Wi-Fi coach” with their row number and seat number if they were having a network problem. The coaches (part of most Extreme Wi-Fi deployments) also followed social media like Twitter to see if fans were reporting network problems.

“It’s fantastic to have the live [performance] data from your fans,” Orr said. With fan and network data and area knowledge in hand, the coaches and the network team could more quickly determine if it was a network or device problem, and respond more quickly to the issue. So more data = better solutions, faster.

“If you don’t have good access to analytics you can’t deal with fan [problems] in real time,” Orr said.

VenueNext and the Niners: Finding out who’s in the building

As one of the newer and more technologically advanced venues, Levi’s Stadium often gets noticed for its wireless networks, which set single-day records of 26 terabytes of data for combined DAS and Wi-Fi usage at Super Bowl 50.

A VenueNext beacon enclosure at Levi's Stadium. Credit: VenueNext

A VenueNext beacon enclosure at Levi’s Stadium. Credit: VenueNext

Though wireless performance is important to teams and fans, the information being gathered by the Levi’s Stadium app — built by VenueNext, the company created by the Niners specifically to construct stadium apps — may end up being among the most valuable digital assets, since it helps teams discover exactly who is coming in the building and how they are spending time, attention and dollars.

“We generate data for analytics,” said VenueNext CEO John Paul, talking about the role VenueNext plays as a stadium app partner. One of the more stunning facts revealed after the Niners’ first year at Levi’s Stadium was that via the stadium app, the team was able to increase its marketing database of fan names from 17,000 to 315,000, with even more impressive success in the details.

“We were able to find out things like how many games fans attended, and who they got the tickets from,” said Paul. Such data, he said, helps teams solve the classic problem of “having no idea who’s in the building on any given day.”

Knowing how many hot dogs can be delivered

While VenueNext’s value proposition may be centered on its ability to help teams gather such valuable marketing data, VenueNext itself relies on internal analytics to ensure the services its apps support — like express food ordering and in-seat food delivery — keep working smoothly during games.

After the first season at Levi’s Stadium, Paul said VenueNext learned that it needed to expose some of its data in real-time to fans — “to improve service during the event,” Paul said. One example is that now, if there are too many orders in a certain section, the app can send a message to fans that wait times might be longer than normal. Conversely, if a certain area of the stadium has idle kitchen capacity and runners, a team might send an in-app notification asking if fans want to order something, to create demand.

Over time, Paul said the VenueNext analytics might help teams find out where walk-up concession stands get overloaded by foot traffic, and maybe reconfigure stadium kitchen placements to assist with food delivery options. In the end, he said, it should be seamless to the fans, so that in-seat delivery becomes a regular part of a game-day experience.

“The fans should have no idea where the food comes from,” Paul said.

Stadium Tech Report: Buffalo Bills bridge Wi-Fi gap with Extreme Networks

Ralph Wilson Stadium in Buffalo. Credit: AP Photo/Scott Boehm

Ralph Wilson Stadium in Buffalo. Credit: AP Photo/Scott Boehm

When the Buffalo Bills undertook a $130 million renovation of Ralph Wilson Stadium in 2014, it was only natural to take took a look at the venue’s communications and networking infrastructure. They knew fans entering a newly modernized sports entertainment facility would have higher expectations about Internet access and connectivity. And there were some back-of-the-house challenges that needed resolution as well, according to Dave Wheat, chief administrative officer for the Buffalo, N.Y.-based National Football League franchise.

“Our own internal staff and the people we bring in — fire, emergency medical services, those groups — were having some real challenges communicating among themselves,” Wheat said. “The key for us was seeing the communications challenges throughout the stadium on game day.”

So a year later when the Buffalo Bills began examining their Wi-Fi infrastructure, they knew they wanted a system that gave fans lots of options – content where and when they wanted it, whether to access social media, the dedicated team app, email or simple web browsing, Wheat said.

“We watched a lot of our [NFL] peers installing Wi-Fi in their stadiums for the last 5 years and saw the investments that different clubs were making,” said Wheat, who added that the extra time gave the Bills the chance to watch the technology develop and evolve and become “more mainstream and reliable.”

The Wi-Fi backend consists of two ExtremeSwitching bonded S4 Chassis; yellow wires are single-mode fiber optic patch cables and the purple cables are for Wi-Fi infrastructure. Credit: Buffalo Bills / Extreme

The Wi-Fi backend consists of two ExtremeSwitching bonded S4 Chassis; yellow wires are single-mode fiber optic patch cables and the purple cables are for Wi-Fi infrastructure. Credit: Buffalo Bills / Extreme

They also got behind-the-scenes tours of competitors’ operations and networks during away games, which also helped shape the Bills’ requirements and buying decisions.

“We saw Gillette Stadium in Foxborough and used that to determine what we wanted and who we wanted to work with,” Wheat said. The Bills recognized the increasing demand for mobile connectivity, with some fans even attending with multiple capable devices. The Wi-Fi network needed to deliver services to these devices anywhere in the 70,000 seat capacity stadium.

The Bills looked at four different Wi-Fi vendors, narrowed it down to two, then chose Extreme Networks for its Wi-Fi and wired networks. “We worked pretty extensively with Extreme — their expertise in this space helped us understand our design better and plan for tremendous growth to cater to our fans,” Wheat said. They also worked with technology integrator Carousel Industries for its managed services expertise, especially at sports facilities.

Cutting through the concrete

Editor’s note: This profile is from our most recent STADIUM TECH REPORT, the Q2 issue which contains a feature story on Wi-Fi analytics, and a sneak peek of the Minnesota Vikings’ new US Bank Stadium. DOWNLOAD YOUR FREE COPY today!

As with any venue as old as Ralph Wilson Stadium (built in 1973), there were some technical challenges to work through before installing the Wi-Fi. The stadium has an extremely limited overhang from the upper decks; there are also a limited number of railings from which to hang equipment, or conceal it. Individual sections in the bowl also tend to be wide and long, according to Wheat.

Under-seat Wi-Fi AP at Ralph WIlson Stadium

Under-seat Wi-Fi AP at Ralph WIlson Stadium

“The biggest design challenge was that the entire lower bowl is 40 feet below ground and sits on shale rock, so it was a real challenge to get APs there,” Wheat added. Extreme came up with the under-seat solution, spacing the antennas about every five to eight rows, and about 13 to 20 seats apart.

Installing 211 under-seat APs in the lower bowl was also the most labor-intensive part of the Wi-Fi upgrade. Seats had to be removed, then concrete was saw-cut to different depths, depending on the number of cables and APs to accommodate. There was more than 5,000 feet of concrete cuts when all was done, which took nearly six weeks –- “a big challenge,” Wheat said.

In other parts of the stadium, the Bills relied on flag poles and overhangs for AP installation; the public Wi-Fi at Ralph Wilson Stadium ended up with 850 APs total, with 211 under-seat APs in the lower bowl. Total cost: About $4 million, according to Wheat.

Social media and internal operations

Fans are required to authenticate with Social Sign In to access Bills’ Wi-Fi on game day. The service allows users to sign in with Facebook or by providing their email address.

“As the network remembers you, we can deliver more personalized content and learn customer likes and dislikes,” Wheat said, adding that they’re still testing different ways to measure customer behavior. “The challenge for us as an entertainment provider is we have 70,000 people who will come see our game live,” he said. “And we only know the names of 14,000 people who purchased those tickets, and that often changes between the original buyer and the actual attendee.”

As the Bills learn more, their intention is to serve up content that’s more and more personalized. For the moment, the online activities of Bills fans mirrors what happens at other NFL stadiums around the league, according to John Brams, director of sports and entertainment for Extreme. Roughly 60 percent is social media usage -– Facebook, Instagram and Snapchat; iPhones hitting iCloud, either to update apps or upload photos, also constitute a significant amount of data usage. A third tier of fans is using apps for fantasy sports or individual NFL teams, Brams added.

So far, the Bills have clocked game-day totals of 24,334 unique users on the network, with a peak concurrent usage rate of 20,067 fans. Peak game-day data volume topped out at 3.4 TB, according to figures from the team.

Internally, the Bills use wireless for ticketing and entry scanning, including secondary scanning at entrances to premium areas; a Bills retailer uses the Wi-Fi for some of its pop-up shops around the stadium. “Our guest services ambassadors have mini tablets to assist customers with questions or if they need to do incident logging or tracking,” Wheat said.

There’s no in-seat ordering via Wi-Fi at Ralph Wilson Stadium, but technology isn’t the issue. “Infrastructure in the back of house is the impediment to that,” Wheat laughed. The Bills also use the Wi-Fi in its fieldhouse practice facility across the parking lot from the stadium, but not in the parking lots themselves.

The Bills will continue to work on giving fans that personalized experience. “If you’ve never been to a game before and are parking a mile away in a satellite lot, we want to give you GPS directions to get you from your car to the gate to your seat,” Wheat said. “We want to be able to tell you where the shorter lines at the concession stands are, using the technology to make the customer experience better.”

Wheat and his group are also in the process of looking at and installing Bluetooth-based beacon technology for the upcoming season. They’re also reworking the Bills’ mobile app for game-day use, adding wayfinding and other features.

“We want fans to experience Buffalo Bills football the way they want to,” Wheat said.

Golden State Warriors’ Wi-Fi network lease part of planned SignalShare assets auction

Screen shot of nGage Fan Feed. Credit: SignalShare

Screen shot of nGage Fan Feed. Credit: SignalShare

The contract covering the operation of the Wi-Fi network at the Golden State Warriors’ Oracle Arena is up for auction, as part of the fallout from a lawsuit involving alleged fraudulent business practices by Wi-Fi deployment concern SignalShare.

UPDATE, 7/7/16: According to the auctioneers, the auction is currently postponed, due to their claims of a bankruptcy filing by SignalShare. More details as we learn more.

SignalShare, which has installed and operated Wi-Fi networks in a number of large sports venues, including arenas used by the Detroit Red Wings, the Houston Rockets, the Sacramento Kings, the University of Maryland and others, is being sued for $7.8 million by NFS Leasing, an equipment leasing company, over a dispute involving allegedly fraudulent leases by SignalShare and SignalShare’s default on an agreement to pay back money obtained through those leases. As part of the ongoing legal proceedings, NFS has apparently scheduled an auction of SignalShare assets it claims, including leases, software code and hardware, for July 14 through Paul E. Saperstein Co., Inc.

So far, Mobile Sports Report has not been able to get any comments on the lawsuit or the auction from SignalShare, NFS, or any of the venues where SignalShare had installed networks. According to the auction site, NFS will make available for auction the contracts between SignalShare and the following list of teams and venues: The Golden State Warriors and Oracle Arena; the Carolina Hurricanes and PNC Arena; the Houston Rockets and Toyota Center Arena; the Detroit Red Wings and Joe Louis Arena; the Jacksonville Jaguars and Everbank Field; the Milwaukee Bucks and Brady Harris Arena; and the Las Vegas Sands Convention Center.

According to sources familiar with some of the SignalShare deals, some of the networks were run under a lease agreement, where the team or venue owners paid SignalShare a monthly fee for operation of the Wi-Fi network, with SignalShare retaining ownership of the actual equipment. According to legal documents filed in the case, NFS provided the financing for many of the existing SignalShare deals, as well as millions more in financing for deals NFS claims never actually existed. So far, there has been no public accounting for where the millions provided by NFS for the allegedly fraudulent leases ended up.

Live-Fi code also up for auction

While the leases are potentially interesting to many possible parties — firms who could take over the network operations, or who might be interested in purchasing the leased equipment — the asset with perhaps the most tangible worth is SignalShare’s “Live-Fi” software, a kind of customer portal program meant to help teams and venues engage more closely with fans and to also facilitate advertising sales. According to legal documents filed in the case, SignalShare owners apparently attempted to transfer the ownership of the Live-Fi code to a subsidiary firm to apparently keep it out of any claim proceedings, a move that was recently blocked when the courts granted an injunction requested by NFS.

Aside from whatever happens in the ongoing legal case and at the auction, for the venues involved the bigger question is more likely what happens to their existing or planned networks. In several of the mentioned deals, including the Jaguars, the University of Maryland and the Detroit Red Wings, SignalShare publicly partnered with Wi-Fi gear provider Extreme Networks; Extreme representatives declined to comment on any specifics of the SignalShare lawsuit.

One common trait shared by several of the SignalShare deals was that they involved Wi-Fi networks at arenas that were scheduled to be replaced or abandoned by the teams in the near future — the Warriors, Kings and Red Wings are all already building or planning to build new stadiums. The SignalShare “leasing” model may have seemed more attractive than spending the potentially millions in upfront costs for a network that may only have been used for a few years. The only thing for sure now is that the future of Wi-Fi at the venues mentioned seems to be on hold until the legal questions around SignalShare’s operations are answered.