New Report: Green Bay’s Lambeau Field leads new NFL Wi-Fi deployments

Wave the flag, Wi-Fi has come to Lambeau Field! Photo: Green Bay Packers

Wave the flag, Wi-Fi has come to Lambeau Field! Photo: Green Bay Packers

When most NFL fans think of the Green Bay Packers and Lambeau Field, they think of frozen tundra — of Vince Lombardi roaming the sideline in his thick glasses and peaked hat, with visible breath coming through the face masks of behemoth linemen on the field. In the stands, they see the venerable fans braving the cold of northern Wisconsin in their snowmobile suits, with mittens wrapped around a bratwurst and a beer.

But do they think of those same Packers fans pulling out their iPhones and Samsungs to take selfies, and posting them to Instagram or Facebook? Maybe not so much.

The reality of 2015, however, finds us with fans in Green Bay being just like fans anywhere else — meaning, they want to be able to use their mobile devices while at the game. As the cover story of our most recent Stadium Tech Report series, we explore the details of bringing Wi-Fi to historic Lambeau Field, where late-season texting might carry the threat of frostbitten fingers.

Our PRO FOOTBALL ISSUE has 50-plus pages of insight and how-to explanations that in addition to Green Bay’s work also cover some interesting Wi-Fi access point hiding tricks practiced by the IT folks at AT&T Stadium, and a recap of Levi’s Stadium plans as it gets ready to host Super Bowl 50. Plus team-by-team capsule descriptions of stadium tech deployments for all 32 NFL franchises. It’s all free to you, so download your copy today!

The NFL haves and have-nots when it comes to Wi-Fi

PRO_FB_ThumbWas it really three long years ago that NFL commissioner Roger Goodell issued an edict calling for Wi-Fi in all 31 NFL stadiums? While we’re almost there, it’s not quite everywhere yet and during the course of preparing this year’s PRO FOOTBALL ISSUE we found ourselves wondering how many of the current NFL stadium Wi-Fi networks are really up to snuff. Sure, there are leaders in the networking space, as teams with lots of money or recent Super Bowl hostings seem to be in a bit of an arms war when it comes to installing robust wireless networks. Teams like the Dallas Cowboys, the San Francisco 49ers, the Miami Dolphins, the New England Patriots and a few others come to mind when you are making a list of top networks, and you can probably add Green Bay’s 1,000-plus AP deployment to that tally.

But what about the balance of the league, which now has some kind of fan-facing Wi-Fi in 25 of its 31 venues? While those that don’t have any Wi-Fi at all are somewhat understandable (mainly due to questions about imminent franchise relocation), what about the stadiums that put in Wi-Fi a few years ago, or only put in a limited amount of technology? With no end in sight to the increasing demands for wireless bandwidth, how soon will the older networks need revamping? Including the DAS deployments? Those are questions we’ll keep asking and looking to answer, as we’ve already seen some public reports about Wi-Fi networks falling down on the job. The best place to start, of course, is with the report, so DOWNLOAD YOUR COPY right now!

Thank the sponsors, who let you read for free

Reporting, writing, editing and producing all this content has a cost, but thanks to our generous (and increasing!) list of sponsors, our editorially objective content remains free for you, the reader. We’d like to take a quick moment to thank the sponsors of the Q3 issue of Stadium Tech Report, which include Mobilitie, Crown Castle, SOLiD, CommScope, TE Connectivity, Aruba Networks, JMA Wireless, Corning, 5 Bars, Extreme Networks, ExteNet Systems. and partners Edgewater Wireless and Zinwave. We’d also like to thank you, our readers for your interest and continued support.

As always, we are here to hear what you have to say: Send me an email to kaps at mobilesportsreport.com and let us know what you think of our STADIUM TECH REPORT series, and whether or not the Wi-Fi at your local NFL stadium is a division winner.

Husker Wi-Fi: Nebraska fans use 4.2 TB of Wi-Fi data during Sept. 12 home game

It looks like we have an early leader in the (unofficial) college football Wi-Fi usage race, as the University of Nebraska folks are claiming that fans at Memorial Stadium in Lincoln, Neb., used 4.2 terabytes of Wi-Fi data during the Huskers’ Sept. 12 victory over South Alabama.

Thanks to Chad Chisea, IT operations manager for the Huskers and Dan Floyd, Nebraska’s director of IT for athletics, we’ve got some stats and tweets to share — of the 4.2 TB, approximately 3.0 TB was downloaded data and 1.2 was uploaded, according to network stats sent to us via email. But if you look at the embedded tweet below, the numbers that really jump out at us are the 34,439 unique connected devices and the 28,290 peak connections at a single time — those are numbers that rival anything we’ve seen in NFL stadiums, and are dwarfed only by Super Bowl or college playoff championship game numbers.

With 89,822 in attendance to watch Nebraska whup up on South Alabama 48-9, it’s perhaps no surprise that there are pro-type numbers being put up on the Wi-Fi scoreboard. With a top deployment from Cisco and CDW put in last year, the Memorial Stadium Wi-Fi should be on par with any other large football stadium, and so far the numbers from Nebraska look to be proof of that idea. The Huskers also seem to have a good handle on promoting the Wi-Fi network, as witnessed by the two tweets below that direct fans to the network and let them know they also have game-day help available.

We’re looking forward to getting some hard stats from other top college venues — so far we’ve heard anecdotal evidence that the fiber-based network at Texas A&M’s Kyle Field is rocking, but no numbers yet — so send them our way, and let’s see how the stadium networks stack up. Right now it’s Big Red in the lead, but if DAS numbers from AT&T are any indication, there is lots more data being used this year in stadiums so let’s start adding up the scores.

Stadium Tech Report: Los Angeles Dodgers hit it out of the park with Cisco, Aruba Wi-Fi

Dodgers Stadium, the SoCal baseball shrine. All photos: Terry Sweeney, MSR (click on any photo for a larger image)

Dodgers Stadium, the SoCal baseball shrine. All photos: Terry Sweeney, MSR (click on any photo for a larger image)

Growing up in the Los Angeles suburb of Norwalk, Ralph Esquibel recalled playing outdoors while inside the Dodger game was on the radio. “I knew from the kinds of noises coming out of the house how the game was going,” he laughed. Esquibel, now vice president of IT for the Los Angeles Dodgers, may have wished for some similar indicators or guideposts as he began the wireless retrofitting of Major League Baseball’s third oldest stadium (after Boston’s Fenway Park and Chicago’s Wrigley Field) in early 2011.

Esquibel faced multiple challenges with Dodger Stadium. First, there was all that concrete to push signals through or around. There was the size of the Chavez Ravine venue and its far-flung parking lots, spanning more than 350 acres. The stadium also has few overhangs, a favorite place to attach Wi-Fi access points or distributed antenna system (DAS) gear. Then there’s Dodger Stadium’s capacity — 56,000 seats – the largest in the league and almost 30 percent larger than the average MLB stadium (42,790).

Esquibel’s biggest hurdle? ” Trying to achieve the network that we wanted but also maintain an appropriate budget for the solution,” he said. While Esquibel would not specify what the Dodgers spent, he did allow that it was “an 8-figure project.”

Coverage challenges in the best seats

Initially, the best seats in the house presented a coverage challenge; field and club level seats along the third- and first-base lines and the dugout lack any overhangs. So while phones in those sections could receive a short, directional beam sent from across the outfield, the upstream signal couldn’t get back to the AP across the field, said Esquibel.

Ralph Esquibel, VP of IT for the Dodgers, with the new Wi-Fi relief pitcher mobile.

Ralph Esquibel, VP of IT for the Dodgers, with the new Wi-Fi relief pitcher mobile.

“We wanted to guarantee a premium experience, regardless of the seat,” said Esquibel, who joined the Dodgers 6 years ago after working in IT at Toyota and Honda. So by using what he calls “a hybrid approach,” Wi-Fi APs and antennas are installed overhead where possible, but also under seats and in staircase handrails that divide the stadium’s steep aisles.

All told, nearly 1,000 APs from Cisco and Aruba Networks blanket Dodger stadium, its concession areas and parking lots. Horizon Communications helped the Dodgers with design and installation of the Wi-Fi and DAS.

The under-seat APs/Wi-Fi antennas on the club level are housed in NEMA enclosures about every 15 seats, set eight rows apart. Esquibel was concerned about losing real estate under those seats; he also didn’t want to create any potential trip hazard for fans. In addition, the Dodgers use Cat 6A cabling, whose thickness and rigidity couldn’t run up a stepped incline. Consequently, they drilled through concrete to snake the cabling through from the clubhouse underneath. “There’s no visible conduit leading into the enclosure,” Esquibel explained. The profile and footprint of the enclosure still leaves space for fans to place belongings.

Handrail Wi-Fi enclosure

Handrail Wi-Fi enclosure

It’s the same modus operandi for the enclosures housed in the stair rails, except there are two APs in larger enclosures at the top of each staircase on the reserve level and upper deck, then a single AP per enclosure as the stairways descend. Some 290 APs offer coverage on the reserve level, which by itself has a greater capacity than nearby Staples Center (18,118 seats), Esquibel told Mobile Sports Report. After 2 years of use, there have been no issues with the AP enclosures. “We power-wash the seats and stands after games and [the enclosures] are very resilient against the sun, water and wind,” Esquibel said.

He also acknowledged some early challenges with Wi-Fi. Part of the issue was working with Cisco’s CleanAir technology, which is supposed to minimize RF interference, if not eliminate it altogether. If an AP starts broadcasting over a frequency in use by another AP, for example, CleanAir helps it find another frequency. It took a few months to fully tune the network; some directional antennas needed a 10-degree adjustment, Esquibel said. Another challenge was having APs from more than one vendor. “If your network is 100 percent Cisco and all leveraging the same controllers, [CleanAir] will work perfectly,” Esquibel said. “If you have a mixed environment that pushes Wi-Fi in certain locations, it becomes a problem — there’s competition for frequencies.”

Coordinating the APs

A third-party leveraging a non-public frequency would switch channels, for example, causing the APs for public use to also switch channels. “What we had was a lot of bouncing back and forth,” Esquibel said, which affected performance. “So we assigned channels and frequencies for each AP, which still requires a lot of coordination.”

Under-seat Wi-Fi enclosure

Under-seat Wi-Fi enclosure

Since 2013, the stadium has been carved into 24 DAS sectors. AT&T, T-Mobile USA and Verizon Wireless are the carriers presently using the DAS; Ericsson makes the DAS antennas. Stubborn Sprint relies on a tower adjacent to the stadium.

Dodger fans average anywhere from 500-655 megabytes of data use per game, according to Esquibel. During a busy game, the wireless networking accommodates 16,000 concurrent users; a slower event clocks in at 4,000-8,000. To test upload speed, Esquibel will push a 50MB video to Facebook. When there’s lots of available bandwidth, he gets 60 Mbps performance; on the low end, it’s closer to 4 Mbps. Esquibel said users are mostly streaming and posting videos and photos to social media; Dodger Stadium is the second most Instagrammed site in southern California, after Disneyland, Esquibel added.

The Dodgers have their own version of Ballpark, the in-stadium MLB app, which offers video replay and highlights; in-seat ordering of food and drink in certain areas; and stadium mapping. Check-ins on Ballpark are handled through a network of 44 iBeacons, which takes advantage of Bluetooth Low Energy (BTLE) technology. Between Ballpark and social media activity, Dodger fans have run up as much as 700 MB data usage during games — and the network is ready if more demand is needed.

“We don’t do any rate limiting, so if we consume all our bandwidth we get a free upgrade, thanks to a clause in our agreement with our ISP, AT&T,” Esquibel explained.

To ensure a family-friendly and wholesome environment, the Dodgers use Palo Alto Networks 5020 firewalls for content filtering. “As we developed our SLAs, it was one of the first issues to pop up — no sexual content, no malware/phishing, and no illegal drug sites,” he said.

What’s on his wish list for the future? “I’d like geo-fencing within the Wi-Fi network so if I see someone enter a club, I can say hi or welcome them, notify them of specials, or flag points of interest around the stadium,” Esquibel said, like the World Series trophy case or giveaway locations for promotional items. Alongside all the other applications, wireless can be used as guideposts for fans and visitors to Dodger Stadium.

Taylor Swift shows card 3+ TB each, and push Levi’s Stadium Wi-Fi past half-million user mark

Taylor Swift at Levi's Stadium. All photos: Levi's Stadium (click on any picture for a larger image)

Taylor Swift at Levi’s Stadium. All photos: Levi’s Stadium (click on any picture for a larger image)

Because her humungous stage kept Levi’s Stadium from filling to capacity, Taylor Swift’s Aug. 14 and 15 shows didn’t come close to setting a Wi-Fi capacity record at the venue, as some had predicted. But with almost 20,000 unique users the first night and almost 24,000 the second, the “1989” tour events pushed the cumulative Levi’s Stadium Wi-Fi user number past the half-million mark, just more than a year after the venue officially opened.

Thanks to Levi’s Stadium owners and operators, the San Francisco 49ers, we have some very interesting statistics about Wi-Fi use at large outdoor stadiums. Now that 509,524 unique users have logged in to the Levi’s Stadium Wi-Fi network during its numerous football and other events, the networking team has some interesting observations, including the fact that concertgoers use more bandwidth than football fans.

Over the 28 events hosted at the stadium in the 54 weeks (not counting this past weekend’s preseason game with the Dallas Cowboys) the Levi’s Stadium Wi-Fi network has carried a total of 56.88 terabytes of data, according to figures sent to us by Roger Hacker, senior manager, corporate communications for the Niners. One of the more interesting tidbits was the sustained connectivity during concert events; from Hacker, these direct notes:

Concerts generate up to 65% more load per fan than sporting events.

o For 2014 NFL season, the average fan on Wi-Fi used 100MB.

o For One Direction and Taylor concerts, the average fan consumed 164MB.

Concerts are generating higher sustained loads on the network than sporting events.

o For One Direction, the network was over 2 Gbps for 1 hour and 15 minutes continuously.

o For Taylor the network was over 2 Gbps for a total of 1 hour and 10 minutes continuously.

At many venues with Wi-Fi, the so-called “take rate” or the number of users actually logging in to the network is one key piece of data about how well the system is working. The idea is, the better it works, the more people log on. At the Taylor Swift concerts, the take rates were among the highest we’ve seen: For the first show with attendance of 50,393, there were 19,963 unique users on the Wi-Fi network; for the second show the numbers were 23,885 out of 52,479 in attendance.

Breaking 3 TB twice in a row

Let it be said, that even though the Swift concerts didn’t beat the all-time Levi’s Stadium Wi-Fi mark of 4.5 TB set by WrestleMania 31 back in March, each of the shows passed the 3 TB total mark, with 3.31 TB used on Aug. 14 and 3.807 TB used on Aug. 15. And remember, that’s with just a little more than 50,000 fans in the house, far fewer than the 76,976 who crammed in for the wrassling on March 31.

But even at other events that might not be as high on the “gotta do it” list, fans at Levi’s used the Wi-Fi in high numbers, as another of Hacker’s notes spells out:

On average, nearly 40% of people attending an event at Levi’s use the Wi-Fi.

For the first year, the average attendance at all events was 46,400 based on the VenueNext valid scans. (including smaller events as well as bigger events)

For the first year, the average unique wifi users per event was 18,200 (or 39% of attendance)

On-field Aruba Networks Wi-Fi AP at Levi's Stadium

On-field Aruba Networks Wi-Fi AP at Levi’s Stadium

Part of what might have made the Swift audience bigger users of Wi-Fi was a new, improved temporary Wi-Fi network for the field seats, an innovation crafted by Wi-Fi gear supplier Aruba Networks and the Levi’s Stadium networking team. By putting temporary Wi-Fi APs on both the walls surrounding the field level, on railings in the temporary seats and even underneath the temporary flooring, the stadium provides service to the premium-seat customers, a challenge for NFL stadiums who must keep regular Sunday-game networks from bleeding into the field area (because that’s where the league operates its own Wi-Fi network for game operations). For what it’s worth, we also heard from DAS supplier DAS Group Professionals who said they were going to have temporary DAS antennas on the field as well for the Swift concerts. No DAS stats yet, however.

Earlier this summer at the annual SEAT Conference, we had a very interesting panel discussion about which stats really matter when it comes to stadium Wi-Fi usage — though I will always remain a sucker for top-line totals, I do agree that there should probably be separate categories for events like the Swift concerts, the WrestleMania events, and regular football games. What is catching my attention more lately are numbers like the sustained connectivity and the unique and concurrent numbers of users, since in the end the bottom line on stadium Wi-Fi network performance has to be how many people are using it. A year in, it looks like the Levi’s Stadium network has passed a crucial test, in that people are both finding it and using it, even if they are not regular attendees or season-ticket holders. Those facts say a lot about how well the Levi’s Stadium system works, and should be numbers stadium tech professionals look to when assessing their own deployments.

More Swift shots below. Enjoy.

ts3

ts4

ts5

Huge jump in stadium Wi-Fi deployments, according to our latest State of the Stadium Technology Survey

2015_SoS_thumbA clear trend toward greater Wi-Fi and DAS deployments is the main point our “State of the Stadium” research tells us this year, especially with Wi-Fi, where now more than half of the venues surveyed said they have fan-facing Wi-Fi services to all seating areas.

To be precise, a full 65 percent of our respondents said they now have full-seating Wi-Fi services in their venues – a huge increase from the 35 percent number we saw in last year’s survey. Full-stadium DAS deployments increased as well, with 80 percent of respondents claiming stadium-wide DAS deployments this year, up from 71 percent in the 2014 survey.

Input for the survey came from more than 100 respondents representing arenas that host the top U.S. professional league teams, including the NFL, Major League Baseball, the NBA, the NHL, as well as top U.S. university facilities for basketball and football, European and U.S. professional soccer teams, professional golf and car-racing venues. The data provide a clear snapshot of how teams are deploying technology to both improve the fan experience while helping increase business opportunities.

And thanks to a sponsorship from Mobilitie, the entire report can be downloaded for free from our website.

Why more Wi-Fi?

Why the sudden shift to more Wi-Fi deployments? Most likely it was the maturation of plans that had been in the works for several years, speeded up no doubt by the ever increasing demands for mobile data connectivity. Even facilities that have had Wi-Fi services for years noticed that over the past year data usage has climbed even as the number of connected users plateaued; what that tells us is that the trend of devices and apps to stay well ahead of the networks’ ability to keep up still has legs, and will likely keep climbing for the near future.

For more analysis and a breakdown for each category, download the report now to get the only independent, numbers-based research available for the stadium technology marketplace.

Stadium Tech Report: World Series set new wireless records at AT&T Park

AT&T Park during the World Series. Photo: SF Giants (click on any photo for a larger image)

AT&T Park during the World Series. Photo: SF Giants (click on any photo for a larger image)

The most-connected park in all of baseball is still finding ways to serve more people more data, as proven by the wireless consumption records set by the San Francisco Giants during last year’s World Series.

The traffic generated at the three games at AT&T Park was “definitely more than anything we had ever experienced before,” said Bill Schlough, senior vice president and chief information officer for the Giants. The combined download and upload numbers for both the park’s Wi-Fi network and AT&T traffic on its DAS network averaged 2.08 terabytes per game, Schlough said, with a high of 2.14 TB of total traffic for Game 4.

Since AT&T Park has had Wi-Fi longer than any sports stadium in the U.S. – this season will be its 12th with stadium-wide Wi-Fi – and since last year was the Giants’ third World Series in five years – Schlough’s team was perhaps a bit more prepared than most IT staffs for the expected demands.

“The traffic followed the standard trend, where each round [of the playoffs] saw successively higher demand,” Schlough said. Upload totals also increase as the team progresses through the playoffs, he said, perhaps more so now that fans of all types are getting more adept at adding multimedia to their messaging.

“You don’t just send a text anymore,” Schlough said. “The expectation is that you will send a picture and or a video.”

Replacing Jay Z and Beyonce at the top

Editor’s note: This story is reprinted from our latest Stadium Tech Report, the BASEBALL (and Soccer!) ISSUE, which is available now for free download from our site. The report includes a focus on baseball and soccer stadium technology deployments, and team-by-team coverage of technology deployments for all 30 MLS teams — AND all 20 MLS teams. DOWNLOAD THE REPORT now and read for yourself!

The spoils of victory. Photo: Paul Kapustka, MSR

The spoils of victory. Photo: Paul Kapustka, MSR

Prior to last year’s games with the Kansas City Royals, the top Wi-Fi traffic event at AT&T Park had been a couple concerts earlier in the sum- mer starring Jay Z and Beyonce, where Schlough and his staff saw upload totals of 410 GB on the second night of the show. The World Series games blew by that previous record total with an average of 700 GB uploaded per game, with a high of 750 GB for Game 5.

Wi-Fi download numbers for the three series games averaged 890 GB, Schlough said, with a max of 940 GB during Game 3. For the AT&T customers on the park’s DAS, download num- bers for the Series averaged 320 GB per game with a maximum of 350 GB for Game 4. DAS upload totals were an average of 170 GB per game.

Not even knowing you’re on Wi-Fi

What amazed or satisfied Schlough even more than the raw data numbers was the Wi-Fi take rate, or the number of fans connected to the network. For the Series it hovered right around 50 percent, meaning that every other fan in the 42,000-seat venue was using the network.

The view from left field corner. Photo: Paul Kapustka, MSR

The view from left field corner. Photo: Paul Kapustka, MSR

Regular-season Wi-Fi take rates, he said, were usually in the 30-percent range, climbing to 40 percent as the playoffs progressed. One thing that helps people connect to the Wi-Fi network at AT&T Park is the overall ubiquity of AT&T hotspots – “If you’ve accessed another AT&T hotspot anywhere else, you get automatically activated when you’re here [at AT&T Park],” Schlough said.

Fan surveys, he said, showed that many people didn’t even know they were connected to the Wi-Fi network instead of the cellular networks. “I think that’s cool,” Schlough said. “Fans should come to an event and be universally connected, without having to think about it. They should just be able to turn on their phones and share.”

More APs for the upper deck

For 2015, Schlough and his team will finish off the latest Wi-Fi upgrade with the installation of another 400 under-seat APs for the stadium’s upper decks, which will bring the park’s AP total to almost 1,700 when it’s finished. Already this year Schlough said that fans at Giants games are using more data than last year – an average of 1.1 TB per game over the first 10 games of the 2015 season, compared to an average of 650 GB per game over the same time period in 2014.

Giants fans check out the three WS trophies, at a 2015 season game. Photo: Paul Kapustka, MSR

Giants fans check out the three WS trophies, at a 2015 season game. Photo: Paul Kapustka, MSR

Though he didn’t want to dive into details, since last year Schlough said the network is seeing “a lot more photos and a lot more videos.” He also said his team is on the lookout for use of livestreaming apps like Periscope and Meerkat,
which he expects might happen at AT&T Park before it happens anywhere else, perhaps due to the overall technological bent of the local populace.

“We feel we have a relatively unique fan base,” Schlough said. “If anyone is going to do it [livestream during games] it’ll probably happen first in this region.”