New Report: Green Bay’s Lambeau Field leads new NFL Wi-Fi deployments

Wave the flag, Wi-Fi has come to Lambeau Field! Photo: Green Bay Packers

Wave the flag, Wi-Fi has come to Lambeau Field! Photo: Green Bay Packers

When most NFL fans think of the Green Bay Packers and Lambeau Field, they think of frozen tundra — of Vince Lombardi roaming the sideline in his thick glasses and peaked hat, with visible breath coming through the face masks of behemoth linemen on the field. In the stands, they see the venerable fans braving the cold of northern Wisconsin in their snowmobile suits, with mittens wrapped around a bratwurst and a beer.

But do they think of those same Packers fans pulling out their iPhones and Samsungs to take selfies, and posting them to Instagram or Facebook? Maybe not so much.

The reality of 2015, however, finds us with fans in Green Bay being just like fans anywhere else — meaning, they want to be able to use their mobile devices while at the game. As the cover story of our most recent Stadium Tech Report series, we explore the details of bringing Wi-Fi to historic Lambeau Field, where late-season texting might carry the threat of frostbitten fingers.

Our PRO FOOTBALL ISSUE has 50-plus pages of insight and how-to explanations that in addition to Green Bay’s work also cover some interesting Wi-Fi access point hiding tricks practiced by the IT folks at AT&T Stadium, and a recap of Levi’s Stadium plans as it gets ready to host Super Bowl 50. Plus team-by-team capsule descriptions of stadium tech deployments for all 32 NFL franchises. It’s all free to you, so download your copy today!

The NFL haves and have-nots when it comes to Wi-Fi

PRO_FB_ThumbWas it really three long years ago that NFL commissioner Roger Goodell issued an edict calling for Wi-Fi in all 31 NFL stadiums? While we’re almost there, it’s not quite everywhere yet and during the course of preparing this year’s PRO FOOTBALL ISSUE we found ourselves wondering how many of the current NFL stadium Wi-Fi networks are really up to snuff. Sure, there are leaders in the networking space, as teams with lots of money or recent Super Bowl hostings seem to be in a bit of an arms war when it comes to installing robust wireless networks. Teams like the Dallas Cowboys, the San Francisco 49ers, the Miami Dolphins, the New England Patriots and a few others come to mind when you are making a list of top networks, and you can probably add Green Bay’s 1,000-plus AP deployment to that tally.

But what about the balance of the league, which now has some kind of fan-facing Wi-Fi in 25 of its 31 venues? While those that don’t have any Wi-Fi at all are somewhat understandable (mainly due to questions about imminent franchise relocation), what about the stadiums that put in Wi-Fi a few years ago, or only put in a limited amount of technology? With no end in sight to the increasing demands for wireless bandwidth, how soon will the older networks need revamping? Including the DAS deployments? Those are questions we’ll keep asking and looking to answer, as we’ve already seen some public reports about Wi-Fi networks falling down on the job. The best place to start, of course, is with the report, so DOWNLOAD YOUR COPY right now!

Thank the sponsors, who let you read for free

Reporting, writing, editing and producing all this content has a cost, but thanks to our generous (and increasing!) list of sponsors, our editorially objective content remains free for you, the reader. We’d like to take a quick moment to thank the sponsors of the Q3 issue of Stadium Tech Report, which include Mobilitie, Crown Castle, SOLiD, CommScope, TE Connectivity, Aruba Networks, JMA Wireless, Corning, 5 Bars, Extreme Networks, ExteNet Systems. and partners Edgewater Wireless and Zinwave. We’d also like to thank you, our readers for your interest and continued support.

As always, we are here to hear what you have to say: Send me an email to kaps at mobilesportsreport.com and let us know what you think of our STADIUM TECH REPORT series, and whether or not the Wi-Fi at your local NFL stadium is a division winner.

Levi’s Stadium Monday Night Football debut sees 2.87 TB of Wi-Fi traffic, 874 GB on AT&T DAS

Levi's Stadium during its inaugural Monday Night Football game. Photo: Levi's Stadium

Levi’s Stadium during its inaugural Monday Night Football game. Photo: Levi’s Stadium

For its first-ever Monday Night Football game, Levi’s Stadium saw 2.87 terabytes of data cross its Wi-Fi network, with an additional 874 GB traversing the AT&T cellular DAS network during the Niners’ somewhat surprising 20-3 victory over the Minnesota Vikings.

With the confirmed numbers bumping up against the 4 TB mark — and if you add in the probable (but unreported) 1 TB or more that was used by Verizon Wireless, Sprint and T-Mobile customers on the Levi’s Stadium DAS — it’s readily apparent that usage of wireless data inside stadiums is only continuing to grow, with no top end yet in sight.

Though the Wi-Fi mark didn’t hit the same heights as the 3.3 TB number recorded at the first regular-season opener at Levi’s Stadium last fall, it’s impressive nonetheless because of the game’s somewhat lower profile given the modest expectations for a Niners team that has suffered through an exceptionally strange offseason that saw its high-profile coach Jim Harbaugh leave for the University of Michigan, and a number of top players retire, like star linebacker Patrick Willis, or depart, like running back Frank Gore, who went to Indianapolis.

And with the new-car buzz somewhat gone from Levi’s Stadium if almost 3 TB of Wi-Fi is a “regular” mark you have to start wondering what the totals are going to be like when Super Bowl 50 comes to the venue in February. On the DAS side of things, the cellular traffic generated by AT&T customers at Levi’s Stadium Monday night was the second-highest in the NFL venues measured by AT&T, trailing only the traffic at namesake AT&T Stadium, where AT&T saw 1.107 TB of DAS traffic during the Cowboys’ opening-game victory over the New York Giants. According to AT&T, DAS traffic at NFL stadiums during the first week of games was up 46 percent compared to the first week of games in 2014. We’ll have a separate post on college DAS traffic tomorrow, which is also up. Thanks to the Niners for the data chart below.

Screen Shot 2015-09-16 at 9.40.52 PM

DGP upgrades Levi’s Stadium DAS in preparation for Super Bowl 50

New 'chiclet' DAS antennas visible on the concourse overhangs at Levi's Stadium. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

New ‘chiclet’ DAS antennas visible on the concourse overhangs at Levi’s Stadium. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

After deploying one of the biggest and most robust stadium DAS networks at Levi’s Stadium last year, what did DAS Group Professionals do for an encore?

How about completely re-deploying a new DAS at Levi’s Stadium during the football offseason, just about doubling the capacity in a construction project that took place at a “live” venue instead of one under construction?

The ambitious renovation of the not-quite-1-year-old Levi’s Stadium DAS was all done in the name of Super Bowl 50, the NFL’s big game that is coming to Levi’s Stadium in February 2016. According to DGP president Steve Dutto, the huge jump in wireless traffic statistics at Super Bowl XLIX last February got the major wireless carriers working early to assure that the NFL’s 50th Super Bowl would have enough cellular capacity.

In addition to new antennas from DAS gear supplier JMA Wireless that can be more finely tuned, and more antennas and remote units to beef up coverage and support more cell sectors inside Levi’s Stadium, DGP and stadium owner the San Francisco 49ers also increased the DAS footprint outside the stadium in the adjacent parking lots, addressing what Niners COO Al Guido told MSR was a “primary concern” heard from fans during the inaugural Levi’s Stadium season.

Close-up of new DAS antennas (from mid-July, before the wires were connected)

Close-up of new DAS antennas (from mid-July, before the wires were connected)

Though work on the DAS will likely continue throughout the season, most of the construction and deployments were expected to be in place for the Niners’ regular season opener tonight versus the Minnesota Vikings.

Ready for another ‘Super’ cellular day

Though the unofficial totals for DAS traffic at the last Super Bowl that were in excess of 6 terabytes were the highest we’ve ever seen reported, DGP’s Dutto said that the top wireless carriers are expecting as much as 2.5 times that amount of traffic at Super Bowl 50, a load that might have swamped even the previously robust Levi’s Stadium DAS.

“The [cellular] traffic at the Super Bowl this past year was greater than anything anybody had seen,” Dutto said in a recent interview at Levi’s Stadium. All the carriers, he said, were “amazed” at the traffic jump from the year before, when AT&T and Verizon Wireless reported a combined total of about 2.5 TB of cellular data at Super Bowl XLVIII at MetLife Stadium in New Jersey. (For what it’s worth the Wi-Fi traffic also just about doubled, from 3.3 TB to 6.23 TB.)

The DGP team at Levi's for our interview included, L to R, Derek Cotton, director of engineering; Steve Dutto, president; and Vince Gamick, VP and COO.

The DGP team at Levi’s for our interview included, L to R, Derek Cotton, director of engineering; Steve Dutto, president; and Vince Gamick, VP and COO.

So even though the Levi’s Stadium DAS performed exceedingly well — according to Dutto “we turned it on and from day one it exceeded expectations” — and that there wasn’t any capacity problems during the Niners’ 2014 season, the expected impending crush coming in February spurred what Dutto called “carrier-driven upgrades” that included the need to take over some previous storage-room space to house the increased amount of head-end gear.

Under seat DAS antennas part of the upgrade

Inside the stadium itself, Dutto said that Verizon will expand its coverage from 60 to 80 sectors, while AT&T will increase from 45 to 54 or more sectors. DGP will increase its “zones” of coverage in the main seating bowl from 23 to 40, and will go from 250 antennas to 400, and from 330 remote units to 450. There will also be plenty of new antennas from JMA that Dutto said have “greater gain and sharper patterns” to enhance coverage.

“It’s almost like deploying twice the network in the same amount of time,” Dutto said. The other big difference from last year, however, was that Levi’s Stadium was open for events this past football offseason, from WrestleMania 31 to a NHL Stadium Series game to concerts with the Grateful Dead and Taylor Swift.

Another view of the antennas on the concourse overhangs

Another view of the antennas on the concourse overhangs

Though there was more than enough cabling areas and pathways to make the retrofit easier, Dutto said working around the live events increased the deployment degree of difficulty. But even with new head end rooms and extras like under-seat DAS antennas (a new deployment method at Levi’s Stadium for DAS), 20-plus new antenna placements in the parking lot and the accomodation of new 2.5-GHz spectrum for Sprint, Dutto said that the new network was expected to be at least “90 percent complete” before the season’s start.

As is regular with lead-ups to Super Bowls, there will likely be even more network tweaking and adjustment up until the last minute before kickoff on Feb. 7, 2016.

A ‘heart-attack” moment and hot dog machines

With any luck, the new network will work as well as the old one, and will hopefully light up without the “heart attack moment” that happened just before opening day last fall. Dutto and his team of Derek Cotton, director of engineering, and Vince Gamick, DGP’s vice president and COO, told of coming to Levi’s Stadium for the regular season opener last September and almost keeling over when they couldn’t detect the DAS network anywhere inside the building.

As it turns out, Dutto said that the major wireless carriers had forgot to turn down the power on their macro towers at the cell sites that surround Levi’s Stadium, which basically overwhelmed the internal network since Dutto said the outdoor cell sites operate at 80 watts, compared to the 2-watt in-stadium DAS network.

“There are six cell sites within hundreds of feet [of Levi’s Stadium] and since they hadn’t turned them down, we couldn’t see the network in the stadium,” said Dutto, whose pulse was revived when the situation was quickly rectified.

Close-up of the back of one of the new antennas

Close-up of the back of one of the new antennas

At least the Levi’s Stadium DAS couldn’t be turned off by stadium workers or blacked out by hot dog machines, problems that DGP encountered at the Niners’ old home, Candlestick Park, where DGP deployed a test DAS in the 49ers’ final season there in 2013. According to Dutto, for some reason the DGP network was connected by utility PG&E to the same electrical routes as the cookers for stadium hot dogs — and when those were turned on one Sunday the system was overloaded, bringing the DAS down with sausages.

The Candlestick DAS also suffered an outage when the park rangers who lived at the stadium thought the equipment air conditioners were too loud, so they turned them off — setting off multiple alarms for Dutto and his network administration team.

Safe to say, similar problems aren’t expected to arise as DGP and the Niners prepare Levi’s Stadium’s cellular network for Super Bowl 50, a tough task but one with rewards on both a personal and business level.

“This has been fun, and the Niners have been great to work with,” said Dutto.

DGP gets deal to extend DAS outside Levi’s Stadium

Franks and DAS: DGP DAS antennas above food station at Levi's Stadium. Photo credit: Paul Kapustka, MSR

Franks and DAS: DGP DAS antennas above food station at Levi’s Stadium. Photo credit: Paul Kapustka, MSR

DAS Group Professionals, the company that installed the neutral-host DAS inside Levi’s Stadium, now has a deal to extend the DAS outside the Levi’s walls, covering parts of the city of Santa Clara, Calif., that surround the stadium.

With next year’s Super Bowl set to take place at Levi’s Stadium, it makes sense that city officials would want to make sure the parking lots and other pre-game gathering areas outside the venue had good cellular connectivity. At the most recent Super Bowl in Glendale, Ariz., neutral host provider Crown Castle did an extensive job of building the “oDAS” or outside DAS in the spaces surrounding the University of Phoenix Stadium.

According to DGP, it will design, build and maintain an oDAS for the City of Santa Clara, initially targeting the area around the Great America theme park and the Santa Clara Convention Center, which sit on the other side of the main Levi’s Stadium parking lots. Like the DAS inside the stadium, access to the network outside the stadium will be offered to all major wireless carriers, who must pay DGP and the city for access to the network.

While the network will definitely come in handy for pre- and post-game connectivity following Levi’s Stadium events, it will also improve overall cellular performance in the area, which is also the home to several large corporate office buildings as well as the busy convention center.

Super Bowl XLIX sets new stadium Wi-Fi record with 6.2 Terabytes of data consumed

University of Phoenix Stadium. Credit: Arizona Cardinals.

University of Phoenix Stadium. Credit: Arizona Cardinals.

The Super Bowl is once again the stadium Wi-Fi champ, as fans at Sunday’s Super Bowl XLIX in Glendale, Ariz., used 6.23 terabytes of data during the contest, according to the team running the network at the University of Phoenix Stadium.

The 6.23 TB mark blew past the most recent entrant in the “most Wi-Fi used at a single-day single-stadium event” sweepstakes, the 4.93 TB used at the Jan. 12 College Football Playoff championship game at AT&T Stadium. Prior to that, pro football games this past season at Levi’s Stadium in Santa Clara, Calif., and at AT&T Stadium had pushed into the 3-plus TB mark to be among the highest totals ever reported.

The live crowd watching the New England Patriots’ 28-24 victory over the Seattle Seahawks also used about as much cellular data as well, with Verizon Wireless, AT&T and Sprint claiming a combined total of 6.56 TB used in and around the stadium on game day. All three carriers were on the in-stadium and outside-the-stadium DAS deployments being run by neutral host Crown Castle. If those figures are correct (more on this later) it would put the total wireless data usage for the event at 12.79 TB, far and away the biggest single day of wireless data use we’ve ever heard of.

Apple OS updates still the application king

Handrails with Wi-Fi antenna enclosures from AmpThink. Credit: Arizona Cardinals.

Handrails with Wi-Fi antenna enclosures from AmpThink. Credit: Arizona Cardinals.

Mark Feller, vice president of information technology for the Arizona Cardinals, and Travis Bugh, senior wireless consultant for CDW, provided Mobile Sports Report with the final Wi-Fi usage numbers, which are pretty stunning for anyone in the stadium networking profession. According to Feller the new CDW-deployed Wi-Fi network with Cisco gear at the UoP Stadium saw 2.499 TB of data downloaded, and 3.714 TB uploaded, for a total of 6.213 TB of Wi-Fi usage. Bugh of CDW said there were 25,936 unique devices connecting to the network on game day, with a peak concurrent usage of 17,322, recorded not surprisingly at halftime.

Peak download usage of 1.3 Gbps was recorded before the game’s start, while peak upload usage of 2.5 Gbps was hit at halftime. The top applications by bandwidth use, Feller said, were Apple (mobile update), Facebook, Dropbox and Snapchat.

DAS numbers also set new record, but clarification needed

The only reason we aren’t yet trumpeting the 6.564 TB of reported DAS use as a verified record is due to the differences in clarity from each of the reporting providers. We also haven’t yet heard any usage totals from T-Mobile, so it’s likely that the final final wireless data use number is somewhere north of 13 TB, if all can be believed.

Parking lot light poles, Westgate entertainment district. Can you spot the DAS?

Parking lot light poles, Westgate entertainment district. Can you spot the DAS?

As reported before, AT&T said it saw 1.7 TB of cellular wireless activity from its customers on game day, with 696 GB of that happening inside the stadium, and the balance coming from the outside areas before and after the game. We’d also like to welcome Sprint to the big-game reporting crew (thanks Sprint!), with its total of 754 GB of all 4G LTE traffic used in and around the stadium on game day. According to Sprint representatives, its Super Bowl coverage efforts included 5 COWs (cell towers on wheels) as well as expanded DAS and macro placements in various Phoenix-area locations. The Sprint coverage included the 2.5 GHz spectrum that uses TDD LTE technology.

As also previously reported, Verizon Wireless claimed 4.1 TB of customer traffic in and around the stadium on game day, which Verizon claims is all cellular traffic and does not reflect any Verizon Wireless customer use of the stadium Wi-Fi network. Verizon also reported some other interesting activity tidbits, which included 46,772 Verizon Wireless devices used at the game, of which just 59.7 percent were smartphones. Verizon also said it saw 10 million emails sent on its networks that day, and 1.9 million websites visited, while also seeing 122.308 videos sent or received over wireless connections.

We’re still waiting to see if we can get usage numbers from the Super Bowl stadium app (we’re especially interested to see if the instant replay feature caught on) but the warning for stadium owners and operators everywhere seems to be clear: If you’re hosting the big game (or any BIG game), make sure your network is ready for 6 TB and beyond!

AT&T sets new DAS traffic records for Super Bowl with 1.7 Terabyte mark

University of Phoenix Stadium

University of Phoenix Stadium

AT&T said its customers set new records for Super Bowl and professional football game wireless data consumption, with a total of 1.7 terabytes of traffic used in and around the University of Phoenix Stadium in Glendale, Ariz., Sunday night.

In a blog post from AT&T senior executive vice president John Donovan AT&T said it saw 696 gigabytes of wireless data used on its in-stadium DAS Sunday night, with an additional 1 TB used in and around the stadium in the surrounding parking lots and the Westgate entertainment district, a mall/restaurant complex that is connected to the UoP stadium area. The 1.7 TB mark surpasses the 1.4 TB DAS mark AT&T saw at the recent College Football Playoff championship game in Arlington, Texas, on Jan. 12.

Donovan’s blog post contains some interesting looks back — with a peak usage of 125 GB per hour Sunday, AT&T saw another new high mark, one that seems to say that usage of wireless data at stadiums is still climbing with no roof (retractable or not) in sight. Here’s a couple quotes:

Since 2011 – inclusive of the last five Big Games – the total data usage on AT&T’s in-stadium network has increased from 177GB to 696GB and peak hour data usage has increased from 30GB to 125GB.

And:

These numbers don’t come as a total shock as we experienced several high marks this season. In total, from 253 games at 31 stadiums, our customers have used more than 85.7TB of mobile data on our venue-specific cellular networks. That’s equivalent to more than 245M social media posts with photos from 253 games (an average of almost 1M social media posts per game).

We are still waiting for results from the stadium Wi-Fi network… will the total break the 6 TB mark set at the CFP championship game? Stay tuned! More AT&T infographic fun below.

click on photo for larger image

click on photo for larger image

ATT_Super_Bowl_Football_GraphicTwitter_R1V4-2-2-2015