Fan-facing Wi-Fi on hold as Coliseum gets ready for Rams’ return to Los Angeles

DAS antennas visible on the LA Coliseum's facade

DAS antennas visible on the LA Coliseum’s facade

Normally when a new professional sports franchise comes to town or opens a new venue, preparations move into overdrive pretty quickly — especially for infrastructure like luxury suites, Wi-Fi and DAS.

But this is the Los Angeles Rams, and nothing about the team’s trajectory is normal, including the technology. So after making good on a longstanding threat to move the team from St. Louis, owner Stan Kroenke in January broke ground on a new, $2.6 billion “NFL Disneyland” venue in the LA suburb of Inglewood. It’s expected to be ready in time for the 2019 season. So for right now, the returning Rams will play at the Los Angeles Coliseum.

While it has an established and robust DAS system, the Coliseum has no dedicated, fan-facing Wi-Fi network, just as the Edward Jones Dome in St. Louis didn’t. But it’s LA, baby. Kanye. Jack. Celebrity sightings means more bandwidth is needed, not less. But the first appearance of Wi-Fi at the venerable Coliseum won’t be for fans, but for operations.

Relying on DAS for fan wireless

Editor’s note: This profile is from our most recent STADIUM TECH REPORT, the Q2 issue which contains a feature story on Wi-Fi analytics, and a sneak peek of the Minnesota Vikings’ new US Bank Stadium. DOWNLOAD YOUR FREE COPY today!

Artist rendering of the proposed new LA football stadium

Artist rendering of the proposed new LA football stadium

So when the Rams play their first pre-season game at the Coliseum in August against the Dallas Cowboys, fans will have to rely on the DAS network for connectivity. AT&T, Verizon Wireless and more recently T-Mobile, are the DAS carriers; Sprint trucks in a COW for USC games.

The Rams will bring in their own Wi-Fi for communications to and from the sidelines, along with their Microsoft Surface tablets, according to Derek Thatcher, IT manager at the Coliseum and an employee of USC, which oversees and administers the venue for Los Angeles County. There will also be private Wi-Fi in the locker rooms and the officiating rooms. Thatcher’s working closely with the Rams and the NFL, including one of the NFL’s frequency coordinators, to ensure everybody has the bandwidth they need.

Separate from the Rams and the NFL, USC is undertaking a major renovation of the Coliseum, home to the university’s storied football team. The $270 million project cost will be funded entirely by USC Athletics from capital gifts, sponsorship revenue, non-USC athletic events at the Coliseum, and donor naming opportunities. “The project will not require any student fees or general university, local, state or federal funds,” the university said on ColiseumRenovation.com.

In addition to significant expansion of luxury suites and press box facilities (which houses most the IT and networking gear for the Coliseum), USC will also be adding public Wi-Fi and is talking with different vendors about their requirements.

DAS antennas inside the concourse

DAS antennas inside the concourse

It is worth noting that Aruba Networks provides wireless networking for most of the adjacent USC campus (more than 5,000 APs), including the Galen Center and USC’s healthcare facilities. Thatcher emphasized the bidding was open to all vendors.

Waiting for the new stadium to be built

The Coliseum’s current capacity is 93,600 and the NFL will use 80,000 seats; post-renovation, capacity will be 77,500, due to replacement of all seats and addition of handrails to the aisles. That’s good news for Wi-Fi engineers, since the Coliseum’s bowl design has no overhangs to speak of; DAS antennas are mounted to poles that ring the stadium, and also above the bowl’s entry and exit tunnels. “Underseat AP design is expensive… we could end up with underseat and handrail,” Thatcher told Mobile Sports Report. “We’re looking at all possible solutions.”

The renovation is scheduled to begin right after USC’s 2017 football season ends and is expected to be done in time for the 2019 opener. The university said it will plan the construction schedule so that 2018 season can still be played at the Coliseum.

The other wild card in the Coliseum’s future is the Los Angeles bid to host the 2024 Summer Olympics. The city has already done the honors before (1932, 1984) and the Coliseum served as the Olympic Stadium both times; a third time hosting would be unprecedented. But because of LA’s experience, coupled with plenty of already-built sporting venues to handle a plethora of events and requirements, it was natural for the US Olympic Committee to turn to LA once Boston bailed.

The city’s bid includes $300 million for additional renovations to the Coliseum.

Rome, Paris and Budapest are also competing to host the games. The winner will be announced by the International Olympic Committee in September 2017.

St. Louis Cardinals team with MLBAM for Busch Stadium Wi-Fi

Busch Stadium, St. Louis, home of a new MLBAM Wi-FI network. Credit all photos: St. Louis Cardinals

Busch Stadium, St. Louis, home of a new MLBAM Wi-FI network. Credit all photos: St. Louis Cardinals

Working closely with Major League Baseball’s Advanced Media initiative, the St. Louis Cardinals activated what’s being billed as the league’s largest wireless deployment, at least if you measure by overall square footage. The system went live in a phased introduction the week before Memorial Day, according to Cardinals’ director of IT Perry Yee.

More than 740 Wi-Fi access points were installed to accommodate fans at Busch Stadium, including the AT&T Rooftop, as well the Busch II Infield and the Budweiser Brewhouse rooftop deck across Clark Ave. from the stadium at Ballpark Village, where the Cardinals played til 2005. The Cards’ wireless deployment was part of a $300 million initiative headed by MLBAM to build out Wi-Fi and DAS in all the league’s ballparks, with MLB, wireless carriers and teams all sharing in the costs. (While Busch may be the largest MLBAM deployment, AT&T Park in San Francisco has baseball’s most-dense Wi-Fi and DAS network by antenna numbers; the networks at AT&T Park are run by the Giants and AT&T.)

Yee said the Cardinals experienced relatively few engineering issues, in part because of the relative newness of the stadium. He also credited MLBAM, which took the Cardinals’ design and selected a systems integrator and an equipment vendor (Cisco).

“It’s a real turnkey solution where you submit the blueprint [to MLBAM] and they start locating the APs,” Yee said. “Where we come in is we have the background experience to tell the design team where people congregate and how often different spaces get used.”

Putting Wi-Fi in the railings

Editor’s note: This profile is from our most recent STADIUM TECH REPORT, the Q2 issue which contains a feature story on Wi-Fi analytics, and a sneak peek of the Minnesota Vikings’ new US Bank Stadium. DOWNLOAD YOUR FREE COPY today!

Wi-Fi railing enclosure.

Wi-Fi railing enclosure.

Busch Stadium, with a capacity of 46,861, is also blessed with good lines of sight to the field; the only real obstructions are near the foul poles. That’s great for fans, but creates challenges in that there are consequently fewer structures on which to mount antennas. That meant getting inventive in some areas, like installing Wi-Fi antennas in the handrails, and drilling the conduits from underneath seats to keep trip hazards and visual distractions to a minimum.

“Over in Ballpark Village, we had brick on the outside of building so we had to be careful — that was an issue for the electricians to figure out,” Yee said, quickly adding that the electricians on projects like these rarely get the recognition they deserve. “The designer can say where the antennas go, but the electricians have to figure out how to get power to that spot and do it in a manner that fits the building,” he said.

Cardinal fans trying to access the network hit a gated page that asks which cell carrier they use and also to accept terms and conditions; the Cardinals can then track usage and capacity by carrier and take that information back to the three carriers with DAS service in Busch Stadium: AT&T, Sprint and Verizon Wireless. “If Sprint, for example, notices they have lot more customers than anticipated, it might be time for them to review their capacity,” Yee said.

Cards director of IT Perry Yee

Cards director of IT Perry Yee

There’s also an option on the gating page for users to share their email if they want to subscribe to the Cardinal newsletter, Yee added.

MLB app the center of activity focus

The Busch Stadium Wi-Fi network is new enough that MLBAM still is in the process of handing over management and oversight to the Cardinals’ organization; that makes it hard to track certain numbers — like what the budget was the for the project and how much money each entity contributed, numbers which MLBAM has not revealed for any of the many deployments it led throughout the league.

The Cardinals also aren’t releasing any official throughput or usage thresholds yet. Yee said he has seen speeds of 100 Mbps up and down and up on his Samsung S5 phone during more exciting parts of a recent game when fewer users were online. That number dipped to 20-30 Mbps during quieter parts of game. “It was a really good game — for half an hour no one was on the Wi-Fi, then when the score went in one direction I began to see speeds going down as people got online,” Yee said.

Lower level seats are covered with APs that shoot backwards into the stands.

Lower level seats are covered with APs that shoot backwards into the stands.

The Cardinals are looking to use the league’s Ballpark mobile app as the focal point for digital ticketing and inseat ordering plans. One hurdle to inseat ordering isn’t technical; it’s making sure the concessionaire can receive the data. “There’s lots of backend components to establish and flesh out before it becomes a real thing — lots of logistics to make these things happen,” Yee said.

The Cardinals have been using Bluetooth-based beacon technology for a couple years now. “We use it at our gates to greet people and it works via the Ballpark app,” Yee said. For now, the beaconing only works with iPhones; they’ll add support for Android devices at some point. But Yee foresees using beacon technology all around Busch Stadium at points of interest like the Stan Musial statue, providing information about who he was, what he did, to fans in proximity of the monument.

The Cardinals are still considering whether to deploy ambassadors in the stands during games to help people with connectivity issues and other questions. Longer term, they’re looking at geofencing with the concession areas or team store for specials and sale items — “Hot dogs on sale for this inning,” Yee mused. That’s way off in the future, the Cardinals’ IT director added.

In the meantime, the focus will be on “infrastructure that allows fans to see more and do more that makes the games more enjoyable,” Yee said.

UPDATE: WrestleMania 32 sets new Wi-Fi mark at AT&T Stadium; total Wi-Fi + DAS hits 8.6 TB

The Undertaker arrives at AT&T Stadium for WrestleMania 32. Photo: WWE.com

The Undertaker arrives at AT&T Stadium for WrestleMania 32. Photo: WWE.com


UPDATE: Fixes an MSR calculation error on DAS figures.

The 101,763 fans who filled AT&T Stadium Sunday for WrestleMania 32 set new stadium records for Wi-Fi, according to figures provided by AT&T Stadium and AT&T, with 6.77 terabytes of Wi-Fi traffic and an additional 1.9 TB on the AT&T network on the stadium’s DAS for a total wireless figure of 8.6 TB.

The Wi-Fi numbers put Sunday’s signature WWE event (also the biggest WrestleMania by attendance) into second place in our unofficial record-keeping of the largest single-day Wi-Fi traffic stadium events. Only Super Bowl 50 at Levi’s Stadium earlier this year was bigger, with 10.1 TB of Wi-Fi traffic. So far, WrestleMania 32 is also now third in combined Wi-Fi and DAS figures, trailing Super Bowl 50 and Super Bowl XLIX (see charts below).

THE NEW TOP 3 TOTAL USAGE

1. Super Bowl 50, Levi’s Stadium, Santa Clara, Calif., Feb. 7, 2016: Wi-Fi: 10.1 TB; DAS: 15.9 TB; Total: 26 TB
2. Super Bowl XLIX, University of Phoenix Stadium, Glendale, Ariz., Feb. 1, 2015: Wi-Fi: 6.23 TB; DAS: 6.56 TB**; Total: 12.79 TB**
3. WrestleMania 32, AT&T Stadium, Arlington, Texas, April 3, 2016: Wi-Fi: 6.77 TB; DAS: 1.9 TB*; Total: 8.6 TB*

* = AT&T DAS stats only
** = AT&T, Verizon Wireless and Sprint DAS stats only

THE NEW TOP 5 FOR WI-FI

1. Super Bowl 50, Levi’s Stadium, Santa Clara, Calif., Feb. 7, 2016: Wi-Fi: 10.1 TB
2. WrestleMania 32, AT&T Stadium, Arlington, Texas, April 3, 2016: Wi-Fi: 6.77 TB
3. Super Bowl XLIX, University of Phoenix Stadium, Glendale, Ariz., Feb. 1, 2015: Wi-Fi: 6.23 TB
4. Alabama vs. Texas A&M, Kyle Field, College Station, Texas, Oct. 17, 2015: Wi-Fi: 5.7 TB
5. College Football Playoff championship game, AT&T Stadium, Arlington, Texas, Jan. 12, 2015: Wi-Fi: 4.93 TB

Wi-Fi not to blame for stadium entry issues

John Winborn, chief information officer for the Dallas Cowboys Football Club, said in an email that the reported claims of the Wi-Fi being offline Sunday — and that being the reason why entry lines were long and slow — were not true. While Winborn did admit that one single Wi-Fi AP (out of the more than 2,000 in the stadium’s network) was offline and there were “a couple issues” with ticket scanners, he said “there were no Wi-Fi issues that would have had a significant impact on ingress.” Other reports have claimed the doors were opened later due to extended show rehearsals, while commenters on MSR’s posts have claimed that a lack of wristbands for stadium-floor seating also led to seating issues even for fans already inside the main building entrances. So far, we have not seen any official explanation for the delays other than the official apology from the stadium and the WWE:

“To ensure the safety of WWE fans, increased security measures were put in place tonight. We apologize that it may have taken some fans longer than usual to get into AT&T Stadium.”

During Sunday’s event Winborn said the Wi-Fi network saw 20,462 concurrent and 34,951 total user connections, some via a network of 150 temporary Wi-Fi APs installed among the seats on the stadium floor.

Betting the Under (Part 2): Putting Wi-Fi antennas under seats is the hot new trend in stadium wireless networks

Under-seat Wi-Fi AP at Levi's Stadium. Photo: Paul Kapustka, MSR (click on any photo for a larger image)

Under-seat Wi-Fi AP at Levi’s Stadium. Photo: Paul Kapustka, MSR (click on any photo for a larger image)

Part 2 of this story picks up with the decision to put Wi-Fi APs under seats at Levi’s Stadium. If you missed it, here is the link to Part 1.

According to Chuck Lukaszewski, now vice president of wireless strategy and standards at Hewlett Packard Enterprise (formerly very high density architect in the CTO Office of Aruba Networks), Aruba had been testing under-seat AP designs since around 2010, “in one form or another.” There were some initial tests of under-seat AP deployments at Turner Field in Atlanta and at American Airlines Arena in Dallas, but nothing on the scale of AT&T Park’s 2013 deployment, or on the scale Aruba planned to have at Levi’s Stadium when it opened in 2014.

Some of the first under-seat Wi-Fi deployments in other arenas were actually deployed completely under the stands, Lukaszewski said, with signals shooting up through the concrete. Though he said “you could get reasonably good throughput through concrete,” especially for 2.4 GHz frequencies, installing antennas above the concrete was “considerably better,” Lukaszewski said.

Curiously, one of the biggest problems in stadium Wi-Fi deployment — especially for those heavy on overhead antenna use — is negotiating interference between antennas; sometimes, clients can “see” antennas and APs that are across the stadium, and will try to connect to those instead of the AP closest to them, a problem that leads to inefficient bandwidth use. Interference also means you can’t place APs too closely together, making it somewhat of an art to find ways to increase coverage without increasing interference.

Dan Williams, former VP of technology for the San Francisco 49ers, talking networking at Levi's Stadium. Photo: Paul Kapustka, MSR

Dan Williams, former VP of technology for the San Francisco 49ers, talking networking at Levi’s Stadium. Photo: Paul Kapustka, MSR

What Aruba found in its testing, Lukaszewski said, was that under-seat Wi-Fi AP deployments could be far more dense than overhead-centric designs, mainly because the human bodies in the seats would provide beneficial “blocking” of signals, allowing network designers to place APs more closely together, and to be able to re-use the same Wi-Fi channels in more antennas.

“If you can use human bodies to contain signals, you can have much smaller cells,” Lukaszewski said. Under-seat deployments, he said, “allows us to re-use the same channel less than 100 yards away.”

With more channels available for each AP, the difference in the metric Lukaszewski calls “megabytes per fan” can be “profound” for an under-seat design versus an overhead design, he said.

“We do see trends [in stadium network data] of under-seat being able to deliver well over 100 MB per fan per event, while overhead designs [deliver] significantly under 100 MB per fan per event,” said Lukaszewski.

Dan Williams, the former vice president of technology for the San Francisco 49ers, said he and Lukaszewski were in agreement that under-seat was the best method to deploy at Levi’s Stadium.

Kyle Field at Texas A&M. White spots in stands are under-seat AP locations. Photo: Paul Kapustka, MSR

Kyle Field at Texas A&M. White spots in stands are under-seat AP locations. Photo: Paul Kapustka, MSR

“I just did not believe in overhead,” said Williams, who said he brainstormed with Aruba’s Lukaszewski on the under-seat idea, which they both brought to the Wi-Fi design at Levi’s. By using under-seat APs, Williams said, the Levi’s Stadium design looked to provide “cones [of bandwidth] around the audience, immersing [fans] in a signal.”

After beating the previous year’s Super Bowl Wi-Fi total at its NFL regular-season opener in 2014, Levi’s Stadium’s Wi-Fi network more than passed its biggest test ever this year, carrying a record 10.1 terabytes of Wi-Fi data during Super Bowl 50. Those numbers are proof of Lukaszewski’s claim: “By far, under seat is better.”

New deployments trending to under-seat

Editor’s note: This excerpt is from our latest STADIUM TECH REPORT, our long-form PDF publication that combines in-depth stadium tech reports with news and analysis of the hottest topics in the world of stadium and large public venue tech deployments. Enjoy this PART 1 of our lead feature, or DOWNLOAD THE REPORT and read the whole story right now!

Even though under-seat deployments can be considerably more expensive, especially in a retrofit situation where deployment requires coring through concrete, many stadiums are now seeming to agree with another Lukaszewski claim, that “the return absolutely justifies the investment.”

At AT&T Stadium in Arlington, Texas, the Cowboys quicked followed their sister park’s lead and installed under-seat APs in force ahead of that venue’s hosting of the inaugural College Football Playoff championship game in January of 2015. John Winborn, chief information officer for the Dallas Cowboys Football Club, said the team worked with AT&T’s “Foundry” innovation centers to produce a smaller, sleeker under-seat AP enclosure that fit well with the stadium’s commitment to aesthetics.

Back on the baseball side, the Giants now have 1,628 Wi-Fi APs in their park, with the vast majority of them under-seat, in all three decks of seating. And the Giants’ main rival to the south, the Los Angeles Dodgers, also used under-seat APs in a recent Wi-Fi upgrade.

Close-up of conduit running to under-seat AP at Kyle Field. Photo: Paul Kapustka, MSR

Close-up of conduit running to under-seat AP at Kyle Field. Photo: Paul Kapustka, MSR

And if Levi’s Stadium led the way for under-seat Wi-Fi, the new mainly under-seat network at the refurbished Kyle Field at Texas A&M might be the QED on the debate, with ultra-fast network speeds and big data-consumption numbers (including 5.7 TB of Wi-Fi at a game versus Alabama) adding measureable momentum to the under-seat trend. Bill Anderson, CEO of Wi-Fi deployment strategy firm AmpThink, said he was an early disbeliever in under-seat Wi-Fi — until he saw the numbers.

“At first we mocked it, made fun of it,” said Anderson, whose firm has been called in to produce Wi-Fi network designs for several recent Super Bowls, as well as for the Kyle Field design. But when Aruba showed AmpThink the data from under-seat tests and deployments, “that was the ‘a-ha’ moment for us,” Anderson said.

Working with Aruba at Kyle Field, AmpThink was able to collect its own data, which convinced Anderson that under-seat was the way to go if you wanted dense, high-performing networks.

“The really important thing is to get APs closer to the people,” said Anderson. “That’s the future.”

Anderson said some doubters may remain, especially those who try to mix a small amount of under-seat APs with existing overhead deployments, a recipe for lowered success due to the potential interference issues. At Texas A&M, Anderson said AmpThink was able to build a design with far less interference and much greater density than an overhead solution, producing numbers that people have to pay attention to.

“We only know what we’ve observed, but we’re evangelistic supporters” of under-seat designs, Anderson said. “If someone says to you under-seat is hocus-pocus, they’re not looking at the data.”

Not for everyone, but more are trying under-seat

Though proponents of under-seat Wi-Fi all agree on its ability to deliver denser, faster networks, they all also agree that under-seat can be considerably more costly than overhead Wi-Fi, especially in a retrofit situation.

In addition to having to core through concrete seating areas to get conduit to the under-seat APs, the devices themselves need to be sealed, to guard them from weather, drink spills, and the power-washing equipment employed by most stadiums to clean seating areas.

Aruba’s Lukaszewski also noted that under-seat deployments generally use more linear feet of cabling to connect the APs than overhead, which also drives up the cost. Then since under-seat designs tend to use more APs, that also means a higher budget to cover a higher number of devices.

A row shot of the under-seat APs at AT&T Stadium. Photo: Dallas Cowboys

A row shot of the under-seat APs at AT&T Stadium. Photo: Dallas Cowboys

For some stadiums, the construction materials used prohibit the under-seat option from even being tried. At the Green Bay Packers’ legendary Lambeau Field, a late-1950s construction design that used lots of concrete and rebar — as well as part of the stadium’s bottom sitting directly in the ground — meant that under-seat Wi-Fi wasn’t an option, according to Wayne Wichlacz, director of information technology for the Packers.

Other stadiums, like the University of Nebraska’s Memorial Stadium, don’t have enough space between the stadium’s bleacher seats and the floor for under-seat APs to be safely installed. And many schools or teams simply don’t have big IT budgets like the $20-million-plus available to Texas A&M that allowed the Kyle Field design to seek the best result possible.

But many of the new stadiums under construction, as well as existing venues that are planning for new best-of-breed networks, have already committed to under-seat Wi-Fi designs, including the Sacramento Kings’ Golden 1 Center, where Ruckus Wireless will implement its first under-seat stadium Wi-Fi network.

Steve Martin, senior vice president and general manager at Ruckus, said the Golden 1 Center design, planned to be the most dense anywhere, will “primarily be underseat,” a choice he said “helps in a lot of ways.”

Foremost is the performance, something Martin said Ruckus has been testing at the Kings’ current home, the Sleep Train Arena. “It [under seat] does give you the isolation for frequency re-use,” he said.

The under-seat design also makes sense in Golden 1 Center since the stadium’s overall design is very open, with lots of glass walls and unobstructed views.

And under-seat deployment is even making inroads into the distributed antenna system (DAS) world, with Verizon Wireless implementing more than 50 under-seat DAS antennas at Levi’s Stadium prior to Super Bowl 50. Mainly installed to cover the bottom-of-the-bowl rows, the under-seat APs helped Verizon manage a record day for DAS traffic, with 7 TB reported on its in-stadium cellular network during the game.

“To get a quality signal, we had to go under seat,” said Brian Mecum, vice president, network, for Verizon Wireless, who said that in that area of the stadium, under seat was the only way to get a quality signal close to the subscriber’s phone. Verizon, he said, helped design the under-seat DAS antenna, and is looking to deploy it in other stadiums soon.

“It’s the first of more,” he said.

END PART 2… HERE IS THE LINK TO PART 1… TO READ THE WHOLE STORY NOW, DOWNLOAD OUR REPORT!

Commentary: Will stadium networks ever catch up with demand?

Dallas fan in mobile action at AT&T Stadium. Photo: Phil Harvey, MSR

Dallas fan in mobile action at AT&T Stadium. Photo: Phil Harvey, MSR

If there’s one thing the stadium-networking industry learned — or should have learned — from the stunning stats from Super Bowl 50, it’s that nobody really knows where or when the demand for in-venue bandwidth will stop growing.

Thanks to the smart folks who built the Levi’s Stadium Wi-Fi network — and the smart deployers and carrier partners who doubled down on the DAS this summer — the venue was able to handle an incredible 26 terabytes of wireless data on Super Sunday, with 15.9 TB amongst the four major cellular carriers on the DAS, and 10.1 TB on the stadium’s Wi-Fi network. Both were easily new records for single-day events, far eclipsing the totals seen at Super Bowl XLIX the year before.

The question, of course, for everyone else is: What comes next? And what does that mean for networks built in the past few years?

Keep building for growth

Editor’s note: This column is an excerpt from our latest STADIUM TECHNOLOGY REPORT, which is available for FREE DOWNLOAD from our site. In addition to this analysis and stadium tech deployment profiles we also take an in-depth look at the new trend of deploying Wi-Fi and DAS antennas under seats, and provide a wireless recap from Super Bowl 50. GET YOUR COPY today!

Texas A&M student recording the halftime show. Photo: Paul Kapustka, MSR

Texas A&M student recording the halftime show. Photo: Paul Kapustka, MSR

In some follow-up discussions with folks from Aruba, the Levi’s Stadium networking team, Verizon Wireless and others in the industry, I wondered out loud if what we are seeing is just yearly growth in demand, or whether expanded networks are merely satisfying pent-up demand that was always there.

To be sure, the “big” events like the Super Bowl, the World Series and the Final Four are always going to generate outsize traffic numbers, especially as more and more fans join the selfie craze and post photos and videos from the stadium, showing the rest of the world how great their life is. According to Verizon, the amount of traffic uploaded on their DAS network at Super Bowl 50 was double the upload traffic the carrier saw at Super Bowl XLIX, validating Verizon’s belief that traffic was going to grow significantly.

For mobile traffic in general, and for in-stadium wireless activity specifically, the continued growth over the past few years can likely be traced to a number of factors, including the ever-increasing power of mobile devices; the new number of bandwidth-hungry apps that incorporate images and video; and a steady increase in familiarity with devices and apps, as older folks — those more likely to purchase event tickets — catch up to their kids in wanting to use, and being able to use, the content-sharing features of social media and other communication apps. And, simply, more people trying and successfully connecting to in-stadium networks, reversing historical thinking that told them from experience in past years that connecting at a stadium was futile. In almost all venues these days, no longer is that the case.

So with no end yet in sight to the general doubling of traffic on a year to year basis, the big question out there for stadium technology teams has to be — is what we have now enough to handle growing needs, and if not, what are we going to do about it? The fact that cellular carriers (correctly) assumed they needed to add capacity to the Levi’s Stadium DAS a year after it was deployed should give pause to anyone building a network right now; though your venue may not be hosting the Super Bowl anytime soon, it might not hurt to re-think the traffic projections that have been made and step them up a bit, just in case.

These signs are up all over AT&T Park. Photo: Paul Kapustka, MSR

These signs are up all over AT&T Park. Photo: Paul Kapustka, MSR

On both the Wi-Fi side and the DAS side, we may also be looking soon at a need for new kinds of technology to help build the networks of the near future. In this issue we take an in-depth look at the trend toward putting antennas for both Wi-Fi and DAS under seats, mainly to build networks that are more dense and can handle more traffic.

Verizon, which pioneered DAS under-seat antennas at Levi’s Stadium this year, said it is already seeing stadiums where just adding another sector of traditional DAS produces diminishing returns; if traffic on both types of networks keeps growing at the current rate, when will existing designs become obsolete? And what will replace them?

Stadium network owners shouldn’t ignore policy

As stadium-networking types ponder the future, it would serve the industry well to start thinking together as well, especially in the areas of telecom policy and standards-setting. As the builders of telecom networks worldwide ponder the future of 5G technologies there are some discussions where stadium networking representatives are nowhere to be found, especially in the potentially troubling direction of LTE-U, the idea of carriers using unlicensed bands for LTE traffic.

It’s still early days in the LTE-U discussions, but a recent decision by the FCC to allow carriers to conduct tests of the technology should be a call to action for the stadium networking industry. While nobody knows quite for sure yet what will happen to an in-venue Wi-Fi network if and when LTE-U traffic appears, it seems to make sense for the industry to get together and at least present some kind of unified voice before decisions get made.

At the recent Silicon Flatirons telecom policy conference in Boulder, Colo., there were exactly zero representatives from the venue Wi-Fi ecosystem present, including no visible representation from the Wi-Fi and DAS gear manufacturers who sell into the stadium networking space. With telecom providers hungry for spectrum of any kind, it’s a fool’s bet to assume that the unlicensed bands used in stadiums worldwide today will remain as free and available as they are now. It’s just another factor that’s needed to be considered, as venue tech professionals plan for a future of ever-increasing needs.

Stadium Tech Report: Connectivity soars at Denver Broncos’ Sports Authority Field at Mile High

Panoramic view of Sports Authority Field at Mile High from the top seats. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

Panoramic view of Sports Authority Field at Mile High from the top seats. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

On most of our stadium visits, we have to wait until we get inside the venue to start testing the network. At Sports Authority Field at Mile High, however, we barely got out of the car before the Wi-Fi auto-connected — at superb speeds. Nothing like a network that announces itself before you get in the door.

The parking-lot connection — at a download speed of 45.48 Mbps and an upload speed of 53.35 — was the first clue that football fan connectivity is taken seriously in Denver, especially so if you have Verizon service. While the stadium’s Wi-Fi network is currently only available to Verizon customers — more on this later — full DAS participation by the three other major U.S. wireless carriers means that pretty much any visitor to the venue is going to have good, if not great, connectivity for their mobile device, no matter which service they use.

Inside the stadium, a trained eye can spot many different types of DAS and Wi-Fi antenna placements, under overhangs, on towers, on ceilings and on walls; and thanks to a first-person stadium tech tour conducted by Russ Trainor, vice president of information systems for the Denver Broncos, we got to learn about a wide range of not-so-noticable antenna deployments, including in railing enclosures and on field-level walls, all part of an ongoing plan to try to stay ahead of the still-growing demand for mobile data from sports fans who come to the games.

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

The day we visited, during the last regular-season game on Jan. 3, was important for the Broncos as a team since their 27-20 victory over the San Diego Chargers gave Denver home-field advantage through the playoffs, an edge that helped the team reach its eighth Super Bowl. But even as he celebrated his team’s win, Trainor was happy for another reason: the bye week gave him and his team more time to light up some new Wi-Fi and DAS antenna placements, to better handle the expected and eventual playoff data crush.

“You can never have enough APs,” Trainor said.

Good Wi-Fi, but still only for Verizon customers

Opened on Aug. 11, 2001, with a concert by the Eagles, the then-named Invesco Field at Mile High replaced the old Mile High Stadium in basically the same spot, sitting at 5,280 feet above sea level. Seen by many on TV when it hosted the 2008 Democratic National Convention and the acceptance speech of then-Sen. Barack Obama, the “new” Mile High has seen more than 12 million fans come through its doors since it opened for a variety of sports and entertainment events.

But true high-speed wireless for fans didn’t take root until 2012, when a revamp led by Verizon Wireless and the Broncos’ IT staff added a Cisco-based Wi-Fi network to the stadium with 500 access points, designed to serve 25,000 concurrent users and also designed to be “open,” allowing any other carrier to provide access to its customers by negotiating a deal with Verizon. While Trainor said the option still remains open and talks with some of the other carriers are underway, none have yet signed on — making the Wi-Fi network a fast playground for Verizon customers, who apparently are in the vast majority in the Denver region.

Sorry, AT&T customer, no soup for you

Sorry, AT&T customer, no soup for you

We don’t have any exact proof of that thinking, but statistics from the recent AFC Championship game at Sports Authority Field — a 20-18 Denver victory over the New England Patriots — seem to show Verizon customers in a bit of a majority. According to Verizon, its customers at the game used a total of 2.87 terabytes of data, with 1.7 TB on the Wi-Fi network and another 1.17 TB on the Verizon LTE DAS network. AT&T, by comparison, said its customers used 819 GB on the AT&T DAS network that day. So either there are more Verizon customers at the stadium on game days, or Verizon customers use more data because they have more network options; take your pick.

With our Verizon iPhone 6 Plus in hand, we found great connectivity on Wi-Fi pretty much everywhere we roamed. After finding our way from the parking lot to the press box, we got a signal of 46.46 Mbps down and 46.90 up, this from the regular fan network in the stands and not from the press-only Wi-Fi network.

While roaming through the plush United Club we got a speed test of 33.36/35.19, a figure that Trainor said could change on any given game day — “when it gets cold outside, this place fills up,” he noted — and then later when we walked up to the top, 5th-level concourse, we still got a Wi-Fi signal of 34.96/30.40 on the walkways behind the seats. During second-quarter action we even sneaked up to the nosebleed seats in section 501, one of the ski-slope steep sections near the stadium’s top edge — and still got a Wi-Fi signal of 10.28 Mbps/5.00 Mbps.

According to Trainor, the upper seats are among the toughest challenges for Wi-Fi reception, especially those in the “bulge” areas in the middle of the stadium where on both sides the sections curve upwards, adding more seats. Though the light structures that wind all the way around the stadium do provide good spots for antenna mounts, the bulge areas are harder to reach, and in the near future Trainor and his team will keep experimenting with other methods of deployment, like railing enclosures and row-end mounts they have used successfully for both Wi-Fi and DAS in other areas of the stadium.

Lots of antennas visible in this overhang area

Lots of antennas visible in this overhang area

One interesting architectural quirk of the stadium — its use of metal decking instead of concrete — actually helps the wireless deployment team, Trainor said. Installed to mimic the metal upper deck at the old Mile High Stadium — where Broncos fans would do the “Denver Stomp” to produce thunderous noise — the metal construction acts as a barrier to keep Wi-Fi signals from the bowl from interfering with those from antennas inside suites and concourses, Trainor said.

While most of the stadium has favorable locations for overhead antennas — there are three main levels of seating, providing two expansive overhangs covering about 80 percent of the seating area — some typical problem places like seats near field level and in the no-overhang South stands have required some creative thinking, an excercise that never really ends.

“We started with 500 Wi-Fi APs, and we’re now at 640, and by the time we get it [the current plan] all built out we’ll have about 850 to 900 total,” Trainor said.

DAS deployments a mix of connectivity

On the DAS side, Trainor said that the four major carriers — Verizon Wireless, AT&T, Sprint and T-Mobile — are all present inside the stadium, with different antenna placements in different numbers. In some instances, all the carriers use “neutral” antennas, mainly in areas where there isn’t enough room for exclusive deployments. But in other areas, the carriers have installed their own antennas, an arrangement that allows them to replace and upgrade them as necessary at their own discretion, Trainor said.

Field-level Wi-Fi AP (small white box next to right leg of Peyton Manning fan)

Field-level Wi-Fi AP (small white box next to right leg of Peyton Manning fan)

We didn’t have a Sprint or T-Mobile device on hand, but our AT&T Android phone had good connectivity everywhere we measured, including a 4G LTE signal of 27.94 Mbps down and 6.86 up in the press box, and signals of 47.83/6.37 on the same 5th-level concourse area where we tested the Verizon Wi-Fi.

All the carrier back-end gear is housed in a brick building built outside the southeast side of the stadium, Trainor said, since there wasn’t room inside the stadium structure itself. DAS and Wi-Fi antennas also exist in great number in the vast parking lots that directly surround the stadium, as well as in the “fan zone” gathering area outside the South stands.

Like with the Wi-Fi, Trainor and his team are always planning for more DAS capacity, even if contracts aren’t signed yet. On the new railing enclosures they are installing, the Denver IT team builds in enough space for both DAS and Wi-Fi, even if only one network is using the deployment to start with. Again, you can never have enough antennas — or enough places to put them.

YinzCam app and Cisco SportsVision

Rounding out the mobile-device offerings is not one but two YinzCam team apps, one for use at outside the stadium and the other one for live game-day offerings, with a geocache feature that allows the team to provide content it has stadium rights to, like the NFL’s RedZone channel. Both apps have live links to the Broncos radio coverage from KOA Radio, and the in-stadium instant replay feature worked superbly during our visit, showing plays in seconds and often before they appeared on the stadium’s big screens.

In the concourses we recognized the split-screen capabilities of Cisco’s StadiumVision technology, which can direct programming to all the TV screens inside a stadium. Another nice touch in the United Club was a circular charging station, with tabletop space so fans could have a place to put food and drink while waiting for their devices to juice back up. “We are always looking for ways and configurations to allow fans to recharge their devices,” Trainor said.

With all its different parts, the wireless deployment at Sports Authority Field at Mile High adds up to a favorable fan experience, one that clearly has the ability to keep getting better on an incremental basis. But like their Super Bowl team, Denver fans should be happy with what they have right now.

MORE PHOTOS BELOW

mihi7

Railing antenna enclosure. Some of these have both Wi-Fi and DAS.

mihi11

App promo on the scoreboard

mihi13

Panoramic view of the stadium and the city

mihi4

South stands have a horse and Wi-Fi antennas on the top of the scoreboard

mihi20

Cisco SportsVision in action on 6-panel display

mihi22

DAS antennas on end-of-row railing area

mihi24

Game on, phones out!

mihi25

Antennas covering the concourse area on second level

mihi26

More SportsVision and Wi-Fi deployment in the United Club

mihi14

Only accept on the scene reporting!