Denver Broncos, Verizon bring Wi-Fi blitz to Sports Authority Field at Mile High

Railing-mounted Wi-Fi enclosures in the lower seating bowl at Sports Authority Field at Mile High in Denver. Credit all photos: Paul Kapustka, MSR (click on any photo for a larger image)

Even as the team on the field seeks to regain its recent Super Bowl champion status, Denver Broncos fans will have something they can all cheer this season — vastly improved Wi-Fi networking service, which is now available to all fans and not just Verizon Wireless customers.

Even though we found good connectivity on our last visit to Sports Authority Field at Mile High two seasons ago, the caveat was that Wi-Fi was only available to Verizon customers, since the carrier couldn’t find another wireless provider who would chip in to fund the system. Fast-forward to later in 2016, when Verizon and the Broncos sought to upgrade the entire system and ended up picking a $6 million bid from Cisco to install a total of 1,470 new Wi-Fi APs, replacing the 640 APs in the old system, which started out with 500 Cisco APs in 2011 and added some more over the years.

The new network, which is scheduled to be fully completed by late October or early November this year, is already live in parts of the lower bowl at Mile High as well as in many concourse, suite and back-of-house locations. The big difference inside the hardware is the Broncos’ and Verizon’s choice of using the new Cisco 3800 APs, which can have two separate antennas in each device, basically doubling the amount of connectivity per unit. The new network will be powered by a new 10-gig backbone pipe provided by CenturyLink, replacing the 1-Gbps pipe that was previously used.

Close-up of a lower-bowl railing AP mount.

Cisco 3800s are proving to be a popular choice in venues lately, being picked for recent deployments at the San Jose Sharks’ SAP Center and at Notre Dame Stadium.

“The 3800 is a game-changer,” said Russ Trainor, the Broncos’ vice president of information technology, during a stadium tour Tuesday, citing its ability to connect more fans per device. Perhaps the most visually striking note of the upgrade is the huge amount of new railing-mounted APs in the building, with several per row not an uncommon sight in the lower bowl. Jason Moore, a senior IT engineer with the Broncos (and as Trainor calls him, a “Wi-Fi wizard”) said the enclosures are all custom designs from a local provider, with some of the fiberglass structures housing not just Wi-Fi but Verizon DAS antennas as well.

(Right now, the DAS situation at Mile High remains unchanged from our last visit, with all cellular carriers basically running their own operations.)

Going under-seat in bright orange fashion

The other new deployment method being used by the Broncos is under-seat placement, a tactic used for about 90 APs so far, half of those in the South end zone seats, where there are no overhead structures at all. Overhead AP placements are also being used in the main seating bowl, mainly to serve rows at the tops of sections.

No mistaking where the under-seat APs are at Broncos games.

Unlike other stadiums, who try to hide the under-seat APs as best they can, the Broncos have gone the opposite direction, painting many a bright Broncos orange to show up under the Broncos blue seats. “The mounting options are just about getting the APs as close to fans as we can,” Moore said. “Railing [mounts] work great. Under seat is new to us, and we’re excited to see how they work.”

In a quick empty-stadium test, MSR found Wi-Fi speeds in the south stands of 46 Mbps download and 47 Mbps up, on both the all-access and Verizon-customer SSIDs. In section 309, right at the 50-yard line, we got a Wi-Fi speed reading of 69 Mbps down, 75 Mbps up. Verizon execs at the tour said that like at other NFL stadiums where Verizon helps provide the Wi-Fi, Verizon customers will have reserved bandwidth and an autoconnect feature that links them to Wi-Fi without any sign-in needed. Trainor said the Broncos are still undecided how to approach the all-access Wi-Fi onboarding, though the team is leaning in the favor of having some kind of portal approach to gather information from fans using the service.

Of the 1,470 planned new APs (that count may change as final tuning is made, Moore said), the Broncos plan to deploy 920 of those in the seating bowl. With many of those devices having two 5 GHz antennas for each AP, the Sports Authority Field at Mile High crowds should enjoy one of the league’s top network experiences when all the work is completed.

Both railing and under-seat deployments are used to bring Wi-Fi to the south stands.

Those readers who closely track MSR stories for such stats should know that we are now working on a new chart to show not just the top numbers of APs in stadiums, but actual radios and antennas thanks to devices like the Cisco 3800 that have more than one per unit. (Any and all help with such counts is appreciated, you know where to find us!)

Like other stadiums, the network in the bowl at Sports Authority Field at Mile High will switch to only 5 GHz connections when complete. Even a few years ago, stadiums needed to still support 2.4 GHz Wi-Fi connections for fans, but the quick shift in consumer devices has shown that almost all mobile devices used these days have a 5 GHz radio.

A string of summer concerts at the stadium (which still bears the name of the now-bankupt and closed sporting-goods business as the Broncos search for a new title sponsor) kept the network deployment from being completed sooner, but Trainor and Moore said the incremental improvements are already being noticed. With both the old system and new system working simultaneously, Moore said that at last week’s college game between the University of Colorado and Colorado State University, the network saw approximately 35,000 unique users — more than the typical 25,000 unique connections during a Broncos game when only Verizon customers could use the network.

“Our goal and challenge is to connect as many fans as possible,” Moore said.

A smaller railing mount seen in the upper deck (300 level) seating section

Wi-Fi antenna mounts in the ceiling of the United Club

Wi-Fi coverage also exists for the fan-gathering area outside the stadium to the south

No Wi-Fi here, just white horses

Alamodome taps AmpThink for new Wi-Fi ahead of 2018 Final Four

A new full-stadium Wi-Fi network installed by AmpThink is coming to the Alamodome, scheduled to be finished just ahead of this year’s Alamo Bowl and well in place for next spring’s men’s NCAA basketball tournament’s Final Four, Alamodome executives said.

Scheduled to be announced publicly by the San Antonio, Texas, venue today, the new network is part of a $50-million-plus renovation project that includes updated video boards, sound systems and TV screens throughout the stadium. Nicholas Langella, general manager of the Alamodome, said the new Wi-Fi network was financed in part by donations from Alamodome customers, including the Valero Alamo Bowl, scheduled this year for Dec. 28. The network will use Wi-Fi gear from Cisco, according to Langella.

According to Langella, approximately $6 million out of the roughly $10 million needed for the Wi-Fi upgrade came from the Alamo Bowl. Langella also said that the venue now has an updated DAS as well, built by Verizon, which will also have AT&T and T-Mobile on board. “We’re very happy about that [the DAS],” said Langella in a phone interview.

Going under seat for Wi-Fi

Though Wi-Fi deployment firm AmpThink has lately preferred railing Wi-Fi enclosures for proximate network builds, such as at Notre Dame, Langella said the Alamodome deployment will use more under-seat AP placements than railings, given the designed mobility of the Alamodome seating areas. “We have so much mobility with the stands, it’s hard to do lots of railing [placements],” Langella said.

According to Langella when the Wi-Fi deployment is finished — the network is scheduled to be fully completed by Dec. 1 — there will be approximately 750 APs installed, allowing the Alamodome to increase coverage from being able to serve 3,500 fans to being able to cover 65,000 fans, meaning every seat in the house. The improvements, he said, were part of a plan to attract the Final Four, which succeeded.

“We always thought we would improve the Wi-Fi,” Langella said. With the Final Four looming, he said, “we took the bull by the horns and got it done.”

Colorado State fans connect with Wi-Fi at stadium-opening victory

For both stadium networks and football teams, the first games are always a challenge; but at Colorado State there was nothing but smiles and celebrations as fans enjoyed great Wi-Fi connectivity at the new CSU stadium during the Rams’s 58-27 win over Oregon State Saturday.

With a sellout crowd of 37,583 crammed into the new on-campus venue for the Fort Collins, Colo., school, the not-yet-finished Wi-Fi network built by 5 Bars nevertheless saw 5,891 unique connections during the day with a peak concurrent number of 3,680 users at 3 p.m. local time. Even with most of the under-seat Wi-Fi connections not yet online (5 Bars said the network was about at 35 percent capacity), the network still saw 2.7 terabytes of data used, an average of 458 MB per connected user.

And though we don’t have the connection locations, some enclosed speedtest screen shots sent to us by the 5 Bars crew seems to show very good connectivity via Wi-Fi, with a top reading of 47.70 Mbps down and 65.88 up, as well as a couple other readings in the 20 Mbps range. Mobile Sports Report plans to be on hand for the next home game at the new CSU stadium on Sept. 9, so stay tuned for more testing. According to 5 Bars the Wi-Fi network should be “75 percent complete” by then, so we will test wherever we can.

The 5 Bars crew also said that Verizon was live on the stadium DAS, but so far we have not seen any stats from Verizon about performance. Stay tuned for more tests and pictures when we head north a couple weeks from now!

Picture of the west stands overhang, with a Wi-Fi antenna highlighted. Credit all photos below: 5 Bars

Gear locations on the east-side lighting stanchions

Under-seat AP enclosure. Pretty sure Wi-Fi can go through popcorn

Good speeds!

Wi-Fi, app ready to go for Falcons’ preseason opener at Mercedes-Benz Stadium

The shiny new stadium gets ready to host its first big event this weekend. Credit all photos: Paul Kapustka, MSR (click on any photo for a larger image)

It might have taken some recent all-nighters, but the crew behind the technology at Mercedes-Benz Stadium said that the Wi-Fi and DAS networks, as well as the new app, will be ready to go when the Atlanta Falcons hold their first big event at their new home this Saturday.

“We’ve spent the last week really tuning the Wi-Fi, and it’s awesome to see the speeds we’re getting as I walk around,” said Jared Miller, chief digital officer for the Falcons, in a phone interview Thursday. “I’m anxious to see how it will do with a full house.”

The first potential for a packed stadium comes Saturday night, when the Falcons host the Arizona Cardinals in an NFL preseason game. With roughly 1,800 Aruba Wi-Fi APs installed inside (1,000 in the seating bowl and the rest in concourses and other areas), Mercedes-Benz Stadium should have excellent Wi-Fi coverage, even if it takes several events to figure out the things you can only figure out once you have live people in the seats.

“We’ll need a few events until we get to a spot where we’re dialed in [with network performance],” said Miller. Though Miller said the Falcons were able to get some network feedback during a recent season-ticket holder open house, real performance stats won’t come until fans are filling the venue for an NFL game.

“You just have to go through a series of events to see actual performance,” Miller said.

Home page of the new Falcons app from IBM

Curiously, Miller would not comment directly when asked if any carriers other than sponsor AT&T had officially signed on to be on the Falcons’ neutral-host DAS. However, he did say that “all the fans who [are at the game] should be able to have cellular connections.” (Any attendees who want to send us speedtests of Wi-Fi or DAS, you know where to find us.)

New app gets its debut

One area that might cause some delays getting into the arena is the Falcons’ decision to go to all-digital ticketing — fans must either have an RFID card (for season tickets) or must download the new team app so they can have digital tickets on their phones. Both the RFID cards and phones can be scanned at the SkiData turnstyle machines.

“We’re encouraging fans to download the app before they get to the game,” Miller said.

The new version of the team app, which was only made live in the last couple days, was built by main IT contractor IBM.

App view of a wayfinding map

The IBM app will do things a little bit differently than other stadium apps; the wayfinding maps are not “blue dot” or interactive like Google Maps or other stadium apps like those at Levi’s Stadium or Sacramento’s Golden 1 Center. Instead, fans must enter both their location and their desired destination to see a path on a stored map.

Food ordering via the app is also handled differently than apps that support in-seat delivery or express window pickup. With the Falcons app fans will select from a list of concession stands, then enter credit card information and their order, which will be scanned at the stand, according to IBM. While such new services always take some time for fans to discover and use, Miller is keen to see if the new systems work as promised.

“We want to see not just quantitative numbers but qualitative data too,” Miller said. “Did it really benefit fans? Were they able to bust the queue? Would they do it again?”

The app also has a Falcons-esque chat bot, called “Ask Arthur” for owner Arthur Blank; while the bot can quickly answer FAQ-type questions about the stadium and its operations, more open-ended queries will require perhaps some time with IBM’s Watson technology (see examples in photos below).

And on a final low-tech note, Miller said the Falcons had been in constant touch with several local groups, including the Georgia Department of Transportation, the city of Atlanta and the MARTA light rail system to get all the pertinent maps and signs updated. On Mobile Sports Report’s visit to Mercedes-Benz Stadium and Atlanta last week, we noticed that there were very few signs up with Mercedes-Benz Stadium on them.

One big map on the wall inside the nearby CNN Center (where there is a large public food court) didn’t even have Mercedes-Benz Stadium on an area map, and a sign over the MARTA station just outside the new stadium’s doors had no mention of Mercedes-Benz Stadium but still did mention the Georgia Dome, the next-door neighbor slated for demolition. The MARTA online map still lists “Dome” but not “Mercedes-Benz Stadium” for the stop outside the venue’s doors.

“We’ve scoured the city looking for anything that still says Georgia Dome,” Miller said. Getting all the new signs up, he said, is “in the process of getting done.”

The ‘Ask Arthur’ bot can answer simple questions about Mercedes-Benz Stadium but…

… don’t ask the app for Super Bowl odds

We’re hoping this map, seen here on a wall inside the nearby CNN Center, has been updated

One of the many under-seat Wi-Fi APs that will be getting their first test this weekend

Chief digital officer Jared Miller answers questions at the recent media day

On Aug. 15, this sign over the nearby MARTA station still didn’t mention Mercedes-Benz Stadium

Ready or not, Unlicensed LTE is here. What should your venue do?

The entry concourse at Atlanta’s new Mercedes-Benz Stadium. Credit all photos: Paul Kapustka, MSR (click on any photo for a larger image)

By Chuck Lukaszewski, Aruba Networks, a Hewlett Packard Enterprise company

There’s much excitement around the coming of “unlicensed LTE” and for good reason. In our anytime, anywhere world the last device many of us use at night, and the first one we pick up in the morning, is a mobile phone, tablet or computer. Although much of the time our devices connect via Wi-Fi, when we’re in transit we depend on cellular.

With consumers quick to express their disappointment when their apps fail to respond – or don’t respond fast enough – on a wireless network, cellular providers are keenly aware they must keep pace with rapidly escalating user experience expectations. Research suggests mobile data traffic will grow by 47 percent annually through 2021. Combine the two and the drivers for expanding network capacity are clear.

While the lure of more bandwidth can be attractive, stadium and venue operators need to carefully evaluate the technological impact and operational overhead unlicensed LTE introduces.

Gigabit cellular coming soon

Editor’s note: This post is part of Mobile Sports Report’s new Voices of the Industry feature, in which industry representatives submit articles, commentary or other information to share with the greater stadium technology marketplace. These are NOT paid advertisements, or infomercials. See our explanation of the feature to understand how it works.

To provide gigabit speeds, the cellular industry has enhanced LTE technology to bond multiple channels together, called “carrier aggregation.” Although originally designed only to combine different licensed frequencies, it has now been extended to aggregate licensed spectrum with 5 GHz unlicensed spectrum (where Wi-Fi operates). Two competing technologies for doing so have emerged, with notable differences when deploying in high-density environments like stadiums in the U.S.

LTE-U (LTE in the Unlicensed Spectrum) is a proprietary technology, developed by the LTE-U Forum, a consortium of several cellular-related companies. It enables simultaneous operation of LTE over both licensed and unlicensed spectrum by aggregating the bands together, resulting in a performance boost. However, the way LTE-U takes control of a channel – while legal in the U.S. – is controversial and may significantly degrade performance of Wi-Fi equipment using the same channel. The Wi-Fi and cellular industries worked together to produce a coexistence test plan, but so far none of the test results for LTE-U equipment authorized by the FCC have been made public.

LAA (Licensed Assisted Access) can be thought of as the standardized version of unlicensed LTE, designed to meet European “listen-before-talk” (LBT) requirements, so it can be deployed anywhere on the planet. It was developed through the 3rd Generation Partnership Project (3GPP) worldwide standards organization, with wide participation including input from the Wi-Fi community.

DAS gear above concession stand at Coors Field

Think of LBT like the telephone party lines of yesteryear, where multiple customers share a communal phone line but only one person can use it at a time for their conversation while others wait. When there is no conversation happening on the party line and two or more people try to speak at once, other customers of the party line graciously “back off” to allow one person to go first. In cellular terms, this makes LAA a more “polite” technology than LTE-U, as it waits to transmit until a channel is clear. The back-off method it uses is compatible with Wi-Fi at least on paper, although 3GPP does not require vendors to perform or publish any kind of test results.

The Road Ahead

Of course what you want to know is how the advent of LTE-U/LAA impacts your stadium and whether to add gigabit cellular to the connectivity mix.

As a robust, stable and mature technology, Wi-Fi’s strength and ability to handle exceptional stadium data traffic loads is well established. To make informed decisions about whether to consider LTE-U/LAA technologies alongside Wi-Fi, here are five essential technical considerations.

Spectrum Availability. The unlicensed radio spectrum is comprised of 24 channels in the U.S., which is analogous to a 24-lane freeway. Until now, only Wi-Fi traffic traveled on that roadway, with many years spent developing technologies to ensure steady traffic flow, particularly in stadiums. Wi-Fi includes its own LBT solution, which helps assure data merges smoothly onto the freeway. It’s been proven at six Super Bowls plus countless other concerts and sporting events.

Most stadium Wi-Fi networks are already spectrum-constrained, meaning they are just managing to carry the existing load – much less new fan technologies like AR/VR. A large body of evidence demonstrates that stadiums and arenas need 20-24 fulltime-equivalent channels to make a 5 GHz system work (regardless of technology). These Wi-Fi networks are carefully optimized to eliminate all unnecessary transmissions.

Adding one or more LTE-U or LAA systems will reduce available capacity for Wi-Fi operations. As of this writing, there are no public technical measurements of deployed systems so the actual impact is unknown. If four separate unlicensed LTE networks are actually deployed, the impact will be even greater.

Number of LTE-U/LAA Networks Required. Visitors to your stadium likely utilize each of the four U.S. cellular operators: AT&T, Sprint, T-Mobile and Verizon. Therefore, to offer gigabit cellular connectivity, you’ll need to permit all four to deploy an LTE-U, or LAA, network. Because the technologies are so new, they lack a “neutral host” methodology, so each operator will require its own separate physical network and spectrum.

DAS gear under seating area at SunTrust Park

Compatibility with Existing DAS. Most stadiums and arenas have either separate antenna systems for each major cellular operator or a converged neutral-host DAS. Although LTE-U and LAA are intended to support “dual connectivity” to a separate macro base station (or “eNodeB”) on paper, the products currently being shipped are intended as co-located small cells that contain two paired LTE radios – one licensed and one unlicensed. Stadium operators should validate whether their DAS systems are compatible with an expansive LTE-U/LAA small cell deployment where the primary cell (or “PCell”) is the DAS and each PCell has dozens of secondary cells (or “SCells”) providing 5 GHz service.

Cost vs. Benefit. Of no small consideration is the added amount of equipment, and the costs, in a hybrid Wi-Fi/cellular situation. If every cellular operator requires a separate LAA/LTE-U overlay, this implies up to four full new sets of equipment must be deployed under seats or on handrails. For a 60,000-seat stadium at typical under-seat densities, it would only require about 850 Wi-Fi access points (APs). In contrast, for LAA/LTE-U stadium operators would need over 3,000 additional small cells– with each one requiring a sturdy waterproof housing, a 30-watt POE connection, Cat-6 cabling, conduit and, of course, a hole drilled in the concrete. Meaning, LTE-U/LAA small cell deployments would require essentially the same physical footprint for each carrier as Wi-Fi which is likely already installed and is inherently a neutral host technolgy.

Risk. It’s also critical to consider the corresponding risks of adding up to four cellular unlicensed LTE networks to your Wi-Fi environment. It took about seven years and three full generations of radio designs for Wi-Fi vendors to perfect high-capacity stadium systems whereas LTE-U/LAA equipment is only beginning to ship. In short, it may be wise to delay comingling Wi-Fi and LTE-U/LAA networks until unlicensed LTE equipment becomes proven in less mission-critical settings than your venue.

Chuck Lukaszewski is Vice President of Wireless Strategy & Standards at Aruba Networks, a Hewlett Packard Enterprise company. For over a decade he has engineered and deployed large-scale 802.11 networks, joining Aruba in 2007.

Chuck has built Wi-Fi systems in stadiums, seaports, rail yards, manufacturing plants and other complex RF environments, including serving as chief engineer for many stadiums ranging from 20,000 to 100,000 seats that provide live video and other online amenities. He is the author of six books and design guides including Very High Density 802.11ac Networks and Outdoor MIMO Wireless Networks.

First Look: Inside the Atlanta Falcons’ new Mercedes-Benz Stadium

Big Bird greets all visitors to Mercedes-Benz Stadium. Credit all photos: Paul Kapustka, MSR (click on any photo for a larger image)

We’ll have much more to report on what we saw at the press day at the Atlanta Falcons’ new Mercedes-Benz Stadium, but we thought it was important to share these views as soon as we could — so here is an extended photo essay from the newest NFL venue (which will also be used for soccer). Unfortunately the Wi-Fi and DAS networks were live but not yet optimized, so we weren’t able to do any comprehensive speed testing (but hey, that’s what a regular season game is for!).

Overalll first impressions, technology wise — this is another well thought out venue specifically from a technology standpoint but also mainly just from a visual feel. The halo board is as impressive as advertised, though we would want to see it in action during a game (while sitting in a seat) to fully judge whether or not it fits in with the flow of an event. For advertisers it’s a wonder, as watching all the video screens in the house go to a synchronized ad video was a big wow factor.

Since much of the stadium interior is unfinished concrete, there wasn’t much of an effort to hide networking components — but given all the other piping and cabling, the equipment does kind of fade out of sight in plain view.

MSR welcomes you to the big house

It’s our educated guess that the AT&T Porch — a wide open gathering area in the end zone opposite the windows toward downtown — is going to be a popular hangout, since you can see the field and have multiple big screen TV options behind you. We also liked the “technology loge suites,” smaller four-person private areas just off the main concourse with their own small TV screens and wireless device charging.

On the app side of things, it’s fair to say that features will iterate over time — both the wayfinding and the food-ordering options are not wirelessly connected yet, but according to IBM beacons are a possible future addition to the mix. And while Mercedes-Benz Stadium is going to all-digital ticketing, season ticket holders will most likely use RFID cards on lanyards instead of mobile phone tickets simply because the RFID is a quicker option. The ticket scanners are by SkiData, fiber backbone by Corning, Wi-Fi APs by Aruba, and DAS by Corning and a mix of antenna providers.

Like we said, more soon! But enjoy these photos today, ahead of the first event on Aug. 26.

The view inside the main entry, with halo board visible above

The view from the other side of the field, from the AT&T Porch

Just hard to fit all this in, but you can see here from field to roof

I spy Wi-Fi, APs point down from seat bottoms to main entry concourse

One of the many under-seat APs

A good look at the roof: Eight “petals” that all pull straight out when open, which is supposed to take 7 minutes according to design

Good place for maximum coverage

View from the field

One of “hundreds” of mini-IDFs, termination points that bring fiber almost right to edge devices

The mega-vertical TV screen, just inside the main entry. 101 feet tall!

Something Falcons fans may like the most: Look at the prices!

MORE SOON!