Stadium Tech Report: Connectivity soars at Denver Broncos’ Sports Authority Field at Mile High

Panoramic view of Sports Authority Field at Mile High from the top seats. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

Panoramic view of Sports Authority Field at Mile High from the top seats. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

On most of our stadium visits, we have to wait until we get inside the venue to start testing the network. At Sports Authority Field at Mile High, however, we barely got out of the car before the Wi-Fi auto-connected — at superb speeds. Nothing like a network that announces itself before you get in the door.

The parking-lot connection — at a download speed of 45.48 Mbps and an upload speed of 53.35 — was the first clue that football fan connectivity is taken seriously in Denver, especially so if you have Verizon service. While the stadium’s Wi-Fi network is currently only available to Verizon customers — more on this later — full DAS participation by the three other major U.S. wireless carriers means that pretty much any visitor to the venue is going to have good, if not great, connectivity for their mobile device, no matter which service they use.

Inside the stadium, a trained eye can spot many different types of DAS and Wi-Fi antenna placements, under overhangs, on towers, on ceilings and on walls; and thanks to a first-person stadium tech tour conducted by Russ Trainor, vice president of information systems for the Denver Broncos, we got to learn about a wide range of not-so-noticable antenna deployments, including in railing enclosures and on field-level walls, all part of an ongoing plan to try to stay ahead of the still-growing demand for mobile data from sports fans who come to the games.

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

The day we visited, during the last regular-season game on Jan. 3, was important for the Broncos as a team since their 27-20 victory over the San Diego Chargers gave Denver home-field advantage through the playoffs, an edge that helped the team reach its eighth Super Bowl. But even as he celebrated his team’s win, Trainor was happy for another reason: the bye week gave him and his team more time to light up some new Wi-Fi and DAS antenna placements, to better handle the expected and eventual playoff data crush.

“You can never have enough APs,” Trainor said.

Good Wi-Fi, but still only for Verizon customers

Opened on Aug. 11, 2001, with a concert by the Eagles, the then-named Invesco Field at Mile High replaced the old Mile High Stadium in basically the same spot, sitting at 5,280 feet above sea level. Seen by many on TV when it hosted the 2008 Democratic National Convention and the acceptance speech of then-Sen. Barack Obama, the “new” Mile High has seen more than 12 million fans come through its doors since it opened for a variety of sports and entertainment events.

But true high-speed wireless for fans didn’t take root until 2012, when a revamp led by Verizon Wireless and the Broncos’ IT staff added a Cisco-based Wi-Fi network to the stadium with 500 access points, designed to serve 25,000 concurrent users and also designed to be “open,” allowing any other carrier to provide access to its customers by negotiating a deal with Verizon. While Trainor said the option still remains open and talks with some of the other carriers are underway, none have yet signed on — making the Wi-Fi network a fast playground for Verizon customers, who apparently are in the vast majority in the Denver region.

Sorry, AT&T customer, no soup for you

Sorry, AT&T customer, no soup for you

We don’t have any exact proof of that thinking, but statistics from the recent AFC Championship game at Sports Authority Field — a 20-18 Denver victory over the New England Patriots — seem to show Verizon customers in a bit of a majority. According to Verizon, its customers at the game used a total of 2.87 terabytes of data, with 1.7 TB on the Wi-Fi network and another 1.17 TB on the Verizon LTE DAS network. AT&T, by comparison, said its customers used 819 GB on the AT&T DAS network that day. So either there are more Verizon customers at the stadium on game days, or Verizon customers use more data because they have more network options; take your pick.

With our Verizon iPhone 6 Plus in hand, we found great connectivity on Wi-Fi pretty much everywhere we roamed. After finding our way from the parking lot to the press box, we got a signal of 46.46 Mbps down and 46.90 up, this from the regular fan network in the stands and not from the press-only Wi-Fi network.

While roaming through the plush United Club we got a speed test of 33.36/35.19, a figure that Trainor said could change on any given game day — “when it gets cold outside, this place fills up,” he noted — and then later when we walked up to the top, 5th-level concourse, we still got a Wi-Fi signal of 34.96/30.40 on the walkways behind the seats. During second-quarter action we even sneaked up to the nosebleed seats in section 501, one of the ski-slope steep sections near the stadium’s top edge — and still got a Wi-Fi signal of 10.28 Mbps/5.00 Mbps.

According to Trainor, the upper seats are among the toughest challenges for Wi-Fi reception, especially those in the “bulge” areas in the middle of the stadium where on both sides the sections curve upwards, adding more seats. Though the light structures that wind all the way around the stadium do provide good spots for antenna mounts, the bulge areas are harder to reach, and in the near future Trainor and his team will keep experimenting with other methods of deployment, like railing enclosures and row-end mounts they have used successfully for both Wi-Fi and DAS in other areas of the stadium.

Lots of antennas visible in this overhang area

Lots of antennas visible in this overhang area

One interesting architectural quirk of the stadium — its use of metal decking instead of concrete — actually helps the wireless deployment team, Trainor said. Installed to mimic the metal upper deck at the old Mile High Stadium — where Broncos fans would do the “Denver Stomp” to produce thunderous noise — the metal construction acts as a barrier to keep Wi-Fi signals from the bowl from interfering with those from antennas inside suites and concourses, Trainor said.

While most of the stadium has favorable locations for overhead antennas — there are three main levels of seating, providing two expansive overhangs covering about 80 percent of the seating area — some typical problem places like seats near field level and in the no-overhang South stands have required some creative thinking, an excercise that never really ends.

“We started with 500 Wi-Fi APs, and we’re now at 640, and by the time we get it [the current plan] all built out we’ll have about 850 to 900 total,” Trainor said.

DAS deployments a mix of connectivity

On the DAS side, Trainor said that the four major carriers — Verizon Wireless, AT&T, Sprint and T-Mobile — are all present inside the stadium, with different antenna placements in different numbers. In some instances, all the carriers use “neutral” antennas, mainly in areas where there isn’t enough room for exclusive deployments. But in other areas, the carriers have installed their own antennas, an arrangement that allows them to replace and upgrade them as necessary at their own discretion, Trainor said.

Field-level Wi-Fi AP (small white box next to right leg of Peyton Manning fan)

Field-level Wi-Fi AP (small white box next to right leg of Peyton Manning fan)

We didn’t have a Sprint or T-Mobile device on hand, but our AT&T Android phone had good connectivity everywhere we measured, including a 4G LTE signal of 27.94 Mbps down and 6.86 up in the press box, and signals of 47.83/6.37 on the same 5th-level concourse area where we tested the Verizon Wi-Fi.

All the carrier back-end gear is housed in a brick building built outside the southeast side of the stadium, Trainor said, since there wasn’t room inside the stadium structure itself. DAS and Wi-Fi antennas also exist in great number in the vast parking lots that directly surround the stadium, as well as in the “fan zone” gathering area outside the South stands.

Like with the Wi-Fi, Trainor and his team are always planning for more DAS capacity, even if contracts aren’t signed yet. On the new railing enclosures they are installing, the Denver IT team builds in enough space for both DAS and Wi-Fi, even if only one network is using the deployment to start with. Again, you can never have enough antennas — or enough places to put them.

YinzCam app and Cisco SportsVision

Rounding out the mobile-device offerings is not one but two YinzCam team apps, one for use at outside the stadium and the other one for live game-day offerings, with a geocache feature that allows the team to provide content it has stadium rights to, like the NFL’s RedZone channel. Both apps have live links to the Broncos radio coverage from KOA Radio, and the in-stadium instant replay feature worked superbly during our visit, showing plays in seconds and often before they appeared on the stadium’s big screens.

In the concourses we recognized the split-screen capabilities of Cisco’s StadiumVision technology, which can direct programming to all the TV screens inside a stadium. Another nice touch in the United Club was a circular charging station, with tabletop space so fans could have a place to put food and drink while waiting for their devices to juice back up. “We are always looking for ways and configurations to allow fans to recharge their devices,” Trainor said.

With all its different parts, the wireless deployment at Sports Authority Field at Mile High adds up to a favorable fan experience, one that clearly has the ability to keep getting better on an incremental basis. But like their Super Bowl team, Denver fans should be happy with what they have right now.

MORE PHOTOS BELOW

mihi7

Railing antenna enclosure. Some of these have both Wi-Fi and DAS.

mihi11

App promo on the scoreboard

mihi13

Panoramic view of the stadium and the city

mihi4

South stands have a horse and Wi-Fi antennas on the top of the scoreboard

mihi20

Cisco SportsVision in action on 6-panel display

mihi22

DAS antennas on end-of-row railing area

mihi24

Game on, phones out!

mihi25

Antennas covering the concourse area on second level

mihi26

More SportsVision and Wi-Fi deployment in the United Club

mihi14

Only accept on the scene reporting!

Verizon puts cellular antennas under seats to improve Levi’s Stadium DAS ahead of Super Bowl 50

New Verizon Wireless under-seat DAS antenna placement at Levi's Stadium. Photo: Verizon Wireless

New Verizon Wireless under-seat DAS antenna placement at Levi’s Stadium. Photo: Verizon Wireless

In a first for distributed antenna system (DAS) deployments, Verizon Wireless is using under-seat antenna deployments in Levi’s Stadium to increase capacity for Super Bowl 50, according to a Verizon executive interviewed Wednesday.

Brian Mecum, vice president, network for Verizon Wireless, said in a phone interview that large amounts of wireless traffic at last year’s Super Bowl spurred Verizon to help fund a full replacement of the Levi’s Stadium DAS, which was only a year old, to make sure that this year’s big game would be able to handle the expected usage growth. Mecum said the carrier expects to see as much as 6 terabytes or more of cellular traffic during the Feb. 7 game between the Denver Broncos and the Carolina Panthers at the San Francisco 49ers’ home field in Santa Clara, Calif.

To increase capacity in the lower seating bowls, Mecum said that Verizon basically invented its own under-seat DAS antenna system and paid for its deployment, adding about “30 percent more capacity” through the under-seat antennas alone. For its Wi-Fi network, Levi’s Stadium already uses under-seat access points, as do a growing number of large outdoor arenas. While Verizon’s deployment of under-seat antennas for cellular DAS is a first we’ve heard of, it achieves the same goals as under-seat Wi-Fi, basically just getting signals closer to users in a place where there isn’t any overhead or side structures to attach antennas to.

Close-up of under-seat DAS antenna system. Photo: Verizon Wireless

Close-up of under-seat DAS antenna system. Photo: Verizon Wireless

“To get a quality signal you have to get [the antenna] closer to the device,” and the only way to do that in the 100-level sections of Levi’s Stadium is to go under the seats, Mecum said. Though stadium DAS integrator DAS Group Professionals also upgraded most of its other cellular antennas this summer throughout Levi’s Stadium, the under-seat antennas are exclusive to Verizon Wireless, Mecum said. According to DGP vice president and COO Vince Gamick, just more than 50 under-seat DAS antennas were deployed, along with more than 700 other DAS antennas in the stadium and parking lots.

Low power to ease safety concerns

Though some fans might wonder about the white boxes under their seats, Mecum said the Verizon antenna deployment operates at “well below FCC standards” for power output, and should not be a cause for health concerns.

“They use about the same amount of power as a cell phone,” Mecum said. The antennas are also mounted to shoot their signals down to bounce off the concrete flooring, further reducing the power signal toward fans’ bodies.

According to Mecum, the antennas have already been doing a lot of work during the regular season — though he didn’t provide an exact number, Mecum did say that the 49ers’ home game against the rival Seattle Seahawks (which produced a lot of Wi-Fi traffic) also had a total of DAS traffic on the Verizon network that surpassed the 4.1 TB that Verizon recorded at last year’s Super Bowl at the University of Phoenix Stadium in Glendale, Ariz.

“We’re pretty excited” to see what the eventual Super Bowl DAS traffic number is, Mecum said. “We expect to see at least 1.5 times as much as we saw at the Super Bowl last year.”

San Franicso advertising kiosk where Verizon Wireless installed extra antennas for Super Bowl crowds. Photo: Screen shot from Verizon Wireless video

San Franicso advertising kiosk where Verizon Wireless installed extra antennas for Super Bowl crowds. Photo: Screen shot from Verizon Wireless video

Antennas on kiosks, and engineering for the Lombardi Trophy traffic

The stadium improvements are just part of a Bay area-wide infrastructure improvement blitz from Verizon, a fairly typical thing ahead of big events for almost all the cellular carriers. But some of the pre-game prep for Verizon has a unique flavor for San Francisco, especially the carrier’s decision to put small cells inside San Francisco’s stylish advertising kiosks that dot the downtown area.

With Super Bowl fan activities taking place from San Francisco’s Ferry Building down to the Moscone Convention Center, Mecum said the kiosks became a perfect place to add cellular capacity.

“They were basically empty [at the top] so we ran fiber to them and put in antennas,” Mecum said.

Verizon’s Super Bowl engineering effort also included extra antenna deployments around the areas where the Super Bowl hardware, the Lombardi Trophy and the Super Bowl winning teams’ rings, will be on display.

“The lines to see the trophy and the rings are so long we actually needed to engineer for them and adjust the antenna density [in the display area],” Mecum said. “So you should be able to connect anywhere and everywhere you go for the Super Bowl.”

Verizon: Denver fans used 2.87 TB of wireless data during AFC championship; AT&T also sets DAS traffic records at both Sunday games

Sports Authority Field at Mile High, during Jan. 3 game vs. San Diego. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

Sports Authority Field at Mile High, during Jan. 3 game vs. San Diego. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

During their team’s exciting 20-18 victory over the New England Patriots Sunday, Denver Broncos fans who are Verizon Wireless customers used 2.87 terabytes of wireless data, according to Verizon. That total includes 1.7 TB used on the Verizon-only Wi-Fi network at Sports Authority Field at Mile High, and another 1.17 TB of data on the Verizon LTE DAS network at the stadium.

Also on Sunday, at the Bank of America Stadium in Charlotte, N.C., where the Carolina Panthers defeated the Arizona Cardinals, Verizon Wireless customers there used 1.3 TB of data on the Verizon LTE DAS network, according to figures sent to us by Verizon.

According to AT&T, its wireless customers set DAS traffic records at both stadiums Sunday, with 819 GB used on AT&T’s DAS network at Sports Authority Field and 739 GB used by AT&T DAS customers at Bank of America Stadium. Both totals are the highest-ever marks seen by AT&T at the respective stadiums, according to AT&T; the Denver total Sunday was 34 percent higher than the total used in the team’s first playoff game this season against Pittsburgh, and was 52 percent more than the average data used during regular-season games. In Charlotte, the DAS traffic total Sunday was 16 percent higher than the number hit during the playoff game a week previous against Seattle, and 50 percent higher than the average regular-season game, according to AT&T.

On the AT&T Wi-Fi network at Bank of America Stadium, fans used 740 GB of data, which AT&T said is also the highest-ever mark for that network, 19 percent higher than the previous playoff game vs. Seattle and 36 percent more than the network saw for average regular season games. The Wi-Fi network at Bank of America Stadium will be replaced this offseason, with a new network built by AmpThink and Aruba.

In Denver, the network situation is somewhat unique since Verizon built the Wi-Fi network inside Sports Authority Field at Mile High, but so far only Verizon customers are allowed access to it. While we’ll describe the situation in more detail in a stadium-visit profile coming very soon, the word from stadium IT types is that the Wi-Fi is open to other carriers but none have yet signed on to allow their customers access to it. There are, however, separate DAS networks for AT&T, Sprint and T-Mobile inside the stadium as well, so at least from a DAS perspective most fans at Mile High are pretty well connected.

Stay tuned for more soon on the networks at Sports Authority Field at Mile High! In the meantime, some pictures from our Jan. 3 visit below.

mihi3

Count the antennas! See if you can spot the AT&T DAS antennas (slightly rounded) and the T-Mobile antenna (big square) among others in this overhang shot at Sports Authority Field at Mile High.

mihi4

Denver’s south end zone scoreboard is topped by a horse… and two Wi-Fi antennas

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

Your Levi’s Stadium technology primer: Everything you need to know about wireless technology at the site of Super Bowl 50!

Scoreboard promo for the Levi's Wi-Fi network, from 2014 season. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

Scoreboard promo for the Levi’s Wi-Fi network, from 2014 season. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

With Super Bowl 50 two weeks away there is going to be increased interest about whether or not Levi’s Stadium in Santa Clara, Calif., host site of the event, is “the most technologically advanced” athletic venue. Here at Mobile Sports Report, we have spent the better part of the last two years researching and reporting on Levi’s Stadium and all its technical components, attending multiple events and compiling all the known statistics we can find, to present as complete and as honest an assessment as possible, from a completely objective perspective, about the technology found at Levi’s Stadium.

So what’s our verdict? As we see it, there are three main features that set Levi’s Stadium aside from most others, and qualify it for consideration as one of the most technologically advanced large public venues: The stadium’s Wi-Fi network, its distributed antenna system, or DAS, and the integrated Levi’s Stadium app, which takes advantage of a large network of beacons to provide wayfinding and other location-based features. Though some of the components, like the Wi-Fi network, may not be the fastest or largest around, it’s our opinion that the sum of the parts puts Levi’s Stadium at or near the top of any well-connected stadium list; but the 2-year-old venue’s real test won’t come until Super Sunday, when we’ll all see if the networks, apps and personnel performance can live up to the stress of one of sport’s biggest events.

For anyone who wants to know the exhaustive details behind the technology, we’ve included in this story links to all of our Levi’s Stadium stories we think are pertinent, to help other writers or interested sports-tech types get a grip on what’s really going to be technologically available to the 72,000 or so fans who show up on Feb. 7 to watch the NFL’s 50th annual big game.

For starters, here is the first part of a feature we did at the start of the season about how Levi’s Stadium was getting ready for Super Bowl 50. Though we expect some more news next week about late additions, this article pretty much sums up the first-year performance and the tweaks the San Francisco 49ers made to their home-stadium’s wireless infrastructure. And here is the second part of the feature, which focuses more on the stadium’s excellent app, which we’ll talk more about later.

Fans take pictures at Levi's Stadium, opening day 2014 season.

Fans take pictures at Levi’s Stadium, opening day 2014 season.

Wi-Fi: It’s good, but is it the best?

The Wi-Fi in the stadium is pretty good, among the best out there anywhere, but probably not the biggest or fastest network in all the land. Though the Aruba-gear network was innovative for its heavy use of under-seat Wi-Fi APs and the 1,200 APs it had for its first year, other stadiums are meeting or beating those numbers, and under-seat deployments are now becoming quite trendy for venues that want fast, wide connectivity. With slightly more than 1,200 APs now, the Levi’s Stadium Wi-Fi network has seen some big-traffic days for Wi-Fi, including the stadium’s NFL regular-season opener and a WrestleMania event last year.

Among stadiums we’ve seen, AT&T Stadium in Arlington, Texas, has more Wi-Fi APs and is a bigger place (by about 30,000 in capacity for football) so it had more overall Wi-Fi traffic than Levi’s the past couple years. And Kyle Field’s new network down at Texas A&M is the fastest we’ve seen anywhere, and already has had a bigger Wi-Fi traffic day than Levi’s Stadium. And we haven’t yet visited Miami’s Sun Life Stadium but they get a lot of wireless traffic there too. So while Levi’s Stadium may be among the best, we’re not quite sure it is at the top of the list, at least when it comes to sheer Wi-Fi connectivity.

We might change our tune if the Super Bowl 50 crowd can top last year’s Super Bowl Wi-Fi traffic total of 6.23 terabytes, recorded at the University of Phoenix Stadium in Glendale, Ariz. But so far the biggest total recorded at Levi’s Stadium was 4.5 TB seen at the WrestleMania 31 event last March. From our unofficial observations, the “top 5” list of most single-day Wi-Fi events we know of are:

1) 6.23 TB — Super Bowl XLIX, University of Phoenix Stadium, Glendale, Ariz., Feb. 1, 2015
2) 5.7 TB — Alabama vs. Texas A&M, Kyle Field, College Station, Texas, Oct. 17, 2015
3) 4.93 TB — College Football Playoff championship game, AT&T Stadium, Arlington, Texas, Jan. 12, 2015
4) 4.9 TB — College Football Playoff championship game, University of Phoenix Stadium, Glendale, Ariz., Jan. 11, 2016
5) 4.5 TB — WrestleMania 31, Levi’s Stadium, Santa Clara, Calif., March 31, 2015

For what it’s worth, Super Bowl XLVIII in MetLife Stadium used only 3.2 TB of Wi-Fi, so it will be interesting to see what happens with the growth curve at SB50. In addition to total data “tonnage” there is also an interesting observation about how much data is used per fan, on average. When we get the stats back from Super Bowl Sunday it will be interesting to see if the smaller crowd at Levi’s Stadium will have used more data per connected person, a good reflection of both the carrying capacity of the network and the ease of connecting and staying connected to the Wi-Fi.

Replacing the entire DAS for better cellular connectivity

What is often confusing to non-tech types who try to write about stadium wireless is realizing that there are often two separate networks, Wi-Fi and cellular, operating in the same venue. While many fans actively seek out Wi-Fi, many game-day attendees either don’t bother or don’t know how to connect to Wi-Fi, and so just use their phones like they do anywhere else. To make sure they still have a strong signal, wireless carriers and venues often team up to deploy a distributed antenna system, or DAS, which is basically a bunch of small antennas located inside the venue that act just like a big cell tower, connecting phones to the nearest antenna.

Close-up of new DAS antennas (from mid-July, before the wires were connected)

Close-up of new DAS antennas (from mid-July, before the wires were connected)

At Levi’s Stadium, integrator DAS Group Professionals (DGP) built a “neutral host” DAS for the stadium, which means the team owns the infrastructure and rents out space to carriers so they can connect to customers inside the building. One of the more interesting twists this past offseason was that DGP ripped out and replaced the entire DAS network it built the year before, at the behest of its customers, the major cellular providers. Why? According to DGP, the cell providers — who paid for the upgrade — are expecting as much as 2.5 times more cellular data at this year’s Super Bowl compared to last year, huge numbers that they were afraid might overwhelm the system installed in 2014.

During a stadium tour this summer, MSR saw that the main Levi’s Stadium head end (where the telecom gear that connects the stadium to the outside networks lives) was being doubled in size, so by any stretch cell connectivity should be good if not great during the big game. DGP was also supposed to be increasing cell coverage outside the stadium in the parking lots, but so far we haven’t heard any reports if reception was better this year than last.

At big events like the Super Bowl, the big wireless carriers will spend like crazy to make sure there are no reports of “phones not working,” so the DAS upgrades have become somewhat par for the course. AT&T said that it spent $25 million on wireless infrastructure improvements in the greater Bay area, including expanding its DAS operations inside Levi’s Stadium to allow them to handle 150 percent more traffic. You can expect that Verizon was spending some similar dollars, so rest assured, if you are there your phone will more likely than not find a signal.

What will the app let you do?

The biggest question remaining about the technological underpinnings of Super Bowl 50 — at least as of Sunday night — is whether or not all the features from the regular-season Levi’s Stadium app will make it into the mix for the Super Bowl, especially the one that really sets Levi’s Stadium apart, the ability to order food to be delivered to any seat in the stadium.

Though we’ve been given a “head nod” that the service will be available for Super Sunday, we haven’t yet received any official notice of what’s going to be in the game-day app either from app provider VenueNext or the NFL. This season Niners fans at home games could not only order food and drinks for themselves, they could order and pay for food to be delivered to friends in the stadium, something we noted in our season preview of changes to the groundbreaking Levi’s Stadium app.

App showing ability to buy pricey parking ticket for your RV

App showing ability to buy pricey parking ticket for your RV

If there is some doubt whether the league and the stadium might not make food-delivery available for the Super Bowl, it might have to do with the fact that at one of last year’s “big events” at Levi’s Stadium, the NHL’s Stadium Series outdoor hockey game, the food-delivery service melted down in the face of a massive amount of orders and a too-low level of human staffing. But our guess is that eventually (maybe this week?) we will hear that the Super Bowl app will embrace all the features of the regular Levi’s Stadium game-day app, including in-seat delivery.

What many fans at the game may find even more useful is the app’s ability to provide wayfinding capabilities through a mapping feature that uses the 2,000+ Bluetooth beacons installed throughout the venue to provide live wayfinding, just like how Google Maps shows your car as a blue dot driving down the highway. With many attendees most likely visiting the stadium for the first time, having the ability to find your way around via your device may be the most welcome reason to download the app. Fans should also be able to watch in-stadium replays seconds after plays happen, and may also be able to watch Super Bowl broadcast commercials via their mobile device. Stay tuned for more “official” announcements of app capabilities as we hear them.

In case you haven’t heard enough, here are a few more links from our in-person visits to Levi’s Stadium for Niners home games during the 2014 season.

Niners’ home opener tops Super Bowl for Wi-Fi data traffic with 3.3 Terabytes (Sept. 16, 2014)

Levi’s Stadium ‘NiNerds’ get high-visibility wardrobe upgrade (Nov. 23, 2014)

Stadium Tech Report: Network finishes season strong at Niners’ Levi’s Stadium (Jan. 12, 2015)

T-Mobile buys naming rights to new MGM/AEG Las Vegas arena

Artist rendering of T-Mobile name across top of new Las Vegas Arena.

Artist rendering of T-Mobile name across top of new Las Vegas Arena.

The new 20,000-seat arena being built by MGM and AEG on the Las Vegas strip now has a title name — the T-Mobile Arena, thanks to a naming sponsorship bought by the wireless carrier for an undisclosed amount.

Set to open in April, the new multi-purpose arena will be draped in the familiar magenta hues of T-Mobile branding and also offer “unique benefits” for T-Mobile customers who attend events there, including unspecified “VIP” fast-track entrances and early notification of ticket sales. Alan Snel of the Las Vegas Review-Journal estimates that the rights fees are probably in the near-$6 million per year range, with a deal length of 10 years.

What’s unclear is whether or not the “un-carrier” will actually have any hand in the wireless deployments inside the venue, since the deal for backbone bandwidth and Wi-Fi for the Vegas arena was given to Cox Communications back in December. Among the stadiums we’ve reviewed over the past few years, T-Mobile has trailed other cellular providers in joining stadium DAS networks but it’s a good bet that T-Mobile will act differently in the first sports venue to carry its name.

On our quick trip to Vegas for CES we didn’t see the stadium, but that was because we were looking in the wrong place — it’s not being built next to the MGM but across the strip and somewhat behind New York New York, according to pictures of the building on the new arena website. More as we learn more…

T-Mobile Arena under construction in Las Vegas. All images: T-Mobile Arena.

T-Mobile Arena under construction in Las Vegas. All images: T-Mobile Arena.

Texas A&M’s Kyle Field: A network built for speed

Full house at Kyle Field. All Photos: Paul Kapustka, MSR (click on any photo for a larger image)

Full house at Kyle Field. All Photos: Paul Kapustka, MSR (click on any photo for a larger image)

Is there a combined stadium Wi-Fi and DAS deployment that is as fast as the one found at Texas A&M’s Kyle Field? If so, we haven’t seen or heard of it.

In fact, after reviewing loads of live network-performance data of Kyle Field’s new Wi-Fi and DAS in action, and after maxing out the top levels on our speed tests time after time during an informal walk-around on a game day, we’ve come to the conclusion that Kyle Field has itself a Spinal Tap of a wireless deployment. Meaning, that if other stadium networks stop at 10, this one goes to 11.

Movie references aside, quite simply, by the numbers Kyle Field’s wireless network performance is unequaled by any other large public venue’s we’ve tested in terms of raw speed and the ability to deliver bandwidth. With DAS and Wi-Fi speed measurements ranging between 40 Mbps and 60+ Mbps pretty much everywhere we roamed inside the 102,512-seat venue, it’s a safe bet to say that the school’s desire to “build the best network” in a stadium hit its goal as best as it could.

Editor’s note: This story is part of our most recent STADIUM TECH REPORT, the COLLEGE FOOTBALL ISSUE. The 40+ page report, which includes profiles of stadium deployments at Texas A&M, Kansas State, Ole Miss and Oklahoma, is available for FREE DOWNLOAD from our site. Get your copy today!

On one hand, the network’s top-line performance is not that much of a surprise, since as part of an overall Kyle Field renovation that has already cost an estimated $485 million, the optical-based Wi-Fi, DAS and IPTV deployment inside the Aggies’ football palace is probably among the most expensive and expansive in-venue networks ever built. According to Phillip Ray, Vice Chancellor for Business Affairs at The Texas A&M University System, the total cost of the optical-based Wi-Fi, DAS and IPTV network was “somewhere north of $20 million.”

Remote optical cabinet and Wi-Fi AP at Kyle Field.

Remote optical cabinet and Wi-Fi AP at Kyle Field.

And even though the nation’s biggest cellular carriers, AT&T and Verizon Wireless, paid nearly half the network’s cost – $10 million, according to Ray – with the dedication and work crews brought to the table by main suppliers IBM and Corning, and Wi-Fi gear vendor Aruba, you have components, expertise and budgetary freedom that perhaps only a small group of venue owners could hope to match.

But just throwing money and technology at a stadium doesn’t necessarily produce a great network. In a venue the size of the new Kyle Field there needs to be great care and innovative thinking behind antenna placement and tuning, and in that arena Texas A&M also had the guiding hand of AmpThink, a small firm with oversized smarts in Wi-Fi deployment, as evidenced by its impressive track record of helping wireless deployments at the biggest events including several recent Super Bowls.

The core decision to go with optical for the network’s guts, and a tactical decision to put a huge chunk of the Wi-Fi APs in under-seat deployments are just part of the strategy that produced a network that – in A&M fan parlance – can “BTHO” (Beat The Hell Out) of most challengers.

Since it’s almost impossible to directly compare stadiums and venue network performances due to all the possible variables, you’ll never hear us at Mobile Sports Report declare a “champion” when it comes to click-bait themes like “the most connected stadium ever.” Given its remote location some three hours south of Dallas in College Station, Texas, Kyle Field will almost certainly never face the ultimate “big game” pressures of a Super Bowl or a College Football Playoff championship, so the network may never know the stress such large, bucket-list gatherings can produce. And so far, there aren’t many ambitious fan-facing applications that use the network, like in-seat food delivery or wayfinding apps found in other stadiums.

But as part of the football-crazy SEC, and as the altar of pigskin worship for some of the most dedicated fans seen anywhere, Kyle Field is sure to see its share of sellout contests against SEC rivals that will push wireless usage to new heights, especially as more fans learn about and use the still-new system. Though total Wi-Fi usage at the Nov. 7 game we attended versus Auburn (a 26-10 Texas A&M loss) was “only” 2.94 terabytes – a total hampered by cold, windy and rainy conditions – an Oct. 17 game earlier in the season against Alabama saw 5.7 TB of Wi-Fi usage on the Kyle Field network, a number surpassed only by last year’s Super Bowl (with 6.2 TB of Wi-Fi use) in terms of total tonnage.

At the very least, the raw numbers of total attendees and the obvious strength of the still-new network is sure to guarantee that Kyle Field’s wireless deployment will be one of the most analyzed stadium networks for the foreseeable future.

Texas A&M student recording the halftime show.

Texas A&M student recording the halftime show.

What follows are some on-the-spot observations from our visit, which was aided by the guidance and hospitality of Corning project manager Sean Heffner, who played “tour guide” for part of the day, giving us behind-the-scenes access and views of the deployment that are unavailable to the general fan audience.

An off-campus DAS head end

This story starts not inside Kyle Field, but in a section of town just over three miles away from the stadium, on a muddy road that curves behind a funky nursery growing strange-looking plants. A gray metal box, like a big warehouse, is our destination, and the only clue as to what’s inside is the big antenna located right next to it. This structure is the Kyle Field DAS head end, where cellular carrier equipment connects to the fiber network that will bring signals to and from fans inside the stadium.

Why is the head end so far away? According to Corning’s Heffner there was no room for this huge space inside the stadium. But thanks to the use of optical fiber, the location is not a problem since signals traveling at the speed of light makes 3.3 miles an insignificant span.

It might be helpful to back up a bit if you haven’t heard the full story of the Kyle Field deployment, which we told last year when the job was halfway completed. Though the rebuilding of the stadium was started with copper-based networks as the original plan, a last-minute audible championed by Texas A&M chancellor John Sharp sent the school on a decidedly untraditional path, by building a stadium network with a single optical-based core for Wi-Fi, DAS and IPTV networks. The kicker? Not only would this network have huge capacity and be future-proof against growth, it would actually cost less than a comparable copper-based deployment. If it got built on time, that is.

Though the pitch for better performance, far more capacity, use of less space, and cheaper costs might sound a bit too good to believe, most of it is just the combination of the simple physics advantages of using fiber over copper, which are well known in the core telecom and large-enterprise networking worlds, applied to a stadium situation.

One of the many maxed-out speed tests we took at Texas A&M's Kyle Field.

One of the many maxed-out speed tests we took at Texas A&M’s Kyle Field.

Without going too deeply into the physics or technology, a simple explanation of the benefits stem from the fact that optical fiber can carry far more bandwidth than copper, at farther distances, using less power. Those advantages are why fiber is used extensively in core backbone networks, and has been creeping slowly closer to the user’s destination, through deployments like Verizon’s FiOS.

And that’s also the reason why Texas A&M could put its DAS head end out in a field where it’s easier to add to (no space constraints), because the speed of fiber makes distance somewhat irrelevant. Corning’s Heffner also said that the DAS can be managed remotely, so that staff doesn’t need to be physically present to monitor the equipment.

Of course, there was the small matter of digging trenches for optical fibers to get from the head end to the stadium, but again, for this project it is apparent that getting things done was more important than strictly worrying about costs. Beyond the cash that the carriers all put in, other vendors and construction partners all put in some extra efforts or resources – in part, probably because the value of positive publicity for being part of such an ambitious undertaking makes any extra costs easy to justify.

Keeping the best fans connected and happy

From the head end, the fiber winds its way past apartment buildings and a golf course to get to Kyle Field, the center of the local universe on football game days. Deep inside the bowels of the venue is where the fiber meets networking gear, in a room chilled to the temperature of firm ice cream. Here is where the human element that helps keep the network running spends its game days, wearing fleece and ski jackets no matter what the temperature is outside.

See the white dots? Those are under-seat Wi-Fi APs

See the white dots? Those are under-seat Wi-Fi APs

In addition to Corning, IBM and AmpThink employees, this room during our visit also had a representative from YinzCam in attendance, a rarity for a company that prides itself on being able to have its stadium and team apps run without local supervision. But with YinzCam recently named as a partner to IBM’s nascent stadium technology practice, it’s apparent that the Kyle Field network is more than just a great service for the fans in the seats – it’s also a proof of concept network that is being closely watched by all the entities that helped bring it together, who for many reasons want to be able to catch any issues before they become problems.

How big and how ambitious is the Kyle Field network? From the outset, Corning and IBM said the Wi-Fi network part was designed to support 100,000 connections at a speed of 2 Mbps, so that if everyone in the stadium decided to log on, they’d all have decent bandwidth. But so far, that upper level hasn’t been tested yet.

What happened through the first season was a “take rate” averaging in the 35,000-37,000 range, meaning that during a game day, roughly one-third of the fans in attendance used the Wi-Fi at some point. The average concurrent user peaks – the highest numbers of fans using the network at the same time – generally averaged in the mid-20,000 range, according to figures provided by Corning and AmpThink; so instead of 100,000 fans connecting at 2 Mbps, this season there was about a quarter of that number connecting at much higher data rates, if our ad hoc speed tests are any proof.

Our first test that Saturday [Nov. 7, 2015], just inside a lower-level service entryway, hit 41.35 Mbps for download and 18.67 on the upload, on a Verizon iPhone 6 Plus over the stadium’s DAS. And yes, that download speed was the slowest we’d record all day, either on the DAS or the Wi-Fi.

Inside the control room we spent some time with AmpThink CEO Bill Anderson, who could probably use up an entire football game talking about Wi-Fi network deployment strategies if he didn’t have a big network to watch. On this Saturday the top things we learned about Kyle Field is that Anderson and AmpThink are solid believers in under-seat AP placements for performance reasons; according to Anderson at Kyle Field, fully 669 of the stadium’s 1,300 APs can be found underneath seats. Anderson also is a stickler for “real” Wi-Fi usage measurements, like trying to weed out devices that may have autoconnected to the Wi-Fi network but not used it from the “unique user” totals – and to take bandwidth measurements at the network firewall, to truly see how much “live” bandwidth is coming and going.

On the road to College Station, Aggie pride is everywhere. Whoop!

On the road to College Station, Aggie pride is everywhere. Whoop!

AmpThink’s attention to detail includes deploying and configuring APs differently depending on which section they are located in – student sections, for example, are more densely packed with people than other sections so the APs need different tuning. Corning’s Heffner also said that the oDAS – the DAS just outside the stadium – got special attention due to the large numbers of tailgating fans, both before and during the games. At the Alabama game, Heffner said there were some 30,000 fans who remained outside the stadium during the contest, never coming inside but still wanting to participate in the scene.

AmpThink, Corning, IBM and others involved at Kyle Field all seem keen on finding out just how much bandwidth stadium fans will use if you give them unlimited access. The guess? According to Corning’s Heffner, the mantra of stadium networks these days seems to be: “If you provide more capacity, it gets consumed.”

The ‘real’ 12th man

After walking through a tunnel with a nearly full cable tray overhead (“It’d be even more loaded if we were using copper,” Heffner said) we went out into the stadium itself, which was just starting to fill. Though the overcast day and intermittment rain squalls might have kept other teams’ fans from showing up for a 5:30 p.m. local start time, that simply wasn’t the case at an A&M home game.

Some of the Wi-FI and DAS download measurements we took at Kyle Field.

Some of the Wi-FI and DAS download measurements we took at Kyle Field.

As someone who’s attended a countless number of football games, small and large – including a Super Bowl and last year’s inaugural College Football Playoff championship game – I can honestly say that the level of fan participation at Texas A&M is like nothing I’d seen before. The student section alone spans two decks on the stadium’s east side and takes up 40,000 seats, according to stadium officials – simply dwarfing anything I’d ever witnessed. (Out of an enrollment of 57,000+, having 40,000 students attend games is incredible.) And outside of small high school crowds I’d never seen an entire full stadium participate in all the school songs, the “yells” (do NOT call them “cheers” here) and the locked-arms back-and-forth “sawing” dance without any need for scoreboard instruction.

Part of the stadium renovation that closed the structure into a bowl was, according to school officials, designed to make Kyle Field even more intimidating than it already was, by increasing the sound levels possible. Unfortunately the night of our visit some early Auburn scores took some of the steam out of the crowd, and a driving, chilling rain that appeared just before halftime sent a good part of the crowd either home or into the concourses looking for warmth and shelter. (The next day, several columnists in the local paper admonished the fans who left early for their transgressions; how dare they depart a game whose outcome was still in doubt?)

But I’ll never forget the power of the synchronized “yells” of tens of thousands of fans during pregame, and the roar that surfaced when former Aggie QB Johnny Manziel made a surprise appearance on the field before kickoff. Seattle Seahawks fans may stake the pro claim to fan support, but if you want to determine the “real” 12th man experience you need to stop by Kyle Field and give your ears a taste of loud.

Controlling the TV with the app

If the students and alumni and other fans outside provide the vocal power, the money power that helped get the stadium rebuilt can be found in the new Kyle Field suites and premium seating areas, some of which are found on the venue’s west side, which was blown up last December and rebuilt in time for this past season.

Conduit reaching to an under-seat AP

Conduit reaching to an under-seat AP

Inside the All American Club – a behind-the-walls gathering area with catered food and bars that would not seem out of place in Levi’s Stadium or AT&T Stadium – we tested the Wi-Fi and got speeds of 63 Mbps down, 69 Mbps up; Verizon’s 4G LTE service on the DAS hit 48 Mbps/14.78 Mbps, while AT&T’s 4G LTE DAS checked in at 40 Mbps/22 Mbps.

In an actual suite where we were allowed to check out the IPTV displays, the speed tests got 67/67 for Wi-Fi and 57/12 for Verizon 4G LTE. So the well-heeled backers of A&M football shouldn’t have any problems when it comes to connectivity.

As for the IPTV controls, the new system from YinzCam solves one of the problems that’s plagued stadium suites since there’s been suites: What do you do with the TV remote? What YinzCam did for Texas A&M was link the TV controls to a Texas A&M “TV Remote” app; by simply punching in a numerical code that appears on the bottom of the screen in front of you, anyone with access to a suite or club area with TVs can change the channel to a long list of selections, including multiple live game-day views (stadium screen, broadcast view) as well as to other channels, like other games on the ESPN SEC network.

By having a static code number for each TV and another set of numbers that randomly scrambles over time, the system smartly builds security into the channel changing system, and prevents someone who had been in a suite previously from being able to change the channels after they leave. The whole remote-control process took less than a minute to learn, and we had fun wandering through the club-level areas our pass gave us access to, changing screens as we saw fit.

Our favorite places to watch the game at Kyle Field were the loge-level lounges, where you could first purchase food and beverages, including alcoholic ones, at an inside bar and then sit at an outside seat with a small-screen TV in front of you for information overload. The Wi-Fi in the southwest corner loge lounge checked in at 67.03/62.93, so it was no problem being connected via mobile device, either.

What comes next for the Kyle Field network?

Even though the rain had started coming down harder, we left the comfort and warmth of the club levels to wander around the stadium’s upper decks, including the student section, where we watched numerous fans taking pictures or videos of the band’s halftime performance. Clearly most everyone in Kyle Field had gotten the message and wasn’t afraid that they won’t connect if they use their mobile device at the game, even among 102,000 of their closest friends.

Antennas on flag poles atop seating

Antennas on flag poles atop seating

The question now for Kyle Field is what does it do next with its network? The most obvious place for innovation or new features is with a stadium-centric app, one that could provide services like a wayfinding map. Maybe it was our round-the-stadium wandering that produced confusion finding our way around, but any building that seats 102,000 plus could use an interactive map. It might also be interesting to tie a map to concessions – the night we visited, there were long lines at the few hot chocolate stands due to the cold weather; in such situations you could conceivably use the network to find out where hot chocolate stands were running low, maybe open new ones and alert fans through the app.

We’re guessing parking and ticketing functions might also be tied to the app in the future, but for now we’ll have to wait and see what happens. One thing in Kyle Field’s favor for the future: thanks to the capacity of the optical network buildout, the stadium already has thousands of spare fiber connections that aren’t currently being used. That means when it’s time to upgrade or add more DAS antennas, Wi-Fi APs or whatever comes next, Kyle Field is already wired to handle it.

For the Nov. 7 game at Kyle Field, the final numbers included 37,121 unique users of the Wi-Fi network, and a peak concurrent user number of 23,101 taken near the end of the 3rd quarter. The total traffic used on the Wi-Fi network that night was 2.94 TB, perhaps low or average for Kyle Field these days but it’s helpful to remember that just three years ago that was right around the total Wi-Fi data used at a Super Bowl.

Until the next IBM/Corning network gets built in Atlanta (at the Falcons’ new Mercedes-Benz Stadium, slated to open in 2017), the Kyle Field network will no doubt be the center of much stadium-technology market attention, especially if they ever do manage to get 100,000 fans to use the Wi-Fi all at once. While A&M’s on-the-field fortunes in the competitive SEC are a yearly question, the performance of the network in the Aggies’ stadium isn’t; right now it would certainly be one of the top four seeds, if not No. 1, if there was such a thing as a college stadium network playoff.

What we’re looking forward to is more data and more reports from a stadium with a network that can provide “that extra push over the edge” when fans want to turn their connectivity dial past 10. Remember, this one goes to 11. It’s one more.

(More photos below! And don’t forget to download your copy of the STADIUM TECH REPORT for more!)

kf7
Panoramic view of Kyle Field before the 102,000 fans fill the seats.

kf2
Some things at Kyle Field operate at ‘traditional’ speeds.

kf1

Outside the south gate before the game begins.

kf3

Overhang antenna in the middle section of the stadium.