Colorado passes on full-stadium Wi-Fi or DAS for Folsom Field

View of the west stands at Folsom Field, home of the University of Colorado football team. Credit all photos: Paul Kapustka, MSR (click on any photo for a larger image)

View of the west stands at Folsom Field, home of the University of Colorado football team. Credit all photos: Paul Kapustka, MSR (click on any photo for a larger image)

With a No. 9 ranking in all the major polls, the University of Colorado football team is experiencing a resurgence this season, which may lead to a rare CU sellout for this weekend’s final home game against Pac-12 rival Utah.

While the Buffs’ on-field performance in 2016 may have ended years of fan frustration, the 50,000+ fans expected to be in attendance at Folsom Field this Saturday may still experience another form of frustration, mainly in trying to get their mobile devices to connect to the Internet. According to school officials, there is no full-stadium, fan-facing Wi-Fi or cellular distributed antenna system (DAS) network in Folsom Field, and no plans to bring either to the venue anytime soon.

Instead, most fans at the on-campus stadium will rely on one of two nearby macro sites, one each from top wireless carriers AT&T and Verizon Wireless. According to Jeff Lipton, director of real estate for CU, the school decided to pass on bringing Wi-Fi or DAS to Folsom for a number of reasons, including the cost of the systems, the infrequent use of the facility, and potential loss of network control to prospective suppliers.

Hard to justify cost of connectivity for lightly used venue

“Nationally, stadiums are challenging” when it comes to cost justifications for installing wireless networks, Lipton said in a recent phone interview. While Lipton claimed that CU “hasn’t been sleeping on this,” saying the school has been reviewing wireless stadium options for several years, he added that CU had concluded that a cellular DAS wasn’t a good fit for Folsom, which has been the home of CU football since 1924.

Error message shown while trying to connect to ESPN's website at Folsom Field on Nov. 19.

Error message shown while trying to connect to ESPN’s website at Folsom Field on Nov. 19.

One of the main problems is, Lipton said, justifying the cost to bring connectivity to a venue that is only used a handful of times a year. In 2016, Colorado had six home games on its football schedule, the main use of 53,613-seat Folsom Field. This past summer there were two concert dates with the Dead and Company band that filled the stadium, and the stadium is also used as the finish line for the annual Bolder Boulder 10k on Memorial Day.

Though the crowds that do come to Folsom would no doubt enjoy better connectivity, right now it’s not in the cards, Lipton said, especially from a cellular DAS perspective.

“We looked at DAS for the main part of the stadium but determined it was not cost effective, and the vendors wanted a connectivity exclusive” for the rest of the campus, a deal Lipton said CU definitely did not want to agree to.

“We like to control our own [wireless] destiny on campus — we’re not going to give that up in a deal to get DAS,” Lipton said. And, Lipton said that “I’m not sure that [technically] in the long term, DAS is the solution” for stadium networks.

The right way to Wi-Fi

What’s more interesting to CU is finding some way or waiting for new technology to emerge to make owning and operating a Wi-Fi network inside Folsom something that makes sense. When it comes to Wi-Fi, Lipton said that CU has been aggressively installing it in many of the 12 billion square feet of building space it manages at the Boulder, Colo., campus.

Around Folsom, there is free public Wi-Fi available at the new Champions Center (an indoor practice field and offices building located adjacent to the east side of Folsom Field) as well as in the attached parking structure. There is also some free Wi-Fi available for suites and club spaces in the newer structures on the east side of Folsom Field, but nothing for the balance of bowl seating in the stadium.

Folsom's east side structure, which does have some Wi-Fi inside suites and club areas.

Folsom’s east side structure, which does have some Wi-Fi inside suites and club areas.

“We have pretty ubiquitous Wi-Fi throughout campus, and we installed it and run it,” said Lipton. In terms of bringing Wi-Fi into the Folsom Field bowl — as well as to the stands at the Coors Event Center, the school’s 11,00-seat basketball facility — that idea is still being studied by an internal working group, Lipton said.

“We recognize that long term, there are some real revenue opportunities [around Wi-Fi networks] that could pay for this later on,” Lipton said. “But it’s not there yet.”

Any Wi-Fi network that does end up getting built inside Folsom would also have to surmount the non-trivial challenge of bringing wireless networks to a facility with parts that are nearly 100 years old. Part of the bowl also sits in the ground, bringing another degree of difficulty to the idea of getting cables underneath the stands (for possible under-seat or railing-based antenna options). But for newer parts of the stadium, including the north end zone structures and the new east side, bringing connectivity to the stands outside wouldn’t be as difficult.

At the 11,064-capacity Coors Event Center, Lipton said there is some CU Wi-Fi inside the building, but it was not designed for full-stadium crowd access down into the seating bowl.

Unable to send texts, or get Internet access

Though Lipton claims that the two Folsom macro sites — one atop the roof on the stadium’s west side building and another on a building across the street — are working “much better” this season, an MSR visit to Folsom for the Nov. 19 home game against Washington State saw almost zero connectivity, on both the cellular and Wi-Fi front.

A look at the newer north and northeast structures at Folsom Field from the east stands.

A look at the newer north and northeast structures at Folsom Field from the east stands.

Though our tests were sporadic, with only one phone in one part of the stadium, our not being able to send a text message with a photo of the stunning Colorado Rockies backdrop was probably something many others experienced inside Folsom last week, where 48,658 fans saw CU beat WSU 38-24. On the Verizon network, we were almost always looking at a “1x” number for connectivity, which pretty much guaranteed no signal all day long.

Even inside the east building’s 5th-floor club area, where we detected the CU campus Wi-Fi network, our phone couldn’t connect, briefly showing a link but then dropping it as soon as we tried to do anything. There was no visible promotion of the CU Wi-Fi, or any instruction about whether fans should use one of two visible SSIDs, one with a “guest” label and one without. Back out in the stands, we tried to get to the ESPN website to see other college scores, but again our device failed to connect.

While Lipton admitted that “traffic on [football] game days can overwhelm” the macro sites, he still thinks any advanced connectivity has to make fiscal sense. As the one who says he signs contracts for such deployments at CU, Lipton said “there’s an art to every deal.” For Folsom Field connectivity, however, that deal hasn’t yet been done.

Verizon: Series fans used 5.3 TB of cellular data at Wrigley Field, 15.4 TB at Progressive Field

Cubs victory celebration in Chicago's Grant Park. Credit: KIICHIRO SATO/AP from Cubs.com.

Cubs victory celebration in Chicago’s Grant Park. Credit: KIICHIRO SATO/AP from Cubs.com.

We now have the Verizon Wireless stats for cellular usage by fans at Chicago’s Wrigley Field during the recent World Series, and according to Verizon fans on its cellular network in the stadium used a total of 5.3 terabytes of data for the three games at the Friendly Confines. Verizon also said that for the four games at the Cleveland Indians’ Progressive Field, Verizon saw a combined 15.4 TB of wireless data on its cellular network.

Combined with AT&T’s previously announced total of 3 TB on its network over the three games gives us a total of 8.3 TB of DAS usage for the top two wireless carriers for the three Chicago Cubs home games during the Fall Classic. For the Cleveland games, the 15.4 TB on Verizon and 2 TB on AT&T comes to 14.4 TB combined.

Outside the games, Verizon added that Chicagoans used another 6.5 TB of data along the World Series celebration parade route last Friday. According to a published report, AT&T’s network in Chicago saw 10.5 TB of traffic during the parade; we haven’t been able to confirm that number with AT&T directly but with 5 million fans estimated taking place in the celebration, 17 TB of combined data usage sounds about right.

AT&T: College and pro football crowds have used 37 TB of cellular data this year, almost double from 2015

mihi24At about the halfway point for the football season, AT&T said it has seen 37 terabytes of traffic on its stadium wireless networks so far, for both college and professional games combined.

If the total tonnage doesn’t mean that much to you, consider a related data point that should give stadium network managers pause: According to AT&T, so far this year it has seen 85 percent more data used at college games and 90 percent more data used at pro games as compared to last year. If you’re wondering why stadiums are already upgrading networks they installed just a few years ago, the data growth is likely the biggest cause. And there’s still no end in sight to the upward usage curve. Even though these numbers represent only AT&T customer traffic on AT&T stadium DAS installations, the totals and the growth are still pretty staggering.

AT&T also enclosed some weekly highlights for pro and college stadiums, enclosed below.

Screen Shot 2016-11-09 at 10.40.41 AM

Screen Shot 2016-11-09 at 10.40.51 AM

Cubs see 3 TB of cellular data on AT&T network during World Series games at Wrigley

Screen Shot 2016-11-03 at 12.08.20 PMAs expected, the just-concluded 2016 World Series between the Chicago Cubs and the Cleveland Indians saw big totals for in-stadium wireless data use, with AT&T customers using almost 5 terabytes of total data, with 3 TB recorded during the three games at Chicago’s Wrigley Field and another 2 TB in the four games at Cleveland’s Progressive Field.

According to statistics of traffic on AT&T’s DAS (distributed antenna system) cellular networks in and around the stadiums, the biggest single-game total was a mark of 1+ TB at Wrigley Field for Game 3. While there was less overall traffic on the AT&T DAS at Progressive Field, some of the fan data in Cleveland most likely was carried by the stadium’s Wi-Fi network; Wrigley’s new Wi-Fi network is still under construction and was not available to fans for this season.

Thanks to AT&T, here is a game-by-game breakdown of the DAS traffic totals. Remember, this is for AT&T customers only on AT&T networks. Any other carriers out there who want to add their totals to this mix, you know where to find us!

Screen Shot 2016-11-03 at 12.17.15 PM

The MSR Interview: San Francisco Giants CIO Bill Schlough

AT&T Park CIO Bill Schlough shows off his World Series bling.

AT&T Park CIO Bill Schlough shows off his World Series bling.

Who better to talk about stadium Wi-Fi than the guy who was there when it all started? Our guest for our first MSR Interview (part of our Stadium Tech Report Podcast series) is San Francisco Giants senior vice president and chief information officer Bill Schlough, who goes old-school talking about stadium Wi-Fi back in 2004… and brings it to the current day with stats from the most recent season at AT&T Park. Plus, his thoughts on game-day apps and why great connectivity is the real winner. Listen in now!

Hear Bill talk about:

— New Wi-Fi records set… during the Warriors’ playoff run

— Why going under-seat with Wi-Fi was a necessary thing to do

— How the Giants are experimenting with virtual reality

— Why he thinks great connectivity matters most (even more than stadium-app features)

Some story links that offer some history about AT&T Park’s networks from MSR:

S.F. Giants add more Wi-Fi, ‘virtual reality experience’ to AT&T Park for 2016 season

SF Giants fans used 78.2 TB of Wi-Fi data at AT&T Park during 2015 season

Stadium Tech Report: World Series set new wireless records at AT&T Park

Stadium Tech Report: San Francisco’s AT&T Park lives up to its wireless reputation

Giants: NLCS stadium Wi-Fi usage at AT&T Park quadrupled since 2012

SUBSCRIBE TO THE PODCAST:

Here is the link to the podcast on iTunes!

Stadium Tech Report: Carolina Panthers take ownership of DAS, Wi-Fi at Bank of America Stadium

James Hammond, director  of IT for the Panthers, poses next to an under-seat Wi-Fi AP. Credit all photos: Carolina Panthers

James Hammond, director of IT for the Panthers, poses next to an under-seat Wi-Fi AP. Credit all photos: Carolina Panthers

“The fan is the most valuable member of our team,” Jerry Richardson, owner of the Carolina Panthers, is fond of saying.

And it’s become the virtual mission statement for the Charlotte, N.C.-based National Football League franchise. So even though its home field, the Bank of America Stadium, was built relatively recently (1996), technology has come a long way in two decades. And as the Panthers began a four-phase renovation in 2014, they did it with fans’ MVP status in mind, according to James Hammond, director of IT for the Panthers. “It was time for some changes,” he said.

While Carolina was among the first NFL stadiums to install fan-facing Wi-Fi and enhanced cellular networks, the previous DAS and Wi-Fi systems weren’t keeping up with demand and that was starting to adversely impact the Panthers fan experience, Hammond said.

“We chose to perform a rip-and-replace on both DAS and Wi-Fi and take ownership in-house,” Hammond explained. Because the Panthers own and operate BofA Stadium, making those moves was a lot easier than if they were tenants.

Time for an upgrade

Editor’s note: This profile is part of our latest STADIUM TECHNOLOGY REPORT, which includes more stadium profiles as well as looks at Wi-Fi at the Mall of America, and analytics software being used by the Cleveland Browns. DOWNLOAD YOUR FREE COPY of the report today!

The first “fan-centric improvements,” as Hammond called them, came in 2014 in the form of escalators, video big boards and a distributed audio system. As part of the second phase of upgrades, the Panthers then used the 2015 offseason to renovate the club-level suites and tore out the old DAS system while they were at it. And after a careful evaluation of different DAS solutions, they shortlisted two vendors: CommScope and Corning.

CommScope ultimately got the nod; the Panthers then had to decide between the vendor’s ION-B and ION-U DAS systems. “We went with the ION-U, which was quite new and cutting edge at that point, since it had NEMA-rated remotes,” Hammond said. Other systems lacked that kind of weatherproofing and would require additional enclosures – and expense.

CommScope's ION-U powers the new DAS at Bank of America Stadium.

CommScope’s ION-U powers the new DAS at Bank of America Stadium.

“We started over with all new fiber and coax. We did the decommissioning and construction in 90 days, which was pretty quick for a ground-up project,” he said. Beam Wireless Inc. did the design, integration and optimization and is handling the ongoing maintenance of the DAS system; Optical Telecom installed the DAS gear. BofA Stadium now has 256 DAS remotes and more than 600 DAS indoor and outdoor antennas.

AT&T, Verizon and Sprint are the participating DAS carriers; T-Mobile is weighing whether to join the mix during the 2017 off-season.

The Panthers have divided BofA Stadium into 48 DAS zones: 16 zones for the upper bowl, 16 in the lower bowl, and another 16 for concourses, suites, clubs, and offices. Not all zones are used exclusively; carriers choose simulcast patterns that place multiple zones into sectors, and can change them as capacity requirements dictate, Hammond told Mobile Sports Report.

“With some minor design changes to the interior areas, we can accommodate nearly 70 zones,” he explained. “At present the most sectors in use by a carrier is 32. This means the carrier simulcasts across a mix of our 48 zones in order to match them up to 32 carrier sectors.”

Once the new DAS was built and the first couple of events were analyzed, carriers began asking for more frequencies and additional DAS sectors to continue meeting ever-growing demand. In response to the new carrier requests, the first round of DAS upgrades were implemented in the spring of 2016, Hammond said. During the 2015 season, DAS bandwidth was running around 2 GB during games. Hammond said, “With these latest DAS upgrades, we expect the bandwidth numbers to be even higher.”

A DAS remote in a NEMA-rated enclosure.

A DAS remote in a NEMA-rated enclosure.

The impact of the new DAS system was felt immediately upon its debut in July 2015. “It was a much better experience for fans who noticed the improved cellular experience,” Hammond said. Another unexpected benefit: The upgraded DAS helped mitigate bottlenecks with the old Wi-Fi system, which Hammond characterized as “under-designed.”

Going under seat for Wi-Fi upgrade

Unfortunately, there wasn’t time to address any Wi-Fi upgrades before the 2015 football season began, but the Panthers issued an RFP for new Wi-Fi in August 2015 in preparation for Phase 3 renovations that would also include security upgrades and renovations to the upper concourse.

Interested vendors needed to ensure high bandwidth rates as well as high take-rates that allowed three different ISPs (Time Warner Cable/Charter, Level 3 Communications and Windstream) to deliver in excess of 10 GB, though Hammond said they’re starting at a 7-GB threshold.

The Wi-Fi award went to Aruba, now HP Enterprise, in December 2015, and construction began in January 2016 after the last postseason game, when the Panthers beat the Arizona Cardinals to win the NFC championship and a trip to Super Bowl 50.

Similar to Levi’s Stadium and the Dodgers Stadium, the Panthers chose underseat AP enclosures; BofA Stadium sports 770 AP enclosures in the upper and lower bowls out of a total of 1,225 APs, all to ensure maximum coverage and minimal dead spots. The Panthers selected AmpThink to do the Wi-Fi integration and construction; the turnkey contractor also designed and fabricated a custom enclosure for the APs.

Indoor access point inside the stadium.

Indoor access point inside the stadium.

One other innovation in the Panthers’ Wi-Fi installation is that the underseat enclosure is mounted to the riser — the vertical part of the step — but looks like it’s on the tread, the horizontal part, which is intended as a waterproofing measure. “The riser is easier to seal and isn’t affected by pressure washers, which you’re doing constantly with an outdoor stadium,” Hammond said. “And by running pipe through the riser, you don’t have gravity working against you,” which helps keep out water, he explained.

Panthers fans access the stadium Wi-Fi through a portal page after accepting the team’s terms and conditions. From there, they are whitelisted and can automatically join the Wi-Fi network for the rest of the season. Hammond said a fan’s email is requested but not required by the portal page, and there’s a small incentive offered to encourage fans’ email subscriptions.

The new Wi-Fi system got a workout with a soccer game at BofA Stadium at the end of July 2016, then with a Panthers’ Fan Fest the following week. “All the indicators were good, and fan feedback about the system was excellent,” Hammond said. But he cautioned that the two events were not “full bowl” events with smaller attendance numbers (~50,000) than a regular season football game (75,000+). “We will continue to optimize and tune settings as we learn more during events with higher attendance,” Hammond said.

Total budget so far for the technology upgrades totals about $16 million; the DAS build-out was just under $10 million; Wi-Fi was a little more than $6 million, which included additional wired infrastructure, according to the team.

Beacons coming next

And the Panthers aren’t done making technology improvements to their stadium. Phase 4 looks to add Bluetooth beacons and do some refinement of the Panthers app. “My goal during the upcoming season is to look at options for location-aware services,” Hammond said. Some APs have beacons built in; other may need to be added to get the granularity the Panthers want for location awareness.

Hammond also wants to give fans more things to do with the Panthers app and also optimize it for push notifications, even with something as basic and useful as restroom and concessions location information. “As we learn more about fans individually, we can direct them to things of particular interest to them,” he added.

“So far, we are very pleased with the performance of the Wi-Fi and DAS systems,” Hammond said, noting the Panthers will continue to tune frequencies, add zones and increase bandwidth where needed. It’s the sort of attention that smart sporting franchises pay to their most valued team members.

car1