Stadium Tech Report: Carolina Panthers take ownership of DAS, Wi-Fi at Bank of America Stadium

James Hammond, director  of IT for the Panthers, poses next to an under-seat Wi-Fi AP. Credit all photos: Carolina Panthers

James Hammond, director of IT for the Panthers, poses next to an under-seat Wi-Fi AP. Credit all photos: Carolina Panthers

“The fan is the most valuable member of our team,” Jerry Richardson, owner of the Carolina Panthers, is fond of saying.

And it’s become the virtual mission statement for the Charlotte, N.C.-based National Football League franchise. So even though its home field, the Bank of America Stadium, was built relatively recently (1996), technology has come a long way in two decades. And as the Panthers began a four-phase renovation in 2014, they did it with fans’ MVP status in mind, according to James Hammond, director of IT for the Panthers. “It was time for some changes,” he said.

While Carolina was among the first NFL stadiums to install fan-facing Wi-Fi and enhanced cellular networks, the previous DAS and Wi-Fi systems weren’t keeping up with demand and that was starting to adversely impact the Panthers fan experience, Hammond said.

“We chose to perform a rip-and-replace on both DAS and Wi-Fi and take ownership in-house,” Hammond explained. Because the Panthers own and operate BofA Stadium, making those moves was a lot easier than if they were tenants.

Time for an upgrade

Editor’s note: This profile is part of our latest STADIUM TECHNOLOGY REPORT, which includes more stadium profiles as well as looks at Wi-Fi at the Mall of America, and analytics software being used by the Cleveland Browns. DOWNLOAD YOUR FREE COPY of the report today!

The first “fan-centric improvements,” as Hammond called them, came in 2014 in the form of escalators, video big boards and a distributed audio system. As part of the second phase of upgrades, the Panthers then used the 2015 offseason to renovate the club-level suites and tore out the old DAS system while they were at it. And after a careful evaluation of different DAS solutions, they shortlisted two vendors: CommScope and Corning.

CommScope ultimately got the nod; the Panthers then had to decide between the vendor’s ION-B and ION-U DAS systems. “We went with the ION-U, which was quite new and cutting edge at that point, since it had NEMA-rated remotes,” Hammond said. Other systems lacked that kind of weatherproofing and would require additional enclosures – and expense.

CommScope's ION-U powers the new DAS at Bank of America Stadium.

CommScope’s ION-U powers the new DAS at Bank of America Stadium.

“We started over with all new fiber and coax. We did the decommissioning and construction in 90 days, which was pretty quick for a ground-up project,” he said. Beam Wireless Inc. did the design, integration and optimization and is handling the ongoing maintenance of the DAS system; Optical Telecom installed the DAS gear. BofA Stadium now has 256 DAS remotes and more than 600 DAS indoor and outdoor antennas.

AT&T, Verizon and Sprint are the participating DAS carriers; T-Mobile is weighing whether to join the mix during the 2017 off-season.

The Panthers have divided BofA Stadium into 48 DAS zones: 16 zones for the upper bowl, 16 in the lower bowl, and another 16 for concourses, suites, clubs, and offices. Not all zones are used exclusively; carriers choose simulcast patterns that place multiple zones into sectors, and can change them as capacity requirements dictate, Hammond told Mobile Sports Report.

“With some minor design changes to the interior areas, we can accommodate nearly 70 zones,” he explained. “At present the most sectors in use by a carrier is 32. This means the carrier simulcasts across a mix of our 48 zones in order to match them up to 32 carrier sectors.”

Once the new DAS was built and the first couple of events were analyzed, carriers began asking for more frequencies and additional DAS sectors to continue meeting ever-growing demand. In response to the new carrier requests, the first round of DAS upgrades were implemented in the spring of 2016, Hammond said. During the 2015 season, DAS bandwidth was running around 2 GB during games. Hammond said, “With these latest DAS upgrades, we expect the bandwidth numbers to be even higher.”

A DAS remote in a NEMA-rated enclosure.

A DAS remote in a NEMA-rated enclosure.

The impact of the new DAS system was felt immediately upon its debut in July 2015. “It was a much better experience for fans who noticed the improved cellular experience,” Hammond said. Another unexpected benefit: The upgraded DAS helped mitigate bottlenecks with the old Wi-Fi system, which Hammond characterized as “under-designed.”

Going under seat for Wi-Fi upgrade

Unfortunately, there wasn’t time to address any Wi-Fi upgrades before the 2015 football season began, but the Panthers issued an RFP for new Wi-Fi in August 2015 in preparation for Phase 3 renovations that would also include security upgrades and renovations to the upper concourse.

Interested vendors needed to ensure high bandwidth rates as well as high take-rates that allowed three different ISPs (Time Warner Cable/Charter, Level 3 Communications and Windstream) to deliver in excess of 10 GB, though Hammond said they’re starting at a 7-GB threshold.

The Wi-Fi award went to Aruba, now HP Enterprise, in December 2015, and construction began in January 2016 after the last postseason game, when the Panthers beat the Arizona Cardinals to win the NFC championship and a trip to Super Bowl 50.

Similar to Levi’s Stadium and the Dodgers Stadium, the Panthers chose underseat AP enclosures; BofA Stadium sports 770 AP enclosures in the upper and lower bowls out of a total of 1,225 APs, all to ensure maximum coverage and minimal dead spots. The Panthers selected AmpThink to do the Wi-Fi integration and construction; the turnkey contractor also designed and fabricated a custom enclosure for the APs.

Indoor access point inside the stadium.

Indoor access point inside the stadium.

One other innovation in the Panthers’ Wi-Fi installation is that the underseat enclosure is mounted to the riser — the vertical part of the step — but looks like it’s on the tread, the horizontal part, which is intended as a waterproofing measure. “The riser is easier to seal and isn’t affected by pressure washers, which you’re doing constantly with an outdoor stadium,” Hammond said. “And by running pipe through the riser, you don’t have gravity working against you,” which helps keep out water, he explained.

Panthers fans access the stadium Wi-Fi through a portal page after accepting the team’s terms and conditions. From there, they are whitelisted and can automatically join the Wi-Fi network for the rest of the season. Hammond said a fan’s email is requested but not required by the portal page, and there’s a small incentive offered to encourage fans’ email subscriptions.

The new Wi-Fi system got a workout with a soccer game at BofA Stadium at the end of July 2016, then with a Panthers’ Fan Fest the following week. “All the indicators were good, and fan feedback about the system was excellent,” Hammond said. But he cautioned that the two events were not “full bowl” events with smaller attendance numbers (~50,000) than a regular season football game (75,000+). “We will continue to optimize and tune settings as we learn more during events with higher attendance,” Hammond said.

Total budget so far for the technology upgrades totals about $16 million; the DAS build-out was just under $10 million; Wi-Fi was a little more than $6 million, which included additional wired infrastructure, according to the team.

Beacons coming next

And the Panthers aren’t done making technology improvements to their stadium. Phase 4 looks to add Bluetooth beacons and do some refinement of the Panthers app. “My goal during the upcoming season is to look at options for location-aware services,” Hammond said. Some APs have beacons built in; other may need to be added to get the granularity the Panthers want for location awareness.

Hammond also wants to give fans more things to do with the Panthers app and also optimize it for push notifications, even with something as basic and useful as restroom and concessions location information. “As we learn more about fans individually, we can direct them to things of particular interest to them,” he added.

“So far, we are very pleased with the performance of the Wi-Fi and DAS systems,” Hammond said, noting the Panthers will continue to tune frequencies, add zones and increase bandwidth where needed. It’s the sort of attention that smart sporting franchises pay to their most valued team members.

car1

Spiking the Surface: Why Microsoft’s NFL deal got thrown for a loss

Bill Belichick’s very thorough takedown of the Microsoft Surface slapped a fried egg on the face of Microsoft’s $400 million deal with the NFL to use the tablets on game-day sidelines. In the latest STADIUM TECH REPORT PODCAST, co-hosts Phil Harvey and Paul Kapustka explore why the deal was flawed from the beginning, and whether or not wireless technology can be counted on to perform in NFL sideline environments.

SUBSCRIBE TO THE PODCAST:

Here is the link to the podcast on iTunes!

Verizon upgrading Wi-Fi at Lions’ Ford Field

Screen Shot 2016-10-21 at 10.52.24 AMVerizon Wireless is upgrading the Wi-Fi network it originally installed at the Detroit Lions’ Ford Field, but is keeping specifics about its upgrade to itself.

In a widely quoted press release Verizon said it is adding capacity to the Wi-Fi network it first installed at Ford Field back in 2013, a network that was only open to Verizon customers. According to a story in the Detroit Free Press, the new network (which won’t be operational until the end of the current NFL season) will be open to all fans, not just Verizon customers. However, like in other stadiums, Verizon customers will be able to access the Wi-Fi over a separate SSID, Verizon said. The Ford Field upgrade joins other NFL stadiums getting a Wi-Fi revamp this season, including the Carolina Panthers, the New England Patriots, the Houston Texans and others.

Verizon would not answer questions about how many Wi-Fi access points the new deployment would have, instead only claiming that coverage would be “quadrupled” from the previous installation. Verizon also claimed it now has Wi-Fi networks in 11 NFL stadiums, but declined to provide a list of which stadiums have Verizon Wi-Fi. MSR has previously reported that the Seattle Seahawks’ CenturyLink Field has a Verizon network, as does FedEx Field in Washington, D.C.. Verizon also claims it will install Wi-Fi in two other NFL stadiums this year, but again declined to name the stadiums.

VenueNext lands $15 million Series B funding, adds San Jose Sharks as 1st NHL client

Screenshot from new San Jose Sharks app developed by VenueNext. Credit: VenueNext

Screenshot from new San Jose Sharks app developed by VenueNext. Credit: VenueNext

Stadium and venue app developer VenueNext has secured a $15 million Series B round of financing, as well as its first National Hockey League client, the San Jose Sharks.

Both announcements were made by VenueNext Tuesday, just ahead of Wednesday’s season opener for the Sharks at SAP Center in San Jose. The new app is ready for fans to download in time for the Sharks’ game against the Los Angeles Kings. Later this season fans will also be able to connect via the arena’s new Wi-Fi network, which will use Wi-Fi gear from Cisco.

The new round of funding brings VenueNext’s total of announced venture capital to $24 million, following a $9 million round raised last summer. Causeway Media Partners, which led the initial round, is also leading the new round; according to VenueNext some of its first-round investors are also participating in the B round, but the company did not yet name any of them other than Causeway. Twitter, Live Nation and Aruba were among the Series A investors in VenueNext.

Adding hospitality and healthcare to market targets

In the increasingly competitive market for stadium and team application development, VenueNext has had a solid year in breaking away from just being the app provider to the San Francisco 49ers and Levi’s Stadium, its initial offering. So far this year, new VenueNext apps have appeared at Super Bowl 50, Yankee Stadium, Churchill Downs for the Kentucky Derby, and at the Minnesota Vikings’ new home, U.S. Bank Stadium. VenueNext also announced a new app being developed for the Saratoga and Belmont horse-racing tracks, which will be launched next year.

SAP Center, home of the San Jose Sharks. Credit: SanJoseSharks.com.

SAP Center, home of the San Jose Sharks. Credit: SanJoseSharks.com.

While VenueNext still hasn’t come close to publicly announcing the 30 clients CEO John Paul said the company would have by the end of 2015, the list of announced clients now includes the 49ers and Super Bowl 50, the Dallas Cowboys, the Orlando Magic, and now the San Jose Sharks. According to VenueNext, it does have clients signed already in the healthcare and hospitality markets, but cannot name them due to confidentiality agreements. VenueNext said it will announce more customers in the next few months.

Sharks fans get beverage, not food, delivery to seats — for now

One of the signature VenueNext services at Levi’s Stadium, the ability for all fans to use the app to order concessions delivered to their seats, will initially only support in-seat beverage ordering and delivery for Sharks fans, according to VenueNext. That service is similar to how the VenueNext app was used at Super Bowl 50. Other new services now available at SAP Center via the app include digital ticketing, with the ability to view, upgrade or transfer tickets; the ability to view and manage parking passes; mobile ticket access via the VenueNext ticket kiosks; and team content.

According to Sharks chief operating officer John Tortora, the team was first introduced to VenueNext during the NHL Stadium Series game at Levi’s Stadium in February of 2015.

“We were impressed with their execution at that event and have witnessed the business success they have generated at sports venues throughout the country,” said Tortora of VenueNext in an email communiction. “We look to bring that standard to SAP Center.” According to Tortora, wayfinding and virtual reality experiences are among features that will be added to the app in the future.

The Sharks app page also says that during the season the app will add a large list of Sharks-related content, including team and league stats, and it will also add in-game trivia contests. So far in most of its deployments, VenueNext has added and improved features in its apps over time.

On the Wi-Fi side, the venue is now getting its first full-scale Wi-Fi network for fans, a deployment that will include the use of Cisco StadiumVision for digital-display controls. According to the Sharks, the Wi-Fi network is expected to be operational by Dec. 1.

“To complement our new Sharks app and the use of it at SAP Center, we are in the process of deploying Cisco Connected Stadium Wi-Fi, a best-in-class Wi-Fi platform used in sports venues around the world,” Tortora said. “We want our patrons to be able to easily and reliably connect while at SAP Center to allow for the best fan experience when attending Sharks games and other events.”

VenueNext said it now has 90 employees, with offices in Santa Clara, Calif., San Francisco, New York and London. The new funds, the company said, will be used to “continue to innovate on our platform,” and also to help launch the new vertical markets as well as expansion to international clients.

Verizon correcting bills to make sure NFL streaming is really ‘free’

Screen Shot 2016-09-09 at 10.29.06 PMIf you are a Verizon Wireless customer who rejoiced at the announcement earlier this fall that the wireless carrier wouldn’t charge for data used for watching live NFL action via its NFL Mobile app, you can stay happy — even if you do initially see some data charges on your monthly bill.

After being alerted by a Mobile Sports Report reader that data charges were still showing up after a user watched some NFL games, we asked Verizon if their promotion was up and working. According to an email response from an unnamed Verizon authority, it appears that while Verizon was making the NFL action free, the streaming of ads was not — but never fear, Verizon said it is crediting users’ bills for any ads watched while watching football via the NFL Mobile app.

Here is the official response we got from Verizon’s PR company, attributed to “someone at Verizon” (seriously, that’s how it was worded to us):

“The NFL live games are automatically zero rated so they won’t count against customer’s data. Ads during the games are not, but we are issuing customers additional data (1GB or more depending on each customer’s NFL streaming) or post bill credits to cover the usage incurred from the ads. That process is disclosed in the disclaimers and the bonus data is issued automatically when a customer streams an NFL live game. We are monitoring this closely to make sure all customers do get a truly free NFL experience with a combination of automatic zero rating, additional data or bill credits as needed.”

As you continue to watch for free maybe give a shout out to Gary for spending 40 minutes on the phone with Verizon customer support, a painful experience we know well. Anyone else still seeing data charges for football with no refunds, let us know.

Analysis: The year of the big stadium Wi-Fi upgrade

Carolina Panthers director of IT James Hammond shows off a new under-seat Wi-Fi AP at Bank of America Stadium. Credit: Carolina Panthers

Carolina Panthers director of IT James Hammond shows off a new under-seat Wi-Fi AP at Bank of America Stadium. Credit: Carolina Panthers

Even in the midst of several brand-new stadium debuts and the future-proofed wireless networks inside them, there is a separate, yet distinct trend emerging in the big-stadium, wireless connectivity world: Call it the year of the big upgrade.

Our profile in our latest STADIUM TECH REPORT of Bank of America Stadium in Charlotte, N.C., is a case in point: Thanks to the never-ending demand for more connectivity for fans, stadiums that deployed networks just a few years ago are now finding that those old systems already need upgrades or replacements, typically at a much higher cost than the original network. In addition to BofA Stadium, the New England Patriots’ home, Gillette Stadium, also got a Wi-Fi makeover this past summer, going from about 400 Wi-Fi APs to well over a thousand, with most of the new ones deployed under seats.

According to Fred Kirsch, who oversees the Gillette Stadium network, some of the under-seat placements there were especially tricky, since granite underneath the stands didn’t allow for the ability to drill through the concrete. A workaround involving an above-ground enclosure was envisioned and manufactured, underlining the custom complexity of network deployment found from stadium to stadium. No two are the same, and what works at one may or may not work at another.

But what is common across all these large venues is the ever-increasing need for bandwidth, a moving target that has yet to slow down or stabilize. Last year the story that turned everyone’s head was the need by carriers to upgrade their DAS infrastructure at Levi’s Stadium ahead of Super Bowl 50 – this coming just a year after the stadium had opened for business. While the demands of a Super Bowl (especially Super Bowl 50, which set records for DAS and Wi-Fi usage) are perhaps much different than everyday events, it’s still a safe bet that for many stadiums with Wi-Fi networks – especially the early movers – 2016 has become a year of reckoning, or biting the bullet and writing more checks for more coverage, perhaps seemingly too soon after the initial rollout.

Getting ready for Super Bowl LI

In Houston, NRG Stadium finally has Wi-Fi, and not a moment too soon, with Super Bowl LI on the near horizon. Since the venue didn’t have Wi-Fi prior to this season it’s not really an upgrade but it’s hard to understate the challenge of putting in a Super Bowl-ready network in just one summer, a construction calendar shortened by the fact that integrator 5 Bars and equipment vendor Extreme Networks had to wait until after the NCAA Men’s Final Four was over to begin installing cabling and APs. At of the start of the NFL season the Wi-Fi network is already live at NRG Stadium, and is sure to go through weekly tweaks as the league marches on toward its championship game.

Gillette Stadium before the Sept. 11 game vs. the Miami Dolphins. Credit: Steve Milne, AP, via Patriots.com

Gillette Stadium before the Sept. 11 game vs. the Miami Dolphins. Credit: Steve Milne, AP, via Patriots.com

And while attention-grabbing new stadiums like US Bank Stadium in Minneapolis and Mercedes-Benz Stadium in Atlanta are planning big network capacity from the get-go, some new stadiums like T-Mobile Arena in Las Vegas have upgrade thinking planned in from the start, with the idea that the network will never really be a finished product, at least until they stop making new phones or developing new apps. Of course, that future isn’t happening anytime soon, with the Apple iPhone 7 announcement with the new double-lens camera coming in just before the start of another football season.

New phones and new apps mean more bandwidth demands, leading even those who already have stadium networks to keep wondering if what they’ve installed is enough. We suspect this may be an ongoing story line for the foreseeable future, so – stay tuned here to Mobile Sports Report for the latest success stories and lessons learned from those who have already jumped in or jumped back in to the deployment fray.

Editor’s note: This column is from our latest STADIUM TECH REPORT, which is available for free download from our site. Read about Wi-Fi deployments at Bank of America Stadium, Mercedes-Benz Stadium and more!