MSR at the College Football Playoff Championships: Send us your speedtests!

ESPN's College Football Playoff Championships stage in downtown Fort Worth, Sunday night. Credit: Paul Kapustka, MSR

ESPN’s College Football Playoff Championships stage in downtown Fort Worth, Sunday night. Credit: Paul Kapustka, MSR

As Twitter followers found out yesterday, MSR is in “north Texas,” aka the Dallas-Fort Worth “Metroplex,” attending tonight’s inaugural College Football Playoff Championship game at AT&T Stadium.

We’re here to see a test of AT&T’s LTE broadcast technology, which will ostensibly make it easier for venues to deliver live video streams via a cellular connection. But we also are going to take advantage of the event to walk around “Jerry’s World” and take speed tests and see how the network at AT&T Stadium performs for a big game. If you are in attendence and know how to do a Wi-Fi or cellular speed test, send us the results (tweet at @paulkaps is the best way, or send me email to kaps at mobilesportsreport.com). Check Twitter for updates during the game.

Washington dropping Huawei for Cisco/Verizon Wi-Fi at FedEx Field, report says

Ming He, Country General Manager for Huawei in the U.S. (left), and Rod Nenner, Vice President of the Washington Redskins (right), pictured together when Huawei announced the team sponsorship and partnership.

Ming He, Country General Manager for Huawei in the U.S. (left), and Rod Nenner, Vice President of the Washington Redskins (right), pictured together when Huawei announced the team sponsorship and partnership.

According to a report from Bill Gertz at the Washington Times, the Washington, D.C. NFL franchise is apparently scrapping a recent deal with Chinese networking gear supplier Huawei to put fan-facing Wi-Fi into FedEx Field, turning instead to U.S. companies Cisco and Verizon.

Gertz, in the “Inside the Ring” column at the Times, said the Washington team’s senior vice president Tony Wyllie said in an email that “We [Washington] are in the process of deploying a stadium-wide Wi-Fi network working with Verizon and Cisco.” Gertz said the team did not elaborate on why the recent deal with Huawei was apparently scrapped before it got started.

Huawei, which claims to have installed Wi-Fi networks in many stadiums worldwide, had not had any large-scale installations at major U.S. venues before announcing the FedEx Field deal. A major competitor to large U.S. networking firms like Cisco, Huawei has been at the center of controversy in recent years, including being tabbed as a security threat by U.S. government officials, and later as a reported target for N.S.A. surveillance.

Under the announced terms of the deal, Huawei was supposed to install Wi-Fi in suite areas this December; a company spokesman said that while there was no official deal announced, Huawei was also supposed to follow that install up with a full-stadium deployment before the 2015 season started. In the initial announcement, the team announced Huawei Enterprise USA as a multi-year team sponsor and “Official Technology Partner.”

We have got calls and emails in to all the interested parties, and will update this story as we hear more.

Stadium Tech Report: THE COLLEGE FOOTBALL ISSUE looks at university Wi-Fi deployments

collegethumbIf there was a college football playoff for stadium wireless network deployments, which four teams would be in? Electing myself to the committee, I think my top picks would be the same venues we’re profiling in our latest Stadium Tech Report – Baylor, Nebraska, Stanford and Texas A&M. All four are pursuing high-end networks to support a better fan experience, leading the way for what may turn out to be the largest “vertical” market in the stadium networking field – sporting venues at institutions of higher learning.

To be sure, network deployments at major universities in the U.S. are still at the earliest stages — in our reporting for our latest long-form report, we found that at two of the top conferences, the SEC and the Pac-12, only four schools total (two in each conference) had fan-facing Wi-Fi, with only one more planned to come online next year. Why is the collegiate market so far behind the pro market when it comes to network deployment? There are several main reasons, but mostly it comes down to money and mindset, with a lack of either keeping schools on the sidelines.

Leaders look for NFL-type experiences

But at our “playoff” schools, it’s clear that with some ready budget and a clear perspective, college stadiums don’t need to take a back seat to anyone, pro stadiums included. The networks, apps and infrastructure deployed for this season at Baylor’s McLane Stadium and Nebraska’s Memorial Stadium are among the tops anywhere in sports, and the all-fiber infrastructure being put in place at Texas A&M should make that school’s Kyle Field among the most-connected if all work gets completed on time for next football season. Read in-depth profiles on these schools’ deployments, along with team-by-team capsule technology descriptions and an exclusive interview with Mississippi State athletic director Scott Stricklin in our latest report, available for free download from our site.

We’d like to take a second here to thank our sponsors, without whom we wouldn’t be able to offer these comprehensive reports to you free of charge. For our fourth-quarter report our sponsors include Crown Castle, SOLiD, Extreme Networks, Aruba Networks, TE Connectivity, and Corning.

Stadium Tech Report: Arizona Cardinals get stadium ready for Super Bowl with Wi-Fi upgrade

University of Phoenix Stadium. Credit all photos: Arizona Cardinals.

University of Phoenix Stadium. Credit all photos: Arizona Cardinals. (click on photos for larger image)

As they get set to host their second Super Bowl this February, the IT team at the University of Phoenix Stadium in Glendale, Ariz., knows now what they didn’t know then: The big game requires a big wireless network. Bigger than you think.

“It’s funny to look back now on that first Super Bowl,” said Mark Feller, vice president of information technology for the Arizona Cardinals, speaking of Roman numeral game XLII, held in the barely 2-year-old facility on Feb. 3, 2008. With a couple Fiesta Bowls and one BCS championship game (2007) under his belt in a facility that opened with Wi-Fi and DAS, Feller said he and his team “thought we had a good handle” on what kind of network was required for a Super Bowl crowd.

The NFL, he said, begged to differ. Those college games might have been big, but the Super Bowl was bigger.

“We had the Fiesta Bowl that year at night, and when the game was over there were people from the NFL there wanting to know when they could set up,” said Feller in a recent phone interview. “This year, we’re much better prepared. We know what the water temperature is this time.”

Rip and replace, with more and better gear

Wi-Fi railing antennas

Wi-Fi railing antennas

For Super Bowl XLIX, scheduled to take place in Glendale on Feb. 1, 2015, Feller and his team have not just tuned up their network — they have done a full rip and replace of the Wi-Fi system, installing new Cisco gear from back end to front, in order to support a wireless game-day demand that is historically second to none. Integrator CDW has led the Wi-Fi effort and Daktronics and ProSound did the installation of new video screens, and neutral host Crown Castle has overseen a revamp of the DAS system, again with more antennas added to bolster coverage. In all, there has been more than $8 million in wireless improvements before this year, Feller said, as well as another $10 million for two new video boards that are each three times larger than what was there before.

“The last three or four years there have been things we knew we needed to improve [before the Super Bowl],” Feller said. After extensive work with the NFL’s technical team — this time well before the Fiesta Bowl — Feller oversaw a “top to bottom” refurbishment that included replacing core Cisco networking gear with newer gear, and new and more Wi-Fi access points that now total somewhere north of the 750 mark, with some more to be added before the big game. The new network, which was in place for the start of the current NFL season, has undergone testing by CDW at each home game, Feller said. CDW also plans to expand the network outside the stadium before the Super Bowl, in part to handle the extra events that take place not just on game day but in the days leading up to the game.

“The plan is to install more [coverage] outside, in the plaza areas,” Feller said.

When it opened in 2006, the $455 million University of Phoenix Stadium was one of the first with full-bowl Wi-Fi, using Cisco gear from the inside out. “Cisco was in here before they called it [their solution] ‘connected stadium’,” Feller said. From core network switches to firewalls to edge switches, this year there is all new Cisco gear in the venue, as well as new 3700 series APs, with panel antennas and antennas in handrails.

“Handrail [antennas] are sometimes a bit of a challenge, because you need to drill through concrete that’s 40 feet up in the air, behind another ceiling,” said Feller, describing one particular design challenge. Another one was mounting antennas on drop rods from the catwalks below the stadium’s retractable roof, to serve the upper-area seating. There are also some new Wi-Fi APs on the front row of the seating bowl, pointing up into the crowd.

“It was a fun project,” Feller said.

Stadium with roof open

Stadium with roof open

All on board for the DAS

The upgrade for the stadium’s DAS, led by Crown Castle, was just finished a few weeks ago, Feller said, and included more coverage outside the stadium as well, with antennas placed on light poles and on the stadium’s shell.

“Crown Castle did a great job of managing the carriers” on what is a 48-sector DAS, Feller said. “It [the upgrade] really required a lot of creative thinking from their engineers.”

Since the stadium was originally designed with wireless in mind, Feller and his team didn’t need to build new head end room for the DAS upgrades. “But I wouldn’t say we have plenty of space left,” he added. “We’ve got a lot of new equipment.”

Though all the major carriers are expected to be on the DAS by the big game, league partner Verizon Wireless should have some special projects up its sleeve for the big game, including another demonstration of its LTE Broadcast technology, which optimizes things like live video over LTE cellular links.

New Cardinals app a preview of Super Bowl version?

The Cardinals also had a new version of the game-day team app for this season, built by stadium-app leader YinzCam. According to Feller the new app supports three different live video feeds, as well as instant replays.

Wi-Fi antenna on railing

Wi-Fi antenna on railing

“It’s really cool to have that ability to watch things like a touchdown pass at the end of the game,” Feller said. And while no details have yet been revealed, in an interview with NFL CIO Michelle McKenna-Doyle earlier this year MSR learned that the league and YinzCam are working on a Super Bowl app with its own new bells and whistles. (Stay tuned for more info on the Super Bowl app.)

In addition to two more regular-season home games in December, the University of Phoenix Stadium will have at least a couple more dry runs to help test the network, during the Dec. 31 Fiesta Bowl and during the NFL’s Pro Bowl, scheduled for Jan. 25. And though the Cardnials lost to the Atlanta Falcons Sunday, at 9-3 they are still tied with the Green Bay Packers for the best record in the NFC, something that has the Phoenix faithful optimistic about the postseason.

“We’re going to get some more test runs, on New Year’s Eve and during the Pro Bowl,” Feller said. “And maybe some home playoff games as well!”

(more photos below)

Wi-Fi antenna in roof rafters

Wi-Fi antenna in roof rafters

More antennas in rafters

More antennas in rafters

Wi-Fin antenna under overhang

Wi-Fi antenna under overhang

NFL’s CIO says teams need to share technology know-how

Michelle McKenna-Doyle, CIO, NFL

Michelle McKenna-Doyle, CIO, NFL

Editor’s note: the following interview with NFL CIO Michelle McKenna-Doyle was featured in our most recent long-form report, THE FOOTBALL ISSUE, which is available for free download. In addition to team-by-team capsules of technology deployments for all 32 NFL teams the issue also has several in-depth stadium technology profiles and an extensive look at the technology behind the San Francisco 49ers’ new Levi’s Stadium. Get your copy today!

In a league known for its intense rivalries, is it possible to get teams to work together and share information for the betterment of all? In the area of stadium technology, that task is on the to-do list for Michelle McKenna-Doyle, who is now in her third season as chief information officer for the National Football League.

In an interview with Mobile Sports Report, McKenna-Doyle outlined some of the league’s recent accomplishments in technology-related areas like instant replay and digital content, while also explaining how the league oversees stadium technology deployments. According to McKenna-Doyle, one of her office’s jobs is to act as a best-practices and lessons-learned clearinghouse, to better move the state of NFL stadium technology forward faster as a whole.

“One of our roles is helping teams help themselves” with technology deployment strategies, McKenna-Doyle said. “What we want is to provide a forum where clubs can share information with each other. If somebody’s done something and learned it doesn’t work, we can tell other clubs not to waste their time doing the same thing.”

The business use of technology

Focusing on the business uses of technology and not the 1s and 0s is somewhat of a natural fit for McKenna-Doyle, who spent 13 years at the Walt Disney Company in disciplines including finance and marketing before becoming a VP in IT for two years. During CIO stints at Centex Homes and Universal Orlando Resort, McKenna-Doyle said she focused on using technology to enhance the guest experience, a goal the NFL sought when it brought her aboard in September 2012.

Wi-Fi access point antennas visible on poles at CenturyLink Field, Seattle. Credit: Extreme Networks

Wi-Fi access point antennas visible on poles at CenturyLink Field, Seattle. Credit: Extreme Networks

“Part of my job is making sure our in-stadium experience for mobility meets the needs of our fans,” McKenna-Doyle said. While it is true that commissioner Roger Goodell said he wanted all NFL stadiums to have fan-facing Wi-Fi, and that the league does expect teams to meet a minimum set of connectivity standards, McKenna-Doyle said the NFL’s overall stadium-tech strategy is to be more of a guide than to dictate exactly which technologies or apps teams should deploy.

“People really are fans of their own team first, and we encourage clubs to have that engagement, and help them interact with fans,” McKenna-Doyle said. “There are minimum standards and we do grade their [technology] experience, and report that back to the club. But they manage it. We are more of a guide.”

Putting out a plan for stadium Wi-Fi

In the area of Wi-Fi, for example, McKenna-Doyle said that last year the league put together “a really deep-dive spec” that laid out all the basics necessary for stadium Wi-Fi deployments. “That was so teams didn’t have to start at square one for design,” McKenna-Doyle said.

The league also helped move Wi-Fi deployments forward faster by signing a preferred-supplier deal with Extreme Networks, under which teams get a discount on pricing in exchange for the league-wide sponsorship exposure. (Editor’s note: This week, the league announced that Extreme is now the official Wi-Fi technology supplier for the NFL.) Though teams are not required to use Extreme’s Wi-Fi gear, new Extreme-based Wi-Fi networks are in use this season at Seattle, Jacksonville, Tennessee, and Cincinnati, joining two previous Extreme installations in Philadelphia and New England.

“It’s a great option if teams choose Extreme, and it [the league deal] also creates a competitive environment for other suppliers like Cisco to step up,” McKenna-Doyle said.

Fans take pictures at Levi's Stadium. Credit: Paul Kapustka, MSR

Fans take pictures at Levi’s Stadium. Credit: Paul Kapustka, MSR

If necessary, the league can also play Wi-Fi matchmaker, as McKenna-Doyle said it did in bringing partner Verizon into Seattle, where the carrier deployed a Wi-Fi network at CenturyLink Field that is live this season.

At the writing of this report, 10 of the NFL’s 32 teams still had no fan-facing Wi-Fi services at their stadiums, a list that includes Green Bay, Baltimore, Cleveland, Buffalo, Houston, Oakland, San Diego, Washington, Minnesota and St. Louis. While McKenna-Doyle said that there are “a few more Wi-Fi announcements coming,” she also noted that some teams with lease uncertanties still don’t have firm plans to deploy Wi-Fi.

Help on the app side with partner YinzCam

McKenna-Doyle said the league also provides assistance in stadium application development via YinzCam Inc., a Pittsburgh-based company that has developed mobile apps for a number of pro sports teams. The NFL, which was an investor in YinzCam, uses the company’s technology in its league mobile apps, and McKenna-Doyle said YinzCam is also developing an app for the upcoming Super Bowl XLIX that will be “like nothing we’ve ever had before.”

Though teams are not required to use YinzCam – the San Francisco 49ers, for example, turned to newcomer VenueNext to develop their Levi’s Stadium app – McKenna-Doyle said that YinzCam may be a fit for other teams.

“For quality and speed to market, [YinzCam’s] product is very strong,” McKenna-Doyle said.

It’s about the fans, not the technology

Following a summer that saw her office overseeing the new method for on-field official review of replay calls – “which meant building a new system for 32 teams, 31 stadiums, and training all the officials” – as well as the launch of the NFL Now digital content site, McKenna-Doyle is back spending time with teams, counseling them on technology deployment resource management – “what to prioritize, and what to put on the back burner,” she said.

That includes technology ideas that might not work operationally, like a food-ordering service that isn’t staffed properly. For the in-seat food ordering feature at Levi’s Stadium, for instance, the 49ers said they did extensive research, hiring and training to make sure they had enough feet on the ground – runners carrying orders – to make the tech-inspired feature work.

If teams don’t do the human engineering behind the scenes, McKenna-Doyle said, the technology may not be that cool.

“We stress that it’s not about the technology, but about the fan experience,” McKenna-Doyle said. “It has to be operationally sound, and it has to be integrated with being at the game. If it’s not something that’s operationally sound, you might be better off not doing it.”

AT&T: Getting busy with multiple college football DAS deployments

In an interview with AT&T’s John Donovan earlier this year the company’s senior executive vice president told us that AT&T would continue to be aggressive in its deployment of stadium DAS systems. True to his word, here are announcements from no fewer than eight new top U.S. universities (and one that was announced earlier in the year) that got an AT&T DAS in time for this fall’s football season.

Included in the list of DAS deployments that AT&T either is leading or has joined another operator’s infrastructure are Baylor University, which has a whole new stadium and a new stadium Wi-Fi network as well; Big Ten schools Indiana University, Ohio State University, Michigan State University, the University of Minnesota and the University of Wisconsin (where AT&T also installed a new Wi-Fi network and some IPTV systems); the University of Missouri from the SEC; and Pac-12 schools the University of Washington as well as the University of California, an installation plan that we covered last year. AT&T also participated alongside Verizon in a unique joint DAS deployment at the University of Oregon, also announced earlier this year.

Why so much DAS? As we are finding out in the process of doing a lot of reporting for our upcoming Q4 Stadium Tech Report on college football stadium technology deployments, Wi-Fi deployments are still somewhat of a rarity, even at some of the biggest schools. As we’ve said before, bringing in a DAS deployment makes a lot of sense for schools since A) you can usually get the carrier to pay for most if not all of the cost of building the DAS; and B) a good DAS goes a long way toward eliminating the feared “no signal” problem that can still be found on many major college campus facilities.

How much have fans already been using the new networks? According to AT&T the new Mizzou DAS has done the biggest amount of traffic so far, with 290 gigabytes of traffic crossing the DAS system with its 150+ antennas at one game this season. Cal was close behind with an average of 253 GB per game so far in 2014, while up in Seattle at UDub the fans are generating an average of 190 GB per game. Remember, these stats represent ONLY AT&T traffic on the AT&T part of the DAS; since we still can’t convince Verizon to provide similar statistics we’ll just have to guess what the total-totals are.

Stay tuned for more information about college stadium deployments… look for our Q4 STR report in early December!