Verizon sees 4.1 Terabytes of cellular data use in and around Super Bowl

We’re still trying to get an only-in-stadium breakdown but the early report from Verizon Wireless said that the company’s customers used 4.1 terabytes of cellular data directly around the Super Bowl Sunday in Glendale, Ariz.

According to the Verizon stats team this figure includes not just Verizon traffic on the stadium DAS, but also from the surrounding Westgate entertainment district that surrounds the University of Phoenix Stadium, a large mall/entertainment area that also includes the stadium parking lots.

Verizon, which like AT&T beefed up its coverage in the Phoenix area ahead of the Super Bowl, said its data totals more than doubled last year’s Super Bowl numbers, an interesting stat since the University of Phoenix Stadium (~72,000) seats fewer fans than last year’s venue, MetLife Stadium in New Jersey (82,500).

We will have a full stadium breakdown as soon as we can get all the Wi-Fi and DAS numbers we can, but for now here’s a Verizon infographic to chew on in the meantime.

SuperBowl_NetworkStats-07

Super DAS: AT&T, Verizon beefed up Phoenix area with mobile cell towers and more DAS

AT&T Cell on Wheels (COW) deployment outside the ESPNZone in Phoenix. Credit all photos: AT&T (click on any photo for a larger image)

AT&T Cell on Wheels (COW) deployment outside the ESPNZone in Phoenix. Credit all photos: AT&T (click on any photo for a larger image)

While we wait for the traffic stats from the incredibly exciting Super Bowl XLIX, here’s the final installment of our Super DAS series — in which the two major U.S. wireless carriers, AT&T and Verizon Wireless, provide some details about how they beefed up coverage in and around Phoenix to handle the expected Super Bowl communications crush.

The lengths to which AT&T and Verizon went to ensure no signals were dropped are interesting from several business points of view; to be sure, no major carrier wants Twitter to erupt with reports of dropped calls from a major event. (AT&T folks still grimace when you bring up the historical benchmark for this type of problem, SXSW and Twitter.)

The flood-the-zone type of temporary enhancements now brought in on a regular basis for big events also point out the ongoing need for distributed antenna system (DAS) deployments: the basic fact of our ever more connected lives simply means that for large public venues, or places where lots of people gather at once, the legacy cellular network designs simply can’t keep up.

ESPNZone DAS gear in underground garage

ESPNZone DAS gear in underground garage

To make sure it could, AT&T said it deployed 10 cell towers on wheels (aka “COWS”) to the Phoenix area in advance of the weekend, while also upgrading its equipment at DAS installations like the one Crown Castle had at the University of Phoenix Stadium as well as at other points around town. AT&T folks were kind enough to supply us with plenty of photos of the deployments — we especially like the DAS built in an underground garage near the ESPNZone outlet in Phoenix.

Verizon also said it deployed 13 COWs and upgraded many DAS deployments in the Phoenix area prior to the Super Bowl, and even said it had a team of network technicians on hand to make sure traffic kept running smoothly.

How did it all work out? So far, we haven’t seen any reports of missed cellular connections during Super Bowl weekend (which also included the Waste Management golf tournament in the area, further adding to cellular pressure). What it does make us wonder about is the economic solution in the future to big-crowd wireless traffic concerns, which clearly aren’t limited to inside the event venue anymore. Are more portable deployments the way forward, or will we see more DAS installations that can be upgraded quickly on the fly?

More photos below!

AT&T COW with box on roof

AT&T COW with box on roof

Another AT&T COW

Another AT&T COW

AT&T COW at Wild Horse Pass

AT&T COW at Wild Horse Pass

AT&T COW deployment in downtown Phoenix

AT&T COW deployment in downtown Phoenix

Downtown COW on a roof

Downtown COW on a roof

ESPNZone DAS cabling run

ESPNZone DAS cabling run

Hyatt Gainey Ranch COW

Hyatt Gainey Ranch COW

Super DAS: Crown Castle’s neutral host infrastructure aims to keep Super Bowl XLIX fans connected, inside and outside the stadium

Editor’s note: This story is part 1 of a series of profiles of the providers of the extensive Distributed Antenna System (DAS) deployment for Super Bowl XLIX at and around the University of Phoenix Stadium in Glendale, Ariz. and other parts of the Phoenix city area as well. Stay tuned all week as we highlight how DAS will keep Super Bowl fans connected, no matter where they roam in and around Phoenix and Glendale this week and weekend.

University of Phoenix Stadium getting its Super Bowl on. (Click any photo for a larger image) Credit all photos: Paul Kapustka, MSR

University of Phoenix Stadium getting its Super Bowl on. (Click any photo for a larger image) Credit all photos: Paul Kapustka, MSR

When is a big air-fan vent not an air-fan vent? When it’s a fake vent covering a hidden cellular antenna, put there to keep people from noticing the technology that’s keeping their cell phones connected. Before kickoff at Super Bowl XLIX Feb. 1 in Glendale, Ariz., many fans outside the University of Phoenix Stadium will walk right by a faux vent and its sheltered equipment, never knowing the attention to detail that goes into a major-venue Distributed Antenna System (DAS) deployment.

But to stadium technology connectivity professionals, such leaps of aesthetic deception are just part of a day’s, or perhaps month’s, DAS deployment work. For neutral host DAS provider Crown Castle, the fake vents on the shell of the University of Phoenix Stadium — and the powerful antennas behind — are just one part of a massive project to ensure there is excellent mobile-device connectivity both inside and outside the Super Bowl stadium, so that fans never get a dropped signal anywhere between the parking lot and their prized seat.

During a recent press tour, a small team of Crown Castle employees showed off some of the upgraded DAS network deployed at the University of Phoenix Stadium as well as in the surrounding Westgate Sports and Entertainment District, a sort of open-air mall that stretches from the UoP Stadium past numerous attached restaurants and stores, also encompassing the Gila River Arena, home of the Phoenix Coyotes of the NHL. Over the past year or so, Crown Castle has been upgrading the DAS inside and outside the arena, throughout the mall areas as well as into the huge parking lots that surround it and the football stadium, bringing connectivity to phones being used by customers from all the four major U.S. wireless carriers.

Since the mall and all its food outlets are conveniently located a short stroll from the stadium, it’s a good bet that a large portion of the Super Bowl crowd will spend time wandering around the Westgate area before and after the big game. Thanks to Crown Castle’s efforts, there shouldn’t be many connectivity problems, as antenna deployments on light poles, building rooftops and — yes, even behind fake vents — should be able to keep devices on the cellular networks without a glitch.

Game day connectivity starts in the parking lot

Since we couldn’t actually spend much time wandering around the stadium itself — even three weeks before the big game, the facility was already on NFL security lockdown — most of the Crown Castle tour consisted of walking around the Westgate mall/neighborhood, hearing about the various methods Crown Castle used to locate the necessary DAS antennas. In all, there are five separate DAS networks Crown Castle is responsible for in the area around the stadium: The football stadium itself; the Gila River Arena (which we will profile in an upcoming feature on hockey stadiums); the Westgate shops and restaurants; the nearby Renaissance Hotel; and the surrounding parking lots.

Parking lot light poles, Westgate entertainment district. Can you spot the DAS?

Parking lot light poles, Westgate entertainment district. Can you spot the DAS?

The curious start of the tour in a far-flung parking lot made sense when we found ourselves next to a small DAS equipment box and a light pole with multiple antennas (which had not yet been covered with their final aesthetic sheaths). Aaron Lamoureux, program manager for Crown Castle’s small cell solutions, served as tour guide, and said that for the Westgate area alone there were 18 individual node locations, with about 52 antennas total. Some were located on light poles, some on rooftops, and some along walkways between buildings, to conquer the unique RF characteristics of the open-air/large building outdoor mallish area that is Westgate. (See photos for DAS geek views)

For the University of Phoenix Stadium itself, Crown Castle deployed 228 DAS antennas inside (more on this in an upcoming profile) and at 21 different locations outside the stadium, 13 of those on parking lot poles and 8 mounted on the building itself. Why building-mounted antennas? If you’ve never been there, the University of Phoenix Stadium has a large plaza area on one side, which is used for pre-game activities like rallies, bands and other walk-up amenities where fans gather before entering. The challenge for Crown Castle was finding places to deploy antennas at a low enough height to cover crowds of people standing in one location. While some parts of the building allowed for regular antenna placements, a big part of the plaza faces part of the stadium wall that is a sheer sheet, with no aesthetic place to mount a DAS antenna — unless you add a fake vent or two to the existing design, that is.

Keeping everyone happy is part of the neutral host job

See the big air vents? Nobody would tell us which ones were 'faux vents,' there to hide DAS antennas

See the big air vents? Nobody would tell us which ones were ‘faux vents,’ there to hide DAS antennas

To people outside the industry it might seem silly to go to such lengths just to keep folks from noticing antennas, but anyone who’s deployed a network for a detail-oriented building owner knows why aesthetics are important. That’s why you paint antenna enclosures to match the surrounding walls, or build sheaths to keep wires and other obvious gear out of main sight. It’s part of the art of wireless network deployment, and not as simple as it sounds. Experience counts.

The complex owner and operator relationships involved in the stadium and surrounding-area DAS also seem tailor-made for a big, experienced provider like Crown Castle, which has a long history of deploying and operating multiple-tenant networks. With five different landlords and four different carriers, being the neutral DAS host for this year’s Super Bowl is a task with many moving parts; but, as Mike Kavanagh, president of sales for Crown Castle’s small cell solutions, said, “We understand how to run networks, how to manage them and deal with carriers. It’s high touch and very fluid. But we know that business.”

COMING UP NEXT: What’s inside the network inside the stadium.

MORE PICTURES BELOW! (Click on any picture for a larger image.)

Sky Harbor Airport: Ready for Super XLIX

Sky Harbor Airport: Ready for Super XLIX

Verizon's NFL Mobile ads were in airport walkways well before the Big Game

Verizon’s NFL Mobile ads were in airport walkways well before the Big Game

If you stumble off the escalator, Bud Light is there to catch you

If you stumble off the escalator, Bud Light is there to catch you

The Westgate uber-mall should see a lot of fan activity (and connectivity) on game day

The Westgate uber-mall should see a lot of fan activity (and connectivity) on game day

Here's the official Super Bowl replay HQ (actually a place with DAS antennas on the roof that you can't see)

Here’s the official Super Bowl replay HQ (actually a place with DAS antennas on the roof that you can’t see)

Mama Gina's will offer you pizza and DAS on the roof

Mama Gina’s will offer you pizza and DAS on the roof

More DAS antennas, on a Westgate walkway

More DAS antennas, on a Westgate walkway

Outside UoP Stadium, where the architecture allows for DAS antenna placement

Outside UoP Stadium, where the architecture allows for DAS antenna placement

Close-up of that placement. Still pretty well hidden.

Close-up of that placement. Still pretty well hidden.

Parking lot light mounts. These will have sheaths by Super Sunday.

Parking lot light mounts. These will have sheaths by Super Sunday.

Here's the remote equipment box that powers the light pole antennas. Also scheduled for more concealment.

Here’s the remote equipment box that powers the light pole antennas. Also scheduled for more concealment.

Every artist leaves a signature...

Every artist leaves a signature…

NBC will live-stream Super Bowl online for tablets, computers; Verizon NFL Mobile will carry for smartphones

Screen shot 2015-01-20 at 10.37.03 AMIf for some reason you are banned from the living room couch for the Super Bowl, NBC has you covered — the network will be streaming the game live online, along with hours of pregame, postgame and halftime festivities — for anyone with an Internet connection and a laptop, PC or tablet.

According to a press release sent out Monday NBC said it will also not require viewers to have a qualifying cable or satellite contract to view the game, thereby eliminating the often annoying login process that accompanies many other online live sports streaming activities. You will, of course, be subject to multiple NBC advertisements but hey — a small price to pay for the convenience of being able to watch the game online.

(Mobile Sports Report is old enough to remember Super Bowl parties where we rented extra TV sets for the bathrooms and the kitchen; now you can just use in-house Wi-Fi and a tablet or laptop, perhaps with a splash guard.)

On the cellular side, if you are stuck somewhere and want to watch on your phone, the only option is having a Verizon contract and using the NFL Mobile app. If you are a More Everything customer the live viewing of the game is free, if not you must pay a $5 monthly charge for the one day in February that you will need premium access. (Pro tip for Verizon customers — don’t forget to cancel that premium access charge the day after the game, since Verizon will happily charge you $5 a month all summer long even though there are no NFL games during that time.)

It’s an easy guess so we will predict right now that this year’s Super Bowl will set new online records for most Internet viewers — without fail this has happened every year since the networks and the league started making the game available online. According to the NBC folks the online stream will have some handy extras, like the DVR feature that lets you scroll back to important plays, as well as additional camera angles and in-game stats.

Now our next dream is for the Shield and its broadcasters to follow ESPN’s lead on the college championships and provide online “Megacast” options for alternate announcers. A man can hope.

All NFL playoff games available online; Verizon only for smartphone watching

vzn_playoffWe’ve come a long way from the days when it was a struggle to even find NFL live action online. This season, all NFL playoff games, including the Super Bowl, will be available for online viewing, via a desktop computer, laptop or tablet, no matter which network is carrying the games. The league has even created a handy single web page to find instant access to the live streams, no small matter since network pages usually make you jump through several clicks to find the actual live stream.

Depending upon the broadcast network, you may need to have a qualifying cable or satellite contract to view the games. For this morning’s CBS game there was no confirmation process but for the afternoon Fox game I needed to submit cable provider info. Let me know what you see.

However, if you want to watch playoff games on your smartphone, your only option is to be a Verizon Wireless customer since that provider is the exclusive NFL live action host for smartphone devices. To view the live action, Verizon customers need to have either a More Everything plan or pay the $5 monthly premium fee for the NFL Mobile application. To be clear, customers of any wireless provider can download the NFL Mobile app to get all the stats and archived video it provides, but only Verizon customers can purchase access to live action on phone platforms.

UPDATE: Congrats to FOX for having the first playoff online broadcast fail:

Screen Shot 2015-01-04 at 3.01.07 PM

No word from Fox yet on when its stream will come back, or why it was down.

Stadium Tech Report: Arizona Cardinals get stadium ready for Super Bowl with Wi-Fi upgrade

University of Phoenix Stadium. Credit all photos: Arizona Cardinals.

University of Phoenix Stadium. Credit all photos: Arizona Cardinals. (click on photos for larger image)

As they get set to host their second Super Bowl this February, the IT team at the University of Phoenix Stadium in Glendale, Ariz., knows now what they didn’t know then: The big game requires a big wireless network. Bigger than you think.

“It’s funny to look back now on that first Super Bowl,” said Mark Feller, vice president of information technology for the Arizona Cardinals, speaking of Roman numeral game XLII, held in the barely 2-year-old facility on Feb. 3, 2008. With a couple Fiesta Bowls and one BCS championship game (2007) under his belt in a facility that opened with Wi-Fi and DAS, Feller said he and his team “thought we had a good handle” on what kind of network was required for a Super Bowl crowd.

The NFL, he said, begged to differ. Those college games might have been big, but the Super Bowl was bigger.

“We had the Fiesta Bowl that year at night, and when the game was over there were people from the NFL there wanting to know when they could set up,” said Feller in a recent phone interview. “This year, we’re much better prepared. We know what the water temperature is this time.”

Rip and replace, with more and better gear

Wi-Fi railing antennas

Wi-Fi railing antennas

For Super Bowl XLIX, scheduled to take place in Glendale on Feb. 1, 2015, Feller and his team have not just tuned up their network — they have done a full rip and replace of the Wi-Fi system, installing new Cisco gear from back end to front, in order to support a wireless game-day demand that is historically second to none. Integrator CDW has led the Wi-Fi effort and Daktronics and ProSound did the installation of new video screens, and neutral host Crown Castle has overseen a revamp of the DAS system, again with more antennas added to bolster coverage. In all, there has been more than $8 million in wireless improvements before this year, Feller said, as well as another $10 million for two new video boards that are each three times larger than what was there before.

“The last three or four years there have been things we knew we needed to improve [before the Super Bowl],” Feller said. After extensive work with the NFL’s technical team — this time well before the Fiesta Bowl — Feller oversaw a “top to bottom” refurbishment that included replacing core Cisco networking gear with newer gear, and new and more Wi-Fi access points that now total somewhere north of the 750 mark, with some more to be added before the big game. The new network, which was in place for the start of the current NFL season, has undergone testing by CDW at each home game, Feller said. CDW also plans to expand the network outside the stadium before the Super Bowl, in part to handle the extra events that take place not just on game day but in the days leading up to the game.

“The plan is to install more [coverage] outside, in the plaza areas,” Feller said.

When it opened in 2006, the $455 million University of Phoenix Stadium was one of the first with full-bowl Wi-Fi, using Cisco gear from the inside out. “Cisco was in here before they called it [their solution] ‘connected stadium’,” Feller said. From core network switches to firewalls to edge switches, this year there is all new Cisco gear in the venue, as well as new 3700 series APs, with panel antennas and antennas in handrails.

“Handrail [antennas] are sometimes a bit of a challenge, because you need to drill through concrete that’s 40 feet up in the air, behind another ceiling,” said Feller, describing one particular design challenge. Another one was mounting antennas on drop rods from the catwalks below the stadium’s retractable roof, to serve the upper-area seating. There are also some new Wi-Fi APs on the front row of the seating bowl, pointing up into the crowd.

“It was a fun project,” Feller said.

Stadium with roof open

Stadium with roof open

All on board for the DAS

The upgrade for the stadium’s DAS, led by Crown Castle, was just finished a few weeks ago, Feller said, and included more coverage outside the stadium as well, with antennas placed on light poles and on the stadium’s shell.

“Crown Castle did a great job of managing the carriers” on what is a 48-sector DAS, Feller said. “It [the upgrade] really required a lot of creative thinking from their engineers.”

Since the stadium was originally designed with wireless in mind, Feller and his team didn’t need to build new head end room for the DAS upgrades. “But I wouldn’t say we have plenty of space left,” he added. “We’ve got a lot of new equipment.”

Though all the major carriers are expected to be on the DAS by the big game, league partner Verizon Wireless should have some special projects up its sleeve for the big game, including another demonstration of its LTE Broadcast technology, which optimizes things like live video over LTE cellular links.

New Cardinals app a preview of Super Bowl version?

The Cardinals also had a new version of the game-day team app for this season, built by stadium-app leader YinzCam. According to Feller the new app supports three different live video feeds, as well as instant replays.

Wi-Fi antenna on railing

Wi-Fi antenna on railing

“It’s really cool to have that ability to watch things like a touchdown pass at the end of the game,” Feller said. And while no details have yet been revealed, in an interview with NFL CIO Michelle McKenna-Doyle earlier this year MSR learned that the league and YinzCam are working on a Super Bowl app with its own new bells and whistles. (Stay tuned for more info on the Super Bowl app.)

In addition to two more regular-season home games in December, the University of Phoenix Stadium will have at least a couple more dry runs to help test the network, during the Dec. 31 Fiesta Bowl and during the NFL’s Pro Bowl, scheduled for Jan. 25. And though the Cardnials lost to the Atlanta Falcons Sunday, at 9-3 they are still tied with the Green Bay Packers for the best record in the NFC, something that has the Phoenix faithful optimistic about the postseason.

“We’re going to get some more test runs, on New Year’s Eve and during the Pro Bowl,” Feller said. “And maybe some home playoff games as well!”

(more photos below)

Wi-Fi antenna in roof rafters

Wi-Fi antenna in roof rafters

More antennas in rafters

More antennas in rafters

Wi-Fin antenna under overhang

Wi-Fi antenna under overhang