Cisco – Page 21

Entry issues snarl start of WrestleMania 32 at AT&T Stadium; crowd uses 6.77 TB of Wi-Fi

April 3, 2016 by Paul Kapustka 2 Comments

While Sunday’s WrestleMania 32 eventually ended up with a record attendance for the WWE’s signature event with 101,763 fans in attendance, some snafus with the entry process had fans reporting as long as three-hour waits to get in, with some pegging a Wi-Fi outage that disabled ticket scanners as one of the roots of the problem.

Here's what it looked liked for fans trying to get into #WrestleMania From @dallasnews photog David Guzman pic.twitter.com/BaZS8U5wxN

— Mark Francescutti (@mfrances100) April 4, 2016

~~We don’t as of yet have any on-the-record comments from AT&T Stadium so we can’t say exactly what the problem was.~~ UPDATE, 4/4/16, 2:50 p.m.: According to John Winborn, chief information officer for the Dallas Cowboys Football Club (and the person who runs the network inside AT&T Stadium), despite the rumors that emerged there was no Wi-Fi outage at the stadium — in fact Winborn said the crowd used 6.77 TB of Wi-Fi data Sunday, a single-day stadium Wi-Fi total second only to the 10.1 TB used at Levi’s Stadium for Super Bowl 50. (More details on the Wi-Fi consumption coming soon.)

Wi-Fi rumors aside, on Sunday night Twitter and Instagram were full of photos from irate fans who were stuck outside the venue, with some reportedly unable to get inside or to their seats before the actual matches started. According to a blog post at the Fort Worth Star-Telegram, the WWE and the Stadium issued a joint press release halfway through the event that read:

“To ensure the safety of WWE fans, increased security measures were put in place tonight. We apologize that it may have taken some fans longer than usual to get into AT&T Stadium.”

Several other news outlets reported the problem, with all seeming to peg the issue on a Wi-Fi problem with the ticket scanners. Winborn, however, said there were no Wi-Fi outages in the stadium or with the ticket scanners.

According to the Fort Worth Star-Telegram report, the vitriol on Twitter and other social media outlets may not have been entirely correct; according to the report:

Do not believe the viral reports and unsightly online pictures that fans could not get to their seats by the time the early matches began — they could but they preferred to walk around, buy food, drink and merchandise. But the images and the complaints that flooded the Internet about #WrestleMania32 were so plentiful, and negative, joint press release was issued halfway through the event.

With 101,763 finally in their seats the event was the fifth-largest crowd inside AT&T Stadium, and as such most likely produced a large Wi-Fi and DAS traffic number once things finally got underway. We will update this post and have another when we get the wireless traffic stats from the stadium.

Filed Under: 4G LTE, AT&T, Cisco, Dallas Cowboys, DAS, NFL, VenueNext, Wi-Fi Tagged With: 4G LTE, AT&T, AT&T Stadium, DAS, Levi's Stadium, Mobile Sports Report, Paul Kapustka, Stadium Wi-Fi, Twitter, Wi-Fi, WrestleMania, WrestleMania 32, WWE

Betting the Under (Part 2): Putting Wi-Fi antennas under seats is the hot new trend in stadium wireless networks

March 29, 2016 by Paul Kapustka Leave a Comment

Under-seat Wi-Fi AP at Levi’s Stadium. Photo: Paul Kapustka, MSR (click on any photo for a larger image)

Part 2 of this story picks up with the decision to put Wi-Fi APs under seats at Levi’s Stadium. If you missed it, here is the link to Part 1.

According to Chuck Lukaszewski, now vice president of wireless strategy and standards at Hewlett Packard Enterprise (formerly very high density architect in the CTO Office of Aruba Networks), Aruba had been testing under-seat AP designs since around 2010, “in one form or another.” There were some initial tests of under-seat AP deployments at Turner Field in Atlanta and at American Airlines Arena in Dallas, but nothing on the scale of AT&T Park’s 2013 deployment, or on the scale Aruba planned to have at Levi’s Stadium when it opened in 2014.

Some of the first under-seat Wi-Fi deployments in other arenas were actually deployed completely under the stands, Lukaszewski said, with signals shooting up through the concrete. Though he said “you could get reasonably good throughput through concrete,” especially for 2.4 GHz frequencies, installing antennas above the concrete was “considerably better,” Lukaszewski said.

Curiously, one of the biggest problems in stadium Wi-Fi deployment — especially for those heavy on overhead antenna use — is negotiating interference between antennas; sometimes, clients can “see” antennas and APs that are across the stadium, and will try to connect to those instead of the AP closest to them, a problem that leads to inefficient bandwidth use. Interference also means you can’t place APs too closely together, making it somewhat of an art to find ways to increase coverage without increasing interference.

Dan Williams, former VP of technology for the San Francisco 49ers, talking networking at Levi’s Stadium. Photo: Paul Kapustka, MSR

What Aruba found in its testing, Lukaszewski said, was that under-seat Wi-Fi AP deployments could be far more dense than overhead-centric designs, mainly because the human bodies in the seats would provide beneficial “blocking” of signals, allowing network designers to place APs more closely together, and to be able to re-use the same Wi-Fi channels in more antennas.

“If you can use human bodies to contain signals, you can have much smaller cells,” Lukaszewski said. Under-seat deployments, he said, “allows us to re-use the same channel less than 100 yards away.”

With more channels available for each AP, the difference in the metric Lukaszewski calls “megabytes per fan” can be “profound” for an under-seat design versus an overhead design, he said.

“We do see trends [in stadium network data] of under-seat being able to deliver well over 100 MB per fan per event, while overhead designs [deliver] significantly under 100 MB per fan per event,” said Lukaszewski.

Dan Williams, the former vice president of technology for the San Francisco 49ers, said he and Lukaszewski were in agreement that under-seat was the best method to deploy at Levi’s Stadium.

Kyle Field at Texas A&M. White spots in stands are under-seat AP locations. Photo: Paul Kapustka, MSR

“I just did not believe in overhead,” said Williams, who said he brainstormed with Aruba’s Lukaszewski on the under-seat idea, which they both brought to the Wi-Fi design at Levi’s. By using under-seat APs, Williams said, the Levi’s Stadium design looked to provide “cones [of bandwidth] around the audience, immersing [fans] in a signal.”

After beating the previous year’s Super Bowl Wi-Fi total at its NFL regular-season opener in 2014, Levi’s Stadium’s Wi-Fi network more than passed its biggest test ever this year, carrying a record 10.1 terabytes of Wi-Fi data during Super Bowl 50. Those numbers are proof of Lukaszewski’s claim: “By far, under seat is better.”

New deployments trending to under-seat

Editor’s note: This excerpt is from our latest STADIUM TECH REPORT, our long-form PDF publication that combines in-depth stadium tech reports with news and analysis of the hottest topics in the world of stadium and large public venue tech deployments. Enjoy this PART 1 of our lead feature, or DOWNLOAD THE REPORT and read the whole story right now!

Even though under-seat deployments can be considerably more expensive, especially in a retrofit situation where deployment requires coring through concrete, many stadiums are now seeming to agree with another Lukaszewski claim, that “the return absolutely justifies the investment.”

At AT&T Stadium in Arlington, Texas, the Cowboys quicked followed their sister park’s lead and installed under-seat APs in force ahead of that venue’s hosting of the inaugural College Football Playoff championship game in January of 2015. John Winborn, chief information officer for the Dallas Cowboys Football Club, said the team worked with AT&T’s “Foundry” innovation centers to produce a smaller, sleeker under-seat AP enclosure that fit well with the stadium’s commitment to aesthetics.

Back on the baseball side, the Giants now have 1,628 Wi-Fi APs in their park, with the vast majority of them under-seat, in all three decks of seating. And the Giants’ main rival to the south, the Los Angeles Dodgers, also used under-seat APs in a recent Wi-Fi upgrade.

Close-up of conduit running to under-seat AP at Kyle Field. Photo: Paul Kapustka, MSR

And if Levi’s Stadium led the way for under-seat Wi-Fi, the new mainly under-seat network at the refurbished Kyle Field at Texas A&M might be the QED on the debate, with ultra-fast network speeds and big data-consumption numbers (including 5.7 TB of Wi-Fi at a game versus Alabama) adding measureable momentum to the under-seat trend. Bill Anderson, CEO of Wi-Fi deployment strategy firm AmpThink, said he was an early disbeliever in under-seat Wi-Fi — until he saw the numbers.

“At first we mocked it, made fun of it,” said Anderson, whose firm has been called in to produce Wi-Fi network designs for several recent Super Bowls, as well as for the Kyle Field design. But when Aruba showed AmpThink the data from under-seat tests and deployments, “that was the ‘a-ha’ moment for us,” Anderson said.

Working with Aruba at Kyle Field, AmpThink was able to collect its own data, which convinced Anderson that under-seat was the way to go if you wanted dense, high-performing networks.

“The really important thing is to get APs closer to the people,” said Anderson. “That’s the future.”

Anderson said some doubters may remain, especially those who try to mix a small amount of under-seat APs with existing overhead deployments, a recipe for lowered success due to the potential interference issues. At Texas A&M, Anderson said AmpThink was able to build a design with far less interference and much greater density than an overhead solution, producing numbers that people have to pay attention to.

“We only know what we’ve observed, but we’re evangelistic supporters” of under-seat designs, Anderson said. “If someone says to you under-seat is hocus-pocus, they’re not looking at the data.”

Not for everyone, but more are trying under-seat

Though proponents of under-seat Wi-Fi all agree on its ability to deliver denser, faster networks, they all also agree that under-seat can be considerably more costly than overhead Wi-Fi, especially in a retrofit situation.

In addition to having to core through concrete seating areas to get conduit to the under-seat APs, the devices themselves need to be sealed, to guard them from weather, drink spills, and the power-washing equipment employed by most stadiums to clean seating areas.

Aruba’s Lukaszewski also noted that under-seat deployments generally use more linear feet of cabling to connect the APs than overhead, which also drives up the cost. Then since under-seat designs tend to use more APs, that also means a higher budget to cover a higher number of devices.

A row shot of the under-seat APs at AT&T Stadium. Photo: Dallas Cowboys

For some stadiums, the construction materials used prohibit the under-seat option from even being tried. At the Green Bay Packers’ legendary Lambeau Field, a late-1950s construction design that used lots of concrete and rebar — as well as part of the stadium’s bottom sitting directly in the ground — meant that under-seat Wi-Fi wasn’t an option, according to Wayne Wichlacz, director of information technology for the Packers.

Other stadiums, like the University of Nebraska’s Memorial Stadium, don’t have enough space between the stadium’s bleacher seats and the floor for under-seat APs to be safely installed. And many schools or teams simply don’t have big IT budgets like the $20-million-plus available to Texas A&M that allowed the Kyle Field design to seek the best result possible.

But many of the new stadiums under construction, as well as existing venues that are planning for new best-of-breed networks, have already committed to under-seat Wi-Fi designs, including the Sacramento Kings’ Golden 1 Center, where Ruckus Wireless will implement its first under-seat stadium Wi-Fi network.

Steve Martin, senior vice president and general manager at Ruckus, said the Golden 1 Center design, planned to be the most dense anywhere, will “primarily be underseat,” a choice he said “helps in a lot of ways.”

Foremost is the performance, something Martin said Ruckus has been testing at the Kings’ current home, the Sleep Train Arena. “It [under seat] does give you the isolation for frequency re-use,” he said.

The under-seat design also makes sense in Golden 1 Center since the stadium’s overall design is very open, with lots of glass walls and unobstructed views.

And under-seat deployment is even making inroads into the distributed antenna system (DAS) world, with Verizon Wireless implementing more than 50 under-seat DAS antennas at Levi’s Stadium prior to Super Bowl 50. Mainly installed to cover the bottom-of-the-bowl rows, the under-seat APs helped Verizon manage a record day for DAS traffic, with 7 TB reported on its in-stadium cellular network during the game.

“To get a quality signal, we had to go under seat,” said Brian Mecum, vice president, network, for Verizon Wireless, who said that in that area of the stadium, under seat was the only way to get a quality signal close to the subscriber’s phone. Verizon, he said, helped design the under-seat DAS antenna, and is looking to deploy it in other stadiums soon.

“It’s the first of more,” he said.

END PART 2… HERE IS THE LINK TO PART 1… TO READ THE WHOLE STORY NOW, DOWNLOAD OUR REPORT!

Filed Under: 4G LTE, AT&T, Cisco, Dallas Cowboys, DAS, Hewlett-Packard, Levi's Stadium, NCAA Football, NFL, Sprint, Super Bowl, T-Mobile, Verizon Wireless, Wi-Fi Tagged With: 4G LTE, Aruba, Aruba Networks, AT&T, AT&T Stadium, Cisco, Dan Williams, DAS, Extreme Networks, Kyle Field, Levi's Stadium, Mobile Sports Report, NCAA, NFL, Paul Kapustka, San Francisco 49ers, Stadium Wi-Fi, Super Bowl, Super Bowl 50, Verizon Wireless, Wi-Fi

S.F. Giants add more Wi-Fi, ‘virtual reality experience’ to AT&T Park for 2016 season

March 29, 2016 by Paul Kapustka Leave a Comment

The view from AT&T Park’s left field corner. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

The first ballpark to bring Wi-Fi to its fans is still padding its networking lead, as AT&T Park will have 543 new or upgraded Wi-Fi access points for the 2016 season, according to the San Francisco Giants.

Most of the new APs are of the under-seat variety, completing the team’s three-year plan to put more APs under seats to increase network density and capacity. According to Bill Schlough, senior vice president and chief information officer for the Giants, the park now has a total of 1,628 Wi-Fi APs, the most of any MLB stadium and more than most big football stadiums as well. With 78.2 terabytes of data used during the baseball season and another 20+ TB used during other events, Schlough said AT&T Park’s Wi-Fi network carried more than 100 TB of data in calendar 2015.

Since it’s an even year, the Giants expect to win the World Series again, so the action on the field should be pretty good. If you want to leave reality, however, the Giants can accomodate you in that realm this season with the addition of a “virtual reality experience” at the team’s @Cafe social media spot, located on the concourse behind the left-field bleachers.

Since it’s an even year, does that mean another one of these is on order for the Giants?

According to the Giants, fans can be “transported” to Scottsdale Stadium to view practice from spring training, or they can see views from the AT&T Park field, the batting cages and “even Sergio Romo’s car” through a VR headset.

The Giants said fans will also notice an upgrade to the stadium’s LED ribbon boards, which circle the park on the facings of the upper decks. The new Mitsubishi screens, the Giants said, offer 150 percent more pixels than their predecessors, meaning that you might not need those reading glasses to get stat updates or read advertising messsages.

On the DAS side of things, AT&T Park finally has all four major U.S. wireless carriers on its in-house cellular network, with the DAS and Wi-Fi serviced by 13 1-Gbps backbone pipes from AT&T.

Filed Under: 4G LTE, AT&T, Baseball, Cisco, DAS, Major League Baseball, San Francisco Giants, Wi-Fi Tagged With: 4G LTE, AT&T, AT&T Park, Bill Schlough, Cisco, DAS, MLB, Mobile Sports Report, Paul Kapustka, San Francisco Giants, Stadium Wi-Fi, under seat Wi-Fi, Virtual Reality

Betting the Under: Putting Wi-Fi antennas under seats is the hot new trend in stadium wireless networks

March 28, 2016 by Paul Kapustka Leave a Comment

White box at bottom is one of the under-the-seat Wi-Fi access points at AT&T Park. Photo: Paul Kapustka, MSR (click on any photo for a larger image)

What do you typically find under stadium seats on game days? The traditional list might include bags and purses, and get-out-of-the-way items like empty popcorn tubs, used hot dog wrappers and drink cups no longer filled with fluids.

And now you can add Wi-Fi access points and DAS antennas to the list.

A growing trend is emerging to use under-seat antenna placements to bring wireless signals closer to fans, for both Wi-Fi networks as well as cellular distributed antenna system (DAS) deployments. First used to compensate for a lack of overhang or railing placement spaces, under-seat deployments are now winning favor in all sorts of arenas for their ability to use human bodies to help build a more dense network, one that proponents say can carry far more capacity than an infrastructure that relies mainly on overhead antenna placements.

With proof points emerging quickly at venues like the San Francisco 49ers’ Levi’s Stadium and Texas A&M’s Kyle Field, as well as at pioneers AT&T Park and AT&T Stadium, under-seat Wi-Fi deployments may soon become more common, as more integrators and equipment suppliers embrace the under-seat method.

New stadiums under construction including the Sacramento Kings’ Golden 1 Center as well as new Wi-Fi deployments at existing stadiums like Houston’s NRG Stadium and the Bank of America Stadium in Charlotte are also planning to primarily use under-seat Wi-Fi deployments, both for the performance and aesthetic benefits. With such high-profile deployments embracing the method, under-seat APs may become the default placement position going forward, especially as stadium mobile-device usage by fans keeps growing.

History: a need required by architecture

Giants senior VP and CIO Bill Schlough, at the office

When Wi-Fi first arrived in stadiums, the obvious solution to questions about antenna and access point placement seemed evident — just mount them on ceilings, overhangs and walls, like they had always been placed historically. Mostly that decision kept the antennas out of sight, and provided good-enough reception for most network deployments.

But as fan Wi-Fi usage started growing, poor reception areas cropped up, most often in the most expensive seats near the courts or playing fields, where there was often little architectural infrastructure other than the seats themselves. At the San Francisco Giants’ AT&T Park, the stadium where fan-facing Wi-Fi was first installed in 2004, the need for more bandwidth was a big problem that needed to be solved during the 2012-13 offseason, after the team’s second World Series title run in two years had produced record wireless usage.

Even with a Wi-Fi AP placed just about everywhere they could be, the San Francisco Giants’ IT team couldn’t keep up with demand. And the trick that has been tried at some stadiums — putting AP enclosures on handrails — wasn’t an option at AT&T Park, since its lower-bowl seating areas have no railings.

With options limited, that’s when an internal battle commenced around the new idea of placing APs under seats, a plan that met fierce resistance on many fronts.

“We got beaten up pretty bad over the idea [of under-seat APs],” said Bill Schlough, senior vice president and chief information officer for the Giants, who described the 2012-13 offseason as “a very stressful time,” with lots of internal strife and discord. With multiple stakeholders checking in on the plan, including the Giants’ facilities group, the marketing group and the ticketing group, concerns about the loss of under-seat space and the potential health concerns fueled opposition to putting APs under chairs.

But without any railings or overhangs for most of the park’s lower-bowl seats, Schlough and his team had “no other alternative” than to try placing Wi-Fi APs under seats. On the possible health issue, Schlough said the Giants were assured by technology partner (and ballpark title sponsor) AT&T that the deployment would be safe and comply with all FCC regulations; “We were assured that having [an antenna] 18 inches from your butt was the [radio] equivalent of having a cell phone in your pocket,” Schlough said.

On the storage-space concern side, Schlough said the Giants’ IT team made models of the antennas out of cardboard and duct tape, and placed them under seats to see how they worked.

“The [walking] flow through the aisles was good, with the AP models tucked under we never kicked them” during testing, Schlough said. With AT&T assuring the Giants that under-seat was “the way of the future,” the team took a leap of faith and added a large number of under-seat Wi-Fi APs in preparation for the 2013 season, more than doubling the number of APs in the park (to 760 total) in the process.

Under-seat Wi-Fi enclosure at Dodgers Stadium. Photo: Terry Sweeney, MSR

Though Schlough and his team “spent a lot of time” communicating with season-ticket holders about the new technology, there was still consternation about what might happen when opening day arrived, and “fans find this box under their seat, and not have a place to put their garlic fries,” Schlough said.

As it turns out, there was almost no resistance to the method; according to Schlough the Giants only had two complaints about the under-seat APs that first day of deployment, which Schlough called “the biggest relief day of my life.”

The success of the under-seat idea was particularly noted at that time by another IT team in the Bay area, the one putting together the wireless plan for the San Francisco 49ers’ new home, Levi’s Stadium, which was being built just to the south in Santa Clara. Testing some under-seat placements of their own at Candlestick Park during that venue’s final season as the Niners’ home, the team building the Levi’s Stadium network became convinced that going under seat was the best way to build the high-density deployment they wanted to have.

END PART 1… HERE IS THE LINK TO PART 2… TO READ THE WHOLE STORY NOW, DOWNLOAD OUR REPORT!

Filed Under: 4G LTE, AT&T, Baseball, Cisco, DAS, Levi's Stadium, Major League Baseball, San Francisco Giants, Wi-Fi Tagged With: 4G LTE, AT&T, AT&T Park, Bill Schlough, Cisco, DAS, Levi's Stadium, MLB, MLBAM, Mobile Sports Report, Paul Kapustka, San Francisco Giants, Stadium Wi-Fi, Wi-Fi

University of Wisconsin takes on Wi-Fi, Badger Game Day app upgrades

March 21, 2016 by Terry Sweeney Leave a Comment

Camp Randall Stadium, University of Wisconsin. Photo: Dave Stluka

Sports fans at the University of Wisconsin have been enjoying a nice technology two-fer for the last 20 months: In addition to new Wi-Fi and beacon technology at its largest sporting venues in Madison, Wisc., the university also released v3.0.2 of its Badger Game Day app which adds live and archived video, among other features, for fans and their smartphones.

Jim Roberts, director of technical services for the university’s athletic department, described this as a happy coincidence as opposed to a larger strategy to bring sports technology to the Badger faithful. “Knowing that the new Wi-Fi system was coming, the group working on the app upgrade was able to incorporate more features, knowing fans could take advantage of the improved Wi-Fi and not rely solely on cellular data plans,” Roberts said.

Editor’s note: This profile is an excerpt from our latest STADIUM TECHNOLOGY REPORT, which is available for FREE DOWNLOAD from our site. In addition to this stadium tech deployment profiles we also take an in-depth look at the new trend of deploying Wi-Fi and DAS antennas under seats, and provide a wireless recap from Super Bowl 50. GET YOUR COPY today!

The Wisconsin venues are Camp Randall, a bowl-style football stadium with a capacity of 80,321; and nearby Kohl Center, used for hockey, basketball, concerts and other live events with room for 17,230. The LeBahn Arena, built for women’s ice hockey with a capacity of 2,273, is also included. In part because of their proximity, Roberts and his team used the upgrades to replace and enhance the underlying infrastructure for the venues – core switching, Wi-Fi access points, an IPTV system, cabling, electrical power and HVAC improvements — $11 million for the whole package, according to Roberts.

“Due to the expected size of the population connecting to Wi-Fi, we had to upgrade the entire network,” he explained, adding that the previous 10/100 Mbps backbone with Gigabit Ethernet uplinks and its 32,000 MAC address capacity was insufficient for the job.

“We upgraded our core to some pretty big Cisco routers at each venue that could handle 128,000 MAC addresses, with 10-gigabit fiber to all 33 telecom rooms within the Camp Randall complex,” he said; they also added about 1,100 wireless APs. Camp Randall got upgraded during the summer of 2014; the Kohl Center and LeBahn were done a year later.

Kohl Center

Camp Randall proved to be the largest test, both from an engineering and design perspective. Built in 1917, its open bowl lacks the overhangs from which RF engineers love to hang antennas and other infrastructure.

“The east side of the bowl became our biggest challenge with getting the signal to penetrate deep enough into the sections,” Roberts said, adding that the problem was especially acute for seats closest to the field, where the first few rows are tarped over. Initially, APs were installed below the tarps, but the signal only carried 10 rows back.

“We ended up mounting the APs on the front, 4-6 feet up from ground level,” and above the tarps, he explained. “They don’t affect the sight lines for spectators. But getting the APs to shoulder height from waist height definitely helped us get it back to row 25.”

APs were also mounted just above the entry tunnels, where the hardware and antenna could be attached to railings and concrete. Cisco is the University of Wisconsin’s AP vendor; the deployment uses Cisco model 3700s.

Wi-Fi install over a VOM at Camp Randall (click on photo for a larger image)

Roberts and his team also ran into some structural issues with waterproofing and cabling that kept them from putting in more APs in the student section. They had to re-calculate where the APs would go; consequently, coverage can be spotty in the student section, which is exacerbated by the high density of phones in that part of the stadium. AT&T and Verizon both have DAS infrastructure in Camp Randall that helps coverage, but Roberts and his team are looking at long-term solutions for Wi-Fi coverage in that section and throughout Camp Randall.

The University of Wisconsin worked closely with AmpThink on a facility-wide Wi-Fi analysis, according to Bob Lahey, a network engineer in the athletic department. AmpThink did the design and tuning and worked out some issues in advance. “Our facilities staff and [AmpThink] discussed locations for best coverage and worked through the aesthetics before we started the project,” Lahey said. AmpThink was also onsite during the first year to see how the Wi-Fi performed with people in the bowl. “You can only figure out so much without people there,” Lahey laughed.

Getting Online at Camp Randall

The stadium’s fan-facing wireless network, Badger WiFi, is a captive portal that asks users for their name, email address and zip code. There are also two boxes: one, users must check to agree to terms and conditions of service; the second allows the university to send them emails, and by default, the second box is checked. “Our plan is to send them email surveys and allow them to remain on the system and not have to re-authenticate every time they come to one of our buildings,” Lahey said. “But if they uncheck, they have to re-authenticate.”

The university does no bandwidth limiting or throttling back usage once users are logged in. “We’ve got dual 10-gigabit links and 100-gigabit to the world, so we’re not too concerned about overall bandwidth,” Lahey said. “We limit each radio in the AP to a maximum of 200 clients. It doesn’t happen often, but we see it occasionally.” Camp Randall users normally get at least 1 Mbps bandwidth — plenty for checking scores or posting to social media, Lahey added. Kohl Center users average 40-60 Mbps because the venue is less dense.

Screen shot of Wi-Fi portal login

At present, 65-70 percent of Badger Wi-Fi clients are on 5 GHz spectrum rather than 2.4 GHz. Roberts finds the 5 GHz band easier to manage, and said users get a better experience. “If we have problems with wireless, it is most times an older couple with their iPhone 4,” Roberts said. “APs can only do so much, but sometimes a phone [using 2.4 GHz spectrum] will want to connect with an AP a half mile across the field rather than one that’s 10 feet away.”

He also said the maximum number of unique clients for Camp Randall is about 26,000, or 37 percent of the crowd. “We assume that’s going to keep growing and we’ll have to augment the system,” he said. “At some point we won’t have enough access points.”

Game Day Gets a Badger Refresh

Concurrently, the Badger Game Day smartphone app was getting new features like live video replay and interaction with Bluetooth-based beacon technology. The app’s first iteration was initially for football, then expanded to all 10 sports that sell tickets; the latest version embraces all 23 sports at the University of Wisconsin, men’s and women’s. “Not many schools have all their sports represented, so while the traffic may not be high on rowing, it’s a great recruitment tool,” said Ben Fraser, director of external engagement for the athletics department. “So it helps there with the coaches sending out links or for parents and other supporters.”

It also helps with fans. “Collegiate and professional sports venues are looking for how to keep fans entertained and also allow them to participate in the game via social media and other methods,” noted Tam Flarup, director of the athletic department’s website services. When there’s break in the action, Badger fans are busy posting to Facebook, Instagram and of course, Wisconsin’s infamous Jump Around. “Twitter’s also allowing Periscope live video in its tweets now,” Flarup added. “Our fans will like that – it keeps them in the stands with a great game day atmosphere and experience.”

The university developed the first two iterations of Badger Game Day internally but chose to outsource the upgrade to sports-app developer YinzCam in June 2015 and gave them a tight deadline to meet — Aug. 30, just in time for Badger football season. YinzCam delivered on time, and then met an Oct. 15 deadline for revisions and tweaks, Fraser said.

Badger Game Day now includes live video replay from four different camera angles; YinzCam’s secret sauce makes streaming video across Wi-Fi more efficient. “Video would have been impossible without the Wi-Fi investment we made,” Fraser said.

Unlike previous iterations that only allowed the participation of a single sponsor, the new Badger Game Day app gives the university the ability to sell individual pages and sports, Fraser said.

Game day beacon message to app

Perhaps the leading edge of Badger Game Day is its use of Bluetooth-based beacon technology and messaging with geo-fencing. Gimbal Inc. worked with the university customize the technology; Fraser and his team did some social media messaging to alert fans to the feature and to remind them to turn it on.

The first remote use of messaging with beacons and geo-fencing was in Dallas for Wisconsin’s season opener in Dallas at AT&T Stadium; the feature was then used continually at both Camp Randall and the Kohl Center.

“We continued to use this messaging on the road for the Holiday Bowl in San Diego,” Fraser said. “Messages varied from welcome messages that were linked to videos from our players, to informational messages that informed fans about events, to scavenger hunts that engaged our fans at these sites.”

When users first download the app, there’s a proximity allowance message that they must activate to receive beacon messages. So far, the university has sent out 46 unique messages, 21 of which were geo-fenced. At each home game, they geo-fence Camp Randall with a welcome video from players; they reached an average number of 1,160 fans per game with these welcome messages and videos.

“We’re still learning how fans are using [beacons and Bluetooth], and we’re trying not to hit them with too many ads,” Fraser said. By building their trust, it encourages fans to leave their Bluetooth on for the signal to find them. “And we are looking for ways to improve it,” he added. Potential future additions: Features that show the length of lines at concession stands and restrooms, and an online lost and found. They’re also looking for more robust scheduling information inside the app — such as which broadcast network is carrying the game, along with links to Wisconsin’s video stream and live stats.

App development and a new server cost the university about $100,000, according to Fraser and Flarup. Since August 2015, there have been 123,000 downloads of Badger Game Day and nearly 1 million page views. Average time spent per game on the audio feature of the app is about 14 minutes. There’s more room to grow as fans continue to download and use the app; there’s plenty of revenue upside as well as sponsors discover multiple avenues for their messaging and content.

Filed Under: Apps, AT&T, Basketball, Cisco, DAS, NCAA Football, NCAA Hoops, Verizon Wireless, Wi-Fi, YinzCam Tagged With: Apps, Badgers, Beacons, Camp Randall Stadium, Cisco, DAS, Mobile Sports Report, NCAA, Paul Kapustka, Stadium Wi-Fi, University of Wisconsin, Wi-Fi

Stadium Tech Report: Connectivity soars at Denver Broncos’ Sports Authority Field at Mile High

January 29, 2016 by Paul Kapustka Leave a Comment

Panoramic view of Sports Authority Field at Mile High from the top seats. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

On most of our stadium visits, we have to wait until we get inside the venue to start testing the network. At Sports Authority Field at Mile High, however, we barely got out of the car before the Wi-Fi auto-connected — at superb speeds. Nothing like a network that announces itself before you get in the door.

The parking-lot connection — at a download speed of 45.48 Mbps and an upload speed of 53.35 — was the first clue that football fan connectivity is taken seriously in Denver, especially so if you have Verizon service. While the stadium’s Wi-Fi network is currently only available to Verizon customers — more on this later — full DAS participation by the three other major U.S. wireless carriers means that pretty much any visitor to the venue is going to have good, if not great, connectivity for their mobile device, no matter which service they use.

Inside the stadium, a trained eye can spot many different types of DAS and Wi-Fi antenna placements, under overhangs, on towers, on ceilings and on walls; and thanks to a first-person stadium tech tour conducted by Russ Trainor, vice president of information systems for the Denver Broncos, we got to learn about a wide range of not-so-noticable antenna deployments, including in railing enclosures and on field-level walls, all part of an ongoing plan to try to stay ahead of the still-growing demand for mobile data from sports fans who come to the games.

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

The day we visited, during the last regular-season game on Jan. 3, was important for the Broncos as a team since their 27-20 victory over the San Diego Chargers gave Denver home-field advantage through the playoffs, an edge that helped the team reach its eighth Super Bowl. But even as he celebrated his team’s win, Trainor was happy for another reason: the bye week gave him and his team more time to light up some new Wi-Fi and DAS antenna placements, to better handle the expected and eventual playoff data crush.

“You can never have enough APs,” Trainor said.

Good Wi-Fi, but still only for Verizon customers

Opened on Aug. 11, 2001, with a concert by the Eagles, the then-named Invesco Field at Mile High replaced the old Mile High Stadium in basically the same spot, sitting at 5,280 feet above sea level. Seen by many on TV when it hosted the 2008 Democratic National Convention and the acceptance speech of then-Sen. Barack Obama, the “new” Mile High has seen more than 12 million fans come through its doors since it opened for a variety of sports and entertainment events.

But true high-speed wireless for fans didn’t take root until 2012, when a revamp led by Verizon Wireless and the Broncos’ IT staff added a Cisco-based Wi-Fi network to the stadium with 500 access points, designed to serve 25,000 concurrent users and also designed to be “open,” allowing any other carrier to provide access to its customers by negotiating a deal with Verizon. While Trainor said the option still remains open and talks with some of the other carriers are underway, none have yet signed on — making the Wi-Fi network a fast playground for Verizon customers, who apparently are in the vast majority in the Denver region.

Sorry, AT&T customer, no soup for you

We don’t have any exact proof of that thinking, but statistics from the recent AFC Championship game at Sports Authority Field — a 20-18 Denver victory over the New England Patriots — seem to show Verizon customers in a bit of a majority. According to Verizon, its customers at the game used a total of 2.87 terabytes of data, with 1.7 TB on the Wi-Fi network and another 1.17 TB on the Verizon LTE DAS network. AT&T, by comparison, said its customers used 819 GB on the AT&T DAS network that day. So either there are more Verizon customers at the stadium on game days, or Verizon customers use more data because they have more network options; take your pick.

With our Verizon iPhone 6 Plus in hand, we found great connectivity on Wi-Fi pretty much everywhere we roamed. After finding our way from the parking lot to the press box, we got a signal of 46.46 Mbps down and 46.90 up, this from the regular fan network in the stands and not from the press-only Wi-Fi network.

While roaming through the plush United Club we got a speed test of 33.36/35.19, a figure that Trainor said could change on any given game day — “when it gets cold outside, this place fills up,” he noted — and then later when we walked up to the top, 5th-level concourse, we still got a Wi-Fi signal of 34.96/30.40 on the walkways behind the seats. During second-quarter action we even sneaked up to the nosebleed seats in section 501, one of the ski-slope steep sections near the stadium’s top edge — and still got a Wi-Fi signal of 10.28 Mbps/5.00 Mbps.

According to Trainor, the upper seats are among the toughest challenges for Wi-Fi reception, especially those in the “bulge” areas in the middle of the stadium where on both sides the sections curve upwards, adding more seats. Though the light structures that wind all the way around the stadium do provide good spots for antenna mounts, the bulge areas are harder to reach, and in the near future Trainor and his team will keep experimenting with other methods of deployment, like railing enclosures and row-end mounts they have used successfully for both Wi-Fi and DAS in other areas of the stadium.

Lots of antennas visible in this overhang area

One interesting architectural quirk of the stadium — its use of metal decking instead of concrete — actually helps the wireless deployment team, Trainor said. Installed to mimic the metal upper deck at the old Mile High Stadium — where Broncos fans would do the “Denver Stomp” to produce thunderous noise — the metal construction acts as a barrier to keep Wi-Fi signals from the bowl from interfering with those from antennas inside suites and concourses, Trainor said.

While most of the stadium has favorable locations for overhead antennas — there are three main levels of seating, providing two expansive overhangs covering about 80 percent of the seating area — some typical problem places like seats near field level and in the no-overhang South stands have required some creative thinking, an excercise that never really ends.

“We started with 500 Wi-Fi APs, and we’re now at 640, and by the time we get it [the current plan] all built out we’ll have about 850 to 900 total,” Trainor said.

DAS deployments a mix of connectivity

On the DAS side, Trainor said that the four major carriers — Verizon Wireless, AT&T, Sprint and T-Mobile — are all present inside the stadium, with different antenna placements in different numbers. In some instances, all the carriers use “neutral” antennas, mainly in areas where there isn’t enough room for exclusive deployments. But in other areas, the carriers have installed their own antennas, an arrangement that allows them to replace and upgrade them as necessary at their own discretion, Trainor said.

Field-level Wi-Fi AP (small white box next to right leg of Peyton Manning fan)

We didn’t have a Sprint or T-Mobile device on hand, but our AT&T Android phone had good connectivity everywhere we measured, including a 4G LTE signal of 27.94 Mbps down and 6.86 up in the press box, and signals of 47.83/6.37 on the same 5th-level concourse area where we tested the Verizon Wi-Fi.

All the carrier back-end gear is housed in a brick building built outside the southeast side of the stadium, Trainor said, since there wasn’t room inside the stadium structure itself. DAS and Wi-Fi antennas also exist in great number in the vast parking lots that directly surround the stadium, as well as in the “fan zone” gathering area outside the South stands.

Like with the Wi-Fi, Trainor and his team are always planning for more DAS capacity, even if contracts aren’t signed yet. On the new railing enclosures they are installing, the Denver IT team builds in enough space for both DAS and Wi-Fi, even if only one network is using the deployment to start with. Again, you can never have enough antennas — or enough places to put them.

YinzCam app and Cisco SportsVision

Rounding out the mobile-device offerings is not one but two YinzCam team apps, one for use at outside the stadium and the other one for live game-day offerings, with a geocache feature that allows the team to provide content it has stadium rights to, like the NFL’s RedZone channel. Both apps have live links to the Broncos radio coverage from KOA Radio, and the in-stadium instant replay feature worked superbly during our visit, showing plays in seconds and often before they appeared on the stadium’s big screens.

In the concourses we recognized the split-screen capabilities of Cisco’s StadiumVision technology, which can direct programming to all the TV screens inside a stadium. Another nice touch in the United Club was a circular charging station, with tabletop space so fans could have a place to put food and drink while waiting for their devices to juice back up. “We are always looking for ways and configurations to allow fans to recharge their devices,” Trainor said.

With all its different parts, the wireless deployment at Sports Authority Field at Mile High adds up to a favorable fan experience, one that clearly has the ability to keep getting better on an incremental basis. But like their Super Bowl team, Denver fans should be happy with what they have right now.

MORE PHOTOS BELOW