Stadium Tech Report: Los Angeles Dodgers hit it out of the park with Cisco, Aruba Wi-Fi

Dodgers Stadium, the SoCal baseball shrine. All photos: Terry Sweeney, MSR (click on any photo for a larger image)

Dodgers Stadium, the SoCal baseball shrine. All photos: Terry Sweeney, MSR (click on any photo for a larger image)

Growing up in the Los Angeles suburb of Norwalk, Ralph Esquibel recalled playing outdoors while inside the Dodger game was on the radio. “I knew from the kinds of noises coming out of the house how the game was going,” he laughed. Esquibel, now vice president of IT for the Los Angeles Dodgers, may have wished for some similar indicators or guideposts as he began the wireless retrofitting of Major League Baseball’s third oldest stadium (after Boston’s Fenway Park and Chicago’s Wrigley Field) in early 2011.

Esquibel faced multiple challenges with Dodger Stadium. First, there was all that concrete to push signals through or around. There was the size of the Chavez Ravine venue and its far-flung parking lots, spanning more than 350 acres. The stadium also has few overhangs, a favorite place to attach Wi-Fi access points or distributed antenna system (DAS) gear. Then there’s Dodger Stadium’s capacity — 56,000 seats – the largest in the league and almost 30 percent larger than the average MLB stadium (42,790).

Esquibel’s biggest hurdle? ” Trying to achieve the network that we wanted but also maintain an appropriate budget for the solution,” he said. While Esquibel would not specify what the Dodgers spent, he did allow that it was “an 8-figure project.”

Coverage challenges in the best seats

Initially, the best seats in the house presented a coverage challenge; field and club level seats along the third- and first-base lines and the dugout lack any overhangs. So while phones in those sections could receive a short, directional beam sent from across the outfield, the upstream signal couldn’t get back to the AP across the field, said Esquibel.

Ralph Esquibel, VP of IT for the Dodgers, with the new Wi-Fi relief pitcher mobile.

Ralph Esquibel, VP of IT for the Dodgers, with the new Wi-Fi relief pitcher mobile.

“We wanted to guarantee a premium experience, regardless of the seat,” said Esquibel, who joined the Dodgers 6 years ago after working in IT at Toyota and Honda. So by using what he calls “a hybrid approach,” Wi-Fi APs and antennas are installed overhead where possible, but also under seats and in staircase handrails that divide the stadium’s steep aisles.

All told, nearly 1,000 APs from Cisco and Aruba Networks blanket Dodger stadium, its concession areas and parking lots. Horizon Communications helped the Dodgers with design and installation of the Wi-Fi and DAS.

The under-seat APs/Wi-Fi antennas on the club level are housed in NEMA enclosures about every 15 seats, set eight rows apart. Esquibel was concerned about losing real estate under those seats; he also didn’t want to create any potential trip hazard for fans. In addition, the Dodgers use Cat 6A cabling, whose thickness and rigidity couldn’t run up a stepped incline. Consequently, they drilled through concrete to snake the cabling through from the clubhouse underneath. “There’s no visible conduit leading into the enclosure,” Esquibel explained. The profile and footprint of the enclosure still leaves space for fans to place belongings.

Handrail Wi-Fi enclosure

Handrail Wi-Fi enclosure

It’s the same modus operandi for the enclosures housed in the stair rails, except there are two APs in larger enclosures at the top of each staircase on the reserve level and upper deck, then a single AP per enclosure as the stairways descend. Some 290 APs offer coverage on the reserve level, which by itself has a greater capacity than nearby Staples Center (18,118 seats), Esquibel told Mobile Sports Report. After 2 years of use, there have been no issues with the AP enclosures. “We power-wash the seats and stands after games and [the enclosures] are very resilient against the sun, water and wind,” Esquibel said.

He also acknowledged some early challenges with Wi-Fi. Part of the issue was working with Cisco’s CleanAir technology, which is supposed to minimize RF interference, if not eliminate it altogether. If an AP starts broadcasting over a frequency in use by another AP, for example, CleanAir helps it find another frequency. It took a few months to fully tune the network; some directional antennas needed a 10-degree adjustment, Esquibel said. Another challenge was having APs from more than one vendor. “If your network is 100 percent Cisco and all leveraging the same controllers, [CleanAir] will work perfectly,” Esquibel said. “If you have a mixed environment that pushes Wi-Fi in certain locations, it becomes a problem — there’s competition for frequencies.”

Coordinating the APs

A third-party leveraging a non-public frequency would switch channels, for example, causing the APs for public use to also switch channels. “What we had was a lot of bouncing back and forth,” Esquibel said, which affected performance. “So we assigned channels and frequencies for each AP, which still requires a lot of coordination.”

Under-seat Wi-Fi enclosure

Under-seat Wi-Fi enclosure

Since 2013, the stadium has been carved into 24 DAS sectors. AT&T, T-Mobile USA and Verizon Wireless are the carriers presently using the DAS; Ericsson makes the DAS antennas. Stubborn Sprint relies on a tower adjacent to the stadium.

Dodger fans average anywhere from 500-655 megabytes of data use per game, according to Esquibel. During a busy game, the wireless networking accommodates 16,000 concurrent users; a slower event clocks in at 4,000-8,000. To test upload speed, Esquibel will push a 50MB video to Facebook. When there’s lots of available bandwidth, he gets 60 Mbps performance; on the low end, it’s closer to 4 Mbps. Esquibel said users are mostly streaming and posting videos and photos to social media; Dodger Stadium is the second most Instagrammed site in southern California, after Disneyland, Esquibel added.

The Dodgers have their own version of Ballpark, the in-stadium MLB app, which offers video replay and highlights; in-seat ordering of food and drink in certain areas; and stadium mapping. Check-ins on Ballpark are handled through a network of 44 iBeacons, which takes advantage of Bluetooth Low Energy (BTLE) technology. Between Ballpark and social media activity, Dodger fans have run up as much as 700 MB data usage during games — and the network is ready if more demand is needed.

“We don’t do any rate limiting, so if we consume all our bandwidth we get a free upgrade, thanks to a clause in our agreement with our ISP, AT&T,” Esquibel explained.

To ensure a family-friendly and wholesome environment, the Dodgers use Palo Alto Networks 5020 firewalls for content filtering. “As we developed our SLAs, it was one of the first issues to pop up — no sexual content, no malware/phishing, and no illegal drug sites,” he said.

What’s on his wish list for the future? “I’d like geo-fencing within the Wi-Fi network so if I see someone enter a club, I can say hi or welcome them, notify them of specials, or flag points of interest around the stadium,” Esquibel said, like the World Series trophy case or giveaway locations for promotional items. Alongside all the other applications, wireless can be used as guideposts for fans and visitors to Dodger Stadium.

Chicago Cubs tap NFL deployment expertise of Extreme, DGP for new Wi-Fi, DAS at Wrigley Field

Artist rendering of the proposed fan plaza outside Wrigley Field. Renderings courtesy of the 1060 Project.

Artist rendering of the proposed fan plaza outside Wrigley Field. Renderings courtesy of the 1060 Project.

The video boards above the historic ivy-covered outfield walls are only the first clue that this isn’t your grandpa’s Wrigley Field anymore.

And though you won’t be able to see it, new Wi-Fi and DAS networks are coming soon to the Friendly Confines, as part of the Ricketts Family’s ambitious remake of Wrigley Field and its surrounding area. And according to Cubs IT executives, the team is tapping firms with NFL stadium expertise to bring not just fast and thorough wireless coverage to fans, but also back-end ownership and analytics so that the team can more effectively track online activity to improve the fan experience as well as improve the team’s return on infrastructure investment.

Though Wrigley Field has had full fan-facing Wi-Fi for longer than most Major League Baseball stadiums — the AT&T-built network arrived in 2012 — with the major overhaul of not just the park itself but the surrounding areas outside beginning this offseason, it was time to rethink the team’s overall approach to wireless connectivity, said Andrew McIntyre, senior director of information technology for the Chicago Cubs.

As part of the team’s ongoing 1060 Project the Ricketts family (which owns the Cubs) is not only adding more concessions and other fan amenities to Wrigley, they are also building a fan plaza outside the main gate as well as building a retail/office building and eventually a boutique hotel on the edge of the famed ballpark property at Chicago’s somewhat slanted corner of Clark and Addison. (If you don’t get the “1060” label, we suggest you ask Elwood Blues what the address of Wrigley Field is.)

“As it all starts looking more like a campus, it changes the dynamics” of how you provide wireless coverage to all areas, said McIntyre. As a regular attendee, speaker and steering council member of the SEAT Conference — the premier gathering of stadium technology professionals — McIntyre was well aware of all the new trends for large-venue Wi-Fi and DAS deployments, some of which were taking place in football stadiums across the country.

“We understood what was happening with other leagues in regards to Wi-Fi and DAS from what we saw at SEAT,” said McIntyre, in an interview at this summer’s SEAT Conference in San Francisco. “We started to evaluate those deployments and ideas as we were getting ready for our restoration.”

The Winners: Wi-Fi with a heavy side of analytics, and team-owned DAS

Cubs fans know how to enjoy a day at the park. Photo: Lisa Farrell, MSR

Cubs fans know how to enjoy a day at the park. Photo: Lisa Farrell, MSR

As major construction took place this past offseason, the Cubs de-activated the AT&T Wi-Fi network that had previously served fans inside the ballpark. Even though it doesn’t sound very old, McIntyre notes that many other stadiums around the country have had to completely overhaul Wi-Fi networks built just several years ago, due to the ever-increasing demand for more bandwidth and the rapid introduction of new phones and devices that fans are bringing to games.

“AT&T had previously controlled both the DAS and the Wi-Fi, and [to them] the Wi-Fi was kind of a ‘check the box thing,’ ” McIntyre said. “The scope [of the network] was just for Wrigley Field only. When we took down the Wi-Fi while we replaced the bleachers, we looked more toward the future.”

What McIntyre and the Cubs IT team saw was a future where Wi-Fi was used not only to provide connectivity, but to also provide a deep link between venue owners and operators and the digital activities of their visitors, through advanced analytics of Wi-Fi traffic. In the end the Cubs selected Wi-Fi provider Extreme Networks for the Wrigley project, in no small part due to Extreme’s experience in deploying Wi-Fi networks and Wi-Fi analytics inside numerous NFL stadiums.

“We saw patterns emerging in other leagues, and especially in the NFL, where the league and teams called out analytics,” said McIntyre. Extreme, which has a partnership deal with the NFL as its preferred provider of Wi-Fi analytics for its Purview software, has provided analytics help at recent Super Bowls in addition to being part of stadium Wi-Fi deployments for the New England Patriots, the Philadelphia Eagles and the Seattle Seahawks, among others.

“A lot of times talk about Wi-Fi is simply about coverage and capacity, and more, more, more,” McIntyre said. “The question of ‘what are you doing with the service’ becomes an afterthought.” McIntyre noted that in some cases, the NFL has deployed Extreme analytics on top of Wi-Fi infrastructure with gear from another manufacturer. “What they [Extreme] are able to provide [with analytics] is night and day compared to the competition,” McIntyre said.

Back of the iconic Wrigley bleachers, circa 2014. Photo: Paul Kapustka, MSR

Back of the iconic Wrigley bleachers, circa 2014. Photo: Paul Kapustka, MSR

John Brams, director of Sports and Entertainment at Extreme Networks, called the coming Wrigley Field network “a signature deployment.” Wrigley Field itself is expected to have Wi-Fi service in time for the 2016 season, McIntyre said.

DAS: Neutral host instead of carrier-led

On the DAS side of the wireless equation, McIntyre and the Cubs team were impressed with the cellular network deployment at the San Francisco 49ers’ new venue, Levi’s Stadium, a deployment done by the lesser-known firm DAS Group Professionals, or DGP. While many may have first heard of DGP for its Levi’s Stadium deployment, DGP does have other large-venue experience, having built previous cellular networks for airports and the San Francisco Bay area’s BART light-rail service.

At Levi’s Stadium, DGP worked with the Niners to build a neutral-host DAS deployment that is owned and controlled by the team, an emerging trend for stadium owners and operators who don’t want to simply concede control to wireless carriers. Under a neutral-host deployment the owner or operator of the DAS typically builds a non-carrier-specific antenna infrastructure, and then charges wireless carriers to connect their systems to the back end of that network.

At a prior SEAT event McIntyre said the Cubs team talked to the Niners about why they went with DGP, and liked what they heard.

“The venue-owned DAS solution was a business model we liked,” McIntyre said, “It perfectly aligns with our strategy of being closer to the fan base and not one step removed.”

Steve Dutto, president of DGP, said the Cubs contract “validates our work at Levi’s Stadium.” The new DAS, McIntyre said, should be fully functional by 2017.

Artist rendering of the home plate view after all construction done.

Artist rendering of the home plate view after all construction done.

All-Star Wi-Fi: Cincinnati crowds used 4.3 TB over All-Star Game activities

Fans at All-Star Game taking pictures of Pete Rose. Photo: Screenshot courtesy Fox Sports/Cincinnati Reds

Fans at All-Star Game taking pictures of Pete Rose. Photo: Screenshot courtesy Fox Sports/Cincinnati Reds

Like a player added to the roster just before game time, the new Wi-Fi network at the Great American Ball Park in Cincinnati handled some all-star traffic levels, carrying a total of 4.3 terabytes of data over the three separate events that made up Major League Baseball’s All-Star Game festivities earlier this week, according to IT execs at the ballpark.

Though it only came online a couple weeks before the big event, the GABP Wi-Fi network held up admirably for the big game, carrying 2.36 TB during Tuesday night’s main event, according to Brian Keys, vice president of technology for the Cincinnati Reds. Almost another 2 TB was recorded during the ancillary events, the futures game and the Home Run Derby, proving once again that “big event” crowds like their Wi-Fi and are adept and finding and using in-stadium wireless networks. We don’t have DAS stats yet but it’s an easy guess that all four DAS deployments inside the stadium also carried significant traffic loads during the All-Star activities.

In a phone interview Friday, Keys said that the peak concurrent Wi-Fi user number hit 9,700 at one point during the actual All-Star Game, with a total of 12,000 unique Wi-Fi connections over all of Tuesday night. And even though the game attracts a national audience, the hometown fans provided the biggest traffic surges during Cincinnati Reds-specific moments — like at the end of Monday’s Home Run Derby when local hero Todd Frazier won in dramatic fashion, and when former Reds star Pete Rose had a brief pre-game introduction.

“Especially when Todd [Frazier] got up to bat, that really tested the limits of our [bandwidth] pipe,” Keys said. The Rose introduction, he said, put similar stress on the 576 Wi-Fi access points, but with Keys’ staff as well as a special group from Wi-Fi gear provider Cisco on hand to help out, the new network performed in big-league fashion, Keys said.

During construction, the IT team had to overcome one structural hurdle, namely the lack of any railings in the lower bowl to mount Wi-Fi APs on. Keys said some of that was solved by putting APs at the bottom of seating rows pointing up, and using overhang space for other antenna mounts. The Great American Ball Park did not use any under-seat APs, Keys said.

Pete Rose. Photo: Screen shot of Fox Sports broadcast courtesy of Cincinnati Reds.

Pete Rose. Photo: Screen shot of Fox Sports broadcast courtesy of Cincinnati Reds.

Though the ballpark had explored putting Wi-Fi in last season, the initial deployment was stalled last summer due to what Keys called contract issues. But with the All-Star game coming this season, the park re-started its Wi-Fi deployment, which was part of the Major League Baseball Advanced Media (MLBAM) plan to bring Wi-Fi to all parks for this season. Keys said the new network deployment began in March and finished up on June 26, giving his team a few home dates to kick the tires and tune it up quickly for its big event.

Going forward, Keys said the four-DAS deployment — with four sets of antennas and four different headends — will be consolidated into a single, neutral host DAS operation. Keys is also looking forward to adding features enabled by the Wi-Fi network, like expanded food ordering and greater use of beacon technology. “It’ll be great to add more things to improve the fan experience,” he said.

All-Star Game has DAS Grand Slam: Four different DAS systems online at Great American Ball Park

Google Map screenshot of Cincinnati riverfront area, showing Paul Brown Stadium and the Great American Ball Park. Somewhere in between is a DAS headend.

Google Map screenshot of Cincinnati riverfront area, showing Paul Brown Stadium and the Great American Ball Park. Somewhere in between is a DAS headend.

Call it the DAS grand slam — to cover wireless customer demand, the Great American Ball Park in Cincinnati has four separate DAS deployments, one for each of the major U.S. wireless carriers, which are probably all getting a workout at tonight’s Major League Baseball All-Star Game.

Brian Keys, vice president of technology for the Cincinnati Reds, confirmed Tuesday that there are four separate DAS (distributed antenna system) networks in the ballpark, one each for AT&T, Verizon Wireless, Sprint and T-Mobile. Through several interviews Mobile Sports Report was able to confirm that Solid is providing the gear for the Verizon DAS, and another source said that ADRF is providing the DAS for Sprint.

We also had a long interview with the folks at TE Connectivity, who initially installed a 2G/3G/4G DAS in the venue in 2011, and recently upgraded that DAS, adding support for the 2100 MHz AWS spectrum. And while TE Connectivity was not at liberty to name the carrier for which it provides the DAS, by process of elimination we are fairly confident that their customer is AT&T. T-Mobile, which is also on its own DAS in the park, is also believed to be a Solid customer but we haven’t yet confirmed that fact.

Why are there four systems in Cincinnati? We haven’t yet had a chance to talk to Brian Keys (he’s been a little bit busy this week) but it’s fairly likely that it was just a fairly normal occurrence in the DAS world — one big carrier doesn’t want to join a DAS already installed by another big carrier, so it just funds its own. At the Great American Ball Park, Verizon’s decision to build its own DAS may have been in part because the carrier already has a DAS headend facility nearby, serving Paul Brown Stadium, the GABP’s riverfront neighbor. In fact, the Solid folks told us Tuesday that both the baseball DAS and the football DAS for Verizon are served out of the same facility, which makes sense.

The TE Connectivity DAS, for the client it couldn’t name (AT&T!), was also recently upgraded to cover areas outside the stadium, including the parking lots, a trend we are seeing more of as venues realize that fans want connectivity the moment they arrive, not just when they’re in their seats. We’ll try to get more details on this somewhat unique DAS situation — which was also apparently approved by the technical and business folks at MLBAM, which helped bring a Wi-Fi upgrade to the park this past offseason — but for the meantime, let’s just be glad that customers of all four of the major U.S. carriers had DAS support at Tuesday’s All-Star Game — in their own private and separate ways.

MLB Stadium Tech and Wi-Fi Reports — AL Central

Editor’s note: The following team-by-team capsule reports of MLB stadium technology deployments are an excerpt from our most recent Stadium Tech Report, THE BASEBALL (And Soccer!) ISSUE. To get all the capsules in one place as well as our featured reports, interviews and analysis, download your free copy of the full report today.

AL Central

Reporting by Paul Kapustka

Kauffman Stadium during 2014 World Series

Kauffman Stadium during 2014 World Series

Kansas City Royals
Kauffman Stadium
Seating Capacity: 37,903
Wi-Fi – Yes
DAS – Yes (under construction)

Venerable Kauffman Stadium turned on its MLBAM-powered Wi-Fi network just in time for the Royals’ historic run to the World Series last year, where it was found and enjoyed by true blue fans even without any promotion. A new four-carrier neutral DAS is under construction for 2015.

Minnesota Twins
Target Field
Seating Capacity: 39,504
Wi-Fi – Yes
DAS – Yes

Host of the 2014 All-Star Game, Twins fans can now enjoy the Wi-Fi and DAS networks installed last year. MLBAM helped install the Wi-Fi network, while the DAS includes gear and expertise from InSite Wireless and TE Connectivity.

Chicago White Sox
U.S. Cellular Field
Seating Capacity: 40,615
Wi-Fi – Yes
DAS – Yes

One of the losers in the MLBAM program to bring Wi-Fi to all stadiums was Boingo Wireless, the previous provider of Wi-Fi at U.S. Cellular Field. For 2015, a MLBAM network will be in place with 550 new APs from MLBAM equipment provider Cisco.

Detroit Tigers
Comerica Park
Seating Capacity: 41,681
Wi-Fi – Yes
DAS – Yes (under construction)

Tigers fans are yet another group who have in-park Wi-Fi this season courtesy of MLB, with 600 APs installed for the 2015 season. A full-featured DAS is currently under construction, and is expected to be operational by mid-season with all four major wireless carriers participating.

Cleveland Indians
Progressive Field
Seating Capacity: 43,545
Wi-Fi – Yes
DAS – Yes

Recognized as one of baseball’s most Twitter-friendly teams, the Cleveland Indians have stepped up their game with not one but two social media areas inside the stadium that will take advantage of the Verizon-powered Wi-Fi. Formerly known as the Social Suite, the Indians now have #Tribelive, which will have its own seating section inside a right-field bar; Cleveland will also have a Family Social Suite, in a more kid-friendly environment.

MLB Stadium Tech Reports — AL West

Editor’s note: The following team-by-team capsule reports of MLB stadium technology deployments are an excerpt from our most recent Stadium Tech Report, THE BASEBALL (And Soccer!) ISSUE. To get all the capsules in one place as well as our featured reports, interviews and analysis, download your free copy of the full report today.

AL West

Reporting by Paul Kapustka

The iconic sign outside the "Big A," aka Angels Stadium of Anaheim. Credit Terry Sweeney, MSR

The iconic sign outside the “Big A,” aka Angels Stadium of Anaheim. Credit Terry Sweeney, MSR

Los Angeles Angels of Anaheim
Angel Stadium of Anaheim
Seating Capacity: 45,050
Wi-Fi – Yes
DAS – Yes

Angel Stadium is now fully wired for Wi-Fi and DAS, courtesy of integrator 5 Bars, which used Wi-Fi gear from Ruckus Wireless and DAS gear from JMA Wireless.

Houston Astros
Minute Maid Park
Seating Capacity: 40,963
Wi-Fi – Yes
DAS – Yes

The Houston Astros were one of the beneficiaries of MLB’s program to bring updated Wi-Fi to all stadiums. The new network was scheduled to be live for the 2015 season.

Seattle Mariners
Safeco Field
Seating Capacity: 47,476
Wi-Fi – Yes
DAS – Yes

Finally, geeks of all stripes in Seattle can stay connected while at Safeco, thanks to a new Wi-Fi network with 560 access points that is scheduled to be live for the 2015 season. New LED lighting for the field should make it easier to watch games through the rain and mist.

Oakland A’s
O.co Coliseum
Seating Capacity: 35,067
Wi-Fi – Yes
DAS – Yes

With Wi-Fi already in place the Oakland A’s spent $10 million in the offseason to upgrade the video boards in aging O.co Coliseum. The two new HD Daktronics screens, 36 feet high and 145 feet wide, were joined by new ribbon LED boards to bring more info to fans.

Texas Rangers
Global Life Park in Arlington
Seating Capacity: 48,114
Wi-Fi – Yes
DAS – Yes

The team isn’t talking about it publicly yet, but according to MLBAM there is now a working Wi- Fi network and a DAS at Global Life Park in Arlington, one of the latest beneficiaries of the MLB program to bring Wi-Fi to all MLB parks.