New Report: Baseball AND Soccer stadium tech updates

MLB_Thumb_2015MOBILE SPORTS REPORT is pleased to announce the second issue in our second year of our STADIUM TECH REPORT series, which includes a focus on baseball and soccer stadium technology deployments, and team-by-team coverage of technology deployments for all 30 MLS teams — AND all 20 MLS teams.

Download your free copy today!

We spent a long time getting this longest-ever report ready, but if you want to get an update on MLB’s Wi-Fi everywhere plan, which is almost complete, we have the most information anywhere about the strategy and the results.

Inside the report — our longest ever — our editorial coverage includes:

— Kauffman Stadium profile: The benefits of Wi-Fi installed before the surprise World Series run by the Kansas City Royals.

— In-depth profiles of new technology deployments at Angels Stadium in Anaheim, and at the new Avaya Stadium for soccer in San Jose.

— MLB stadium tech research: This editorial research provides a technology update and analysis for stadiums used by all 30 MLB teams, gauging the level of deployment of Wi-Fi and DAS.

— MSR exclusive stadium tech analysis: The report also includes an exclusive interview with MLBAM’s Joe Inzerillo, architect of the “Wi-Fi everywhere” plan MLB is completing this season.

Download your free copy today!

Stadium Tech Report: Los Angeles Angels and 5 Bars build ‘wireless halo’ of Wi-Fi & DAS for Angels Stadium

The iconic sign outside the "Big A," aka Angels Stadium of Anaheim. Credit all photos, even tilted ones: Terry Sweeney, MSR (click on any photo for a larger image)

The iconic sign outside the “Big A,” aka Angels Stadium of Anaheim. Credit all photos, even tilted ones: Terry Sweeney, MSR (click on any photo for a larger image)

Every baseball team wants to notch a win on opening day, and the Los Angeles Angels of Anaheim are no exception. So while the number of runs scored was important to Al Castro, the franchise’s IT director, his eye was also on wireless performance in Angels Stadium, since 2015 will be the first full season with both Wi-Fi and DAS technology in place. The Angels may have lost their opener against the Kansas City Royals, but their wireless networks scored big by handling more than 1.3 TB of data that afternoon.

“Fans expect connectivity these days,” Castro told Mobile Sports Report during a tour of Angels Stadium, aka the Big A, which was built in 1966. Once the home of the NFL’s Los Angeles Rams, the stadium went through and extensive renovation in 1997-98 and now seats about 44,000 for baseball and serves 3 million visitors annually. “If they’re going to come to a ballgame for four hours,” said Castro of today’s fans, “they won’t tolerate not being connected.”

Adding wireless to the ‘Big A’

To get the wireless ball rolling last year, teams of engineers on scaffolding started on the uppermost tier of the Big A (the “View Level”) to mount DAS and Wi-Fi antennas to the stadium canopy. Working from outermost edges of the C-shaped stadium, two sets at of scaffolding at each end leapfrogged each til they met in the middle – a five-week process, according to Castro.

Angels IT director Al Castro, in front of his wireless deployment map

Angels IT director Al Castro, in front of his wireless deployment map

The 15-zone DAS network went live in June 2014 with Verizon, AT&T and T-Mobile on board; Sprint is expected to add some antennas in the next several weeks. Currently, there are 122 DAS remotes in 33 locations. Angel Stadium Wi-Fi went live in September 2014 and now counts more than 400 access points around the stadium, according to team figures. Ruckus Wireless is the Wi-Fi vendor; the DAS gear is from Teko Telecom, now part of JMA Wireless.

The Angels worked closely with technology partner 5 Bars, a builder of turnkey wireless networks for sports venues’ wireless needs. Castro would not disclose the budget for the wireless upgrades at Angels Stadium.

In addition to using Major League Baseball’s Ballpark app, Angels fans can post to social media, surf the Web and check email from the stadium’s wireless networks. On the stadium’s club level, spectators can wirelessly order food and beverage from their seats; Legends, which operates the stadium’s concessions, uses an unpublished SSID for 150 wireless-enabled moveable cash registers and more than two dozen handheld point-of-sale devices. Similarly, TicketMaster has its own invisible SSID for wireless scanning of tickets at the stadium’s entry gates; the SSID for the press box is also masked, according to Castro.

Hiding in plain (or painted) sight

The DAS antennas and APs have been strategically installed and well concealed; they’re as discrete as chameleons. Working with Ruckus gear, 5 Bars installed narrow-beam, sectorized-beam and high-capacity APs, all centrally managed by Ruckus’s SmartCell Gateway 200.

A nice view of the field -- with antennas in silhouette

A nice view of the field — with antennas in silhouette

The Angels also use SmartCell Insight, a reporting and analytics package that helps the team track number of unique connections to the Wi-Fi during the course of a game, device types, total and average data uploaded and downloaded, and their speeds, Castro said.

Angel Stadium Wi-Fi has been engineered for 20,000 simultaneous users; there’s no throttling of user bandwidth and no filtering for streaming media like Spotify — “yet,” Castro was quick to add with a laugh. Download speeds vary depending on crowd size, according to Tommy Taylor, senior manager, engineering services for 5 Bars. For a game with 36,000 in attendance, for example, average download speed for devices using 2.4 GHz bandwidth is 8-12 Mbps, while 5 GHz connections can run as fast as 18-24 Mbps. On the traffic side, currently the network is seeing upload volume of about 20 percent of the download average volume, Taylor said, in an email to Mobile Sports Report.

The Angels will continue to fine-tune the network and add or re-point APs as necessary. “We are in the process of adding additional APs to cover some areas that, when the stadium is full, do not receive the high level of coverage we are targeting to provide,” Castro said. Those additions should be done by mid-June. Management has an eye on monetizing the network through sponsorships, and extending the in-seat ordering system beyond the club level of the ballpark, according to Castro.

He also wants to add streaming video to the network so that fans can watch replay from multiple angles, which Castro described as “a good incentive — something you can’t get at home.” He also intends to expand his use of analytics and report generation on a game-by-game basis. It’s the sort of thing that the owners and managers of the team are increasingly interested in, Castro added.

Bowman: MLB won’t stop fans from using Meerkat or Periscope at games — for now

Bob Bowman, president of business and media for Major League Baseball. Credit: Paul Kapustka, MSR

Bob Bowman, president of business and media for Major League Baseball. Credit: Paul Kapustka, MSR

At a Major League Baseball game and feeling the need to livestream some live game action via Periscope or Meerkat? Go right ahead, because the powers that be at MLB aren’t going to stop you — at least not yet.

While the nascent livestreaming services — which basically allow users to broadcast live video of what their phone cameras can see — potentially create conflicts with both broadcast rights and available network bandwidth, they aren’t yet a problem at MLB ballparks, according to Bob Bowman, president of business and media for Major League Baseball and the CEO of MLB’s advanced media operations.

Bowman, who was at Harding Park Golf Course Tuesday to announce a joint deal between MLBAM and the PGA, spoke briefly with MSR to address the livestreaming question, which surfaced earlier this month when fans started using Periscope and Meerkat to “broadcast” live video from MLB games. Though showing live video “without the express written consent” of MLB games is “strictly prohibited” (as anyone who’s ever watched a MLB game broadcast knows), Bowman said Tuesday that he and MLB don’t see livestreaming as a problem that needs to be addressed by policing fans or blocking the services.

Screen Shot 2015-03-29 at 11.20.55 PM“I just don’t think our fans come to games with the idea of holding their phones up to stream video for 20 minutes,” Bowman said. While he does note that such streaming is patently illegal, Bowman also said that MLB didn’t want to alienate fans over something that wasn’t yet causing any big problems. Currently, he said, most fans are using social media to share photos of themselves at baseball games, a type of free promotion MLB and teams go out of their way to encourage.

What hasn’t happened — yet — is large numbers of fans using the livestreaming services, something that could potentially clog up the cellular and Wi-Fi networks inside the stadiums since live, streaming video inherently uses up a large amount of bandwidth. Bowman, whose MLBAM operation spent some $300 million over the past couple years in a project that is bringing advanced cellular and Wi-Fi networks to all MLB parks, said that if livestreaming becomes a bandwidth issue, it will be addressed.

“We just put all these new networks in, and the last thing the stadiums want is [people] using the network for these types of activities,” Bowman said. “If we’re wrong, we’ll review it. But I just don’t think our fans are there to stream the game.”

Fans are using Periscope and Meerkat to stream Opening Day baseball action — how will MLB respond?

Ended Meerkat stream from MLB opening day

Ended Meerkat stream from MLB opening day

The question we asked about how the use of livestreaming apps like Meerkat and Periscope might affect stadium networks is getting some real-world trials today, as fans are clearly using the apps to show live video from the various opening day games for Major League Baseball. So far, we’ve seen reports that fans are using the apps from the New York Yankees’ home opener agains the Toronto Blue Jays, and at the Detroit Tigers’ home opener against the Minnesota Twins. We’ve tried to catch a live broadcast of game action, but so far no luck!

UPDATE: The Wall Street Journal got MLB to comment… and the league doesn’t like live streaming, no wonder. Still, no word on how it’s going to be enforced.

UPDATE 2: Well, MLB has responded… and it made the WSJ issue a correction, no small thing there. According to MLBAM’s Bob Bowman the league will “monitor” people who are livestreaming, but won’t take any action. Apparently Bowman thinks that fans won’t spend their time at games livestreaming, which we would put in the “remains to be seen” category. Also, nothing has been said so far about how livestreaming might affect stadium wireless network performance; so we are still betting that we haven’t yet heard the end of potential Meerkat/Periscope bans, especially from other sports like football. Stay tuned!

Though live streaming of game action seems to be in direct violation of MLB broadcast rights, we still haven’t heard back from the league about what it plans to do, if anything, about livestream feeds from games. For what it’s worth, the Meerkat terms of service seem to absolve the app or the company from any infraction, saying it’s the user’s responsibility to not use it to show content that is copyrighted or otherwise protected. But we all remember YouTube, right?

Since the live streams aren’t archived it’s possible that the league may just let them slide; and there probably aren’t more than a handful of people streaming yet at each game, so it’s doubtful that stadium networks are yet feeling any huge strain from the apps.

Small text snippet from Meerkat TOS... you are own your own when it comes to rights violations!

Small text snippet from Meerkat TOS… you are own your own when it comes to rights violations!

But it’s also not too hard to look into the near future at a “big game” and see hundreds or thousands of fans bringing a stadium network to its knees with live video streaming. So far, none of our stadium sources seems willing to talk publicly about the potential problem; we also have calls and emails in to both Twitter and Meerkat, and will update this post as we hear more.

Twitter, which bought Periscope, has a relationship with MLB so you are likely to see lots of Vines and photos from teams. But so far on the Twitter Sports blog, no Periscope. Let us know if you see any live action streams… we will keep updating this post as we hear more.

UPDATE: Just saw some live video of introductions in DC thanks to our old pal David Joachim (hey Dave!)…

UPDATE 2: Just had to add this tweet from one of my favorite writers, Steve Rushin… of course they’re on their phones!

Half a stadium using Wi-Fi? 2014 World Series games set Wi-Fi records at AT&T Park

Though this year’s World Series still isn’t over, the three games this past weekend in San Francisco set new local records for fan Wi-Fi use, arguably cementing AT&T Park’s title as baseball’s best-connected stadium.

One of the more stunning numbers provided to us by Bill Schlough, the senior vice president and CIO for the Giants, was that nearly half the crowd in AT&T Park for Sunday’s Game 5 used the Wi-Fi network — a “take rate” we’ve never heard of before at any sporting event of any kind. According to Schlough the Giants’ Wi-Fi network had 20,638 unique users during Game 5, a take rate of 49.7 percent. (According to the official box score, the attendance that night was 43,087, give or take a Panda or four.) The game also set an AT&T Park record for Wi-Fi upload traffic with 747 Gigabytes of data being sent by people at the park out to the network.

(Including one special photo tweeted out by a former Giant you may have heard of.)

A record for downloaded Wi-Fi traffic, of 937 GB, was set during Friday night’s Game 3, and the record for a combination of download and upload traffic was set during Game 4, with a total of 1.62 Terabytes. According to Schlough, the total data transferred during the eight games of the 2014 postseason at AT&T Park was 10.3 TB, which is easily the high-water mark for any baseball stadium network we know of. (The total does not include regular cellular traffic carried by the AT&T Park DAS; we will add those numbers in when they become available.)

And while individual games such as Super Bowls or contests down the San Francisco Peninsula at Levi’s Stadium may set higher overall marks (thanks in part to having more fans in the building), we’d be interested to see if any other stadium networks can match AT&T Park’s season-long data number of 59.3 TB, which according to Schlough represents total traffic over 2014’s 91 games (regular and postseason) at AT&T Park.

One more fun number from Schlough: Since the park turned on its fan-facing Wi-Fi network in 2004, there have been 4,085,518 Wi-Fi connections during the 946 game days. Kansas City’s Kaufmann Stadium, which just got its Wi-Fi network installed recently, has one or two more games this season to start trying to catch the Giants from a networking perspective.

UPDATE: Looks like the Royals are claiming some already-decent usage of their Wi-Fi network:

Confirmed: KC has MLB-provided Wi-Fi, part of plan to bring Wi-Fi and DAS to all MLB stadiums

KC fans at seriesThe rumored Wi-Fi network at Kansas City’s Kauffman Stadium is now a confirmed entity, according to Joe Inzerillo, executive vice president and CTO for MLB.com. In a phone call Thursday, Inzerillo said the Royals’ new network is part of a league-wide effort to bring Wi-Fi to all MLB parks, a task he thinks may take another year or two to complete.

As MLB.com has stated publicly before — but maybe not spelled out in so much detail — it has a program under which Major League Baseball teams can “opt in” to have MLB.com and the nation’s top four wireless carriers participate in the funding and building of both Wi-Fi and DAS networks in MLB stadiums. Though he wouldn’t divulge the specific financial commitments for specific deals, Inzerillo said that under the program “everyone has some skin in the game,” though he did allow that the league and the carriers, not the teams, foot the bulk of the bills.

Still, Inzerillo stressed that individual teams play a huge role in the Wi-Fi deployments, from design to deployment to management on site. “It’s not just like we show up and we’re the Wi-Fi fairies,” Inzerillo said. “This program wouldn’t be possible without the teams and the work they do.”

Screen Shot 2014-10-23 at 1.27.54 PMGiven MLB’s synchronized digital strategy of having the same app — and only the same app — available for fans in each ballpark, Inzerillo said that having high-quality cellular and Wi-Fi networks in each stadium was a key necessity, especially to make MLB app functions like seat upgrades and concession purchases work.

“You need to have the right [network] plumbing in place or none of the other stuff matters,” Inzerillo said.

Kauffman Stadium’s network, Inzerillo said, was just one of about a dozen MLB Wi-Fi projects that got underway this year. That it was finished in time for postseason play was just luck, and not some last-minute installation due to the Royals’ on-field successes. “It was just a fortuitous thing that it was ready,” said Inzerillo, who said that construction of Wi-Fi at Kauffman had been ongoing for the past 5 to 6 months.

Though the network wasn’t promoted on the team’s website or anywhere else on the Internet, Inzerillo said the Royals were promoting it at the stadium. Even without a lot of advertising, fans found the network, he said, claiming “tens of thousands” of Wi-Fi connections during the Royals’ postseason run. However, Inzerillo also said some extra cellular trucks were brought in by some of the carriers for the Royals playoff games because the DAS at Kauffman isn’t quite finished yet.

Inzerillo said that anywhere from 22 to 26 teams will eventually end up using some combination of league-provided Wi-Fi and/or DAS. The league’s goal of having every stadium fully wired should be nearly complete by opening day of 2015, he said, with a more likely “final” goal of complete coverage reached sometime in 2016. In 2014, Mobile Sports Report research showed that 10 of the 30 MLB stadiums didn’t have fan-facing Wi-Fi; some of those teams (like Kansas City) will be getting MLB networks, while some other program participants are upgrading existing systems, Inzerillo said.

Under the MLB network deal, the DAS in each stadium will be a neutral-host deployment hosted by one of the four major U.S. cellular carriers — AT&T, Verizon Wireless, Sprint and T-Mobile — depending on things like regional/historical market share and existing contracts, Inzerillo said. While the carriers will operate the MLB DAS deployments, the Wi-Fi networks will be deployed, run and monitored by MLB, either from its New York City or San Francisco network operation centers, Inzerillo said. Wi-Fi gear will come mainly from Cisco, though Inzerillo said there is also a small percentage of Meru Networks gear based on teams that had previously installed Meru equipment.

Though Inzerillo said MLB may make some overall announcement once the league-wide project is completed, he didn’t necessarily think that having working Wi-Fi and DAS in stadiums was such a big deal.

“It’s kind of a weird thing to think about bragging about,” said Inzerillo, who compared Wi-Fi and DAS to plumbing as a basic stadium necessity, not an amenity. Having high quality networks, he said, “are table stakes for a modern facility.”

UPDATE: The Kansas City Royals are now officially promoting the Wi-Fi, with some usage claims: