Levi’s Stadium ‘NiNerds’ get high-visibility wardrobe upgrade

NiNerd sporting the new neon vest. (Click on any photo for a larger image) Credit all photos: Paul Kapustka, MSR

NiNerd sporting the new neon vest. (Click on any photo for a larger image) Credit all photos: Paul Kapustka, MSR

I needed a one of the NiNerds Sunday at Levi’s Stadium, and thanks to a new wardrobe addition, they were a lot easier to find.

Discovering a problem with the Levi’s Stadium app, I looked around for one of the stadium’s walk-around technology helpers — aka the “NiNerds” — and found one quickly thanks to the new neon-yellow vests many were wearing during Sunday’s game between the San Francisco 49ers and Washington.

Earlier this season, the NiNerds were much more nattily dressed in their gingham-check shirts, bow ties and fake horn-rim glasses. While cool and cute, the outfits proved hard to recognize in the crowd, especially since the NiNerds’ red check shirts looked a lot like the jerseys and t-shirts worn by the many faithful fans. Perhaps in order to make the NiNerds stand out more, the team dressed them in neon Sunday, like Wi-Fi “coaches” in other stadiums have done.

Unfortunately, the NiNerd I talked to wasn’t able to solve my problem (it seems to be related to a known bug in the newest Android release of the stadium app) but I did notice during my visit Sunday that the NiNerds in general seemed to be more numerous and visible, and they even got a nice shout-out on the Levi’s Stadium big screen (see photos below). Below are some thoughts and observations on the network performance, the app performance and the overall fan experience at Levi’s, which I hadn’t visited since the season opener back on Sept. 14.

Wi-Fi network struggles at 2.4 GHz, soars at 5 GHz

Speed test results from outer concourse location, Levi's Stadium, pregame

Speed test results from outer concourse location, Levi’s Stadium, pregame

On the network-performance side of things, the Wi-Fi system seemed as robust as ever for new devices, including the AT&T LG Optimus G Pro I’ve been test-driving lately. With the Android device and its 5 GHz Wi-Fi connection I hit speeds of 31 Mbps download and 29 upload before the game on the Levi’s Stadium outside concourse, and then had a 14 Mbps download connection in my seat in section 229 (south end zone) at kickoff. In the third quarter I wandered up to the top (7th) level of seats, and got a 28/33 Mbps reading while waiting in a concession-stand line.

With my older Verizon Droid 4 device, however, I struggled to connect to the Wi-Fi network. Since the phone only runs on the 2.4 GHz Wi-Fi frequency, it doesn’t do well at Levi’s Stadium, where the Wi-Fi is more heavily tuned for newer, 5 GHz-capable devices.

Full charging station... before the game starts

Full charging station… before the game starts

The best Wi-Fi speedtest I could get with the Droid device was a 1.06 Mbps download/3.04 Mbps upload mark, from the same spot on the outer concourse where the newer device recorded blazing connectivity speeds. Switching it over to its Verizon 4G LTE radio, I was able to get much faster connectivity, including one mark of 21.60/9.58 on the main level inside concourse.

I also ran out of juice on the Verizon phone before the end of the game — which could have been either the device draining due to its inability to get a solid connection, or due to the fact that it’s getting old and the battery doesn’t hold a charge that well anymore. Judging from the crowds around the Levi’s Stadium recharging stations (the picture to the left was taken during pregame) I am not alone in my device-energy woes.

App problem derails beacon test

One of the main tasks I had planned for Sunday was to see how well the beacon-assisted wayfinding feature in the stadium app worked. Only problem was, in the new update of the Levi’s Stadium app that was released in the past week (which I downloaded to both devices Sunday morning), several features were missing, including the “Maps” feature.

Picture of app fail in Levi's Stadium Android app

Picture of app fail in Levi’s Stadium Android app

A NiNerd I talked to outside the stadium on the Faithful Mile area showed me the maps/wayfinding feature on his iPhone, and pulled up a GPS-supported direction message that pointed the correct way for me to enter the stadium. But neither he nor the NiNerd I talked to inside the stadium could figure out why both of my Android devices weren’t showing the maps feature, or several other features on the left-upper-corner pull-down menu.

According to Louise Callagy, vice president of marketing for app developer VenueNext, the new version of the app released this week did have a known Android bug. In an email response Sunday night Callagy said, “we know we have a bug where Android gets confused and won’t return results from the network,” adding that rebooting the device might have fixed the problem; however, I did reboot both devices during the game and the problems were not corrected.

Callagy said the Levi’s network also had issues Sunday in getting location information from the beacons. “Our plan is to re-write the code [for the app] and solve this issue, releasing a new version before the Seahawks game on Thursday,” Callagy added in her email.

On the good-news side the replay function of the Levi’s Stadium app was more impressive than earlier versions, with highlights appearing in the app in mere seconds after the original play had concluded. After the Niners’ first TD pass of the day, I was able to view the highlight of the Colin Kaepernick-to-Anquan Boldin pass just after the extra point had been kicked.

I was also able to see the red light/green light system for restroom wait times that drew so much attention when it was talked about earlier this year. However, in real-life practice it’s doubtful anyone thinks of looking at the app when it’s time to go. (It’s also quite likely that while you are looking at the app for a short restroom line, a natural break in action will occur and restroom lines will predictably lengthen everywhere.) I found a quick trick for Levi’s attendees that might pay off in the future: If the restroom you’re aiming toward has a long line, walk a small bit farther to find its back door — where there is often no line at all.

New version of app, with clearer icons on main screen

New version of app, with clearer icons on main screen

I messed up later in the game, however, in thinking that it would just be easier to find a concession stand than to use the app’s express window ordering function. At least the 10 minutes I spent in line behind three women who were apparently ordering for their entire row (five hotdogs, six orders of fries, two orders of wings, two beers and one large ice water) gave me time to conduct a couple more Wi-Fi speedtests. Next time, I’m using the food-ordering features on the app.

I also made great use of the app’s ability to let fans watch live game action (I chose the feed from the main video screens in the stadium). Since I had to leave early I was on the first VTA train when I saw the game-winning TD run courtesy of the app’s live action broadcast. The live video, incidentally, kept playing seamlessly over the AT&T 4G LTE network as I sped away from the stadium, allowing me to watch the final game-sealing sack as I beat most of the traffic home.

VTA trains a smooth ride, once you figure out how to get on

I also had another smooth ride to and from the stadium using the VTA light rail trains from Mountain View — once I was on the train it took just a little over 30 stress-free minutes both coming and going. Getting on the trains, however, is a process that could still use some work. The Mountain View station, which is logistically hampered by having to share space with the Caltrain tracks and station, has very little signage on game day, and has a lot of confusing temporary gates to try to flow foot traffic toward the ticket-verification checkers.

Packed VTA train en route to Levi's Stadium

Packed VTA train en route to Levi’s Stadium

Once I figured out the maze and was guided across the Caltrain tracks I was directed to one of two waiting trains — but then a VTA staffer looked into the train and told people there were also express buses that wouldn’t stop on the way to Levi’s (unlike the trains, which stop at numerous stations en route). The quizzical advice — nobody said if it was any faster to take the buses — had many people wondering what they were supposed to do, causing a delay in closing the train doors as people made up their minds without any more information.

Once we arrived at the stadium, on the exit platform there was no person or sign directing fans in the proper direction. Good thing for the many newbies on the train (the train 2 hours before kickoff was packed) several of us were veterans and directed everyone down the proper ramp. For the return trip the Mountain View line suffered from similar lack of information and signing — and after one train passed the station with signs that said “Not in Service” we got on a second train that also said “Not in service” but whose doors opened anyway and one person in a yellow vest told everyone, “get inside.”

Overall impressions: Levi’s experience and technology still a work in progress

While I continue to be impressed by the network and app performance at Levi’s Stadium, I also felt several times on Sunday like the technology, the stadium and the entire fan experience is still a work in progress — perhaps something to be expected for a venue in its first year of events. But I have to wonder a bit about releasing a new version of the app in midseason, without apparently testing it enough to make sure it worked well on all devices that might want to use it.

I’m also still skeptical on how well the wayfinding feature will work in real world situations; though it sounds great to be able to get GPS-like directions to places inside the stadium, the reality of trying to walk around looking down at your phone on one of Levi’s Stadium crowded concourses is more likely to lead you into someone’s backside. Anyone with tales to tell of Levi’s Stadium technology experiences, please chime in below in the comments or send me an email to kaps at mobilesportsreport.com. I’d be especially interested to know if anyone else saw my app problems Sunday on Android phones. More Levi’s pictures below.

A NiNerd (no vest) helps fans outside the stadium.

A NiNerd (no vest) helps fans outside the stadium.

Kickoff view from Section 229. Thanks to the Niners for the free media access.

Kickoff view from Section 229. Thanks to the Niners for the free media access.

Niners fans get their phone cameras busy for kickoff ceremonies.

Niners fans get their phone cameras busy for kickoff ceremonies.

Scoreboard plug for the app.

Scoreboard plug for the app.

Scoreboard promo for the NiNerds (one in a series)

Scoreboard promo for the NiNerds (one in a series)

Second in the series. This one got laughs from the crowd.

Second in the series. This one got laughs from the crowd.

Probably the first time many fans heard the term "NiNerds"

Probably the first time many fans heard the term “NiNerds”

Nothing says geek like a bow tie

Nothing says geek like a bow tie

Stadium Tech Report — NFL stadium technology reports — NFC West

Editor’s note: The following team-by-team capsule reports of NFL stadium technology deployments are an excerpt from our most recent Stadium Tech Report, THE FOOTBALL ISSUE. To get all the capsules in one place as well as our featured reports, interviews and analysis, download your free copy of the full report today.

NFC WEST

Reporting by Chris Gallo

Arizona Cardinals
University of Phoenix Stadium
Seating Capacity: 65,000
Wi-Fi – Yes
DAS-Yes
Beaconing – No

As the site of the Super Bowl XLIX, University of Phoenix Stadium is in the midst of massive upgrades. This includes an $8 million dollar improvement for faster wireless and larger video boards. The stadium turned eight this summer and already has one Super Bowl under its belt. Look for more information about upgrades throughout the year before the big game on Feb. 1, 2015.

San Francisco 49ers
Levi’s Stadium
Seating Capacity: 68,500
Wi-Fi – Yes, 1,200 access points
DAS – Yes, 700 antennas
Beaconing – Yes

View from the Pepsi seating porch at the north end of Levi's Stadium

View from the Pepsi seating porch at the north end of Levi’s Stadium

There is a buzz around Levi’s Stadium entering this season. And for good reason. The brand-new venue boasts more than 1,200 Wi-Fi access points and 700 DAS antennas. Aruba Networks (Wi-Fi) and DAS Group Professionals are hoping to fulfill the 49ers’ desires to own the most-connected stadium in all of sports. An ambitious new team app, with replays and food ordering and delivery to all seats is also part of the technology offerings.

Early tests of the stadium network during the preseason and regular season opener were promising, with Wi-Fi performance at Super Bowl-surpassing levels. On the DAS side, strong cellular signals were reached, with a nearly full house of fans. The question for the network, like the team itself – can it keep performing at a high level during the full season?

Seattle Seahawks
CenturyLink Field
Seating Capacity: 72,000
Wi-Fi-Yes
DAS-Yes
Beaconing – No

Fans of the reigning Super Bowl Champions will have more to cheer for in coming years at CenturyLink Field. The question is, will the loudest stadium in the NFL stay that way if Seahawks fans are using their phones more often, now that Verizon has installed stadium-wide Wi-Fi using Extreme Networks equipment?

St. Louis Rams
Edward Jones Dome
Seating Capacity: 66,000
Wi-Fi – No
DAS-Yes
Beaconing – No

After a proposed $700 million dollar upgrade was rejected, the St. Louis Rams are still seeking to improve the Edward Jones Dome. The good news for fans is that for this season a Mobilitie neutral-host DAS should significantly improve cellular communications not just in the stadium itself, but also in the adjacent convention center.

Levi’s Stadium Wi-Fi update: Usage down from record, but still strong — 2.4 TB for Eagles game

Fans take pictures of opening-day kickoff from southwest concourse. Credit, all photos: Paul Kapustka, MSR

Fans take pictures of opening-day kickoff from southwest concourse. Credit, all photos: Paul Kapustka, MSR

With three NFL games and now one collegiate contest under its belt, the Wi-Fi network at Levi’s Stadium is still handling big loads of data traffic, though not quite at the Super Bowl-beating level of 3.3 Terabytes recorded at the Niners’ home opener.

According to statistics provided by Niners vice president for technology Dan Williams, the Levi’s Wi-Fi network carried 2.4 TB of data during the Sept. 28 game against the Philadelphia Eagles, and another 2 TB during the Oct. 5 contest with the Kansas City Chiefs, both sellouts with reported attendance of 70,799 fans. And at an Oct. 24 college game between Cal and Oregon with 55,575 fans in attendance, the network carried 1.5 TB of traffic.

As you might be able to guess from the bandwidth numbers, the Sept. 28 game also had more users on the network, with 22,942 unique users, compared to 21,133 at the Oct. 5 game. For Cal-Oregon the unique user count was 13,508. At the Niners’ regular-season home opener on Sept. 14, there were more than 30,000 fans using the Wi-Fi network, with a peak of 19,000 simultaneous connections. The peak numbers for the later dates were 15,500 for the Sept. 28 game, 14,500 for the Oct. 5 game, and 8,400 for the college game.

In-seat and express food orders stay strong

Screen grab from Levi's Stadium app showing in-seat food delivery option. Credit all photos: Paul Kapustka, Mobile Sports Report.

Screen grab from Levi’s Stadium app showing in-seat food delivery option.

Food and beverage orders via the Levi’s Stadium app appear to be holding between the 1,500-2,000 order level for both the express pickup option as well as the in-seat delivery, with both features available to every seat in the house. According to Williams, for the Eagles game there were 1,144 express pick-up orders and 1,652 in-seat deliveries, while at the Kansas City game there were 1,162 express and 1,712 in-seat orders. At the Cal-Oregon game, where many fans were probably not as aware of the Levi’s app functionality, there were 551 express orders and 308 in-seat deliveries.

The video-replay feature of the Levi’s app also continues to attract solid interest, with 1,297 unique users watching 5,089 replays at the Eagles game and 1,111 unique users watching 4,986 replays at the Kansas City game. At the college game 234 fans used the replay feature, watching 1,059 replays.

One more interesting stat provided by Williams — the Apple iPhone is the overwhelming favorite device for connecting to the Levi’s Stadium Wi-Fi, with 62 percent share for the Sept. 28 game, and 60 percent share for both the Oct. 5 and Oct. 24 games. Android devices represented 24 percent (Sept. 28), 26 percent (Oct. 5) and 25 percent of all devices, while Apple iPads accounted for 2 percent, 3 percent and 1 percent of devices for the respective Sept. 28, Oct. 5 and Oct. 24 games.

DAS usage also remains strong, but not tops in NFL

On the DAS side of the Levi’s Stadium network we have some stats from AT&T to share (but none from any other carriers). According to AT&T, for the Oct. 5 game AT&T customers used 549 GB of data, which was only the fourth-highest AT&T DAS total for that weekend. Dallas (827 GB), San Diego (716 GB) and New Orleans (598 GB) all had higher AT&T DAS traffic totals for that weekend’s games. (Remember, these are results for stadiums with AT&T DAS networks only, not for all stadiums.) For the Sept. 14 game, the Levi’s Stadium AT&T DAS recorded 673 GB of traffic, according to AT&T.

Stadium Tech Report: THE FOOTBALL ISSUE arrives, with extensive coverage of Levi’s Stadium launch and tech reports on all 31 NFL stadiums

STR3_ThumbMobile Sports Report is pleased to announce the arrival of THE FOOTBALL ISSUE, our third Stadium Tech Report for 2014. As the title suggests this long-form report focuses on technology deployments at U.S. football stadiums, with an extensive inside look at the technology inside Levi’s Stadium, the new facility for the San Francisco 49ers. The report is available for free download from our site.

In addition to our Levi’s coverage, the Q3 issue of Stadium Tech Report also includes team-by-team reports on all 31 NFL stadiums, with a focus on Wi-Fi and DAS deployments. Our research found that while there are still 10 stadiums without fan-facing Wi-Fi, there is a lot of innovation going around league-wide, including big new digital displays in Jacksonville and Dallas, and new Wi-Fi and app deployments in other facilities.

Included in the report is an exclusive MSR interview with Michelle McKenna-Doyle, the NFL’s chief information officer, who talks about how the league office acts as a guide to helping teams with their tech deployments. We also have additional insight, analysis, and more tech profiles, and the good news is it is all free to read! Simply head to our report download page and get your free copy today!

We’d like to take a quick moment to thank our report sponsors, without whom we wouldn’t be able to offer such extensive original reporting and analysis free of charge. Our list for the Q3 2014 report includes SOLiD, Crown Castle, TE Connectivity, Extreme Networks, Aruba Networks, Mobilitie and DAS Group Professionals.

VenueNext, developer of Levi’s Stadium app, formally launches

Instant replay feature on Levi's Stadium app. Credit: Paul Kapustka, MSR

Instant replay feature on Levi’s Stadium app. Credit: Paul Kapustka, MSR

VenueNext, the startup firm that developed the Levi’s Stadium app for the San Francisco 49ers, is formally hanging out its own banner Thursday, with announcements of not just application design but a full stadium-specific suite of analytic tools designed to give stadium owners and operators more control and better visibility into what’s happening when fans use the app. The firm will be seeking to license its software and develop apps for other stadiums and large public venues, like outdoor concert arenas.

We’ll have a longer profile with VenueNext soon — since there are no other customers yet other than the Niners, we have a little time to catch up. What’s most interesting is that the team actually helped create VenueNext, funding its start through Aurum Partners LLC, the investment firm of the team’s ownership. VenueNext is led by COO and founder John Paul, a name we remember well since we interviewed him way back in his days at networking pioneer Banyan (when yours truly worked for a publication called Unix Today!). Mr. Paul’s decorated tech career includes stops at Netscape and Slingbox, among others.

In a phone interview with Paul, he said VenueNext hopes to sign up as many as 30 customers for its app development efforts over the next year. The VenueNext team, which has offices in Palo Alto, Calif. and New York, has 25 engineers and is also seeking additional venture funding, Paul said.

John Paul, COO and founder, VenueNext

John Paul, COO and founder, VenueNext

“Jed [York, the Niners’ CEO] is very open to us using Levi’s as a Petri dish,” Paul said. “Now the goal is to go license this to the world.”

Paul also corrected our reporting that the instant replay feature on the Levi’s app (which was live for the first time at Sunday’s home opener) didn’t have multiple camera angles available; the multiple angles were there, Paul said, in the upper left corner of the video window, visible when the player was stopped and a user tapped on the screen.

We did counter with an opinion that the feature was hard to find, and after a brief debate about application UI preferences Paul did say that the app was being constantly updated and that fans would “see enhancements” as the season continues.

As we said, look for a longer profile of VenueNext and its Levi’s app efforts soon.

Niners’ home opener tops Super Bowl for Wi-Fi data traffic with 3.3 Terabytes

Fans take pictures of opening kickoff from southwest concourse. Credit, all photos: Paul Kapustka, MSR

Fans take pictures of opening kickoff from southwest concourse. Credit, all photos: Paul Kapustka, MSR

The first regular-season home game for the San Francisco 49ers in their new home, Levi’s Stadium, produced more Wi-Fi traffic and far more actual fan-to-network connections than the most-recent Super Bowl, according to statistics from the Niners’ tech team.

Dan Williams, vice president of technology for the 49ers, said the Levi’s Stadium Wi-Fi network carried 3.3 Terabytes of data during Sunday night’s game between the Niners and the Chicago Bears, topping the 3.2 TB mark reported from Super Bowl XLVIII in February. According to Williams, out of the 70,799 that filled Levi’s Stadium Sunday, more than 30,000 fans connected to the Wi-Fi network at some point, with peak usage of 19,000 fans all connecting at one time occurring just before the 5:30 p.m. local time kickoff. According to the Super Bowl stats, the peak number of fans on Wi-Fi at that game at MetLife Stadium in New Jersey was 13,500.

“We’re pretty excited by Sunday,” said Williams, who said that the Wi-Fi network stood up well even under peak data transfer rates of 3.1 Gbps right before kickoff, and another 2.6 Gbps peak around 7:30 p.m. Around the peaks, network traffic stayed “well over 1 gig per second for three and a half hours,” Williams said.

North scoreboard screen at Levi's Stadium.

North scoreboard screen at Levi’s Stadium.

During the Niners’ first preseason game against the Denver Broncos, the Levi’s Stadium Wi-Fi network carried 2.13 TB of data, and during the Aug. 24 preseason game against San Diego there was another 1.96 TB of Wi-Fi data. The figures do not include any reporting from the stadium’s DAS network, which carries cellular traffic from AT&T, Verizon Wireless and T-Mobile customer. If preseason games are any indication, Williams expects to see numbers in the terabyte range for DAS traffic as well.

The Wi-Fi numbers from Sunday showed that fans quickly figured out a name change in the network name (or SSID). During preseason games, the Wi-Fi network was identified as “Levi’s Stadium” in a device list of available networks; on Sunday the free stadium network used the name “xfinitywifi,” reflecting the brand of Wi-Fi sponsor Comcast. Some fans might have been confused since the “xfinitywifi” SSID is the same one used by Comcast for its public Wi-Fi networks.

“Some folks may have been scratching their heads,” said Williams. “We changed the name last Monday before the opener.”

Replay app gets 7,800 views

As previously reported by MSR, the instant replay feature of the Levi’s Stadium app had its debut Sunday, and according to Williams fans watched 7,800 replays via the app. The top replay view was of the early touchdown pass from Niners QB Colin Kaepernick to Michael Crabtree, which Williams said was viewed more than 1,000 times.

Fans on southwest concourse take photos of live action.

Fans on southwest concourse take photos of live action.

As MSR reported, the replay feature was somewhat limited in functionality, not working at all until late in the first half and then only offering the last two plays plus some scoring highlights for viewing. Previously, team executives had said the replay feature would offer multiple camera angles and multiple replay reviews all at the same time. According to Williams, more features will be added to the replay function in the near future.

“It’s not the finished product, by any means,” Williams said. “You’ll see some more polish on it.”

The most-used feature in the stadium app, Williams said, continues to be the food and beverage features, which allow fans to either purchase concessions for express line pickup, or to have their orders delivered to their seats. Williams said the Niners delivered 2,100 food orders to fans Sunday, the most for the Niners so far.

Perhaps the best news for Williams was the lack of complaints about the wireless network, which the team had asked fans to tweet about if they were experiencing problems. Though some fans with older devices that only work on the 2.4 GHz wireless bands might not see the same speeds as those with newer devices (which use the more roomy 5 GHz bands), Williams said his team only got a couple complaints about network issues, and one of those was solved before they could respond.

“Overall it just was a really good [wireless] experience,” said Williams, who always ends by noting that networks are never completely finished products. But with its Super Bowl-beating performance Sunday, the Levi’s Stadium network appears in midseason form.

“I think we’re close,” Williams said.

View from the north porch

View from the north porch