Thousands of Developers. No Physical Office. Ship the Game. Go.

March 2020. EA operates game studios worldwide. Thousands of developers, artists, testers, and dev directors all working from studio locations across the globe. Everything that mattered was physically in those offices. Perforce servers. Dev and test consoles. Workstations that weren't laptops. QA servers. Daily in-person design sessions where artists and directors would crowd around screens, argue about frame rates, and make the hundred small decisions that turn code into a game.

Then everyone went home.

Game development broke first. Developers left with their knowledge but not their tools. Code now lived in thousands of locations, each one trying to connect remotely to on-prem Perforce servers that were built for studio networks, not home broadband. Commits failed. Connections dropped. The servers buckled under load they were never designed to handle.

If you didn't have code, you didn't have a game.

And nobody had a console. The dev and test consoles that developers needed to build, test, and debug their work were sitting in offices nobody could enter. You can't test a game you can't run. You can't ship a game you can't test.

Studios tried to solve it themselves first. Within days it was clear that wasn't going to work. The problem was too big, too interconnected, and too urgent for any single studio to tackle alone.

What followed was one of the largest centrally coordinated mobilizations in EA's history. IT, Game Dev, Security, Facilities, Site IT, and a dozen other teams pulled together across time zones and studio locations. The mandate was simple and impossible at the same time: get thousands of developers operational, remotely, without breaking the pipeline that shipped games.

I was part of that mobilization from the Game Dev side. My piece covered dev machine provisioning, console access, and the build pipeline. Each one was its own crisis.

Of everything we tackled, remote console access was the hardest.

Dev and test consoles are specialized hardware. They contain diagnostic logs, troubleshooting data, and information about the inner workings of platforms that first party manufacturers — PlayStation, Xbox, Nintendo — guard closely. Shipping them to developers' homes was a non-starter. The security and IP concerns were real and the platform holders weren't going to bend on them.

So we built console farms. Centralized racks of dev and test hardware in secure, connected facilities that developers could access remotely. The concept was straightforward. The execution was brutal.

Remote game development isn't remote streaming in the traditional sense. The developer's inputs — controller, keyboard, debug commands — had to travel to the console and the console's output had to travel back in real time. Frame rates are unforgiving. A few milliseconds of latency that wouldn't matter in a video call would break a game. Technology built for remote streaming wasn't built for this.

We had to solve for how you push a build to a console you can't touch. How you reset it when a build crashes and the console goes non-responsive. How you manage build queues and console reservations when you have more developers than consoles. How you move builds through the pipeline without anyone physically in the room.

None of this had an off the shelf answer. We built it under pressure, iterating as we went.

The console problem was the most visible but it wasn't the only one.

Developers needed workstations. The machines they'd left behind in the office were configured for game development in ways that took specialist knowledge and physical access to replicate. We couldn't ship those machines home. Even the people who configured them weren't at the sites.

The answer was cloud-based developer workstations. Preconfigured remote machines in the cloud, in the same regional network as the console farms and QA servers. A developer could connect from home and work inside an environment that behaved like being in the studio — because everything was connected to the same pipeline in the same region, not scattered across home broadband connections reaching back to on-prem infrastructure.

We also moved EA's Perforce footprint. The on-prem servers that everyone was hammering with remote connections were failing under the load. Failed commits, corrupted syncs, servers going dark. We expanded to cloud nodes and edge locations so developers were connecting to infrastructure closer to them geographically. Code check-ins got faster. The failures dropped.

We had some release slippages. Given what we were operating under, that's not a failure — that's a miracle of execution.

EA shipped games during COVID. Not all of them on the original schedule but we shipped. Studios that had never operated remotely for a single day figured out how to build, test, and release content with their entire workforce at home, on consumer internet connections, accessing hardware they couldn't physically touch.

What made it work wasn't any single technical solution. It was the people. There was friction, yes. There were nights when nothing worked and mornings when we started again. But something happened in that period that I haven't seen replicated since. Thousands of people across dozens of studios and functions locked onto a single goal — we will operate as a fully remote company and we will still ship great content — and pulled in the same direction with an intensity that was rare and genuine.

I've heard people talk about those years with a mix of exhaustion and nostalgia. Honestly, me too. Hard problem, great team, mission accomplished.

What it taught me about large-scale organizational change I still carry today: people can absorb enormous disruption when the mission is clear, the leadership is present, and nobody is pretending the problem is smaller than it is.