A lot of good thoughts in this episode – highly recommend listening to this one.
This episode explores how Spotify transitioned to a “platform” culture. Spotify has a culture of autonomous hard-working super-hero coders. But as they grew (quadrupling their size every year), this mode of operation did not scale, yet they wanted to retain the benefits of autonomy.
They had learned that from the external auditors, that we had too many vulnerabilities in our CI infrastructure. IE, we had over 200, um, bespoke Jenkins is running at various stages of cleanliness, sort of.
Pia Nilsson was hired to fix this CI mess. Due to the Spotify culture, they did not have the authority to force any platform changes on anyone. So instead they focused on being useful. So they came up with this approach:
And it’s still a part of our, our main engineering practices, uh, for the platform mission, which we called the platform takes the pain. It really helped us actually, because it’s short and snappy and everyone knew. What that really means, because it’s tedious to migrate someone off a Jenkins to another CI engine.
This is not sort of the lovely ivory tower work that maybe some platform teams had loved doing. Deep, deeply thinking about sort of orchestration and creating some fantastic product. But this is actually sort of going out there to some feature teams. 34, 50 feature teams. And sitting with them, and helping them, because they have different challenges, all of them.
Where is the autonomy really? We started to learn how to nuance that understanding because autonomy has to do with impact. We’re not interested in being all alone in a room being autonomous. That’s useless. When we say autonomy in the engineering industry, right, we mean actually impact.
The platform team learned to own the adoption of their tools.
Adam: This change shifted the team. They went from ivory tower architects, dreaming up theoretical solutions, customer focused engineers.
Pia: Because we really weren’t customer oriented at all when I joined. Nobody had asked the engineers to be customer oriented. It wasn’t that they didn’t want to, they, nobody had thought about it basically. But this owning your own adoption, they have to become and they will become customer oriented because they’re gonna go out there and fix that adoption.
In the end, they built a developer Platform called Backstage.
Backstage has a plug-in architecture with a number of open source and commercial plugins.
They open-sourced Backstage and the reason is quite interesting:
And when something becomes that critical, you got to protect it as well. And one of the fears that was starting to arise was Hold on. This is now super important to us. What if someone builds an internal developer portal and it becomes the industry standard? At the time, there were no developer portals, basically, in the industry.
But we were sort of thinking we would have to migrate off of backstage because we are not going to sit there with a bespoke system. And that’s going to be so painful for us because now we have every single team. On backstage and so many plugins over 200 plugins actually internally built. So this was a real threat to sort of our speed again.
And so we, we made the decision, like we’re going to actually donate this. We’re going to give it away and make sure to keep investing in open source backstage because we need it. We need it to be the industry leader for ourselves, for our speed. So that was why we open sourced back in 2020. That we’re really celebrating.
