Platform Thinking

Support from the top

In a recent discussion with Matthew Rassi about Lean, I asked Matthew if he has ever witnessed Lean being successful in organizations where there is not support at the top. He answered with a resounding no.

Likewise, I’ve never observed “Platform Thinking” being successful company-wide without support from management.

If Platform thinking does not start at the top, it is a hard go.

These are cultural issues, and perhaps the most important influence those in leadership positions have is culture.

However, you can still apply platform thinking at the personal or sometimes team level. levels

Where do reliability/stability problems typically come from?

99% of the time, system bugs/problems come from the stuff you do.

The code you wrote.

The hardware you designed.

The system you configured.

The custom PRC system you created.

Rarely are we affected by stability problems in the Linux kernel, the Zephyr RTOS, an OSS library, or a bug in a chip.

Why? Because these components have many more users and developers than the in-house pieces we develop.

“Given enough eyeballs, all bugs are shallow.”

Why are we so hesitant to update the open source system software in our product, but deploy new application updates that one developer created without a thought?

And ironically, the best way to avoid the rare problem in OSS components is to keep up-to-date.

We should be most suspect of the parts of the systems with few developers and users.

This is why YOUR Platform is so important – because you reuse it, more developers work on it, and more users use it.

And by extension why it is so beneficial to open source parts of what you do. Even if you get only a few more users, it will get a lot better, making your products better.

The easiest path to reliability …

… is simplicity.

Not that we dumb things down and avoid new technology.

That is false simplicity.

As we discussed previously, the problem is most often not with the underlying technology, but rather the stuff we do.

Therefore, if we can simplify what we do, it will get better.

This is a fundamental purpose of YOUR Platform – to simplify what you do. To make it more repeatable, deployable, reliable, and stable.

Technology and tools can help, but in the end what you do and how you do it is your responsibility. You can’t buy a solution for this, nor can AI do it for you.

1 Like

What is the purpose of technology?

To do more with less effort.

To simplify our tasks.

Sometimes this backfires and technology ends up giving us more to do and complicates our lives.

It becomes a monster and we are consumed feeding it.

Thus, the selection and implementation of technology is critical.

How do you objectively measure this?

Perhaps we need a term “technical efficiency”

technical efficiency = time saved/time feeding

Hard to measure, but important to reflect on.

A number greater than 1 is good, less than 1 is bad. Ideally, the number should be much greater than one and also increasing.

Like with so many things, the key metric is not where we are now, but where we are heading.

Kayaks and Platforms

Recently we purchased a few used kayaks that had been built 20 years ago from Pygmy Boats kits. (Unfortunately, Pygmy is not longer supplying kits, but similar kits can be purchased from Chesepeake Light Craft)

We had the pleasure of visiting with the former owner (Mike) for an hour or so and he generously shared his knowledge about the boats, and kayaking in general.

The more I listened to Mike, the more I realized these boats were his platform for getting on the water.

Building a boat like this from a kit is a lot of work, but in the end you have:

  • a nice boat – performs as good as boats 4x the cost
  • one that is customized to exactly what you need (features, rigging, seat, bracing, etc).
  • can be easily modified in the future

Additionally, you have the know-how and tools to repair and maintain it.

If the hull is damaged, you can get your extra plywood, fiberglass cloth, and epoxy, and fix it.

If you don’t like the rigging, you can fasten new points and fill the old holes.

If you don’t like the seat, and can hand-carve a new one out of a block of foam.

If the boat is scratched up, a little sandpaper and varnish will have it looking like new.

It’s not for everyone, but for those who want to own their kayak platform, a stitch and glue boat is a nice option.

What are the pillars of a good platform?

Simplicity - you can make changes when needed, and understand the system enough to do so.

Consistency - useful patterns are reused throughout the system where appropriate.

Reliability - the design/implementation is sound and there is enough testing that you don’t worry too much when deploying updates.

Velocity - iterations are fast and easy to deploy.

What can kids teach us about platforms?

When helping my children with math, one of the biggest struggles is to teach them to break math problems into small steps. They are smart and think they can do it in their head, and inevitably make mistakes because everything gets muddled.

When you break a math problem into steps, it is easy – because each step is easy.

And writing out the steps is easy. It’s not rocket science.

But what is not easy is the discipline to write out the problem step-by-step instead of rushing to the answer.

YOUR Platform is the same way – it is breaking down problems into simple steps.

It’s not rocket science.

But it does take discipline to create a checklist, document a process, break things down into smaller steps, automate some things, look for patterns, etc.

Getting to simplicity is hard, but once you get there, everything is easier.

Advanced technology and small players

Integrated circuits (IC) vary widely in complexity from the small devices that are easily documented and understood to complex beasts that would require a stack of books and years to fully document.

If you are a high-volume manufacturer of consumer goods, you can easily get all the attention and support you need from IC vendors. They will help you design your product.

However, if you are a low volume manufacturer, it is nearly impossible to get direct support for complex ICs. Support resources are limited and IC vendors can’t help everyone in the world design their products.

So they allocate support engineers to a small number of higher-volume customers.

It does not matter how important you think you are, or how much money you have, if you don’t have volume, it will be hard to get direct support from complex IC vendors. Their margins are small and they make it up in volume.

However, there is a path forward for low-volume manufacturers.

You can buy system-on-modules (SOM) from vendors who do the hard parts of integrating a processor, RAM, power supplies, and other complex ICs on a module.

Additionally, they provide open-source software for these modules. Open-source has changed the game for software by providing everyone access to advanced technology.

The third aspect of this is community. If a technology vendor has a vibrant community around their products, then the community can support the low-volume customers. This is the best and most sustainable model.

The difficult part in all this is discerning where the line is. What technology is accessible to low-volume manufacturers and what is not?

A few red flags:

  • SOM or other module technologies that don’t have a clear support model for industrial customers.
  • The technology is not open and requires NDAs to access documentation.
  • The documentation is inadequate and difficult to understand. The only way you’ll figure it out is by the vendor helping you.
  • There is not a vibrant community around the products.
  • Supporting software is not open-source.

The accessibility of technology today has spurred tremendous innovation. Open-source and community have changed the game enabling even small players to use very advanced technology in their products. But you must understand your limits. There is technology that is accessible to small players and technology that is not.

IC vendors who put the pieces in place (openness, open-source, and community) can benefit from the long tail of distribution, which increasingly is where innovation is happening.

Habits – a critical ingredient in YOUR Personal Platform

We defined your platform as the collection of technology, tools, and processes that enable you to differentiate and consistently deliver your products and services.

At a personal level, habits are critical- they enable us to do the boring/hard stuff that makes us effective.

Getting up early gives us undistracted time to think.

Regularly exercising helps us in so many ways.

Patterns and tools that help us plan and focus are sometimes necessary.

Bad habits (sleeping in, overeating, reading instead of working, etc.) are easy to fall into.

Good habits are amazingly hard to establish, but worth it, and a big part of YOUR Personal Platform.

Platforms vs Architecture Astronautics

The excellent Superstruct Manifesto addresses the tendency for developers to want to “rewrite” things, wholesale switch to new frameworks, jump on new tech, etc. vs. delivering real business value.

Another description for these people is Architecture Astronauts.

Platforms, at least the way I envision them, are the opposite of this.

They are incremental, not wholesale.

They are a part of our daily thinking, not some separate grandiose effort that will solve all of our problems.

They are largely invisible.

They leverage what we already have and make it better.

They are the responsibility of everyone, from the bottom to the top, not some elite team.

However, like personal habits, company platforms take discipline to implement, and are sometimes difficult to get established.

There are exceptions, but if platform work is exciting and interesting, this is a warning flag – you may be venturing into architecture astronautics. Generally we need to block out time in our schedule to work on platforms as it does not just happen because it is fun – kind of like brushing your teeth and exercising.

But in the end, platforms make the difference whether you have a sustainable business or just some talented individuals. And just like exercising, or cleaning up our workspace, it is very satisfying afterward.

How can you make your commercial source code customers happy?

When engaging with a community around an open-source project, Github is likely the lowest friction option. It makes sense.

But there are other scenarios – one is where you are delivering closed-source commercial source code to customers.

Git makes sense for delivering any source code, no matter what the context – closed, open, etc.

Using zip files to deliver source is antiquated. Since Subversion (2000) and Git (2005), there are now other options.

With Git, is easy to deliver updates, manage changes, tag release, and track what has happened.

Tools like Gitea/Forgejo, that wrap Git, give us nice ways to interact around the source code – issues, pull-requests, projects, etc.

Gitea/Forgejo give us several things that Github does not – unlimited organizations and users that we can create and manage.

What this means is every customer can have their own organization. You can mirror the source code they have purchased to their own organizational space, and continue updating it as long as they are a customer.

If they decide to quit purchasing updates, you can leave their Gitea organization and source code intact, but simply quit updating it. This is ensures you do break build processes they already have in place and is a measure of good will.

Why not deliver the source-code to the customers Github organization? With Gitea, you are in control of the repo visibility. It is less likely someone might accidentally make a repo public in the future.

Would you like the velocity, stability, and collaboration that OSS projects have? With tools like Gitea/Forgejo, you can mirror these workflows into any software development effort.

How to get the little things done?

Previously, we discussed habits. Habits are the way we get the small, but important, things done.

Some people are naturally neat, organized, and focused – I’m not. Stuff tends to pile up in my workspace and I don’t even notice it.

One thing I’ve been using some is the Pomodoro technique. This is a time management method where you use a timer to break down work into intervals – typically 25 minutes in length separated by 5m breaks.

The primary goal of the Pomodoro technique is to increase focus, and that is a worthy goal.

However, another benefit of this technique is to allocate time for the little things, like organizing your workspace.

These are the little things that clear space to do the big things – part of YOUR Personal Platform.

Two ways to scale

Scaling is the goal of many businesses – grow revenue, serve more people, sell more devices, grow the product line, etc.

There are two ways to scale:

  1. Increase volume with a small number of direct customers. You might be making an integrated circuit that is sold to a company making consumer devices, or you might be making a consumer product that is sold through a few outlets like Amazon and Walmart. Although you might have thousands of end customers, you only have a few direct customers.

  2. Increase the number of direct customers. Many of these customers may be low volume, but the aggregate can be large, sometimes described as the long tail.

(For this discussion, we’ll assume you already have a product or technology that meets a real need).

With approach #1, you can brute force about anything. As long as the product is something people want, you can dedicate application engineers to help your customers make it happen. Sometimes the design/development team interacts directly with customers. This is expensive, and the reason many industrial products are costly is that they rely on direct support.

However, this does not scale past a handful of customers. Direct application engineering support is expensive. If you want to scale the number of customers, you need to do something different.

Open-source projects are an interesting study in this regard. They address well the needs of many users. How do they do this?

There are two primary aspects: A) Quality, and B) Community.

For any complex technology to scale without a lot of hand-holding, the quality needs to be high. There has to be good documentation. The source code or design needs to be readable. The architecture needs to be simple and solid so it can be understood. You need consistency (data structures, code organization, etc.). Testing needs to be automated and robust so there are not a lot of unexpected problems.

The second thing that is needed is community. A community is the only way to support a large number of users or customers of a complex technology. With complex systems, the problems you run into are often things you have never seen before. If all of these problems need to be solved from scratch with direct one-on-one support, you will quickly become buried. Additionally, you’ll find that experienced users are often better at solving user problems than technology creators. If this support is done in a public place, you soon build up a knowledge base that can be reused.

Quality and Community are the ways to scale the number of direct customers or users, and OSS-style workflows and communities are the best way to accomplish this.

If you want to address the long tail, the days of NDAs with every customer are past. The future is open. Not necessarily 100% open source, but certainly open documentation and support. And “open” provides feedback that increases quality – it all works together.

Rituals and Culture

Recently, we discussed personal habits.

Organizations have habits too – we call them rituals and culture.

How does the organization react when things don’t go as planned?

How does the organization motivate and demotivate people?

How does the organization encourage openness and transparency?

A lot of this probably comes down to those at the top – the art of leadership.

Organizations are made of people, so the struggles we face personally we also face organizationally.

We need some structure and tooling to help us do our job.

To ensure the boring parts are taken care of.

To catch errors we overlook.

Platforms.

Does your organization celebrate platforms? If so, how?

Improve development efficiency, reduce technical debt

The term “platform” can mean a lot of things, but this is primarily what I’m talking about in the context of product development:

Improve development efficiency, reduce technical debt

Some examples:

  • do more with what you have

  • iterate faster

  • make less mistakes

  • regularly update to new software versions

  • collaborate better

  • use better tools

  • automate where you can

  • reduce legacy (untested) code

  • deploy with confidence

  • react quickly to new customer needs or problems

Of course, everyone wants these things and claims to be doing them. But in your gut, you know if these things are getting better or worse.

And the only way I know to get better is to work on YOUR Platform.

What is a good abstraction?

When writing a program, abstraction is a powerful tool.

Every abstraction has a benefit and a cost. Hopefully, the value > cost.

There are good abstractions and bad abstractions.

Good abstractions break things down into simpler steps that others can understand.

Bad abstractions may be brilliant but are nearly impossible for anyone but the author to understand.

Who are you optimizing for? Yourself, or everyone else?

This often tells the difference between a good and bad abstraction.

A platform stuck in time

Vintage is great in some contexts—I like old woodworking hand tools. Even in our modern jobs, some vintage tools, like a pencil and paper, are still very useful.

But information technology moves forward, and we rarely benefit from using decade-old phones, computers, programs, etc.

New information technology and tools are often simpler to use and more powerful. Some examples:

  • C → Go

  • Apache → Caddy

  • Vim → Helix

  • x86 → ARM → RISCv (MPU)

  • 8051 → PIC → ARM → RISCv (MCU)

  • FreeRTOS → Zephyr

  • Kafka → NATS

  • Rotating storage → Solid state

Information technology, and perhaps computer technology in general, is different. The physical constraints are largely removed as we scale into GHz and Terabytes, which opens the doors to human innovation and collaboration.

Innovation is largely a process of discovery, so you can’t predict where it will go and what will happen. It is a result of development iterations that are now happening at a massive scale and distributed across the globe.

Therefore, when you lock yourself into some version of technology in the name of stability, cost savings, or whatever, you are also limiting the potential for your product – for interesting things to happen that no one could have predicted.

It takes humility to acknowledge this – that you can’t see the future, the best you can do is prepare for it.

YOUR Platform is the vehicle that keeps your product development moving forward with technology so that you are not stuck in time as you watch it pass you by.

Coupling

Good platforms emphasize loose coupling where one part of the system can change without affecting another.

Not abstractions for the sake of abstraction.

But rather well-thought-out data structures and patterns.

A message bus like NATS is an excellent way to decouple a program. One part of the system publishes information, and any other interested part can subscribe.

But for this to work, you must have simple and flexible data structures that don’t change often. Otherwise, your system is now tightly coupled again.

Simple data structures and messaging – a sane way to scale YOUR Platform.

Creativity

“Creativity is just connecting things. When you ask creative people how they did something, they feel a little guilty because they didn’t really do it, they just saw something. It seemed obvious to them after a while. That’s because they were able to connect experiences they’ve had and synthesize new things" – Steve Jobs

(Thanks Rod for this quote!)

As technology advances, being good at making connections (integration) is the critical skill, as we can’t create it all ourselves.

Part of YOUR Platform is building a high-quality toolkit of things to connect.

And knowing how, when, and where to connect.

Architecture vs. Optimization

In 1968, “The Art of Computer Programming” by Donald Knuth was published with the following quote:

“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.”

This was in the age when computer resources were scarce.

We now live in the age of abundant computing resources, so it seems this statement would be many times truer now.

Performance is important when we select technologies. For instance, Go or Rust is significantly faster than Python and it is important to think about issues like this up front. This is an architectural issue.

Deciding if we will use a message bus or not is an architectural issue.

However, if we receive data over a network, especially in distributed IoT systems, the network is relatively slow. So it rarely pays to optimize how we process data from remote systems until we reach a pretty large scale. This is an optimization issue.

Creating an abstraction to reduce some duplicated code is an optimization.

With architecture, we need to think about it up front.

It rarely pays to optimize up front – that is best done later after some learning.

And if we spend too much time optimizing, we never get to the learning. And if we don’t learn from testing a working system, we’re likely optimizing the wrong things.

Architecture vs Optimization – there is a difference.