Platform Thinking

✅ Each release is 100% done

2026-03-12T00:00:00+00:00

One common problem in product development is developers not finishing a feature or task. They get it to 70% done and rush off to the next thing. Bugs linger, edge cases go unhandled, testing falls short, and the release process remains incomplete. "100% done" does not mean the product is 100% done — it means each release is complete, tested, and high quality. Quality products emerge from many small high-quality steps, not from a pile of slop that somehow gets polished by QA at the end of the development cycle.

Release First is a simple way to ensure each step in the journey is 100% done and high quality.

💡 A Simple Example

2026-03-11T00:00:00+00:00

The Release First mindset starts with how you handle reasonably complex information (a procedure, technical details, etc). There are two approaches:

Email or message the information.</li>
Put it in Git and send a link or PR.</li> </ol>
The first is transient, inaccessible, and difficult to build on or improve.
The second is permanent, reusable, accessible, and something people can build on. It is an asset.
Sure, there are times when email or messaging is the appropriate forum for discussions. But the minute there is something of value, turn that into a permanent, team-accessible asset. This is how teams build value.

🏷️ The simple approach to versioning

2026-03-10T00:00:00+00:00

The biggest barrier to starting something is an unwillingness to start simple. Let's use versioning as an example. The natural tendency is to think that all software components need their own version, which can be a daunting task to implement, track, and manage. However, until a team is large enough to have multiple groups dedicated to different tasks, instead put everything in a monorepo</a> and use a single version for everything. Not every component needs to be released with every version, only the ones that have changed. There are several advantages:

There is a single changelog to review.</li>

It is easy to coordinate things - components of the same version should work well together.</li>

Much easier to use AI to execute tasks that involve multiple components. If there are several repos, you are always fighting security checks when crossing repos (at least in Claude Code).</li>

It is easy and simple.</li> </ul>

"But!!!," you say, "The versions won't always be sequential, some will be skipped for various components!!!" Why does this matter? If it bothers you, release every component for every version. Automation can easily do this.

The resulting changelog looks something like this:

2026-03-06 v3.0.18</span></span>
</span>
[Server]</span></span>
</span>
- Fix mongo iterator stopping on decode errors instead of skipping bad documents</span></span>
  Add bson truncate tags to all int fields to handle float64 values from</span></span>
  JavaScript</span></span>
</span>
[GW]</span></span>
</span>
- Improve response of Modbus polling.</span></span>
</span>
[Portal]</span></span>
</span>
- Add UI for e2e test status.</span></span>
</span>
2026-03-06 v3.0.17</span></span>
</span>
[Server]</span></span>
</span>
- Log rule count each processing cycle to aid debugging</span></span>
</span>
2026-03-06 v3.0.16</span></span>
</span>
[E2E testing]</span></span>
</span>
- Improved ntfy error messages to include response body for easier debugging.</span></span>
- Reduced log noise: only log pump and cycle mirror state changes instead of</span></span>
  continuous tank level.</span></span>
- Added logging of test status on initial check and state changes.</span></span></code></pre>
Start simple, and add complexity when there is a need.</p>
</p>


📦 The bare minimum to call something a release
2026-03-09T00:00:00+00:00
What is the bare minimum to call something a release? Consider the following:</p>

A version</a> that increments with every release.</li>
A changelog</a>
that describes what has changed for every version and why it matters.</li>
A clear identification of any assets
(IPNs</a>).</li>
Unique IDs for anything built
(HFIDs</a>).</li>
A place to store stuff where all this gets tracked over time
(Git</a>).</li>
</ol>
The concrete benefits:</p>

Exact knowledge of what exists.</li>
Clear record of who did what when.</li>
Clear communication to everyone involved.</li>
Nothing gets lost.</li>
</ul>
Doing the above for all design assets (including documentation) from day one</strong>
in the development process improves thinking, communication, and
effectiveness — and reduces untold confusion and wasted time.</p>
This does not need to be hard. Simple text files, spreadsheets, and CSV files
work great. It is perfectly fine to start small.</p>
</p>


❓ Release Questions
2026-03-06T00:00:00+00:00
Some questions to consider about a release:</p>

When is it created?</li>
Where is it stored?</li>
What does it contain?</li>
How is it created?</li>
Who uses it?</li>
Why is it used?</li>
</ul>
The "Wh- questions"</a>
are a powerful technique for analyzing anything we do.</p>
</p>


🤔 Two types of unknowns
2026-03-05T00:00:00+00:00
There are two types of unknowns:</p>

Things we don't know.</li>
Things we think are true, but are not. 1</a></sup></li>
</ol>
Both can inflict havoc on our plans, but the second is the most damaging. Why?
The first is more associated with doing nothing. Bugs not known to be there,
etc. But the second prompts people to take action and go down a path that can be
difficult to undo (promised ship dates, features, etc.). It is not too hard to
say "oops, I did not know about that problem - we'll fix it." But it is really
hard to say "sorry, I was completely wrong about that." - this takes a great
deal of humility.</p>
While avoiding #1 is important, avoiding #2 is critical, because that forces
people to retrace their steps and admit they were wrong. While humbling oneself
is beneficial for character development, practically, it is difficult (if not
impossible) for most people to do, especially those in leadership positions. So
let's
Release First</a>, and
learn where we are wrong in little ways, rather than have a
BIG 1.0</a> humiliation.</p>
^{1</sup>
"It ain't what you don't know that gets you into trouble. It's what you know
for sure that just ain't so." - Mark Twain (maybe)</p>
</div>
</p>}


🧊 The hardest thing to manage
2026-03-04T00:00:00+00:00
The things we know and understand are easy to manage. We can hire managers who
are good at this. Tools can effectively track this. Life is good. The difficulty
arises in the unknown. The iceberg analogy illustrates this well. The things we
see, understand, and touch are above water. Everything else is underwater. This
is often where unexpected problems arise. As systems get more complex, the size
of the unknown grows. We can't predict what will happen in the unknown; the only
thing we can do is release and integrate early.</p>
</p>


🎉 The BIG 1.0 Release
2026-03-03T00:00:00+00:00
BIG releases are appealing — the magical moment when everything comes together.
The months or years of hard work that finally come to fruition. The excitement
and fanfare of the BIG announcement. The accolades and admiration of customers
and the press.</p>
... except, it
rarely works this way</a>
unless you are Apple or Tesla. And what we don't see with high-end consumer
products is the rigorous internal release and testing programs ... something
lower-volume industrial products don't have the luxury and resources to do.</p>
The BIG Release mindset assumes all the code and designs will magically ship
when the BIG day comes. It never goes this smoothly. Sure, we may build
prototypes and do testing, but unless we do these with release processes we can
reuse, it is hard to be confident we can do it again quickly, easily, and
accurately. One-off prototype processes don't scale. Only release processes
scale. And the reality is that the rigor of releases in the prototype phase
works better anyway.</p>
The solution is of course ... Release First. The problem is that The BIG 1.0
Release fits human nature better (procrastination, excitement, and glory).
Release First is boring, tedious at first, and no one cares. The effort is
invisible and rarely gets rewarded. Very few developers are wired for Release
First. Thus, Release First is not a technical problem, but rather a cultural
one.</p>
</p>


❓ What is the difference between "releasing" and "shipping"?
2026-03-02T00:00:00+00:00
Someone asked: "What is the difference between releasing</strong> and shipping</strong>?"
Are they the same thing?</p>
In one sense they are. However, drawing a distinction here—though somewhat
arbitrary—is helpful in this context.</p>
Release:</strong> Make something available.</p>
Ship:</strong> Deliver to the customer for sale.</p>
Releasing is part of the process to get something to the point where it provides
enough value that customers will pay for it. Releasing is the way useful work
gets shared with others and the delivery process gets refined. Releasing enables
early feedback needed to know exactly what to build. The Release is the way
different disciplines communicate around what the team is building. Releases are
a concrete measure of development progress. Releases build confidence in the
product.</p>
The problem arises when the two are considered equivalent—when there is no
release until ship. Then inevitable surprises pile up.</p>
</p>


🚨 Problems with not Releasing First
2026-02-27T00:00:00+00:00
What is the downside of not
Releasing First</a>?</p>

Integration problems</li>
Manufacturing issues</li>
Last minute bugs appearing</li>
Lack of product fit (does not solve customer's problem)</li>
</ul>
The common theme is unforeseen problems that typically surface late in the
development cycle. The point of Release First is to expose unknowns early, when
there is still time to address them.</p>
</p>


🌅 Why optimizing the user release experience early is so important
2026-02-25T00:00:00+00:00
With Release First, the goal is to optimize the
Big Blue Button</a> (make
it easy for users to use a release) early. Why is this important? There are many
reasons, but the most important is so that users actually use the release and
provide needed feedback early.</p>
Without optimizing the release experience early (when the
circle</a> is small),
developers soon become inundated with support issues as the circle grows.
Pretty soon the developers are doing nothing but support. Optimizing the user
experience early enables scaling.</p>
Fix things early, when the stakes are low, not after the BIG 1.0 Release.</p>
</p>


🔵 The Big Blue Button
2026-02-24T00:00:00+00:00
In the past days, we've been focusing on the
Big Green Button</a>,
but there is another side of this - what happens after the release? Does the
release get used? If not, why not? If a release does not get used, there is
little point in releasing.</p>
There are several aspects to consider in the usability of a release:</p>

Communicating to users that a release is available and relevant to them.</li>
Making it easy to consume the release.</li>
An efficient way to support and get feedback from users.</li>
</ol>
We have a Big Green Button to create a release, so let's continue the analogy
with a Big Blue Button that makes it easy for a user to consume a release. We
can't just throw something over the wall (that may be technically very good and
complete) and expect magic to happen without any consideration for the user.
What does the Big Blue Button include?</p>

Clear documentation, tutorials, videos.</li>
Well organized assets that are consistent from release to release.</li>
An easy way to obtain/install/update the release.</li>
Built-in help functionality, diagnostics, automated error reporting.</li>
A community where the user feels included, supported, and part of something
bigger than themselves. This is important to have, even early on when the
internal team is the only user base.</li>
</ul>
One of the key warning flags is when a bunch of hand-holding and personalized
help is required for someone to use a release. Can manufacturing take what the
team produced and run with it? Can someone use a software service without
personalized support? Can someone set up and configure a product without a dozen
support calls? The point is not to avoid talking to users, but rather to talk to
them after they have used the product and can add value to the development
process.</p>
There needs to be a symmetry between the Big Green Button and the Big Blue
Button - easy to produce and easy to consume, otherwise the flow will stall.
These are the factors that will differentiate developers and products in the AI
age.</p>
</p>


🧠 The Human Factors
2026-02-23T00:00:00+00:00
Gerald M. Weinberg</a> once
said:</p>

No matter what the problem is, it's always a people problem. — Gerald M.
Weinberg</p>
</blockquote>
At its heart, release-first addresses the human element:</p>

Our propensity to procrastinate in getting releases out.</li>
Our fear that something might be broken in the release.</li>
Our tendency to think short-term instead of long-term.</li>
Our tendency to rush on to the next feature before finishing the one we
started.</li>
Our tendency to skip tests when writing code.</li>
Our OCD in needing to perfect something before anyone else can see it.</li>
And the list goes on ad infinitum</em> ...</li>
</ul>
Release-first at its core is a way to address human weaknesses. It is a simple
tweak in our behavior that anyone can do. It is a language that everyone can
understand. It can be measured by anyone without fancy tools. It is so absurdly
simple that most people will write it off as "obvious" or "too simple" and
instead reach to the
merchants of complexity</a>.
But true effectiveness comes when we reduce things to the simplest form
possible, and remove all the accidental complexity.</p>
So the next time you're struggling with a release process, a testing gap, or a
feature that never quite ships — remember Weinberg. The problem isn't the tools,
the framework, or the process. It's us. Release-first works because it doesn't
try to engineer around human nature — it simply makes the right behavior the
default behavior.</p>
</p>


☝️ What do we mean by "First"?
2026-02-20T00:00:00+00:00
What do we mean by Release "First"? Several things:</p>

First in the timeline</strong> of development.</li>
First thing we focus</strong> on during implementation.</li>
First priority</strong>.</li>
</ul>
Simply put, implement the
Big Green Button</a>
before implementing the actual deliverable. This might include:</p>

Tests</li>
CI/CD</li>
Build automation</li>
Deployment scripts</li>
PCB release automation</li>
Clear user documentation</li>
etc.</li>
</ul>
This becomes part of the review process, part of the culture, part of the
expectation, and part of the daily habit — make it easy to release.</p>
Because the
Big Green Button</a>
does not ship to customers, forgetting and neglecting this step comes naturally.
Release First is a simple hack to ensure it gets done before the rush to do
other things.</p>
</p>


🟢 What is in the Big Green Button?
2026-02-18T00:00:00+00:00
As previously stated, the
Big Green Button</a>
enables teams to quickly and confidently release. But what is the Big Green
Button? Several things come to mind:</p>

Automation that translates code/design/docs/models into assets others can use.</li>
Automated testing that builds confidence in the release.</li>
</ul>
These save developers time by avoiding repetitive, tedious tasks, avoiding
mistakes, and reducing support load. These investments have a very quick
payback.</p>
The point with the Big Green Button analogy is that once we have a Big Green
Button, we don't think about what is inside the button every time we release,
our minds are free to focus on other things.</p>
</p>


🟢 The Big Green Button
2026-02-17T00:00:00+00:00
The word "release" can be a noun (what gets released) or a verb (the process of
producing a release). "What" and "How".</p>
Many teams produce complete high-quality releases that allow them to produce or
deploy something at some point in time. However, each new release requires
significant manual effort:</p>

Manually testing software to make sure it is solid.</li>
Manually going through the process of generating output formats, and updating
drawings for manufacturing. This might involve double-checking the output
files.</li>
Manually updating BOMs and top level information.</li>
Manually updating instructions, manuals, and other released documentation.</li>
</ul>
An alternative to the above is the "Big Green Button" - a single step that can
convert designs/code/models/documentation into something others (manufacturing,
sales, customers, stakeholders, etc.) can use. This is the critical point - do
you have a "Big Green Button"? Without one, releases naturally get delayed until
the last minute — losing opportunities for critical early feedback and the
ability to quickly response to new issues or needs.</p>
The Big Green Button is what gets the work from developers to users - quickly
and confidently. And this is something BEC</a> can help
with.</p>
</p>


🔄 The release circle
2026-02-16T00:00:00+00:00
As previously discussed, a release is
useful and quality</a>.
What counts as a release? One way to think about this is to view the release as
a circle with two attributes:</p>

What (it contains)</li>
Who (it's for)</li>
</ol>
Initially, there is not much in the circle (perhaps only a specification), and
only the development team and a few others in the organization are interested.
But it is something that can be used and built upon. The team will change and
expand it, but it will continue to exist in some form in all future releases.
This might start with a spec, then a UI prototype in the cloud, then a device
prototype based on a single-board-computer, then a custom system-on-module PCB,
and so on. As time goes on, the functionality expands and improves.</p>
The circle of who uses a release also expands over time, to include sales and
marketing, management, beta-customers, and finally a general product release.
Over time, the number of customers also grows to include more varied use cases.</p>
The key point is to release a small slice of permanent</em> functionality early,
and then continue to grow/add slices and the number of people who use the
product during the product development cycle</em>. Waiting until the BIG Release is
too late. It's OK to start small. As long as it's
a release</a>.</p>
But there are things that are not a release:</p>

Emails</li>
Messages</li>
Meetings</li>
Memos</li>
Experiments</li>
Prototypes (that are not released)</li>
</ul>
Why? Because this information is not permanent. While others consume it and find
it very useful, it is transitory and cannot be continually used, refined, and
improved over time. It is one-off communication.</p>
</p>


🧩 Why things fall apart at the end
2026-02-11T00:00:00+00:00
As discussed in the previous two posts, things often go haywire near the end of
a project cycle, right before the BIG Release. Why is this? Things seem to go so
well at the beginning of the project. While it is impossible to detail every
reason this happens, a few are listed below.</p>
Technical debt</strong> occurs when problems are half-solved. Things appear to work,
but edge cases and inadequate testing lurk beneath the surface. As time
progresses, the system becomes more fragile, and significant amounts of time go
toward fighting bugs and unplanned problems.</p>
Scope creep</strong> is a common phenomenon where features continually accumulate
throughout the project. Some of this is natural if it's based on user feedback
and learning while implementing. However, there is an unnatural scope creep
rooted in engineering whims, just-in-case thinking, and a developer's natural
propensity toward complexity. Just because something can be done does not mean
it should be.</p>
Integration issues</strong> occur when one piece of work combines with another. This
is often when new edge cases surface.</p>
Not correctly anticipating user needs</strong> is another problem. A team may build
something thinking the requirements are understood, but it is not until people
start using it that the real needs emerge.</p>
Lastly, there are unknowns</strong> that simply cannot be anticipated. Complex
systems are just that — impossible to completely understand. People involved at
every level are fallible and unpredictable. A large amount of uncertainty simply
cannot be known or predicted up front.</p>
The simplest way to address all of these issues is to move the end closer to the
beginning — Release First.</p>
</p>


🪞 The illusion of project meta-data
2026-02-10T00:00:00+00:00
Many tools exist for managing and tracking projects: Jira, Trello, GitHub
projects/issues, Notion, Airtable, Asana, MS Project, etc. These all can be very
useful. Why then does effective project tracking remain so elusive? As mentioned
in the
previous post</a>,
in a typical scenario, things appear to progress well. Issues get checked off.
The burn-down chart tracks the line. But out of nowhere, monsters (bugs,
unanticipated feature requests, and integration problems) appear that catch
everyone off guard.</p>
Several factors contribute to this problem. Foremost, the meta-data is exactly
that — "meta." It is data about something; it is not the thing. There is no
guarantee it is complete and accurate. If the focus stays on the meta-data, the
meta-data will (perhaps unknowingly) be gamed to look good. If developers get
rewarded for checking issues off every sprint, they will eventually make sure
that happens, even though quality, maintainability, and other factors suffer.</p>
I love tools, and use them every chance I get. I write
new tools</a> to solve problems. Tools are what
amplify our efforts. But tools in themselves don't seem to be the answer. How
can we ensure our meta-data is accurate and not missing things? How can we know
our tools are not lying to us?</p>
</p>


🚢 What is the #1 problem in product development?
2026-02-09T00:00:00+00:00
I'll go first ... a very common challenge is shipping on time and the resulting
uncertainty to the business. We often add a 2-3x factor on any estimates, but
sometimes even that is not enough.</p>
Shipping (delivering value) is the bottom line — it generates revenue and
sustains the business. Delivering new products on time and providing subsequent
updates are critical. Without shipping, there is no value delivery. Often
businesses collaborate with customers throughout the development process. When
things are late, customers and partners lose confidence. Development team morale
tanks. It's a downward spiral.</p>
This problem has an even more painful manifestation — finding out just before the
BIG Release date that things are not coming together. As the saying goes, the
last 10% takes 90% of the time.
Surprises like this are devastating to everyone involved. Why do things fall
apart near the "end" of a project? Why don't these issues surface sooner? These
are complex and perplexing problems, but worth exploring in the coming days.
Thoughts on this are always welcome.</p>
</p>


📦 More definitions of a release
2026-02-04T00:00:00+00:00
A release must be: reliable, useful, repeatable, and predictable</em>. This
delivers outcomes of: quality, value, timeliness, and consistency</em>.</p>
The second group is the outcome we want, the first group is what we focus on to
get there.</p>
</p>


🛠️ Do things that don't scale
2026-02-03T00:00:00+00:00
From Paul Graham's popular essay
Do Things That Don't Scale</a>:</p>

Over-engaging with early users is not just a permissible technique for getting
growth rolling. For most successful startups it's a necessary part of the
feedback loop that makes the product good. Making a better mousetrap is not an
atomic operation. Even if you start the way most successful startups have, by
building something you yourself need, the first thing you build is never quite
right. And except in domains with big penalties for making mistakes, it's
often better not to aim for perfection initially. In software, especially, it
usually works best to get something in front of users as soon as it has a
quantum of utility, and then see what they do with it. Perfectionism is often
an excuse for procrastination, and in any case your initial model of users is
always inaccurate, even if you're one of
them. 7</a></p>
The feedback you get from engaging directly with your earliest users will be
the best you ever get. When you're so big you have to resort to focus groups,
you'll wish you could go over to your users' homes and offices and watch them
use your stuff like you did when there were only a handful of them.</p>
</blockquote>

I should mention one sort of initial tactic that usually doesn't work: the
Big Launch</strong>. I occasionally meet founders who seem to believe startups are
projectiles rather than powered aircraft, and that they'll make it big if and
only if they're launched with sufficient initial velocity. They want to launch
simultaneously in 8 different publications, with embargoes. And on a Tuesday,
of course, since they read somewhere that's the optimum day to launch
something.</p>
It's easy to see how little launches matter. Think of some successful
startups. How many of their launches do you remember? All you need from a
launch is some initial core of users. How well you're doing a few months later
will depend more on how happy you made those users than how many there were of
them. 10</a></p>
So why do founders think launches matter? A combination of solipsism and
laziness. They think what they're building is so great that everyone who hears
about it will immediately sign up. Plus it would be so much less work if you
could get users merely by broadcasting your existence, rather than recruiting
them one at a time. But even if what you're building really is great, getting
users will always be a gradual process — partly because great things are
usually also novel, but mainly because users have other things to think about.</p>
Partnerships too usually don't work. They don't work for startups in general,
but they especially don't work as a way to get growth started. It's a common
mistake among inexperienced founders to believe that a partnership with a big
company will be their big break. Six months later they're all saying the same
thing: that was way more work than we expected, and we ended up getting
practically nothing out of it. 11</a></p>
It's not enough just to do something extraordinary initially. You have to make
an extraordinary effort</em> initially. Any strategy that omits the effort —
whether it's expecting a big launch to get you users, or a big partner — is
ipso facto suspect.</p>
</blockquote>
What Paul is talking about here is
Release First</a> — the
extremely tedious, manual effort required to get usable stuff in front of real
users. There is no substitute.</p>
</p>


🔮 Can we predict product development schedules?
2026-02-02T00:00:00+00:00
Estimating or predicting product development is difficult. Teams often end up
adding a 2-3x padding to any estimate, because bitter experience has taught them
not to depend on their own estimates.</p>
There will always be a high degree of variability in product development,
especially software development. Product development involves doing something
new. Teams discover exactly what they are building along the way. No one can
completely plan it up front.</p>
However, we can do better. What is most damaging is not that a product is six
months late, rather it is that we did not figure this out until shortly before
the original anticipated release date.</p>
Release First is a simple solution. By the time the planned release date comes
around, you have released dozens if not hundreds of releases. Some releases may
be documents, some may be hardware, and others software. But there have been
many opportunities for others to use something and provide feedback.
Additionally, you have had lots of practice releasing so there are no surprises.
We may have learned that it will take an extra 6 months to finish the product,
but we learned that early and the business has had time to adjust. Or we may
have gained so much efficiency through automation, and reduced wasted effort to
a level that we come out ahead of schedule (what we really want).</p>
Release First eliminates surprises, and that is what matters in business.</p>
</p>


🚀 How to make releases painless
2026-01-29T00:00:00+00:00
In the olden days, releases were a lot of work. The process involved manually
building/testing, uploading to a distribution location, sending out
notifications to interested parties, etc. Supporting multiple platforms meant
even more work.</p>
Today, there are better ways. With modern CI/CD workflows, pushing a tag to a
repo triggers GitHub Actions to handle the rest.</p>
As an example, in the BRun</a> project, there are
several key pieces:</p>

Tests</a> run
on every PR commit to ensure nothing breaks.</li>
Go makes it
super easy</a>
to build for other platforms. Anyone who has cross-compiled C/C++ knows the
pain.</li>
The amazing GoReleaser</a> tool takes care of building
the release binaries and uploading them to GitHub.</li>
A
script</a>
automatically extracts the Changelog for the release that gets added to the
Release.</li>
A GitHub action runs GoReleaser whenever the repo is tagged with 'v*'.</li>
GitHub sends notifications to watchers of the project. (Consider watching the
BRun project!)</li>
</ul>
The result is beautiful releases</a> for
every platform (Windows, MacOS, Linux, ARM, x86, RISC-V) created whenever
anything is improved, no matter how small. There is no friction to creating a
release.</p>
Part of making all this happen is choosing the right technology. When it comes
to release and deployment, Go makes things really easy (compared to C++, Python,
Ruby, Java, etc.). There are trade-offs so make sure release is factored into
technology decisions. One way to ensure this happens is
do it first</a>.</p>
</p>


📖 When should you write the user manual?
2026-01-27T00:00:00+00:00
Most of the time, teams write the user manual at the end of a product
development cycle right before the
Big 1.0 Release</a>.
It is one of those things that is not much fun, and teams subconsciously put it
off while fighting bigger fires. But what would happen if we write the user
manual first - even before the product spec? Some things that might happen:</p>

We might think a little more from the user's perspective during the
development.</li>
Our upfront thinking would improve through the process of writing.</li>
Internal/beta releases would have documentation, allowing others to use them
without a lot of hand-holding (saving developers time).</li>
We would get feedback on the user manual early in the development process so
it could be improved before release.</li>
</ul>
Good user documentation is part of a good release - it helps make the release
useful</a>. With
doc-driven development</a>,
I typically start at the repository
README.md</a> or other documentation close
to the code, but why not start with the user manual and derive more from that.
Start with the experience you want the user to have. Reverse the process.</p>
</p>


🎬 Is a Demo the same as a Release?
2026-01-26T00:00:00+00:00
Some development teams have a weekly demo where they might show what they are
working on to others in the company or the customer, which is a great idea as it
encourages hitting milestones and early feedback. However, is the "demo" just
another name for an "internal release?"</p>
The "release" and "demo" mindsets have some differences. For a demo, quality or
automation may not be a priority. Engineers may record a video or present the
demo. Again, all of these are great ideas and should be done, but they are not a
release.</p>
The key thing about a release is that someone else, other than the developers,
uses it. And the developers are confident enough to let other people use it.
This difference seems small on the surface, but is huge underneath. A release
comes with an expectation that drives automation and quality. The demo can
become a crutch that allows teams to avoid the hard work of delivering something
others can use. The mindset matters.</p>
Again, the first internal release need not be feature complete and polished, but
whatever slice of functionality the team delivers should be usable by others
without a lot of props and hand-holding. Beware of proxies for delivering real
value: demos, closing tickets, milestones, etc. While these are all useful
tools, they are not an end in themselves. Measure value, not proxies.</p>
</p>


⏱️ How do you find time for higher value tasks?
2026-01-23T00:00:00+00:00
Do you (and by extension your organization) find yourself mired in minutia and
struggling to find time for higher value tasks? What is the solution?</p>
You can hire more people to do the lower-value stuff (a perfectly valid
approach). You can also own
YOUR Platform</a>.</p>
One of the goals of
Release First</a>/Platform
Thinking is to shift time from low-value tasks (tedious repetitive work) to
high-value tasks (creative, innovative work).</p>
</p>


⚖️ Reliability tradeoffs
2026-01-22T00:00:00+00:00
Recently, one of our servers running Arch Linux refused to boot after updates.
The fix was to comment out a line in the /etc/default/grub</code> file. It took
about 15 minutes to find and everything was done in a rescue terminal — no need
to boot from a rescue image. This server has been running for several
years, been updated hundreds of times, and this is the only time it would not boot.</p>
Many dismiss Arch as a server OS because there is supposedly a higher likelihood
of things like the above happening. But in practice, they rarely happen and are
fairly quick to fix. So on one side of the equation:</p>

Easy to administer OS, latest software, no OS upgrades, etc. This saves a ton
of time.</li>
</ul>
On the other side:</p>

Occasional effort to fix an issue like the above.</li>
Downtime while doing the above.</li>
</ul>
For a high-profile eCommerce site, downtime would matter, but for a small
business, 30 minutes of downtime makes no practical difference. The time saved
running Arch greatly swamps (by an order of magnitude or two) the time spent
fixing things that break. Admin time is many times more valuable than server
up-time. Even spending half a day rebuilding the site from scratch (which is
mostly automated) would not matter.</p>
There is no one-size-fits-all. Just because Summit Federal Bank runs Crucible
Linux Enterprise edition does not mean everyone else needs to. Always consider
the
benefit-cost ratio (BCR)</a>.</p>
</p>


⚖️ Reliability, Cost, and Value
2026-01-19T00:00:00+00:00
Everything has a cost and value to the customer. How much value does the
customer put on 99.999% reliability vs the value of other features they need? If
one system out of a thousand fails every 3 years (0.033% failure rate per year),
what does that cost compared to a set of features that allows the customer to
reduce operating costs 10%, or improve their general yearly reliability by 1%?</p>
Certainly, safety-critical systems where human lives are involved are impossible
to quantify with simple equations. But when considering operational equipment
involving machines, understanding the real concerns of the customer and
delivering the most value becomes paramount.
Reliability/security/performance/whatever is just one factor in that equation,
especially when it costs them other things they need.</p>
</p>


🔗 What defines reliability?
2026-01-16T00:00:00+00:00
Reliability is a common concern - this thing is critical - it needs to work!</p>
Many things affect the reliability of a system:</p>

Bugs in software we write</li>
Bugs in 3rd party components (OSS, etc.)</li>
Hardware failures</li>
Edge condition failures (things we have not tested)</li>
Tolerance stack-ups</li>
Damage in the field</li>
The list goes on ...</em></li>
</ul>
Some common assumptions:</p>

Hardware is more reliable than software.</li>
Safety certified operating systems are more reliable than non-certified.</li>
US-made components are more reliable than those from other countries.</li>
</ul>
Here's the thing ... for 99% of products, most of the above does not matter. The
most likely source of bugs and reliability problems is
the stuff we did</a></strong>.
If SafeRTOS is 99.999% verified and Zephyr is only 99.9% verified, it does not
make much difference if the software we wrote is only 90% verified. Likewise, if
the tricky analog hardware circuit has not been tested in every edge condition
and verified at every combination of component tolerances, it may not be as
reliable as a simpler RTOS application running in an MCU.</p>
Practical reliability is what matters, not theoretical reliability. What defines
reliability in most systems is that the design can be verified. A common MCU and
RTOS (like Zephyr) has already been extensively verified and tested to be
reliable in most conditions by many different people. Writing a relatively
simple application on top of a complex, but verified system makes the job
easier. The difficult verification work gets pushed off on someone else. This
same analysis also applies to concerns like performance and security. The system
is only as good as the weakest link. Reliability does not always directly
correlate to system complexity, but rather what percent can be proven correct.</p>
There are no easy answers here, but don't strain at a gnat and swallow a camel -
latching onto theoretical reliability/performance/security/whatever, but
ignoring the practical concerns.</p>
</p>


🚀 Shipping faster - what are the stakes?
2026-01-14T00:00:00+00:00
What do project delays cost? The obvious thing is the loss of revenue. Time is
money.</p>
However, there are other less tangible consequences:</p>

Customers lose interest.</li>
Team loses motivation.</li>
Supporting organizations (vendors/communities) may not take the team as seriously.</li>
Automating and improving the process never happens as there is more and more
pressure to ship and no time to do anything else.</li>
</ul>
Think of this as a flywheel, either speeding up or slowing down. The forces on this flywheel are imperceptible - a small push here, some
drag there.</p>
The simplest way to speed up the product development flywheel is to ship faster
by focusing on quality releases</a>.
Start by shipping quality
internal releases</a>.</p>
</p>


🔍 It is often more work to review something than to do it
2026-01-13T00:00:00+00:00
As an experienced person, I am sometimes called on to review something. This can
be a frustrating, seemingly hopeless task. With one developer, I commented that
he could write code faster than I could review it. While there are some issues
you can spot from a cursory look, most issues are subtle in nature—otherwise the
original designer would have probably caught them. Therefore, you pretty much
have to go through the entire design exercise again (especially with hardware)
to find issues. In many cases, it would have been faster to simply do it
yourself. This is the quandary many experienced developers find themselves in
when working with junior people. They are responsible (because they are the
reviewer), but feel helpless to ensure quality.</p>
This is also the quandary we find ourselves in with AI. AI agents can write code
much faster than any human can practically review. Therefore, the thought that
"with AI code, as long as we review it, we're good" is delusional. We might
pretend we are doing it, but deep inside we know we are not. AI coding has
brought this into stark reality.</p>
Too often, reviewers receive schematics or source code with little context. What
are the goals? What is the spec? What trade-offs were considered? What
calculations were done? What edge conditions were considered? What is the
architecture? Details without the big-picture context are very difficult to work
with. While I've not often been party to formal code reviews in a meeting
context, I suspect these could work better than offline asynchronous reviews
because you can ask the designer questions and perhaps get some insight into the
potentially weak areas of the system.</p>
There are ways we can help reviewers:</p>

Clear documentation about the requirements located in the same repository as
the design/code.</li>
All assumptions, decisions, calculations, etc. are documented. With
schematics, as much of this as possible should be included directly in the
schematic. Use more pages if needed.</li>
Architecture is clearly defined and documented.</li>
All testing is documented.</li>
All assumptions are documented.</li>
Automated tests are created where possible.</li>
Performance metrics and resource usage are documented and automatically tested
if possible.</li>
Any concerns, potential weak spots, or things we're not sure of are clearly
documented. Often the designer is the only one who has insight into this.</li>
</ul>
The point many miss here is that if we do the above, we're mostly helping
ourselves 6 months from now.</p>
As systems become more complex, it seems the only practical path forward is
automated testing. Chip designers figured this out a long time ago as the cost
of errors is high with full-custom silicon. Software developers are pretty good
at this with TDD (test-driven development). But, the industry needs to go one
step further and embrace end-to-end testing (which includes both hardware and
software). Perhaps the most important role of a reviewer going forward is to
ensure adequate documentation exists and tests are in place. Anything else is
futile and we're just pretending.</p>
</p>


🚀 How far do you take releases?
2026-01-12T00:00:00+00:00
Thus far, we have defined a release as something that is
useful and high-quality</a>.
Typically, we think of a release as something formal - v1.3.8 that is packaged,
built, and announced. But we can push this further. What about pull requests? Is
every pull request that is merged at the same standard? Or do we merge them and
hope that something later in the process will magically transform the work into
something better? "Later never comes" - especially in complex systems.</p>
</p>
Open-source projects have figured this out. The standard for getting a PR merged
is incredibly high in most projects. Why? Because there is no business pressure
to merge it, but rather the fear of having to maintain something that is
substandard. The maintainers are looking out for their own selfish interests,
but these interests happen to align with the realities of this kind of work.
Fixing something after the fact is so much harder than doing it right in the
first place. As a result, most OSS projects maintain fairly high quality at any
point in their development. This is one reason Arch Linux
works so well</a>.</p>
While there is a place for brainstorming, prototyping, experimenting, etc., this
will not get you very far developing a real product. The earlier you push the
"release mentality" into the development process, the faster you'll move. Maybe
not in the initial steps, but for sure once you are a third of the way through
the development cycle. Do you want to save time on the first third, or the last
two thirds of the development cycle? Hint, the Law of Compounding is at work
here ...</p>
</p>


🎯 Standards for internal vs external releases
2026-01-09T00:00:00+00:00
It is natural to think of internal releases as "low stakes" - if it crashes and
burns, no big deal. It's just an experiment, right? Near production release,
we'll toss it over the wall to QA/Manufacturing and find all the issues.</p>
This is partly true, and partly false. While it is true a product should get
better over the development cycle, it is helpful to apply the same standards for
internal releases as external for the following reasons:</p>

It forces an emphasis on quality early.</li>
It results in automated testing.</li>
There will be fewer surprises late in the development cycle.</li>
Integration with other groups happens earlier (Quality, Sales, Manufacturing,
Customers).</li>
It is much easier to build prototypes when quality is high.</li>
</ul>
This does not mean the entire product needs to be perfect on day one - that is
impossible, as the product is not done - just the work already completed,
however small that may be. Is this tiny bit of work done to high standards? Is
it testable, manufacturable, etc.? Is it
useful</a> to someone?</p>
Here's the thing - it is much easier to apply high standards to small bits of
work as you go, than retroactively fix it late in the game. It is not a waste of
time, rather it is only way that works in developing modern systems.</p>
If internal releases are not at the same standard as external releases, they are
not releases, they are something else.</p>
</p>


📦 What do we release?
2026-01-08T00:00:00+00:00
Everything!</strong></p>

Documentation</li>
Hardware</li>
Software</li>
Prototypes</li>
Products</li>
Lunch</li>
</ul>
Engineering is fundamentally the act of creating something new. And that
something new needs to be useful to someone or something. And the way we deliver
"useful" is the release.</a>
If we're not releasing, we're not delivering.</p>
We might be polishing, researching, thinking, perfecting, stalling, loafing,
dreaming, but we're not delivering.</p>
The "release" mindset brings the focus to delivering value.</p>
</p>


🤝 A release is what unifies Engineering and Business
2026-01-07T00:00:00+00:00
Some feedback on yesterday's post
(What is a release</a>):</p>

What specifically flows into the two points? Value? Effort?</p>
</blockquote>
Perhaps a better way to phrase this is these are the two points of focus that
unify the development team with the rest of the business organization. And the
release is the container.</p>
Documentation, specifications, innovation, new technology, etc. flow into the
useful</strong> point.</p>
Testing and automation flow into the confidence</strong> point.</p>
The business value that flows out of the useful</strong> point is user value, profit,
increasing sales, etc.</p>
From the confidence</strong> point flows quality and responsiveness.</p>
But these two points give us a simple way to focus and evaluate a release - is
it any good? Is everything we are doing as engineers contributing to these
points?</p>
</p>


📦 What is a release?
2026-01-06T00:00:00+00:00
There are two aspects of a product release:</p>

Something useful</strong> that others can experience.</li>
Produced with a high level of confidence</strong>.</li>
</ol>
Everything flows into these two points.</p>
</p>


🚀 Shipping faster leads to predictability
2026-01-05T00:00:00+00:00
Shipping faster does not only mean getting to v1.0 faster (or even on time), but
also many internal releases at a good rate. This is the only way</strong> to bring
predictability into the development process.</p>
</p>


🔗 Platform thinking does not mean we don't use other platforms
2026-01-02T00:00:00+00:00
On the contrary, platform thinking readily embraces other tools and platforms.
The difference is:</p>

We add our own tooling, integration, and workflows into the mix.</li>
We own the integration points so we can readily swap out/in 3rd party
components as needed.</li>
We are not overly dependent on someone else's platform that we don't have
control over.</li>
We value openness and flexibility.</li>
</ol>
Too often, the tendency is to lean toward all-encompassing platforms that we
think will solve all of our problems. This rarely gets us to where we need to be
to provide exceptional value.</p>
</p>


🔍 AI coding requires better inputs and robust verification
2025-12-31T00:00:00+00:00
Some great feedback on
the previous post (Impressive is not always effective)</a>:</p>

"I agree. It would be interesting, however, to compare the progression of AI
code generation with that of compilers - were there star assembly-code
engineers saying that compilers were just a way to brute force the generation
of assembly code?"</p>
"I feel like there is a fundamental difference — perhaps that AI always lacks
the determinism of a good compiler. And it can completely ignore the best of
documentation without throwing an error code."</p>
</blockquote>
Having lived through several of these transitions (Assembly -> C -> C++ ->
Python -> Go -> Rust -> Zig -> AI ...) there are always those who prefer to
stick with the current technology thinking the new adds no value or is even
detrimental.</p>
AI is more just an example in the original article as it can make an impressive
mess much quicker than any human. Humans can make messes as well (made plenty
myself). So the point is not so much that AI is bad (or good), but rather
question: "Are values in place?" And, is there a process/system/platform
(whatever you want to call it) to ensure these values happen. AI can do good
work with the proper constraints and workflows, as can humans. But neither does
very good work without them.</p>
That all said, as the above feedback suggested, AI is much less deterministic
than any compiler (two consecutive runs do not produce the same result), thus
needs clear guidance and robust verification to ensure the end result is
quality. All this can be accomplished, but needs to be thought through.</p>
While compile warnings, linters, code review, and unit tests help ensure
compiled code is quality, AI is a different beast. If humans are too slow to
write the code originally, they are for sure too slow to review it. It is like
reviewing machine code from compilers - at some point you need to trust the
compiler. The errors from AI generated code will likely be more architectural
deficiencies or logical errors at a macro level. Bad architecture can bury you
in technical debt, but it takes time - initially it works fine. Thus several
things are critical:</p>

Getting started with a good architecture. AI can help with this, but an
experienced developer needs to have input.</li>
Good, concise documentation. As opposed to machine generated code, humans can
review this.</li>
Automated end-to-end testing is critical. This is the only way to prove and
maintain correctness.</li>
</ul>
With AI development, better inputs and more verification are critical. Trying to
work in the middle of the AI coding process is futile and exhausting. The
beginning and the end of the process need shored up. But this requires
better workflows</a>
and more discipline. While undisciplined traditional development is a nightmare,
undisciplined AI development is a train wreck.</p>
</p>


✨ Impressive is not always effective
2025-12-30T00:00:00+00:00
When AI generates something, the result is often "impressive." We might marvel
at how quickly lots of code is generated that mostly works. Generated
documentation is extremely detailed and well formatted. AI-created diagrams are
colorful, use emojis effectively and every last detail is included. However,
"impressive" is often the antithesis of "effective" for one simple reason -
simplicity is missing.</p>

It is difficult to understand the architecture of the code because there is so
much of it.</li>
"Why" is nearly impossible to draw from the expansive details in the
documentation. Who has time to dig it out?</li>
A quick glance at a diagram reveals nothing about the big-picture without
laboriously processing many details.</li>
The complexity of machine-generated designs is often a brute-force approach
that lacks style, elegance, and consistency.</li>
</ul>
Architecture, Why, Big-picture, and Elegance - these are still the foundations
of great work. Learn to look past impressive things. Is the architecture clear?
Are the reasons obvious? Is it easy to understand at a high-level what is going
on? Is the design and implementation clean and simple? There are two paths:</p>

"Impressive" is the route to compounding technical debt.</li>
"Effective" is the only way to compounding improvements.</li>
</ol>
</p>


🥄 The beauty of human creativity
2025-12-29T00:00:00+00:00
In this age of machine-generated perfection, something is still deeply rewarding
about grabbing a stick of wood and making something by hand. This past
Christmas, with the help of kids and grandkids, we made a few mixing spoons for
gifts. Starting from the same pattern but with varying levels of skill and
technique, each spoon turned out a bit different. However, each one is special
and beautiful because it has a human touch. And sometimes with variety and
imperfection, a new pattern emerges that actually works better. This is one
source of innovation.</p>
In an age of AI-generated code, preserving the human touch remains important.
Perhaps it is the well-thought-out documentation that explains exactly how this
feature will help the user. Or the hand-coded API that is a pleasure to use.
Explorations in new data structures and algorithms coded by hand sometimes
result in amazing discoveries because thinking accompanies the creative process.
Machines (including AI) can mostly only do what has been done before. But the
real value lies in the new connections, the
newly discovered truths</a>,
the well-understood needs of the user, and the exceptional quality. These still
require human impetus.</p>
</p>


📖 The definition of the word 'Platform'
2025-12-24T00:00:00+00:00
The definition of the word "platform" includes things like the following:</p>

Acts as foundational for other products or services, not the final end product
itself.</li>
Provides a stable base.</li>
Typically supports multiple, distinct applications or use cases.</li>
A formal declaration of the principles of a group.</li>
</ul>
What are the aspects of a physical platform?</p>

Provide a supporting surface: Examples include train station platforms,
stages, loading docks, scaffolds, and cargo pallets, all of which physically
support people or objects at a useful height or position.</li>
Enable safe and efficient interaction: Platforms make boarding, loading,
performing, or working safer and more efficient by aligning heights, improving
visibility, and organizing space.</li>
</ul>
Stable, foundational, supportive, efficient, visibility, safety, organization
... does this provide any inspiration for YOUR Platform?</p>
</p>


🔄 Platform Thinking is a habit
2025-12-23T00:00:00+00:00
Platform Thinking in the context of product development has many parallels to
effective habits in personal life. With personal exercise, healthy eating, getting
enough sleep, and reading good books - these are difficult to start, but easy once
started, and rewarding once finished. These things work best when they are
habits - a regular behavior pattern that a person tends to do automatically,
often without much thought.</p>
Documentation, testing, and automation in a product development context are
similar. It won't just happen. A framework is required to nudge us into a
pattern that turns into an effective habit. In many ways, Platform Thinking is
just Atomic Habits</a> for product
development teams.</p>
</p>


🎯 Who is Platform Thinking for?
2025-12-22T00:00:00+00:00
Platform Thinking is for anyone who wants to take the friction out of creating,
publishing, testing, integrating, and deploying products.</p>
The result? Value. Quality. Timeliness.</p>
</p>


⏳ The long-term compression timeline
2025-12-18T00:00:00+00:00
Below are several timelines illustrating
long-term compression</a>
comparing a product development timeline:</p>

20 years ago</li>
Today</li>
Today with Platform Thinking</li>
</ol>
The big effect of long-term compression is that negative consequences can hit
much earlier, delaying initial shipment (the middle timeline).</p>
While it is very difficult to compress a product development timeline to initial
release, with Platform Thinking, v1.0 can still arrive in a year as in the olden
days. The big difference is subsequent releases ship much quicker than in any
previous scenario.</p>
So while the complexity of modern technology presents new challenges and
requires a different approach, leveraging technology to build a platform can
produce even better results than any previous scenario.</p>
Long-term compression can have both compounding negative or compounding positive
effects. Both the negative costs and the positive benefits come back much
quicker. Which comes back depends on YOUR Platform.</p>
</p>


🗜️ Long-term compression
2025-12-17T00:00:00+00:00
</p>
One reality of developing products with modern technology is the compression of
the short and long-term. Today, the consequences of actions come back on the
timeframe of months instead of years, as it might have been ten or twenty years
ago. No longer does it work to take shortcuts with the hope of shipping v1.0,
making some money, and cleaning things up later. The product may never even see
v1.0.</p>
This is why YOUR Platform is so important. Platform thinking enables you to
reach the v1.0 release in a timely manner, and efficiently maintain and improve
beyond that.</p>


🚀 Platforms are not just for developers
2025-12-16T00:00:00+00:00
</p>
This is an
interesting post</a>
from a Marketing person creating website pages:</p>

I built 19 website pages in a single day while my team was out for US
thanksgiving. Not mockups. Not placeholders. Actual, shippable pages: a full
partner ecosystem landing page plus individual pages for every major partner
we’re building with right now. It felt surreal. Mostly because a year+ ago,
there’s no universe in which I would’ve believed I could do that</strong>. And it
definitely wouldn’t have happened without the engineers around me who’ve
taught me how to think more like a builder—patiently walking me through GitHub
(😒), tools, workflows, and a way of working that used to feel miles outside
my world.</p>
</blockquote>
I then asked how long it took before she stopped hating Git?</p>

I mean it took about 5 pull requests before I finally managed to create one
without an error 🥵… it’s been two weeks and I finally feel like the
lightbulbs are going off and I’m understanding the flow (pulling things from
main, branches, commits, PRs, basic commands I have on a sticky note lol,
etc).</p>
</blockquote>
This is a great example of someone who took ownership of their website creation,
editing, and publishing workflow. Using
local files</a> and
text based formats</a>
unleashed the power of AI resulting in the productivity described above. Using
AI,
anyone can now write scripts</a>,
create web pages, and code to some level. Text based formats and command-line
tools are now within reach for anyone willing to learn a bit. All it takes is a
willingness to break out of the cozy cocoon of GUI based tools, learn how to use
an editor and terminal, and invest a little effort in creating a workflow.</p>


🔄 Breaking the AI slop cycle
2025-12-15T00:00:00+00:00
</p>
"AI Slop" is a common expression describing a familiar phenomenon. It refers to
sloppy code (or other assets) generated by AI that mostly works, lacks proper
review, and may be poorly architected or implemented. The primary characteristic
of AI Slop is that it is hard to read, understand, and maintain.</p>
Like anything else, fixing problems early in the cycle works best—not generating
slop in the first place instead of trying to clean it up later. Even AI tools,
as powerful as they are, have trouble cleaning up slop. Once the slop is cast in
code, AI (like a human) is very reluctant to throw away all the hard work and
start over.</p>
So how does one break the cycle? The best method I've found is a lightweight
formal documentation and planning workflow</a>.
The documentation ensures good thinking about what the problem is, what needs to
be built, and why. The planning stage helps ensure the architecture is good. It
is much easier to think about and rework a plan than code.</p>
The documentation/planning steps also clearly delineate the responsibility of
humans and AI. While AI can help explore options, it still does not have the
entire context that humans have when it comes to clearly defining the problem.</p>
When slop appears, this means the up-front thinking was not very good. One of
the most useful skills when working with AI is recognizing when the slop zone is
approaching, backing up, and getting more formal.</p>


🤔 Isn't this just agile?
2025-12-12T00:00:00+00:00
</p>
A reader responded to the last post with a comment:</p>

Isn't "agile" a part of product thinking too? That doesn't seem linear?</p>
</blockquote>
This is a good question. Agile covers a lot of ground. The Agile Manifesto
states:</p>

We are uncovering better ways of developing software by doing it and helping
others do it. Through this work we have come to value:</p>

Individuals and interactions over processes and tools</li>
Working software over comprehensive documentation</li>
Customer collaboration over contract negotiation</li>
Responding to change over following a plan</li>
</ul>
That is, while there is value in the items on the right, we value the items on
the left more.</p>
</blockquote>
Typically agile focuses on fast iterations and quick feedback, which is
definitely part of platform thinking. Most software methodologies focus on
creating software. Platform thinking expands to include more of product
development including mechanical, hardware, and integration. Much of development
these days is integration of complex systems, including vast amounts of
open-source software. As system complexity grows, teams need ways to deal with
this complexity. Individual interactions are great and very important, but they
don't scale with complexity. And perhaps the real goal of YOUR Platform is to
free people from tedious repetitious work, so individuals will have more time to
interact in creative ways.</p>


🏗️ Platform thinking vs 📦 Product thinking
2025-12-10T00:00:00+00:00
</p>
Platform vs. product thinking is a difference in perspective. Platform thinking
does not ignore the product, but rather approaches it from a slightly different
angle.</p>


🏆 What is the ultimate platform success?
2025-12-09T00:00:00+00:00
</p>
The ultimate platform success is when an internal platform turns into a sellable
product/service, or an open source project.</p>
Perhaps some companies succeed by starting out building a platform as a product,
but the
greatest platform successes</a>
started as internal platforms:</p>

AWS</li>
Kafka</li>
Kubernetes</li>
Slack</li>
Shopify</li>
</ul>
Why does this work? Perhaps because developers building internal platforms have
the "platform mindset." Starting out building a platform to sell means building
with the "product" mindset instead of the "platform" mindset.</p>
Compare AWS, Azure, and GCP. Amazon and Microsoft use these platforms
extensively internally. Google
does not</a>.
Google likely has valid reasons for this as GCP is not their primary product
(compared to search, YouTube, etc.). But the results speak for themselves:</p>
Feature</th> AWS (Amazon)</strong></th> Microsoft Azure</strong></th> Google Cloud (GCP)</strong></th></tr></thead>

Market Share</strong></td> ~29-31%</strong></td> ~20%</strong></td> ~13%</strong></td></tr>
Revenue Run Rate</strong></td> ~$132 Billion</strong></td> ~$123 Billion</strong></td> ~$61 Billion</strong></td></tr>
Primary Internal Use</strong></td> High.</strong> Amazon.com is heavily migrated to AWS, including retail databases and Prime Video.</td> High.</strong> LinkedIn, Office 365, and GitHub are actively migrating or fully migrated to Azure.</td> Mixed.</strong> Core search/ads/YouTube/Gmail/Docs runs on internal-only PROD infrastructure.</td></tr>
</tbody></table>


🔀 The division between group and personal workflow
2025-12-05T00:00:00+00:00
</p>
When thinking about developer workflows, tools, and resources, dividing them
into two categories proves useful: personal</strong> and group</strong>. A group resource
is something that multiple people use (or should use): SDKs, libraries, linters,
formatters, AI commands/context, etc. Personal tools are things that others do
not use: pencil, computer, phone, editor, notes system, etc.</p>
Why is this so important? Why not just make it simple and force everyone to use
the same computer, editor, notes system, chair, desk, etc.?</p>
Humans are users and creators of tools—naturally inclined to make, modify, and
optimize tools for specific needs. Innovation happens where there is freedom.
Many improvements start from a single person scratching their own itch. Over
time it becomes apparent that a tool or improvement will benefit the group, so
it migrates to a container or some other shared resource and is integrated into
the group workflow.</p>
As previously mentioned, any shared resource should
be containerized</a>
when possible. Containers allow the personal computer to be the dividing line
between personal and group workflows.</p>
No two humans do exactly the same thing. No two humans think and work in exactly
the same way. While there are many benefits to having group workflows, the
personal workflows must also be nourished, as these are a primary source of
innovation. The key is to have a culture where the migration from personal
workflow to group workflow is encouraged, supported, and rewarded.</p>
To scale, an organization must have:</p>

A clear division between personal and group workflows.</li>
An efficient way to reuse personal workflows as group workflows where
appropriate.</li>
</ul>


📦 CDPD - Container Driven Product Development
2025-12-04T00:00:00+00:00
</p>
When updating the Simple IoT documentation</a>, the
question arose - how to install mdbook</code> and a few plugins? Of course pacman</code>
has part of what is needed. In the past cargo</code> has been used, but that is slow
and not a good fit for shared tools. Then it became clear:</p>
Any shared tools used by the team (build, linting, formatting, CI/CD) etc.
should be put in containers up front (before development starts)</strong></p>
.. and use them for development builds as well as CI/CD.
(this container</a> is the result)</p>
Why?</p>

It is pretty easy these days with Claude Code.</li>
Gitea, GitHub, and likely other Git hosting platforms now provide convenient
"package" registries</a>. You no
longer need to go through the hassle of using the Docker registry.</li>
It is easy to onboard new devs - no need to spend time installing SDKs and a
host of tools.</li>
Versions are locked down, eliminating the possibility of different people
using different versions.</li>
They get exercised a lot, so quality is high.</li>
It is much easier to move into CI/CD.</li>
A Dockerfile is easier to maintain than a long document describing how to
manually install all the tools needed.</li>
NPM and Python packages are a pain to install locally, so why not just
eliminate that pain?</li>
It is easier to get everyone on the team using the entire toolset (linting,
formatting, etc.).</li>
No longer fighting differences between developers' machines. If one developer
wants to run Arch and another Ubuntu, no problem.</li>
</ul>
Developers are still free to use their favorite editor, Git client, and other
local tools.</p>
This is another case</a> where
a little time spent up front is quickly regained, then compounded by not
continually redoing stuff, and fighting problems that can be solved once.</p>


📝 Tips for working with Markdown
2025-11-26T00:00:00+00:00
</p>
Markdown is now a nearly universal format for documentation in many workflows.
A few tips for being effective with Markdown:</p>
Use a standard editor most of the time (VS Code, Vim, Helix, etc.). This is
faster than graphical "Markdown" editors for most things. Additionally, this
builds familiarity with Markdown formatting codes, which aids in creating nicely
formatted comments in Jira, GitHub, Trello, etc.</p>
Use Typora</a> to create more complex things like tables and
images. This tool is well worth the few $ it costs.</p>
Put images in an /assets</code> sub-directory and name image files descriptively
(Typora makes this easy).</p>
Use prettier</a> to format the file and wrap long lines. Add
a .prettierrc</code> to every repo with the following contents: proseWrap: always</code></p>
Use Harper</a>
(language server</a>
in your editor is best) to check for grammar and formatting issues. One
unexpected side effect of this is that it encourages marking technical words
that don't have standard spelling as code, rather than adding them all to your
dictionary.</p>
Add dictionary words to a .harper-dictionary.txt</code> in the repo that contains the
markdown file. This way others can benefit from a common dictionary. Again, this
is easy with a harper-ls</code> quick-action.</p>
Add a TOC if the file is long (the
harper-ls</code></a>
quick-action makes this easy).</p>
Use the Marksman</a> language server.
This makes navigating and refactoring in your editor easy.</p>
Use Markdown Here</a> or
Markdown Here Revival</a>
in your browser or email client to write better emails.</p>
While Markdown has its formatting limitations, with a little tooling it is much
easier and faster to work with than anything else.</p>


🤔 Why > How > What
2025-11-25T00:00:00+00:00

Why?</strong> Is this the right thing to build?</li>
How?</strong> Can it be done efficiently, repeatedly?</li>
What?</strong> The deliverable.</li>
</ul>
These are listed in order of importance, which also corresponds to the level of
difficulty.</p>
The trouble with "How" and "Why" is these are less visible, which contributes to
their difficulty as it takes discipline to prioritize and do them.</p>
</p>


🥉 Good, fast, cheap - choose three
2025-11-24T00:00:00+00:00
A familiar phrase states:</p>

"Good, fast, cheap. Choose two."</p>
</blockquote>
This is similar to the
development triangle</a>.</p>

Which states that everything is a tradeoff.</p>
However, improving all three becomes possible when doing things differently.
That is the fundamental purpose of YOUR platform - to improve "how" teams build
things. To develop a synergistic relationship between
value, timeliness, and quality,</a>
without compromise.</p>



⏳ The "long-term" may be closer than you think...
2025-11-21T00:00:00+00:00
A common objection to "Platform Thinking" (proactively build your platform
from the start</a>) is: there
isn't time right now; this can be done after the
1.0 release</a>
when there is ample time and resources.</p>
The "long-term" can be defined as when the consequences of actions come back -
when the seeds sown are reaped.</p>
When building complex systems with modern technology, the gap between the short-
and long-term is months instead of years</strong>. There is a compression of the
short- and long-term. Neglecting the platform may have worked a decade or two
ago, but with the complexity of even basic embedded systems today (Zephyr,
Embedded Linux, connectivity, data, etc.), the "long-term" is much more
imminent. The pain from shortcuts emerges in months instead of years. The slope
of the technical debt curve is much steeper.</p>
The reality with complex systems today is there is no short- and long-term -
they have morphed into one and the same. Without building for the long-term now,
there will not be even a short-term, let alone a long-term.</p>



📝 DDPD - 🧪 TDPD - 🚀 RDPD
2025-11-20T00:00:00+00:00
We have been discussing document, test, and release-driven development (doing
these things first before implementing features).</p>
This ensures the product:</p>

Meets the customer's needs
(DDPD</a>).</li>
Is quality (TDPD</a>).</li>
Can be modified any time and released quickly
(RDPD</a>).</li>
</ul>
Without these, teams can quickly become buried in mundane repetitive work, bugs,
and technical debt.</p>
But perhaps the less obvious fact is these things often go undone if not
prioritized from the start. Why? Because once a feature is done, there is
pressure to move on to the next one. There is no time for "cleaning things up."
To ensure these things happen, they must be done first.</p>

(2025-12-05 update: And we now also have
CDPD</a>.)</p>


📝 DDPD - Document Driven Product Development
2025-11-17T00:00:00+00:00
</p>
Building on TDPD</a> and
RDPD</a>, Document-driven product
development (DDPD) offers another perspective.</p>
Ideas:</strong></p>

Document the feature from the user's perspective before implementing anything.</li>
Keep the documentation close to the code (same repo, etc.).</li>
Use a
doc-driven</a>
AI plugin when applicable.</li>
</ul>
Benefits:</strong></p>

Better thought-through features.</li>
Non-technical people can review before implementation.</li>
Higher likelihood of implementing what users actually need.</li>
Documentation becomes a pleasant/creative/thinking task, rather than a boring
requirement after implementation.</li>
</ul>


🚀 RDPD - Release Driven Product Development
2025-11-13T00:00:00+00:00
</p>
Yesterday we talked about
Test-Driven Product Development</a>.
Release-driven product development is another perspective.</p>

Build the release/deployment mechanism before implementing the design.</li>
Automate as much of the release as possible.</li>
This includes all assets needed to build/deploy the product.</li>
</ul>
While this idea is somewhat common in cloud software (continuous delivery),
embedded, hardware, and mechanical release mechanisms are usually lacking.</p>
The benefits:</p>

More prototypes will be built (because it is easy).</li>
More testing will occur early (because prototypes are available).</li>
Lots will be learned (happens when people are using prototypes).</li>
Releasing to manufacturing becomes easy (because it has already been done many
times).</li>
</ul>
Implementing release mechanisms first is a strategic approach, because without
doing it first, it likely won't get done. Once something is "finished," there is
pressure (from above) to move on to the next development.</p>


🧪 TDPD - Test Driven Product Development
2025-11-12T00:00:00+00:00
</p>
We're familiar with TDD (test-driven development) in the software world.
Basically the idea is to create unit tests before implementing a feature.</p>
What if we extend this to systems, hardware, and even mechanical?</p>

Write end-to-end system tests before developing a feature.</li>
Design/build test fixtures before designing the PCB. PCBs are
cheap and fast</a>
to build now - so build an additional one for testing.</li>
Create automated checks on 3D CAD models for fit/clearance, etc. Does the new
PCB fit in the product?</li>
Do you have additional ideas?</li>
</ul>
And then automate all these so they run continually. This takes the tedium out
of testing and helps ensure every development change is a
BIG Release</a>.
Testing first also leads to better design.</p>


🍄 Mushroom Management
2025-11-07T00:00:00+00:00
I recall a quote from the book The Soul of a New Machine</em> that describes
"Mushroom Management" and goes something like this:</p>

"Keep them in the dark, feed them sh**, and watch them grow."</p>
</blockquote>
(This book chronicles the experiences of a computer engineering team racing to
build the next-generation computer at a blistering pace under tremendous
pressure at Data General in the late 1970s.)</p>
Contrast this with Jensen Huang</a>:</p>

"They never hear me say something to them that is only for them to know. There
is not one piece of information that I secretly tell the staff that I don't
tell the rest of the company. In that way, our company was designed for
agility, for information to flow as quickly as possible, for people to be
empowered by what they are able to do, not what they know</strong>. That's the
architecture of our company."</p>
</blockquote>
Where is Data General today? Where is Nvidia? Perhaps this is comparing apples
and oranges though, because as system complexity grows, Huang's approach is even
more necessary. Systems in the 2020s are many times more complex than those of
the 1970s.</p>


🚀 The BIG Release vs. 🌊 a Continuous flow of Value
2025-11-05T00:00:00+00:00
How do we look at a product release? As a big one-time event, where we spend
heroic efforts to pull it over the finish line? Or as a continuous flow of
high-quality value into the product (even before the customer ever sees it) that
is maintained with every change?</p>
This is a mindset difference that leads to two different approaches:</p>
The BIG v1.0 Release:</strong></p>

Quality is not a priority during development, and gets hammered in with
extreme effort right before the BIG Release.</li>
Tools that make releasing a product easy are not built, rather everything is
done manually before the BIG Release.</li>
Because there is so much work to do for the BIG Release, much of it is not
anticipated so schedules slip.</li>
Issues surface late in the process, which causes further delays.</li>
Stakes are high, pressure mounts, and people become stressed.</li>
</ul>
Continuous flow of Value:</strong></p>

Every pull request (PR) and internal release is treated like an important
customer release.</li>
Tools automatically check quality, eliminating manual verification.</li>
Releases are automated to enable frequent deployments without mistakes.</li>
As a result, the product is always ready to release - internally or to
customers.</li>
Releases happen often (full manufacturing package) because it is easy.</li>
Test fixtures are already done because they are created to test each feature
when the feature was developed.</li>
Prototypes (which are just scaled back manufacturing) are easy to build and
test because a manufacturing release process already exists.</li>
More prototypes get built and more gets learned from actually using the
product.</li>
Issues and manufacturing problems surface early in the design cycle.</li>
Beta releases can be sent to early customers for feedback without hesitation
because the finished features are high quality.</li>
Production release is not a big engineering event, but rather a point where
people start paying for the product.</li>
Schedules are predictable because there is no extra work for production
release, but rather just a focus on the features needed for that incremental
release.</li>
Stress is low, people are happy.</li>
</ul>
The second scenario is desirable, but what is perhaps not so obvious is the
mindset shift required to get there - a relentless focus on quality throughout
the development process, automating anything done repeatedly, and thorough
testing early in the process. Every internal release, every PR, every day is a
BIG Release.</p>


🔀 The Power of Git* workflows
2025-11-04T00:00:00+00:00
Git +
Git hosting platforms</a>
enable both</strong> local and cloud-based workflows.</p>
Local:</strong></p>

Lightning fast access to files - leverage the power of your local computer</li>
Use powerful native tools (editor, e-cad, m-cad, AI agents, command-line
tools, scripts, etc.)</li>
Test on local hardware</li>
Work offline</li>
Organize/document changes</li>
</ul>
Cloud:</strong></p>

Collaboration</li>
Distributed development</li>
CI/CD</li>
Secure storage</li>
Immutability</li>
</ul>
These approaches are not mutually exclusive - both can be leveraged
simultaneously. Software developers take this for granted, but other disciplines
can also benefit from better file management tools. It is beneficial to consider
how both cloud and local domains can be utilized to their fullest potential. In
the end, design assets (data) need to be created, documented, managed, verified,
tested, deployed, secured, and easily synchronized</strong> between both domains.</p>


✍️ Why writing is important
2025-11-03T00:00:00+00:00
Some people are wildly successful without writing. So, if the goal is only to
make a lot of money or have a lot of power, this advice may not apply.</p>
However, for those who want to:</p>

Think better</li>
Scale ideas beyond direct personal contact</li>
Capture and preserve ideas</li>
Grow by building new ideas on old ones</li>
Have influence beyond force and charisma</li>
Effectively
manage up</a></li>
Not have to redo things over and over</li>
</ul>
Then capturing your ideas in a permanent, organized, discoverable place is
essential.</p>


🔄 Three levels of workflow
2025-10-31T00:00:00+00:00
There are three levels of workflow:</p>

Personal (what you do to get things done)</li>
Department/Discipline (How multiple people who do the same thing get things
done – Example: PCB design)</li>
Cross-functional (How people who do different jobs interact - Example:
Engineering, Support, Manufacturing, Sales, and Marketing)</li>
</ol>
Effective workflow is increasingly difficult to implement as you go from 1 to 3,
as perspectives, needs, and comfort with various tools differ more. But
cross-functional workflows are also where the greatest value</strong> lies.</p>


🤝 The benefits of sharing
2025-10-30T00:00:00+00:00
I wrote my first Claude plugin</a> that
packages up some commands that I use in about every project I work on. This
plugin allows me to have these commands globally so I don't have to keep copying
them from project to project.</p>
Although the commands originally worked pretty well, the process of releasing
them publicly made them a lot better. A new implement-tests</code> command was added.
And the commands themselves were cleaned up to be clearer, which will likely
help Claude do a better job.</p>
I am continually amazed by this process. Even if no one else uses or contributes
to this project, I have a better tool for myself, because I shared it.</p>


🛠️ The new Artisan Age
2025-10-29T00:00:00+00:00
We live in a world of powerful tools. Milwaukee battery-powered hand tools,
high-end PCB design suites, cloud platforms, open-source tools, etc. And while
these tools get us a good way toward the goal, they do not do everything.
Additionally, if all you ever use is the mainstream tools, what differentiates
you from anyone else? What prevents others from doing exactly what you are
doing - at a lower cost?</p>
In a previous era, artisan woodworkers built many of their own tools (fixtures,
jigs, squares, planes, storage systems, etc.). As a result, they could highly
optimize for their needs and projects. Tools were not very complex, so it was
possible to make them. Today, things are much more complex. The things we can
buy, or in the case of Open Source use for free, are endless. Many of them are
very useful, but not always exactly what is needed. There is still friction,
waste, manual steps, things that don't work together, integration pain points,
etc.</p>
In the past, it was very time-consuming to solve integration and automation
problems in complex systems. However, today we are entering a new era where we
can again build tools for complex systems - with AI. AI is excellent at
scripting, automation, testing, configuration, using APIs, generating fixtures,
processing data, etc. Things that would have taken days to implement and figure
out, we can now do in a few minutes. In many ways, we are entering a new artisan
age where we can once again craft custom tools tailored to our specific needs
and workflows. The biggest win with AI is not building your products, but rather
using it to build a custom platform to build your products.</p>


🏗️ The difference between Product and Platform
2025-10-27T00:00:00+00:00
Product is what</strong> you build.</p>
Platform is how</strong> you build.</p>
Most people build the first and outsource the second. After all, the product has
the IP—the secret sauce—the valuable part.</p>
However, we
observe in successful companies</a>
a focus on their internal platform. They build their own tools and workflows
optimized for them. How you build is where the rubber hits the road. If you
don't master the "how," getting the "what" done is frustratingly difficult. YOUR
Platform is the engine that keeps product development running smoothly and
allows you to keep ahead of the game. It does not matter so much if someone
steals your ideas because you are already implementing the next ones.</p>
It is relatively easy to copy a physical product. It is nearly impossible to
copy the internal process, culture, workflows, and tools—even if they are freely
shared. Why? Because so few companies have the discipline and will to
own their platform</a>.</p>


🎯 A single source of truth
2025-10-23T00:00:00+00:00
An intense discussion about part/version numbers</a>
driving application logic surfaced an idea to create a simplified version of the
product metadata that simplified the programming logic. However, this rarely
works out long-term, because several things happen:</p>

The two sets of data must be related and kept in sync.</li>
This approach ends up replicating the first set of data with the second.</li>
</ol>
Fix the problems with the first set of data, rather than trying to band-aid it
with additional data. Data can be derived by machine, but it should never be
created in parallel. There needs to be a single source of truth. Otherwise
mistakes will happen. More mistakes than just dealing with the first set of
data, no matter how painful.</p>
Manually duplicating data is symptomatic of organizational, not technical
problems. Duplicated data often indicates that organizational silos exist caused
by a lack of:</p>

transparency</li>
collaboration</li>
cross functional teams</li>
common standards</li>
common workflows</li>
common tools</li>
</ul>
Don't make excuses for these, fix them.</p>


📈 Scale vs. Value
2025-10-17T00:00:00+00:00
As scale increases (# of people, # of technologies, # of products, # of teams,
life cycle of product, experience, amount of knowledge, etc.), what does the
value produced look like?</p>

Metcalfe's law</a> states that
the financial value or influence of a telecommunications network is
proportional to the square of the number of connected users of the system.</p>
</blockquote>
In communication and economic systems, we often find Metcalfe's law to be true.
However, personally and organizationally, the results often plateau as
projects/systems/organizations grow.</p>
Which of the below graphs describes you personally or organizationally? How do
we get from average -> great?</p>
</p>
</p>
</p>


💻 Local First Dev
2025-10-14T00:00:00+00:00
In the last couple decades, we've observed the success of SaaS web applications
(Google Docs, Salesforce, Basecamp, Trello, etc.). These applications have
unleashed collaboration using the web platform. Before this, users might have
emailed documents around (many still do this) or shared access to a file-server
or used a terminal server to access common infrastructure. However, nothing can
match the web for low friction/cost access.</p>
One of the downsides of SaaS is that users need to re-create the context of
whatever they are doing in the web application (upload files, copy images,
etc.). For many applications, the benefits to collaboration greatly outweighed
the cost of re-creating context. For things like issue tracking, SaaS
applications work well.</p>
However, for more complex workflows (like CI/CD/PDM/PLM), the drawbacks of SaaS
become more apparent because they require much more context. Some products,
like Onshape</a>,
solve this problem by moving everything to the cloud. This works great for pure
mechanical products, but what about electrical and software? Should these all
operate in their own silos?</p>
A more compelling vision is a mixture of SaaS and local tooling. Examples
include:</p>
Claude Code</a></strong>: while
using AI in web sites is very useful, running AI locally in your terminal is a
huge step-level improvement for actually doing work (not just answering
questions).</p>
Model Context Protocol (MCP)</a></strong>: allows AI
to connect to local applications and data.</p>
Git hosting apps</a></strong>:
Gitea/Github/Gitlab have demonstrated amazing utility for software developers by
mixing local tooling (Git, Actions, etc.) with an accessible cloud web
application.</p>
Pushing data to the edge is how to scale, and files are the obvious way to do
this. While this has
always been true</a>,
the rise of AI is making this fact crystal clear.</p>


💬 The importance of effective communication
2025-10-13T00:00:00+00:00
Recently</a>,
effective communication has been discussed. But let's step back -- why is
communication so important? As developers, isn't coding what we do? Think for a
minute -- how much time gets spent coding/designing vs. everything else? For
most, it is probably too little time on producing and way too much on everything
else. Becoming a more effective communicator is the one way to spend less time
on "everything else" leaving more time to produce.</p>


🧠 Context is everything
2025-10-10T00:00:00+00:00
In conversation, I find myself saying: "what is the context of your comment?" At
times we find ourselves responding before we really understand something.</p>
AI does the same thing -- it just spouts off, even if it does not understand
your context.</p>
This is one reason local agents like Claude Code work so well, vs AI in a web
browser. Local agents at least have the context of your code and local files.
However, it still does not have access to your complete domain context -- that
is your part.</p>


💡 Reflections from AI-coding a 10,000 line app
2025-10-08T00:00:00+00:00
Lessons from AI-coding a 10,000 line app</p>
BRun</a> is getting usable. Claude Code wrote
almost all of it using
Doc-driven development</a>
(manually write/update the docs, and have Claude write the code).</p>
Some thoughts:</strong></p>

Claude is fast -- there is no way I could have written this app manually in a
day or so.</li>
Large changes are easy with AI.</li>
Keeping docs up to date is easy.</li>
Having AI do the coding freed up more time to think about the app user
interface, etc.</li>
Claude is a nice partner to bounce ideas off of.</li>
</ul>
Some things to watch out for:</strong></p>

Look out for duplicated/unnecessary code. There were multiple cases where
Claude was calling the same function multiple times.</li>
Refactoring is not always complete. If things get restructured, check that
everything got cleaned up.</li>
Check the tests -- Claude really goes crazy generating tests. A lot of work
remains on this repo reviewing and cleaning up tests.</li>
Claude often tends toward complex approaches and a lot more code than needed
-- like many junior devs. Close monitoring is needed to ensure the
architecture takes the simple path.
Code is a liability, not an asset</a>
-- even with AI.</li>
Docs updated by Claude tend to be overly verbose. I was continually trimming
them down to remove duplicated/unnecessary content.</li>
Critical things like life-cycle and state management often get overlooked.</li>
Race conditions are also easy to miss in AI-generated code.</li>
</ul>
In the end, this project went well, but it still needed expert guidance at
times. Perhaps some of the above could have been improved with more specific
rules. Like many things, the greatest strength is also the greatest weakness.
Claude is really good at generating code -- so good that it tends to generate
way too much at times. The hardest part of an app like this is figuring out what
it should do, how it should be configured, what the UI looks like, and debugging
problems when it does not work. This takes experience and is still a lot of work
to get right.</p>


📏 Less code is not the ultimate goal
2025-10-03T00:00:00+00:00
Less code is not the ultimate goal</p>
Consider the following
byte to binary C program</a>:</p>
int</span> main</span>(</span>int</span> b</span>,</span>char</span>*</span>*</span>i</span>)</span>{</span>long</span> long</span> n</span>=</span>B</span>,</span>a</span>=</span>I</span>^</span>n</span>,</span>r</span>=</span>(</span>a</span>/</span>b</span>&</span>a</span>)</span>>></span>4</span>,</span>y</span>=</span>atoi</span>(</span>*</span>++</span>i</span>)</span>,</span>_</span>=</span>(</span>(</span>(</span>a</span>^</span>n</span>/</span>b</span>)</span>*</span>(</span>y</span>>></span>T</span>)</span>|</span></span>
y</span>>></span>S</span>)</span>&</span>r</span>)</span>|</span>(</span>a</span>^</span>r</span>)</span>;</span>printf</span>(</span>"</span>%.8s</span>\n</span>"</span>,</span>(</span>char</span>*</span>)</span>&</span>_</span>)</span>;</span>}</span></span></code></pre>
Unless you are trying to win the International Obfuscated C Code Contest, this
is not good code. In spite of what I've
said before</a>,
less code is not always better. The highest goal is readable maintainable code.
Sometimes we have to write more code to have readable code. An example is
aligning the happy path</a>.</p>
Less code is more about not implementing unneeded features, cleaning up dead
code, refactoring to simplify and eliminate duplicated code, etc. The goal of
less code should never sacrifice readability or maintainability.</p>


😊 Other ways to align the happy path
2025-10-01T00:00:00+00:00
Yesterday</a>, we discussed
aligning the happy path in code. Are there other ways to align the "happy path"?
(The happy path is the desired path where things are functioning as expected.)</em></p>
Some thoughts:</p>
Focus on your productive people - as a manager focus your efforts on making life
better for the productive 95% of your team, rather than trying to fix the
unproductive 5% who probably cannot be helped anyway. At least make your efforts
proportional. We should be willing to help people who sincerely want help, but
trying to help those who do not want help drains our soul.</p>
Processes should focus on making the primary work go faster and easier. Sure,
problems must be addressed, but returns are higher on improving the work
desired, vs. knocking down problems. If the former is done better, less time
will be spent on the latter.</p>
Proactively write tests to ensure the system is doing what you want it to do,
rather than wait for bugs to appear and then write tests to cover those.</p>
In summary, if energy is proactively focused on what is wanted to happen, a lot
less time will be spent battling things not wanted.</p>


😊 Align the "happy path" in your code
2025-09-30T00:00:00+00:00
An idiom that I learned in the Go community is to align the happy path. An
example in C below:</p>
happy path not aligned</h2>
int</span> process_file</span>(</span>const</span> char</span>*</span> filename</span>)</span> {</span></span>
    FILE</span>*</span> f </span>=</span> fopen</span>(</span>filename</span>,</span> "</span>r</span>"</span>)</span>;</span></span>
    if</span> (</span>f </span>!=</span> NULL</span>)</span> {</span></span>
        char</span> buffer</span>[</span>100</span>]</span>;</span></span>
        if</span> (</span>fgets</span>(</span>buffer</span>,</span> sizeof</span>(</span>buffer</span>)</span>,</span> f</span>)</span> !=</span> NULL</span>)</span> {</span></span>
            //</span> process buffer</span></span>
            fclose</span>(</span>f</span>)</span>;</span></span>
            return</span> 0</span>;</span> //</span> success</span></span>
        }</span> else</span> {</span></span>
            fclose</span>(</span>f</span>)</span>;</span></span>
            return</span> -</span>2</span>;</span> //</span> error: unable to read</span></span>
        }</span></span>
    }</span> else</span> {</span></span>
        return</span> -</span>1</span>;</span> //</span> error: unable to open</span></span>
    }</span></span>
}</span></span></code></pre>happy path aligned</h2>
int</span> process_file</span>(</span>const</span> char</span>*</span> filename</span>)</span> {</span></span>
    FILE</span>*</span> f </span>=</span> fopen</span>(</span>filename</span>,</span> "</span>r</span>"</span>)</span>;</span></span>
    if</span> (</span>f </span>==</span> NULL</span>)</span></span>
        return</span> -</span>1</span>;</span> //</span> error: unable to open</span></span>
</span>
    char</span> buffer</span>[</span>100</span>]</span>;</span></span>
    if</span> (</span>fgets</span>(</span>buffer</span>,</span> sizeof</span>(</span>buffer</span>)</span>,</span> f</span>)</span> ==</span> NULL</span>)</span> {</span></span>
        fclose</span>(</span>f</span>)</span>;</span></span>
        return</span> -</span>2</span>;</span> //</span> error: unable to read</span></span>
    }</span></span>
</span>
    //</span> process buffer</span></span>
</span>
    fclose</span>(</span>f</span>)</span>;</span></span>
    return</span> 0</span>;</span> //</span> success</span></span>
}</span></span></code></pre>
In the first example, it is more difficult to follow the logic due to nested if
statements. In the second example, it is easy to scan down through the code
aligned to the left and sequentially see what it is supposed to do.</p>


🔓 You should not be needed
2025-09-29T00:00:00+00:00
An effective process, team, or organization should be able to continue on if any
individual steps away for a bit, or forever. But, most people are insecure and
like to be needed, so subconsciously structure things to require them to be
present for things to proceed -- the gatekeeper or authoritarian mindset.</p>
A few ideas:</p>

Share everything you know</a>
with the widest possible audience.</li>
Document why</a>.</li>
Automate tedious processes so anyone can execute them. This also serves as
good documentation.</li>
Encourage others to do things you do.</li>
Prefer transparency for accountability rather than oversight or status
reports.</li>
</ul>
Creating a process or culture that can function just fine without a single point
of dependency makes that person more valuable than ever, and frees them up to do
other higher-value things.</p>


🚧 Three things that hinder product development
2025-09-24T00:00:00+00:00

Not using new tech when applicable.</li>
Doing the same work over and over again instead of automating.</li>
Lack of collaboration.</li>
</ol>


⚙️ Three aspects of product development
2025-09-23T00:00:00+00:00

Finding and leveraging existing technology.</li>
Developing some of your own technology.</li>
Mixing it together.</li>
</ol>
All three are necessary.</p>


⚖️ Technical debt compounds with you doing nothing
2025-09-22T00:00:00+00:00
Technical debt is like any other kind of debt -- it accrues when nothing is
done. This is perhaps the most insidious aspect of debt.</p>
How does technical debt work? Mainly that software (and technology in general)
moves on (especially OSS), and we do not keep up.</p>

"It works, why do we need to update? That just introduces risk."</p>
</blockquote>
But the longer we put this off, it becomes exponentially harder to ever catch up
-- just like compound interest on financial debt never receiving payments.</p>
The other aspect of doing nothing is that infrastructure rots -- the tools,
automation, testing, and know-how that even allow making changes and improve
things. So when the gap is huge and there is no infrastructure (way to close the
gap), it is nearly impossible, so we declare bankruptcy and start over.</p>
There is another path: keep up-to-date. This forces the infrastructure to stay
fresh. Iterations expose areas that can be automated and improved. Investing in
YOUR platform compounds benefits over time.</p>


📚 Without docs
2025-09-12T00:00:00+00:00
Design files (mechanical, electrical, or software) contain the technical details
that allow us to manufacture something in the moment, but they often don't
contain the "why"</a>.</p>

Why is this mechanical feature present?</li>
Why is this circuit necessary?</li>
Why is the code structured this way?</li>
</ul>
Most designs exist this way -- without adequate documentation. What happens two
years later when something quits working? What happens when a new feature needs
added? What happens when someone leaves?</p>
World-class engineering requires world-class documentation. There is no other
way.</p>


⚖️ Code is a liability, not an asset
2025-09-11T00:00:00+00:00
A wise developer
once said</a>:</p>

Code is a liability, not an asset. Aim to have as little of it as possible.</p>
</blockquote>
This is perhaps the #1 rule for clean code. Some examples:</p>

Don't implement "just in case" features. Remember
YAGNI</a>.</li>
Premature abstractions</a>
often do more harm than good.</li>
Premature optimization is the
root of all evil</a>.</li>
Simple data structures scale</a>.</li>
Find patterns</a> and
refactor often.</li>
</ul>
Just-in-case development (implementing stuff not needed right now) is a powerful
force, especially with less experienced developers who have not spent
significant time maintaining software.</p>

Understand the problem before starting implementation.</li>
If at all possible, reproduce the bug before trying to fix it.</li>
</ul>


✂️ YAGNI
2025-09-10T00:00:00+00:00
YAGNI</a> - "You ain't
gonna need it!"</p>
If you don't need it now, the chances you'll need it in the future are small, or
at least not the current implementation. Do the simplest thing that works now. Get
feedback. Repeat.</p>
Iterations</a> are
the key -- get better at doing iterations faster (including testing, and
feedback), not implementing more stuff now.</p>


🐛Kernighan's Law
2025-09-09T00:00:00+00:00
Kernighan's Law:</strong> Debugging is twice as hard as writing the code in the
first place. Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it.</em></p>
Cliff's Law:</strong> In three months, it will be three times harder to read the
code you just wrote than writing the code in the first place. With AI, add a 10x
factor.</em></p>
Maintaining code over a product lifecycle is many times harder than the initial
implementation. With AI,
doc</a> or
spec</a> driven development is critical.
Otherwise, you will bury yourself in technical debt. Make it easy on others and
your future self.</p>


🌀 Complex systems are hard to predict
2025-09-08T00:00:00+00:00
My wife makes sourdough, which is a multi-day process and at times
unpredictable. Recently, the current batch was a little sluggish and not
expanding like it should, so she "fed" it and left it on the counter. It took
off during the night and was all over the counter in the morning.</p>
Sourdough is a complex living system. Because of this, sourdough is difficult to
predict 100% of the time. As computer systems become more complex, they take on
some of the same characteristics of living systems. You can't completely predict
everything. What worked yesterday may not work today, due to various external
factors that cannot be controlled or fully understood. Especially with networked
systems -- we cannot control everything else on the network, or involved with
the network. We cannot control what new feature customers are going to ask for
that might require us to update dependencies. We can't predict what new edge
case will trigger a bug we have not seen before.</p>
The best we can do is keep tools and processes flexible, up-to-date, and as
automated as possible (YOUR Platform). This enhances the ability to adapt to the
external forces that cannot be controlled.</p>


📊 Simple data formats + powerful tools scale
2025-09-04T00:00:00+00:00
There is an ongoing shift in data analysis tools. Another example is
datavzrd</a>. datavzrd is a zero/low-code,
interactive, visual, server-free, browser-based reporting tool for tabular
datasets.</p>
The
paper</a>
makes an interesting statement:</p>

Most spreadsheet applications, including Excel, lack the capability to quickly
and easily visualize different columns of a dataset. They also do not
support reproducibility of results for similar datasets without repeating the
work</strong>.</p>
</blockquote>
This last point is key. For example, if data needs monthly review, tools like
this make a lot of sense. Complete automation becomes possible. Manual
spreadsheet work every month becomes unnecessary, eliminating tedious and
error-prone processes.</p>
This is another example of the trend toward
simple data formats</a>
and flexible tools.</p>
YAML, CSV, and Markdown are the file formats of the future.</p>
Simple data formats + powerful tools is the formula that scales.</p>


📚 Documentation never has to go out of date again
2025-09-03T00:00:00+00:00
AI is really good at updating documentation after implementation. While I
strongly advocate
writing the documentation to drive AI</a>,
inevitably, the implementation takes some unanticipated turns, so making sure
everything is in sync after implementation is a good task for AI.</p>
Are the docs up to date? With Claude Code (or something similar), there is
really no excuse.</p>
However, it helps to standardize on the format the AI knows best:
Markdown</a>, and it helps a lot to
keep the
docs close to the design</a>.</p>


📚 Keep documentation close to the design/code
2025-09-02T00:00:00+00:00
I find it works best to keep documentation as close to the code or design files
as possible. There are several benefits:</p>

You can update, commit, and review documentation as part of the standard
working process (pull requests, CI, etc.)</li>
You can use tools you are already using (like your favorite editor)</li>
It encourages the use of Markdown (the format AI works with)</li>
Claude Code, which you run from your working directory, has access to it. This
has always been important, but with AI tools, it is even more important—more
on that in future.</li>
</ul>


📝 Document "why" not "how"
2025-08-29T00:00:00+00:00
Be kind to others (and your future self) and document why you did something.</p>
AI is pretty good at generating the "how" any time you need it, but only you
know the "why" because you have that context.</p>


🔄 Closed-loop engineering
2025-08-28T00:00:00+00:00
What are some ways we can close the loop in engineering efforts?</p>

Systematic testing</li>
User feedback</li>
</ul>
These ensure whatever we build works</strong> and is useful</strong>.</p>


🚴 The power of movement
2025-08-27T00:00:00+00:00
Rod's thoughts on
Riding a Bike</a> are thought
provoking.</p>

... A decision that might be bad. Yet, with it, you can steer. Just like
riding a bike. If it's standing with no movement, you can direct the handle
and still be where you are. It's when you're moving on the bike that you give
direction.</p>
</blockquote>
I've often experienced this in software development where I don't really know
how I'm going to solve a problem until I start coding, and then things start to
emerge. The same happens with
writing</a> and
sketching</a>.</p>
There is value in up-front planning, but there is power in movement, especially
when you iterate</a> and
close-the-loop</a>.</p>


🔄 Closed-loop humans
2025-08-26T00:00:00+00:00
What are some ways we as humans can close the loop?</p>

Listen</li>
Reflect</li>
Write</li>
</ul>
Without these, we operate in a largely open-loop mode without the feedback we
need to improve or correct our course.</p>
Action bias is good, but it helps to know we are doing the right things and
going in the right direction.</p>


🔄 Look for closed loops
2025-08-25T00:00:00+00:00
Closed loop systems are stable. Why? Because they learn and self-correct.</p>
In any system, process, person, or organization, look for closed loops. If
something is not working, chances are there is an open loop.</p>


🔄 Close the loop with AI
2025-08-22T00:00:00+00:00
There is a fascinating statement in a
recent MIT study</a>
that claims 95% of organizations are getting zero return from GenAI investments.</p>

The core barrier to scaling is not infrastructure, regulation, or talent. It
is learning. Most GenAI systems do not retain feedback, adapt to context, or
improve over time.</p>
</blockquote>
AI proves useful to software developers, perhaps because so much context is
available, giving it a better starting point than in most other industries.</p>
However, I think we can still do better by closing the loop. The below workflow
is currently working well for me:</p>
</p>
Closing the loop means retaining and re-using what you have learned.
Interestingly, it does not really matter if humans or AI are doing the work—the
same process works well with both.</p>
Notice the absence of AI prompts -- while an AI tool may remember prompts for a
session, they are lost on the next. They are "open-loop".</p>
Code is not the most valuable asset you own, but rather the ability to generate,
maintain, and improve it—the process. It is kind of like personal wealth and
skills. Even if we lose the first, we can always generate more with the second.</p>


💰 Ways to reduce your AI costs
2025-08-21T00:00:00+00:00
AI is not free, as there are significant costs associated with training and
hosting modern AI models. Some ideas on how to reduce costs with tools like
Claude Code:</p>

If you do something that is somewhat repetitive, have AI generate scripts or
programs that do the work instead of using AI directly. AI excels at creating
automation tools.</li>
Capture context. Instead of relying solely on prompts, update documentation
and have AI implement the doc changes. This approach captures a lot of context
instead of losing context in prompts.</li>
Start with more complete and accurate descriptions of what needs to be done.</li>
Use cheaper models. In my experience, Claude Sonnet performs just as well as
Claude Opus for most tasks I need to complete. Sonnet is 1/5 the cost and is
generally faster.</li>
If the agent is spinning its wheels, stop it, provide more details, and
resume.</li>
</ul>


💰 What is the cheapest PDM tool out there?
2025-08-19T00:00:00+00:00
Product Data Management (PDM)</a>
is how you manage and publish your product data. The goal is to ensure all the
stakeholders share a common understanding, confusion is minimized, and quality
is maintained.</p>
The simplest PDM system is to put a bunch of files on a fileserver and talk
about them in meetings, emails, or other messages. This is a start, but it is
error-prone and does not scale.</p>
There are many commercial PDM tools available. CAD systems,
like Onshape</a>,
provide PDM features. Software developers have been doing PDM to various degrees
for decades using open-source tools. You have SCCS (1972), RCS (1982), CVS
(1986), SVN (2000), and Git (2005).</p>
Git and its various
hosting platforms</a>
support a number of the
features</a>
needed for PDM. The cost is minimal and you probably already have people on
staff who are experts at using it. For small teams, Git may just be the PDM
solution you need. However, there is a learning curve, and some additional
tooling may be required to make it accessible—more on that in the future.</p>


🗂️ Git has an unfortunate name
2025-08-18T00:00:00+00:00
Git</a>, which was created 20 years, and
various
hosting platforms</a>,
has revolutionized</a> software development
workflow and collaboration. The name Git was chosen by its creator, Linus
Torvalds, and has a humorous backstory rooted in both practicality and
world-class trolling. In British slang, "git" is an insult meaning
an unpleasant or silly person, which Torvalds acknowledged with characteristic
Finnish bluntness: “I'm an egotistical ___</strong>, and I name all my projects
after myself. First Linux, now Git.”</p>
Most of us software developers, especially in OSS culture, appreciate a little
humor and self-deprecation (it's basically a job requirement at this point).
However, those from other disciplines seem to have an inexplicable aversion to
anything with the word "git" in it. Perhaps this is understandable -- why would
I ever use a tool that literally calls me stupid in the name? That must mean I'm
stupid if I use it, right? This appears to be a subconscious psychological
defense mechanism that is surprisingly hard to overcome. It unfolds something
like this:</p>

Developer (D)</strong>: Hey, why don't we use
Git*</a>
for this project? It handles all our
file management problems</a>
with some project management tools thrown in.</li>
Project Manager (P)</strong>: Great idea! (Famous last words)</li>
D</strong>: Can you comment on that "Git" PR I just posted?</li>
P</strong>: Sure.</li>
<deafening silence></code></li>
D</strong>: Can you push the new product specification you wrote to the "Git" repo
in Markdown so we can track and review changes?</li>
P</strong>: Sure! (Proceeds to email a Word doc like it's still 2003)</li>
<developer dies a little inside></code></li>
D</strong>: Can you test the new release in "Git"? (link to Git* release).</li>
P</strong>: (a little while later) Can you just email the files to Nick? (Git? What
Git?)</li>
D</strong>: <internal screaming intensifies></code></li>
D</strong>: (at the weekly status meeting) Why am I the only one tracking my tasks
in "GitHub" issues?</li>
P:</strong> Good question, we should all do that. (Immediately forgets and goes
back to sticky notes)</li>
<the sound of one developer crying></code></li>
D:</strong> Why don't people use "Git" when it has most of the file management,
collaboration, and PM tools we need?</li>
P:</strong> It's too hard. (Says the person who maintains a 47-tab Gantt chart in
Excel)</li>
D:</strong> It's a lot simpler than a mega Gantt chart in Excel, or a 3D model in
SolidWorks.</li>
P:</strong> True. (Goes back to Excel anyway)</li>
<the void stares back></code></li>
</ul>
The only rational conclusion I can come up with is that the word "Git" somehow
triggers a Pavlovian response in most people's brains that makes them physically
unable to click on anything Git-related. It's like some kind of linguistic
kryptonite. Only a minority of battle-hardened software nerds, especially those
who've been sufficiently traumatized by OSS projects, have managed to overcome
the existential crisis of using a tool that openly mocks them.</p>


📁 Git* solves the problem of managing files
2025-08-15T00:00:00+00:00
The thing you get with Git*</a> is a
sane way to manage files:</p>

organization</strong> (team/repo/directories)</li>
documentation</strong> (excellent support for Markdown)</li>
efficient navigation</strong> (links in READMEs and other markdown files)</li>
history</strong> (who changed what when)</li>
collaboration</strong> (issues, pull requests)</li>
distributed</strong> (anyone can work anywhere)</li>
automation</strong> (continuous integreation (CI), continuous deployment (CD),
hooks, etc.)</li>
reviews</strong> (PRs)</li>
workflow</strong> (enforces workflow such as requiring PRs, approvals, etc.)</li>
versions</strong> (feature and maintenance branches)</li>
releases</strong> (built in way to do formal releases)</li>
security</strong> (granular access controls and workflow permissions, bank level
security for web access (https), and better for Git operations (ssh keys))</li>
</ul>
Git* is not primarily a way to manage projects, but rather a way to manage and
collaborate around files, with some project management thrown in. It is centered
around the work, which is the main point.</p>
Other tools have more advanced project management features, which are needed at
times. But they are missing a key ingredient: tight integration with your files.
The closer you can keep your project management and other metadata to the actual
work, the better.</p>


📁 The file dilemma
2025-08-14T00:00:00+00:00
There is this movement against files in most disciplines. Files are hard to
manage, track, version, and secure (unless you use Git). Thus, a lot of tools
have moved to SaaS (software-as-a-service) applications in the cloud. Jon
Hirschtick
articulates this point of view</a>
very well. Cloud and APIs are the new "file."</p>
I'd like to present a few counterpoints.</p>

Integration:</strong> One of the ways we add value fast is integrating systems.
This is due to the
network effect</a>, where
Metcalfe</a> defines the value as
the square of the number of nodes (or in our case integrated systems or types
of data). When we can integrate multiple types of data (say hardware,
software, and mechanical), the value increase is much greater than just
linear.</li>
AI</strong>: AI needs context to be effective. The easiest way to provide context
is to run a local agent that has access to files that are in an open format.
MCP</a> (Model Context Protocol) may be
another way.</li>
Accessibility:</strong> One of the problems with SaaS and APIs is you don't really
own the data. Ten years down the road, are you still going to have access to
it? Are you still going to have the tools needed to use the data?</li>
</ul>
Files are flexible and scale. The success of Linux evidences this
(which adopts the Unix philosophy that everything is a file), the web (it still uses a
bunch of plain-text files for every site), and software development (anyone
ever heard of source code?). Software is by far the largest and most complex systems
humans have created, and it is still 99% based on the lowly text file. Since its
inception in 1991, the Linux kernel has had more than 20,000 contributors.
Onshape</a>
is impressive, but it is a long way from scaling to projects of this size and
longevity.</p>
The reality is that a mix of files and APIs is no doubt required. However, if
you want scalability and flexibility, I'm not sure you want to throw out files. GUI
technologies have been trying to do this for decades, but
HTML and friends</a>
are still king (anyone remember Flash?).</p>


🔄 AI and documentation - close the loop
2025-08-13T00:00:00+00:00
Starting with good documentation is often a
better way to use AI</a>.
However, you can also close the loop and have AI update the documentation when
it is finished. It can fill in details, clean things up, etc. This closes the
loop. Then repeat.</p>
</p>


🤖 AI is the new compiler
2025-08-12T00:00:00+00:00
There are two ways to use AI:</p>

Start with a vague prompt and keep prompting until we get the result we want.</li>
Point the AI at documentation, and keep updating the documentation until it is
clear enough that AI can do the right thing.</li>
</ul>
Even though it takes more time up front, in the end, the second approach will be
faster and more efficient because building a cohesive picture from a long string
of sloppy prompts is difficult, even with something as capable as AI. AI coding
engines are not free, so efficiency matters.</p>
AI is the new compiler -- IF we choose to use it that way.</p>
Writing clean code is a skill. So is writing clean documentation.</p>


✨ Anyone can write scripts now
2025-08-11T00:00:00+00:00
With Claude Code (and likely many other AI technologies), anyone can write
scripts now. Instead of writing in a technical language like Python or Bash, you
can now write a sequence of commands with some rules in plain English, and
Claude transforms these into the technical bits to make it happen.
Low-code</a> is
finally a reality. You can keep these as Claude commands, or have Claude write
permanent scripts, programs, tools, etc.</p>
It is now easier than ever to build YOUR Platform.</p>
See
this summary</a>
or
this example</a>
for more information.</p>


✏️ Effective Communication (Sketching)
2025-08-08T00:00:00+00:00
Sketching is kind of like writing cursive—it engages a dynamic in our brains
that inspires creativity. As we mostly use computers, it takes extra effort to
pick up pen or pencil, but surprising things happen when we do. It engages
broader areas of our brain, and there is a flow that comes from the lack of
restraints our digital tools have. So perhaps sketching is mostly about
communicating with ourselves, but we can still easily snap a photo with a phone
and share it. This is perhaps the lowest friction form of drawing.</p>
Here is an example</a> in
thinking through a hard problem where sketching really helped.</p>
One thing that helps me cultivate a sketching habit is to always have paper and
a pencil or pen out on the desk when working. The
daily worksheet</a> is
always on my desk—the back is a good place to sketch and write notes. I also
need to get into the habit of using a
journal</a>
more.</p>


🎨 Effective Communication (Drawing)
2025-08-07T00:00:00+00:00
We've already explored 4 modes of
communication</a> (audio,
written, images, video). Drawings are another, and today it is even easier to do
with tools like Excalidraw</a> and
Draw.io</a>. It is easy to copy drawings from both into other
applications.</p>
A clean drawing/diagram brings another level of professionalism to your work.</p>
Drawing with AI is another option, but I have not figured out how to use that
yet.</p>
One thing that has worked well for me is to use a single Excalidraw drawing as
an infinite canvas. This way I can easily see content I've created before, reuse
it, or use it for inspiration.</p>
</p>


📹 Effective Communication (Video)
2025-08-06T00:00:00+00:00
Video is now an important way to communicate. The most obvious type is Zoom
calls. But there are other ways we can use video such as screencasting,
recording demos, etc.</p>
Position a decent camera at a good angle. Shooting up your nose probably isn't a
good camera position.</p>
Screencasts are a scaling superpower. Record how you do something and share it
with people learning. Have them write the documentation or checklist.</p>
Screencasts may be a better way to demo. Record and share the demo ahead of
time, then discuss it in a meeting.</p>
With a screencast, it's even more important to have good audio. In a meeting,
people will overlook bad audio because you have a captive audience. But in a
recorded video, people will likely quit watching if the audio is bad. So get a
decent
podcasting mic</a>
or headset</a>.</p>
Learn OBS Studio - this is a professional way to record screencasts that is
free. I've found that
setting up a 1080p (1920x1080) section</a>
of my screen to record works well. Drag, resize, and stack the applications you
want to display in this section of the screen, then just record that single
scene. This is much simpler than setting up a separate scene for every
application.</p>
If you are working with hardware, get a document camera to video stuff on the
lab bench.</p>
And don't forget your phone takes excellent video - we all should probably make
better use of this capability.</p>
When possible, record in MP4 format. This format plays easily in most browsers,
so you can simply upload a file to a file sharing site, and people can
automatically play it in their browser.</p>
I've yet to figure out lighting, green screens, etc. - lots to learn yet.
Systems like this can always be improved, so keep improving.</p>


📸 Effective Communication (Images)
2025-08-05T00:00:00+00:00

"A picture is worth a thousand words."</p>
</blockquote>
There are many times when a screenshot or a photograph can drastically improve
the rate of communication. Therefore, figure out ways to do this efficiently.</p>
Make sure your computer system has an efficient way to capture and mark up
screenshots -- preferably in the same application. Spectacle in KDE is
excellent. I'm sure there are good options for other environments.</p>
Also have a quick way to get photographs off your phone into a message. I use
the Google Magnifier app on Android, sharing to a "Note to self" in
Signal, then copying the image in Signal desktop on my PC. This is much faster
than waiting for the image to upload to Google Photos.</p>
What works for you?</p>


✍️ Effective communication (Written)
2025-08-04T00:00:00+00:00
For starters, consider Michael Lynch's
"Underused Techniques for Effective Emails"</a>.</p>
A few points in summary:</p>

Put the most important information first.</li>
Use headings, code blocks, and other formatting.</li>
Be succinct.</li>
Provide context by quoting and replying
(bottom posting</a>
in emails and quoting in PR/issue threads). Don't make your readers read the
entire discussion and try to extrapolate the context of what you are trying to
say.</li>
</ul>


🤖 How I use Claude Code to help publish this blog
2025-08-01T00:00:00+00:00
I have been using Claude Code to help with the following tasks on this blog:</p>

proofread (including detecting
passive voice</a>)</li>
add tags</li>
add an emoji to the title</li>
</ul>
These are not rocket science, but they also take time and are something AI does
pretty well.</p>
This video</a> shows how the process works.</p>


🎧 Effective communication (Audio)
2025-07-31T00:00:00+00:00
Effective communication requires that those receiving information from us can
easily process it. I've been on way too many calls where you can barely hear the
speaker due to echo, background noise, or other audio problems. Some things I've
found helpful:</p>

Use a headset. This eliminates any possibility of feedback from speakers into
the mic.</li>
Use a good noise-canceling/rejecting mic, especially in noisy environments.
Some headsets (like the
Sennheiser SC series</a>)
block background noise amazingly well. Good podcast mics (like the
AT2020</a>)
are usually directional and block noise pretty well, especially when
positioned close to your mouth</a>.</li>
Use a
Bluetooth headset</a>
with your phone that has noise blocking in the microphone. Most
noise-canceling headsets focus on blocking background noise for your benefit.
On calls, optimize for your listener's benefit.</li>
Pro tip: never use the speakers and mic in your laptop, even if it is a Mac.</li>
Pro tip 2: don't use speakerphone mode on your phone -- especially for group
calls.</li>
</ul>
Don't be the one who disrupts a group call with crummy audio. You are a
professional -- use professional gear.</p>


📧 Top vs bottom posting in email
2025-07-29T00:00:00+00:00
I was thinking about computing history this morning, and I wonder how many young
people even know that
bottom posting in email</a> is even a
thing? Some of the older OSS projects
still have this culture</a>,
but I suspect that most emails sent today use top posting.</p>
I find myself getting lazy and top posting, but it is so much more considerate
of the reader (especially with modern email readers that thread messages) to
bottom post and only keep the text needed to understand the context or your
reply. The old emails are still in the thread and easily viewable -- there is no
reason to repeat the entire discussion chain in every email.</p>
Smart clients like Gmail somewhat alleviate this concern by hiding previous
message details, but it often hides too much, and you have to expand parts of
the old message to see what a response is referencing. And with top posting, you
often have to repeat the context of your reply vs. just including your reply
directly below the relevant context.</p>
This is one reason discussions in Git* issues and other forum type threads can
be so much more effective -- you can more easily follow the discussion over
time. But you can get near the same efficiency with email if you take a little
care.</p>


🎉 200 posts
2025-07-28T00:00:00+00:00
This marks the 201st post I've sent -- 2/3's of the way through the
300 bad posts</a>.</p>
How is it going? I still enjoy the process and the thinking it brings. However,
it is getting a little bit more difficult recently to keep the discipline,
especially with busy project and summer schedules.</p>
I continue to have many more ideas than time to write, so coming up with ideas
is not a problem.</p>
The publishing process has been further streamlined using Claude Code -- will
share more details on that in the near future.</p>
If you have any feedback, please reply -- always welcome!</p>


🔗 Integration, not isolation (part 2)
2025-07-24T00:00:00+00:00
We recently
discussed integration</a>,
not isolation from a technical perspective.</p>
In an organizational context, we can have integrated cross-functional teams, or
isolated silos.</p>
And the easiest way to
avoid silos</a>
is transparency and common systems. These things are hard -- they don't just
happen automatically.</p>


🔗 Integration, not isolation
2025-07-23T00:00:00+00:00
In a recent discussion</a>, the comment was made that
containers are not always a good fit for edge/embedded devices, as they are
designed for isolation (which is useful at times for cloud workloads), but with
embedded, we want tight integration. Justin Schneck advocates file-system layers
as an alternative. Isolation from the kernel is a feature in the cloud, but in
embedded systems, we often need direct access to the kernel (integration).</p>
We, as an industry, perhaps focus too much on isolation (security, protection,
every app bundles its own stack of dependencies, etc.). Short term, this is
convenient. Long term, this can lead to bloat, inefficiencies, and brittle
systems.</p>
Can we have integration AND decoupling? That is a topic for future thinking...</p>


📈 What do we mean by scaling?
2025-07-22T00:00:00+00:00
This can mean many things:</p>

More units sold</li>
More customers served</li>
More people doing the work</li>
More features</li>
More innovation</li>
More models produced</li>
More model options</li>
More quality</li>
</ul>
We could go on, but basically we mean more of something over time. And the "over
time" part means we need to be thinking long-term. You cannot scale with
short-term thinking—this is a logical fallacy. This is why YOUR Platform is so
important.</p>


🔄 Open loops -> technical debt
2025-07-21T00:00:00+00:00
If something is corrected, it needs to be fed back and incorporated, so you
don't keep redoing the same stuff over and over and eventually bury yourself in
technical debt.</p>
Effective organizations close the loop.</p>


🧩 Composing is hard, complexity is easy
2025-07-18T00:00:00+00:00
We've recently explored the idea that
composing simple things is the way to scale</a>.
But this is harder than just cranking something out. Some examples:</p>

It is much easier to create an Excel spreadsheet than write a Python script to
process a CSV file and create a chart which can be later automated in a CI
process.</li>
It is much easier to create lots of custom structs with lots of fields than to
create simple structs that can be used for multiple purposes.</li>
It is much easier to add additional functions to an interface than to create
additional well-thought-out interfaces.</li>
It is quicker to hack on a Raspbian image than to set up a proper Yocto build
for a Raspberry Pi single board computer that can be more easily maintained
long-term.</li>
It is quicker to draw a schematic with generic parts and manually assign
manufacturer part numbers/footprints, than set up a proper parts library that
will make subsequent designs and changes easier.</li>
</ul>
If your task is a one-off project, or something only you will ever interact
with, then the easy path may be fine. However, if you want to scale, the hard
path is the way forward.</p>


🎼 Compose software like music
2025-07-17T00:00:00+00:00
Music is an interesting thing -- a beautiful arrangement is composed of very
simple constructs -- a handful of simple notes on a relatively simple scale. The
brilliance is in how these simple constructs are arranged.</p>
Likewise, brilliant software is composed of simple data structures and
interfaces. Just like music, there is no limit to the ways these simple
constructs can be arranged.</p>
Conversely, complex constructs are very limited in how they can be used.</p>
Put the complexity in the arrangement, not the constructs.</p>


🌱📈 Simplicity scales
2025-07-16T00:00:00+00:00
When programming, we learn that
simple interfaces</a>
and
simple data structures</a>
are good, because they are flexible.</p>
Simple file formats</a>
are the same. We can do way more with a CSV or Markdown file in terms of
presentation and automation than we can with Excel or Word documents.</p>
This seems counterintuitive -- Excel is a very powerful tool, and the right
tool for many uses. However, an Excel file is not as composable as simple
formats. As an example, consider the
assets</a>
of a Perplexity labs project. They are Python scripts, CSV, Markdown, etc. Why?
Because a powerful tool, like AI, can do more with simple formats.</p>
Complex formats (Ex: Excel) allow a single person to do a lot in the short term,
but simple formats allow a team and automation to do way more over time.</p>
The way to scale YOUR Platform is to compose simple interfaces, data structures,
and file formats. A little more work short term, but way more powerful
long-term.</p>


📏 Goodhart's law
2025-07-15T00:00:00+00:00
Goodhart's law</a> states: "When a
measure becomes a target, it ceases to be a good measure".</p>
Metrics are useful, but they are just a tool, not the goal. If Metrics become
the goal, then they undermine collaboration, trust, and transparency.</p>


📈 Why GUI development tools don't scale
2025-07-10T00:00:00+00:00
This is an interesting read:
GUIs are Antisocial</a></p>
This hits on an important point... GUI tools don't scale, because they are not
social. Two of the most complex efforts humans have accomplished are chip design
and software development -- both are firmly entrenched in
text based formats and command line tools</a>.</p>


🔭 Text based formats and command-line tools are the future
2025-07-08T00:00:00+00:00
For the past three decades, we have been told that graphical user interfaces
(GUIs) will revolutionize computer use and even programming. In many ways, they
have made computers very accessible for the average person -- especially phone
UIs.</p>
However, the command-line tools and text files are still the standard for
building stuff. Most useful programming is done in text based programming
languages. Even most UIs are still implemented in text based languages such as
HTML and CSS. In spite of the many graphical options that have come and gone --
HTML remains the king.</p>
Why do text based formats and command line tools work so well?</p>

Text-based formats are easy to work with (collaborate, edit, tooling).</li>
Command-line based tools are easy to automate (build systems, CI, CD).</li>
</ul>
The coming AI storm makes these factors even more important. Because AI coding
is all about tooling, collaboration, and automation.</p>
If we dig a little deeper, text-based formats and command-line tools are "open"
-- you can do things that were not originally planned. Open formats and tools do
not lock you into a single paradigm, but rather allow you to build YOUR Platform
to meet your needs.</p>


🔄 Flipping the playbook for AI coding
2025-07-02T00:00:00+00:00
A lot of focus these days is on "good AI prompts."</p>
Some of the best prompts may be documentation and tests. Write your
CHANGELOG.md</code> entries first, then update the documentation, then write the
tests, then tell the AI to update the code to match.</p>
We tend to get it backwards—asking AI to generate tests and documentation
after the code. Flip this and do the docs and tests first.</p>
Your thinking will be better, and you'll move faster.</p>


🧑‍💻 The first step in AI coding
2025-07-01T00:00:00+00:00
✔️ Should be "How does this work?"</p>
❌ NOT "Implement this feature."</p>
Understanding should precede any work, especially in complex systems.</p>


👥 Alternatives to meetings
2025-06-26T00:00:00+00:00
✔️ Instead of presenting in a meeting, record a screencast and share it.</p>
✔️ Publish your work often.</a></p>
✔️ Utilize notifications in workflow tools.</p>
✔️ Utilize a team chat tool such as Signal, Slack, Discourse chat, Discord, etc.. This does wonders for bonding teams (especially distributed teams). You can always mute it when doing deep work.</p>


🥣 The two ingredients for agility
2025-06-24T00:00:00+00:00
✔️ Low friction</strong>: tooling and workflows that allow anyone to make changes
easily. (good build systems, clean libraries, release automation, etc.)</p>
✔️ Confidence</strong>: what we are doing is not going to break things. (CI,
end-to-end testing, review, etc.)</p>
#1 is obvious. #2 is subtle -- if we don't have confidence in the work, we
will avoid it.</p>


♻️ How to iterate quickly on hardware
2025-06-23T00:00:00+00:00
Today, we mostly think of iterating quickly on software, but it is becoming
increasingly possible to also do this with hardware. A few ideas:</p>

Printed circuit boards (PCBs) are now relatively cheap and quick to
manufacture in low quantities from companies like JLCPCB (you can have a new
PCB in less than a week at a very reasonable cost).</li>
Companies like JLCPCB and Macrofab offer prototype assembly services.</li>
It is practical to
assemble your own</a>
prototype PCBs quickly.</li>
You can now do more CI/CD type things with PCB design tools. KiCad has a
powerful DRC rules engine and the ability to script a lot of operations
(including generating outputs) from the command line. Run ERC/DRC in CI and
generate outputs in CD. This allows you to make and deploy changes with a high
level of confidence.</li>
Verified parts libraries</a> allow you to
easily reuse electrical components in new designs.</li>
Spice simulation tools allow you to verify designs before building a PCB --
again, you can have a higher confidence that things will work.</li>
3D printing in production</a> allows you to make
mechanical improvements without expensive tooling changes.</li>
Intelligent part numbering schemes</a> and
PLM tools</a> allow you to efficiently and
reliably manage changes.</li>
Use OSS tools
like KiCad</a>
where possible so anyone can make changes.</li>
Use Git workflows</a>.</li>
</ul>
The above is YOUR Platform. Here's the thing -- the more complex and the higher
value your product is, the more important it is to be able to iterate quickly
and confidently on both hardware and software. As an example, Tesla updates the
software in their cars frequently and continuously makes mechanical improvements
(sometimes as many as 120-160 changes per week). If Tesla can do it with cars,
you can probably do it with your low-volume industrial product.</p>


🤝 Modern product development is a collaborative endeavor
2025-06-20T00:00:00+00:00
There is no way around this -- even if you are a
Company of One</a>.
You must collaborate with:</p>
▶ OSS communities who provide the tools and software you use,</p>
▶ Your customers -- if you are not learning from them, you probably won't
understand and fulfill their needs,</p>
▶ and Your team.</p>
While the last item on the list is obvious, the first two are less obvious, but
perhaps just as important.</p>


✨ Use the most advanced technology that is comfortable in your product design
2025-06-19T00:00:00+00:00
Some examples:</p>
▶ When designing a PCB, put down the smallest packages you are comfortable
working with. Example: put down 0603's instead of 0805's, even if you don't need
the space right now.</p>
▶ Use a MCU with the most performance and RAM/Flash that is reasonable. Use a
STM32 instead of a PIC16.</p>
▶ Everything else being pretty much equal, use Go instead of Python.</p>
Why?</p>
✅ You may not need the space now, but the next iteration or variant of the
product will likely be smaller, so why not plan for that now if it does not cost
you anything?</p>
✅ Modern embedded systems that support field updates are flexible and can do a
lot. New features will be requested. Plan for that now.</p>
✅ Python may be good enough now, but what happens when you need to add a
handful more threads, more features, deploy to other architectures, and need
higher performance?</p>
Here's the thing -- it is impossible to predict all the directions a product
will take. Set yourself up now to be able to reuse what you're doing.</p>
🔭 Product VISION == Smart technology choices</p>


🔄 When to update application dependencies?
2025-06-18T00:00:00+00:00
Most modern languages (Node.js, Python, Rust, Go, Zig, Elm, etc.) have their own
package manager. So when should we update package dependencies? How about at the
beginning of every PR cycle? I'm working on making this my standard practice
when starting a new feature in an application -- first, update all dependencies.
There are several advantages to this flow:</p>

Updating frequently means you are dealing with small changes.</li>
Updating at the start of a dev cycle also gives you a standard time to do
this. Otherwise, things like updating dependencies tend to get neglected.</li>
Updating at the start of a dev cycle (vs end) gives you some time to exercise
new modules during development before merging.</li>
You automatically get security updates without having to track these too
closely.</li>
It feels right.</li>
</ul>
This also requires good tests, so that you can update with confidence. There
might also be an issue with a team if multiple developers are updating
dependencies at the same time. So in larger teams, perhaps one person or an
automated process should do this. The key thing is that it does get done, and
implement the needed automation to make this easy.</p>


➡️ Frontload the boring stuff
2025-06-16T00:00:00+00:00
Some examples:</p>

Think about and write tests before</em> implementing a new software feature.</li>
Automate deployment of a new service before</em> developing it.</li>
Create eCAD libraries before</em> doing the schematic/layout work.</li>
Document the architecture</a>
before</em> implementing it.</li>
Open a PR before</em> doing the work, and explain what is going to be done.</li>
Create a Changelog</a> entry describing the
benefit to the user</a>
of what you are doing, before</em> writing the code.</li>
</ul>
In the end, you will go faster and do better work. Why? Because you are
frontloading the tedious details that are difficult to find time for after</em> the
work is done.</p>
This is the difference between the prototyping and product development mindset.</p>


🌿 A natural way to get more testing/automation
2025-06-06T00:00:00+00:00
One thing I've noticed is when I release more frequently, I tend to automate. If
releases only happen once every year, it is easy to neglect automation, CI/CD,
etc. Automated tests are not needed as it is just quicker to manually test
everything. But, with frequent releases, there is a subtle push toward scripting
stuff, getting CI/CD running, etc. Instead of demanding more tests/automation,
perhaps decide to release more often, and the automation will naturally follow.</p>


🌒🌓🌔The four phases of product development
2025-06-05T00:00:00+00:00
</p>
Each phase is produced by someone or something. And the problems arise when we
mix up who does what.</p>

Great/innovative ideas usually don't come out of a committee. Truth is a solo
pursuit. Evaluate ideas in meetings, but don't try to come up with them there.</li>
When a team is involved, collaboration is required. The better your
collaboration culture/workflow, the more efficient your team will be.</li>
Manually building, testing, and deploying software is a waste of human time --
leave that to machines when possible.</li>
Developers are horribly inept at testing/evaluating their own work. Any
outside viewpoint is useful -- especially with user interfaces.</li>
</ul>
The faster you can do this loop, the faster you'll learn, and the quicker you
can build something useful.</p>


⏩ Get on with it ...
2025-06-04T00:00:00+00:00
While many great ideas and algorithms germinate in deep work and solo thinking,
bringing a modern product (especially one containing software) to life require
many iterations with feedback and learning. So lets propose a new release
every week</a>.
But, the common objection is: we can't do a quality release every week? A
monthly release seems more reasonable, then we have plenty of time to get it
right.</p>
You can't build a great product until you've built 52 bad ones.* If you only do
an iteration every month, then you are looking at 4.3 years to get to good. If
you do an iteration every week, then it only takes 1 year.</p>
So instead of pining away about how can I possibly build a good iteration every
week, quit worrying about building a perfect release, and instead focus on
getting through the bad ones as quickly as possible.</p>
This is a parody on Jonathan Stark's
"Weekly vs. Daily"</a>
post.</p>
* This may be an exaggeration, but if we are honest with ourselves, it is often
astounding how many revisions it takes to get something working, let alone
great.</p>


👥 Who are users?
2025-06-03T00:00:00+00:00
To get really valuable feedback, we need users. If the product has not been
released yet, find someone -- external CI system, CEO, beta users, manufacturing
people, accountants, your mom ...</p>
Often, when it leaves the developer's computer, we find it does not even build,
let alone run and do something useful!</p>
A user is anyone</em></strong> other than the developer.</p>


♻️ How to define an iteration?
2025-06-02T00:00:00+00:00
I've found it useful to define an iteration as a formal release (software that
is packaged and users use, a PCB that you build, a prototype, a web app that you
deploy, etc.). You don't learn many things until you build and, more
importantly, others use - even if your users are only internal. Practice formal
releases during development, and then the process will be polished when you go
into production.</p>


🧑‍💻 How can we increase developer productivity?
2025-05-29T00:00:00+00:00

"The way you get programmer productivity is not by increasing the lines of
code per programmer per day. That doesn’t work. The way you get programmer
productivity is by eliminating lines of code you have to write.The line of
code that’s the fastest to write, that never breaks, that doesn’t need
maintenance, is the line you never had to write." --
Apple WWDC '97 Steve Jobs Closing Keynote</a></p>
</blockquote>
How can we reduce the lines of code we need to write? Several ways come to mind:</p>

Use OSS components</li>
Build on someone else's platform</li>
Don't implement "just-in-case" features</li>
Removed unused code (linting tools can find this for you)</li>
Don't implement unnecessary abstractions</li>
Simplify</li>
Reuse what you have already done (YOUR Platform)</li>
</ul>


📈 What is the quickest way to store and track data?
2025-05-28T00:00:00+00:00
Do you need a quick way to track some data? Serial numbers, part numbers, test
results, defect metrics, etc.?</p>
There are many ways, but one of the quickest and most flexible is
CSV</a> files in Gitea. You
are already using
Git*</a>,
right?</p>
With CSV files in Gitea, you can:</p>

View the data as a table</li>
Easily edit the data using a variety of tools (Libreoffice, Gitea web UI,
etc.)</li>
Track and review changes</li>
Manage access/permissions</li>
Work in a distributed fashion</li>
</ul>
Additionally, it is easy to add automation to CSV files in Go or Python. While
databases can be very useful for larger datasets, they are often overkill for a
lot of things today, as most datasets (especially human-generated ones) now
easily fit in memory. And when it comes time to use a database, it is easy to
import your existing CSV data into a database.</p>


🌱 OSS can decrease or increase your code liability
2025-05-27T00:00:00+00:00
A wise person
once said</a>:</p>

"Code is a liability, not an asset. Aim to have as little of it as possible."</p>
</blockquote>
Whether the open-source code in your projects is an asset or a liability depends
on how you handle it. Almost all code in complex systems
needs to be maintained</a>.</p>
If you do not update the OSS code in your projects, you are essentially
forking it</a>, and it is now "your code" and you own it.
It is a static entity that is not gaining value unless you put direct effort
into it. Additionally, there will likely be issues that you now need to solve.
Very few OSS project maintainers are going to be interested in helping you fix a
problem in a 5-year-old snapshot.</p>
Alternatively, if you keep the OSS code in your projects up-to-date, then many
issues automatically get fixed (security, compatibility, performance, bugs, new
features, etc.). Additionally, if there is a problem, then maintainers will
likely work with you to fix it. In this scenario, OSS code is an asset -- it is
gaining much more value than your direct effort. However, you must commit to:</p>


updating OSS components regularly</p>
</li>

having an automated way to comprehensively test the system</p>
</li>

engage with the OSS projects you use, build relationships, and contribute
where you can (documentation and bug reports are contributions)</p>
</li>
</ul>
Some effort is required on your part -- it's not a free lunch. However, if you
do this right, the benefit-cost ratio (BCR) is high --
well worth it in the long term</a>.</p>


🤝🔍 Trust, but verify
2025-05-23T00:00:00+00:00
Apparently, this is an
old Russian proverb that Ronald Reagan popularized</a>.</p>
This is good general advice in life -- don't automatically distrust people and
go looking for problems, but when something does not look or feel right, verify.</p>
As engineers, when we are building something new, this is important.</p>
✅ Are our assumptions correct?</p>
✅ Have we checked the edge conditions?</p>
✅ Did this change break anything else?</p>
✅ What are we missing?</p>
As systems become more complex, two tools come to mind:</p>
▶ Simulation (a faster/cheaper way to verify)</p>
▶ Automation (testing, deployment, etc. -- do things humans don't have time
for)</p>
In complex systems, blind trust does not get us very far, especially in
ourselves. As engineers, the work we should verify the most is our own.</p>


🌽 What is the easiest way to avoid silos?
2025-05-21T00:00:00+00:00
Silos occur when a person or group of people operate in isolation from the rest
of the organization. One of the reasons you have failure in organizations is one
silo does not know that the other silo is doing.</p>
Silos are deadly in our organizations. How can we avoid them? The efficient way
is to use open, transparent, and common systems where possible for:</p>

Issue/project tracking</li>
Part number management</a></li>
Information storage (design files should be versioned in Git)</li>
Releases</li>
Change control and review (Pull requests)</li>
Documentation</a> (Markdown,
CSV, etc)</li>
</ul>
Software developers build really good workflow tools. Why, because they can
build their OWN tools. For this reason, no matter what your discipline, I
recommend starting with
Git*</a>.
For small teams, this can get you most of the way there. For larger teams, you
may need something more comprehensive like odoo</a>. But,
start small with Git*, cover the basics, and add more complex systems when
needed.</p>


🤖 The biggest problem with AI
2025-05-20T00:00:00+00:00

"It ain't what you don't know that gets you into trouble. It's what you know
for sure that just ain't so." - Mark Twain (maybe)</p>
</blockquote>
AI is really good at summarizing existing information. It is less good at
thinking, although it can do some amazing things. The biggest deficiency I see
right now is AI does not know how to ask questions and doesn't know what it
doesn't know. It just makes assumptions and pretends like it knows what it is
doing. People like this are some of the hardest ones to work with. They act like
they know everything and never think they are wrong. Smart people, blind to
their own weaknesses, make the subtlest, hardest problems to find.</p>
Until AI is self-aware of its own weaknesses and knows how to ask questions, we
cannot fully trust it to operate like a professional human. It needs watched
very carefully. AI can still be very useful, and it will likely get better, but
for now, I think this is the current state.</p>


🤖 What is AI good for?
2025-05-19T00:00:00+00:00
AI is really good at summarizing existing information -- Perplexity Pro mostly
has replaced Google/Duckduckgo, etc., and is worth every penny. AI is really
good at finding API documentation -- much quicker than searching through docs,
and often provides a useful snippet of how to use an API. Occasionally, I'll use
AI to write a quick Python script where I don't care how it works.</p>
The most useful scenario is when you are having an intelligent back-and-forth
conversation with AI, and you understand what is going on. What does not work is
just lazily expecting it to do all your work so you don't have to understand it
-- especially the hard parts like architecture. If AI is doing things that you
don't understand, ask it to teach you. Write tests so you can be sure it is
right. Trust, but verify.</p>
I'm currently working on a project with a smart intern working on the frontend
code. He is working fast and appears to be vibe coding some of it. Some of it is
a bit of a mess, but it mostly works. Some parts have obviously never even been
tested. Is vibe coding in this case a net gain? Probably, but I'm not sure yet
-- will know more once we try to maintain this code.</p>
On a project that is complex enough to require some architecture,
coding is not the hard part</a>.
Architecture, requirements, integration, testing, debugging, support ... all
this stuff that is required to get it right swamps any coding effort. As
one person said</a>,
"Code is a liability, not an asset. Aim to have as little of it as possible."</p>
At this point, don't aim to have AI write all your code for you. Use AI to help
you write better code faster that YOU understand.</p>


📝 What is my process for capturing experience? (part 3)
2025-05-16T00:00:00+00:00
Reflection is another important aspect of the experience capture process. Do we
carefully evaluate and critically think about what happened? We don't do this to
blame ourselves or others, but rather to recognize areas where we might improve.
Reflection can also include recognizing things that went well and what we need
to keep doing.</p>
Some examples:</p>

Personally spend 5m journalling at the end of each day. What went well? What
did not? What are we thankful for?</li>
Blameless postmortems</a></li>
Periodic project review -- what have we learned? Is what we learned captured?</li>
Pull requests -- have we captured everything we have learned when we did this
work? Are there documentation and tests so the work can be extended and
reused?</li>
Look for opportunities to automate. Automation is one of the best ways to
capture experience.</li>
</ul>
If we do not reflect, we are destined to keep repeating the same mistakes and
needlessly doing the same work over and over.</p>
One of the most tragic and common forms of human nature is to just keep doing
what we are doing, even if it is not working.</p>


📝 What is my process for capturing experience? (part 2)
2025-05-15T00:00:00+00:00
In
part 1</a>,
we discussed the importance of getting information into permanent mediums. Below
are a some that I use:</p>

Workflowy. This is my 2nd brain with 1000's of notes, links, howtos, etc. It
is easy to share Workflowy notes with others as needed and multiple people can
collaborate on a note with just a link -- no account needed. I can't recommend
Workflowy enough -- it is so flexible and powerful.</li>
Checklists</li>
Templates (typically a Google Doc)</li>
OSS projects

Github</a></li>
Simple IoT</a></li>
Yoe Distro</a></li>
GitPLM</a></li>
</ul>
</li>
My daily newsletter</a></li>
TMPDIR Community Forum</a></li>
TMPDIR Podcast</a></li>
Sketches</a></li>
Youtube

General</a></li>
SimpleIoT</a></li>
Yoe Distro</a></li>
</ul>
</li>
Automation (Ansible, scripts, CI, CD, etc.)</li>
</ul>
I try to not directly email useful information if possible, but rather write it
down somewhere permanent and then share the link.</p>


📝 What is my process for capturing experience?
2025-05-14T00:00:00+00:00
After
yesterday's post</a>,
I was asked: "What's your process to capture experience?"</p>
The most important thing is to get information out of transient mediums into
permanent mediums.</p>
What are the attributes of a permanent medium?</p>

Searchable</li>
Persistent (exists over time, backed up)</li>
Easily accessible to all relevant parties over time, even those who join late</li>
Well organized</li>
Has context</li>
Improved over time</li>
Turned into automation where possible</li>
</ol>
The first two attributes are the easiest. While email, chat, and meetings are
useful in teams for communication and notifications, they are NOT permanent
mediums for information. Even though they may be persistent and searchable, they
fail on the other attributes.</p>


🧠 How to leverage experience?
2025-05-13T00:00:00+00:00
To leverage experience, we need to capture it. And to capture it, we need
process. Without this, leveraging the experience of our best people is at best
random. If we can capture experience, then we have it forever, even when our
best people inevitably move on.</p>


🔭 How to encourage vision?
2025-05-09T00:00:00+00:00
Innovation is fueled by vision. Without vision, things can be directionless,
drifting toward obscurity. So, how to encourage vision? Perhaps the first thing
is not to discourage it. Ironically, the more successful an organization is, the
more likely this is to happen. This happens when people mistakenly confuse
experience with vision. In the realm of technology, how things were done
yesterday, while a useful guide and foundation, will not get us to where we need
to go tomorrow.</p>


🧠🔭 The tension of experience and vision
2025-05-08T00:00:00+00:00
🧠 Experience</strong>: Guardrails for our journey, temperament for our actions.</p>
🔭 Vision</strong>: The view of where we are going, the source of energy and
enthusiasm for our enterprise.</p>
Experience is negative, a restraint. Vision is positive, a motivator. Both are
necessary in the right balance.</p>
Again, dysfunction ensues when we mix these up.</p>
Guardrails based purely on what we think will happen (vision) are not very
effective.</p>
Building a product only on what has worked in the past (experience) is not very
compelling.</p>


🛣️ Two approaches to life
2025-05-07T00:00:00+00:00
There are two ways we can approach life:</p>

Take responsibility and try to solve some of our own problems.</li>
Expect others to solve all of our problems.</li>
</ol>
Some examples:</p>

Owning the
integration points</a>
in our product vs. outsourcing everything.</li>
Critically evaluate what we hear vs. blind trust.</li>
Growing/raising some of our food vs. buying all of it.</li>
Learning to find fulfillment in doing vs. allowing others to fulfill us by
entertaining.</li>
Making/building some of what we need vs. buying everything.</li>
Running Arch Linux on our computers, where we control the system vs. an OS
vendor.</li>
Proactively learning about nutrition and health vs. total dependence on
mainstream medicine.</li>
Owning YOUR Platform vs. expecting someone else's platform to solve all our
needs.</li>
</ul>
In the short term, the first approach rarely makes business sense. However,
long-term, we learn that the second approach is an illusion.</p>


🏆 The entire story
2025-05-06T00:00:00+00:00
In debates about technology, people
bring up points</a>,
such as an Arch Linux update occasionally breaking and not booting; thus, Arch
Linux is bad.</p>
While the above point is true, it is also irrelevant because it is not the
entire story. What about:</p>


The hours upon hours I've saved updating and installing stuff because pacman
is so fast.</p>
</li>

The 100's of packaged tools and applications that save me from installing them
manually.</p>
</li>

The AUR, which has everything that is not already packaged, making it easy to
install even the most obscure piece of software.</p>
</li>

The benefits of using the latest and best versions of everything. After all,
if something is getting better, may as well benefit from it now vs later.</p>
</li>

The simplicity of PKGBUILD, which allows me to create and modify my own
packages easily.</p>
</li>
</ul>
I could go on, but you get the point. After running Arch for over 10 years,
let's conservatively say I've saved an hour a week by using Arch -- that is 520
hours. I've spent maybe 5 hours working through broken stuff in Arch in 10
years. That is a benefit-cost ratio (BCR) of 104. So yes, the 5 hours are real,
but they are irrelevant.</p>


👤 It's all about the people
2025-05-05T00:00:00+00:00
It is very true, we need great people in our organizations. And I have the
utmost respect for those who can motivate and lead people. I also have the
utmost respect for the capabilities of humans -- that divine spark of creativity
and the ability of the human brain to solve problems and make new connections.</p>
However, to do much in this age requires people to work together and build on
the work of the past. What one person does, another person needs to continue.
The lone accomplishments of any one individual may be impressive, but without
continuity and scale, there is likely to be very little lasting impact. Without
systems in place (YOUR Platform), the brilliant work of your people will not be
utilized to their full potential.</p>


🧠🔭 Experience and Vision
2025-04-29T00:00:00+00:00
In the realm of product development, there are two powerful forces -- experience
and vision.</p>
Experience (the past) is a guide -- a help to avoid serious blunders and
mistakes. Experience can help us avoid dead-end tangents. Experience gives us a
gut feel that is critical at times. Experience helps us be more efficient if we
can make fewer mistakes.</p>
Vision (the future) is where we are going. It is seeing things that have not
been done before. It is exploring new combinations of technology to build
compelling products. It is adopting new technology, including tools, software,
components, and workflows. It is about understanding the needs of your
customers.</p>
Without experience, we are on the path to collapse.</p>
Without vision, we are on the path to obscurity.</p>


📊 Past performance ...
2025-04-28T00:00:00+00:00

"past performance is not indicative of future results"</p>
</blockquote>
The above phrase is a disclaimer we often see associated with investments.</p>
However, this applies to more than investments ...</p>

A technology that was cutting edge in the past may not be the best option for
the future (example building web applications in PHP vs Go).</li>
People who did good work in the past usually continue to do good work, but not
always. (people can lose interest, suffer burnout, have personal problems,
etc.)</li>
Workflows used in the past are not always the best option for the future
(example, requiring patches on mail-lists will probably alienate a significant
portion of the developer population who were raised on pull-requests).</li>
As the level of integration in integrated circuits continues, the IC we used
in the past may not be the best solution compared to more integrated solutions
available in the future.</li>
</ul>
How we proceed in the future needs to be a combination of what we learned in the
past (experience) and the technology available in the future (vision). Again,
dysfunction arises when we mix these two modes up.</p>


✔️ Legacy code
2025-04-25T00:00:00+00:00
Michael Feathers defines legacy code as code without tests. He further
elaborates in
his book</a>:</p>

“Code without tests is bad code. It doesn’t matter how well written it is; it
doesn’t matter how pretty or object-oriented or how well-encapsulated it is.
With tests, we can change the behavior of our code quickly and verifiably.
Without them, we really don’t know if our code is getting better or worse.”</p>
</blockquote>
Without automated tests, any modification to the codebase is risky, as there is
no reliable way to know if a change introduces defects. This lack of feedback
forces us to rely on manual testing or institutional knowledge -- often lost
over time -- which makes maintenance difficult and error-prone. It is not about
the age of the code or who wrote it, but about whether the code can be changed
safely. If you cannot verify behavior through tests, the code is "legacy"
regardless of its age or origin.</p>


🚀 Git driven CD -- you know what got deployed
2025-04-24T00:00:00+00:00
Git driven CD (Continuous Deployment) typically means whenever you merge to
main</code> , or tag a repo, something automatically gets deployed. It can be
something as simple as a static website being updated.</p>
One of the biggest advantages to this is that you know for sure the code was
checked in and tagged. And you know exactly what code was deployed. There is no
manual process involved to mess it up.</p>
This may not seem like a big deal, but it is. As humans, we get lazy and forget
to do the simple things like manually tagging a repo -- especially on small
teams where process is not emphasized. However, CD is a way to have a process
without having a process. You do it in the name of saving time, but it also
gives you enough process that things are now also consistent.</p>


💸 Technical debt
2025-04-23T00:00:00+00:00
I like the definition of technical debt from
this article:</a></p>

Technical debt is anything that slows down your ability to update and upgrade
your work</p>
</blockquote>


🛠️ Another reason to self-host Gitea
2025-04-22T00:00:00+00:00
You own your data -- forever. You don't need to keep paying monthly fees if you
decide to switch to a different system just to access your historical data and
conversations.</p>
I've been advocating that development discussions should happen in
Git*</a>
issues/PRs/projects whenever possible, so the conversations have context. One
requirement for this to work long term is that you need access to these
conversations for the duration of a product, and perhaps beyond.</p>
There is a hosting cost, but it is minimal ($5/mo on a Linode server). There is
also some ops cost to keep things running and up to date, but in my experience,
Gitea is cake to self-host compared to things like WordPress or Email. The
benefit-to-effort ratio is huge.</p>


👷 The tension between solo and collaborative work
2025-04-21T00:00:00+00:00
As systems become more complex, we emphasize collaboration more and more. But we
must also remember that truth is still largely a solo pursuit. Truth by
committee is almost an oxymoron. The new algorithms, the keen insights, and the
disruptive inventions are usually not the work of a committee, but rather a lone
person exploring the boundaries of knowledge, careful observation, and making
new connections from what already exists. And really, invention is not anything
new, but rather a discovery of the universal truths that exist in our universe
-- the laws of nature.</p>
Increasingly, we need to be good at both -- the focus and solitude required for
deep work, and the collaboration required to build complex systems. The great
ideas will generally come from the former, but the ability to do anything useful
with them requires the latter.</p>
Dysfunction arises when we mix these two modes up.</p>


🕹️ What can we control?
2025-04-18T00:00:00+00:00
There are some things we can control:</p>

Products have a robust field update mechanism.</li>
Automated testing is in place.</li>
We have automated deployments.</li>
We have processes to quickly and confidently make and release hardware
changes.</li>
</ul>
And some things we can't control:</p>

A customer asking for a new feature we did not anticipate.</li>
Bugs showing up in the field.</li>
Security problems in a dependency.</li>
Cloud vendor infrastructure problems.</li>
Supply chain issues (have we forgotten COVID already)?</li>
</ul>
But if we do the first list, then we are prepared to handle the second. However,
we tend to focus most of our energy on the 2nd list (onerous contracts with
vendors, extensive manual testing of the "final" release, etc.) and think we are
controlling the situation, when in reality we are not -- it's just an illusion.
Really, it is change that we fear, as change is associated with a lack of
control. But it is change that gives us control.</p>


😩 Boring, error-prone, repetitive, tedious work
2025-04-17T00:00:00+00:00
Some examples:</p>

manually testing software</li>
manually constructing a bill of materials</li>
searching your email for some bit of information</li>
scrolling through logs looking for potential problems</li>
manually configuring a new server</li>
manually deploying software updates</li>
manually checking for spacing violations in a PCB layout</li>
building physical prototype circuits and trying a range of different
components</li>
organizing parts or physical materials without an easy way to identify or sort
them</li>
</ul>
If this describes what we are doing, there may be an opportunity for workflow or
automation improvements. This is the kind of work computers do best -- to free
us up for what humans do best.</p>


🧑‍💻 Will AI cause coding to disappear?
2025-04-16T00:00:00+00:00
AI is not a binary off/on thing, like many of the industry commentators like to
think. Many of us are already using AI to improve our productivity. For me,
Perplexity has largely replaced Google search.</p>
Technology has always brought productivity gains: ASM -> C -> Code generators ->
Libraries -> OSS -> Google search -> Zephyr -> etc. AI is just another step.</p>
Coding has always been the easy part. Requirements, innovation, collaboration,
architecture, integration, debugging, etc. -- these are the hard parts. There
will always be a place for skilled humans ... people with intuition, creativity,
experience, taste, vision, and style. People who see and can do what does not
already exist.</p>
If all you do is translate well-defined requirements into code, then yeah, I'd
be worried. However, there is a lot more that needs to be done that is of much
higher value, and not enough people to do it. Step up!</p>


✅ Guarantees
2025-04-15T00:00:00+00:00
Sometimes we want guarantees:</p>

Are you sure this will work?</li>
Can we lock this down to a rock-solid stable release?</li>
Is it fully tested?</li>
Will this vendor be able to provide this part forever?</li>
Do we have all the features the product will ever need?</li>
</ul>
The reality is that in complex systems, nothing is guaranteed. There will be
problems, especially during integration and new use scenarios. No supplier is
guaranteed. We can't anticipate all the use cases of a complex system.</p>
So the more important questions are:</p>

Can we start integration and testing earlier?</li>
Do we have automated testing?</li>
Can we easily update deployed systems?</li>
Can we pivot to new components as required?</li>
Do we have processes and workflows in place to quickly and confidently make
any needed changes (hardware or software)?</li>
</ul>


🔄 The 2nd key to effective process
2025-04-14T00:00:00+00:00
If the people at the top don't use the process, hardly anyone else will either.
What we do often speaks way louder than what we say.</p>
(1st key ...</a>)</p>


🛠️ Git* -- the new standard for workflow
2025-04-11T00:00:00+00:00
It started with Github, a place to host your OSS projects. Over time, several
other options emerged, and each has its place:</p>

Github</a>: excellent for OSS projects where you want to
attract the most developers.</li>
Gitlab</a>: great for a large hosted enterprise solution.</li>
Gitea</a>: a great solution for small/midsize teams
building products. Can be run on a $5/mo Linode or Digital Ocean cloud server.</li>
Forgejo</a>: is a fork of Gitea focused more on community
and security.</li>
</ul>
I'll call the above Git*</strong>. These tools set a new standard for workflow
integration</a>.</p>


🛠️ The defining characteristic of a good workflow tool
2025-04-10T00:00:00+00:00
As mentioned in the
previous post</a>,
modern workflow tools provide efficient access to information, collaboration,
and automation.</p>
The real power is when these three things come together</strong> in one environment.
The defining characteristic of a good workflow tool is ... integration</strong>.</p>
There are powerful project board tools like Jira or Trello. There are great
communication tools like Discourse, Discord, Slack, and Email. There are popular
code review tools like Gerrit. But good things happen when you bring all these
things together into one workflow tool, which I will call Git*: Github, Gitlab,
Gitea, or Forgejo. Then everything has context, and relationships between
different types of information are easy to create. Integration in a tool is a
powerful thing where the overall value is much greater than the sum of the
individual features.</p>


🛠️ The critical features of workflow tools
2025-04-09T00:00:00+00:00
Tools are what enable knowledge work in the information age. Several critical
features:</p>

Efficient access to information</li>
Collaboration</li>
Automation</li>
</ol>
We'll discuss more in the future posts ...</p>


✉️ Email filters
2025-04-08T00:00:00+00:00
Email is an amazing technology -- the only widespread decentralized
communication tool we have. But most of us are flooded with email. The
fundamental problem is when we get emails from so many different sources,
projects, and contexts flowing into one inbox. The context switching as we go
from one email to the next is very expensive.</p>
Email filters which automatically put emails into folders can help tame this
problem. Gmail already tries to do this automatically with their Primary,
Promotions, Social, and Updates. However, we can also do things like the
following:</p>

filter emails for every project/context into a separate folder. You can
trigger of senders, subject, etc.</li>
filter notifications from workflow tools (Github, Trello, etc.) into folders.
If there are separate projects, then create separate folders for each project.</li>
filter email newsletters into two folders: news-high-priority, and
news-low-priority -- the former is the ones you want to read, and the latter
you just browse occasionally.</li>
create a "personal" folder that emails outside of work get filtered into.</li>
</ul>
This is automation that makes information easier to process. Of course some
emails will still slip into your inbox, but for most of our email, everything is
grouped into a single context. This greatly improves our efficiency in
processing them.</p>
The critical thing is to use an email client that makes it easy to create
filters. Thunderbird works pretty well</a>.</p>


👥 Everybody Works and the impact on meetings
2025-04-07T00:00:00+00:00
In a recent podcast titled
Everybody Works</a>, the owners of
a 60-person company discuss the impact of full-time managers and one of them is
more meetings:</p>

And the primary problem with management is when it’s full time and it’s only
that it can’t do anything by itself. So the way 40 hours are manifested is
through other people. It’s, hey, can you join this meeting? Hey, can we do a
one-on-one? Hey, can we do some planning? Hey, can we do? Every bit of it is
something that pulls someone who’s actually working on things away from that
work and into the managerial orb. And I think we just finally came to the
realization we did not need that. We do not need that at our scale, that we
can supplement the work that we’re doing with a handful of managerial
practices or protocols, things that don’t even require human to run.</p>
</blockquote>
The following article also discusses the subject:</p>
We once more have no full-time managers at 37signals</a></p>


👥 Efficient meetings
2025-04-04T00:00:00+00:00
There are a lot of good ideas for effective meetings:</p>

Whose meeting is this? A simple checklist</a></li>
Who has time for meetings?</a></li>
</ul>
There are good reasons to have a meeting (hanging out, brainstorming, setting
priorities, making decisions), but in this age sharing information is probably
not one of them. People should have access to information before the meeting.
The goal of the meeting should also be known ahead of time. It is like planning
your day the night before — your subconscious has time to work on it and you are
prepared. Prepared people are efficient.</p>


👥 Meetings and human relationships
2025-04-03T00:00:00+00:00
Improving human relationships is sometimes used as a justification for meetings,
and face-to-face meetings can certainly improve human relationships.</p>
But more generally, we can say that positive interactions improve human
relationships. And the more positive interactions we have, the stronger our
relationships will be.</p>
Modern workflow tools provide a tremendous opportunity for human interactions at
a scale that is impossible to accomplish with meetings. They provide
transparency. The stream of activity is visible to all, and anyone can
participate (or ignore) at any time. There are so many more opportunities for
positive interaction than a weekly, or even daily status meeting.</p>
However, it is necessary to have a culture that encourages and promotes
transparency and frequent, positive interactions. It starts with the example of
those at the top. Do you provide positive feedback, give helpful suggestions,
and ask thoughtful questions in the course of work? Do you follow the
development activity and use the communication features of the workflow tools
such as Issue/PRs/project boards comments?</p>


👥 Why meetings?
2025-04-02T00:00:00+00:00
There are many reasons to call meetings:</p>

Accountability</li>
Knowledge transfer</li>
Transparency</li>
Status</li>
Review</li>
Oversight</li>
Improve/maintain relationships</li>
Brainstorming</li>
</ol>
However, in this age of excellent collaboration tools (Git workflows, project
boards, etc.), meetings are an inefficient way to handle the first 6 items on
the above list. If our workflow does not provide complete transparency into what
everyone is doing, then there may be opportunities to improve our workflow.</p>


⚖️ Horizontal vs Vertical Scaling
2025-04-01T00:00:00+00:00
In the context of product development:</p>

Horizontal Scaling</strong>: Selling more units, getting more customers.</li>
Vertical Scaling</strong>: Add value to already sold/deployed units.</li>
</ul>
Bigger vs. Better -- both are valid ways to scale and produce value. In this age
of powerful connected devices, vertical scaling is becoming increasingly
important. And that is where YOUR Platform matters.</p>


👌 Struggling to scale your IoT system? Simplify your data.
2025-03-31T00:00:00+00:00
I recently published a new blog article about IoT data. Have you struggled with
adapting your IoT system to new applications and requirements? Is handling
configuration updates at both the cloud and edge a challenge? Is synchronizing
required data between cloud and edge instances a challenge? This article
describes a simple way to model data that eliminates many of these difficulties.</p>
Read here ...</a></p>


🔄 The secret to process
2025-03-28T00:00:00+00:00
Process/workflows/tools should be good enough that they help the people doing
the work. They are not just an administrative tool. Some examples:</p>

Git helps me manage my own work, so I use it even on personal projects where
collaboration is not needed.</li>
Gitea issues are so fast and easy to use that when working on hard problems, I
automatically create an issue to log progress. It is easier and faster than
keeping notes anywhere else.</li>
Gitea releases are so quick and easy to use that I use them even for internal
releases there rather than composing an email.</li>
Jitsi Meet is easier than a phone call, so we use it for most calls and have
the benefit of screen sharing, video, multiple participants, etc.</li>
Code formatting tools help me as a developer, so I gladly integrate them into
my editor.</li>
Part numbers are so easy to pull and use that they help me manage parts and
assemblies for my prototype builds, so I pull them early in the design process
and integrate them in CAD libraries.</li>
Tests are easy to create and run locally, so I create them as part of my
development process.</li>
Documentation is integrated with the design files and easy to create, edit,
and track. Thus, documentation is an integral part of the development process
vs. an afterthought.</li>
End to end tests are easy to run by anyone, so when there are issues to debug,
they are tool that is regularly used by developers.</li>
It is easy to purchase things needed to do my job and improve the company as
everyone is issued a credit card, and expense reports are generated
automatically. There is no motivation to bypass the process.</li>
</ul>
Tools/workflows/process should help the people doing the work go faster, not
slow them down. If this is the case, you won't need to force them on people. If
this is not the case, the process needs work.</p>


⚙️ Fix systems, not people
2025-03-27T00:00:00+00:00
We can rarely "fix" other people. Short of a divine conversion experience or a
life-changing event, people don't fundamentally change, no matter how much we
plead, beg, threaten, cajole, encourage, etc. Certainly, some are more gifted
with the ability to motivate, nurture, and mentor others. But, even this has
limits. So we are left with two options:</p>

Get the right people on the bus.</a></li>
Work on the systems and processes that help all of us work and communicate
more effectively.</li>
</ol>


📈 Scaling
2025-03-26T00:00:00+00:00
We often think of scaling as growing a company's size or the number of units
sold/deployed/etc. That is one way to provide value to the world.</p>
However, there are other ways to scale:</p>

improving the experience</li>
improving reliability/security</li>
new markets</li>
deploying updates to existing systems</li>
better customer support</li>
giving users more control/flexibility/customization</li>
higher profit margins</li>
</ul>
Think carefully about this as scaling in one dimension often limits how far you
can scale in another.</p>


🌅 Simple stuff is boring
2025-03-25T00:00:00+00:00
That may be. But doing the simple stuff well is also rewarding. And which is
better: exciting or rewarding?</p>
To do the simple stuff well feels good. It feels right. That is because it is
right. And what gets the simple stuff done? YOUR Platform.</p>


👌 The simple/basic things are the most important to get right
2025-03-24T00:00:00+00:00
It pretty much any endeavor, consistently doing the simple things are what make
the difference.</p>

Doing simple exercises consistently.</li>
Eating whole foods.</li>
Not overeating.</li>
Taking time to share and communicate.</li>
Showing up.</li>
Being on time.</li>
Meeting commitments.</li>
Not overcomitting.</li>
Using a headset when communicating.</li>
Fixing bugs.</li>
Keeping code clean.</li>
Following the process.</li>
Having a process.</li>
Working transparently.</li>
Making things easier for others.</li>
</ul>
Simple things are foundational -- if you don't get these right, the rest
crumbles.</p>


✅ The problem with software
2025-03-21T00:00:00+00:00
I'm currently reading
The Checklist Manifesto </a>
by Atul Gawande. Gawande is a surgeon who wants to find a way to reduce the
mistakes and their catastrophic consequences during surgery. So he investigates
other industries such as air travel and the construction of large buildings.
What he finds is the checklist is at the center of their processes. (more on
this later ...)</p>
With software, our failures (with some exceptions) are usually not as
catastrophic as people dying in surgery, airplanes falling out of the sky, and
large buildings collapsing. Another difference with software is that is easy and
cheap to fix by deploying an update (hence the prefix "soft"). People,
airplanes, and buildings are not this way.</p>
Thus we can get sloppy. Motivation to "get it right" can wane as pressure to
ship increases and we can "always fix it later." The problem is that this
approach encourages accumulating technical debt. Technical debt will just as
surely crush the soul of a programmer and eventually the company, just not in
dramatic news-catching ways.</p>
Building a sustainable technology company requires an extreme level of
discipline because there is no external force requiring you to do it right. For
many technology industries, there is not any regulatory agency watching over
them -- it is entirely up to YOU, and that is where YOUR Platform comes in.</p>


🌻 Downstream, things should be better
2025-03-20T00:00:00+00:00
For everyone downstream of us who uses or is influenced by what we create or do,
things should be simpler/better/easier -- because we did something. This is the
formula for creating wealth.</p>
The alternative is that we produce no value for others. It would be easier for
someone to do whatever we're doing themselves, or in extreme cases what we are
doing should not be done at all. This is the formula for debt.</p>
Wealth can include more than money -- good will, positive energy, intellectual
property, spiritual well-being, relationships, community, health, etc.</p>
Whenever debt creation is greater than wealth creation for too long, the
family/organization/company/country we're a part of collapses.</p>
In whatever sphere we're a part of, do we create wealth or debt.</p>


🪙 Complexity and Simplicity -- two sides of the same coin
2025-03-19T00:00:00+00:00
To get to simple for someone, it often has to get more complex for someone else.</p>
An elegant, easy to use UI was a lot of work for someone to design.</p>
A powerful, modern programming language that makes it easy for software
developers was no doubt very complex to implement.</p>
Modern CPUs that allow us to do so much are very complex to design and
manufacture.</p>
In technology, anything that appears simple/nice/easy likely has something very
complex behind the scenes.</p>
Again, this makes business sense if many people can benefit from the simplicity
for every person that deals with the complexity.</p>
What does not work is if we push complexity onto our users/customers in an
effort to keep things simple for ourselves.</p>


⬆️ The business case for improvement
2025-03-18T00:00:00+00:00
As discussed previously, improvement is sometimes hard -- really hard. So how do
we justify it?</p>
The payback comes when we improve something once, and then something else gets
lots easier many times.</p>
An example might be automating a release process. If it takes 1 hour to manually
do a release, and it takes 6 hours to automate it, then in 6 releases we break
even and releases after that is gain.</p>
We want the balance to tip heavily toward the gains from repeatedly using the
improvement, not the effort spent implementing improvements.</p>
</p>
For this to work out, we need a way to leverage improvements in the future. We
need tools, workflows, automation, and processes to capture improvements -- YOUR
Platform.</p>


⬆️ The path of improvement
2025-03-17T00:00:00+00:00
Improvement is not always easy, actually it is usually hard -- really hard, and
sometimes a complex path. But in the end, something likely
got simpler for someone</a>.</p>
Getting to complex is easy and the natural state of things (chaos).</p>
The path and the resulting state are two different things.</p>
We can do the hard/complex things and get to a simple/nice/sustainable state.</p>
Or we can only do the the easy things which results in a
complex/messy/unsustainable state.</p>
Take your pick.</p>
</p>


⬆️ What is improvement?
2025-03-14T00:00:00+00:00
Improvement often means something got simpler for someone:</p>

We refactored the code and it is now easier to read.</li>
The build system is more reliable.</li>
The automated tests are better, so less manual testing is needed.</li>
The documentation is better, so it is easier to understand something.</li>
The circuit is more reliable, so support has less field problems to deal with.</li>
A product has more needed features, so it is easier to sell.</li>
</ul>
If things are not getting simpler for someone, we need to question -- are they
improving?</p>


👷 Make less work for others, part 2
2025-03-13T00:00:00+00:00
A few examples of how to make "less work" for others:</p>

try not to introduce bugs or break stuff</li>
communicate in workflows like Tickets/PRs where there is context and messages
can be easily processed</li>
document your work -- including "why"</li>
focus on the business priorities</li>
be self-directed -- see what needs done and do it</li>
work in public
(commit early and often</a>
and
release your work often</a>).</li>
document how work was tested</li>
better yet, provide examples/scripts/tests of how to use/test your work</li>
</ul>
It should be obvious:</p>

what we are doing</li>
why we are doing it</li>
how we did it</li>
how we tested/verified it</li>
and how to use it</li>
</ol>


👷 Do we make more work for others or less?
2025-03-12T00:00:00+00:00
This question really gets to the heart of our value to others. In everything we
do, we need to think -- am I delivering this work or information in a way that
is easy for others to understand and use? Or do they need to do a bunch of
unneeded/extra work?</p>
Organizations composed of people who make less work for others scale.</p>
Organizations composed of people who make more work for others collapse.</p>


🔄 The business model for updates
2025-03-11T00:00:00+00:00
In the old'n days, a project was shipped and done, and we moved on to the next
one.</p>
However, with modern connected systems, we can
deliver a lot of value</a>
after the initial release. Most of our engineering effort may be spent after the
initial release maintaining and improving a product. How do we fund this? Some
ideas:</p>

Build the cost into the product up front (many consumer products use this
model)</li>
Pay for maintenance with new sales (be careful this is not a pyramid scheme)</li>
Monthly/yearly subscription fees (might include a cloud service)</li>
Fees for add-on features (high-end test equipment like oscilloscopes often use
this model)</li>
Pay for every update (hard to get customer buy-in)</li>
</ul>
The biggest mistake is not to plan for maintenance costs, or to surprise your
customers with them. Be up front with the costs, but be more up front by
communicating the value that you are continuously delivering.</p>


🔄 What is the best software release frequency?
2025-03-10T00:00:00+00:00
Different projects/products have different needs, but generally the trend in our
industry is to release new software more often. There are several reasons for
this:</p>

Modern processors (even embedded microcontrollers) are very powerful and
expandable, so it is possible to add additional features after the initial
release.</li>
Many devices are now connected, so updates can be deployed "over the air"
automatically, rather than executing some manual operation.</li>
Continuous Integration (CI) and other workflows are improving the quality of
software such that we are more confident the software is always tested and
working. Long manual testing/QA periods are no longer the best practice.</li>
People are used to software updating on their phones/computers, so it is
generally more accepted.</li>
Connected systems
need more updates</a>.</li>
Connected systems bring security concerns which require updates.</li>
</ul>
Perhaps the main reason the release frequency is increasing is more cultural
than anything. In previous times, software updates were perceived as "breaking
things." Especially in embedded systems, the mantra was: test it extensively,
get it right, and lock things down. Updates are risky. However, the world has
changed.</p>
In the end, a software release means more value is being delivered to your
users. And if our goal is to deliver value, then that includes releasing new
software more often.</p>


🚀 How can Arch Linux get by without releases?
2025-03-07T00:00:00+00:00
Arch Linux does not have discrete releases -- it is a rolling distribution where
individual packages are continuously updated. Most people still raise an eyebrow
at the mention of Arch Linux -- how can that possibly work and be stable?</p>
Having run Arch for many years, I can affirm
it works very well</a>. It is fast, efficient, and lets
me run the latest software of everything with minimal fuss.</p>
Today, a Linux distribution release is an artificial notion. A distribution's
main job is to deliver a collection of software to a computer. Each application
has its own release cadence. Imposing a distribution release cadence on top of
the package release cycle does not make sense.</p>
But the distribution's job is ensuring everything is stable and works together?
Really? Does a distribution really test every one of the 1000's of packages
before every release?</p>
If there is a serious problem in an application, it will be fixed soon in the
main development branch. Requiring these fixes to be back-ported to an old
release in a distribution is a lot of work. In some cases, it is nearly
impossible to backport a change if things have changed significantly in the
codebase.</p>
A Linux distribution delivers the most value when each software package can be
deployed to its users as soon as it is ready. Arch does this and then gets out
of the way.</p>


⚖️ Truth is discovered, not invented
2025-03-06T00:00:00+00:00
I once advised a young man -- If you have some down time on a plane, in a hotel,
etc., you might consider writing an essay or two about some of the issues facing
you personally, or your organization. This is one of the best ways to think and
discover truths. I'm often surprised and amazed what comes out.</p>
Meetings would be much more effective if writing were part of the preparation
(Amazon does this</a>).
But, alas, most think we can make progress by more discussion, meetings, emails,
etc.</p>
Truth is discovered, not invented. Truth cannot be forced. It is only when we
approach the creative process with this humility that we can make progress.
Designs devoid of truth are unwieldy, unreliable, hard to maintain, etc.</p>
Modern, powerful tools like AI, CAD, computer languages, etc, may give us the
illusion that great inventions start with us, but truly great
designs/algorithms/processes are discovered, a simple reflection of truths that
have existed for eternity. In the process of simplifying our designs, we often
come closer to truth.</p>
I wish I had understood the power of writing and the essence of truth earlier in
my career.</p>


🔗 Connected systems are different
2025-03-05T00:00:00+00:00
In distributed computer systems, change is the constant.</p>
Because a system can be networked, its capabilities are endless. The ability to
connect to new and different systems means new systems (and value) can always be
created, and things will change</em></strong>.</p>
If you don't, somebody else will.</p>


🛡️ How to get to secure and reliable
2025-03-04T00:00:00+00:00
The best way to get to secure and reliable software is to fix the core issues.</p>
If your programming language is not reasonably secure and reliable, adding more
external tools to check and track down problems is helpful, but is likely not
the most direct approach.</p>
Likewise with architecture -- getting the core architecture right is important.
Adding external "watchers" and "fixers" to compensate for an architectural
problem only complicates things, and complexity is the #1 enemy to both security
and reliability.</p>


🏭 Semi-structured part numbers
2025-02-28T00:00:00+00:00
Some time back I recorded a video on Semi-structured part numbers.</p>
https://youtu.be/tlsl01uxUfE</p>
Be more efficient.</p>
Make less mistakes.</p>
We tend to ignore simple things like part numbering systems.</p>
But doing the simple things right is what frees us up to work on the hard
things.</p>


👤 Who is our "customer"?
2025-02-27T00:00:00+00:00
Your "customer" is someone who derives value and uses what you produce.
Customers are both internal and external to your organization.</p>
While it is good to focus on the flow of primary business value to the
end-customer (that is what pays the bills), we can also think of everyone in the
organization as being our customer. How can we provide value to everyone?</p>


⚠️ Toward a Theory of Medical Fallibility
2025-02-26T00:00:00+00:00
I recently skimmed an essay from 1975 titled
Toward a Theory of Medical Fallibility</a>.
A few quotes from the article:</p>


Therefore, where there is scientific activity, there is a partial ignorance
-- the ignorance that exists as a precondition for scientific progress. And
since ignorance is a precondition of progress, where there is the
possibility of progress there is the possibility of error. This ignorance of
what is not yet known is the permanent state of all science and a source of
error even when all the internal norms of science have been fully respected.</li>
Error may indeed arise from the present state of scientific ignorance or
from willfulness or negligence. But it may also arise precisely from this
third factor which we have called necessary fallibility in respect to
particulars.</li>
The encouragement of this inflated belief in the competence of the physician
is of course reinforced by the practice of not keeping systematic and
accessible records of medical error. Yet everyone knows that this is a false
confidence.</li>
Indeed, the only profession of which we know which fully and publicly
documents predictive successes and failures is that of horse-racing
correspondents in Great Britain.</li>
The first reaction of physicians to the invitation to dispense with the
masks of infallibility is likely to be a humane alarm at the insecurity
which a frank acceptance of medical fallibility might engender in the
patient. But we wonder whether the present situation, in which the
expectations of patients are so very often disappointed during medical
treatment, is not a worse source of insecurity.</li>
</ul>
</blockquote>
As computer/software systems get more complex, are there parallels?</p>


🛤️ Two paths with experience
2025-02-24T00:00:00+00:00
As we gain experience, we can take two paths:</p>

learn from mistakes and all the things that went wrong, and apply this
knowledge to do better in the future.</li>
assume because we've done something before the next time we do something
similar it will "just work."</li>
</ol>
If I've learned anything working in technology, things rarely "just work."
Nothing is easy.</p>
Some can accomplish #1 because they have a great memory. For the rest of us, we
need some process to capture what we have learned and a mechanism for applying
this in the future.</p>
A checklist is a great place to start.</p>


🛤️ Two paths
2025-02-21T00:00:00+00:00
There are two paths we can follow as a team or an organization:</p>

what we do makes it easier</strong> for everyone else to do their job.</li>
what we do makes it harder</strong> for everyone else to do their job.</li>
</ol>
#1 is the formula for scaling</strong>. #2 is the formula for stagnation and
technical debt</strong>.</p>
And which path a team or organization takes depends on culture</strong>. What are the
expectations</strong>? Are they documented</strong>? And most importantly, what
example</strong> do we set?</p>


👥 Your team as the customer
2025-02-20T00:00:00+00:00
Previously we
defined our "customer"</a>
as someone who derives value from what we do. While it is useful to think of how
the primary business value is flowing, hopefully we are providing value to our
team as well as people outside our team. Some ideas:</p>

Are Git commit messages helpful and easy to understand?</li>
Are commits and PRs small enough to comprehend?</li>
Is our work transparent?</li>
Do we deliver what we said we would?</li>
Is the code we write easy to follow and maintain?</li>
Do we document what we are doing and why?</li>
Do we share tests scripts and other automation that helps people use what we
created?</li>
</ul>


🚀 Why are releases important?
2025-02-19T00:00:00+00:00
Since many products/systems run software these days, they can be improved after
the initial release. Most software projects have "releases" when a new version
of the software is deployed or made available.</p>
Why are releases so important? Some reasons:</p>


Communicates to customers an update is available and what changed.</p>
</li>

Provides a cadence for doing work.</p>
</li>

Encourages development teams to "ship". Shipping is motivating.</p>
</li>

Delivers more value to the customer.</p>
</li>

Helps us isolate problems if we know what is running and what changed.</p>
</li>
</ul>


📄 Who is a changelog for?
2025-02-18T00:00:00+00:00
Most projects have releases, and with each release, we describe what is
new/changed/fixed in a changelog file.
keep a changelog</a>
has a lot of good ideas.</p>
Who is a changelog for? It is for our customers -- those who use whatever we
produce. The changelog communicates to our customers what is new and valuable
for them</em></strong> in each release.</p>
Once we realize who the changelog is for, then this can influence how we write
changelog entries. A few ideas:</p>


keep it is as succinct as possible (don't waste people's time)</p>
</li>

don't use technical jargon that our customers don't understand</p>
</li>

be humble -- if there are bugs that got fixed, acknowledge they were bugs.
Don't try to embellish or gloss over bug fixes by calling them features, etc.
People can see right through this.</p>
</li>

explain changes from the lens of benefits to the customer/user, not from the
developer's perspective</p>
</li>
</ul>


👥 Who is the customer in organizational hierarchy?
2025-02-17T00:00:00+00:00
In a business organization hierarchy, where E (engineer) reports to M (manager),
who is the customer in the E - M relationship?</p>
Our first inclination may be to think M is the customer -- M is the boss, E
should make M happy. But, lets think about this a bit more. Who does E create
business value for? Is E writing code that benefits M, or producing a printed
circuit board that M will directly use?</p>
Hmm, does not seem so. E's customer is probably purchasing and manufacturing (in
the case of a PCB). M does not directly produce tangible outputs. M's job is to
help E get the job done. Therefore, it would follow that E is M's customer.
Certainly, M has authority and needs to make decisions at times. But, in the
end, the job must get done. Business value must be produced. In product
development, authority or decision-making in themselves do not produce business
value. The more people we have under us, the more people we serve.</p>
Do we exist to serve, or be served? The higher up we go, the more important it
is to understand servant-leadership</a>.</p>


👤 Confusion about when we are the customer
2025-02-06T00:00:00+00:00
In our modern, entitled, self-serving consumer culture, we tend to be confused
about when we are the customer or not. If we work for pay, our job is to do
something to benefit someone else -- our customer. We are not the customer in
this scenario, someone else is. We are supposed to make someone else's life
easier or better. In our job role, are we serving our customer, or do we expect
to be served? This mindset probably does more than anything to determine if we
are useful.</p>


🕊️ Simple -- from whose perspective?
2025-02-05T00:00:00+00:00
"Just keep it simple" -- we sometimes hear this. But from whose perspective?</p>

Legal/IP</li>
Management</li>
IT</li>
Financial</li>
R&D</li>
Security</li>
Sustaining engineering</li>
Purchasing</li>
Manufacturing</li>
Quality</li>
Sales/Marketing</li>
Customer support</li>
End User/Customer</li>
</ul>
How do we prioritize simplicity for the above list?</p>
Hint, the one at the bottom is probably the most important. Do you constantly
keep your customers in mind as you make decisions? "Customer" may include
multiple from the above list.</p>


🕊️ Simple to maintain vs simple to implement
2025-02-04T00:00:00+00:00
Recently</a>
we talked about the tension between simple to use vs. simple to implement.</p>
Maintenance is similar. If you want something that is easy to maintain, you need
to put in the work upfront.</p>
The Elm programming is a classic example of this. The initial coding takes
longer than generating some Javascript from AI prompt. You need to figure out
how to best represent the problem with Types. But after you get it right, it is
so easy to maintain</a>.</p>


🤝 Integrate early
2025-02-03T00:00:00+00:00
Integration is where we bring together separate parts of a system.</p>
When things come together, problems appear: performance problems, stability
problems, electromagnetic noise issues, mechanical interference,
incompatibilities, manufacturing issues, usability issues, aesthetic issues,
etc.</p>
It is much better to discover these problems early, and the only way to do this
is to integrate everything early in the project. And the only way this happens
is if it is prioritized by management and supported by process.</p>
Iterate Hardware Like Software</a></p>


🕊️ Simple to use vs simple to implement
2025-01-30T00:00:00+00:00
To make something simple to use, often it is not simple to implement. For
example, in the SimpleIoT UI, the following line of code is all that is required
to add a text input field to a node. No handlers need to be attached, etc.</p>
textInput Point.typeDescription "Description" ""</code></p>
However, the
code that implements the textInput</a>
function is a little more involved.</p>
Generally, the better you want the user experience to be (even if your users are
developers using your API or library), the more work you'll need to put into
things behind the scenes. There is a fundamental tension between "Simple to use"
vs. "Simple to implement" -- you can have one or the other -- usually not both.
We all use many "components" in product development, but an excessive
off-the-shelf mental, especially at the top hardware and software levels of your
product, won't get you very far.</p>
Simple != Easy</p>


🤏 Handling the small details
2025-01-29T00:00:00+00:00
We've discussed</a> the
necessity to fulfill your small commitments as a stepping stone for fulfilling
larger commitments. We can expand this thinking -- handling the small details
well enables you to focus on bigger things.</p>
In product development, we have many details to be concerned about:</p>

consistent code formatting</li>
testing new code</li>
testing the system</li>
ensuring there are no errors or DRC violations on a PCB layout</li>
generating a documentation package (including BOM) for each PCB release</li>
generating documentation for each project release</li>
notifying relevant people when things change</li>
reviewing code changes</li>
organizing parts in the lab and in production</li>
deploying software updates</li>
publishing various communications</li>
ensuring everyone on the team has access to needed information</li>
</ul>
We can do all of the above manually, and it is not a big deal to do once.
However, these details soon start to pile up until pretty soon that can be all a
person does. Fortunately, there is a better way -- all of the above can be
handled to some degree with intelligent workflow, automation, and tooling.</p>
We have so many good options for automation available today -- scripting
languages, APIs, CI/CD systems, flexible DRC engines in CAD tools, Git based
workflows, commercial tools and services, etc. Implementing and using these
tools is not always easy and requires an investment in time and learning.
However, this is much easier than manually managing tons of details forever and
frees you to focus on more important things.</p>


🛠️ Selecting tools
2025-01-28T00:00:00+00:00
There are many considerations when selecting a tool:</p>

will it do the job?</li>
what does it cost?</li>
can I get support?</li>
is there a community around it?</li>
will it be available and maintained long-term?</li>
what does the future look like for new features?</li>
is it accessible/available to others?</strong></li>
</ul>
The last point is often overlooked -- "That is not my problem." However, if you
are building a platform, this should be a primary consideration as tool
accessibility is important for collaboration and maintenance. This is the
difference between a sustainable business and a few talented individuals.</p>


🛠️ Two classes of tools
2025-01-27T00:00:00+00:00
There are two classes of tools: Personal</strong> tools and Common</strong> tools.</p>
A personal tool might be your editor -- each developer has his own preferences
and what editor one person uses does not affect another.</p>
However, it is important to have a common tool for formatting code. So you
decide on a formatting tool and everyone configures their editor to use that.</p>
Some more examples:</p>
Personal</th> Common</th></tr></thead>

Computer OS</td> File/doc formats</td></tr>
Phone</td> Git PR/Issue system</td></tr>
Notes system</td> Build systems</td></tr>
Email client</td> CI/CD</td></tr>
Browser</td> CAD Tools</td></tr>
Git client/setup</td> Compilers</td></tr>
</tbody></table>
Sometimes sticking with the common stuff is slightly less convenient than doing
it "my way." However, you need to decide if you want to be part of a sustainable
business or just a few talented individuals.</p>


📝 Documentation is part of the process, not an afterthought
2025-01-24T00:00:00+00:00
One of the best productivity hacks ever is to change your thinking about
documentation. Most documentation is done after the fact, which is a rather
distasteful task at this phase.</p>
If we learn to use documentation as part of our thinking/design/implementation
process, it can be enjoyable, and help us do a better job in the first place. A
few ideas:</p>

Write code comments before we write the code, describe what we are going to
do, etc.</li>
In a schematic, document constraints and design criteria as a note in the
schematic before drawing a new section of the schematic. As you go, think in
your notes.</li>
Write the user documentation before implementing a feature -- this may help us
think more from the user perspective upfront.</li>
If we are stuck on a problem, write documentation about it and check it into
Git. This allows us to still
ship something</a>.</li>
As we are doing something, create careful notes about all the resources we
need to do the job, summarize important parts with screenshots, etc. It will
be much easier next time.</li>
Create a checklist before you do something.</li>
When someone asks you about something, instead of writing an answer in an
email, create a document and
point them do it</a>.</li>
</ul>
A small shift in our habits can have a big impact!</p>


✨ Demos vs. Products
2025-01-23T00:00:00+00:00
I'm a huge fan</a> of demos --
so is Khem</a>. They are a very
important part of product development. They are a way to ship early and get
critical feedback. Demos during the course of real product development are
essential.</p>
There are some challenges or risks with demos. One is that when managers see a
demo, they sometimes assume the product is nearly done when this is rarely the
case.</p>
Other demos are for the purpose of evaluating some technology with little
concern as to how it is implemented. The world has exploded with "maker" stuff
(Arduinos, Raspberry PIs, etc). These are very useful for one-off lab
experiments, but experience has taught me that it is generally a bad idea to
ship products (at least industrial products) based on "maker" technology.</p>
Products have considerations like cost, manufacturability, maintainability,
reliability, and availability over a lifecycle that can be 5, 10, or more years
for industrial products. We can ignore these concerns for the demo, but they
can't be ignored for the product.</p>


🧩 The three legs of YOUR Platform
2025-01-22T00:00:00+00:00
Automation</strong> -- start with a checklist.</p>
Collaboration</strong> -- #1 rule is don't view email as a permanent place to store
information.</p>
Process</strong> -- the habits of your organization.</p>


🔍 The importance of small Git commits/pull-requests
2025-01-21T00:00:00+00:00
Why are small Git commits/pull-requests important?</p>

small changes are easier to review. Large PRs are painful to review, thus
they tend to be rubber-stamped and not reviewed, or rejected in OSS projects.</li>
you get to
ship more often</a>.</li>
other people are motivated by seeing work get done (positive peer pressure).</li>
if we are heading down the wrong path, this can be corrected earlier before
wasting a lot of time.</li>
our work is more transparent, and transparency is a powerful tool.</li>
</ol>
Even if we are working on a personal project that no one else sees, small
commits helps us organize our thoughts, reflect on what we've done, and receive
the motivation boost that comes from incrementally shipping, even if only to
ourself.</p>


🎆 100 daily emails
2025-01-20T00:00:00+00:00
Today marks the 100th "daily" email I've sent out. (I don't actually send on
weekends, or days when I'm out of the office). Some observations:</p>

Daily is easier than weekly -- I was skeptical when others said this, but I
think it is true.</li>
If I skip a few days, it can be hard to get going again. There is something
about the process of doing something daily that energizes the creative
process and keeps it going.</li>
Ideas are everywhere -- when I write consistently, the mind starts to see
ideas everywhere. There are 32 article ideas in my backlog -- Ideas come in
much faster than they go out.</li>
I spend 10-20 minutes a day writing these posts. Often writing one will spur
the idea for the next, and I'll write that one too. I think it could be even
faster.</li>
Tooling is important. I have scripts that automate creating a post and
publishing it. Kit is configured to pick up the post from my site's RSS feed
and then mail it out. There is nothing slow or painful about the process that
pushes me away from doing it.</li>
</ol>
If you have feedback, please let me know! I'd love to know what topics are
interesting to you or not.</p>


🔁 The importance of Git Commit/Push often
2025-01-15T00:00:00+00:00
There are many forms of "shipping" your work as we
recently discussed</a></p>
Shipping does not necessarily mean a product release, but rather releasing your
work external to you. It can be a Git push, running tests in CI, a prototype, a
document that others can read -- basically something of lasting value</em></strong> that
others can interact with. (Email does not count).</p>
As a developer, the simplest way to ship your work is a Git push. It does not
have to be perfect. In most cases, it does not even have to be done. That is the
beauty of software, it is soft -- it can be improved/finished later. A Git
commit does not even have to be code -- it can be documentation, specs, etc. A
Git commit/push is the simplest and most effective way to demonstrate you are
providing value to the project or organization. Yet how many of us are consumed
with ephemeral activities that in the end provide no lasting value?</p>


💭 Thinking, Designing, Developing, Engineering ...
2025-01-13T00:00:00+00:00
If we are thinking without writing, are we really thinking?</p>
If we are designing without sketching/drawing, are we really designing?</p>
If we are developing software without iterations/testing, are we really
developing?</p>
If we are engineering without building prototypes, are we really engineering?</p>
Something happens when our work is implemented in an external form that is
staring back at us. It speaks to us, exposes the flaws, and shows us where
improvements are possible. When our work is good, we can recognize it and find
motivation for the next phase.</p>
Yes, it is possible to do some things without shipping, but when we ship our
work in the form of writing, drawing, prototyping, and iterating, a world opens
up that we could never see before. New connections are made.</p>
Optimize your habits/processes for shipping something -- your work will get
better.</p>


🔧 envsetup.sh -- a very useful automation technique
2025-01-10T00:00:00+00:00
During a Android development stint, I learned about the
envsetup.sh</code></a>
concept. This is a script that contains numerous functions get populated in a
shell envrionment when the script is sourced. There two ways to source the file
-- you can type:</p>
source envsetup.sh</code></p>
Or use a '.' for a shortcut for source and type:</p>
. envsetup.sh</code></p>
Most of my projects and Git repos now contain a envsetup.sh</code> file (examples:
Simple IoT</a>,
Yoe</a>,
TMPDIR Podcast</a>).
Typically all of the functions in the file have a common prefix. This gives them
a namespace and allows you to see all the functions quickly in your terminal by
typing <prefix><tab><tab></code>.</p>
Why shell functions instead of something like Make, Earthly, Nix, Task, Mage,
Pants, Bazel, Python ...? Shell script is about the simplest abstraction on top
of typing commands. Thus, if mostly what you need to do is execute a sequence of
terminal commands, then shell functions are very efficient at doing this. A
shell function is often the best documentation for how to do something -- it is
almost like a powerful checklist. There are no dependencies -- every Linux and
MacOS system comes with a shell built-in. envsetup.sh</code> functions can often be
used directly in CI/CD and other build systems (like Yocto/Bitbake tasks).</p>
There are some common objections ...</p>
Shell is not cross-platform to Windows. That is unfortunate, but I rarely use
Windows, and most developers I interact with use either Linux or MacOS systems.
However, this may change with Zephyr as MCU developers traditionally have used
Windows.</p>
Why not use something more powerful like Make? With modern programming languages
and build systems like Go, Rust, and Yocto, the is no need for yet another build
system. The complex issues like dependencies are already handled. Thus, all you
need to do is remember how to kick it off.</p>
Why not use Python -- everyone uses Python for everything -- it is the most
popular programming language ever! Python is great if you need to do something
more complex, but it is a more complex programming abstraction. And mostly what
you need to do is record a sequence of commands, any abstraction costs you more
than benefits.</p>
Why not a shell script that you execute instead of sourcing a script that
populates functions? I'm not sure -- the sourcing just seems more ergonomic to
me and simpler. If you need to run a script to execute a command, then you need
to process command line arguments, etc. You are now programming, and programming
is what we are trying to avoid in envsetup.sh.</code> Also, the functions are not as
discoverable as when they are populated in your environment.</p>
Shell scripting is not without its tradeoffs -- when you do need to program
something in shell, it can be very painful and errors can be subtle and hard to
catch. However, AI tools are very effective at writing shell script, so this can
be a big help. Linters and formatters are also very good at catching problems
early and making your scripts more readable (more on this later).</p>
In summary, think of your envsetup.sh</code> file primarily as documentation. Create
one in every project and instead of documentation how to do something, create a
shell function. Your future self will thank you.</p>


📧 Effective email is either a pointer or notification
2025-01-08T00:00:00+00:00
Email is the only decentralized communication platform that has widespread
usage. And this is more amazing considering it was invented in 1971. Email is an
amazing technology and has stood the test of time.</p>
However, there are challenges with email:</p>

It can be difficult to find information in emails at a later date.</li>
Visibility is limited to the recipes of the email.</li>
Discussions often don't happen in the context of the work being done.</li>
Information is not organized very well.</li>
</ol>
I follow a few rules regarding email:</p>

Whenever possible, don't send information in email, but rather point people to
information in a more permanent location (Git repo, Wiki, Discourse forum,
etc.). If the information does not exist, create it, and then send the email.</li>
Keep project-related discussions in Git pull requests, issues, forums, etc.</li>
For transient discussions, use chat tools like Discourse chat, Signal,
Discord, Slack, etc. This is more fun, and keeps email from clogging up with
chatty stuff. This keeps the information accessible to everyone involved. It
is also way more efficient as discussions and information are stored in the
context of the work.</li>
</ul>
It is still very useful to receive email notifications when a piece of
information or a discussion is updated. This is quick to process and gives
everyone interested the opportunity to watch the stream of activity as it flows
by. Most systems give you options for tuning how many email notifications you
receive based on your time/interest.</p>
Accessibility and context -- the keys to effective communication and
information.</p>


🎓 The problem with experts
2025-01-07T00:00:00+00:00
At a recent family holiday gathering, we happened to be discussing how crazy
family life gets at times -- you are halfway through cooking something and
realize you don't have the ingredients on hand, so you need to improvise. Some
of the men in the group described how they always get everything out first
before starting, which ironically
I also do</a>.
Some of the females commented this was unnecessary as they had done it thousands
of times, so it was easier to get stuff as needed. They have a pretty good sense
of where stuff is and what they have on hand -- 99.9% of the time it works
pretty well.</p>
Perhaps this is a general difference between men and women -- women are
generally better at multitasking. But I think there is also another factor at
play -- with food preparation, the women in our families are experts, and the
men are amateurs. Experts are very good at what they do. They don't need a
process to prop them up. However, in the cooking example it is more difficult to
scale these methods to their husbands or children.</p>
The real value in expertise is in scaling it. Do we document what we do? Can we
make it easy for others to follow? Can we make it easy to find the things we
need? Can we create processes that reduce errors and blockers? In the kitchen,
these differences between the amateurs and experts are mostly amusing and not
all that critical, but in business and product development it is. Experts need
to be creating and contributing to platforms. This is the difference between a
sustainable business and a few talented individuals. Unfortunately, many experts
don't personally feel they need a platform, so have little motivation to
contribute to one.</p>
When you hire, expertise is important, but so is Platform Thinking.</p>


🍳 Preparation - lessons from the kitchen
2025-01-06T00:00:00+00:00
Most Sunday mornings, I make gluten-free waffles for breakfast (see recipe at
the end of this post). I like to get everything out (ingredients and utensils)
before I start.</p>
</p>
This seems to make the process go faster and smoother. There is an efficiency in
focusing on getting everything first, and then cooking.</p>
So I was wondering if there is an analogy to product development. What are the
ingredients for product development?</p>

clear definition of the goal and next steps</li>
any relevant documentation</li>
tools (editor, CAD tool, issue tracking systems, build systems, tests,
release processes, etc.)</li>
collaboration with others on the team (issues, PRs, discussions, etc.)</li>
</ol>
Is this scattered all over the place, or can you quickly assemble all of this
and hit the ground running?</p>
It helps to
organize your work in a notes system</a>.
This has links to relevant information and notes so you immediately have all the
relevant context.</p>
For some tasks like code review, tools can prepare the task for us.
Github/Gitea/Gitlab pull request (PR) workflows have the code, discussion, and
proposed changes all in one place.</p>
With any task, having all the ingredients at hand makes for a nice result!
Preparation leads to focus.</p>
</p>
Gluten-free waffle recipe:</p>
</p>


😊 Joy
2025-01-03T00:00:00+00:00
Platforms help bring the joy back into engineering.</p>
How? -- by assisting you with the tedious error-prone tasks, and freeing you to
focus on the valuable creative work.</p>
Satisfaction comes from being useful and creating value. We need to get to the
place where we can do that work.</p>


🪞 Platform reflection
2025-01-02T00:00:00+00:00
How has YOUR Platform done this past year?</p>
Have you burned down technical debt?</p>
Have you automated repetitious or tedious tasks?</p>
Is your test coverage increasing or decreasing?</p>
Are you releasing more often?</p>
Are deployments getting easier?</p>
Are you deploying with more confidence?</p>
Is your documentation getting better?</p>
Is it easier for new people to get up to speed?</p>
Do you have any new checklists?</p>
Have any existing checklists been automated?</p>


🙏 Humility
2024-12-27T00:00:00+00:00
An essential mindset for platform thinking is humility -- the realization that
we as humans are finite, we make mistakes, we don't always perform at the same
level, we get lazy and tend to take shortcuts.</p>
YOUR Platform can compensate for much of this as it does the boring tedious
work, checks for mistakes, deploys reliably, provides structure, etc.</p>
But we need to acknowledge this first, and that is the first step.</p>


🔄 The biggest benefit to frequent updates
2024-12-23T00:00:00+00:00
Perhaps the biggest benefit of frequent updates is that the changes are small,
making them easier for humans to absorb and adapt to.</p>
The conventional wisdom in our industry is that frequent updates mean things are
always breaking. This has not been my experience; rather, things are
continuously improving in small ways, and the changes are much easier to adapt
to.</p>
Arch Linux is a classic example -- always improving, rarely breaks, and when it
does, you can usually find a fix in a few minutes. Zephyr is another example --
always improving, and rarely problems merging in the new changes.</p>
When things break or change unexpectedly, you are only dealing with one issue at
a time, not dozens or even hundreds.</p>
This is more organic and enjoyable -- the way natural systems work. Small
continuous changes (either good or bad) are better than large step changes and
much easier on the system.</p>


📅 Dates in filenames
2024-12-20T00:00:00+00:00
One of the most common labels we use today is dates. We use them in file names,
reports, database fields, etc. There are many ways to enter a date:</p>

2024-12-20</code></li>
20 Dec 24</code></li>
12/20/2024</code></li>
20/12/2024</code></li>
Dec 20, 2024</code></li>
etc.</li>
</ul>
RFC 3339</a> (YYYY-MM-DDTHH:MM:SS+HH:MM</code>)
is increasingly becoming an international standard. Many programming languages
(Go, Rust, Javascript, Elm) support RFC 3339.</p>
What about when naming files? It is often helpful to put a date in directory or
file names so you can easily sort to find the latest or construct a timeline of
events. In this case it makes sense to format the date
general to specific</a>
YYYY-MM-DD and put the date at the beginning of the filename:</p>

2024-10-20_project-requirements.docx</code></li>
2023-11-01_architecture-rev1.pdf</code></li>
2024-11-11_architecture-rev2.pdf</code></li>
2024-12-15_release-v1/</code></li>
</ul>
With a little thought, even simple things like file and directory names can
become useful metadata that is easy to sort and query. Consistency and standards
-- simple things you can do to make things better.</p>


🔄 Lifecycle management in dynamic systems
2024-12-18T00:00:00+00:00
One of the hard parts of dynamic systems is the lifecycle management of various
bits of running code.</p>
It is easy to create stuff, but are you cleaning up properly when things
disappear or change?</p>
We can order these in difficulty:</p>

Create something: (easy)</li>
Delete something: (hard)</li>
Change something already running: (very hard)</li>
</ul>
On the surface, this looks easy -- what is the big deal, right?</p>
But we soon realize differently. Have you ever heard of memory/resource leaks
and race conditions? Even worse, code that we don't know is still running.</p>
Implementations usually start simple -- no problem.</p>
But the scope grows, the system becomes more dynamic, and we start to scale.
Scaling requires concurrency. Concurrency is hard, but when the system changes
while running, concurrency gets really hard.</p>
This is the nature of IoT systems -- because they reflect the real world, and
the real world is always changing.</p>


🧪 Abstractions and Platforms
2024-12-17T00:00:00+00:00
I've come to appreciate the Go communities advice on abstractions.</p>

Avoid creating abstractions prematurely. Every abstraction has a cost, and
often duplicated code is simpler and easier to maintain than a premature
abstraction.</p>
</blockquote>
Don't take DRY (Don't Repeat Yourself) to an extreme. It is perfectly fine to
duplicate code several times and only create an abstraction once you see a clear
pattern and understand better the problem the abstraction is solving.</p>
For instance, in Simple IoT, we implemented a number of subsystems (rules,
modbus, 1-wire, etc.) before we noticed a pattern -- all of these systems needed
lifecycle management, which is very tricky to get right. So it made sense to
create a
client abstraction</a>
that handled lifecycle in one place. But it was not clear what we really needed
until we had implemented several subsystems.</p>
Follow the YAGNI (You Ain't Gonna Need It) principle -- only create abstractions
when you need them, not for hypothetical future use cases.</p>
The same applies to platforms.</p>
Other than avoiding architectural pitfalls and selecting the wrong technology,
focus on building out YOUR Platform to solve your current needs, not your future
needs. This is one of the best ways to know if your architecture and technology
are right.</p>
Platforms for the sake of platforms are detrimental and counter-productive --
kind of like the IT department that loses its focus that they are there to serve
the business needs and becomes a barrier to getting things done. They must serve
the business goals, not themselves.</p>
A platform in itself provides no value if you are not shipping something of
value.</p>
The same can be said of any supporting role or metric -- finance, performance,
etc.</p>
If you are not shipping (internal prototypes and test versions count as
shipping), nothing else matters.</p>


🤝 Keeping commitments
2024-12-16T00:00:00+00:00
When we commit to something, do we follow through? How do we get better at this
when we always have more to do than time to do it?</p>
The thought came recently that it probably starts with keeping commitments to
ourselves, which is the essence of self-discipline. If we can't keep promises to
ourselves, likely we won't keep them to others either.</p>
If we plan to get up at 6 AM, do we?</p>
If we plan to walk every day, do we?</p>
If we plan to work on something today, do we block out the time and make it
happen?</p>
Seemingly simple things, but important and sometimes hard in this busy life.</p>
If we can't do the small things, we'll likely not succeed in the bigger things.</p>
This is foundational in Platform Thinking -- at the personal, group, or
organization level. If you do the small things right, this will enable you to do
the big things.</p>


📝 Personal vs. collaborative documentation
2024-12-13T00:00:00+00:00
Yesterday</a>,
we discussed the importance of a personal note-taking tool. We can also extend
this concept to a team or company using Wikis, Markdown in Git, shared Google
Docs, etc.</p>
Collaborative documentation is very valuable -- IF you can get group
participation. Over the years at various companies, I've set up Wikis as
collaborative documentation efforts. Most of them resulted in only me
contributing. Group initiatives like this need to come from the top, otherwise
there is very little incentive for people to contribute. As a result, I've
pretty much given up on Wikis for small teams/companies -- it is nearly
impossible to get momentum and maintain the information. This is sad because so
much knowledge is lost and effort wasted as the same things are rehashed,
relearned, and re-communicated.</p>
As a result, I now focus primarily on:</p>

My own personal notes system</li>
Markdown in project Git repos</li>
</ul>
If I'm going to be the only one creating and reading notes, it may as well be in
my own personal notes system which is most convenient for me.</p>
For project documentation, this is best maintained in the Git repo as close as
possible to the source code or design files. There it has a better chance of
documentation being maintained if it is worked into the PR workflow and review
process.</p>
In the course of product development, documentation is the hardest deliverable
to produce because its effects are hard to measure. Initially, the product still
functions without documentation. Documentation costs short-term but produces
compounding gains long-term. It is an investment in the future. If your team or
organization is not interested in making this investment, you can always make it
personally.</p>


🔧 What is your most important tool?
2024-12-12T00:00:00+00:00
Your editor? Your CAD tool? Your 2GHz oscilloscope? Your phone? Your computer?</p>
In the information age, we are flooded with information and opportunities.</p>
Many of the problems we solve are ones we've never seen before, or at least not
very often.</p>
Most of us have more to do than we can ever hope to get done in multiple
lifetimes.</p>
We are exposed to way more knowledge than we can ever fit in our finite
memories. Therefore, a tool to efficiently capture, organize, and find your
personal knowledge may be your most important tool.</p>
If we figure out something, we can make a note so we don't have to figure it out
again the next time.</p>
If we repeatedly do something that requires multiple steps, a checklist can make
this easier.</p>
If we learn something valuable, a note makes this useful in the future.</p>
Creating notes encourages reflection.</p>
With a good notes system, we can organize and prioritize what we do.</p>
Over time this helps build a picture or model of what we know and have
experienced. This model helps us leverage what we have done in the past in the
future.</p>
There are many good notetaking tools and methods (Notion, Workflowy, Logseq,
Bullet Journal, etc.) so there is very likely one that will fit you. Some things
I look for:</p>

low friction to use</li>
stores text and images</li>
can be used on multiple mobile and desktop platforms</li>
can export the information so that it can be moved to another system if
necessary</li>
has a stable history and likely future -- something you can use for 5-10+
years</li>
easy to share information with others</li>
and most importantly -- you own the information and can take it with you. IE,
don't store your personal knowledge base in your employer's MS Teams OneNote
app.</li>
</ul>
Workflowy has been perfect for me, but there are lots of other good options.</p>
Over time your personal notetaking tool becomes a valuable knowledge base that
you can take with you anywhere you go -- part of YOUR
Personal</a> Platform.</p>


📧 Inbox zero
2024-12-11T00:00:00+00:00
Thanks to a nudge from Khem</a>,
I'm back to Inbox-zero.</p>
The big question is what to do with emails that need action or have valuable
information in them? Some put them in a special folder. Others tag them. Others
forward them to lots of people.</p>
The best solution is to move the action or information to another system such as
a project ticketing system, or Workflowy (my personal notes system). I then
archive the email and dig it out of the archive if I need to reply later.</p>
Email is a great notification system, but it does not work very well at the
following:</p>

collaboration</li>
knowledge capture</li>
workflow</li>
</ul>
Find other solutions for the above such as:</p>

Github/Gitea/Gitlab issues, projects, wikis, etc.</li>
Markdown in Git is a great way to capture knowledge from emails</li>
Trello</li>
Personal notes system</li>
</ul>
Collaboration, knowledge capture, open access to knowledge, and workflow are key
components of YOUR Platform. Getting information out of email is critical, and
following an Inbox-zero policy is a great way to help make this happen.</p>


🏆 Goals vs. Systems
2024-12-10T00:00:00+00:00

“You do not rise to the level of your goals. You fall to the level of your
systems.” -- James Clear, Atomic Habits</p>
</blockquote>
Goals and vision are important, but we also need a way to get there.</p>


🏁 Easy to start vs. easy to finish
2024-12-09T00:00:00+00:00
There are many ways to classify how we do things. One way is "easy to start" or
"easy to finish."</p>
We may try to use off-the-shelf "maker" style embedded boards (Arduino or
Raspberry PI) instead of a SOM (System-on-module) designed for industrial use.</p>
An interpreted language like Python is easy to use and definitely has its place
on your workstation, tooling, etc. But deploying, cross-compiling, upgrading,
and maintaining a Python stack on thousands of embedded Linux edge nodes is not
easy -- Go is much better over the long haul.</p>
Skipping documentation, CI/CD, and other basic best practices is certainly
quicker to start, but long-term will slow you down.</p>
We can skip tests when writing code (often faster up front), or we can take the
time to write tests as we implement features. Likewise, when we debug a problem,
we can just fix the problem, or we can reproduce the problem first with a test,
and then fix it to ensure it does not happen again.</p>
Ignoring architecture up front is easier initially, but taking the time to think
through architectural issues like decoupling, life-cycle management, and
high-level performance, pays huge dividends long-term.</p>
Outsourcing PCB design is easy to start, but owning the custom PCB designs in
your product (especially the integration points) gives you the flexibility to
easily fix, improve, and adapt.</p>
When designing a PCB with constraints (such as high-speed or high-voltage), we
can manually route the board or invest time in creating DRC rules and
documentation that clearly details the board's constraints. The first approach
might be quicker to get rev 1 out, but the latter makes it much easier to
maintain the design over time, onboard new people, etc.</p>
When designing hardware, we can skip creating CAD libraries, assigning house
part numbers and instead hand-create BOMs. But long term it becomes very
difficult to manage even a small inventory parts in the lab, let alone
production. We have duplicates, things are lost, and mistakes are made.</p>
We can do things like we've always done them (certainly easier to start), but
there are new technologies, tools, and workflows that can greatly improve our
productivity.</p>
We certainly need to push hard to get to an MVP (minimal viable product), but
this should be done by cutting features, not the approach. However, most people
do exactly the opposite -- they want all the features up front and rather cut
out the foundational investments in technology and process.</p>
Here's the thing -- "easy to start" puts you on the path that accumulates
technical debt. As time goes on, things become more complex, fragile, and
difficult to manage. This approach does not scale.</p>
"Easy to finish" puts you on the path to burn down technical debt. Over time you
look for patterns and ways to simplify things. You continuously improve all
aspects of the product. You select the best technologies at the time and adopt
new technologies when it makes sense. This approach scales.</p>
Starting is easy, and finishing is hard. What are you optimizing for?</p>


🎧 Use a headset optimized for calls
2024-12-06T00:00:00+00:00
One thing most of us do routinely is voice calls or video conferencing. For this
to work, each party needs to have a microphone and a speaker.</p>
The simplest setup is to use the microphone and speaker built into your phone,
laptop, or car. Sometimes this works pretty well.</p>
But, often we are environments where this is background noise. Simple
microphones, especially those some distance from our mouths, cannot discern
background noise from our voice. AGC (automatic gain control) algorithms tend to
amplify any sound to capture your voice. The most baffling scenario is those
walking through a busy airport holding a phone in front of them in speakerphone
mode. Another disruptive problem is echo which sometimes occurs when using a
laptop mic/speaker for video conferencing. Trying to talk when your voice is
being echoed back is nearly impossible.</p>
Conversely, our ears are very good at differentiating what we want to hear from
local background noise. Our brain and ears are
very sophisticated instruments</a>
and include features such as using the time delay and intensity difference
between each ear to pinpoint a source, selective focus, frequency tuning,
adaptive processing, etc.</p>
Some headsets attempt to block out background noise to the microphone. The most
effective ones have a direction microphone located very close to your mouth (on
a boom), or a microphone array and DSP processing. Two examples I've used are
the Plantronics - Voyager 5200</a>
(Bluetooth) and the
Sennheiser SC 160</a>
(USB).</p>
</p>
</p>
There are likely many other good options. I use the Plantronics with my phone
and the Sennheiser with my computer. Both are very effective at blocking out
background noise and I'm sure there are many other effective models.</p>
The result is whomever I am communicating with can clearly hear what I am saying
without being distracted by background noise in my environment. This makes me
more effective.</p>
Some headsets have noise noise cancellation for the speakers. This is very nice
for deep work when we are trying to focus and need to block out all sounds other
than perhaps some music. But they are of marginal use during a call because our
ears are already pretty good at focusing on one sound and during a call we have
the discussion to focus on.</p>
Consider having separate headsets for deep work (noise-cancelling speakers) and
calls (noise-reducing microphones) as part of YOUR
Personal Platform</a>. This may seem
like a minor thing, but the small things add up. Using a good microphone is a
courtesy to those listening to us. It is a courtesy to the entire group on the
call that our background noise is not interrupting the call. It is professional.
It shows we care about others.</p>
For more discussion and review, see
this thread</a>.</p>


⚡ Are your processes proactive or reactive?
2024-12-05T00:00:00+00:00
There are two ways to implement a process.</p>
The traditional way is as a check to make sure people did everything right. This
could be required documentation that is written after a project is finished, a
separate QA department that runs tests after each release, or various
certifications like UL, FCC, CE, etc. There are certainly times when things like
this are necessary; however, writing documentation after the fact is never much
fun, and dealing with failure reports from the QA department is not all that
pleasant, and the FCC testing facility has been called "the house of pain."
We'll call this the "reactive" process approach.</p>
An alternative approach is the "proactive" process. Structure your documentation
efforts as part of the development process -- as a way to help you think and as
a vehicle for collaboration. QA can also be implemented proactively as automated
tests that are run during development.</p>
Proactive processes focus on well-designed tools that help the people doing the
work get it right vs. tools that help managers who oversee the work. Proactive
process tools are well-designed, fast, and a pleasure to use.</p>
Think what could happen if UL focused on providing KiCad templates and DRC rule
sets for various high-voltage design scenarios that would help engineers do
their work instead of a bureaucratic morass that is very painful and expensive
to work through. The first approach encourages innovation, whereas the current
approach greatly hinders it.</p>
Some reactive processes are necessary, but where is your focus? The more you can
shift your processes from reactive to proactive, the better off you'll be. This
is what Platform Thinking is all about -- building YOUR Proactive Platform.</p>


⚖️ Architecture vs. Optimization
2024-11-27T00:00:00+00:00
In 1968, "The Art of Computer Programming" by Donald Knuth was published with
the following quote:</p>

"We should forget about small efficiencies, say about 97% of the time:
premature optimization is the root of all evil."</p>
</blockquote>
This was in the age when computer resources were scarce.</p>
We now live in the age of abundant computing resources, so it seems this
statement would be many times truer now.</p>
Performance is important when we select technologies. For instance, Go or Rust
is significantly faster than Python and it is important to think about issues
like this up front. This is an architectural issue.</p>
Deciding if we will use a message bus or not is an architectural issue.</p>
However, if we receive data over a network, especially in distributed IoT
systems, the network is relatively slow. So it rarely pays to optimize how we
process data from remote systems until we reach a pretty large scale. This is an
optimization issue.</p>
Creating an abstraction to reduce some duplicated code is an optimization.</p>
With architecture, we need to think about it up front.</p>
It rarely pays to optimize up front -- that is best done later after some
learning.</p>
And if we spend too much time optimizing, we never get to the learning. And if
we don't learn from testing a working system, we're likely optimizing the wrong
things.</p>
Architecture vs Optimization -- there is a difference.</p>


Creativity
2024-11-25T00:00:00+00:00

“Creativity is just connecting things. When you ask creative people how they
did something, they feel a little guilty because they didn't really do it,
they just saw something. It seemed obvious to them after a while. That's
because they were able to connect experiences they've had and synthesize new
things" -- Steve Jobs</p>
</blockquote>
(Thanks Rod</a> for this quote!)</p>
As technology advances, being good at making connections (integration) is the
critical skill, as we can't create it all ourselves.</p>
Part of YOUR Platform is building a high-quality toolkit of things to connect.</p>
And knowing how, when, and where to connect.</p>


🔗 Coupling
2024-11-22T00:00:00+00:00
Good platforms emphasize loose coupling where one part of the system can change
without affecting another.</p>
Not abstractions for the sake of abstraction.</p>
But rather well-thought-out data structures and patterns.</p>
A message bus like NATS is an excellent way to decouple a program. One part of
the system publishes information, and any other interested part can subscribe.</p>
But for this to work, you must have simple and flexible data structures that
don't change often. Otherwise, your system is now tightly coupled again.</p>
Simple data structures and messaging</a>
-- a sane way to scale YOUR Platform.</p>


⏰ A platform stuck in time
2024-11-21T00:00:00+00:00
Vintage is great in some contexts—I like old woodworking hand tools. Even in our
modern jobs, some vintage tools, like a pencil and paper, are still very useful.</p>
But information technology moves forward, and we rarely benefit from using
decade-old phones, computers, programs, etc.</p>
New information technology and tools are often simpler to use and more powerful.
Some examples:</p>


C -> Go/Rust/Zig</p>
</li>

Apache -> Caddy</p>
</li>

Vim -> Helix</p>
</li>

x86 -> ARM -> RISCv (MPU)</p>
</li>

8051 -> PIC -> ARM -> RISCv (MCU)</p>
</li>

FreeRTOS -> Zephyr</p>
</li>

Kafka -> NATS</p>
</li>

Rotating storage -> Solid state</p>
</li>
</ul>
Information technology, and perhaps computer technology in general, is
different. The physical constraints are largely removed as we scale into GHz and
Terabytes, which opens the doors to human innovation and collaboration.</p>
Innovation is largely a process of discovery, so you can't predict where it will
go and what will happen. It is a result of development iterations that are now
happening at a massive scale and distributed across the globe.</p>
Therefore, when you lock yourself into some version of technology in the name of
stability, cost savings, or whatever, you are also limiting the potential for
your product -- for interesting things to happen that no one could have
predicted.</p>
It takes humility to acknowledge this -- that you can't see the future, the best
you can do is prepare for it.</p>
YOUR Platform is the vehicle that keeps your product development moving forward
with technology so that you are not stuck in time as you watch it pass you by.</p>


🧪 What is a good abstraction?
2024-11-20T00:00:00+00:00
When writing a program, abstraction is a powerful tool.</p>
Every abstraction has a benefit and a cost. Hopefully, the value > cost.</p>
There are good abstractions and bad abstractions.</p>
Good abstractions break things down into simpler steps that others can
understand.</p>
Bad abstractions may be brilliant but are nearly impossible for anyone but the
author to understand.</p>
Who are you optimizing for? Yourself, or everyone else?</p>
This often tells the difference between a good and bad abstraction.</p>


⚡ Improve development efficiency, reduce technical debt
2024-11-18T00:00:00+00:00
The term "platform" can mean a lot of things, but this is primarily what I'm
talking about in the context of product development:</p>
Improve development efficiency, reduce technical debt</em></strong></p>
Some examples:</p>


do more with what you have</p>
</li>

iterate faster</p>
</li>

make less mistakes</p>
</li>

regularly update to new software versions</p>
</li>

collaborate better</p>
</li>

use better tools</p>
</li>

automate where you can</p>
</li>

reduce legacy (untested) code</p>
</li>

deploy with confidence</p>
</li>

react quickly to new customer needs or problems</p>
</li>
</ul>
Of course, everyone wants these things and claims to be doing them. But in your
gut, you know if these things are getting better or worse.</p>
And the only way I know to get better is to work on YOUR Platform.</p>


🎯 Rituals and Culture
2024-11-14T00:00:00+00:00
Recently,
we discussed</a>
personal habits.</p>
Organizations have habits too -- we call them rituals and culture.</p>
How does the organization react when things don't go as planned?</p>
How does the organization motivate and demotivate people?</p>
How does the organization encourage openness and transparency?</p>
A lot of this probably comes down to those at the top -- the art of leadership.</p>
Organizations are made of people, so the struggles we face personally we also
face organizationally.</p>
We need some structure and tooling to help us do our job.</p>
To ensure the boring parts are taken care of.</p>
To catch errors we overlook.</p>
Platforms.</p>
Does your organization celebrate platforms? If so, how?</p>


📈 Two ways to scale
2024-11-13T00:00:00+00:00
Scaling is the goal of many businesses -- grow revenue, serve more people, sell
more devices, grow the product line, etc.</p>
There are two ways to scale:</p>


Increase volume with a small number of direct customers. You might be making
an integrated circuit that is sold to a company making consumer devices, or
you might be making a consumer product that is sold through a few outlets
like Amazon and Walmart. Although you might have thousands of end customers,
you only have a few direct customers.</p>
</li>

Increase the number of direct customers. Many of these customers may be low
volume, but the aggregate can be large, sometimes described as the
long tail</a>.</p>
</li>
</ol>
(For this discussion, we'll assume you already have a product or technology that
meets a real need).</p>
With approach #1, you can brute force about anything. As long as the product is
something people want, you can dedicate application engineers to help your
customers make it happen. Sometimes the design/development team interacts
directly with customers. This is expensive, and the reason many industrial
products are costly is that they rely on direct support.</p>
However, this does not scale past a handful of customers. Direct application
engineering support is expensive. If you want to scale the number of customers,
you need to do something different.</p>
Open-source projects are an interesting study in this regard. They address well
the needs of many users. How do they do this?</p>
There are two primary aspects: A) Quality, and B) Community.</p>
For any complex technology to scale without a lot of hand-holding, the quality
needs to be high. There has to be good documentation. The source code or design
needs to be readable. The architecture needs to be simple and solid so it can be
understood. You need consistency (data structures, code organization, etc.).
Testing needs to be automated and robust so there are not a lot of unexpected
problems.</p>
The second thing that is needed is community. A community is the only way to
support a large number of users or customers of a complex technology. With
complex systems, the problems you run into are often things you have never seen
before. If all of these problems need to be solved from scratch with direct
one-on-one support, you will quickly become buried. Additionally, you'll find
that experienced users are often better at solving user problems than technology
creators. If this support is done in a public place, you soon build up a
knowledge base that can be reused.</p>
Quality and Community are the ways to scale the number of direct customers or
users, and OSS-style workflows and communities are the best way to accomplish
this.</p>
If you want to address the long tail, the days of NDAs with every customer are
past. The future is open. Not necessarily 100% open source, but certainly open
documentation and support. And "open" provides feedback that increases quality
-- it all works together.</p>


🔍 How to get the little things done?
2024-11-11T00:00:00+00:00
Previously, we
discussed habits</a>.
Habits are the way we get the small, but important, things done.</p>
Some people are naturally neat, organized, and focused -- I'm not. Stuff tends
to pile up in my workspace and I don't even notice it.</p>
One thing I've been using some is the
Pomodoro technique</a>. This is
a time management method where you use a timer to break down work into intervals
-- typically 25 minutes in length separated by 5m breaks.</p>
The primary goal of the Pomodoro technique is to increase focus, and that is a
worthy goal.</p>
However, another benefit of this technique is to allocate time for the little
things, like organizing your workspace which is a good use of your 5m breaks.</p>
These are the little things that clear space to do the big things -- part of
YOUR Personal Platform.</p>


😊 How can you make your commercial source code customers happy?
2024-11-08T00:00:00+00:00
When engaging with a community around an open-source project, Github is likely
the lowest friction option. It makes sense.</p>
But there are other scenarios -- one is where you are delivering closed-source
commercial source code to customers.</p>
Git makes sense for delivering any source code, no matter what the context --
closed, open, etc.</p>
Using zip files to deliver source is antiquated. Since Subversion (2000) and Git
(2005), there are now other options.</p>
With Git, is easy to deliver updates, manage changes, tag releases, and track
what has happened.</p>
Tools like Gitea/Forgejo, that wrap Git, provide nice ways to interact around
the source code -- issues, pull-requests, projects, etc. Issues and
pull-requests are much better ways to communicate about code than email or other
messaging methods. The communication is close to the code and preserved instead
of being disconnected, and eventually lost.</p>
Gitea/Forgejo also give us unlimited organizations and users that we can create
and manage. What this means is every customer can have their own organization.
This provides issolation between customers. Each now has their own space to
raise issues, discussion PRs, etc. You can mirror the source code they have
purchased to their own organizational space, and continue updating it as long as
they are a customer. If they decide to quit purchasing updates, you can leave
their Gitea organization and source code intact, but simply quit updating it.
This is ensures you do break build processes they already have in place and is a
measure of good will.</p>
Why not deliver the source-code to the customer's Github organization? You have
less control in Github. Simple things like users not being able to create an
account are now under your control. With Gitea, you have more control of the
repo visibility. It is less likely someone might accidentally make a repo public
in the future.</p>
Would you like the velocity, stability, and collaboration that OSS projects
have? With tools like Gitea/Forgejo, you can mirror these workflows into any
software development effort.</p>


🚀 Platforms vs Architecture Astronautics
2024-11-07T00:00:00+00:00
The excellent Superstruct Manifesto</a>
addresses the tendency for developers to want to "rewrite" things, wholesale
switch to new frameworks, jump on new tech, etc. vs. delivering real business
value.</p>
Another description for these people is
Architecture Astronauts</a>.</p>
Platforms, at least the way I envision them, are the opposite of this.</p>
They are incremental, not wholesale.</p>
They are a part of our daily thinking, not some separate grandiose effort that
will solve all of our problems.</p>
They are largely invisible.</p>
They leverage what we already have and make it better.</p>
They are the responsibility of everyone, from the bottom to the top, not some
elite team.</p>
However, like personal habits, company platforms take discipline to implement
and are sometimes difficult to get established.</p>
There are exceptions, but if platform work is exciting and interesting, this is
a warning flag -- you may be venturing into architecture astronautics.
Generally, we need to block out time in our schedule to work on platforms as it
does not just happen because it is fun -- kind of like brushing your teeth and
exercising.</p>
But in the end, platforms make the difference whether you have a sustainable
business or just some talented individuals. And just like exercising, or
cleaning up our workspace, it is very satisfying afterward.</p>


💪 Habits -- a critical ingredient in YOUR Personal Platform
2024-11-06T00:00:00+00:00
We defined</a> your
platform as the collection of technology, tools, and processes that enable you
to differentiate and consistently deliver your products and services.</p>
At a
personal level</a>,
habits are critical -- they enable us to do the boring/hard stuff that makes us
effective.</p>
Getting up early gives us undistracted time to think.</p>
Regularly exercising helps us in so many ways.</p>
Patterns and tools</a>
that help us plan and focus are sometimes necessary.</p>
Bad habits (sleeping in, overeating, reading instead of working, etc.) are easy
to fall into.</p>
Good habits are amazingly hard to establish, but worth it, and a big part of
YOUR Personal Platform.</p>


🚀 Advanced technology and small players
2024-11-05T00:00:00+00:00
Integrated circuits (IC) vary widely in complexity from the small devices that
are easily documented and understood to complex beasts that would require a
stack of books and years to fully document.</p>
If you are a high-volume manufacturer of consumer goods, you can easily get all
the attention and support you need from IC vendors. They will help you design
your product.</p>
However, if you are a low volume manufacturer, it is nearly impossible to get
direct support for complex ICs. Support resources are limited and IC vendors
can't help everyone in the world design their products.</p>
So they allocate support engineers to a small number of higher-volume customers.</p>
It does not matter how important you think you are, or how much money you have,
if you don't have volume, it will be hard to get direct support from complex IC
vendors. Their margins are small and they make it up in volume.</p>
However, there is a path forward for low-volume manufacturers.</p>
You can buy system-on-modules (SOM) from vendors who do the hard parts of
integrating a processor, RAM, power supplies, and other complex ICs on a module.</p>
Additionally, they provide open-source software for these modules. Open-source
has changed the game for software by providing everyone access to advanced
technology.</p>
The third aspect of this is community. If a technology vendor has a vibrant
community around their products, then the community can support the low-volume
customers. This is the best and most sustainable model.</p>
The difficult part in all this is discerning where the line is. What technology
is accessible to low-volume manufacturers and what is not?</p>
A few red flags:</p>

SOM or other module technologies that don't have a clear support model for
industrial customers.</li>
The technology is not open and requires NDAs to access documentation.</li>
The documentation is inadequate and difficult to understand. The only way
you'll figure it out is by the vendor helping you.</li>
There is not a vibrant community around the products.</li>
Supporting software is not open-source.</li>
</ul>
The accessibility of technology today has spurred tremendous innovation.
Open-source and community have changed the game enabling even small players to
use very advanced technology in their products. But you must understand your
limits. There is technology that is accessible to small players and technology
that is not.</p>
IC vendors who put the pieces in place (openness, open-source, and community)
can benefit from the long tail</a> of
distribution, which increasingly is where innovation is happening.</p>


What can kids teach us about platforms?
2024-11-01T00:00:00+00:00
When helping my children with math, one of the biggest struggles is to teach
them to break math problems into small steps. They are smart and think they can
do it in their head, and inevitably make mistakes because everything gets
muddled.</p>
When you break a math problem into steps, it is easy -- because each step is
easy.</p>
And writing out the steps is easy. It's not rocket science.</p>
But what is not easy is the discipline to write out the problem step-by-step
instead of rushing to the answer.</p>
YOUR Platform is the same way -- it is breaking down problems into simple steps.</p>
It's not rocket science.</p>
But it does take discipline to create a checklist, document a process, break
things down into smaller steps, automate some things, look for patterns, etc.</p>
Getting to simplicity is hard, but once you get there, everything is easier.</p>


🏛️ What are the pillars of a good platform?
2024-10-31T00:00:00+00:00
Simplicity</strong> - you can make changes when needed, and understand the system enough to do so.</p>
Consistency</strong> - useful patterns are reused throughout the system where appropriate.</p>
Reliability</strong> - the design/implementation is sound and there is enough testing that you don't worry too much when deploying updates.</p>
Velocity</strong> - iterations are fast and easy to deploy.</p>


🛣️ Kayaks and Platforms
2024-10-30T00:00:00+00:00
Recently we purchased a few used kayaks that had been built 20 years ago from
Pygmy Boats kits</a>. (Unfortunately, Pygmy is not longer
supplying kits, but similar kits can be purchased from
Chesapeake Light Craft</a>)</p>
</p>
We had the pleasure of visiting with the former owner (Mike) for an hour or so
and he generously shared his knowledge about the boats, and kayaking in general.</p>
The more I listened to Mike, the more I realized these boats were his platform
for getting on the water.</p>
Building a boat like this from a kit is a lot of work, but in the end you have:</p>

a nice boat -- performs as good as boats 4x the cost</li>
one that is customized to exactly what you need (features, rigging, seat,
bracing, etc).</li>
can be easily modified in the future</li>
</ul>
Additionally, you have the know-how and tools to repair and maintain it.</p>
If the hull is damaged, you can get your extra plywood, fiberglass cloth, and
epoxy, and fix it.</p>
If you don't like the rigging, you can fasten new points and fill the old holes.</p>
If you don't like the seat, and can hand-carve a new one out of a block of foam.</p>
If the boat is scratched up, a little sandpaper and varnish will have it looking
like new.</p>
It's not for everyone, but for those who want to own their kayak platform, a
stitch and glue</a> boat is nice
option.</p>


💻 What is the purpose of technology?
2024-10-29T00:00:00+00:00
To do more with less effort.</p>
To simplify our tasks.</p>
Sometimes this backfires and technology ends up giving us more to do and
complicates our lives.</p>
It becomes a monster and we are consumed feeding it.</p>
Thus, the selection and implementation of technology is critical.</p>
How do you objectively measure this?</p>
Perhaps we need a term "technical efficiency"</p>
technical efficiency = time saved/time feeding</code></p>
Hard to measure, but important to reflect on.</p>
A number greater than 1 is good, less than 1 is bad. Ideally, the number should
be much greater than one and also increasing.</p>
Like with so many things, the key metric is not where we are now, but where we
are heading.</p>


✨ The easiest path to reliability ...
2024-10-25T00:00:00+00:00
... is simplicity.</p>
Not that we dumb things down and avoid new technology.</p>
That is false simplicity.</p>
As we
discussed previously</a>,
the problem is most often not with the underlying technology, but rather the
stuff we do.</p>
Therefore, if we can simplify what we do, it will get better.</p>
This is a fundamental purpose of YOUR Platform -- to simplify what you do. To
make it more repeatable, deployable, reliable, and stable.</p>
Technology and tools can help, but in the end what you do and how you do it is
your responsibility. You can't buy a solution for this, nor can AI do it all for
you.</p>


🔍 Where do reliability/stability problems typically come from?
2024-10-24T00:00:00+00:00
99% of the time, system bugs/problems come from the stuff you do.</p>
The code you wrote.</p>
The hardware you designed.</p>
The system you configured.</p>
The custom PRC system you created.</p>
Rarely are we affected by stability problems in the Linux kernel, the Zephyr
RTOS, an OSS library, or a bug in a chip.</p>
Why? Because these components have many more users and developers than the
in-house pieces we develop.</p>
"Given enough eyeballs, all bugs are shallow."</a></p>
Why are we so hesitant to update the open source system software in our product,
but deploy new application updates that one developer created without a thought?</p>
And ironically, the best way to avoid the rare problem in OSS components is to
keep up-to-date.</p>
We should be most suspect of the parts of the systems with few developers and
users.</p>
This is why YOUR Platform is so important -- because you reuse it, more
developers work on it, and more users use it.</p>
And by extension why it is so beneficial to open source parts of what you do.
Even if you get only a few more users, it will get a lot better, making your
products better.</p>


🔝 Support from the top
2024-10-23T00:00:00+00:00
In a recent discussion with Matthew Rassi</a> about Lean,
I asked Matthew if he has ever witnessed Lean being successful in organizations
where there is not support at the top. He answered with a resounding no.</p>
Likewise, I've never observed "Platform Thinking" being successful company-wide
without support from management.</p>
If Platform thinking does not start at the top, it is a hard go.</p>
These are cultural issues, and perhaps the most important influence those in
leadership positions have is culture.</p>
However, you can still apply platform thinking at the
personal or sometimes team level</a>.</p>



🎯 Doing more with less, rather than less with more
2024-10-22T00:00:00+00:00
Building your platform does not mean you need to invent a bunch of stuff from scratch, or purchase expensive tools.</p>
Rather, it is making better use of what you already have.</p>
Automating your current workflows and deployment.</p>
Reusing the designs and IP you already have.</p>
Increasing the rate of iterations.</p>
Automated testing so that you can deploy with confidence.</p>
Platforms are not rocket science, and you don't need fancy tools to get started.</p>
A checklist can do wonders.</p>
A few shell scripts and a cron job can get you a long ways.</p>
Automation can be written in the languages you are already using on the project.</p>
Building testing and diagnostics into your products rather than needing expensive, external fixtures.</p>
Leverage the CI features built into tools like Github and Gitea.</p>
Platforms are the boring work that clear the path and you give you time and space to do the interesting work that adds value to your products.</p>


🆘 How to ask for help
2024-10-21T00:00:00+00:00
Modern systems are too complex for any of us to know everything about them.</p>
Many of the problems we encounter we have never seen before.</p>
Communities built around various technologies are what fill this gap.</p>
So the superpower in this age is first having the humility to ask for help, and
then knowing who and how to ask for help.</p>
Before asking anything, do a reasonable amount of work.</p>
Someone once told me he puts thirty minutes into a problem before asking a
co-worker for help, which seems like a good baseline. Any less than that you are
likely needlessly interrupting and distracting someone from their work. Any more
than that you may be spinning your wheels needlessly. The nature of the problem
may justify more at times.</p>
Sometimes our questions are met with silence.</p>
Don't just remind people that you need help.</p>
Do more work, and bring more information, and then ask again.</p>
If you ask more questions, always bring more information.</p>
Help people help you.</p>


🔧 How to keep a product maintainable?
2024-10-18T00:00:00+00:00
How do you know if you have the ability to maintain your product?</p>
With complex systems, having all the design files and source code does not
always mean a lot.</p>
Do you know how to build the source code?</p>
Do you know why the hardware is designed the way it is?</p>
Do you know how to test and verify changes?</p>
Do you regularly use these tools?</p>
The only way to know for sure if you can efficiently make a change to your
product is to practice doing so.</p>
If you don't, things soon ossify to the point where the cost and risk of any
change are prohibitive.</p>


🤝 What happens when someone else tries to use your work?
2024-10-17T00:00:00+00:00
It does not build on machine X ...</p>
It does not work in scenario Y ...</p>
It is hard to understand by developer Z ...</p>
As developers, we naturally avoid the rough edges of whatever we are working on -- it hurts.</p>
When other developers try to build/run our stuff, it is amazing what they find.</p>
The more people you have building and running your stuff, the better it will be.</p>
Even two people vs. one makes a huge difference.</p>
The platform approach says anyone should be able to easily check out, build, and run any code on any machine.</p>
We've run this experiment with OSS projects for a long time -- it works.</p>
Encourage others to build and run your code -- it will make YOUR Platform better.</p>


📌 Where do you put stuff?
2024-10-16T00:00:00+00:00
There are a lot of ways to organize product development information.</p>
You could do something like:</p>

productA</strong>

firmware</li>
hardware</li>
yocto-build</li>
docs</li>
manufacturing</li>
</ul>
</li>
productB</strong>

firmware</li>
hardware</li>
yocto-build</li>
docs</li>
manufacturing</li>
</ul>
</li>
</ul>
The above is nice in that it keeps all the stuff for each product together, but
discourages re-use between projects.</p>
If you want to build a platform, then it may make more sense to do something
like this:</p>

Firmware</strong>

Boards

productA</li>
productB</li>
</ul>
</li>
Apps</li>
Common</li>
</ul>
</li>
Hardware</strong>

Part Libs</li>
productA</li>
productB</li>
</ul>
</li>
Yocto</strong>

...</li>
</ul>
</li>
Manufacturing</strong>

Common testing framework</li>
</ul>
</li>
Product</strong>

productA</li>
productB</li>
</ul>
</li>
Doc</strong>

(general technology information)</li>
(general processes, etc)</li>
</ul>
</li>
</ul>
Each type of code/design files live in a common location if possible. The
Product</code> location is only for information that is truly unique to a specific
product (documentation, top-level BOMs, etc) that does not fit anywhere else. In
many cases, each of the top level categories are a separate repo. The Product</code>
repo may pull the other repos in as submodules, so you can version everything
together.</p>
If you want to build a platform, it helps to be intentional about this. Reuse
and cross-product learning won't just happen.</p>


🌱 When edge nodes grow up
2024-10-15T00:00:00+00:00
Yesterday</a> we
discussed how to partition functionality in hierarchical distributed systems.</p>
In the past, it was difficult to do much at the edge nodes because they could
not do much.</p>
Resources were limited.</p>
It was difficult to update software in them without being on-site and connecting
a PC-based programmer, or before that replacing an OTP (one-time-programmable)
chip.</p>
But today things are different.</p>
A powerful 32-bit MCU is as now as cheap as your father's 8-bit PIC and your
grandfather's 8051.</p>
It has plenty of memory for network/communication stacks, and in-system update
routines.</p>
We now have powerful operating systems like Zephyr that we can run on these edge
nodes.</p>
If we continue our analogy to human systems, when a person is young or
inexperienced, we don't push a lot of responsibility on them.</p>
However, when they grow up, we want them to do more, make decisions, and look
around and do what needs done.</p>
Modern MCU's are capable of doing a lot including
running AI algorithms</a> -- maybe it is time our edge
nodes grow up.</p>


🔠 How do you partition systems?
2024-10-14T00:00:00+00:00
In modern networked systems, there is often the debate how to partition the
system.</p>
Below is one example where you have three levels: cloud, gateway, and IO
(input/output) nodes.</p>
</p>
All of these devices have a processor and can run software and do stuff.</p>
So where do you do stuff? There are two basic philosophies:</p>


Push as much of the processing upstream to the cloud (where you have
"infinite" resources) and never update downstream devices.</p>
</li>

Do as much at the Gateway and IO nodes as possible, make decisions when
possible with local data, and only push processed data and events upstream.
This requires routinely updating the downstream devices as needs change and
better algorithms are developed.</p>
</li>
</ol>
Initially, approach #1 seems simpler and may be appropriate in some cases.</p>
You can centralize logic and decisions as much as possible.</p>
It is easy to update the system -- you only have to update one central piece.</p>
Until you need to scale, or there is a network disruption ...</p>
And have 50 IO nodes per site that are sending frequent unprocessed data all the
way to the cloud, and decisions are coming back down, even when nothing is
changing.</p>
And then you eventually have 1000 sites.</p>
And then there is a cloud outage and the system is in limbo until the cloud
issue is fixed. All data during the outage is lost.</p>
And someone damages the CAN bus cable and an actuator is left in the ON state
and a tank overflows because no one is making decisions.</p>
And you are storing Gigabytes of unprocessed historical data that you never use.</p>
Distributed systems are hard, but in the end they scale and are more resilient,
especially when you are forced to be distributed by device locations. Within
reason, the more processing you push to the edge, the better. "Distributed" in
these systems is not an architectural choice, but is forced on us by topology.
Why not leverage this since we need to deal with it anyway.</p>
We can compare this to human systems. How well do autocratic, centralized
organizations work? Where someone at the top makes all the decisions and the
rest are minions?</p>
They may work OK at a small scale, but even that is questionable.</p>
A student of history soon observes that successful organizations push
responsibility down. People at the top provide oversight but are only involved
in local decision-making when coordination is required. Trusted people feed
necessary information up as required.</p>
Is it any different with hierarchical, distributed digital systems?</p>


🏷️ How do you consistently name things?
2024-10-10T00:00:00+00:00

There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton</p>
</blockquote>
Plenty of opinions abound on style, naming conventions, formatting, etc.</p>
But there is one principle I try to follow when naming things -- go from general to specific. Example:</p>

UserEmailAddress</code></li>
</ul>
For functions, you may have a module that has multiple operations. In this case, the operation is a specific operation of a more general module.</p>

atmtcp_v_dev_open()</code></li>
atmtcp_v_dev_init()</code></li>
atmtcp_v_dev_close()</code></li>
</ul>
Even though open_atmtcp_v_dev()</code> reads better as an English sentence. Programs can be thought as both prose and data, and sometimes there are advantages to considering function and variable names as data.</p>
General->specific naturally organizes things. It is easy to browse variables in an editor that may list all the variables in a module. Searching/refactoring is also easier.</p>
Sometimes when dealing with two unrelated types of information, it is not obvious which is more specific. In the below example, VINST/VPEAK</code> are measurements (type), and AX/AY/...</code> are channels (instance). Which should go first?</p>

VINST_AX</code></li>
VINST_AY</code></li>
VINST_AZ</code></li>
VPEAK_AX</code></li>
...</li>
</ul>
Generally I prefer to put the "type" first, and then the "instance" -- especially if there are multiple instances.  But this could probably go either way.</p>
Another way to think about this is VINST</code> may be a map/dictionary/hash-table property of an object, and AX/AY/AZ</code> may be keys into this map.</p>
Use nouns for variables and verbs for functions.</p>
Does this make sense? How do you name things?</p>


🔄 How can we be more consistent?
2024-10-09T00:00:00+00:00
What does not work very well is long standards and endless code reviews where we shame people into compliance. There are better ways.</p>
We now have tools that can proactively format our code. We have linting tools that check things. Use them. Even if no one else reads our code, they help us.</p>
Write tests. Perhaps their greatest value is that they give us a new perspective on our work, which leads to consistency.</p>
We can have CI hooks that check for various things -- check out the Zephyr project if you want a good example of this.</p>
Being nudged by a CI tool to be consistent is a much better experience than being pulled over by the consistency police.</p>
Does YOUR Platform have tooling that encourages consistency?</p>


🔄 Does consistency matter?
2024-10-08T00:00:00+00:00
If you have a single developer on a single project, then perhaps consistency does not matter too much.</p>
However, if you want to scale, either products or developers, then consistency matters.</p>
Why?</p>
So that code does not get drastically reformatted every time someone makes a change, making Git diffs impossible to review.</p>
So that any developer can easily understand and make changes in any part of the codebase.</p>
So that new products can leverage previous efforts.</p>
So that new developers can be more easily on-boarded.</p>
So that we can see patterns and simplify systems.</p>
So that our systems are tested.</p>
So that documentation can be easily found.</p>
Linux Torvalds is being lambasted</a> for encouraging some consistency in Git commits.</p>
But if you read his actual email</a>, the request seems quite reasonable.</p>
The Linux kernel has a well-defined coding style</a> that all contributors are expected to follow.</p>
Have we considered the impact this emphasis on consistency has had on the Linux Kernel's success?</p>
The irony of all this is that consistency is usually done in the name of the "team" or "reuse". But if we reflect a bit, we are mostly just helping ourselves.</p>
We can read and understand our own code in 6 months.</p>
We can find stuff.</p>
We can more easily make changes and improvements.</p>
We have tools helping us.</p>
A little bit of consistency goes a long way in building YOUR Platform.</p>


🎲 Isn't it risky to update your dependencies?
2024-10-07T00:00:00+00:00
This is a common objection I hear when building industrial systems: "We want to
lock things down to a super stable/tested LTS (Long Term Support) release and
then stay on that release for a long time -- it's risky to update dependencies."</p>
Is it?</p>
How often do you update your browser?</p>
Your phone?</p>
How often does Windows or MacOS force you to update your computer OS?</p>
Do you worry every time it updates?</p>
I've run Arch Linux for years and update routinely without worrying.</p>
I update to new versions of Gitea every time they come without a concern.</p>
I routinely update to the latest HEAD of Zephyr on projects during development
and have rarely had a problem.</p>
The same with about every software component I use.</p>
Yes, there are safety-critical control systems that have stringent testing
requirements, but we're talking about complex connected systems that are mainly
concerned with moving data around.</p>
Where security is a concern.</p>
With rare exceptions, modern OSS projects get more stable with each release, and
to a lesser extent with each Git commit.</p>
They have defied the laws of entropy.</p>
How? With OSS workflows, testing, continuous integration (CI), more real-world
usage, more user feedback and contributions, etc.</p>
With good CI, changes don't get merged to main until they are tested pretty
well.</p>
Transparency, community, and OSS workflows are powerful -- really the only
practical way to build complex technology.</p>
The next time you seek the cozy cocoon of an LTS release for a dependency in
YOUR Platform, think about what you might be giving up ... features,
improvements, community connection, and likely also stability.</p>


🏗️ Platforms are for building systems
2024-10-04T00:00:00+00:00
If you are building a one-off, non-connected device, you can get by without a
platform.</p>
This is why so many design-shops don't get platforms -- they are designing
something then moving on to the next project.</p>
But if you are building a connected system, you have a much bigger problem to
solve.</p>
You now have a distributed system, and distributed systems are hard.</p>
You are now living on the Internet with all its associated security concerns.</p>
You have a system that has the potential to do so much more than it is doing
now.</p>
You have a system that has almost unlimited potential to be expanded.</p>
You (potentially) have a platform.</p>


💸 The cost of updating dependencies, or not
2024-10-02T00:00:00+00:00
As developers, we are often lazy when it comes to updating dependencies.</p>
A short-term productivity hack is to not update them.</p>
Leave our Yocto build at an old version.</p>
Never touch go.mod</code> or package.json</code> -- everything is working and I can keep
focusing on coding features.</p>
Don't update our tools -- we don't have time.</p>
... until things break, there is a security problem in a dependency, or we need
a feature in a new version of something, etc.</p>
And then things grind to a halt.</p>
As Khem recently shared,
Maintenance is costlier than development</a>,
so even though development is important today, maintenance is more important --
for tomorrow.</p>
Part of YOUR Platform should be selecting technologies that can be updated
regularly with little pain, and a process to do this.</p>
It is the question of paying a little bit continuously, or a lot all at once
later, and the latter is often so painful that in many cases it is impractical.</p>
Investing in YOUR Platforming compounds positive gains -- neglecting technical
debt compounds negative gains.</p>


🆚 What is the difference between YOUR Platform and other platforms?
2024-10-01T00:00:00+00:00
We all use other platforms -- operating systems, cloud services, middle-ware,
hardware modules, etc.</p>
It is tempting when building a product to piggyback entirely on someone else's
platform (AWS, .NET, one of the 100's of IoT Platforms, etc.)</p>
Society tells us -- you can't host your own service, deploy your own updates,
design your own hardware, implement reliable systems, etc.</p>
But at a small/medium scale, none of these things are very hard.</p>
If you do them, you can simplify and optimize for your needs.</p>
YOUR Platform is partly the ability to leverage other platforms, but also to
build your own -- where you are in control of the critical integration points.</p>


💪 Good enough is not good enough
2024-09-27T00:00:00+00:00
</p>
My grandfather had this sign on his desk for years, and it embodied his
philosophy for work.</p>
He did things right.</p>
He read manuals for his equipment cover to cover.</p>
He understood how things worked.</p>
And because of this, he could fix and maintain machines. Lots of people brought
their machines to him.</p>
Is your product development process trapped in "good enough?"</p>
Do you understand the technology used to build your products?</p>
Have you read the manuals?</p>
Do you know how to fix things when they break?</p>
Is your process maintained? Can you update the pieces?</p>
Platforms are long-term prospects.</p>
And because YOUR Platform is never good enough, over time it gets really good.</p>


⚠️ When things go wrong ...
2024-09-26T00:00:00+00:00
What do we do?</p>
Do we focus on who/what to blame?</p>
Or do we figure out a path forward.</p>
How we are going to prevent this problem in the future?</p>
Not by shaming someone into paralysis, but by fixing the process.</p>
By improving YOUR Platform.</p>
The opportunity for the individual/organization who made the mistake to help fix
the process is a graceful way out and preserves their dignity.</p>


🚀 Do you own your deployment?
2024-09-23T00:00:00+00:00
Saturday morning, I got a call from a customer -- something was not working due
to a bug we had deployed Friday (no we don't have very good tests) 🤫.</p>
The fix was easy, I tested it locally, and then tried to push it to a Git
hosting service we are using, but the Git service was down.</p>
Now what? Our Ansible deployment script pulled directly from git, built the
program, and then deployed it.</p>
While I could reverse engineer the build from the Ansible scripts and do it
manually, that would have taken time and introduced the possibility of another
error.</p>
So I pushed the repo to my Gitea server, tweaked the repo line in the Ansible
script, and deployed the update -- not a big deal.</p>
This brings up a question though -- we don't usually think of deployment as
critical infrastructure -- not a big deal if it is not working -- until you need
to fix something quickly in production.</p>
What if the deployment was wrapped up in some CI/CD workflow that only worked in
vendor X's cloud service?</p>
Maybe simple deployments are actually a better -- a shell script that lives in
the project repo that you can run anywhere. This could still be called by a CI
process for the normal workflow.</p>
All computing systems have the potential to fail -- it does not matter how big
vendor X is -- their stuff can still fail.</p>
Networks occasionally have problems.</p>
DNS can have issues.</p>
Systems get hacked.</p>
No matter how many layers of complexity we pile on top of this.</p>
In networked computer systems, the simplest path to resiliency is the ability to
QUICKLY rebuild systems, whether that is your workstation, laptop, server, or
deployment system.</p>


🔧 What will you improve today?
2024-09-20T00:00:00+00:00
Platforms are all about improvement -- at the personal, team, and company
levels.</p>
One approach is to each day write down something you are going to improve and
set aside a small block of time daily to work on it. Make this part of your
personal process.</p>
This does not have to be something big -- clean your workspace, create a
checklist for something you don't enjoy doing, automate something, write some
documentation to help others on the team, write a test for some troublesome
code, refactor something, improve CI/CD, ...</p>
The internal improvements you do today, while not directly seen by your
customers, will help you deliver something better tomorrow.</p>
What will you improve in YOUR Platform today? Reply and I'll compile a list and
share it in a future post.</p>


What is the role of vision?
2024-09-19T00:00:00+00:00
Yesterday</a>, we
discussed the difficulty in predicting the future and the folly of
"just-in-case" development.</p>
However, we should not discount the need for vision -- the sense for where we
need to go in the future.</p>
Not necessarily specifics, but a gut-feel for where things are heading.</p>
Clearly seeing problems that could be solved.</p>
To understand the changes that are happening in our respective industries.</p>
To realize the potential of new technologies that most don't understand.</p>
This vision does not translate to "just-in-case" feature development, but rather
building a platform that prepares you to meet the future.</p>
Where you can quickly build the features your customers need as you learn the
specifics.</p>
Where iterations cost very little.</p>
Where new features and technologies can be integrated, validated, and deployed
-- confidently.</p>


🔮 Preparing for the future
2024-09-18T00:00:00+00:00
Yesterday</a>, we discussed
a fundamental characteristic of a platform: it will go places you never
imagined.</p>
How do we prepare for this?</p>
There are two ways:</p>

Try to predict where things will go in the future and build specific features
into your platform now -- just in case ...</li>
Build the tooling, processes, and workflows so that you can easily and
confidently add functionality when it is needed.</li>
</ol>
It is obvious which approach works better -- if we try to predict the future (at
least the specifics), we are more often wrong.</p>
We can either try to build the future now, or prepare to meet it as it comes.</p>
The line between these two is often difficult to discern.</p>


✅ The Platform Test
2024-09-17T00:00:00+00:00
How do you know if you own YOUR Platform?</p>
If a customer needs a new hardware interface or connector on a product, can you
easily add that?</p>
If a security vulnerability is found in a piece of software in your stack, can
you fix it?</p>
If a customer wants to use a new USB peripheral in a product, can you release a
new SW version with that driver included?</p>
Can you do all the above quickly, easily, and with confidence?</p>
If so, you likely own YOUR Platform.</p>
Here's the thing -- embedded systems today are general purpose, updatable, and
expandable. That means people will do things with them that you never imagined.</p>
A fundamental characteristic of a platform is that it will go places you never
imagined.</p>
Are you prepared for that?</p>


🤖 What is the role of AI in your platform?
2024-09-16T00:00:00+00:00
We hear much today about how AI is going to do our jobs better than us.</p>
AI is powerful and has access to a lot of information and does a lot of things
humans can't do.</p>
It is really great at writing shell scripts, boilerplate in programs, figuring
out how to use an API, summarizing public information on the Internet, etc.</p>
But these tasks are not the differentiator.</p>
The real value is going deep.</p>
Information found only in books, which AI does not have access to.</p>
Experience found only in people and encoded in platforms.</p>
AI in a product can be useful for extracting information from YOUR data.</p>
General AI tools are platforms in themselves, but probably not YOUR Platform,
and probably not the creator of the primary value of YOUR Platform.</p>
YOUR Platform is for extracting and leveraging the deep -- the value people
need.</p>


🔄 What is the first thing you should implement?
2024-09-06T00:00:00+00:00
When building a new system, what is the first thing you implement?</p>
There are a lot of approaches -- one is to implement a minimal proof of concept.</p>
However, I think an easy way to update software in any part of the system  (Linux edge devices, microcontrollers, cloud applications) is something that should be done as early as practical.</p>
Requiring a technician to go on-site with a computer and specialized software does not count.</p>
Neither does a multi-step manual process of manually building something, scp'ing files to a server, manually restarting stuff, etc.</p>
Deploying new software to any system should be easy and quick.</p>
A single click, or putting a single file on USB disk and rebooting a device, or merging to main and continuous delivery (CD) delivers our new software to the cloud.</p>
We may think nice update mechanisms are for our customers in production, and they certainly are.</p>
But update tooling also helps us during development.</p>
An easy way to update systems encourages us to do it more often, because it is easy.</p>
We iterate faster.</p>
Fixes and improvements are deployed instantly.</p>
We test more systems during development.</p>
We can easily adapt as we learn more about the problem we are trying to solve.</p>
We can involve more non-technical people in development and testing.</p>
What helps us scale in production also helps us scale during development.</p>
Instead of doing update last, do it first -- as an integral part of YOUR Platform.</p>


🚫 No Golden Machines
2024-09-05T00:00:00+00:00
As humans, we can become attached to "golden" machines.</p>
A very expensive bicycle set up just right, a well-tuned tool, a nicely
configured workstation, a server that we have set up just right ...</p>
We like buying expensive things or the iterative process of tweaking things just
right.</p>
However, this generally does not move YOUR Platform forward.</p>
Everything on this earth is ephemeral.</p>
Laptops are damaged. Bicycles are stolen. Tools break. Severs crash.</p>
So then we become obsessed with protecting our golden machines -- locks,
excessive security, redundancy, monitoring, etc.</p>
The problem with being over-protective is that it is time-consuming and hinders
our using something in the first place.</p>
We're afraid to use it in case we might mess it up.</p>
We're afraid to change it because we don't understand the history of tweaks.</p>
What if instead we said: "NO GOLDEN MACHINES"</p>
If our laptop gets run over, we can quickly set up a new one.</p>
If the server crashes, we can quickly deploy a new one.</p>
If we need to use a favorite editor on a different computer, it only takes 2
minutes to set it up.</p>
If our bicycle is stolen, we buy another reasonably priced one and have the
skills to set it up.</p>
YOUR Platform is best built from machines that we can easily use, replicate, and
scale, not golden machines that we have to protect.</p>


🙋 How to help yourself?
2024-09-04T00:00:00+00:00
Did you ever have the experience going back into a project you have not touched
for 6 months -- why did I make this change?</p>
Or did you ever struggle to get all the dependencies installed to build a
project you wrote?</p>
Or did you ever wonder -- will this change break something I'm not thinking of?</p>
Or did you ever question -- how do we actually deploy this now that it is
updated?</p>
Ironically, when you try really hard to help others use your work --
documentation, tests, CI/CD -- you are mostly helping your future self.</p>


🔎 Seeing patterns
2024-09-03T00:00:00+00:00
Patterns are important.</p>
Seeing patterns allows us to simplify things by using common data structures,
re-using code/design, finding solutions to common problems, etc.</p>
Being able to see patterns also allows us to troubleshoot effectively.</p>
But, to see patterns, we need to have history and data in a form that is easy to
see and process.</p>
For troubleshooting, this may be a common place to collect notes -- say every
time a system fails, we record the incident in a single shared Google Doc.</p>
Tagging is also effective -- we might tag a dataset for different types of
events, and then be able to quickly filter on different combinations of tags.</p>
To see patterns in source code, we need to have source code organized such that
it is easy to see and navigate through all of it -- for most of this, this means
a monorepo</a>.</p>
Transparency is critical -- can we easily see things other people are working
on, or is each person or team working in walled-off silos?</p>
Patterns are not invented, but rather discovered.</p>
Platforms are all about identifying and leveraging patterns.</p>
And seeing patterns requires transparency, thoughtful organization of our
assets, and rigorous logging of what happens over time.</p>
Do these things, and patterns will naturally emerge, for YOUR Platform.</p>


🗝️ What is the best tool for private Git repos?
2024-09-02T00:00:00+00:00
Several days ago, I
made a case</a> for using
Github to engage users around your OSS project.</p>
But is this the hammer you should use to drive every nail?</p>
For private projects, I think Gitea is actually better in most respects.</p>
It is very fast, clean, and most of the basic functions work as good or better
than Github.</p>
And you can host all but the largest repos on a $5/mo Linode with unlimited
users.</p>
Groups and permissions are very flexible.</p>
Branch protection works.</p>
In all, it is very good!</p>
I wrote an Ansible role</a> that
works very well for updating your Gitea instance. I've updated through ~70 Gitea
versions with almost no problems.</p>
Gitea may be a better option for YOUR internal Platform -- more details to
follow ...</p>


🐶 Lessons from a 1-year old dog
2024-08-30T00:00:00+00:00
As I was going through my morning routine with our 1.3 year old dog, Reese, who
is very energetic, it occurred to me how well short training sessions every</em>
morning are working.</p>
A 12hr session with her once a month will do virtually nothing.</p>
Improvement is not really what my dog wants to be doing -- she would rather
chase squirrels, dig holes, jump on people -- do something heroic.</p>
Likewise, most of us don't really want to clean our office, write a CI script,
refactor some messy code -- we want to design things, code a new feature, make a
sale, build 1000 widgets, etc.</p>
Improvement is hard and sometimes painful.</p>
And the pain is proportional to the size of the improvement dose we are facing.</p>
If we can break it down into small enough chunks, it is manageable, and actually
enjoyable.</p>
Platform improvements are rarely moments of deep inspiration, but rather just
buckling down and doing what you know should be done.</p>
Discipline.</p>
And this sometimes works best if you schedule a time block every day -- 25
minutes, start the timer, go ...</p>
You only have 25m, get going, now!</p>
And after our morning walk, Reese now walks toward where her leash is hanging --
she is actually looking forward to these short, regular sessions.</p>
Improvement is best continuous.</p>
</p>


🔍 Solving problems or symptoms?
2024-08-29T00:00:00+00:00
The following quotes from the book "The One-Straw Revolution" caught my eye:</p>

The more elaborate the countermeasures, the more complicated the problems
become. ... When a decision is made to cope with the symptoms of a problem, it
is generally assumed that the corrective measures will solve the problem
itself. They seldom do. Engineers cannot seem to get this through their heads.
These countermeasures are all based on too narrow a definition of what is
wrong.</em></p>
</blockquote>
Although Masanobu Fukuoka's book is about agricultural systems, the concepts
apply to systems in general.</p>
With security, do we pile on additional layers of checking, detection, etc., or
do we use a simpler and more secure technology to start with?</p>
For reliability, do we implement elaborate distributed/redundant systems that
check each other, or improve our testing of a single system so that it rarely
fails?</p>
When deployment mistakes are made, do we add more bureaucracy and red tape, or
do we improve the process/tooling such that it is difficult to make the mistake
in the first place?</p>
When something is not working, do we add on layers of complexity to fix it, or
try to simplify and identify the root cause?</p>
One way to improve this is ask "why?" five times.</p>


🐙 Github and why does it matter?
2024-08-28T00:00:00+00:00
For the last</a> two
days</a>, we have been
discussing aspects of several high-quality Yocto BSPs.</p>
Another characteristic of these three BSPs is that they are all hosted on
Github.</p>
Yes, there are many other platforms you can host Git repositories on.</p>
You can easily do your own Git hosting (I personally use Gitea for private Git
repos).</p>
But, for better or worse, if you want to engage with a world-wide community
around a software project, Github is the easiest, lowest friction place to do
that.</p>
Most developers have a Github account, and this means they can easily create
issue tickets, submit pull requests etc.</p>
Some projects, like the Linux kernel, are so popular and well-established it
does not really matter how they interact with the community. They can be rude,
use older methods of handling changes like patches on a mail-list, etc. and it
does not matter. It might even be argued that these "barriers to entry" are
helpful in weeding out the noise in a very large project. Perhaps this is true.</p>
But for most of us, we are not at that scale. We don't have that luxury. Even
most downstream Linux trees are hosted on Github these days.</p>
If users find it difficult to engage and interact with us, they won't bother,
especially the younger generation of developers.</p>
Most developers are (rightly) focused on their projects -- their work, not
yours. Thus, it is an act of generosity if they take the time to interact with
or contribute to a supplier's project. They are doing it because they want to --
not because they have to.</p>
And if users can't easily interact with a supplier, does that supplier really
understand their concerns?</p>
Github has set a new standard for transparency, tooling, and interaction. It is
the lowest friction platform for social coding. And social matters today.</p>
This all has implications for how users/customers interact with YOUR Platform
(if public) and how you select technology for YOUR Platform.</p>


🔁 Tracking upstream and why does it matter?
2024-08-27T00:00:00+00:00
Yesterday, we discussed three Yocto BSP layers that are exceptional:</p>

meta-tegra</a> (69 contributors)</li>
meta-freescale</a> (174
contributors)</li>
meta-raspberrypi</a> (157
contributors)</li>
</ul>
The above three layers make an effort to keep up with upstream developments.
This may mean regularly merging upstream, doing a build, fixing issues --
keeping up.</p>
As a result, they are always ready for the next release of the Linux kernel,
Yocto, whatever.</p>
A little bit of continuous effort is much easier than a monumental effort every
four years.</p>
Why does this matter for small teams/companies?</p>
There are many reasons, but when working with complex open-source software,
we'll eventually need support/help.</p>
And this help typically comes from the community around open-source projects.</p>
And the community is focused on the current development, not a 4-year LTS
release.</p>
Additionally, the latest releases are where security problems get fixed,
features implemented, and value added.</p>
YOUR Platform benefits most from being where the value is being created.</p>


🔧 What makes a good Yocto BSP?
2024-08-26T00:00:00+00:00
As we evaluate technology to use in our platforms, Yocto Embedded Linux BSPs
often come into the mix.</p>
Recently we talked</a> with Matt Madison, who maintains the
meta-tegra</a> BSP layer, which provides Yocto
support for Nvidia's embedded processors, which are increasingly becoming
popular in edge AI applications.</p>
We are using meta-tegra in several projects, and it has been a good experience.</p>
What came out of our discussion is that user/community involvement</strong> is what
makes a good BSP.</p>
meta-freescale</a> and
meta-raspberrypi</a> are other
examples of community-oriented BSP layers that are very high quality.</p>
Chip makers have different priorities and concerns than users. And the only way
to understand your users is to get them involved. And the best way to get them
involved is to work on an OSS project together.</p>
We've run the experiment in the Yocto BSP space for 14 years, and the results
are in.</p>
When evaluating a complex technology like processors that rely on open-source,
if a supplier does include their users in the development process, there is a
good chance they don't really understand your concerns.</p>


⏰ The present or the future?
2024-08-23T00:00:00+00:00
Planning has its place -- we need a vision for where we are going.</p>
But YOUR Platform is best focused on the present, not the future.</p>
Because the future of a complex system is pretty hard to predict.</p>
When we focus on improving our current efforts with refactoring, testing,
automation, documentation, CI/CD, etc. in the simplest way possible ...</p>
We pave the way to the future, by reducing friction in the present.</p>


👥 Platform != People
2024-08-22T00:00:00+00:00
The right people in your organization is absolutely essential -- no question
about that.</p>
And dealing with "people issues" is a critical and essential skill.</p>
But, if your organization is dependent on a star individual and you are in big
trouble when he/she is gone, then you don't have a platform.</p>
We need great people, but if they can't work in the context of YOUR Platform,
then you will have trouble scaling and hit a major speed bump when they leave.</p>
People come and go.</p>
YOUR Platform is what gives your business the consistency and resiliency to keep
going -- smoothly.</p>


✅ What is the simplest form of automation?
2024-08-21T00:00:00+00:00
The simplest form of automation is a checklist or playbook.</p>
If there is something you need to do more than a couple times that involves
multiple steps, a checklist is a great place to start.</p>
Then we no longer have to dread doing a multi-step task because it is easy --
all we need to do is go through the documented steps.</p>
Checklists are surprisingly effective at reducing friction and mistakes. Many of
us do this personally for things like travelling as it is easy to forget
something if we don't.</p>
And the beautiful thing about checklists is they can easily be turned into more
advanced forms of automation (scripts, continuous delivery, etc) because you
have already thought through and debugged the process, which is often the hard
part and where we get stuck.</p>
Checklists can turn into scripts, which can turn into continuous delivery.</p>
Part of YOUR Platform.</p>


📈 How does your product value increase after the initial sale?
2024-08-20T00:00:00+00:00
How does your product increase in value after the sale?</p>
Software/Firmware/Cloud updates.</p>
Software is soft for a reason -- it is meant to be changed, improved.</p>
And sometimes we install "apps" to add new functionality.</p>
And with this improvement comes increased value.</p>
It is kind of like your phone -- with each new OS update, things generally get
better.</p>
Each app you install (potentially) provides some utility.</p>
Why, because modern phones are platforms.</p>
Is your product a platform, or is it at least built and deployed using a
platform?</p>
Does its value increase over time?</p>
At industrial product scale, you don't necessarily need to support user
installed apps, but you need to at least be able to add functionality, fix
things, and deploy these updates -- efficiently.</p>
How? With YOUR Platform.</p>


⚠️ Hack-it, ship-it, forget-it ...
2024-08-19T00:00:00+00:00
Hack-it, ship-it, forget-it ... is a race to the bottom where cost is the only
thing that matters.</p>
The initial delivery is what your customer gets -- forever.</p>
These types of products only depreciate.</p>
And cannot be maintained.</p>

Alternatively, the value of your product can increase</em> over time.</p>
Where you fix problems quickly.</p>
And add features in a timely fashion as needs/markets change.</p>
Where you overcome the laws of chaos and improve instead of decay.</p>
Because you own YOUR Platform.</p>
Which business do you want to be in?</p>


🏆 What is the #1 purpose of YOUR Platform?
2024-08-16T00:00:00+00:00
The purpose of YOUR Platform is to reduce friction in delivering value to your
customers.</p>
We are in business to deliver value to our customers.</p>
If we can do this better, we can deliver more value.</p>
There are lots of ways we can do this.</p>
Automation.</p>
Better tools.</p>
Additional features.</p>
Quality/stability improvements.</p>
Release more often.</p>
Respond more quickly.</p>


🌱 Culture
2024-08-15T00:00:00+00:00
What type of culture do platforms thrive in?</p>
Where ideas are valued and evaluated on merit and truth.</p>
Where initiative is rewarded.</p>
Where mistakes result in process improvements, not scoldings.</p>
Where openness and transparency are the standard.</p>
Where anyone can improve anything in the system -- there is no protecting turf.</p>
Where OSS tools are used when possible.</p>
Where things are continuously improved.</p>
Platform cultures are organic.</p>
We can learn a lot from Open Source projects here ...</p>


🏗️ Three levels of YOUR Platform
2024-08-14T00:00:00+00:00
We can look at platforms at three levels.</p>

Your personal platform is the know-how that you can personally reuse from
task-to-task, project-to-project, and job-to-job. But, it is not real useful
outside yourself.</p>
A product platform may be what a team uses to efficiently build, maintain, and
produce variants of a product.</p>
A company platform is when you are able to reuse know-how, technology, tools,
and processes on one product and leverage that on the next product, and when an
improvement on one product improves another.</p>
All three are valuable, but the higher up the stack you can go, the higher the
value.</p>


👣 What are the first steps?
2024-08-13T00:00:00+00:00
What are the first steps to building YOUR Platform?</p>
Ironically, the first step is not some grandiose new Platform vision, but rather
to set your house in order.</p>
Clean up what you already have.</p>
Fix legacy code (code without tests).</p>
Set up CI/CD.</p>
Implement end-to-end testing so that you can deploy with confidence.</p>
Write documentation.</p>
This paves the highway for efficiently moving forward, instead of slogging
through the mud.</p>


🔄 What is YOUR Platform? (v2)
2024-08-12T00:00:00+00:00
Had a nice visit with Rod</a> and he asked me to
describe what a Platform is in 20 words or less, so here we go:</p>
YOUR Platform is the collection of technology, tools, and processes that
enable</em></strong> you to differentiate and consistently deliver your products and
services.</em></strong></p>
sorry Rod, that is 22 words :-)</p>
We have discussed</a> this
before, but I want to revisit this question occasionally and see if the
definition improves over time.</p>
What got added this time is "differentiate" and "consistently".</p>
Thoughts?</p>


❓ But what if ...
2024-08-08T00:00:00+00:00
In the last post</a>, we
discussed the fallacy of using complex technology when the scale does not justify
it.</p>
But, what if we get a lot more customers, or sell a lot more systems? Should we
not plan for the future now? Use technologies now that will work at large scale?
We don't want to paint ourselves into a corner such that we end up with a system
that won't meet our needs in the future.</p>
Certainly, we should keep the future in mind, but the most important way to plan
for the future is to streamline your processes and development workflow now so
that you can continually move toward the future as it comes.</p>
NOT try to implement the future now. That is costly, and you'll never get it
right that way.</p>
Some decisions are important now and some are not.</p>
You don't need to use Kubernetes now -- you can add that in later. But it is
important to use some type of automated configuration and deployment -- perhaps
a simpler solution like a simple Github Action, or Ansible.</p>
The same with micro-services. It is easy to split things apart later as it makes
sense. But it is difficult to deal with a mess that is prematurely split up.</p>
You can plan by using a decoupled architecture, data structures that are
flexible, and modern programming languages that are efficient.</p>
If you have an efficient workflow where it is easy to make and deploy changes,
the correct architecture will naturally emerge. You will continually simplify
things as you see common patterns.</p>
And simple patterns are what scale to any problem.</p>


🔍 You are not Google
2024-08-07T00:00:00+00:00
A blog article published 2017 titled
You Are Not Google</a>
is a good read. This article makes the case that most companies are not
operating at the size of Google, Amazon, and LinkedIn. Thus, most of us are
solving different problems than the big 5 tech companies.</p>
YOUR Platform needs to fit your problems at your scale.</p>
If one server will work, then you are saving yourself untold pain and suffering
by not pre-maturely building a distributed system.</p>
If one monolothic application can run your product, then you are saving a lot of
hassle not implementing microservices up front.</p>
Keep it simple -- you can move to distributed micro-services later -- AFTER you
succeed at a smaller scale.</p>
And if you succeed at small scale, then there will be plenty of resources to
adjust your architecture later.</p>
The quicker you can get to a MVP, the better, and simple gets you there quicker.</p>
But what if ??? ... stay tuned :-)</p>


💰 Owning YOUR platform is not more expensive
2024-08-06T00:00:00+00:00
Owning your platform sounds expensive.</p>
"We can't afford ops people to manage servers."</p>
"Security is scary."</p>
"We don't want to manage a database."</p>
However, at the scale of most industrial products, none of these things are very
hard.</p>
A couple servers can easily handle all your traffic.</p>
A backup and restore plan can handle failures in most cases. Redundancy and
high-availability are overkill.</p>
Cloud services require management too -- often you end of managing the cost
instead of the fundamental issues listed above. Six of one, half-dozen of the
other -- take your pick. And then you still have to deal with problems -- often
due to the distributed nature of cloud services where the network is the weak
link.</p>
See DHH's posts on
"leaving the cloud"</a>
for an example. They are saving millions owning and managing their own servers
and it is not any more work.</p>
You can hitch up to one of the major cloud vendors and go all in on their
services -- that is what everyone else is doing.</p>
Or you can do something different. Take the technology that is being developed
for the cloud, use the cloud where it make sense, and then optimize for YOUR
Platform.</p>


⚖️ False vs Real Platform Development
2024-08-05T00:00:00+00:00
A common objection to investing in a platform is that it is "too expensive."</p>
"We want to get it right, lock it down, and then crank out our product and make
some $$$."</p>
Many managers and owners are wary of platform development because they have
experienced false</em> platform development -- endless tinkering, the costly
pursuit of perfection, etc.</p>
However, real platform development is focused on shipping as soon as possible
and then improving.</p>
Real platform development is realizing that complex, connected systems are
dynamic, living things that need maintained. They do not operate in isolation.</p>
Real platform developers have the humility to realize they can't see and
understand the big picture up front until they build something and get feedback.</p>
The key to controlling costs is not locking things down such they cannot change,
but rather the opposite -- improving your process so that improvements are
efficient to develop and and painless to test and deploy.</p>


🎆 Do something different
2024-08-02T00:00:00+00:00
If you only build on other people's platforms, how are you different?</p>
And if you are not different, what value are you providing?</p>
We are not talking different for the sake of being different -- outlandish,
rebel, etc.</p>
We are talking being different in the sense of offering something better --
something that no one else is doing.</p>
Something you can do pretty well.</p>
Something that helps others.</p>
And to do this, you need to own YOUR Platform, otherwise, why would anyone come
to you?</p>


🐜 The smaller you are, the more import YOUR Platform is
2024-08-01T00:00:00+00:00
This may seem like a contadiction -- how can small companies afford to own and
control their Platform? Should not this all be outsourced? "Platforms is what
large companies do."</p>
Actually, the reverse is true. Netflix can afford to build on AWS and be locked
into the AWS platform because they have enough volume that they have clout and
the ability to negotiate. Amazon will make sure Netflix gets what they need.</p>
Small companies cannot get this type of support.</p>
Does this mean "small company" should never use AWS? No, certainly not -- AWS is
a good option for many things.</p>
But it does mean that we only use stuff in AWS that we can migrate elsewhere if
needed.</p>
Use managed Postgres instead of some proprietary AWS.db.</p>
Use NATS running on EC2 or managed at Synadia instead of a proprietary event
system.</p>
Owning YOUR Platform does not mean you build all of it, but rather you have some
level of control over it. You have options.</p>
And the smaller you are, the more important this is.</p>


😨 Control is a scary thing
2024-07-31T00:00:00+00:00
Control is a scary thing -- because then we are responsible.</p>
And if we are responsible, then we can't blame someone else.</p>
But if we are not responsible for anything, then what are we doing?</p>
How are we adding value?</p>
Why would anyone want what we are doing?</p>


🎛️ What is control?
2024-07-30T00:00:00+00:00
What is control in the context of Your Platform?</p>
Control is simply the ability to respond to opportunities and problems.</p>
In modern systems we are very dependent on others as there is no way we can do
everything ourselves.</p>
But we can also strategically retain enough control so that we can adjust and
adapt as conditions require.</p>
We have more options today than ever before.</p>
We may think putting all of our eggs in the "big company" basket is the path to
safety.</p>
But if we open our eyes, we soon realize this does not always work.</p>
"big company" may discontinue a service we depend on.</p>
"big company" may have an outage.</p>
"big company" may have security problems.</p>
There is nothing wrong with using "big company" as long as we have the option to
move when problems or opportunities require it.</p>
Lock-in is a good short-term strategy, but is rarely beneficial long-term.</p>


🎯 Why do we need some level of control?
2024-07-29T00:00:00+00:00
In today's connected systems, some level of control of your Platform is needed.</p>
Security problems needed fixed.</p>
Problems needed debugged.</p>
Problematic subsystems need swapped out.</p>
Suppliers go out of business.</p>
Parts go obsolete.</p>
New customers require new features.</p>
We may want to leverage new technologies.</p>
New Open Source projects become available.</p>
We obviously can't control every detail, but we at least need enough control to
adapt and respond to change or the unexpected.</p>
Control is required because we can't predict everything up front.</p>
With complex systems, this is more true than ever.</p>


🏁 The long game
2024-07-26T00:00:00+00:00
Platforms are long games.</p>
If you are building a one-off consumer product that will not be maintained,
improved, or ever change, then you probably don't need a platform.</p>
However, if you are building an long-lived industrial product that will be
maintained, or a series of related products, then it gets very expensive to
start over every time.</p>
To effectively maintain multiple unrelated products, or a system that that is
difficult to update, improve, test, and deploy is a recipe for cripling
technical debt.</p>
The expectation today is things will improve over time, because software can be
updated. It is not really optional any more.</p>
And with the complexity of systems today, the odds you'll get perfect on the
first release are near zero.</p>
Build for the long game.</p>


♾️ It's about not starting over every time
2024-07-25T00:00:00+00:00
Platforms are about reuse.</p>
Not starting over on every new product.</p>
Leveraging what you have done before is one way to scale your efforts.</p>
Reuse takes practice and experience to get it right.</p>
Effective platforms are an art.</p>


🌐 Complex Systems, Simple Platforms
2024-07-24T00:00:00+00:00
The word "simple" in the context of modern systems may seem like an oxymoron.</p>
We can't deny that modern systems are complex -- they need to be to do the
things they do.</p>
The role of a Platform is to reign in this complexity and create a way to
effectively manage and interact with the system.</p>
Platforms narrow our focus to the problem we are solving.</p>
Platforms decouple us from being too entangled or dependent on any one technology
or system.</p>
Platforms are the abstraction on top of complex systems that make them workable.</p>


🔹 Simplicity
2024-07-22T00:00:00+00:00
What is the role of simplicity in your platform?</p>
Simplicity makes it easier for people to use your platform.</p>
Simplicity helps ensure maintainability.</p>
Simplicity moves us toward reliable and secure platforms.</p>
Simple is not easy, actually it's much</strong> harder than complex. But platforms
are a long game.</p>


🏢 What is your Platform?
2024-07-18T00:00:00+00:00
What is a Platform?</p>
There are many ways to look at this. Some might think of AWS, GCP, iOS, Android,
etc.</p>
These are certainly platforms, but they are not your</em> platform.</p>
Your platform is the collection of technology, tools, and processes you use to
do what you do.</p>
This may be building products, delivering services, developing people, etc.</p>


✨ The Beauty of Constraints
2024-07-17T00:00:00+00:00
Constraints are what guide us to good solutions.</p>
Software that is simple enough that average developers can actually understand
it.</p>
A user interface that is intuitive.</p>
A system we can deploy without a dedicated SRE team.</p>
Hardware created with open tools that anyone in our organization can maintain.</p>
An architecture that can scale in unforeseen directions.</p>
Flexible core data structures that rarely change.</p>
Platforms are all about constraints.</strong></p>


😱 The tragedy of no constraints
2024-07-16T00:00:00+00:00
We never have enough time, resources, funding, CPU, storage, etc.</p>
However, think what would happen if we had infinite resources.</p>
We would never get started because we have forever to do it.</p>
With unlimitted resources, we tend to make a mess and get sloppy.</p>
Getting everything we want leads to bloat and lethargy.</p>
Platforms are a long game, and but no contraints encourages short-term thinking.</p>


👋 Introduction
2024-07-15T00:00:00+00:00
Product development is hard.</p>
Technical debt tends to accrue at a frightening pace.</p>
A "Platform" approach helps you reign in the chaos.</p>
And provides a framework to reuse, scale, and leverage your efforts.</p>
This newsletter provides tips to help you in the journey to
"Own your Platform."</a></p>