Just another acronym TBD

Working at scale I am constantly aware of how much we decide upfront. Before it gets anywhere near a team a lot of time goes into looking at what it is, what will change and who will be involved. In some cases, whole designs are considered before a team even see's it.

On the face of it, there is good reason. It costs a lot of money to build things: better make sure it will give a good return. Things taking longer costs even more: better make sure we know what we are getting into. It takes a lot of people to build stuff: better make sure we know who is involved so we can make sure they can actually make it.

We loose something important in doing this - our competitive edge. Every week we take in understanding the risk and cost is another week our customers don't have our product and our competitors have a window of opportunity.

Working with teams, I am aware of how much we assume. We build architectures based on our understanding at the time, which often include a lot of assumptions. I like assumptions because we can actually prove them out - but we usually don't.

We often build more that we actually need since we don't or can't prove out these assumptions. After a while of a service running, I have seen teams reduce the system in lots of different ways. This can sometimes be by removing caches, services that store data or using different scaling patterns that we discover over time.

We would benefit massively from building something small because we can see how it responds in the real world. We get feedback using monitoring and telemetry to understand what is going on and we can make better decisions on it's design and architecture based on that information.

But we need to start somewhere...... so how about we just focus on getting this data in the easiest way we can.

Imagine we took our best guess at an architecture that would suit our intended audience and built the services and deployed them. We make sure we add no logic whatsoever, only the bare minimum to allow the system to interact and we focus only on the monitoring and telemetry.

We can then load test this in a live environment and we could call this a 'best case' system. Without the logic this is the fastest it could operate - anything we add will slow it down. See it as an extreme case, where we are looking at the skinniest skeleton we could possibly get away with.

We can load the system and see what happens. We could also introduce waits in areas we can anticipate more logic and see what happens under load. We can add more monitoring where we have poor visibility. We can stub 3rd parties and make them 'misbehave' to see what happens.

Since there is not much too this, we can quickly move things around and see what happens and fix problems we can see - essentially we searching for a baseline test that we are happy with before we add anything else. We can easily remove things that don't have a measurable impact in the scenarios we are testing.

Since we don't have an logic there is no need for unit tests, meaning changes can be quick. As it does not do anything and contains/accesses no data, it is benign in a live environment so should not constitute a security risk either.

When we do start to add logic, we have a baseline we can compare to and a suite of tests that we use to monitor KPIs that we can actually monitor from the beginning. We also have an architecture that is better than a guess - it's already got some data to support why this is the right place to start.

I call this Telemetry Biased Design but it just sounds like a cool way of making sure you starting with just the right amount of architecture to solve the problem you have.

In full disclosure: at time of writing, I have never tried this. I am no longer an engineer and I work with smart people who get things done in their own way. It's just an idea.

Lessons from my favourite angle grinder

I am a DIY nut. I do a lot of jobs around my house and there are some tools which are indispensable.

So when my Makita Angle Grinder stopped working, a little part of me died. The switch had not felt 'right' for a while so I suspected that it had finally given up.

My beaten, battered, Makita GA9020

It is the single most used tool at the moment since I am building patios, walls and outside kitchens. All of this involves stone and brick, which always needs to be cut at some point. Usually many points.

This tool was particularly expensive. Makita tools generally are. I could have bought a much cheaper version from another manufacturer. This was a significant capital investment for me at the time. I spent more since I had previously bought cheaper tools which had then failed.

I hate throwing things away. And I still need the tool and now I have to buy another one! If I buy another cheap one then the same will likely happen since I will be doing similar if not more complex or heavy work. I can always buy a better (more expensive tool) but my total investment will be higher than it could have been.

It is a popular belief that anything can be repaired. This is not true. Many things are not designed to be repaired and the cost of repairing them would be equivalent to replacing the item with a new one. Cheap angle grinders are like this. If you can find the parts, which is hard in itself, they won't be cheap. Something like the switch might be up to a third of the price of the device. LCD TV's are another good example - replacing a panel can easily be the same or more than the cost of the TV in the first place.

There is also a noticeable difference in build quality. My Makita feels solid and weighs a tonne. It is intended to last and perform it's task even in harsh building environments. Most trades people I know wont turn up with shoddy tools. If one fails, you have lost a days pay for starters. If they do go wrong, you want to have them up and running ASAP for minimal expense.

My angle grinder did have a problem with the switch. I found a new one for under £20 on ebay in about 10 minutes. Replacing it was pretty painless - 4 screws, take off case, swap power, swap motor leads, transfer fuse unit - it took about 15 minutes and most of that was just prising off the case.

I also noted that the brushes can be replaced and the motor be taken out for servicing. The Makita is designed to be fixed and repaired, it was a part of the design. The most likely parts are trivial to replace and required only a screwdriver. It even comes with a guide on how to perform routine maintenance.

As it is designed to be repaired, parts are easy to come by. I could have got my switch for about 30% cheaper if I chose to wait for a generic part from Estonia. We see the same thing happen with car parts - if you are unlucky enough to own a car that was not popular then generic parts might not be made for it, meaning you have to buy the more expensive genuine parts from the manufacturer.

So how does this apply to software?

Good software design should not need a specialist or the original developer to fix or maintain it. Anybody should be able to figure out what to do if we need to play with it.

Investing time to make sure something can be fixed or changed easily will pay up in the future, not now. Identify where that investment is needed most - acknowledge that some components or services wont't change that often.

Making it complex does not help the next person. Making it as simple as you can is more difficult than it seems.

Understand your investment. If this were to stop working how much would it cost you? If you need it to keep working, invest in making sure it can be maintained, diagnosed and fixed.

If you are building just enough to get the job done, understand that more investment will be required in the future if you still want that service.

If you do decide to cut corners, it will end up costing you more. Understand how your whole product cycle works - from cradle to grave - so you can make the right decisions.

To help people out we can produce simple documentation that helps the next person. It does not have to contain everything but should include what we have already thought about.

Think about the components or services you are using and try to use ones which are commodities already, meaning they are well supported and understood.

Understand your environment and build appropriate for that. Building something for 100 users will not be the same as something for 100,000 users.

Routinely check your solution and make sure it is fit for purpose. Don't wait until it goes wrong to fix it.