Tuesday, August 28, 2018

What goes into a technical design?

Part 4 of a 4 part series (originally posted 20 Aug 2018)

In the first three posts of this series we decided we need a technical design, figured out who would be doing it, and how we’ll be sending it out and getting input. But what is the content of that communication (for example, what sections would we put into a written design document)?

What to include will vary depending on your organization and the needs of a particular design. For an early stage startup, anything relating to scaling and operations may take a back seat to “am I building something people want and how can I most quickly validate my hypothesis?”. For a company in a highly regulated space, there may be a lot of requirements specific to your field.

The same applies to an individual design. Does my design concern a server with a high or low need to be available? Does my design concern data which is sensitive? Does this design change anything related to this topic? (If not there’s probably little to say on the subject). For that reason, I’d suggest treating templates (including this article) as guidelines, and omitting sections which don’t seem relevant. One of the fastest ways to lose an audience is to include a bunch of material that you aren’t very interested in (and probably didn’t do a very good job with). And of course to prioritize everything is to prioritize nothing, a good motto in a variety of contexts.

So, what might we include?

Goals and non-goals
These are perhaps the most important sections. If you can figure out what your design achieves and what you are leaving for another day or deciding is not worth doing, you are well down the path of figuring out how to do it.
Description of the proposed solution
What changes will we make to code, data, networks, and hardware? How does this design achieve the goals? Give enough detail that people can see some of the implications of various choices, but try to avoid the kinds of details which can easily be fleshed out during implementation.
Security
What data is stored and sent where? How is access controlled? If cryptography is involved, how are keys managed and have we chosen appropriate algorithms? Are some parts of the system isolated from others and if so how?
Reliability
Is there redundancy? What are the consequences of network outages? If data is stored in a master-slave setup, how do we elect a new master? If data is written multiple places how do we reconcile them? Are there rate limits or other ways of keeping a problem one place from cascading elsewhere?
Capacity
What is the expected load on the various systems involved? Does load ramp up gradually or do we expect a sudden spike in traffic? What needs to be handled manually and is there sufficient staffing to do it?
Monitoring
Do we need to report new metrics? How will we know about errors?
Data analytics
How will we measure usage of the new functionality? What kind of analysis might we want to do?
History
Has the company considered this problem before? What previous decisions got us here? If there are documents describing previous designs, I tend to just link to them rather going into a lot of detail about what has gone before.
Storage
What database(s) are involved (new or existing)? What changes in database schemas are required?
Interfaces between systems
Defining these can help clarify the design and is particularly helpful if one of the functions of your design is to coordinate between different teams or companies who are responsible for different pieces.
Alternatives
How else did we consider solving the problem? Why did we choose the solution we are proposing?
Open questions
This section is particularly helpful if you know certain topics are controversial or warrant further discussion. As questions are resolved, move items from here into the main design section or the alternatives section.
Rollout
In what order are we building this? Are we shipping it continuously? In a series of phases? Is it rolled out selectively to certain users?

These questions can be taken as a template for a design document, but they also can be used to figure out who to go talk to, what to put into a presentation, or what anticipated questions to prepare for.

There’s a lot in this series of blog posts about things to do: Did you talk to X? Did you consider Y? What if we did Z? And those are all very helpful up to a point. But only do those things which seem necessary for your particular organizational culture and problem you are trying to solve. The purpose of all these suggestions is to help you build things and solve problems, so as you go, don’t be afraid to keep asking yourself and others: Are people on the same page now? Is this enough specificity to build this? Is my technical design sufficient for what I need?

No comments: