Chirag Anand and I had written this note together in 2019, for a client for whom we were consulting. The client had ambitions of building a sophisticated business productivity system, but had no expertise in software development or software vendor outsourcing, and wanted to hand over the requirements lock, stock and barrel to one of the Big Five IT services companies of the land. We persuaded our client to take ownership of the design, and just outsource the development.
I read it today, and look around at our clients, and am surprised at how the essay remains relevant, and how something which I considered obvious still needs to be told.
Some basic concepts
Software design is like the blueprint of a building that is to be built. In building construction, these blueprints would specify sizes and positions of pillars, height of the ceiling, exact layout of rooms on a floor, water/sewage pipelines paths, exact positions of passages, doors and windows, etc. Construction can start only after the blueprints are ready.
The analogy may be extended quite effectively:
- It’s very hard to build a large or complex building without detailed blueprints. Ditto, software.
- It’s far easier to correct mistakes in the design phase than in the build phase.
- Most of the exceptional buildings we see have exceptional design followed by a faithful build. Poor or average design does not build great buildings, however good the build. Ditto, software.
- The skills and engineering knowledge needed for the design phase are far deeper than what is needed for the build.
- There is little use for a high-level design. The real value of the design activity comes from a detailed design and spec, so that the build team can build their house/software to conform faithfully to the design and spec.
- If there is a “vision” behind the building or software system, this is captured in the UX (the visual design, user features) and the engineering design. It’s impossible to capture a “vision” without careful and excellent design. Ergo, all systems which embody a vision must have careful and detailed design to capture it.
- Good design makes the building easier to maintain, which means cheaper to maintain, in the long run. Ditto, software.
- Good design can reduce costs during the build phase. Poor or vague design may lead to sharply escalating costs during build, because each unknown or vague piece triggers errors of fitment or compatibility in related pieces. For example: costs incurred for redoing plumbing of a building because things did not “fit” well.
- Some of the more unusual features simply cannot be built without a detailed design. Unusual curves or structures in buildings and intricate behaviour or algorithms in software are both in this league. The order matching engine of a transaction processing system comes to mind.
In rural India, many small houses are built by simply employing a labour contractor and giving him rough specs verbally. Bridges were built before log tables or the slide rule were invented, with primitive or no engineering design. Should we consider architectural blueprints optional or an avoidable overhead in civil construction today? Should we, in the 21st century, consider that “simple” bridges can be built without engg design for “faster” implementation?
In the software community, some inexperienced software professionals feel that good software can be built by directly engaging with a team of coders and discussing design verbally on whiteboards. We do not understand this point of view at all.
Why good design adds value
Detailed spec reduces project uncertainties
Most important: What is called “design” is actually 10-20% of actual technical design and 80% of detailed specification writing. A project’s progress and time slippages can be tracked accurately only if the detailed “bill of material” in the form of detailed spec is written. This spec converts directly into an accurate WBS (work breakdown structure, which is a fancy name for a detailed and explicit list of tasks for the build phase.)
In our experience, many lingering project slippages which remain open even after 70-80% of the system is built, are because of grey areas about some modules whose spec has been allowed to remain unclear right till the end. The last 10-20% of development can double the project timeline.
Maintainable design saves costs long term
Design done carefully pays attention to code reuse. It also lays down specifications about how the development needs to proceed in order to achieve this reusability.
For instance, front-end programmers are usually careless about their CSS and HTML code and repeat code fragments multiple times. A good design will insist that the dev team creates a single global CSS file and prevents duplicate style definitions. It is almost guaranteed that a dev team will not choose these paths if they are not given a detailed technical design.
Good design reduces development effort
Careful design can often specify a minimal spec of the software to implement the full functionality desired. This is the earlier principle of reuse, but at a higher level. The functional requirements can be analysed to identify how code may be structured to maximise common components. Basically, careful design marks out clear cases for code reuse.
Some components do not get done at all
In our experience, some modules do not get developed correctly unless they are designed and spec’d by talented designers.
We saw one large project where a Very Large Consulting Firm (VLCF) was doing the design and build of the system, but even after 12 whole months, did not know how do build an enterprise-wide search system, and a configurable forms interface. An enterprise search facility would let a user do a Google-like search through the entire in the system database across all entities. A configurable forms module would let a non-programmer quickly mash up a data entry form without code change, for simple data capture tasks. The VLCF did not have a single senior engineer who could design either of these two modules. If we simply hand over the project blindly to such a contractor, we will never know till the end of the project that some key pieces will never be built.
Maintain control of what is developed
The only way to maintain control over what is developed is to create a good, detailed, technical design. Words like “maintainable”, “extensible”, “robust”, etc, are meaningless unless the client ensures that the technical design embodies these qualities. Giving the dev team the design responsibility gives the client no visibility into the quality of the software system. Giving the dev team the design responsibility is as illogical as asking your vendor (in manufacturing) to also operate the QC team which does the acceptance testing of what the vendor is supplying.
To summarise, we find that in the software projects industry, design continues to be a terribly under-rated phase.
(Also published at https://www.linkedin.com/article/edit/7414551360769253377/)
Leave a Reply