I recently started working on my fourth book, “Mastering Software Engineering in C++”. This led me to a few thoughts on the role and state of software engineering. My thoughts are still in flux, so I expect to change some of my views in time, but I found it useful to share them and to ask for your opinions. More articles may come.
I’ve been involved in the software craft movement pretty much since the beginning. So an interesting question is: are these two paradigms, software craft and software engineering, compatible?
To approach this problem, I first looked at the history. The Middle Age could be seen as the age of craftsmanship and guilds. Each craftsman (and they were all men), was able to produce unique objects. There was no standardization, because the objects didn’t really need to fit.
However, Middle Ages also had engineers, mostly in the army and for civil projects like public buildings, bridges and such. The two worlds had to work together from time to time, since the large projects required engineering to ensure that the structure could be built and could stand on its own, and craftsmanship, from building and laying bricks to painting the ceilings.
These things changed when factories and the modern assembly line appeared. Factories allowed building things much faster, provided that they were building roughly the same item. As they used to say, you can have your Ford Model T in any color as long as it’s black.
Factories led to the quasi-extinction of craftsmen, due to productivity and economy of scale leading to better pricing. Standardization became the norm, since factories require exchangeable parts and products that can fit or work together.
Factories increased the distance between engineers and workers. Engineers were able to design or improve the production process, build and test new prototypes. Workers had a very specific task in the process, which they had to perform repeatedly.
Enough about history though. Can we learn anything that applies to software from this?
Well, what if history repeats itself for software development? We now have a bunch of programmers who create unique software artifacts – programs, libraries, services etc. Some of them also make them reusable by licensing them (open source or proprietary) and releasing them to the world. Some of these artifacts are very useful and very used, but there’s no standardization. It looks as if we’re in a period equivalent with the Middle Ages of programming.
It seems therefore as though software engineers and software crafters are both needed. We see that crafting software is useful for building components, but we also see the emergence of Site Reliability Engineers who ensure that products work as expected and stand the test of time. Perhaps software engineers should take care of structural integrity and operational needs – concerns currently dealt with by different roles in different companies.
It doesn’t help however that software universities don’t prepare for this type of role, so the industry is compensating for it in various ways. It also doesn’t help that software development has copied a construction metaphor, which doesn’t fit at all the reality of teams that are knowledge refineries.
What about the future? Could we head then towards software factories, or some kind of “industrial revolution”-like change? How would that look like?
This question takes us back to standardization. Factories require a higher level of standardization and exchangeable parts. Imagine being able to switch from one email sending library to another without any problem. Or from one authentication library to another. Or from one MVC framework to another (I can dream, can’t I?).
This standardization may have started already. Microservices allow us to use components written in any technology, as long as they use some standard protocols. Their interfaces tend to use the REST quasi-standard. All that remains is to standardize their behavior – and that’s relatively easy to do with automated tests. Imagine being able to exchange a microservice with another, both taken from the shelf. Imagine avoiding duplicate work by picking up from a shop the microservices that you need and connecting them.
If the change happens in the same way, engineers will be required to define the parameters of these exchangeable parts, to organize or improve the system, and to pick the right parts for the job. Crafting software would be limited to those fringe domains where standards or exchangeable parts don’t exist yet, or to clients who still require a very niche product.
Will this possible future ever happen? I don’t know. On one hand, it seems that many of the domains where software development happens resist standardization. Every company has their own systems, and structures, both leading the architecture. On the other hand, as a consultant who visits many companies, I can’t help but notice the similarities between their work. Standardizing on exchangeable parts seems not to be fundamental, but rather cultural.
There’s so much more to talk about, but I believe it’s time to stop for now. What do you think? How do software engineers fit with software craft? And what could we expect from the future? Drop a comment below to let me know.