{"id":1334,"date":"2018-11-11T01:59:45","date_gmt":"2018-11-11T01:59:45","guid":{"rendered":"http:\/\/sdi.thoughtstorms.info\/?p=1334"},"modified":"2018-11-11T01:59:45","modified_gmt":"2018-11-11T01:59:45","slug":"on-architecture-and-modules","status":"publish","type":"post","link":"https:\/\/sdi.thoughtstorms.info\/?p=1334","title":{"rendered":"On Architecture and Modules"},"content":{"rendered":"

Another long Quora Answer<\/a>
\nWhy is it important to agree on software architecture principles?<\/em>
\nIn a sense, some of this is an update on my thinking on “modularity” (eg. ThoughtStorms:DecompositionByLanguageIsProbablyAModularityMistake<\/a><\/p>\n

Well, possibly it\u2019s only important in an \u201corganizational\u201d sense.
\nIn that people in your team or project need to be aligned in their conception of the architectural principles or they\u2019ll be working against each other or fighting.
\nOTOH, are absolute \/ global architectural principles important?
\nI\u2019m inclined to think that it\u2019s the same as in the case of \u201creal\u201d architecture. Of buildings and cities.
\nIn architecture I\u2019m a big fan of Christopher Alexander (inventor of \u201cPattern Languages\u201d) The point about Alexander\u2019s \u201cTimeless Way of Building\u201d is that it IS universal, but part of the universalism is intense awareness of local conditions. It champions the traditional and \u201cvernacular\u201d. It\u2019s small-c \u201cconservative\u201d, in the Burkean sense. Believing that \u201cwhat people have been doing a lot of around here for last few hundred years is probably a good idea\u201d
\nOTOH, taking a bunch of ideas that have become successful in France and dumbly re-applying them in Brazil in the name of some bogus \u201cuniversality\u201d can be, at best ridiculous, and at worst disastrous.
\nSo there are many heuristics and patterns that are useful, but have to be seen in context. How much abstraction and indirection you might need can depend on the kind of project, the language you are using, the type of user, type of interaction, platform the software is hosted on etc.
\nI increasingly believe that writing good software is like being a good butcher.
\nButchery is about knowing where the natural joints of the animal are and carving along them. This allows you, in the minimal number of cuts with minimal effort, to produce the maximum useful pieces of meat.
\nThe same is true of software. It\u2019s about getting a feel for exactly where the natural module boundaries of your application are. What things need to be decoupled and how much decoupling \/ indirection is needed at each boundary.
\nGetting that wrong is what leads to expensive problems. And being \u201cwrong\u201d can mean both too close coupling of things that should be more loosely coupled with extra layers of indirection. And ALSO too much unnecessary indirection \/ abstraction between things that naturally should be more cohesive.
\nWe tend to teach the importance of putting in abstractions \/ layers of indirection, but ignore the cost of doing it when we don\u2019t need it.
\nPeople sometimes cite Richard Gabriel\u2019s Worse Is Better as a rather vague principle. But read it carefully and a LOT of it is about exactly this problem. How much a function should expose the caller to its own failures.
\nThe intuition we are all taught to cultivate, \u201cDo the Right Thing\u201d is that the module should protect the caller as much as possible. \u201cWorse is better\u201d argues that in this case, the module incurs too high a cost (in complexity) from trying to protect the caller from this failure. And that it\u2019s both \u201cworse\u201d but, in fact, \u201cbetter\u201d to let the caller deal with the failure.
\nThe important point is here is that the general principle is not \u201cnever protect the caller from your failure\u201d. Nor is it \u201calways protect the caller from your failure\u201d. The important lesson is that \u201cbetter\u201d is to recognise what is the right answer in this particular situation.
\nIf \u201cagreed architectural principles\u201d is taken to mean dumb applications of heuristics : \u201calways use an extra abstraction layer\u201d then they are worse than useless. They\u2019re positively dangerous.
\nIf the \u201cagreed architectural principle\u201d is \u201clook at what people have been doing here for a decade, understand why, and follow that\u201d then we\u2019re on to something.
\nThere are patterns that are nearly essential in Java. But pointless in Python.
\nJava and C++ are extremely similar languages in many way \u2026 BUT if you write Java in C++ or C++ in Java you are doing things very wrong.
\nThey are not substitutes.
\nJava IS suitable for writing huge systems in a way which C++ just isn\u2019t. If you try to write the kind of mega-application that Java is used for in C++ it\u2019s going to be horrible. Juggling that much memory allocation by hand is intractable.
\nUse C \/ C++ for small, independent low-level programs, and glue them together in something else (eg. write small independent tools orchestrated within the operating system, or as libraries called from a Python script).
\nOTOH, trying to write the small programs for which C \/ C++ are good in Java is overkill. You are going to put too many abstract boundaries and the cost of garbage collection into something that should be smaller, simpler and faster.
\nOne thing which is particularly egregious about Java (and the C++ heritage it comes from) is that these languages have an impoverished vocabulary for talking about \u201cboundaries\u201d between modules. They see classes and objects as a universal solution for everything.
\nIn fact almost all programming languages have fairly poor vocabularies.
\nWhen we think of software in terms of architecture, or in terms of the \u201cnatural joints\u201d we start to see many kinds of joints \/ membranes between the parts of our systems. With many degrees of permeability. Many features of languages are about this : the inlineability of functions, hygiene in macros, scope rules, \u201creferential transparency\u201d, data-hiding, lazy vs. eager evaluation of function arguments, the access rules for classes, synchronous messages to objects vs. asynchronous messages to actors, go-routines, sockets, the Unix pipe, internet protocols, microservices, integration \u201cat the glass\u201d, integration in the database, centralization \/ decentralization of databases.
\nAll these are issues to do with \u201cwhat kind of boundary is there between THIS and THAT?\u201d How permeable is the boundary? How much do dependencies leak through? What obligations does it incur? What timing commitments does it need? How is the data communicated represented? How are errors checked and controlled?
\nWe recognise this huge variety. But our languages often don\u2019t. Most languages try to hide the variety behind a single principle : everything is a function. Or everything is passing messages between objects defined with classes. Or everything is an actor with async. messages.
\nWhile there is something very attractive about this simplicity and uniformity. Sooner or later you find yourself somewhere where the kinds of boundaries you want between the parts DON\u2019T correspond well to the kinds of boundaries that your simple principle defines. You think actors and immutability are way cooler than mutable objects. And then you try to write a photo-editing program that applies filters to huge bitmaps.
\nSo one architectural issue is that our languages try to enforce a single type of boundary when we want many.
\nThe other is arbitrary and unnecessary boundaries. The biggest culprit is the difference between what is inside the program and what is outside it. Inside the program we have function calls, messages, possibly go-routines and internal queues \/ async. channels. Outside we have async. pipes from the OS. And synchronous socket communication. And a separation of database engine and cache engine. And search engine. And front-end web-server. And client. And server. And microservices. And XML-RPC and SOAP. And Amazon Lambda and similar \u201cfunction as a service\u201d etc etc.
\nOur program is divided into files. And various languages and frameworks insist that they should be divided within the file system in a particular way.
\nOur systems are arbitrarily divided by the underlying architecture of our platforms. Regardless of the natural joints of our applications.
\nI believe that one task for the next generation of languages is to be able to describe a range of boundary \/ membrane types and their permeability, timing and protocol requirements, all within a coherent and simple vocabulary. And where questions like \u201cis this communication synchronous or asynchronous\u201d, \u201clazy or eager\u201d etc. are explicit \u201cparameters\u201d or alternatives within our language.
\nThe other things these languages must do is transcend the \u201cinside the program\u201d \/ \u201coutside the program\u201d distinction. Today we have applications which have maybe a couple of lines of code doing calculations and other \u201cbusiness\u201d logic. And a huge external \u201cextro-structure\u201d, often dozens of config files and MVC separated directories, to represent routing, caching, permissioning etc. architecture.
\nWe need programming languages where the large scale architecture is a first class citizen of the program. Not something which has to be laboriously built around it. And when the program compiles, it compiles to not only object code, but to automated architecture : Puppet or Ansible scripts, Continuous Integration, containerization, orchestration of Kubernetes pods. This is where the DevOps revolution needs to end up. Languages that are as fluent in talking about all the parts of our systems, and all the kinds of communication between them.
\nSo why am I talking about this in a question about \u201cprinciples\u201d?
\nBecause software as an art \/ discipline \/ profession \/ culture advances when we take informal ideas and turn them into formal code that can be executed. Heuristics and \u201cgood practice\u201d become patterns, become libraries, and ultimately become language features.
\nArchitectural principles will ultimately be recognised and \u201cagreed upon\u201d parts of our practice when they finally become part of the languages we use everyday.<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"

Another long Quora Answer Why is it important to agree on software architecture principles? In a sense, some of this is an update on my thinking on “modularity” (eg. ThoughtStorms:DecompositionByLanguageIsProbablyAModularityMistake Well, possibly it\u2019s only important in an \u201corganizational\u201d sense. In that people in your team or project need to be aligned in their conception of […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[24,287],"class_list":["post-1334","post","type-post","status-publish","format-standard","hentry","category-opinion","tag-architecture","tag-modules"],"_links":{"self":[{"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=\/wp\/v2\/posts\/1334","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1334"}],"version-history":[{"count":0,"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=\/wp\/v2\/posts\/1334\/revisions"}],"wp:attachment":[{"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1334"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1334"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1334"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}