Transcript
Stefan Tilkov: Welcome, listeners, to a new episode of The CaSE Podcast, another conversation about software engineering. In this episode I talk to Simon Brown, author of Software Architecture for Developers, who I happened to talk to for the GOTO Book Club. They kindly allowed us to record the full audio; you can view the video that's been distilled down to the essential parts on the GOTO Book Club website, and I'm looking really forward to your feedback regarding this episode. Let's dive right in.
Stefan Tilkov: Welcome, Simon. It's great to talk to you. You know I enjoy that anyway, but in fact using it for dual purposes - actually, triple purposes, in fact... I'm getting a nice conversation and I also get an audio track for the CaSE Podcast, and we get a video track for the CaSE Podcast, and we get a video track for the GOTO Book Club. This is just about perfect. Simon, why don't you start by telling everyone who you actually are and what you do for a living?
Simon Brown: I'm an independent consultant, mostly specializing in software architecture, so my background is as a software developer, building software either for or without customers; I used to work in a consulting environment. Now I get to do two things - I get to hopefully one day again fly around the world and run architecture workshops, and I also have another company called Structurizr, which is a set of tooling to help people create software architecture diagrams, essentially... So that's me.
Stefan Tilkov: Very cool. Okay, so as I'm a software architect myself, I will try to just ask questions, but I'm very sure I'm going to just have to agree or disagree - most likely agree - with some of the things you say... So let's just start and discuss a few things.
Stefan Tilkov: First of all, the tools and your books which we're going to talk about as well - they all sort of seem to make a point of making their target audience developers. It's as if you want to draw attention to the fact that architecture should matter to developers as well. Can you elaborate a bit on that? Why do you think that is, or why do you think people could think that it's not important for developers to consider architecture?
Simon Brown: So the whole software architecture for developers thing really came about -- so when I worked for consulting companies, in order to scale and grow a consulting company, you need more teams to go service more customers. And for every extra team you need, you need more tech leads and more architects. So basically, a lot of this stuff came from me teaching our developers how to think about software architecture, to do software architecture, to communicate software architecture... And when I was going through this myself earlier on in my career, I didn't really find the existing literature helpful in terms of "I'm a developer now, and I've been thrown into an architecture role. What do I need to do?"
Simon Brown: There were lots of books out at the time, you've got all the SEI in Practice books, and so on and so forth... But I just found they were very research-based and academic-focused, and it didn't really give me a "If you're an architect, this is the sort of stuff you should do." So that was really where my focus came from, and hence the kind of software architecture for developers theme.
Simon Brown: To answer the same question from another angle, I guess, when the Agile Manifesto came around in early 2000, we saw lots of people jumping on that, which is great, and there's a lot of benefit that came out of the Agile movement... However, a lot of people started dropping some of the more design-focused, documentation-focused, architecture-focused techniques and practices and processes... And I wanted to figure out "How do we get this stuff back into the way that teams work, without seeming like that horrible dictator-style architect that people don't like?"
Simon Brown: And also, I want to reintroduce some of the existing ways of working, but with a little bit more of a developer focus, so that developers potentially pay more attention to them, if that makes sense. So that's really the kind of developer focus, I guess.
Stefan Tilkov: So do you think architect should be a role, or is it simply a set of tasks that somebody just has to do?
Simon Brown: Kind of both, I guess... If you look back 15, 20+ years, every team would probably have an architect on the team who would do all of that stuff, and they would really get involved in the code and they would tell people what to do, and we learned that that's not a contesting way to work.
Simon Brown: Those tasks still need to be done, but perhaps nowadays with our more modern Agile approaches, they're much more collaborative; trust is an inherent factor in many teams... Maybe we don't need that single person who's looking after all of those tasks, and that's why I like to think about it as a role. So it is a role that the collective team needs to do. It doesn't need to be one person, it could be many people, but that set of tasks does still need to be done in order to hopefully get a better, more successful end result.
Stefan Tilkov: Yes, I think I completely agree with that assessment. After all, if you sort of have to do it, because otherwise what you end up will just be a matter of -- whatever happened to you, it'll be sort of accidental, because some people make decisions and you just end up with some sort of architecture that may or may not be what you wanted.
Simon Brown: Yes, it's either the kind of accidental structure thing, so a big ball of mud, or some of the quality attributes tend to get forgotten about, because they kind of fall between the gaps between the people like performance and security and scaling... It's like "Oh, I thought someone else was looking after that stuff", but no, it's our job collectively.
Stefan Tilkov: I've also seen people build something that works perfectly well in some architectural dimension. The problem was just that that dimension was not one anybody really cared about. I mean, nobody wanted that system to be that super-scalable, it just was a lot of fun to make it -- I mean, I can't say that I've seen too many systems that were too scalable; that would be a bit of an exaggeration as well... But of course, as you mentioned, the quality attributes are something that you need to take into account to make a decision what actually matches those requirements... So you have to know them first, and then people often actually don't. Yes, I completely agree.
Stefan Tilkov: So you mentioned the Agile Manifesto as sort of an influence, sort of something that led some people (I think we both agree) mistakenly to dismiss all of the architectural ideas and all of those things... But I think some of the reasons for that were that it was very often at least associated with doing a lot of work upfront. Somebody designs the system, builds the architecture in the form of a lot of diagrams, and prose, and big documents, and then just hands it off to somebody else to actually build it according to those guidelines and decisions and rules that are in the big architectural master document. Obviously, that is not a good way to build things... But maybe the other extreme isn't that great as well. So how much architectural work needs to be done before you actually start coding?
Simon Brown: Yes, there's a great quote by Dave Thomas, he says "Big design upfront is dumb, and doing no design upfront is even dumber." And it's that kind of flip-flop from one extreme to the other which is exactly what I've seen teams do over the past 20 years now... So yes, I completely agree - doing too much locks you down, it's too rigid, you spend lots of time getting there, you spend lots of time potentially solving problems you're never going to have... Then you have to order and make sure all of the people are doing all of the things according to the document you wrote four years ago, and the world's moved on...
Simon Brown: I think a lot of people misinterpreted the Agile Manifesto, and because the Agile Manifesto doesn't explicitly talk about doing upfront design, a lot of people, I think, have interpreted that as the Agile Manifesto says "Don't do upfront design." And if you read things like Extreme Programming and if you read some of the principles, it is easy to kind of get that view of what the Agile Manifesto is saying, but I don't think that's the case.
Simon Brown: What I find amusing about all this is when I worked for consulting companies back in the early 2000's, and the Agile Manifesto was kind of coming out, I looked at it and I thought "Well, that's kind of what we're doing anyway." I was very hesitant in doing the big upfront design, because I knew that -- well, first of all, it's really boring. I did work on a few projects prior to that, where we do like six months of just doing design, and using Rational Rose and putting UML diagrams into Rose... And it was very interesting from a kind of domain analysis perspective, but we wrote zero code, and I just got bored of doing that. So that was never really my approach anyway... My approach was "Let's get the big building blocks in place. Let's understand the major driving factors, the major important quality attributes like security and performance and so on. And then let's build a design around that, and then we'll fill in the blanks as we go along." So when the Agile Manifesto came in, I was thinking "Well, that's what I do anyway."
Simon Brown: So from my perspective, nothing changed too much, but I guess, depending on where you were coming from at the time, your perspective was very different, and I think that's what led a lot of teams down the no-design route... And unfortunately in some cases, fortunately in others of course, those same teams are now realizing that was potentially a bad idea, and they're now starting to go back and think about "Well, what sort of design should we do upfront?" How much should we do, and what sort of architecture process should we put around making sure that people are doing the things we think they're doing, to fit into the constraints of the environment and the guidelines and the principles and everything else.
Stefan Tilkov: Basically, it seems the answer to that question is you should do just enough upfront design. Just the right amount. That's a very easy answer - you just do the right amount of design.
Simon Brown: Well, it sounds obvious, but it's super-hard to quantify.
Stefan Tilkov: Yes, indeed. But I think we agree on that as well. There was an interesting thing that you've just mentioned, which is this impression that sometimes when you read a book, or some sort of paper, or some blog post, or whatever - you look at it and you say "Well, that all sounds totally obvious. That's what we're doing anyway, so why is everybody making such a big fuss about the whole thing?" But I've come to think that that, in some cases at least, is the sign that this is a very great contribution, a very valuable contribution, because it actually articulates something that should be obvious, and maybe is obvious to you as a reader, but actually it may not be. To be perfectly honest, I had some of those feelings when I first read your book. That's one of the reasons I said we'll probably agree. "This all seems very reasonable. It seems so reasonable", and I keep saying "Yeah. D'oh... Sure. That's what you should be doing." But I still think they're excellent books [unintelligible 01:12:06.23] are great, because they actually provide a lot of value to people who may not have that experience, who may not have arrived at those conclusions, and they get it in a form that actually makes it tangible, and gives some actual guidelines of doing things.
Stefan Tilkov: I think what I'm trying to say is that if you read a book or if you're looking at something and it seems kind of like what you're doing anyway, that's not a sufficient reason to dismiss it. It may be an excellent thing, just what you need to convince other people.
Simon Brown: Yes. And of course, there are a lot of people now entering the industry, and they didn't see the stuff that pre-dated the Agile Manifesto; they've kind of missed that whole journey, and now they're just getting the end bit, where it's "We move fast, we break things, we don't do design", and like, "Hang on a second... That's not what's happening here. We should maybe step back and plug some of the gaps that these people don't have in terms of knowledge and experience."
Stefan Tilkov: Yes, good point. One of the things that you emphasize a lot is the role of visuals, the role of diagrams and visualizing architecture. Can we talk a little bit about that? Why is it not enough to just sketch something on a whiteboard somewhere and then wipe it away 30 seconds later?
Simon Brown: That is one of the big things that the Agile Manifesto and the Agile movement has trended teams towards... It's the "You don't need these big design documents, you don't need to do these big, heavy upfront design processes, and you don't need to use expensive tooling. Grab a few people, draw some sketches on a whiteboard, have that conversation, and then erase it." And the emphasis here is the values and the conversation. And I completely agree... If we've got a bunch of people around a whiteboard and we're talking about different ideas and different designs and different approaches, and we're assessing trade-offs and stuff - that's a super, super-valuable conversation, and we should definitely keep having those things.
Simon Brown: The problem I've seen is that once people erase these diagrams, a lot of that knowledge gets lost. Now, sometimes people take a photo and they save it in Confluence and that's all good, but...
Stefan Tilkov: But nobody ever looks at that.
Simon Brown: Right. Nobody looks at it because nobody can find it, first of all, because the architect quit three years ago, or something... If the diagram doesn't make sense from a visual perspective, it's very easy to interpret what you think the diagram means, and that might be completely different to what everybody else thinks. I'm a fan of whiteboards and I definitely recommend teams use whiteboards for doing upfront design... I just want people to add a little bit more - and I have to be careful how I phrase this - formality and structure around what they're drawing... Because it's far too easy to draw two boxes on a whiteboard, stick and arrow between them, and that can literally mean anything. I want the stuff that we're drawing to have a little bit more meaning.
Simon Brown: Now, of course, you might ask "Well, why don't we just use UML?", and that's a very good question. Up until 2004-2005 I was a big UML user. All of the documentation I did, all of the sketching I did was using UML. But it went out of fashion, and people dropped it like a hot potato in the mid-2000(ish) to 2010 years... And it's not really bounced back.
Simon Brown: When I got into teams and I say "What sort of notations are you using to draw your architecture?" they literally just say boxes and lines on a whiteboard. There's no mention of UML. Some of the people who have now been through university and college and their apprenticeships - they're not being taught UML. So again, you have a whole bunch of people in the industry who have missed out on all that stuff, and they just think "Well, let's draw some random boxes and lines, because that's what everybody else does."
Simon Brown: To get back to your question, I think visuals are super-important. I think you can literally hang all of the other stuff related to the architecture around a good set of visuals, because that good set of visuals allows you to tell stories, allows you to have design discussions, you can make design decisions, you can assess trade-offs, and it just opens up all of that information to a much wider audience. So that's why I place so much importance on the visuals.
Stefan Tilkov: Do you think that the role or -- let me think of how I want to phrase this... Because everybody's working remotely these days, do you think things have changed because tools are more important, and the things that we create just are more structured because the tools typically offer some structure as a default?
Simon Brown: I think, and especially if you look at remote working and especially with the whole pandemic thing, what teams have done - and I've definitely seen this - they've taken their whiteboards in the office, because they're not allowed in the office anymore, and they're using tools like Miro because they can basically fire these things up -- you can get a bunch of people looking at them, you can all interact and collaborate at once... And Miro is a fantastic tool, and I also use it myself for other things, but it's not going to give you structure. If you approach a whiteboard, it's not gonna provide you any assistance in drawing an architecture diagram; it's not going to help you explain what types of abstractions you should be drawing, and the semantics of the visual language you're using. Miro is the same - it's just a great way to draw boxes and arrows collaboratively. It's a collaborative online whiteboard.
Simon Brown: So I think we're missing a trick of tooling here. And again, I have to be careful what I say here, because it's very easy to interpret what I'm saying as "We should go back to doing what we did 20 years ago, with Rational Rose, and have lots of rules and semantics, and it was all very, very precise." Maybe that's too far, so maybe we need to rein that back a bit, but have some tooling that actually provides some assistance. It's the same with AutoCAD now - if you want to sketch out a building blueprint, you don't fire Visio up; Visio is going to offer you no assistance at all, apart from "You've got this box and this box, and you can put them together and group them." You're going to use some of that AutoCAD, because it's going to provide you assistance and rules and different things like that. So yes, I think there's still some more work to be done there.
Stefan Tilkov: Yes, I think what people might be missing is that -- the assumption somehow seems to be that if you use something like UML or any sort of structured CASE tool, whatever it happens to be, then in the end you might have to specify things at a level that is very close to programming. So people fear that they have to go into all of the details and fulfill all of the requirements of something that could then be automatically turned into code... Which is not something that you have to do. I mean, you could possibly do that, but whether it's a good idea or not is probably beyond our scope today, even though you could do that.
Stefan Tilkov: You can just as well use a tool like that to model something at a very high level - just draw the high-level structure of your system, just with a little bit more meaning, more semantics as to "This is what this kind of line means. If it's a dashed line, it means something different than a solid line, because that's why we make it a dashed and not a solid line." You could invent your own notation...
Simon Brown: Yes, totally.
Stefan Tilkov: ...and then stick to it. That would be perfectly fine. But you could also use one of the notations that's already there. For example UML, which of course gives you a ton of variation and lots of ways to customize it. And because it's a super-powerful, super-customizable tool, it may end up being super-complex and way too powerful for what you want to do, as usual... And then you might end up with alternatives, which I'm sure we're going to talk about very soon.
Simon Brown: Yes.
Stefan Tilkov: Let's talk a bit about your efforts here. I think there are two things that we should talk about. One is C4 and one is Structurizr. Maybe we can start with the C4 model and you can go into a bit more detail what that is and how it differs from UML.
Simon Brown: The C4 model is essentially a formalization of how I've always drawn software architecture diagrams. So in that kind of post-UML phase, where the organizations I was working for and with didn't want to use UML, and so they kind of threw us back into Visio. I had a specific way that I drew software architecture diagrams. So if I'm building a software system, I want a box in the middle of my diagram saying "This is the thing I'm building." I want to list out all the different types of users, their roles, the actors, the personas, and then I want to also show my key system dependencies, because of course, as you well know, anywhere you've got a system dependency, there's some sort of interface, and there's always some sort of risk associated with doing that.
Simon Brown: So that was really my starting point for drawing out software architecture. And then I wanted to kind of zoom into the system I was building and show deployable, runnable things. If we were building a web application toolkit or database, I would literally draw two boxes. "Here's a web application, and here's a database", with an arrow between them. And again, this is really reflecting how I thought about the architecture from a developer's perspective, as someone who ends up coding on those sorts of projects.
Simon Brown: When I started teaching people how to use software architecture, I was quite focused on getting people to do the stuff we talked about before - understanding architecture principles and guidelines and constraints and quality attributes... And I had a little case study in my workshop where people would group together, they'd go and do a small amount of upfront design for like an hour, and the output was one or more diagrams to describe their solution, essentially. And after doing that a bunch of times, I realized that I couldn't understand any of the diagrams, and neither could anybody else in the workshop. So now I thought to myself "Well, I know how I do this... Why don't I figure out how to teach that to other people?" Because I just naively thought that everybody else did the same thing, but it turns out they don't.
Simon Brown: So that's where the C4 model essentially came from. It was formalized in the latter half of 2005 up to 2010, something like that... And the C4 model is a hierarchical set of diagrams to describe software architecture. The C4 stands for Context, Containers, Components and Code. So the context is a system context diagram, and it's basically what I've described already - it's a very high-level diagram. It's a single box in the middle, representing the system you are building - maybe like an interbanking system; it's got the different types of users and actors and roles and personas around it, with arrows, using the system... And then a set of other boxes representing your system dependencies.
Simon Brown: Then you zoom into this system boundary and you show what I'm calling containers - and I'll come back to that in a second, because it's a little bit controversial. So I call these things containers, but basically, they are deployable and runnable applications and data stores. So if you're building an Angular single-page app that's sending JSON across the internet to a backend Ruby on Rails app which is sticking stuff in a MySQL database, you draw those three boxes and the arrows between them. So again, it's really reflecting the realities.
Simon Brown: And then if you want to, you can zoom into an individual application or data store and show components inside them. What do I mean by component? For me it's just a grouping of stuff. A grouping of stuff, nice, well-defined interface. The components often or usually relate to how you are structuring your code from a higher level. So if you think about your codebase as a set of components in a layered architecture, then your component diagram essentially represents boxes and layers. If you're doing ports and adapters, or hexagonal architectures, or a package by feature, then that's what your component diagram is essentially showing you. And then if you want to get really into detail, you can zoom into an individual component to show the code inside it.
Simon Brown: There's a few things to unpack here. Number one, why did I choose the term "container"? I think I got there before Docker. That doesn't make this right, but I tried to find a term that didn't have many associations, and I obviously failed very badly... I didn't want to use the word "process", because that's not what I was trying to show. I didn't want to use the word "application", because that's not what I wanted to show. At that time I was very big into J2EE and I just liked that container metaphor. It's like "Here's a thing that runs and stuff goes inside it." So that's why I chose the term "containers". And although the C4 model has the number 4 in the name, I definitely don't recommend doing all four levels. For most uses and most teams, the top two levels are more than sufficient, and if you want to get more detailed, you can do. Then you get into the whole question of "Well, should you sell automating diagram generation?" and so on and so forth... But again, that's a bit abstract for this discussion.
Simon Brown: So that's the C4 model in a nutshell. It's a hierarchical set of diagrams to describe software architecture, at different levels of abstraction, and then those different levels of abstraction allow you to tell different stories to different audiences.
Stefan Tilkov: Okay. So my understanding - and I'll be interested to see what you think of that - is that the major difference between that and what UML does, for example, is that UML is far more generic. It's got all of those things, and somehow you could absolutely 100% use UML to do the exact same thing, it just doesn't recommend anything. It just gives you tons of options and you have to decide what to do with them.
Stefan Tilkov: You could use a component diagram, or some sort of package diagram where the packages have sub-packages, and you can drill down all those in the same way, finally ending up at classes, or whatever it is... And I've seen you actually recommend combining the C4 model with other diagram types that are in UML, like a sequence diagram or a collaboration diagram, so it could totally do that.
Stefan Tilkov: What I do see as the major thing that people find attractive, that people find useful beyond the UML part is that you actually made some decisions; not all of them, but some decisions, to say "This is the useful level. Not any arbitrary number, it's 4. Not any level, it's these." This may really be a stretch, but I want to run this analogy by you... It reminded me, weirdly enough, of something Eric Evans did in the DDD book. Because even before the DDD book -- and that obviously was also one of those "Oh, I'm doing that already" while I'm reading. "This book's for me." – he sort of proposed a number of what in UML I would call stereotypes. You know, "These are the kinds of things beyond just classes. We have value objects, and we have entities, and we have services, and we have those things..." And he made a very explicit choice to include these (whatever) ten things. And it's not a real difference from just saying you have stereotypes, and you can actually customize the generic UML class diagram, or the class concept to mean something more specific... But still, it's immensely useful, because it gives people more to hang on to; it gives them a starting point, as opposed to this super-generic thing that then somebody else would have to customize to actually become useful. Is that a fair analogy, or am I totally off?
Simon Brown: No, that's completely fair. I was just going to ask you, do you remember Peter Coad's UML Modeling in Color?
Stefan Tilkov: Yes.
Simon Brown: It's the same thing - you have these four different types of classes, and you make them different colors.
Stefan Tilkov: Exactly.
Simon Brown: So what's really interesting is that the C4 model does not prescribe a notation, so there is no C4 model notation. You can do whatever one, and you can apply whatever visual semantics, whatever notation you choose; different shapes, and colors, and [unintelligible 01:28:36.08] and all that sort of thing.
Simon Brown: Something I always do in my workshops is I say to teams "You could use UML to do these diagrams, you're exactly right." You come up with a set of conventions and rules; you maybe use a use case diagram for a system context diagram, or you maybe have a component diagram and you apply appropriate stereotyping, and that sort of thing... And I do offer this as a thing teams can do, and no one does it. And I still find that surprising.
Simon Brown: I think you're right, when I introduce the C4 model, I often get a lot of skepticism, because -- and again, I have to be careful how I phrase things initially, because it tends to put a blocker up immediately. If I say "I'm gonna teach you a diagramming technique. It's a framework, and it has a bunch of rules, and it'll make drawing diagrams really easy", everyone will instantly switch off. So that's not the approach I can go in with.
Simon Brown: If we have the discussion after the workshop and I say "So you've gone through this. You've created a bunch of useful diagrams. I can see you really like them. What feedback do you have? What do you think is the selling point for this?", a lot of the people actually say "Well, you've given us a framework to work in." And it's the same thing, I've given them something to hang their ideas off. I've taken away a lot of the silly decisions that really don't matter too much...
Stefan Tilkov: Exactly, yes.
Simon Brown: It's just a way to draw a bunch of pictures. We know what pictures are, and we can move on and we can get focused on something more important.
Stefan Tilkov: That's awesome. I think that the key point here is there are tons of places where your creativity is way more useful than in the kind of lines or the way you draw a box. I mean, sure, you can be super-creative with your architecture diagram notation, but no user is going to be happier because you decided to use color this way. Just getting it out of the way is a very useful thing there.
Simon Brown: Yes.
Stefan Tilkov: Okay, very good. So what do you think about -- I'm hesitant to even ask that question, but what do you think about UML? Should we still be learning it and teaching it? Should it be part of a CS education?
Simon Brown: I'm going to say yes, actually... I think it's very useful. Again, if you go back to the early UML 1.0, 1.1, which was '989(ish), maybe something around that sort of timescale.
Stefan Tilkov: Late '90s, yeah.
Simon Brown: Yeah... You can actually still find some videos of Grady Booch and the others teaching UML, and I think the best version of UML was actually those early versions, because they were nice and lightweight. There was a smaller number of diagrams --
Stefan Tilkov: They were simple.
Simon Brown: Right, it was much simpler. And I've seen this at play lots of times - over the years, the vendors have all gotten involved, they've all wanted their specific features for their specific tooling, they're all focusing on Executable UML and defining complete architectures, and you magically press a button and all of the code pops out. The whole UML thing has become side-tracked by vendors, essentially.
Simon Brown: I think if you look at the core of UML, there's a lot of really good stuff in there. I don't like the fact that you have to read 750 pages to understand the entirety of it. The biggest thing I don't like about UML, beside from some tooling, which is awful - different story... But the biggest thing I don't like about UML is it doesn't help you. And this is exactly what you were saying earlier - if you fire up a UML tool and you say "I want to describe an architecture", it gives you no assistance. And furthermore, there are no good examples or case studies about if you have a modern microservices-based architecture, think about using these sorts of UML diagrams in this sort of way to describe what it is you're actually building, and make it reflect reality and reflect the code, and that sort of stuff.
Simon Brown: So those are the major problems with UML... But having said all that, I still think it's a useful thing to teach. The C4 model - it doesn't cover state diagrams, it doesn't cover activity diagrams, it doesn't cover business process diagrams or entity relationship diagrams... So even those C4 model diagrams I still think should be supplemented where necessary with a whole bunch of other stuff from UML, from ArchiMate, from good old-fashioned entity relationship diagrams. So I definitely think we should still be teaching it... But maybe with the emphasis that it's a tool in a toolbox. There's some useful stuff, and there's some maybe less useful stuff.
Stefan Tilkov: It's almost kind of a little annoying how much we agree, right? Maybe we have to talk about microservices at some point and try to find something we disagree about.
Simon Brown: Yes, it's the world's most boring discussion where people agree.
Stefan Tilkov: No, I totally agree with everything you said. I think it's useful, I think it's way too big, and way too complicated. I think the mere fact that, at least with a lot of people, you can draw a diagram in UML notation, and a significant number of people will understand the semantics of what you just drew. That's an extremely useful thing.
Simon Brown: In theory.
Stefan Tilkov: It's kind of obvious, right? Having a common language for those things... There are no points to be gained by having a different one than all the others because your class diagram is super-special, or your entity relationship diagram, which is typically shown as a class diagram these days.
Simon Brown: Yes. Just to actually pick on something you said there - one of my recommendations is even if people are using UML, they should still stick a diagram key, a legend, to explain notation, because not everybody knows it.
Stefan Tilkov: Oh, that's true. That's a very good point.
Simon Brown: And that's one of the biggest things - once they see that complicated set of boxes and lines they've never seen before, they're like "That's too much for me."
Stefan Tilkov: Very good point, yes. I also think that even within those diagram types there are certain things that you might not want to use for certain audiences... Like using inheritance in a diagram intended to communicate with a businessperson is not going to help you in any way. Maybe it's a refinement if you wanna do it at some later point in time, but it's not gonna ease things at that stage. Neither are UML stereotypes or types on attributes, or whatever.
Stefan Tilkov: Okay, so one of the things that comes up quite often is the question about how do you keep documentation in general and diagrams specifically in sync with the actual code? Because the code is obviously what matters, and nobody gains anything if you have a diagram that's out of sync with actual reality. What are your strategies there?
Simon Brown: You've got two basic options, and in many cases option one is the most simplest... And unfortunately, at the moment, the most effective. So option one is just update it. So if you've got a definition done for your task, so your work items, you add a line to the bottom and it says "Have you updated the diagram and the documentation given this (hopefully) small change that you've made to the codebase?" And that's often easy; it's just a process thing, and it's done, and we can forget about it.
Simon Brown: Option two is much more complicated, and that's really "Well, let's auto-generate the diagrams and the documentation of the thing we're actually building." That thing we're building could be the code itself, so we could autogenerate diagrams off the code, or we could perhaps autogenerate diagrams off build scripts, or infrastructure provisioning scripts, or maybe the live infrastructure, or maybe things like distributed logging and tracing.
Simon Brown: So you've got a bunch of different inputs you can use to potentially automate and autogenerate diagrams. However -- and this is where things start to get a little bit complicated... I'm sure you've seen this yourself - if you open up an IDE with like a million lines of code and you ask it to draw you a picture, what do you get?
Stefan Tilkov: A lot of inter-crossing lines, and...
Simon Brown: Just an utter mess, right. And it's not because your code is a mess, hopefully... It's because it's trying to show you too much. It's trying to show you exactly what the code looks like. It's reflecting that level of detail. And that's not useful to us as humans. We need to come and chunk it up and zoom out, which is where the C4 model comes into play again.
Simon Brown: And you often get the same thing if you start autogenerating for build scripts and running infrastructure as well, because if you've got like a hundred versions of a microservice running, do you want to show a hundred things or do you want to show one thing with some variability somewhere else?
Simon Brown: So the whole autogeneration thing is a bit complicated, and it's a bit tricky at the moment... And also, a typical codebase doesn't tell you the whole picture. So if you think about the system context diagram that I kind of briefly outlined earlier - it's got a box in the middle representing the system, different users, different system dependencies - it's really hard to generate that high-level diagram just from using the code as an input/source. That's some of the real-world challenges we face here. There's not enough metadata in most codebases to generate high-level pictures, and if you generate the low-level pictures from codebases, you often get too much. And there's often no easy way to get that happy medium... So that's why option one is often the easiest, unfortunately.
Stefan Tilkov: Yes. I'm old enough, as you are probably, to remember things like Together C++, TogetherJ, or whatever...
Simon Brown: Yeah.
Stefan Tilkov: The Together CASE tool that simply -- you can't really say reverse-engineered... It simply showed you your code as a UML.
Simon Brown: It was a roundtrip--
Stefan Tilkov: Yes, completely roundtrip engineering, where you could actually edit the code within the diagram; you could seamlessly switch between the IDE, because essentially the CASE tool was your IDE... And I never found that an appealing thing, because it didn't give me the abstraction that I want from a model. I want the model to focus on one aspect, like for example the structure that you mentioned... And then all of the details - if I wanted to focus on all of the details, I would look at the code, and not at a high-level diagram...
Stefan Tilkov: But there was another extremely important point that you made that I also want to highlight, which is that the code, while being super-important, is not everything; especially not in modern architectures. Maybe - I highly doubt it even then - maybe back then, 20-30 years ago when you wrote a program that essentially was a standalone thing that you ran on the command line and then it produced a file, took a file as an input, produced another file - then maybe the whole truth was within the code of that particular program. But these days, everything is a complicated collection of independent, deployable units (or containers, in your terminology, if I remember correctly). So all of those things communicate, and the communication paths are sometimes only discoverable at runtime. They're not even in any configuration file or in anything that resembles code. It's through something that you could only discover if you could look into the minds of the architects, or the coders/developers, or actually observe or survey the system while it runs... So that is definitely a very complicated thing.
Stefan Tilkov: Have you, by the way, found a solution to (I think) a related problem - even if you can figure out, say, dependencies between artifacts of some kind, between deployables for example... Once you visualize them using something like Graphviz or any sort of thing, I think while the quality has gotten better in recent years, it's never as perfect as something that you would have drawn yourself. If you can actually realign the boxes, and you move this line a little bit there, and this box a bit over here, then it actually looks better... But the problem, of course, is once you regenerate the whole thing, so you have to do that over and over again - has anybody yet come up with a solution that allows you to apply the visual changes, like a diff to the autogenerated graphic? Am I making any sense? Do you have any idea what I mean?
Simon Brown: Yeah, kind of... I must say, I'm not a huge fan of the auto-layout algorithms, like you get in GraphViz. I always find they put things in the wrong place, and when I'm telling a story, if I'm doing a presentation or something, I want to point things or I want to group things visually, because they kind of belong together; they might be the same type, but from a nodes and edges graph perspective, you've got your typical "all dependencies flow downwards" in tools like GraphViz and they give you this weird layout.
Simon Brown: Something I did a while ago was I started creating some tooling called Structurizr App, and I wanted to build something that would let people draw diagrams easily, and again, would take some control away from them... Does that make sense? No. To take some freedom away from them in terms of a fixed notation... But I still wanted the ability to move boxes around myself. So as a part of the Structurizr tooling, you can upload a new definition of your model and the views in the model, and the Structurizr tooling will attempt - and it's not perfect by any means - to retain the existing layout. So I'm trying to do a blended approach there.
Stefan Tilkov: That's basically what I was asking for, so that sounds very interesting. So can you tell us a little bit more about Structurizr? Maybe a brief intro.
Simon Brown: When I was doing my workshops on the C4 model, up until about five years ago people would always ask me "What tooling do you recommend?" and my answer was always "Just use Visio", and you have no idea how irritating or frustrating that is. So I figured, "Well, I should try and do something. If I'm promoting this way of drawing diagrams, I should try and build some tooling myself." So I sat down and I started to put together - and this sounds horrible, and it was - a HTML web-based modeling tool, specifically tailored around the C4 model, and it was horrible. It was the worst UX you've ever experienced in your life.
Simon Brown: But the interesting thing about that was that the stuff behind it, the actual framework and code I'd written, was a really nice way to succinctly define a model and different views on that model; that was essentially a set of Java classes. So there was a set of Java classes sitting behind this web application, representing people and software systems and containers and components. And I figured out "Well, let's ditch the UI, let's only have the UI for drawing the diagrams, and let's allow people to describe their architecture using code."
Simon Brown: So you'd basically create a bunch of objects in memory, wire them together with a nice little API, and that would create you essentially a directed graph. And once you have that directed graph, you can export it to different formats and visualize it in different ways... And that's actually where the Structurizr tooling came from. That was about five years ago. Now there's a whole bunch of different ways you can use the tooling, and there's a community of different tools that have popped up around it.
Simon Brown: My original version of tooling was you write some Java code and you get some diagrams. Now there are libraries for all sorts of different languages that people have written and they've open-sourced. When we were locked down last April, I put together -- so if you're familiar with something like PlantUML, where you write text and get diagrams... I created something I call the Structurizr DSL (domain-specific language), and it allows you to define a model and a bunch of [unintelligible 01:44:15.09] as a single DSL file.
Simon Brown: Also, whilst we were in lockdown and I wasn't doing very much traveling, I came up with a bunch of open source tooling, and there's something called the Structurizr CLI... So what you can do is you can define a C4 model to describe your architecture, and you can describe a set of views in that single DSL file. And then using the CLI, you can export it to my Structurizr tooling, which is available at Structurizr.com, or you can export the views in that DSL file to PlantUML, or to Mermaid, or to WebSequenceDiagrams. What it's become now is a way to describe an architecture using a set of hierarchical diagrams and hierarchical viewpoints and stuff, but with independence in the way you actually visualize that model. So that's kind of what it is today.
Stefan Tilkov: Very cool. Before we wrap up, one last question I have is -- obviously, we're going to point people to your books, and to Structurizr, to your website, which has lots of resources... Are there any other resources that you think are useful for developers getting into architectural work?
Simon Brown: Developers getting into architectural work... There's a bunch of really good books that have come out recently. Neil Ford and Mark Richards have a book, and the name completely escapes me.
Stefan Tilkov: I think it's Fundamentals of Software Architecture.
Simon Brown: Yes, Fundamentals of Software Architecture. Correct, yeah. I can visualize it with the picture on the front. That's a very good book for people looking to get into architecture.
Simon Brown: Gregor Hohpe has got a book out, it's called The Software Architect Elevator. That's a really interesting book, because it talks about the software architecture role not just from a technical perspective, but in terms of what that role means to the entire organization... And it focuses a bit more on soft skills and presentation skills, and influence, and that sort of thing. So that's another really good resource that I'd point people to.
Simon Brown: Plus you've got Michael Nygard’s "Design It", you've got George Fairbanks' "Just Enough Software Architecture", Eoin Woods and Nick Rozanski's book "Software Systems Architecture"... There's lots of really good stuff out there focused now at software developers.
Stefan Tilkov: Excellent. Well, I'll definitely put them all in the show notes, and then our listeners can definitely check them out. Thank you very much for all your time, Simon. It was an awesome conversation; I really, really enjoyed it. Thanks for being with me, thanks for being with us, and enjoy the rest of your day.
Simon Brown: You too, thank you.