Transcript
Stefan Tilkov: Hi, listeners. Welcome, everyone, to a new episode of the CaSE Podcast, another conversation about software engineering. Today I have a guest that I know pretty well, because we have been doing this podcast thing for a long time together, even though it's been a long pause since we recorded the last one... Welcome, Markus Voelter!
Markus Voelter: Hi, Stefan! Nice to be here.
Stefan Tilkov: Markus, great to have you on the show.
Markus Voelter: Yes, absolutely. I haven't podcasted about software in a long time.
Stefan Tilkov: But you're a very regular podcaster, as many people will know.
Markus Voelter: Absolutely.
Stefan Tilkov: Maybe you can just start by introducing yourself.
Markus Voelter: Yes. I'm basically an independent consultant, spending much of my time working for itemis, and I deal mostly with language engineering, building domain-specific languages and the tools around them. It all started 15 years ago maybe; at that time it was called model-driven development. I wrote a book about it, and this kind of evolved into building custom languages. We've been doing this for a while in healthcare, in engineering disciplines, in finance, in government, in currently payroll and tax... A very wide range of domains. But the commonality is that we tried to capture the essence of these domains in languages and allow people to express business logic, or whatever you want to call it (Fachlichkeit) with this language.
Stefan Tilkov: You've basically introduced the topic as well, because that's obviously going to be what we're going to talk about today. We're going to do a little bit of that, and we're also going to contrast it, or maybe combine it, or discuss it in conjunction with domain-driven design, with DDD, and with maybe other current topics such as microservices... No podcast without microservices...
Markus Voelter: Yes! Of course not!
Stefan Tilkov: And that's very interesting, because this is a topic that interests both of us, and we'll both have lots of opinions, so the audience should be prepared for us sharing lots of opinions... Which at least in the past were not often the exact same ones, which I think is a very nice thing.
Markus Voelter: Exactly. I think we agree on many of the fundamentals and basic ideas, at least based on past conversations... But then at some point we kind of branch off into different subdirections or something, where we start disagreeing. I'm looking forward to seeing if this is going to happen again.
Stefan Tilkov: Yes, it's going to be interesting. Okay, so my suggestion is why don't you start us off by introducing what language engineering (as I think you called it) is all about.
Markus Voelter: Well, language engineering is the art and science of -- no, it's basically custom language development. Assuming we have decided we need a custom programming language, or a custom modeling language - it doesn't really matter how you call it - then language engineering is the profession or the activity of building the language. In the traditional case, it would have been defining a grammar, implementing a parser, maybe name resolution, type checking, and then perhaps an interpreter or code generator... And fundamentally, this hasn't changed, except that tools have evolved a bit, so depending on the kind of tools you use - let's say Xtext, then your grammar definition also gives you a basic IDE, with code completion, syntax highlighting... You can then further customize it if you, for example, choose to use MPS (Meta Programming System from JetBrains) for your language definition, then you don't even define a grammar, because it's a projectional editor and you define the language in a different way.
Markus Voelter: Language engineering basically says "Let's start with a domain, let's analyze what is relevant to that domain, what do people in that domain want to express in terms of behaviors, structures of software systems in that domain, and then let's implement the language with a suitable set of abstractions and a suitable notation, which is very important. Often people see language more as notation than about the abstractions that's behind them. And then people can efficiently and effectively model stuff, and then we can generate or analyze, interpret and run. That's basically the nutshell.
Stefan Tilkov: Very nice. For some of our listeners this might be completely old news, nothing new about anything you just said, but some people might come from a different perspective. I think some listeners might equate this with the design of programming languages. If you invent a new programming language, then of course what you will be doing to large degrees - language engineering will probably be the essence of what you're doing. You're designing a language -- all that you've just said: It has a syntax, it needs some kind of tooling to operate, it needs to figure out how to express things... There are lots of things that are involved there. Maybe some listeners have been doing that as part of their studies, their education, but many of them will say "How is that relevant to me if I'm not a programming language designer?" Can you elaborate a bit on that?
Markus Voelter: Yes. This is where the notion of domain-specific languages comes in. The reason why I explained it a bit more generally is that language engineering really doesn't necessarily imply that the language you build is somehow specific to a particular niche. It could be a general-purpose language. As you say, the process is exactly the same. By the way, the economic constraints are a bit different. If you are Microsoft and you design a language called C#, then you can easily afford to invest 20 person-years into an interpreter or a generator/compiler. But if you do that in the context of a project for a particular business domain, then 20 years for a generator is probably a bit too much.
Markus Voelter: So the trade-offs are different, but domain-specific languages - the idea is that if you want to express something, and typically a lot of "somethings", in a particular domain, then it's worth building your own language... Because you can be more succinct, you can describe things in a more abstract way, with more domain semantics, which gives you a more interesting analysis. Usually, another benefit is that you're independent of a particular deployment technology or implementation language, so there's an aspect of longevity... You basically avoid the legacy system trap, because when you want to change the implementation technology, you don't have to re-extract the business logic from the implementation code, and then kind of lift it and implement it in a different language. You have it already in its purest form as a model, expressed with a DSL. So that is basically the idea.
Markus Voelter: Let me add one more sentence... You might say "Well, what's the point? I can express everything with a programming language. After all, it's Turing-complete. Why? Why should I build my own language?" And it turns out that if you want to, for example, get doctors to specify diagnostics or treatment algorithms formally and executably, so that they can be checked and verified, or if you want to make tax experts express tax calculations - they really don't want a programming language with which they can build general-purpose abstractions, with all the complexity that goes along with it. They want a tailored language, where the language itself knows about the domain. That is where DSLs come in, the programs become shorter, more understandable, maybe have a bunch of tables, and fraction bars to make it more readable... Stuff like that.
Stefan Tilkov: I have a lot of things that I disagree with, but I think I'm going to save them for a bit, because I want to make sure we have the basics down first. I think that was a very good explanation of the domain-specific aspect; generally, people won't be designing general-purpose programming languages, unless it's maybe a hobby project that you want to do for the fun of it, which is a great motivation and a very good reason for doing it. But most people won't be doing that as part of their day job, because they're not language designers at a company that does programming languages.
Markus Voelter: There are some exceptions... For example, we have a general-purpose functional programming language which we have built with MPS for the explicit purpose of extending it domain-specifically, so that you have basically a domain-specific language, but you reuse basic expressions, operators, comparators, and stuff like that. So that's kind of a middle ground.
Stefan Tilkov: Yes, but let me add something to maybe connect it to something that I would guess most of our listeners that are programmers have experienced, which is you very often design quick little language, not with a very formal process or very formal tooling, when you for example design a little configuration language, or something. You want to have some external file that you can read in at runtime to set some key-value pairs, or something like that. And as everybody knows, these things grow, and become bothersome, and then you regret that you didn't use the proper tooling to do that, but whatever. Most people have done something like that.
Stefan Tilkov: Or it could be some language that you accept from the user, where they maybe enter something into some UI, or it could be something that they even enter with the help of some UI control specifically built for that purpose; not exactly like an IDE, but a little bit like an IDE, because it helps them enter the information that conforms to whatever format or model it is that you represent with that. So at least a little bit, I think most people will have been exposed to that idea at least. And what you've been spending the last two decades on is essentially just that - you've completely focused, as far as I gather from what I read from you, and what I hear from when we talk to each other, you've completely focused on this particular thing, the engineering aspect of the language.
Markus Voelter: Well, yes, with two additions. One, a language is never developed in isolation. There is always an architecture of the overall system. There are business processes associated with it, and of course, you always take that into account and design all of that together, so that things make sense. And the second comment here is that although you can say "Well, a sophisticated language for payroll calculation with semantic versioning, data inheritance, incremental calculation and temporal data is a language", and you know, "a little configuration language with three different language concepts is also a language" - yes, but comparing the two in terms of size, complexity, the associated tools, the usability concerns, or user experience concerns for the prospective users, it doesn't really make too much sense to put them into the same basket.
Stefan Tilkov: That makes perfect sense. Very good.
Markus Voelter: I was suggesting, because we want to put this into the context of DDD, would this be a good time for you to recap/refresh my mind a little bit about DDD?
Stefan Tilkov: Sure. I can try.
Markus Voelter: For me, as context - I read the big book back in the day, and I've been confronted with DDD again and again, in various projects, and I remember things like bounded contexts, or the ubiquitous language... But I'm probably a DDD beginner, so...
Stefan Tilkov: Well, I wouldn't consider myself a complete expert - I have some colleagues who are more into this - but I think I can do well enough for the intro, for this level...
Markus Voelter: For me...
Stefan Tilkov: Well, not that... But it's probably good enough for somebody who wants to get an intro to the whole thing; I'll refer you to a DDD-focused episode or even a DDD-focused podcast if you want to get more details about that. But to just give people a general introduction who don't know what this is about, or maybe have read the books, or read an article a while ago - essentially, domain-driven design is nothing new as well; it's been there for 15 years... It's been sort of rediscovered with the whole microservices emergence, because people found out that some of its ideas are a very good match with what many people do with microservices architectures.
Stefan Tilkov: In general, I think the core things are still the same, with maybe one or two additions. In the original book by Eric Evans there are two parts. One is the tactical DDD part, one is the strategic DDD part, and they're both relevant for our discussion, very much so... Although for at least the microservices part, and what most people these days do with domain-driven design, the strategic part is the more important one. And in the strategic part of the book, the idea is that there are some general, high-level concepts that can help you structure your software or your architecture in a way that makes it more maintainable, easier to evolve, easier to manage, easier to develop, easier to collaborate about with your business stakeholders... And all those things are embodied in a few concepts. You've mentioned a few of them, and maybe we can pick two.
Stefan Tilkov: One is the idea of the ubiquitous language. That is essentially the idea that you capture the terminology and the concepts and the relationships between those concepts that actually hold for the business as a whole; not something isolated to development, to programmers, to implementation, but something that connects everybody within the company. Ideally, that language would be the language that everybody understands, that everybody can speak, so that you can get the transfer of concepts and ideas and the possibility to discuss things among different stakeholders in the whole game. So you can talk to a business person about something, because the language you use, the words you use to express concepts are the same ones that they use when they talk about what it is that they do in their daily work. The idea of expressing things with good words, and finding good names or extracting/discovering the good names--
Markus Voelter: And giving a good meaning, a well-defined meaning to all these words...
Stefan Tilkov: Exactly... It lends itself to this idea that you have a core model that is very much technology-independent; so it's in contrast to what you've just proposed - it's written in a programming language, typically, but not necessarily, an object-oriented language... But that's sort of where the roots of the whole DDD thing are. Although there are people doing it with functional programming; we had an episode on that as well, with Mike Sperber.
Stefan Tilkov: Essentially, the idea is that you put a lot of emphasis on creating abstractions, in the case of OO classes and methods and variables and parameters, with good names, that actually help you express what it actually is the business does, in a way that reads as if it's a description of what somebody told you, as opposed to something transcoded from one world into another.
Markus Voelter: Yes. Before you go on to the second one, it's probably obvious to everybody how this ubiquitous language in my mind is half the way to a domain-specific language. That's kind of an obvious connection point. Now go on.
Stefan Tilkov: Well, yes and no. I didn't want to interrupt your flow at the beginning; I didn't want to distract you with the concept of internal DSLs. And I suggest that maybe we pick that up again after we've done the DDD thing, because that's sort of probably where the connection is. But let's maybe postpone that for a minute.
Markus Voelter: Yes.
Stefan Tilkov: But I definitely agree, there is a connection here. There is some similarity, and it's essentially the probably good idea that it makes sense to talk to the business if you're a developer, if you're a programmer, and to actually use the words that they use, instead of picking others. That is a very obvious thing.
Markus Voelter: That's kind of a trivial statement, yes.
Stefan Tilkov: It is. But as we both know, those trivial statements have the problem that they're very often not known, and discounted, and not viewed with the importance that they actually have.
Markus Voelter: Yes, alright.
Stefan Tilkov: Ubiquitous language - that, I think, is one thing. The second thing from the strategic part that I find very important is the one that you mentioned as well, which is the idea of a bounded context. Now, if you use OO methodology to engineer a model, with all of those concepts, then at some point in time that model becomes really big; at some point in time it becomes too big for its own good. It becomes so huge that it's hard to understand it. What also happens is that you have an overlap of words. You might call something an account, and I might call something an account, but because the two of us work in different apartments, we might mean completely different things.
Markus Voelter: Did you just say "work in different apartments", instead of departments? Was that a Covid--
Stefan Tilkov: Did I say apartment? I meant department.
Markus Voelter: I think you did... But we actually do work in different apartments.
Stefan Tilkov: That's very true as well. Okay, so this is a completely socially-distant podcast recording. Okay, so "departments" is what I wanted to say. So if people in different departments or different branches of the business use the same word, then that does not necessarily mean that it's the same concept that they're talking about... And again, this is a completely trivial statement, but people who've been in this industry for a while have probably suffered through some variant of the enterprise data model fallacy, or the canonical data model fallacy; whatever you wanna call it. The idea that you just walk around the company and talk to everybody, and once you've done that, you have gathered everything that there is to be known about a customer, and then you define the one and only customer class, or record, or data type, or whatever... Which then ends up having 12,568 attributes, and becomes completely unmanageable, because every change requires synchronizing with all of those people again.
Stefan Tilkov: To make a long story short, the idea of a bounded context is to explicitly model isolated parts to acknowledge that it's a good idea to have islands of meaning, that are coherent, that actually form a reasonable boundary across a bunch of knowledge, a bunch of concepts and their relationships... And then you can have another bounded context that is connected somehow, that has maybe some overlap or some relation between its concepts and the concepts of another bounded context, but it's allowed to, for example, reuse the same words, it's allowed to assign different meaning to some of them, and it's allowed to focus on what it is that it's about, as opposed to being about everything at once.
Stefan Tilkov: And if you have different bounded contexts, that might actually end up being owned by different organizational units, teams. Then you need some way to manage the connection between those things, which is also a huge part of the whole strategic DDD part of the book, and there are multiple strategies of dealing with that. One bounded context might be owned by a team that can just dictate its terms to another one. Another situation might be where you explicitly create what's called an anti-corruption layer to isolate yourself against changes occurring in another place...
Markus Voelter: Basically mappings, right?
Stefan Tilkov: Mappings, and ways to address the boundaries between the bounded contexts in different ways. You might share code - not something I would advise - but typically, you have some strategy of doing that. That is the strategic part.
Stefan Tilkov: Then for the tactical part, typically if I explain DDD, especially to business people, or while talking about high-level architecture, I say "Don't worry about the tactical DDD part now. We can talk about that later." But it actually is very relevant to our discussion, I think, because it also has this language aspect. In the tactical DDD part there's a bunch of -- being an old UML person, I tend to call them stereotypes, although that's not the terminology being used there, right?
Stefan Tilkov: There's a bunch of concepts, like for example an entity, or an aggregate, or a value object, or a service, which are all -- you could call them roles that classes play, or you could call them classes in their own right... Or meta-classes, or stereotypes...
Markus Voelter: It's basically a simple meta-model, right?
Stefan Tilkov: You're right, it is a simple meta-model. The problem that I've found in the past is that once you use the word "meta-model", you lose about half of your audience.
Markus Voelter: Yes... I don't care, as long as the other one stays around... Sorry.
Stefan Tilkov: Again, one of the nice differences between those two worlds... You are, of course, right, as you know; it is a meta-model. It sort of defines the structure that you can then use to define the actual model. So it's a model for the model that defines -- again, some people will now see why so many people don't want to talk at that level. And not only am I an old UML person, I'm also an old OMG/meta-modeling/whatever creep... So I can remember multiple meta-model layers, like having a meta-model, and a meta-meta-model, and a meta-meta-meta-model...
Markus Voelter: Yes. And if we say that M3 is self-defining, we've lost the other half of the audience.
Stefan Tilkov: Yes. Which is why I try to avoid that... Which doesn't make it any less true that what's in the DDD book is actually just a set of stereotypes, in the tactical DDD part, that just happen to work for Eric Evans. And that is actually one of my critiques of DDD - a lot of people sort of assume that this is THE canonical set of stereotypes; it's the only way to do it, because it's in the book... Which is limiting yourself way too much, even if you're not into the meta-model thing at all, even if you're not into DSLs or MDD or anything like that. It's kind of really sad that people think just because this was the snapshot of the number of stereotypes that he had when he wrote the book, this somehow makes them magically appropriate or true. They're not. They were a good start, but there are many others that might make much more sense for your particular work.
Markus Voelter: See, this is really interesting, because I think also this meta-model - now we've used the word many times - is almost useless, except for as the basis of basically data-in/data-out applications. It is not in any way interesting to capture interesting business logic. On the other hand, we have the ubiquitous language, which is supposed to capture the essence of the domain abstractions, correct?
Stefan Tilkov: No, I disagree. Maybe this is where the disagreements start, right?
Markus Voelter: Finally!
Stefan Tilkov: Finally... So what you've just said sort of puzzled me a bit, because what you've just said was that this meta-model is not really important, except for maybe data in/data out applications.
Markus Voelter: I didn't say it's not important, I said it's insufficient for...
Stefan Tilkov: Okay, then I agree with that. Because I often point people to an old article that you wrote - when was that, 10 years ago? 15 years ago? The Architecture as a Language thing that you did for InfoQ. That was ages ago...
Markus Voelter: Yes.
Stefan Tilkov: I still remember that, because it's a very good article... And I send people to read it, because it is to me a very important idea; it doesn't necessarily mean that I buy the toolchain idea, not at all. But the idea is that -- well, maybe you should tell me what the idea of that article is, before I paraphrase it for you.
Markus Voelter: Well, as far as I remember - because I have since changed my view quite a bit... Okay, there are two different things you can build modeling languages/DSLs for. One is technically-motivated; so you want to describe as meaningful first-class abstractions, the architecturally relevant things in your system. And that is the architecture as language idea. I start with interface, component, connector - the usual stuff that everybody has. But then I point out that if you're not limited to these five abstractions, and it might be the other five from Eric Evans here... But if you're able to define your own set of architectural abstractions, like for example a replication interface for data, then you can come up with much more meaningful and relevant architectural descriptions that are beyond the stupid box/line abstraction that everybody seems to use.
Stefan Tilkov: Exactly.
Markus Voelter: Now, however -- and I think this is still very relevant. A colleague/friend of mine has used this approach to describe the architecture of satellite onboard systems, and he has done it in a way where these abstractions are directly integrated into C. Of course, this gives him the ability to directly express architecturally relevant concepts directly into code, with all the possible automation for code generation beyond that. But I have since moved on almost completely.
Markus Voelter: I haven't built one of these architectural languages in probably eight years, because I have since moved on to building languages for the business, for the actual business domain core of businesses.
Stefan Tilkov: It makes perfect sense.
Markus Voelter: So there you don't model any of these technical considerations. You really talk about a payroll version, about temporal data, about a yearly deduction, and stuff like that.
Stefan Tilkov: Yes. I get that difference. Let me just argue why I still think the first one is relevant. I'm not necessarily suggesting that you build a DSL for that, as your article suggests, but what I'm observing is that that is essentially technical architecture work defined. What I think you do as a technical architect - maybe not as a business architect, but as a technical architect - you come up with patterns, drawing on a set of patterns that you already know, you adapt them to the situation at hand, and you come up with a language for expressing the architecture of the system, whether that is a formal language that you have a DSL for, and an interpreter, and a code generator or whatever; it doesn't matter. The fact that you use those words -- when I'm in a project, I say "Let's build an SCS (self-contained system)." That's one example of that. A self-contained system means that. "Let's give that self-contained system an asynchronous interface, or let's use UI integration here. Let's use a business event stream here", or something like that.
Stefan Tilkov: I use those words, and those words are the jargon, the language of the architecture of this particular system, with a lot of overlap with other systems, definitely, because it would be stupid to invent everything from scratch, at least if the systems have some similarity.
Markus Voelter: Yes, of course.
Stefan Tilkov: But no two systems are 100% alike, because no two sets of architecture requirements are alike. So that is why I point people to that idea, and that, I think, is a very valuable concept, completely regardless of whether or not you're using a formal DSL to do these things.
Markus Voelter: Yes, because one thing you often do is you represent these concepts in your frameworks and platforms. You might have base classes, or stuff that makes these architectural abstractions more concrete, even if you don't build a language.
Stefan Tilkov: Well, yes. And that is maybe another interesting aspect here - I think it could be a variety of things, but that doesn't change the concept.
Markus Voelter: Yes, of course.
Stefan Tilkov: It could be, as you say, a base class; it could be an interface, it could be a set of related interfaces. It could be a description in a file, it could be some XML or JSON these days, or it could be a particular way to structure the outer observable surface of a system. There are so many ways to represent it... That's another discussion. The basic idea is that you abstract from all of those concrete details to something that has a catchy name, that people can relate to and can use, to talk about those concepts on a different level.
Markus Voelter: Well, it is more than a name though... Because one problem if you do it only with the means of your programming language and frameworks, it's very hard to make these things declarative, which makes analysis much harder, assuming you want to analyze -- there are all kinds of reasons why you want to do, or why you maybe don't want to do.
Stefan Tilkov: Absolutely true.
Markus Voelter: So that's where formalizing it into an actual language has definite advantages. I'm not saying you always need it, of course, but you know...
Stefan Tilkov: Sorry, brief interruption. I'm back.
Markus Voelter: You fell from the chair?
Stefan Tilkov: No, I just had somebody walk into the room. Don't worry.
Markus Voelter: Okay.
Stefan Tilkov: Yes, you are right. But that's another very interesting thing. So maybe let's address things one after the other. So I understand why you said you've moved on, because you sort of changed -- I would phrase it differently. I would say you've changed the domain you address. You use the same means, but you don't use them to address the domain of a technical architect, you use the means to address the needs of a business person... Which is completely valid, fair enough, but you've sort of moved on from my domain to a tax accountant's domain. Is that a fair way to phrase it?
Markus Voelter: Yes. I mean, it's obviously not specifically a tax accountant; it might be all kinds of things.
Stefan Tilkov: Sure.
Markus Voelter: So you're right, and of course, you can combine the two layers very nicely... But the reason why I've done that is because - or why I've moved on - ultimately, if you look at the total increase in productivity, it is much more leverage (I've used the word; please congratulate) if you can get your domain people who ordinarily might write documents and crap to "program". It's a much bigger deal than if you can improve the efficiency of your architecture team by (whatever) 20% or 30%.
Stefan Tilkov: I understand.
Markus Voelter: So that's where I've moved on.
Stefan Tilkov: So from your perspective, you've sort of moved on to more interesting domains, right?
Markus Voelter: No, actually -- if you think about taxes, I wouldn't call them interesting at all. So it's really not about interesting. It's about the impact. Seriously, it's about the impact the approach can have.
Stefan Tilkov: More impactful. Fair enough. Okay. But regardless of the domain, there is a question of "Is this a good idea or not?" and if it's not a good idea, why not? Maybe some context, as well - I spent a lot of my professional career doing those things, too... So I think I'm a little older than you, I think I might have even started a little earlier, but I've sort of abandoned it a bit in the middle of my career. So I've moved on to different things. I used to believe many of the same things that you believe, but I've sort of changed my mind a bit as well. One of the things that changed my mind was the rediscovery, just for me personally, of internal DSLs. So do you want to define that, or should I define that?
Markus Voelter: Yes, I'd like to define it, because there is also a misunderstanding. So an internal DSL, at first glance, is defined as one where the code you write in the DSL, the sentences of the language, if you will, lives inside a program expressed in a general-purpose language. So you have a bunch of Java code, or Ruby code, or Groovy code, or Scala code, and it has the usual Java/Ruby/whatever keywords, but then somewhere in there there is something that reads more like, for example, a sentence. Or where you chain a bunch of thought expressions which is what I would call the fluent API; other people call it an internal DSL, which I think is a joke, but okay... That is how people usually define it. Would you agree to that definition?
Stefan Tilkov: Well, yes. I would point out that different general-purpose languages are differently suited to it …
Markus Voelter: Yes, of course.
Stefan Tilkov: But yes, I would have given a similar definition.
Markus Voelter: But I think the definition is wrong, or at least it is not complete... Because what is an additional - in my opinion - condition for calling something an internal DSL is that the language definition is done with the means of the host language. So that is why different general-purpose languages have different capabilities of defining the internal DSLs, because you basically need some means of meta-programming, of introspecting the code, of rewriting ASTs, or if you do it in SmallTalk, you intercept calls, and do meta-object protocol etc.
Markus Voelter: The reason why I emphasize that is because we have built lots of extensions to a general-purpose programming language like C, or Java, or KernelF, a functional programming language where you do mix general-purpose and domain-specific stuff... But the meta-programming is not done with the host language. I mean, C obviously has no means of defining internal DSLs. So we have done the meta level with an external tool. Resulting is something that looks like an internal DSL, but it is defined with the means and techniques and tools of external DSLs, and this gives you much better support in terms of code completion, type-checking and all these other nice things you want to have usually. So that's why I make that distinction.
Stefan Tilkov: So in those particular examples, are the programs in that internal DSL still valid C programs, for example?
Markus Voelter: No. Of course not. Well, yes in the sense that their semantics is defined by a translation to C. So in that sense, yes.
Stefan Tilkov: But that's not the same thing. Okay, I'm not sure why you say that the first definition you gave is wrong...
Markus Voelter: It's incomplete.
Stefan Tilkov: I think you sort of extended it to mean something that I don't think anybody intended it to mean in the first place, and then had to add something to make it what it was in the beginning... So I'm not 100% sure about that. I think basically everybody who did internal DSLs would have implicitly expected them to still be valid whatever-programs. If you do a Ruby internal DSL, then you expect the code to be valid Ruby code. If you do the SmallTalk, same thing.
Markus Voelter: Well, but that's exactly what I'm saying. That is part of the mainstream definition of internal DSLs.
Stefan Tilkov: Okay. So I agree with you; let's add that to a necessary part of the internal DSL thing. So as you pointed out, if one uses that definition of an internal DSL, there is a very fluid boundary between what to call internal DSL and what not to call one... Which is why I don't really use the term anyway. I don't use it a lot, because many people would argue that that is just an API, and they'd be right, because that's what it is.
Markus Voelter: Exactly.
Stefan Tilkov: You write some library code, and that gives you some abstractions, built with the means that the programming language offers you... So obviously, that's just a valid program; if it happens to be a library, then you would call it a library. Well, why does it need a different name? And I agree, it doesn't really need a different name. It could just be called a program, or an API, or whatever makes sense for you.
Stefan Tilkov: The interesting thing for me though was that I became interested in code generation and in all of those model-driven things at a time where it was really hard to write programs, specifically mainstream programs, in a certain mainstream programming language.
Markus Voelter: Yes.
Stefan Tilkov: And it was possible to up the productivity dramatically. That's what got me interested in the first place. And that sort of went away a bit as the environment became a little bit more usable. The user experience improved... That doesn't change anything fundamental about the whole concept. It doesn't devalue the concept at all... But it reduced the friction. Maybe it's easier to get by without that concept.
Markus Voelter: Yeah, the classical example was the early J2EE crap.
Stefan Tilkov: Exactly.
Markus Voelter: There you had so much verbosity and repetition in all these deployment descriptors, and interfaces, and base classes... You basically had to generate to retain your sanity. Also, programming languages like C really aren't very good at building higher-level abstractions. Just think about implementing a stupid state machine in C. You basically need a code generator if it becomes any degree of complex, the state machine.
Markus Voelter: So the worse your target architecture is, the less capable your means of expressing things, the more pain you experience and the more there is the need to build a custom language. That's basically your point.
Stefan Tilkov: Yes, you are right. It reduced the need drastically.
Markus Voelter: Yes. And I agree to that, by the way. I would not recommend people to build a DSL, for example, on top of Spring, or Spring Boot. It doesn't make much sense... A technical DSL.
Stefan Tilkov: Yes, the technical DSL. Okay. The other thing that I discovered - or rediscovered; some people found out - was that once you start questioning the whole thing, you really have to see it as something that is a thing with a typical architectural trade-off... Which reflects my current view of the whole concept.
Markus Voelter: What is the whole thing you question? I forgot the context.
Stefan Tilkov: So the whole thing I'm talking about is the idea of building and using a language for a particular purpose.
Markus Voelter: Okay.
Stefan Tilkov: So I'm not at all saying that this is always a bad idea. Not at all. I think it is sometimes an excellent idea to do that. I think sometimes an internal DSL is a very good option. I think sometimes it's not. I think sometimes you're much better off with an external language, which you might sometimes best build with traditional, boring tools, or sometimes with a full-featured language engineering platform...
Markus Voelter: Called language workbench, yes.
Stefan Tilkov: Whatever, right. The opinion that I have arrived at is that it is an absolutely useful concept to have in your toolchain, and you should be able to apply that if it matches your requirements... But it really has to match your requirements, because it's not for free.
Markus Voelter: But how is that not true for--
Stefan Tilkov: It has a huge, huge cost.
Markus Voelter: But Stefan, how is this statement not true for everything?
Stefan Tilkov: It is. It is. That's all I'm saying. It's true for everything.
Markus Voelter: Yes, of course.
Stefan Tilkov: That's exactly the point. But of course, if I talk with somebody who is an advocate or an evangelist for one particular approach...
Markus Voelter: Like me.
Stefan Tilkov: ...I've found that people who do that tend to over-emphasize the applicability and tend to use it in many places where it's not a good idea. They tend to de-emphasize or undervalue the cost of the whole thing.
Markus Voelter: True.
Stefan Tilkov: So I may be doing it wrong here, but that's -- and I think people need to be aware of that as well.
Markus Voelter: Yes. But this is also a true statement, right? I mean, it's also true for microservice architects.
Stefan Tilkov: Oh, yes. I completely agree.
Markus Voelter: I'm trying to provoke you a little bit, but so far you've made general statements about stuff that's always true for every technology and software.
Stefan Tilkov: So we can agree. That is very good. We can agree that all things have downsides, have hopefully some benefits - hopefully, at least some - but they also have downsides. And now we can maybe talk about the particular applicability of these particular approaches in some example situations, because that is essentially what now every architect who is aware of those things has to do. If they look at something, they have to consider all the options that they have, and then they have to make an informed decision... So maybe we can talk a bit about the pros and cons, and about the applicability and lack of applicability of each approach, in different scenarios.
Markus Voelter: Yes. Should I start with the cons of DSLs?
Stefan Tilkov: Sure. Good idea.
Markus Voelter: Alright. So you need this tool to build the language, right? These are usually not the most mainstream tools you can get, so you have some kind of tool dependency. That's obvious. You might choose open source software, which makes this dependency less painful.
Markus Voelter: The next drawback is it's not a technique with which most developers you have on your team or you can easily get on the market have a lot of experience... So it's easy to screw up; not because it's rocket science, but just because people have less experience.
Markus Voelter: So these two things are rather obvious. I think the next two drawbacks are more interesting. One is that -- well, let me give you an anecdote. When the iPhone was invented or came on the market, obviously we all know Nokia ceased to be the dominant smartphone -- not smartphone, but mobile phone manufacturer. They had a hard time switching their focus to smartphones. And something I heard - I think it's true, but I don't have proof - is that they have put so much effort into very efficiently developing the old style of mobile phones that they were resisting to throw away all their tooling - I don't wanna call it a language, but their tooling - in favor of something completely different.
Markus Voelter: So basically, building a DSL and associated tools, because of the investment, and maybe your slightly narrow focus on what it is you do - it might narrow your focus to not see changing business requirements, or a completely different, changing world around you. That is certainly an argument.
Markus Voelter: And then the final one that is a drawback in my opinion is that depending on how you organize development of such a language -- I mean, it's very easy to make it waterfall. You first developed a language for two years, and then let's see. Of course, that is a stupid idea, and you should do it differently, and you can do it differently, but often, building infrastructure in that sense is often something that is an excuse of elfenbeinturming for...
Stefan Tilkov: Ivory tower...
Markus Voelter: Ivory towering, yes... For a while. And that's not a good idea. So these sort of drawbacks I come up with off the top of my head. Do you have additional ones?
Stefan Tilkov: Well, those are very good. I like the third one very much, because that is one of my pet peeves, the whole -- I like to call it "emotional mis-attachment". You invest so much into something that you become so attached to it that it becomes increasingly hard to throw it away. It's related to the sunk costs fallacy - you invested so much time in that particular thing... It was something that I briefly thought of in your introduction at the beginning, when you mentioned those things. I think that there is a certain legacy aspect to the thing. You have to be aware that what you're building is something that you will have to treat as legacy... In the sense that you've inherited it; you own it now, and you have to take care of it. You're responsible for that thing.
Stefan Tilkov: If I pick a general-purpose programming language, it's somebody else's problem to maintain that language. It's somebody else's problem to evolve it, right? I don't have to do it.
Markus Voelter: See, but this is one of those typical (I think) misalignments people make... Because it's not in this case about maintaining the language, it's about maintaining all kinds of libraries and other abstractions you build to build your applications, and those you have to maintain. So the comparison of a DSL is really more the framework or the set of libraries or the base platform (or whatever you want to call it) that you build for your domain. That is something people almost always do. So that's what you compared DSL to.
Stefan Tilkov: Yes. It's hard to define the exact line, where on the continuum do you sit.
Markus Voelter: True.
Stefan Tilkov: You can have the same discussions when you talk to people about whether it's a better idea to pick a mainstream programming language, a mainstream, general-purpose programming language, or something more fancy. That's a discussion that you have with, let's say, functional programming advocates... Which I am one; I like functional programming languages. I think they're an awesome idea; it's a really good thing. You just have to be aware that you get this additional level of expressiveness... If you believe -- I think most functional programming advocates do believe that their languages have additional expressive power, that you can write better programs because you're using a more powerful programming language.
Markus Voelter: True.
Stefan Tilkov: Regardless of whether that's true or not, even if it were true, or even if it is true, you also pay for the fact that you didn't use this other language. That's not really the point; I mean, somebody else is maintaining both of them... You should really be thinking about the code that you maintain in those languages, and maybe the code that you have to maintain if you use a more powerful language is easier, and fewer lines, so that's a benefit on the other hand; maybe fewer people can understand it when they just look at it, because they first have to learn this exotic programming language that they use. I'm just saying it's not a simple thing. It's a very complicated and a few nuanced things that you have to make your mind up about.
Markus Voelter: Yes. And of course, the criterion that you typically use is a trade-off in terms of the benefit and the cost. If you are, let's say, 30% or 50% more effective with your functional programming language, assuming you can get the good developers that know how to use it, then maybe that's worth the cost of using a less mainstream language. I would say for 30% to 50% probably not. With DSLs, we have had cases of a factor of 5 or 10 - not percent, factors of 5 or 10 - of increased effectiveness, or efficiency, or speed... And then, of course, there might be a little bit of a different trade-off. So the benefit has to be big enough to bite the bullet.
Stefan Tilkov: I wanted to add one or two more things... One thing is when you -- well, I'm jumping ahead of myself, but let me go back to the first thing I wanted to say... In addition to the fact that you have to maintain this thing, and you have to bear the costs of maintaining that thing for the foreseeable future, for the time where you want to maintain the programs in that particular DSL, you also might end up having to support multiple versions of the damn thing; something that happened to myself on one particular project... Differently from other projects where we did that approach. The idea of having a team that provided this language for multiple other teams.
Stefan Tilkov: So the other teams upgraded to newer versions of the language at their own speed... So the central team ended up having to maintain multiple versions, and then having to be able to do a migration from version n-3 to version n. And then "Oh, increase the costs. It's all doable. Nothing's impossible here, it's all doable..." But you just increase the cost, and you have to be aware that the benefits don't have to be great only at this particular moment, when you do a measurement, but also over the complete lifecycle of the whole thing...
Stefan Tilkov: Which brings me to the second thing, which is the 5x to 10x increase in productivity that you mentioned. I totally buy that, but the question is 5x to 10x compared to what? Is it 5x to 10x compared to a naive approach? I can very well imagine if I have a bunch of C programmers that just use the C language with its abstractions to build something that's quite complicated... You can increase their productivity drastically. Maybe you could have also increased it by moving to a different general-purpose language. Maybe you could have increased it by building some more abstractions into their core programming language... I don't know. I'm just saying it's not as easy and as straightforward as it might seem, that's all I'm saying.
Markus Voelter: Absolutely. Yes. And as everybody knows, to make good propaganda you have to choose the right comparison.
Stefan Tilkov: Oh, yes.
Markus Voelter: Yes, potentially changing to another programming language instead of C might solve some of these problems. On the other hand, in my contexts that's not an option, because you don't have another language, a different compiler for your processor... So you're forced to use C, and you have to make do with it.
Stefan Tilkov: Fair enough. That could be an excellent argument. I completely buy that argument.
Markus Voelter: Your customer, for example ESA, forces you to use C, and you have just no question. Or even worse, I've just recently learned that even for the most modern helicopters of the German armed forces, they still use Ada 83! I'm sure you could use some abstractions on top of that. But that is, if you will, a little bit of a special case. But there are other reasons... For example, let's say you think it is useful that your domain experts, who are not programmers - they might be doctors, healthcare professionals, they might be tech specialists, they might be public benefit civil servants... Let's presume you think it's sensible that they are able to express programming logic instead of writing requirements documents, or user stories, or something like that. You're not going to give them Clojure++ to build high-level function of function-as-value, mutable-immutable stuff... It's not going to work. You need to give them something that has less apparent complexity, it has language constructs that are more aligned with their world, it might use - again, as I said before - tables or mathematical functions, or the occasional diagram mixed with text to be able to express these things...
Markus Voelter: And it also turns out that in my domains, the actual complexity isn't the surface complexity. For example, in text calculations - well, it's a bunch of decisions and a bunch of math. Where is the problem, right? The problem is that you operate on temporal data, you get new versions of this stuff every year, you have to do basically semantic version-checking between subsequent versions, to make sure the calculations, if they cross multiple years that have different calculation rules, are still kind of consistent... How do you do that with a general-purpose language? You don't. Or at least I have no idea how to do this.
Stefan Tilkov: I agree. I agree. On the other hand, I've seen projects where this was the promise given in the beginning - the business, the domain experts can actually write code then, instead of writing a document that a developer uses... And in the end, it didn't end up that way, because the domain people couldn't do that. They just couldn't. It depends on the kind of domain people that you have.
Markus Voelter: Of course.
Stefan Tilkov: And I'm not dissing them, I'm not saying they're --
Markus Voelter: Obviously, you have to say in my case it works, obviously... But more seriously, the tools to build these languages have changed quite a bit. So it is more likely today that the approach can work than it was maybe 10-15 years ago with UML and a bunch of picture-drawing tools.
Stefan Tilkov: I completely buy that, yes. I'm sure the tools have gotten a lot better, and I definitely do think there are certain domains where these domain experts are really technical - and I'm not saying technical in terms of whether or not they can debate whether this or that microservices approach is the better one. I'm talking technical in their domain; they're actually very much able to express things completely consistently. They can use an appropriate tool for the whole thing.
Stefan Tilkov: You see people who are absolute wizards at tools like Microsoft Excel. They may not be great C programmers or Visual Basic programmers, but they just absolutely fantastically work with their thing there... Maybe controllers, or they may have some other job where their domain knowledge maps really nicely to the way Excel expresses stuff. I don't care. There are definitely people who can work with different kinds of tools. What I've observed a lot of times though is that this ended up being a benefit more for the reading than for the writing part... Which I think is still a good benefit; it's still a very good thing if the domain expert can read code that a developer wrote. "Let me show you this code. Look at it and tell me whether this looks as if it's correct." And domain people can spot mistakes there.
Stefan Tilkov: But of course, as you know, I can argue that that works for internal DSLs as well, because the problems that you rightly point out for the usability of internal DSLs or general-purpose code for domain experts is, of course, that the tooling sucks, and is not at all suitable for use by somebody who's not also an expert in the general-purpose programming language.
Markus Voelter: And not just the tooling, also the degrees of freedom you have to customize the notation.
Stefan Tilkov: True, yes.
Markus Voelter: It depends on the host language, but it's usually quite limited.
Stefan Tilkov: Yes. On the other hand, if the person who then ends up writing that stuff is the developer - and I've seen that as well; there's a developer who would be perfectly fine writing Java, Python, Ruby, C#, whatever code, but instead, they have to use some half-assed external DSL-based thing, and they keep cursing all day, throwing insults at the language engineer who built that thing... "Just give me my programming language. I know my IDE, I know all the shortcuts... I'm much more efficient with that standard programming language that I'm using all the time, instead of having to use that particular, super-specific tool." I'm not saying that's always true, I'm just saying that's sometimes true.
Markus Voelter: Yes, but you see - first of all, maybe half-assed is the problem...
Stefan Tilkov: True.
Markus Voelter: The other thing is you talked about maintenance before... So the use case of the guy who did the satellite software based on DSLs - and I can give you a link to a paper if you want to put it into the show notes...
Stefan Tilkov: Oh, yes. Please.
Markus Voelter: The point was that it's not just about expressing the code so the software runs, it's also about expressing things in a way that allows you to generate all the billions of pages of documentation that ESA wants, and a bus encoding for the communication protocol, and a mock ground station to command the satellite, and so on. So all of these things you can only automatically derive if you describe suitably, declaratively, in a suitably abstract way. You just cannot do that if you implement it in C. It's just impossible. So it doesn't really matter...
Stefan Tilkov: Well, maybe not in C... But if you have a decent -- well, I'm not dissing C either; it's a reasonable, useful programming language.
Markus Voelter: It doesn't matter about dissing, you cannot do that in C.
Stefan Tilkov: Well, I think what you can always do - and I seem to recall that; maybe I'm just imagining this now, but I seem to recall that's even in your book... I don't know. But of course, you can build a model with whatever technology you have; you can build a declarative model. It may become very unwieldy to do that, but you can design whatever it is that you want to model as an explicit model, that's addressable, that's data. Maybe in an OO language you'll build a suitable object model for expressing that thing; it may be a pain in the ass to do all of that work...
Markus Voelter: Yes, exactly.
Stefan Tilkov: ...and it might be much easier to just use the external DSL tool. I completely buy that. It might be, but it also might not be, right? So you can, of course, do it. It's then part of your program, and it's written in one of those general-purpose languages... Again, you can argue - is it better or worse that it's written in that language? It's now additional code that is there... But it's understandable by everybody who can use that thing. It doesn't add any additional dependency on any external tools. You have no dependency on an open source provider, or a commercial provider, or anybody. You have a dependency on yourself; it's like saying "I'd rather write this myself than use that library or that tool from somebody else that I may or may not trust." It's a decision that you have to make.
Markus Voelter: Yes, of course. And I'm not suggesting you shouldn't carefully decide whether investing into a DSL is a good idea. I do think that is true, as is for any other technology and approach you use. Maybe here it's a little bit more pronounced, because it's such a "niche" technology. But again, you just cannot ask your doctor, healthcare guy to write Java code that assembles some kind of object graph, which is then the input to a generator... Which is what you'd have to do if you have no language. It just doesn't fly.
Markus Voelter: In theory, what you say is completely true, you could do that, but in practice you can't. If you want your domain expert to do that, you need to give them a good language, with a good tool.
Stefan Tilkov: Again, I completely agree if they are supposed to write the whole thing. You can't give a domain expert, you can't give anybody who's not--
Markus Voelter: Even reading.
Stefan Tilkov: Reading, I don't agree with that. That depends on the power and expressiveness of the programming language that you're using. It's absolutely possible if you've got a high-level language, let's say a Lisp, or a Ruby, or maybe even a SmallTalk. Then I think you could get away with that. You can write code that domain experts can read and assess and judge...
Markus Voelter: Yes, but it works quite well for structures, it works less well for behaviors.
Stefan Tilkov: Again, I would argue it depends on the kind of behavior, right?
Markus Voelter: Of course it does. But the kind of stuff I do in my domains - it's maybe not impossible but it's hard.
Stefan Tilkov: True... Which is maybe also a good, positive example of selection bias. Maybe it's just that your knowledge and your experience gets applied to domains where it makes perfect sense, so that's perfectly fine then.
Markus Voelter: Which also -- instead of calling it selection bias, I make sure I don't sell people a hammer if they don't have nails.
Stefan Tilkov: I intended it to be positive... I mean, of course --for example we build a lot of web-based systems. We only occasionally build native apps. So most of the time, that works perfectly well. Maybe that's just because web apps are better than native apps, or maybe …
Markus Voelter: I don't understand how that relates.
Stefan Tilkov: Well, I'm saying the selection bias means that you're chosen for it. You know something you're good at, that works well, and that sort of reinforces itself. I didn't mean it as an insult. Other things maybe, but not that.
Markus Voelter: [laughs] Thank you.
Stefan Tilkov: The other thing -- what was it that I wanted to say...? I do think that the cost is also generalizable. This cost discussion, cost-benefit thing, as you said, is true for everything. I think the cost-benefit thing is also in general true for specific versus generic tooling in general. You can have a thousand examples where there is a benefit to picking some general-purpose thing versus a specific thing, and the other way around as well, and you just have to find the right balance for your particular situation there. So we could definitely agree on that. And I trust that you don't sell stuff in a snake oil fashion to make everything, everywhere always require the engineering of a new language.
Markus Voelter: No.
Stefan Tilkov: On the other hand, I do think - and maybe that is another observation from a little more distance... I think that, as with many hypes, the model-driven thing was way more hyped for a certain amount of time than it is today.
Markus Voelter: Yes.
Stefan Tilkov: When it was very popular, expectations were way too high, overblown... Not by you, but by other people. And then when it got the inevitable backlash, it sort of ended up the same way as many other things as well, sort of throwing out the baby with the bathwater. There's a lot of situations where this simple concept of having something that's declarative, something that's an engine, something that's interpreted, something that allows you to reason about that thing that you want to express is an absolutely awesome idea, independently of how you express it, and many people don't have that idea anymore. That's a little sad, I think.
Markus Voelter: True.
Stefan Tilkov: I'm not sure whether I'm making myself clear, but that's one thing that really annoys me - people don't even have this concept as part of their toolchain anymore. I'm probably an angry old man shouting at the clouds.
Markus Voelter: So if we go back to try to connect this to DDD, which about an hour ago was our starting point... So let me express my hypothesis here, which is if you have a ubiquitous language the intent of which is to make sense to business people and to technical people, and to faithfully represent what your business is about, then I think - not in all, but in many considerations, it makes sense to allow the business people to express themselves in that language. You want them, if the language is ubiquitous, to be able to write sentences in that language, and basically to define and analyze and test (by the way, that's important) the business logic of the system they're building themselves.
Markus Voelter: Of course, this only makes sense if you have a domain where there is enough core business logic amount and complexity. And that's not always the case. But if you buy it so far, then I think building a DSL with appropriate abstractions/annotations is a valid - if you will - extension or implementation of the idea of DDD. And my experience is that many DDD people just don't see that like that at all. That was the starting point for my thoughts on this topic. Why is that?
Stefan Tilkov: So I would agree with you. I would at least in principle agree with you, because you're right that the core model that you're supposed to define - which can include behavior, if your language covers behavior as well - should be something that is shared, that is owned, that can be understood and reasoned about and talked about to the business people... So I think that is an actually perfectly valid implementation. Probably I say that because I have the background in that language engineering thing, a bit at least, as well. You know I used to do a lot of MDA type stuff, or whatever. Not the same tooling, but the same general concept. People who don't share that will take much longer to convince. That could be just because they don't know it, and it could also be because they don't see the additional value.
Stefan Tilkov: Maybe there are two things related here. One is that if you're following the DDD idea, then the domain model that you implement will be relatively free of technical details; something that I used to use as an argument for a model-driven approach always was that you don't want your business knowledge mixed with all of the technical details...
Markus Voelter: POJO, blah-blah-blah.
Stefan Tilkov: Exactly. Right. So in the DDD world it's ideal - not always achieved, but it's definitely a goal – to have a business model, a domain model that really has no technical dependency to any database or UI framework or whatever it is that surrounds this core set of abstractions. So people said "Yeah, of course, I could switch Java and use external DSL XYZ instead, but I don't see much additional benefit. To me, it's just changing a very popular programming language to another one..." And of course, you would argue that is a completely invalid comparison, but that is I think what goes through people's minds there. Sure, I have a language that's more useful, but it's not changing anything fundamental.
Markus Voelter: It does, and that is the thing I seem to be unable to get over, or to get across... Not just to you, or not to you, but I once submitted -- go ahead.
Stefan Tilkov: I definitely think it changes something. It might even change a lot if it's a good language, that allows you to express things much more concisely than what your general-purpose language would allow you to do... Which is not necessarily the case; it depends, obviously, on how well the language is designed.
Markus Voelter: The point is it allows other people to do it, and that is the core benefit.
Stefan Tilkov: Right.
Markus Voelter: It brings these people from document-writers into concrete use producers; that's stupid wording.
Stefan Tilkov: As you say, I think that is true if they are experts in something that merits this. I think in many cases - and that's my personal experience as well - you have domain experts who really know a lot, but they're really not programmers, and the knowledge that they have is not expressible in... It doesn't matter which language you gave them because there exists no language that matches their knowledge, because their knowledge isn't as finely structurable, if that's a word... They're not physicists who talk about the formulas of some observable system. They're not health experts who talk about the symptoms and treatments of something; they're more general-purpose...
Markus Voelter: Experts...
Stefan Tilkov: ...than that. Yes, exactly. That's what they are. They know a lot of different aspects of different parts of the domain, so it's very hard to find additional benefit beyond what a general-purpose language would provide.
Markus Voelter: So this is where I just totally and fundamentally disagree. For example, you just talked about doctors, right? The domain or market of digital therapeutics is a huge and growing market... And the thing you do there is you formally - and you have to do it formally, because of FDA certification - define treatments and diagnoses. Almost everything that has to do with financial products is of that sort. Everything that relates to law, government, public benefits calculations, tax calculations is like that. Not even talking about technical domains or science-oriented domains.
Markus Voelter: So while I totally agree that there are domains or applications where the knowledge isn't structurable or formalizable, there is a huge set of domains where this is possible and people still write stupid documents.
Stefan Tilkov: I would agree with that.
Markus Voelter: And that is where I'm interested in improving the situation.
Stefan Tilkov: Yes, I agree with that. Maybe we can agree to this -- let me try to phrase this in a different way. There are certain things -- I think I had a similar discussion with Mike, actually, in one of the last episodes. I'll look that up and link to it. I think there are some parts, some domains, or maybe even some parts of a system where this is absolutely true, where there are rigorous definitions, where there is absolute truth, where there is something that's right and something that's wrong, and something that may be legally required; so the guideline, or a law, or a regulation of some kind. There might be some fundamental laws of physics that we're talking about here; there might be some mathematical relationship here.
Markus Voelter: Even the marketing guy who defines pricing strategies for train companies... It doesn't have to be about physics.
Stefan Tilkov: Let me try to draw the boundary here. I think the problem begins once the humans enter. Once the human part enters into this whole thing, it becomes less well-defined, and it becomes much softer, much more squishy. It becomes like this annoying, un-mathematical part of the system. And I think that is very often the major part of the system.
Markus Voelter: Give me an example.
Stefan Tilkov: That's the part where all of the other things start to matter... I can phrase this as a critique of domain-driven design, if you want me to. I think the whole concept of having this model, this core model in a general-purpose programming language, without any relation to UIs and databases is completely overblown. It's a very useful thing - I'm not saying it's not useful - but to assume that that model in itself is the core value of your system is a fundamental misrepresentation of basically every system I've seen in the last 30 years... Because basically, every system I've seen was way more defined by its usability or lack of usability at the UI part. The way it made you jump through hoops when you wanted to do something. That complete lack of being usable for anybody.
Stefan Tilkov: All of these things -- like, you don't know what you want to have unless you've seen one version of it, and they say "Well, that sucks. Give me that differently." And this means that your system evolves into something that is more usable, or maybe you do a study with some people beforehand to figure out what it is they want to do... This is not a mathematical thing, and I don't think it's addressed by these things. It doesn't mean that they're not useful, or anything; it just means that -- and we can probably take that tax system as an example.
Stefan Tilkov: I don't doubt that the German tax system is probably the most complicated one in the world, and it's absolutely regulated and there are tons of rules and it makes zero sense to write that down in documentation form and then have somebody manually translate... I totally get why you would want to formalize that. The interesting question is "Is that 98% of the resulting system, or is it 20%?" And do you address the problems - without talking about that particular system - do you address the long-term problems, the evolution of that system with that engineering approach, or with a bunch of other engineering approaches? That, I think, is an important discussion.
Markus Voelter: I can give you an example that may give a hint... So first of all, I agree - there's no point in having a correct (let's say) core in terms of data model and behavior if the UX is so bad that nobody wants to use it. True, absolutely. Let me give you a little anecdote. The Dutch tax agency - they use this approach; they have implemented all their tax law as models expressed with a bunch of DSLs. And there is this nice anecdote where years ago the government, if they wanted to make a change to tax law, they called the tax agency and said "Hey guys, if we make this change, can you change the software by the beginning of next year to make sure we can actually run this new law?" Because it took them time to evolve the software.
Markus Voelter: Today they still call, and they're like "Hey, we're thinking about making this change to the law. Can you by next week basically create a branch of your rule set, make a change, run it and see how much it benefits or costs the government?" So they have become so much more agile in terms of changing their business logic that the whole process of interacting with the lawmakers completely changes. So it's really about changing the actual behavior at the core of the system. It doesn't relate at all to UX at this point, or something like that.
Stefan Tilkov: Right. That could be something that I could absolutely see as an ideal model; that's not going to happen any time soon, but it could be an ideal model. The law itself should be written in a form that is declarative and provable and machine-checkable.
Markus Voelter: That will never happen.
Stefan Tilkov: I know it won't, but it would be cool if it did, because then that would be shipped to everybody who wanted to use it. They could have an interpreter to run that thing, or they could translate it to C, or they could do whatever... But they still - and that's my point - would have to build an actual system around that thing, and I think large parts of that thing would not be addressable by that particular aspect.
Markus Voelter: That is true.
Stefan Tilkov: But I think we can agree on so many things that I think that is a good chance to sort of wrap this up, because we're at the limit of our time. If there is something that you really, really want to discuss before we do that, let's do it now.
Markus Voelter: Well, I would briefly like to connect to one thing you said before, and it won't take more than five minutes, I hope. You said that one reason why many DDD people don't even think about building a DSL, even in the cases where we probably both agree it would make sense, is that they don't have exposure to the technology; they did it when they were at university, using some crappy, old tool, and it was complicated... And you don't even have to talk about usability, so they just forgot it again. So do you think it would be useful for people to -- I mean, is it a matter of exposure, of knowing that this technology exists? Or is there a more fundamental issue? And assume that it is a case where we would probably agree it makes sense to use the approach, like it's a suitable case.
Stefan Tilkov: I'm not sure. Maybe it's something that you would have to generalize again, to a different level. Talking about communities, and whether something from another community can enter -- what are the conditions that something from a different community can enter one particular one?
Stefan Tilkov: Let me draw a completely weird analogy... For historical reasons, I was part of the Java community, as well as the Ruby community. So that was one of the reasons I liked something called JRuby, which is a JVM environment for running Ruby programs. And I respect the people who built that thing; I liked it a lot, and we've used it on a number of projects, and it sort of worked, so we were kind of happy with that... It's close to impossible to get people in one community to even look at that thing, to get the Java people to look at Ruby as a valid option for writing programs, or the Ruby people as using the JVM to run actual Ruby programs. It's as if you're Bozo the Clown doing a stupid joke.
Stefan Tilkov: I'm not saying that is the same thing here. I think the DDD community is way more open and diverse. It now sounds more negative than I wanted it to sound... It is kind of an open community, so I think it has open eyes, and open ears, and would listen to stuff like that... But I think it's skeptical, as probably many other communities are as well. And I'm not sure it's just that. I think there are many experienced people who probably have -- maybe that's one of the problems. Maybe one of the problems is that there are experienced people in the DDD community who've seen code generation go wrong - which, of course, can happen, but is not a fundamental critique of the concept; it's just something that people -- if once you have a negative experience with something, it really becomes tainted a bit. The whole idea gets sort of tainted with that particular experience and it's hard to get away from that. So in my personal point of view, I wouldn't hold my breath for a strong adoption of any of those concepts there.
Markus Voelter: No, of course, I don't. That makes me kind of sad... Because I think there is potential quite a bit of synergy in many - not all, of course - cases.
Stefan Tilkov: Well, personally, I think it really depends on the kind of approach you take in general. I personally am more of a generalist. I don't want to identify too much with any of those one particular things, and I think that makes it easier to use different things in different situations. Once you strongly identify with whatever concept it is, it's kind of natural, and only human, and reasonable, and commercially viable to stick to that particular thing... So it's really a matter of strategy. I'm not sure what to do about that. It sounds like we're ending on a sad note here...
Markus Voelter: Yes, we do.
Stefan Tilkov: But maybe we can get fantastic reader feedback...
Markus Voelter: Listener.
Stefan Tilkov: Listener feedback, by awesome listeners from the domain-driven design, or language engineering community, who will tell us how this is actually working for them and their particular projects. That would be cool.
Markus Voelter: Yes, that would be a good outcome, indeed.
Stefan Tilkov: Markus, it was great and awesome fun to have you on the show. I would love to repeat that again.
Markus Voelter: Yes... But we need to find a different topic, right? ...which of course won't work, because this is the only thing I know about, as you just kind of blamed me for!
Stefan Tilkov: Oh, you're just a liar. No, we'll manage something. It was great, thank you for your time.
Markus Voelter: Yes, absolutely.
Stefan Tilkov: Thanks to our listeners for listening. Please check the show notes, and until next time, bye-bye.
Markus Voelter: Ciao!