This fits my gut feeling every time I see an ECS system that videogame design has gotten stuck in a local maxima abstraction/pattern. Often what they really want is a Monadic abstraction of a data/state transformation process, but they are often stuck in languages ("for performance reasons") that make it hard to impossible to get good Monadic abstractions. So instead they use the hammers that to make nails of the abstractions that they can get. ECS feels to me like a strange attempt to build Self-like OO dynamic prototypes in a class-based OO language, and that's almost exactly what you would expect for an industry only just now taking baby steps outside of C/C++.
C# has some good tools to head towards that direction (async/await is a powerful Monadic transformer, for instance; it's not a generic enough transformer on its own of course, but an interesting start), but as this article points out most videogames work in C# today still has to keep the back foot in C/C++ land at all times and C/C++ mentalities are still going to clip the wings of abstraction work.
(ETA: Local maxima are still useful of course! Just that I'd like to point out that they can also be a trap.)
The quotes imply that this is a bad reason, but in soft realtime systems you often want complete control of memory allocation.
Even in the case of something like Unity--in order to give developers the performance they want--they've designed subset of C# they call high performance C# where memory is manually allocated.
In most cases if you're using an ECS, it's because you care so much about performance that you want to organize most of your data around cache locality. If you don't care about performance, something like the classic Unity Game Object component architecture is a lot easier to work with.
Yea, you're right, I think the previous poster seriouspy underestimates videogames as performance critical (and performance consistent!) apps. In the modern days of desktop Java and C# (and even more in web dev) the vast majority of coders just don't come across the need to "do everything you need to do" in 33ms or less, consistently.
I'm not implying it is a bad reason with the quotes, I'm trying to imply that it is a misguided reason (even if it has good intentions).
The "rule" that C/C++ is always "more performant" is just wrong. It's a bit of a sunk cost fallacy that because the games industry has a lot of (constantly reinvented) experience in performance optimizing C/C++ that they can't get the same or better benefits if they used better languages and higher abstractions. (It's the exact same sunk cost fallacy that a previous games industry generation said C/C++ would never beat hand-tuned Assembly and it wasn't worth trying.)
In Enterprise day jobs I've seen a ton of "high performance" C# with regular garbage collection. Performance optimizing C# and garbage collection is a different art than performance optimizing manually allocated memory code, but it is an art/science that exists. I've even seen some very high performance games written entirely in C# and not "high performance C#" but the real thing with honest garbage collection.
(It's a different art to performance optimize C# code but it isn't even that different, at a high level a lot of the techniques are very similar like knowing when to use shared pools or deciding when you can entirely stack allocate a structure instead of pushing it elsewhere in memory, etc.)
The implication in the discussion above is that a possible huge sweet spot for a lot of game development would actually be a language a lot more like Haskell, if not just Haskell. A lot of the "ECS" abstraction boils away into the ether if you have proper Monads and a nice do-notation for working with them. You'd get something of the best of both worlds that you could write what looks like the usual imperative code games have "always" been written in, but with the additional power of a higher abstraction and more complex combinators than what are often written by hand (many, many times over) in ECS systems.
So far I've not seen any production videogame even flirt with a language like Haskell. It clearly doesn't look anything like C/C++ so there's no imagination for how performant it might actually be to write a game in it (outside of hobbyist toys). But there are High Frequency Trading companies out there using Haskell in production. It can clearly hit some strong performant numbers. The art to doing so is even more different from C/C++ than C#'s is, but it exists and there are experts out there doing it.
Performance is a good reason to do things, but I think the videogames industry tends to especially lean on "performance" as a crutch to avoid learning new things. I think as an industry there's a lot of reason to avoid engaging more experts and expertise in programming languages and their performance optimization methodologies when it is far easier to train "passionate" teens extremely over-simplified (and generally wrong) maxims like "C++ will always be more performant than C#" than to keep up with the actual state of the art. I think the games industry is happiest, for a number of reasons, not exploring better options outside of local maxima and "performance" is an easily available excuse.
I think you may be indexing a bit into your exposure and painting the industry in a broad brush.
I've seen impressive things done with Lua, from literate AI programming with coroutines to building compostable component based language constructs instead of standard OOP. You have things like GOAL[1] which ran on crazy small systems (the Lua I saw ran in a 400kb block as well).
On performance, data oriented design and efficient use of caches is the way you get faster. I've done it in Java, I've done it in C#, I've done it in Rust and C++. Certain languages have better primitives for data layout and so you see gamedev index into them. We used to do things like "in-place seek-free" loading where an object was directly serialized to disk and pointers were written as offsets that were fixed up post load. Techniques like this easily net 10-30x performance benefits. It's the same reason database engines run circles around standard language constructs.
You are correct that I am using a broad brush, but so far it is a broad brush sort of conversation. I realize I'm a (bitter?) cynic at this point and don't have a lot of respect for the videogames industry as a whole from a technical perspective, because it is an industry that prides itself on reinventing wheels, not sharing a lot of efforts between projects, and not trusting nor retaining expertise in the long run. I realize there are a lot of great and novel approaches such as the ones you mention (I appreciate that), but so much of the novelty is siloed and a lot of what I see as a certain kind of outsider is the tiniest slices of things that escaped the siloes such as Unity and Unreal. I realize they aren't accurate to the state of the technical art in some siloes, but these days given the number of games using one or the other of those two common engines today it certainly reflects the "state of the technical median".
> It's the exact same sunk cost fallacy that a previous games industry generation said C/C++ would never beat hand-tuned Assembly and it wasn't worth trying.
C/C++ hasn't beat the performance of hand-tuned assembly - it has simply gotten close enough that the cost of hand-tuned assembly is not worth it in most cases.
Seconding this - some esoteric JVM garbage collector tuning is required to build a high performance Java (or Clojure, etc.) system, but it can be done.
It's arguably significantly less work to learn how to tune the GC and then optimizing it for your situation than it is to deal with manual memory allocation and all of its fallout.
If memory serves, you (only) need two or three times the memory for GC which work well(low pause) than for manual allocation --> I'm surprised that developers of PC games didn't switch to GCs 'en masse'..
And no, I'm not joking: I work in C++ and I know exactly how annoying memory errors can be.. Thanks a lot valgrind|ASAN developers!
If your optimization goal is "use the least memory possible," then sure, manual memory allocation is the way to go. I was addressing a different optimization goal: the "high performance" case, meaning approximately "high throughput, low latency operation."
There is a common misconception that GC invariably precludes the construction of a "high performance" system, which is not true. If your use case allows you to not care as much about larger memory consumption -- 2x to 3x does seem like a reasonable first approximation of "larger" -- then GC is indeed a viable option for building "high performance" systems.
This case is not uncommon. Not everyone is targeting a memory constrained console or embedded system.
In many (though of course not all) cases, the tradeoff is well worth it -- consume more memory at runtime, spend some time tuning the GC, and in exchange developers can ship a product faster, by having to spend significantly less time dealing with manual memory allocation.
>If your optimization goal is "use the least memory possible," then sure, manual memory allocation is the way to go. I was addressing a different optimization goal: the "high performance" case, meaning approximately "high throughput, low latency operation."
Ignoring the amount of memory used, GC tuning a managed language doesn't give you the flexibility to control memory layout needed for maximum cache locality.
>If your use case allows you to not care as much about larger memory consumption -- 2x to 3x does seem like a reasonable first approximation of "larger" -- then GC is indeed a viable option for building "high performance" systems.
Not ignoring amount of memory used. In the context of this thread--video games specifically "high performance" video games--2x to 3x is almost never going to be acceptable.
I can tell you why this switch didn't happen: 2x to 3x the memory usage is just absolutely abysmal for a process that is barely fitting into memory as it is. Most of the games that run up against these constraints are multiplatform titles targeting consoles that are notoriously stingy with main memory to reduce cost.
>The "rule" that C/C++ is always "more performant" is just wrong.
Who said it's a rule? What C/C++ gets you is the ability to manually allocate memory without jumping through hoops.
> Performance optimizing C# and garbage collection is a different art than performance optimizing manually allocated memory code, but it is an art/science that exists. I've even seen some very high performance games written entirely in C# and not "high performance C#" but the real thing with honest garbage collection.
Performance optimizing C# with garbage collection for high performance soft realtime systems (I've done it) relies on tricks like object pooling to avoid triggering GC along with avoiding many of the more advanced language features. Even then you don't get the same level of control. I'm also almost completely certain that the high performance C# games you're talking about aren't using C# for the engine, but feel free to provide examples so I can take a look.
If your game (or parts of your game) doesn't need the performance that comes with a higher degree of memory layout control, then by all means use whatever tools you want to.
I've written game logic in C#, F#, Ruby, Haxe, Python, Lua, Java, JavaScript and Elixir.
>The implication in the discussion above is that a possible huge sweet spot for a lot of game development would actually be a language a lot more like Haskell, if not just Haskell.
There almost certainly is for game logic. Many modern game engines provide higher level scripting languages.
However, if what you are working on is in that sweet spot, you likely didn't need an ECS to begin with and a classic component architecture would have probably been a lot easier to deal with.
>But there are High Frequency Trading companies out there using Haskell in production.
HFT is not game dev. "Performance" in HFT doesn't mean the same thing as performance in games.
I haven't used Haskell specifically, but I've toyed with using Elixir for gamedev. It's reliance on linked lists makes it extremely difficult to iterate quickly enough. There are work arounds of course, but the work arounds remove most of what is nice about Elixir in the first place.
>Performance is a good reason to do things, but I think the videogames industry tends to especially lean on "performance" as a crutch to avoid learning new things. I think as an industry there's a lot of reason to avoid engaging more experts and expertise in programming languages and their performance optimization methodologies when it is far easier to train "passionate" teens extremely over-simplified (and generally wrong) maxims like "C++ will always be more performant than C#" than to keep up with the actual state of the art. I think the games industry is happiest, for a number of reasons, not exploring better options outside of local maxima and "performance" is an easily available excuse.
The average engine coder writing high performance code in C++ isn't a "passionate teen". They are experienced software engineers who want to stick as close to the metal as they feasibly can.
The games industry (outside of AAA games) also has an extremely low barrier to entry, and it's something that nearly every programmer has thought about doing at some point--if Haskell turns out to be a fantastic language for making games, it will almost certainly happen sooner or later.
> The average engine coder writing high performance code in C++ isn't a "passionate teen". They are experienced software engineers who want to stick as close to the metal as they feasibly can.
Statistically the median age in the games industry is 25 and always has been. It's a perpetually young industry not known for retaining experienced talent. I know that statistically the median doesn't tell you a lot about how long of a tail there is of senior talent, you need the standard deviation for that, but given what I've seen as mostly an outside observer with a strong interest the burn out rate in the industry remains as high as ever and senior developers with decades of experience are most likely to be an anomaly and an exception that proves the rule than a reality. In terms of anecdata all of the senior software developers I've ever followed the careers of on blogs and/or LinkedIn are all in management positions or entirely different industries after 30. I realize my sample size is biased by the people I chose to follow (for whichever reason) and anecdata is not data, but statistically it's really hard for me to square "experienced software engineers" with "in practice, it looks like no one over 30".
>Statistically the median age in the games industry is 25 and always has been.
Where are you getting this information from? The only hard data I can find is from self selected survey responses, but this survey from IGDA shows only 10% of employed game developers are under 25 [1]. My guess is that (as you've acknowledged is possible) there's some serious selection bias going on. You said you have an interest in burn out rate, so I'm guessing you're more likely to follow/notice game devs who discuss this topic. This group is more likely to be suffering from burn out I'd wager.
Another poster already mentioned that engine devs (the one's writing most of the C++) tend to be older than the industry average.
In game dev, there has been a really serious split between engine development and game (logic and content) development. Most of the talented and experienced programmers seem to drift towards engine development. That's where the hard problems are and where these guys can have the most impact. As a bonus, engine development cycles are not so closely coupled to game release dates anymore, so crunch is less of an issue in engine teams.
C# has some good tools to head towards that direction (async/await is a powerful Monadic transformer, for instance; it's not a generic enough transformer on its own of course, but an interesting start), but as this article points out most videogames work in C# today still has to keep the back foot in C/C++ land at all times and C/C++ mentalities are still going to clip the wings of abstraction work.
(ETA: Local maxima are still useful of course! Just that I'd like to point out that they can also be a trap.)