how much of the executable is understood?

[sthalik]

Hey,

I'm yet another C89/C++17 programmer with some knowledge of assembly and x64dbg. My IDA skills aren't up to par but this is a matter of time, given the amount of information available online.

The question -- how much of the binary is understood? That includes the data strutures which layout and semantics are understood. Do you consider it feasible exporting the common logic (AI, stat modifier calculation, core drawing of everything, anything else that is useful) into a language higher-level than assembly?

Ideally that kind of work would be done privately at first, with functions transcribed word-by-word from the IDA decompiler. Only then the logic could be improved.

My aforementioned approach is the same one as the one taken by PRACX where the code is hooked, rather than written in x86 assembly by hand. I was surprised how well IDA's decompilation and structure mapping actually works. That can speed up the process greatly.

Is there any similar work done already? Have you had trouble mapping out the data structures and isolating the individual code bits?

I'd also like to point you toward Starcraft 1 for ARM, done entirely as a machine translation with some bits rewritten by hand. But this was done without any intermediate high-level language, thus not getting understanding. However, seeing Starcraft for the Raspberry Pi shows this isn't a pipe dream.

The especially interesting bits here are:

- making drawing more efficient at outer zoom levels (caching the immobile map parts, rather than drawing each sprite each frame)
- changing the AI to be "pluggable" as an external library, in the spirit of BWAI
- moving more code out of the assembly in general
- new layouts for HiDPI screens
- active legal threats
- possibly indepth modding in the long run

Is that feasible at all? Have you documented the existing understanding of the binary?

sh

[bvanevery]

Since nobody has answered so far, I will chime in.  I have no idea about the question you asked.  My chime:

At what point would you personally accept that it is more feasible to write a brand new game?

At what point are you willing to merely steal the art assets of an old game?  I mean, I'm not legally sure in all countries, but I think technically the binary patches to the game are a violation of copyright anyways?  I'm asking, at what point are any principles being stood upon.  Why not just make a clone SMAC project and if you want to CYA, first say "you must first install SMAC to use this" ?

I've dealt with a lot of open source projects over the years, including trying to involve myself with FreeCiv at one point.  Someone tried a "mod it for SMAC" effort at one point, but they didn't get very far and abandoned it.  This is a pattern with most people in open source: they propose pie-in-the-sky projects, they get started, they run into trouble, they get bored, and finally real life intervenes with money and job pressures.  There's a very limited timeline that anyone has for success.  A lot of people think they have lots of time to work on things because they're "not being paid and doing it in their free time".  What such people usually fail to realize, is that personal concentration is a finite resource that is evaporating.

In short, how feasible is it for you to do any of the things you suggested?  Do you have a track record of actually shipping open source projects, and having them be successful for some period of time?

Myself, I managed it once.  Basked in the glory for 6 months of wonderfully working code.  Then had a fight with the project lead, who dumped my code.  A man-year of effort down the drain on my part.  Well, that taught some things about self-destiny.

[sthalik]

At what point would you personally accept that it is more feasible to write a brand new game?

At what point are you willing to merely steal the art assets of an old game?  I mean, I'm not legally sure in all countries, but I think technically the binary patches to the game are a violation of copyright anyways?  I'm asking, at what point are any principles being stood upon.  Why not just make a clone SMAC project and if you want to CYA, first say "you must first install SMAC to use this" ?

They are in violation of copyright, but no copyright holder cares. This is good.

Let's consider this a thought experiment. What would make for a worthy successor?

For writing a brand new game we'd need great story designers and writers. Also some voice actors but that pales in comparison to the amount of work done in SMAC on significant and memorable quotes as well as the consistent and interesting universe. I won't ever have *that* kind of fun with Pandora or Beyond Earth due to the typical trite. Just listen to BE's main menu theme.

I've thought of S. Lem's "Invincible" as similar, in a way. The clanking replicators have a certain scariness to them that wouldn't come from an intelligent enemy.

Also consider Underrail's bioengineering catastrophe as a similar backstory. Then see the Hollow Earth complex from the same game as replacement for the game's area. That could work.

There's a certain breed of player that considers the atmosphere and narrative first and foremost. I particularly love the motif of amoral power-grabs from techs and AI diplomacy. It's overarching through a campaign. Unless any team was to improve upon (or just plain replicate) the atmosphere and narrative of SMAC, the game won't be a successor. Seemingly other games haven't even tried, due to the great amount of manpower that wouldn't be needed otherwise.

I've heard it said "If not for ideology, Hitler and Stalin would agree on the economy approach a lot". It's funny, but also true in a way. The factions' ideology-based distinction in SMAC is what couldn't be done better.

I've dealt with a lot of open source projects over the years, including trying to involve myself with FreeCiv at one point.  Someone tried a "mod it for SMAC" effort at one point, but they didn't get very far and abandoned it.  This is a pattern with most people in open source: they propose pie-in-the-sky projects, they get started, they run into trouble, they get bored, and finally real life intervenes with money and job pressures.  There's a very limited timeline that anyone has for success.  A lot of people think they have lots of time to work on things because they're "not being paid and doing it in their free time".  What such people usually fail to realize, is that personal concentration is a finite resource that is evaporating.

In short, how feasible is it for you to do any of the things you suggested?  Do you have a track record of actually shipping open source projects, and having them be successful for some period of time?

I've been maintaining a fully-featured open-source program for 6 years. The issue with involvement is 80/20 -- first 80% are fun to write. The last 20% are critical fixes and they take 80% of the time. Maintaining a larger team with diverse skillsets is another can of worms though. Look up opentrack/opentrack on Github, and my own username. After a stable release it has 15k downloads a month. Normally it's some below 10k a month.

Myself, I managed it once.  Basked in the glory for 6 months of wonderfully working code.  Then had a fight with the project lead, who dumped my code.  A man-year of effort down the drain on my part.  Well, that taught some things about self-destiny.

It takes a great amount of humility (in addition to plain responsibility) to be a project lead and not fragment/demoralize the team long-term. Dumping working code is unprofessional at best. I've only _replaced_ code that was bad or architecturally unsound... Keeping the team's relationship strictly professional could alleviate problems you experienced. To a degree, of course. When a policy's to be made, there's a difference between a firm stance and an abusive flamefest. Well, that's what helped me survive as a maintainer/lead programmer for six years.

Why haven't you forked the project? In fact my project started as a fork in 2012.

[bvanevery]

For writing a brand new game we'd need great story designers and writers.

I agree.

Also some voice actors

Not necessary.  It's a production bell and whistle.  Write the best dialogue in the 1st place, don't worry about whether someone is speaking it.  Some of the quotes and voicings are clever and hilarious in SMAC.  The exemplar for that would be "Recon Rover Rick".  Others are clearly just filler, had to get the project done for a long tech tree and didn't have anything interesting to say.

I've been maintaining a fully-featured open-source program for 6 years.

Well good, then we don't need to go through any more "amateur hour" admonitions.  The question still stands though.  When do you accept that it's easier to write a new game?

Why haven't you forked the project? In fact my project started as a fork in 2012.

Because I was the CMake build system guy, not the Scheme compiler writer guy.  I did not have enough understanding of compilers at the time to develop the core compiler.  The strategic issue is that although the guy allowed me to create the fully working Visual Studio build, he had no personal committment, investment, or incentive towards Visual Studio or CMake.  His concern with Windows was minimal as well.  He would complain about all the extra work CMake was causing him.  Since of course I had done the heavy lifting to make it all work, my reaction is like what am I, chopped liver?  He threw out the code; I had hoped the small community around the compiler, would see the value of keeping this thing working with the CMake build.  But push come to shove, nobody else wanted to do the ongoing support for CMake and Visual Studio.  Windows wasn't sufficiently important to them.

CMake has become standard drill for so many projects since then.  No it is not a lovely build language, and I won't be doing any more of that, as that part of my so-called career is long since over.  But what a waste, back then.  I learned that I can't drag people into support situations they basically don't care about.

[sthalik]

I've been maintaining a fully-featured open-source program for 6 years.

Well good, then we don't need to go through any more "amateur hour" admonitions.  The question still stands though.  When do you accept that it's easier to write a new game?

Could it be that you yourself are interested in writing a game from scratch? That's very interesting in itself, but for the "what's easier" argument's sake:

I'm sorry but so far you haven't proven your case. Creating a new codebase for the same assets (either based on the original executable to the point of being a derived work, clean-room engineering, or not at all based upon it) is by far easier, and has guaranteed results.

Also note the finished Diablo 1 complete rewrite "Tchernobog" as well as "freeablo" in earlier development phase. Is it proof enough? Few repeatable bugs notwithstanding, Tchernobog actually works as a widescreen Diablo 1 multiplayer replacement, with few modernized features like mousewheel zoom.

As far as SMAC or "Tchernobog" are concerned, a code rewrite for SMAC can be done by a single person, only with drive-by contributions. There may be multiple code contributors but it's not essential in any degree.

On the other hand, writing a new game altogether is an endeavor fraught with peril. See how many S.T.A.L.K.E.R. total conversions fell apart due to the need for a large skillset, i.e. modelling, texturing, level design, some writing, and voiceovers. Lost Alpha actually came out, and it was laughed off as vaporware, deservingly.

Let me give you a few examples for entirely new titles.

This is a wall of text, you may want to skip or skim it. Marked as spoiler.

(click to show/hide)

(1) Underrail is an isometric RPG, mostly for Fallout and Arcanum dinosaurs like me. It was made by a few people. It has a great consistent art direction and complex combat. Then, imagine all the effort that went into world-building, individual areas, encounter design, quests and dialogue, and even a writer putting accurate bits of modern medicine into some quests and logs. All that man-hours and skill of individual team members.

See felipepe's review book for a short introduction -- https://crpgbook.wordpress.com

Still, some dislike it for skill cooldowns, poor writing in plenty of places, animations worse than Fallout or even Arcanum, and the main story being easy to miss on purpose.

(2) Torment: Numenera is a "spiritual successor" to the original Planescape: Torment. Made by a professional studio, it had its own share of good reviews from sites like IGN, etc.

Dinosaurs who remember Planescape: Torment, talk of "purple prose", and general lack of creativity. You can visit the very-NSFW "rpgcodex" site for the general negative consensus.

(3) Age of Decadence

Pretty much the first RPG announced after RPGs weren't made anymore after 2002-ish. An indie team. Unique worldbuilding, Lovecraft-inspired story, its own brand of Very-Hard-only game difficulty, some improvements and changes on the old-style formula. I believe the prototype-aka-combat-demo came out about 2008? The game came out in 2015.

It has its own cult following but "rpgcodex" consensus is mixed.

It hasn't sold well, probably due to poor graphics and the difficulty level.

Why haven't you forked the project? In fact my project started as a fork in 2012.

Because I was the CMake build system guy, not the Scheme compiler writer guy.  I did not have enough understanding of compilers at the time to develop the core compiler.  The strategic issue is that although the guy allowed me to create the fully working Visual Studio build, he had no personal committment, investment, or incentive towards Visual Studio or CMake.  His concern with Windows was minimal as well.  He would complain about all the extra work CMake was causing him.  Since of course I had done the heavy lifting to make it all work, my reaction is like what am I, chopped liver?  He threw out the code; I had hoped the small community around the compiler, would see the value of keeping this thing working with the CMake build.  But push come to shove, nobody else wanted to do the ongoing support for CMake and Visual Studio.  Windows wasn't sufficiently important to them.

CMake has become standard drill for so many projects since then.  No it is not a lovely build language, and I won't be doing any more of that, as that part of my so-called career is long since over.  But what a waste, back then. I learned that I can't drag people into support situations they basically don't care about.

Now that's a coincidence. The final straw in forking "opentrack" was that after having received the go-ahead for porting to Linux, the project owner refused to accept the code. The rationale was that he was comfortable with ".sln" files and CMake was new. Having done the porting work, the build system choice was either autotools, SCons or CMake.

Does the project still exist? At least it's not newLISP. Man I regret giving up CL due to the lack of work. No one's hiring except for ITA Software.

[bvanevery]

Could it be that you yourself are interested in writing a game from scratch?

It was more than an interest.  I tried to write such a thing during the dot.com bust and ultimately went bankrupt.  It won't stop me from trying again, but many core design issues of the 4X TBS genre have eluded me over the years.  Making something very similar to SMAC is doable, as it's been done before.  But making something that is an improvement, game mechanically, that doesn't devolve into tedious unit pushing... I've never figured it out.  It may be an intractable problem.  I wouldn't say I've given up, but after all these years of contemplation, I'm most certainly stymied.

I've also contemplated the best way to narrate something like a SMAC universe.  It leads to different problems.  Although SMAC's naration is good, and half of why we still play it, it is also uneven in quality and needs to be regarded objectively for its faults.  They got some things right and other things are just filler.  It is sad that nobody had any incentive to try again.

I'm sorry but so far you haven't proven your case. Creating a new codebase for the same assets (either based on the original executable to the point of being a derived work, clean-room engineering, or not at all based upon it) is by far easier, and has guaranteed results.

Newsflash: you don't have the source code.  Twiddling an AI you don't have source code for?  Good luck with that.  I'm writing up a Yitzi game in the AAR subforum right now.  I see no evidence of it playing any better.  AI does the same dumb things it always does.

Also note the finished Diablo 1 complete rewrite "Tchernobog"

A complete rewrite is a "new game".  Not hacking on the binary of an old game.  If you thought I meant you must design a completely new game experience you are mistaken.  That's up to your ethics of how many copyright violations you want to make with a derivative work.  You could rip off the story and art assets of SMAC and call it a day.  You could CYA by saying "you must install SMAC first".  If you are content with a hobbyist level of effort, probably no one will sue you.

No one has sued FreeCiv.  Game mechanically it's almost a straight copy of Civ II.  There are very few changes.  There's even a "Civ II compatibility" ruleset option if you want to make sure you're playing straight Civ II.  They did write their own code and provide their own art assets.  I don't recommend using their code though.  Unless something changed dramatically over the years, it's an ugly mess, and the AI isn't that bright anyways.  Not what I'd call a treat to work from.

Now that's a coincidence. The final straw in forking "opentrack" was that after having received the go-ahead for porting to Linux, the project owner refused to accept the code. The rationale was that he was comfortable with ".sln" files and CMake was new. Having done the porting work, the build system choice was either autotools, SCons or CMake.

Project leads are #######s when it comes to build systems.  It's a permanent cultural problem with programmers, because in the C/C++ ecology, the build system is made to be a completely different thing than the rest of the programming.  So programmers don't like new, tedious learning curves beyond their current skill set.  If they are project leads they handle this problem by executing build guys that come along to do ports.  You can do the greatest port ever; the project lead will say "waah I don't like it, I have to lift a finger to have this capability."  They don't care about the new capability, you cared.

So then you're faced with taking on the whole project yourself, without any help.  And the extant community is probably going to stick with the old "official" project anyways in most cases, so you won't get anyone else helping you.  Gotta handle all the coding and build a community too.  For many starter projects that's a completely losing proposition, and certainly was in my case.

A possible answer is to have the build system technology deeply tied to the language, so that nobody is fighting and gnashing teeth about learning how to do builds.  Rust has this kind of design.  To be honest, their included build and package system is the only thing I'm sure I like about Rust.  I'm not convinced that the safety dances they want programmers to do, actually buy an independent game developer anything.  My jury is out on the question, and I reevaluate the Rust community from time to time, but so far they haven't produced anything tangible that would make me change my mind.

I have worked on designing my own programming language, which I'd eventually integrate a conceptually similar build system with.  But I have not gotten far in the language design effort.  Frankly I've gotten stuck a lot.

BTW I didn't fork CMake either.  That would have been a sheer suicide mission.  Yes I had a falling out with their principal developers, I was kicked off their mailing list.  A little too interested in forward looking ways of making things better.  Consequently my willingness to touch anything related to CMake nowadays is pretty limited.  Conceivably I would do it if some open source project actually had value to me.  I have bugfixed a few people's CMake builds here and there over the years.  But in practice, no open source project has had any ultimate value to me.

Especially, 3d graphics engines are generally made on wrong principles that aren't sustainable for my needs, so I've given up on people who try to write them open source style.  Most of them make the same basic mistake of trying to do DirectX and OpenGL at the same time.  I really wish OpenGL would hurry up and die.  It's sad, and I was on the OpenGL side of things earlier in my career, but things evolved as they did.  It's a painful, crippled API nowadays and it needs to be shot and left in a ditch.  Of course the Linux crowd doesn't know what they're doing with graphics stacks, so they'll piddle with Vulkan and nobody else is gonna care.  Windows has DX12, Apple has Metal, they don't want Vulkan.  If Vulkan could finally kill OpenGL on Linux that would be a good thing at least for Linux, but they will foot drag that for quite awhile.  That's pretty much the Linux way in my experience.  You get diversity but you also get inertia and low QA.

Man I regret giving up CL due to the lack of work. No one's hiring except for ITA Software.

I've never thought that jobs depended on what compiler one knows.  But then historically, I had 3d graphics software development as my specialty area.  It was niche, and I was hired based on what I could do with that.  I didn't hang out in industry very long before going my own way either.  Logical next stage would have been to go work as a principal at somewhere like NVIDIA, but I realized it would be way too dry and boring for me.  I tried to find my star as an independent game developer... and 20 years later, I'm still trying.

[sthalik]

Could it be that you yourself are interested in writing a game from scratch?

It was more than an interest.  I tried to write such a thing during the dot.com bust and ultimately went bankrupt.  It won't stop me from trying again, but many core design issues of the 4X TBS genre have eluded me over the years.  Making something very similar to SMAC is doable, as it's been done before.  But making something that is an improvement, game mechanically, that doesn't devolve into tedious unit pushing... I've never figured it out.  It may be an intractable problem.  I wouldn't say I've given up, but after all these years of contemplation, I'm most certainly stymied.

First off, SMAC has serious user interface problems. Turning on warnings leaves one unable to do anything other than interact with the thing being warned upon -- "cannot be done in upkeep mode". "View mode" centers on the unit, and switch to the next unit is automatic and centers the view again.

Most of it has workarounds but overall more the area the bases and units take, the worse it gets.

I've also contemplated the best way to narrate something like a SMAC universe.  It leads to different problems.  Although SMAC's naration is good, and half of why we still play it, it is also uneven in quality and needs to be regarded objectively for its faults.  They got some things right and other things are just filler.  It is sad that nobody had any incentive to try again.

That's entirely doable as an RPG. Said by someone skipping classes to play Fallout 2 and 1 in elementary school. Take a look at some of the basic late-90's classics (Fallout 1, Planescape: Torment) and see how well it fits narrating a well-built universe.

There's more to "something like the SMAC universe". It's possible to be entirely derivative. A new title can as well be set in an all-different place, keeping SMAC's motifs and expanding upon them.

I remembered the "I Have No Mouth and I Must Scream" adventure game. There's a parallel here. AM covers the entire planet and creates psychodramas to torment five people he left alive. They can game the system to their benefit, but they're inside the all-powerful AM all the time anyway. We can game with Planet's responses to eco-damage, but we can't sadly nuke it from the orbit, only live on it.

I'm sorry but so far you haven't proven your case. Creating a new codebase for the same assets (either based on the original executable to the point of being a derived work, clean-room engineering, or not at all based upon it) is by far easier, and has guaranteed results.

Newsflash: you don't have the source code.  Twiddling an AI you don't have source code for?  Good luck with that.  I'm writing up a Yitzi game in the AAR subforum right now.  I see no evidence of it playing any better.  AI does the same dumb things it always does.

Let's assume for a second we're talking about a completely new codebase using the same assets, *without* disassembly:

(0) We're interested in "what it does" rather than "how it does it". Compare genotype and phenotype.

(1) We're not interested in using old AI's architecture. It's inferior anyway. We might use MCTS for some level of abstraction. A planning algorithm for another layer like some BWAI bots do. Role scoring and heuristics here and there. We're interested in keeping the good bits while replacing the bad bits.

(2) We know, or *should* know what individual attributes (PLANET, POLICE, unit abilities, attack, defense, speed, terrain modifiers, etc.) do. There was a huge Diablo 1 .pdf describing game mechanics. They've overdone it but it's a good example.

(3) We should know what kind of phenomena can occur at all, e.g. artillery fire, fast unit disengagement, response to eco-damage, diplomacy actions, drone riots, global warming... The list needn't be complete from the start while prototyping basic functionality. Together with (2) it's enough.

(4) Arbitrary data formats are already described by third parties. It doesn't matter if they used disassembly themselves since it's clean-room engineering. I've decoded Fallout and Arcanum sprites and animations from .pak files. Old sprite-based games are so notorious for using arbitrary data formats, their own RLE schemes, etc. Fallout movement animations are a real piece of work WRT the unit center deltas, etc. It's all documented and been done anyway though. See also Fallout Online. These guys did it from scratch.

(5) What's important but can't be inferred as reasonably deterministic from using the game, should be decompiled with IDA by a third party, then used through clean-room engineering.

Also note the finished Diablo 1 complete rewrite "Tchernobog"

A complete rewrite is a "new game".  Not hacking on the binary of an old game.  If you thought I meant you must design a completely new game experience you are mistaken.  That's up to your ethics of how many copyright violations you want to make with a derivative work.  You could rip off the story and art assets of SMAC and call it a day.  You could CYA by saying "you must install SMAC first".  If you are content with a hobbyist level of effort, probably no one will sue you.

No one has sued FreeCiv.  Game mechanically it's almost a straight copy of Civ II.  There are very few changes.  There's even a "Civ II compatibility" ruleset option if you want to make sure you're playing straight Civ II.  They did write their own code and provide their own art assets.  I don't recommend using their code though.  Unless something changed dramatically over the years, it's an ugly mess, and the AI isn't that bright anyways.  Not what I'd call a treat to work from.

1. Tchernobog is a completely new codebase. As is freeablo. They have the "you must install Diablo assets" provision, of course.

2. There's a hobbyist team maintaining Falcon 4 (1998) to this day, based on leaked source code. Proper aerodynamic model, rendering code rewrite, you name it. It's as if Falcon 4.0 with bugs fixed, and improved in all directions. It's on par with newly-written commercial combat sims. The features don't fully overlap but Falcon 4 did some great things no other game even bothered, i.e. the missions aren't scripted, the air and ground war are all dynamic.

They got a go-ahead from the current copyright holder. Truth is no one buys Falcon 4 except for the copyright check in the modern maintained version.

Now that's a coincidence. The final straw in forking "opentrack" was that after having received the go-ahead for porting to Linux, the project owner refused to accept the code. The rationale was that he was comfortable with ".sln" files and CMake was new. Having done the porting work, the build system choice was either autotools, SCons or CMake.

Project leads are #######s when it comes to build systems.  It's a permanent cultural problem with programmers, because in the C/C++ ecology, the build system is made to be a completely different thing than the rest of the programming.  So programmers don't like new, tedious learning curves beyond their current skill set.  If they are project leads they handle this problem by executing build guys that come along to do ports.  You can do the greatest port ever; the project lead will say "waah I don't like it, I have to lift a finger to have this capability."  They don't care about the new capability, you cared.

So then you're faced with taking on the whole project yourself, without any help.  And the extant community is probably going to stick with the old "official" project anyways in most cases, so you won't get anyone else helping you.  Gotta handle all the coding and build a community too.  For many starter projects that's a completely losing proposition, and certainly was in my case.

Yes and no. I said, it must work on Linux and OSX or be excluded from the build, but only for good reasons. Modern C++ is of great help here! It worked so far and "opentrack" has multiple contributed modules on all platforms. In my case it works since being project lead, maintainer, and "build system dude" all at once.

Truth is multiplatform capability helps with code quality by itself. If only Windows already had a decent conformant toolchain. But it's almost there.

A possible answer is to have the build system technology deeply tied to the language, so that nobody is fighting and gnashing teeth about learning how to do builds.  Rust has this kind of design.  To be honest, their included build and package system is the only thing I'm sure I like about Rust.  I'm not convinced that the safety dances they want programmers to do, actually buy an independent game developer anything.  My jury is out on the question, and I reevaluate the Rust community from time to time, but so far they haven't produced anything tangible that would make me change my mind.

I have worked on designing my own programming language, which I'd eventually integrate a conceptually similar build system with.  But I have not gotten far in the language design effort.  Frankly I've gotten stuck a lot.

Rust has a big interop library ecosystem already. I'm not even touching it until they fix MIR+borrowck fully though.

Flightgear has a serviceable build system with multiplatform builds. OpenCV has a monstrosity but it can be worked with. "Less is more" I guess. "opentrack" has 800 lines for tooling and 1-5 lines for each module.

BTW I didn't fork CMake either.  That would have been a sheer suicide mission.  Yes I had a falling out with their principal developers, I was kicked off their mailing list.  A little too interested in forward looking ways of making things better.  Consequently my willingness to touch anything related to CMake nowadays is pretty limited.  Conceivably I would do it if some open source project actually had value to me.  I have bugfixed a few people's CMake builds here and there over the years.  But in practice, no open source project has had any ultimate value to me.

Especially, 3d graphics engines are generally made on wrong principles that aren't sustainable for my needs, so I've given up on people who try to write them open source style.  Most of them make the same basic mistake of trying to do DirectX and OpenGL at the same time.  I really wish OpenGL would hurry up and die.  It's sad, and I was on the OpenGL side of things earlier in my career, but things evolved as they did.  It's a painful, crippled API nowadays and it needs to be shot and left in a ditch.  Of course the Linux crowd doesn't know what they're doing with graphics stacks, so they'll piddle with Vulkan and nobody else is gonna care.  Windows has DX12, Apple has Metal, they don't want Vulkan.  If Vulkan could finally kill OpenGL on Linux that would be a good thing at least for Linux, but they will foot drag that for quite awhile.  That's pretty much the Linux way in my experience.  You get diversity but you also get inertia and low QA.

It's pointless to fork CMake. The point of CMake is to be maintained continually, and support all platforms without a fuss.

Actually modern OpenGL 4.5+ is finally decent, see the extensions. Linux has GL4 too so it's usable. It took Mesa a while to go from 8.0 to GL3 to GL4. Now AMD pays few Mesa experts and Intel does even more. It works except for nouveau.

I'm not aware how much more boilerplate-ish is Vulkan. From reading the API it looks sensible. Given that for GL it's necessary to make a proper typed wrapper, Vulkan looks like less work. Soon Linux radv will be on par with Gallium opengl anyway.

Man I regret giving up CL due to the lack of work. No one's hiring except for ITA Software.

I've never thought that jobs depended on what compiler one knows.  But then historically, I had 3d graphics software development as my specialty area.  It was niche, and I was hired based on what I could do with that.  I didn't hang out in industry very long before going my own way either.  Logical next stage would have been to go work as a principal at somewhere like NVIDIA, but I realized it would be way too dry and boring for me.  I tried to find my star as an independent game developer... and 20 years later, I'm still trying.

It's not the compiler, it's the language. Even small companies won't go near niche stuff. Whether it's Smalltalk or OCaml. See how long it took closures to become mainstream.

I can't work on a salary due to health for few more years. Had big-name corporations try to snipe me. Had a great offer just a few miles from where I live. No can do.

Since you're into game development we might be onto something. Send me a PM, I'll send you an email. As long as the scope isn't too great it has a good chance of working out actually.

[bvanevery]

I believe there's an option to turn off the automatic centering behavior.  If it doesn't leap out at you, some options are buried under the [MORE] selection.

To be honest as a game designer I don't think the data structures of SMAC are even remotely interesting.  If I'm designing a game, I can design any given feature of a game out of existence.  Why sit around thinking about writing AIs that solve SMAC problems, if the game doesn't even have to be SMAC in the 1st place?  The only value in SMAC data structures is if one intends to make stuff work with the SMAC binary.  I don't think it's remotely hard to conceptualize the data structures of a 4X TBS game from scratch.  The hard part is the design of such a game and doing all the work.  I also don't think there's any reward in slavishly cloning SMAC until one's code is "feature complete".  Yes we all love SMAC around here, but it's not like it was the biggest commercial success when it was released, or the direction that made Firaxis money over the long haul.  It's a critical success with a fan following that demonstrates some of what is possible in 4X TBS.

What was Firaxis' original process?  They skinned Civ II.  It shows in all kinds of ways if you think about what really should be possible on a planet.  One can go down many rabbit holes if seduced by realism though.  Game design is also deciding what to leave out.  What to direct the player's focus upon.

As for the Common Lisp ecology, been there done that, trying to assess it for Windows game development.  They never could provide anything.  It kept me looking for other languages.  Eventually I gave up and decided to try to write a language.  Which I've had little success with.  Anyways, the people in the CL community are all too old to make Lisp a "great thing" again.  New languages borrowing from Lisp have a chance to be accepted, but the old timers who really do Lisp proper, they're done.  They don't have the energy to fight the standards and marketing battles to make Lisp a thing again.

As for PM, nothing personal but I prefer public discussion at this point.  I've always been public discussing "what's needed" for 4X TBS game designs, as they contain hard problems and not much in the way of secrets.  I don't know what kind of design discussion can ensue around here, but at least everyone is familiar with SMAC.  I think a thread about "what would you NOT do in a remake?" is in order, to get people thinking.  I'll go start it.  I put it in the Rec Commons forum because that has more traffic.

For the record, I will never work on extending the SMAC binary.  I'm impressed by the people who choose to deal with software that way, but for me personally, it is commercial suicide.  I've wasted years of my life on endeavors that have no money at the end of the rainbow.  I can't afford to do that sort of Quixotic thing anymore.  Really I'm only interested in the kind of game development that would actually make me a living.  That means choosing partners very carefully, or avoiding them entirely.  So I feel a need to be clear up front with my intentions, and the bar I set for partnering with people.  If we're not even on the same page about some things, for instance the Windows vs. Linux gaming ecologies, there's no point in even wasting the time.  Which is why I prefer the public discussion processes to shake those things out, to get to know people better.

[sthalik]

As for the Common Lisp ecology, been there done that, trying to assess it for Windows game development.  They never could provide anything.  It kept me looking for other languages.  Eventually I gave up and decided to try to write a language.  Which I've had little success with.  Anyways, the people in the CL community are all too old to make Lisp a "great thing" again.  New languages borrowing from Lisp have a chance to be accepted, but the old timers who really do Lisp proper, they're done.  They don't have the energy to fight the standards and marketing battles to make Lisp a thing again.

You can't make a 60 Hz game on SBCL or any other implementation. Their GC's aren't robust, with long collections, unrestrained memory growth and wall time during the collections. It's not feasible to avoid consing altogether either. Writing idiomatic Lisp, even abusing DECLARE DYNAMIC-EXTENT doesn't work for a game. It's only JVM that allows for consing freely and wastefully while still keeping deterministic 60 Hz.

TL;DR version: I agree with you on CL's obsolescence. These are anecdotal technical reasons not even related to language marketing.

Long boring version:

For many years modern CL had no actual working user interface library. McCLIM isn't a working GUI library. Just like Closure isn't a working web browser.

They hold some nonsensical positions, e.g. that FFI to C code isn't the right choice because of C code implicitly connecting to Unix signals. Which pure hogwash because nothing except ncurses does this. They have other faux-reasons, and as result we have either half-working native Lisp interop libraries, or none at all. CL-PPCRE isn't faster than libpcre. DRAKMA isn't any better than libcurl.

CL has a sequence type limited to vectors and lists. In fact CL predates the concept of user-made sequence types. However, there's no momentum toward standardizing language extensions at all. It's set in stone and considered proper "because macros". Too bad macros won't help with DEFTYPE/SEQUENCE and ITERATE/LOOP. Plenty of #:CL namespace functions should be generic as well, but then again there are no extensions or future versions of the standard.

For the record, I will never work on extending the SMAC binary.  I'm impressed by the people who choose to deal with software that way, but for me personally, it is commercial suicide.  I've wasted years of my life on endeavors that have no money at the end of the rainbow.  I can't afford to do that sort of Quixotic thing anymore.  Really I'm only interested in the kind of game development that would actually make me a living.  That means choosing partners very carefully, or avoiding them entirely.  So I feel a need to be clear up front with my intentions, and the bar I set for partnering with people.  If we're not even on the same page about some things, for instance the Windows vs. Linux gaming ecologies, there's no point in even wasting the time.  Which is why I prefer the public discussion processes to shake those things out, to get to know people better.

I don't see money in games fixed to the modern "genre" convention, with the unduly-praised "evolutionary improvements". I indeed have feasible ideas for something experimental and transgressing the "genre" boundaries. There's no money in that and it's fine. If it takes profit and revenue stream into account too much, the actual quality will suffer anyway.

Microprose did the Falcon 4 dynamic campaign "because they didn't know better". Their words. Revolutionary effort doesn't sell.

TL;DR I'm disillusioned with the game industry. Rant follows.

With regard to "evolutionary improvement":
- Skyrim has better graphics and slightly better combat than Oblivion
- This new shooter has better shadow algorithm and it really adds up
- HBAO+ is better than SSAO
- The sequel has hires wall textures and more of the same.
- Life is Strange is like Telltale but done right (note, I love LIS)

All in all, Quake 2 has better wall textures if using MIPMAP_NEAREST as the magnifying filter, while keeping MIPMAP_LINEAR as the minifying filter. A single-line change in a source port.

[bvanevery]

You can't make a 60 Hz game on SBCL or any other implementation.

I never really imagined this as a hard requirement for a 4X TBS project.  However I have never planned to limit myself to only that kind of game, and the industry consensus is that GC simply doesn't work for 3d that needs to go fast.

It's only JVM that allows for consing freely and wastefully while still keeping deterministic 60 Hz.

Actually I'm surprised to hear that anyone believes that.  Do any AAA titles prove that in practice?  Do any indie titles?

Just like Closure isn't a working web browser.

I have wondered about the Clojure food chain, but never really delved into it due to the JVM or the CLR.

They hold some nonsensical positions, e.g. that FFI to C code isn't the right choice because of C code implicitly connecting to Unix signals.

I did notice that C FFIs were rather bad.  As were native code generating Windows open source implementations.

- Skyrim has better graphics and slightly better combat than Oblivion

I gave up on Bethesda.  Someone did some game development writeup somewhere saying, they're trying to appeal to millions of people, this is why they don't do such-and-such.  Casualification?  My main takeaway from Skyrim is the landscape is indeed pretty.  Also that they know what they're doing about building a moddable game and community.  It's like the most moddable thing ever, if you go to sites like Nexus Modding.  People still buy Skyrim new.

On the other hand I looked at the actual programmability of Skyrim mods and it looks like baby stuff, filling out very limited forms and templates.  I don't think you can fundamentally change how the game works or plays.  To be sure, it's not reasonable to expect "fundamental" changes out of a mod, but I seem to recall thinking that it would be a waste of time for whatever I imagined.  Acutally I may not have wanted to do anything, I may have been just evaluating their scripting language for its usability.  I remember there being "nothing to see".

- Life is Strange is like Telltale but done right (note, I love LIS)

I'm peripherally aware of these adventure titles but haven't experienced them.