Decompilation coordination thread

[ete]

As i said in the other thread:

I would *really* like to get the active decompilers sharing data with each other, since otherwise there's just massive duplication of effort which could instead be directed towards more improvements to AC. The initial work of merging discovered functions would be significant, but I suspect would be quickly payed back by having three times as many people working on the project.

I want to see this happen because I like it when things get fixed and improved and think this would be a major step forward. I don't know exactly what I can do to help as a non-programmer, but if there's anything I can do to make it more likely I'm in (e.g. teaching people github, talking to people, organizing things).

PlotinuxRedux and Dio have both offered to share their research with the community, scient definitely interested in sharing with at least the other disassemblers, and kyrub has previously given indication he would share his SMAC AI disassembly at some point after he finished work (I'll contact him when the rest of this is set up). Yitzi, I have a half memory of talking to you about this sometime long ago, but am not certain what you said. Would you like to participate?

If everyone with information they're happy sharing with the community could post it in this thread that would be awesome. I also expect you guys will have things to talk about related to the specifics of your work, seeing what's been duplicated, figuring out how to merge, and deciding on a versioning system for easy syncing. I'll be hanging around trying to be useful, would be happy to set up a github organization or something like that if you guys want it.

Several of you have independently come to the conclusion that decompiling large chunks of the game is the only way to make the improvements you want to see in the game. If we can link those chunks up we'll have an opensource clone on our hands, and the possibilities from there are endless.

[Yitzi]

I don't have decompilation in the normal sense; what I do have is notes on what various functions and variables do, and a step-by-step walkthrough of some functions.

However, I'm pretty sure that source code coming from decompiling the actual program would not be open source in the usual sense, but would be subject to all the copyright rules of the .exe.  So we could mod it more easily, but that's about it.

[ete]

I don't have decompilation in the normal sense; what I do have is notes on what various functions and variables do, and a step-by-step walkthrough of some functions.

Okay, I'm sure that would be of interest to the others. Would you be willing to share?

However, I'm pretty sure that source code coming from decompiling the actual program would not be open source in the usual sense, but would be subject to all the copyright rules of the .exe.  So we could mod it more easily, but that's about it.

It would not be free software, but it would in a literal sense be open source, and increasing ease of modifications is a very major benefit.

[Yitzi]

I don't have decompilation in the normal sense; what I do have is notes on what various functions and variables do, and a step-by-step walkthrough of some functions.

Okay, I'm sure that would be of interest to the others. Would you be willing to share?

Sure.  I'm attaching the most important stuff to this post.

[scient]

Thank you for sharing Yitzi, that has some very useful information I don't have documented.

I could easily integrate this information into my IDA database with additional enums and structures. I have started to do a similar thing as I've gone along. It really enhances the readability of the asm.

Example:

[Yitzi]

If you could send me whatever information of that sort you've got (especially anything on AI), that would also be great.

[scient]

I don't have anything on AI specifically however there are some flags I've found that identify AI vs human units and then toggle AI specific code accordingly.

For example, here:


Everything I've done is bundled in my IDA database. I will see about cleaning it up and making it available to you and whoever else who would like it in a couple weeks. I have spent a lot of time into making this database very comprehensive.

[Buster's Uncle]

I wouldn't be surprised if Impaler and Mart, old-timers who have made much study of AI and manipulating it from the .txt end, couldn't be of use, here.

[Dio]

I posted my labels file in the wrong topic ...

Here is my for the most part complete list of all the labels present in the assembly code and their addresses. I could also create a list that shows where individual facilities and their associated effects are located.

Plus a list of bugs that I know exist:

1. This bug occurs when you put SOCIAL, TALENT as a Social Model Effect and look in the datalinks under Society Model Effects. The TALENT bonus or penalty will appear linked to the Society Effects section of the datalinks. If you click on the link, then you can only navigate by pressing “Back.” You can locate the sections in the code for this bug by clicking search for all referenced text strings and then “HELPSOC” and “HELPEFFECT”.
2. This bug occurs after you select a faction with Shift + F9 option in the scenario editor. Once you do that, the diplomacy windows shrink until the text and Faction leader potrait run over the sides. The potential starting points to look for the bug in the code are 004DB9AE, 004DBB12, and 004DB080.
3. This bug is in the world mini map right click menu. The menu has an option called “Recenter this window here” that does not function properly or function at all. The code controlling this label is located at address 004C3C66.
4. This bug occurs when you try to give two or more free starting technologies to a faction. One of the faction’s technologies must grant a free upgrade of the faction’s initial scout unit (i.e. Doctrine: Mobility) and the other technology a free prototyping of a unit design (i.e. Applied Physics). This bug will cause the faction to not get the free scout unit upgrade and only occurs if the technology granting the free prototype is above the technology in the alpha(x) file tech list that gives the free scout unit upgrade. An applicable example of this is giving a faction Doctrine: Mobility and Applied Physics as free starting techs.

[Buster's Uncle]

No harm - go ahead and repost here.

[scient]

For labels, I have the logic down pretty nicely. Although, I haven't gone through my IDA db and made a comment which offset is which. However, based on structure it is easy to identify what goes where like so.



Then if we go to a random offset like so:



And follow the guidelines, you get with line 420 which is "then". This matches up with your list.

004B48A8   mov eax, dword ptr ds:[edx+664]           then (0x664 == 1636)

It would be helpful to integrate all of them into the the database however to save having to look them up. I have been doing this on my own as I go, so a full list is useful. As for facilities, I have created an enum for them and identified instances like so.

[Dio]

In addition, if it might help, I can make a list of the procedures and memory addresses for the .cvr files in the code that control the display and assembly of chassises, weapons, armor, and special abilities.

[scient]

I can sync any specific info about caviar data into what I have already. While I have identified most of the functions with my analysis, I haven't gone any further.

For example, I have functions but I don't have any of global data as to what is being added.

[Yitzi]

Everything I've done is bundled in my IDA database. I will see about cleaning it up and making it available to you and whoever else who would like it in a couple weeks. I have spent a lot of time into making this database very comprehensive.

That would be great, thanks.

[PlotinusRedux]

Scient,

I understand you wanting to clean up your IDA DB before making it available--I'm the same way, generally--but would it be possible to get a .map in the meantime?  That will just list the function names and addresses--I didn't know about the MAC executable having the function names, and just that would be a huge help to me.

You can export a .map file from IDA (File\Produce File\Map File)--either post it here or email it.

I'll be glad to share anything anyone is interested in, but everything I have so far is SMAC, I'm just starting on SMAX tonight.  I have IDA, which can spit out .map and similar files for those that don't have (actually it can even split out the entire disassembly with the comments and all to an html file), and the structures from that in a C header file along with the changes I've made in cpp files.

I've mostly been working on the GUI classes--I've got the base image and surface classes, window/control base class, menu class, and a few other things mapped out with enough data known to use them.