Decompilation coordination thread

[PlotinusRedux]

Kyrub--thanks for sharing your IDA DB.  I don't know how much Scient may have identified in the AI routines, but I know I haven't identified any of them myself, as I've focused on the UI, so this will all be information I didn't have before.

I just downloaded your IDA db to look at tomorrow, but just to be clear on one thing--your DB is called terran.idc and your post references the SMAC exe.  My own IDA work started with SMAC and only after that did I look to find the equivalent addresses in SMAX, but everyone else's work here seems to have been solely on SMAX.  Is your DB really for terran.exe, the original Alpha Centauri, as it would seem from the name, or did it just get renamed and is really for Alien XFire?  If the former, did you ever identify in the same routines in Alien Xfire?  (It's OK if you didn't, I've already done some work mapping routines between the two that would help me in making a SMAX translation of your SMAC db.)

[PlotinusRedux]

Scient--

I'm getting more and more optimistic about our ability to release a compileable, drastically improved version of SMAX in a matter of months rather than years.

A large part of the class structure is devoted to UI controls.  Just combining what I had worked out with your .map file and the function prototypes in the MAC version you shared already lays out 60% of the class prototypes for the UI.  That's past the tipping point where finishing the prototypes becomes easy if tedious.

I already have a DirectX 9 control library I wrote that implements 80% of what I've seen in the UI, and the other 20% would be trivial to add.  Once we have the UI prototypes worked out, we're talking weeks for me to completely replace the entire UI code with my own DX9 control code implementing the same interfaces.

The next thing I'd attack is the map and unit rendering code.  The hardest part there will be converting the cavier files to a standard mesh format, though the terrain rending code will also take me some time.  I'm guessing 3-4 months to accomplish a complete conversion of that code to DX9, which combined with the control library will replace all the UI and graphics code and allow textures and meshes of any resolution and color depth.  From there the only limit to how good the game looks is what mesh and graphical artists we can round up.  And some of the non-technical people here who want to contribute could probably download Blender, a free mesh creating program, and produce meshes much better than the current ones--anyone who can draw a decent stick figure could improve them.  I just happen to be in the lowest 1% who can't even draw decent stick figures.

That basically takes care of the entire Jackal engine code, making it easier to identify the rules engine, game logic, AI, etc., code.  I'd assume you and Yitzi et al would have continued to work on that while I was working on replacing the Jackal code.  At that point I could join ya'll in working on that part. 

I really think by a conservative estimate within a year would could have a complete C++ de-compilation/rewrite completed using DX9 for the UI and mesh rendering on the graphics side and a fully customizable game engine on the other side.

[ete]

Thank you very much kyrub!

@Plotinus: kyrub's work was indeed based on SMAC not SMAX. A few of his changes were transferred to SMAX, as listed here: http://alphacentauri2.info/index.php?action=downloads;sa=view;down=108 but the vast majority of improvements and work was SMAC only. We had plans to create a mod for which would clone SMAC within SMAX, at which point he would consider switching over, but it was never finished.

That is amazing news about having the game possibly fully decompiled and significantly improvable within months . Especially excited about what can be done with the AI, modability, and full color depth.

[Dio]

Thank you Kyrub for releasing your IDA database .

[PlotinusRedux]

Dio--Any information on any routines or data will be useful.  I think between what I have and what Scient has we're going to have 99% of the UI identified, but I've done nothing on the game logic, ai, etc., myself, and the Mac executable Scient found seems to mainly identify the UI classes rather than the game logic and AI routines that were probably straight C.

So really, I think any areas you want to focus on other than UI will be helpful.

Scient and any other C++ coders--so the main part that is going to suck is MSVC only allows the "naked" specification, which suppresses generation of function prologues and epilogues, outside of classes--you can't even use the specification for static members of a class.  I looked to see if GNU C was any better, and it doesn't even allow the "naked" specification for 80x86 processors at all.  I've always preferred Borland, which gives much more control over--well, everything--but that's a dead end long term since they basically went out of business nearly a decade ago, so I'm forcing myself to not use them.

The only solution I can think of is to wrap the .exe assembly in straight C __thiscall specifications and have a script add thunks for every non-virtual class method to the classes themselves.  It's not a problem for virtual functions since those can be pointed directly at naked __thiscall's that aren't defined as class members, even though we know they really are class members.

[Dio]

Dio--Any information on any routines or data will be useful.  I think between what I have and what Scient has we're going to have 99% of the UI identified, but I've done nothing on the game logic, ai, etc., myself, and the Mac executable Scient found seems to mainly identify the UI classes rather than the game logic and AI routines that were probably straight C.

So really, I think any areas you want to focus on other than UI will be helpful.

Scient and any other C++ coders--so the main part that is going to suck is MSVC only allows the "naked" specification, which suppresses generation of function prologues and epilogues, outside of classes--you can't even use the specification for static members of a class.  I looked to see if GNU C was any better, and it doesn't even allow the "naked" specification for 80x86 processors at all.  I've always preferred Borland, which gives much more control over--well, everything--but that's a dead end long term since they basically went out of business nearly a decade ago, so I'm forcing myself to not use them.

The only solution I can think of is to wrap the .exe assembly in straight C __thiscall specifications and have a script add thunks for every non-virtual class method to the classes themselves.  It's not a problem for virtual functions since those can be pointed directly at naked __thiscall's that aren't defined as class members, even though we know they really are class members.

Good to hear that my work will not go to waste. I have discovered a few things worth mentioning in the UI interface code but alot of what I found so far is related to script controls and AI behavior in regards to those scripts.

The number of different memory addresses and there associated flags that control AI behavior keeps increasing. I now know that their is at least 4 different memory addresses along with various constant flags they are compared to control various aspects of AI behavior in scenarios and during regular gameplay. The long version will be in the text file I am going to update.

[Lord Avalon]

...I've always preferred Borland, which gives much more control over--well, everything--but that's a dead end long term since they basically went out of business nearly a decade ago, so I'm forcing myself to not use them. ...

It looks like Turbo C++ is now C++Builder XE7 from Embarcadero Techonologies, which acquired CodeGear from Borland in 2008. I am not a coder, so I have no idea if it's any good, but there's a 30-day free trial.

[Dio]

Attached below is most of my work so far plus a slightly revised and updated version of my Labels in the assembly code file.

Edit: Nobody is interested .

[Dio]

I am currently going through the AI controls for the Diplomacy scripts. Their are alot of immediate constants involved in the running of this game. In addition, I now have a somewhat clear idea of how portions of the faction specific AIs for  , , , , , and work . If people have more memory addresses/data segements identified and are willing to share that is always helpful. I will continue  this code.

[PlotinusRedux]

Attached below is most of my work so far plus a slightly revised and updated version of my Labels in the assembly code file.

Edit: Nobody is interested .

I'm interested, Dio, I've just suspended all work until Scient's release of his IDA database to avoid duplication of effort.  Once he releases that and I spend some time turning it into compilable C, I'll want to incorporate all the information every one has into a GIT C++ project.

Be assured, no one's research they're willing to share will go to waste--I'm just not ready for it yet.

The addresses/routines I have identified are mainly related to the UI and/or are for  terran.exe rather than terranx.exe, so I don't think they'd be of much help to you with the AI.

[Yitzi]

By the way, I found the landmark creation procedures:
  5C5EF0: jungle
  5C5C70: crater
  5C6DB0: volcano
  5C7F40: mesa
  5C8150: ridge
  5C69E0: uranium
  5C64A0: ruins
  5C7750: unity
  5C7A80: fossil
  5C7540: nexus
  5C7020: borehole
  5C6200: sargasso
  5C6740: dunes
  5C6C40: fresh
  5C83B0: geothermal
In all cases, arg1 and arg2 are position, with -1,-1 for random.  Volcano has arg3 to determine if it comes from a special
event as opposed to map creation or scenario editor

[Dio]

The procedure at 005002F0 moves arg 1 and 2 into the EAX and EDX registers respectively. Arg 3 takes a push value and moves it into the ECX register. The memory address of the diplomatic relationship with faction x is moved into the register EAX for these operations. Then it performs an AND command with the  memory address for faction diplomatic relationships against the value in ECX. The final result for the AND command on the Diplomatic relationship address is then returned and often has a TEST command performed against itself with a conditional jump following.

This is just one of the many procedures that control AI diplomatic behavior but appears in many of the procedures for AI diplomatic behavior.

[Dio]

Most of the call commands in the sections around the Script Names for AI Diplomatic interactions simply control either the appearance of words from the playerfiles in strings like FACTIONNAME0, FACTIONADJ2, NUM0 or the diplomatic conditions neccesary for these scripts to occur. What I have not figured out yet is the meaning or use of some of the memory addresses that appear frequently in the script sections for AI diplomatic interactions. Because I have very little reference to their meaning, it is difficult to ascertain their function in the procedure. I know some of the addresses store values with unknown meaning. A few examples of these addresses include 93F800, 93FAA8, 93F80C, 0093F7CC, and so on. Identifying the function of these addresses would greatly ease my reading the procedures.

Lastly, the procedure I described in the previous post opens a whole slew of new diplomatic relationships values that I have not figured out yet.

[PlotinusRedux]

For anyone interested, while I wait for Scient's IDA release, I've been working with Ford_Prefect to identify the parts of the .cvr files he hadn't already identified.  I think we've got the entire file format pretty much worked out now, including newly identified 8 and 16 bit shader tables, the normal information in the voxel data itself, and all the tags used in the files--enough to easily rebuilt the entire rendering engine in full 32-bit color and create a full model/animation editor.

No formal write up yet, it's still a WIP, and I only just now figured out what the "material" tag was for--it's really a 16-bit shading table, not "material" in any normal 3d graphics sense--but all the information is spread across the thread starting around http://alphacentauri2.info/index.php?topic=6909.45.  I'm sure Ford or I will write it up in the not too distant future once we verify my hypothesis about the normals really is correct by encoding it in a rendering engine and seeing it looks right.

My hope is the same engine is being used for the terrain with the models being built in memory by combining a voxel map of the terrain with the terrain textures in the .pcx files instead of being read from a file--if so, I'm a lot closer to my 32-bit conversion than I thought I would be.  The Caviar engine itself can already handle 16-bit color, which would itself be a massive improvement over 8-bit paletted, so it's just the graphics primitives--including the render target of the engine--that are hard-coded to 8-bit, and they will be fairly easy to replace.

[Yitzi]

The procedure at 005002F0 moves arg 1 and 2 into the EAX and EDX registers respectively. Arg 3 takes a push value and moves it into the ECX register. The memory address of the diplomatic relationship with faction x is moved into the register EAX for these operations. Then it performs an AND command with the  memory address for faction diplomatic relationships against the value in ECX. The final result for the AND command on the Diplomatic relationship address is then returned and often has a TEST command performed against itself with a conditional jump following.

This is just one of the many procedures that control AI diplomatic behavior but appears in many of the procedures for AI diplomatic behavior.

I think it's usually more useful to give it as a more succinct result; we don't need to know what registers are used.  For instance, here 005002F0 simply returns the relationship between faction arg1 and faction arg2, masked by arg3 (i.e. only showing bits present in arg3).