Decompilation coordination thread

[Yitzi]

By the way, note that the alphax variables are their values in my patch, which differ from those in the original, so you'd have to use the right choice for whichever .exe you're analyzing.

[scient]

Ahhhh, ok that makes sense why some things differed. I already have all alphax variables labeled in my database.

I was curious what you meant by Nessus inside faction struct and instead I found an array with pointers to addresses inside the string table. It doesn't really look like these are used and redundant since most everywhere else just references labels.txt offset.

I was confused by you listing them in faction struct.

I also figured out what you mean by SUPPORT and marked that up.

I'm still not sure what virtual mineral count is tho.

[Yitzi]

Ahhhh, ok that makes sense why some things differed. I already have all alphax variables labeled in my database.

I was curious what you meant by Nessus inside faction struct and instead I found an array with pointers to addresses inside the string table.

Not sure why; that value should be where the number of Nessus mining stations for that faction are located.  (Unless I mistyped the address; I found it from its reference in the minerals ecodamage formula, so you can always look up there.)  The other satellites are nearby as well IIRC.

Actually, there seem to be a few faction structures, which is part of why enabling more than 7 factions would be such a huge job, though decompilation would make it feasible.

I'm still not sure what virtual mineral count is tho.

Originally, that location recorded the number of times the faction had used tectonic missiles (which count like PBs for ecodamage purposes, but not faction reputation/sanctions.)  When I redid ecodamage, I used it for the bonus minerals' worth of ecodamage from all history-based sources: Tectonic missiles, PBs (depending on setting; the default counts nerve staples after IIRC the 15th because they're major atrocities, but not PBs used with no UN charter, I provided the option to change that, and PBs only count toward this value if it's set to work by PBs instead of major atrociites), and pops.  (In version 1.x it included ecological facilities built, but now it's based on ecological facilities owned instead.)

[PlotinusRedux]

Scient:

Do your decompiled classes follow the same prototypes as the original classes?

I'm thinking--once you get your IDA db ready to share, I can pull in the function prototypes from the Mac/Carbon version (I've been playing around with the Python scripting in IDA enough to do that), and do a dump to a .asm file.  Then, running parsers on the .asm file, I can convert it to a compile-able C++ application initially with all the functions naked inline assembly.

In theory, that gives us C source code that can be compiled into a new .exe, which would already make things like assembly level patching by Yitzi easier--he could add as much code as he wanted anywhere he wanted to with all the pointers automatically fixed up, since they will be symbolic names ("loc_XXXXXXXX", "sub_XXXXXXXX", etc.) rather than absolute addresses and offsets.

Then, if your new classes followed the prototypes of the existing classes, we could literally just plug them in in place of the existing assembly versions and recompile.

[scient]

Yes and no. In some places I have, others not.  I was planning on going through and fixing them up after my initial pass through. I haven't created structure placeholders for some of classes that are used as parameters.

That sounds good. It is at least worth a shot to see if it works. The method you are currently using with DLL injecting is another possibility to start replacing existing game logic functions to fix bugs.

[PlotinusRedux]

I'd like to try to pull all the assembly into compile-able C source, with my DLL injection as a backup.  The advantage over my DLL injection is the latter requires defines for the address of every function overridden, and if you want to override an existing function and still call the existing function from within the override, you need defines for every place that calls the function so you can change the address they call--putting a jmp instruction at the top of the existing function destroys it.

Currently the dump to .asm is spitting out over 4 million lines, so manually massaging them is out of the question, but I think I can write automated routines to turn that into compile-able C.

Are you using MSVC or GNU, or something else?  I'm using VS 2013 Express, which unfortunately doesn't allow naked functions as class members (not even static ones), so I'd need to define all the functions as straight C __thiscall functions, and have the class members just be stubs that called them.  I haven't looked to see if GNU has that limitation--if it doesn't, that would almost be reason enough for me to switch to GNU.  If you're using MSVC, though, I'll keep it all MSVC compatible.

[scient]

Ahhh ok I see. That does make sense.

I was originally using MSVC 2012 and just recently updated to MSVC 2015.

[DrazharLn]

Plotinux, are none of the available asm -> C converters suitable for your purposes?

Here are a couple:
http://decompiler.fit.vutbr.cz/decompilation/
http://www2.onlinedisassembler.com/odaweb/

There's also Hex-Rays and Hopper, but they cost money.

[scient]

Hex-Ray's is by far the best out there, however for more complex software this is not feasible. There will be too many errors and it would be time consuming to go through and fix them up. It is better to decompile each function and go from there. While this is also time consuming, you end up with a better product than a mangle of code. The decompilation routine is useful as a first pass in some cases to then go through and rework the function manually.

I have used Hopper as well, but consider it more of an "IDA" light than a real competitor. In some cases it is useful to have, however it is no where near as polished and robust as IDA.

[PlotinusRedux]

As Scient says.  Hex-rays is particularly bad with BP-based frames combined with __thiscall functions, from what I've seen.

If we can get a working build in C with C prototypes around naked assembly to start with, that will be no small feat.

[PlotinusRedux]

On a different note, Scient and/or Yitzi--have either of you found a simple method of determining the topmost window in the game?  The method I'm using in my patch is only working some of the time.

What I'm using now to decide if the map is topmost (i.e., mouse should scroll the map, etc.) is what I took from the scroll check timer:

bool IsCityShowing(void)
{
   bool fRet =
      m_fGlobalTimersEnabled &&
      !*m_pAC->pfGameNotStarted &&
      m_pAC->pfncWinIsVisible(m_pAC->pCityWindow);

   return fRet;
}


bool IsMapShowing(void)
{
   bool fRet =
      m_fGlobalTimersEnabled &&
      !*m_pAC->pfGameNotStarted &&
      !IsCityShowing() &&
      !m_pAC->pfncWinIsVisible(m_pAC->pAnotherWindow) &&
      !m_pAC->pfncWinIsVisible(m_pAC->pAnotherWindow2);

   return fRet;
}

Where pAnotherWindow & pAnotherWindow2 where globals the timer was checking before scrolling.

[scient]

I will have to look tomorrow, but there is a main "Win" class that handles all the window functionality of other classes. There is one function about whether something is visible but I'm not sure about top most.

[PlotinusRedux]

Yeah, the base Win class all the others inherit from has an IsVisible member, but I haven't identified all the global instances of Win classes to call IsVisible for all of them.  I did notice 4 globals for unparented windows, but saw at least one case where the map was the topmost window and more than one of those globals was != NULL.  I would have thought the code in the scroll timer call back should be pretty foolproof--I'll look at it again to see if I missed something in translating its checks.

Back to the original topic--once we get the whole thing compile-able, I've got a DX9 control library I wrote for a game I never finished I could adapt to replace the rendering parts of all the UI windows.  The terrain itself screams for DX/OpenGL rendering.  From there, converting the Cavier files to a standard mesh format is all that would stand in the way of a full DX/OpenGL conversion.

[PlotinusRedux]

Scient, have you done anything with the vtables?  From the .map file, it looks like most of what you have defined is missing the virtual functions.  I was working on a vtable struct for all the Win descendents, but have been holding off until I see what your final IDB has.  If it's missing them too, I should be able to add a lot of function defs in pretty short order.

[scient]

The one I sent you hasn't had a lot of work on Win class. I have since worked on it a lot more going through and defining virtual functions for each set of classes.

This next week I will have a lot more free time to try and finish it all up. If you want to wait until then, you can see what I've done.