scient replied

[ete]

Contact successful!

Hey!

I apologize for my absence from the community. You are correct, I have been extremely busy. Various things in life have kept me occupied. I still do have the patch stored with a large number of changes I never released. I forget if I ported all changes over to SMAC.

Even though I have been absent, I have be actively working on SMAC/X project. I have shifted from fixing bugs to complete decompilation of the game. There are just too many things I'd like to change and resolve that patching at the assembly level is no longer an option. This will take a number of years, at least if I work on it solo in my free time, however I have made a lot of progress. I have identified almost all the major functions inside the binary which makes stepping through the code and looking at logic a lot easier.  Also, I have decompiled a few of the base internal classes back into C++. I've been doing them in blocks and building upon each other with subclasses.

Perhaps in a couple weeks when I have some time off I will make a post on forums about my work along with the unreleased patch. If I don't have time, then I will email you everything and let Yitzi sort it out.

I hope all is well. If you don't hear back from me on boards or by email in a few weeks, feel free to remind me.

Brendan (scient)

I just had a quick look and it seems like I pretty much finished the patch entirely. I don't know why I never released it. The text files were being cleaned up for typos and mistakes by James Ó Nuanáin (Guv'ner). I'm not sure if he ever released his work, so I'll have to see if he posted a final version. I know I do have a few more revisions that I never included up until ~6/2013.

I took a few screenshots of everything. Since it looks like all the work is pretty much done, I will definitely post a public draft the installer for people to test in about two weeks. Then troubleshoot any problems.

Good news .

I would *really* like to get the active decompilers sharing data with each other, since otherwise there's just massive duplication of effort which could instead be directed towards more improvements to AC. The initial work of merging discovered functions would be significant, but I suspect would be quickly payed back by having three times as many people working on the project.

Yitzi, PlotuxRedux, Dio, and scient when you get here:

Would you be willing to share your data if it was frictionless to sync?
What form is your decompiling work stored in, and what program do you use?
Are you willing to help merge everyone's information?

I want to see this happen because I like it when things get fixed and improved and think this would be a major step forward. I don't know exactly what I can do to help as a non-programmer, but if there's anything I can do to make it more likely I'm in (e.g. teaching people github, talking to people, organizing things).

[PlotinusRedux]

I'm actually working along the same lines as Scient, reconstructing the original C++ class structure.  Unfortunately they severely abused multiple inheritance which makes detangling them particularly messy.

I'd love to swap notes with him, and I'm happy to share my research with anyone.  I use IDA Pro, but for people not using that I also have C header files, map files, etc.

My modifications are actually being done in C++ code directly referencing the decompiled classes, with assembly just used for hooks and thunks.

[BFG]

I continue to be impressed by your dedication and passion on this.  I've actually programmed some video games but nothing anywhere close to the complexity of SMAC - I wouldn't even know where to begin with this.  And to think that some of it's in assembly...

[ete]

I sent PlotuxRedux's post to him

That's awesome! I will have to touch base with PlotinusRedux. I too am decompiling the asm back into c++ source code more or less 1-1. I've been optimizing the code and updating it as I see fit. So far, I've mainly been working on start up code to parse the initial files. While I have identified all the major game logic, I wanted to have a working engine shell first.

As for code analysis, I first started identifying functions on my own about two years ago. Then I realized a nice trick. The mac binary, due to how gcc (I think that was compiler used) and macosx works, contains all the ORIGINAL function names. As in, what the SMAC/X devs used. This makes identifying the logic of what a function is suppose to do a lot easier. Granted, it has taken a lot of time and effort to match the PPC code up with the intel code (mac code base vs win code base). There are also a number functions not referenced directly which do not have this information. However, I have almost the entire binary mapped out in IDA. I still have much work to do, I've started going through and cleaning it up. This analysis has taken up the bulk of my time, countless hours have gone into my database. I would be happy to sync up work with whoever.

Overall, it would be great to set up a git (not necessarily on github) to help with working on this project. My goal is to have an open source version of SMAC/X in the end, similar to OpenXCom and Riven X.

I will see about getting myself on the boards sometime around 12/20 when I'll have some time off. I look forward to chatting with everyone, it's been too long!

This is looking very promising

[Dio]

Contact successful!

Hey!

I apologize for my absence from the community. You are correct, I have been extremely busy. Various things in life have kept me occupied. I still do have the patch stored with a large number of changes I never released. I forget if I ported all changes over to SMAC.

Even though I have been absent, I have be actively working on SMAC/X project. I have shifted from fixing bugs to complete decompilation of the game. There are just too many things I'd like to change and resolve that patching at the assembly level is no longer an option. This will take a number of years, at least if I work on it solo in my free time, however I have made a lot of progress. I have identified almost all the major functions inside the binary which makes stepping through the code and looking at logic a lot easier.  Also, I have decompiled a few of the base internal classes back into C++. I've been doing them in blocks and building upon each other with subclasses.

Perhaps in a couple weeks when I have some time off I will make a post on forums about my work along with the unreleased patch. If I don't have time, then I will email you everything and let Yitzi sort it out.

I hope all is well. If you don't hear back from me on boards or by email in a few weeks, feel free to remind me.

Brendan (scient)

I just had a quick look and it seems like I pretty much finished the patch entirely. I don't know why I never released it. The text files were being cleaned up for typos and mistakes by James Ó Nuanáin (Guv'ner). I'm not sure if he ever released his work, so I'll have to see if he posted a final version. I know I do have a few more revisions that I never included up until ~6/2013.

I took a few screenshots of everything. Since it looks like all the work is pretty much done, I will definitely post a public draft the installer for people to test in about two weeks. Then troubleshoot any problems.

Good news .

I would *really* like to get the active decompilers sharing data with each other, since otherwise there's just massive duplication of effort which could instead be directed towards more improvements to AC. The initial work of merging discovered functions would be significant, but I suspect would be quickly payed back by having three times as many people working on the project.

Yitzi, PlotuxRedux, Dio, and scient when you get here:

Would you be willing to share your data if it was frictionless to sync?
What form is your decompiling work stored in, and what program do you use?
Are you willing to help merge everyone's information?

I want to see this happen because I like it when things get fixed and improved and think this would be a major step forward. I don't know exactly what I can do to help as a non-programmer, but if there's anything I can do to make it more likely I'm in (e.g. teaching people github, talking to people, organizing things).

I have a whole slew of user comments on things that I have identified using the program Ollydebug. I also released a notepad file that listed all the addresses and names of the labels from labels.txt that are present in the assembly game code. I have never used C++ coding but woud be willing to look into it for this game.
I would be more than willing to share what I do know about the game code and the state of various sound effects, voices, scripts, and pcx's within the said code with those that are interested.

[Lord Avalon]

    

[ete]

@PlotinusRedux & Dio: Okay, cool . I think it'd be a good idea for you to both post the notes and decompilation progress you have here if you're okay with that, that way all the interested people can just download rather than having to each send them out as requested.

[PlotinusRedux]

...While I have identified all the major game logic, I wanted to have a working engine shell first.

Any idea where I could get a copy of that Mac executable?

To Scient: I've approached it as a hybrid project.  My C++ code compiles into a DLL that gets loaded into the same address space as the .exe.  In the C++ code I start by stubbing out the classes with the methods initially just being jmp's to the exe image in memory.  Then as I need to modify a method, I reverse engineer it and replace the jmp with actually C++ code and overwrite the exe image version with a jmp to my C++ code.

That way I don't have to wait until it's all converted for it to be useful, and over time more and more of the code will migrate over to the C++/dll side.  In the end, all of it could do so and replace the original .exe altogether.

[Buster's Uncle]

Note that scient was just by...

[scient]

I decided to pop on here sooner than I planned heh. The mac binary is from the mac port of the game by Aspyr, not the GOG version which is similar to a wine port (I believe).  The actual version I'm using is a beta patch by one of the port developers Brad Oliver. I've attached it below. However, I would wait for me to finish cleaning up my IDA database. I have spent a lot of time porting over the function information into the intel binary so it would be a wasted effort to do it all over again. They're not 1-1 but you can see patterns of function calls and once you start identifying things it becomes easier and easier. The mac binary is in PPC asm, which I personally cannot understand as well as intel. On a side note, one of my goals with SMAC/X decompilation projection would be to create a native mac version that runs without the need for wine or emulation.

Here is screenshot of function names for mac SMACX binary in IDA:


Here is example of my copy of WinMain:


I haven't figured out how to get IDA to display the class double colon properly, so it changes it to "__" when displayed. However, as you can see it makes stepping through the code a lot easier. 

It also makes finding connections a lot easier, especially for the internal engine code with all the jumble of inheritance. I have started to create the internal structures and enums in IDA as well.

...

The hybrid method sounds interesting. I guess since I've been working on my own, I have been more working on just decompiling classes as I go. Most of my work has been going into analysis of binary for now, so I have finished a few basic classes. I need to split some of my code as I have a lot of functions in general.cpp related to parsing the technology and text files. I'm almost done with reading all text files and loading that information into the globals for example, like labels. Also, I've been trying to ditch some of their usage of globals where ever possible.

The ones in done are "complete" and have gone over a clean up with comments so they look like the next screenshots.


In end, you get:



The next two weeks are going to be quite busy but I will try and stop by from time to time. And I will see about putting out a beta patch with my v2.0 with a lot of fixes and changes I finished about a year ago but never released.

[scient]

Also this was a reply to ete that I'll quote here as well.

I haven't used github much, pretty indifferent. I liked the idea of launchpad because it also included the ability to track bugs in a more robust way than github. Something I had wanted to have set up for SMAC/X project as well.

For example, see here: https://launchpad.net/rivenx

Riven X project doesn't have bug tracking but if you look into launchpad vs github there is some discussion about it.

From what I looked into, legally decompiling it isn't an issue. However, resources it might get hazy.

I agree, I will wait to make a test release until after they are merged.

I've attached the installer script. It's a very easy scripting language to create the installer.

Perhaps others have a better idea about which we should use. I thought having repository as well as bug tracking system would be a great way to index bugs as well as start decompiling the code. Actually, as I have been working on a few functions I have found a number of bugs in engine. They will happen under rare circumstances but still exist. For example, there is a set of functions designed to index the text files to speed up reading from them which it does. However, the code has bug that breaks the implementation completely so it never works. I have since corrected this. 

edit:
Hey BU, hope all is well. ^^

[Buster's Uncle]

I've been better and been worse. 

Obviously, my big hobby -this community- is rolling along robustly, if not so much as I aspire to.    

We've missed you.  Did you not see all my skypes this summer, or did I keep catching you at a bad time?

[scient]

Likewise.

Yes, this community seems to be doing well!

One of my best friends past away suddenly at the start of the summer, so I didn't do much of anything.

I will see about popping on Skype more in the future. It would be great to chat.

[Buster's Uncle]

And I just want to say to ete, publically:

   

You've done a man's job, sir; and I'm not just talking about getting scient active - we've needed to get our patchers talking to each other for a long time.  Name your accomplishment badge; you've got one coming, for all your organizing activities lately.

[Dio]

Wrong topic to post this in...