Decompilation coordination thread

[ete]

The OpenSMACX repository is ready for use! I'm still fiddling with the bug tracker and a few other things, but if everyone interested could register on github and set up github for windows (or other OS equivalent) that would be great. Post here with your username and I'll add you as a contributor to the repository.

I'm guessing we're probably better off with the shared repository model rather than fork and pull, but am open to switching.

Also I'll be hanging out on #smac a lot over the next few days if anyone feels like chatting about plans or wants some help getting used to github. Just say my name in the channel and my client will alert me, otherwise I may not see you've arrived for an hour or two.

[ete]

The bug tracker now has a bunch of bugs in: https://github.com/OpenSMACX/OpenSMACX/issues

Ones labeled possible-bug need some discussion.

[scient]

I think shared model is good for now. Once we reach alpha or beta stage, maybe switching it over. But that won't be for some time.

I just set up github id:  b-casey

Also, I wonder if I should keep a document of any bugs that I've already fixed in my source. There are about 3-4 that come to mind as part of engine code.

[ete]

Okay, invited you to the organization.

That's probably a good idea. Maybe best to collect up everyone's individual changelogs in a text file in the repository, so we've got one unified changelist?

[scient]

Well, I don't really have a change log myself. I have made some comments at the top of my header files where I've been keeping comments.

One thing we'll need to worry about at some point is having a unified code format.

For example:

/*
 * heap.h
 * SMAC/X Project
 * Brendan Casey
 *
 *  v1.0 (6/30/14)
 *
 * Class to handle the low level heap operations to manage or allocate memory
 * Removed an unused error flag (+0) and an unused parameter from squeeze()
 */

[ete]

Well, I don't really have a change log myself. I have made some comments at the top of my header files where I've been keeping comments.

One thing we'll need to worry about at some point is having a unified code format.

For example:

/*
 * heap.h
 * SMAC/X Project
 * Brendan Casey
 *
 *  v1.0 (6/30/14)
 *
 * Class to handle the low level heap operations to manage or allocate memory
 * Removed an unused error flag (+0) and an unused parameter from squeeze()
 */

Right, hm. It probably would be worth collecting information on that kind of change up somewhere.

And what do you mean exactly by unified code format? If everyone's stable changes get merged into the shared repository, does that achieve that?

[scient]

I mean, for the final product I think all the source files should follow certain guidelines of how the code looks. For example, I've been storing all the function comments in header file except for a few runtime bits in the cpp file. Having some unified header format and such. For now, I think just getting it all up onto github is first priority and then sorting it out as we go along.

Example of my heap class decompiled from game.

/*
 * heap.h
 * SMAC/X Project
 * Brendan Casey
 *
 *  v1.0 (6/30/14)
 *
 * Class to handle the low level heap operations to manage or allocate memory
 * Removed an unused error flag (+0) and an unused parameter from squeeze()
 */

#pragma once
#include "stdafx.h"
#include "general.h" // mem_get()

class Heap
{
   private:
      LPVOID basePtr;    // (+4) -> pointer to base memory address
      LPVOID currentPtr; // (+8) -> current memory address position
      size_t baseSize;   // (+12) -> size of total memory
      size_t freeSize;   // (+16) -> size of free available memory

   public:
      // Constructor and Destructor
      Heap(): basePtr(0), currentPtr(0), baseSize(0), freeSize(0) { }
      ~Heap() { shutdown(); }

      // Get base total memory size
      size_t getBaseSize() { return baseSize; }
      // Get base memory address pointer
      LPVOID getBasePtr() { return basePtr; }

      // Destroy current heap and zero out all class variables
      // LOCATION: 005D4580
      void shutdown();

      // Deflate heap of any free memory
      // LOCATION: 005D45E0
      void squeeze();

      // Allocate memory based on requested size from parameter
      // Return TRUE if memory allocation failed, otherwise FALSE
      // LOCATION: 005D4620
      BOOL init(size_t reqSize);

      // Check if there is currently enough free memory for requested size from parameter
      // If not, allocate more memory to heap in blocks of 1024 bytes until there is enough
      // Return a memory pointer with address to requested size free
      // LOCATION: 005D4680
      LPVOID get(size_t reqSize);
};

/*
 * heap.cpp
 * SMAC/X Project
 * Brendan Casey
 *
 *  v1.0 (6/30/14)
 *
 * Class to handle the low level heap operations to manage or allocate memory
 */

#include "heap.h"

void Heap::shutdown()
{
   if(basePtr){
      free(basePtr);
   }
   basePtr = currentPtr = 0;
   baseSize = freeSize = 0;
}

void Heap::squeeze()
{
   size_t usedSize = baseSize - freeSize;
   LPVOID newAddr = realloc(basePtr, usedSize);
   if(newAddr){
      basePtr = newAddr;
      baseSize = usedSize;
      freeSize = 0;
   }
}

BOOL Heap::init(size_t reqSize)
{
   if(basePtr){
      shutdown();
   }
   basePtr = mem_get(reqSize);
   if(basePtr){
      currentPtr = basePtr;
      baseSize = freeSize = reqSize;
      return FALSE;
   }
   return TRUE; // failed to allocate memory
}

LPVOID Heap::get(size_t reqSize)
{
   while(freeSize < reqSize){
      LPVOID newAddr = realloc(basePtr, baseSize + 1024);
      if(!newAddr){
         CHAR szError[150]; // max size of string + three int(s) + extra
         wsprintf(szError, "Aborting due to a heap shortage!\nBase size: %d\nFree size: %d\nRequested size: %d",
            baseSize, freeSize, reqSize);
         MessageBox(NULL, szError, "Heap Notice!!", MB_OK);
         exit(3);
      }
      basePtr = newAddr;
      currentPtr = LPVOID(size_t(basePtr) + baseSize - freeSize); // fix in case realloc shifts memory
      baseSize += 1024;
      freeSize += 1024;
   }
   LPVOID freeMemAddr = currentPtr;
   freeSize -= reqSize;
   currentPtr = LPVOID(size_t(currentPtr) + reqSize);
   return freeMemAddr;
}

[ete]

Right, coding conventions? And yes, I agree that getting your (and Plotinus's) current work up and shared is the next step, can decide on exactly how to organize things and which conventions to follow after. When everyone's work is public I'll contact kyrub.

Edit: Contacted Guv'ner.

[Yitzi]

By the way, because of how I did my major changes they may not be so easy to decompile normally; in such case, feel free to decompile Kyrub's version or earlier and then ask me to edit the result to fit my changes.

[scient]

I think the easiest thing is to decompile the original code and then apply any updates or changes from there. If it's minor bug fix, then it would be simple to apply it as you go along. For more intensive patches, it would probably be best to apply after. Smaller room for error. Granted, I've mainly been focusing on engine code and little on game logic where most of my own patches reside.

[scient]

@Yitiz:
Taking a break from work, I started to look over your notes to see if there is any info to add into my IDA database. So far, there has been a few things. Although, I noticed a mistake and wanted to let you know.

Inside base structure:
"Turns left to revolt: +7"

This is actually the number of turns left until assimilation of the base is complete. Once it goes down to zero, the former faction is set to current faction and the base no longer looks like the previous owners bases.

The info about base flags is useful, I only had rioting and golden age.

[Yitzi]

Inside base structure:
"Turns left to revolt: +7"

This is actually the number of turns left until assimilation of the base is complete. Once it goes down to zero, the former faction is set to current faction and the base no longer looks like the previous owners bases.

The info about base flags is useful, I only had rioting and golden age.

Yeah, I just mis-typed what I meant.

[ete]

There are now 29 issues in the tracker. I'd like some feedback on tagging from anyone who's interested at some point. There are also a bunch of possible bugs that need discussion.

Yitzi, would you be okay with me tracking feature requests from your thread on this?

Also, progress update on text files. Mostly done, but conceptsx has massively changed and specifically reordered which makes comparing diffs much slower. There's a whole lot of good changes, but there's also a few formatting errors and other things to fix so I need to check over it properly.

[Yitzi]

There are now 29 issues in the tracker. I'd like some feedback on tagging from anyone who's interested at some point. There are also a bunch of possible bugs that need discussion.

Yitzi, would you be okay with me tracking feature requests from your thread on this?

Go ahead.

[scient]

Yitzi, could you clarify these from variables.txt?

Factions: Size 20CC.  Faction 0 would have:
  Nessus at 96D1EC
  AI value for SUPPORT x at 96D080+4*x.


virtual mineral count: 946138+4Xfac#

Otherwise, I merged in any missing details from all your text files into my IDA database. Most of it I already had, but every bit helps! At the moment, I have most of all the internal structures from text files mapped out. Also, unit and base are almost entirely complete except for a few members.  These make a big difference in poking around functions. I have portions of the large faction struct (includes diplomacy related info) but that will take some time. The same goes for world map. Granted, having all the game logic functions identified with original names will make this analysis go a lot faster.

By the end of the year, I should have my database all nice and tidy to share. Of course going through each function and marking it up with the enums and structures will be required as well as local variables. The same goes for analysis on the remaining structures. However, the more complete the database is the easier it is start decompilation of game logic code.