SMAC Movie and other Upscaling Improvements

[DrazharLn]

I've been looking into enabling some clever image scaling stuff in SMAC and I believe this can probably be done with the ddhack project. A preliminary investigation with that library shows that the hack doesn't quite work for SMAC.

At the moment, the ddhack displays the game in the top left corner (at the correct aspect ratio) and won't stretch it. Some other features of the hack including oldlcd mode, etc also don't work, but the game itself does. That suggests to me that it might be quite simple to fix.

Anyway, one problem is that if you start SMAC in DirectDraw mode (which you have to to get the ddhack working), it will crash the first time a video is shown. That's because the videos are displayed by a subprocess that forces a particular resolution.

If you run with DirectDraw=0, you can get the game to run at your desktop resolution, but only until the first video is played and it drops back into DirectDraw mode.

The proposed fix deals with both of these problems.

If a friendly .exe modder were to replace that subprocess with a call to open an external .exe and wait on it returning, I could trivially replace the external .exe with an .exe of my own to parse the input arguments and call ffplay. As terranx.exe seems to contain a complete copy of play uv (which could be removed), this could also claim a lot of extra space for new mod stuff.

With ffplay, I can display the videos at whatever resolution I please. I can also enable some clever upsampling and other filters to make the videos look a lot nicer at large resolutions.

This would both be a good first step towards enabling scaling throughout the game and also a good feature on its own.

If anyone wants to take a crack at it, the string "Play uV/uV2 Version 1.50, Martin Griffiths, Electronic Arts UK 1997" exists in terranx.exe on line 409970 (.data: 00691f08) and in terran.exe on line 430174.

The strings ".wve", "movies\virtualWorld.wve", "mplaynow.exe", "play()", "play_frame()" also exist in both files and might also be good starting points.

[DrazharLn]

@Assembly-nauts, Disassemblers and terran-spelunkers

I had a bit of a go with IDA today, but I can't find the code that plays a movie. I was trying to find where the opening movie was called but, confusingly, aDisableOpening and aDisableopeni_0 (as IDA calls them) aroundabouts 0068BF18 aren't cross referenced anywhere.

I think I'll leave hunting the code to you lot for now. Hopefully you'll be able to find the appropriate code quicker with your greater familiarity with the code base and IDA and assembly in general.

Best of luck and thanks if you do decide to have a look at how to do this.

[Yitzi]

i/o is probably the hardest thing to do in assembly modding, and calling an external .exe counts there.  If you can write (or copy from elsewhere in a way that will work) the assembly code to call an external .exe, I can put it in there, but it's not something I'm touching.

And in terms of freeing up space it wouldn't help much; there's plenty of space in the executable portion, it's the memory portion where it gets tricky.

[DrazharLn]

I/O in assembly is a matter of calling the appropriate library and kernel functions, no?

Edit: found a better call below, though ShellExecuteA is still interesting.

terranx.exe has a likely I/O call baked in via an external call to ShellExecuteA in shell32.dll.

ShellExecuteA takes a bunch of parameters including the path to a file. It then asks windows to either print the file back or open that file with whatever windows deems best. This almost certainly means that you can use it to call external .exe or .bat files.

In the existing code, the path to open is a URL eastore or alphacentauri.com and shell32 presumably invokes the default browser with it. Maybe it's code left over from the Demo?

Anyway, there's an example in the code at 0052476C. I don't know where that gets called from in the game at the moment, but you can presumably get some part of the code you control to call it. Then all you need to do is change the value of the address that currently reads "http:\\\\www.alphacentauri.com" to "test.bat" and see what happens.

An example test.bat to put in the same directory as terranx.exe might be:

Code: [Select]

@echo off
date /t >> shellexecutea_log.txt
time /t >>  shellexecutea_log.txt
echo %* >> shellexecutea_log.txt

If that works, then we just need to make a wee subroutine that'll take a movie name and pass it to a batch file. That's presumably just a matter of hijacking whichever bit of code currently opens movies.

If that works, then there are only a few more low hurdles to pass.

I think it's worth a try.

[DrazharLn]

In addition to ShellExecuteA, terranx.exe also includes the WinExec call from Kernel32.dll

WinExec takes two strings and runs the binary that corresponds to the first string with the parameters in the second.

terranx.exe uses it to call the process "mplaynow.exe" at 0062E2ED.

[Yitzi]

I/O in assembly is a matter of calling the appropriate library and kernel functions, no?

Edit: found a better call below, though ShellExecuteA is still interesting.

terranx.exe has a likely I/O call baked in via an external call to ShellExecuteA in shell32.dll.

ShellExecuteA takes a bunch of parameters including the path to a file. It then asks windows to either print the file back or open that file with whatever windows deems best. This almost certainly means that you can use it to call external .exe or .bat files.

In the existing code, the path to open is a URL eastore or alphacentauri.com and shell32 presumably invokes the default browser with it. Maybe it's code left over from the Demo?

Anyway, there's an example in the code at 0052476C. I don't know where that gets called from in the game at the moment, but you can presumably get some part of the code you control to call it. Then all you need to do is change the value of the address that currently reads "http:\\\\www.alphacentauri.com" to "test.bat" and see what happens.

An example test.bat to put in the same directory as terranx.exe might be:

Code: [Select]

@echo off
date /t >> shellexecutea_log.txt
time /t >>  shellexecutea_log.txt
echo %* >> shellexecutea_log.txt

If that works, then we just need to make a wee subroutine that'll take a movie name and pass it to a batch file. That's presumably just a matter of hijacking whichever bit of code currently opens movies.

If that works, then there are only a few more low hurdles to pass.

I think it's worth a try.

Maybe, though it's not going to be a high priority for me to do.  Image scaling is nice, but not as important as, say, an alternate tech rate formula.

[DrazharLn]

Well, thanks for considering it.

Here are the steps I think are necessary to patch in the feature I'm describing:

1. Identify if WinExec works

Put this at the start point of the program (I don't know enough about editing binary files to do this properly).
cmd db "sometest.exe"
push 1
push offset cmd
call ds:WinExec

put sometest.exe in the same directory as terranx.exe

2. Identify where the code to show films is

Run the game with enough breakpoints to find where the movies are played from.
Replace the movie playing bit with:

cmd db "filmplayer.exe $filmname" *
push 1
push offset cmd
call ds:WinExec
call ds:sleep $length_of_film_in_milliseconds

The top line there isn't immediately convertable into real assembly. I guess the way to do it would be to generate a string long enough for all possible film names and then copy in whatever array they're currently using for the film name (which they must have in order to open the right file).

3. Some more small changes dependent on how exactly SMAC handles films.

I could then make filmplayer.exe and hopefully hack on ddhack to make everything prettier, make alt tab work properly and so on.

The assembly to do this isn't so hard, but I don't know how to edit these binaries. What's your workflow?
* Do you disassemble, edit assembly code and then re-assemble? (Doesn't look like it)
* Do you disassemble, find the code you want to change and then edit in a hex editor?
* Do you do something else?

I've been messing with IDA and olydbg. What tools do you use?

If any of you would like to have a go at it, you're more than welcome.

[Yitzi]

The assembly to do this isn't so hard, but I don't know how to edit these binaries. What's your workflow?
* Do you disassemble, edit assembly code and then re-assemble? (Doesn't look like it)
* Do you disassemble, find the code you want to change and then edit in a hex editor?
* Do you do something else?

I use ollydbg to disassemble, and you can change things right in there, and then just copy to the executable.  It does the assembly-language-to-machine-code for you, so editing the binary is actually fairly easy.  The hard part is making room for the new code if it's longer than the old one (not that hard in most cases, but in some places it is) and making sure you don't mess up any calls or jumps in the process.  And of course finding the spot in the first place and figuring out how it currently works.

[DrazharLn]

I found a call to the existing video playing code at 0052ab6d (in yitzi2.5g). It takes one argument, the name of the movie to play without path or file extension.

The code that seems to handle project movies calls the same thing.

I followed the code through for a while and found a string concatenation subroutine and, eventually, the movie playing thread starting (at which point ollydbg stopped being able to step through the code).

I've managed to patch in a call to WinExec that can open a binary I made or cmd.exe (so I can run .bats, if I like). I've checked that parameters are passed correctly.

#todo soon:
Write an assembly function to accept some parameters and send them to a hardcoded binary.
Replace the existing calls to the play movie thing with a call to mine.
Write some external files to play the movies.
Test if this still upsets directdraw mode.

#todo later:
Make terranx.exe wait on the external call finishing. (Could do this by writing a temporary file with the external and checking for it with terranx.exe kernel32.readfile or something).
Put a switch into alphax or (better) Alpha Centauri.Ini to select film player (internal/external)
Find appropriate scaling and other filters to use on the videos.
Possibly generate the upscaled movies ahead of time, depending on speed of filters.

Yitzi, et. al, if you could tell me where and how you read new values from alphax and test them, I'd appreciate that.

Edit: Saw your post in the yitzi2.5g thread. Could you go into a little more detail about the process of adding a new alphax.txt variable for me?

[Yitzi]

Edit: Saw your post in the yitzi2.5g thread. Could you go into a little more detail about the process of adding a new alphax.txt variable for me?

It's really simple: Add it to the text file, add an appropriate function call (depending on size) at the end of the RULES section of the procedure (or in the middle, but then everything after it has to be adjusted for the new memory location), and it will be loaded to the next memory location.

I can do that part if you write the procedure.  Alternatively, this might be better working based not on an alphax variable, but rather on whether directdraw is enabled.

[vv221]

Not sure if I understand what's going on here…
Are you going to hack the .exe to have it use ffplay to display the videos?

If that's the case, does that mean we should be able to use other formats than .wve for the videos, like .mpg which is used for the Linux version?
It would be a huge step forward in modding possibilities!
(conversion from .wve is easy as cake pie, but I still don't know if conversion to .wve is doable)

A shame that I don't know anything about Windows binaries hacking, all the help I can give you is my full support

[DrazharLn]

@vv221,
I'm trying to hack the .exe to make it ask some external program to play the videos. It's the first step of a plan to ultimately improve the appearance of the whole game by hacking the directdraw library (which will probably be easier than this bit).

If it turns out that I can write good generic code for calling external binaries, then there's also a possibility of serialising bits of the game's internal state and sending it to other external programs that could be used to decide when to show new interludes, for example, depending on the game state.

That's for another project though (and could be far too much bother). This one is just for replacing the video drawing code and later the directdraw library.

I think it will make the game look a lot nicer on modern computers, fix the aspect ratio on widescreen monitors and allow alt-tabbing to work as people expect it to.

@Yitzi,
Thanks. I'd rather use the .ini, but I haven't worked that out yet.

[DrazharLn]

@Yitzi
What do you know about SMAX's runtime memory management? I need to allocate a decent size buffer (~64 bytes) for building a string for winexec.

The simplest approach that occurs to me is just to write to use the next 64 bytes of the stack as a buffer, but if SMAX uses the stack inappropriately that will cause problems.

Where in memory do you put your new variables?

Also, where in the file do you put your new constants and functions? There's a lot of empty space around, but I don't know if SMAC writes to it or not.

@The record

My (working) assembly so far:

Code: [Select]

play_video:
; Subroutine prologue
push ebp ; Store the caller's base pointer for later.
mov ebp, esp ; Replace base pointer with stack pointer for easier
; calls.
push ebx ; Store caller's registers we might use
push edi
push esi

; Subroutine body
; Accept one parameter, concatenate it with a constant string and pass
; to WinExec.

; Need a large buffer for string concatenation. Longest original movie
; name is 20 characters, player name is 12 characters.
; Allocate 64 bytes to allow modders not to care too much and think
; about checking with strlen sometime.
sub esp, 64
push offset command
push esp
call strcpy

; eax and esp should be the same thing here.
push [ebp+8]
push eax
call strcat

push 5
push eax
call winexec

; Remove buffer
add esp, 64

; Wait for some indicator file to appear

; Subroutine epilogue
pop esi ; Restore caller's registers.
pop edi
pop ebx
mov esp, ebp ; Deallocate our local variables (if we used any)
; and restore esp.
pop ebp ; Restore ebp
ret

It's got some boilerplate function stuff in it at the moment that I might later purge to recover space. It also allows a buffer overflow attack from movlist.txt and doesn't yet wait for the external binary to finish.

[Yitzi]

@Yitzi
What do you know about SMAX's runtime memory management? I need to allocate a decent size buffer (~64 bytes) for building a string for winexec.

The simplest approach that occurs to me is just to write to use the next 64 bytes of the stack as a buffer, but if SMAX uses the stack inappropriately that will cause problems.

By "inappropriately", you mean using stuff past ESP?  I'm pretty sure that it doesn't do that, and for that matter that I haven't done it either.  (I've done some pretty out-of-the-way things, but I don't think that's one of them.)

Where in memory do you put your new variables?

I cleared some space in the 949738-867 area, but that's getting somewhat full.  However, I have found a buffer at 9B86A0, and it seems to be fairly large (more than 64 bytes); you'll have to make sure it's not currently in use, but that seems your best bet.

Also, where in the file do you put your new constants and functions? There's a lot of empty space around, but I don't know if SMAC writes to it or not.

Generally, it does.  New stuff in the code is just put wherever I find room; there's a lot of empty space in the executable portion of the code.  But that's not writable by the program, and free writable memory is quite a bit harder to get.  I'd say use the buffer at 9B86A0 if you can, and otherwise just use the stack.

Code: [Select]

play_video:
; Subroutine prologue
push ebp ; Store the caller's base pointer for later.
mov ebp, esp ; Replace base pointer with stack pointer for easier
; calls.
push ebx ; Store caller's registers we might use
push edi
push esi

; Subroutine body
; Accept one parameter, concatenate it with a constant string and pass
; to WinExec.

; Need a large buffer for string concatenation.

Yeah, that's what 9B86A0 is usually used for.

It's got some boilerplate function stuff in it at the moment that I might later purge to recover space. It also allows a buffer overflow attack from movlist.txt and doesn't yet wait for the external binary to finish.

If you want me to add it, you'd also have to replace the external calls with something that I can actually put it.  (Well, the call to winexec, at least.  I know where strcat is, and that can be used to imitate strcpy just by setting the first byte to 0 and then calling strcpy.)

[DrazharLn]

Re: stack,
Yes, I did mean using data past ESP.

Re: unused writable memory,
Thanks for the suggestion, I may use that for a "which film interface" type switch.

Re: unused executable memory,
I'll just dump everything at the end of the executable section for now, then.

However, I have found a buffer at 9B86A0, and it seems to be fairly large (more than 64 bytes); you'll have to make sure it's not currently in use, but that seems your best bet.

Need a large buffer for string concatenation.

Yeah, that's what 9B86A0 is usually used for.

That's contradictory. Have you ever seen SMAC use any of the writable memory around 9B86A0?

If you want me to add it, you'd also have to replace the external calls with something that I can actually put it.  (Well, the call to winexec, at least.  I know where strcat is, and that can be used to imitate strcpy just by setting the first byte to 0 and then calling strcpy.)

Thanks for the offer. The code isn't ready yet (not feature complete, not tested), but when it is I'll be sure to send you what you need with addresses based off your latest patch.

strcat and strcpy are standard C functions that have been rendered in the main body of the code. The call to WinExec goes into Kernel32.dll. I find them all with IDA Pro, but I can give you exact addresses to use once the code is ready to include.