Alpha Centauri 2

Sid Meier's Alpha Centauri & Alien Crossfire => Modding => Topic started by: Ford_Prefect on February 06, 2014, 04:02:33 AM

Title: First CVR mod!
Post by: Ford_Prefect on February 06, 2014, 04:02:33 AM
Update: !  2014-03-16    See original post bellow.
Link to the download file: http://alphacentauri2.info/index.php?action=downloads;sa=view;down=158 (http://alphacentauri2.info/index.php?action=downloads;sa=view;down=158)


(http://sites.google.com/site/jordanprogramming/home/FirstEverCVRmod.png)

-------------- Original post------------------

I've had a breakthrough in deciphering the CVR file format.  ;zak;

What I've figured out: (Updated list on 2014-02-26)
I haven't figured out (yet):


Ok. Here is the nitty gritty details of what has gotten me so excited.  :D
Example hex dump from parts section:
Code: [Select]
6F D2 95 6F D2 95 6F D1 95 6F D1 95 6F D1 95
6F D1 95 6F D1 95 6F D0 95 6F D0 95 6F 76 95
6F 75 95 6F 1C 95 6F 1C 95 B6 C3 95 6E C2 95
5E C2 95 7E 69 95 B5 B6 95 15 5D 95 B4 AC 95
B4 AC 95 6C AB 95 ......
See that 95? That is the color that the voxel will use.
That was easy part.

Ok.  The part that was driving me nuts:  Position:  How was it calculated for each voxel?  First I thought it was an z, y with x being incremented every once and a while... but I was wrong.... So I got to toying with the numbers and ......

byte1 byte2 byte3  If I lay the data out like this.... with byte 3 being the color code.
60      D2     95

Lets look closer at byte 1 shall we?
x60 = 01100000
01100   000       
|-----| < this part, the first 5 bits, when it changes, changes the direction the next voxel will be located.  With 22 possible vectors.  (WARNING: Using a value larger than 11001 will cause access violations or crashes with the model viewer)

Yes, each voxel is a set distance from the last, with the first 5 bits of data telling which direction (in 45 degree increments) to go.

The 3 bits + byte 2 is something else.  Figuring out what they are used for is on my todo list.  :)

Title: Re: CVR... I've had a major breakthrough today.
Post by: Dio on February 06, 2014, 07:00:28 PM
First, your post was very insightful and I hope you are successful in deciphering the caviar file format for this game.

If you are interested, this is how the cvr files are actually used inside  the game code. The method they used to create the multipart chassises in the game was to assemble them in a single procedure that had the skeleton of the chassis in one procedure and from there they would add other pieces like armor and weapons. In addition, they added a memory address and call function to some of the caviar pieces that had an animation associated with them.
Title: Re: CVR... I've had a major breakthrough today.
Post by: Ford_Prefect on February 10, 2014, 02:03:27 AM
I've learned how single part objects store their coordinates.   ;excite;

The general layout:
zz zz xx xx yy yy E8 FF C7 FF EB FF 31 00 74 00 2B 00 zz zz xx xx yy yy E5 12 00 00 00 00 00 00
                         |---? ? ? ?? ? ? ? ? ? ? ? ? ? ? ? -------|                          |--Voxels---|
The voxel number is the number of voxels to the next jump.
zz, xx, yy are signed shorts.

I'm probably missing something with the coordinates as both sets have different numbers, but modifying them seemed to have the same results.

Also, multipart pieces have a different data structure so I'm gonna have to do some more work for figuring out their layout. 

I'm gonna take a break from looking at the hex files for a bit and try writing an exporter for single pieces.  After that I'll try to pin down where the data stores the center-point (I'll be able to do some trial and error with my own file viewer) and then start work on multi-part files.


Title: Re: CVR Progress
Post by: Ford_Prefect on February 12, 2014, 03:04:01 AM
(click to show/hide)

My first attempt at importing the data.  :D

I haven't programmed in the color, so I set it to random.
I need to get lighting to work properly in the viewer.  Learning how to program in this 3d engine will be interesting.
Title: Re: CVR Progress - Now with images!
Post by: Buster's Uncle on February 12, 2014, 04:28:12 AM
 ;excite;
Title: Re: CVR Progress - Now with images!
Post by: Ford_Prefect on February 13, 2014, 08:57:47 PM
(http://sites.google.com/site/jordanprogramming/home/progress2.png)

Status Update:

I've got my first cvr to convert and display!  I did try loading in some other pieces... but some of them failed halfway through the conversion.  So I guess one of my assumptions was wrong. Gonna have to take a look.

In other news.... Looking at the performance of just plopping in cubes for each voxel seems to give really bad performance.  So I'm gonna need a better method of rendering the data.  I mean seriously, the original CVR viewer is rendering the same data at 2000 fps!  If anyone has some suggestions, I'd be happy to hear them as I'm a complete noob when it comes to working with 3d graphics.
Title: Re: CVR Progress - Now with images!
Post by: Ford_Prefect on February 14, 2014, 12:58:46 AM
(http://sites.google.com/site/jordanprogramming/home/progress3.png)

Woohooo!  Can now load more one part models!  Both the ship and the missle now load.

Downside is that these high voxel count models really drive home the fact I need to come up with a better method of rendering them.
Title: Re: CVR Progress - Now with images!
Post by: Ford_Prefect on February 14, 2014, 04:12:34 AM
Good news, Bad news.
Good news - I've changed the way the engine renders the files and it can now handle them.  I've also got it loading the colors.

Bad news - the palate color codes from the palate file are not the same as the color codes in the CVR file.   Which means I'm gonna have to reverse engineer the palette location.   :mad:

TODO to get models into another game engine:

TODO:  To make it easier to get into another game engine
Title: Update
Post by: Ford_Prefect on February 26, 2014, 07:11:34 PM
I'm back!

I've been playing dungeon keeper 2 in my free time... which is why you guys haven't heard from me in a while. :-D

So, I spent half a day trying to pin down why the positional data appears and disapears in this one part.... and I haven't figured it out yet, but I did pin down some other data.

New stuff I can read:

I also know where the value for units per 3d voxel is stored.  I don't know how to read it yet.

What's next?
I'll probably try to read in the new data, and try to get both non-animated files to convert properly.  Sadly, I think I already figured out all the easy stuff.  I think all that's left is manually changing values or formulas and seeing what changes, which is slow.  I'll probably make some changes to my renderer so that I don't have to open and close it to view different models.
Title: Re: CVR Progress - Now with images!
Post by: Buster's Uncle on February 26, 2014, 07:23:11 PM
If this ultimately results in a .cvr editor so art modders could, for example, make appropriate-looking units for a Middle Earth mod --- the SMACX modding world will be laid at your feet.  I can't do a lot to sweeten the deal for you, but there's definitely a .cvr King badge for your postbit in it.  And the community would respect and owe you.
Title: Re: CVR Progress - Now with images!
Post by: Geo on February 26, 2014, 08:15:42 PM
Quote from: BUncle on February 26, 2014, 07:23:11 PM
...and awe you.

Corrected. ;nod
Title: Re: CVR Progress - Now with images!
Post by: Buster's Uncle on February 26, 2014, 08:26:31 PM
;b;
Title: CVR Editor?
Post by: Ford_Prefect on February 26, 2014, 10:02:30 PM
Picture of my work with ACP00.cvr

(http://sites.google.com/site/jordanprogramming/home/progress4.png)

Spent some more time on it and I think I've figured out how to grab the color palate.   :D
I'm also making progress on the positional data.  The main issue currently is that I think its supposed to have a different rotation location.  This is the simplist of the cvr files in alpha centari.  It was converted from a single mesh.  Infuriating me is the fact that this stores its location data differently than every other single part (but converted from multiple meshes) files.  So... I have to figure out how to detect this anomoly, or just stick in an exception for the ACP0# files.

Quote
.cvr editor

  Ok.  A .cvr editor... I wasn't originally planning on writing one.... but....
With what I know currently, I or someone else could write a "texture" editor for the models. 
I could also probably make it possible to move some parts relative to each other.

Without knowing how normals are stored, mesh editing wouldn't work.  As things would be very, very, very, ugly.




 

Title: Re: CVR Progress - Now with images!
Post by: Buster's Uncle on February 26, 2014, 10:15:41 PM
Y'know, I gather from your work that the files are true 3D - all I could tell from the .cvr viewer lo, all those many years ago, was that eight different positions displayed.  That's all the game needs anyway.  If, then, it was possible to substitute eight 2D flat shots of a covered wagon in place of the eight angles the colony pods display depending on which way they went - it would be a lot of work, but totally doable, and make that wild west mod someone dreamed of SO much better...
Title: Re: CVR Progress - Now with images!
Post by: Ford_Prefect on February 26, 2014, 10:21:59 PM
Actually, it would be more like spray painting the desired image onto the model using the palette colors.  Not sure what a different color palette would do to the object models in the game.  They both use a 256 color palette.  If you change the games color palette, does it change the colors of the units as well or does the game just crash?
Title: Re: CVR Progress - Now with images!
Post by: Buster's Uncle on February 26, 2014, 10:55:39 PM
Quote from: Ford_Prefect on February 26, 2014, 10:21:59 PM
Actually, it would be more like spray painting the desired image onto the model using the palette colors.  Not sure what a different color palette would do to the object models in the game.  They both use a 256 color palette.  If you change the games color palette, does it change the colors of the units as well or does the game just crash?
I believe you get an LSD trip, but no crash.
Title: Re: CVR Progress - Now with images!
Post by: Buster's Uncle on February 26, 2014, 10:58:29 PM
I'd love to see Dio kibbitz with you - he's and excellent art modder who is also the haxxor.

Kilkakon is also a fantastic artist who actually altered the pallet for his mod.  I do the art, but the tech stuff with the pallet is outside my interests.
Title: Re: CVR Progress - Normals. Help.
Post by: Ford_Prefect on February 27, 2014, 07:28:40 PM

Long version:
(click to show/hide)

Short Version:
I would really like it if someone else figured out the normals in the CVR file format as they are driving me nuts.
If no one does, I'll still be able to export the models to other programs, it just won't have the correct reflections on it.  (I'll whip up a formula to get it somewhat correct normals). 
If you do decide to take a shot at figuring it out, please tell me! 
Title: Re: CVR Progress - Now with images!
Post by: Dio on March 01, 2014, 02:33:19 AM
When you say shininess, do you mean how the light looks after striking the caviar parts? As far as I know, the game grabs the vector value that determines how the vehicles are lit and thus shininess from the #VEH_LIGHT_DIR_MAP in the system.txt file. I imagine the first number controls what value goes into x, the second y and the third z. You can find the coding for said feature starting at address 00451953. I hope this helps somewhat. I am unfamiliar with the exact mathematical formulas you might use (in particular, the trigonmetric formulas). However, I am willing to help you in any that I can within my present time constraints.
Title: Re: CVR Progress - Now with images!
Post by: Ford_Prefect on March 01, 2014, 03:37:22 AM
I meant the normals.  http://en.wikipedia.org/wiki/Normal_%28geometry%29 (http://en.wikipedia.org/wiki/Normal_%28geometry%29).  Normals tell the 3d renderer how light reflects off of an object.   So as the object is rotated, the correct parts reflect the light.  I should of used better terminology in my previous post.  If I don't have normals, the object will appear flat if it uses only one color.  You can't tell where corners are at, see overlapping parts, etc.

I wish I had paid attention back when I was a CS major (I switched) when we did reverse engineering.... I don't remember anything.   :-P  I would have to start from scratch.

Ok.  Here are my current notes on the CVR file format.  Not complete and probably wrong in several places. :-D  I'll post my code and an updated specification when I get all the models loading their geometry correctly.
(click to show/hide)

On the request for assistance,  helping me pinpoint which bits are used for the normal lookup would be very helpful.  The normals are stored with the voxel points.  In the notes I've identified the voxel lookup bits, and 3 of the bits used to identify the normals.  If someone could even find out which other specific bits are also part of the normal reference number, it would help me significantly. 
Title: Re: CVR Progress - Milestone Reached!
Post by: Ford_Prefect on March 03, 2014, 12:07:42 AM
Made some more progress today!

All (I think) single part non-animated files I can now load their geometry and their colors.
Also, I reworked the code to make it much more reliable and readable. 

Todo:

This image below shows why normals are important. :-D   Allot of the units are only 1 or 2 colors.  (As usual, the smaller bottom right image is what it's supposed to look like)
(click to show/hide)

EDIT:
Ick.  Just found a file that the colors are wrong on.... sigh.  Need to figure out what I assumed wrong this time.  ::)
Title: Re: CVR Progress - Milestone Reached!
Post by: Geo on March 03, 2014, 05:34:56 PM
Quote from: Ford_Prefect on March 03, 2014, 12:07:42 AM
EDIT:
Ick.  Just found a file that the colors are wrong on.... sigh.  Need to figure out what I assumed wrong this time.  ::)

Which file? Perhaps its a non-used one?
Title: Re: CVR Progress - Now with images!
Post by: Ford_Prefect on March 03, 2014, 09:34:16 PM
Bad news:  I decided to take a gander at normals.

Unless I'm mistaken, all 11 bits are being used for the normal data.  11 bits means 2048 possibilities.   :-\

Yeah, I'm not gonna be able to figure them out by brute forcing it.  I've looked at a good number of points and no useful patterns are appearing.

Any ideas?

Title: Re: CVR Progress - 2048 normals... :-(
Post by: Ford_Prefect on March 04, 2014, 02:53:56 AM
Oh.  Just found out some good news.   :D I just tried my code on select.cvr which is a single part animated file... and it worked (except for the colors were slightly off).   ;) 

I did some more looking at normals... and I'm not seeing any patterns that I could use to solve it.   :(

Help.  I can't solve normals by slowly modifying values and seeing what changes.... not when there are up to 2048 possible values and in 8-10 degree increments.  :(  (especially not at the resolutions I'm having to deal with)



Other news....
I'm gonna examine multipart files next and try to figure out how they are put together.
Then all I have left is getting the colors correct and
normals.  :( 

Title: Re: CVR Progress - 2048 normals... :-(
Post by: Ford_Prefect on March 05, 2014, 05:07:48 PM
Ok.  This is just weird.  I figured out where the color is going wrong.
Palette code 236.  All the other colors are correct, but sometimes the file will call on color 236, which I thought should be 0,0,0 (black).  But!  On some files, it isn't supposed to be black, but a light gray color... while on others, it is black!  The real fun bit is that the front section of the file where the palette codes are stored are identical.... So I have no clue as to why it is choosing one color over another. ???

So yeah.  Updated list of things I would like help on figuring out:
* Normals
* What determines the color of 236 or 246? (236 for A, Select, VWNAA.  245+ for Vwntu)
Title: Re: CVR Progress - 2048 normals... :-(
Post by: Buster's Uncle on March 05, 2014, 05:27:36 PM
True black (and a peach shade fdbd76) is semi-transparent in the game - they make the base shadows you don't notice in-game because the flag is in the way.  Those particular shades are unfortunate for custom faction art, because it's all too easy to end up with all the lines on a base see-through, and black bits on the leaderheads.
Title: Re: CVR Progress - 2048 normals... :-(
Post by: Ford_Prefect on March 06, 2014, 01:18:22 AM
Yeah.  I'm stuck.  Without someone who can figure out the proper values for the color issue and more importantly the normals, I won't be able to make any further progress on CVR files.

I'm going to work on tidying up my code and notes and release them here for anyone who is curious or wants to try to help fill in the blanks.

I'll see if with what I do know, if I can do some limited modding of the CVR files... but without all the color and normal data.... it will be really limited. 

I'm sorry I wasn't able to do more.
Title: Re: CVR Progress - 2048 normals... :-(
Post by: Geo on March 06, 2014, 07:55:55 PM
You already did more regarding caviar files then anyone else I remember on figuring out them. And for that I thank you, ;b;
Title: First CVR mod!
Post by: Ford_Prefect on March 06, 2014, 10:12:11 PM
(http://sites.google.com/site/jordanprogramming/home/FirstEverCVRmod.png)

http://alphacentauri2.info/index.php?action=downloads;sa=view;down=158 (http://alphacentauri2.info/index.php?action=downloads;sa=view;down=158)

I'll be working on making the color mod tool more user friendly and powerful before releasing it.  Right now I can tell it "replace all of color x with color y".  Without any hints as to what x or y will be. :-D   I'm working on fixing that as well as adding a few features.
Title: Re: First CVR mod!
Post by: Geo on March 06, 2014, 10:15:51 PM
Heh, color-changed faction-unique units?  :unworthy:
Title: Re: First CVR mod!
Post by: Buster's Uncle on March 06, 2014, 10:20:02 PM
I doubt that's doable, but  :adore: for doing anything at all.
Title: Re: First CVR mod!
Post by: Buster's Uncle on March 09, 2014, 01:58:30 AM
BTW, I don't get any notice that something's been posted to Downloads, so files will get approved a lot faster if you tell me...

;b; for posting the purple tank mod to Downloads.
Title: Re: First CVR mod!
Post by: Ford_Prefect on March 11, 2014, 05:16:24 PM
Just thought I post that I'm not dead.  With the weather becoming nice, I'm becoming busy with my job.  So I have less free time working on this project.  That said....

Status:
I'm writing up a program for editing the 'texture' data of cvr files.  Currently you can load a file and see the left side of it displayed in the viewer.  You can also see what colors are available (minus the 10-20 that are unknown) for painting.  I'm still working on getting the object to display correctly.  Progress is slow because... its python.  It  sucks for anything more complex than a simple linear script and the documentation for tkinter is pretty bad.
Title: Re: First CVR mod!
Post by: Geo on March 11, 2014, 06:17:33 PM
Quote from: Ford_Prefect on March 11, 2014, 05:16:24 PM
Just thought I post that I'm not dead.  With the weather becoming nice, I'm becoming busy with my job.  So I have less free time working on this project.  That said....

Oh, you're selling ice cream on a road stall? ;cute ;)
Title: Re: First CVR mod!
Post by: Ford_Prefect on March 11, 2014, 10:51:03 PM
No, I'm a farmer.  So I'm outside fixing my combine, planter, etc... for the upcoming harvests, planting, etc... while the weather is nice.
Title: Re: First CVR mod!
Post by: Geo on March 12, 2014, 05:28:51 PM
I take it you're living somewhere in West Europe then. I've been having a blast with 15-18° C weather lately.
Title: Re: First CVR mod!
Post by: Ford_Prefect on March 12, 2014, 07:13:31 PM
Lol.  No, Kansas, USA.
Title: Re: First CVR mod!
Post by: Kilkakon on March 15, 2014, 11:08:10 AM
Pretty impressive to see someone start to unpack the format. :) Good on ya!
Title: Re: First CVR mod!
Post by: Ford_Prefect on March 22, 2014, 09:06:58 PM
Progress Report:
What's done:

Todo:
Title: Re: First CVR mod!
Post by: Buster's Uncle on March 22, 2014, 09:32:42 PM
Impaler was telling me the other night how impressed he is with what you've done. ;b;
Title: Re: First CVR mod!
Post by: Ford_Prefect on March 23, 2014, 10:02:00 PM
I've released the tool
http://alphacentauri2.info/index.php?topic=7866.0 (http://alphacentauri2.info/index.php?topic=7866.0)

 :D :danc: :danc: :danc: :danc: :danc: :danc: :danc:
Title: Re: First CVR mod!
Post by: Geo on March 24, 2014, 03:52:42 PM
 ;worship
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 02, 2015, 09:52:38 AM
Ford--I don't know if you're still working on this, but I'm interested in the work you've done.  I'm considering a DX/OpenGL conversion of the graphics engine, where I might convert the .cvr files into a vertex format.

Did you ever get anywhere with the normals?

I'm just starting to look at the data, but looking at the lighting in the Caviar Player, something I've noticed is that edge voxels are not sharp.  I.e., where you have two planes that meet at right angles, if you can adjust the viewing angle so that you can see both planes, you get 3 colors--1 for the 1st plane, 1 for the 2nd, and 1, between the other 2, for the edge or seam between the planes.

That makes me suspect the normals may be calculated rather than stored in the file, along the lines of vertex formats that don't contain normals.  In such vertex formats, the normal is computed from the angles of the planes that meet at it (with the order of the triangles, clockwise or counterclockwise, determining which of 2 possible directions it could point, which is only needed if you distinguish interior from exterior).  Without the proper shading model, that gives precisely the effect I'm seeing in the cavier viewer--that edges seem to be filed down, so where planes meet at right angles, there's a single line of voxels at a 45 degree to them.

If so, there are a few ways that could be accomplished.  One would be using the direction vectors to the next voxel themselves to compute the normals--if you take the position of the previous voxel, the current voxel, and the next voxel in that chain, that gives you 3 points to define a plane with the normal being perpendicular to it.  Another possibility would be once the voxels are loaded, to give each voxel a normal based on the N nearest voxels to it.  In either case, the 11 bits you haven't identified could contain material information--how reflective the material was, etc.  It could even have something to do with palette stepping, since this is shading going on in a very restricted color space.

Alternately, the 11 bits might give addition information used in combination with the direction vector in the first 5 bits to determine the normal.

Or--I haven't looked at the data yet, do parts correspond to planes?  If so, the part itself may have normal information that applies to all the voxels it contains.  Since many neighbor voxels would have identical normals, it would kind of make sense to provide a single normal applying to a whole group of voxels rather than a separate normal for each one.

Really, the data you've already interpreted is enough for a conversion to vertex format, since it's enough to build a 3d block map I can then write code (if I can't find some already written somewhere) to convert into planes and thus vertexes--reducing thousands of voxels to 10's of vertexes that will themselves allow OpenGL or DX to compute the normals.  Though having the normals would simplify the conversion code, by making plane detection as simple as searching for adjacent voxels with the same normal.

And kudos on working out that direction translation table.  With just 6 bits they could have given each dimension its own 2 bits, it wouldn't have occurred to me they would go to a look up table to save 1 bit.
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 02, 2015, 07:38:29 PM
 
Quote
Ford--I don't know if you're still working on this, but I'm interested in the work you've done.
Its on hiatis.  If someone figures out the rest of the color data or the normals, I'll work on it again.  I'm glad to answer any questions you may have.

Quote
I'm considering a DX/OpenGL conversion of the graphics engine, where I might convert the .cvr files into a vertex format.
Oh cool.  8)  That would be really neat.

Quote
Did you ever get anywhere with the normals?
No.  I gave up.  :)

Quote
That makes me suspect the normals may be calculated rather than stored in the file, along the lines of vertex formats that don't contain normals.

The reason I think those 11 bits affect the normal is because when I modified them, I could reliably get the normal to be identical regardless of the slope. 

Quote
Or--I haven't looked at the data yet, do parts correspond to planes?
No.  There can by quite complex 1 part models.  Parts can have multiple planes.  Parts just mean that the file can move them in an animation.

Quote
If so, there are a few ways that could be accomplished.  One would be using the direction vectors to the next voxel themselves to compute the normals--if you take the position of the previous voxel, the current voxel, and the next voxel in that chain, that gives you 3 points to define a plane with the normal being perpendicular to it.  Another possibility would be once the voxels are loaded, to give each voxel a normal based on the N nearest voxels to it.  In either case, the 11 bits you haven't identified could contain material information--how reflective the material was, etc.  It could even have something to do with palette stepping, since this is shading going on in a very restricted color space.
The 3 points wouldn't get you the same gradient that the original model has for curved slopes (since each voxel is at 45 degree increments).   The n voxel idea could work, I had a similar idea, but I was rather burned out at that point and wanted to work on something different.


Title: Re: First CVR mod!
Post by: Ford_Prefect on January 02, 2015, 07:40:12 PM
Oh.  Most of my notes on the format is here.

http://alphacentauri2.info/wiki/Caviar (http://alphacentauri2.info/wiki/Caviar)
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 03, 2015, 06:11:48 PM
Forum for the python based editor I wrote.
http://alphacentauri2.info/index.php?topic=7866.0 (http://alphacentauri2.info/index.php?topic=7866.0)

Source code:
https://github.com/uaEquals42/CVR-Colorizer (https://github.com/uaEquals42/CVR-Colorizer)
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 03, 2015, 11:35:39 PM
Ford--

After ruling out a lot of possible formats, I'm convinced the 11 bits are an index into a hard-coded array of normals, so I'm now reverse engineering the code to locate it.  I haven't found it yet, but I've found some general information on the file format from the possible error messages you might be interested in.

The entire file is a series of chunks, consisting of:

4 byte Chunk Type
4 byte Chunk Size (including the Type above)
(Chunk Size - 8 bytes) bytes of data
Next Chunk...

Each chunk is immediately followed by another chunk with no alignment.

Some chunk types are containers, whose data consists of child chunks.  For instance, the Palette Container chunk can contain a Palette Name chunk, a Palette Entry chunk, and 0+ Material chunks.

You had wondered in one place how the code knew if some data was there or not--it's just a matter of it if that chunk type appears in the chain of chunks or not.  A number of the chunks are optional and/or can be present multiple times.

The chunk types are:

00525643  Main Container (Contains the entire file)
01000000  Version
02000000  Database Name

03000000  Palette Container
01010000  Palette Name
01020000  Palette Entry (0x1B02 of data)
01030000  Material (0x3004 of data)

04000000  Scene Container
04010000  Scene Name
04020000  Object Counter
04030000  Frame Counter

04040000  Object Container
04040100  Object Name

04040200  Geometry Container
04040201  Voxel Object (0x2A Header + Voxel data)
04040202  Voxel Object Size

04040300  Animation Container
04040301  Object Flag
04040302  Object Location
04040303  Object Matrix

04040400  Link Tree Container
04040401  Link Flags
04040402  Object Parent
04040403  Object Pivot

04050000  Aux Object Container
04050001  Aux Object ID
04050002  Aux Object (0x2A Header + Aux data)

So, for example, the beginning of the file is interpreted:

00000000  00525643 = Main Container Chunk
00000004  00001A76 = Main Container Chunk Size = Size of file
00000008  01000000 = Version Chunk
0000000C  0000000C = Version Chunk Size 
00000010  41200000 = Version
00000014  02000000 = Database name
00000018  00000015 = Database name Size
0000001C  "ACAC-WEP.max"
00000029  03000000 = Palette Container
0000002D 00004B2D = Palette Container Size
00000031  01010000 = Palette Name
...
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 03, 2015, 11:49:17 PM
Thats pretty cool.

I'm glad someone who knows how to read assembly is working on this.    :D

If you figure the normals and where the heck the rest of the color palette is at, I'll pause my JAC project and work on an updated mod tool.

I was thinking of writing code that would try to calculate the normals by grouping voxels by their slope (11 bit key) and then calculating the slope from those groupings.  But you finding the lookup values would be allot more percise. (and there wouldn't be any missing values)  I don't envy you trying to figure out 3d code in assembly.  :(
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 04, 2015, 12:10:58 AM
What were you missing from the palette specifically, so I can be on the look out for it?

The files I'm looking at (sp-marined.cvr and vr03.cvr) have 0A and F5 at the start of the palette info, which I take to mean the valid palette indexes run from 0A to F5.  It's followed by 256 3 byte runs of which the first 10 and last 10 are all 0's, those being the ones outside the 0A to F5 range.  Were you seeing color indexes in the voxel data outside that range?

That is followed by 24 tables of 256 bytes I assume have something to do with palette stepping--they don't have a separate chunk type id and are included in the palette entry chunk's size.  I haven't looked into exactly how they work yet.

The material chunks are also in the Palette container and have 24 tables of 256 words it looks like--how the material actually works I haven't looked at yet.
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 04, 2015, 02:09:24 AM
I've been staring at numbers too long.

2^11 = 2048 normal indexes * 3 bytes per normal = 6144 bytes for the normal table

There are 6144 bytes left in the palette entry chunk after the last actual color entry.  But, they're in 256 * 24 blocks with a run of 20 0's at the end of each block, exactly the number of unused colors indexes in the palette.  So I know what has to be palette stepping of some sort rather than the normal tables.  But still, that coincidence of both being 6144 bytes is taunting me....
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 04, 2015, 03:52:18 AM
Color pallete.  What I did to get it to work mostly is weird...   :)
Code: [Select]
0A F5       ????? A = 10.  F5 = 245   10+245 = 255 
                For reading in the values, I had to move 10 forward before starting to count.
                245 for 245 values used?
    zz F5 00 00 00 FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC....
    zz F5 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 AC 80 50 A0 7C .... Yeah....
    Ok.  I've learned this:
        zz*3 (usually 1*3 or 10*3) tells how far to right to being start counting up from 0 for color numbers.
        F5 (245) is the length of this sections data. 
        so values greater than 245 - zz. are fetched by some other method. 
        Colors are stored as R G B values.  1 byte each.
Basically, there are allot of 00 00 00 or other junk data at the start of the pallete that you skip using the first number and the second number tells you how many colors there are that you can use.   The problem is, there are color codes greater than 245 that are being used.  If I try to go off the end and keep in reading values past the 245 rgb values, I get values that are incorrect.  (For example, the terraformer isn't yellow if I try that)

I had no idea about a materials system being in the file.  Maybe its getting all the color values for those items from that instead.  ???

EDIT: Fixed the stupid mistake I made with the numbers.  :P
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 04, 2015, 10:07:57 AM
Ok, here's how I think it works.  (BTW, I think you meant F5 = 245, 10+245=255, etc.)

0A F5
The 0A is the first palette index actually used, and the F5 is the last palette index actually used.

The physical palette itself starts immediately after the 0A F5, not 3*10 bytes later, and is always exactly 256*3 bytes long.  It's just the first 10 entries aren't used, so they're 0's.  But you still need to count them--so the first actual color @3*10 is palette index 10, not palette index 0.  By treating it as palette index 0, you're offsetting your entire palette by 10, and that's why values like F5 appear to overrun your palette.

So in terms of your comments--
zz tells the first valid color index, not how far to the right to start counting--you always start counting right after the F5.

Separate from that, I think I've figure out how the palette stepping for shading works.

Immediately after the palette entries themselves, there are 24 sets of 256 bytes.  Consider them as an array Shades[24][256] of byte.  The first index is the light level, with 0 being darkest, 23 being lightest, and 11 being neutral lighting.  The 2nd index is the palette index of the color your shading minus zz from your comments above (usually 10, occasionally 1).  The byte pointed to by those indexes is the palette index for that color at that level of lighting.

So take sp-marined.cvr for example.  It has zz = 0x0A = 10.  Almost all the voxels in it have palette index 0x46=70.  So to find that color's neutral shaded color, Shades[11][70-10=60], which has the value 0x45=69.  So voxels with palette index 0x46 actually use palette index 0x45 at neutral lighting.  At Shades[12][60] we have 0x46, so it shows it's true color at just slightly above neutral lighting.
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 04, 2015, 01:59:44 PM
Opps.  I meant 246.   ::)
The problem is that 255 - 10 = 245.  And then they go ahead and use 247-255 as a reference number on some of the voxels.  ???

So if i'm getting it right, the large block of data after the color palette is pre-computed shading data.  Telling it what colors to use when a surface is bright or dark. 
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 04, 2015, 02:20:58 PM
Quote from: Ford_Prefect on January 04, 2015, 01:59:44 PM
Opps.  I meant 246.   ::)
The problem is that 255 - 10 = 245.  And then they go ahead and use 247-255 as a reference number on some of the voxels.  ???

You're missing what I'm saying--255 is a valid palette index, and all the palettes have 256 entries--even RGB(0,0,0) still counts as an entry, even though it's never used.

The 10 means for that file, none of the palette indexes in the voxels should be less than 10 (except 0, which means translucent).  It doesn't mean to offset the palette by 10.

In CVR.py, change:
Code: [Select]
pos=pos + 1 + CVRfile[pos-1]*3  #10 was the right number for ACP which had 145 for its number. 155-145 = 10
# Color is wrong on select for the knob parts... as well as the drill arm
# It isn't getting the right colors when it is # 145 or higher
inttmp = pos + 3*(245-CVRfile[pos-1]-1)  #it won't be reading in the right values for 155-x... yeah.
colorcount = 0;
#TODO: I think this might be where it is adding one color to many sometimes.... I'm gonna have to pin it down sometime.
while pos <= inttmp:to:
Code: [Select]
pos=pos + 1
inttmp = pos + 3*255
colorcount = 0;
while pos <= inttmp:assuming pos is starting 1 byte past the 0x0A (at the 0xF5).  Unless you using the shading information I found, you can completely ignore both the 0x0A and the 0xF5.

On your other question--yeah, the 24*256 bytes right after the palette is definitely a shading array.  And I've found in the code now where the 11 bit normal index is used to look up a shading value between 0 and 23--which fits with the 24 shading tables.  Now I just need to work out, the normal index is added to an offset computed from the viewing and light angles--once I work out exactly how that offset is calculated I should be able to translate the normal indexes into actual normal values.
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 04, 2015, 02:59:45 PM
I tried your suggested code change.

Vwntu.cvr now is yellow.
Acolonypod.cvr has incorrect colors.
Vw12 cvr has incorrect colors
etc.

A side note:  There are at least 2 different version numbers for the cvr files in SMACX.  Didn't know if you noticed that.  There could be a differences between files because of that.
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 04, 2015, 06:52:47 PM
So, this has all the colors correct by looking them up in the shading tables in all the files I've tested, including the ones you listed above:

Code: [Select]
# Palette colors
pos += 16
logging.debug("palette num" + str(CVRfile[pos-1]))
pos += 1

# Physical Palette
inttmp = pos + 3 * 255
colorcount = 0;
while pos <= inttmp:
self.physical_palette[colorcount] = str(CVRfile[pos]) + "," + str(CVRfile[pos+1]) +"," + str(CVRfile[pos+2])
colorcount+=1
pos+=3

# Shader Palette
# 12 below is the shade level: 0 = dark 23 = light
pos += 12 * 256
inttmp = pos + 255
colorcount = 0;
while pos <= inttmp:
self.dict_colors[colorcount] = self.physical_palette[CVRfile[pos]]
colorcount += 1
pos += 1

# Body Parts 3D data
...

And added self.physical_palette = {} to __init__()

The only issue is I'm missing some ambient light information that may be encoded in the normal table, or in a header somewhere, so for some of the files all the colors look dim.  That could be offset a bit by adding a slider to adjust the hard-coded 12 shading level above.
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 06, 2015, 01:53:27 AM
If this isn't the computation to retrieve the normals, it's close to it.  It's actually quite clever, using exactly 11 bits to store the angles between Z and the X-axis and Y and the X-axis.  Oddly, they seem have normalized to Y > 0 rather than the usual Z > 0, so Y is always positive--they may have internally used a system with the Z and Y axes.
Code: [Select]
void DecodeNormal(DWORD n, double* x, double* y, double* z)
{
    // A1(0-23)*89+A2(0-88)
    // Normally the highest number would be 23 * 89 + 88 = 2136,
    // but because in the case of A1=0, we know Y=1.0, so X=Z=0.0, so A2 = 0,
    // we compressed the range 0-88 down to 0
    // In the case of A1=0, sin(A1)=0, so x & y will be 0 for any value
    // from 0 to 88 below
    n = n + 88;

    // extract the original angles
    // Angle1 = Z to X-axis
    // Angle2 = Y to X-axis
    DWORD dwAngle1 = 88 - n % 89;
    DWORD dwAngle2 = n / 89;

    // convert the angles to radians
    double dAngle1 = dwAngle1 / 88.0 * 2.0 * PI;
    double dAngle2 = dwAngle2 / 23.0 * PI / 2.0;

    // convert the angles to coords in the range [0, -1]
    // Using standard 3d axes, for cavier axes reorder to x, y, z below
    *z = sin(dAngle1) * sin(dAngle2);
    *x = cos(dAngle1) * sin(dAngle2);
    *y = cos(dAngle2);
}

I've attached a program I'm using to test.  At the command line, cvrtest <direction> <normal>, where <direction> is the 1st byte of the voxel and <normal> is the 2nd byte.  The output is x y z.

The program itself doesn't actually decode the normals, it just looks them up in a table, but from what it was doing to the light vector as an offset into that table, the above is how I think they would be decoded, and seems to be working in testing.  I modified your python to show direction and normal instead of painting to input to cvrtest--I'm not a python coder, I'd have to look up how to do the trig with it.
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 06, 2015, 03:34:36 AM
Honestly, I'm not much of a python programmer as well.   Writing that program sucked.  I'm seriously thinking of remaking it in another language.  (And since you are figuring out the normals, it could be done in 3d  :D ) 

Anyways...
Keep up the great work!  ;rockon
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 06, 2015, 12:33:06 PM
I edited the code above to remove a line that did nothing and add a comment about the axes.

Now that I think about it, of course they're using non-standard 3d axes.  They're using Y for left-right, and Z for up-down, and X for front-back, which is why you found their coordinates in front-back (normal Z, their X), left-right (normal X, their Y), top-down (normal Y, their Z) order in the file and wrote it as ZZ XX YY--converting it to standard 3d axes in your interpretation just as I've down in the code above.

That's going to be confusing working with their code, since everything will be in ZXY order instead of XYZ, including their matrices.

It's giving plausible normals for everything I've sight-checked--the next step will be to build a 3d renderer using them and comparing that to cvrplay to make sure they match.

It looks like their renderer makes a pass through the voxels, for each one each one writing to the Z buffer a 16-bit depth, a 1-byte lighting value from 0-23 (looked up from the tables based on viewXlight(converted to the same format as the normals in the file) + normal), and a 1 byte palette code (if the depth of the current voxel < depth value already in the Z buffer, of course).  Then possibly another pass comes through and adds to all the lighting values an ambient light value from 0-23, with the total lighting value clamped at 23.  So only specular and ambient light, no diffuse.  Then finally it reads through the Z-buffer and uses the lighting value * 0x100 + palette code to look up the physical palette code in the shading tables stored in the file, and reads the RGB values from the physical palette table using the physical palette code.

That's why you had apparently missing palette colors, btw--the palette codes in the Voxels are never directly used as indexes into the physical palette--only into the shading tables, which themselves contain the indexes into the physical palette and never use the missing colors.

I still need to interpret the material block I saw in sp-marined.cvr to see how that plays into it all--most of the files I've looked at don't have a material block, but for those that do we should know what it does.
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 06, 2015, 12:54:20 PM
Could you post/attach the normal lookuptable? 
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 06, 2015, 01:09:50 PM
You meaning the lighting table the normals are used as in index into?  I had expected to find an actual normal look-up table (i.e., the indexes would give you an X, Y, Z normal), but instead there's just the table that gives the lighting value using the normal + viewXlightsource as an index.

I'll try to post it tonight after work.

Ultimately I'd want to replace that table look up with actual lighting calculations based on the extracted X, Y, Z normal from my code fragment above, in order to get shading levels finer than the current 0-23 system, to go along with a 32-bit color conversion also replacing the 8-bit palette indexes.
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 06, 2015, 02:25:42 PM
Quote
You meaning the lighting table the normals are used as in index into?
No...

What I meant to ask is how is cvrtest coming up with x,y,z ... and then I realized that you already had (I'm feeling rather stupid right now. :-\ ).   
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 06, 2015, 05:54:47 PM
Oh, yeah, it's just using that function I posted, where the "n" parameter being passed in = (( D & 7 ) << 8 ) | N if you label the 3 bytes of each voxel as D, N, P, and with "PI" just being a #define I made for the value of pi.

If you want to do the opposite and *encode* a normal--i.e., convert a standard X, Y, Z normal into their format to store in a .cvr file to modify the existing normals, make a new mesh, or whatever, it should be:

Code: [Select]
#define PI 3.141592653589793238463

DWORD EncodeNormal(double x, double y, double z)
{
if (y < 0.0)
{
x = -x;
y = -y;
z = -z;
}

double dAngle1 = atan2(z, x);
if (dAngle1 < 0.0)
dAngle1 += 2 * PI;

DWORD dwAngle1 = (DWORD)round(dAngle1 / (2.0 * PI) * 89.0) % 89;
dwAngle1 = 88 - dwAngle1;

double dAngle2 = acos(y);

DWORD dwAngle2 = (DWORD)round(dAngle2 / (PI / 2.0) * 23.0);

DWORD dwRet = dwAngle2 * 89 + dwAngle1;

// This works because in the case of y = 1.0, x & z will be 0,
// so 23 * 89 is the highest number we need
dwRet = (dwRet >= 89) ? dwRet - 88 : 0;

return dwRet;
}

Which is what I saw the code basically doing to the lighting vector except for the part about subtracting 88 at the end since internally it doesn't need to fit it into 11 bits, and from which I derived DecodeNormal() as it's inverse.
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 09, 2015, 04:07:34 AM
Ford--

I ran any analysis of all the tags in all the .cvr files to get an idea of the tree structure and see what tags we actually need to worry about.  It's attached here.  It looks like we can ignore the following tags because they aren't actually used in any of the SMACX .cvr files:

   CVRCHUNK_VOXEL_OBJECT_SIZE      = 0x04040202,
   CVRCHUNK_LINK_TREE_CONTAINER   = 0x04040400,
   CVRCHUNK_LINK_FLAGS         = 0x04040401,
   CVRCHUNK_OBJECT_PARENT         = 0x04040402,
   CVRCHUNK_OBJECT_PIVOT         = 0x04040403,
   CVRCHUNK_AUX_OBJECT_CONTAINER   = 0x04050000,
   CVRCHUNK_AUX_OBJECT_ID         = 0x04050001,
   CVRCHUNK_AUX_OBJECT         = 0x04050002

I believe I've figured out the material chunk, which was the last tag we hadn't deciphered.  It appears to be a 16-bit color (555) version of the shading tables to complement the 8-bit paletted version I identified earlier, and consists of a 4 byte integer followed by 24*256 16 bit colors.  It gives a bit finer shading than the 8-bit table, which you can see in CVRPlay.exe by switching the "Mode" between "High Color" and "Index Color".  It's not a huge difference, but it does look slightly better, and would be a superior input to a Gaussian shader.  The CVRPlay code can handle 555 or 565 16-bit color, and there's probably a bit somewhere in the file saying which to use, but all the ones I sight-checked the color value tops out at 0x7FFF, pointing to 555 since the highest bit is always 0.
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 09, 2015, 04:25:54 AM
Quote
It's attached here. 
  Ummm.   Where?

Great to know that there is a higher quality color source I can use.  That will come in real handy when I make a cvr viewer for JAC.  Sounds like the only information needed for exporting is the positioning data for voxel meshes.  (See the terraformer model, Vwntu.cvr, which has the arm, drill, body as separate pieces)
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 09, 2015, 04:37:37 AM
D'oh, it's attached now.

The general file format seems to go:

VW14.cvr
CVRCHUNK_FILE_CONTAINER 00000000 0003dce8
    CVRCHUNK_VERSION 00000008 0000000c
    CVRCHUNK_DB_NAME "SMACX_FINAL.max"  00000014 00000017
    CVRCHUNK_PALETTE_CONTAINER 0000002B 00004b2d
        CVRCHUNK_PALETTE_NAME "cavcols"  00000033 0000000f
        CVRCHUNK_PALETTE_DATA 00000042 00001b0a
        CVRCHUNK_MATERIAL 00001B4C 0000300c
    CVRCHUNK_SCENE_CONTAINER 00004B58 00039190
        CVRCHUNK_SCENE_NAME "SMACX_FINAL.max"  00004B60 00000017
        CVRCHUNK_OBJECT_COUNTER (40)  00004B77 0000000c
        CVRCHUNK_FRAME_COUNTER (15)  00004B83 0000000c
        CVRCHUNK_OBJECT_CONTAINER 00004B8F 00002fd3
            CVRCHUNK_OBJECT_NAME "Barrel Brace 01"  00004B97 00000017
            CVRCHUNK_GEOMETRY_CONTAINER 00004BAE 00002cb5
                CVRCHUNK_VOXEL_OBJECT 00004BB6 00002cad
            CVRCHUNK_ANIMATION_CONTAINER 00007863 000002ff
                CVRCHUNK_OBJECT_FLAG 0000786B 00000017
                CVRCHUNK_OBJECT_LOCATION 00007882 000000bc
                CVRCHUNK_OBJECT_MATRIX 0000793E 00000224
        CVRCHUNK_OBJECT_CONTAINER 00007B62 00000cad
            CVRCHUNK_OBJECT_NAME "Barrel02"  00007B6A 00000010
            CVRCHUNK_GEOMETRY_CONTAINER 00007B7A 00000996
                CVRCHUNK_VOXEL_OBJECT 00007B82 0000098e
            CVRCHUNK_ANIMATION_CONTAINER 00008510 000002ff
                CVRCHUNK_OBJECT_FLAG 00008518 00000017
                CVRCHUNK_OBJECT_LOCATION 0000852F 000000bc
                CVRCHUNK_OBJECT_MATRIX 000085EB 00000224
        CVRCHUNK_OBJECT_CONTAINER 0000880F 00000b09
            CVRCHUNK_OBJECT_NAME "Barrel03"  00008817 00000010
            CVRCHUNK_GEOMETRY_CONTAINER 00008827 000007f2
                CVRCHUNK_VOXEL_OBJECT 0000882F 000007ea
            CVRCHUNK_ANIMATION_CONTAINER 00009019 000002ff
                CVRCHUNK_OBJECT_FLAG 00009021 00000017
                CVRCHUNK_OBJECT_LOCATION 00009038 000000bc
                CVRCHUNK_OBJECT_MATRIX 000090F4 00000224
...

FLAG, LOCATION, and MATRIX are included for each mesh separately, and are arrays of length CVRCHUNK_FRAME_COUNTER.  So if you take the first location of each array and apply the first matrix of each array, that should align the meshes properly.  I don't know what all is in the flags--it may be as simple as a 1 to show or hide the mesh in each frame.

I'm working on a full file parser in C++ I'll post when it's done sometime this weekend.  You'd mentioned maybe re-rewriting your editor in a language other than python--were you thinking C++, Java, or something else?
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 09, 2015, 12:34:34 PM
I was thinking doing it in Java since JAC is in Java (I'll be able to reuse the code).   In fact, I already created a voxel viewer in Java for reading the export files CVRColorizer produces (I didn't release it because it couldn't show the normals). So that means I  just need to write a cvr file reader and a gui.

Title: Re: First CVR mod!
Post by: Ford_Prefect on January 09, 2015, 01:24:47 PM
Ack.  I just realized the toolkit i used wont work in a canvas element. (Can't use swing).  So I might need to redo the renderer as well.  Oh well.  I'll save analysis of the different toolkits for a latter date.
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 11, 2015, 12:54:06 PM
Ford--I've attached a write up of the .cvr file format, just for our use.  Once we get everything confirmed and nailed down, you can update wiki entry, using my document or your own write up as you wish--this is just a WIP while we nail down the last parts of the file.

New since my last report are:

First 4 bytes of Voxel header is a flag--
flag & 2 means a jump header follows and each voxel group will have another jump header, otherwise there is no jump header (answering a question you had asked in your cvr file write up);
flag & 8 does something in the code, but I haven't figured out what exactly yet;
flag & 1 is true in only 1 file, VLIGHTS.CVR, and it means instead of each jump group having a jump header, each group has a voxel header without a jump header.  I.e., the 32 bytes at the start of the voxel data repeat inside the voxel data the way the jump headers do in other files.

The last 4 bytes of the jump header is actually a flag.  It may be 0 in all files, but if it is ever 1, it means an extra 4-byte integer will follow the header giving the byte offset of the next jump header.

Offset 8 in the voxel header is a 4 byte float giving the "units per 3d pixel".

The CVRCHUNK_OBJECT_LOCATION array in the animation area is an array[FrameCount] of VECTOR3's, VECTOR3 being defined as 4-byte float Z, 4-byte float X, 4-byte float Y.

The CVRCHUNK_OBJECT_MATRIX array in the animation area is an array[FrameCount] of 3x3 Matrices of 4-byte float, in order to multiple against [Z,X,Y] VECTOR3's.

Aligning multiple objects (meshes) should be a matter of placing their origin at CVRCHUNK_OBJECT_LOCATION[0] * CVRCHUNK_OBJECT_MATRIX[0].

I'm not happy with definition of the voxel jump headers.  They contain 4 SHORTVECTOR3's (call them v1, v2, v3, and v4), and v1 + v4 does seem to give the start position as you use in CVRColorizer.  But that doesn't explain why they are separate vectors, or what 2 and 3 are for.  I'm looking at some code that uses all 4, that seems to be determining the boundaries of each voxel group, but I haven't reverse compiled it well enough to really understand what it's doing yet.  Oddly, the first thing it does is compute 2 vectors A and B as A = v1 + v4 and B as v2 - v4, and it passes (B, v3) to the same routine it passes from the voxel header itself (Position, Scale).  But v3 does not seem to be a scale, and v2-v4 doesn't seem to be a meaningful position.  If it really is just computing boundaries, it's replaceable with the method you used of actually rendering the voxels to find the boundaries, and changing v2 and v3 seems to have no effect on cvrplay.exe, but it bugs me that I don't understand what they are.

Let me know you have any thoughts on what v2 and v3 might be.

Title: Re: First CVR mod!
Post by: Ford_Prefect on January 11, 2015, 03:32:20 PM
I'll find a stopping spot in my current work on JAC and start working on CVR editor 2.

Quote
Let me know you have any thoughts on what v2 and v3 might be.
  No clue.  If I come up with an idea, I'll tell you.

Title: Re: First CVR mod!
Post by: PlotinusRedux on January 17, 2015, 08:25:10 AM
How committed to Java are you, Ford?  I just have a fundamental dislike for interpreted languages and things that happen "behind the coder's back" like Java's garbage collection, and I've never seen a really smooth Java game under Windows.  C++ using OpenGL can easily be coded to cross-compile for Linux if that's your concern.
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 17, 2015, 06:23:14 PM
Quote
How committed to Java are you, Ford?

It's mostly because I'm used to it and I thought I would be doing most of the coding. 

Quote
I've never seen a really smooth Java game under Windows

I have.  Blocks that matter, Far Sky, Minecraft.


A side note:  I'm pretty sure it would be legally questionable in several countries if you (and the other exe decompilers) were to contribute code.  I'm wanting the work I'm doing to be completely legal in case EA ever notices it.  Which means following http://en.wikipedia.org/wiki/Clean_room_design. (http://en.wikipedia.org/wiki/Clean_room_design.)
Title: Re: First CVR mod!
Post by: PlotinusRedux on January 24, 2015, 09:36:18 AM
Interesting, I hadn't heard of that clean room thing before.

As long as any mod/rewrite is (1) free and (2) requires SMACX to be installed, I'd be very surprised if Firaxis would have an issue with it--they've always encouraged modding, and if you still have to buy their game, no financial harm is done to them.

For a rewrite to be completely free of any liability to Firaxis (or whoever owns the copyright today), though, I think you'd have to not use their artwork.  Game rules are explicitly not copyright-able, but images definitely are--even more so than code.
Title: Re: First CVR mod!
Post by: Ford_Prefect on January 24, 2015, 04:45:35 PM
Yeah.  I'm not worried about Frixas, but EA owns the game and they are not known for being nice.  So I'm doing it by the book.  Its why writing JAC takes so much effort is because instead of just programing it to be the same, I'm making it read all the config files and dialog, etc from SMAC.

Also, I've almost got a cvr viewer ready.  I think its reading in all the data correctly, I just need to hook it up to a renderer and see if it actually produces the models correctly. 

Templates: 1: Printpage (default).
Sub templates: 4: init, print_above, main, print_below.
Language files: 4: index+Modifications.english (default), TopicRating/.english (default), PortaMx/PortaMx.english (default), OharaYTEmbed.english (default).
Style sheets: 0: .
Files included: 31 - 840KB. (show)
Queries used: 14.

[Show Queries]