Fun with Caviar Palettes

[rmolina]

Hi. I'm new here.

I've been doing some CVR palette manipulation experiments that I would like to share.  I know there is a tool by Ford Prefect that allows some model "repainting".  However, I would like to describe another way to perform palette transformations.

Summary: Caviar supports an 8-bit "Index Color" palette and a 15-bit "High Color" palette.  If a high color palette is not present, it can be estimated from the index color palette.  If the high-color palette is present, it is used.  Different approaches are possible to derive a high-color palette.  That's what i have been trying...

However, let's start with a quick review...

Index Color Table

The "index color" palette supports up to 256 colors and is stored as an array of 256x3 uint8 values.   This is the table called "Physical Palette" in PlotinusRedux's CVR Specification and is located at the PALETTE_DATA chunk. Looks like only 236 colors are used in the game.

Here is a plot of the "index color" palette, extracted from AX.CVR (Note: Only 236 valid colors are shown.  Also note: color does not change in the vertical axis):


And here is the model rendition using this color mode:



High Color Table

The "high color" palette supports up to 6144 colors and is stored as an array of 24x256 uint16 values.   This is the table called "ShadeArray" in PlotinusRedux's CVR Specification and is located at the MATERIAL chunk. Looks like only 24x236 colors are used in the game.

Note: It looks like the MATERIAL chunk description in PlotinusRedux's CVR Specification needs to be corrected.
    offset=0000: uint16 Number of valid colors in the color index palette
    offset=0002: uint16 Unknown
    offset=0004: uint16 High color palette, array[24][256] of 16 bit R5G5B5 color values

Here is the high color palette extracted from AX.CVR:


And here is the model rendition using this color mode:



High Color Table generated from the Index Color Table

The High Color Table (Material table) is not mandatory.  If the chunk is not present in the file, a High Color Table is generated from the Index Color Table.  There is an easy way to study these "generated" high color tables: just remove the MATERIAL chunk using a hex-editor and update the sizes in the   FILE_CONTAINER and PALETTE_CONTAINER chunks.  I did it on AX.CVR and attached the files: AX_ORIGINAL.CVR and AX_REMOVE.CVR.

Here is the model rendition using the "generated" high color table on the AX_REMOVE.CVR.  It looks kinda blueish:



I tried to reproduce this "high color from index color" generation, and the first thing I tried was the process described here for the shades generation and  the process described here in order to repack the colors.  I got this new palette:



When we look at the model rendition, we found it is not "blueish" as the one generated by the Caviar Player, and the colors are closer to "index color" palette, but the model is way too dark:



I attached the file as AX_RECOLOR.CVR.


Finally, I reverse engineered this "high color from index color" generation, and found it uses the last row of the "Shade" array from the PALETTE_DATA chunk.  Please note: This  "Shade" array (from the PALETTE_DATA) is not the same  "ShadeArray"from the MATERIAL chunk that we mentioned before.
This agrees with how PlotinusRedux describes the 24 shades as "light levels".  Here is how the generated palette looks like:



I reimplemented the process and created another model with a recreated palette that emulates the one generated on-the-fly (attached as AX_REPLACE.CVR).

Since we can use different rows of the "Shade" array, and different "fractions" for the shading process described here it is really easy to generate a set of new high color palettes using scripting.

Here is just an example of further manipulation using an inverted palette (I attached the file AX_PINK.CVR):


That's all for now.

[Geo]

Hi, and welcome.

Let me get this straight, you managed to recolor the alien scout by changing the color palette? And doing so didn't affect the rest of the caviar files used ingame?

In any case, the result is already amazing.

Are these recolored models available for use in a mod? I certainly would like to do so.

[rmolina]

Hi there.  Thanks for your comments.

Let me get this straight, you managed to recolor the alien scout by changing the color palette?

Yes.  What I'm trying to do is to write some code for the generation of new color palettes that fit the models well...  For example, here is a rendition of the model using a "wrong" palette, that does not fit the model (actually, I kinda like it for an alien.   Looks like it's "cloaked"):



I believe the trick to generate nice palettes is to preserve the "distribution" of the colors on the palette.  Let's look at the high color palette again:



There are some shades of brown, some shades of green, and so on... after a while, there are some blues that represent about 1/5 of the palette... I haven't counted carefully, but there might be about 25 "base colors", and each one of them has a number of shades.  That's the "distribution" that I'm intending to preserve: how many shades of the first color?, how many of the second?...

Now, let's look again at the high color palette:



This is just an extension of the index color palette: for each color in the index palette we have 24 shades of that color (on the vertical).  This is the part that I already implemented.  Justo to be clear: I already know how to recreate the vertical distribution, and I'm currently working on the horizontal distribution.

Since I haven't implemente the horizontal distribution yet, I am currently performing some numerical transformations on the index palette to create new index palettes, and then I am generating new high color palettes dynamically.  But what I want is to implement some code that will accept a user defined "base color palette" (those 25 or so colors that I haven't counted yet) and generate both the index color palette (236 colors) and the high color palette (24x236 colors)...

And doing so didn't affect the rest of the caviar files used ingame?

As each caviar file stores its own palette, modifying one file should not affect the others. However, I haven't performed any testing ingame.  I have been working only on the Caviar Player.

Are these recolored models available for use in a mod? I certainly would like to do so.

Please Feel free to use them if they work for you.  I did I quick test and didn't worked on my SMAX installation.   I replaced AX.CVR with AX_PINK.CVR an launched the game.  The model was rendition was green instead of pink.  Since I didn't modify the index color palette on the file (I only modified the high color palette) this would imply that my installation has been using the (inferior) index color mode.  Will need some extra testing.  Please let me know if the color variations are shown in your own installation.   In any case, will update my code to replace both: the index color and high color palettes, instead of replacing just the high color palette.

[Geo]

Please Feel free to use them if they work for you.  I did I quick test and didn't worked on my SMAX installation.   I replaced AX.CVR with AX_PINK.CVR an launched the game.  The model was rendition was green instead of pink.  Since I didn't modify the index color palette on the file (I only modified the high color palette) this would imply that my installation has been using the (inferior) index color mode.  Will need some extra testing.  Please let me know if the color variations are shown in your own installation.   In any case, will update my code to replace both: the index color and high color palettes, instead of replacing just the high color palette.

Well, a quick test gave the same result as your test. No pink aliens to be seen ingame.

[rmolina]

i have no idea what's going on...

I first thought the game was using index color instead of high color mode.  So, I modified the index color palette on my pink alien (but leave the high color palette unchanged).  For the index color I used a different tranformation.  This time I did a green/red band swapping and I got a purple alien.  This is how the model rendition looks like on index color mode:



And of course, on high color mode it still looks pink...



So. I was thinking: purple aliens imply the game is using index color and pink aliens imply high color... I replaced the file, launched the game, and got:



The same old green aliens!  Where are those greens coming from? I have no idea.

Just to be sure, I tried loading my file on CVR Colorizer.  It parses the file properly and let me explore the parts.  Looks like it only supports the index color palette (i.e. it sees purple aliens)



I'm attaching the new AX_PURPLE_PINK.CVR file.

[rmolina]

This small post isn't about the Caviar palettes but I wanted to share this anyway...

I've been looking at the rendition process of Caviar files and I was able to create an animation of the AX.CVR model, frame by frame and part by part.  Normally, you won't be able to see this process because the screen is updated only at the frame level, not the part level.

The AX.CVR model includes 15 animation frames and 12 model parts (torso, arm_l, hand_l, arm_r, hand_r, head, thigh_l, calf_l, foot_l, thigh_r, calf_r, and foot_r).  That makes 180 frames in total for the part-by-part full animation:



GIFV link for mobile users.