Difference between revisions of "Caviar"

Latest revision as of 10:29, 6 December 2020

Caviar is a voxel based 3D model format with animation capabilities developed by the now-defunct AnimaTek International, since purchased by Digital Element Inc.

The .cvr files in Alpha Centauri contain Caviar models that are combined to produce graphics for most units in the game.

Ford Prefect partially deciphered the file format and wrote a new and open source Caviar viewer and paint tool, CVR-Colorizer.

A proprietary Caviar player, CvrPlay.exe, is also available. CvrPlay.exe supports animations and allows rotation of the models, unlike Ford's Colorizer. It cannot edit .cvr files, however.

Overview

Ford identified 3 major sections in the caviar file format.

File start - this section includes the model name and some of the palette codes.
Unknown - Currently unknown what this section of data is for.
3D data - Contains the 3D voxel information. Position, Color and the normal vector.

CVR Specification

PlotinusRedux's work on reverse engineering it:

The file is a set of variable-length chunks with some chunks nested inside each other. Each chunk begins with the header:

0000    uint32       Tag
0004    uint32       Size including this header
0008    <data>
Size    <next chunk>

Some chunks contain only other chunks (the ones with "Container" in their name), while others contain actual data.

The tag values are:

typedef enum CVRCHUNK_E {
    CVRCHUNK_FILE_CONTAINER         = 0x00525643,
    CVRCHUNK_VERSION                = 0x01000000,
    CVRCHUNK_DB_NAME                = 0x02000000,

    CVRCHUNK_PALETTE_CONTAINER      = 0x03000000,
    CVRCHUNK_PALETTE_NAME           = 0x01010000,
    CVRCHUNK_PALETTE_DATA           = 0x01020000,
    CVRCHUNK_MATERIAL               = 0x01030000,

    CVRCHUNK_SCENE_CONTAINER        = 0x04000000,
    CVRCHUNK_SCENE_NAME             = 0x04010000,
    CVRCHUNK_OBJECT_COUNTER         = 0x04020000,
    CVRCHUNK_FRAME_COUNTER          = 0x04030000,

    CVRCHUNK_OBJECT_CONTAINER       = 0x04040000,
    CVRCHUNK_OBJECT_NAME            = 0x04040100,

    CVRCHUNK_GEOMETRY_CONTAINER     = 0x04040200,
    CVRCHUNK_VOXEL_OBJECT           = 0x04040201,

    CVRCHUNK_ANIMATION_CONTAINER    = 0x04040300,
    CVRCHUNK_OBJECT_FLAG            = 0x04040301,
    CVRCHUNK_OBJECT_LOCATION        = 0x04040302,
    CVRCHUNK_OBJECT_MATRIX          = 0x04040303,
} CVR_CHUNK_E;

Chunks are described below.

CVRCHUNK_FILE_CONTAINER (0x00525643)

Always at the start of the file, and contains all the other chunks in the file.

Child chunks:

   CVRCHUNK_VERSION (1)
   CVRCHUNK_DB_NAME (1)
   CVRCHUNK_PALETTE_CONTAINER (1)
   CVRCHUNK_SCENE_CONTAINER (1)

CVRCHUNK_VERSION (0x01000000)

File version.

   Data:
   0000   float32    Version #

CVRCHUNK_DB_NAME (0x02000000)

Original .max file name

   Data:
   0000   ascii       Name, length given by chunk size.

CVRCHUNK_PALETTE_CONTAINER (0x03000000)

Contains Palette related chunks.

Child chunks:

   CVRCHUNK_PALETTE_NAME (1)
   CVRCHUNK_PALETTE_DATA (1)
   CVRCHUNK_MATERIAL (0-1)

CVRCHUNK_PALETTE_NAME (0x01010000)

Text name for palette

Data:

   0000    ascii       Name, length given by chunk size.

CVRCHUNK_PALETTE_DATA (0x01020000)

8-bit palette color tables

0000    uint8           First color index used.
0001    uint8           Last color index used.
0002    0x100*RGB24     Physical Palette as array[256] of 3-byte RGB values.
0102    0x18*0x100*uint8 Shade Array[24][256] of byte indexes into Physical Palette.
                        <Light Level(0-23)>*256 + <Voxel Color Index> gives index 
                        of color in Physical Palette.

CVRCHUNK_MATERIAL (0x01030000)

16-bit color tables

0000    0x18*0x100*uint16 ShadeArray[24][256] of 16 bit R5G5B5 color values.
                          <Light Level(0-23)>*512 + <Voxel Color Index>*2 gives
                          R5G5B5 color value for voxel.

CVRCHUNK_SCENE_CONTAINER (0x04000000)

Contains voxel and animation chunks.

Child Chunks:

   CVRCHUNK_SCENE_NAME (1)
   CVRCHUNK_OBJECT_COUNTER (0-1)
   CVRCHUNK_FRAME_COUNTER (0-1)
   CVRCHUNK_OBJECT_CONTAINER (0+)

CVRCHUNK_SCENE_NAME (0x04010000)

Text name for scene (usually same as DB name)

Data:
0000    ascii       Name, length given by chunk size.

CVRCHUNK_OBJECT_COUNTER (0x04020000)

Number of objects in this scene

   Data:
   0000    uint32      # of objects in scene

CVRCHUNK_FRAME_COUNTER (0x04030000)

Number of frames in the animation of this scene

   Data:
   0000    uint32      # of frames in scene (1+)

CVRCHUNK_OBJECT_CONTAINER (0x04040000)

Container for chunks related to a single object

Child chunks:

   CVRCHUNK_OBJECT_NAME (1)
   CVRCHUNK_GEOMETRY_CONTAINER (0-1)
   CVRCHUNK_ANIMATION_CONTAINER (1)

CVRCHUNK_OBJECT_NAME (0x04040100)

Text name for object

Data:

   0000    ascii       Name, length given by chunk size.

CVRCHUNK_GEOMETRY_CONTAINER (0x04040200)

Container for voxel information

Child Chunks:

   CVRCHUNK_VOXEL_OBJECT

CVRCHUNK_VOXEL_OBJECT (0x04040201)

Voxel information for object

Data Types:

SVECTOR3(6)
0000    int16       Z
0002    int16       X
0004    int16       Y

VOXEL_HEADER(0x2A)
0000    uint32      Flags.  1 = Next voxel group has its own VOXEL_HEADER
                            2 = Next voxel group has its own JUMP_HEADER
                            4 = Unused
                            8 = Used but unknown what it means
0004    uint32      Size of voxel object from start of first VOXEL_HEADER
0008    float32     Units per 3D pixel
000C    uint32      If Flags & 1: bytes to next VOXEL_HEADER from start of
                        this header
                    Else: same of offset 0004.
0010    uint32      If Flags & 1: bytes to last byte of this voxel group
                    Else: bytes to last byte of voxel object
0014    SVECTOR3    Scale
001A    SVECTOR3    Position
0020    SVECTOR3    Rotation Center
0026    uint32      If Flags & 1: # voxels until next VOXEL_HEADER
                    Else: # voxels in object

JUMP_HEADER(0x20 or 0x24)
0000    SVECTOR3    Position
0006    SVECTOR3    v2--unknown vector, used determining max size
000C    SVECTOR3    v3--unknown vector, used determining max size
0012    SVECTOR3    PositionOffset--start drawing at Position+PositionOffset
0018    uint32      # of voxels in this jump group
001C    uint32      JumpFlags.  1 = Has Offset
<020    uint32      Offset to next JUMP_HEADER.  Present only if
                    JumpFlags & 1>

VOXEL_DATA
0000    uint8       Direction (see addendum)
0004    uint8       Normal (see addendum)
0008    uint8       Color index into shading tables

Data layout depends on Flags.

Flags & 1:
VOXEL_HEADER
array of VOXEL_DATA
VOXEL_HEADER
array of VOXEL_DATA
....

Flags & 2:
VOXEL_HEADER
JUMP_HEADER
array of VOXEL_DATA
JUMP_HEADER
array of VOXEL_DATA
...

!(Flags & 3):
VOXEL_HEADER
array of VOXEL_DATA

CVRCHUNK_ANIMATION_CONTAINER (0x04040300)

Container for animation data.

Child chunks:

   CVRCHUNK_OBJECT_FLAG (0-1)
   CVRCHUNK_OBJECT_LOCATION (0-1)
   CVRCHUNK_OBJECT_MATRIX (0-1)

CVRCHUNK_OBJECT_FLAG (0x04040301)

Array[FrameCount] of flags

0000    uint8      ObjectFlag
0004... repeated FrameCount times

CVRCHUNK_OBJECT_LOCATION (0x04040302)

Array[FrameCount] of location vectors

0000    float32     Z
0004    float32     X
0008    float32     Y
001C... repeated FrameCount times

CVRCHUNK_OBJECT_MATRIX (0x04040303)

Array[FrameCount] of 3x3 matrixes of float32 in Z,X,Y order

0000    3*3*float32     Matrix
0024... repeated FrameCount times

Appendix

Example file layout

ACOLPOD.cvr

CVRCHUNK_FILE_CONTAINER 00000000 00091120
    CVRCHUNK_VERSION 00000008 0000000c
    CVRCHUNK_DB_NAME "MB_Alien_Pod2.max"  00000014 00000019
    CVRCHUNK_PALETTE_CONTAINER 0000002D 00004b2d
        CVRCHUNK_PALETTE_NAME "cavcols"  00000035 0000000f
        CVRCHUNK_PALETTE_DATA 00000044 00001b0a
        CVRCHUNK_MATERIAL 00001B4E 0000300c
    CVRCHUNK_SCENE_CONTAINER 00004B5A 0008c5c6
        CVRCHUNK_SCENE_NAME "MB_Alien_Pod2.max"  00004B62 00000019
        CVRCHUNK_OBJECT_COUNTER (2)  00004B7B 0000000c
        CVRCHUNK_FRAME_COUNTER (15)  00004B87 0000000c
        CVRCHUNK_OBJECT_CONTAINER 00004B93 0008bdb6
            CVRCHUNK_OBJECT_NAME "ACOLPOD"  00004B9B 0000000f
            CVRCHUNK_GEOMETRY_CONTAINER 00004BAA 0008baa0
                CVRCHUNK_VOXEL_OBJECT 00004BB2 0008ba98
            CVRCHUNK_ANIMATION_CONTAINER 0009064A 000002ff
                CVRCHUNK_OBJECT_FLAG 00090652 00000017
                CVRCHUNK_OBJECT_LOCATION 00090669 000000bc
                CVRCHUNK_OBJECT_MATRIX 00090725 00000224
        CVRCHUNK_OBJECT_CONTAINER 00090949 000007d7
            CVRCHUNK_OBJECT_NAME "Lit Objects"  00090951 00000013
            CVRCHUNK_GEOMETRY_CONTAINER 00090964 000004bd
                CVRCHUNK_VOXEL_OBJECT 0009096C 000004b5
            CVRCHUNK_ANIMATION_CONTAINER 00090E21 000002ff
                CVRCHUNK_OBJECT_FLAG 00090E29 00000017
                CVRCHUNK_OBJECT_LOCATION 00090E40 000000bc
                CVRCHUNK_OBJECT_MATRIX 00090EFC 00000224

Voxel Direction Element

The upper 5 bits control the direct of the next voxel.

d = Direction >> 3

    z   x   y
00 -1  -1  -1
01 -1  -1   0
02 -1  -1   1
03 -1   0  -1
04 -1   0   0
05 -1   0   1
06 -1   1  -1
07 -1   1   0
08 -1   1   1

09  0  -1  -1
0A  0  -1   0
0B  0  -1   1
0C  0   0  -1
0D  0   0   1
0E  0   1  -1
0F  0   1   0
10  0   1   1

11  1  -1  -1
12  1  -1   0
13  1  -1   1
14  1   0  -1
15  1   0   0
16  1   0   1
17  1   1  -1
18  1   1   0
19  1   1   1

1A  STOP

Voxel Normal Element

n = ((Direction & 7) << 8) | Normal

void DecodeNormal(DWORD n, double* x, double* y, double* z)
{
	// A1(0-23)*89+A2(0-88)
	// Normally the highest number would be 23 * 89 + 88 = 2136,
	// but because in the case of A1=0, we know Y=1.0, so X=Z=0.0, so A2 = 0,
	// we compressed the range 0-88 down to 0
	// In the case of A1=0, sin(A1)=0, so x & z will be 0 for any value
	// from 0 to 88 below
	n = n + 88;

	// extract the original angles
	// Angle1 = Z to X-axis
	// Angle2 = Y to X-axis
	DWORD dwAngle1 = 88 - n % 89;
	DWORD dwAngle2 = n / 89;

	// convert the angles to radians
	double dAngle1 = dwAngle1 / 88.0 * 2.0 * PI;
	double dAngle2 = dwAngle2 / 23.0 * PI / 2.0;

	// convert the angles to coords in the range [-1, 1]
	*z = sin(dAngle1) * sin(dAngle2);
	*x = cos(dAngle1) * sin(dAngle2);
	*y = cos(dAngle2);
}

FordPrefect's Original Analysis. Contains errors.

Work in progress

Values on the left are the hexadecimal values found in the caviar file format. Description of what they represent on the right.

Data that is tabbed is extra examples and/or values.

File start

43 56 52 00      is the characters "CVR"  

xx xx xx xx <- file size in bytes. Unsigned? 

00 00 00 01 <- Flag saying next is the version number.  
    
Next part        |-------|  Version number for the file is stored here, not sure if it stores anything else as well.
0C 00 00 00 9A 99 21 41 
        9A 99 21 41 - 10.10
        33 33 23 41   version 10.20     
    
00 00 00 02     Constant ID

11 00 00 00     Name length + 8
42 41 53 49 43 2E 4D 41 58  Name of file it was converted from?
    
00 00 00 03     Constant ID

xx xx xx xx  
    The distance in bytes to the 3d description section. Relative to this point!  Points a ways before the name of body part.
    What's interesting this distance seems to be exactly the same for the different palates for non-animated files.  
    Need to check to see if number changes for animated files or not.            
                 
00 00 01 01     Constant ID
0F 00 00 00     = Palette name length + 8
55 6E 6E 61 6D 65 64  Palette name

                       
00 00 02 01     Constant ID
0A 1B 00 00 <- Points to 3d section label 04.  Distance from here to there.

0A F5       ????? A = 10.  F5 = 145   10+145 = 155  
                For reading in the values, I had to move 10 forward before starting to count.
                145 for 145 values used?  Allot of guess work here
    aa F5 00 00 00 FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC....
    aa F5 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 AC 80 50 A0 7C .... Yeah....
    Ok.  I've learned this:
        aa*3 (usually 1*3 or 10*3) tells how far to right to being start counting up from 0 for color numbers.
        F5 (145) is the length of this sections data.  
        so values greater than 145 - aa. are fetched by some other method.  
        Colors are stored as R G B values.  1 byte each.


00 00 02 01 xx xx xx xx < points to 3d section label 04

0A F5 00 00 00 <  Why is this duplicated!? 
    01 F5 < unnamed.... Animated. 5 body parts
    01 F5 < Unnamed... no-anime. 14 body parts
        /\- 245 hmmm  The next two bits = 245*3 in length.  
        Ok, I (think) this is saying how big the palette is in size.

Large Amount of unknown data

After that there is a large section of data that is currently unknown as to what it does.

Lets skip to the 3d data section.

3d Voxel Data Section


00 00 00 04     Constant ID, Earlier Reference points here
11 9D 00 00     ok this is the distance to the end of the file in bytes + 4.  If you measure from the 04.   

00 00 01 04     Constant ID 
14 00 00 00     Name length + 8
21 41 43 41 43 2D 41 56 2E 6D 61 78 00 NAME 

------------If this is an animation only file this will be missing-----
00 02 04 0C     Constant ID 
00 00 00        ????
01 00 00 00     Number of body parts for object


00              ?? Spacer?
----------- End optional section ------------------

00 03 04 0C     Constant ID ?  
00 00 00        ????<br />
01 00 00 00     # of frames  Never set to 00.  Always 1 or more

00 00 04 04     Constant ID 
DD 9C 00 00     Distance    

00 01 04 04     Constant ID 
10 00 00 00     Length of Part Name + 8 
41 41 2D 52 4F 56 45 52    NAME


----------- On select.cvr, this section wasn't here.  select.cvr was 10.10 version. was animated but only had one "part" 
    This section was also missing from AVHT-LightsReposition, an animation only file.
00 02 04 04     Constant ID 
74 9C 00 00     Distance 
-----------
or
--              Not sure if here all the time if 02 04 04 is missing.  
                Just need to write a function to find ID lines.
00              Probably and artifact of their save function when skiping unused data.
                Might explain some of the weird bits that I've found solo laying around.
--

01 02 04 04     Constant ID 
6C 9C 00 00     Distance 

0A 00 00 00         Ok, currently I have no idea what this is.  If i change it, CVR will crash.
    
    

64 9C 00 00 00 00 80 40 64 9C 00 00 63 9C 00 00 
     most of this has to deal with the number of units per 3d pixel  
     I have no clue how it actually works.  
                      
7F 00 B3 00 2D 00     ZZ sz xx sx yy sy   
        zz,xx,yy didn't change object position.
        sz, sx, sy when i changed them reduced the size and offset the item in the view
        zz, xx, yy I think this tells the viewer the dimensions of the item on screen so that it knows when to render

C1 FF A7 FF EA FF  <- Positional Data. I believe it is the starting point for rendering voxels.  Not sure how it's supposed to interact with the rotation point, yet.
    zz zz xx xx yy yy 

24 00 26 00 0A 00 <- Related to the rotation point for object, Not sure on the math yet for it.
    zz zz xx xx yy yy

E7 33 00 00 < Number of 3d voxels in this part

--- This part changes, can either be here or isn't ---  Wish I knew how the program knows this part exists. 
zz zz xx xx yy yy ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? zz zz xx xx yy yy nn nn nn nn
    nn nn nn nn <- Number of voxels until the next location jump.

00 00 00 00 
--- END Section ---

26 C1 95 66 C2 95 66 C3 95 ... < 3d data

How 3D data is stored. Each voxel has 3 bytes of data. If we represent the 3 bytes with "gg ff pp" then:

pp is the palate code. This tells what color this voxel should be.
gg contains the direction and some of the normal vector info.
- if we represent gg's binary data as "dddd dnnn":
  - dddd d Is used in a lookup chart to know where the voxel should be placed relative to the last voxel.
  - nnn <- is used for finding the normal vector
ff - I believe this is also used for finding the normal vector.

Multi-part Models

If this is a multi part file......

F5 A8 00 F5 D0 00 F5  end of 3d data
00 00 00 00 00 

00 03 04 04     Constant ID
51 00 00 00     ???
01 03 04 04
09 00 00 00 
00 02 03 04         is it 2 3 4: 4 14  or is it 2 3 4 4: 14????  heck what could it be pointing to or representing?
04 14 00 00 00 40 1E E0 BE F0 DB 7C 43 FF DD CB 43 03 03 04 04 2C 00 00 00 64 71 12 34 0F 96 59 34 00 00 80 BF 52 CF 7D 3F F2 A8 05 3E 4D 97 2D 34 F2 A8 05 BE 52 CF 7D 3F 1F 9C 44 34 00 00 04 04 84 76 00 00 

00 01 04 04     Constant ID
13 00 00 00     Length of Part's Name + 8
44 72 69 6C 6C 5F 41 72 6D 5F 31 
00 02 04 04     Constant ID
18 76 00 00     Distance to next 03 04 04 marker
01 02 04 04     Constant ID
10 76 00 00 
0A 00 00 00     ????? 

08 76 00 00 00 00 A0 40 08 76 00 00 07 76 00 00 

AF 00 37 00 21 00   If this is the same as the other section, then this is the scale data

A9 FF E5 FF F0 FF   ???Position?  Check!

4D 00 07 00 F7 FF   ???Rotation Check it out.
 31 27 00 00        3d voxel count?
 
ok no guess for here.... will have to try out stuff and see
 4D 00 11 00 F6 FF F7 FF F6 FF FD FF 14 00 15 00 08 00 00 00 F6 FF 01 00 60 02 00 
 00 00 00 00 00 
 
 ok, 3d data begins again.
 26 B0 F5 B7 E9 F5 6F E9 F5 2E

Lookup chart for voxel direction data

       +Y
       |  /z+
       | /
       |/
+x---------- (-x) 
      /|
     / |
    /  |

                            x, y, z 
dict_directions = {'00000':[-1,-1,-1]}  00
dict_directions['00001'] = [-1, 0,-1]   08
dict_directions['00010'] = [-1,+1,-1]   10
dict_directions['00011'] = [ 0,-1,-1]   18
dict_directions['00100'] = [ 0, 0,-1]
dict_directions['00101'] = [ 0,+1,-1]   28
dict_directions['00110'] = [+1,-1,-1]   30
dict_directions['00111'] = [+1, 0,-1]   38
dict_directions['01000'] = [+1,+1,-1]   40
dict_directions['01001'] = [-1,-1, 0]   48
dict_directions['01010'] = [-1, 0, 0]   50
dict_directions['01011'] = [-1,+1, 0]   58
dict_directions['01100'] = [ 0,-1, 0]   60
dict_directions['01101'] = [ 0,+1, 0]   68
dict_directions['01110'] = [+1,-1, 0]   70
dict_directions['01111'] = [+1, 0, 0]   78
dict_directions['10000'] = [+1,+1, 0]   80
dict_directions['10001'] = [-1,-1,+1]   88
dict_directions['10010'] = [-1, 0,+1]   90
dict_directions['10011'] = [-1,+1,+1]   98         
dict_directions['10100'] = [ 0,-1,+1 ]  A0
dict_directions['10101'] = [ 0, 0,+1 ]  A8 
dict_directions['10110'] = [ 0,+1,+1 ]  B0
dict_directions['10111'] = [+1,-1,+1 ]  B8
dict_directions['11000'] = [+1, 0,+1]   C0
dict_directions['11001'] = [+1,+1,+1]   C8
dict_directions['11010'] = [ ?, ?, ?]   d0 ??????!!!!!  Noticed this appears only at the end of parts sections for direction for some files!

Other files don't have it. I think it is a marker that they used for certain versions of the CVR file.

Difference between revisions of "Caviar"

Latest revision as of 10:29, 6 December 2020

Contents

Overview

CVR Specification

Appendix

File start

Large Amount of unknown data

3d Voxel Data Section

Multi-part Models

Lookup chart for voxel direction data

Navigation menu

Personal tools

Namespaces

Variants

Views

Actions

Search

Navigation

AC2

Toolbox

Datalinks

@@ Line 288: / Line 288: @@
 </pre>
-Appendix
+=== Appendix ===
 Example file layout
 -------------------