Difference between revisions of "Caviar"

Revision as of 10:23, 6 December 2020

Caviar is a voxel based 3D model format with animation capabilities developed by the now-defunct AnimaTek International, since purchased by Digital Element Inc.

The .cvr files in Alpha Centauri contain Caviar models that are combined to produce graphics for most units in the game.

Ford Prefect partially deciphered the file format and wrote a new and open source Caviar viewer and paint tool, CVR-Colorizer.

A proprietary Caviar player, CvrPlay.exe, is also available. CvrPlay.exe supports animations and allows rotation of the models, unlike Ford's Colorizer. It cannot edit .cvr files, however.

Overview

Ford identified 3 major sections in the caviar file format.

File start - this section includes the model name and some of the palette codes.
Unknown - Currently unknown what this section of data is for.
3D data - Contains the 3D voxel information. Position, Color and the normal vector.

CVR Specification

PlotinusRedux's work on reverse engineering it:

The file is a set of variable-length chunks with some chunks nested inside each other. Each chunk begins with the header:

0000    uint32       Tag
0004    uint32       Size including this header
0008    <data>
Size    <next chunk>

Some chunks contain only other chunks (the ones with "Container" in their name), while others contain actual data.

The tag values are:

typedef enum CVRCHUNK_E {
    CVRCHUNK_FILE_CONTAINER         = 0x00525643,
    CVRCHUNK_VERSION                = 0x01000000,
    CVRCHUNK_DB_NAME                = 0x02000000,

    CVRCHUNK_PALETTE_CONTAINER      = 0x03000000,
    CVRCHUNK_PALETTE_NAME           = 0x01010000,
    CVRCHUNK_PALETTE_DATA           = 0x01020000,
    CVRCHUNK_MATERIAL               = 0x01030000,

    CVRCHUNK_SCENE_CONTAINER        = 0x04000000,
    CVRCHUNK_SCENE_NAME             = 0x04010000,
    CVRCHUNK_OBJECT_COUNTER         = 0x04020000,
    CVRCHUNK_FRAME_COUNTER          = 0x04030000,

    CVRCHUNK_OBJECT_CONTAINER       = 0x04040000,
    CVRCHUNK_OBJECT_NAME            = 0x04040100,

    CVRCHUNK_GEOMETRY_CONTAINER     = 0x04040200,
    CVRCHUNK_VOXEL_OBJECT           = 0x04040201,

    CVRCHUNK_ANIMATION_CONTAINER    = 0x04040300,
    CVRCHUNK_OBJECT_FLAG            = 0x04040301,
    CVRCHUNK_OBJECT_LOCATION        = 0x04040302,
    CVRCHUNK_OBJECT_MATRIX          = 0x04040303,
} CVR_CHUNK_E;

Chunks are described below.

CVRCHUNK_FILE_CONTAINER (0x00525643)

Always at the start of the file, and contains all the other chunks in the file.

Child chunks:

   CVRCHUNK_VERSION (1)
   CVRCHUNK_DB_NAME (1)
   CVRCHUNK_PALETTE_CONTAINER (1)
   CVRCHUNK_SCENE_CONTAINER (1)

CVRCHUNK_VERSION (0x01000000)

File version.

Data: 0000 float32 Version #

CVRCHUNK_DB_NAME (0x02000000)

Original .max file name

Data: 0000 ascii Name, length given by chunk size.

CVRCHUNK_PALETTE_CONTAINER (0x03000000)

Contains Palette related chunks.

Child chunks:

   CVRCHUNK_PALETTE_NAME (1)
   CVRCHUNK_PALETTE_DATA (1)
   CVRCHUNK_MATERIAL (0-1)

CVRCHUNK_PALETTE_NAME (0x01010000)

Text name for palette

Data: 0000 ascii Name, length given by chunk size.

CVRCHUNK_PALETTE_DATA (0x01020000)

8-bit palette color tables

0000 uint8 First color index used. 0001 uint8 Last color index used. 0002 0x100*RGB24 Physical Palette as array[256] of 3-byte RGB values. 0102 0x18*0x100*uint8 Shade Array[24][256] of byte indexes into Physical Palette.

                       <Light Level(0-23)>*256 + <Voxel Color Index> gives index 
                       of color in Physical Palette.

CVRCHUNK_MATERIAL (0x01030000)

16-bit color tables

0000 0x18*0x100*uint16 ShadeArray[24][256] of 16 bit R5G5B5 color values.

                         <Light Level(0-23)>*512 + <Voxel Color Index>*2 gives
                         R5G5B5 color value for voxel.

CVRCHUNK_SCENE_CONTAINER (0x04000000)

Contains voxel and animation chunks.

Child Chunks:

   CVRCHUNK_SCENE_NAME (1)
   CVRCHUNK_OBJECT_COUNTER (0-1)
   CVRCHUNK_FRAME_COUNTER (0-1)
   CVRCHUNK_OBJECT_CONTAINER (0+)

CVRCHUNK_SCENE_NAME (0x04010000)

Text name for scene (usually same as DB name)

Data: 0000 ascii Name, length given by chunk size.

CVRCHUNK_OBJECT_COUNTER (0x04020000)

Number of objects in this scene

Data: 0000 uint32 # of objects in scene

CVRCHUNK_FRAME_COUNTER (0x04030000)

Number of frames in the animation of this scene

Data: 0000 uint32 # of frames in scene (1+)

CVRCHUNK_OBJECT_CONTAINER (0x04040000)

Container for chunks related to a single object

Child chunks:

   CVRCHUNK_OBJECT_NAME (1)
   CVRCHUNK_GEOMETRY_CONTAINER (0-1)
   CVRCHUNK_ANIMATION_CONTAINER (1)

CVRCHUNK_OBJECT_NAME (0x04040100)

Text name for object

Data: 0000 ascii Name, length given by chunk size.

CVRCHUNK_GEOMETRY_CONTAINER (0x04040200)

Container for voxel information

Child Chunks:

   CVRCHUNK_VOXEL_OBJECT

CVRCHUNK_VOXEL_OBJECT (0x04040201)

Voxel information for object

Data Types:

SVECTOR3(6) 0000 int16 Z 0002 int16 X 0004 int16 Y

VOXEL_HEADER(0x2A) 0000 uint32 Flags. 1 = Next voxel group has its own VOXEL_HEADER

                           2 = Next voxel group has its own JUMP_HEADER
                           4 = Unused
                           8 = Used but unknown what it means

0004 uint32 Size of voxel object from start of first VOXEL_HEADER 0008 float32 Units per 3D pixel 000C uint32 If Flags & 1: bytes to next VOXEL_HEADER from start of

                       this header
                   Else: same of offset 0004.

0010 uint32 If Flags & 1: bytes to last byte of this voxel group

                   Else: bytes to last byte of voxel object

0014 SVECTOR3 Scale 001A SVECTOR3 Position 0020 SVECTOR3 Rotation Center 0026 uint32 If Flags & 1: # voxels until next VOXEL_HEADER

                   Else: # voxels in object

JUMP_HEADER(0x20 or 0x24) 0000 SVECTOR3 Position 0006 SVECTOR3 v2--unknown vector, used determining max size 000C SVECTOR3 v3--unknown vector, used determining max size 0012 SVECTOR3 PositionOffset--start drawing at Position+PositionOffset 0018 uint32 # of voxels in this jump group 001C uint32 JumpFlags. 1 = Has Offset <020 uint32 Offset to next JUMP_HEADER. Present only if

                   JumpFlags & 1>

VOXEL_DATA 0000 uint8 Direction (see addendum) 0004 uint8 Normal (see addendum) 0008 uint8 Color index into shading tables

Data layout depends on Flags.

Flags & 1: VOXEL_HEADER array of VOXEL_DATA VOXEL_HEADER array of VOXEL_DATA ....

Flags & 2: VOXEL_HEADER JUMP_HEADER array of VOXEL_DATA JUMP_HEADER array of VOXEL_DATA ...

!(Flags & 3): VOXEL_HEADER array of VOXEL_DATA

CVRCHUNK_ANIMATION_CONTAINER (0x04040300)

Container for animation data.

Child chunks: CVRCHUNK_OBJECT_FLAG (0-1) CVRCHUNK_OBJECT_LOCATION (0-1) CVRCHUNK_OBJECT_MATRIX (0-1)

CVRCHUNK_OBJECT_FLAG (0x04040301)

Array[FrameCount] of flags

0000 uint8 ObjectFlag 0004... repeated FrameCount times

CVRCHUNK_OBJECT_LOCATION (0x04040302)

Array[FrameCount] of location vectors

0000 float32 Z 0004 float32 X 0008 float32 Y 001C... repeated FrameCount times

CVRCHUNK_OBJECT_MATRIX (0x04040303)

Array[FrameCount] of 3x3 matrixes of float32 in Z,X,Y order

0000 3*3*float32 Matrix 0024... repeated FrameCount times

Appendix

==

Example file layout

ACOLPOD.cvr CVRCHUNK_FILE_CONTAINER 00000000 00091120

   CVRCHUNK_VERSION 00000008 0000000c
   CVRCHUNK_DB_NAME "MB_Alien_Pod2.max"  00000014 00000019
   CVRCHUNK_PALETTE_CONTAINER 0000002D 00004b2d
       CVRCHUNK_PALETTE_NAME "cavcols"  00000035 0000000f
       CVRCHUNK_PALETTE_DATA 00000044 00001b0a
       CVRCHUNK_MATERIAL 00001B4E 0000300c
   CVRCHUNK_SCENE_CONTAINER 00004B5A 0008c5c6
       CVRCHUNK_SCENE_NAME "MB_Alien_Pod2.max"  00004B62 00000019
       CVRCHUNK_OBJECT_COUNTER (2)  00004B7B 0000000c
       CVRCHUNK_FRAME_COUNTER (15)  00004B87 0000000c
       CVRCHUNK_OBJECT_CONTAINER 00004B93 0008bdb6
           CVRCHUNK_OBJECT_NAME "ACOLPOD"  00004B9B 0000000f
           CVRCHUNK_GEOMETRY_CONTAINER 00004BAA 0008baa0
               CVRCHUNK_VOXEL_OBJECT 00004BB2 0008ba98
           CVRCHUNK_ANIMATION_CONTAINER 0009064A 000002ff
               CVRCHUNK_OBJECT_FLAG 00090652 00000017
               CVRCHUNK_OBJECT_LOCATION 00090669 000000bc
               CVRCHUNK_OBJECT_MATRIX 00090725 00000224
       CVRCHUNK_OBJECT_CONTAINER 00090949 000007d7
           CVRCHUNK_OBJECT_NAME "Lit Objects"  00090951 00000013
           CVRCHUNK_GEOMETRY_CONTAINER 00090964 000004bd
               CVRCHUNK_VOXEL_OBJECT 0009096C 000004b5
           CVRCHUNK_ANIMATION_CONTAINER 00090E21 000002ff
               CVRCHUNK_OBJECT_FLAG 00090E29 00000017
               CVRCHUNK_OBJECT_LOCATION 00090E40 000000bc
               CVRCHUNK_OBJECT_MATRIX 00090EFC 00000224

Voxel Direction Element

The upper 5 bits control the direct of the next voxel.

d = Direction >> 3

    z   x   y
00 -1  -1  -1
01 -1  -1   0
02 -1  -1   1
03 -1   0  -1
04 -1   0   0
05 -1   0   1
06 -1   1  -1
07 -1   1   0
08 -1   1   1

09  0  -1  -1
0A  0  -1   0
0B  0  -1   1
0C  0   0  -1
0D  0   0   1
0E  0   1  -1
0F  0   1   0
10  0   1   1

11  1  -1  -1
12  1  -1   0
13  1  -1   1
14  1   0  -1
15  1   0   0
16  1   0   1
17  1   1  -1
18  1   1   0
19  1   1   1

1A  STOP

Voxel Normal Element

n = ((Direction & 7) << 8) | Normal

void DecodeNormal(DWORD n, double* x, double* y, double* z) { // A1(0-23)*89+A2(0-88) // Normally the highest number would be 23 * 89 + 88 = 2136, // but because in the case of A1=0, we know Y=1.0, so X=Z=0.0, so A2 = 0, // we compressed the range 0-88 down to 0 // In the case of A1=0, sin(A1)=0, so x & z will be 0 for any value // from 0 to 88 below n = n + 88;

// extract the original angles // Angle1 = Z to X-axis // Angle2 = Y to X-axis DWORD dwAngle1 = 88 - n % 89; DWORD dwAngle2 = n / 89;

// convert the angles to radians double dAngle1 = dwAngle1 / 88.0 * 2.0 * PI; double dAngle2 = dwAngle2 / 23.0 * PI / 2.0;

// convert the angles to coords in the range [-1, 1] *z = sin(dAngle1) * sin(dAngle2); *x = cos(dAngle1) * sin(dAngle2); *y = cos(dAngle2); }

FordPrefect's Original Analysis. Contains errors.

Work in progress

Values on the left are the hexadecimal values found in the caviar file format. Description of what they represent on the right.

Data that is tabbed is extra examples and/or values.

File start

43 56 52 00      is the characters "CVR"  

xx xx xx xx <- file size in bytes. Unsigned? 

00 00 00 01 <- Flag saying next is the version number.  
    
Next part        |-------|  Version number for the file is stored here, not sure if it stores anything else as well.
0C 00 00 00 9A 99 21 41 
        9A 99 21 41 - 10.10
        33 33 23 41   version 10.20     
    
00 00 00 02     Constant ID

11 00 00 00     Name length + 8
42 41 53 49 43 2E 4D 41 58  Name of file it was converted from?
    
00 00 00 03     Constant ID

xx xx xx xx  
    The distance in bytes to the 3d description section. Relative to this point!  Points a ways before the name of body part.
    What's interesting this distance seems to be exactly the same for the different palates for non-animated files.  
    Need to check to see if number changes for animated files or not.            
                 
00 00 01 01     Constant ID
0F 00 00 00     = Palette name length + 8
55 6E 6E 61 6D 65 64  Palette name

                       
00 00 02 01     Constant ID
0A 1B 00 00 <- Points to 3d section label 04.  Distance from here to there.

0A F5       ????? A = 10.  F5 = 145   10+145 = 155  
                For reading in the values, I had to move 10 forward before starting to count.
                145 for 145 values used?  Allot of guess work here
    aa F5 00 00 00 FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC FC 00 FC....
    aa F5 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 AC 80 50 A0 7C .... Yeah....
    Ok.  I've learned this:
        aa*3 (usually 1*3 or 10*3) tells how far to right to being start counting up from 0 for color numbers.
        F5 (145) is the length of this sections data.  
        so values greater than 145 - aa. are fetched by some other method.  
        Colors are stored as R G B values.  1 byte each.


00 00 02 01 xx xx xx xx < points to 3d section label 04

0A F5 00 00 00 <  Why is this duplicated!? 
    01 F5 < unnamed.... Animated. 5 body parts
    01 F5 < Unnamed... no-anime. 14 body parts
        /\- 245 hmmm  The next two bits = 245*3 in length.  
        Ok, I (think) this is saying how big the palette is in size.

Large Amount of unknown data

After that there is a large section of data that is currently unknown as to what it does.

Lets skip to the 3d data section.

3d Voxel Data Section


00 00 00 04     Constant ID, Earlier Reference points here
11 9D 00 00     ok this is the distance to the end of the file in bytes + 4.  If you measure from the 04.   

00 00 01 04     Constant ID 
14 00 00 00     Name length + 8
21 41 43 41 43 2D 41 56 2E 6D 61 78 00 NAME 

------------If this is an animation only file this will be missing-----
00 02 04 0C     Constant ID 
00 00 00        ????
01 00 00 00     Number of body parts for object


00              ?? Spacer?
----------- End optional section ------------------

00 03 04 0C     Constant ID ?  
00 00 00        ????<br />
01 00 00 00     # of frames  Never set to 00.  Always 1 or more

00 00 04 04     Constant ID 
DD 9C 00 00     Distance    

00 01 04 04     Constant ID 
10 00 00 00     Length of Part Name + 8 
41 41 2D 52 4F 56 45 52    NAME


----------- On select.cvr, this section wasn't here.  select.cvr was 10.10 version. was animated but only had one "part" 
    This section was also missing from AVHT-LightsReposition, an animation only file.
00 02 04 04     Constant ID 
74 9C 00 00     Distance 
-----------
or
--              Not sure if here all the time if 02 04 04 is missing.  
                Just need to write a function to find ID lines.
00              Probably and artifact of their save function when skiping unused data.
                Might explain some of the weird bits that I've found solo laying around.
--

01 02 04 04     Constant ID 
6C 9C 00 00     Distance 

0A 00 00 00         Ok, currently I have no idea what this is.  If i change it, CVR will crash.
    
    

64 9C 00 00 00 00 80 40 64 9C 00 00 63 9C 00 00 
     most of this has to deal with the number of units per 3d pixel  
     I have no clue how it actually works.  
                      
7F 00 B3 00 2D 00     ZZ sz xx sx yy sy   
        zz,xx,yy didn't change object position.
        sz, sx, sy when i changed them reduced the size and offset the item in the view
        zz, xx, yy I think this tells the viewer the dimensions of the item on screen so that it knows when to render

C1 FF A7 FF EA FF  <- Positional Data. I believe it is the starting point for rendering voxels.  Not sure how it's supposed to interact with the rotation point, yet.
    zz zz xx xx yy yy 

24 00 26 00 0A 00 <- Related to the rotation point for object, Not sure on the math yet for it.
    zz zz xx xx yy yy

E7 33 00 00 < Number of 3d voxels in this part

--- This part changes, can either be here or isn't ---  Wish I knew how the program knows this part exists. 
zz zz xx xx yy yy ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? zz zz xx xx yy yy nn nn nn nn
    nn nn nn nn <- Number of voxels until the next location jump.

00 00 00 00 
--- END Section ---

26 C1 95 66 C2 95 66 C3 95 ... < 3d data

How 3D data is stored. Each voxel has 3 bytes of data. If we represent the 3 bytes with "gg ff pp" then:

pp is the palate code. This tells what color this voxel should be.
gg contains the direction and some of the normal vector info.
- if we represent gg's binary data as "dddd dnnn":
  - dddd d Is used in a lookup chart to know where the voxel should be placed relative to the last voxel.
  - nnn <- is used for finding the normal vector
ff - I believe this is also used for finding the normal vector.

Multi-part Models

If this is a multi part file......

F5 A8 00 F5 D0 00 F5  end of 3d data
00 00 00 00 00 

00 03 04 04     Constant ID
51 00 00 00     ???
01 03 04 04
09 00 00 00 
00 02 03 04         is it 2 3 4: 4 14  or is it 2 3 4 4: 14????  heck what could it be pointing to or representing?
04 14 00 00 00 40 1E E0 BE F0 DB 7C 43 FF DD CB 43 03 03 04 04 2C 00 00 00 64 71 12 34 0F 96 59 34 00 00 80 BF 52 CF 7D 3F F2 A8 05 3E 4D 97 2D 34 F2 A8 05 BE 52 CF 7D 3F 1F 9C 44 34 00 00 04 04 84 76 00 00 

00 01 04 04     Constant ID
13 00 00 00     Length of Part's Name + 8
44 72 69 6C 6C 5F 41 72 6D 5F 31 
00 02 04 04     Constant ID
18 76 00 00     Distance to next 03 04 04 marker
01 02 04 04     Constant ID
10 76 00 00 
0A 00 00 00     ????? 

08 76 00 00 00 00 A0 40 08 76 00 00 07 76 00 00 

AF 00 37 00 21 00   If this is the same as the other section, then this is the scale data

A9 FF E5 FF F0 FF   ???Position?  Check!

4D 00 07 00 F7 FF   ???Rotation Check it out.
 31 27 00 00        3d voxel count?
 
ok no guess for here.... will have to try out stuff and see
 4D 00 11 00 F6 FF F7 FF F6 FF FD FF 14 00 15 00 08 00 00 00 F6 FF 01 00 60 02 00 
 00 00 00 00 00 
 
 ok, 3d data begins again.
 26 B0 F5 B7 E9 F5 6F E9 F5 2E

Lookup chart for voxel direction data

       +Y
       |  /z+
       | /
       |/
+x---------- (-x) 
      /|
     / |
    /  |

                            x, y, z 
dict_directions = {'00000':[-1,-1,-1]}  00
dict_directions['00001'] = [-1, 0,-1]   08
dict_directions['00010'] = [-1,+1,-1]   10
dict_directions['00011'] = [ 0,-1,-1]   18
dict_directions['00100'] = [ 0, 0,-1]
dict_directions['00101'] = [ 0,+1,-1]   28
dict_directions['00110'] = [+1,-1,-1]   30
dict_directions['00111'] = [+1, 0,-1]   38
dict_directions['01000'] = [+1,+1,-1]   40
dict_directions['01001'] = [-1,-1, 0]   48
dict_directions['01010'] = [-1, 0, 0]   50
dict_directions['01011'] = [-1,+1, 0]   58
dict_directions['01100'] = [ 0,-1, 0]   60
dict_directions['01101'] = [ 0,+1, 0]   68
dict_directions['01110'] = [+1,-1, 0]   70
dict_directions['01111'] = [+1, 0, 0]   78
dict_directions['10000'] = [+1,+1, 0]   80
dict_directions['10001'] = [-1,-1,+1]   88
dict_directions['10010'] = [-1, 0,+1]   90
dict_directions['10011'] = [-1,+1,+1]   98         
dict_directions['10100'] = [ 0,-1,+1 ]  A0
dict_directions['10101'] = [ 0, 0,+1 ]  A8 
dict_directions['10110'] = [ 0,+1,+1 ]  B0
dict_directions['10111'] = [+1,-1,+1 ]  B8
dict_directions['11000'] = [+1, 0,+1]   C0
dict_directions['11001'] = [+1,+1,+1]   C8
dict_directions['11010'] = [ ?, ?, ?]   d0 ??????!!!!!  Noticed this appears only at the end of parts sections for direction for some files!

Other files don't have it. I think it is a marker that they used for certain versions of the CVR file.

Difference between revisions of "Caviar"

Revision as of 10:23, 6 December 2020

Contents

Overview

CVR Specification

==

File start

Large Amount of unknown data

3d Voxel Data Section

Multi-part Models

Lookup chart for voxel direction data

Navigation menu

Personal tools

Namespaces

Variants

Views

Actions

Search

Navigation

AC2

Toolbox

Datalinks