Unprotecting a protected MIX

This tutorial shows the reader how to unprotect a MIX file protected by the methods of MIX Protector or XRML Leechkiller as of spring 2009. Please note that due to the public availability of this information, the tools in question may change their approach, invalidating this technique.

Background

TS/RA2 Mix files consist of a header, an index, and a body.

The header includes basic information about the MIX (does it have a checksum, is it encrypted, how many files are in it, how big is it).
The index includes the ID, Address/Location, and size of each file in the mix.
The body contains the actual files.

There are two main "MIX protector" or "Leechkiller" programs out there that will mangle the header (specifically, the internal file size) in such a way that RA2 reads the mix properly but XCC Mixer will refuse to open it. They both operate under essentially the same principle.

First, it is necessary to understand how the headers are laid out. The first four bytes are the encryption and checksum flags. If the MIX is has encrypted headers (not to be confused with "protected": "encrypted" refers to the Blowfish encryption that Westwood used on its own mixes; XCC breaks this just fine) the flags are 00 02 00 00. If the MIX has a checksum, the flags are 00 01 00 00. Neither of the current MIX Protector programs work on files with encrypted headers, (although they could be modified to do so with some effort), so I won't go into that right now.

On unencrypted MIXes (i.e. all the ones you will need to break), after the flags comes the "header". The header consists of a 16-bit integer that equals the number of files in the MIX, and a 32-bit integer that equals the size of the body of the MIX (the size of the MIX file minus the size of the header and index). These numbers are "little-endian", which means the bytes are arranged backwards (but the bits within each byte are arranged normally). (This is because the Intel x86 architecture is semi-retarded). So if bytes 04 and 05 (the number of files) are 0xB801, you need to rearrange this to 0x01B8 to get the actual number of files. (0x01B8 in decimal is 440, you can use Google Calculator to convert if necessary.)

Header summary

So, here's the big picture so far: The first few bytes of the MIX are

byte 00   : always 00
byte 01   : is 01 if checksummed, 02 if encrypted, 00 otherwise. Should always be 00 on protected mixes.
byte 02   : always 00 (VK's Mix Protector will overwrite this with garbage)
byte 03   : always 00 (VK's Mix Protector will overwrite this with garbage)
byte 04-05: little-endian number containing the number of files.
byte 06-09: little-endian number containing MIX body size. (LeechKiller and VK's Mix Protector will both overwrite bytes 06 and 07 with garbage).

Then you have the index (12 bytes per index entry) and then the body.

How protection works

All the "MIX protectors" do is add some number to the body size. XCC Mixer checks the body size first when reading a MIX, and rejects the MIX if the body size in the header is larger than the actual body size. It does this to avoid accessing unallocated memory and crashing with a segmentation fault.

Red Alert 2, on the other hand, apparently just ignores the body size in the header, and operates entirely on the information in the index. In theory, anyway. I suppose it's possible that "protected" mixes could cause RA2 to access memory out of bounds and crash, but I haven't seen it happen yet.

Anyway, VK's Mix Protector apparently just adds 2 to byte 06, causing XCCU to fail but making it easy to repair the MIX file. The XRML LeechKiller will add a random 16-bit value to bytes 06-07, instead, making it more difficult. To repair a "protected" MIX, you need to recalculate the body size and write it into bytes 06-09 with a hex editor.

To calculate the body size, take the actual file size by right-clicking on the file and clicking Properties. Windows should give you Size: n MB (x bytes). (Ignore "Size on Disk"). Take the number of bytes and apply the following formula:

BodySize = FileSize - ( (NumberOfFilesInMix * 12) + 10)

You see, the index has twelve bytes for each file, the header has four bytes for flags, two bytes for the number of files, and four bytes for the body size.

Don't forget, the body size and the number of files are all in little-endian form, and must be inserted as such.

In short, for the impatient

Grab your favorite hex editor and open the protected mix.
Ensure that the first four bytes (bytes 00 - 03) are 0x00000000.
Read the next two bytes (bytes 04-05) and reverse the order (e.g. b8 01 becomes 01 b8). This is the number of files. Convert it from hex to decimal.
Multiply the number of files by 12. Then add 10. We'll call this number H.
Take the file size. We'll call this number F.
F - H = B.
Convert B to hex. Pad the front with zeroes until it is eight hex digits long. Rearrange the bytes in little-endian order (e.g. 00 94 26 7F becomes 7F 26 94 00)
Overwrite the next four bytes (bytes 06-09) with this corrected value.
Save the MIX file and exit your hex editor.

File Names Inside a MIX

All MIX files that was created or modified by Olaf's XCC MIX Utilities like "XCC MIX Editor" and "XCC Mixer" appends a "local mix database.dat"(LMD for short). "local mix database.dat" is used by XCC MIX Utilities to identify all the possible file name entries that are found inside the MIX File, thus translating these file names into a CCRC generated ID. In more simple terms:

If a corresponding file's name is NOT listed at "local mix database.dat" then XCC MIX Utility Tools will only let you see the corresponding file's name in hexadecimal format which is its "ID".
If a corresponding file's name is listed at "local mix database.dat" then XCC MIX Utility Tools will show this corresponding file's name properly.

Understanding the MIX Index

All files have ID entries(aka index), they are always found Between the MIX Header(starting 10 bytes that we fixed above) and the Body. Each File ID entries occupy 12 bytes.

byte 1 - 4   : (Little-Endian) - File ID
byte 5 - 8   : (Little-Endian) - File Body Offset Address/Location
byte 9 - 12  : (Little-Endian) - File Size

How File Name Corruption Works

As stated earlier, All files have an index and body, so do the LMD too. Mix protectors tend to corrupt either:

The index(id reference entry) of the LMD - LeechKiller sets the 12 byte entry of the LMD to a pad of zeros, modern mix protectors set this to a random pattern style aka scrambling. Using the Method below will surely recover the File Names inside the MIX File.
The body(local mix database.dat) of the LMD - modern mix protectors removes the LMD.dat from the mix aka mangling. If Mix file has a Mangled LMD then it is Impossible to recover the File Names using the method below since the Original LMD was mangled. But it is possible to recover its file names if you create a new "local mix database.dat" with all possible file names inside that mix(although it is hazzle and time consuming but worth it if you try). This method is called LMD Fabrication.

Recovering all File Names inside the MIX

This is a little bit harder and need some tool.
1) Make sure that the MIX header is UNPROTECTED(can be opened by xcc mixer or xcc mix editor), if not, then unprotect it first by following the steps above.
2) Download and install XCC Mix Editor.
3) OpenXCC MIX Editor and browse the MIX file you want to recover the filenames.
4) Click on "Compact" Button(this will append a new index suited for the LMD incase that there are no presend LMD ID), then close the XCC MIX Editor.
5) Grab your Favorite Hex Editor and "String Search" for "XCC by Olaf van der Spek" without quotation marks.
6) If an Instance was found, clarify if the arrangement of byte is like this:

Important 32 bytes field that should exist																																LMDSize
58	43	43	20	62	79	20	4F	6C	61	66	20	76	61	6E	20	64	65	72	20	53	70	65	6B	1A	04	17	27	10	19	80	00	nn	nn	nn	nn
X	C	C		b	y		O	l	a	f		v	a	n		d	e	r		S	p	e	k

7) Read the "nn nn nn nn", then reorder the bytes in BIG-ENDIAN order(E0 53 00 00 becomes 00 00 53 E0). Remember it as LMDSize.
8) Determine the current position(Address/Location) of the first character we searched(position of the "X" or byte "58"). Common Hex editors projects the "Address" on its left-side (Your Hex Editor must support projecting Address/Location). Remember this as LMDOffset.
9) Remember the BodySize?(7th(06) byte to 10th(09) byte of the Header) If not, use your hex editor and go to very first line of this mix file, read the 7th(06) byte to 10th(09) byte of the Header, reorder the bytes into BIG-endian order(E0 53 03 00 becomes 00 03 53 E0). Remember this as BodySize.
10) Use the Formula below to and calculate the BodyOffset.
BodyOffset = LMDOffset - BodySize
11) Convert the calculated result into little-endian(00 0F 21 23 becomes 23 21 0F 00), remember this as BodyOffset.
12) MOVE KEYBOARD FOCUS of your Hex Editor at the starting of this mix file(topleft-most), just simply left-click the very first byte to do this. then put your Hex Editor in "Hex Search Mode"(Your Hex Editor must support that) and search for "1f 05 6e 36" without quotation marks.("1f 05 6e 36" is the ID of the LMD that is always present)
13) when the first occurence was found, the format you will see is like this:

1F	05	6E	36	nn	nn	nn	nn	NN	NN	NN	NN
LMD ID				LMD Body Offset				LMD File Size

Change the "nn nn nn nn" into the BodyOffset. Change then "NN NN NN NN" into the LMDSize(Little Endian Order).
14) Save the Hex of the MIX, and try opening it with XCC Mixer, the File Names inside this MIX are now Recovered.

Recovering the MIX using an Application

If you find it hard to do all of these technical stuffs, you can use RA2YR MIX Unprotector Master to completely recover the MIX's header and the file names inside it. But this application cannot recover file names if the MIX File's LMD was mangled.