Tutorial: Extracting Raw Texture Data and Calculating Hash Values From TPL Files + Naming PNG Image Files As Per Dolphin Standards

I’ve been playing around with the “Dump Texture” feature in Dolphin recently.

When the emulator dumps textures, it creates a file name that includes a hash of the texture data. This hash uses the xxhash64 algorithm.

I’ve found that, using information about the TPL file format, I’m able to extract the raw image data from the TPL files.

Once I extracted this data (via manually copy/pasting in a hex editor), that I’m was able to feed it into the xxhash64 algorithm. I used QuickHashGUI for this.

To my surprise, this actually worked and the hashes matched!

Extracting the image data from the first image contained in the logo.tpl file (the Marvelous logo) and comparing hash values to the dumped Dolphin texture filename.

I’m going to try creating a QuickBMS script to extract raw image data from TPL files to somewhat automate the process.

Eventually this means that I should be able to generate hashed filenames for every texture on the game’s disc. This bypasses the need to typically play through and manually come across every texture in-game.


Update

I’ve created a QuickBMS script that can take any input TPL file, and output the extracted raw texture data.

endian big

idstring "\x00\x20\xAF\x30"

get filesize asize
get openedFile filename

get numberOfImages long
get imageTableOff long

if numberOfImages == 1
    goto imageTableOff
    get imageHeaderOff long
    goto imageHeaderOff
    get height short
    get width short
    get imageFormat long
    get imageAddress long
    
    set name string openedFile
    string name - ".tpl"
    string name + "_Texture0_"
    string name + width
    string name + "x"
    string name + height
    string name + "_"
    string name + imageFormat
    string name + ".bin"
    
    math filesize - imageAddress
    
    log name imageAddress filesize
else
    for i = 1 < numberOfImages
        goto imageTableOff
        get imageHeaderOff long
        get null long
        get nexImageHeaderOff long
        
        goto imageHeaderOff
        get height short
        get width short
        get imageFormat long
        get imageAddress long
        
        goto nexImageHeaderOff
        get null long
        get null long
        get nexImageAddress long
        
        set imageSize long nexImageAddress
        math imageSize - imageAddress
        
        set name string openedFile
        string name - ".tpl"
        string name + "_Texture"
        set texNum long i
        math texNum - 1
        string name + texNum
        string name + "_"
        string name + width
        string name + "x"
        string name + height
        string name + "_"
        string name + imageFormat
        string name + ".bin"
        
        log name imageAddress imageSize
        
        math imageTableOff + 0x08
    next i
    
    goto imageTableOff
    get imageHeaderOff long
    goto imageHeaderOff
    get height short
    get width short
    get imageFormat long
    get imageAddress long
    
    set name string openedFile
    string name - ".tpl"
    string name + "_Texture"
    math numberOfImages - 1
    string name + numberOfImages
    string name + "_"
    string name + width
    string name + "x"
    string name + height
    string name + "_"
    string name + imageFormat
    string name + ".bin"
    
    math filesize - imageAddress
    
    log name imageAddress filesize
endif

Once extracted, these files can be hashed with a tool (e.g. QuickHash GUI) using the xxHash64 algorithm.

Combining the data from the script with the hash will provide you with the full filename that Dolphin would normally generate.

Comparison of BMS-extracted texture data vs Dolphin-dumped files

Once you’ve determined the file hashes, you can extract the textures from your TPL as PNG files using a tool like BrawlBox or Wexos’s Toolbox.

The naming convention will be as follows:

  • Naming Type – denotes the texture naming scheme used by Dolphin, should always be “tex1”
  • width x height – dimensions of the original texture, can be found in the file output from the BMS script above
  • hash – xxHash64 of the raw original texture data (as output from the BMS script once fed through a tool such as QuickHash GUI)
  • format – original texture image format as named in output from the BMS script above
  • Output file extention – output image file, should always be “.png”

Here’s a visual demonstration of the file naming convention, and where the different parts can be found.

Extracting a texture to a png image using BrawlBox, using naming information provided by QuickBMS + QuickHash

Update #2

After going through every CLZ, ARC, and TPL file, I beleive I’ve been able to dump/rename (almost) every texture in the game.

There are a total of 4115 unique texture images from Harvest Moon: A Proud Life (lesbian version).

I’ve compiled all of these dumped textures into a Dolphin-compatible pack, which can be downloaded [here].

Player Model Swapping

Now that I’m able to access the contents of the previously inaccessible CLZ files, It’s time to experiment with model files.

My first test was to decompress the mainchapter archives (e.g. mainchapter0.arc.clz), then extract the boy archives from there (e.g. boy_0.arc).

Then, I went about swapping the GPL files (e.g. boy_0.gpl). Based on my previous research, these should contain all of the 3D model data needed.

My first test at importing the girl model into AWL was… less than successful.

All hail the nebulous genderless void!

But at least the game didn’t crash, so that’s a start.

After some further tests, I found that I needed to swap all of the girl/boy-associated model files (act, gpl, skn, tpl, and tam) with their boy/girl counterparts.

Time to replace some additional data!

After replacing the model files, replacing boy_0.arc and girl_0.arc in the corresponding mainchapter0.arc files with their updated versions, and repacking those into their corresponding mainchapter0.arc.clz formats, it was time to test them out.

To my surprise, the models worked with little to no issue.

We now have the female model in AWL and male character in AnWL!

Note the mirror (AnWL) or lack thereof (AWL)

The animations seem to rig fairly well to each character, with a few exceptions (e.g. standing still while holding an animal such as a dog or chicken as the girl in AWL will cause the ponytail to do weird things).

I’ll be doing some more testing regarding the animations, with the hopes of fixing any discrepancies (e.g. the above animal holding glitch, the wider/narrower stance difference between the characters, etc). I know James in the Discord has been very active in experimenting with the character animations lately.

Exploring the GameCube Service Disc gpl models

Preliminary Analysis of gpl files on the Service Disc

After a bit of research on gpl models, I’ve been lead to the Gamecube Service Disc v1.0/03.

Upon analysis of this disc, I’ve found that it contains gpl files created using version 12012000 of the geometry library (same as the Dolphin SDK), as well as version 6012001 (same as A Wonderful Life).

When booting the disc normally, it seems to load the v12012000 gpl files (e.g. disc:/gxDemos/CarDemo/f50.gpl) for the default Car Demo program (disc:/CarDemo/cardemo.dol).

Attempting to Swap Models

However, I can get it to load the 6012001 versions (e.g. disc:/CarDemo/f50.gpl) by replacing the main executable on the disc (disc:/main.dol; naming might vary depending on the tool used) with a special version of the Car Demo program (disc:/ntd/cardemoN.dol).

With this in mind, I should be able to (in theory) replace some of the models in the cardemo with files from AWL.

E.g. If I were to look at bass.arc from AWL, it contains all of the needed files (an act, a gpl, and tpl).

If I were to extract these files and adjust the files on the service disc as follows:

  • rename disc:/CarDemo/f50.act to disc:/CarDemo/f50.act.backup
  • rename disc:/main.dol to disc:/main.dol.backup
  • copy disc:/ntd/cardemoN.dol to disc:/main.dol
  • copy bass.act to disc:/CarDemo/f50.act
  • copy bass.gpl to disc/CarDemo/bass.gpl
  • copy bass.tpl to disc/CarDemo/bass.tpl

It should load the bass model when booting the disc, instead of the default ferrari.

However, it seems that the new model (in this case, bass.gpl) does not load correctly.

The bass model should be above the rectangular shadow in the middle.

I’m not sure why this is currently the case, but I plan on doing some further tests as I can.

Why a lot of AWL models won’t work

A lot of gpl files on the root of the AWL disc (which, it’s worth mentioning, aren’t actually used by the game) don’t seem to be configured correctly.

As an example, let’s look at boy_0.arc.

While this file contains a valid act and gpl, it has 3 seperate texture (tpl) files for the body (b0), eyes (f0_e), and mouth (f0_m).

This is a problem because most gpl files are coded to load one tpl file. In this case, we can open boy_0.gpl in a hex editor and see that it’s supposed to load all of it’s textures from a nonexistent Boy_0.tpl file.

As such, attempting to load the boy_0 model using the above method I tried for bass, will not work. The program will simply give an error “Could not find CarDemo/Boy_0.tpl” and freeze/crash.

Attempting to rename any of the other texture files (e.g. renaming boy_0_b0.tpl to Boy_0.tpl) also doesn’t work, it just causes a crash. This is likely because the elusive Boy_0.tpl file would include all body, eye, and mouth textures in one single tpl file.

More Research Required

I plan on doing more testing with the Service Disc and the CarDemoN program.

There’s also other demos, disc:/ntd/zebraA.dol and disc:/ntd/zebraS.dol, that could be useful for viewing models/animations.

There’s also a program disc:/ntd/preview.dol that might be useful for previewing models.

Hopefully I’ll be able to come up with something over the next few weeks.

Exploring File Formats

After scouring the internet for any documentation on the file formats used for the character models (namely the .gpl files) I’ve found the one (and seemingly only) source of information.

The Documentation for the official Nintendo Gamecube SDK / Character Pipeline / Dolphin Emulator

There are some executables included with the sdk that enable previewing of GPL (geometry), ACT (hierarchy), ANM (animation), SKN (skinning information) and TPL (texture) files. However, it seems that these require some sort of PowerPC configuration through either the dolphin development hardware or an old powerpc mac. There is a previewer.exe in the sdk, but I haven’t been able to get it to load a gpl without crashing so far.

Honestly my best chance will probably be emulating an old build of Mac OS X (e.g. 10.5 Leopard) and trying to load the preview scripts on that platform.

I will post back with my results.

HM: A Proud Life – Face Textures

After looking through all of the files that I could find, I’ve been unable to locate the face textures for the main character in AnWL; I’ve only been able to locate the body/outfit textures.

The face texture files are easy enough to find in in AWL:

  • Young Character:  disc:\boy_0.arc (U8 Archive)
    • Eye Textures: boy_0.arc\boy_0_f0_e.tpl
    • Mouth Textures: boy_0.arc\boy_0_f0_m.tpl
  • Adult Character: disc:\boy_1.arc
    • Eye Textures: boy_1.arc\boy_0_f1_e.tpl
    • Mouth Textures: boy_1.arc\boy_0_f1_m.tpl
  • Senior Character:  disc:\boy_2.arc
    • Eye Textures: boy_2.arc\boy_0_f2_e.tpl
    • Mouth Textures: boy_2.arc\boy_0_f2_m.tpl

However, the structure seems to be different for AnWL. I’ve created a diff comparison for the disc filesystems to hopefully help with finding the newly located files.

I’ve run the game through dolphin and can see that the textures do exist somewhere, but I can’t seem to find the source that they’re being loaded from.

If anyone can give any insight, it would be much appreciated.

For now I’ll work on the swapping out the body/outfit textures.