Custom Save Blocks

After some research, I’ve figured out how to customize the save file header information for both AWL and AnWL.

Each game has a file named wlp1.rel, which is a relocatable module (similar to how .dll files function for Windows programs). This file has the information which dictates the header info when creating or overwriting a save file.

This information includes the chapter names, the save filename, the game / save title, and the banner / icon to be used.

By manipulating this file, I’ve changed the game / save title for each game to HM: APL (for the wlw mod) and HM: AnPL (for the mlm mod).

I’ve considered changing the filename (originally bokujyo4.dat or bokujyoA.dat, depending on the game), but changing this would invalidate any previous save files for users of earlier alpha/beta builds of the mod.

Save Block Textures

In addition to modifying names / variables in this file, we can also modify the textures used. While I had already modified banner_card.tpl, I decided to try my hand at modifying card_icon.tpl. This file is a bit trickier since it uses a texture format that utilizes pallettes.

The important thing to note hear is that all of the individual textues must utilize the same pallette for the textures to render properly when displayed by the console hardware.

It’s also important to note that the order the textures are displayed is in bouncing order. That is to say that they go in order of 1->2->3->2->1->2->3, as opposed to a normal GIF loop order of 1->2->3->1->2->3. This is most noticable when seeing how the dog mouth opens / closes on the otiginal save icon

Standard Loop
Bounce Loop

Creating a new Save Block Texture

I started by deciding what I wanted to make for the new save icon. After some internal debate, I decided on using the blue feather (the symbolic item for proposing to one of the romance options) with a LGBT flag background. I created a version with a standard 6-stripe rainbow flag (1979 6-stripe variant), as well as one with the lesbian flag (2018 5-stripe variant).

I also created versions with the Philadelphia pride flag (2017 8-stripe variant) and Progress flag (2018 variant). While these show up ok with increased internal resolutions (i.e. emulator), they appears blurry / pixelated on original interlaced resolutions (i.e. original hardware). For this reason, I have made them available as optional, which can be used to replace the card_icon.tpl texture file on the disc.

Blue Feather (from commonall.arc.clz\symbol.tpl\Texture5), 1979 Rainbow Flag, 2018 Lesbian Flag, 2017 Philadelphia Flag, 2018 Progress Flag
Progress and Philadelphia Pride Flag Mockups

Philadelphia Flag card_icon.tpl [DOWNLOAD]
Progress Flag card_icon.tpl [DOWNLOAD]


To ensure that all images used the same pallette, I saved each texture as a single-frame 256-colour .gif file. When saving the first texture, I saved the generated colour table as a Photoshop palette. Then, when I saved the second and third textures, I loaded the colour table from the previously saved palette file. This would ensure that all three saved .gif image files had the same embedded colour table / palette.

When loading a .gif with an embedded colour palette into BrawlBox, it will give the option to import the palette. By selecting this option when replacing each texture in the card_icon.tpl file, it ensures they’ll all match.

Once the game is loaded with the edited wlp1.rel, card_icon.tpl, and banner_card.tpl files, and the game diary is re-saved, the changes will take effect.

AWL/AnWL vs APL/AnPL

Save File Checksums

After some research, we’ve determined that the save files for A(n)WL use a checksum to verify data integrity.

If the save data is modified and this checksum isn’t accounted for, a data corruption error will be presented when attempting to load the data.


This checksum is a 4-byte (32-bit) value stored at 0x40

Note: save files can be when using Dolphin emulator are located in “C:\Users\{USER}\Documents\Dolphin Emulator\GC\USA\Card A\”

The value can be calculated by taking all of the data after the checksum position, i.e., 0x44 to the end of the save file, and then calculating the checksum using the CRC32_bzip2 algorithm.

I’ve found that the easiest way to accomplish this is by saving the save data (from 0x44 onwards) to a seperate file (in this case, I’ve named it as hashtest.bin)

Once created, the save data (hashtest.bin) can be analyzed using a supported tool such as Jacksum. For Jacksum, I used the following hash command:

java -jar jacksum.jar -a all -F "ALGONAME{i} (#FILENAME) = #CHECKSUM{i}" hashtest.bin

Then just look at the output of the crc32_bzip2 hash and voila! It should match the previous 0x40 checksum value.


Using this knowledge, we can now edit various aspects of the save file including:

  • Character names
  • Animals
  • Items
  • Buildings (e.g. milking room, seed maker)
  • Money

The process for making these edits should be fairly straightforward

1. Extract the save data from your GCI file (from 0x44 onwards) and save it to a seperate file (e.g. hashtest.bin)

2. Find the value(s) you want to edit (e.g. money) and edit them

In this case, my money value was previously 999999 (hex = 0F423F). The value seems to be stored at 0x1962e (in hashtest.bin). We’re gonna try changing this to 123456 (hex = 01E240).

3. Calculate a new checksum of your save data using Jacksum (as shown above) by analyzing your seperate save data file (i.e. hashtest.bin)

CRC32_bzip2 checksum of our new data (money = 123456) is 34efd17e

4. Open your original GCI, overwrite with your new checksum at 0x40

5. Overwrite the save data fro 0x44 onwards with your newly edited data (copied from hashtest.bin)

6. Save your new GCI file

Voila!

Update

As an alternative to Jacksum, you can actually calculate the checksum right from within Hex Editor Neo (if you’re using a different hex editor, you’ll need to follow the Jacksum directions above).

Simply go to View -> Tool Windows -> Checksum

Then add, select the “Add Algorithm button to create an algorithm with the following settings:

  • CRC-32
  • Initial Value: ffffffff
  • Polynomial: 4c11db7
  • XOR Out: ffffffff
  • Uncheck Reflection In
  • Uncheck Reflection Out
  • Name: CRC-32/BZip2

Once saved, simply highlight the affected data in your .gci save file (i.e. 0x44 to the end of the file). Then, select the “Selection Only” option from the Checksum tool window, and the appropriate value should be generated in the CRC-32/BZip2 row.

Voila! You can now modify any data directly within the GCI file, regenerate the checksum (by selecting data from 0x44 to the end off the file), and then insert the new checksum (at 0x40).

Special thanks to James for figuring this out.

Tutorial: Extracting Raw Texture Data and Calculating Hash Values From TPL Files + Naming PNG Image Files As Per Dolphin Standards

I’ve been playing around with the “Dump Texture” feature in Dolphin recently.

When the emulator dumps textures, it creates a file name that includes a hash of the texture data. This hash uses the xxhash64 algorithm.

I’ve found that, using information about the TPL file format, I’m able to extract the raw image data from the TPL files.

Once I extracted this data (via manually copy/pasting in a hex editor), that I’m was able to feed it into the xxhash64 algorithm. I used QuickHashGUI for this.

To my surprise, this actually worked and the hashes matched!

Extracting the image data from the first image contained in the logo.tpl file (the Marvelous logo) and comparing hash values to the dumped Dolphin texture filename.

I’m going to try creating a QuickBMS script to extract raw image data from TPL files to somewhat automate the process.

Eventually this means that I should be able to generate hashed filenames for every texture on the game’s disc. This bypasses the need to typically play through and manually come across every texture in-game.


Update

I’ve created a QuickBMS script that can take any input TPL file, and output the extracted raw texture data.

endian big

idstring "\x00\x20\xAF\x30"

get filesize asize
get openedFile filename

get numberOfImages long
get imageTableOff long

if numberOfImages == 1
    goto imageTableOff
    get imageHeaderOff long
    goto imageHeaderOff
    get height short
    get width short
    get imageFormat long
    get imageAddress long
    
    set name string openedFile
    string name - ".tpl"
    string name + "_Texture0_"
    string name + width
    string name + "x"
    string name + height
    string name + "_"
    string name + imageFormat
    string name + ".bin"
    
    math filesize - imageAddress
    
    log name imageAddress filesize
else
    for i = 1 < numberOfImages
        goto imageTableOff
        get imageHeaderOff long
        get null long
        get nexImageHeaderOff long
        
        goto imageHeaderOff
        get height short
        get width short
        get imageFormat long
        get imageAddress long
        
        goto nexImageHeaderOff
        get null long
        get null long
        get nexImageAddress long
        
        set imageSize long nexImageAddress
        math imageSize - imageAddress
        
        set name string openedFile
        string name - ".tpl"
        string name + "_Texture"
        set texNum long i
        math texNum - 1
        string name + texNum
        string name + "_"
        string name + width
        string name + "x"
        string name + height
        string name + "_"
        string name + imageFormat
        string name + ".bin"
        
        log name imageAddress imageSize
        
        math imageTableOff + 0x08
    next i
    
    goto imageTableOff
    get imageHeaderOff long
    goto imageHeaderOff
    get height short
    get width short
    get imageFormat long
    get imageAddress long
    
    set name string openedFile
    string name - ".tpl"
    string name + "_Texture"
    math numberOfImages - 1
    string name + numberOfImages
    string name + "_"
    string name + width
    string name + "x"
    string name + height
    string name + "_"
    string name + imageFormat
    string name + ".bin"
    
    math filesize - imageAddress
    
    log name imageAddress filesize
endif

Once extracted, these files can be hashed with a tool (e.g. QuickHash GUI) using the xxHash64 algorithm.

Combining the data from the script with the hash will provide you with the full filename that Dolphin would normally generate.

Comparison of BMS-extracted texture data vs Dolphin-dumped files

Once you’ve determined the file hashes, you can extract the textures from your TPL as PNG files using a tool like BrawlBox or Wexos’s Toolbox.

The naming convention will be as follows:

  • Naming Type – denotes the texture naming scheme used by Dolphin, should always be “tex1”
  • width x height – dimensions of the original texture, can be found in the file output from the BMS script above
  • hash – xxHash64 of the raw original texture data (as output from the BMS script once fed through a tool such as QuickHash GUI)
  • format – original texture image format as named in output from the BMS script above
  • Output file extention – output image file, should always be “.png”

Here’s a visual demonstration of the file naming convention, and where the different parts can be found.

Extracting a texture to a png image using BrawlBox, using naming information provided by QuickBMS + QuickHash

Update #2

After going through every CLZ, ARC, and TPL file, I beleive I’ve been able to dump/rename (almost) every texture in the game.

There are a total of 4115 unique texture images from Harvest Moon: A Proud Life (lesbian version).

I’ve compiled all of these dumped textures into a Dolphin-compatible pack, which can be downloaded [here].

The Unwritten Rules of .mes Files

Last time I went over dialog, I detailed how to edit index entries.

Upon review, I’ve found that there are some additional “unwritten rules” on how .mes dialog files in A Wonderful Life are structured.

Index Entries Aligned as Hex “Words”

After looking at the index values in any given .mes file, I began to notice a pattern.

Note every dialog index ends in a 0, 4, 8, or c

Every single message starts at a position ending in a multiple of 4 in hex (i.e. 0, 4, 8, c).

At first I thought this was a weird coincidence. However, upon review I’ve determined that this is not on accident.

Different Types of Hex Grouping

There are different ways of grouping hex values.

Bytes: A pair of hex values
Words: 2 bytes
Double Words: 4 bytes

Message #1 in badog.mes, illustrated as bytes, words, and double words

What this means for MES files

It seems that .mes files are read in memory as “double words”. This means that a new message needs to start at a double word position (i.e. a hex position multiple of 4).

Every message needs at least one “00” end-of-message {EOM} byte to tell the system that it’s the end of that message. Additional EOM bytes are then added to pad out the message so that the next message in the file is pushed to start at the next double-word position.

This explains why some messages seem to have only 1 {EOM} byte, while some messages can have up to 4 {EOM} bytes.

File Endings

The Double Words pattern above still didn’t explain why the last messages in each dialog file seemed to have a random number of EOM bytes.

After reviewing the file endings, I found that the last byte position in any .mes file was at the “f” position on an odd line.
i.e. 1f, 3f, 5f, 7f, 9f, bf, df, ff

The last bytes for carter.mes, badog.mes, and bahn.mes are all at an odd _f position

This means that each file is padded out with EOM bytes until they reach a valid ending position.

Why bother with any of this?

These groupings are both seemingly due to how the game stores dialog in memory.

If dialog files weren’t configured properly with these rules, there is potential for dialog to overflow into other areas of memory (or other areas of memory to overflow into allocated “dialog space”) and wreak havoc.

This was evidenced during a recent Twitch stream, in which the dialog for Muffy (muumuu.mes) was improperly configured.

In this case, the game would experience numerous glitches following any interaction with Muffy (e.g. random words/phrases showing up where they shouldn’t, freezing, crashes, etc.).

To prevent any of these types of issues, dialog files must follow the above positioning rules.

A Brief Rundown of CLZ Files Thus Far

CLZ-Compressed files remain one of the biggest hurdles for the Proud Life mod.

For this reason, I’ve decided to document everything currently known about these files.

Archives vs. Compression

The easiest way to understand how the game handles compression is to understand the difference between “archiving” and “compression”.

Most compression tools on Windows will do both of these in the same step (e.g. .zip files).

However, archiving is the act of taking multiple files, and putting them into one file.

Compression is when this “archive” is compressed to save space.

Both A Wonderful Life and Another Wonderful Life separate these two steps by creating archives (in the form of U8 archive or .arc files), then compressing them (using clz compression).

Note how the U8 Archive is the same size as it’s parts (5+2+7 = 14) , and it’s the CLZ file that’s actually smaller in size

Essentially, we need to figure out how to undo the compression step, do whatever we want to the uncompressed data, then redo the compression step.

File Header

All CLZ files will follow a similar format.

  • File header (“CLZ”)
  • The size of uncompressed data in hex (twice). This is useful since we can use it to confirm if an attempt at uncompressing a CLZ file was successful or not.
  • Compressed data
In this example, an uncompressed mainchapter0.arc file should be 6.4MB

Compression Algorithm

While I haven’t been able to find a match for the compression used, the file extension alludes to it being a Custom Lempel-Ziv (CLZ) algorithm.

The files utilize a big endian byte encoding.

They seem to use some sort of dynamic dictionary generation. Which can be most seen when viewing the chapter arc.clz files. It’ll display the first occurrence of a pattern, but not subsequent occurrences. In this case _0.arc is listed for “boy0_0.arc” but not the files afterwards.

Note _o.arc only appears for the first entry

It seems to be generating some sort of dynamic dictionary.

Comparing Compressed / Decompressed Files

Certain data on the disc can be used to see both compressed and decompressed versions of some files.

The disc:\test\Scripts folder contains numerous script (.sb) files.

Of note, this folder also contains a U8 archive test.arc.
While this U8 archive is formatted in such a way that you can’t directly see the filenames, they can be viewed in a hex editor.

In this case, we can see that the first file in the U8 archive is a compressed CLZ file that, when decompressed, would yeild a sb script of 2a79 bytes in size.

By comparing the file size in the CLZ header, we can match up these CLZ files with the uncompressed sb files.

In this case, we can find that the first compressed “@” clz file in test.arc actually matches up with the uncompressed 0000_Animal_Navigation.sb (2a79 bytes)

I’ve managed to match up the compressed/uncompressed files (and have renamed them accordingly). They can be found [here] for comparison/research purposes.

The Search for Help

I’ve tried searching on various forums (e.g. ZenHAX, XeNTaX, and Romhacking.net), but haven’t really come up with much.

If anyone has any ideas on how to tackle this compression algorithm, it would be greatly appreciated.

Tutorial: How to Manage Dialogue Indexes with Excel

Background Info

One issue with how A Wonderful Life manages it’s dialogue is that it sorts each dialogue entry by indexes.

Each index value is a 4-byte hex value (also known as a double-word) that shows the place of the message in the file.

As an example, here’s nami.mes (dialogue for Nami):

In this case, Message #1 would be located at 0x000008d8, Message #2 would be located at 0x0000091c, etc.

This means, when we modify a dialogue entry manually in hex (e.g. substituting a word for a longer or shorter word), we need to fix the index accordingly.

Using the nami.mes dialogue file as an example, lets try changing the word “Blue” (hex “800c 80e3 80ec 80dc”) to “Purple” (hex = “801a 80ec 80e9 80e7 80e3 80dc”)

This is what we’ll get if we don’t change any of the indexes. Note that the {EOM} portion of the message gets pushed over to the next message

This can lead to a lot of deformed dialogue later on in the file.

Adjusting Indexes (the long way)

My initial method of fixing indexes was to go and manually fix the index of each of the following entries in the dialogue file.

In the above case, I need to increase every single index by a hex value of 4 (e.g. 0000091c + 4 = 00000920)

This can become extremely tedious and time-consuming, considering that some dialogue files will have hundreds of indexes you’d need to fix in this manner. But it works, and doesn’t require anything except a hex editor.

Adjusting Indexes (the Excel way)

An easier method of fixing indexes is to use a spreadsheet.

Requirements

Instructions

1.Open your dialogue file (e.g. nami.mes) in your hex editor.

2. Set it to use “Big Endian” byte order and group by “Double Words”

3. Select all of the indexes (starting at 0x00000008) and Copy

4. Open Notepad++ and paste the copied values
5. Open the Replace window (Search -> Replace…”
6. Set it to find ” ” (without quotes) and replace it with “\n”
7. Set the Search Mode to “Extended”
8. Click on “Replace All”

Replace all spaces ” ” with newline “\n”

9. Select your nice new list and Copy

10. Open up a blank spreadsheet in Excel
11. Select a cell and click Paste -> Use Text Import Wizard

12. The first 2 windows don’t really matter, but on the 3rd window, you’l want to make sure you select “Text” formatting.

13. Feel free to adjust your spreadsheet at this point. I added a row so I could keep track of what each column is, as well as a “Message #” column.
14. In a new column (beside your column of pasted indexes), type the following formula: =DEC2HEX(HEX2DEC(B2)+HEX2DEC(0),8)
Note, B2 is my first index value in my spreadsheet.

15. Select the bottom-right corner of your new formula cell and click+drag down to copy the formula to all subsequent cells.

16. Next, find the cell you need to start adding to, in this case we need to change our Message #2 (and every subsequent cell) by adding 4 to them.
17. Select the cell, and in the formula bar, change the second Hex2Dec entry from 0 to whatever you need to add/subtract. In my example case, I need to change to add 4. Then press enter.
18. Select the bottom-right corner of your new “fixed cell” and drag it all the way down to apply the +4 addition formula. You should end up with a full list of corrected indexes.

19. Select all of the entries in your new list and hit Copy.

20. Go back into your hex editor. Go to the first index location.
21. Right-click and Paste your new index values, overwriting the old ones.

22. Save
23. Open up your newly edited file in Harrison’s MES Editor to verify that everything looks good.

No more pushing the {EOM} bits to the next message. Yay!

24. Repeat as necessary with any other dialogue edits.

Edit:Technically, we’re not done yet!

Upon review, I’ve found there are a few more unwritten technical rules on how mes files and their indexes are structured.

Each edited message needs to be padded out with one to four “00”, a.k.a. {EOM}, bytes so that the next dialog message starts at a double-word position.

Each file also needs to have an appropriate amount of {EOM} bytes at the very end of the file so that it ends at an odd “_f” position.

Example Spreadsheet (Download)

Here’s an example spreadsheet that I used while making some edits to dialogue in numerous .mes files.

The End

As always, feel free to reach out if you have any questions on how to edit dialogue indexes using the above method.

We also have a great Discord community who I’m sure would be happy to offer any help if needed.

GPL Model Trial and Error and Error and Error

I tried running some of the .gpl files from A Wonderful Life through the previewer program on my iBook.

I attempted to load farm.gpl, since it’s texture configuration is one of the simplest (one tpl + one gpl) amongst the AWL gpl models.

Unfortunately, all of the gpl files from AWL caused a version-mismatch error and I’m not well-versed enough to recompile the program for mac at this point.

Error when attempting to load gpl files from AWL.
Occurs on both SheepShaver and iBook.

Fortunately, after examining the gpl files, I was able to find that the version number can actually be changed in a hex editor by setting the view options to display as decimal, group by double words, and use a big endian byte order. In this case, the version of the gpl was 6012001. I changed the version field to what previewerD requires (version 12012000), and reran previewer.

Changing the gpl library version from 6012001 to 12012000.

This allows the file to open, but leads to a new error:


GXSetVtxAttrFmt(): Only valid scale for 8b pos or texcoord data is 0 on HW1 in “GXGeoVert.c” on line 263.
Occurs on both SheepShaver and iBook.

This is likely due to the gpl being initially exported for HW2 (retail Gamecube) which uses an updated graphics driver compared to HW1 (initial dev kits). Since the mac version of previewer appears to be built on the HW1 code, it leads to the error above.

It might be possible to get around this error by running the HW2 version of previewer.elf via Dolphin (fan emulator) or Swiss (Gamecube homebrew loader), but it would take some time to figure out the proper configuration. Further research will be needed.


I also attempted to load a more complex 3d model (boy_0.gpl, extracted from boy_0.arc). This file has multiple textures associated with it (boy_0_b0.tpl, boy_0_f0_e.tpl, and boy_0_f0_m.tpl).

Unfortunately it seems that the gpl is improperly configured. When loaded through previewer on iBook, it looks for a single nonexistent boy_0.tpl texture.

It’s worth noting that the files from this version of boy_0.arc aren’t actually used in-game (the actual models/textures are loaded from one of the clz archives). These are likely leftovers from some point in the games development.

For now, I’ll be sticking to research on the simpler models first (e.g. farm.gpl, grass.gpl, and various fish models) to see if I can get past the HW1/HW2 limitations.


I also attempted to change encoding of farm.gpl from big to little endian (grouped by double-words).

I ran it through my Windows version of previewerD.exe (recompiled with version-check removed). However, I got a new “component count” error.

This is likely due to some sort of error when I attempted to convert the file’s endianness. Further research will be needed.

Tutorial: Dialogue Part II – Simple AWL Dialogue Editing

Once you’ve been able to read some dialogue files, it’s time to start editing any sections you want to change.

Locating Phrases

In this example, I’m going to alter the word “husband” in a phrase said by Griffin.

First things first.  You’re going to open up your .mes file in a hex editor of your choice.  I prefer Hex Editor Neo on Windows, but any hex editor should do.

Once open, you’ll want to locate the phrase in the file.  The easiest way to do this is to copy the hex code displayed in Harrison’s tool, then search for it in your file.

Replacing Words

For simplicity, it’s best to replace words with alternatives that have an equal or lesser amount of characters.

In this case, we’ll replace “husband” (7 letters) with “wife” (4 letters, 3 letters less than the original).  In this case we’ll need to add a few spaces onto the end of the line.  Spaces are encoded as “02″ in hex.  Since they only take up 1 hex slot as opposed to 2 (which most letters do), we’ll need to double the amount of needed spaces to 6 to make up for 3 missing letters.  So we’ll replace “a great husband for” with  “a great wife for      ”.  In the end, your replacement phrase should be just as long in hex as your original.  This keeps us from needing to modify the list of indexes at the start of the file and possibly creating any bugs.

While we might need to modify the file at times where the new text is longer than the original, we won’t worry about it right now.