New Discord Server for Harvest Moon / Story of Seasons Translation Projects

It’s recently come to my attention that some other projects are now using the tutorials I’ve written on this blog. This was always my intent and the primary reason why I’ve been documenting all of my findings while researching AWL.

I’ve now created a [Harvest Moon Translations Discord] where people can collaborate and discuss translating AWL, as well as other HM/SoS games into various languages.

I hope that having a dedicated server will offer people a common place to help work on these projects, and get in touch with others who might have helpful expertise, whether it’s knowing another language, or technical knowledge about the game in question.

Currently I’ve created channels for a few AWL, FoMT, and DS projects, but more will be added as they come up.

If you or someone you know would be interested in helping translate some of these games, feel free to [join]!

Dialog Editing with Sukharah’s MES Unpacker

Introducing Sukharah’s MES Unpacker

After months of painstakingly editing dialog using a combination of Harrison’s MES Editor, Hex Editing, and Spreadsheets, I’m happy to say that we now have a much easier method for editing dialog (mes) files from Harvest Moon: A Wonderful Life and Another Wonderful Life.

Sukharah, the brilliant individual behind the CLZ Compression Tool, has created a new tool for manipulating MES files. This tool has been aptly named the MES Unpacker (although it can also repack).

Compiling the MES Unpacker

Prerequisites

The tool is released as source code, meaning you’ll need to have a C++ compiler such as g++ (included with MinGW, which can be installed using the mingw-get-setup.exe installer from the MinGW releases page.

Once you have MinGW/g++ installed, it’s just a matter of downloading the MES Unpacker repository, and building the program using your terminal of choice.

Compilation Options

There are 3 main modes you can use while compiling the MES unpacker.

The normal mode uses a fairly basic compilation command:

g++ -o mes.exe -std=c++11 -iquote header "source/main.cpp" "source/MES.cpp"

The next mode, known as ddupe, will allow the app to group sequential duplicate messages together in the final repacked .mes file. This is needed for certain dialog files (e.g. muumuu.mes) that approach the 73,376-byte file size limit. Attempting to load a .mes file larger than this in-game will result in a crash.

g++ -o mes_ddupe_seq.exe -DDUPE -std=c++11 -iquote header "source/main.cpp" "source/MES.cpp"

The final mode, known as ddupe2, will group together all duplicate messages from anywhere in the dialog file. This breaks compatibility with Harrison’s MES Editor, but should otherwise work in-game. It’s not advised to use this option unless you’re unable to get below the 73,376-byte limit after using ddupe.

g++ -o mes_ddupe2_nonseq.exe -DDUPE2 -std=c++11 -iquote header "source/main.cpp" "source/MES.cpp"

Precompiled MES Unpacker (Windows 10 only)

I have also created a compiled version. The limitation of this precompiled version is that it’s limited to Windows 10 systems, since that’s the type of system I compiled it on.

[Download]

mes.exe – Standard compilation

This version is recommended for almost all .mes editing use cases

mes_ddupe_seq.exe – Compiled with -DDUPE flag

This version is recommended for certain large .mes files that approach the 73,376-byte filesize limit.

e.g. muumuu.mes, son_talk.mes

It’ll group any sequential (appearing directly after one another) duplicate dialog messages together in the final output .mes file.

mes_ddupe2_nonseq.exe – Compiled with -DDUPE2 flag

This version is experimental. It’ll group any duplicate messages from anywhere in the dialog file.

This is only recommended if you’re having difficulty outputting a .mes file below the 73,376-byte file size after using mes_ddupe_seq.exe, which is possible if you plan on editing Muffy’s dialog (muumuu.mes).

Using Sukharah’s MES Unpacker

MES to TXT

Once you’ve compiled or downloaded the MES Unpacker, using it is fairly trivial.

Simply place your .mes file (e.g. badog.mes) in the same folder as your compiled program. Note this folder must also include the tool’s “data” folder and it’s contents.

Then, launch a command prompt in that folder. The simplest way to do this is by typing “cmd” in the navigation bar and pressing Enter.

From there, just type in the appropriate unpack command.

mes_program unpack file.mes file.txt

Where mes_program is either mes, mes_ddupe_seq, or mes_ddupe2_nonseq and file.mes is your dialog file.

Unpacking badog.mes and muumuu.mes using their respective builds.

Editing the TXT files

After you’ve run the commands, you should have some new text files in your folder.

You can edit these using any tool you like. Personally, I prefer Notepad++.

For this example, we’ll be replacing one of Dr. Hardy’s lines (badog.mes).

Simply locate your desired line in the appropriate text file, and edit it at will. Note that a single line (not counting actions like {PAUSE}, {WIPE}, {SOUND}, etc.) can only be 21 characters (letters) long. Anything beyond that will cause issues when displaying in-game.

TXT to MES

Once you’ve edited and saved the txt file, simply run the appropriate pack command.

mes_program pack file.txt file.mes

Where mes_program is your build of the tool, and file.txt is your edited dialog text file.

This will overwrite the mes file in your folder

You can then put the newly packed mes file back into your ISO or game folder, replacing the previous one.

From there, the game should load your new dialog in place of the old one.

And voila! You’ve successfully edited dialog.

The Unwritten Rules of .mes Files

Last time I went over dialog, I detailed how to edit index entries.

Upon review, I’ve found that there are some additional “unwritten rules” on how .mes dialog files in A Wonderful Life are structured.

Index Entries Aligned as Hex “Words”

After looking at the index values in any given .mes file, I began to notice a pattern.

Note every dialog index ends in a 0, 4, 8, or c

Every single message starts at a position ending in a multiple of 4 in hex (i.e. 0, 4, 8, c).

At first I thought this was a weird coincidence. However, upon review I’ve determined that this is not on accident.

Different Types of Hex Grouping

There are different ways of grouping hex values.

Bytes: A pair of hex values
Words: 2 bytes
Double Words: 4 bytes

Message #1 in badog.mes, illustrated as bytes, words, and double words

What this means for MES files

It seems that .mes files are read in memory as “double words”. This means that a new message needs to start at a double word position (i.e. a hex position multiple of 4).

Every message needs at least one “00” end-of-message {EOM} byte to tell the system that it’s the end of that message. Additional EOM bytes are then added to pad out the message so that the next message in the file is pushed to start at the next double-word position.

This explains why some messages seem to have only 1 {EOM} byte, while some messages can have up to 4 {EOM} bytes.

File Endings

The Double Words pattern above still didn’t explain why the last messages in each dialog file seemed to have a random number of EOM bytes.

After reviewing the file endings, I found that the last byte position in any .mes file was at the “f” position on an odd line.
i.e. 1f, 3f, 5f, 7f, 9f, bf, df, ff

The last bytes for carter.mes, badog.mes, and bahn.mes are all at an odd _f position

This means that each file is padded out with EOM bytes until they reach a valid ending position.

Why bother with any of this?

These groupings are both seemingly due to how the game stores dialog in memory.

If dialog files weren’t configured properly with these rules, there is potential for dialog to overflow into other areas of memory (or other areas of memory to overflow into allocated “dialog space”) and wreak havoc.

This was evidenced during a recent Twitch stream, in which the dialog for Muffy (muumuu.mes) was improperly configured.

In this case, the game would experience numerous glitches following any interaction with Muffy (e.g. random words/phrases showing up where they shouldn’t, freezing, crashes, etc.).

To prevent any of these types of issues, dialog files must follow the above positioning rules.

Opening Up Dialog for Community Collaboration

Dialogue Dumps

To help facilitate dialog review/editing, I’ve synced some of my work-in-progress dialog dumps via Google Drive.

This means that anyone interested in helping with the project can view .txt format dialog dumps, which will be continuously updated as I edit the dialog for the game.


Community Dialogue Review

I’ve also created some Google Docs listing any dialogue that I plan on editing, but haven’t gotten around to yet.

If you come across any dialogue in need of fixing (e.g. gender pronouns, grammatical/spelling mistakes), feel free to add it to these docs.

A screenshot of the Google Doc noting dialog from A Proud Life (A Wonderful Life) to be fixed

A Note About File Names

Note that the dialog file names are sometimes different from the in-game character names. This is because the files utilize the original Japanese character names.

English NameJapanese Name (name of dialogue file)
CeliaSepilia
ChrisKris
CodyGodey
FlakFlat
GalenGhali
GrantGurat
GriffinGarfun
HardyBadog
KasseyKesaran
MarlinMash
MuffyMuumuu
MurreyMoi
NakDavid
NicEbony
PatrickPasaran
RubyRou
SamanthaSan
TartanTsurutan
TimTai / Tay
VanBahn
WallySuary

Tutorial: How to Manage Dialogue Indexes with Excel

Background Info

One issue with how A Wonderful Life manages it’s dialogue is that it sorts each dialogue entry by indexes.

Each index value is a 4-byte hex value (also known as a double-word) that shows the place of the message in the file.

As an example, here’s nami.mes (dialogue for Nami):

In this case, Message #1 would be located at 0x000008d8, Message #2 would be located at 0x0000091c, etc.

This means, when we modify a dialogue entry manually in hex (e.g. substituting a word for a longer or shorter word), we need to fix the index accordingly.

Using the nami.mes dialogue file as an example, lets try changing the word “Blue” (hex “800c 80e3 80ec 80dc”) to “Purple” (hex = “801a 80ec 80e9 80e7 80e3 80dc”)

This is what we’ll get if we don’t change any of the indexes. Note that the {EOM} portion of the message gets pushed over to the next message

This can lead to a lot of deformed dialogue later on in the file.

Adjusting Indexes (the long way)

My initial method of fixing indexes was to go and manually fix the index of each of the following entries in the dialogue file.

In the above case, I need to increase every single index by a hex value of 4 (e.g. 0000091c + 4 = 00000920)

This can become extremely tedious and time-consuming, considering that some dialogue files will have hundreds of indexes you’d need to fix in this manner. But it works, and doesn’t require anything except a hex editor.

Adjusting Indexes (the Excel way)

An easier method of fixing indexes is to use a spreadsheet.

Requirements

Instructions

1.Open your dialogue file (e.g. nami.mes) in your hex editor.

2. Set it to use “Big Endian” byte order and group by “Double Words”

3. Select all of the indexes (starting at 0x00000008) and Copy

4. Open Notepad++ and paste the copied values
5. Open the Replace window (Search -> Replace…”
6. Set it to find ” ” (without quotes) and replace it with “\n”
7. Set the Search Mode to “Extended”
8. Click on “Replace All”

Replace all spaces ” ” with newline “\n”

9. Select your nice new list and Copy

10. Open up a blank spreadsheet in Excel
11. Select a cell and click Paste -> Use Text Import Wizard

12. The first 2 windows don’t really matter, but on the 3rd window, you’l want to make sure you select “Text” formatting.

13. Feel free to adjust your spreadsheet at this point. I added a row so I could keep track of what each column is, as well as a “Message #” column.
14. In a new column (beside your column of pasted indexes), type the following formula: =DEC2HEX(HEX2DEC(B2)+HEX2DEC(0),8)
Note, B2 is my first index value in my spreadsheet.

15. Select the bottom-right corner of your new formula cell and click+drag down to copy the formula to all subsequent cells.

16. Next, find the cell you need to start adding to, in this case we need to change our Message #2 (and every subsequent cell) by adding 4 to them.
17. Select the cell, and in the formula bar, change the second Hex2Dec entry from 0 to whatever you need to add/subtract. In my example case, I need to change to add 4. Then press enter.
18. Select the bottom-right corner of your new “fixed cell” and drag it all the way down to apply the +4 addition formula. You should end up with a full list of corrected indexes.

19. Select all of the entries in your new list and hit Copy.

20. Go back into your hex editor. Go to the first index location.
21. Right-click and Paste your new index values, overwriting the old ones.

22. Save
23. Open up your newly edited file in Harrison’s MES Editor to verify that everything looks good.

No more pushing the {EOM} bits to the next message. Yay!

24. Repeat as necessary with any other dialogue edits.

Edit:Technically, we’re not done yet!

Upon review, I’ve found there are a few more unwritten technical rules on how mes files and their indexes are structured.

Each edited message needs to be padded out with one to four “00”, a.k.a. {EOM}, bytes so that the next dialog message starts at a double-word position.

Each file also needs to have an appropriate amount of {EOM} bytes at the very end of the file so that it ends at an odd “_f” position.

Example Spreadsheet (Download)

Here’s an example spreadsheet that I used while making some edits to dialogue in numerous .mes files.

The End

As always, feel free to reach out if you have any questions on how to edit dialogue indexes using the above method.

We also have a great Discord community who I’m sure would be happy to offer any help if needed.

Poll: Which version of the mod are you most excited for?

Thus far, I’ve been working on both A Proud Life and Another Proud Life fairly equally.

As I continue my work on the mod, and focus in on certain aspects (e.g. dialogue), I found it would be best if I work on one game at a time.

This doesn’t mean that I’m abandoning either game, it’s just that I’d like to polish off one of the mods, then polish off the other.

I’m currently leaning towards Another Proud Life for a couple reasons.

  1. Dialogue: It’s easier to replace text with shorter words (e.g. replacing “she” with “he” in any dialogue where the main character is referred to).
  2. Textures: I also am able to edit at least the outfit textures, since those aren’t hidden in a clz archive.

As always, any input is appreciated.

Endian Differences between the Gamecube and PS2

Upon review, it seems that the Gamecube primarily stores data using big-endian (a type of data encoding), whereas the PS2 primarily stores data as little-endian.

This means that some game data (e.g. dialogue files) might appear to have their data “flipped” when comparing the GCN and PS2 versions.

As an example, a section of data might be “cdc3b0b0” for a file in the PS2 version, but “b0b0c3cd” in the GCN version.

This means that certain files were re-encoded when the game was ported to the PS2 to avoid them from being read as corrupt.

Shoutout to Dahni in the Discord chat for making this discovery.

It’s unclear at this time exactly which files were re-encoded. It seems that dialogue files were altered at the very least, but this could extend to other file types (textures, models, etc).

With that said, I’ll be focusing exclusively on the GCN versions of the game files to prevent any confusion of endian encoding going forward.

Tutorial: Dialogue Part II – Simple AWL Dialogue Editing

Once you’ve been able to read some dialogue files, it’s time to start editing any sections you want to change.

Locating Phrases

In this example, I’m going to alter the word “husband” in a phrase said by Griffin.

First things first.  You’re going to open up your .mes file in a hex editor of your choice.  I prefer Hex Editor Neo on Windows, but any hex editor should do.

Once open, you’ll want to locate the phrase in the file.  The easiest way to do this is to copy the hex code displayed in Harrison’s tool, then search for it in your file.

Replacing Words

For simplicity, it’s best to replace words with alternatives that have an equal or lesser amount of characters.

In this case, we’ll replace “husband” (7 letters) with “wife” (4 letters, 3 letters less than the original).  In this case we’ll need to add a few spaces onto the end of the line.  Spaces are encoded as “02″ in hex.  Since they only take up 1 hex slot as opposed to 2 (which most letters do), we’ll need to double the amount of needed spaces to 6 to make up for 3 missing letters.  So we’ll replace “a great husband for” with  “a great wife for      ”.  In the end, your replacement phrase should be just as long in hex as your original.  This keeps us from needing to modify the list of indexes at the start of the file and possibly creating any bugs.

While we might need to modify the file at times where the new text is longer than the original, we won’t worry about it right now.

Tutorial: Dialogue Part I – Viewing and Understanding AWL Dialogue Files

In this tutorial, I’ll review the dialogue files in A Wonderful Life, and how to open, edit, and save them.

Finding Dialogue Files

First things first, you’ll want to extract your dumped ISO file to an easy-to-find directory.

In the root of your extracted ISO, you’ll find several files ending in “.mes”. It’ll be easiest to sort by File Type to find these files.

Most of these files are named after the character that says the included dialogue (e.g. Nami’s lines are included in “nami.mes” and “nami_wife.mes”). Some characters are referred to by their Japanese names (e.g. Celia’s lines are included in “sepilia_wife.mes” and “sepilia_wife.mes”

Dialogue File Format

All dialogue files follow a similar format when viewed in a hex editor.

In this example, we’ll look at the “other.mes” file.

All dialogue files will start with a file header/identifier at 0x00000000 of “cd c3 b0 b0″.

At 0x00000008, there will be a list of indexes which denote where each “phrase” exists in the dialogue file.  In this example, the first phrase exists at 0x00000734, the second exists at  0x0000075c, and so on.

Opening Dialogue Files

For viewing dialogue, I recommend using Harrison’s MES Editor.

Despite the name, the tool can only view files (not edit) at the time of writing this tutorial.

Harrison’s tool will display all messages in both plain text and hex.  All characters (letters, numbers, etc.) in the game are encoded as hex values.  Below is the chart for these values used in AWL (note AnWL uses different values).

Going by the above chart, “He” would be converted to “8012 80dc″

Referencing this table will be integral to editing dialogue files later on.

Fortunately, Harrison’s tool can extract dialogue to easy-to-read plain text files which will make searching for phrases easier.  I’ve created dumps of all of the AWL and AnWL dialogue, which can be viewed at my A(n)PL GitHub repository.

APL/AWL Dialogue Offsets

I’m almost done adjusting the message offsets for all of the edited AWL files.  I just have a couple I need to run another pass or two on.

I should be able to finish those off tonight and will begin working on the AnWL files tomorrow.

I’ve been using the github repo as a way to easily mass-compare all of the files from their source contents to the edited versions.  This has helped me to easily identify which files need offset fixes and which can be ignored.

Now that I’m more aware of how editing text affects the offsets, I should be able to adjust the offsets at the same time as I make any further changes in the future to avoid having to go through this again.