Tutorial: How to Manage Dialogue Indexes with Excel

Background Info

One issue with how A Wonderful Life manages it’s dialogue is that it sorts each dialogue entry by indexes.

Each index value is a 4-byte hex value (also known as a double-word) that shows the place of the message in the file.

As an example, here’s nami.mes (dialogue for Nami):

In this case, Message #1 would be located at 0x000008d8, Message #2 would be located at 0x0000091c, etc.

This means, when we modify a dialogue entry manually in hex (e.g. substituting a word for a longer or shorter word), we need to fix the index accordingly.

Using the nami.mes dialogue file as an example, lets try changing the word “Blue” (hex “800c 80e3 80ec 80dc”) to “Purple” (hex = “801a 80ec 80e9 80e7 80e3 80dc”)

This is what we’ll get if we don’t change any of the indexes. Note that the {EOM} portion of the message gets pushed over to the next message

This can lead to a lot of deformed dialogue later on in the file.

Adjusting Indexes (the long way)

My initial method of fixing indexes was to go and manually fix the index of each of the following entries in the dialogue file.

In the above case, I need to increase every single index by a hex value of 4 (e.g. 0000091c + 4 = 00000920)

This can become extremely tedious and time-consuming, considering that some dialogue files will have hundreds of indexes you’d need to fix in this manner. But it works, and doesn’t require anything except a hex editor.

Adjusting Indexes (the Excel way)

An easier method of fixing indexes is to use a spreadsheet.

Requirements

Instructions

1.Open your dialogue file (e.g. nami.mes) in your hex editor.

2. Set it to use “Big Endian” byte order and group by “Double Words”

3. Select all of the indexes (starting at 0x00000008) and Copy

4. Open Notepad++ and paste the copied values
5. Open the Replace window (Search -> Replace…”
6. Set it to find ” ” (without quotes) and replace it with “\n”
7. Set the Search Mode to “Extended”
8. Click on “Replace All”

Replace all spaces ” ” with newline “\n”

9. Select your nice new list and Copy

10. Open up a blank spreadsheet in Excel
11. Select a cell and click Paste -> Use Text Import Wizard

12. The first 2 windows don’t really matter, but on the 3rd window, you’l want to make sure you select “Text” formatting.

13. Feel free to adjust your spreadsheet at this point. I added a row so I could keep track of what each column is, as well as a “Message #” column.
14. In a new column (beside your column of pasted indexes), type the following formula: =DEC2HEX(HEX2DEC(B2)+HEX2DEC(0),8)
Note, B2 is my first index value in my spreadsheet.

15. Select the bottom-right corner of your new formula cell and click+drag down to copy the formula to all subsequent cells.

16. Next, find the cell you need to start adding to, in this case we need to change our Message #2 (and every subsequent cell) by adding 4 to them.
17. Select the cell, and in the formula bar, change the second Hex2Dec entry from 0 to whatever you need to add/subtract. In my example case, I need to change to add 4. Then press enter.
18. Select the bottom-right corner of your new “fixed cell” and drag it all the way down to apply the +4 addition formula. You should end up with a full list of corrected indexes.

19. Select all of the entries in your new list and hit Copy.

20. Go back into your hex editor. Go to the first index location.
21. Right-click and Paste your new index values, overwriting the old ones.

22. Save
23. Open up your newly edited file in Harrison’s MES Editor to verify that everything looks good.

No more pushing the {EOM} bits to the next message. Yay!

24. Repeat as necessary with any other dialogue edits.

Edit:Technically, we’re not done yet!

Upon review, I’ve found there are a few more unwritten technical rules on how mes files and their indexes are structured.

Each edited message needs to be padded out with one to four “00”, a.k.a. {EOM}, bytes so that the next dialog message starts at a double-word position.

Each file also needs to have an appropriate amount of {EOM} bytes at the very end of the file so that it ends at an odd “_f” position.

Example Spreadsheet (Download)

Here’s an example spreadsheet that I used while making some edits to dialogue in numerous .mes files.

The End

As always, feel free to reach out if you have any questions on how to edit dialogue indexes using the above method.

We also have a great Discord community who I’m sure would be happy to offer any help if needed.

Tutorial: Dialogue Part II – Simple AWL Dialogue Editing

Once you’ve been able to read some dialogue files, it’s time to start editing any sections you want to change.

Locating Phrases

In this example, I’m going to alter the word “husband” in a phrase said by Griffin.

First things first.  You’re going to open up your .mes file in a hex editor of your choice.  I prefer Hex Editor Neo on Windows, but any hex editor should do.

Once open, you’ll want to locate the phrase in the file.  The easiest way to do this is to copy the hex code displayed in Harrison’s tool, then search for it in your file.

Replacing Words

For simplicity, it’s best to replace words with alternatives that have an equal or lesser amount of characters.

In this case, we’ll replace “husband” (7 letters) with “wife” (4 letters, 3 letters less than the original).  In this case we’ll need to add a few spaces onto the end of the line.  Spaces are encoded as “02″ in hex.  Since they only take up 1 hex slot as opposed to 2 (which most letters do), we’ll need to double the amount of needed spaces to 6 to make up for 3 missing letters.  So we’ll replace “a great husband for” with  “a great wife for      ”.  In the end, your replacement phrase should be just as long in hex as your original.  This keeps us from needing to modify the list of indexes at the start of the file and possibly creating any bugs.

While we might need to modify the file at times where the new text is longer than the original, we won’t worry about it right now.

Tutorial: Dialogue Part I – Viewing and Understanding AWL Dialogue Files

In this tutorial, I’ll review the dialogue files in A Wonderful Life, and how to open, edit, and save them.

Finding Dialogue Files

First things first, you’ll want to extract your dumped ISO file to an easy-to-find directory.

In the root of your extracted ISO, you’ll find several files ending in “.mes”. It’ll be easiest to sort by File Type to find these files.

Most of these files are named after the character that says the included dialogue (e.g. Nami’s lines are included in “nami.mes” and “nami_wife.mes”). Some characters are referred to by their Japanese names (e.g. Celia’s lines are included in “sepilia_wife.mes” and “sepilia_wife.mes”

Dialogue File Format

All dialogue files follow a similar format when viewed in a hex editor.

In this example, we’ll look at the “other.mes” file.

All dialogue files will start with a file header/identifier at 0x00000000 of “cd c3 b0 b0″.

At 0x00000008, there will be a list of indexes which denote where each “phrase” exists in the dialogue file.  In this example, the first phrase exists at 0x00000734, the second exists at  0x0000075c, and so on.

Opening Dialogue Files

For viewing dialogue, I recommend using Harrison’s MES Editor.

Despite the name, the tool can only view files (not edit) at the time of writing this tutorial.

Harrison’s tool will display all messages in both plain text and hex.  All characters (letters, numbers, etc.) in the game are encoded as hex values.  Below is the chart for these values used in AWL (note AnWL uses different values).

Going by the above chart, “He” would be converted to “8012 80dc″

Referencing this table will be integral to editing dialogue files later on.

Fortunately, Harrison’s tool can extract dialogue to easy-to-read plain text files which will make searching for phrases easier.  I’ve created dumps of all of the AWL and AnWL dialogue, which can be viewed at my A(n)PL GitHub repository.

AnWL Dialogue Dumped

Harrison has been able to implement a switch into the MES Reader program to read files as AnWL format (different hex cipher than AWL). [link to new version]

This means I’ve been able to fully dump the AnWL dialogue files. [link]

As I’ve said before, I’ll probably start with basic gendered terms (husband/wife, mom/dad, he/she, etc.).  If you know of any specific dialogue lines that should be changes, please let me know. [contact link]


I’ll basically just be able to work with the dialogue until I can decipher the CLZ archive format that contain a lot of the game’s data (including some character models/textures).

I might try and consult some of the folks over on either the AssemblerGames or GBATemp forums for some help.

Worst case scenario, I might even try to get in contact with someone from the original team over at Natsume to see if they’d be willing to provide any insight into the file format spec.

AWL Dialogue Dumped, AnWL Dialogue Cipher Broken

The MES Editor is now at v0.2 [link] and now includes export features.

With this, I’ve been able to fully dump all of the dialogue from AWL. [link]

The dialogue files can be viewed for anyone interested. [link].

The only things that aren’t dumped from AWL are some kanji characters that haven’t been added to the tool yet. But this is being worked on.  I’ve been progressing through each of the characters from the font texture.

The red characters are those that I haven’t been able to identify.  If anyone knows what they are, please let me know so we can add them to the program.

I’ve just broken the AnWL hex cipher, so I should hopefully be able to coordinate with Harrison to create an alternate version of the Editor for AnWL MES files.

If you notice any dialogue lines in-game that need to be changed, please let me know so I can find and edit them.

I’ll start by looking for individual gender identifiers (he/she, husband/wife, boy/girl. mom/dad, etc) and will go from there.  But it will likely be a little slow without full context of the conversations.

Dumping Dialogue

I’ve had a good amount of success with Harrison’s MES Editor [v0.1 stable]

I’ve set up a google doc [link] where I’ve been dumping all of the decrypted tool.

I’ll continue dumping messages manually and then will look into adjusting dialogue (i.e. pronouns, etc) for the mod once all of the scripts are in one place

I’ve made the google doc link editable (I have a seperate version for myself as a backup), so if you want to use the tool and contribute, it would be extremekly appreciated.

Also, if you can find any areas of dialogue in either game that will require adjusting for the Proud Life mod, please let me know so I can make a note of it.

Conversations with Harrison have been very productive around the tool.  Currently, he’s updating the tool with addition characters (symbols, kanji, etc.), as well as refactoring chunks of the code.  Most excitingly, the new development version seems to allow dumping of full decrypted output to a file, which will make things a lot easier rather than me copy/pasting each individual message.  Currently this new version is still under development and won’t really display what you want it to.  But it should be ready in the coming weeks. 

Further Examination of CLZ archives, Discovery of MES Reader, and Personal Stuff

At this point I’m just trying to understand the CLZ header.  So far I’ve noticed a trend between the varying files.

I’ll be using mainchapter0.arc.clz for this example

Offset   |   Size   |   Description
0x00   |   4 bytes   |   CLZ file identifier [43 4c 5a 00]
0x04   |   4 bytes   |   unknown variable [00 63 ad 40]
0x08   |   4 bytes   |   zero space [00 00 00 00]
0x0c   |   4 bytes   | unknown variable, repeats that of 0x04 [00 63 ad 40]
0x10   |   1 byte   |   zero space [00]
0x11   |   variable amount   |   First file in clz archive, in this case a U8 arc file [55 aa 38 2d …]

The main differentiator seems to be the section at 0x04/0x0c.  My best guess is this likely references the file size and/or the number of files contained in the clz archive and/or the offset(s) of said files.


In other news, I’ve found a MES Editor Reader (courtesy of Harrison’s seemingly abandoned translation project).

I’ve emailed Harrison to see if he has any additional insights into the mes format (i.e. encoding formats) so that I might be able to manually decode/encode text changes for pronouns, etc.  Here’s hoping that he’s still using the same email 5 years after posting the tool.


Btw if you’re wondering why progress has been a little slow, here’s your reason.

Update: Harrison has replied

Hey Sean! Exciting – I’m assuming you’re working to fix the typos and reversed yes/no options that are throughout the game? That’s what my goal ended up being, but I quickly reached the the limits of my programming knowledge.

Check out main.pas file on GitHub – at the top, I have an array named CharMap that maps out all the characters that are used. Based on this, I think they’re stored as two bytes from 0x8000 to 0x81FF. Near the bottom, within the function “ConvertFromHM”, you’ll see some of the other characters that I found that are used for page breaks, character names, and other inserts.

Let me know how it goes! If you find out any additional special characters let me know, I wouldn’t be averse to playing around with finishing this tool sometime.

With this info, I should have the basic character information to decode/encode basic text for dialogue.

Going through all of the dialogue to find pronouns might take some time though, so if you see anything that should be changed in either game, let me know.