Actor code, animation and matrices, RAM segments...

[xdaniel]

I'm no good with MIPS assembly at all and wouldn't even know where to start, so I'm hoping someone else here who's more versed in it can help me out...
 
Basically, I'm trying to get actors to render properly in a viewer, including ex. their limbs' connecting vertices being positioned correctly, if applicable. I know that actors that do have such smooth connections between their limbs, like most human(oid) actors, achieve this by translating/rotating certain vertices of one limb according to the translation/rotation of another limb. They get the other limb's modelview matrix from RAM segment 0x0D via a Mtx (0xDA) command near the start of their display list, then load the connecting vertices (thus positioning them using that matrix), then get their own matrix to finally position and render the rest of the vertices like normal.
 
Now, after much trial and error I've succeeded in correctly rendering the cow...
 

 
...but only the cow. Everything else is more nightmare fuel than actor...
 

 
I (most likely!) correctly create OpenTK-side matrices from each limb's/animation's translation and rotation values, relationships between childs, parents and siblings included, and I can create valid "N64-format" matrix data - there's not a single glitch with the cow, as far as I can see. I partially emulate the gSPMatrix / 0xDA Mtx command (not the projection stuff, not needed here; works for the cow and ex. Jabu-Jabu's wobbly walls, albeit rendering them stationary of course), I correctly transform the position of each vertex loaded according to the current modelview matrix (bless OpenTK's Vector3d.Transform). There's two big problems I'm seeing or anticipating:
 
Problem 1: At what address in segment 0x0D does each limb's matrix have to be stored? I'm currently writing each limb's matrix data into the segment successively, so that limb #1's is at 0x0, limb #2's at 0x40, limb #3's at 0x80, and so on. That's working fine for the cow, but for none of the other actors I've tried so far, like Sheik, Stalfos, Saria, young Zelda, Wallmaster and a few others.

Problem 2: I'm currently relying solely on the Mtx commands to try and position each limb correctly, however some actors don't appear to have any inside their display lists, like the Stalfos. How does the game render those? Or rather, how does the game know to render those differently from actors that do contain Mtx commands?

Well, by this point I'm stumped. I imagine I'd find some more answers if I look into the actors' or the game's code, but as mentioned above, I'm no good with MIPS at all. So I reluctantly have to ask: Has anyone else looked into any of this before, and has something to share about how this all works?

 

[SanguinettiMods]

 

Problem 2: I'm currently relying solely on the Mtx commands to try and position each limb correctly, however some actors don't appear to have any inside their display lists, like the Stalfos. How does the game render those? Or rather, how does the game know to render those differently from actors that do contain Mtx commands?

 

 

I know that the Stalfos doesn't contain matrices. If you were to null the DA (And D8?) Commands in Link's Hierarchy Display Lists, you would see in-game that his joints would not be connected. I don't think I'm really providing an answer to your problem, but if I'm understanding it correctly, actors are built soley on their Hierarchy and rely on animations to rotate the limbs. Matrices, at least, the ones that "fill" in joints don't position the display lists themselves. There are matrices, however, that do position certain display lists. An example of such can be found in MM Link's file, to rotate the Hero's Shield 180 degrees to use on his back, rather than a whole new display list.

 

 

Problem 1: At what address in segment 0x0D does each limb's matrix have to be stored? I'm currently writing each limb's matrix data into the segment successively, so that limb #1's is at 0x0, limb #2's at 0x40, limb #3's at 0x80, and so on. That's working fine for the cow, but for none of the other actors I've tried so far, like Sheik, Stalfos, Saria, young Zelda, Wallmaster and a few others.

 

I think I was told once that the matrix data was stored inside the actor's file; To be used later, but I haven't had much luck finding relative to where.

[xdaniel]

I know that the Stalfos doesn't contain matrices. If you were to null the DA (And D8?) Commands in Link's Hierarchy Display Lists, you would see in-game that his joints would not be connected. I don't think I'm really providing an answer to your problem, but if I'm understanding it correctly, actors are built soley on their Hierarchy and rely on animations to rotate the limbs. Matrices, at least, the ones that "fill" in joints don't position the display lists themselves. There are matrices, however, that do position certain display lists. An example of such can be found in MM Link's file, to rotate the Hero's Shield 180 degrees to use on his back, rather than a whole new display list.

From the looks of it, there are actors that do position the display lists via matrix operations, like the cow. Most (if not all) of its display lists first load the previous limb's matrix to position the joint's vertices, then they load the current limb's matrix to position the rest of the display list. If I'd now render it the "usual" way as done by ex. ZSaten, I'd get limbs that have been translated and rotated twice, once by the matrix command, and again by the viewer's hierarchy rendering, resulting in a sort-of exploded view of the actor.

 

That also doesn't cover actors that appear to be using both variants (with and without matrices) at the same time, like the bazaar guy (object_ossan). His torso and head do not appear to use matrices, while his arms do - rendering him by relying just on the matrices, the torso doesn't get transformed and his head is stuck to his right hand, the last matrix to be loaded before the head is supposed to be rendered, like so:

 

 

 

 

I think I was told once that the matrix data was stored inside the actor's file; To be used later, but I haven't had much luck finding relative to where.

But wouldn't that mean there's matrix data stored for each limb and each animation frame? That's something like (((0x40 bytes per matrix * limb count) * frame count) * animation count) I think, and a huge waste of space.

[xdaniel]

Well, EX-FUCKING-CUUUUUUUUSE ME, PRINCESS!

 

And for the moment, I'm so unbelievably sick of matrices, RAM segments, overlay files and all the other shit. Taking a bloody well-deserved break of this for a day or two or three.

[Twili]

Well, EX-FUCKING-CUUUUUUUUSE ME, PRINCESS!

 

[images]

And for the moment, I'm so unbelievably sick of matrices, RAM segments, overlay files and all the other shit. Taking a bloody well-deserved break of this for a day or two or three.

Zelda hacking accomplishment of the decade! Seriously! This is something that has been missing in viewers for years. We all appreciate the time you took to finally figure this out and I'm very interested in the details of how it works. DO WANT this program/source code.

[xdaniel]

Zelda hacking accomplishment of the decade! Seriously! This is something that has been missing in viewers for years. We all appreciate the time you took to finally figure this out and I'm very interested in the details of how it works. DO WANT this program/source code.

 

I guess program and source code are forthcoming, but don't expect clean code - the program's was was supposed to become "ZSaten2", but turned into a testbed of sorts when I started work on SorataVSE (see other threads). I will also document this properly, as opposed to just leave it at the code; here's the gist of it, as program/source release and proper documentation will take a bit longer:

 

The "magic" is basically creating the contents for RAM segment 0x0D at runtime, which holds the matrix data for each limb that has a display list attached. I'm preparing the segment by parsing the hierarchy in advance before actually rendering the model. The basic modelview matrix is identity multiplied with the global translation (the first 3 shorts in the rotation index data), then the limbs are parsed starting at #0 and address 0 in segment 0x0D.

 

The limb parsing function makes a local copy of the previously loaded matrix - so at limb #0 the aforementioned (identity * global translation), at the first child (identity * global translation * limb #0), etc., at the first sibling (identity * global translation), and so on - and then generates the current limb's matrix based on its translation and rotation values (depending on selected hierarchy, animation and frame number). Now, if the current limb has a display list to be rendered, it writes the current matrix to the current address in segment 0x0D and then increases the address by 0x40; if it's not supposed to be rendered, the matrix is not written to the segment, the address not increased. Lastly, if there is a child, the function recursively calls itself again, with this limb's matrix as a parameter, to parse the child, and if there's a sibling, it calls itself with the previous matrix as a parameter.

 

The matrix data's format as written to segment 0x0D is explained in the Functions Reference, on the gSPMatrix page.

 

Now the actor's display lists that use gSPMatrix (0xDA) commands - so those with joint vertices connecting two limbs - will happily read the matrix from the segment, which can then be used to transform the vertices loaded by gSPVertex (0x01) commands. My ucode interpreter library thingy has a matrix stack, which is filled/emptied by gSPMatrix and gSPPopMatrix, the matrix at the top of which is taken whenever a vertex gets loaded and applied to the vertex position.

 

What I'm also doing to get any actor display lists to render that do not contain gSPMatrix commands - ex. the whole Stalfos, Saria's body and head, Zelda's body, head and boots if I recall - is that, right before a display list is given to the interpreter library for parsing/rendering, the corresponding matrix is manually pushed onto the matrix stack. Display lists without gSPMatrix commands get the correct matrix, while those with them will get the limb's matrix, then (usually) directly load another limb's matrix for the joint vertices, then reload its own limb's matrix again for the rest of the display list. No need for manual glTranslate/glRotate/glPushMatrix/glPopMatrix in OpenGL, as used in ZSaten and likely every other viewer as well as described on the wiki, anymore.

 

Whew, hope that's making some sense already... again, program, source and some better documentation (probably with an example or so) coming a little later, next week or so at the latest - I hope.

[SanguinettiMods]

XDAN, XDAN, XDAN, YOU DID IT!

[xdaniel]

 

Download: http://magicstone.de/dzd/random/ZSaten2-POC.rar

 

Program & library binary (ZSaten2ZSaten2binRelease), program source. No library source yet, still very much WIP and will instead be released with SorataVSE in presumably a few months. Potentially buggy proof-of-concept, read and heed the Help/About message.

[Conker]

Amazing job, man!

[Zeth Ryder]

I cannot get it to run, just crashes. Guessing this is related to being on 64 bit OS?

[Librarian]

I cannot get it to run, just crashes. Guessing this is related to being on 64 bit OS?

Probably not, I have a 64bit OS (Windows 7) and I can run it perfectly fine. 

[SanguinettiMods]

I just played around with it a little bit, testing switching between Hierarchies and playing Animations. This works great! It didn't even lag for me.

[Mallos31]

That's really amazing! Works great on my computer. Will the new program be changing display lists when e.g. link pulls out a sword? 

[xdaniel]

I cannot get it to run, just crashes. Guessing this is related to being on 64 bit OS?

 

I'm actually developing on Windows 7 HP 64-bit, so that shouldn't be a problem. Are you getting an error message or something? If so, could you post any details etc.?

 

Now as for feature requests and such, I'll look into them when the time comes to integrate and clean up this code for use in Sorata. There's even code in already this that's not being used yet, like a C# "port-ish variant/rewrite" of spinout's mips-eval as used in ZSaten.

[Secant]

I don't normally get excited over just viewers, but holy damn xdaniel, this isn't "just" a viewer. Seeing actors rendered properly like this is quite a feat (I'm especially looking at Link), and it sounds like you've made a major contribution in terms of getting matrices down. As I've said in the past, you do damn good work and I'm very excited to see where this takes off.

 

Also, to address Zeth's problem, it's definitely not because of running a 64-bit OS, I'm running Windows 7 64-bit and this works fine without even needing to run in compatibility mode. Speaking of, the program works beautifully for what it's meant to be--I can't stress enough how awesome a job you're doing, man.

[Zeth Ryder]

I'm actually developing on Windows 7 HP 64-bit, so that shouldn't be a problem. Are you getting an error message or something? If so, could you post any details etc.?

 

Now as for feature requests and such, I'll look into them when the time comes to integrate and clean up this code for use in Sorata. There's even code in already this that's not being used yet, like a C# "port-ish variant/rewrite" of spinout's mips-eval as used in ZSaten.

Well ok, its not that but it doesn't give me an error except it would just close for no reason but now suddenly trying it today, it works! xD

[Twili]

[SanguinettiMods]

I did try to preview Dark Link and I ended up getting an error like the one in Twili's video above, however, I think this is because Dark Link requires extra files like link himself, and for example, He uses the file link_animetion for his animations. I just thought that might be something worth looking into.

[xdaniel]

Dark Link's an easy fix: http://magicstone.de/dzd/random/ZSaten2-POC-b.rar

 

if (ObjectFN.ToLowerInvariant() == "object_link_boy" || ObjectFN.ToLowerInvariant() == "object_link_child") goto dolink;

...became...

if (ObjectFN.ToLowerInvariant() == "object_link_boy" || ObjectFN.ToLowerInvariant() == "object_link_child" || ObjectFN.ToLowerInvariant() == "object_torch2") goto dolink;

 

[SanguinettiMods]

I don't really know where else to report problems if I find any, so I guess I'll do it here.

 

Another one I just found is object_ru2 and ovl_En_Ru2. The only display list that is rendered is her head but everything else is apparently invisible. I don't know if this is a problem with the file or if it is you program, especially because object_ru1/En_Ru1 works perfectly fine. :3

[Twili]

I see from the source code that it reads any rom and it seems as though MM may work. The only problem is that it needs filenames. Could you add support for offsets if a filename table is absent?

 

XXXXXXXX-YYYYYYYY

[xdaniel]

I see from the source code that it reads any rom and it seems as though MM may work. The only problem is that it needs filenames. Could you add support for offsets if a filename table is absent?

 

XXXXXXXX-YYYYYYYY

 

That might already work, actually... If no filename table has been found before reading the DMA table, it assigns a default name of "V<virtual start>_P<physical start>" (see DMATableEntry.cs, "Name = string.Format("V{0:X}_P{1:X}", VStart, PStart);"), both in hex, not prefixed and without leading zeroes, so something like ex. "VCA8930_PBAC0B0". Insert the correct values and try that as the filename.

[Twili]

That might already work, actually... If no filename table has been found before reading the DMA table, it assigns a default name of "V<virtual start>_P<physical start>" (see DMATableEntry.cs, "Name = string.Format("V{0:X}_P{1:X}", VStart, PStart);"), both in hex, not prefixed and without leading zeroes, so something like ex. "VCA8930_PBAC0B0". Insert the correct values and try that as the filename.

That is the correct way to do it, but it still gives an error because of the lack of a filename table and doesn't load anything.

[xdaniel]

That is the correct way to do it, but it still gives an error because of the lack of a filename table and doesn't load anything.

 

Ohhh, "Cannot load to segment by filename; no filename table found", I guess? Yeah, gonna change that and post another build later. Link still won't work without a filename table tho, as "gameplay_keep", "link_animetion" and "ovl_player_actor" are hardcoded, and I probably am not gonna change that; the program's meant to be a proof-of-concept after all, not an ongoing project as such.

[SanguinettiMods]

Right, since this is just a proof of concept; it's going to be implemented into SorataVSE, right?

 

As far as the loading Majora's Mask stuff, I probably did it wrong. I never got an error, I just never made it past the "load object filename" dialog box. :3

[xdaniel]

Exactly. Basically, what you have here in ZSaten2 will be a tab in Sorata, next to the scene and file system lists. You select an actor from the actor table, the program will load everything the actor needs, look for hierarchies and animations, etc., - pretty much like "ZSaten1", only with a proper GUI and eventually editing.