N64/Zelda hacking tools (DList conversion etc)

[xdaniel]

You know what, Heavy? You just made my day - and it's only 1:32 am! Oh god, this is so awesome to see!

[HeavyZ]

I'm pretty stoked as well! Now we will have hardware compatible maps, and maybe models too!

[sairugoth]

HELL YAH! You guys did it!!!

[xdaniel]

I'd actually be willing to bet that spinout's converter worked perfectly fine all along, and all this is just my fault, having fucked up my code that was based on his working code.

 

Oh what the hell, here we go, my code's working fine now, too; now it's time for cleaning it up and improving conversion (everything's super-hacky right now), then making sure nothing will trip up the converter (ex. materials without texture maps), then adding features like user-changeable parameters for ex. backface culling, etc...

[Airikita]

Hooray! Can't wait, I am SO pumped to start working on things over the holidays. MERRY XMAS!

 

EDIT:

Can I have that awesome cube for my mod? I'll put a sign next to it commemorating this day.

[xdaniel]

SketchUp 8 file, http://magicstone.de/dzd/random/test.skp, and goodnight!

[DeathBasket]

Well I just saw xdaniel's tweet about this. Came here to say really good job on getting this to work! I hope in the future everything we do will actually work on hardware. That would really be something to look forward to.

[HeavyZ]

I actually don't even have Sketchup yet lol I used too on my old PC, and I still have Sharp Ocarina. I'll go to the Master tools and tutorials thread to refresh myself on how to do that so I can run the tests for that Sketchup 8 file. I ended up not going to work today becuase my alarm clock was just not effective, and I'm still sick, so I can test out some things earlier than usual

[xdaniel]

I'll be cleaning up the display list generation in a bit (ex. make the order of commands more akin to ex. object_timeblock and Bmori1_room_0), so there is a chance for breakage...

[Airikita]

The only -real- breakage with gi models is the size of the display list. Obviously more vertices will make the game crash, because I -think- I remember around 0x2000 in length was the limit, or that was the file I tested that was too big. Majora's Clock was a real squeeze.

 

I think those two will be fine, but more testing in the future will be needed.

 

I will be buying myself an ED64 as soon as I get the money for it.

[HeavyZ]

Oh I didn't know what that sketchup 8 file was until I opened it in sketchup, my bad haha. I thought it was something for more testing at first.

[xdaniel]

Haven't actually worked on the converter's code again yet, but have thought about user interaction, like how to specify what parts of a model should ex. have backface culling, where lighting should be enabled, etc. I was thinking, what about using the .obj format's groups for that purpose? More specifically: their names.

 

As with SO and Model2N64 previously, display lists are created by the converter on a per-group basis, so everything that's part of one group in the .obj file ends up in the same display list. Unlike SO and Model2N64 tho, this is a command line application, so user interaction is limited. Thus, to 1) avoid ridiculously long command lines (say, converter.exe test.obj -g "group name here" "nocull" -g "another group name here" "nocull lighting") and 2) use features already available in the .obj format and thus supported by modelling programs, I think specifying settings such as culling via the names of .obj groups is a good idea.

 

The way I'm currently planning for this to work is as follows, using SketchUp as the example: You group your model the way you need, then set the name of each group to the properties you want to set, delimited by underscores or spaces, like ex. "nocull lighting" (no quotation marks). Now, you export the model to .obj. SketchUp adds some stuff to the names you gave the groups, so they'll look like "Mesh1 nocull_lighting Model" - note that SketchUp appears to turn most non-alphanumeric characters into underscores, so you could technically say "nocull?lighting" and it'll work. Do use underscores or spaces, tho, also for compatibility with other modelling programs.

 

When the converter now begins to create a display list from a group, it looks at the group name, splits it at underscores and spaces (so the above example becomes a list of "Mesh1", "nocull", "lighting", "Model"), checks if the resulting strings are known properties (so the list is now "nocull", "lighting"), and then acts on each property by ex. noting that this display list shouldn't have culling enabled.

 

This might sound complicated, but I think it's pretty intuitive:

 

 

Note that this isn't fully implemented yet as it's still in the planning stages. The actual conversion process doesn't yet do anything with the information gathered; that's part of the de-suckification the currently rather hacky code will receive soon.

 

Underlying, preliminary, experimental code for parameter detection as of now for those who're curious:

 

 

class Attributes
{
	public bool NoCulling { get; set; }
	public bool Lighting { get; set; }
}

static Dictionary<string, Action<Attributes>> attribDict = new Dictionary<string, Action<Attributes>>()
{
	{ "nocull", new Action<Attributes>((attribs) => { attribs.NoCulling = true; }) },
	{ "lighting", new Action<Attributes>((attribs) => { attribs.Lighting = true; }) },
};

(... ... ... ...)

/* Get attribs from group name */
Attributes attribs = new Attributes();
List<string> attribList = group.Name.ToLowerInvariant().Split(new char[] { '_', ' ' }, StringSplitOptions.RemoveEmptyEntries).Where(x => attribDict.ContainsKey(x)).Distinct().ToList();
foreach (string attrib in attribList) attribDict[attrib](attribs);

 

Hope that makes sense so far!

[Airikita]

Is this specified in the sketchup models, or also part of any model in obj format?

[xdaniel]

Is this specified in the sketchup models, or also part of any model in obj format?

Part of the .obj format. That's why I said ".obj format's groups", ".obj groups", "features already available in the .obj format" and the like

[xdaniel]

Another patch for HW testing: http://magicstone.de/dzd/random/besitu-hacked-7.ppf - Been working on cleaning up the conversion code, proper mode settings, combiner setups, etc.; hope this didn't break anything!

 

 

(Note that Link will appear in front of the translucent cube regardless of position, a limitation of how I hacked the display list into the game...)

 

Speaking of the combiner, I really need to port more macros from the SDK to C#... SetCombineLERP makes generating combiner setups easy as pie (if you know what you're doing): GBI.SetCombineLERP("TEXEL0", "0", "SHADE", "0", "0", "0", "0", "TEXEL0", "COMBINED", "0", "PRIMITIVE", "0", "COMBINED", "0", "PRIMITIVE", "0"); - I believe there's more where that came from, ex. for SetRenderMode.

[Zeth Ryder]

BOOM!!

 

This is pretty exciting to see! Congrats to everyone for their hard work! Seriously keep it up!

[xdaniel]

Tsk, the moment I post, Zeth posts and pushes my post off to the previous page!

 

Another patch for HW testing: http://magicstone.de/dzd/random/besitu-hacked-7.ppf - Been working on cleaning up the conversion code, proper mode settings, combiner setups, etc.; hope this didn't break anything!

 

 

(Note that Link will appear in front of the translucent cube regardless of position, a limitation of how I hacked the display list into the game...)

 

Speaking of the combiner, I really need to port more macros from the SDK to C#... SetCombineLERP makes generating combiner setups easy as pie (if you know what you're doing): GBI.SetCombineLERP("TEXEL0", "0", "SHADE", "0", "0", "0", "0", "TEXEL0", "COMBINED", "0", "PRIMITIVE", "0", "COMBINED", "0", "PRIMITIVE", "0"); - I believe there's more where that came from, ex. for SetRenderMode.

EDIT: Another patch! http://magicstone.de/dzd/random/besitu-hacked-8.ppf

 

Also updated the DL comparison table a bit, including the above patch 8: http://magicstone.de/dzd/random/DLCompare.htm

Edited December 19, 2013 by xdaniel

[HeavyZ]

Going to test those patches soon xdan I gotta run somewhere and I'll be back in literally about 12mins, then I'll get straight to testing!

 

edit:

On patch 7 the model seems is a semi white clearish? square has the model inside it, and the model is white now too with no textures. I can walk through both the big square and the model, although I could walk through the model before.

 

Ptach 8 has the textures the big square that has the model inside is now orange and clearish, and the model has its textures again

[xdaniel]

On patch 7 the model seems is a semi white clearish? square has the model inside it, and the model is white now too with no textures. I can walk through both the big square and the model, although I could walk through the model before.

 

Ptach 8 has the textures the big square that has the model inside is now orange and clearish, and the model has its textures again

Yeah, this not having proper collision is normal; the converter's just a simple model converter for now, not a map creation tool like SharpOcarina, so it doesn't create collision, etc. at all.

 

So, it sounds like patch 8 looks like it's supposed to on hardware? Like the screenshot I posted with patch 7? (Model itself was identical between the two, just the way it was converted differs, mainly regarding rendering modes and combiner stuff)

[HeavyZ]

Yeah it's identical to the screenshot you posted and doing what it should be. My phone no longer has service for a while, and is not recognized by my PC or I would post more pics

Edit:

Although if you would really like pics now or sometime before I get my service back on I could always go down the street to my fathers and use his Iphone

[xdaniel]

Oh, don't worry. At this point pics would be most important when something glitches on hardware, so seeing how patch 8 did work fine, I don't think pics are needed right now

 

Alright, doing more cleanup work, etc. tomorrow. Not sure if I'll be able to get a release of the converter out before or during Christmas - will be with my family over the holidays, and while I will bring my laptop, I don't think I'll be using it much -, early to mid January should be realistic, tho.

[HeavyZ]

That's awesome news! and do you already have any plans for Sharp Ocarina, or anything regarding custom maps in future? I could always test collision!

[xdaniel]

No exact plans as such. Obviously this thing could be extended into, or used as the base for a map importer, but that's something to think about after getting it polished up and released...

 

That said, I do kinda have the idea that, if I would spin this off into a custom map tool, I'd leave it command line-based as well, and it probably would not inject maps into ROMs, either (= just create separate scene and room files). That said, it would allow the user to specify how many ex. actors, transitions, waterboxes, etc. they want via some command line switches or something, it would insert some default values for those into the file(s), which could then be edited in ex. SceneNavi like any other map. Something like that.

 

Again, if that happens, it's still quite some time away. First things first, that being the simple model converter it's planned as right now.

[Airikita]

Another patch for HW testing: http://magicstone.de/dzd/random/besitu-hacked-7.ppf - Been working on cleaning up the conversion code, proper mode settings, combiner setups, etc.; hope this didn't break anything!

 

 

(Note that Link will appear in front of the translucent cube regardless of position, a limitation of how I hacked the display list into the game...)

 

Speaking of the combiner, I really need to port more macros from the SDK to C#... SetCombineLERP makes generating combiner setups easy as pie (if you know what you're doing): GBI.SetCombineLERP("TEXEL0", "0", "SHADE", "0", "0", "0", "0", "TEXEL0", "COMBINED", "0", "PRIMITIVE", "0", "COMBINED", "0", "PRIMITIVE", "0"); - I believe there's more where that came from, ex. for SetRenderMode.

Are you saying that the cube will never cover Link, but just grow larger and larger as Link approaches it?

[Arcaith]

He means the zbuffer always draws Link after drawing the translucent cube, so even if you're inside the cube's interior space, it'll still draw Link as if he's standing outside of it. I had the reverse problem with maps imported with SharpOcarina on hardware, it was always drawing Link first, so I could never actually see where he was.