Universal Model Converter (UMC) Development thread - Deceased

Tcll · Jan 14, 2011

I've been working on this program for a while now, and am trying to make a release :/

Universal Model Converter does just as it states...
converting between any file format.

it converts between the formats using format-specific plugins
(similar to the plugin feature used in Winamp)

the formats I'm working on for the release version are:
MDL0 (brawl)
DAT (melee)
PMD (MMD)
-
BLEND (Blender 2.49)
3DS
-
RAW (export only)
OBJ (export only)

the conversion process is simple (for users)
1: open file to read
2: save file to convert to
(you're done)

a future thing for me to work on is batch operations :/

and also, a small bug IDK how to fix yet is you have to type in the file format :/

I'm not the best of all proggers, but I'm learning...

Q/A:
why isn't this program released yet?
the answer is I've not gotten any farther than converting the vectors... (verts, normals, and uv's)
I am UD-ing progress on my forum: http://tcll5850.proboards.com/index.cgi?board=python&action=display&thread=74&page=1
(you must be a member to view) :/

here's further documentation on my program:

MDL0 format:
figured out: (converted)
verts
normals
uv's

to figure out:
faces (objects) (face-points)
bones (figured out, not converted yet)
weights
materials (mat2)
shaders (mat1)
texture references (tex1)
pallet references (tex2)

I've been working on trying to figure out the MDL0 format for almost half a year now...
it's a shame I've only gotten this far -_-*

if you have any Q's, feel free to ask

also, I don't want any fame for doing this... (like fans and stuff)
I just want to be widely known

I only do this stuff for fun is all 8)

EDIT:
for further notice:
I will definatly be multi-posting here

EDIT: boy do I need to update this OP or what XD
man was I such a posting noob XDD

anyways...
UMC 3x dev5 is my current project...
dev4, my last release...
both of which completely overtop and knock down this version.

dev5 is a scrapped/rebuilt version of dev4, so it has many mew features and upgrades (including it's own GUI), and may also get beefed up to an editor by dev6.
(the official release will be able to do alot more for your scripts and libraries for UMC)
^more on that when I redo this post

EDIT2: meh...
actually, due to the old posts on here from past progress...
I'm somewhat thinking of creating a new thread and just linking this one to it in the op of that thread >_>

Tcll · Jan 14, 2011

I've gotten help with this post:

hey...
anyone know anything about materials other than this:

here's a single material: (FitPikachu_main)

00 00 02 00 FF FF 7E 40 00 00 00 00 03 00 00 00
00 FF FF FF FF FF FF FF 00 00 00 00 00 00 00 00
61 FE 00 00 0F 61 F6 00 00 04 61 FE 00 00 0F 61
F7 00 00 0E 61 FE 00 00 0F 61 F8 00 00 00 61 FE
00 00 0F 61 F9 00 00 0C 61 FE 00 00 0F 61 FA 00
00 05 61 FE 00 00 0F 61 FB 00 00 0D 61 FE 00 00
0F 61 FC 00 00 0A 61 FE 00 00 0F 61 FD 00 00 0E
61 27 FF FF FF 00 00 00 00 00 00 00 00 00 00 00
61 FE FF FF F0 61 F6 E3 78 C0 61 28 03 F0 40 61
C0 28 F8 AF 61 C2 08 FE B0 61 C1 08 F2 F0 61 C3
08 1F F0 61 10 00 00 00 61 11 00 00 00 00 00 00
61 FE FF FF F0 61 F7 00 38 C0 61 29 3B F3 BF 61
C4 08 06 EF 00 00 00 00 00 61 C5 08 1F F0 00 00
00 00 00 61 12 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

block size
MDL0 offset
ID
unk

the data is in Materials2 (materials)

Tcll · Jan 15, 2011

well I spent all night last night trying to figure out more info while working on the GUI...

here's an image of the finished GUI:

^actual size =D

anyways...
the GUI istelf is usless...
but I couldn't get rid of the blank window that kept popping up,
so I made use of it

Dantarion · Jan 15, 2011

How far are you along?
Have you figured out Melee animation format?

Tcll · Jan 15, 2011

Dantarion said:
How far are you along?
Have you figured out Melee animation format?

umm...
I havn't really looked into the animation formats...
but I know it's in Pl**Aj.dat, and it's composed of multiple dat files
^(the data is stacked)

I'm just working on the model format as of right now...
I'll prbly UG this converter for animations... (actually put some use to the GUI)
here's a few formats I know of:
dat (melee)
chr0 (duh)
VMD (MMD)

there's more, but I'd have to see them again :/

Tcll · Jan 15, 2011

btw, here's my UD'd tmp format:

Code:

< vertices name X,Y,Z/X,Y,Z >
< normals name X,Y,Z/X,Y,Z >
< uvs name X,Y/X,Y >
< faces name vnu points lines line_strips tri's tri_strips tri_fans quads >
points: v,n,u/v,n,u
lines: v,n,u|v,n,u/v,n,u|v,n,u
line_strips: v,n,u|v,n,u|v,n,u|v,n,u|v,n,u/
tri's: v,n,u|v,n,u|v,n,u/v,n,u|v,n,u|v,n,u
tri_strips: v,n,u|v,n,u|v,n,u|v,n,u|v,n,u|v,n,u/
tri_fans: v,n,u|v,n,u|v,n,u|v,n,u|v,n,u|v,n,u|v,n,u/
quads: v,n,u|v,n,u|v,n,u|v,n,u/v,n,u|v,n,u|v,n,u|v,n,u
 
< bone bone_name parent,child(grid) is_connected(bool) locX,Y,Z rotX,Y,Z scaX,Y,Z bind inv_bind >
if bone has no parent:
< bone Top 0,0 0 0.0,0.0,0.0 0.0,0.0,0.0 1.0,1.0,1.0 1,0,0,0|0,1,0,0|0,0,1,0 1,0,0,0|0,1,0,0|0,0,1,0 >
 
< weights ... > ;unfinished
< object name parent verts,normals,uv's,faces bones > ;unfinished
< material name > ;unfinished

ignoring the very bottom block...
is there anything needed that I'm missing??
eg: anything in bones...

note:
the < and > are useless...
you could put any char in that area but space...
they just make the data easier to read

Kryal · Jan 15, 2011

Lots of good information can be found on the Dolphin EMU SVN on how the GPU registers are set up and emulated. The Revolution SDK fills in most of the rest. I cringe at giving out links in forums, but if you google these two you'll find good links at the top.

The GPU operates very closely to OpenGL 2.1, but the data files are really just storage (except for display lists). The developer tools used to create these games use built-in libraries that decode and transfer the data to the GPU for you. These tools are based on Maya 7.

Most of the features in modern 3D programs either don't exist or are watered down on the Wii. A good example is anim interpolation, which uses only hermite point-to-point smoothing. When converting model files, you may find that some features are rather simple to export. When importing, many features will either need to be ignored or interpreted for the Wii.

Tcll · Jan 15, 2011

Kryal said:
Lots of good information can be found on the Dolphin EMU SVN on how the GPU registers are set up and emulated. The Revolution SDK fills in most of the rest. I cringe at giving out links in forums, but if you google these two you'll find good links at the top.

I had the 3089 SVN src once...
I'd still have it now, but it was eaten by my cpu (HDD went RAW) and I havn't DL'd it since...
that stuff is complicated as heck o.O

I tried to understand how models were formatted or read on the GCN...
(didn't work out)

I do at least have the YAGCD :/
that's helped me out with understanding alot of stuff about the GCN,
but theres still alot more to go -_-

well...
I guess I'll go search up those things then

The GPU operates very closely to OpenGL 2.1, but the data files are really just storage (except for display lists). The developer tools used to create these games use built-in libraries that decode and transfer the data to the GPU for you. These tools are based on Maya 7.

Most of the features in modern 3D programs either don't exist or are watered down on the Wii. A good example is anim interpolation, which uses only hermite point-to-point smoothing. When converting model files, you may find that some features are rather simple to export. When importing, many features will either need to be ignored or interpreted for the Wii.

lol... you've successfully made me feel like a noob

I understand what you're saying above

if so, would you mind if I asked you for things when I needed them??

you don't have to provide any resources or anything if you don't want to,
and I usually ask for simple things that the user knows or may know...

like one thing right now,
how do the "nodes" work or what do they do??
(the nodes just under the MDL0 header)

I know they handle the weights, but that's all I know...

also, with me, can you try to speak with hex... plz...
I tend to understand things alot easier that way :/

EDIT:
oh btw, "Shaders" don't sound like a good name... :/
just sayin... from what they do ("define" shaders)
do you think "Settings" would be a better name for them??
since they are basically a configuration like ini to exe :/

Tcll · Jan 15, 2011

oh hey...
here's another one for ya...
this in I believe to be more right on...

you know the Tex1 and Tex2 right...
well, I've only seen Tex2 when the mdl0 was linked to images that had pallets
(I've been looking into this one for a while (with different mdl0's))

I think the Tex2 is actually a pallet :/

IDK if you've looked into it or what not, but I have...

hope it helps

Tcll · Jan 16, 2011

so here's a layout of everything dealing with conversion from MDL0:
(green = fully working, red = not working)

Verts
Normals
UV's
Bones (grid_positioning problem (parent/child))
Weights (unfinished formatting)
Faces (not understanding the face-points)
texture ref
Materials/Shaders (extended from the shader config)
^(thanx to Kryal for correction and a little clarification on that)

also...
what are the definitions anyways :/
what do they do??

Tcll · Jan 18, 2011

do you guys want this converter or not??

0% progress made sinde my last post

I'm out of resources:
nobody wants to tell me how their code works...
other people I show the src's to are clueless and have no idea WTF I want to do...
documentation sucks...
and the people that actually know something are selfish and/or rude as heck... >:/
(excluding RTB (still need to ask him))

I post info about what I know all over the net...
why can't other people do the same >:/

I mean... there are other people that want to learn this stuff... (not just me)

uugh...
sry bout a h8 post...
it just gets all over my nerves when nobody wants to help each other... :/
^(in any situation)

standardtoaster · Jan 18, 2011

I am interested in where this project is heading. I'm not knowledgeable in those fields, so I won't really be of that much help. :/

Tcll · Jan 18, 2011

standardtoaster said:
I am interested in where this project is heading. I'm not knowledgeable in those fields, so I won't really be of that much help. :/

it's supposed to do just as it's title states...

any model format you make a plugin for, it'll convert

but what good is the plugin if you can't prog the format into it...
I suppose I'll just hault on MDL0 and work on DAT a little more...

or maybe complete a little of my project list...
whatever... DX

btw, I'll UD this to convert any format...
(not just models)

libertyernie · Jan 18, 2011

It seems like texture conversion would be easier to program than model conversion, especially since the GameCube and Wii share texture formats.

Tcll · Jan 18, 2011

libertyernie said:
It seems like texture conversion would be easier to program than model conversion, especially since the GameCube and Wii share texture formats.

it is... but it's a little confusing on some formats :/

Tcll · Jan 19, 2011

any experts out there know what this offset (the 0 offset) is for:
'FFFF 0000 #### #### 00000000 00000000'

I know it does something (as shown by the unknowns)
but I'm not sure what that "something" is...

'unk zero unk unk string data'

also...
I still need to know what the nodes do (under the mdl0 header)

and face-points are another big issue...
how do I find the index lengths, and how many indexes each point contains??

IDK how many times I've asked about face-points...

gawd...
H8RS are LZRS... I H8 H8RS

Tcll · Jan 22, 2011

yo kryal...
I think I may have a thing to suggest :/

you rmbr the shader configs right?
well, I did a little data comparison,
and found out that the padding fits w/in 30 byte blocks...

I hope this may be of help to you...
I'll continue to research them even more...

pikazz · Jan 22, 2011

looks great so far :D
but I lack of skillz when it comes to programming D:

wish I could help on some way or another

pokelover980 · Jan 23, 2011

Please, for the love of god, use the edit button and don't triple/quadruple post.

Tcll · Jan 23, 2011

pokelover980 said:
Please, for the love of god, use the edit button and don't triple/quadruple post.

hey I made it clear in my first post that I'd be doing it...

honestly... why do people worry about such small things like that

and please don't hand me "well it's in the rules"
that's exactly why I made it clear...

jsyk:
I'm multi-posting to let people know of progress, findings of the format,
or to simply ask a Q and hope for a reply...
it's my intention to bump the thread up,
otherwise, it gets knocked down to another page and loses attention...

but that's not really all of it...
there's people who have subscribed to this thread and
(unlike my forum) will get a notification of a new post...

so I think I have quite a few valad reasons...

JetAurion · Jan 23, 2011

I'm fairly certain that for something of this caliber, multi-posting would be fine since barely anyone else can even contribute to such a project.

Tcll · Jan 23, 2011

JetAurion said:
I'm fairly certain that for something of this caliber, multi-posting would be fine since barely anyone else can even contribute to such a project.

Thank you.

Ls777 · Jan 23, 2011

Tcll said:
-
BLEND (Blender 2.49)
3DS
-
RAW (export only)
OBJ (export only)

the conversion process is simple (for users)
1: open file to read
2: save file to convert to
(you're done)

Ok, you have the right idea, but you should change your goals slightly
.blends are nearly impossible to support, since they are more of a program dump then
anything. I highly suggest you concetrate on just exporting/importing objs
since most 3d programs can use that format

Im gonna be waatching this thread, gooood luck :cool:

Tcll · Jan 23, 2011

Ls777 said:
Ok, you have the right idea, but you should change your goals slightly
.blends are nearly impossible to support, since they are more of a program dump then
anything. I highly suggest you concetrate on just exporting/importing objs
since most 3d programs can use that format

Im gonna be waatching this thread, gooood luck

I have the blender 2.48 src if that should help explain anything :/

I know blends just support everything possible,
so if I can export the needed info into the certain places in the blend,
it should work :/

I want 2.49b though :/

and thanx

Ls777 · Jan 24, 2011

Tcll said:
I have the blender 2.48 src if that should help explain anything :/

I know blends just support everything possible,
so if I can export the needed info into the certain places in the blend,
it should work :/

I want 2.49b though :/

and thanx

im saying, blender can export objs, so youd be better off supporting obj then blends
(All versions of blender support obj, but i believe the blends change slightly between version)

This may help if you insist to read blends:
http://en.wikipedia.org/wiki/Readblend

Id like to use 2.5+ to import stuff, im an avid blender user =P

Tcll · Jan 24, 2011

Ls777 said:
im saying, blender can export objs, so youd be better off supporting obj then blends
(All versions of blender support obj, but i believe the blends change slightly between version)

This may help if you insist to read blends:
http://en.wikipedia.org/wiki/Readblend

Id like to use 2.5+ to import stuff, im an avid blender user =P

OBJ sux D:
along with RAW
I'm almost done with that format (not MTL) and about to move onto DAE or X... :/

I use 2.49b... (stable)
but I'm waiting for the 2.6 release (since there'll be no bugs, unlike 2.5)
I do intend to add support once they release it.

and the blender importers and exporters suck D:
it's almost like I have to redo them to get them correct...
comon people

Tcll · Jan 25, 2011

well, I hope to further extend my MDL0 info search with this thread:
http://www.smashboards.com/showthread.php?t=296356

this is my last MDL0 thread I'm creating on this forum btw >:/
(it's like my 5th one here (there's more that weren't created by me)) D:

@ anyone thinking:
and don't tell me to go look at documentation...
everything I've figured out up to this point is from the docs >:|

EDIT:
oops... lol

didn't realize the thread wasn't in a public place XDD
I've asked the mods if they could move it

EDIT2:
I've moved it myself >:/
http://www.smashboards.com/showthread.php?p=12060199#post12060199
heh...

Tcll · Jan 26, 2011

so I've finally gotten further on deciphering the face-points...
FINALLY!

found this odd block of hex after the weights:
00 00 00 00 00 00 00 00 00 00 08 50 00 00 5E 05
08 60 00 00 00 03 10 00 00 10 08 00 00 00 15 00
08 70 41 37 70 09 08 80 C8 24 12 09 08 90 04 82
41 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00

so yea...
working on deciphering it now... :/

EDIT:
well...
I figured out what that mystery hex was :/
it was the shader instrictions :/

08 50 00 00 5E 05
08 60 00 00 00 03
10 00 00 10
08 00 00 00 15 00
08 70 41 37 70 09
08 80 C8 24 12 09
08 90 04 82 41 20

well that got me nowhere DX
all that excitement for nothing...
whatever...

back to reading...

Tcll · Jan 30, 2011

alright :D

Definitions (what do these do??)
Bones
Verts
Normals
Colors (not sure how they're used)
UV's
Materials (figuring)
Shader_Config (figuring)
Objects (face-points almost complete)
Textures (not sure how they're used)
Pallets (not sure how they're used)

other stuff:
nodes (figuring)
weights (unconverted)
anything I may have forgotten

Tcll · Feb 2, 2011

Tcll said:

KK
this data looks like it really is face-point attributes...
basically the key to defining the lengths of the facepoints

now I need some good notes on how it works, and how to convert it...

these notes suck:

Code:

vertex position data block {
 0x40 byte header {
  uint32_t length of this block
  int32_t offset to beginning of file (always negative)
  ...
  uint32_t at 0x10:
   index # of block (starts at 0, should be no gaps)
  uint32_t at 0x14: {
   0x01 = positions, has bounding box + zero pad
   0x00 = normals, no bounding box data + zero pad
  }
  ...
  uint16_t at 0x1c: bytes per vertex (0x04 * #components) {
   0x08 = UVs
   0x0c = vertex and normal data
  }
  uint16_t at 0x1e = number of vertices to follow
  0x20 {
  float[6] (bounding box?)
  uint32_t[2] zero
  } (not present for normals)
 } 
 component data follows: #vertices * #components float32
}
 
vertex array header: {
 0x140 bytes long, almost completely unknown
 u8 @ 0x0e {
  0x14 or 0x54 for u8 datas
  0x5e for u16 datas
 }
 u16 @ 0x1e (vertex count for a position data block) is:
  0x80 for a 4 or 7 byte vertex struct layout
  0xc0 for an 8 byte vertex struct layout
 u16 @ 0x94 = byte length of 3rd component in the struct
  only if @0x1e != 0xc0 and @0x0e != 0x14
}
vertex array data: {
 u8 list type to follow {
  0x00 == stop processing
  0xB8 == points
  0xA8 == lines
  0xB0 == line strip
  0x90 == triangles
  0x98 == triangle strip
  0xA0 == triangle fan
  0x80 == quads
 }
 u16 = #verts to follow
 if (@0x1e in header == 0xc0) {
  u8 position idx
  u8 unknown1
  u16 unknown2
  u8[4] more unknowns
 } else {
  component lengths defined in header (@0x0e, @0x94):
  position idx
  (uv or normal idx)
  (uv or normal idx)
 }
}

and Pharrox's notes arn't really much help in that area :/

here's that hex as it would be viewed in a hex editor: (padding included)
00 00 00 00 00 00 00 00 00 00 08 50 00 00 5E 05
08 60 00 00 00 03 10 00 00 10 08 00 00 00 15 00
08 70 41 37 70 09 08 80 C8 24 12 09 08 90 04 82
41 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00

^(just in case some people find viewing it this way better) ;P

original post in the community MDL0 thread

Tcll · Feb 6, 2011

well, here's some progress:

bad codes and no instruction from experts is to blame for this :/

I had to use will's code to at least get it that far...
but I have no idea how it works...
I'd know how to fix the prob if I did :/

EDIT:
heh...
got it this far after adding 1 to each index:

I figured it out since there were faces at 0|0|0 in my global format...

SqLeon · Feb 6, 2011

yay progess

Tcll · Feb 6, 2011

yeh...
sry it took so long...
I had to pull teeth just to get it that far

it only writes the vert indexes of the face-points...
I still need to add the normals and uv's and everything...

the faces currently look like this: f 1 2 3
they should look like when finished: f 1/3/2 2/4/8 3/6/5

the materials arn't even supported yet... (mtl files)
this is just simple mdl0 to obj mesh conversion

Tcll · Feb 11, 2011

here's an update:
http://www.smashboards.com/showthread.php?p=12165939#post12165939
I went back...
so it doesn't export atm...

but it should export perfectly for any mdl0 when it does

uugh...
most complicated part yet...

heh...
just wait until I get to the material/shader block...

the TEV's should be easy to figure out...
all they are is '61 ##' commands

Tcll · Mar 4, 2011

got some progress to show off:
http://brawlimports.proboards.com/index.cgi?action=gotopost&board=214666&thread=3&post=81

the code is 100% mine

not sure what's causing it to look so bad,
but I'll figure it out

I'll just look into Will's code to see wut's up... heh

Tcll · Mar 5, 2011

I'm re-modding my program a little...

I'm adding a safety check window
this will cancel the use of the original safety checks,
and will easily allow me to build different plugins for the same filetype.

the window will appear after selecting the import and export files (appearing twice),
and will contain buttons to select the conversion for the type of game or format version.
(eg: collada 140 or collada 141)

I got the idea since melee's format doesn't have an easy safety check that can be performed,
and brawl's safety check can be easily mistaken on a different type of MDL0 file.

so the SCW is an absolute method that will be implemented

but it will require that I redo alot of the main program,
as well as the info and initiators in every plugin.

Tcll · Mar 6, 2011

well...
I've gotten the idea almost fully working...

the multi-reference conversion is the prob...
the buttons display in order as they should...
but right now I can't seem to get the command workng correctly

the command for the button activates as it's created...
this is causing major problems such as crashing

single reference conversion works perfectly though...
so I can still get away with conversion on that end >_>

but I need to get the multi-reference conversion working <_<

Tcll · Apr 10, 2011

thank you Revel8n ;D

yes, he has come back, and I have talked with him
he was the one who fixed my code

I'm still trying to figure out what he did to it... XD
^only verts are supported :/

Photos · Apr 14, 2011

this is icredible. so does that mean it's close to being done? i can't wait.

Tcll · Apr 14, 2011

Photos said:
this is icredible. so does that mean it's close to being done? i can't wait.

meh...

lets see:
~60% for the mdl0 import process
~20% for the obj export process
~80% for the main program
and
~40% for the global format

figuring the values would be:
~50% for the entire converter
^(current mdl0 to obj conversion anyways)

Universal Model Converter (UMC) Development thread - Deceased

Smash Lord

Smash Lord

Smash Lord

Smash Champion

Smash Lord

Smash Lord

Smash Ace

Smash Lord

Smash Lord

Smash Lord

Smash Lord

Tubacabra

Smash Lord

Smash Ace

Smash Lord

Smash Lord

Smash Lord

Smash Lord

Smash Ace

Smash Lord

Smash Journeyman

Smash Lord

Smash Cadet

Smash Lord

Smash Cadet

Smash Lord

Smash Lord

Smash Lord

Smash Lord

Smash Lord

Smash Lord

Smash Journeyman

Smash Lord

Smash Lord

Smash Lord

Smash Lord

Smash Lord

Smash Lord

Smash Journeyman

Smash Lord

Information

Network