Author Topic: Material Library Generator error + VMF importer exception  (Read 15029 times)

joewa

  • Newbie
  • *
  • Posts: 14
  • Karma: 0
  • Acolyte
    • View Profile
Material Library Generator error + VMF importer exception
« on: May 12, 2014, 09:59:38 PM »
Posting this here at Shawn's request:

I get what looks like about 90% of the way through generating a material library and throw this error:
"--Unable to convert: undefined to type: Integer"

matgenfuncs.ms opens with this highlighted:

   
Code: [Select]
if ((matchPattern theLine pattern:"*$phong*") == true) then (

addMat.shaderByName = "Phong"
matType = "model"
addMat.forModel = true

if ((matchPattern theLine pattern:"*$phongboost*") == true) then (
addMat.phongboost = val as float
continue
)

if ((matchPattern theLine pattern:"*$phongexponent*") == true) then (
>>>THIS LINE HIGHLIGHTED addMat.phongexponent = val as float
continue
)

if ((matchPattern theLine pattern:"*$phongfresnelranges*") == true) then (
addMat.phongfresnelranges = val as string
continue
)


continue

)
and F11 has this to say:

Code: [Select]
Welcome to MAXScript.

-- Error occurred in wallworm_generateMaterialFromVMT(); filename: C:\Program Files\Autodesk\3ds Max 2013\scripts\WallWorm.com\common\matgenfuncs.ms; position: 18417; line: 796
--  Frame:
--   modelColor: (color 155 155 0)
--   selfIllum: false
--   theLine: ""$phongexponenttexture" "truck/truckspec""
--   vmt: "F:\Steam\steamapps\SourceMods\ShotgunSunrise_dev\materials\truck\Truck.vmt"
--   dispColor: (color 0 155 0)
--   selfIllumName: ""
--   kwvars: undefined
--   dispColor2: (color 155 155 155)
--   extraRootPaths: undefined
--   nocull: false
--   theVMT: <File:F:\Steam\steamapps\SourceMods\ShotgunSunrise_dev\materials\truck\Truck.vmt>
--   overWriteName: undefined
--   pos: undefined
--   rootPath: "F:\Steam\steamapps\SourceMods\ShotgunSunrise_dev\materials"
--   color1: undefined
--   addMat: truck/Truck:Standard
--   val: "truck/truckspec"
--   displayMat: undefined
--   diffuseName: "truck/Truck"
--   c: undefined
--   theTex: #("F:\Steam\steamapps\SourceMods\ShotgunSunrise_dev\materials\\truck/TruckEnv.tga", true)
--   dName: undefined
--   envmapmask: "truck/TruckEnv"
--   param: "$phongexponenttexture"
--   bumpName: "truck/trucknormal"
--   op: undefined
--   blendV: Vertex_Color:Vertex Color
--   CanClose: true
--   ambientocclusionName: undefined
--   displayMatnameB: undefined
--   opacityName: ""
--   color2: undefined
--   mp2: Standardmaterial:Standard
--   displayMatname: undefined
--   addMatB: Blend:Blend
--   tex1: "F:\Steam\steamapps\SourceMods\ShotgunSunrise_dev\materials\\truck/TruckEnv.tga"
--   matType: "model"
--   useMat: true
--   oldm1: undefined
--   lineNum: 16
--   lineArray: #(""$phongexponenttexture"", ""truck/truckspec"")
--   tex2: undefined
--   brushColor: (color 155 155 155)
--   extraPath: undefined
--   isTranslucent: false
--   maxfogdensityscalar: undefined
--   called in vmt loop; filename: C:\Program Files\Autodesk\3ds Max 2013\scripts\WallWorm.com\common\wwmt_material_library_generator.ms; position: 2963; line: 134
--  Frame:
--   vmt: "F:\Steam\steamapps\SourceMods\ShotgunSunrise_dev\materials\truck\Truck.vmt"
--   mat: undefined
--   useMat: undefined
--   called in wwmt_ceate_material_library(); filename: C:\Program Files\Autodesk\3ds Max 2013\scripts\WallWorm.com\common\wwmt_material_library_generator.ms; position: 4443; line: 204
--  Frame:
--   fileFilter: "*"
--   count: 433
--   total: 468
--   recurse: false
--   project: "SGSR"
--   br: true
--   bl: true
--   oneLibrary: true
--   folderFilter: "*"
--   BrushMaterials: undefined
--   mdl: true
--   ModelMaterials: undefined
--   matContinue: undefined
--   BlendMaterials: undefined
--   message: undefined
--   p: "F:\Steam\steamapps\SourceMods\ShotgunSunrise_dev\materials"
--   AllMaterials: #materialLibrary()
--   called in btnGenerate.pressed(); filename: C:\Program Files\Autodesk\3ds Max 2013\scripts\WallWorm.com\common\wwmt_material_library_generator.ms; position: 11288; line: 397
--  Frame:
--   errors: #()
--   message: undefined
>> MAXScript Rollout Handler Exception:
-- Unable to convert: undefined to type: Integer <<



When importing a VMF - seemingly any VMF - I get this:
Code: [Select]
"Error parsing VMF attempt 1:-- Unknown system exception

Will attempt another 1 times..."
"Error parsing VMF attempt 2:-- Unknown system exception

Will attempt another 0 times..."
There is a common problem with parsing VMF files that causes an *unknown system exception*. This error seems to be random and may be due to a memory buffer overflow in Max.

Wall Worm defaults to trying 20 times to parse a VMF, but you can set this to your own preference.

If you get this failure, try running the import again and/or setting the retry limit higher. Repeated attempts will likely succeed eventually.

It does eventually succeed if I put retry on 999 and leave it overnight.

joewa

  • Newbie
  • *
  • Posts: 14
  • Karma: 0
  • Acolyte
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #1 on: May 12, 2014, 10:04:53 PM »
The material library generator error actually went away when I removed Truck.vmt (non-essential anyway) but here's the offending VMT in case it helps:
Code: [Select]
"VertexlitGeneric"
{
"$basetexture" "truck/Truck"
"$bumpmap" "truck/trucknormal"
"$envmap" "env_cubemap"
$envmapmask "truck/TruckEnv"
"$envmapcontrast" 1
"$model" 1

"$halflambert" 1
"$model" "1"

"$phong" "1"
"$phongexponenttexture" "truck/truckspec"
"$phongboost" "6"
"$phongfresnelranges" "[0.05 0.5 1]"
}



I'm out of touch with Source stuff and haven't touched this particular VMT in yeeeeeeeeears so if there's anything obviously wrong with that I'm not picking up on it.

wallworm

  • Administrator
  • Hero Member
  • *****
  • Posts: 3622
  • Karma: 100
  • Behind me?
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #2 on: May 12, 2014, 10:13:39 PM »
The error with the VMT is now patched in the current version of WW that I just loaded.

As for the VMF import error, I think I'd need you to send me a VMF that is failing. So far, I've never needed to put the reparse limit above 20, even when I was importing the Dear Esther levels which were insane. Perhaps it has something to do with the amount of RAM available? I currently have 12GB.

joewa

  • Newbie
  • *
  • Posts: 14
  • Karma: 0
  • Acolyte
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #3 on: May 12, 2014, 10:32:45 PM »
That's probably not it, I have 16GB and it's not getting close to being all used. From what I remember there is a lot that's unconventional/crazy/wrong about this map, but I had the same thing happen with some Ep2 ones... I'll do some tests on simpler maps and if it's not an "every map always does this" thing I'll send you the problem vmf.

joewa

  • Newbie
  • *
  • Posts: 14
  • Karma: 0
  • Acolyte
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #4 on: May 14, 2014, 01:35:11 AM »
The map eventually got imported after the 208th retry. Materials are all in there working fine. Problems:

-when I hit "import props from scene entities" I get "there are no entities in the scene with MDLs listed", but the map is riddled with prop_statics, and I'm pretty darn sure I have my modelsrc stuff set up right. Ideas anyone?

-There's bunch of func_precipitation in my map which don't delete when I hit the "tools to hidden layer" button. Admittedly not sure what material I used on them so probably my bad and no big deal anyway.

-A few patches of displacement are missing - weirdly enough mostly big flat ones, my more elaborate ones came in fine. For my purposes also no big deal and unfortunately I can't open Hammer to get a better look at what might have been atypical about those displacements/their brushes.

-Some brushes are crazy F#$@ed up, like this:



which may or may not end up being a problem for me. Haven't yet figured out if there was anything particularly weird about those brushes.

Main thing is the prop bug. :( Still, this tool is amazingly great. Thanks!

Joris Ceoen

  • Global Moderator
  • Full Member
  • *****
  • Posts: 227
  • Karma: 27
  • WallWorm Ambassador
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #5 on: May 14, 2014, 04:20:28 AM »
Because this is the first time I ever saw someone using the level importer (which is really cool to see!) it's probably some errors that have been lingering around somewhere that Shawn hasn't been able to test. Although he'll notice this thread quickly I'll ping him so he can look into this :) just stay tuned!

wallworm

  • Administrator
  • Hero Member
  • *****
  • Posts: 3622
  • Karma: 100
  • Behind me?
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #6 on: May 14, 2014, 08:18:47 AM »
I'm sorry you are having these problems :( Without the actual files, it's hard to say.

The whole problem behind the reparse requirements is very odd... and exposes a fundamental flaw in either MAXScript or a method I'm employing to parse the VMF. What is so odd about it is that the failure always happens at random places and eventually works. I have a feeling it is due to memory usage/allocation. Sometimes a file will fail 4 times but work on the 5th... sometimes that same file will work on the first... and sometimes the 11th try. In most cases, the failure is 3-11 in the cases I've tested. My CS 1.6 levels import fairly quickly... and the one displacement heavy level I did for CS:S imports in a few minutes. The Dear Esther levels could take an hour. What is causing your level to fail specifically or take so long is something I really have no answer for.

Regarding the messed up geometry. This is due, ironically, to rounding errors. You can sometimes fix some of these by selecting the  messed up brushes, changing the current setting for Use Default Brush Generation and clicking the button to Rebuild Selected. Unfortunately, this function only works during the same session Max is open when you imported the level. And it is dependent on the version of Max you have. (The alternative method uses MassFX functions to create a hull from the imported brush points and is not guaranteed to have 100% precise vertex locations).

Oddly enough, the messed up brushes (and probably the missing displacements) is due to rounding errors. Yep. And there is nothing I can do about this (unless there is some math ninja that someone can teach me). You can help the cause by Voting here on Autodesk Uservoice for Double Float Precision. Essentially, what is happening is that all the slicing of an object via the planes has to check which side of a slice to delete geometry... and sometimes this rounding error causes the wrong side to be thrown away. I do have an idea for a work-around... but it's going to cause even more slowness in the importer... but I can make it an option.

As for the models... this is odd. When you select an entity in the scene, can you see its properties when you open the modify panel? Select one and see what the model path is. WW assumes they are relative paths. So if the model is mymaps/model1.mdl then WW will look for sdk_content_folder/modelsrc/mymaps/mymodel1.qc . This can be a problem because the QC can actually tell the model to export to any random path and name... which means that there is no guarantee that the QC is in the same location in modelsrc as the output model... but without parsing all QCs in the entire modelsrc there is no way to know this... which could be a very long process. Perhaps this sheds some light?

joewa

  • Newbie
  • *
  • Posts: 14
  • Karma: 0
  • Acolyte
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #7 on: May 16, 2014, 01:18:28 AM »
Hey guys, thanks for the mega great support! This might be the first time in Source modding history that anybody has properly supported their software.

If you guys want to check out the exact maps/content I'm working with, I actually open sourced the mod a while ago. You can download it here if you want. The map pictured above is in there, it's called ss_revolver, and the other one I was trying and had the same problems with, ss_shotgun. The model sources aren't in there, because they don't exist anymore, which is why I batch decompiled them with Crowbar.

When I click a prop_static in the scene, my modify panel just says "Box", no other properties in there. It seems like I just have different coloured little cubes that don't mean anything.

As it happens, none of these problems probably matter much for my purposes - what I'm trying to do is port these old maps to Unreal Engine 4 for shits and giggles (and potentially towards prototyping a new version of Shotgun Sunrise on UE4). I deleted all my screwed up brushes and there's nothing there that I can see that wasn't pretty much just a cube (?!?) so those are easily replaceable, and the prop_statics were probably never going to  be easily importable into the new engine so I guess I'll just place them all again.

I am totally up for helping test this stuff though, I think it's super great that there's a way to get your Source content away from Source now. The Dear Esther Unity port put me onto it, though I was following WWMT from the newsletter for ages.

Let me know if I can help you guys test more or help in any other way! I'll keep trying to do what I'm trying to do. :)

Joris Ceoen

  • Global Moderator
  • Full Member
  • *****
  • Posts: 227
  • Karma: 27
  • WallWorm Ambassador
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #8 on: May 16, 2014, 04:44:46 AM »
It's really great to see people opening to the Wall Worm Model Tools :) You are entirely right! Every Source level can now be ported to other engines!

joewa

  • Newbie
  • *
  • Posts: 14
  • Karma: 0
  • Acolyte
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #9 on: May 16, 2014, 05:42:18 AM »
Yeah, it's super great.
The problem I'm having at this particular moment is A) UE4 has a maximum of 64 materials per mesh (so I can't just export out the entire level as one mesh, damn) and B) when I export an individual building, say, using Export Selected, as .FBX, when I import it into UE4 it's using all the wrong materials - still materials from my map, just like, SS_WOOD_WALL_002 instead of _022 and stuff like that.

I'll just keep whackin' my head against it.

wallworm

  • Administrator
  • Hero Member
  • *****
  • Posts: 3622
  • Karma: 100
  • Behind me?
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #10 on: May 16, 2014, 09:44:49 AM »
I'm working on some updates to the VMF exporter this morning. I'll hopefully have an update shortly that might alleviate almost all broken geometry. I'm trying to get the Unknown System Exception to disappear. Unfortunately, the message "Unknown System Exception" is about as useful as a Port-a-Potty on the moon.

As for props not having the entity values, my best guess (at the moment) is that your entity cache in WW is missing or broken.

Make sure that your global settings have your FGD path ... and that if the FGD includes other FGD files, the include FGDs exist in the same folder as your FGD. (So usually, base.fgd must exist in the folder with the FGD for your mod).

Regarding the material names not exporting with the FBX exporter... that sounds more like you may need up update FBX. Just search for the latest FBX update for your version of Max.

PS. Now that you are getting into WW some, how about making some content for Source still using WW???? :)

wallworm

  • Administrator
  • Hero Member
  • *****
  • Posts: 3622
  • Karma: 100
  • Behind me?
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #11 on: May 17, 2014, 01:55:46 AM »
So I haven't had a chance to download your levels but was looking into the parse error problem. I haven't solved it 100% but I have updated a bunch of the code in an attempt to alleviate it. Can you download the latest WW and try to parse a level. Please let me know if it finished before 20 parse attempts.

Here is more info on today's update: http://www.wallworm.net/index.php/topic,1028.msg2758.html

joewa

  • Newbie
  • *
  • Posts: 14
  • Karma: 0
  • Acolyte
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #12 on: May 17, 2014, 03:38:56 AM »
Heyo. I've just updated to the new version (assuming it's not a problem to just paste the new version over the old one?) and I'm still failing after 20 retries. I'm gonna leave one going even so to see if reparsing my fgd cache thing helped with the prop_static issue. Also updated to the latest FBX version so hopefully that helps my material issue, I'll let you know.

joewa

  • Newbie
  • *
  • Posts: 14
  • Karma: 0
  • Acolyte
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #13 on: May 17, 2014, 09:43:35 PM »
I just got an import from the new WWMT version and got a slightly different result, I still have pretty much all the same missing or messed up brushes but now the texturing on my displacements is also wrong:

You can see how it's supposed to look in the last screenshot.

Still no luck getting prop_statics to have .mdls listed, does this look obviously wrong or anything?

wallworm

  • Administrator
  • Hero Member
  • *****
  • Posts: 3622
  • Karma: 100
  • Behind me?
    • View Profile
Re: Material Library Generator error + VMF importer exception
« Reply #14 on: May 17, 2014, 10:30:14 PM »
I'm still pounding my head about this whole error. I imported 3 complex level yesterday, and all worked as expected (despite the odd Unknown System Exceptions that I am closing in on). Although there are some bad brushes, they are not often bad and the displacements seem to be almost always good.

Some models have failed to import for me... but this is only if the gamedir/path/to/modelname.mdl in the game did not match the QC in the modelsrc like  modelsrc/path/to/modelname.qc . So in some attempts, I've had to occasionally move a QC and/or change its name to match.

It could also be that the WW entities cache is not made yet.

Test this: click Wall Worm > Wall Worm Level Design > Point Entities. Is the list populated and can you add a point entity yet? If not, you need to parse the entities... which you can do in the global settings by clicking the Reparse button above your FGD. Unfortunately, this will probably need to be done before the import... as the model import only uses proper WW entities (and not the un-editable entities that are stored using an older format if you don't have the entity cache already).