Wall Worm Forums

Design Tools => Wall Worm Level Design in 3ds Max => Topic started by: SairesArt on June 30, 2015, 06:49:38 PM

Title: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on June 30, 2015, 06:49:38 PM
Hey,
when importing a VMT, almost everything is fine and cookies.
The Overlays however, come incredibly out of scale (ca. five times larger) and all horizontal and with wrong rotation.
My units are set up correctly and everything else seems to work just fine, but the overlays just go wild.

It's a 2006 Engine Game, any help?

Also I'd like to reconstruct the Skybox from it's 1/16 scale miniature to it's rescaled "original".
I just can't seem to get it to work, exept painful reallignment and rescaling of each entity to the original map gemotry. Just placing the sky_camera at the origin and scaling *16 always results in a wrong scale and placement. Is there a way to reproduce it consitently?

Couldnt find any info on both topics, either in search or documentation.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on June 30, 2015, 08:04:43 PM
Welcome to WW :)

I'll have a look at the VMF Importer and see about fixing the overlay issue. The scale is likely a result of the fact that the VMT>Material importer ignores the basetexturetransform at this time, and the initial scale is always based off of the physical dimension of the bitmap rather than any VMT transformations. The orientation is another issue, and I'll have a look.

Regarding rescaling the tiny skybox to true scale, that is nothing I've ever done. But here are some steps I'd try:

1) Group the entire 3D Sky with the Group command (Except the sky_camera... leave it out of the group).
2) With the group selected, open the Hierarchy Tab.
3) Click Affect Pivot Only.
4) Align the Group Pivot to the Sky Camera pivot.
5) Exit Affect Pivot Only mode.
6) Move the Group to the Origin.
7) Open the Utilities Tab
8) Add Rescale World Units to the Utilities if it is not present (with the More... button)
9) Click the Rescale... button
10) In the Scale Factor, enter 16.
11) For the Affect, chose Selection.
12) Click OK
13) Ungroup the Group
14) Add to a new layer to keep separate.
15) With sky objects selected, click Wall Worm > Wall Worm Level Design > Set Selection as Skybox Item

Now when you re-export, the skybox should still rescale to 1/16 around wherever the sky camera is (so keep the sky camera out in the middle of no-mans-land).
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 01, 2015, 07:48:01 AM
Wow, many thx, your skybox reconstruction instruction worked perfectly.

And thx, that you take the time to look at overlays^^
Have a level where there are around 40 or so in one room for extra detail, after import it all becomes a flickering mess of 40 roomzized floating color carpets.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 06, 2015, 02:11:33 PM
Quote from: wallworm on June 30, 2015, 08:04:43 PMThe orientation is another issue, and I'll have a look.
I'd like to add, that decals don't ever import either.

Also big map imports always fail. Even after 300 tries over 24h of retries.
I split the map up into skybox, brushes, props and overlays/decals. Import times are still big, but it at least manages to import.

The thing with the overlays is really a hassle though, any idea when you can update WW to fix overlays (and mebee decals <3 )?
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 06, 2015, 02:59:00 PM
I'll try to give it some attention this week. It will help if you send me a ZIP file of your VMF and any VMT/VTF that go with it.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 07, 2015, 02:19:29 PM
Quote from: wallworm on July 06, 2015, 02:59:00 PM
I'll try to give it some attention this week. It will help if you send me a ZIP file of your VMF and any VMT/VTF that go with it.
Since you are monitoring this thread, I'll give you a quick comment on the bugs of "Validate geometry", since I'm too lazy to type: an infographic as an attachment below.
That bug made be import the map with BOTH options and then import the broken geo from the without Validate geometry to the one with validate geometry. Not a big deal, just so you know.

(https://www.wallworm.net/proxy.php?request=http%3A%2F%2Fwww.saires.de%2Fserver%2Fbbugsbunny.jpg&hash=6402971ae5eddf29cada37c8f493b01c8efb651b)

Here are the not-decompiled Textures and Models (Neotokyo°+SourceBase2006) as an Ultra Compressed 7z (1gb) (http://www.saires.de/server/wormystuff.7z)
decompiled would be 12+gb
And the Mapfiles with their divided counterparts
(http://www.saires.de/server/rise_vmfs.7z)
I do rerenderings of maps in an offline renderer and preparing/restructuring/remodeling them as scenes for an animated short.
What I usually do is: Delete all lights (since I retexture and rematerial the lights to be physically based) and all brushes, that are purely skybox or clips/hints/blocklight
Delete all env_, logic_, ambient_ and so on... everything that is not visible, including sprites.
Pray to the gods, it will import. If not, devide up into: (Skybox, Brushes, Props, Overlays) Which works. Sometimes (after building Prop library) I even have to devide up the prop import, since too many props still dont import after 200 retries)

Attached below what I do with the maps from earlier tests (manually placed overlays, props, reimported textures, since your importer didnt work so well earlier this year)
Also Animation test (https://www.youtube.com/watch?v=HNptAX8FE8E)


(https://www.wallworm.net/proxy.php?request=http%3A%2F%2Fwww.saires.de%2Fserver%2F1.jpg&hash=8a5d5a1453fcb396bcc1b68da9554a1c634f94aa)

(https://www.wallworm.net/proxy.php?request=http%3A%2F%2Fwww.saires.de%2Fserver%2F2.jpg&hash=1b698162d752f3c1b9eb7b19bae73d2f01c8663c)

(https://www.wallworm.net/proxy.php?request=http%3A%2F%2Fwww.saires.de%2Fserver%2F3.jpg&hash=1247a419871f1f260df4f2afd402a05e996719aa)

(https://www.wallworm.net/proxy.php?request=http%3A%2F%2Fwww.saires.de%2Fserver%2F4.jpg&hash=c8c2c337461cbe07e3bdaaa5bc2250a5573cdfa5)
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 07, 2015, 09:07:58 PM
I'm downloading the files. It's likely I won't get a chance to look at it until tomorrow. Hopefully I can find why your files are choking. At this point, all my imports work fine within a few minutes... so perhaps there is some special case I need to unearth. I'll figure it out.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 08, 2015, 01:13:58 AM
An update. Two things.

1) I've discovered the bugs in the VMF Importer. One related to the displacements not having some properties that are in newer VMF files related to displacements. The second related to a typo in the VMF importer code--not sure how it got in there but must have crept in during some phase where I got distracted on some other chore and inadvertently left the code in.

I will push the fix for this sometime tomorrow.

2) Regarding decals and overlays:

The fix for these is not going to be as quick. While going through my code, I found this comment:

QuoteAs it turns out... infodecal does not have a list of sides. This poses a problem--because there is no fast way to determine what brush face to align to.
--!TODO Finish the Decal Plugin you've been thinking about for the last few years

I have some ideas on addressing this, but it's not likely to be addressed in the update coming tomorrow.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 08, 2015, 12:05:12 PM
I have updated WW today. It does two things: 1, your level will now import now without the errors (the raw level). On my system, it took about 15 minutes to fully import.

I also updated infodecal and info_overlay entities to mostly orient correctly. There is still not a guarantee on orientation because infodecal must guess which brush to align to--and info overlays just align to the first side in the sidelist.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 10, 2015, 01:09:41 AM
There is now a pending update to WW that should address almost all your issues. See http://www.wallworm.net/index.php/topic,1358.msg4029.html#msg4029
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 10, 2015, 08:43:18 PM
Now its all kinds of messed up.
Import works now solid, was plesantly surprised to see 4 as the default for Parse retries now :]
Wanted to make a post about the unexpeted error thingy, but you figured it out.
Props and Overlay import however, is completly broken. The props rotation is screwed and overlay postition is just somewhere in the skies.
This is with the new dev build.

I also had a new problem: When importing with my Old Props library, it would parse, but never finish importing. Had it running for 6 hours. only after deleting props library, did it work, tried it three times and verified it two times.

PS: Decals still dont import, but again, its not a big deal for me personaly
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 10, 2015, 09:05:32 PM
Let me run a few tests. ALl the prop positions/orientations are correct in my tests the last couple of days.... but let me see if I can replicate the problem. And I've got all the overalys and decals coming in now. Overlays are working in my tests... and decals are generally correct. But I'll see if I can replicate.

Also, here is some new info on the Unknown System Exception error (http://dev.wallworm.com/document/179/unknown_system_exception.html).

And I'll look at the prop importing. Do you mean that you cannot batch import if the props already exist?
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 10, 2015, 09:23:41 PM
What I do is
>import props from map
and then importing the map itself
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 10, 2015, 09:25:56 PM
OK, I'm looking into it to see what could have changed how that was working for you.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 11, 2015, 12:36:46 AM
I've confirmed the bug with the rotations. Unfortunately it's late. I'll try to figure it out tomorrow.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 11, 2015, 01:17:23 AM
Well I figured the problem out... the code I added for infodecal looking for the closest brush and orienting to it was inappropriately being applied to ALL entities. I just loaded the fix for that.

EDIT:

So I expect the props to be all correct now.

In terms of the long loads with a prop library... I have an idea what is causing this. My best advice at the moment is to click the Prop Library From VMF button in the VMF Importer before importing the level, keep those objects in the scene, then import the level. This might reduce file system lookups per-prop in the VMF importer as I don't think this method will use XRef objects.

I'll optimize it for less file lookups as that's probably the problem.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 11, 2015, 01:38:59 PM
Quote from: wallworm on July 11, 2015, 01:17:23 AM
So I expect the props to be all correct now.
Indeed, they are all fixed.
However, the Overlays still float in space and decals still don't import.
Used the last official build from today.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 11, 2015, 01:56:57 PM
In my simpler tests all the overlays and decals are mainly correct. So I will have to test more complex scenes now.

The position of the overlays/decals is directly based on where it's origin is inside the VMF. Hammer often displays the decals far from their origins. This is something that I'll address in the plugin (see below).

Overlays are almost always correct for me. But their accuracy in the viewport is dependent on their origin being close to the surface they are supposed to be applied to and the number of brush faces the overlay is applied to (if the overlay is using multiple faces or has faces that no longer exist, there can be problems--as the overlay aligns itself to the normal of the first face in the sides it is assigned to).

Decals are often aligned incorrectly because the decals are just points in space and I have to get the closest brush based on surface (which can be a guesswork and can have more than one valid brush to choose from). The fastest way to do this uses an intersect() function... but this is flawed because the decals are planar in WW ... so I need to rework it some.

Even once I get it "done" the decals are still planes... so if the decal needs to wrap on multiple surfaces... it won't look right. I've started a new plugin for decals in Max that will handle this (and more). As it will be a general purpose decal plugin and is going to take me a week (or more) to develop, it'll be a commercial plugin that will be no more than $10.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 11, 2015, 02:02:18 PM
thx, you deserve a cookie :]
(https://www.wallworm.net/proxy.php?request=http%3A%2F%2Fwww.saires.de%2Fserver%2Fcookie_small.png&hash=2435f1c3f21febb11508f4ac6464330c27ef326e)

Would it be possible to revert to how Overlays were handeled prior to 1.62? aka, always correct position, but wrong scale and rotation?
The Map import is fixable with those, but with the wrong positions now it's too inconsistent.
But keep the awesome VMF import code, that for whatever reason does not have missing brushes anymore and is stable?
(also the disappearing verticies on small brushes with "validate geometry" are still there)
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 11, 2015, 02:20:52 PM
As for the positions, I'll simply need to find out why they are currently off for you. The position of an overlay should not have changed from before... so I will just find out why it is incorrect in your case (I'm not getting any overlays with incorrect locations in any tests ATM).

In terms of  the "validate geometry" error. I actually recommend NOT using that function. See if the brushes are all correct without that option. The validate geometry function is destined to be removed eventually as the errors that it was used for should be disappearing. I'd leave it off. After an import you can click the "Select Imported Geometry that Needs Attention" button in the VMF Importer to see brushes that the importer thinks might be wrong--and you can fix as needed (if needed).

At this point, I import with:

Use Default Brush Generation = ON
Validate Geometry = OFF

I'm off for the day to go fishing with my son, so I won't have any updates until tonight or tomorrow.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 11, 2015, 03:39:16 PM
Quote from: wallworm on July 11, 2015, 02:20:52 PM
I'm off for the day to go fishing with my son, so I won't have any updates until tonight or tomorrow.
Have a fun day :]
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 12, 2015, 12:05:39 PM
So I've figured out the overlay positions problem. info_overlays have 2 parameters that relate to their position in world space. Like all other point entities, they have an origin property that is their position in the world. But they also use a property called BasisOrigin. The BasisOrigin overrids the origin. IN all the files I've tested, these two values are identical. But when I imported your level, the Z position is inverted (so the origin might be [128,64,512] but the basis origin is [128,64,-512] . So now I know. I'll fix this in the next update.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 14, 2015, 03:39:48 PM
Quote from: wallworm on July 12, 2015, 12:05:39 PM
So I've figured out the overlay positions problem.
My body is ready. Hope you killed the local fish populus.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 15, 2015, 05:42:45 PM
was busy for a couple days. And tied up ATM... I haven't forgotten about this. I'll post a fix this week.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 16, 2015, 03:46:57 PM
OK, I believe you can now get all the correct positions and usually the correct orientation. However, there are times when the overlay is flipped, so you may need to rotate some on occasion and/or flip the UVs.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 17, 2015, 12:08:36 PM
Quote from: wallworm on July 16, 2015, 03:46:57 PM
OK, I believe you can now get all the correct positions and usually the correct orientation.
Many Many thx.
I see your confidence has risen again as you dropped the parse retries to 0 :D

Well, it's finally as good as I needed it to be.

Bugs include:
-Overlays are Double or half the actual ingame size
-rotation arond the X axis being -33.164 (for whatever reason) Its no problem to set it back to zero, but a couple of overlays are just randomly rotated around thex axis by 33.164 degrees
-decals still don't import, although I set it up correctly
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 17, 2015, 01:02:23 PM
Thanks for the response.

I was importing the full level from what you sent me and the overlays and decals are all importing for me (I'm importing from the full scene). Are you trying it from the same full file? It may not work for a file with only decals... it might need to include the brushes in the imported vmf.

The rotations of the overlays is still something I intend to look at. I'm not sure I'll get it 100% because of how the data is stored/interpreted/converted. The issue, I believe, is that the overlays are determined in world space but then also determined by the U/V scaling ... and there is some more math I haven't truly figured out when aligning it to a face in the correct coordinate system. I think the issue will be solved by the decal plugin I'm working on that won't need to worry about face orientation (and order of the vertices in the face).

In terms of the overlay size, it may be related to which axis is used. In the tests I've done in my own scenes, the sizes seem correct. So I haven't determined what is the cause for some to be correct and others not.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 17, 2015, 02:06:06 PM
Quote from: wallworm on July 17, 2015, 01:02:23 PM
I was importing the full level from what you sent me and the overlays and decals are all importing for me (I'm importing from the full scene). Are you trying it from the same full file? It may not work for a file with only decals... it might need to include the brushes in the imported vmf.
I tried the full map two times, the decals just always stay proxy objects.
It just says infodecal in the propties, so it's not a Base FGD parse issue.

(different map, but same for others)
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 17, 2015, 02:18:24 PM
When you click Wall Worm > Wall Worm Level Design > Point Entities, does it let you create a infodecal or prop_overlay ? (although in WW you'd want to use Worm Face to create both, this is just a test to see if the entities are actually available). If not, WW embeds the raw entity data (if you right-click the imported decal and choose Object Properties and the User Defined Tab, you'll see the embedded data).

WW won't do all the necessary things for importing decals/overlays with embedded data as it was going to take more time to update the legacy entity info import code--and if the entity cache is valid it isn't necessary anyway. Supporting the legacy entity info (that is based on support for Maple 3D's Convexity entity format) is probably not something I will continue to do with updated functions.

So if WW doesn't have the entities for those, then the entities do need to be updated/parsed. However, if the entities are parsed, then it might be a difference in the entity data in the entities in your FGD than those I've ever dealt with. I'm currently testing with the CS:GO FGD. In that case, send my your FGD files you are using.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 17, 2015, 04:54:27 PM
ye i can create it.
I have also deleted the MOD's FGD, since I don't need its spawn entities.
It still wont import decals exept squares

So i really only use the base.fgd
considering the proxy imports corerctly, the fgd shoudn't be the issue.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 17, 2015, 05:43:09 PM
Well it's indeed odd. Sorry you are experiencing that... but it's hard for me to troubleshoot this specific issue since the decals are always importing when I try here. But I'll let my brain sit on it and hopefully I can think of something that might cause my installation to work and yours not.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 18, 2015, 05:14:38 AM
It's broken :[
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 18, 2015, 09:58:01 AM
[EDITED] Go to this link: http://wallworm.com/store/index.php?route=product/product&path=59&product_id=55
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 19, 2015, 06:54:24 AM
Quote from: wallworm on July 18, 2015, 09:58:01 AMGo to this link: https://wallworm.com/store/index.php?route=product/product&path=59&product_id=55 (with the SSL) and the download should work.
nope :|

First image firefox (not a number error)
second image chrome (never ending Arrow Animation)
third image mobile [Chrome] (never ending Arrow Animation)

PS: First and foremost make sure WWpro downloads work for paying customers
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 19, 2015, 08:18:35 AM
Also created a test case for oddities with short import times.
Image 1: Tiled overlays get streched wrong. (Height ignored)
Image 2: Double Sized overlays, which should be half their size.
Image 3: No decal import. (Windows on the left side)

This is nitpicky at this point, since the general overlay import is kinda correct, which is all that matters. although decal import is still a hmpf :|
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 19, 2015, 09:39:12 AM
Sorry, after I posted that I found that there were other problems. So I removed the SSL enforcement. Now the Non-SSL link (http://wallworm.com/store/index.php?route=product/product&path=59&product_id=55) is what you should use again until I figure it out.

I'll give the map a look. But I think you may need to send me the exact FGD you are using, because it's possible that there is a different parameter name or missing one that might explain why the decals work here.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 19, 2015, 10:28:27 AM
Quote from: wallworm on July 19, 2015, 09:39:12 AM
I'll give the map a look. But I think you may need to send me the exact FGD you are using, because it's possible that there is a different parameter name or missing one that might explain why the decals work here.
Here's the fgd, also included the parsed .ms file from C:\Users\Saires\AppData\Local\Autodesk\3dsMax\2015 - 64bit\ENU\scripts\WallWorm.com\entities

It just seems odd, for this to be a fgd issues, since I'm using the standart base.fgd from Episode one
(F:\Games\steamapps\common\SourceSDK\bin\ep1\bin)
(meaning the 2006 Base source SDK)

Forgot the zip  ;D
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 19, 2015, 11:26:25 AM
Quote from: SairesArt on July 19, 2015, 10:28:27 AM
It just seems odd, for this to be a fgd issues, since I'm using the standart base.fgd from Episode one

I agree... but I'm trying to eliminate the variables.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: SairesArt on July 22, 2015, 04:33:32 PM
Quote from: wallworm on July 19, 2015, 11:26:25 AM
I agree... but I'm trying to eliminate the variables.
Any new magic insight?
Just wanted to know if you plan on fixing this. If not I'll continue doing what I do, not a big problem, if you have a plan I'll wait a touch.
Title: Re: Overlays imported wrong & Skybox reconstrucion
Post by: wallworm on July 22, 2015, 09:20:24 PM
I'm pretty tied up ATM. I do want to figure out why the decals are so wrong for you because I'm largely getting the decals into my scenes. But I didn't get to test with the FGD you sent yet. I probably won't realistically get to look more into it until after the weekend.

Don't think I've abandoned it, just have other priorities at the moment.