UFO:Alien Invasion

Development => Coding => Topic started by: Another Guy on July 14, 2009, 06:28:31 am

Title: Compiling maps too slow
Post by: Another Guy on July 14, 2009, 06:28:31 am
I used to compile all maps in about 1-1.5 hour. Now the process is taking ridiculous 16 hours. No relevant changes on my PC. It was so weird I even checked for virus with 3 distinct antivirus. I even tried shuting down my PC and unplugging power cable for about 5 min to ensure all RAM and buffers were lost, then starting windows and terminating all non critical processes. Then I did the process again. Same slow result.

Was the way maps are compiled changed between now and and july 6th (my last full compile)?

Anyone else with same issue?
Title: Re: Compiling maps too slow
Post by: odie on July 14, 2009, 06:44:06 am
I used to compile all maps in about 1-1.5 hour. Now the process is taking ridiculous 16 hours. No relevant changes on my PC. It was so weird I even checked for virus with 3 distinct antivirus. I even tried shuting down my PC and unplugging power cable for about 5 min to ensure all RAM and buffers were lost, then starting windows and terminating all non critical processes. Then I did the process again. Same slow result.

Was the way maps are compiled changed between now and and july 6th (my last full compile)?

Anyone else with same issue?

Yoz Another Guy. Nope it did not take me tat long. It did take me longer than previously (i believe another 45 mins to an hr plus, i just let it run).

Usually when i start in the morning, i end the map compilation task by abt 1 plus 2pm. (which means abt 3-4hrs). That day, it ended slightly later. I realised its because of quite a few new maps being added. Hence the slight delay even for my upload.

But not the 16 hrs u mentioned.

Did u have hidden maintainance tasks running in the background?

A good tool to check might be:
Tweaknow Powerpack 2009 (http://www.tweaknow.com/powerPack.html)

Goto the Process Manager and check your active processes.
(See screenshot - Notice my mouse cursor, click it and the popup with your current processes will appear).

U can try running the new function Ram Manager within too. It let u see ur system remaning ram in %.

U can also see mani mani MANI MORE of the system stuff within. Check it out.



Sidetrack:
Ildamos: U can do a review on this too. lol. Its EXCELLENT. :P



Quick note - For those who have seen me advocating Win-Secret previous, this IS THE UPGRADED version with a WHOLE tons more goody stuff. :D Check it out, its freeware too.
Title: Re: Compiling maps too slow
Post by: Another Guy on July 14, 2009, 06:54:43 am
Did u have hidden maintainance tasks running in the background?
Nope. As I said on first post, I terminated all non vista essential process. And checked manager from time to time to see if vista would launch any new one (only windows update and search index related. not anything much memmory consuming).

About the app u suggested, it seems very handy. I'll try it out.  :P

Still, the problem remains. Somewhere in the last revisions (from july 6th to now), something really capped my map compiling processing power.
Title: Re: Compiling maps too slow
Post by: geever on July 14, 2009, 10:55:51 am
Some pathfinding changes happened recently, I dunno if they slowed it down that much.

-geever
Title: Re: Compiling maps too slow
Post by: Duke on July 14, 2009, 11:00:28 am
Are you using svn ?
If so, you can do an 'update to revision' as of July 5th and see if the prob is gone.

Then update to a rev from say July 10th and so on and finally nail down the exact rev number which broke it.
Title: Re: Compiling maps too slow
Post by: odie on July 14, 2009, 11:17:02 am
Nope. As I said on first post, I terminated all non vista essential process. And checked manager from time to time to see if vista would launch any new one (only windows update and search index related. not anything much memmory consuming).

Still, the problem remains. Somewhere in the last revisions (from july 6th to now), something really capped my map compiling processing power.

Being curious, have u configured your pc to maximise your CPU CORES?? (As in if u r using duo core, u need to change the number of processors to 2, which btw, still does not explain the GREAT disparity in the time diff).


Some pathfinding changes happened recently, I dunno if they slowed it down that much.

-geever

geever, i highly doubt tat. :P

Even if pathfinding makes things 200% slower, tat will probably be maxing out compilation time from 3 hrs to 9..... not 16. Logically logical reasoning? :P

Are you using svn ?
If so, you can do an 'update to revision' as of July 5th and see if the prob is gone.

Then update to a rev from say July 10th and so on and finally nail down the exact rev number which broke it.


I supposed what DUKE is suggesting is for the SVN revision to be backdated to at least 25017 (Thats the revision i made for 6th July). If u wan to trace what has been changed, u might want to drop by my sticky thread and browse thru the revision change notes. I made it a point to track it there, except for losing revisions 24913 to 24918 (due to being sick and >100 revisions).

25021 to 25023 - Something about maps making using nos of processors is something i suspect for ur case? I dunno. U might consider omitting that for a start (i.e. going before 25021).

After 25023, there are tons of prefabs updates and i doubt they will affect ur processing speed.

Try a few diff revisions (maybe going by day by day using my 'not-so-daily tracker'). Hehe.

Hope ur issues get resolved!



About the app u suggested, it seems very handy. I'll try it out.  :P

Yupz, its useful, maybe u can tweak up ur system quite a bit on that. :D
Title: Re: Compiling maps too slow
Post by: geever on July 14, 2009, 12:32:41 pm
geever, i highly doubt tat. :P

Even if pathfinding makes things 200% slower, tat will probably be maxing out compilation time from 3 hrs to 9..... not 16. Logically logical reasoning? :P

You shouldn't. Happened before. If tracing changes everything change. Pathfinding uses traces but creating lightmaps uses them as well and creating lightmaps is the major task during map compilation.

Anyway, I'm looking forward, the revision-check result.

-geever
Title: Re: Compiling maps too slow
Post by: odie on July 15, 2009, 04:18:14 am
You shouldn't. Happened before. If tracing changes everything change. Pathfinding uses traces but creating lightmaps uses them as well and creating lightmaps is the major task during map compilation.

Anyway, I'm looking forward, the revision-check result.

-geever

Tracing.... I believed that it refers to (src/tools/ufo2map/routing.c) or for that purpose, anything within (src/tools/ufo2map/) changes? So long as those changes, ufo2map.exe should be changed, hence, meant we should recompile all maps?

I started a thread on Request for Notifications For Maps Recompilation (http://ufoai.ninex.info/forum/index.php?topic=3799.msg27992#new) here, but so far, i guess no one really paid attention to it; though Mattn did mentioned something abt a script (map-get), but in the contrib/scripts directory, it makes no sense to me how to compile this to get it working.

Hmmmm.....

Btw, what do u mean by looking forward to the revision-check results? Referring to Another Guy? :)
Title: Re: Compiling maps too slow
Post by: geever on July 15, 2009, 07:48:59 am
Mattn did mentioned something abt a script (map-get), but in the contrib/scripts directory, it makes no sense to me how to compile this to get it working.

No compilation but a python (http://www.python.org) is needed.

Btw, what do u mean by looking forward to the revision-check results? Referring to Another Guy? :)

Yeah, the revision at the compilation time changed that much (and what happened in that).

-geever
Title: Re: Compiling maps too slow
Post by: odie on July 15, 2009, 08:50:44 am
No compilation but a python (http://www.python.org) is needed.
-geever

Okie geever,

Thanks so far for the clarifications.

I am new to python alrite. Never used it b4.
I have downloaded the latest 3.1 from its homepage, installed it on winxp pro SP3.

Question is, how do i use it to check it?

I found mattn's site: http://mattn.ninex.info/ but it does not say anything. lol.
I wiki google search, forum search, dun seem to find the windows version of how it works. My eyes must be playing tricks on me......

Any chance geever u could guide me on this one pls?

Thanks!
Title: Re: Compiling maps too slow
Post by: Another Guy on July 15, 2009, 05:57:26 pm
OK, SVN checkout testing is taking longer than I expected (not that much free time). I'll post back here when I have narrowed it down.
Title: Re: Compiling maps too slow
Post by: Duke on July 15, 2009, 10:52:07 pm
Plz give us an intermediate report as soon as you have managed to test with a rev of July 5/6th, ie. once you can prove that it's NOT your system.
I guess that would lauch additional analysis/hints from the community ;)