compile_maps.bat improvements

[Kildor]

> why a_hangar and bunker maps take sooo long to compile.
I know nothing about bunker, but a_hangar has *many* glowing surfaces. If you up verbose of ufo2map, you can see, the main time ufo2map spends on Lightning calculation.

[Muton]

I suggest reading this
http://ufoai.ninex.info/forum/index.php?topic=3604.0

and
http://ufoai.ninex.info/forum/index.php?topic=3583.0

[Duke]

Thx, Muton, very interesting.
Your findings differ a bit from mine. As I wrote above, for me -t4 was better than -t2, which was better than -t1.
Any idea why ?

I didn't use -O3 or -march=core2.
btw as there is no -march=i7, do you expect core2 to be better than nothing ?

[Muton]

> Thx, Muton, very interesting.
> Your findings differ a bit from mine. As I wrote above, for me -t4 was better than -t2, which was better than -t1.
> Any idea why ?

You are missleaded
If you call ufo2map with -t 1 and generate another ufo2map also with -t 1 .....
until you run out of cores (So you run n ufo2map process simultaneously)
You are faster than using -t n, If you set the affinitymask correctly (which my script does)
btw: There is no magic thats normal

> I didn't use -O3 or -march=core2.
> btw as there is no -march=i7, do you expect core2 to be better than nothing ?

You own a Core7  
Yes there is no core7 in gcc but the important thing is SSE
If you use SSE you can remove option -ffloat-store
Just use core2duo optimation
btw. There is a option for SSE4.1

[Borsti67]

Hm, the next time I need a full recompilation I'll try -t 1: yesterday I did w/o parameters (2 cores correctly found) and it took about 5 hours, leaving the PC nearly unusable most of the time. Especially when alienb/hangar was compiled, I couldn't even move the mouse! I never noticed such a severe impact before, were there changes in the process itself?

[Muton]

set ufo2map priority to low
or just use my script, if you want

[Borsti67]

priority on windoze?
I'll check out your script, thanks.

[Duke]

Muton,
I just compiled a_hangar.map without optimization, -o3 and -march=core2. The results are 73s, 56s and 50s, respectively. Thx for your hints

But I really doubt that running just as many processes as you have processors can be the optimum. Because ufo2map has a single-threaded part that doesn't use much cpu. With or without affinity mask: if you don't have additional waiting processes to fill that gap, the cpu will be unused in that period.

So you may want to increase the # of processes in your script and see what happens

[Muton]

I'm shure it will take longer
The problem is the cache must be filled with data from process|thread a <-> b
which is very time consuming.
And this for only a fiew cycles ...

Even if you dont set the affinity mask for a ufo2map process the compile time will increase
'couse windows (xp--) reaisign threads from core n to core x and vice versa all the time
which is very time consuming.
They've made some improvements on win7 in that point ...

On a core7 this idle times are'nt a real problem
'couse a core7 is dynamically overclocked if other cores are at "idle"
Its not realy that easy and yes there are some improvements on win7

No i'm not a friend of win7

[Duke]

A little update on compile times:

Using an ufo2map compiled with release, -o3 and -core2.
Using msys, maps.mk forced to -t 4, -j 50.
(my old approach with -j 500 ran into memory alloc probs, so I had to reduce -j)
Result: 26 minutes on my machine (i7).

Quite a lot of new big maps have been added recently, so the result seems reasonable.

Another interesting result: I deleted alienb/a_hangar, bunker and england.bsp and ran make again. Those three maps alone took 6:48 to complete.

[geever]

A little update on compile times:

Using an ufo2map compiled with release, -o3 and -core2.
Using msys, maps.mk forced to -t 4, -j 50.
(my old approach with -j 500 ran into memory alloc probs, so I had to reduce -j)
Result: 26 minutes on my machine (i7).

Quite a lot of new big maps have been added recently, so the result seems reasonable.

Another interesting result: I deleted alienb/a_hangar, bunker and england.bsp and ran make again. Those three maps alone took 6:48 to complete.

We have a list of Map compiling times on the wiki. We should update it regularly.

ps. -j 50 isn't a usual setting....

-geever

[Duke]

Tried again with -j 5, but it didn't really make a difference. The idea behind -j 500 was this: because of -t4, the last job running can only make use of half of the 8 'processors' of the i7. So I wanted the 'big maps' to be started asap.

I also updated the table you linked to.

Currently running compile_maps.bat for a comparison

[Muton]

The core7 own HT again right?
can you test it with my map compile script too
http://ufoai.ninex.info/forum/index.php?action=dlattach;topic=3583.0;attach=1302

/clean /1t4core

[Duke]

Where do I have to unzip it to ?

[Duke]

Maybe I should just look into the zip before asking
Found it.