project-navigation
Personal tools

Author Topic: compile_maps.bat improvements  (Read 20193 times)

Offline Duke

  • Administrator
  • PHALANX veteran
  • *****
  • Posts: 1037
    • View Profile
Re: compile_maps.bat improvements
« Reply #30 on: November 30, 2009, 11:14:05 pm »
Built diff and affinity, then got this:
C:\UFO>\ufo\contrib\scripts\compile_maps_Muton /clean /1t4core
C:\Users\me\AppData\Local\Temp\ufo2map_core*.* konnte nicht gefunden werden
Its seems this is not the root folder of Ufoai

Offline Muton

  • Sergeant
  • *****
  • Posts: 496
    • View Profile
Re: compile_maps.bat improvements
« Reply #31 on: December 01, 2009, 06:15:22 pm »

Offline Duke

  • Administrator
  • PHALANX veteran
  • *****
  • Posts: 1037
    • View Profile
Re: compile_maps.bat improvements
« Reply #32 on: December 01, 2009, 08:28:05 pm »
Same result, but it's probably nor your fault.
I figured out that for some unknow reason (Vista?) the %tmp% var is set to "C:\Users...", but the dir is actually named C:\Benutzer....
I changed your script to set tmp explicitely, but only to get:

C:\UFO>cd /d C:\UFO\contrib\scripts\
Das System kann den angegebenen Pfad nicht finden.
Its seems this is not the root folder of Ufoai

Looks like there is more than one problem to solve before this runs on my machine.
Well, be both are convinced that out method is better than the other's method and we both have a good reasoning. I at least gave your method a try. Guess now it's your turn to try mine ;)

Offline Muton

  • Sergeant
  • *****
  • Posts: 496
    • View Profile
Re: compile_maps.bat improvements
« Reply #33 on: December 01, 2009, 11:19:09 pm »
the %tmp% var is ok
this code troubles you
Code: [Select]
if /I NOT "%this%" == "ufoai\" (
@echo Its seems this is not the root folder of Ufoai
pause
exit /b 1
)
It seems you stored your source under c:\ufo
and not c:\ufoai
I've replaced it with some reliable code
Code: [Select]
if NOT EXIST "%UfoAIroot%INSTALL" (
@echo Its seems this is not the root folder of Ufoai
@echo Because im unable to find the file %UfoAIroot%INSTALL
pause
exit /b 1
)

http://ufoai.ninex.info/forum/index.php?action=dlattach;topic=3583.0;attach=2550

What is this option -j for
im unable to find information in the help output

btw  i have some troubles on my machine using -o3 or -o2 on ufo2map
C::B reports some warnings on both optimations
and ufo2map crash (lightning) on my machine if i run it with -t 2 (-t 1 generate no errors during "maping")
only -o1 is working during "maping" and there are no warnings by C::B

I've just copmpleted a testrun
ufo2map compiler options
Code: [Select]
<Compiler>
<Add option="-march=k8" />
<Add option="-O1" />
<Add option="-Wall" />
<Add option="-msse3" />
<Add option="-mfpmath=sse" />
<Add option="-mieee-fp" />
<Add option="-D__GNUWIN32__" />
<Add option="-DWINVER=0x501" />
<Add option="-DNODEBUG" />
</Compiler>
took using -t 2
02:09:00
and using my script
02:00:34

yesterday i've compiled with this options
using my script
01:37:43
using -t 2 crash ufo2map during "maping"
Code: [Select]
<Compiler>
<Add option="-march=k8" />
<Add option="-O3" />
<Add option="-fexpensive-optimizations" />
<Add option="-Wall" />
<Add option="-msse3" />
<Add option="-mfpmath=sse" />
<Add option="-mieee-fp" />
<Add option="-D__GNUWIN32__" />
<Add option="-DWINVER=0x501" />
<Add option="-DNODEBUG" />
</Compiler>

Offline Duke

  • Administrator
  • PHALANX veteran
  • *****
  • Posts: 1037
    • View Profile
Re: compile_maps.bat improvements
« Reply #34 on: December 02, 2009, 10:38:34 pm »

It seems you stored your source under c:\ufo
and not c:\ufoai
Correct.

Quote
What is this option -j for
im unable to find information in the help output
-j is the number of make-jobs to start.

Quote
btw  i have some troubles on my machine using -o3 or -o2 on ufo2map
C::B reports some warnings on both optimations
Yes. Same here. But works.


Ok, I tried your update. It managed to delete the bsps, but then got into trouble:
Code: [Select]
475 maps to go
current C:\UFO\base\maps\bunker.map bunker.bsp

ufo2map.exe -v 4 -extra  -t 1  maps\bunker.map
"\Acer\Empowering" kann syntaktisch an dieser Stelle nicht verarbeitet werden.

Why does try to scan the programs dir ?
And WTH does it not react if I set 'echo on' in line 2 ??

Offline Mattn

  • Administrator
  • PHALANX Commander
  • *****
  • Posts: 4831
  • https://github.com/mgerhardy/vengi
    • View Profile
    • Vengi Voxel Tools
Re: compile_maps.bat improvements
« Reply #35 on: December 03, 2009, 07:31:02 am »
what are those warnings? i don't get any

Offline Muton

  • Sergeant
  • *****
  • Posts: 496
    • View Profile
Re: compile_maps.bat improvements
« Reply #36 on: December 03, 2009, 07:33:29 pm »
Quote from: Duke
ufo2map.exe -v 4 -extra  -t 1  maps\bunker.map
"\Acer\Empowering" kann syntaktisch an dieser Stelle nicht verarbeitet werden.

Why does try to scan the programs dir ?
And WTH does it not react if I set 'echo on' in line 2 ??

Why does try to scan the programs dir ?
I dont
the script is somwhere at
if "%_1t4core%" == "true" (
you can add a pause after
call :ufo2map_check %%a
@echo ufo2map_check %%a
pause

or
call :ufo2mapcmd "%tmp%\ufo2map_core%%a.cmd" %%a
@echo ufo2mapcmd "%tmp%\ufo2map_core%%a.cmd" %%a
pause

And WTH does it not react if I set 'echo on' in line 2 ??
turn it off this way
:: @echo off
or
rem @echo off

=================================

Code: [Select]
<Compiler>
<Add option="-march=k8" />
<Add option="-O3" />
<Add option="-fexpensive-optimizations" />
<Add option="-Wall" />
<Add option="-msse3" />
<Add option="-mfpmath=sse" />
<Add option="-mieee-fp" />
<Add option="-D__GNUWIN32__" />
<Add option="-DWINVER=0x501" />
<Add option="-DNODEBUG" />
</Compiler>

-------------- Clean: windows in ufo2map ---------------

Cleaned "ufo2map - windows"

-------------- Build: windows in ufo2map ---------------

Compiling: ..\..\src\tools\ufo2map\common\bspfile.c
Compiling: ..\..\src\common\ioapi.c
Compiling: ..\..\src\common\mem.c
Compiling: ..\..\src\common\routing.c
Compiling: ..\..\src\common\tracing.c
Compiling: ..\..\src\common\unzip.c
V:\MinGW\ufoai\src\common\unzip.c: In function 'unzlocal_getShort':
V:\MinGW\ufoai\src\common\unzip.c:205: warning: 'i' may be used uninitialized in this function
V:\MinGW\ufoai\src\common\unzip.c: In function 'unzlocal_getLong':
V:\MinGW\ufoai\src\common\unzip.c:233: warning: 'i' may be used uninitialized in this function
Compiling: ..\..\src\ports\windows\win_shared.c
Compiling: ..\..\src\shared\byte.c
Compiling: ..\..\src\shared\entitiesdef.c
Compiling: ..\..\src\shared\mathlib.c
Compiling: ..\..\src\shared\parse.c
Compiling: ..\..\src\shared\shared.c
Compiling: ..\..\src\tools\ufo2map\brushbsp.c
Compiling: ..\..\src\tools\ufo2map\bsp.c
Compiling: ..\..\src\tools\ufo2map\check\check.c
Compiling: ..\..\src\tools\ufo2map\check\checkentities.c
Compiling: ..\..\src\tools\ufo2map\check\checklib.c
Compiling: ..\..\src\tools\ufo2map\common\aselib.c
Compiling: ..\..\src\common\files.c
Compiling: ..\..\src\tools\ufo2map\common\imagelib.c
V:\MinGW\ufoai\src\tools\ufo2map\common\imagelib.c: In function 'LoadTGA':
V:\MinGW\ufoai\src\tools\ufo2map\common\imagelib.c:74: warning: dereferencing type-punned pointer will break strict-aliasing rules
V:\MinGW\ufoai\src\tools\ufo2map\common\imagelib.c:78: warning: dereferencing type-punned pointer will break strict-aliasing rules
Compiling: ..\..\src\tools\ufo2map\common\polylib.c
Compiling: ..\..\src\tools\ufo2map\common\scriplib.c
Compiling: ..\..\src\tools\ufo2map\common\trace.c
Compiling: ..\..\src\tools\ufo2map\csg.c
Compiling: ..\..\src\tools\ufo2map\faces.c
Compiling: ..\..\src\tools\ufo2map\levels.c
Compiling: ..\..\src\tools\ufo2map\lighting.c
Compiling: ..\..\src\tools\ufo2map\lightmap.c
Compiling: ..\..\src\tools\ufo2map\map.c
Compiling: ..\..\src\tools\ufo2map\patches.c
Compiling: ..\..\src\tools\ufo2map\portals.c
Compiling: ..\..\src\tools\ufo2map\routing.c
Compiling: ..\..\src\tools\ufo2map\textures.c
Compiling: ..\..\src\tools\ufo2map\threads.c
Compiling: ..\..\src\tools\ufo2map\tree.c
Compiling: ..\..\src\tools\ufo2map\ufo2map.c
Compiling: ..\..\src\tools\ufo2map\writebsp.c
Linking console executable: ..\..\ufo2map.exe
Output size is 288,00 KB
Process terminated with status 0 (0 minutes, 44 seconds)
0 errors, 4 warnings

-------------------------

on -o2 NO -fexpensive-optimizations
....
V:\MinGW\ufoai\src\tools\ufo2map\common\imagelib.c: In function 'LoadTGA':
V:\MinGW\ufoai\src\tools\ufo2map\common\imagelib.c:74: warning: dereferencing type-punned pointer will break strict-aliasing rules
V:\MinGW\ufoai\src\tools\ufo2map\common\imagelib.c:78: warning: dereferencing type-punned pointer will break strict-aliasing rules
....

-------------------------
on -o1  -fexpensive-optimizations
no warning

Offline Duke

  • Administrator
  • PHALANX veteran
  • *****
  • Posts: 1037
    • View Profile
Re: compile_maps.bat improvements
« Reply #37 on: December 03, 2009, 08:03:43 pm »
Warnings confirmed.

Offline Duke

  • Administrator
  • PHALANX veteran
  • *****
  • Posts: 1037
    • View Profile
Re: compile_maps.bat improvements
« Reply #38 on: December 03, 2009, 08:16:08 pm »
ok, :: worked.

It seems to have a problem with ";C:\Program Files (x86)\Acer\Empowering Techno" when trying to set the path.

Offline Muton

  • Sergeant
  • *****
  • Posts: 496
    • View Profile
Re: compile_maps.bat improvements
« Reply #39 on: December 03, 2009, 10:37:39 pm »
oh microsoft ;(

http://ufoai.ninex.info/forum/index.php?action=dlattach;topic=3583.0;attach=2551

added
a short algo to replace path dirs with 8.3 naming
Code: [Select]
set path=%path%;%this%;
set this=
:pathloop
for /F "tokens=1* delims=;" %%a in ("%path%") do (
set path=%%b
if exist "%%a" (
set this=%%~dpsnxa;%this%
)
)
if NOT "%path%" == "" goto pathloop
set path=%this%
set this=

Offline Duke

  • Administrator
  • PHALANX veteran
  • *****
  • Posts: 1037
    • View Profile
Re: compile_maps.bat improvements
« Reply #40 on: December 03, 2009, 11:34:22 pm »
Congrats, that did it :)

Recompile maps is currently running....

Offline Duke

  • Administrator
  • PHALANX veteran
  • *****
  • Posts: 1037
    • View Profile
Re: compile_maps.bat improvements
« Reply #41 on: December 04, 2009, 12:13:54 am »
finished
started at 23:05:07,34
finished at 23:54:02,46

There is something wrong with the starting time. My stopwatch said it took 30m13s.

It started as a tight race. CPU was at 100% much more often than I thought. That's because you start with the big 'static' maps in \maps. Then a little slowdown for \maps\africa, but not much. 100% with the big alienb tiles again.

When we come to the RMA maps with the tiny tiles (eg. farm), cpu drops to below 80% sometimes.

At 26:30, you're running out of maps to compile.
Around 27:30, alienb\a_hangar is the last map still running (on one puny core, no, half of a core) for the remaining time.

I think the script could do better if the maps were sorted by size, running the big ones first, but I still believe you can't afford the idle times.

EDITED: typos
« Last Edit: December 04, 2009, 12:25:38 am by Duke »

Offline Borsti67

  • Squad Leader
  • ****
  • Posts: 164
    • View Profile
Re: compile_maps.bat improvements
« Reply #42 on: December 06, 2009, 06:11:04 pm »
Hi Muton,

oh microsoft ;(
added
a short algo to replace path dirs with 8.3 naming
This had still issues with path elements containing <"> (whyever, one of the MS-XML-Patches did so).
I removed the quote signs from the path, afterwards I got one step further, but...
Code: [Select]
Its seems this is not the root folder of Ufoai
Because im unable to find the file C:\PROGRA~1\Spiele\UFOAI-~1\INSTALL

A "dir /x" reveals here is no "ufoai-~1":
Code: [Select]
01.01.2009  16:16    <DIR>          UFOAI-~1.1   UFOAI-2.2.1
06.12.2009  14:48    <DIR>          UFOAI-~1.3_D ufoai-2.3_dev

Is this a problem specific to my installation (German version of XP) or is there more room for improvements to the path detection routine? ;)

€dit: Before the "pathloop" the variable %this% is still correctly set to UFOAI-~1.3_D...?!

€²: After setting UfoAIroot manually in the script, it continues to the next error - in my case, code::blocks is not found via path, so "affinity" could not be compiled. I also "hardcoded" the path into the script, now it finally starts... Anyway a lot of messages scroll through:
Code: [Select]
find: /I: no medium
find: /V: No such file or directory
find: /I: no medium
find: /V: No such file or directory
find: prefabs: No such file or directory
48 map/s to compile
Only 48? Seems not enough to me...?
Afterwards error messages pop up, stating e.g. "file .svn/.map doesn't exist" or "file .sv/prop-base/UFOAIBot.pl doesn't exist"... Seemingly always 2 per map to compile.

Would be nice if a complete compilation would only take about 5 mins (including clicking away the error windows), but I can't believe it. :-P
« Last Edit: December 06, 2009, 07:05:04 pm by Borsti67 »

Offline Muton

  • Sergeant
  • *****
  • Posts: 496
    • View Profile
Re: compile_maps.bat improvements
« Reply #43 on: December 07, 2009, 04:02:25 pm »
I've updated the script the last time
http://ufoai.ninex.info/forum/index.php?action=dlattach;topic=3583.0;attach=2558

>This had still issues with path elements containing <"> (whyever, one of the MS-XML-Patches did so).
In that case i give up  :P

> find: /I: no medium
Because of my last modification a query is broken
find had to be replaced with findstr
something i realy do not understand
Somethimes i think cmd.exe has its own (female)AI
(fixed in uploaded one)

> UFOAI-~1.3_D
> Is this a problem specific to my installation (German version of XP) or is there more room for improvements to the path detection routine?
The 8.3 name has an extension
set UfoAIroot=%%~dpsna\
replaced by
set UfoAIroot=%%~dpsnxa\
should fix it (fixed in uploaded one)

And the bad thing is
on my system not all maps have been build
3-4 where missed
( NOT fixed in uploaded one)

I dont want to put more effort into it
because i have another fine working script
that im currently working on.
http://ufoai.ninex.info/forum/index.php?topic=3843.0

Offline Borsti67

  • Squad Leader
  • ****
  • Posts: 164
    • View Profile
Re: compile_maps.bat improvements
« Reply #44 on: December 07, 2009, 06:03:11 pm »
Thanks for you efforts, Muton. Obviously it's better not to follow this one any longer. ;)

Is the EXE-file linked in 1st post still valid, or is there already a new one?
If I understand correctly, nothing else is needed (or comes from SVN)?