UFO:Alien Invasion

Technical support => Mac => Topic started by: keybounce on May 19, 2010, 06:20:23 am

Title: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 19, 2010, 06:20:23 am
I'm having trouble with 2.3 building universal on a PPC.

Code: [Select]
stbmac:23 Michael$ svn info
Path: .
URL: https://ufoai.svn.sourceforge.net/svnroot/ufoai/ufoai/branches/ufoai_2.3
Repository Root: https://ufoai.svn.sourceforge.net/svnroot/ufoai
Repository UUID: 39b98707-f80d-0410-bc7e-c6b02f5409da
Revision: 30059
Node Kind: directory
Schedule: normal
Last Changed Author: tlh2000
Last Changed Rev: 30046
Last Changed Date: 2010-05-17 23:57:30 -0700 (Mon, 17 May 2010)

stbmac:23 Michael$ svn st
?      Makefile.in.orig
?      svn-diff.txt
?      port-readme.txt
?      svn-diff.nosignal.txt
?      svn-diff-2.3.txt
stbmac:23 Michael$ make models
 * [MOD] src/tools/ufomodel/ufomodel.c
lipo: can't create output file: debug-darwin-universal/tools/ufomodel/tools/ufomodel/ufomodel.o (No such file or directory)
make: *** [debug-darwin-universal/tools/ufomodel/tools/ufomodel/ufomodel.o] Error 1
stbmac:23 Michael$
stbmac:23 Michael$ make
 * [GAM] src/game/g_actor.c
 * [GAM] src/game/g_ai.c
 * [GAM] src/game/g_ai_lua.c
 * [GAM] src/game/g_client.c
 * [GAM] src/game/g_combat.c
 * [GAM] src/game/g_cmds.c
 * [GAM] src/game/g_edicts.c
^CInterrupted by signal 2
make: *** [debug-darwin-universal/game/game/g_edicts.o] Interrupt

stbmac:23 Michael$ make models
 * [MOD] src/tools/ufomodel/ufomodel.c
 * [MOD] src/shared/mathlib.c
 * [MOD] src/shared/byte.c
 * [MOD] src/shared/images.c
 * [MOD] src/shared/parse.c
 * [MOD] src/shared/shared.c
src/shared/shared.c: In function ‘Com_sprintf’:
src/shared/shared.c:473: warning: comparison is always false due to limited range of data type
src/shared/shared.c:475: warning: comparison is always true due to limited range of data type
src/shared/shared.c: In function ‘Com_sprintf’:
src/shared/shared.c:473: warning: comparison is always false due to limited range of data type
src/shared/shared.c:475: warning: comparison is always true due to limited range of data type
 * [MOD] src/common/files.c
 * [MOD] src/common/list.c
 * [MOD] src/common/mem.c
 * [MOD] src/common/unzip.c
 * [MOD] src/common/ioapi.c
 * [MOD] src/client/renderer/r_model_alias.c
 * [MOD] src/client/renderer/r_model_dpm.c
 * [MOD] src/client/renderer/r_model_md2.c
 * [MOD] src/client/renderer/r_model_md3.c
 * [MOD] src/client/renderer/r_model_obj.c
 * [MOD] src/ports/unix/unix_main.c
 * [MOD] ... linking -rdynamic (-lz  -ljpeg  -L/opt/local/lib -lpng12 -lz   -L/opt/local/lib -lSDLmain -lSDL -Wl,-framework,Cocoa -lSDL_image  -L/opt/local/lib -lSDLmain -lSDL -Wl,-framework,Cocoa)
./ufomodel -mdx -overwrite -v  -s 0.5 -f models/aircraft/craft_arm_plate/plating.md2
---- ufomodel 0.2 ----

---- filesystem initialization -----
Adding game dir: /usr/local/share/ufoai/base
Adding game dir: /Users/Michael/Documents/UFOAI-2.3-dev/base
Adding game dir: ./base
using ./base for writing
- model 'models/aircraft/craft_arm_plate/plating.md2'
Error: R_ModForName: unknown fileid for models/aircraft/craft_arm_plate/plating.md2
make: *** [base/models/aircraft/craft_arm_plate/plating.mdx] Error 1
stbmac:23 Michael$

First, I'm looking at 30046. Starting with an absolutely stock, clean copy.

'make models' will fail -- the directories for the .o files don't exist.
'make' will set them up. And then, 'make models' will compile.

But it doesn't run. It fails. And doesn't give any indication as to why.

Gdb begins ...

Edit 1:
Code: [Select]
Breakpoint 1, Com_Error (code=0, fmt=0x2455c "R_ModForName: unknown fileid for %s") at src/tools/ufomodel/ufomodel.c:178
178 va_start(argptr, fmt);
(gdb) bt
#0  Com_Error (code=0, fmt=0x2455c "R_ModForName: unknown fileid for %s") at src/tools/ufomodel/ufomodel.c:178
#1  0x00002590 in LoadModel (name=0x277bc "models/aircraft/craft_arm_plate/plating.md2") at src/tools/ufomodel/ufomodel.c:227
#2  0x00002814 in PrecalcNormalsAndTangents (filename=0x277bc "models/aircraft/craft_arm_plate/plating.md2") at src/tools/ufomodel/ufomodel.c:284
#3  0x00003ed8 in SDL_main (argc=8, argv=0x25095c0) at src/tools/ufomodel/ufomodel.c:678
#4  0x000236ec in -[SDLMain applicationDidFinishLaunching:] ()
#5  0x96d39688 in _nsnote_callback ()
#6  0x94ce243c in _CFXNotificationPostNotification ()
#7  0x96d36dc4 in -[NSNotificationCenter postNotificationName:object:userInfo:] ()
#8  0x93b64858 in -[NSApplication _postDidFinishNotification] ()
Not very deep in the code...
Title: Re: Current issues with 2.3 PPC build
Post by: keybounce on May 19, 2010, 06:41:01 am
Alright, do you see an obvious byteswap issue here?

stbmac:23 Michael$ egrep IDALIASHEADER src/*/*.h
src/common/qfiles.h:#define IDALIASHEADER   (('2'<<24)+('P'<<16)+('D'<<8)+'I')

Code: [Select]
(gdb) p buf
$8 = (byte *) 0x282be2c "IDP2\b"

(gdb) p LittleLong(*(unsigned *) buf)
$10 = 1229213746

(gdb) p  (('2'<<24)+('P'<<16)+('D'<<8)+'I')
$13 = 844121161


207 /* call the appropriate loader */
208 switch (LittleLong(*(unsigned *) buf)) {
209 case IDALIASHEADER:
210 /* MD2 header */
211 R_ModLoadAliasMD2Model(mod, buf, modfilelen, qfalse);
212 break;
(gdb)
213
214 case DPMHEADER:
215 R_ModLoadAliasDPMModel(mod, buf, modfilelen);
216 break;
217
218 case IDMD3HEADER:
219 /* MD3 header */
220 R_ModLoadAliasMD3Model(mod, buf, modfilelen);
221 break;
222
(gdb)
223 default:
224 if (!Q_strcasecmp(mod->name + strlen(mod->name) - 4, ".obj"))
225 R_LoadObjModel(mod, buf, modfilelen);
226 else
227 Com_Error(ERR_FATAL, "R_ModForName: unknown fileid for %s", mod->name);
Title: Re: Current issues with 2.3 PPC build
Post by: keybounce on May 19, 2010, 07:09:57 am
Definitely byteswap.

- model 'models/aircraft/craft_arm_plate/plating.md2'
Debug: IDAliasHeader is 32504449, swapped is 32504449
Debug: Buf is 49445032, swapped is 49445032
Error: R_ModForName: unknown fileid for models/aircraft/craft_arm_plate/plating.md2

LittleLong() is a no-op on PPC.
The bytes are backwards (those are hex numbers).

What's the best way to solve this? Is it:
1. Redefine LittleLong to do the swapping
2. Conditional #defines so that the value is different on PPC?
3. Something else?
Title: Re: Current issues with 2.3 PPC build
Post by: keybounce on May 19, 2010, 07:34:04 am
Ok, it looks like the answer is "Something else".

- LittleLong() is used everywhere.
- IDALIASHEADER is from the quake engine, and playing with it is likely to have trouble elsewhere.

Code: [Select]
stbmac:23 Michael$ egrep IDALIASHEADER src/*/*.c
src/server/sv_world.c: case IDALIASHEADER:
stbmac:23 Michael$ egrep IDALIASHEADER src/*/*/*.c
src/client/renderer/r_model.c: case IDALIASHEADER:
src/tools/ufomodel/ufomodel.c:Com_Printf("Debug: IDAliasHeader is %x, swapped is %x\n", IDALIASHEADER, LittleLong(IDALIASHEADER));
src/tools/ufomodel/ufomodel.c: case IDALIASHEADER:
src/tools/ufomodel/ufomodel.c: case IDALIASHEADER:
Or perhaps not -- it's used once in the server, once in the client rendering, and in the ufomodel code where I'm finding it broken.

The other option is to byteswap all the data being loaded in from the file before processing. And then, it looks like the file is in i386 format as "native", and the PPC is the only thing that's going to need swapping.

Which leads to the next question: How much is the data not being properly swapped in/out during generation and processing over all of the code? Is the code supposed to be byte order fixed, or is it supposed to be "Lets assume x86"?
Title: Re: Current issues with 2.3 PPC build
Post by: Mattn on May 19, 2010, 11:33:39 am
maybe SwapInit() isn't calles for ufomodel?
Title: Re: Current issues with 2.3 PPC build
Post by: keybounce on May 19, 2010, 05:11:21 pm
stbmac:23 Michael$ find src -name '*.c' -print | xargs grep SwapInit
stbmac:23 Michael$

Can't find any call to SwapInit anywhere.
Title: Re: Current issues with 2.3 PPC build
Post by: keybounce on May 21, 2010, 11:07:21 pm
Problem has been traced to the libsdl port in Macports.
Workaround testing in progress.
Title: Re: Current issues with 2.3 PPC build
Post by: keybounce on May 22, 2010, 06:51:09 am
Initial report, 30076, PPC mac.

First, I have one locally compiled map -- wilderness -- for testing. Most of the time I can start a skirmish. About 25% of the time, I get an error like this:

Code: [Select]
Change gametype to 'Skirmish mode'
------- Loading game.dylib -------
Reading symbols for shared libraries . done
LoadLibrary (./base/game.dylib)
==== InitGame ====
checksum for the map 'wilderness': 3167195999
ufo script checksum 229509907
Created AI player (team 7)
-------------------------------------
Connecting to localhost...
connection attempt from loopback connection
load material file: 'materials/wilderness.mat'
S_LoadSound: Unrecognized sound file type.
S_LoadSound: Could not find sound file: 'ambience/firesoft'
added light, ambient=0.476837
Starting the game...
Michael has joined team 1
music change to PsymongN5 (from van_theme)
(player 0) It's team 1's round
Michael has taken control over team 1.
********************
ERROR: Unknown model type in R_DrawEntities entity chain: 0
********************
Shutdown server: Server crashed.
==== ShutdownGame ====
Shutdown gametype 'Skirmish mode'
0   ufo                                 0x001c4a3c Sys_Backtrace + 36
1   ufo                                 0x001378fc Com_Error + 400
2   ufo                                 0x0017d414 R_DrawEntities + 608
3   ufo                                 0x00189224 R_RenderFrame + 548
4   ufo                                 0x00047b70 CL_ViewRender + 428
5   ufo                                 0x00019aec SCR_UpdateScreen + 384
6   ufo                                 0x00017d68 CL_Frame + 416
7   ufo                                 0x00139af8 Qcommon_Init + 2596
8   ufo                                 0x0013a1c4 Qcommon_Frame + 180
9   ufo                                 0x001c4e68 main + 356
10  ufo                                 0x00002874 start + 68
music change to van_theme (from PsymongN5)
music change to van_theme (from van_theme)
GDB does not trigger (I'm running it from gdb), so I can't actually investigate what's happened.

2. Screen resolutions are things like 1024x768, while in window mode. There is no subtraction for the two menu bars, so the bottom of the map does not display. This didn't happen in the past -- not only were there many more resolutions available, I could manually go into the config file and put 960x720 in. Not anymore.

3. I cannot specify GLSL shaders. Attempting gives me the error messages
Invalid value for cvar r_programs
Invalid value for cvar r_programs
Modified render related cvars
not using GLSL shaders
Invalid value for cvar r_programs
Invalid value for cvar r_programs
Modified render related cvars
not using GLSL shaders

4. Graphic display is messed up. The map ground is funky garbage. Again.

5. Default display settings are too dark. Again, macs tend to have calibrated displays because the OS includes a good calibration tool -- the default should be "use system default". Looking in the config file, there is a line for

set vid_ignoregamma "1" a

However, setting that to 1 (default was 0) did not actually work. Raising brightness seemed to be more important than gamma.

6. Attempting to start a campaign game brings up a world map where everything on the globe is too dark to see.

More information as I test more.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 22, 2010, 11:51:49 pm
Previously, I had built 26262 (trunk) and it seemed to be working OK. Now, I am unable to get 30076 (branch) to work correctly. My test map is wilderness.

26262 was able to display a map pretty much OK. There were occasional lighting issues, with no rhyme or reason to them -- things would appear dark. Sometimes it could be fixed by moving the mouse off window and back on. Sometimes by going down to level 1 and then back up. Never anything consistent. With wilderness specifically, in the upper right corner I can tell that there is a natural rock staircase, but there's also some strange stuff that I can't figure out by looking at it.

30076 is giving graphics that are mostly garbled, constantly too dark. But oddly, the water and wood dock staircase of Wilderness is almost perfect.

Here are some screen shots showing what I'm referring to.

(NB: going into a skirmish will sometimes give me a crash from the battle server, with no indication of why.)

Hardware: PPC mac mini, 9300 video card, 9200 driver. I don't think it has core image support. Will not let me select GLSL shaders.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 22, 2010, 11:55:18 pm
Sadly, I have to upload each screenshot separately.

Note that in 26262, I could specify 960 x 720 for the window size. At the time, I was using a 1024x768 monitor, and needed that. (Now I'm using a widescreen TV and the DVD drive ...)

Here's the water / dock corner, with an example of the old lighting issue.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 23, 2010, 12:00:26 am
And here is the new (30076) build with the graphics.

Sadly, the water corner makes too big a file to upload here.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 23, 2010, 04:33:39 am
Looking at 30076 on a different machine.

This one is an iBook g4, with a 9550. This supports GLSL shaders and core image.

Turning on GLSL shaders fixes the lighting nicely. But it raises the seconds per frame up to 25.

No shaders works ... well, more-or-less the same. I did find one camera position where the lighting flicker was very apparent. Here are two screen shots, with no motion, key pressing, or anything -- just an example of the back-and-forth lighting issue.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 23, 2010, 04:35:59 am
EDIT: Make that a 9550 mobility. (Apparently, "mobility" is an important distinction.)

Finally, here is a look at the rear strip on the wilderness map. This back portion looks so different -- like a sharp knife -- that I'm hoping it gives someone who knows what's happening a forehead slap.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: Mattn on May 23, 2010, 08:33:01 am
if you have the time - please do some revision hoping to get the revision that reintroduced the lightmap artifacts on the ppc - did you try to disable glsl on the ppc, too? or only on the intel mac?
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 23, 2010, 08:45:05 am
I do not have an intel mac. Both machines are PPC.

I will do revision hoping tomorrow. The machine that could use GLSL shaders was too slow -- 25 seconds per frame -- to be usable.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 23, 2010, 06:28:06 pm
First revision hop -- {20090930} - 26389 - waits forever for the game server.

===
The opening movie that plays when you start UfoAi for the first time plays static noise instead of sound. Playback is very jerky on both of these machines.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 23, 2010, 07:02:31 pm
Just a quickie.

I get lots of messages like
S_LoadSound: Unrecognized sound file type.
S_LoadSound: Could not find sound file: 'geoscape/mail'
S_StartLocalSample: Failed to load geoscape/mail

Yet I've got those files:
stbmac:trunk Michael$ ls base/sound/geoscape/
.svn/           info.wav        newmission.wav  standard.wav   
attack.wav      mail.wav        promotion.wav   ufospotted.wav 

I'm not building the zip files; is that the issue?
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: Mattn on May 23, 2010, 08:07:30 pm
no, the problem is with libogg.dylib and libvorbis.dylib - sdl_mixer can't load them - see http://ufoai.ninex.info/wiki/index.php/Compile_for_Mac#If_OGG.2FVorbis_Is_Broken_.28Some_Sounds_Won.27t_Play.29
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 23, 2010, 08:43:00 pm
Still seeing
S_LoadSound: Unrecognized sound file type.
S_LoadSound: Could not find sound file: 'ambience/firesoft'

===
26425 2009-10-03 plays ok
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 23, 2010, 09:55:22 pm
{20091019} (26701) plays fine. Map compiler has byteswap issues, but existing map works.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 23, 2010, 11:16:57 pm
{20091120} - {20091124} all have byteswap issues.
{20091130} fails to link -- _R_BindTexture not found.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 24, 2010, 12:20:35 am
... And rm -rf debug-darwin-universal makes it recompile everything, and work.

There is a makefile flaw somewhere; I'm guessing some header change that should, but does not, trigger some recompile somewhere.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 24, 2010, 01:12:03 am
Trunk 28000, dated 2010-01-13, works just fine.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 24, 2010, 04:38:00 am
Trunk 29900.
Battlescape works just fine.
However, the 3d geoscape is always dark, never sees sunlight.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 24, 2010, 05:29:59 am
Trunk 29910.

Battlescape works.
"Abort" button does NOT -- the first soldier gets skill notes printed on the terminal for skills 0-7, and starts to get a note printed for skill 8. Then the game goes into infinite CPU.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 24, 2010, 06:16:20 am
"Confirmed".

29900, one test, abort worked.
29910, four tests, abort worked once.

Yet the code doesn't seem to change between those two. The only code difference is in the base defense callbacks.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 24, 2010, 07:00:01 am
29918, graphics working.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 24, 2010, 07:32:37 pm
29921, graphics "working".

Here's the first problem. The spectacular failure of 30076 isn't there, but the excessive shadow issue isn't going away, and seems to be getting worse -- the back strip is starting to show issues in this build. The excessive shadows are so bad in this version that I spent about 20 minutes trying various things to make them go away; nothing was successful. I can generate "flicker views", although not as spectacular as the 30076 "all wrong / all right" flicker.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: Mattn on May 24, 2010, 08:44:04 pm
http://ufoai.svn.sourceforge.net/viewvc/ufoai?view=rev&revision=29921 doesn't make sense - only comments were changed.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 25, 2010, 12:52:49 am
29926, branch, 9550 mobility, mac mini, skirmish working perfectly. No bad shadows. 3D geoscape is dark

29926 branch, 9300, iBook G4, skirmish shows bad shadows. 3d geoscape is dark

===
I'm not testing every revision. If I list revision A and B, then any change between A and B may have happened.

Right now I'm stepping once per day, as I'm almost at "today", and the spectacular failure isn't too far in the "future".
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 25, 2010, 01:50:54 am
29998, reversed: iBook g4 is fine, mac mini has bad shadows.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 25, 2010, 02:09:41 am
30030, mac mini good, ibook has bad shadows.
Retesting 29998.
... and the list of what gets recompiled is NOT the same on both machines.

Full wipe and recompile next.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 25, 2010, 03:15:38 am
29998: iBook G4 initially was perfect display. After either f8 or f9 (expose/spaces), it changed and added in some shadows.

It altered shadows if I used f8, f9, or f10. It never removed the shadows completely.

Macmini had extra shadows the entire time.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 28, 2010, 08:03:17 am
NAARG.

I thought I had tracked down the spectacular failure case to being the downloaded maps. But "farm" displays perfectly with the downloaded map.

Sigh. The downloadeds seem to be worse, but not guaranteed bad.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: Mattn on May 28, 2010, 08:49:54 am
that sounds like there is an endian issue in loading or saving the lightmap data - because they are calculated on a little endian system
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 28, 2010, 09:36:29 pm
Which is what I thought, and was hoping to find.

But as I said, I got a perfect load on "Farm" (downloaded). Other than it looks like completely flat ground, I didn't see anything else different from the other maps.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 29, 2010, 01:23:57 am
... and 4 more loads of farm all gave the "spectacular failure" that I'm used to.

Compiling all the farm tiles locally now...
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: keybounce on May 31, 2010, 07:30:06 pm
that sounds like there is an endian issue in loading or saving the lightmap data - because they are calculated on a little endian system

Ok, where in ufo is the lightmap data read in, and what values should be seen? Do the lightmap values change as you rotate the map? (What's displayed does change as the map is rotated, and sometimes on a per-frame toggle.)

Lets see if we can get some debugging patches, and then see what values are read/used on ppc/x86.

Oh yea -- Local compiled farm maps are "mostly good" with minor issues, and the rare, occasional good.
Title: Re: Current issues with 2.3 PPC build (Now with Graphics!)
Post by: Mattn on May 31, 2010, 09:18:15 pm
the lightmap data does not change - it's loaded to a texture in r_model_brush.c R_ModLoadLighting - but this data is only stored in byte - so there can't be any endian issue - so it must be later (when uploading - maybe wrong opengl pixel format) - R_CreateSurfaceLightmap or somethign like that.