UFO:Alien Invasion

Development => Coding => Topic started by: arisian on March 18, 2010, 12:36:46 am

Title: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 18, 2010, 12:36:46 am
So, the 2D geoscape has these nice lights that show up on the "night" portion of the globe.  They're nice for several reasons, ranging from "it's pretty" to "it improves immersion" to "it marks the population centers" to "it helps you figure out where stuff is in the dark" (helping to counteract the issues with low ambient lighting).

The method used for the 2D geoscape isn't directly applicable to the 3D version, but I took a look at it this afternoon and it's fairly straightforward to render the lights as emissive sources, which makes them glow nicely at night.  The only trouble with this is that they don't "turn off" during the day, though they're a lot less obvious during the day because of the lower contrast with the background.

I've attached a diff (based on the current trunk; it's the output of `svn diff` run in src/client/ ) of the simple changes I made; I make no claim that my little hack is the best way to implement this feature, but it was the quickest way for me to get nightlights.  I've also attached the image that I used as an emissive light texture; it's basically just a thresholded version of the standard nightime geoscape texture.

If someone who's actually familiar with the overall code structure wants to add this feature properly (eg. as an option in the menu, or as a selectable "overlay" like the nation borders or radar overlays, etc.) feel free to use it; if the devs don't think this feature is something they want, that's fine too, I'll just keep it around for my personal use.
Title: Re: Nightime city lights in 3D geoscape
Post by: Mattn on March 18, 2010, 08:20:22 am
applied this patch to trunk - thanks. if you know opengl and have the time and the interest... we are searching for new coders and could really need some opengl coders, too.
Title: Re: Nightime city lights in 3D geoscape
Post by: MCR on March 18, 2010, 09:11:49 am
Great idea, arisian & thanx 4 helping here also. 8)

Had the same idea already written down on paper (I love glowing lights  ;)), but no knowledge about implementing it  ;)
Great to see improvements happening on the geoscape also...

My feature requests regarding graphical upgrades to the geoscape would be also this:

- We could add a normalmap to the geoscape also to enhance visual quality. The engine seems to be very capable of great bump-mapping, so this should not be a big problem, but for me it is  ;). Mattn, any ideas !?

- This idea is actually stolen from here: http://colabti.org/irclogger/irclogger_log/ufoai?date=2010-03-16
Look @ 15:05. (The discussion starts 14:42.)
So it is basically the idea of reflecting the season-change onto the globe  8)
I think this is a great idea & would enhance immersion into the game for the player big time !!!

We do not need to use 12 different pictures for each month, but could blend 4 pictures, although I do not know if this would be a technical problem when we have to blend day/night & seasons & add a bumpmap to all of this.
Mattn, I think your technical expertise is needed once again here  ;)
Title: Re: Nightime city lights in 3D geoscape
Post by: arisian on March 18, 2010, 07:15:55 pm
Actually, this the first time I'd done any openGL coding (I'm a fairly experienced C/C++ coder, and I've done a decent amount of graphics work, just not with openGL).  That's why it took me all afternoon; I was learning the relevant parts of openGL as I went :)

I'd also like to state again that the patch I posted is a somewhat brittle way of implementing the feature, and has the downside of making the cities look white-ish during the day (ie. everyone keeps their lights on 24/7).

Anyway, I've certainly got interest; I like the ideas behind ufoai a lot.  Time, on the other hand, is not something I'm exactly swimming in.  I might be able to do small things from time to time, but I'm just not in a place where I can claim I'll be reliable in doing things in a timely fashion.

The other trouble is that the things that I'd most like to fix/improve about the current development version are things which would require re-writing substantial portions of the codebase (for example, .50 caliber sniper rounds bouncing off rice paper dividers, or allowing missions to be fought in pseudo-realtime a la the UFO:After* games as well as the turn-based mode.  I love turn-based, but it makes thing so darned *slow*, especially given the frequency of terror missions.  I find myself using the "auto-resolve" frequently just because I'm sick of trying to balance good tactics with the desire to explore the map and find the enemy before my hair turns gray).

That said, I may have time to occasionally implement some small feature or other; do you use any formal feature/bug tracking system that I could look at to see what things are awaiting implementation/fixing, or should I just use the wiki? Nevermind, I found the sourceforge tracking systems, though things there seem more like random feature requests than an ordered list of things that need doing.

Also, what's the best way of submitting changes?  I'm currently using SVN anonymously, so I assume I can't do commits that way, and posting patches in the forums seems sub-optimal if I'm going to do this more than once.
Title: Re: Nightime city lights in 3D geoscape
Post by: Mattn on March 18, 2010, 08:42:08 pm
either use our patch tracker https://sourceforge.net/tracker/?atid=805244&group_id=157793&func=browse or submit one more patch that fits the coding guidelines (http://ufoai.ninex.info/wiki/index.php/Coding_guidelines) and i can grant you write access to the svn

about the feature tracker: i know, it's that everyone is posting his ideas, but we have way too little heads and helping hands to implement them. there will be a roadmap after 2.3 is released. but it's not yet carved into stone.

i know the time issue myself, too and i think everyone who is no student anymore knows it, too - but for me ufoai is my obsession ;)

it would be quite cool to welcome you in our team - we could need every helping hand.
Title: Re: Nightime city lights in 3D geoscape
Post by: arisian on March 19, 2010, 01:32:54 am
Hmm, looks like my previous patch didn't follow all the guidelines; I hadn't read them because I wasn't really expecting it to get put into trunk without revision.  It won't be hard to fix though (mostly things like tabs vs. spaces, comment style); I'll try to post a better version tomorrow.
Title: Re: Nightime city lights in 3D geoscape
Post by: MCR on March 19, 2010, 08:19:54 am
 8) to see you motivated  ;D
& sorry for misspelling your name in my last post (I've corrected it).
I think there is a lot of stuff which could have been done, but cannot by me, because of my lacking programming-skills.
So I try to improve this project on other fronts  ;), but it is really great to see another programmer here.

Title: Re: Nightime city lights in 3D geoscape
Post by: Mattn on March 19, 2010, 01:17:35 pm
Hmm, looks like my previous patch didn't follow all the guidelines; I hadn't read them because I wasn't really expecting it to get put into trunk without revision.  It won't be hard to fix though (mostly things like tabs vs. spaces, comment style); I'll try to post a better version tomorrow.

you don't have to - as your patch is already in the svn... ;)
Title: Re: Nightime city lights in 3D geoscape
Post by: MCR on March 21, 2010, 06:42:37 pm
I like the lights  ;D
Title: Re: Nightime city lights in 3D geoscape
Post by: arisian on March 23, 2010, 03:08:00 pm
I decided that if I'm actually going to be the one to implement this feature, I want to do it right, rather than just the proof-of-concept I posted before, so this is a new, vastly improved version.

Now the city lights only show up where it's actually nighttime, and are prettier to boot.  Also, I've implemented bump-mapping and specularity-mapping and texture-blending as well.  This means that the 3D geoscape now has topographic rendering with lightsource-dependent shadows, and sunlight reflects off the water.  It also means that the visuals now correctly represent seasonal changes with smooth, seamless changes over the course of the year.

These new rendering features are all handled by a GSLS shader program, but the older rendering methods all still work the way they always did (ie. there's no change to the way that nation overlays and the like appear).  There is also a new option in the "options->game" menu that lets you switch between the new globe renderer and the old one (though you need to restart the program after switching the option before it will take effect).

This is particularly important because the new version has higher requirements than the old one.  If your graphics card doesn't have the features required, it will automatically fall back on the old rendering system, but I think that any reasonably modern graphics card should work (if anyone has issues, please let me know).  It also requires more system memory, especially if you use the high-res textures.

Speaking of textures, I've been using higher resolution images for the globe surface, which look much crisper and prettier when you zoom all the way in.  The disadvantage is that they are, well, bigger.  This means that they take longer to load into memory, and they take up more memory once loaded.  It takes several extra seconds on my system to bring up the geoscape, with an additional ~3 second pause near the beginning of a new month while it loads the texture for that month.

That said, I can't upload the high res images here because they're ~25Mb a piece, and there are 12 of them.  I have therefore uploaded lower resolution versions so people can actually see the system working, but I really think the high-res versions are prettier; I just don't know what the best way to distribute them is.

The current season system consists of one texture image per month; the images I'm using come from NASA's Blue Marble project (as did the images I started with when making the bump map, city lights, and gloss textures, though those required some modification).  The system blends two textures, one associated with the beginning of the month, and the other associated with the end of the month (ie. the beginning of the following month).  The two textures are combined according to how far through the month it is, so things change slowly and smoothly.  If you want to see it in action, I recommend modifying the campaign.ufo to set the "minimum happiness" to something negative; otherwise, you'll generally loose after just a few months if you don't do anything but watch the globe spin, and you really need to speed things up to the fastest speed and watch for a few months to get the full effect :)

I've attached a diff relative to the current svn, along with a zipfile that contains the new images needed (extremely low-res versions only, I'm afraid, since the forum only  accepts up to 1Mb of attachments; if someone wants to give me svn access so I can upload the high-res versions, my sourceforge username is "arisian"...;)).

Please let me know what you think, or any issues you might have, especially if you experience anything that looks "wrong;" I've only been able to test this on my own system.  It looks very pretty for me, but I'm not 100% confident that it will look the same on other brands of graphics cards (my current one is an nvidia geforce 9800gt).
Title: Re: Nightime city lights in 3D geoscape
Post by: Kildor on March 23, 2010, 03:48:40 pm
I can`t compile the game with your patch, since I get this errors:
Code: [Select]
\src\client\renderer\r_program.c||In function 'R_LoadShader':|
\src\client\renderer\r_program.c|349|warning: passing argument 3 of 'qglShaderSource' from incompatible pointer type|
\src\client\renderer\r_sphere.c||In function 'R_InitBumpmap':|
\src\client\renderer\r_sphere.c|213|error: 'uint' undeclared (first use in this function)|
\src\client\renderer\r_sphere.c|213|error: (Each undeclared identifier is reported only once|
\src\client\renderer\r_sphere.c|213|error: for each function it appears in.)|
\src\client\renderer\r_sphere.c|213|error: expected ';' before 'max_instructions'|
\src\client\renderer\r_sphere.c|217|error: expected ';' before 'res'|
\src\client\renderer\r_sphere.c|223|error: 'max_instructions' undeclared (first use in this function)|
\src\client\renderer\r_sphere.c|226|error: 'max_indirections' undeclared (first use in this function)|
\src\client\renderer\r_sphere.c|233|error: 'res' undeclared (first use in this function)|
\src\client\renderer\r_sphere.c||In function 'R_SphereRender':|
\src\client\renderer\r_sphere.c|316|warning: implicit declaration of function 'glActiveTexture'|
\src\client\renderer\r_sphere.c|353|warning: implicit declaration of function 'glClientActiveTexture'|
||=== Build finished: 8 errors, 3 warnings ===|

But I like the idea, it will be excellent addition to the game.
Title: Re: Nightime city lights in 3D geoscape
Post by: BTAxis on March 23, 2010, 03:58:15 pm
The one thing that the seaons system as presented here is lacking is additional terrain maps that correspond with the seasonal textures. The current terrain map is base/pics/geoscape/map_earth_terrain.tga (http://ufoai.svn.sourceforge.net/viewvc/ufoai/ufoai/trunk/base/pics/geoscape/map_earth_terrain.tga?view=log). Its legend can be found here (http://ufoai.ninex.info/wiki/index.php/Mapping/Terrain_Types). More terrain maps will have to be created before the seasonal changes are committed.
Title: Re: Nightime city lights in 3D geoscape
Post by: Kildor on March 23, 2010, 04:00:37 pm
I think, we no need in smouth change of terrain map, 12 discret maps will be enought, will not?
Title: Re: Nightime city lights in 3D geoscape
Post by: arisian on March 23, 2010, 04:01:14 pm
Hmm, looks like you're using a different build environment; are you building on Windows by any chance?  I run Linux, so I appreciate feedback from those on other platforms.  In any case, try this version; hopefully it will be more friendly with non-gcc compilers (the only difference is using slightly longer forms of names in a few places).
Title: Re: Nightime city lights in 3D geoscape
Post by: BTAxis on March 23, 2010, 04:01:46 pm
I think, we no need in smouth change of terrain map, 12 discret maps will be enought, will not?

Yes.
Title: Re: Nightime city lights in 3D geoscape
Post by: arisian on March 23, 2010, 04:03:50 pm
The "smooth change" is really just so that there isn't a visually abrupt change at the end of the month; there are still only 12 discrete maps, it's just that the change from one map to the next is gradual rather than instant.
Title: Re: Nightime city lights in 3D geoscape
Post by: Kildor on March 23, 2010, 04:07:05 pm
Yes, I use WinXP and code::blocks package
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 23, 2010, 04:34:52 pm
In terms of terrain maps, it's really just an issue of expanding cold/arctic regions during the winter.  I can do it myself, but I'm unclear as to the difference between "cold" regions and "arctic" regions, and also on the overlap between either of these and "mountain" regions.  The current texture image (map_earth_day.jpg) seems to have been taken sometime around March.  There are clearly "mountain" regions that have snow on them, and the "cold" region overlaps both green and snow-covered areas.

I can just make it up as I go, but if there are actual conventions I'd rather follow them.

Also, if anyone is still having issues compiling from the latest posted patch, please let me know.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on March 23, 2010, 05:03:37 pm
There are no conventions that I know of. The difference between arctic and cold is that arctic has no vegetation while cold may still have evergreens and such. It's basically a gradation in coldness.

Also, the terrain map may not necessarily be that accurate. Some allowances may have been made for the cold-themed maps. With seasonal change that becomes less necessary.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 23, 2010, 05:57:35 pm
 8) 8) 8) Arisian !!!

BTW, have you read my feature request regarding this ?
It will be no problem to get the maps I suppose  ;D

https://sourceforge.net/tracker/?func=detail&aid=2972849&group_id=157793&atid=805245
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 23, 2010, 05:59:54 pm
The original was from NASA, in a moment comes the link:
http://visibleearth.nasa.gov/view_set.php?categoryID=2363
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 23, 2010, 06:03:51 pm
nice - i will give it a try - but please extract the shaders to base/shaders and use the functions in r_program.c to load and compile them.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 23, 2010, 06:19:26 pm
https://sourceforge.net/tracker/?func=detail&aid=2975340&group_id=157793&atid=805244

patch tracker item created
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 23, 2010, 06:26:09 pm
some of your changes are applied to trunk (not the feature, but shader stuff like qglUniform2fv)

you should have a look at the R_UseProgram function
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 23, 2010, 07:27:57 pm
https://sourceforge.net/tracker/download.php?group_id=157793&atid=805244&file_id=367909&aid=2975340

updated version of the patch
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 23, 2010, 07:34:40 pm
Can't wait to test thiz one  ;D
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 23, 2010, 07:43:25 pm
I'm working on integrating my code with the functions in r_program.c; if I'd known they were there, I probably would have used them in the first place.  Oh well, this is what comes of working on a totally unfamiliar codebase while simultaneously teaching myself openGL and GLSL ;)

I'll post another version of the patch once I've got things working properly using existing functions.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on March 23, 2010, 08:28:21 pm
Arisian, would it be possible to do what you did with the city lights to the nation overlay as well? I'm thinking of making the edges of the nation's territories glow a little so they stand out on the dark side. I think it would look quite good.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 23, 2010, 08:32:38 pm
Great folks, to see all this tuning happening  ;D
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 23, 2010, 08:37:09 pm
here is a version i tried to hack together using the existing stuff - but i might have missed something here and there - untested
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 23, 2010, 08:38:02 pm
btw. the biggest problem i see with this is that most people out there won't have more than 4 texunits
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: geever on March 23, 2010, 10:01:22 pm
btw. the biggest problem i see with this is that most people out there won't have more than 4 texunits

And if it hangs for 3 secs on a gf 9800gt it will be unusable for quite some players (and devs). 25Mb images are huge IMHO. The current memory usage of UFO:AI is already too high.

-geever
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 23, 2010, 10:29:31 pm
that's why he's using lower res textures - the high res textures won't be included but will be offered as a separate pk3 that can be downloaded. also this feature isn't activated for none-glsl cards anyway.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Yatta on March 23, 2010, 11:53:51 pm
I really dont know much about graphical optimisation, but cant the lights picture be divided in sub parts for continents (also you dont need light texture for the oceans) ? It would allow higher res - which is quite needed to keep the 'crisp' feeling of light souces (the blur does not really fit).

Also, since its just white light, maybe there some way somewhere to optimise the file/process by reducing the palette ? Once again I dont know much in that domain.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 23, 2010, 11:59:45 pm
Okay, here's another new-and-improved version; it integrates better with the existing systems.  Also, for the record, you don't actually need more than 4 texture *units*, you just need more than 4 texture addressable *coordinates.*  The major problem I was running into earlier, as it turned out, was that the code in r_program.c was refusing to give me the texture references I was asking for because it was capping them at the 4 units that my card has.  The code now checks both GL_MAX_TEXTURE_UNITS and GL_MAX_TEXTURE_COORDS, and uses the latter when deciding whether functions like R_SelectTexture succeed or not.

It would also be easy to get rid of the changing seasons, which would require fewer texture units, less memory, and so forth.  It seemed like a nifty feature, but it may not be worth the resources required.  It would also be fairly easy to do the rendering of the city lights as a separate pass, which would mean another texture address that wouldn't be required, but it seemed better to do it all in one pass.  Besides, I had just learned GLSL, so I wanted to flex its muscles a little :)

In terms of the light textures, the light is actually off-white at the moment (slightly yellow looks more correct).  It would be easy enough to simply use a grayscale image, which would certainly mean a smaller file on disk, but I don't think it would help much in terms of graphics card memory (someone correct me if I'm wrong).
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 24, 2010, 12:06:55 am
The high-res textures are now available at:

http://cs.jhu.edu/~ben/graphics/ufoai/
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Kildor on March 24, 2010, 05:04:13 am
arisian, with last version of your patch I get the same problems with compiling:
Code: [Select]
||=== ufo, windows_debug ===|
ufoai\src\client\renderer\r_program.c||In function 'R_LoadShader':|
ufoai\src\client\renderer\r_program.c|360|warning: passing argument 3 of 'qglShaderSource' from incompatible pointer type|
ufoai\src\client\renderer\r_program.c|360|note: expected 'const GLchar **' but argument is of type 'char **'|
ufoai\src\client\renderer\r_sphere.c||In function 'R_SphereRender':|
ufoai\src\client\renderer\r_sphere.c|228|warning: implicit declaration of function 'glClientActiveTexture'|
.objs\client\src\client\renderer\r_sphere.o||In function `R_SphereRender':|
ufoai\src\client\renderer\r_sphere.c|228|undefined reference to `glClientActiveTexture'|
||=== Build finished: 1 errors, 2 warnings ===|
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 24, 2010, 08:08:56 am
applied to trunk now - some todo comments are left and some cleanup should still be made though.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 24, 2010, 08:29:33 am
the specular lighting needs some work imo. it's not realistic to have a bright light spot on one particular part of the globe
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 24, 2010, 08:36:18 am
some screenshots

http://mattn.ninex.info/screenshots/ufo01.png
http://mattn.ninex.info/screenshots/ufo02.png
http://mattn.ninex.info/screenshots/ufo03.png
http://mattn.ninex.info/screenshots/ufo04.png
http://mattn.ninex.info/screenshots/ufo05.png

03 this is the specular light spot i was refering to in my previous post
note the snow border on 04 and 05
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on March 24, 2010, 11:53:31 am
Okay, I've tried what was committed in trunk up until r29121, and I must say it looks absolutely appalling. The textures have huge jpg compression artifacts. Since the memory usage for the texture is going to be the same in any case, I must insist on better quality textures (at that resolution). I will see what I can do about this.

How did you make the lights texture? Simple level adjustments over the original night texture?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 24, 2010, 12:47:01 pm
Wow !
This screenshots looks great !
Even better than I imagined it to be  ;)
Immediately have to test this...  ;D
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 24, 2010, 12:52:43 pm
BTW, IMHO we should include the highest quality textures also, we can always implement a downgrade option in the graphic-options for those who do not have fast gfx-cards, but this will be a top feature for people with modern middle to high-end cards  ;)

I can't wait to test this  ;D
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 24, 2010, 12:58:41 pm
Okay, I've tried what was committed in trunk up until r29121, and I must say it looks absolutely appalling. The textures have huge jpg compression artifacts. Since the memory usage for the texture is going to be the same in any case, I must insist on better quality textures (at that resolution). I will see what I can do about this.


jpg is really often bad in terms of artifacts, maybe you could try maximally compressed pngs ?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 24, 2010, 01:11:55 pm
I have made some experiments with a very detailed normalmap of map_earth_day @ a resolution of 4096x2048:

1. jpg Full Compression: ugly, many artifacts, 1.18 MB
2. jpg HQ: still ugly, but much better now, 8.31 MB
3. png Full Compression: almost no artifacts, but 14.3 MB

Hope that helps in the decision finding process  ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 24, 2010, 01:15:44 pm
the decision is that we won't include such high res big images - because we want that older machines are still able to load them in acceptable times.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 24, 2010, 01:51:01 pm
FYI, the "high-res" textures that I started with are, in fact, .pngs; with optimized lossless compression, the "season" images are ~20MB each at 8192x4096.  The gloss and night_lights maps can be losslessly compressed to 280Kb and 600Kb respectively.  The issue is less how much space they take up on disk, and more how much space they take up in system memory when the game is running.  The amount of compression doesn't affect runtime memory useage much; the only thing that's really important at runtime is the resolution.

Also, if you haven't yet downloaded the high-res textures from the link I posted, wait another 24 hours and I'll have replaced them with versions that are the exact same quality, but smaller on disk (fully optimizing PNGs takes a long time, but it's worth it for lossless compression; the season images are being reduced from ~30Mb each to ~20Mb each).

In terms of what winds up in the SVN trunk, I think it should be somewhere in the middle.  The versions that I attached to my post yesterday were *extremely* low quality because of the fact that I could only attach 1Mb total to the forum post.  I think we can probably afford more than 60Kb per globe image in the "final" version ;)

Feel free to mess with the specularity; there's the specularExp variable that defines how "tight" the spot is, and then there's map_earth_gloss.jpg which defines how shiny different parts of the globe are.  Currently, water is maximally shiny, which should probably be tuned down.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on March 24, 2010, 05:22:36 pm
So any thoughts about the nation overlay?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 24, 2010, 07:14:57 pm
What about the nation overlay; has it stopped working?  I admit I haven't checked since a few versions back, but I thought that the new rendering system hadn't affected it.  If it's broken, I'll take a look at, though I probably won't have time to do anything about it until tomorrow.  It shouldn't be hard to fix, given that I know it was working with the new renderer just fine the other day.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on March 24, 2010, 07:16:20 pm
I was referring to my earlier question (http://ufoai.ninex.info/forum/index.php?topic=4561.msg35326#msg35326).
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 24, 2010, 07:20:40 pm
Ah, sorry, I'd missed that post.  In any case, yes, it should be fairly straightforward.  Are you looking for edges that always glow, or only at night?  Also, do you just want the edges to glow, or the whole overlay? (the latter is actually easier to implement; I suspect the former will require either an extra texture or some real fancy GLSL).
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 24, 2010, 07:51:19 pm
imo always glow would be cooler - and only the edges should glow.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on March 24, 2010, 08:28:06 pm
Agreed.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 24, 2010, 09:43:01 pm
Sounds good; I'll let you know when I've got something.  Incidentally, is it better to post such things here, or to the tracking system, or both?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 24, 2010, 09:45:07 pm
if you have a patch, please publish it on the tracker - discussion for features is better here. but if we are discussing about the patch => tracker please - complicated eh? ;)

btw. attached is a patch of a water shader from nullad - would be cool if you could integrate something like that, too
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 25, 2010, 03:22:50 am
Here's a screenshot of a first attempt at glowing nation-overlay borders; is this what you had in mind?  It requires all of about 3 lines of extra code, and an extra texture.  The texture I've used here could probably use some work, but I'd like feedback on what direction people want to go with it first (ie. should glowing borders be more/less bright? more/less wide? more/less opaque? etc.).
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Destructavator on March 25, 2010, 05:35:55 am
Looks nice IMO.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Bartleby on March 25, 2010, 07:20:35 am
..., but I'd like feedback on what direction people want to go with it first (ie. should glowing borders be more/less bright? more/less wide? more/less opaque? etc.).
hm... u will be able to switch that feature off, right? than it would be cool. well. nice work anyway :).
i would make it somewhat less bright and less wide. opaque seems ok.
is the texture near the pole still blurred?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 25, 2010, 07:37:26 am
the borders could maybe be a little bit thinner imo. but it's looking already quite nice.

btw. if you could have a look at the specular light stuff again that would be sweet. playing with the specularExp value didn't turn out into reasonable results.

while you are on glsling the geoscape rendering, athmospheric scattering, glowing for the sun and other neat stuff would be damn sweet ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 25, 2010, 09:43:37 am
I have to say it once again: Arisian, it is great to see all that stuff happening  ;D
 8)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: H-Hour on March 25, 2010, 10:20:16 am
Looks great arisian. I would agree with mattn, thinner borders. Brightness is good. Opaque-ness, I think good but will have to see it in thinner borders.

Also, if you go back to revisit the ocean reflection, I think it should be less bright and cover a larger area on the globe.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 25, 2010, 01:45:53 pm
First off, yes, you will be able to turn it on/off; it works the same way the nation overlay always has, except that the colored patches for nations now have glowing borders.

And I will work on improving the specularity and adding other rendering features, but that might take me a bit longer to have time for (the nation borders don't actually require much work).
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: misiek on March 26, 2010, 06:27:54 pm
I haven't seen new ufo: ai globe, but check this out:

http://www.youtube.com/watch?v=eS6tbUEn5_I

Maybe this could work as inspiration.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 27, 2010, 03:25:01 am
Just a quick status update; here's a couple of screenshots from my latest version.  I've toned down the specularity a bit, and worked on the glow texture.  I've also made some less visible improvements, including making my GLSL code not need so many texture units, reducing memory footprint, and having a proper tangentspace for calculating bump-map lighting.  I'm going to take a look at trying implement post-processing effects like rendered glow (as opposed to "glow" which is actually a hand-painted texture, which is what I've been using up to now). 
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 27, 2010, 07:28:52 am
very nice. please show us your patch - i like small commits ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: H-Hour on March 27, 2010, 10:46:57 am
Looks great arisian.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Bartleby on March 27, 2010, 11:01:43 am
that looks very good.
but now the sun also needs a texture ;).
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on March 27, 2010, 11:47:00 am
Actually, the sun needs to be done in shaders, probably.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 27, 2010, 04:06:40 pm
Okay, the patch is posted over at the patchtracker, though I'll post it here as well.  I have to post the texture-images here because the patch-tracker will only accept 256k of attachments.

I've only included one season texture in this tarball; just use symlinks (or copies on windows) to make all the season map names point to this file, or go grab updated versions of all the season textures from http://cs.jhu.edu/~ben/graphics/ufoai/ (the new files aren't there yet, but the links are; the files will be there in a little bit, when I'm done converting/compositing/resizing them all).
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 27, 2010, 04:08:40 pm
Code: [Select]
R_LoadShader: geoscape_fs.glsl: Fragment shader failed to compile with the following errors:
ERROR: 0:46: error(#202) No matching overloaded function found dot
ERROR: 0:59: error(#202) No matching overloaded function found dot

users are reporting this error
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 27, 2010, 04:16:24 pm
Is this error only with the newest patch, or with any version of the GLSL shaders?  Can you post any info about their graphics cards?  It works fine for me, so it's a little difficult to debug; I suppose it might be some sort of difference between what version of GLSL their cards support, though I thought the dot product operator was always available.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 27, 2010, 04:17:06 pm
it's about the patch that is currently in the svn

Quote
GL_VENDOR: ATI Technologies Inc.
 GL_RENDERER: ATI Mobility Radeon HD 3670
 GL_VERSION: 2.1.9016
 GL_EXTENSIONS: GL_AMDX_vertex_shader_tessellator GL_AMD_draw_buffers_blend GL_AMD_performance_monitor GL_AMD_vertex_shader_tessellator GL_ARB_color_buffer_float GL_ARB_copy_buffer GL_ARB_depth_buffer_float GL_ARB_depth_texture GL_ARB_draw_buffers GL_ARB_draw_buffers_blend GL_ARB_draw_instanced GL_ARB_fragment_program GL_ARB_fragment_program_shadow GL_ARB_fragment_shader GL_ARB_framebuffer_object GL_ARB_framebuffer_sRGB GL_ARB_geometry_
 shader4 GL_ARB_half_float_pixel GL_ARB_half_float_vertex GL_ARB_instanced_arrays GL_ARB_map_buffer_range GL_ARB_multisample GL_ARB_multitexture GL_ARB_occlusion_query GL_ARB_pixel_buffer_object GL_ARB_point_parameters GL_ARB_point_sprite GL_ARB_provoking_vertex GL_ARB_shader_objects GL_ARB_shader_texture_lod GL_ARB_shading_language_100 GL_ARB_shadow GL_ARB_shadow_ambient GL_ARB_texture_border_clamp GL_ARB_texture_buffer_object GL_ARB_t
 exture_compression GL_ARB_texture_compression_rgtc GL_ARB_texture_cube_map GL_ARB_texture_env_add GL_ARB_texture_env_combine GL_ARB_texture_env_crossbar GL_ARB_texture_env_dot3 GL_ARB_texture_float GL_ARB_texture_mirrored_repeat GL_ARB_texture_non_power_of_two GL_ARB_texture_rectangle GL_ARB_texture_rg GL_ARB_texture_snorm GL_ARB_transpose_matrix GL_ARB_uniform_buffer_object GL_ARB_vertex_array_object GL_ARB_vertex_buffer_object GL_ARB
 _vertex_program GL_ARB_vertex_shader GL_ARB_window_pos GL_ATI_draw_buffers GL_ATI_envmap_bumpmap GL_ATI_fragment_shader GL_ATI_meminfo GL_ATI_separate_stencil GL_ATI_texture_compression_3dc GL_ATI_texture_env_combine3 GL_ATI_texture_float GL_ATI_texture_mirror_once GL_EXT_abgr GL_EXT_bgra GL_EXT_bindable_uniform GL_EXT_blend_color GL_EXT_blend_equation_separate GL_EXT_blend_func_separate GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT
 _compiled_vertex_array GL_EXT_copy_buffer GL_EXT_copy_texture GL_EXT_draw_buffers2 GL_EXT_draw_instanced GL_EXT_draw_range_elements GL_EXT_fog_coord GL_EXT_framebuffer_blit GL_EXT_framebuffer_multisample GL_EXT_framebuffer_object GL_EXT_framebuffer_sRGB GL_EXT_geometry_shader4 GL_EXT_gpu_program_parameters GL_EXT_gpu_shader4 GL_EXT_multi_draw_arrays GL_EXT_packed_depth_stencil GL_EXT_packed_float GL_EXT_packed_pixels GL_EXT_pixel_buffe
 r_object GL_EXT_point_parameters GL_EXT_provoking_vertex GL_EXT_rescale_normal GL_EXT_secondary_color GL_EXT_separate_specular_color GL_EXT_shadow_funcs GL_EXT_stencil_wrap GL_EXT_subtexture GL_EXT_texgen_reflection GL_EXT_texture3D GL_EXT_texture_array GL_EXT_texture_buffer_object GL_EXT_texture_compression_latc GL_EXT_texture_compression_rgtc GL_EXT_texture_compression_s3tc GL_EXT_texture_cube_map GL_EXT_texture_edge_clamp GL_EXT_tex
 ture_env_add GL_EXT_texture_env_combine GL_EXT_texture_env_dot3 GL_EXT_texture_filter_anisotropic GL_EXT_texture_integer GL_EXT_texture_lod GL_EXT_texture_lod_bias GL_EXT_texture_mirror_clamp GL_EXT_texture_object GL_EXT_texture_rectangle GL_EXT_texture_sRGB GL_EXT_texture_shared_exponent GL_EXT_texture_snorm GL_EXT_texture_swizzle GL_EXT_transform_feedback GL_EXT_vertex_array GL_EXT_vertex_array_bgra GL_IBM_texture_mirrored_repeat GL_
 KTX_buffer_region GL_NV_blend_square GL_NV_conditional_render GL_NV_copy_depth_to_color GL_NV_explicit_multisample GL_NV_primitive_restart GL_NV_texgen_reflection GL_SGIS_generate_mipmap GL_SGIS_texture_edge_clamp GL_SGIS_texture_lod GL_SUN_multi_draw_arrays GL_WIN_swap_hint WGL_EXT_swap_control
 using GL_ARB_texture_non_power_of_two
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 28, 2010, 12:42:26 am
bayo fixed it in current trunk: https://sourceforge.net/apps/trac/ufoai/changeset/29155
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 28, 2010, 01:59:23 am
Okay, I'm glad it got fixed.  I started trying to fix something else, and got bogged down; I'm having trouble fixing specularity properly because things like vertex and light positions don't seem to be relative to view coordinates, or the view isn't where it should be, or something.  Anyway, I'll let you know when I get it figured out; I suspect it's an issue that's being caused by the fact that we're not using a projective system, which means the "view location" doesn't have the same meaning it normally would.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 28, 2010, 09:52:22 am
can you please attach the textures? (those that are missing in the patch https://sourceforge.net/tracker/?func=detail&aid=2977653&group_id=157793&atid=805244
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on March 28, 2010, 10:54:31 am
nevermind, found them in your previous post - but the whole geoscape now only renders white for me.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on March 28, 2010, 05:53:33 pm
Are you sure that the new textures are being read rather than the old ones?  My guess would be that it's reading the old versions of the "season" textures, which would account for the all-white behavior.  Either download the full set of new textures from www.cs.jhu.edu/~ben/graphics/ufoai/ or make symlinks/copies from the season_00 texture from the forum post to all the other season_XX files.  Also, the game seems to prefer .PNG files over .JPG files; if there are files with the same name but different extensions, it loads the .PNG one and ignores the .JPG; if you've got versions of both, this might be why it's not loading the image you were expecting.

Honestly, though, you might not want to put that patch into the SVN; feel free to play with it, but there's still a bug in the way that specularities are calculated.  I'm still trying to figure out exactly what's going on with the view vector to get it "right"; it's close in that patch, but the specular highlights just aren't quite right.  I'll let you know when I get it fixed.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 31, 2010, 12:12:37 am
Thanx arisian for investing so much energy here.

So when this is finished we will have UFO-Fights over bump-mapped ever-changing territory with shining cities & glowing nation-borders & a glowing sun in the background to summarize this here a little bit  :D

I do not know ANY game with such features for the geoscape !!! ;)   8)

&
because the patch-tracker will only accept 256k of attachments.

Had to smile here, because I had the same problem  :P...
Great to see you achieving & optimizing your goals, I wonder what your next project regarding UFO:AI will be ?
I would have quite a few suggestions here, if you are interested...  ;)

Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: freegamer on March 31, 2010, 02:56:10 pm
I would consider having no bump map over the oceans as it looks wrong.  They should only change hue according to depth, not cast shadows.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on March 31, 2010, 06:06:41 pm
I would consider having no bump map over the oceans as it looks wrong.  They should only change hue according to depth, not cast shadows.
You have seen the final version already ?  ???
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: freegamer on April 01, 2010, 03:46:45 am
No, only the screenshots on the previous page.  There is bump mapping happening in the visible ocean.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 01, 2010, 01:43:31 pm
Just wanted to tell you that there are still massive problems with this patch in current trunk !?
Tested v29210 & there are massive problems:

1. The textures still look ugly.

2. The sunspot is not moving over the earth when time goes on like it should, but is always centered in players view, so if you move the earth around you also move the sunspot.

3. No Glowing Borders

Will attach some screens so you can see what is going on - Watch the time !
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 01, 2010, 01:47:01 pm

Here some additional screens which describe maybe a bit better what is going on:

I will try to integrate the high-res textures from you arisian, maybe this is the fault why it is not working ?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 01, 2010, 01:48:20 pm
But maybe this all is happening just because I am testing the version from April 1st  ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 01, 2010, 03:07:50 pm
I tested v29210 again, this time with arisian's new textures & IMHO we should really integrate the high-res versions. The details are just looking phantastic !

If it is important for us to keep the installer as small as possible we should rather lower the number of seasons & transitions. We could change the transition period from 1 to 2 months & save a lot of space, but still have the high-quality in-game.
I tested this on a really slow gfx-card (ATI Radeon 3400 Series with passive cooling even) & it worked with this low-end card, so performance cannot be the problem, the loading times might be, but I am sure there will be a way to solve this also, time will solve that problem for sure (with new generations of HDD & RAM & SSDs) ;)

I made some screenshots to show the difference, unfortunately the problem with the code persists even with the new textures & IMHO the city-lights are still too strong & there are all the other lightning problems & problems with the quality of the bumpmapping & the glowing of the nation borders, but man, look @ the details !!!  8)
 ::) We could even enhance the zoom with this & make the distances @ which airfights take place more realistic ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Muton on April 02, 2010, 08:02:55 am
> I tested this on a really slow gfx-card

please dont forget some people do use the onboard gfx-card
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 02, 2010, 09:58:55 am

please dont forget some people do use the onboard gfx-card
Some people only use a cell-phone ;)

I just do not want to read anymore that graphics in UFO:AI are oldschool & not up-to-date, based on the stone-old Quake II engine, because this tuned engine CAN make great graphics happen... ;)
Do not misunderstand me here, I am all for more graphic-options being built in, to be able to play this great game on old laptops also, but I think we also should deliver for people with modern equipment & today you can get a gfx-card with 1 Gigabyte of RAM for 35 €, so I wonder what will be 'tomorrow', when UFO:AI finally comes out ?!?!?

Read my feature-request regarding this issue here:
https://sourceforge.net/tracker/?func=detail&aid=2975351&group_id=157793&atid=805245
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 02, 2010, 10:25:26 am
we won't add the high res textures due to the reason muton rised above. we can offer another download of a pk3 file. and we will not remove the support for old cards
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: DiDiT on April 02, 2010, 12:11:54 pm
we can offer another download of a pk3 file.

what, like a downloadable high-res pack? that would be cool.

didn't they do that for half-life?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: bayo on April 02, 2010, 11:47:42 pm
... for deus ex, duke nukem 3D... and many more games...
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: freegamer on April 05, 2010, 01:42:23 pm
we won't add the high res textures due to the reason muton rised above. we can offer another download of a pk3 file. and we will not remove the support for old cards
I think this might be an untenable position if you want the game to really look good in today's (or tomorrow's) gaming world.

You might want to consider distributing UFO:AI with a low-res and hi-res data pack option.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on April 05, 2010, 05:45:10 pm
I've replaced the geoscape textures with better ones. The night geoscape is still broken though. Can somebody please fix that?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 05, 2010, 05:59:42 pm
Thanx a lot, BTAxis !
Can somebody please fix that?
& thanx a lot for that one  ;), because I  :P :-[ :-X
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 05, 2010, 06:38:48 pm
Hey guys, sorry I haven't been around for a bit.  I am still working on this stuff, but I'm continuing to add new features and improve the way things are implemented, so it doesn't really make sense to put lots of temporary stuff into the SVN, which is why I haven't been posting patches.

The issue with the nighttime illumination is caused by the fact that JPGs can't handle transparency.  We'll need to use .PNGs or .TIFFs or something instead.  I'm still working on trying to make textures which are relatively small on disk but also look pretty; I've thought of several things I could potentially do to improve the texture file size, but haven't had time to implement them yet.

Also, don't worry about the nation borders; the way I did it before was the quick and dirty version (which was fast to implement).  Right now I'm working on adding proper real-time glow/bloom effects, which should negate the need for having a separate "glow" texture for the nation overlay.

Finally, in terms of this game running on different systems, I'm making an effort to implement everything in such a way that all the new features are "optional," and can be disabled on systems with lower specs.  None of the old rendering code is going anywhere, and new rendering code will have menu options to disable, as well as being automatically disabled on graphics cards which don't support the required features.  This way, we should be able to get the game to look pretty on new systems without changing the "minimum" system requirements.

In terms of texture size, there are essentially two completely distinct issues, and I'm still not sure how important each one is.  First, there's the size of the image files on disk.  This is important to how long it will take people to download UFO:AI, and how much hard drive space it will take up.  Frankly, I don't think an extra 30Mb of disk space will make that big a difference for any system that's modern enough to run the game at all.

The second issue is how much memory the textures take up at runtime; this is both an issue of system memory (RAM), and of dedicated graphics card memory.  Here, the size of the textures *will* make a difference in terms of framerate on systems with different levels of graphics hardware.  If this is the real concern, however, it should be relatively straightforward to allow the user to select what size textures they want to use, again allowing the game to be played on both high-end and low-end systems.  This would still require us to have the high-res textures in the distributable, however.

Finally, as to bump-maps for ocean floors, they're easy enough to remove if people don't like them; once I'm done improving the rendering code I'll look at working on the textures.  Do other people have thoughts about whether oceans should have topographic depth or not?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 05, 2010, 06:46:05 pm
Finally, as to bump-maps for ocean floors, they're easy enough to remove if people don't like them; once I'm done improving the rendering code I'll look at working on the textures.  Do other people have thoughts about whether oceans should have topographic depth or not?

imo it should not - but as it's very easy to get removed .. no prio until the rest is finished imo.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 05, 2010, 07:55:56 pm
Hey arisian,
nice to see you're still workin' on that one  :)
Forget the jpgs, throw them out, use pngs instead, afaik we do not have support for TIFF (yet).
Great to see that you will implement additional graphic options !
Regarding the texture-size I will  :-X.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on April 05, 2010, 08:02:57 pm
Thing with PNG is that it's a good deal bigger on disk than JPG for images like these. I used a PNG for the replacement night lights however, since it's actually smaller than JPG because it only contains black and white.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 05, 2010, 09:20:20 pm
Sorry BTAxis, I just started to  >:( those jpgs because I am actually cleaning the ufo_tex jpgs from their ugly artifacts & really think we should convert all of the jpgs to pngs, because this format has a better design (lossless).
Somebody also mentioned to use tiff as it reduces the filesize further without reducing the quality, but I do not know how hard it would be to implement support for this. I learned that tga is more or less a simple version of png with a lesser rate of compression, so the lossless designs are (from big filesize to small):

tga->png->tiff

 ::)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 05, 2010, 09:52:37 pm
tiff is not and maybe won't ever be supported - but png is - and i've already done tests converting the tgas to pngs to reduce the size of the svn. for the jpegs, a simple convert won't work of course (due to the artifacts)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 06, 2010, 02:55:05 am
Just a couple screenshots of the "glow" effect for the sun I'm working on; the first one is just real-time bloom, the second is real-time bloom combined with a static texture for the "rays of light."  It still needs some work; the current implementation is inefficient (my 9800gt is only getting 30FPS), and also screws up the cursor image (for reasons I have not yet been able to figure out), but at least now you have some pretty pictures to look at while you wait :)

Also, as to image formats, I personally am a big fan of lossless compression; I think Mattn calculated that we could save something like 100Mb by converting the .TGA images to .PPM, so my personal preference would be to use half of that savings to include decent looking globe textures.  Though I might be a bit biased, given that I made them ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Kildor on April 06, 2010, 03:56:56 am
about "old cards and new features", nations border and daytime is not new, and need to be worked at all cards, don't forget.
and one more, could you look to the XVI overlay?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 06, 2010, 04:43:04 am
Nation borders and day/night work the same way as they always did if you disable the new features.  I haven't even looked at the XVI overlay, though I don't think any of the changes I've made should have affected it.  Did you have specific thoughts about changes you'd like to see in the XVI overlay?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Kildor on April 06, 2010, 09:27:17 am
Well, now XVI is not looks as beauty as it can, and I believe in your talents :-)
May be some neon-green light? Or even slight animation, if it possible? I can provide save with activated XVI if you need in it.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: H-Hour on April 06, 2010, 10:52:01 am
Screenshots look good!
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: freegamer on April 06, 2010, 11:29:43 am
Finally, as to bump-maps for ocean floors, they're easy enough to remove if people don't like them; once I'm done improving the rendering code I'll look at working on the textures.  Do other people have thoughts about whether oceans should have topographic depth or not?
The thing is, water is not totally translucent.  There's a reason you don't see any hue variation - other than in shallow waters - in images of the globe (1024x1024 image (http://earthobservatory.nasa.gov/Features/BlueMarble/Images/BlueMarble_2005_SAm_09_1024.jpg), source (http://earthobservatory.nasa.gov/Features/BlueMarble/)).  So it looks odd to have the topgraphic depth represented by bump maps.

Light mainly travels to ~100m in the ocean.  From there to ~1000m it is very dim, and from there downwards it is no light at all (reference (http://www.waterencyclopedia.com/La-Mi/Light-Transmission-in-the-Ocean.html)).

There are two questions:
1. does the depth of the ocean affect UFO:AI gameplay?
2. if it does - is there a better way to represent it?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on April 06, 2010, 11:58:08 am
It does not.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 06, 2010, 02:47:56 pm
Kildor: if you could provide a save with XVI active, that would be helpful.

In terms of topographic rendering, I had been going more for the "classroom globe" look than "earth from space."  Changing the ocean bumpmap is easy enough; we can tone down the amount of topographic depth for the land as well, if people want more realism.  I'd like to introduce clouds/weather systems eventually, but that's likely to be a fairly complex and involved project, and I'm not sure how feasible it is at the moment.

There are no oceanic missions right now anyway, so the ocean rendering is purely aesthetic.  On land, mountains/plains generate different battlescape maps, so there is an actual gameplay effect.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Kildor on April 06, 2010, 03:10:35 pm
Ok, I`ll share it tomorrow.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 07, 2010, 01:28:20 am
Just a quick update; I've solved the cursor bug, and improved the efficiency.  It's still not "finished," but I'm making good progress.  Here are some screenshots from the latest version.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Legendman3 on April 07, 2010, 02:18:58 am
How far along are we to finding out what happened to night and day in 2.3? (r29004 and up that is)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 07, 2010, 02:23:58 am
I'm not certain which day/night issue you're referring to, but I'm pretty sure that they're all fixed in my personal latest version.  However, there are some other things that need to be dealt with before I'm ready to release this version.  I'm sorry that the current svn version isn't working for you, but it is the unstable branch ;)  All will be fixed in time, I promise.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on April 07, 2010, 02:49:05 am
Looking nice so far. Hope you'll have it commit ready soon.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 07, 2010, 03:18:00 am
Man, that looks so sweet. I love it already. Pushing the geoscape into 2084  ;D !
Also hope that you are ready soon.  ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Legendman3 on April 07, 2010, 04:09:54 am
What i meant was it is either day or night on my game when i finish a mission depending on the time but will stay that way till the next mission. Help much?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 07, 2010, 04:13:33 am
Actually, that is helpful; I haven't really tried playing the game with the new features yet, since I've been too busy coding them.  I haven't played much with going back and forth between the new geoscape and the battlescape, so I'll definitely give it a look.  As long as I can replicate the problem, I'm sure I can fix it.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 07, 2010, 04:50:30 am
Also, for those who are saying we can't include high-res textures because old systems can't handle them, I have to admit I'm kind of confused.  The game already includes an option in the "graphics" menu that lets you pick what you want to cap the maximum texture resolution at; if you're having problems with high res textures, can't you just force the game to use scaled down versions?  The original data file size has nothing to do with the size of the texture in system/graphics memory, which is controlled entirely by the resolution of the image, so how big the files are on disk is really only relevant when you're downloading them.  Is there something about this that I'm missunderstanding?  Because I just don't get the "large textures won't work on older systems" thing.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Kildor on April 07, 2010, 05:55:09 am
Ok, here is a savegame with XVI actived.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 07, 2010, 08:01:29 am
Also, for those who are saying we can't include high-res textures because old systems can't handle them, I have to admit I'm kind of confused.  The game already includes an option in the "graphics" menu that lets you pick what you want to cap the maximum texture resolution at; if you're having problems with high res textures, can't you just force the game to use scaled down versions?  The original data file size has nothing to do with the size of the texture in system/graphics memory, which is controlled entirely by the resolution of the image, so how big the files are on disk is really only relevant when you're downloading them.  Is there something about this that I'm missunderstanding?  Because I just don't get the "large textures won't work on older systems" thing.

no, it's not about "they won't work" - it's about "loading them will take ages".
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Muton on April 07, 2010, 09:06:37 am
> it's about "loading them will take ages".

There is always a price to pay

Its the same as the "black models" problem
Default setting med res textures
If you select high res you need to wait or a SSD (http://www.anandtech.com/show/2944/10)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 07, 2010, 09:26:49 am
even if you select low res the image has to be scaled down - which can take ages already (though i haven't done any tests on this). ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 07, 2010, 12:00:27 pm
Maybe we need this for UFO:AI when it is finished  ;) :D ;D:
http://www.tomshardware.co.uk/10tb-hdd-raid,review-31629-5.html
 8)
Please, do not take this serious. It is just a  ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Muton on April 07, 2010, 06:46:29 pm
even if you select low res the image has to be scaled down - which can take ages already (though i haven't done any tests on this). ;)

I thought the idea is to provide a low res and a high res texture?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 07, 2010, 08:15:34 pm
Also, for those who are saying we can't include high-res textures because old systems can't handle them, I have to admit I'm kind of confused.  The game already includes an option in the "graphics" menu that lets you pick what you want to cap the maximum texture resolution at; if you're having problems with high res textures, can't you just force the game to use scaled down versions?  The original data file size has nothing to do with the size of the texture in system/graphics memory, which is controlled entirely by the resolution of the image, so how big the files are on disk is really only relevant when you're downloading them.  Is there something about this that I'm missunderstanding?  Because I just don't get the "large textures won't work on older systems" thing.

@muton: my post was referencing to this
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 08, 2010, 11:53:11 pm
Okay, a new patch has been posted to the patch tracker (https://sourceforge.net/tracker/index.php?func=detail&aid=2977653&group_id=157793&atid=805244 ); it works well on my system, but it's a large enough patch that it's pretty unlikely that it will work flawlessly for everyone.  If you run into issues, please let me know.  In addition to the patch, you will also need a number of new files; they can be obtained from (http://cs.jhu.edu/~ben/graphics/ufoai/).

After the discussions here about texture file size, I have introduced a system with several different versions of each texture in different sizes.  The game will automatically load the images with the size that best matches the setting for "texture quality" in the graphics options menu.  If those images don't exist (ie. you ask for high-res textures but you haven't downloaded them), it will automatically fall back on the best match it can find.  For this reason, the lowest resolution textures should always be present, but these are quite small.

I've also changed from having one texture per month to having one texture per "season" of winter, spring, summer, fall.  This means we only need 4 textures rather than 12, which reduces the size of the download by a factor of 3, and still provides pretty good visual quality in terms of being able to watch the seasons change.

The new post-processing effects are pretty, but also relatively demanding, so if you're not happy with the framerate, simply turn off the postprocessing effects in the graphics menu.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 08, 2010, 11:59:57 pm
Great, arisian !
Wanted to go to sleep, but now I have to test this immediately  ;) :D ;D :) 8)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 09, 2010, 07:49:51 am
looks nice - but... ;)

* i'm using 1024x768 and the sun post processing effect is a little bit off.
* the gloss of the sun is a little bit too bright imo. if you could make it disappear from the earth surface that would be the most realistic imo.
** (i like the athmospheric scattering one can see on ufo08 though [even if it should be blue, not yellow ;) ] - but it's not that important to keep the whole gloss)

about the cvar for the post process - this can be made to be switchable at runtime (currently you have to restart the game for this).
Code: [Select]
static qboolean CL_CvarWorldLevel (cvar_t *cvar)
{
const int maxLevel = cl.mapMaxLevel ? cl.mapMaxLevel - 1 : PATHFINDING_HEIGHT - 1;
return Cvar_AssertValue(cvar, 0, maxLevel, qtrue);
}

[....]
cl_worldlevel = Cvar_Get("cl_worldlevel", "0", 0, "Current worldlevel in tactical mode");
Cvar_SetCheckFunction("cl_worldlevel", CL_CvarWorldLevel);
[....]

take e.g. this code - you could add your own check function and check for the r_config.frameBufferObject boolean in it - if set, allow a change, otherwise fix to zero. then you can query the r_postprocess->integer value and remove the r_config.postProcessing.

great to have different texture resolution support - but ... ;) our filesystem supports overwriting textures. that means that if a gamer has a 0geoscape.pk3 with low res, a 1geoscape.pk3 with high res, the high res textures will be taken (taken into account that the files have the same name and are in the same (zip) directory of the pk3). You know how pk3 (http://ufoai.ninex.info/wiki/index.php/Pk3) files are working? E.g. you could provide a zip file with all the textures for the medium or high res packs (as the low res pack will go into svn anyway) with a folder structure like this:
Code: [Select]
-pics/
  \- geoscape
      \- map_earth_bump.png
      \- map_earth_season00.png
      \- map_earth_season01.png
      \- map_earth_season02.png
      \- map_earth_season03.png

all in all great version of the geoscape rendering. just a few minor tweaks and we can add it to the svn. thanks for the hard work you put into this.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: geever on April 09, 2010, 11:00:33 am
Mattn, maybe we should create a new branch for arisian. I would like to test if it would even work on my sys (and how fast) but I doubt I will patch my copy from two or more sources by hand... Am I lazy? :D

-geever
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 09, 2010, 11:11:48 am
the feature will be deactivated by default for the first time. so nothing should change for "normal" users.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on April 09, 2010, 12:52:19 pm
Since the game collects information about the user's system on startup, wouldn't it be better if the feature were turned on or off depending on the user's hardware on first run? I realize that the game should still be playable on low end systems, but that does not mean that the initial configuration should always be geared for those low end systems.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 09, 2010, 04:08:36 pm
It's certainly easy enough to just get rid of specular reflection entirely if we want.  I can rip out all the specularity code entirely if you want, or I can just disable specular rendering for the moment.  I'm inclined to do the latter, but let me know what you think.

In terms of the "overwriting textures" issue, part of the reason I did things the way I did was so that users could actually switch which textures they were loading without having to deal with actually deleting or re-naming files.  Unless I don't understand the way that the pk3 files work, if you have the high-res 1geoscape.pk3 file on your system, there's no way to tell the game to use the low-res 0geoscape.pk3 without actually removing the 1geoscape.pk3 file from the search path.  It's still entirely feasible to have different pk3 files for the different texture resolutions the way I've done it, but if you have several of them on your system, my method uses the correct one based on the in-game options settings, rather than automatically using the highest resolution one that exists.  This way, you could (for example) include 0geoscape.pk3 *and* 1geoscape.pk3 in the "standard" download, and then have 2geoscape.pk3 as an "optional" addon with the very-high-res textures.  You could even include all three .pk3 files in the "standard" download without having to fear that you would be forcing people to use the high-res textures.  It would make a difference in terms of hard-drive space and download bandwidth, but that would be the only difference for players with low-end systems.

I'll take a look at the issue with the sun effects being in the wrong place; I didn't try running the game at different resolutions, which I'm pretty sure is causing that problem.  I don't think it should be too hard to fix.

I'll also go ahead and make the post-processing switchable at runtime.  In terms of the "default" behavior, I'm not sure how easy it would be to enable/disable things by default based on auto-detection; is there an easy way of doing that?  It will disable things if the system can't handle the new features at all (ie. the graphics card doesn't support the opengl extensions we need), but I'm not quite sure what to do beyond that.  Though once I make it possible to turn post-processing on or off on-the-fly, it may not matter that much.

I'll let you know when I've got a new patch; in the mean time, let me know if anyone runs into any other issues.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 09, 2010, 04:25:39 pm
I realize that the game should still be playable on low end systems, but that does not mean that the initial configuration should always be geared for those low end systems.
I agree 100%.

Most of the players of a computer-game never bother going into any graphic-options-menu.
They just start playing the thingy @ whatever configuration it starts.
I have witnessed this fact very often as I am the type of guy who does care & before starting the game actually does adjust his graphic settings ;)

So the best thing for us to have would imho be a reasonable auto-configuration, like BTAxis suggested.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 09, 2010, 04:46:34 pm
@arisian:
 Unfortunately I cannot tell you how I like it (yet), because I had problems with the compilation yesterday.  :-\
I will wait with shouting out my opinion until a new version of this will be added into trunk.  ;)

Please do not throw out any code that can be optimized & adjusted, I think Mattn just meant to remove the gloss from the parts of the earth which are covered by land & not by water, if I correctly understood him ?!

Regarding the textures & their sizes: I think you have done enough regarding filesize by reducing the seasons by factor 3  ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 09, 2010, 05:51:18 pm
I agree 100%.

Most of the players of a computer-game never bother going into any graphic-options-menu.
They just start playing the thingy @ whatever configuration it starts.
I have witnessed this fact very often as I am the type of guy who does care & before starting the game actually does adjust his graphic settings ;)

So the best thing for us to have would imho be a reasonable auto-configuration, like BTAxis suggested.

hehe - believe me, i've seen hundrets of players that are exactly doing the opposite. They will tweak the graphic settings if they have a high end machine anyway.

but i also agree with btaxis that this could be activated via autodetection - but that is a totally different feature request and should not be handled here. this is only about geoscape rendering.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 09, 2010, 05:54:10 pm
It's certainly easy enough to just get rid of specular reflection entirely if we want.  I can rip out all the specularity code entirely if you want, or I can just disable specular rendering for the moment.  I'm inclined to do the latter, but let me know what you think.

whatever looks best - the bright yellow sunlight spot does just not look realistic. if you see space photos of the earth you won't see a bright yellow spot on the earth ;)
if you could disable it for now that would be nice. but i fear that the bumpmapping might take some losings from that, no?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: geever on April 09, 2010, 08:06:20 pm
but i also agree with btaxis that this could be activated via autodetection

I'm afraid about autodetection a bit, it doesn't work always. Quite some players had problems with GLSL autodetection too, in the world of videocards "supported" != "useable".

-geever
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on April 09, 2010, 08:14:02 pm
Just err on the side of safety. It doesn't have to be perfect, people can always manually adjust the options.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 09, 2010, 08:18:32 pm
I'll go ahead and disable specularity for now; to be honest, I don't really like the way it currently looks either.  I think it looks nice from an oblique angle, but when the specularity is right in the middle of the view it looks wrong.

In terms of "earth from space has no specularity," it depends on the images you look at.  For some examples of earth from space with specular reflection:

http://www.losgazquez.com/blog/wp-content/uploads/2007/12/earth-from-space.jpg
http://www.daviddarling.info/images/Earth_living_planet.jpg
http://researchpark.arc.nasa.gov/lecture%20series/images/Earth.jpg
http://www.softpedia.com/progScreenshots/Earth-3D-Space-Tour-Screenshot-5607.html

Also, note the atmospheric scattering in this one:
http://multimedia4everyone.com/faith_science_commentary/media/photo_earth-from-space.jpg

At some point, I'd like to improve the atmosphere rendering.  I'm also not totally happy with the way the sun looks right now, but I can continue to work on those things; I don't think it's worth delaying this patch for them, I'll just make another smaller patch for them when I get around to it.

In terms of auto-detection, it should already handle the "unsupported" case; it's the "supported but unusable" case that I'm not sure how to test for automatically.  If anyone has suggestions, I'm open to them.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 10, 2010, 02:43:07 am
Okay, a new version is now up at the patchtracker; I think it should fix the issue with different screen resolutions.  Also, in the process of fixing that, I discovered that it didn't handle changing the screen resolution without a restart, so I fixed that, too.  Now you can turn the post-processing on and off and/or change resolution, all with out restarting :)

Be sure to re-download the "common" textures, as one of them has changed; you shouldn't need to re-download anything else.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 10, 2010, 09:01:42 am
applied to trunk https://sourceforge.net/apps/trac/ufoai/changeset/29282

are all glsl shaders needed? i can only see one init function for combine2?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 10, 2010, 09:03:20 am
oh, btw. it would be nice if we would use uppercase variable names for varyings in the shaders. just to keep the style to the existing shaders consistent.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Destructavator on April 10, 2010, 09:07:59 am
There seems to be a problem with putting the new code in:

Code: [Select]
error: 'for' loop initial declarations are only allowed in C99 mode|
...pops up in numerous places when trying to compile now, breaking the compile process.

This happened after I just updated from SVN a minute ago.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Destructavator on April 10, 2010, 09:39:07 am
@Everyone else reading this: This issue has now been fixed in the SVN.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 10, 2010, 12:57:22 pm
Compiling a whole new version for a test now (including maps)...
Will report my findings soon.
Can't wait to see this stuff in action  ;D
Nevertheless I make a whole version (including maps), because I also want to test the different other map-bugfixes as well...
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 10, 2010, 04:06:26 pm
Arisian, compliment.
You really pushed the geoscape into 2084 !!!
This looks awesome. Never seen a game with such a greatly looking earth before...
Great to watch the terminator-line moving over the earth dividing night & day, together with the bumpmapping happening. RESPECT !
I just realized that I just tested the version with the low-res textures, have to check the hi-res textures, too.
But I think that will completely blow me away  ;)...

  8)RESPECT AGAIN, TOP WORK !!!  8)

btw, tested dev29315.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 10, 2010, 04:30:18 pm
@Mattn: There is only the base/pics/geoscape/low directory in trunk @ the moment (low res version only)...  ::)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 10, 2010, 05:06:05 pm
@Mattn: I'll try to keep shader "varying" variables capitalized; I hadn't realized that was the style.  With respect to the "convolution" shaders, you're correct that the current version only uses one of them.  The others are there in case you want to play with the FILTER_SIZE macro; basically, using the larger filters gives better quality at the cost of performance.  Actually, I think that probably the best way to do it would be to generate the shader code procedurally based on the FILTER_SIZE, which could possibly then be made into a user-changable option.  I haven't done that yet, though, so I just included the extra shaders in case people wanted to try them out.

@Destructavator: Sorry about that; I hadn't realized that style of for-loop syntax was only introduced in C99.  My compiler doesn't give me any sort of warnings at all, so I hadn't given it any thought.  In the future, I'll be sure not to declare my counters in my loop statements.

Let me know if anyone runs into any other bugs or issues; in the mean time, I'll see if I can't do something about the sun specularity...
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 11, 2010, 07:38:14 pm
Here's another small patch; this one offers an improvement on rendering the earth's atmosphere, including both specular and diffuse styles of reflection/refraction.  It also fixes an issue with the positioning of the moon.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 11, 2010, 07:46:20 pm
Code: [Select]
R_LoadShader: atmosphere_fs.glsl: 0(65) : error C1115: unable to find compatible overloaded function "pow(float, int)"
0(65) : error C1115: unable to find compatible overloaded function "pow(float, int)"

R_LoadProgram: atmosphere: Vertex info
-----------
0(41) : warning C7549: OpenGL does not allow C style initializers
0(42) : warning C7011: implicit cast from "vec4" to "vec3"
0(43) : warning C7011: implicit cast from "vec4" to "vec3"
0(44) : warning C7011: implicit cast from "vec4" to "vec3"
0(41) : warning C7549: OpenGL does not allow C style initializers
0(41) : warning C7549: OpenGL does not allow C style initializers
0(41) : warning C7549: OpenGL does not allow C style initializers

Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 11, 2010, 07:51:57 pm
fixed it in my commit. see r29345
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 11, 2010, 08:15:14 pm
 http://mattn.ninex.info/screenshots/geoscape_improved1.png
 http://mattn.ninex.info/screenshots/geoscape_improved2.png
 http://mattn.ninex.info/screenshots/geoscape_improved3.png

looks really great.

the scattering could be a little bit less bright at the dark side (in other words the fallout might be a little bit higher)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 11, 2010, 08:23:04 pm
Yeah, in those screenshots I agree; it looks less bright in mine.  I guess maybe I should hard-code a value instead of using the standard "ambient" light as a baseline.  I'll add a patch to the SVN in just a minute.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 11, 2010, 09:15:09 pm
Hey, arisian !
Maybe you could manage to integrate another option:
A possibility to turn off the text-overlay that covers that beautiful earth so superficial...
That would be nice. We could place the button for that right there where the button for the nations-borders-overlay also is. There is even some place left there, like designed for this idea ;)
If you make the code I would volunteer in making the button ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on April 11, 2010, 09:39:12 pm
I rather think that space is used.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 11, 2010, 09:45:49 pm
Sorry, I did not know that.
But don't you also think that the player should have the ability to turn off the nation-text-overlay ?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on April 11, 2010, 10:20:34 pm
Maybe it should be turned off by default and shown only when the nation overlay is toggled?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 11, 2010, 10:32:21 pm
Also a good idea.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Kildor on April 12, 2010, 02:38:49 am
I like this idea. And turn this overlay on at campaign start..
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 12, 2010, 02:47:11 am
btw, does someone else have problems with the latest version of this ?
I just started dev29353 & the globe is transparent again ?!

Sorry, wrong alarm. I somehow f**k*d up the installer, because I was doing too much stuff @ the same time during compilation yesterday (was creating a normalmap of the earth's landmasses with 7 Overlays @ the same time ;)), the computer was slowing down to a crawl & the installer somehow made a 0pics.pk3 with errors hidden inside.
The funny thing is that the installer did not warn me that something was wrong when installing, but when I tried on another computer it began having write errors...
Sorry again for raising the alarm here...   :-[
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: BTAxis on April 12, 2010, 02:53:34 am
Works fine here.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 12, 2010, 06:53:28 pm
Just a thought:
Maybe we could combine our nation flags somehow with the text overlay, this plain text just looks to 'simple' IMHO  ::)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 12, 2010, 08:37:08 pm
 :-[ Sorry, wrong topic, I did not want to post this here.
Did I really post this here ?!
It was a mistake. This was supposed to be in UFO Animations. If administrators like they can delete this here & Mattn's response also, I got the answer now ;)

@Mattn: Could you help me in telling me in which limit I ran to when I change just a few more buttons (=load more textures) & the battlescape won't start anymore (I get thrown back to Skirmish Level Selection) ?!
I can imagine this being this nasty '1024 texture slots' limit you were writing about...
Because normally, when something is set up the wrong way in the material file, the level will start anyway.
If I touched one of these limits, is there a way of upgrading those or is there no other way out, than to use less different textures ?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 12, 2010, 08:41:44 pm
If I touched one of these limits, is there a way of upgrading those or is there no other way out, than to use less different textures ?

exactly - use less different textures, try to combine them or reduce the animation frames - often the animations are already looking good with only 4 frames and some color effect or scretching.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 13, 2010, 07:04:14 pm
Hey,
I made a ultra-high-quality normalmap of the earth's landmasses using the 'multiple-overlay' technique & 8 layers, which could be used for the high quality version of this, as arisian's normalmap included in the 'high' version has 'just' 4096x2048 resolution. So if someone wants to try this, feel free to shout out, then I can prepare a ultra.pk3 to test  ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MXcom on April 14, 2010, 01:46:55 am
I'll go ahead and disable specularity for now; to be honest, I don't really like the way it currently looks either.  I think it looks nice from an oblique angle, but when the specularity is right in the middle of the view it looks wrong.

In terms of "earth from space has no specularity," it depends on the images you look at.  For some examples of earth from space with specular reflection:
[...]


Dropping in to give my 2 cents to that topic one page back...

I also thought that the sun reflection on the earth was looking wrong, so I'm glad you decided to remove it - but looking at the pictures you linked to in the quoted post, I had an idea how you could maybe - instead of removing it - improve it (so it does not look wrong anymore).
My idea is: let the reflection only alter the brightness (aka intensity or value or lightness in various colour spaces (http://en.wikipedia.org/wiki/HSL_and_HSV)), not the colour.
Actually, this made me think further - maybe the real cause is that the sun is much too yellow? Looking at your third link (http://researchpark.arc.nasa.gov/lecture%20series/images/Earth.jpg) (and also other space images (http://twitpic.com/1eqe9m)), the sun seems to be much much more white when viewed from space.
Maybe the real solution is not making the reflection only apply to brightness, but making the sun much less yellow in the first place?


Just some thoughts...
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 14, 2010, 08:04:46 am
please also see: http://ufoai.ninex.info/forum/index.php?topic=4680.0
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Muton on April 18, 2010, 01:25:22 pm
onboard gfx-cards

I do own a Nvida 7050/630 chipset

Texture quality makes no difference
extreme is as fast as low
Some shader effects do came from my CPU ~50%
Lightmap setting do change things from ~7fps to ~11fps (high to low)

If i disable glshaders and than reactivate them i got ~45fps
and the glowing is gone

In battle i still have black models even if i disable an re-enable gls

rev 29455
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: arisian on April 18, 2010, 05:27:59 pm
If you disable GLSL, it automatically disables postprocessing (which is what does the glow), since it depends on GLSL.  Re-enabling GLSL doesn't automatically re-enable postprocessing; you have to re-select the "postprocessing" checkbox in the graphics menu as well (it's all the way at the bottom) before you'll get the glow effects again.  The thing with the black models in the battlescape is unrelated to the new features I've added, so far as I know, and I'm not sure what's causing it, as I can't replicate it (it seems to only be happening to a couple of people).  I'll try to figure out what's going on, but it's not my top priority.  If someone who's actually seeing the problem wants to take a crack at it (or whoever wrote the code in the first place), that would be great ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 24, 2010, 04:53:08 pm
Just some screenshots of the modified 'high' version from arisian (you can call it ultra)... ;D

Features here:
Original seasons from arisian's 'high' version with resolution 8192x4096 pixel
Ultra-High Quality Multi-Layered Normalmap of the Earth's Landmasses (Original: NASA) 8192x4096 pixel
High-Quality version of the Earth's City Night Lights 4096x2048 pixel

I will try to optimize filesizes further & hope we can make this available for other people with modern gfx-cards also, because IMHO the effect justifies the filesizes...
I hope developers agree, I feel  :-[ to have this version just for me  ;)

Take a look (night):
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 24, 2010, 04:54:41 pm
Day:
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 24, 2010, 04:56:50 pm
Day again (need more zoom ability ;)):
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 24, 2010, 05:00:52 pm
I am sure I am able to squeeze filesizes by reducing the color palettes, just like I did with the glowmaps, so we can get high quality, small filesizes & short loading times...

btw, there is not even 'med' in trunk afaik...

Maybe someone could correct this for those poor other players with modern hardware out there ?!  ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 25, 2010, 03:36:17 pm
& when I find out how to best squeeze all the filesizes down, I also want to have my 12 seasons back...  ;) :'(
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 25, 2010, 05:42:41 pm
& when I find out how to best squeeze all the filesizes down, I also want to have my 12 seasons back...  ;) :'(

not in svn... the difference isn't really that big.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 25, 2010, 05:51:35 pm
Which max filesizes would be acceptable for the whole high version to be included into trunk ?

My actual ultra version described above has (without any optimizations regarding the color palettes & with 4 seasons) around 120MB...
I can only speculate @ this moment how far down my optimizations can bring this, but I suppose 50% should be possible...

btw, afaik there is only the low res version of this in trunk. Is it okay if I prepare a med version to be included also ?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on April 25, 2010, 10:48:44 pm
Which max filesizes would be acceptable for the whole high version to be included into trunk ?

My actual ultra version described above has (without any optimizations regarding the color palettes & with 4 seasons) around 120MB...
I can only speculate @ this moment how far down my optimizations can bring this, but I suppose 50% should be possible...

btw, afaik there is only the low res version of this in trunk. Is it okay if I prepare a med version to be included also ?

i would really vote against it - nothing against offering them as separate pk3 files for download - but including them into trunk... the low version is already 8 mb - added another 20 mb isn't an option imo. but please prepare them, i will gladly upload and share them.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 25, 2010, 11:22:47 pm
I am sure we can squeeze the low version down also without loss of visual quality, but thank you very much for the offer to host the files, I will prepare them (but this is not my highest priority @ the moment so this will take a while...).
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 27, 2010, 12:05:11 am
First experiments to get the files small are delivering some weird results I wanted to share here for amusement... :P
Unfortunately I cannot report any successes here @ the moment  :'(
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 27, 2010, 12:51:02 am
IMHO we should enable the middle mouse button for navigation on the geoscape in addition to the right mouse button also...

Everytime I am back on the geoscape now I first try to navigate with holding the middle mouse button instead of the right right one  :D ;)

I love our new navigation system arisian coded for the battlescape, but IMHO the user interface should be consistent & intuitive, so we should enable the unused middle mouse button for the geoscape also for the same function it has on the battlescape...

It would be nice if someone knowing how to do that could nake this possible & enable this...
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Sarcanth on April 27, 2010, 01:17:27 am
Just wanted to ask, are you guys going to release a separate pack for the full resolution geoscape with all twelve seasons?
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on April 27, 2010, 11:36:03 am
Hello & Welcome To The Forums !  :)

There will be a downloadable add-on with 4 seasons & high-res textures for sure, I cannot tell you what the final decision on the 12 seasons will be, but it seems that those are gone as this feature would...

1.) Require a code change also & not only a pack of high-res textures...
2.) The difference in graphics does not justify the long loading times & ultrabig files (x12) needed for all of the 12 seasons...

 ::) Maybe for 2.6+ in 2014... /joke ;) :D ;D
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on May 15, 2010, 02:45:05 am
I uploaded the medium res version to current trunk, because I remembered that somewhere here we agreed that the medium res package will make it to trunk, but the high res package will only be available as additional download...

But now I am not sure if I will get beaten for doing this  :-[

The low res version has 28.6 MB, so I thought this is really nothing compared to the 120 MB the version has, that I am currently using fluently on both of my systems (still not succeeded in squeezing filesize)...

Please do not hate me for loving good gfx & high resolutions  ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Mattn on May 15, 2010, 08:13:34 am
that is something i will remove from the final installers anyway and offer as different download - some people has to pay for every megabyte they are downloading.... please keep that in mind.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on May 15, 2010, 09:35:45 am
Sorry, I really thought initially that you just 'forgot' those...
I did not want to make extra-work...
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on July 15, 2010, 12:52:46 pm
IMHO it was a very good decision to keep the medium-res globe textures for v2.3.  8)

Still there are phantastically, phenomenal looking high quality versions available also, which I would like to share with others too.  :-[

Is it okay if I commit those @ least to data_source so others interested could download those from there ?!  ::)  :-\
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: Kildor on July 16, 2010, 05:31:55 am
No. Big textures should be somewhere on site, but not in repo. It can just simply destroy SF SVN.

PS: btw, you were wrong in message of r31015. You shouldn`t manually break strings, only different phrases should be newlined, and * should be in begining of sentence, not each line.
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on July 16, 2010, 04:07:32 pm
No. Big textures should be somewhere on site, but not in repo. It can just simply destroy SF SVN.
Those are not THAT big, Kildor.  :D
We are talking about 120MB ;)
But okay, if you say so. We can sometime later upload those somewhere else as this has no big priority...

Sorry 'bout my wrong commit-comments. Will do it like you described it in future-commits...

It is also easier & faster that way, so thanks for the comment ;)
Title: Re: Improved 3D Geoscape Rendering [was: Nightime city lights in 3D geoscape]
Post by: MCR on December 09, 2010, 09:25:04 pm
For those who want to check out the globe in highest resolution  ;)- here is the link to download the pk3:

http://www.mediafire.com/?0x3iyqfn8h60w0o

 8) 8) 8)

CHECK IT OUT - YO !