UFO:Alien Invasion

Development => Artwork => Topic started by: fhstoica on December 27, 2012, 11:37:26 pm

Title: Tools to generate textures
Post by: fhstoica on December 27, 2012, 11:37:26 pm
Hi,

I have some tools that can be used to generate textures like the ones attached. I made a sample using a color map with only 3 colors, Red, Green and Blue, the same sample on grayscale only, and the normalmap, which I simply generated with the GIMP plug-in.

The colors can be changed, more can be added, the roughness can be increased or decreased. The attachments are just a sample of what can be done. Since the texture is generated using a Fast Fourier Transform, the opposite edges of the images match, and they can be easily tiled.

If anyone is interested please let me know and I will make the tools available to everybody.
Title: Re: Tools to generate textures
Post by: krilain on January 07, 2013, 04:10:17 pm
Hello, are you the author of the gimp plug-in that produces those effects?
Title: Re: Tools to generate textures
Post by: fhstoica on January 08, 2013, 12:01:32 am
No way, I am not at the point of writing GIMP plug-ins.

Only the last image, the normal map, I generated from the grayscale image using the Normalmap GIMP plugin. I simply installed the plugin on my Linux distro using Synaptic.

I believe you can contact the original author at this URL:  http://code.google.com/p/gimp-normalmap/

I generated the images using a Fourier Transform of a 2D array of random numbers (filtering out the high-frequency noise), and plotted the resulting surface (seen from above) using POV-RAY.
Title: Re: Tools to generate textures
Post by: krilain on January 08, 2013, 07:54:55 pm
Ok, thanks for the answer. It is not I wanted to contact the author of the plug'in at all. It was to know until what point you can control your texture production process in order to ask you to produce a wider set of samples with no hand working, which would be long I think.
Title: Re: Tools to generate textures
Post by: fhstoica on January 10, 2013, 12:16:25 am
This week I am a bit tied up with some work, but I will post the tools. Basically you need GSL (the GNU Scientific Library) for the FFT and then you will need POV-RAY. The 2D FFT code (GSL offers only 1D FFT if I remember correctly) is written in C++ and the code for generating the POV-RAY script is written in Python.

Basically you can specify a few Fourier modes by hand (or let Python generate them for you as random numbers), so it takes very little time to generate the textures you see. The color and the grayscale ones take about 5 seconds to generate on my machine (Athlon II X4, 3GHz).  You have very granular control over each step.

The normalmap requires hand work in GIMP.

I will post a short how-to on this forum, but I will write a more detailed document on what each script/program does. Will also generate more samples, and they require no hand work, just to launch a script from the command line.
Title: Re: Tools to generate textures
Post by: fhstoica on January 10, 2013, 06:15:25 am
Finally, I added the tools I mentioned. I also included a new sample texture with the tools, (I did not crop the image).

You'll have to unpack the archive and first modify the Makefile so that it points to your installation of GSL (http://www.gnu.org/software/gsl/), needed for the FFT.

If you work in Linux and have POV-RAY (http://www.povray.org/download/) installed you can uncomment the last line in the start.py file so that the same script will run POV-RAY as well. I am currently working on a Windows machine and I am running start.py under Cygwin. I have to run POV-RAY outside of Cygwin, and process the surface.pov file generated by the scripts.

The quick way to generate a texture is:

./start.py <random_seed> <central_frequency> <decay_power> <output_file>

At the moment it is set up to generate a grid of 128x128 and render the image as seen from above, making basically a contour map.

For example, the image I attached I generated with:

 ./start.py 8991 20 1 surface.png

The central frequency can be somewhere between 0 and 64 (since the grid is 128^2), and the decay power between 1 (for a rough image) and 3 (for a smooth image).  Feel free to experiment with these values. 8991 is just some value I used as seed for the random number generator.

I will write a more detailed description of what each tool does.
Title: Re: Tools to generate textures
Post by: krilain on January 10, 2013, 08:08:58 am
Thank you.

I will try this. It could be very useful for me for landscape purpose. I've to get rid off the first difficulty as I'm on windows and not on linux.

I'll give you feedback.
Title: Re: Tools to generate textures
Post by: fhstoica on January 14, 2013, 12:27:44 pm
Thanks, I'll wait for your feedback.
You may know this already, but just to make sure, the stock installation of Cygwin comes with GSL included.
Title: Re: Tools to generate textures
Post by: krilain on January 18, 2013, 12:09:42 am
 :P Omg, I've been in trouble with cygwin. That sys is revenging for I've usually neglected it for 2 reasons, the first is I don't like a soft that tries to write in C:/ directly, I find this strange under the feeble Windows that in my opinion may get troubled for less than that. Second thing is I've always found unsafe to make a downloading bridge for hours (hours taken just for downloading very tiny packages...). In fact I've always thought that cygwin was made for schools where administrators can use a dedicated computer just for download purpose.

Ok, so as I said I've been in trouble, for I had to retrieve GCC I made the same time a "massive" download to the local disk (that was taking days so I had to abort). Then, I tried to install what I've got, but some of the packages were corrupted !... From this point, I've encountered badly encoded directory names (such as 04 ü˜vK箬Nû]ç§ÆÛ§„®è‡Ö­fjý…%«N¦2¦oZ+µ1ë=‰]ɱI —œ fµò¯«î¨U¿›.[W]× ...) that I try to remove right now. And I dont know if I mistake but I'm looking for about 4 Go gone in smoke in 20 minutes when I've let cygwin trying to abort the install....

That's a pity because I wanted really to test your stuff. I plan something that may ask for textured tiles.

Big recursive rm -r -R is waiting instead ....

ps: anyway a search on the web throwed me there : http://gforge.enseeiht.fr/scm/viewvc.php/SPa1/main.pdf?pathrev=106&sortby=author&root=spacify-dsml&view=diff&r1=73&r2=74 ... does that show some try to write files names in Tex???


Title: Re: Tools to generate textures
Post by: fhstoica on January 18, 2013, 03:35:49 pm
Drat, I am really sorry to hear that you cannot install Cygwin.

One possibility is for me to try and port the 2D FFT program to FFTW instead of GSL.
That is because FFTW comes with pre-compiled Windows binaries and can be used from MinGW.
You can also install Python and POV-RAY for Windows, and then there will be no need for Cygwin anymore.

In the meantime, please let me know what kind of textures you would like. Please look at the Python script Generate_Texture.py at the self.color_map definition in line 15 of the script. This will tell you which colors will be used in generating the image. You can then tell me which colors to use and I will generate some samples.

It's not ideal, but it is a way I can help, and you can tell me what changes to make, until I get a painless way for you to use the tools directly.
Title: Re: Tools to generate textures
Post by: krilain on January 18, 2013, 05:20:57 pm
Don't worry, in fact the problems come from the fact that I'm using the computer to test some modeling so that it was not a very good news to get this cygwin bug.

Anyway I think I've found the reason of this. Here for information : as far as I remember, I aborted a big part of a Tex package in download. When I asked cygwin after that to install all the packages I'd finally downloaded, cygwin possibly has met this uncomplete Tex package and unfortunately was unable to handle it. So it created a lot of directories named in Tex format which is properly undeletable with Windows - but , good point, removable in the cygwin console with rm. For my 4 Go, it is that I'd downloaded too much packages, moreover unuseful because I had most of them (like Perl) in spare parts somewhere else.
Ok, for the textures, what I would need it is quite simple. I'm looking for landscapes tiles that could in any maneer imitate (when melted properly) a top view over the earth.
Unless you're interested on training on this concrete case, I wouldn't ask you to make a sample because I still want to try another approach to compile your C file with the help of visual studio and the GSL library. I want this to work
Title: Re: Tools to generate textures
Post by: krilain on February 03, 2013, 10:39:14 pm
I just post a feedback here. Don't worry about the previous post, it's outdated.

So I've finally compiled the c++ part with gsl inside. Let me tell you that I've again encountered some strange problems. I tried first with QT4, which has a so nice editor, but linking was impossible (using visual C in fact). Then jumped to Visual studio directly, and again encountered some weird issue. The good new, is that, I have been following the first steps provided in the UFOAI wiki for compiling the game and downloaded the great Code Block. This one compiled the file almost immediatly. It uses directly some gcc, that apparently helps. 

Ok, I have now to run it with your python script even if I wonder what new surprises are pending ;)

I would like to add some information here about the UFOAI wiki pages dealing with textures. I've read some part, some days ago, so I'm not sure about the links but this one is an example : Wiki (http://http://ufoai.org/wiki/Where_to_start_as_an_artist#...create_textures_for_3D_environments_.28maps.29_using_Gimp.2C_Photoshop_or_another_program)..

I discovered there, that there could be a "pick effect" when tiling a texture, just like in tiling sounds. That requires to remove the low frequencies from there, and a gimp plugin helps for that. Anyway if you are fluent in gsl as you seem to be, you may remove those frequencies by advance in your tool when you get the time for that.

ps : compiled version of GSL here : Geldreich compiled gsl (http://david.geldreich.free.fr/dev.html)
Title: Re: Tools to generate textures
Post by: fhstoica on February 04, 2013, 04:37:50 pm
Hi,

I'm glad that you managed to compile the code, after a bit of digging I found that Python supports 2-dimensional FFTs directly via the numpy package:

http://docs.scipy.org/doc/numpy/reference/generated/numpy.fft.fft2.html

http://sourceforge.net/projects/numpy/files/NumPy/1.6.2/

I may be able to put everything inside a single Python script that you can run from the Command Prompt in Windows.
Anyway, I tested the executable you posted, and it works perfectly.

The roughness you are mentioning can be controlled via the last parameter, the "power". The higher the value, (2 to 5) the smoother the image. A value between 1 and 2 will result in a rough image.

I have to read a bit more about the "pick effect" you mentioned, but the low frequencies can be removed by choosing a central_freq different than zero (for example 10, to 20).

Please let me know if you encounter any problems running the Python script, it should work with Python 2.7.
Title: Re: Tools to generate textures
Post by: krilain on February 04, 2013, 07:40:03 pm
I may be able to put everything inside a single Python script that you can run from the Command Prompt in Windows.
That's would be easier indeed. But why don't you use some more dedicated engine, like Scilab (powerful matlab clone) dedicated to matrix and including all FFT you could dream of :)
I have to read a bit more about the "pick effect" you mentioned, but the low frequencies can be removed by choosing a central_freq different than zero (for example 10, to 20).
Sorry that's more a "peak" effect. I'm not an english first language guy ;)
As far as I've understood, it is the same effect that you meet when you melt 2 or more identical sounds, the low frequencies make blank sound...

Edit : I've finally obtained my first texture generated by your process, attached below. I removed the part with sys.argv entry at start.py, and I'm entering them manually. I removed also the "make" part, and I let start.py use the exe directly. I don't use povray inside the .py, I just let it generate the surface.pov and open the povray editor. That's fine ;)
   
Title: Re: Tools to generate textures
Post by: fhstoica on February 07, 2013, 01:31:42 pm
Well, done! You got it working now. The two diagrams you posted explain the flow of the texture generation process very well. You basically wrote the documentation for it.

Time to go a bit more into the details:

First, in the povray.ini file the line:
+W1024 +H1024
controls the width and height of the rendered image, in pixels. As it stands it is set to 1024x1024.

You will notice in the Generate_Texture.py script in line 128:

outstring += 'object{surfface scale<1.33, 1, 1>}\n'

This scales the surface horizontally (the x-z plane is the horizontal, y is the vertical) so that the final texture has the same numbers of pixels horizontally and vertically. I found the 1.33 value by trial and error.
Feel free to modify the parameters in this line if you want to get a different shape, but it is a good idea to keep the value for the y-direction to 1:

scale<x_scale, 1, z_scale>

At the moment the surface is shifted and scaled vertically so that it takes values between 0 and 1 along the y-direction. This way it appears flat when seen from a narrow-angle camera located high above the surface (line 88 in Generate_Texture.py). Changing the y_scale can make perspective effects visible, so it is better to leave that value unchanged.

As for the color_map, at the moment it is defined as:

        self.color_map       = \
"""[0.0  color Green]
[0.125  color Blue]
[0.25  color Red]
[0.375  color Orange]
[0.5  color Sienna]
[0.625  color SteelBlue]
[0.75  color Cyan]
[0.875  color Yellow]
[1.0  color Magenta]"""

It specifies at each contour level (between 0 and 1) which color to use. POV-RAY will interpolate the RGB values between these levels. The predefined colors are specified in the colors.inc file of the POV-RAY installation. They can also be found here:
http://www.buckosoft.com/~dick/pov/colors.php

You can specify the colors in rgb format, for example "color rgb<1, 0, 0>" or rgbf format, for example:

        color_map {
          [0.00, 0.33 color rgb <0, 0, 1>
                      color rgb <0, 0, 1>]
          [0.33, 0.66 color rgbf <1, 1, 1, 1>
                      color rgbf <1, 1, 1, 1>]
          [0.66, 1.00 color rgb <0, 0, 1>
                      color rgb <0, 0, 1>]

Again, very good documentation (for pigments, in general) can be found here:

http://www.povray.org/documentation/view/3.6.1/77/

I still have to rewrite Generate_Texture.py to include the FFT as well.
Title: Re: Tools to generate textures
Post by: krilain on February 07, 2013, 07:01:07 pm
Well, done! You got it working now. (...)
Yes, at last ;*)

Thanks for the details. Fortunately I can count on french documentation (like this page http://khayyam.developpez.com/articles/3d/povray/?page=page_8#LVIII-E-1) in the hope to understand all of that.

Before I go further into it, you may want to know what kind of texture it may help to generate and what for. From my side, the interest was not too much to use computed texture on maps. I think that there, that 's photographical textures that are needed more. But I've been from one month looking on some classical implementation of flight combat, you know like 1942 and so on. This shouldn't have been too much difficult to code a first thing, but I'm damn slow so it will probably never see the dawn of day ;_) Anyway, I'm attaching here some stuff that will show you this.

I still have to rewrite Generate_Texture.py to include the FFT as well.
Let me know when it's done. Moreover if I've time enough I will see if some tweak helps to get predictable results.


ps : if you love diagrams, using Dia I've made this topic readable in this way About psi and other stories (http://ufoai.org/forum/index.php/topic,7096.0.html)
Title: Re: Tools to generate textures
Post by: fhstoica on February 09, 2013, 06:43:19 pm
Finally, I wrote the script that does the FFT and generates the POV-RAY file in one go:

./Generate_Texture.py --size=128 --central_freq=0 --power=1.8 --seed=90980

It requires python 2.7 and numpy. I use optparse which doesn't work with Python 3 series, but it is a trivial change to switch to argparse if you prefer the newer series.

I attached a sample image generated with this new script, the first time flattened and seen from above, the second time scaled up vertically, and seen in perspective.

With this script it is trivial to render a large grid (say, 512^2), and render it a high resolution (4k x 4k) and then cut a piece of the image, instead of tiling small texture pieces.
With the old tools you would have to change the code in a few places to update for the new size.

However, for very large grids, POV-RAY will start to use large amounts of memory, and the surface.pov file will grow like N^2. On an old machine with 512MB of RAM the practical limit for the grid size is 256^2.
Title: Re: Tools to generate textures
Post by: Sandro on February 09, 2013, 09:00:05 pm
Can you generate a good velocity field for a sinewave (or better) water? Random for every map loading? That could help a project of mine to use cheap-math wave simulator for UFO:AI water: http://ufoai.org/forum/index.php/topic,6932.0.html
Title: Re: Tools to generate textures
Post by: krilain on February 10, 2013, 10:47:35 pm
I've tested your fluid program - or more precisely your program about fluids ;_). That's quite impressive. It would be funny to be able to shoot in the water (after a miss for example) and see it having such movement.
Title: Re: Tools to generate textures
Post by: fhstoica on February 11, 2013, 02:21:11 am
Regarding the waves on the water, all I have is an old simulation of a wave in 2 dimensions. It is not a proper hydrodynamics simulation, just the wave equation in 2D. I don't know if this is what you are looking for. Anyway, the code is attached.

You will find the following components:
One script for generating the initial data. At the moment it generates a Gaussian, with center and width that are user-specified. Since the grid is 128^2, I generated the initial data with:

./Initial_Data.py 64 0.07

surface_waves.cpp: The actual simulation. You can simply type "make" to compile it. The file Constant.cpp contains parameters like grid size, surface tension, lattice spacing, time step, damping, number of frames to generate and number of time steps between frames.
Running the resulting executable will produce a bunch of "frame###.txt" files following the evolution of the wave.

Finally, you can use POV-RAY to render each frame, with Plot_Frame.py, for example:

/Plot_Frame.py --in_file=frame010.txt --pov_file=frame010.pov --png_file=frame010.png

I also included the txt and png files for the initial data and one of the frames.