This document describes progress on a library for procedural texture
synthesis. It is somewhere between a lab notebook
and an informal diary. The entries are listed in “blog order” with the
most recent material at the top. Or read
from the bottom up beginning with an overview.
The sample
images represent a unit
diameter disk centered at the
origin of 2d texture space, rendered at a resolution of 300
pixels. This library is part of a larger project still in its early
stages. The round image shape is an artifact of that project,
or
can be seen as an homage to
the space
cookies
from [Witkin
& Kass 1991].
While this library can be used in the normal way by a human
programmer (as in early samples below) it is
primarily intended for use by an
automatic
programming
system like genetic
programming. GP is a technique for evolutionary computation related
to
the genetic
algorithm. GA operates on bit strings while GP uses tree structured
“genes”
making it suitable for evolving expressions of nested functions, such
as those in this library. This general approach is similar to Karl
Sims' work on interactive evolution of textures [Sims 1991].
Sims used GP to evolve a function that computes a pixel's color from
its coordinates. My approach differs slightly: evolved program do not
reference pixel coordinates, they effectively return an entire texture.
The functions in this library are
divided into “generators” which produce texture given simple
parameters, and
“operators” which take one or more textures as inputs. Some timing
information is given below but should be considered
preliminary and useful only for relative comparisons.
Probably at some point this will be rewritten to use a GPGPU
programming methodology.
Notes for geeks on the
C++ code snippets below: entries at the top of this page, after October 16, 2009, were evolved with GP using Open
Beagle. Code before/below that date was hand-written. (Some
having been lightly edited for readability. When code is shown as
assignments,
the
value assigned to the variable texture is the
displayed result.) Some of these
diary entries have been revised after the
date given, they are marked with “Update”. The most recent To Do list
was posted December 1, 2009.
Send comments or questions to Craig
Reynolds. This work is supported by my employer, the
US
R&D group of Sony
Computer Entertainment.
Interim spots
February 8, 2010
I updated the
LotsOfSpots
prototype from
July 26 but was concerned about
the way the spot centers
were restricted to the unit circle around the origin (the part shown in
these renderings). Scaling down or translating
LotsOfSpots reveals a circle
containing spots and empty black outside of that.
Ideally I would prefer an unbounded, space filling pattern. This might
use an approach like the grid-based random seeding described in the
paper [
Lagae
et al. 2009] about sparse Gabor convolution, mentioned here last
May 9. I decided to side-step this
limitation with a name change. Eventually there may be a space filling
version of
LotsOfSpots, but
for now this interim operator will be called
SpotsInCircle. In the top image
spots are
distributed uniformly in the circle with their radii proportional to a
Noise texture.
SpotsInCircle (30, // number of spots
0.1, // max spot radius
0.3, // softness
UniformColor (white), // likelihood (uniform)
Add (UniformColor (gray (0.3)), // radius variation
Tint (gray (0.77),
Noise (.2, Vec2 (7,11)))))
In this example both the likelihood of a spot appearing, and the
relative size of each spot, vary over the image. They are controlled by
the
two input textures. In this case the relative size varies horizontally
across the
texture: largest in the center and down to zero at the left and right
sides. Conversely, the likelihood of a spot appearing varies
vertically: high near the center and low at the top and bottom of the
image.
SpotsInCircle (200, // number of spots
0.025, // max spot radius
0.6, // softness
SoftEdgeBar (0.1, 0.4, Vec2(), 0), // likelihood: horizontal bar
SoftEdgeBar (0.1, 0.4, Vec2(), pi/2)) // relative radius: vertical bar
I plan to let GP choose maximum spot radius on the range [0, 1]. This
is what it looks like when 20 spots of radius 0.5 try to fit inside a
circle of the same size.
Clipped difference of Noise
February 6, 2010
No evolved textures today, this is about a new
generator for the texture synthesis library. Originally inspired by the
third image on
July 19, this labyrinth-like
texture is similar to
applying a soft threshold to
Noise.
The implementation used here produces a more organic shape by clipping
the
difference between two
Noise
patterns, offset in position and rotation.
The parameters for this new
NoiseDiffClip
generator include a scale and position (like
Noise) plus a softness value
between 0 and 1. Softness is 0.05 in this first texture.
Another
NoiseDiffClip with a
softness of 0.8 and 2/3 the scale. This has a different style than
basic
Noise, compare
this image to those on
May 4. Looking at
brightness histograms, this texture has more pixels in the midrange and
fewer at the bright and dark ends of the scale.
December in February
February 2, 2010
During December I experimented with new
kinds of fitness and population replacement strategies for GP in Open
Beagle.
Novel textures are created as a side effect of this testing. I
collected some of
the interesting ones, but neglected to post them until now.
This first texture is my favorite. The intense edges seem to be from Wrapulence with a strong
dose of EdgeEnhance. ColorNoise provides the basic
hue variation. Or so it seems, the evolved code is difficult to
comprehend:
// coc20091217a -- very edgy multicolor pattern
Subtract (EdgeEnhance (0.040705, 4.58566,
Add (Wrapulence (2.61481, Vec2 (1.16699, -2.27901)),
Furbulence (3.66211, Vec2 (-2.12694, -1.26397)))),
Subtract (Subtract (EdgeEnhance (0.0420333, 4.58566,
Add (Subtract (Add (Furbulence (0.323467, Vec2 (-2.12694, 1.10331)),
Furbulence (3.66211, Vec2 (-2.12694, 1.10331))),
Furbulence (3.66211, Vec2 (-2.12694, 1.10331))),
EdgeEnhance (0.042568, 4.58566,
Add (Wrapulence (2.61537, Vec2 (-2.94796, -2.94796)),
Furbulence (3.90532, Vec2 (-2.94796, 0.965091)))))),
Furbulence (3.66211, Vec2 (-2.12694, 1.10331))),
HueOnly (Subtract (EdgeEnhance (0.042568, 4.58566,
Add (Wrapulence (2.84277, Vec2 (-2.94796, 0.965091)),
Wrapulence (2.84277, Vec2 (-2.94796, 0.740041)))),
Subtract (Add (Wrapulence (3.05225, Vec2 (1.16699, -2.27901)),
Furbulence (3.66211, Vec2 (-2.12694, 1.10331))),
ColorNoise (0.5612, Vec2 (1.44605, -2.03616)))))))
Looking down into a container as gloppy green
liquid is poured in -- or something like that.
Woodgrain-ish texture from Stretch,
Wrapulence and others.
Psychedelic fractal cabbage. (Not to be confused
with
fractal
broccoli.)
Clouds and contrails.
Updated to do list
December 1, 2009
I wrote the previous To Do list on
September
20, 2008. Now that the initial version
of this library is largely complete, I made this updated list to
keep track of
still unfinished business:
- Not yet implemented:
- Reaction-diffusion textures, see this
and discussion on September 20, 2008
- Generalized warp: brightness of one image causes localized
expansion or contraction of another texture (like StretchSpot, see March 15, 2009)
- “Polar blurs” along constant angle or radius (related to September 11, 2009).
- Prototyped but not finished:
- Marble, maybe “generalized marble” as described on May 28, 2009
- Sparse Gabor Convolution, as prototyped on May 9,
2009
- LotsOfSpots, as
prototyped on July 26, 2009.
- Generator based on the “Max
of Noise” labyrinth, see
third image on July 19, 2009.
- Working but too slow (waiting for GPGPU?):
Exponentiated absolute difference, and an end
for now
November 30, 2009
This marks having completed a pass over the
entire library of texture generators and operators, connecting them to
the
GP
engine of
Open
Beagle. It is also just about a year since I started working on
this project in earnest. Other candidates for the library exist only as
prototypes or unimplemented ideas. I will list those in an updated To
Do list then relegate them to the
back burner while moving to the next phase of this project. Things may
get quiet here for a while.
The final addition is
ExpAbsDiff,
which does a
Gamma-like
exponentiation of the absolute value of the difference of two textures.
Sorry for the abbreviated name, but spelling it out just seemed too
long...
This texture uses
ExpAbsDiff
with several
Multiplies,
noise generators and
SoftEdgeSpots.
In all of today's examples the fitness function
is biased toward inclusion of
ExpAbsDiff
and the grating generators were excluded. (I'd gotten tired of all
those stripy patterns favored by
GrayTheHardWay.)
As a result today's textures are primarily based on the various noise
generators.
This texture's evolved program consists of these operators:
ExpAbsDiff,
Multiply,
HueOnly,
Wrapulence and
ColorNoise. The red “background”
looks to me like the
ExpAbsDiff
of noise and a constant value. It is a bit like the textures in the
September 12, 2009 post, which makes sense since
ExpAbsDiff is a generalization of
the
DifferenceSquared
operator prototyped there.
I rescued this very cool texture from a run
that crashed. I ran into the same problem using the
Gamma
operator with GP. The exponentiation is well behaved for pixel values
in the normal [0,1] range but produces NaNs for negative values and can
produce INFs for large values. In both cases I took the simple, if
heavy-handed, approach of clipping each pixel's RGB values to the
positive unit cube before exponentiation.
This texture was the best of run at generation 42 with a fitness of
0.83 when it hit the error. The second parameter to the outermost
ExpAbsDiff (see code below)
produces the mottled pattern (it looks a bit like the “composition book
cover” of
Mar. 28), and the third parameter
is just a gradation from green to to cyan to dark blue. From those
elements the
ExpAbsDiff
produces this
'60s
Fillmore poster color style.
// coc20091129h -- mottled blobs of one color transition over another
ExpAbsDiff (6.15328,
EdgeEnhance (0.0296748, 6.04062,
EdgeEnhance (0.0296748, 6.04062,
Turbulence (0.292052, Vec2 (-1.52492, -0.617385)))),
Multiply (ColorNoise (1.6958, Vec2 (-1.52492, -0.603703)),
Blur (0.0359943,
Blur (0.0359943,
HueOnly (ExpAbsDiff (6.15328,
Blur (0.0359943,
ColorNoise (1.6958, Vec2 (-1.52492, -0.615891))),
Multiply (ColorNoise (1.6958, Vec2 (-1.52492, -0.603703)),
Gamma (4.41839,
ColorNoise (1.6958,
Vec2 (-2.8845, -0.137089))))))))))
GP Gaussians: good news, bad news, good news
November 29, 2009
The good news is that I extended the GP function
set to include the three texture operators based on Gaussian filters:
Blur (
Aug. 14),
EdgeDetect and
EdgeEnhance (
Aug.
24). The bad news is that these operators are an order of magnitude
or two slower than most of the other operators. (Fractal noise is the
next slowest type, roughly 10 times faster.) They
result
in very slow GP runs, hours instead of minutes. While I did my best
to find some evolved textures featuring these operators, I may leave
them out of future runs. The other good news is that I noticed I had
broken my recent fix to the memory leak, now replugged and stress
tested. I found and fixed an unrelated leak in
EdgeEnhance.
Here an
EdgeEnhance modifies
Furbulence, inside a
Twist, producing the
arc-shaped green streaks. Perhaps the high frequency “ringing”
along those streaks correspond to the Gaussian kernel size (0.02 ≈ 6
pixels)?
// coc20091128a -- irregular bars with ringing arc-shaped green streaks
Subtract (Tint (Gray (1.81729),
Twist (2.42196, Vec2 (0.649625, 2.89499),
EdgeEnhance (0.0237886, 6.21132,
Furbulence (3.43298, Vec2 (2.98064, -0.901972))))),
Subtract (Tint (Gray (1.81729), SineGrating (26.1235, 3.7236)),
Gradation (Vec2 (0.649625, 2.73791),
Pixel (0.335664, 0.834494, 0.00240446),
Vec2 (0.41379, 2.89499),
Pixel (0.335664, 0.834494, 0.00240446))))
In this texture
EdgeEnhance
modifies triangle waves (curved by
Wrap,
see code below)
to produce a “bright bump” and “negative then clipped to black stripe.”
As mentioned on
August 19, I added a hack to
the 1d utility
GaussianBlurX
to basically ignore filter sizes greater than 0.25 by simply returning
the average pixel value of the input texture (as if the filter size was
infinite). The goal was to prevent evolution from supplying a huge
filter
size that would take much too long to compute. Upon reflection I
decided I did not like that approach and removed it. The source code
might suggest that a
Blur was
being computed with a filter size of 0.5,
but it would be silently reinterpreted. If this invisible max size
parameter was ever
changed, old programs would produce different results. Instead I added
yet another kind of GP ephemeral constant for specifying the size of
Gaussian kernels. I limit the range of values it can take during
evolution
(currently [0.011, 0.05]) rather than make ad
hoc adjustments once a too-large value is passed into the operator.
// coc20091129a -- edge enhanced wrapped triangle waves
Tint (Pixel (0.183097, 0.50817, 0.966571),
EdgeEnhance (0.0247835, 5.38926,
Wrap (2.07002, 4.42434,
Vec2 (0.825071, -1.40845),
TriangleWaveGrating (36.499, 0.799096, 0.550199))))
Max, Min and Mirror in GP
November 26, 2009
I added
Mirror
(
July 18) plus
Max and
Min (
July
19) to the GP function set. I was having a hard time finding a test
GP run that illustrated these operators. So in addition to biasing
fitness toward these three operators, I also removed the grating
generators and got more interesting results.
This very cool pattern reminded me a bit of “Bozo's donut” from [Perlin
1985]. The blobby transparent white shape is probably
Max acting as a threshold on
Noise. (Something like
blobs in my
August
24 post.) The background is
ColorNoise.
The wispy curved pattern must be formed by the 6 calls to
Twist in the large (size 287)
evolved program. It also includes 12 calls to
Mirror (of which I can see no
visible evidence) and 16 calls to
Max.
This texture also contains calls to Mirror that produce no apparent
result. Min is used to
threshold the dark gray noise texture into the blue, green and purple
patterns.
This is the only example I saw where the effect
of
Mirror is obvious. There
is a roughly vertical line of reflection passing very close to the
origin
(see the outermost call to
Mirror
in the evolved code below). There are two other fold lines arranged
symmetrically to the right and left of the main fold. So I suspect we
are seeing the result of two nested calls to
Mirror. The program actually
contains four calls to
Mirror,
three have identical angles and three have identical centers. Those
curved black “lines” appear to be
level sets of the
Noise function.
// coc20091126b -- mirrored black curved lines on green
Mirror (-0.137124, Vec2 (0.00175132, -0.00273174),
Tint (Pixel (0.918163, 0.998836, 0.0841015),
BrightnessToHue
(Gamma (2.60127,
BrightnessToHue
(Mirror (-0.13362, Vec2 (0.00175132, -0.00273174),
Mirror (-0.137124, Vec2 (0.427678, -0.137124),
Mirror (0.159892, Vec2 (0.00175132, -0.00273174),
BrightnessToHue
(BrightnessToHue
(Gamma (2.57371,
BrightnessToHue
(Noise (4.69424,
Vec2 (0.427678, -0.137124))))))))))))))
Adjusting brightness, saturation and hue in GP
November 25, 2009
I added
Gamma
(
July 7),
AdjustSaturation
(
July 16) and
AdjustHue
(
July 17) to the set of
functions used by GP to evolve textures. The texture to the left uses
AdjustSaturation and the curvy
edges come from
Wrapulence
which was added to the set yesterday.
This evolved texture combines some triangle and
sine gratings with AdjustSaturation,
VortexSpot and BrightnessWrap.
This has an interesting look, as if it were a
cut
paper
construction (
see) with
lots of little paper flaps each casting a soft
shadow. The
irregularity of the grid is also cool. Note in the code below that
inside the
Gamma there is an
Array of the product of
VortexSpot and
Twist. The
VortexSpot operates on another
Array of a
TriangleWaveGrating.
// coc20091124e -- yellow "cut paper construction"
Multiply (Gamma (1.80957,
Array (Vec2 (-1.85723, 1.37489),
Vec2 (-0.06757, 0.166059),
Multiply (VortexSpot (2.42049, 2.30038,
Vec2 (0.44465, 1.69864),
Array (Vec2 (0.0913006, -2.168),
Vec2 (-0.06757, 2.31276),
TriangleWaveGrating (163.345, 0.980238, 2.91205))),
Twist (2.20778, Vec2 (-1.12413, -1.45564),
Tint (Gray (2.41503),
ColorNoise (2.28193, Vec2 (0.00296707, -2.23223))))))),
Multiply (Tint (Gray (2.57423),
TriangleWaveGrating (169.038, 0.657956, 3.337)),
Gradation (Vec2 (0.940063, 0.900727), Pixel (0.889256, 0.79022, 0.226771),
Vec2 (0.0913006, -2.168), Gray (2.57423))))
GP support for the 1d texture operators
November 23, 2009
The
July 4 entry
describes defining 1d textures as the y=0 “slice” of a 2d texture. It
shows examples of using these 1d textures to construct gratings, radial
patterns, nonlinear shears and colorizations. I connected these
operators (
SliceGrating,
SliceToRadial,
SliceShear and
Colorize) to the GP function set
after adding a missing
angle
parameter to
SliceGrating and
a
center parameter to
SliceToRadial. I also belatedly
added the
Wrapulence
generator to the GP function set.
This first image shows a spoked pattern based on
SliceToRadial.
It is a little hard to tell by looking at the
huge evolved program (size 329) but this seems to be a SliceShear applied to ColorNoise, perhaps using Noise as the shearing profile.
It is hard for me not to interpret this as a 3d
rendering of textured surface with a soft highlight -- like a molded
plastic surface, or quilted textured satin. It seems to be mostly the
product of
several TriangleWaveGrating,
see code below. The role of the SliceShear
is hard to fathom but may provide the high frequency detail overlaid on
the
lozenge pattern.
// coc20091123f -- pink textured bumps
Multiply (Gradation (Vec2 (-3.08321, -2.57112), Gray (3.14785), Vec2 (2.69166, -0.34243), Gray (3.03289)),
Multiply (TriangleWaveGrating (26.9361, 0.383393, 0.932858),
Multiply (Tint (Gray (2.81785),
TriangleWaveGrating (29.3164, 0.383393, 6.17656)),
Tint (Pixel (0.766134, 0.272602, 0.617417),
Tint (Gray (3.03289),
Tint (Pixel (0.722761, 0.272602, 0.617417),
SliceShear (-2.60991, 1.95581, 1.98506,
Vec2 (2.69166, -0.34243),
TriangleWaveGrating (26.9361, 0.383393, 0.932858),
TriangleWaveGrating (29.3164, 0.383393, 5.9962))))))))
Array, BrightnessToHue, BrightnessWrap in GP
November 22, 2009
The top two textures feature the
Array operator (
Apr.
26) evolved, like other recent results, with
GrayTheHardWay and a bias for
including
a given operator in the genetic program. This first image has a nice
“pen and ink” style
of “shading”.
Here I again used a bias to include a call
to Array but only a
single call was allowed. That led to this simple parallelogram tiling
of
texture space.
This texture shows the effect of a call to
BrightnessToHue (
June
8). Evolving programs with this operator also found another bug in
the RGB↔HSV
conversion routines related to hue wrap around.
The large swirls come from VortexSpot, the scratchy patterns
on top of it are apparently the result of BrightnessWrap.
Karl Sims noted that evolutionary computation is very good at finding
bugs. I have certainly been having that experience. One of my first
runs with BrightnessWrap
showed that it failed when the two brightness thresholds are equal
(leading to a division by zero and NaN). Not the sort of thing a human
programmer would ever do, but evolved programs don't know what makes
sense and what does not.
I also added a new constructor called Gray
based on Pixel::gray which (unlike the existing Pixel constructor) permits values
larger than 1, allowing Tint
to scale up brightness.
Note that at least for the time being I decided to leave these
texture operators out of the GP function set: ExperimentalRandomStamper (too slow
for now); ExperimentalDensitySlicer,
DensitySliceTenRamps and DensitySliceTenSteps (replaced by Colorize)
CacheTest and ExperimentalTextureToWaveform
(intended only for testing).
SoftThreshold, Twist and VortexSpot now in GP
November 20, 2009
I continue to work through the library of texture
synthesis operators,
cleaning up and
interfacing them to genetic programming plus watching the test GP runs
for cool results. I continue to use the fitness bias to test each new
operator.
This
texture shows
SoftThreshold
(
Mar. 28) applied to
Furbulence (
May
18) noise. Indeed it appears to be “sliced off” with a bright top,
a dark bottom, and fractal swirls in between.
(Note: for now I decided
against adding RemapContrast
and
AutomaticExposure to the GP
interface. They were early experiments and
overlap with SoftThreshold
and Gamma.)
This lovely pattern is based on
Twist (
Apr. 3).
I first
assumed it also used
Row but that operator does not
appear in its genetic program (below). In fact the “rows” seem to be
subtly different.
// coc20091120a -- "rows" of magenta swirls made with Twist
Subtract (Add (SineGrating (22.1262, 3.28543),
SineGrating (22.1262, 3.28543)),
Tint (Pixel (0.107084, 0.824255, 0.0604564),
Twist (0.497524,
Vec2 (0.0176352, 0.298148),
Add (Twist (0.411332,
Vec2 (0.0176352, -2.94306),
Rotate (3.11221, Vec2 (0.0176352, -2.94306),
TriangleWaveGrating (59.8382, 0.758788, 2.70904))),
UniformColor (Pixel (0.107084, 0.888559, 0.00960653))))))
This texture clearly shows its nested
calls to
VortexSpot (
Apr. 3) operating
on a tinted
SineGrating.
// coc20091120c -- green nested vortices
Subtract (Add (VortexSpot (-1.65995, 5.71535, Vec2 (0.284611, -0.385227),
HueOnly (Add (Gradation (Vec2 (-1.72737, 3.04276),Pixel (0.603485, 0.78617, 0.173666),
Vec2 (0.284611, -0.385227),Pixel (0.603485, 0.793609, 0.173666)),
VortexSpot (-1.65995, 5.71535, Vec2 (0.284611, -0.385227),
HueOnly (Add (Gradation (Vec2 (-1.72737, 2.92979),
Pixel (0.594164, 0.793609, 0.173666),
Vec2 (-1.33214, 2.43265),
Pixel (0.631017, 0.825947, 0.173493)),
VortexSpot (-1.42391, 6.04581,
Vec2 (0.383372, -0.385227),
SineGrating (76.6329, 4.81461)))))))),
VortexSpot (-1.42391, 6.04581, Vec2 (0.284611, -0.385227), SineGrating (76.6329, 4.81461))),
UniformColor (Pixel (0.736725, 0.783351, 0.16919)))
Wrap, Ring, HueOnly and NaN
November 19, 2009
Adding new operators to the GP function set has
been slow due to lingering issues in the slap dash implementation of
the texture synthesis library. While
Wrap
had a
center parameter
since it was written
(Feb. 6) it was being
ignored. I fixed that and tried adding a “bias” in the fitness metric
to
favor programs containing a given operator, to help create a texture to
illustrate its effect. This was only marginally successful.
The texture to the left shows ray-like features converging on a spot in
the lower right. This appears to be formed by nested calls to
Wrap in the evolved program.
I added
Ring
(
Mar. 19) to the GP function set and gave it a
center
parameter.
For both
Wrap and
Ring I made the sector count round
instead of floor (a parameter of 5.9 produces 6 sectors not 5).
This texture was supposed to be a demo of Ring, and indeed there is a
6-fold division of the image. But instead of 6 copies of the same
“pie slice” there seem to be 3 different patterns, each appearing twice
on opposite side of the center point. Also odd is that the center of
“rotation” is really close to the origin. Yet none of the
Rings in the
source code specify a center near there.
GP's evolved programs are often hard to understand. My application to
texture synthesis is fortunate to have an alternate representation that
is easily understood by our visual perception.
// coc20091118g -- six-fold black and brown pattern above
Add (Subtract (Subtract (ColorNoise (3.63764, Vec2 (-1.43951, -3.11809)),
ColorNoise (5.51281, Vec2 (2.59751, -1.10615))),
Subtract (SineGrating (145.714, 0.704762),
Ring (5.95409, Vec2 (-1.43951, -3.11809),
Ring (5.92754, Vec2 (2.59751, -1.10615),
Subtract (ColorNoise (3.37386, Vec2 (-1.43951, -3.04099)),
Subtract (Subtract (SineGrating (153.042, 5.17587),
SineGrating (148.731, 0.582571)),
Ring (6.23521, Vec2 (-1.10615, -1.53761),
Subtract (SineGrating (151.743, 5.17587),
ColorNoise (3.63764, Vec2 (-1.43951, -1.98898)))))))))),
SineGrating (153.042, 5.17587))
Adding
HueOnly
(Mar. 23)
seemed simple, its only parameter is the input texture.
Shortly into the first test run I got an error in the RGB↔HSV
conversion routines. It turned out to be caused by the dreaded
NaN,
usually the result of a divide by zero. The color space transformation
were not
at fault, they were just one of the few places in my code that
did not silently ignore such results. Using
HueOnly as a detector, I was able
to track down three different bugs in my code. I fixed a divide by zero
in
Gradation when the two
“vertices” were coincident, which can easily happen as a result of GP
crossover. There was an issue in the
Noise
generators for a scale of zero which was really a bug in the
SmallPosFloat type. Finally the
NaNs lead me to a bug in
Ring
due to a misplaced “&”. The saturated
cyan and red patches in this texture are apparently produced by
HueOnly.
Fixing leaks and mismatch
November 17, 2009
On
November 12 I
described a mismatch between a synthesized texture and the numeric
constants in the printed version of the same evolved program. I found
the problem went away when I turned off my new “jiggle mutations.” I
had failed to understand that Open Beagle's representation of GP trees
make extensive use of sharing. I had been mutating shared constants in
place, causing unintended side effects. Prof. Gagné suggested I
clone the constant, link the new one into the tree, then jiggle it.
The image to the left is the best of a run where the population
converged on programs containing several nested invocations of the
Wrap operator.
I plugged the “memory leak” where Texture
objects created during each fitness test were not being deleted. My
first quick and dirty fix managed to reclaim 97% of the objects (according
to counters I put into the base class). I might have left it at
that but got to worrying
about the 3% causing a problem at a very bad time. So did the Right
Thing and improved the code to reclaim 100% of allocated Textures. I also fixed CachedTexture to allocate pixel
arrays only when caching is enabled and StretchSpot to allocate its look-up
table only when needed.
Its hard to tell but this texture's evolved
program contains several StretchSpot
operators. I was fiddling with the function set to try to get a more
obvious example of StretchSpot
but it refused to show itself.
An interesting pattern based on several RadialGrads.
White and purple
Brownian layers made with
SoftMatte.
A combination of gratings produce the basic
sheared grid near the center. What caught my eye was the aperiodic
squeezing of the texture on the left and right. The evolved program
contained several calls to Translate
and one to Row.
Adding more texture operators to STGP set
November 12, 2009
I really like this first texture. I had just
added
SoftMatte to the set of
functions used in evolution. I turned off
the gratings to look for other kinds of patterns. This looks like a
green
pattern was
composited
over a monochrome
background pattern. When I looked at the evolved program, that was
exactly what
it was, see the code box below.
But as I typed this post, I noticed the code does not match the image!
For example, the RGBs of the
Pixel
parameter to
UniformColor
indicate a desaturated magenta, not the pea green seen in this image.
Similarly while the image
shows a portion of a
SoftEdgeSpot,
it is not the same portion I see
when I feed this evolved code back into my texture renderer. Could
there be a problem with the printing
of evolved programs? Specifically a problem printing the numbers?
// coc20091111b -- Brownian clouds over a large SoftEdgeSpot
SoftMatte (Brownian (0.56191, Vec2 (-1.25605, 2.11111)),
UniformColor (Pixel (0.562399, 0.34044, 0.504399)),
SoftEdgeSpot (1.64468, 3.00466, Vec2 (-1.25605, 2.11111)))
In addition to the functions mentioned in the
November 8 post, these new texture operators
have been added to the
STGP function set:
SoftMatte,
Scale,
Translate,
Rotate,
Row,
Invert,
Tint,
StretchSpot and
Wrap. These are all described in
earlier posts in this diary. The stand-in fitness function
GrayTheHardWay only looks at pixels
in isolation. Texture operators that just move pixels around (like
Translate) tend to have little
effect on fitness. So they don't get amplified by evolution and may be
discarded by mutations. On the other hand
SoftMatte,
Invert and
Tint, do change pixel values and so
appear more frequently in the “best of run” individuals. One such best
program retained a
Rotate of
ColorNoise, probably an
accidental “hitchhiker,” the
ColorNoise
was helpful, the Rotate didn't hurt and came along for the ride.
An example with
SoftMatte and some
SineGratings.
Other recent changes: I decided to remove the now prehistoric
Monochrome operator (
November 28) since the
AdjustSaturation operator (
July 16) now subsumes that functionality. I added
a
center parameter to both
Rotate and
RadialGrad. They had previously
defaulted
to the
origin. See discussion on
March 29 about why
explicit parameters seem better than nested transformations in the
context of genetic programming.
I have also been reminded that I have a “memory leak” in my Open Beagle
GP fitness function. That needs to be fixed.
Nothing technically interesting here, I just
liked this “glowing shiny yellow rope” pattern.
Avoiding midrange pixels
November 9, 2009
GrayTheHardWay,
my stand-in fitness function, has an strong preference for gratings. I
tried adding a penalty for pixels with midrange brightness to see if it
helped. It didn't. Mostly what evolved where combination of gratings
arranged to produce high contrast patterns containing bright or dark
pixels with few midrange pixels.
I had to remove the gratings form the STGP function set to get other
kinds of results, like this top image. Several noise patterns are
transformed to produce dark blue lines on a white background.
The other textures are the more typical results, ranging from simple
tinted SineGratings to more
interesting combinations like these:
// coc20091109f -- evolved code for "black spots on red spots on white"
Subtract (Add (UniformColor (Pixel (0.513956, 0.572782, 0.355166)),
UniformColor (Pixel (0.963484, 0.382174, 0.87067))),
Add (Subtract (UniformColor (Pixel (0.513956, 0.572782, 0.861566)),
UniformColor (Pixel (0.650672, 0.701417, 0.614527))),
Multiply (Add (Subtract (Add (Add (Subtract (SineGrating (173.7, 1.49421),
SineGrating (102.063, 0.681468)),
Multiply (Add (UniformColor (Pixel (0.382174, 0.6852, 0.870534)),
Add (UniformColor (Pixel (0.513956, 0.572782, 0.355166)),
UniformColor (Pixel (0.513956, 0.572782, 0.861566)))),
SineGrating (122.196, 2.56581))),
Add (Subtract (UniformColor (Pixel (0.513956, 0.572782, 0.861566)),
UniformColor (Pixel (0.963484, 0.382174, 0.87067))),
SineGrating (173.7, 1.49421))),
SineGrating (102.063, 0.681468)),
UniformColor (Pixel (0.650672, 0.701417, 0.614527))),
SineGrating (122.196, 2.56581))))
New generators in GP function set
November 8, 2009
I have added most of the texture generators
previously defined in
this library to the genetic programming function set specified in
Open
Beagle. The generators now include:
UniformColor,
SoftEdgeSpot,
Gradation,
RingedSpot,
SineGrating,
SawtoothGrating,
SquareWaveGrating,
TriangleWaveGrating,
SoftEdgedSquareWaveGrating,
RadialGrad,
Noise,
Brownian,
Turbulence,
Furbulence and
ColorNoise. The textures in
today's post
were made with the same small set of
texture operators:
Add,
Subtract and
Multiply.
After I added
RingedSpot (see
Dec.
4) this texture was discovered by
GrayTheHardWay
(see
Nov. 4). I decided that
RingedSpot was too visually
distinct and removed it. I think rotational patterns based on “1d
textures” are a better approach (see
July 4).
A dark red
Brownian brightened by Noise
to produce the pink and white highlights.
A mixture of SineGrating,
RadialGrad and several types
of noise.
As soon as I introduced these generators I started getting results that
were just a single instance of one or the other. Basically they provide
exactly
the sort of texture that GrayTheHardWay
is looking for. A slight
evolutionary pressure to reduce evolved program size would soon
optimize down to a single bare generator. I got around that by adding
another criterion to the fitness function, requiring that a texture
have at least 10% average saturation. A monochrome grad has zero
saturation so can't compete. As a result I started getting lots of
pale, pastel, 10% saturation gratings. We must choose which battles to
fight, so I decided that was good enough.
This is some combination of RadialGrad and SineGrating.
The five grating generators presented a problem. I realized that the
typical usages in this diary have frequency
parameters up to 200. This is outside the range of my existing types of
STGP ephemeral constants. I considered changing the parameterization of
the gratings from frequency
to wavelength. In the end I
decided to introduce a new class of BigPosFloat
values.
This is Furbulence noise combined with some
Gradations to produce this texture. To me it looks like a dyed anodized
finish on
textured aluminum.
This was pretty surprising! That curved diagonal
edge is apparently a large SoftEdgeSpot.
The bar “behind” it is probably a SineGrating.
Then the whole thing is softly tinted. Who knew?
This combines SineGrating
creating the vertical bars and Turbulence
providing the smaller swirls. The light areas are pink and darks are
cyan. See evolved code below.
Next on the agenda is the rest of the texture operators described below.
I turned off the “parsimony pressure” mentioned above that favors
smaller evolved programs. (Its called a shrink mutation in Open
Beagle.) In fact for this stage I would be happier to have larger
programs which I expect to produce more complex images. I have had some
trouble initializing the evolutionary population with larger programs
in Open Beagle. I will be looking into that.
// coc20091107d
Multiply (Add (ColorNoise (2.64887, Vec2 (1.09611, 2.41177)),
UniformColor (Pixel (0.484014, 0.152003, 0.0768818))),
Add (SineGrating (0.493139, 4.79085),
Subtract (Multiply (Gradation (Vec2 (0.935534, 2.35055),
Pixel (0.484014, 0.152003, 0.0768818),
Vec2 (2.35055, 1.62926),
Pixel (0.153538, 0.250275, 0.653235)),
SineGrating (0.493139, 4.79085)),
Subtract (UniformColor (Pixel (0.484014, 0.152003, 0.0768818)),
Turbulence (0.198022, Vec2 (2.35055, 1.62926))))))
Ephemeral constants and jiggle mutation
November 4, 2009
Again I've been toiling in the trenches of
genetic programming with images generated only as a by-product. Today
some progress reports and a few images made during GP testing.
Recent work focused on getting my GP “ephemeral constants” and their
mutation to work correctly within the
Open
Beagle framework. These “ephemeral constants” correspond to the
numbers in the code fragments in this document. I continued to receive
a lot of helpful and timely advice from Professor
Christian
Gagné.
(By the way these discussions are archived on Open Beagle's forum, for
example: this.)
I defined
three types of floating point values for my strongly typed genetic
programming: Fraction
with values on the interval [0,1], SmallFloat
on [-π,π] and SmallPosFloat on [0,2π].
Open Beagle provides a mechanism for choosing
initial random values for these ephemeral constants, and for
occasionally
replacing them with new random
values as a form of mutation during the evolutionary computation. I
extended that mechanism to mutate these values “incrementally” -- to
replace them with nearby values -- which is sometimes called “constant
jiggling”.
This turned out to be the most challenging part of connecting Open
Beagle to my library. The rest of the
interface was surprisingly easy. For the most part it Just Worked.
// coc20091026c -- evolved code for yellow-orange texture above
Add (Subtract (Multiply (UniformColor (Pixel (0.0743597, 0.71178, 0.14506)),
UniformColor (Pixel (0.72951, 0.0934074, 0.305996))),
Add (UniformColor (Pixel (0.82025, 0.36801, 0.2939)),
UniformColor (Pixel (0.30592, 0.0375148, 0.81569)))),
Add (Add (SoftEdgeSpot (0.32795, 1.76537, Vec2 (-0.7325, -0.491798)),
SoftEdgeSpot (0.158763, 1.84967, Vec2 (-0.7325, -0.51546))),
Multiply (UniformColor (Pixel (0.14023, 0.12566, 0.71178)),
Subtract (UniformColor (Pixel (0.0743597, 0.71178, 0.28708)),
UniformColor (Pixel (0.30592, 0.0375148, 0.82953))))))
As before these textures were evolved using
GrayTheHardWay and a small set of
two generators, three operators, two constructors, and numbers. The
fitness function was modified since the previous post to require a
nearly flat histogram of pixel brightness. That is: nearly the same
number of pixels in the brightest bucket as in the middle bucket as in
the darkest bucket. In these examples, a texture gets a “point” for
each histogram bucket that is 95% full. Since then I changed it again
to give “partial credit” for partially full buckets.
I reverted the change mentioned on
October 23
regarding taking the absolute value of some evolved parameters. This
was solved by the
SmallPosFloat
type.
I think this texture was made after I added Gradation to the list of generators
used in evolution. It contains three distinct hues in addition to
clearly ranging from black to white. Its fitness is nearly perfect:
0.991926
The next step is to work through this texture synthesis library writing
the short bit of “glue code” needed to add each element to the GP
function set.
Evolved textures: “gray the hard way”
October 23, 2009
This is the first batch of test images from
evolution runs using a small subset of the texture synthesis and a
simple prototype fitness function I call GrayTheHardWay. Its goal is a
texture whose average brightness is about 50% but
which is composed of mostly non-midrange pixels. This top image is a
simplistic, visually stark, solution discovered in early testing.
As I tinkered with the fitness function I replaced the “non-midrange
pixels" criteria with one requiring “no large areas of flat unchanging
pixel values” which produced the other five more visually interesting
textures below.
They may now be “too midrange” so I will try adding another criteria
that scores higher for images having a wider distribution of brightness
values.
The GP function set used here consists of the
generators UniformColor and SoftEdgeSpot, the operators Add, Subtract and Multiply, constructors for Pixel and Vec2, plus numerical constants. The
numbers were of type Fraction
randomized on [0,1]. Thats fine for RGB components. But when used for
Vec2 positions they all end up
in the first (upper right) quadrant. As a result these look a bit like
shaded spheres (planets?) with highlights in that corner.
I tried fixing this with two different types of “ephemeral constants”
but that seemed to confuse Open Beagle, probably because they were both
based on the same underlying C type. I'm looking into that.
I made some minor tweaks to the library during
this work:
(1) added
center parameter
for
SoftEdgeSpot as discussed
on
March 29.
(2) some of the existing code assumed positive values for certain
parameters (e.g. for a radius), rather than add yet another numeric
type for small positive values I changed the code to take the absolute
value of the evolved parameter. (So far changed generators
SoftEdgeSpot and
SoftEdgeBar. Likewise these
routines already “sort” their first two parameters to be in the correct
order.)
(The phrase “gray the hard way” just
popped into my head as I was writing the code. I'm pretty sure it goes
back to a joke in a cartoon I saw as a kid in the 1960s where someone
brags they can skate so well they can “do a figure 8 the hard way: two
4s!” When I Googled it today it looks like the joke goes back at least
to Milton
Berle on the radio in January 1948.)
I picked the texture with the smallest evolved
program to reformat and display here:
// coc20091023b -- evolved code for brown texture above
Add (Add (SoftEdgeSpot (0.87495, 0.0574257, Vec2 (0.00256992, 0.179738)),
Add (SoftEdgeSpot (0.350079, 0.598863, Vec2 (0.674037, 0.835846)),
UniformColor (Pixel (0.990439, 0.751919, 0.500734)))),
Subtract (Subtract (SoftEdgeSpot (0.87495, 0.0574257, Vec2 (0.239699, 0.235976)),
SoftEdgeSpot (0.941648, 0.339242, Vec2 (0.146319, 0.313731))),
SoftEdgeSpot (0.487757, 0.6734, Vec2 (0.239699, 0.235976))));
In the interest of full disclosure I should point out that these 6
images were hand picked for visual interest from 15-20 evolution runs.
The GP runs used a population of 100 in 5 demes of 20 individuals, with
steady-state reproduction, for 100 generations. They took roughly 1 to
5 minutes to run on my laptop.
Evolution of texture with Open Beagle
October 16, 2009
In case anyone has been checking this page regularly, I apologize for
the lack of posts over the last month. I have been learning about and
experimenting with the excellent
Open
Beagle library for evolutionary
computation. After a long period as a confused newbie, I have finally
performed test evolutions of procedural textures using a subset of my
library and a simple fitness function. I was guided through this
process with a lot of helpful, patient advice from Open Beagle's
principle author Professor
Christian
Gagné at Université Laval in Quebec, and from another
experienced Open Beagle user, Sergiy Dubovik of Helsinki.
Thus far the evolved textures are uninteresting (imagine a middle gray
circle on a middle gray background) so no picture with
today's post. The subset of my texture synthesis library that is
“known” to Open Beagle consists only of the generators
UniformColor and
SoftEdgeSpot, the operator
Add, constructors for
Pixel and
Vec2, plus numerical constants.
This subset was intended merely to test the operation of the type
matching required in my application. Open Beagle provides support for
many styles of
evolutionary
computation. I am using
genetic
programming, particularly
strongly
typed genetic programming (“STGP”, which allows for mixtures of
types
like
Texture,
Pixel,
Vec2 and numbers) using a steady
state population (where individuals are replaced one by one rather than
as synchronized “generations”).
A small bit of programming is needed to make each texture synthesis
generator or operator known to Open Beagle. So it will take a little
time to include each of the 60 (!) types that have been defined in this
library. I will use this opportunity to review the library: some
operators will be removed, some arguments will be changed (like adding
a
center parameter for
SoftEdgeSpot
as discussed on
March 29). I will also be
refining the numeric types and the way they are mutated during
evolutionary computation.
Difference squared
September 12, 2009
Related to the roundness metric mentioned
yesterday: an operator to take the square of the difference between two
textures. This “difference squared” is a widely used mathematical tool
to measure similarity, often by integrating
difference squared over a range. Larger results mean less similarity.
Squaring serves two purposes, it makes all differences positive and
emphasizes larger differences. Although in the context of this texture
synthesis library, values are normally on the range [0, 1] so
differences are normally on the same range. Squaring such fractional
differences makes them smaller. Possibly it would be better to
take the absolute value of the difference? Or perhaps in either case
the result should be adjusted by an exponentiation within the same
operation?
Update: I replaced
DifferenceSquared
with a slightly more general
ExpAbsDiff
on
November 30.
BrightnessToHue (Gamma (0.3,
DifferenceSquared (UniformColor (Pixel::gray(0.5)),
Noise (0.09, Vec2 (3, 5)))));
In both of these examples, the result of DifferenceSquared is adjusted using
Gamma and then colorized with BrightnessToHue. The top image
shows DifferenceSquared
of Noise and middle gray. The
bottom image shows DifferenceSquared
of two different
portions of Noise.
BrightnessToHue (Gamma (0.055,
DifferenceSquared (Noise (.07, Vec2 (3, 5)),
Noise (.07, Vec2 (5, 3)))));
Experimental spin blur and other side projects
September 11, 2009
I am working on a side project spun off from
thinking about the “unround” nature of finite/truncated Gaussian
filters. Ken Perlin suggested “spin blur” of an image as a basis of
comparison for measuring its roundness. I have prototyped that using
this texture synthesis library. I am not sure if this would be a useful
operator for evolutionary texture synthesis since its result is very
similar to existing operator (see discussion for the bottom image on
July 4).
This experimental
SpinBlur
operator produces an effect like spinning an image on a turntable while
taking a time exposure photo of it. It averages the pixel values along
a given radius. This is implemented by scanning a grid of pixel samples
and adding the values into a 1d texture accumulator. A count is
associated with the pixel accumulator for each radius interval which is
used to normalize the result. The output for a pixel is looked up in
this 1d table indexed by radius. Note the aliasing evident in this
image, especially near the bright spot in the middle and along the main
axes. I assume this is a moire/interference between the pixel grid and
rotated 1d pixel lookup table. This may warrant an improved
implementation.
Speaking of side projects, work on texture synthesis has been slow
recently as I have been spending time learning about
Open Beagle, the open source
engine for
evolutionary
computation I plan to use with this texture synthesis library.
This SoftEdgedSquareWaveGrating
texture was used as the input to SpinBlur
above.
Edge detection and enhancement
August 24, 2009
Building on the recently improved
Blur operator, I
revisited the edge enhancement prototyped on
July
20. Today's results made with two new operators are almost
identical to
those earlier experiments, except that these images show the effect of
the
operators on
colored images. This
blobs
texture was made by thresholding three sections of
Noise and using them to matte three
colors over gray, see code below.
This image shows the result of the new
EdgeDetect operator with a filter
width of 0.02 applied to the top image. As in the
July 20 example, the
EdgeDetect operator is a high pass
filter that returns the
signed edge detection offset by (added to) middle gray. Note how the
inside of the blob boundaries have a gradation of the blob's color,
while
outside the boundaries have the inverse color. So the red, orange and
yellow blobs have “halos” of cyans and blues.
(Orange is the
only tertiary color that has an unambiguous name in English. Its
inverse hue is the color halfway between cyan and blue. We may have
more hue discrimination near orange because human skin tones fall in
this part
of color space. These are also the hues most relevant to finding ripe
fruit.)
This is the result of the new
EdgeEnhance operator using filter
width of 0.02, and a value of 2.5 for enhancement “strength”. The
signed edge detection signal is multiplied by strength then added
to the input to produce this result. As mentioned below,
this approach is also known as
unsharp masking.
a = SoftThreshold (0.60, 0.64, Rotate (1, Noise (.09, Vec2 (3, 5))));
b = SoftThreshold (0.60, 0.64, Rotate (3, Noise (.09, Vec2 (6, 4))));
c = SoftThreshold (0.60, 0.64, Rotate (5, Noise (.09, Vec2 (1, 7))));
blobs = Tint (gray (0.833),
SoftMatte (a,
SoftMatte (b,
SoftMatte (c,
UniformColor (gray (0.6)),
UniformColor (yellow)),
UniformColor (orange)),
UniformColor (red)));
edges = EdgeDetect (0.02, blobs);
enhanced = EdgeEnhance (0.02, 2.5, blobs);
Better Blur
August 19, 2009
OK, the mysteries from the previous entry have
been cleared up and there is now a better
Blur operator. As noted on
August 14 (second and third images) the blurred
result was not radially symmetric as expected. The cause turned out to
be my overzealous clipping of the “tails” of the Gaussian filter in
order to limit the size of the convolution computed for each pixel.
This contour-map image shows an improved “rounder” version, although
you can still see a
bit of four-fold lumpiness near the center. I've
decided this is a reasonable trade-off between quality and
computation speed. Similarly the blurred version of
ResolutionTest now has
less (but not zero) high frequency content.
I renamed this new Gaussian filter
Blur
and renamed the old operator to be
Old3x3Blur.
Blur is now called with two
parameters: a filter width (radius) and the texture to be blurred.
I added a second constructor to
Blur
to provide the old 1-arg calling sequence with 0.005 for the default
width. I
added a hack to avoid long computation times for huge filter sizes: if
the
filter width is greater than 0.25 then
Blur will simply return the
texture's average
pixel value. This is pretty
ad hoc
and subject to further
review.
Colorize (SineGrating (150, pi/2), Blur (0.2, SoftEdgeSpot (0.295, 0.3)));
Post-SIGGRAPH fuzziness: wide blur kernels
August 14, 2009
It has been three weeks since the last post. That
has something to do with
SIGGRAPH
2009, but only one of those weeks was the conference itself.
Perhaps SIGGRAPH's “schedule distortion field”
extends out another week on each side. Which brings me to wide
blur kernels. I wrote most of this code on the flight back from New
Orleans but didn't finish it up until today.
This is the input image before blurring.
This is a wide low pass filter kernel applied to
the white spot in the top image. The sharp-edged spot has a radius of
0.3 and the blur
kernel has a radius of 0.2 (for context, the entire image has a radius
of 0.5).
The blur kernel is formed with two perpendicular applications of a a 1d
Gaussian.
I used the Gaussian because this separability leads to a fast
implementation and because the resulting
impulse response or
point spread function is nice
and round.
Except that near the center it didn't look so round to me...
I used Colorize
with a SineGrating to
make this contour map of the blurred spot above. It is hard to ignore
the presence of some squarishness in there. Near the center, and near
the
boundary of this texture image, the contours I expected to be circular
instead show four-fold undulations along the main diagonals of texture
space.
That can't be right. I
haven't yet found a problem in my code.
I tried to compare the performance of the old and
new
Blur. I
found the effective filter width of repeated applications of the old
3x3 filter was less than I expected (which was one pixel of radius
increase per iteration).
Tests show a Gaussian filter width of 0.05 is visually comparable with
40
iterations of my old 3x3
Blur
filter (yet the transitions in this image seem to be about 26 pixels).
The Gaussian blur was about ten times faster at this filter
size: 0.43 versus 4.03 seconds. The Gaussian's cost is
proportional to the filter width so I am considering how to limit that
value to a reasonable range. Otherwise evolution (GP) could blithely
decide to
use mile-wide filter kernels. I may just say that beyond some limit,
assume the filter is “really big” and return a single average color.
Yet another mystery is that a surprising amount of high frequency
information is preserved in this blurred
ResolutionTest
from
March 28. While the
original pattern did have infinitely sharp edges, compare this to the
image of
June 28 made with multiple
applications of the 3x3 filter.
Experiments with lots of spots
July 26, 2009
I have been looking at ways to parameterize
patterns with “lots of spots” using SoftEdgedSpot
as the drawing element. In this top image the centers of 60 spots are
uniformly distributed over a given disk. The outer radii are uniformly
distributed between two given bounds. (The inner radius for the
transition from black to white is 80% of the outer radius producing the
relatively hard edged spots shown here.) A simple iterative relaxation
procedure is used to avoid overlap by
pushing the spots apart.
Here the likelihood of a spot appearing at a
given location is specified by a texture parameter. In this case that
texture has zero likelihood
(brightness) at the
center and gradually climbs up to one near the outside. (Yet the
transition
between spots and no spots seems surprisingly abrupt. This may result
from spots near the crowded outside “pushing” their neighbors toward
the center.) This image contains 70
spots.
Here the distribution of spot centers is again
uniform. A second texture parameter is used to modulate the radii of
100 spots. In this case the radius map is bright
in the
center and darkens toward the edge.
Perhaps the final
LotsOfSpots
operator will have texture parameters for both likelihood and radius.
Similarly, instead of
drawing all spots in white, it could take another texture to provides a
color for a spot drawn at any given location. (As in Paul Haeberli's
1990
Paint by Numbers:
pdf,
pictures,
applet.) Or
rather than draw
featureless
spots, the operator might take another parameter to provide color
texture inside each spot. (As in the random stamper prototype described
on
May 9.) A given disk of this
texture would be scaled to fit into each spot.
Prototype edge enhance
July 20, 2009
My original
September 20
to-do list included two convolution-based operators: blur and edge
enhancement. I wrote a simple 3x3
Blur
on
November 28 and a
3x3 edge detector on
November 29.
Blur could stand in for low pass
filters with larger kernels by repeated (nested) applications of the
3x3 filter. But a 3x3 edge detector is too small to be useful. I still
need large kernel high pass filters
for edge detection and enhancement.
Today I experimented with
unsharp masking.
A high pass “edge detecting” filter is created by taking an input image
then subtracting off a blurred
version of itself, leaving a
residue of high frequency edges. Adding the high frequencies back into
the original image serves to “enhance” the edges.
This top image is the input. Brightness is 80% in the center and drops
by half each step.
The middle image is the result of prototype edge
enhancement applied to the top image. It is the sum of the original
input and the high frequencies seen in the third image. In this example
the low pass filter used had a kernel diameter of 21
pixels (10 iterations of the 3x3
Blur
operator). The
high
frequency texture was scaled by 2 before being added to the original
input.
My daughter, a frequent contributor to this diary, remarked that the
top image looked “flat” while the enhanced middle image “really looked
3d.” I
was not aiming for a 3d look but indeed its not hard to
interpret this as the top view of a
stepped
cone. The bright “lip” on the
inside of the edges could be seen as highlights on rounded corners. The
dark “trough” on the outside of the edges could be seen as shadow.
This bottom image shows the high frequency edge
detection. It is a signed value which is shown here added to 50%
gray. The middle image shows the sum of this
edginess texture (see
code below) and the top
image.
In the process of this experiment I wrote the long neglected
Subtract operator.
(Random aside: while I originally studied these digital signal
processing filters in school, I tend to associate this idea of edge
enhancement with Richard
Taylor my friend, former boss at triple-I
and graphic design mentor. He told me that skilled airbrush
artists would make the edges of key graphic element pop by painting a thin bright stripe
just inside the element's edge, and a thin dark stripe on the
background just
outside the edge. In the 1970s, Richard and his colleagues at Robert
Abel & Associates had adapted this and related techniques in
the development of their ground-breaking candy
apple neon motion
graphics techniques.)
lowpass = Blur (Blur (Blur (Blur (Blur (Blur (Blur (Blur (Blur (Blur (input))))))))));
highpass = Subtract (input, lowpass);
edginess = Tint (Pixel::gray(2), highpass); // scale enhancement by 2
texture = Add (input, edginess);
Max and Min
July 19, 2009
I added a Max
(and Min) operator that
combines two textures by selecting at each pixel the input that is
brighter (or darker). In this case two half-brightness, perpendicular,
diagonal SineGrating
textures are combined with Max.
For comparison, here are the same two input
textures combined with
Add
(on the left) and
Multiply
(on the right). Adding the two textures together leads to something
like a
foam
“egg carton” pattern with peaks twice as bright as the input
texture.
Multiplication leads to isolated peaks at half the brightness of the
input. In contrast the
Max
operator simply merges the two inputs, preserving the geometry and
brightness of at least one input.
Here two portions of Perlin Noise are tinted yellow and cyan
then combined with Max (see
code below). Think of this as two undulating intersecting surfaces. We
see shades of cyan where it is brighter (higher) and shades of yellow
where it is brighter.
Max (Tint (yellow, Rotate (2, Noise (.06, Vec2 (13, 5)))),
Tint (cyan, Rotate (4, Noise (.06, Vec2 (7, 11)))));
Mirror
July 18, 2009
This Mirror
operator simply reflects a texture about an axis specified by a point
and an angle. In this example, colored Turbulence is used as the input
texture and zeros for angle and center specify the Y axis for mirroring.
Adjusting hue
July 17, 2009
After adding operators for adjusting brightness
and saturation, the
HSV adjustment suite is now rounded out with AdjustHue which takes a hue offset
and adds it to the hue of each pixel (wrapped back onto the range
[0,1]).
This first image is the input. Note the hue progression along the
spokes goes from red at the center through cyan at mid-radius and back
to
red at the outside.
This is the result of applying AdjustHue with a
parameter of 0.5 to the top image. The gray background gradation is
unchanged while
the hue progression now goes from cyan at the center through red at
mid-radius and back to cyan at the outside.
rainbowSpokes = SoftMatte (RadialGrad (12),
Invert (SoftEdgeSpot (0, 0.6)),
Wrap (1, 0.0001, Vec2 (),
BrightnessToHue (SawtoothGrating (4*pi, 0))));
texture = AdjustHue (0.5, rainbowSpokes);
Adjusting adjusting saturation
July 16, 2009
Upon reflection I decided that my
July 14
implementation for
AdjustSaturation
was utter hogwash. Because the RGB↔HSV conversion wants saturation to
be on [0,1], adjusting saturation with a gamma-like exponentiation
seemed like a good idea. But it has the minor drawback of doing the
Wrong Thing. If a pixel in the input texture is fully saturated (has a
value of 1.0) it
could never be reduced by exponentiation.
I changed
AdjustSaturation to
interpret its numeric parameter as a scale factor. Each pixel's
saturation is multiplied by this factor, then clipped onto the interval
[0,1]. While this could lead to Mach band artifacts when saturation
goes out of range, I think it is closer to what the operator should do.
Nonetheless, the difference between the new and old implementation is
very subtle for most textures like
the pastel test case used here. Today's image shows the second image
from
July 14 with its saturation increased by
a factor of 1.5. It is very similar to July 14's third image.
Adjusting saturation
July 14, 2009
While thinking about HDR representation on
December 5, I made a list of
additional To Do items, like operators for brighten and darken (now
covered by the
Gamma
operator) as well as tweaking hue and saturation. Today I wrote an
AdjustSaturation operator that
works like
Gamma to push
mid-range values up or down.
Saturation values are defined by HSV to be on the interval [0,1] so
gamma style exponentiation alters the mid-range while
leaving the ends of the scale unchanged.
This image shows
AdjustSaturation
applied to the middle image, with its saturation pushed down
(desaturated) using an exponent of 1.5
This middle image is the starting point for the
other two. It is
the colored
Furbulence
texture (third image on
July 9) “averaged”
with white: the two textures were added together then divided by 2.
This image shows AdjustSaturation applied to the
middle image, pushing up its saturation
using an exponent of 1/1.5
Colorful non-API
July 9, 2009
On
May 8 I
prototyped
ColorNoise but
didn't decide to officially add it to the library until July 3. Today I
prototyped these
three colorful versions of fractal noise and then decided
not to add them to
the library as predefined named functions. While one generator for a
multicolored
pattern seemed like a good idea, four seems like overkill. Nonetheless,
I thought they were attractive and unique enough to mention here.
In this example, three different portions of
Brownian noise were tinted in red,
green and blue then added together. See the code snippet below. All
three of today's images were made with analogous code. See
May 18 for the scalar grayscale versions of these
three textures.
Three portions of Perlin Turbulence, tinted in red,
green and blue then added together.
Three portions of Furbulence, tinted in red,
green and blue then added together.
Add (Add (Tint (red, Brownian (0.15, Vec2 (4, 3))),
Tint (green, Brownian (0.15, Vec2 (2, 5)))),
Tint (blue, Brownian (0.15, Vec2 (7, 6))));
Gamma
July 7, 2009
The
April 12 post
mentioned adjusting image contrast by “remapping pixel
intensities through an exponential as in
gamma correction”
before going off on another tangent. Today I returned to that original
intention and added a
Gamma
operator. A common use of gamma correction is adjusting the abstract
“perceptual” or “linear” brightness values used in computer graphics to
better match the contrast response of a given display device. In this
library it is being used simply to alter a texture by pushing more of
its dynamic range into lighter or darker values.
This is a portion of Noise texture, it is
unadulterated and so equivalent to a gamma of 1. The bottom image is
the same bit of Noise with
gamma of 2.2 and the top image has gamma of
1/2.2
(2.2 is sort of a cliché value for a discussion of gamma,
it is the typical value used to model video displays. It has no special
meaning in the context of this library.)
Note that in all three cases the total dynamic
range of the texture is unchanged, spanning almost all of [0,1]. The
darkest parts remain dark and the brightest parts remain bright. It is
the midrange (intermediate brightness values) that shift up or down.
See these
plots
of the
contrast curves.
Using 1d slices of 2d textures as “waveforms”
July 4, 2009
Of the two approaches discussed in the previous
entry I seem to be leaning toward defining “waveforms” (1d textures) as
the y=0 slice through a 2d texture rather than as a separate type.
Certainly that approach requires fewer changes to the library and its
interface to genetic programming.
Today's images represent four new operators that each take a “waveform”
or
1d texture as a input. The corresponding parameter is actually a 2d
texture (of c++ type
Texture&). In these examples the
1d
texture comes from
ColorNoise,
as in the first image of
the
May 8 entry. The image to the left shows
the generalized
SliceGrating
operator applied to a
waveform defined
by the y=0 slice of
ColorNoise.
(Update:
Grating was renamed
SliceGrating on
Nov. 23.)
In this image the 1d texture is indexed by the
angle of each pixel by the new SliceToRadial
operator. As a result the colors along the 1d slice of ColorNoise become colored rays
emanating from the center.
Here the brightness of one texture (see
June 8
and
9) is used to index into a 1d texture
(from
ColorNoise) to select a
color for the result. This new operator is called
Colorize.
Brightness values on the interval [0,1] in the texture being colorized
will map to x values on [0,1] along the y=0 slice of the texture
providing the colors. Brightness values are nominally on [0,1] but
this operator does not depend upon that.
This
image shows the result of a new operator called
SliceShear, analogous to
SineShear, but using a 1d texture
to
modulate the nonlinear shift. Compare this to the first image on
April
7. In this case the horizontal
displacements come from the brightness of the
Noise texture along its y=0
slice. (Update:
CurveShear
was renamed
SliceShear on
Nov. 23.)
Since I've shown no reluctance to endlessly grow
this
library, it may come as a surprise that I decided
against adding an operator to “sweep” a 1d-texture around a point
creating rings of the colors along the input texture's positive x
axis. I wrote an experimental version then realized the same
effect can be obtained with the
Wrap
operator described on
February
6.
The two code fragments below both produce the image to the left.
texture = WaveformSweep (Vec2 (),
ColorNoise (0.05, Vec2 ()));
texture = Wrap (1,
0.0001,
Vec2 (),
Rotate (pi/2, ColorNoise (0.05, Vec2 ())));
Formalizing “waveforms”
July 1, 2009
Since
December 20, there
have been several
references to what I later called the
“waveform as first class object” design issue. This post is just
discussion before any changes are made. The concept of
waveform first came up in the
context of
gratings.
It seemed clear that “gratingness” was independent from the choice of
“waveform” the variation of brightness that defines the nature of the
grating. So perhaps there should be a
Grating
operator that takes a
waveform as a parameter. Since back then there was no abstraction for
waveform,
as a temporary measure, I defined four generators of gratings for the
four basic
periodic scalar waveforms from signal processing:
SineGrating,
SawtoothGrating,
TriangleWaveGrating and
SquareWaveGrating.
So there
could be a
Waveform
base class which could be specialized for those four waveforms. However
it
seems like an important degree of visual expressiveness would be
provided if waveforms could be defined just like textures: by
the composition of generators and operators. Instead of a scalar
“waveform” perhaps the new first class object is a “1d texture” or
“colored string” or “pixel row”. The top image is the row of pixels
through the middle of a texture (see the second image for
April 7) with
the
rest of the texture colored like the mid-range gray background. Imagine
this being the result of applying a
Convert2dTextureTo1dTexture
operator.
Then you could take that waveform (or 1d
texture, as you can tell I'm struggling with nomenclature) and pass it
to a generic
Grating operator
to produce this second image. The operator would stretch
(smear?) the 1d texture perpendicular to its axis to create a 2d
texture. So an equally good name for
Grating
would be
Convert1dTextureTo2dTexture.
But wait, what if this generic
Grating
operator took a
Texture
rather than (lets just call it) a
Waveform?
What if it implicitly converted (
coerced
in programming language terms) the texture to a waveform by the simple
expedient of ignoring all but that center row of pixels? So this second
image could just as well be the result of passing that
April 7 texture
to a generic
Grating operator
which takes a texture for its “waveform” parameter.
That is what I am wrestling with now. Is it better to have a
Waveform class with operators to
convert between
Textures and
Waveforms? Or is it better to have
just one
Texture class which
in certain circumstances is interpreted as a 1d texture/waveform?
Simplicity is one argument for one rather than two classes. Conversely
it
might be seen as overloading the concept of 2d texture to include 1d
texture
functionality. From the standpoint of
GP, having
two classes requires using “strongly-typed GP” (which might be
desirable for other reasons) and also creates some genetic separation
between the two classes (which might be a good thing).
One ironic outcome from the “all textures are 2d, but are sometimes
interpreted as 1d” approach would be that I am left with the original
issue
from last December: how do I define a sine wave (etc.) grating? If I
use the generic
Grating
operator, what is the input texture? It seems like the only solution is
to define stand-alone texture generators that have a scalar (which is
to say, gray scale) waveform along their y=0 axis. And that, it turns
out, exactly describes those “just temporary” generators from
December 20:
SineGrating,
SawtoothGrating,
TriangleWaveGrating and
SquareWaveGrating (and
subsequently on
April 3, the oddly named
SoftEdgedSquareWaveGrating).
Laptop upgrade
June 28, 2009
My old laptop was getting slow and decrepit, just
like its owner. So last week I upgraded from my 4.5 year old
PowerBook G4 to a new MacBook Pro. The new machine is much faster along
several dimensions (higher clock speed, 2 cores, more cache, etc.) and
I wanted to know how much faster it was on the calculations used in
this library. I made a test case with
Blur
applied 20 times to the
ResolutionTest
texture generator
from
March 28. On the old machine it took
12.27 seconds. On the new
machine it took 1.77 seconds, roughly
7
times faster.
In the process I fixed an old pixel addressing bug in the
CachedTexture class that also broke
the
ExperimentalRandomStamper
used for “Sparse Gabor Convolution” on
May 9.
Generalized wrap-around in brightness
June 11, 2009
While fiddling with
Wrapulence I tried a variation then
realized it was “just fmod of Brownian.” Not much to do with turbulence
or
fractal noise, but it seemed like a good idea in general to provide an
operator to do a
wrap-around (remainder, modulus) on the brightness of a given texture.
This image shows
BrightnessWrap
applied to
Brownian noise,
with the bounds of the wrapped region of intensity set to 0.3 and 0.7.
It converts the full range and smooth input to mid-range values with
discontinuous jumps. The irregular boundaries and hard edges give this
somewhat of a
grunge
quality like layers of corrosion and peeling paint.
This is
BrightnessWrap
applied to an input like the “Brownian sky” in the
May
15 entry. See
code below.
BrightnessWrap (0.4,
0.6,
SoftMatte (Brownian (0.15, Vec2 (27, 27)),
UniformColor (blue),
UniformColor (white)));
Tweaking Turbulence: “Wrapulence”
June 10, 2009
Describing the
May 18 Furbulence generator to a friend I
said that tweaking
Turbulence
to include a second “fold” suggested other ways to “add discontinuities
at all scales” as Perlin originally described
Turbulence. One of those was to
simply scale up the amplitude of the underlying
Noise signal, then apply
wrap-around on the interval [0,1]. For this image Perlin Noise on the
range [-1,1] is scaled up to [-3,3] then wrapped onto the range [0,1],
using something like the modulus or reminder operator (literally:
v-floor(v)). The rather wacky result may not have any
practical
applications but it is certainly unique. My daughter especially liked
it (“that is
soooooo cool!”)
which is justification enough to describe it here.
While
Turbulence contains
first order (slope) discontinuities at all scales, this
Wrapulence texture has zero order
discontinuities (sharp boundaries between light and dark) at all
scales. As a result the new texture is much “edgier” with sharp changes
in brightness everywhere. In general that is considered a bad thing
from the perspectives of signal processing, image quality and bandwidth.
(I had an encounter with a real world object that was “sharp at
all scales.” Getting out of my car in a parking lot I noticed that the
next car must have hit something really hard right on the edge of its magnesium
wheel. A bit of the alloy had ripped away looking almost like a
strobe picture of splashing water. The highly irregular fractal surface
glittered alluringly in the sunlight. Of course I had to reach out and
touch it--ever so lightly--making several bloody pin-pricks in my
finger. At that moment I was enlightened. Not only was it “sharp
at all scales” but also “pointy
in all directions.”)
Density slicing: ramps and soft steps
June 9, 2009
As mentioned on
May 15,
the
ad hoc density slicing
operator had
undesirable hard edges between its color steps. Here are some textures
produced with two new and improved density slicing operators. Each take
10 colors as parameters. (In these top two images, the colors are 8
copies of 30% gray plus yellow and white in intermediate positions.) In
the top image the
DensitySliceTenRamps
operator maps brightness into one of 9 linear ramps between the 10
given
colors. You can see bright
Mach Bands in the
yellow regions caused by slope discontinuity between two ramps (one
going from gray to yellow adjacent to one going from yellow to gray).
This is a call to DensitySliceTenSteps which maps
brightness into one of 10 flat color steps. It differs from the earlier
experimental density slicer by providing soft edges between the steps
with a softness parameter
between 0 and 1 specifying the width of the transition zone. It is
0.3 in this example. It uses the same 10 colors as in the top image.
DensitySliceTenSteps
with the same input texture and different parameters: softness is
0.7 and there are 7 unique colors, as shown in the code box below.
While these two operators are both fine ways to colorize, they seem a
bit
ad hoc since there are so
many other approaches that could be used. Ideally the sequence of
colors should be defined in a way as flexible as for textures
themselves. This seems another example of the “waveforms as first class
objects” issue. (Update: see new
Colorize
operator described on
July 4.)
DensitySliceTenSteps (yellow, gray90, green, gray90, black,
gray90, magenta, gray90, red, orange,
0.7,
graygratings);
Brightness to hue
June 8, 2009
Some of the recent experiments with noise
textures were portrayed with scalar (B&W) noise colorized by a
simple hard-edge
density
slicer. I later realized that there was a
very simple way to map brightness values onto a continuous hue scale
using
the existing RGB/HSV conversion routines.
This sample gray scale texture is the sum of three
sine wave gratings, see code below.
Here the gray scale input is colorized by mapping
brightness values onto the continuous hue scale using the new BrightnessToHue operator. Because
hue values are cyclic, this “fmod”-like mapping
conveniently handles brightness values outside the range of [0,1].
graygratings = Add (Tint (Pixel::gray (0.5),
Translate (Vec2 (0, 0.5),
SineGrating (pi * 2, 0))),
Tint (Pixel::gray (0.25f),
Add (SineGrating (50, pi * 0.6),
SineGrating (50, pi * -0.6))));
texture = BrightnessToHue (graygratings);
Preliminary Perlin Marble
May 28, 2009
I have been experimenting with the Marble texture described in
Perlin's 1985 paper. He takes a (potentially colorized) vertical sine
grating and at each pixel shifts its phase horizontally by the value of
Turbulence. The result ranges
from a pure sine grating, to a noisy grating, to a strongly warped
grating, to a fairly chaotic pattern of warped dots and rings. The
bottom image shows about 1.5 cycles of the sinusoid with a strong
turbulent component.
(I like the phrase “Perlin marble” as if it was in the same
category as “Carrera marble” or “Parian marble”.)
I haven't given code fragments or other concrete
descriptions of these images because I not sure I fully understand the
parameterization of this texture. (Even if we
ignore coloring issues, as in these B&W examples.) In addition to
scale and translation for the
underlying Turbulence, there
is also the frequency and angle of the sine grating. Perhaps also
a scale factor relating the [0,1] range of Turbulence to an angular phase.
That is, my code has such a factor but maybe it could be made a
constant without loss of generality?
Ken's definition of
Marble using a sine wave and
Turbulence has been widely applied
in computer graphics and other fields. Yet as was noted in the
April 7
discussion of SineShear, there are interesting waveforms beyond sine,
such as
described on
December 20 (square, triangle,
sawtooth) and many others
that could be defined procedurally. This seems to suggest another
argument for “waveforms as first class objects” in this library.
Perhaps another version of
Marble
could turbulently displace the phase of an arbitrary waveform passed in
as
a parameter. An even more generalized
Marble
operator could take a
waveform (playing the part of
SineGrating)
and a texture (to be used like
Turbulence)
then form the “marbleized” combination of the two.
Disalignment (“I happen to have Prof.
Perlin right
here”)
May 23, 2009
As noted on
May 15, my
implementation of Perlin's
1985
Turbulence texture
seemed to be full of annoyingly anisotropic vertical, horizontal and
diagonal features, as in the bottom image. Since “I happen to have
Prof. Perlin right here” (*) I asked and Ken said
that to avoid this artifact, he simply rotates the texture between each
level of the fractal summation, as shown in the nicely isotropic top
image.
(*) In his 1977 film Annie
Hall, Woody Allen's character Alvy Singer famously recruits noted
1960s media theorist Marshall McLuhan
to help make a point by pulling him into the
scene from off stage, saying “...I happen to have Mr. McLuhan right
here...”
Note that features in both images are the same at
the fundamental (lowest) frequency. For example, there is a nearly
circular features just below and right of the center of both textures.
I now rotate each subsequent octave of Noise by 2 radians (plus, as
before: doubling the frequency and
halving the amplitude). I chose 2 radians because it is roughly 1/3 of
a
revolution
but will never re-align with the irrational value (2 pi) of a full
rotation measured in radians.
Tweaking Turbulence: “Furbulence”
May 18, 2009
As mentioned in the
May 15
post, Perlin
Turbulence is
similar to
Brownian noise
except that it
“folds”
Noise using an
absolute value in order to add discontinuities at all scales. A result
of that is that
Turbulence
has sharp “valleys” with soft “peaks.” Thinking about that asymmetry I
wondered what would happen if I used
two
absolute values to make two folds in the
Noise signal. Folding the tops down
and the bottoms up produces a curve that has sharp
and soft features at both the top
and bottom of its range. Here we
see a density slicing of this new variation on
Turbulence.
This image shows the new texture in its raw
grayscale scalar form. I had copied the code for my Turbulence generator, renamed it
and tweaked it to try the double folds. Calling it “turbulence 2”
seemed unimaginative. I thought that the dark and light “tendrils” give
the texture a wispy, furry appearance, so I started calling it Furbulence.
Here for comparison is raw Perlin Turbulence.
Note how all the sharp features are dark. The bright features are soft
and puffy, broken by dark cracks.
(Update: changed this to show the improved “disaligned” version from
May
23, replacing the second image in the
May 15
entry.)
Here for comparison is raw Brownian Noise, where
both the peaks and valleys are soft and smooth.
Fractal Noise: Brownian and Turbulence
May 15, 2009
More bounty from Ken Perlin's 1985 SIGGRAPH paper
An Image Synthesizer.
Here we see
Brownian noise, a
sum of octaves of
Noise: a
fundamental frequency plus additional copies, each with twice the
frequency and half the amplitude of the previous copy. In the abstract,
a texture constructed this way has features at all scales. (While an
image of this resolution can only represent about 8 octaves.) I am not
certain of its origin but I associate
the use of “Brownian” to describe this kind of 2d fractal texture with
Benoît
Mandelbrot. In this image the noise pattern is colored to suggest a
layer of high thin clouds beneath a blue sky.
This is Perlin's
Turbulence texture, often used to
model marble, flame and other chaotic phenomenon.
Turbulence
differs only slightly from
Brownian
texture, summing octaves of the
absolute
value of Perlin Noise, which
is defined on the range [-1,-1]. The absolute value “folds” the
negative peaks of the smooth noise signal into the positive range,
causing sharp valleys between smooth peaks. As a result
Turbulence contains sharp details
(discontinuities), and because of its fractal nature these occur at all
scales. This image shows the raw
Turbulence
texture. I am surprised to see the visually prominent
dark features along the horizontal, vertical and to a lesser extent
diagonal. This bears further investigation. (Update: see entry
for
May 23)
This is the same Turbulence texture colored with an
experimental operator for density slicing. It divides the intensity
range of [0,1] into ten regions and assigns a color to each. (Here a
selection of four colors are each assigned to multiple bands.) I think
a
better density slicing operator should provide either soft transition
zones between the bands, or else color each band with a ramp between
its two neighboring colors.
Sparse Gabor Convolution
May 9, 2009
According to the
authoritative
but unofficial list by Ke-Sen Huang a paper called
Procedural
Noise using Sparse Gabor Convolution by Lagae
et al. will be published at
SIGGRAPH 2009. (As of this
writing the official list of papers has not been released.) So far only
an intriguing video is available on the project's website. Beyond
describing this novel type of textured noise, the video demonstrates a
user interface for authoring noise, and techniques for mapping the
texture onto arbitrary dynamic 3d shapes. The implementation is
apparently quite efficient, running in real time.
The video inspired me to make a prototype
implementation in my texture
synthesis library. The top image is the first result. This is the sum
of 2000
Gabor
kernels each with a random position and a rotation by an angle
uniformly distributed between two given angles. The middle image shows
the sum of the first few kernels.
Lagae
et al. also allow the
frequency of the kernel to vary between two bounds. Mine were all
the same.
(Sorry for the pixel addressing artifacts along the x and y axes in
these images. (Update: fixed on
June 28.) I
wrote this code in a rush
on a Saturday afternoon to
convince myself that such cool textures could be actually produced by
this surprisingly
simple method.)
This is the input texture used as the kernel. A
Gabor kernel is defined as the product of a sine wave grating and a
Gaussian spot.
This is in fact a cosine wave grating times a (co)sinusoid spot, since
those two generators already existed in the library and are visually
similar.
The kernel is shifted down in intensity from [0,1] to [-0.5,0.5] then
passed to an experimental random “stamper” which adds a given number of
copies into an accumulation buffer. Each copy is rotated by a random
angle (between two given bounds) and translated by a random vector (in
a given circle specified by radius and center). My prototype is so not real
time, the top image took several minutes to complete compared to
interactive
rates for the Lagae et al.
system.
gaborKernel = Multiply (SoftEdgeSpot (0, 0.1),
Add (UniformColor (Pixel::gray (-0.5)),
SineGrating (200, 0)));
texture = Add (UniformColor (Pixel::gray (0.5)),
ExperimentalRandomStamper (2000, 0, pi/4, .7, Vec2 (), gaborKernel));
Color Noise
May 8, 2009
Just playing with my new toy. This
image is made
with three copies of
Noise
shifted relative to each other. Each copy is then tinted with red,
green
or blue and the three are added together. Compare this to other
kinds of “blurry confetti” in the
April 12 and
March 23 entries.
(Update: on
June 30 I added a generator called
ColorNoise to produce this
kind of
RGB color Perlin Noise texture.)
Again three copies of Noise tinted with the primary
colors and added together. In this case it is three “octaves” of Noise. The lowest frequency and
highest amplitude is tinted blue. The green layer is half the size
(twice the frequency) and half the amplitude as blue. The red layer is
half the size and half the amplitude of green.
Without the tinting seen here, this summing of octaves of band limited
noise produces 2D Brownian noise. This classic fractal with 1/f power
spectrum was described by Mandelbrot. In his 1985 paper, Perlin gave
the name Turbulence to the
summation of the absolute value
of octaves of Noise.
Noise
May 4, 2009
The original specification for this
library
included a
generator for
Perlin
Noise, an enormously influential concept in procedural modeling
introduced by
Professor Ken
Perlin
in his landmark 1985 SIGGRAPH paper
An Image Synthesizer.
Perlin's Noise has had a huge impact, but is just one
of a stream of innovation that has flowed from the mind of this
modern polymath. If I gush too much it is because I am proud to
consider Ken a
friend and to have been his colleague since we met in the early 1980s
as young
geeks laboring on the production of the cult classic computer animated
film
TRON.
This image shows a unit diameter circle on the
z=0 plane of
Noise scaled down by 16. The
top image is the same but four times
bigger. The bottom image is 4 times smaller.
A key property of Perlin Noise is that its scalar
value is defined over continuous three dimensional space (ℝ3) making it
a “solid texture”. For the purposes of this 2d texture synthesis
library,
I use only the z=0 slice through the solid texture. I started with the
reference implementation from Ken's 2002 paper
Improving Noise and
later took out the computation for the unused z dimension.
Ken writes in his papers that
Noise is equivalent to white noise
that has been smoothed with a Gaussian low pass filter. That is almost
exactly what is shown in the first image of
April
12. One difference is
that this image is pure gray scale since Noise is a scalar value. The
April 12 image is gently colored because it began from the color
confetti of
Grain texture.
Stretch
May 1, 2009
After working to eliminate unwanted stretch from
the Array operator, I now
restore balance by introducing a Stretch
operator. It basically
scales an input texture in the x direction while leaving y unchanged.
In
addition to that scale factor it also includes parameters for rotation
and
translation.
The black ellipse is a Stretch
(by 0.5) of the white circle. With no rotation or translation, scaling
is applied along the horizontal. The red dot shows the center of the
transform. The blue line indicates the angle parameter and corresponds
to the fixed points of the transform.
It took me an embarrassing number of tries to get the 5 transforms
applied in the right order, so here are some more of the test cases I
used:
Stretch
with rotation only.
Stretch with rotation and
translation
More caching woes
April 28, 2009
At the end of March I fixed the blurring OpenGL
had been doing on the textures at display time. That allowed me to
remove the supersampling I'd been doing to work around it. So instead
of caching 300x300 textures at 600x600 I used what seemed The Right
Thing, a 300x300 cache. Everything looked fine until I started working
on rotated arrays and saw really nasty pixel position round-off errors.
The top image with no caching shows how this “standard benchmark”
should look
This image shows the artifacts caused by using
the (non-supersampled) cache in the presence of rotation. Round-off
errors in pixel coordinates cause small positioning errors that add
high frequency “fuzz” in the image.
Without a radically different scheme, caching is just not working out.
I will leave it turned off until performance issues require a different
implementation. I will leave it in place for Blur and related operators
that need to access each pixel of the input texture multiple times (and
do not involve rotation).
Obliging oblique arrays
April 26, 2009
As noted for
April 21's
third image, the skew
Array
produced from
non-perpendicular basis vectors was nice looking but “wrong”. The image
to the left is the corrected version of
Array applied to the same
parameters. The input texture is no longer stretched and all cells are
rigid translations of the original input texture, as seen in the center
cell. These two basis vectors have the same length so each cell is a
rhombus.
This fix involved an excursion into non-orthogonal coordinate systems,
where projecting a point onto an axis cannot use a dot product's
perpendicular projection (as in the previous buggy version). I
implemented the required skew projection using a line intersection
calculation for lines defined in terms of a point and a tangent vector.
Another oblique
Array
using
April 23's input texture (plus a
horizontal blue bar), showing
that the input remains unrotated. In this image the
Array basis vectors have none of
the “special properties”: not aligned with the global axes, not
mutually perpendicular, nor of equal length. As a result the cells
shown here are parallelograms. For randomly selected basis vectors, as
in the context of evolutionary computation, this would be the typical
case.
Row redux
April 23, 2009
As mentioned in the previous entry, the existing Row operator was broken. It only
worked for a horizontal basis vector, and made things easy on itself by
ignoring the y component. For the source image to the left, the old Row operator would produce a series
of copies of it translated to the left and right with horizontal
spacing equal to the x component of the basis vector.
This
is a “near miss” -- not quite the right thing. (Irony alert: for a
library whose goal is as ill-defined as “make pretty pictures” it can
be tricky to tell right from wrong.) In this version the non-horizontal
basis is
handled correctly but the source texture is rotated to align with the
basis. This violates the idea (see previous entry) that the “center
cell” should remain identical to the input (above).
This is the “correct” version, at least according
to my arbitrary definition. The center cell (a strip running roughly
from west-northwest to east-southeast) is identical to the
corresponding region in the source texture (top left image). The other
four visible cells are rigid translations of the center cell.
Note that these images were made with caching turned off. More on that
later.
Arrays
April 21, 2009
The entry for
November 26
shows an array
formed by two nested
Row
operators and speculates about an
Array
operator. I decided to give that a try, using the
Row code as a starting point. Oops,
Row turned out to be a quick
and unfinished prototype: it only handled horizontal basis vectors.
More about that later. (Update: fixed
April 23)
The image to the left is an
Array
whose two basis vectors are perpendicular, equi-length and aligned with
the global axes. The source is a scaled and tinted
RingedSpot.
(Note the prominate alising artifacts due to point sampling: dark
“circles” on the edges and corners of each cell. The
increasingly fine rings of the RingedSpot
pattern are reminescent of the test pattern used by Frank Crow in
his
pioneering work in sampling kernels for “antialiasing”.
See Figure
3 of [Crow 1977])
The
basis vectors of this
Array
are perpendicular and equi-length but rotated, producing a rotated
square grid. If the bases had different lengths the grid would be
rectangular.
This Array
has basis vectors of the same length which are not perpendicular,
producing this skewed or oblique grid with rhombus shaped cells.
This result was visually pleasing enough to seem correct at first, but
eventually cold logic convinced me that it was a bug. That conclusion
is based on an admittedly arbitrary principle that Row and Array should produce a collection
of cells, each of which is a rigid translation of the “center” cell,
which should remain identical to the source texture. More on the rigid
version of oblique Array
later. Still, this operator is too cool not to preserve somehow in the
library. If not as its own operator, this effect could be
obtained by applying the (not yet implemented) Stretch operator to an Array.
Found art: intensity redefined
April 13, 2009
Forgive me gentle reader, today's entry is merely
about
the fun of fiddling with code and accidentally making a beautiful
mistake. The image to the left is the same as yesterday's entry except
that the
intensity
(brightness) of a pixel is defined (rather nonsensically) as the
minimum of its red, green and
blue components. To me this looks a like a 3D shaded rendering of a
lot of small pastel colored blobs. My daughter said it looked like
Floam.
In
this image
intensity is
defined as the
maximum of the
RGB components. This was the first variant I tried while thinking about
yesterday's version and how it might lead to unintended clipping of RGB
values. This image
was surprisingly different from yesterday's with white threads between
blobs
of dark desaturated colors. Sensing I was on to something interesting,
I then tried the
min version above and the distance version below.
In this image intensity is defined as the
distance in 3D RGB space from
black to the given color (divided by √
3, the
distance from black to white). Note that it is very similar to
yesterday's image where intensity was defined as the
average of red,
green and blue values. I think this version is slightly darker than
yesterday's
because it is less prone to create colors outside the unit RGB cube.
The four images from today and yesterday hark back to
Karl Sims'
evolved textures from 1991 where each pixel was generated by an
evolved program whose arguments were the x and y pixel coordinates.
That work used a “microscopic” approach while my library is more
“macroscopic” where evolved programs process whole images.
Remapping contrast
April 12, 2009
This is an experimental
AutomaticExposure operator applied
to fuzzy grain. (
Blur
applied 8 times to
Grain, as
in the second image for
March 23.) I had been
thinking about
operators
to adjust the “contrast” of textures. For example remapping pixel
intensities through an exponential as in
gamma correction.
Another example would be, as in this case, remapping a small range of
intensity values (roughly from 48% to 52%) to the full range dynamic
range of the display, from black to white.
This
AutomaticExposure
operator accomplishes that without any parameters by measuring the
input texture. It remaps the pixel
intensities with the lowest value becoming black and the highest value
becoming white. It gets those values from a new capability of
CachedTexture that randomly
samples a given circular region of the input texture
and estimates the max, min and average pixel values.
Additive and subtractive color mixing
April 8, 2009
My to-do list included “additive and
subtractive color mixing” harking back to features I implemented in
Symbolics S-Paint,
my previous foray into 2D graphics. See this
page
from a
1984
brochure showing images like those to the left created
in S-Paint (
more
here). High end paint systems back then used soft mattes (alpha
blending) to combine opaque colors, but at that time
additive and
subtractive
color
mixing was probably unique to S-Paint. I think my SGD
coworker Eric Weaver microcoded these operations in firmware allowing
them to run at interactive speeds.
After a little thought I realized that additive and subtractive color
mixing could be obtained using operators already in this texture
synthesis library. In fact the images in the entry for
December 26 show the additive mixing of three
gratings tinted in primary
colors. The top image can be thought of as a white wall in a
dark room illuminated by three overlapping spotlights, each tinted with
a colored “
gel” in
red, green and blue. Secondary colors are produced where two primaries
overlap, white is produced where all three overlap in the center.
This image can be thought of as a white field
(like a light box) on which three gels colored cyan, magenta and yellow
have been overlaid. (Or a white sheet of paper tinted with three
overlapping dots of
colored printer ink.) Primary colors are produced where two secondaries
overlap, black is produced where all three overlap in the center. For
example, yellow subtracts (blocks) blue light from the white
background, magenta subtracts green, so the overlap of yellow and
magenta filters produce red. The subtractive effect is obtained by
multiplying together fractional RGB color components which typically
range
between 0 and 1. This is not a physically correct model, but is good
enough to produce this visual effect.
In the first image three primary color spots on black backgrounds are
added together. In the second image the spots are inverted to make
secondary color spots on white backgrounds and these are multiplied
together. It is interesting that the sum of the spots is equal to the
inverse of the product of the inverse spots. If I was more
mathematically inclined I could probably come up with a more specific
word for this than “interesting”.
spot = Translate (Vec2 (0, 0.15), SoftEdgeSpot (0.28, 0.35));
redSpot = Tint (red, spot);
greenSpot = Tint (green, Rotate (4*pi/3, spot));
blueSpot = Tint (blue, Rotate (2*pi/3, spot));
additive = Add (redSpot, Add (greenSpot, blueSpot));
subtractive = Multiply (Invert (redSpot),
Multiply (Invert (greenSpot),
Invert (blueSpot)));
“I remember it like it was
yesterday...”
April 7, 2009
There is a cliché used in TV comedy to set
up a flashback: a character says “
I
remember it like it was yesterday...” followed by a wavy line video
effect and perhaps
a
harp
flourish. In honor of the cheesy video effect, the image to
the left is a
SineShear
operator applied to vertical lines. In this simple version, the texture
is shifted horizontally by a sine wave parameterized by amplitude,
frequency and vertical position.
SineShear
applied to another vertical stripe pattern, this time with an offset
center and an angle. I'm not sure if a translation parameter makes more
sense than a scalar phase parameter. The two seem roughly equivalent.
(One might wonder why this works
for
sine waves but not other waveforms like square, triangle, sawtooth, let
alone arbitrary procedurally defined ones. In other words: “why aren't
waveforms first class types in this
library?” (Update: fixed, see July 4.))
seswg = SoftEdgedSquareWaveGrating (50, 0.25, 0.08, pi/2);
stripes = SoftMatte (seswg,
UniformColor (blue),
SoftMatte (Translate (Vec2 (0.08, 0), seswg),
UniformColor (Pixel (0, 0.8, 0)),
UniformColor (white)));
texture = SineShear (.08, 40, pi/4, Vec2 (0.2, 0), stripes);
Peppermint
April 3, 2009
Today we have two new operators inspired by
spirals, twists and peppermint candy. The image to the left shows the VortexSpot operator applied to a
pattern of alternating red and white radial stripes (shown in the third
image). This is a local operator whose parameters include a center
point and a radius (in this case, the origin and 0.4). Outside that
circle the input texture is unchanged. Inside a rotation accelerates
towards the center. Currently there is a parameter to control the total
amount of rotation, while the exponent controlling the rate of increase
is fixed (at 3). The exponent could be exposed as a parameter but only
a small range of values produce a visually interesting effect, so maybe
not?
This related Twist
operator has infinite extent, applying a rotation proportional to the
distance of each pixel from a given center. An “angular scale”
parameter adjusts the ratio of distance to rotation.
This is the input texture used in the previous
two examples. It is a
Wrap
applied to the new “slightly soft-edged square wave grating” generator
with the oxymoronic name
SoftEdgedSquareWaveGrating.
Compare this to the “radial grad” in the
December
10 entry. In
that, the intensity profile along a circle is a sine wave. The profile
here is generally like a square wave (as in
SquareWaveGrating) but with
“narrow” sinusoids at the transitions between 0 and 1. So if you are
keeping score at home, this would be the fifth “waveform” for gratings
defined in this library. Since the soft transition width is a parameter
SoftEdgedSquareWaveGrating is
actually a continuum of waveforms between the extremes of
SineGrating and
SquareWaveGrating. Maybe these
three waveforms can be folded together in some future revision.
I debugged the operators in the first two images using a sine wave
grating. The soft edged results looked nice but not much like the
peppermint candy I had in mind when I started. I tried using the square
wave grating. That made the edges sharp. In fact it was too sharp,
causing aliasing and moire patterns similar to the center of the image
from
December 26. I tried fixing it with
Blur but as
Rocky
tells
Bullwinkle,
“that trick never works.” The new soft/hard grating acts as filtering
before sampling. It is still not the ideal approach, in this radial
transformation the transition between red and white is too hard near
the center and too soft further out. (A more principled approach would
be stochastic super-sampling but I have been hesitant to take that
plunge.)
Grads and API issues
March 29, 2009
Gradations/gradients of color, or other image
properties, seem a likely component of a library for texture synthesis.
While grads are often
constructed
with a linear transition, these have infinite extent so use a sinusoid
to avoid
Mach bands
at the edges of the transition zone. The image to the left shows a
minimal grad definition:
a transition from black to white, with a single parameter giving the
width of a transition zone centered around the Y axis.
(Note:
the Bar
generators described on January 11 can be seen
as a pair of such
gradations. A sawtooth grating (December 20)
could be seen as a series
of linear gradations.)
This
simple generator brings up some issues of API design that so far have
been
unresolved -- or rather have been resolved inconsistently. Should the
API design goal be to minimize the number of parameters for
simplicity or should all potentially useful parameters be included for
generality? For example, the
SoftEdgeSpot
generator does not currently have a “center” parameter to specify
translation. The assumption was that the
Translate
operator provides that functionality, so why complicate
SoftEdgeSpot
with an extra parameter? Modularity, orthogonality and simplicity all
argue against it.
On the other hand, this library is designed for use by
GP (an
automatic programming technique inspired by evolution) which changes
some design
criteria. For example, unless a
SoftEdgeSpot
happens to appear within the scope of a
Translate operator it will always
be located right at the center of the final texture. Adding a
center parameter to the
SoftEdgeSpot generator effectively
forces GP to pick a location for the spot. I think this will generally
lead to less regularity and more interesting, complex
textures being evolved by GP.
So what would a more elaborate, GP-friendly Gradation generator look like?
Perhaps it would be like the one
described above, with a parameter for transition width, plus parameters
for rotation and translation.
Or perhaps it could fold together translation, rotation AND
colorization by specifying two positions and two colors. Two endpoints
define a line segment indicating the orientation and width of the
transition. Two colors correspond to the value of the gradient at each
endpoint. In the first image above, the two points would be on the X
axis and equidistant from the Y axis, the colors would be black and
white.
These three candidate parameterizations of the Gradation generator are illustrated
in the code box below. Each produce the same tilted, offset, yellow-red
grad shown to the left. The key point to notice is that the
“minimalist” version 1 turns out to require the largest amount of code
while the more “elaborate” version 3 is the most compact expression.
Version 1 is not just larger, but it seems more “genetically fragile”
since the three helper operators must all appear together lest the grad
be centered or vertical or monochrome.
For now Gradation is defined
as in version 3. I think I am leaning toward adding a “center”
parameter to SoftEdgeSpot but am conflicted about whether to add two
color parameters. Possibly I could do both with multiple constructors
for the generator classes.
SoftMatte (Translate (center, // version 1
Rotate (angle,
Gradation_1 (width))),
yellow,
red);
SoftMatte (Gradation_2 (center, angle, width), // version 2
yellow,
red);
Gradation_3 (v1, yellow, v2, red); // version 3
Composition book cover
March 28, 2009
Riffing on the
HueOnly
version of blurry grain, I tried applying
HardThreshold to the same input and
got a nice texture that reminded me of an old-fashioned
composition book
cover.
The
December 13 poppy seed texture used a
similar idea. Annoyingly I
noticed that
HardThreshold
produced a texture that appeared to have some gray values in it.
That was good
news and bad news. Soft “filtered” edges are generally a good thing but
that
operator should
have produced only black or white pixels. That lead to the OpenGL
texture
filtering fix discussed in the previous entry. Once I got a bug-free
version of
hard-edged thresholding working, I wrote a new soft-edged thresholding
operator that takes two floating point values to delimit the start and
end of the soft transition zone.
SoftThreshold (0.497,
0.503,
Blur (Blur (Blur (Blur (Blur (Blur (Blur (Blur (Grain ())))))))));
Avoiding MIP-mappery
March 28, 2009
I fixed a bug in my display code which had caused
occasional localized blurring. This happened when OpenGL tried to
“help” me by filtering my textures for use in 3D. I am still using
OpenGL,
certainly overkill for this application, but have convinced it not to
do any filtering. This texture is a hard-edged resolution test chart.
Along the center row pixels alternate between black and white. The
wavelength of this square-wave doubles (decreases one octave) in each
adjacent row.
Extracting hue
March 23, 2009
This is a blurry grain texture (see second image)
from which the pure hue component has been extracted with the new HueOnly operator. It takes a pixel
value
in RGB space, converts it to HSV, then throws away the original
saturation
and value, replacing each with the max value. So the result is composed
of fully saturated, fully bright hues of the original image.
This is a Grain
texture to which Blur has
been applied 8 times. That is similar to a low pass filter
(Gaussian-ish) whose kernel is 17 pixels in diameter. The first Blur is 3x3, subsequent
convolutions extend the support radius by 1 pixel (17 = 3+2×7).
Ouch!
March 22, 2009
Yikes! The
March 19
band-aid fix to the
cache-miss problem caused an
exponential explosion doing a series of nested Blur operators.
Apparently the 3x3 LPF kernel's support fell outside the cache in a one
pixel
wide border around it. The iterated Blur operators caused an
exponential growth of uncached pixels in a region 8 pixels wide
all
being recomputed many times. I think each
such miss would cause 8 more accesses to the input texture, so perhaps 8
7
(2 million!) additional getPixel calls?
I changed the cache to be twice as big in both directions or 4 times as
many pixels cached. It had previously extended from -1/2 to +1/2 in
both directions, now it extends from -1 to +1. With that change all is
well for now. The painful 8X nested Blur case now runs 90
times faster. It also correctly handles the failure
from
March 18. But the texture caching remains
non-adaptive so this
problem still needs a more robust solution
Another color space
March 20,2009
This image is just to test new routines to
transform color values between the RGB and
HSV color spaces. RGB (red, green, blue) is the color space used by
most display hardware and HSV (hue, saturation, value) is a “computer
graphics friendly” version of IHS (intensity, hue, saturation) space
that corresponds more closely to human color perception. The
HSV and HSL
spaces were first described in the literature by
Alvy Ray Smith in 1978.
In this image, hue varies from left to right, value (intensity)
increases from bottom to top and saturation increases radially from the
center.
Ring
March 19, 2009
I added a
Ring
operator similar to both
Row
and
Wrap. Like
Row it replicates a rigid portion
of the input multiple times into the output. Like
Wrap these copies are pie-shaped
sectors rotated around a given point. Compare this image made with
Ring to the
February
6 image of
Wrap applied
to the same texture
(purpleRSpot).
Historical note: my
ASAS
procedural animation and modeling system
included analogous
ring and
row operators back in 1978.
This fractal pattern was made with Ring and a SoftEdgeSpot. There are three
levels of recursion, each ring has seven copies, for a total of 343
spots. This image demonstrates a band-aid fix for the “cache miss” bug.
The CachedTexture object simply passes through the original
texture if outside the cached region.
r1 = Ring (7, Translate (Vec2 (0, 0.15), SoftEdgeSpot (0.04, 0.1)));
r2 = Ring (7, Translate (Vec2 (0, 0.5), r1));
r3 = Ring (7, Translate (Vec2 (0, 1.66), r2));
texture = Scale (0.2, r3);
Pay no attention to that man behind the curtain
March 18, 2009
While testing the new
Ring operator, I discovered a bug
in the caching mechanism. Here is the “standard benchmark” (see entry
for
November 26) scaled down by half. Oops!
The current cache implementation assumes a finite region of interest.
The size is not adaptive so sometimes its too big and sometimes too
small. Worse, “cache misses” are handled ineptly, as shown in
this image. Finally the spatial resolution of the cache is fixed which
can lead to loss of image quality.
Recently I added input caching to the
Wrap
operator. I noticed that for a simple case (
February
6, bottom image)
the caching code was actually 15% slower which begged the question: why
use it? However metering
Ring
with a somewhat more complicated texture
(test4dots) showed a 60%
speedup. Presumably the speed-up would be significant for the sort of
complex textures that this library is intended to provide. Caching
remains a requirement so a new implementation is
required.
Anti-fisheye
March 15, 2009
Finally fixed the problem seen in
the
January 4 entry: the “local non-linear radial
scale” operator correctly
handled expansion but not contraction. The new version of the operator
uses a look-up table to
invert the scale-
as-a-function-of-radius
curve. I renamed the operator
from RadialMagnification
to StretchSpot. as well as
adding a “center” parameter and removing the “inner radius” parameter.
(The first draft had parameters like SoftEdgedSpot.)
Compare this image to the “local fisheye expansion” in the January 1
image.
Here the same background pattern is first
contracted
about a point near the top and then
expanded about a point near the bottom. The two circular regions of
stretch just touch at the center.
StretchSpot (10, // center scale
0.4, // radius
Vec2 (0, -0.4), // position
StretchSpot (0.1, // center scale
0.4, // radius
Vec2 (0, 0.4), // position
bg3way));
Better wrap with angular scale
February 6, 2009
I tried adding an angular scale parameter to the
previous version. It maps from horizontal coordinate in the source
image to angular coordinate in the polar wrapped output. The results
are
more
what I had in mind but I worry about adding too many parameters that
might be troublesome to set. Here is the same five-fold Wrap
applied to a SoftEdgeSpot and
a tinted RingedSpot.
purpleRSpot = Tint (Pixel (1.5, 0.4, 1.2),
Translate (Vec2 (0, 0.25),
RingedSpot ()));
texture = Wrap (5, 0.14, Vec2 (), purpleRSpot);
Proto wrap
January 13, 2009
A first pass at a Wrap operator that takes a
rectangular region of its input and wraps it in pie-shaped sectors
around a given point.
Bars
January 11, 2008
This shows three bar-like textures tinted with
primary colors and added together. These bars are extruded versions of
spots described below. The two
vertical elements are made with the SoftEdgeBar
generator which has
parameters like SoftEdgeSpot
plus position and angle. The blue one has a wide stripe of full
brightness with soft edges. The green one (seen here as cyan) has a
cosine brightness profile. The horizontal red texture was
generated with RingingBar
an extruded version of RingedSpot.
Local fisheye contraction (not)
January 4, 2009
The local expansion worked well and so it was
disappointing that inverting that one parameter did not produce the
inverse fisheye I was expecting. After a little investigation I
realized I needed the inverse function of the radius remapping used in
the previous example. That has so far eluded my limited algebraic
skills. If nothing else I will have to use a look-up table, or a
spline, to approximate the inverse function.
Local fisheye expansion
January 1, 2009
This RadialMagnification[?]
operator scales the input texture at a given point, with the effect
smoothly tapering off to a given radius. Outside that radius the
texture is unchanged.
BTW, I see a fairly strong illusory motion in this
pattern
where the center of the image seems to wobble relative to its
background
in response to my head
movement.
Cool, but no cigar
December 26, 2008
This was an early attempt at a localized
contraction warping of a texture. It was not what I was looking for but
I thought it was cool nonetheless. The input texture was a linear
grating of wide yellow and narrower blue stripes. Note also the
aliasing errors near the center. So far all sampling is based on single
point samples. Using multiple jittered samples per texel would help in
this case, using them all the time might be overkill. A simple criteria
for deciding when to supersample would be helpful.
Three grating shapes and a naming conundrum
December 26, 2008
This
texture is the sum of three grating-like generators each tinted a
primary color. I am unsure what to call them. The
red horizontal stripes are a sinusoid
linear
grating. The green target-like rings are an
annular grating where the sinusoid
is applied to the radius of a pixel (its distance from the center), a
slight variation on the
RingedSpot
mentioned below. The blue pie-shaped sectors result from applying the
sinusoid to the angle formed by a pixel, the center and a reference
direction. Is that an
angular grating,
or do the ray-like features suggest it should be called a
radial grating?
If there are 3 shapes and 4 waveforms for gratings, should there be 12
distinct generators, or one generator with two selector/enumerator
arguments? (Update: see
July 1 entry about
this issue.)
Stripes, gratings and four waveforms
December 20, 2008
Originally wanting a generator for stripes, I
generalized to
gratings
with four waveforms. In this image you might be able
to make out four distinct gratings added together: a horizontal red
sine
wave, a vertical green triangle wave, and on the diagonals, a yellow
sawtooth
and a green square wave. The triangle and square waves have asymmetric
“duty cycle”. Currently there are four separate generators for the four
waveforms (
SineGrating,
SawtoothGrating,
TriangleWaveGrating and
SquareWaveGrating) but this
may change. (Update: or maybe not, see
July 1
entry about this issue.)
Poppy seeds
December 13, 2008
This is the only image here intended to be
representational. It is meant to look like poppy seeds scattered on a
white plate. I was thinking about the types of images that could be
generated with this library, perhaps more worrying about which kinds
could not. While cleaning up after breakfast I found myself looking at
the real version of this and wondering if it could be portrayed using
this library.
This attempt is moderately successful. I used blurring and thresholding
to get
the effect of the
morphological
image operator called
dilation.
Perhaps some of
the basic morphological operators should be added to this library?
grain = Grain ();
hardGrain = HardThreshold (0.96f, grain);
blur1 = Blur (hardGrain);
blur2 = Blur (blur1);
blur3 = Blur (blur2);
blur4 = Blur (blur3);
hardDots = HardThreshold (0.01f, blur4);
invert = Invert (hardDots);
softDots = Blur (invert);
texture = softDots;
Radial (or is it angular?) grad (or is it
grating?)
December 10, 2008
Thinking about other basic geometries I want the
library to support, some sort of “radial” or “spoke-like” pattern
seemed important. For each pixel in this image an angle is determined
from the center. The angle is passed through a sinusoid curve to
determine brightness. Since the brightness varies with angle, that
suggests
the name “angular”. Along any ray/radius the brightness is constant.
The bright “rays” suggest the term “radial.” I called it RadialGrad for now.
Product of ringed spots and “HDR”
December 5, 2008
Always on the lookout for complexity on the
cheap, I tried to make something like an
interference pattern.
I multiplied two shifted copies of yesterday's
RingedSpot. Indeed the
resulting shapes were nice but the image was very dark. This image is
the result of scaling up the brightness of the product of spots. That
description sounds simple enough but required several changes in the
API to allow color values outside the unit RGB cube, and to clip rather
than wrap around at the final 8 bit conversions. Color values are now
allowed to get large and are clipped just before display. Super-bright
colors are harmless as intermediate values but will be lost if they
persist to the output. The four largest blobs in this image have been
clipped to white and so lack detail at their centers.
Ringed spot
December 4, 2008
Thinking about generators like
SoftEdgeSpot, a round pattern based
on a
sinc function
came to mind. I liked this variation I called
RingedSpot where both the amplitude
and the wavelength of the rings decrease with distance. This
implementation seems a bit
ad hoc
since there are two hidden parameters related to frequency and
falloff. Maybe they should be exposed? And of course beyond
SoftEdgeSpot and
RingedSpot there are plenty of
other interesting functions that could define the profile of a radially
symmetric texture.
Edge finder
November 29, 2008
In the original to-do list I had both
Blur
and some kind of “sharpen” or “edge enhance” operator. This
result, based on a 3x3 convolution kernel, looks more like an “edge
detector.” I'd like an operator that will emphasize edges in the input
image but otherwise leave it largely unchanged.
(Update: see
July 20 and
August
24.)
Grain and desaturation
November 28, 2008
The center of this texture shows the Grain generator where each pixel
has a random color chosen from a uniform distribution in the unit RGB
color cube. Caching is required for Grain
to return consistent results
for subsequent getPixel calls
with identical arguments.
The outside of this texture shows the Monochrome operator applied
to the same Grain texture. In
between is a soft transition provided by
a SoftEdgeSpot.
At least to my eyes, the contrast of the result seems similar at the
center and the periphery, but seems oddly lower in a ring in the
transition
zone, roughly 80% of
the outer diameter. I have no idea what produces that effect.
(Update: I noticed much later that I apparently wrote the Blur operator the same day I wrote Grain and Monochrome but neglected to mention
it here. The source code for this test includes a commented-out verison
which used a blurred Grain.)
grain = Grain ();
mono = Monochrome (grain);
spot = SoftEdgeSpot (0.15, 0.5);
texture = SoftMatte (spot, mono, grain);
Rows and arrays
November 26, 2008
With the previous texture as a starting
point, a
Row operator
produced a row of copies. Applying
Row
a second
time produced this array. Perhaps there should be a built in “array”
operator? (Update: see
April 21, 2009) This
texture also allowed
experiments with an input caching
strategy
in operators to prevent recomputing input pixel values when they are
referenced multiple times while creating the output texture.
(The introduction of caching lead to some c++ issues. Using
nested
expressions of operators and generators (which are in fact class
constructors) lead to short lifetimes of those “temporary”
instances. For example in texture=Row(a,Row(a2,...))
the compiler
would construct and destroy the inner Row
during that one assignment, leaving it dead when the texture was later
evaluated.)
Rotate (pi/8,
Row (Vec2 (0.15, 0),
Rotate (pi/2,
Row (Vec2 (0.15, 0),
Add (Translate (Vec2 (0.015, 0.015),
Scale (0.15,
Tint (green,
SoftEdgeSpot (0.2, 0.5)))),
Translate (Vec2 (-0.015, -0.015),
Scale (0.15,
Tint (magenta,
SoftEdgeSpot (0.2, 0.5)))))))));
Composed operators
November 20, 2008
This screen shot shows a simple composition of
texture operators. Two soft edged spots are colored and added together.
(Update: the coloring was done with SoftMatte
rather than Tint which hadn't
been written yet, the addition used a clipping addition operator before
the
switch to “high dynamic range” representation.)
AddClip01 (Translate (Vec2 (0.1, 0.1),
Scale (0.9,
SoftMatte (SoftEdgeSpot (0.2, 0.5),
UniformColor (black),
UniformColor (green)))),
Translate (Vec2 (-0.1, -0.1),
Scale (0.9,
SoftMatte (SoftEdgeSpot (0.2, 0.5),
UniformColor (black),
UniformColor (magenta)))));
First light
November 17, 2008
In early November I started learning how to write
a simple OSX application using the Cocoa framework. I got one running
and displaying texture
mapped OpenGL triangles. This is the
first display of a texture generated via the
procedural texture
synthesis library described in this document. In this case the texture
was generated by SoftEdgeSpot.
The texture shown
applied to both a square and
a circle. Since I plan to use textured disks in my experiments I
stopped using the square format. I used a 50% gray background as
the least likely to bias perception of the synthesized textures.
Preliminary design
September 20, 2008
This commentary written later, in retrospect:
By early 2008 I had a vague conceptual
design of a library for
procedural texture synthesis, suitable for use with a genetic
programming (GP) system for automatic program discovery. This
combination is intended to allow for goal-oriented texture synthesis.
The basic idea
was to structure the texture synthesis library as a
collection of composable units. It was essentially a data flow
model, or seen another way, a collection of functions whose output were
color
texture objects. These textures represented a mapping from a continuous
2d plane (ℝ2) to
color values.
The inputs for these texture producing functions would be parameters
like real (floating
point) numbers, 2D position vectors, 3D RGB colors and optionally,
other
textures. Functions with
textures as inputs would be called operators
and those with no texture inputs would be called generators. I went back and forth
about whether the textures should have fixed extent (like a texture
map, an array of discrete pixel values) or whether they should have
infinite
extent (returns a value for any
pixel coordinates).
Some potential generators, like reaction diffusion textures, would be
expensive to compute so it seemed like caching of texture would be
needed. Some operators, for example a low pass filter, would need to
access multiple input pixels for each output pixel, so again caching
seemed an important feature. I have an email note from April 2008
discussing some ideas about
caching strategy. Previously I had assumed the texture functions would
cache their outputs (as in memoization) but this note suggested putting
the cache on the inputs of operators that needed them.
On September 20, I drafted an initial specification for the texture
synthesis library:
-
texture operators:
- image ops (on 2d images of RGB
pixels):
- add (with rescale or clip on
pixel values?)
- subtract
- multiply (divide? mult two
textures
pixel by pixel, scale one image by scaler)
- blend/matte
- 3 arg version: foreground, background, matte (take brightness as alpha value)
- 2 arg self matting version
- pure black is transparent (hard
matte)
- some way to have soft matte?
- convolve (two args, a source
image and a kernel image, how to prefer small kernels?)
- blur (default to 3x3?)
- sharpen/edge-enhance/edge-detect
- 2d transformations on
image plane
- image warps
- stretch (scale along a given
axis leaving the perpendicular direction unchanged)
- expand/contract around point
- groups
- grid
-
primitive textures (generators):
- single color
- gradation
- spot (Gaussian by default?
“soft
brush” parameters)
- bar (like a 1d Gaussian
extruded
along perpendicular axis)
- speckle/grain (what parameters?)
- noise (use Ken Perlin's ImprovedNoise?)
Note that this list failed to mention
reaction
diffusion textures which I had initially imagined as the primary
source of primitive texture in this project. Indeed it was to model
natural textures that Turing first proposed such systems in his
landmark 1953 paper
The
Chemical Basis of Morphogenesis [PDF].
