CREATING IMAGE FILTERS IN HLSL (FOR VC# PROJECTS)

PAUL HARTZER

C ONTENTS
Overview 1
Background 2
HLSL 2
Color representations 3
Basic software 5
A simple shader 5
A more complex shader 7
Shaders that relocate pixels 10
Using HLSL filters in VC# 11
Conclusion 13
End notes 13

O VERVIEW
HLSL is a C-related language designed to allow for manipulation of images in the
DirectX model. One of its functions is to create filters, from simple color tinting to
more advanced image manipulations.

Fig. 1: Original image Fig. 2: Inverted

HLSL Image Filters 1 Rev. 9/11/10 – Paul Hartzer

Fig. 3: Desaturated Fig. 4: Distorted
In this article, I discuss the basics of using HLSL for this purpose. First, I provide some
background on HLSL, color, and the suggested software.

Next, I’ll go through three simple examples taken from the sample shaders provided
by Shazzam. These examples are intended to give you an idea of some of the basics of
HLSL as applied to pixel shading.

I’ll wrap up by briefly detailing how I implemented a shader within VC# 2010. The
biggest part of the work, though, is creating the shader within Shazzam; once that’s
been done, applying it within VC# is fairly easy.

B ACKGROUND
HLSL
HLSL stands for High Level Shader Language. In general, a shader language provides a
means for describing how the computer (specifically, the GPU, or Graphics Processing
Unit) is to manipulate basic input images for various effects. This consists of three
basic components:

Vertex shading: A computer-generated image (such as those seen in movies like

the Toy Story series) consists of objects which mimic real world analogues; each
object consists of a collection of geometric shapes described by vectors with
vertices. A vertex shader gives various instructions on how to make a scene look
more solid or realistic, such as how lights behave, the effect of fog, and so on. Also,
as Wikipedia1 describes, vertex shaders “transform each vertex’s 3D position in
virtual space to the 2D coordinate at which it appears on the screen.”

HLSL Image Filters 2 Rev. 9/11/10 – Paul Hartzer

 Geometry shading: Geometry shading can be used to smooth out or add
complexity to objects. Once an object has been shaded, it is then rasterized.
Rasterization consists of converting a collection of vertices into a set of pixels. The
advantage of creating an image as a set of vertices is that resizing creates
consistently smooth images; once an image has been rasterized, the computer
perceives of it as a flat collection of squares (pixels) rather than as a flexible
collection of objects. Most basic computer images, such as all digital photographs,
begin as pixel-based and not as vector-based images.

Pixel shading: A flat, pixel-based image can be further manipulated by changing

colors, moving pixels around, and so on. Photo-manipulation software such as
PhotoShop calls this filtering. Pixel shaders apply the same formula to each pixel in
an image, although how a formula is applied may depend on the pixel’s
coordinates. For the purposes of calculations, the image’s coordinates are rescaled
to values of 0-1 in each direction, so that the same shader when applied to 500x500
and 1000x1000 images will be twice as large for the latter, while shaders applied to
non-square images will be non-square themselves.

This article is restricted to pixel shading.

C OLOR REPRESENTATIONS
The color of a specific pixel in an image file is described by four parameters. Three of
these relate to color (red, green, and blue), and the fourth to the alpha channel. These
values are typically referred to by their initials (r, g, b, and a, respectively).

Fig. 1: Original image (image.rgba) Fig. 5: Red channel (image.r)

HLSL Image Filters 3 Rev. 9/11/10 – Paul Hartzer

Fig. 6: Green channel (image.g) Fig. 7: Blue channel (image.b)

In HLSL representations, RGB values vary from 0 (no color) to 1 (full color). For
instance, if r = 1, g = 0, and b = 0, the pixel is bright red. If r = 0.75, g = 0.5, and b =
0.25, the pixel is orange. An HLSL value of 1.0 corresponds to a hexadecimal value of
#FF.

The alpha channel represents the degree of transparency; as Webopedia2 describes, “it
specifies how the pixel’s colors should be merged with another pixel when the two are
overlaid, one on top of the other.” Images that are completely opaque (that is, nothing
shows through when placed over other objects) have an alpha value of 1. Invisible
images have an alpha value of 0. Figure 9 illustrates applying variable levels of
transparency to the original image (alpha = min{red * green * blue * 5, 1}), and then
superimposing it onto a second image:

Fig. 8: Second image Fig. 9: Variable alpha channel

HLSL Image Filters 4 Rev. 9/11/10 – Paul Hartzer

 B ASIC SOFTWARE
HLSL is not written directly in VC#. Instead, it is written and compiled separately, and
the compiled file is added as a resource into VC#. All of the software involved is free:

DirectX SDK: The most recent DirectX SDK can be obtained from Microsoft3. This
is required for HLSL.

Shazzam: While not required, Shazzam4 greatly simplifies the process of learning,
developing, and compiling pixel shaders in HLSL. It also comes with a large
collection of sample shaders, which can then be easily further modified as desired.

Visual C# 2010: The Express5 version is completely free.

A SIMPLE SHADER
Because the pixel shader applies the same formula to each pixel in an image, there is
no looping required. Here is the code for InvertColor, which creates the “negative”
image in figure 3 (line numbers are provided for reference only and should not be
included in the actual code; the variable names have been changed from Shazzam’s
code for expository purposes):

01 sampler2D InputImage : register(S0);

02 float4 main(float2 xy : TEXCOORD) : COLOR
03 {
04 float4 InputColor = tex2D( InputImage, xy );
05 float4 OutputColor = float4(InputColor.a - InputColor.rgb,
InputColor.a);
06 return OutputColor;
07 }

Code snippet 1: Inverted image (based on Shazzam sample shader)

Line 01 labels the input image, referenced in line 04. InputImage is a variable name,
and can be replaced with any suitable name. sampler2D simply indicates that the
program will be operating in a two-dimensional image file. Registers will be discussed
below.

Line 02 indicates that the input will consist of a two-dimensional set of pixel
coordinates, to be referenced by the variable named xy. The output will be a four-
value array representing a color.

In HLSL pixel shaders, these are the most common variable types:

HLSL Image Filters 5 Rev. 9/11/10 – Paul Hartzer

 float: A single value

float2: A two-dimensional array, used for xy coordinates

float3: A three-dimensional array, used for rgb values (hence, excluding the alpha
channel)

float4: A four-dimensional array, used for rgba values

Line 04 takes the color information of a specific pixel in the image and places it into a
variable called InputColor.

Line 05 calculates the new value of the pixel. HLSL allows a special shorthand that can
be confusing. float4() takes four arguments, but these can be grouped together. In
this case, InputColor.a - InputColor.rgb is equivalent to InputColor.a -
InputColor.r, InputColor.a - InputColor.g, InputColor.a - InputColor.b. In
fact, all of these lines are equivalent:

float4 OutputColor = float4(InputColor.a - InputColor.r, InputColor.a -

InputColor.g, InputColor.a - InputColor.b, InputColor.a);
float4 OutputColor = float4(InputColor.a - InputColor.rg, InputColor.a
- InputColor.b, InputColor.a);
float4 OutputColor = float4(InputColor.a - InputColor.r, InputColor.a -
InputColor.gb, InputColor.a);
float4 OutputColor = float4(InputColor.a - InputColor.rgb,
InputColor.a);

Note that order is important, so this line, while valid, is not equivalent to the ones
above:

float4 OutputColor = float4(InputColor.a - InputColor.rb, InputColor.a

- InputColor.g, InputColor.a);

Since the default value for the alpha channel is 1.0, line 05 will usually be equivalent to
this:

float4 OutputColor = float4(1 - InputColor.rgb, InputColor.a);

In other words, for the red, green, and blue channels, take the inverse of the value.
Using InputColor.a instead of 1 in line 05 takes any transparency into account.

If the formula is adjusted, the output image changes. For instance, you could take just
the inverse of the red channel while swapping the blue and green, or you could apply
different formulas to each channel (pow() is the HLSL function for powers;
pow(x,0.5) takes the square root of x):

HLSL Image Filters 6 Rev. 9/11/10 – Paul Hartzer

 float4 OutputColor = float4(InputColor.a - InputColor.r, InputColor.b,
InputColor.g, InputColor.a);
float4 OutputColor = float4(sin(InputColor.r), pow(InputColor.g,0.5),
2*tan(InputColor.b), InputColor.a);

These generate the following images, respectively:

Fig. 10: Red inverted, blue/green swapped Fig. 11: Various simple tweaks

Finally, line 06 returns the value. The procedure is then repeated on each pixel in the
image.

As you can see, you can create some fairly interesting filters by using code snippet 1 as
a template and simply changing line 05; Microsoft maintains a complete list of
intrinsic HLSL functions6. More complex filters, such as the distortion in figure 4
above, require more programming.

A MORE COMPLEX SHADER

The ColorTone shader creates a mask, as if the image were lit by a colored gel, and
allows for various user adjustments (such as the base color of the mask). It uses
registers to allow the user to communicate these settings:

01 float Desaturation : register(C0);

02 float Toned : register(C1);
03 float4 LightColor : register(C2);
04 float4 DarkColor : register(C3);
05 sampler2D implicitInputSampler : register(S0);
06 float4 main(float2 uv : TEXCOORD) : COLOR
07 {
08 float4 color = tex2D(implicitInputSampler, uv);
09 float3 scnColor = LightColor.rgb * (color.rgb / color.a);
10 float gray = dot(float3(0.3, 0.59, 0.11), scnColor);

HLSL Image Filters 7 Rev. 9/11/10 – Paul Hartzer

 11 float3 muted = lerp(scnColor, gray.xxx, Desaturation);
12 float3 middle = lerp(DarkColor.rgb, LightColor.rgb, gray);
13 scnColor = lerp(muted, middle, Toned);
14 return float4(scnColor * color.a, color.a);
15 }

Code snippet 2: Color Tone image (based on Shazzam sample shader)

Lines 01-04 represent user-set variables. To see how each of these work, select the
ColorTone sample shader in Shazzam, and then select the Change Shader Settings tab.
Each piece of user input requires a register, including the input image itself.

Obviously, since these are user settings, any of them could be replaced with a fixed
value in the code itself. For instance, if you want to have a fixed desaturation of 0.5,
delete line 01 and insert this before line 08:

float Desaturation = 0.5;

Additionally, you could create other user settings, set as allowing for variable levels of
gray (line 10) rather than using the standard values.

Lines 06 and 08 are the same as in code snippet 1.

Lines 09 to 13 represent the transformation applied to the image. This uses two
common functions, dot() and lerp(). dot() calculates the dot product of two vector
arrays; this is a technical way of saying that each value in the first array is multiplied
by the corresponding value in the second array, with the results being added together.
Thus, line 10 is a shorthand version of this:

float gray = 0.3*scnColor.r + 0.59*scnColor.g + 0.11*scnColor.b;

The values (0.3, 0.59, 0.11) are standard values for making a color image gray, meant to
account for the varying brightnesses of red, green, and blue. These are not the only
standard values; Wikipedia7, for instance, uses (0.2126, 0.7152, 0.0722). Feel free to
adjust as desired.

The function lerp() returns the linear interpolation of two points at a specified
distance. That is to say, the function draws a line between the two points, and then
calculates a new point somewhere on that line, the specified distance (as a
percentage) away from the first point. The formula is x + s(y-x), where x and y
represent the endpoints and s is the distance. For instance, if you merely wanted to
make an image look washed out, you could use this code:

sampler2D implicitInputSampler : register(S0);

HLSL Image Filters 8 Rev. 9/11/10 – Paul Hartzer

 float4 main(float2 uv : TEXCOORD) : COLOR
{
float4 color = tex2D(implicitInputSampler, uv);
float gray = dot(float3(0.3, 0.59, 0.11), color.rgb);
float Desaturation = 0.75;
float3 muted = lerp(color.rgb, gray.xxx, Desaturation);
return float4(muted, color.a);
}

which results in the image in figure 12. The Color Tone effect creates a more complex
desaturation and colorization, seen in figure 3 (repeated here for comparison):

Fig. 12: Washed out Fig. 3: Desaturated

As with dot(), lerp() can be replaced with the underlying formula with the same
result, as with line 11:

float3 muted = scnColor + Desaturation*(gray.xxx - scnColor);

If you choose, you can even expand it fully, still with the same result:

float3 muted;
muted.r = scnColor.r + Desaturation*(gray.x - scnColor.r);
muted.g = scnColor.g + Desaturation*(gray.x - scnColor.g);
muted.b = scnColor.b + Desaturation*(gray.x - scnColor.b);

Overall, then, ColorTone creates the following:

scnColor: a copy of the image tinted strongly towards the lighter of the two
specified colors. For instance, if LightColor is yellow, then whites and lighter
pixels in the original image will become yellow, but darker colors will be less
affected. To see the effect, comment out line 13 and reapply it using F5.

gray: a grayscale multiplier.

HLSL Image Filters 9 Rev. 9/11/10 – Paul Hartzer

 muted: a copy of scnColor desaturated towards gray to the degree specified by
Desaturation.

middle: a monochrome image in the color range between DarkColor and

LightColor, as determined by the gray value for the pixel.

Finally, an adjusted scnColor based on mixing muted and middle.

Note that the function uses predominantly float3 rather than float4 values, because
this effect does not affect the degree of transparency (i.e., alpha channel).

S HADERS THAT RELOCATE PIXELS

So far, the code examples have shown how to change the color values in an image.
While different pixels have been affected by differing amounts, the location of input
and output pixels have been the same. As illustrated in figure 4 above, it is also
possible to move certain pixels around to create effects such as magnification. The
shader illustrated in figure 3 is Pinch.

01 float2 Center : register(C0);

02 float Radius : register(C1);
03 float Strength : register(C2);
04 float AspectRatio : register(C3);
05 sampler2D implicitInputSampler : register(S0);
06 float4 main(float2 uv : TEXCOORD) : COLOR
07 {
08 float2 dir = Center - uv;
09 float2 scaledDir = dir;
10 scaledDir.y /= AspectRatio;
11 float dist = length(scaledDir);
12 float range = saturate(1 - (dist / (abs(-sin(Radius * 8) * Radius)
+ 0.00000001F)));
13 float2 samplePoint = uv + dir * range * Strength;
14 return tex2D(implicitInputSampler, samplePoint);
15 }

Code snippet 3: Pinch image (based on Shazzam sample shader)

Lines 08-10 determine the location of the pixel in relation to the center point specified
in the user setting. The y coordinate is adjusted if something other than 1 is specified
for AspectRatio.

Line 11 calculates the actual distance between the current pixel and the center point.

HLSL Image Filters 10 Rev. 9/11/10 – Paul Hartzer

 Line 12 calculates whether the current pixel is in range of the magnification effect, and
if so, the degree of the effect. saturate() limits an input value to the range (0,1), so if
dist is too great, range will be 0 and hence no adjustment will be made in line 13.

The important difference between this shader and the previous examples is in the last
line of the function, line 14. Instead of returning the adjusted color of each pixel in the
image, the function returns information for the appropriate pixel in the original image
based on the adjustments. The function tex2d() returns the pixel of the specified
image at the specified co-ordinate, which may or may not be the same as the co-
ordinates currently under consideration. This function appears in the first line of the
previous two examples because no pixels are moved in those cases.

Here is the shader with different values specified for the user-specified variables. Note
that in figure 13, an aspect ratio of 1 creates an oval rather than a circle; this is because,
as noted earlier, the image is remapped virtually onto (0,1) scales in each direction, so
effects with an aspect ratio of 1.0 will create ovals, not circles, unless the base image is
square.

Fig. 4: Distorted Fig. 13: Distorted, version 2

U SING HLSL FILTERS IN VC#

Shazzam provides auto-generated code for use in VC# and VB projects. Once you’ve
settled on a filter that you like, click on the Generated Shader - C# tab. This is the
code you will copy into your project.

These are the steps I followed to add a red-orange filter to selected cards in my
Canfield8 program:

HLSL Image Filters 11 Rev. 9/11/10 – Paul Hartzer

 a. I created a new class file called ColorTone.cs, which contains Shazzam’s auto-
generated code. Because of where I stored the resource file, I changed this line:

pixelShader.UriSource = new
Uri("/Shazzam.Shaders;component/ColorTone.ps", UriKind.Relative);

to this:

pixelShader.UriSource = new Uri("Resources/ColorTone.ps",

UriKind.Relative);

b. Using Tools>Explore Compiled Shaders within Shazzam, I located the

ColorTone.ps file, and copied it to the Resources subdirectory within my VC#
project.

c. Using Project>Properties...>Resources within VC# 2010, I added

ColorTone.ps as a resource.

d. Within the Solution Explorer, I set the Build Action for ColorTone.ps to
Resource.

The above steps added the appropriate file to the project’s resources and provided the
basic code for applying the filter. Additionally, certain code was added to the main
project code file.

To the using list, I added:

using Shazzam.Shaders;

To the beginning of the function that redraws the screen after a card selection, I
added:

ColorToneEffect pfxCardUp = new ColorToneEffect();

pfxCardUp.LightColor =
(Color)ColorConverter.ConvertFromString("#FFCA7E7E");
pfxCardUp.DarkColor =
(Color)ColorConverter.ConvertFromString("#FF837474");

Note that the extra color conversion is needed within the WPF framework in which
the program was written, because WPF treats graphics differently than traditional
VC# projects. Recall that the Color Tone effect takes four variables; since only two are
defined, the other two will take their default values.

Finally, I directed the program to apply the filter to the appropriate image, based on
which card has been selected. For instance, this code applies the filter to the top of the
waste pile if that card is selected; otherwise, all effects are turned off.

HLSL Image Filters 12 Rev. 9/11/10 – Paul Hartzer

 imgWaste[intCardtoBlur].Effect = (strCardInHandSource == "Waste" ?
pfxCardUp : null);

Once you’ve used Shazzam to do the heavy lifting, the actual implementation of a
pixel shader within VC# 2010 is fairly straightforward.

C ONCLUSION
This article is intended as a basic overview on using HLSL to create pixel shader
effects, also known as filters, and to implement these effects within VC#. HLSL goes
far beyond the examples discussed above, but hopefully this basic tutorial has
provided enough foundation for you to now explore the other sample effects provided
by Shazzam, as well as generating your own filters.

E ND NOTES
URIs to links provided in text (as of September 10, 2010):

1
http://en.wikipedia.org/wiki/Shader
2
http://www.webopedia.com/TERM/A/alpha_channel.html
3
http://msdn.microsoft.com/en-us/directx/aa937788.aspx
4
http://shazzam-tool.com/
5
http://www.microsoft.com/express/Downloads/
6
http://msdn.microsoft.com/en-us/library/ff471376(v=VS.85).aspx
7
http://en.wikipedia.org/wiki/Luminance_(relative)
8
http://paulhartzer.com/programming/csharp/canfield-wpf/

HLSL Image Filters 13 Rev. 9/11/10 – Paul Hartzer