Transform Pane operations in the Filter template utilize SQL functions that compute their results using a matrix filter. Matrix filters are also known as convolution matrix filters. Manifold jargon is to just call them filters, at least in contexts where it is clear we mean a matrix filter. Search for the term filter definition matrix in the SQL functions topic to find functions that use such filters. The video version of this topic is the Manifold Viewer - How Matrix Filters Work video.
Matrix filters are used in computations where a new image is created from an source image, and the value that goes into each pixel in the new image is calculated by considering what the value is of the corresponding pixel in the old image as adjusted by computations based on surrounding pixels.
A matrix filter is just a matrix of numbers, which could be positive or negative numbers. They are usually small, whole numbers, but some filters may use decimal fractions or much larger numbers.
The illustration above shows a typical filter that is a 3 x 3 matrix. It has three rows of numbers with three columns in each row. The center cell in the illustration is shown with a red border to emphasize that is the center of the filter. The numbers in the cells guide how the computation should weight surrounding pixels in a calculation that combines the values of those surrounding pixels with whatever is the value of the center pixel. The particular example is a matrix filter that detects edges in a raster image.
Another way to say a matrix is a 3 x 3 matrix is to say it has a radius of 1. That means the center of the filter is surrounded by cells in one neighboring layer.
A radius of 1 means a 3 x 3 matrix, like our first illustration, or like the yellow bordered region in the illustration above.. A radius of 1 means to go one layer of cells surrounding the central cell. A computation using a 3 x 3 matrix, that is, a matrix with a radius of one, will perform a computation using a total of 9 pixels: the central pixel plus the eight pixels that surround it. A radius of 2 means a 5 x 5 matrix, like the green bordered region in the illustration above. A calculation using a 5 x 5, or radius 2, matrix will perform a calculation using a total of 25 pixels. The outer, blue bordered region in the illustration above shows a 7 x 7 matrix, with a radius of 3. A calculation using a matrix with a radius of 3 will compute using a total of 49 pixels.
If such calculations have to be repeated for each pixel in an image, in the case of images that are gigabytes in size the difference between having to perform 49 computations for each pixel in the image compared to only 9 computations can be a big difference in time. Therefore, filters with radius 2 or 3 are usually used only in machines that have GPGPU acceleration. GIS packages which are not GPU parallel usually only provide 3 x 3, radius 1 forms of matrix filters.
We can see how matrix filter computations work by doing an example manually. We will use a small, 3 x 3 matrix (radius 1) and small numbers to make the calculations easy and obvious. In the illustrations that follow, we illustrate the filter matrix being used on the lower left of the illustration, just for reference. The matrix used detects edges.
The Source Image in the illustration represents a portion of a raster image, with each cell representing a pixel. The small numbers in the corners of the pixels represent the values of each pixel. For simplicity, we assume we are working with a single channel image, where the numbers are a grayscale color or a height in a terrain surface.
All of the numbers in the Source Image pixels are either 0 or 1, to make calculations easier. They have been arranged and the cells colored so we can see there are two regions of lighter or darker color, with the boundary between them being an edge. To the right we see an illustration of what the Result image will be after the matrix filter calculation is done. That image has been temporarily colored similar to the Source Image to better help us keep track of which pixel is being calculated.
The calculation happens by overlaying the Filter matrix onto each pixel in the Source Image, doing a calculation for that pixel, and then placing the result of the calculation into the corresponding pixel in the Result image. To keep this example simple, we will compute only a few pixels in the middle of the image on both sides of the edge between the region of lighter pixels and the region of lower pixels. In the illustration above, we have overlaid the filter matrix onto the first pixel in the Source Image we will compute: that pixel is in the center of the matrix, marked in red, and the corresponding pixel in the Result image is also marked in red.
The calculation is absurdly easy: For each cell in the filter matrix, multiply the number in the matrix cell by the number for that pixel in the source image. Add up the results and put the sum into the corresponding pixel in the Result image. In the above case, the calculation is:
0*0 + 0*-1 + 0*0 + 0*-1 + 0*4 + 0*-1 + 0*0 + 0*-1 + 1*0 = 0
0 is the value we put into the Result pixel.
Next, we move the filter matrix one pixel to the right and repeat the calculation with the numbers it covers:
0*0 + 0*-1 + 0*0 + 0*-1 + 0*4 + 1*-1 + 0*0 + 1*-1 + 1*0 = -2
-2 is the value we put into the Result pixel.
We move the filter matrix one more pixel to the right and repeat the calculation with the numbers it covers:
0*0 + 0*-1 + 1*0 + 0*-1 + 1*4 + 1*-1 + 1*0 + 1*-1 + 1*0 = 2
This is the first time the filter matrix is positioned so that a value of 1 in the Source Image coincides with the 4 cell in the filter. With the multiplication of 1*4 in the values added together, we get a larger grand total. 2 is the value we put into the Result pixel.
We move the filter matrix one more pixel to the right and repeat the calculation with the numbers it covers:
0*0 + 1*-1 + 1*0 + 1*-1 + 1*4 + 1*-1 + 1*0 + 1*-1 + 1*0 = 0
0 is the value we put into the Result pixel.
We move the filter matrix one more pixel to the right and repeat the calculation with the numbers it covers:
1*0 + 1*-1 + 1*0 + 1*-1 + 1*4 + 1*-1 + 1*0 + 1*-1 + 1*0 = 0
0 is the value we put into the Result pixel. At this point, it is clear what will happen if we compute pixels on the next row down. We will get a similar pattern.
We move the filter matrix down one row of pixels and then begin calculating again at the pixel shown:
0*0 + 0*-1 + 0*0 + 0*-1 + 0*4 + 0*-1 + 0*0 + 0*-1 + 1*0 = 0
0 is the value we put into the Result pixel.
We move the filter matrix one pixel to the right and repeat the calculation with the numbers it covers:
0*0 + 0*-1 + 0*0 + 0*-1 + 0*4 + 1*-1 + 0*0 + 1*-1 + 1*0 = -2
-2 is the value we put into the Result pixel.
We move the filter matrix one more pixel to the right and repeat the calculation with the numbers it covers:
0*0 + 0*-1 + 1*0 + 0*-1 + 1*4 + 1*-1 + 1*0 + 1*-1 + 1*0 = 2
2 is the value we put into the Result pixel.
We move the filter matrix one more pixel to the right and repeat the calculation with the numbers it covers:
0*0 + 1*-1 + 1*0 + 1*-1 + 1*4 + 1*-1 + 1*0 + 1*-1 + 1*0 = 0
0 is the value we put into the Result pixel.
We move the filter matrix one more pixel to the right and repeat the calculation with the numbers it covers:
1*0 + 1*-1 + 1*0 + 1*-1 + 1*4 + 1*-1 + 1*0 + 1*-1 + 1*0 = 0
0 is the value we put into the Result pixel. We can see that if we move the filter matrix to the left or right of the inclined edge between the lighter region and the darker region, the computation will result in -2 or 2. For any other pixel location the computation will result in 0.
If we computed the numbers for every pixel, ignoring those on the outermost portion of the Source Image, we would get the numbers shown above. We could color the Result image based on those numbers.
Suppose we colored every Result pixel that had a value of 0 in light gray, we colored every Result pixel with a value of -2 in white, and every Result pixel with a value of 2 in dark gray. We would produce a Result image as seen above.
If we take away the numbers to better show the Result pixel colors, the Result image consists of a line of dark pixels highlighted by white pixels, exactly where the edge between the lighter region and darker region was located in the Source Image. Like magic, a simple calculation using a filter matrix with a few numbers has detected where the edge was in the Source and drawn a line along that edge in the Result.
The example above showed how to apply the filter to a Source Image that had just one number for each pixel, that is one channel. If the Source Image had three numbers for each pixel, for example, if it was an RGB image, we could apply the same operation to each of the three channels. That would allow us to take an RGB image like the one above and to find edges like those that appear in the results image below.
Filter matrix computations are the basis for very many visual effects and analytic operations done with rasters, both in graphics arts editing packages like Photoshop and in GIS. The astonishing range of effects is possible simply by manipulating the numbers used in the filter matrix.
What about those outermost pixels? - The example did not compute the values for the outermost pixels because those results depend on pixels outside of what is shown in the example. Consider an illustration of the filter matrix positioned on an outermost pixel:
To compute values for five of the filter matrix cells we would need pixel values not illustrated in the Source Image. In real life, of course, every image has a given extent beyond which there are no pixel values, so every function that uses a filter matrix must decide what to do when the matrix overhangs the extent of the image. Manifold simply computes using NULL values for such absent pixels. Other systems will "wrap" pixels from the opposite edge, extend the outermost pixels another layer or two, or other approximations. In any case, the results of filter matrix computations within a border of the matrix's radius must be considered an approximation.
Normalization - The matrix filter illustrated is a simple filter for a simple function. Other functions may use a slightly different procedure, for example, normalizing the result placed into each pixel by dividing the result shown in this example by the sum of the non-zero elements in the matrix.
Manifold Viewer - How Matrix Filters Work - The easy, simple way to learn how filters work! Watch this action-packed video using Manifold Viewer that illustrates how matrix filters, also known as convolution filters, work. In addition to explaining filters, the video provides a real-life look at simple Manifold techniques for moving objects around in drawings using the Shift transform, and fast and easy use of Selection and tables to quickly put desired values into attributes. Sound technical? Sure, but in a very easy and simple way.
SQL Example: Process Images with 3x3 Filters - Shows a step-by-step example of developing an SQL query that takes a query written by the Edit Query button and then modifies that query into a general purpose query that can apply any 3x3 filter. This makes it easy to use matrix filters we find on the web for custom image processing. We extend the query by using parameters and adding a function, and then show how it can be adapted to use a 5x5 filter.