In this example we use the Join dialog to convert a drawing into an image, a classic vector to raster conversion. Drawings and images are very different from each other, so by "conversion" we not mean we alter the form of the drawing; instead, our task is to create an image that shows the same information as the drawing in more or less the same detail as the drawing.
Given a drawing of areas where each area has some attribute of interest, we create an equivalent image with a single channel, where the value in that channel for each pixel is the attribute of the area within which that pixel is located. We choose the size of the pixels in the image so pixels are small enough to show approximately the same detail as the drawing, but not so small that the image becomes far larger than necessary.
Our example starts with a vector drawing of areas where each area has a classification attribute, a numeric code for the land cover within that area. The result is a raster image where each pixel has a numeric value that is the same as the classification code for the area containing that pixel. The example uses actual land cover data from New Zealand, which includes many large and complex polygonal area objects.
Our workflow:
We begin by opening the drawing of interest. We have created a map with the drawing as a layer, so we open that map:
Our drawing uses a coordinate system
that is meter-based and which provides reasonable accuracy for measurements.
Do not
use Latitude / Longitude projection. If the drawing of interest
is in Latitude / Longitude, take a moment to re-project it into some sensible
projection using meters or feet or some other linear unit of measure.
The drawing contains areas (polygons) showing regions of different land cover types in approximately the center of the North Island of New Zealand, just to the West of Lake Taupo. The example uses actual land cover data from New Zealand, which includes many large and complex polygonal area objects.
For example, the selected area above is defined by over 52400 coordinates.
The selected area, like many other areas in the drawing, also is spatially very complex on many scales, as we can see by zooming into the view. What we see above is all part of the same, very complicated area. There is no simple algorithm that can isolate only those pixels that fall within that area and not other areas.
Each area has many attribute fields, including fields which give a classification number encoding the type of land cover. Attributes include classifications as used in various years, 2012 being the latest. In addition to the numeric code, a text description for each type of land cover is provided. For example, classification code 41 in 1996 is used for areas that represent regions of Low Producing Grassland.
The example drawing has been thematically formatted using values in the Class_2012 field, which are the classification numbers we will transfer into the raster image we create.
Rasters (images) are different from vectors (drawings) in many ways, but one of the most important ways is how images use a sea of pixels to convey information. When we create an image that purports to show the same information as a drawing, we have to decide
what size of pixel to use, so the image we create can show what the drawing shows in approximately the same level of detail. Manifold will automatically resize the image generated by the join based on the pixel size we choose and the extent of the drawing.
Measuring the extents of the drawing, to get an idea of what size image will be created, is optional. But It is wise to get an approximate idea of the size image that will be created by first measuring the extents of the drawing. That will give us early warning if the size pixel we have chosen will result in an unreasonably large image.
We will first measure the extents of the drawing, and then we will measure the size of small details in the drawing, using path measurement.
With the focus on the open drawing window, in the cursor mode button we choose Draw Path. We click once at the left edge where objects are located and then we click again at the right edge of where objects are located in the drawing.
We Right-click and choose Copy Length. That copies the length of the path segment onto the Windows Clipboard.
Next, open a Notepad document, or open a Comments component in our project, and Paste into that, to save as text the length we have just measured.
Right-click and choose Clear Path, for a fresh start in making another measurement.
Next, we click once at the upper edge where objects are located and then we click again at the lower edge of where objects are located in the drawing.
We Right-click and choose Copy Length. That copies the length of the path segment onto the Windows Clipboard.
We switch to our Notepad document, or our open Comments window, and Paste into that, to save as text the length we have just measured. Our text document now has two measurement in it:
58809.51484172177
49953.95605251737
The first measurement gives the width, that is, the West-East extent, of the objects in our drawing in meters. The second measurement gives the height, that is, the North-South extent, of the objects in our drawing in meters. We can see at a glance that our drawing shows land cover areas in a region approximately 59 kilometers wide and 50 kilometers high.
Next, we will take a look at how detailed our data seems to be.
We zoom into various representative regions of our drawing, looking for areas that seem to be the most detailed in terms of the intricacy and small detail of their boundaries.
For example, we can zoom far into an area that seems to have typically small detail. We right-click and choose Clear Path, and then we click the path tool to measure the size of smaller details. The detail seen above is about 12 meters in length.
It seems that using pixels that are 10 meters in size will give us about the same level of detail as seen in the drawing. Using pixels that are 10 meters in size to cover a region that is 59000 meters by 50000 meters in size will result in an image 5900 by 5000 pixels in size. That seems reasonable, even small by Manifold standards.
If we want to be more precise, we could use pixels that are 5 meters in size, resulting in an image 11800 by 10000 pixels in size. Using pixels that are 1 meter in size would result in an image that is 59000 by 50000 meters in size. That seems excessive, since the drawing seems to have accuracy visibly lower than that.
If we have any metadata we can read that describes the drawing, we should consider that as well. It could be that the publisher of the drawing mentions that the areas are only accurate give or take ten meters. In that case we should not feel a need to use pixels smaller than ten meters in size, since in any event the drawing is already an approximation at that level of accuracy.
We no longer need path measurement,
so we restore the cursor to default, navigation mode.
As part of this workflow, we will create a new image that uses the same coordinate system as our starting drawing. Strictly speaking, we do not need to use the same coordinate system in the image as in the drawing, since the Join dialog can accomplish joins between a source, joined component and a target, original component that are in different coordinate systems. However, it is easier to get a clear idea of what will happen for pixel size and resulting size of the image when the same coordinate systems are used. We therefore will use the same coordinate systems for the image as the drawing. That is easy to do.
The easiest way to pick the same coordinate system for the image is to first Copy the coordinate system used by the drawing, so when we create the image we can Paste that same coordinate system as the one to use for the image. We will use that method in this example:
Another way to use the same coordinate system as the starting drawing is to first save the coordinate system used by the drawing as one of our Favorite Coordinate Systems, and then when we create the image we can choose that coordinate system from a list of Favorites. That takes a few extra clicks, but if we plan on using that same coordinate system again it is convenient to have it in the list of Favorites.
We have all the information we need to create a new image. We right-click into the Project pane and choose Create - New Image.
The first thing we do, while we have the coordinate
system copied to the Clipboard (before we inadvertently overwrite it with
something else copied to the Clipboard), is to press the coordinate picker
button and to choose Paste. That
pastes the coordinate system we copied from the drawing into the Coord system to be used by the image.
Next, we provide a name for the image, Raster Landcover. We know that the Class_2012 field is an integer, so we choose int16 as the data type for pixels in the Tile. That saves some space with many pixels in the image as compared to using int32 or int64. If we expected some really big numbers to be used as classification codes, we might have chosen int32.
Choosing int16 as the data type is also choosing a single channel for the image, as opposed to int16x3 or some other multiple integer data type.
Our task now is to set the desired size of pixels, by adjusting the metrics in the coordinate system.
Once again, we press the coordinate system
picker button, and in the pull down menu we choose More... to
launch the full Coordinate
System dialog.
In the Coordinate System dialog we click the Custom tab.
We press the XY metrics picker button and choose Edit.
In the Coordinate System Metrics dialog, we enter 10 for both the Local scale X and the Local scale Y values, to specify that pixels will be ten meters wide and ten meters high. Press OK.
Back in the Coordinate System dialog, press OK.
Back in the New Image dialog, we check all settings and press Create Image.
A new image called Raster Landcover and a new table called Raster Landcover Table are created in the Project.
To do spatial joins using the Join dialog, we must have all components which participate in the join as layers in the same map.
We drag and drop the Raster Landcover image into the map. The new image is blank, so we see no visual change.
With the focus on the Raster Landcover layer in the map, in the main menu we choose Edit - Join.
The Raster Landcover image by default is the target, original component. In the upper right-hand box we choose the Landcover drawing as the source, joined component.
We check the Clear image pixels and Resize image boxes, so we do not forget to do that later on. This will clear the image before the join operation and will enable the dialog to resize the image as needed for the join. The dialog will use the extents of the drawing and the pixel size in the image to calculate how the image should be resized. That will be different than the rough guess we calculated by measuring the extents of the drawing, but our estimate will not be wildly off.
The Raster Landcover image has only one channel, Channel 0, We double-click into the far right cell in that row, to pick the joined field to use.
The pull down menu offers all of the numeric fields in the Landcover drawing that can be used in a join with the numeric Channel 0 field. We choose Class_2012.
Next, we double-click into the center cell in the row, to choose the transfer method to use.
In the pull down menu we choose max. The transfer method specifies how values in the Class_2012 field should be combined in cases where there is more than one object in the Landcover source drawing that cover the same pixel in the Raster Landcover image. We choose max, so the largest value from any overlapping areas will be used.
We believe there are no overlapping areas in the drawing, but just in case we have to pick some method of choosing one of them. If there are no overlapping areas, then choosing max, or min, or sum will all have the same effect, transferring the single value from the single area that covers the pixel.
We check our work, and press Join to launch the join.
Manifold runs the join operation using all available threads. On an older computer using a four core CPU, providing eight threads, the process is virtually instantaneous. It is so fast that we decide that next time we will use a smaller pixel size, about 5 meters.
When the join finishes, the Raster Landcover image becomes filled with pixels using default style. Before the join operation was launched, pixels were cleared, that is, assigned NULL values, in regions outside any areas in the drawing. The image therefore is transparent in those regions, giving the impression there are no pixels in such regions.
To compare our result to the original drawing, we will Style the image using the same thematic formatting as the drawing.
With the focus on the Raster Landcover layer, we choose the Style pane. In the Style pane we choose Channel 0, unique values, and 100 interval values. We press Tally, and then we use the palette button to assign a palette.
The palette we choose is the Classic - Land Cover palette, with reversed color ordering. We also reverse the colors used for values 2 and 16. This is an entirely arbitrary coloring, but it is the same used in the drawing and provides reasonably green colors in classifications that are used for forest and similarly dense vegetation. Values 2 and 16 are reversed to provide better distinction in classifications next to each other where similar colors tend to look too much the same.
Press Update Style.
The image layer immediately redraws using a color scheme like that used in the drawing.
We can zoom further into the image to compare the colors of pixels, taken from the classification assigned to each pixel by the join, to see how detail and classification compare to the drawing.
The image layer is seem above. The drawing layer for the same view is seen below.
The drawing, of course, is sharper, because vectors are infinitely detailed compared to raster pixels. The drawing also looks sharper because each area has a black border line, in this case a line that is drawn with 0.5 point thickness.
We can style the area border lines in the drawing using the same thematic format as used for the fill color for areas, to create a visual effect more directly comparable to the raster image.
With area border line color the same thematic format as area fill colors, we see the drawing layer above. The image layer is shown below to make comparison easier.
Carefully comparing the drawing and the image we see that our choice of approximately ten meter pixels was a reasonable choice. Five meter pixels would have been a better choice, resulting in finer detail, with fewer jagged edges.
The process is so fast, we can repeat the above workflow, creating an image where we alter the metrics to use Local scale X and Local scale Y of 5 and rerun the join on that image. The result is seen above, using pixels that are five meters in height and width.
Attribution - The Landcover drawing is a small section of land cover data for all of New Zealand downloaded from the LINZ Data Service. The original contains data sourced from the LINZ Data Service licensed for reuse under CC BY 4.0.
Terminology - The original table also may be called the target table, and the table providing data also may be called the source table.
Quick reference - With the Join dialog open, press F1 to open a quick reference guide in a browser window.
Why not have a copy transfer method for drawing to image joins? - In the examples above we chose as the transfer method either max when we expected no area overlaps, or sum when we expected overlaps. Why not have a copy transfer method that in the case of multiple areas containing the same pixel, the system just pulls the first value from one of the areas it encounters? That is not done because when areas overlap they usually overlap in a way that covers more than one pixel. Suppose a few hundred pixels are in the region of overlap. If copy existed, it would simply pull values from whatever one of the overlapping areas it first encountered. But because there is no ordering in such things, different pixels within the region of overlap might have values pulled from different areas, resulting in different values for different pixels within the region of overlap.
If count, min, max, or sum are not sufficient transfer methods to handle the aggregate effect of multiple areas covering the same pixels, the solution is to first clean up the overlaps in the drawing. One way to do that is to perform a Merge Areas transform, then to convert the resulting areas to lines, and then finally to create Bounded Areas on those lines. The result will be independent area objects in all regions of intersection of overlapping areas. Transferring attribute values to those regions of intersection will still require some sort of logic to choose which area value to assign, but at least such logic can be done using arbitrarily complex expressions instead of the simple list of options within the Join dialog.
Guessing the key field - When we launch the dialog with an original field and when we choose a joined field, the dialog will try to guess what fields we might want to use as key fields and will load those first by default. If it guesses wrong, we can choose the field we want. For the original table, the dialog tries to use a field (other than mfd_id) that has a BTREE / BTREENULL index, with a data type preference for numbers, then text, then everything else, and a name preference for field names with id in them, such as ... id or ..._id, then ...id and then everything else. For the joined table, the dialog uses similar logic, but first tries to use a field with the same name as in the original table.
Added fields are static - Added fields are static, that is, if data in the joined table changes the fields in the original table that received data in the Join operation from the joined table will not automatically be updated. We can easily update the table any time we want using a saved update query.
Spatial joins are automatically parallel - Joining data between drawings uses parallel GeomOverlay function variants. Joins from images to drawings are parallelized through a nested SELECT using THREADS.
See the Join Videos list of videos showing how to use the Join dialog.