Projections

The terms projection and coordinate system are used as synonyms by Manifold.  The term projection tends to be favored by cartographers, such as the great master J.P. Snyder, while programmers often prefer coordinate system.  They mean exactly the same thing in this documentation.   

 

Projections are used for four main reasons:

 

 

Manifold understands and can work with over 7500 coordinate systems, including all coordinate systems and transformations specified by the vast EPSG system up through the latest EPSG standard.   

 

The system automatically works with a variety of different standardized systems that define and specify projections.  For example, when importing or linking a data source that conveys coordinate system information the system will automatically use the coordinate system specified for the data, including any projections specific to the data source, such as SRID designations of coordinate systems used by databases like Oracle, SQL Server, MySQL, or PostgreSQL/PostGIS.  

 

See the discussion in the About Coordinate Systems  topic for information on the various systems known to and automatically used by Manifold.

 

 To convert a coordinate system into a different coordinate system Manifold can use enhanced accuracy transformation methods that use grid files, including NADCON 5, NADCON (including HARN and HPGN), and NTv2 transformations.  See the Enhanced Accuracy topic to learn how to use enhanced accuracy transformation methods.

Automatic Use of Coordinate Systems

Modern GIS file formats and data sources that are designed for spatial data will specify the coordinate system to be used for the data they provide.  Manifold reads automatically reads such information.   For example, when a shapefile ensemble includes a .prj file that specifies the coordinate system used, Manifold will automatically read that.   When we import or link from such modern formats or sources, drawings and images will appear in Manifold with the correct coordinate system automatically assigned.    

 

We can then use the data in our workflow without worrying about the coordinate system in use.   Drawings and images will appear in the right places in maps with other layers, we can reproject them if we like and so on.  The coordinate system they use will appear in the Info pane using normal, black text to indicate the coordinate system has been automatically specified by the format or data source.

 

When Manifold exports to modern formats, it will automatically write projection information as well.   For example, when exporting to shapefiles, Manifold will automatically write a .prj file with projection information.

 

The Info pane displays projection information for the active layer or window for maps, drawings, images and labels.   When the active component is a map, the Info pane will report both the projection used by the map window and also the projection used by the active layer in the map.

 

 

The illustration above shows the Info pane when the focus is on a map window called Map with an active layer called buildings.  It shows that the Map uses Pseudo-Mercator projection while the buildings layer is in Latitude / Longitude projection.  

 

Map windows can use any projection desired no matter what projections are used by the components within them. The view of the components within the map will be reprojected on the fly into the projection used by the map.

 

 To change a map's projection, in the Info pane click on the coordinate picker button for the map's coordinate system.

 

Map windows for display purposes will reproject on the fly all layers they contain into the projection used by the map window.  In the example above, the buildings layer will appear in the map window as if it was in Pseudo-Mercator projection.   The buildings drawing will still be in Latitude / Longitude projection, but it will be displayed in the map window as if it was in Pseudo-Mercator projection.

Formats that Do Not Specify Coordinate Systems

Some older formats and formats poorly suited for spatial work, such as CAD formats, do not specify the coordinate system used by the data they contain.  

 

Manifold will detect that a source failed to provide coordinate system information, will assign a temporary placeholder coordinate system of Pseudo-Mercator, and will report that coordinate system in the Info pane using red text to alert us that it is just a placeholder.

 

Before we use that data we must specify the projection manually by launching Assign Initial Coordinate System in the Info pane.

 

When Importing or Linking from non-spatial formats:

 

  1. Import or Link the file.

  2. Open the drawing or image.

  3. In the Info pane, click the coordinate system picker button to launch Assign Initial Coordinate System.

  4. Specify the coordinate system used by the data.

 

Always do the above procedure when importing or linking drawings or images from formats which do not specify the projection used for the data.  This includes all shapefiles not accompanied by a .prj file.  

 

See the Example: Import a Shapefile topic.

The Info Pane

The Info pane is our primary interface for seeing the coordinate systems assigned to components and to edit those coordinate systems.  In Manifold, the terms "projection" and "coordinate system" are synonyms and are used interchangeably.

 

Manifold handles all projection information for us automatically, for example, automatically reprojecting layers in a map on the fly to match the projection use by the map.  Manifold also will automatically assign a projection to components such as drawings and images that are imported or linked into Manifold.

 

Modern formats and data sources that are well-designed for spatial data will automatically specify the coordinate system to be used for the data they provide.  When we import or link from such modern formats or sources, drawings and images will appear in Manifold with the correct coordinate system automatically assigned.  

 

We can then use them in our workflow without worrying about the projection they use.  They will appear in the right places in maps with other layers, we can reproject them if we like and so on.  The coordinate system they use will appear in the Info pane using normal, black text to indicate the coordinate system has been automatically specified by the format or data source.

 

Older formats and data sources or those which are poorly-designed for spatial data, such as CAD formats, often will not specify the coordinate system to be used for the data they provide, or they will do so in an incomplete way.  When Manifold brings in such data it will assign the default coordinate system, WGS 84 / Pseudo-Mercator (EPSG:3857), as a placeholder, and will use red text in the Info pane to indicate the coordinate system must be manually specified before that data can be used.

 

To summarize:

 

 

 

We use the Info pane for three key activities involving coordinate systems:

 

 

 

Reproject Component

Launched from a layer's coordinate picker button in the Info pane, the Reproject Component command is used to reproject an image or drawing that already has had an initial coordinate system assigned, either automatically as a result of import from a spatially-aware source, or as a result of manual application of the Assign Initial Coordinate System command in the event of import from a spatially brain dead source.  

 

If the initial coordinate system has not yet been assigned, Manifold will display the coordinate system for the layer in red text and the Reproject Component command will not be enabled.  The only choice in the coordinate picker button's menu will be Assign Initial Coordinate System.

 

To change the projection of an image or drawing:

 

  1. Open the image or drawing.

  2. In the Info pane click the coordinate picker button and choose Reproject Component.

  3. In the Reproject Component dialog click the coordinate picker button for the New system to use, and choose the new coordinate system desired, either from the favorites listed or by choosing More.

  4. Press the Add Component button to create a copy of this component using the new coordinate system, or..

  5. Press the Update Component button (drawings only) to update the drawing in place using the new coordinate system.

 

For drawings, changing the projection changes the coordinate numbers embedded in the drawing's geometry.  For images, changing the projection will change the number of pixels to interpolate coverage by pixels over the new shape of the image in the new coordinate system.  

 

Because the number of pixels and thus, the number of tiles, will change, changing the projection of an image is always an Add Component operation.  That creates a copy of the image in the new projection form.  See the Reprojection Creates a New Image topic for why reprojecting an image changes the number of pixels.

Assign Initial Coordinate System

If we work with data from modern formats and sources that are spatially aware, we will rarely need to use this command.  If we work with data from older formats, CAD formats, or other spatially-unaware formats we frequently will use this command.

 

Launched from a layer's coordinate picker button in the Info pane,  the Assign Initial Coordinate System command is used to specify the correct initial coordinate system for an image or drawing that was brought into Manifold from a spatially brain-dead source.  If the originating format or data source failed to specify the coordinate system used, Manifold assigns a provisional placeholder, the default Pseudo_Mercator coordinate system, and then warns us the data has not had an initial coordinate system assigned by reporting the coordinate system in red text.

 

 

Assigning an initial coordinate system does not change the coordinate numbers embedded in a drawing's geometry and it does not change pixels in images.   It only specifies the correct coordinate system property to be associated with that drawing or image so the data they contain makes sense as intended.

 

To assign an initial coordinate system:

 

  1. Open the image or drawing.

  2. In the Info pane, click the coordinate picker button and choose Assign Initial Coordinate System.

  3. In the menu that appears, choose the new coordinate system desired, either from the favorites listed or by choosing More.

 

 

Manifold comes pre-loaded with default favorite coordinate systems of Latitude / Longitude and Pseudo-Mercator, the most frequently used in most GIS work.  Latitude / Longitude is used by older data and Pseudo-Mercator by most projected web and modern data, since it is the universal standard for web server cartographic displays.

 

 

Tech Tip:   We should make sure that anyone who we allow to work with our data understands that Assign Initial Coordinate System is to be used only to specify the correct starting coordinate system when importing from a spatially-unaware source.   It is not to be used to change the projection of a correctly imported image or drawing.    

 

Everyone on our team must understand they only should use Reproject Component to change the projection of a correctly imported image or drawing, and not to assign a coordinate system initially shown in red.

Coordinate System Problems

Problems with coordinate systems as a result of import almost always happen for one of five reasons, all of which are caused by bringing data in from spatially-unaware or defective sources.   When using modern, spatially-aware, competently programmed sources these problems do not happen:

 

 

 

 

In addition, problems with coordinate systems can arise when the data we import has already been damaged:

 

 

 

Verify the Coordinate System

The easiest way to avoid problems from data with incorrectly specified initial coordinate systems getting into our projects and then being replicated throughout all our workflow is to check the coordinate system when we first bring in the data.   Even if the source is a modern one that specifies initial coordinate system, the author of the data might have made a mistake when publishing it.  

 

Surprisingly, even data downloaded from national cartographic bureaus as prestigious as IGN in France or USGS in the US sometimes will have the wrong initial coordinate system specified. Ouch.

 

Opening a drawing or an image in its own window is not enough, because damaged data can look perfectly correct when viewed by itself  To detect obvious errors the data must be displayed with a known good layers, for example, a layer taken from an imageserver like Bing or Google.

 

It might not be obvious that we have brought data into Manifold with an incorrectly specified initial coordinate system. Sometimes there may be obvious distortion visible, but in more subtle cases if we open a drawing in its own window it may look perfectly OK so long as it is not compared with good data.    We might only be able to tell that the coordinate system was incorrectly initialized by seeing it in a map as a layer compared to a "known good" layer.

 

That is easy to do.  

 

How to quickly check coordinate systems:

 

  1. Create a data source for an image server like Google streets or Bing street.

  2. Create a map with that image server image as the first layer.

  3. Bring new data into Manifold.

  4. Drag and drop that new data into the map.

  5. See if the new data aligns as expected with the "known good" background layer of the image server.

 

In most cases when projected data has the wrong initial coordinate system specified it will be obviously not aligned with the background layer in the map.   It may be so wildly off that it won't even be visible in the map.

 

See the Example: Detecting and Correcting a Wrong Projection  topic for a step by step, illustrated example.

 

The above technique is so convenient, so fast and so important that, for all the sophistication of Manifold analytics, in a production environment involving a lot of spatial data the simple checking of newly-imported data for incorrectly specified coordinate systems may be the first, most important task.   

Coordinate System Formats and Accessory Files

Many different formats and notations have emerged for conveying coordinate systems.  These range from the supremely expert and unambiguous, such as EPSG, to hodgepodge notations like "world files."  When connecting to data sources, for example, when importing files, connecting to database servers or connecting to online sources such as web servers, Manifold recognizes many different formats for specifying coordinate systems:

 

 

Projections from Accessory Files

When reading file formats Manifold will automatically read coordinate system information from the source file and from any accompanying files, if they exist, to acquire information on what coordinate system should be used for the data.   Accompanying files are also called sidecar files in GIS jargon.   

 

Using sidecar files to convey coordinate system information is generally a Stone Age practice, because of the poor standardization for such files and the obvious risk of losing coordinate system info when somebody copies and forwards the main file but does not pass on the accompanying files.  Modern formats, like GeoTIFF, embed all necessary coordinate system within the main file itself, with no need for any sidecar files.  But we still should know how to handle sidecar files in order to work with data that uses them.   Sometimes even with modern formats, like GeoTIFF, we may encounter sidecar files.

 

Why would auxiliary files accompany a data file, such as a GeoTIFF file, that already provides projection information within the data file?   Auxiliary files may be used to provide more detailed projection information than the data format itself can handle.  Auxiliary files may also be used to correct inaccuracies in the information provided by the data file, for example, in cases where the provider of the data file does not have means to correct the information within the data file.

 

The general rule is that if a data file format is spatially aware, and thus provides coordinate system info, the use of auxiliary files is optional.  If optional auxiliary files accompany the data file, the logical inference is that they are there for a reason, such as correcting errors in the main data file or providing more detailed information.   Therefore, when auxiliary files are provided in addition to a spatially-aware data file it is important that the information in the auxiliary file overrides, that is, takes precedence, over, whatever the data file says should be used for a coordinate system.

 

Sometimes accessory files which accompany the main file specify erroneous coordinate system information.  When inaccurate information in accessory files overrides correct information in the main file, the result is an incorrect coordinate system.  

 

Manifold reads files in the following order to acquire coordinate system information.   Data found in latter steps will override what is found in prior steps.  For example, MAPMETA files will override anything that came before.

 

  1. Primary data file - Manifold reads whatever the primary data file says should be used and uses that as the coordinate system.  For example, if a TIFF file is a GeoTIFF, Manifold will read the coordinate system the file itself says should be used.

  2. PRJ files - PRJ files evolved to provide coordinate system information that shapefiles themselves could not provide.  They are now widely used for other formats besides shapefiles.  Manifold reads whatever is in an accompanying PRJ file and overrides existing coordinate system information.

  3. PAM XML files - Read PAM (Persistent Auxiliary Metadata) XML files created by ESRI ArcGIS, ESRI ArcMap, GDAL and similar packages.  PAM files are often used with raster files.  If the raster file is named MyRaster the PAM file will usually be named MyRaster.aux.xml.

  4. "World" files - Read the world file and use it to adjust existing coordinate system info. World files only contain scales and offsets so they can only be used to adjust existing information. World files should also use file extensions that match the primary data file.  Manifold tries to detect world files dynamically using a two step, case-insensitive, process that is similar to ESRI rules for world file names:  First, try the original extension adding w at the end.  For example, if an image file is called niagara.jpg Manifold will look for a niagara.jpgw file as a possible world file.   Second,  if the extension has three letters, try removing the second letter and adding w at the end.   For example, if an if an image file is called niagara.jpg and there is no niagara.jpgw file in the same folder, Manifold will look for a niagara.jgw file as a possible world file.  So far, that matches ESRI conventions which cover three letter extensions but are silent on four or more letter extensions.  In the case of four or more letters in the extension, Manifold takes the first and last letter in the extension and tries adding w at the end.  That works for combinations like having an image file called niagara.jpeg and a world file called niagara.jgw.   A much better idea than using world files is to use a sensible format like GeoTIFF and skip the mess of relying on sidecar files to provide coordinate system info.

  5. Manifold Release 8 XML - XML auxiliary files produced by Manifold 8 are precise and high accuracy and, if present, will override existing coordinate system specifications harvested up to that point.

  6. Manifold Release 9 MAPMETA files - Manifold MAPMETA  accessory files are produced by Manifold 9 and are the gold standard for Release 9 use. They override any other coordinate system specification.

 

In the event of conflicting coordinate definitions the last file wins, since the read order is in order of more rigorous definitions, or of files that are presumed to be there to correct an error.   If an incorrect coordinate system definition results, the first step in correcting the situation is to remove all accessory files and read the data file only.  If the result is a correct coordinate system, then it is clear one or more of the accessory files are the cause of the error.

Notes

Synonyms -  Cartographers favor the term projection while programmers seem to prefer coordinate system.  This documentation uses the two terms interchangeably, with the term projection tending to be used more in GIS or display contexts and the term coordinate system tending to be used more when discussing programming, SQL or standards.

 

Why those particular world file rules? - There is no standard for naming world files, but given that ESRI introduced world files most archival data organizations recommend doing what ESRI does.  As the loc.gov website puts it, using text taken verbatim from the ESRI description: "In general, world files use the same name as the image, with a "w" appended. For example, the world file for the image file mytown.tif would be called mytown.tifw, and the world file for redlands.rlc would be redlands.rlcw. However, since some older software only accepts 3-letter file extensions, the first and third characters of the image file's suffix and a final "w" are sometimes used for the world file suffix. Therefore the world files for mytown.tif and redlands.rlc might be mytown.tfw and redlands.rcw, respectively."

 

Coordinate System Property - The coordinate system in use for a drawing or image can be seen in the Properties dialog for the table that hosts the drawing geometry or the tile.  Right-click on the table in the Project pane and choose Properties.  The FieldCoordSystem.Tile property for images or the FieldCoordSystem.Geom property for drawings will contain the coordinate system.

 

Image projections limited to maximum hyperthreads - Image reprojection using multiple threads limits the number of total threads to the number of CPUs * 2, the maximum possible number of hyperthreads.   This is a safety measure to protect against trying to create too many threads when using Manifold programmatically.

 

Reprojections and Accuracy - Reprojecting a component from one coordinate system into another coordinate system may or may not be lossy depending upon the mathematical difference between the two coordinate systems.   Some coordinate systems use the same basic projection and are different only in terms of shifts, scaling or the order of axes.  When a reprojection only requires component data to be shifted, scaled or have axes swapped, the result is a high accuracy transformation.   When two different coordinate systems have two different underlying mathematical models the recomputation of coordinate data to reproject a data set from one coordinate system into the other can require curvilinear transformations that are lossy.  

 

There is generally no need to panic over "lossy" reprojections because the computational accuracy of Manifold is so high that even in the case of lossy reprojections any resulting inaccuracies usually will be far less than the intrinsic accuracy of the data.  If our data only is accurate to plus or minus a meter, it makes no difference if a recomputation introduces jitter plus or minus a millimeter.  

 

If we are worried about lossy reprojections, that usually is easy to avoid: leave the data in the original projection and simply display it in a map that uses whatever projection is desired, to reproject the data on the fly for viewing, doing all computation in the original projection or converting on the fly within the computation.  That avoids piling up of any possible inaccuracies from doing many reprojections.

 

EPSG is not Universal -  Modern GIS software will support the use of EPSG codes, although not all packages which claim to support EPSG do so correctly.  See, for example, the discussion in the That YX Thing essay.

 

EPSG and WKT / WKT2 - Writing a coordinate system to WKT or WKT2 text fields (such as when using the Compose text transform) writes authority codes for EPSG systems. Parsing a coordinate system from WKT or WKT2 text fields (such as when using the Copy text transform with wkt geometry in the Use parameter)  reads authority codes for EPSG systems and, if the definition of the parsed system matches that of the referenced EPSG system, adds the EPSG code to the parsed system.  That allows using transforms for that system specified in the EPSG database, such as transforms based on grid files.

 

See Also

Enhanced Accuracy

 

Rotated Views

 

Coordinates

 

Info Pane

 

Assign Initial Coordinate System

 

Repair Initial Coordinate System

 

Reproject Component

 

Map Projection

 

Coordinate System

 

Base Coordinate System

 

Favorite Coordinate Systems

 

Favorite Base Coordinate Systems

 

Example: Import a Shapefile - ESRI shapefiles are a very popular format for publishing GIS and other spatial data.  Unfortunately, shapefiles often will not specify what projection should be used.  This example shows how to deal with that quickly and easily.

 

Example: Convert a 0 to 360 Degree Projection - We often encounter data, both images and drawings, using latitude and longitude degrees that appears to be in Latitude / Longitude projection but which has longitude values from 0 degrees to 360 degrees and latitude values from 0 degrees to 180 degrees, instead of the usual arrangement of -180 degrees to 180 degrees for longitude centered on the Prime Meridian, and -90 degrees to 90 degrees for latitude centered on the Equator.  This example shows how to utilize such data by assigning the correct projection.

 

Example: Assign Initial Coordinate System - Use the Info pane to manually assign an initial coordinate system when importing from a format that does not specify the coordinate system.

 

Example: Reproject a Drawing - An essential example on changing the projection of a drawing, either within the drawing itself, or by changing the projection of a map window that shows the drawing and on the fly reprojects the drawing for display.

 

Example: Change Projection of an Image - Use the Reproject Component command to change the projection of an image, raster data showing terrain elevations in a region of Florida, from Latitude / Longitude to Orthographic centered on Florida.

 

Example: Adding a Favorite Coordinate System - Step by step example showing how to add a frequently used coordinate system to the Favorites system.

 

Example: Detecting and Correcting a Wrong Projection - A lengthy example exploring projection dialogs and a classic projection problem.  We save a drawing into projected shapefiles and then show on import how a projection can be quickly and easily checked and corrected if it is wrong.

 

SQL Example: List Transformation Grids - In this example we use a few snippets of easy SQL to list NTv2 and NADCON transformation grids that are available within the grids.dat compressed collection of transformation grid files.   Grid transformation files can be used when creating custom base coordinate systems, for NADCON / HARN / HPGN and NTv2 high accuracy transformations that use grid files.

 

Reprojection Creates a New Image - Why changing the projection of an image creates a new image.

 

About Coordinate Systems

 

Projections Tutorial

 

The Earth as an Ellipsoid

 

Latitude and Longitude are Not Enough

 

Shapefiles Strangely Out of Shape

 

That YX Thing