File - Import

The File - Import command launches the Import dialog, very similar to the standard Windows File - Open dialog, which allows browsing the Windows file system to locate and to choose the file to be imported. The Import dialog allows importing of file-based data sources such as files, file databases, scripts and text comments. Manifold can read virtually every format and data source in existence. See the Big List of Formats and Data Sources topic for the huge list.

Manifold uses Microsoft facilities to import from all Microsoft Office formats, including legacy Office formats such as .db, .html. .mdb, .xls, and .wkx, together with newer Office formats such as .xlsx and .accdb. If Manifold cannot import from such formats, that means the Windows system we are using is missing the necessary facilities. Please see the Microsoft Office Formats - MDB, XLS and Friends topic for a solution.

Controls

In addition to the usual Windows File - Open style controls the Link dialog provides these controls:

File name - Enter the name of the file to be imported, or populate the box by clicking on a file in the display pane. Ctrl-click more than one file to choose more than one file (or use the usual Windows moves to choose more than one file) and to populate the box with more than one file name, or enter more than one file name into the box with each name enclosed in double quote " characters. For convenience, the pull-down list for the box provides a list of recently-opened files.

(File types) - Use All Files (*.*) by default or choose a specific file type to filter the types of files shown in the display pane. For example, choosing JS Files (*.js) will only show files with a .js extension, that is, JScript script files.

Options - For many file types provides access to Cache Data option, to create and utilize an accessory .MAPCACHE file for improved performance. The Cache Data option is checked by default. The Cache Data options has two options, both checked by default, one to cache only data that is expensive to dynamically compute and the other to save cached data between sessions. It only makes sense to uncheck either option if we are desperately low on free disk space. Note that although these options may appear as part of the Link dialog for all file types a .MAPCACHE file is not created or used with file types such as .js for JScript scripts or other file types not supporting use of .MAPCACHE.

Automatic detection of file type

The Import dialog by default browses the Windows file system using a setting of All Files (*.*) to display all files in folder regardless of the file type as specified by a three letter extension. We can click on whatever file is of interest and the system will automatically utilize the correct dataport for that file type.

This works remarkably well even in cases where different formats use the same three letter extension, or when no three letter extension is used. For example, clicking on an .img file for importing will automatically cause the right dataport to be used whether that .img file is in ENVI, ERDAS, IDRISI or IMDISP .img format. Likewise, clicking on a file with no extension will autodetect if it is a GIRAS, GLOBE or GRASS format file and will automatically use the correct dataport for import.

At times despite the system's best efforts we might have to manually specify a particular file type by choosing it from the list, but usually the automatic choice works best.

Projections

The system will automatically read projection (coordinate system) information from all formats that provide such information. Some formats, however do not provide projection information or, like shapefiles, may or may not provide projection information. When components are imported from formats that do not assign a coordinate system we must launch Assign Initial Coordinate System in the Info pane to specify the correct initial projection for the imported component.

Passwords

Some formats from which we might want to import files, such as Access MDB, may allow the use of passwords to control access to the file. In such cases instead of using File - Import we should connect to the file using OLE DB (for MDB files) or other data source connection option available in File - Create - New Data Source where the dialog allows us to specify a password.

Notes

Highlighting imported items - Some file imports will create many components in the project pane. For convenience, Manifold tries to highlight the main imported component, deciding which is the "main" created component based on the following order from most important to least: maps, map layers (drawings, images, labels), tables, and then all other components.

Importing Data from Sources other than Files - Sometimes we would like to import data into a project from a source other than a file-based source. For example, we might like to import a table from a DBMS server such as an Oracle server. We can do that by creating a new data source using the File - Create - New Data Source command, expanding the data source and then using Copy and Paste to copy the desired table and to paste it into our project.

The New Data Source dialog allows us to connect to file-based data sources as well but in addition also allows connection database servers like PostgreSQL or Oracle, generic DBMS connection technologies such as ADONET, OLEDB and ODBC and also to web servers of various kinds. While Copy and Paste is a good approach to make local copies of tables stored on external DBMS servers it is not usually a realistic approach to attempting to make a copy of the almost limitless data stored in servers such as Virtual Earth or Google Earth.

For example, if we create a data source that is a Virtual Earth satellite image when we open that image and zoom in and pan the remote web server will send us the tiles we need to see content in the current zoom / pan viewport we are using. But if we Copy the image's table in the data source and then Paste it as a table in our local project hierarchy, we are commanding Manifold to grab all tiles for the entire, worldwide extent of satellite imagery content in Virtual Earth. Even if we had the hundreds of terabytes of disk space required it would take a very long time to fetch so much data via whatever Internet connection we likely have.

A big part of the idea of linking data from web servers like Virtual Earth is to avoid having local copies. But if we do want to capture tiles for whatever views we are using we can use specialized tools other than Copy and Paste to do so.

Three Letter Extensions - Most file names in Windows end in what is called a three letter extension, which is usually three letters at the end of the file name following a dot . character. The three letter extension is one way Windows at times (but not always) keeps track of what a file is supposed to be. Unfortunately, by default Windows hides the three letter extensions of files and instead tries to associate files with whatever program it thinks should be used to open that file. This is confusing when working with the many file formats that Manifold and similar products utilize.

Therefore, please turn off the hiding of extensions by Windows. A typical way to do so in most versions of Windows would be from Windows Explorer, choose Tools - Folder options, press the View tab and then in the Advanced Settings pane ensure that the Hide extensions for known file types is unchecked. Press Apply to Folders and then press OK. You will then be able to see extensions such as .map and others. See also the Essay on three letter extensions and why the default hiding of them by Windows is such a bad thing.

Geometry collections - Reading geometry collection values automatically merges individual values of the same underlying type used in Manifold geometry, such as area, line or point, with differences between subtypes such as line and multiline being ignored. The result of the merge is returned. This applies to all data which support geometry collection values, including WKB, GeoJSON, JSON, native geometry in database-specific formats, and so on.

Reading geometry collection values with individual values of mixed underlying types automatically converts areas to lines and lines to points in order to return all coordinates. Example: reading a geometry collection with an area and several points will return a multipoint containing all coordinates of all individual values.

3D conversions - Geometry values with mixed 2Dand 3Dcoordinates in GML, GeoJSON, and TopoJSON are automatically converted to 3D with 2D coordinates padded with zeros.

Drag and Drop - Dragging one or more files from a Windows Explorer window and dropping the files into the Project pane will import the files into the project, just like File - Import. Dragging one or more files from a Windows Explorer window and Shift-dropping the files into the Project pane will link the files into the project, just like File - Link.

The origin of imported components - We can get the path to a file that was imported by looking at the Log Window, which logs the path used for import operations.

2021-06-29 12:01:32 -- Import: C:\data\OSM Venice\shapefiles\wiki landmarks.shp (0.007 sec)

That can be convenient, since text can be copied from the Log Window, and then pasted into any comments components giving the origin of a particular layer.

Videos

Manifold 9 - Re-Project a Shapefile - New coordinate system dialogs make it easier than ever to re-project data, often in only one click. This video shows how to import a shapefile and then rapidly re-project it into different coordinate systems. We then show how maps re-project their contents on the fly for display and how to exploit that to rapidly show data in different projections.