The MAPCACHE File

When linking a file or creating a data source using a file Manifold will create a persistent cache in the form of a cache file.  Such files are called .MAPCACHE files.  The cache file can provide dramatically increased performance with many different types of data and data sources.  The New Data Source dialog will automatically hide the Use cache option and related controls for dataports which do not use cache.

 

Note that cache options for web servers in the File - Create - New Data Source dialog will create cache tables in the parent database (the .map project file itself if the data source is not created within another data source) within a Cache subfolder within the parent database's System Data folder in the project.

 

Manifold .MAPCACHE  technology can provide dramatic improvements in user experience. For example, once the cache is built  .MAPCACHE allows Manifold to render very large, linked shapefiles very rapidly, often faster than ESRI's own, native ArcGIS shapefile can.   The technology therefore gives Manifold users the connectivity of ESRI to its own format but with the internal performance of Manifold.

 

Caching data also allows flexibility with Style when linking data from read-only files.  For example, some TIFF images linked from read-only .tif files might not display if Save cache is not checked.

 

Manifold cache tables for vector data will support the same rendering optimizations Manifold uses for vector data in .MAP files if the dataport for the file format in use has been optimized for local cache, as most of the dataports for popular GIS vector formats have been optimized.  Therefore, when linking a MapInfo .MIF drawing file or an ESRI .SHP drawing file with cache turned on, Manifold will render vector data with  performance comparable to Manifold rendering of drawings within Manifold's own .MAP files.

 

When the data source is first used the cache file is created in the same folder as the subject file and is named the same as the subject file with a .mapcache extension appended.   For example, if we linked a shapefile called mexico.shp into our project when we open the data source Manifold will create a file called mexico.shp.mapcache in the same folder as the mexico.shp file.

 

Importing or linking a secondary file in a data set alters the name of the created .MAPCACHE file so that it is bound to the data set and not to the file being linked.   That is useful given that some data file formats involve more than one file, an example being, how a MapInfo MIF "file" involves two files, a .mif file and a .mid file.  

 

If Windows permissions or other settings do not allow Manifold to create the .MAPCACHE file or open it in read-write mode, the system switches to using a temporary cache instead.

 

The Log window will report the time used for creating, recreating or saving a .MAPCACHE file reports the time it took to complete the operation in the log.  The Log window will also report the time required for building or rebuilding a .MAPCACHE cache for a table.

 

Cache tables support writes for read/write tables.   If the table being cached is read-only the cache for that table will also be read-only. If the table being cached supports inserting, deleting or updating records, the cache for that table will also support inserting, deleting or updating records, at all times communicating with the table being cached and updating cache structures accordingly.  We can see that in action with formats such as shapefiles, which involve a DBF file as well as a SHP file and where edits can involve writes to the DBF file for tables as well as to the SHP file.  

 

.MAPCACHE files rebuild themselves if they detect changes to their source files.   

 

Caching data in an individual table tracks progress and can be canceled, but cancelling a caching operation for a table linked with a Use cache option will result in behavior as if the table did not exist.

 

Layers in a map that are currently being cached will have a black line at the bottom of the layer tab.

Notes

Once the cache is built Manifold can render external, linked files like shapefiles with tremendous speed.   But building the cache the first time can take a minute or more for a big shapefile.    Consider for example, working with a shapefile that contains all parcels in Los Angeles county, a shapefile distributed by the LA county assessor's office that consists of a one gigabyte .shp file and a 660 megabyte .dbf file.  Linking that file into a Manifold project on a slower, older generation computer requires a full 100 seconds, about a minute and a half, and then saving the project file takes about 12 seconds.  After that, saves and opens are essentially instantaneous.

 

After the overhead of first linking the file Manifold can render the drawing, even though the data continues to reside "in place" in the original shapefile, with performance approaching that of a native Manifold map file.    A Zoom to Fit takes about one half second while zooms into the most dense sections of parcels in LA County typically take about two seconds, very good performance for a shapefile with 2.4 million areas.  Doing the same renders with the same data imported and stored within a native Manifold .map file is under half a second to Zoom to Fit and under one and one half seconds for zooms into very dense regions of the drawing.

 

The comparison does have its limits, of course, since shapefiles are limited in size while the Manifold .map will continue providing about the same rendering performance even with data more than ten times as large.   But still, the speed that .MAPCACHE provides makes it possible to realistically and comfortably edit and view very large shapefiles and MapInfo files "in place" as linked files.

See Also

Getting Started

 

How to Edit a Single File

 

Where Manifold Data is Stored