Performance Tips

Manifold includes many capabilities previously found only on software that costs tens of thousands of dollars per license. That the software includes such sophisticated capabilities does not imply that any personal computer, no matter how limited or slow, will be able to adequately support use of such capabilities. In particular, work with large images will require lots of RAM and a fast processor. The thousands of dollars we save by purchasing Manifold can be invested into more RAM and more powerful processors that can serve us in other applications as well.

Most Manifold projects using vector drawings on modern computers will operate so fast that thinking about performance is not necessary. When working with large projects or large images, though, we may want to squeeze every bit of performance out of our systems. The following tips will help you achieve the greatest possible performance on your desktop when working with large maps. Many of these suggestions are generic suggestions that will help you create and operate the fastest Windows system possible.

Important Configuration Notice

Very Important: Before starting Manifold, go to the Windows Control Panel Display dialog Effects tab and make sure that the Show window contents while dragging check box is not checked.

In Windows XP this option is found in the Control Panel's Appearances and Themes - Display choice under the Appearance tab by pressing the Effects button.
In Windows XP or Windows 2003, from the Start button open the Control panel and then open the Display dialog. Click on the Appearances tab and then press the Effects button. Uncheck the Show window contents while dragging check box.
In Windows 2000, Windows ME and Windows 98 open the Control panel and then open the Display dialog. Click on the Effects tab, and uncheck the Show window contents while dragging check box.
In Windows Vista, open the Control Panel and click Appearance and Personalization. Click Customize colors, then click Open classic appearance properties for more color options and click the Effects button. Uncheck the Show window contents while dragging check box.

Checking this box will greatly slow down the system when displaying complex maps and images because it forces a refresh of the window contents with each minor change in mouse position while dragging the window. Make sure the box is unchecked.

Hardware

Install lots of RAM memory, enough to maintain your entire project in RAM memory if possible. It does no good to have a fast processor if the fast processor is waiting around for virtual memory to be paged back in from hard disk. Don't forget to include RAM required by Windows and background processes. Memory is very inexpensive, so install as many gigabytes as you can. On newer systems, install as much as you can afford, using four gigabytes if you plan on working with large projects or multiple Manifold instances. RAM has become very inexpensive: a few hundred dollars can buy massive amounts of RAM, multiple gigabytes.
Since 64-bit Manifold can make use of far larger amounts of RAM than 32-bit Manifold, it makes all the more sense to install lots of RAM (3 GB, 4 GB or more if possible) and / or increase the size of the Windows pagefile.
Install a fast processor. If your budget allows a choice between substantially more RAM or a slightly faster processor, get more RAM and a slightly slower processor. If you are working with very large images and surfaces that will not fit into RAM, then it is more important to install a significantly faster processor and a really fast disk drive.
Install a 64-bit Intel or AMD processor, preferably a quad core or dual core processor. Modern multi core processors run very, very fast and will offload some tasks at a price that is barely more than charged for single-core 32 bit processors. They are remarkably inexpensive for the speed and quality they give. Shop around and often you'll find motherboard and very fast multi core, 64-bit processor combos for $300 or less.
Use multiprocessor machines or multi-core processors. Each new release of Manifold adds more functions that are multi-threaded. For example, Manifold will use more than one thread to render image libraries if more than one processor or processor core is available on the computer system. Therefore, image libraries will render faster on multiprocessor or on multi-core processor systems such as those using a quad-core or dual-core Intel or AMD processor.
Use a modern motherboard with performance features. For example as of this writing some motherboards support full-bandwidth SLI for multiple graphics cards, RAID SATA and can accept 8 GB of RAM. Nice! Extremists may want to try out the over-clocking features available with modern motherboards, cool their CPUs with water-cooling and other tricks. See the gaming sites for tips and tricks on maxing out motherboard performance.
Install an NVIDIA GPU card supported by NVIDIA CUDA. Manifold can use CUDA to dramatically accelerate certain tasks. CUDA-enabled NVIDIA GPUs are so ubiquitous that no matter what your budget you can probably get a CUDA-enabled system for no additional cost. Whatever system you buy will have graphics capability in any event, so you may as well procure NVIDIA-based capability that can also dramatically increase performance through CUDA.
Install a very large disk drive. Large disk drives are faster than smaller drives, because the average access time given is calculated over a larger capacity. It's better to install a larger, reasonably fast drive than a smaller drive that promises extra-fast transfer. Better still, get enough memory to never have to hit disk. An additional bonus to a large disk is that if you have 300 gigabytes of free space you will not hesitate to save interim versions of a project. Frequently saving projects is cheap insurance against undoable user errors and other time wasters.
Get a fast hard disk, spinning at 7200 RPM or faster and using ATA100 or more recent interface such as SATA. This is especially important if you will work with large images or drawings, which will involve a lot of disk accesses.
Install a second or third hard disk, so you can keep your Windows page file on a separate disk volume. If your entire job does not fit into available RAM Windows will have to swap it out to disk. Swapping often occurs at the same time that other disk accesses need to be made. With two disk drives in play Windows can launch the heads on one disk drive for read/write operations for swapping at the same time that the heads on the other drive are moving into position for other read/write operations, thus reducing the average access time. This is less important if you have lots of RAM, but as inexpensive as RAM has become disks can be even cheaper: it can be very inexpensive (under $50) to add a second, small, fast hard disk to host the Windows page file. An ideal situation is to have three disk drives, one for the page file, one for TEMP files and one for ordinary working files.
Think twice before buying SCSI. It's often faster to use the latest generation SATA drives and to spend the money saved over SCSI on larger drives and more drives (for independent action when using page files and temp files). Keep your system page file, TEMP folder and user profile directories on a drive separate from your working files.
Consider installing multiple disks in a striped RAID configuration for maximum disk throughput. For very large images and surfaces this might not be such an expensive option as it sounds, since large, fast hard disks can now be had for $100 each or less. Whether or not it is faster to use striped RAID than it is to use multiple hard disks for different Windows folders and page files is often a toss-up. Most technical users at manifold.net use striped RAID arrays when they want maximum speed because that requires less thought.
For 3D terrain windows get a fast graphics card with OpenGL support in hardware and lots of local graphics memory. These are now very cheap, with supercomputer-class GPUs and 256 MB of RAM being sold in graphics cards for as little as $100. It is critically important to use good drivers. Quite often the fastest drivers for your version of Windows will be supplied by the chip vendor who makes the graphics chip in your graphics card. This can make the difference between smooth "fly through" motion and very jerky a-few-seconds-per-frame motion in terrain views. Manifold requires a functioning OpenGL subsystem to display terrains. If there are no OpenGL capabilities in the system terrain windows will be blank when opened. See the discussion in the Terrains topic.
For maximum terrain viewing performance, use SLI-capable nVidia PCI Express graphics cards in an SLI-capable motherboard to team multiple graphics cards for rendering. Prices on graphics cards are dropping rapidly: as of this writing, installing two high-end, SLI-capable nVidia graphics cards with 256 MB of RAM each costs a total of $250, an amazing deal for the resultant throughput. It is often faster to use two cards via SLI than it is to spend disproportionately more money for a single card that uses the very latest, super-hot graphics chip. For example, two SLI cards using slightly downrev, but still awesome chips might be had for $150 each and the combination could end up being as fast as or faster than the latest superchip board at $750 each. Of course, if money is no object, get two of the latest boards!

Use a large video display with high resolution. Larger, high-resolution displays will allow you to keep more panes, toolbars and windows open at the same time. Work goes faster when all controls are in sight. If you are stuck with a small screen or low resolution, learn to use the ALT-SHIFT keyboard shortcuts to rapidly open and close panes. See the Windows topic for tips on keyboard shortcuts.
Get a flat panel LCD display that accepts DVI direct digital video input and install a graphics card with DVI output. The direct digital connection will drive the LCD flat panel at superb clarity and crispness to reduce eyestrain and fatigue, allowing you to work longer and more effectively.
Use a wheel mouse with additional buttons like the Microsoft Intellimouse. The additional controls can be very useful for navigating within windows.

Operating System and other Software

Very important: Go to the Windows Control Panel Display dialog Effects tab and make sure that the Show window contents while dragging check box is not checked. Checking this box will greatly slow down the system when displaying complex maps and images.
Use any version of Windows 7, Windows Server 2008, Windows Server 2003 or Windows XP. More recent Windows editions provide much better utilization of large memory than do Windows Me or Windows 98. Vista is great, truly superb, but it is not at the present writing as fast as XP.
Use a 64-bit processor and install an x64 Windows version such as Windows 7 x64, Windows Server 2008 x64, Windows Vista x64, Windows XP x64 or Windows Server 2003 x64. Run 64-bit Manifold x64 editions.
Install the latest updates and service packs for the software you are using. Windows, DBMS vendors and others spend billions of dollars on improving the quality and performance of their software by issuing updates, often free updates. Take advantage of that.
Use NTFS for the file system. Do not use FAT or FAT32 (FAT file systems have serious drawbacks in terms of security, safety and performance).
Avoid creating many partitions. Ideally, set up one partition for the entire drive C:.
Set up a fixed-size Windows page file that is far larger than ever will be necessary, several gigabytes or larger in the case of projects involving very large images. This is faster than a dynamically re-sized page file. In Windows 2000 open System in the Control Panel. In the Advanced tab click Performance Options and under Virtual memory click Change. Set the Initial size the same as the Maximum size. In Windows XP, open System in the Control Panel, then in the Advanced tab's Performance section click Settings. In the resulting Performance Options dialog's Advanced tab's Virtual Memory section click Change and then set the Initial size the same as the Maximum size. In both cases the size should be very large, over 2000 MB if possible.
Allow a large enough page file for multiple instances of Manifold if you will be launching more than one instance of Manifold and copying and pasting between them. More than 2.5 gigabytes per Manifold instance is not necessary.
Install the latest version of Internet Explorer. IE installs fresh versions of Windows system facilities such as VBScript and JScript scripting engines, the XML parser and other facilities that are used by Manifold. As newer versions of IE install faster and better facilities, your Manifold installation will also become faster and better.
Avoid running other applications in background.
Minimize the use of memory-resident services. Keep only absolutely vital services on the traybar. The only program sitting on traybars at manifold.net is a readout for system time. A few systems have a volume control icon and some systems have an icon indicating the status of a local SQL Server or Oracle DBMS server. Compare that to the collection of seven or eight icons often seen in traybars when many intrusive, memory-resident consumer applications have been installed.
Very important: Do not operate a virus checker in real-time scanning mode. Instead, schedule a regular weekly or nightly virus check. Real-time virus scanning will have a serious impact on system performance because the virus scanner will analyze all of the many temp files created by Manifold in the normal course of work. This tip is especially important when working with large images or surfaces.
When running WinAmp, Windows Media Player or other audio players at the same time as Manifold, switch off visualization modes to reduce processing overhead. Switch off the equalizer or effects as well if you need every processor cycle. If you do not have enough RAM to avoid frequent paging to disk, run your audio playlist over your local network using a different machine as a disk server to avoid tying up the local disk.
If you have multiple hard disks, keep your TEMP directory and page file on one disk and your applications and .map files on a different disk. When working with very large images and surfaces it helps to keep the TEMP directory and system page file on physically separate hard disks. This allows the disk drive heads to seek independently from each other as various files are used. This is less of a factor if you have so much RAM that disk accesses are minimized, and more of a factor when working with large projects that don't fit into RAM.
When working with server-based OLE DB providers such as SQL Server or Oracle, users are strongly encouraged to maintain primary keys in all tables linked into the Manifold project. A side effect of how such servers interact through OLE DB is that if the table does not have a primary key, performance will be greatly reduced.
Use ADO .NET to connect to DBMS servers instead of ODBC or OLE DB. Use the native Oracle Call Interface (OCI) when connecting to Oracle.
If you have installed a CUDA-capable NVIDIA graphics engine, make sure you also install the CUDA libraries so that Manifold can take advantage of the graphics engine to accelerate computation. CUDA libraries are a built in part of NVIDIA-supplied GeForce drivers, but might not be a part of default Windows drivers. Install the latest drivers from NVIDIA to be sure.
Learn how to operate and administer Windows proficiently. Use a good Internet search engine (such as google.com) to find web sites that teach how to tune your Windows system for maximum performance. If your Windows system is tuned for faster operation then applications that run within Windows, such as Manifold System, will also be able to run faster. Simple Windows maintenance such as defragmenting hard disk storage can result in noticeable improvement in Windows performance.

Manifold System

Run 64-bit x64 editions of Manifold System within 64-bit x64 Windows.
Turn on the Render data progressively option in the Tools - Options dialog (on by default). This allows the system to be responsive while large drawings, themes, images and surfaces are rendered in stages. Quite often we are rendering a drawing simply to orient ourselves with the attention of zooming in to some smaller region, so being able to begin the zoom command before the drawing has finished rendering is a big time saver. When a component is small enough to render very quickly, progressive rendering will not be noticed because the component will be rendered in a single pass. Note: Lengthy operations such as some transforms automatically suspend progressive rendering and resume it upon completion, so that the system does not lose time rendering what might be obsolete data.
Re-project drawings, images, and surfaces into the same projection specified in maps that use these drawings, images and surfaces. All projection parameters must be the same for this to help and not just the name of the projection in use. This is critically important when working with very large images and surfaces. If this is not done then everything from displaying maps in their own window or displaying a layout that includes the map can end up being very slow.
If a map will consist of a large image (or surface) and several drawings in a different projection, create the map using the image and then add the drawing layers. The map will then use the same projection as the image. Large images and surfaces are much slower to re-project on the fly than are drawings, so it is faster for the map to use the same projection and to re-project the drawings for display than the other way around. When working with large images, it is critically important to use the projection of the image for any maps that contain that image.
Turn off any layers you don't need to see in a map.
Close any unnecessary map, drawing, or image windows. Every window that is open will need to be redisplayed on any changes.
When doing analytic work that does not need much visual interaction, use smaller map windows and zoom far into the map so relatively few objects are on screen. The fewer objects that are visible, the fewer need to be computed for rendering and redrawn.
Uncheck View - Refresh Auto so no refreshes occur unless you command them with Refresh.
Don't use high-resolution data when you will zoom out to levels where the details blur together. Resample the data with Normalize Topology (for drawings) using a lower Location Precision value to create a lower resolution equivalent that makes sense at the zoom level you need. For example, don't use high-resolution shoreline data if the map will show an entire world at once. The details will not be visible on screen, but Manifold will still need to re-compute the display based on a huge level of detail. Working with lower resolution data will also make many commands, such as Dissolve , operate much faster. Many drawings, such as those destined for thematic presentation, can be made ten times less complex/smaller without any objectionable visual effects.
Don't use raster images when drawings are more accurate, faster, and provide more information content. See the Images can be Inefficient essay.
When working with images stored in "lossy" formats such as .jpeg, resample (resize) the image down to the level of resolution truly kept with the .jpeg compression. Expanding the compressed image to a large size with many pixels in X and Y is a mirage, because the detail for high resolution has already been lost. You may as well enjoy the faster speed obtained by using image at a pixel resolution commensurate with the true image information it actually contains.
If you only need to work with part of a large data set, take a moment to cut that part out as a separate drawing or image or terrain and then work with only the necessary part.
Uncheck the Preview box in image commands with large images if you don't need to see a preview.
RGBa images are larger and take more time to display than ordinary RGB images. Don't use RGBa images unless you need to use RGBa pixel transparency , or if ordinary invisible pixels within an RGB image can do the job.
Use Zoom Ranges to present the level of detail required - Suppose we would like to show a national map for user orientation when zoomed out but we would also like to show very detailed shorelines when users zoom in. Create a map using two drawings: a highly detailed drawing plus a second, low resolution drawing created from the first by generalizing with Normalize Topology to a lower Location Precision value. Assign zoom ranges to the drawings so that the low-resolution drawing is the only drawing displayed when zoomed out and the high-resolution drawing is the only drawing displayed when zoomed in. When users browse the map in a zoomed out view the display will be fast because a fewer number of coordinates need be used to display the low-resolution drawing. When users are zoomed in the high-resolution drawing will become visible; however, only a portion of the high-resolution drawing will be seen in a zoomed in view so the display will still be fast.
Store the .map file used on a fast, local hard disk - When Manifold operates it accesses the .map file and any temp files created. If the .map file is located on a slow hard disk or on a different machine that must be accessed via a local area network then performance will not be as fast as if the .map file was immediately available on a local, fast hard disk. If you have a very fast network and fast servers this effect might not be significant.
When importing drawings, don't import data fields that will never be used. Many data fields will slow the system down. If a drawing has been imported with superfluous data fields, edit tables using Design to eliminate the unnecessary fields. When working with commands such as Dissolve , check the Transfer Rules for the tables being used to make sure that no fields are being transferred that are not necessary to transfer.
Uncheck the Compress .map files to save space option in Tools - Options to eliminate .map file compression. This will result in larger .map files on hard disk, but saving .map files and opening .map files usually will be a much faster process. Oddly enough, if a project includes very large images or surfaces and you have a very fast processor, it may be faster to use compression because the time to fetch and decompress a smaller sized file may be shorter than the time required to fetch a larger, uncompressed file.
Always acquire and install the latest edition of Manifold System or the latest Service Pack. Each new build of Manifold System includes optimizations and other improvements. Some optimizations will result in dramatically faster performance in the functions that have been optimized.
When working with large images and surfaces, keep the image or surface stored in a Manifold .map file. When a large image or surface is opened for the first time after it is imported Manifold will build a series of intermediate views that are used for faster panning and zooming at less than full resolution. Building these intermediate views takes time, so the first time a large image or surface is displayed the window will open very slowly. After that, viewing will be fast. If the image or surface is stored in a .map file, the intermediate views will be stored in the .map file as well (as a built-in part of the saved image or surface). Opening an image or surface already saved in a .map is therefore fast even the very first time the image or surface is opened.
When working with large images, if they need only to be viewed in a read-only way, consider storing them as compressed images so they load and display nearly instantly.
Store large images within a fast DBMS server such as DB2, MySQL, Oracle, PostgreSQL or SQL Server.
Complex queries take longer to accomplish than simple queries. A SELECT clause with some simple computations in the WHERE clause will scale linearly; that is, handling 100,000 records will take about 10 times longer than handling 10,000 records. A SELECT with a GROUP BY or ORDER BY clause will scale slightly worse, perhaps using n log n scaling where handling 100,000 records may take, for example, 12 times longer than handling 10,000 records. Joined SELECTs with spatial computations may scale even worse.
When re-projecting using the Clip coordinates option, make sure to read and take heed of the notes at the end of the Orthographic topic.
Use native tables instead of linking tables. Tables within the Manifold project are the fastest, shared tables (with caching turned on) from an Enterprise server will be slightly slower and linked tables from an external DBMS provider are the slowest. Retrieving data from a table linked to a fast external data source will take at least 10 times as long as retrieving the same data from a table inside the Manifold project.
DBMS software other than SQL Server may not always favor OLE DB over ODBC. Some databases have slow OLE DB drivers and fast ODBC drivers, and other databases (like SQL Server) have fast OLE DB drivers and slow ODBC drivers. The performance of an ADO.NET provider might or might not be better than OLE DB or ODBC drivers, depending on whether the provider is entirely implemented in managed code or not and on a multitude of other factors.
When using external DBMS tables, use the fastest connection possible, which may vary depending on the DBMS in use. For example, when connecting to Microsoft's SQL Server, ADO.NET is faster than OLE DB and OLE DB is faster than ODBC. ADO.NET is much faster than ODBC. Consider that connecting to a remote SQL Server can take much longer than connecting to a local SQL Server. For example, suppose that connecting to a table inside the Manifold project takes 1 unit of time. A very rough guide to equivalent times to access the same table using different connection and DBMS technologies is listed in the accompanying table. Clearly, it is very unwise to connect to remote SQL Server installations using ODBC. Important: Tables linked from ADO .NET data sources are always read-only. If read-only access is acceptable, ADO .NET is often the fastest possible connection and should be used in preference to ODBC or OLE DB for that reason.

Time	Source for Table
1	Native table (inside the project)
8	Linked from a local MDB file
10	Linked from a local SQL Server via ADO.NET
40	Linked from a local SQL Server via OLE DB
10	Linked from a remote SQL Server via ADO.NET
250	Linked from a remote SQL Server via OLE DB
6000	Linked from a remote SQL Server via ODBC

There is no performance difference between storing geometry in external tables (linked drawings) or internal geometry, once external geometry is refreshed. Refreshing geometry from an external table will likely to take as much time as it takes to run a fairly modest SELECT query, with the external data source being the bottleneck.
When querying external tables, running a Manifold query analyzes the entire table and fetches each record from the server, so it might sometimes be faster to reduce the number of records to be analyzed with a server query. The performance gain will depend on the relative complexity of the server query compared to the Manifold query. The more records that can be trimmed on the server side, the better the performance. Transforming data on the server (as opposite to trimming the number of records) will not gain anything.

The User

Read the documentation and learn as much as you can so you are always using optimal methods.
Take a few minutes to think about a task before launching into it. There are often many different paths to the same end within Manifold. A good plan will help you choose the best path and avoid unnecessary work.
Don't fall into the trap of making projects more complicated than they need to be. Most GIS projects follow the 80 / 20 rule, where 80% of the desired result comes from 20% of the implementation effort. Do that 20% first and then try it out. You might find that you are happy with the result.
Try to find a pre-existing data set that provides what you need before you invest time into building your own. Some people waste weeks of time digitizing raster scans to get a local map without realizing they could download 1:24,000-scale SDTS maps from USGS that provide the same thing for free.
Try to find the data you need in modern formats that automatically convey all necessary information in a user-friendly way. Dealing with improperly used antique formats like shapefiles or non-GIS formats like DXF can be a huge waste of time if you can find the same data in a modern format allows instant, automatic usage. See Projections and Legacy Formats for a discussion of hassles that can be avoided.
Become expert at using image servers to fetch satellite imagery on demand.
Save your work regularly in separate projects so you never have to waste time redoing an entire project after a system crash or undoable user error at some stage.
Memorize keyboard shortcuts and use them in combination with mouse moves. For example, experts will keep the left hand on the keyboard for CTRL-C and CTRL-V to Copy and Paste while the right hand moves the mouse to select items and manipulate windows and other mouse-based controls. The ALT-SHIFT keyboard shortcuts used to open and close Manifold panes are especially important if you have a small screen. See the Windows topic for tips on keyboard shortcuts.
Learn to write scripts. Automating a task so that it takes care of itself while you are at lunch or at home is a wonderful time saver. At times a very simple script can replace a long sequence of commands using pre-built tools. When scripting, write the simplest code that works and use it. Write scripts so their internal functioning is obvious and include plenty of comments to boot. The objective is simplicity and maintainability.
Sometimes you may be called upon to do a job under time pressure. The more time pressure you feel to complete a project, the more important it is to work systematically, steadily and carefully. Don't panic. Take it step by step in a steady pace. If you are short on time you don't have time for errors. Measure twice, cut once.
Get plenty of sleep and exercise regularly. Fatigue causes errors and panic. Good health will help you think clearly and execute with authority.

People are sometimes amused that we include the user as part of our performance tips. However, the greatest gains in performance are usually achieved by using a better method or algorithm. More often than not the sole factor in whether a better method is used is the expertise and clarity of mind that can be mustered by the user. A healthy, well-rested, expert user is the best performance accelerator around.

General Education

Manifold works with many standard Microsoft facilities and products as well as with other industry standard products. Here are some examples drawn from many such features:

Built into Manifold are the standard dialogs we would use to connect to data sources using ADO .NET, OLE DB and other such connection technologies.
The Manifold Internet Map Server works with standard Microsoft Internet web serving technologies such as Internet Information Server (IIS), ASP and ASP .NET.
We can write scripts in Manifold using standard Microsoft .NET languages or ActiveX scripting languages.
We can use standard Structured Query Language (SQL) to write queries.
Manifold can work with database products like SQL Server or Oracle.

One of Manifold's great strengths is exactly that Manifold uses a wide variety of industry standard products and technologies. Doing so makes it possible for people to apply in a familiar way within Manifold the expertise they have already acquired through the use of standard Microsoft products, and it also makes it possible for Manifold applications to leverage the power and flexibility of many other products and technologies in an industry-standard way.

However, this documentation does not teach such industry-standard products and technologies: it is assumed that if users want to employ such things they either already know how to use them or they have the ability to take advantage of the vast number of publications, web pages and training products that already exist for such industry standard products and technologies. There is no point in duplicating within this documentation those educational and support resources that already exist to a much greater degree.

Therefore, if we want to use a technology such as ADO .NET or ASP .NET or SQL Server we can certainly do so with Manifold. But if we don't know enough about such technologies to use them we should browse the web or go to our local high tech book store and buy one of the hundreds of books that teach the use of such technologies and then dig in and learn how to do so. Part of the point of using such technologies is exactly that they are already very well documented in a huge selection of publications.

Another benefit to using such standard technologies is that if we are unfamiliar with them and then have to learn about them to use them with Microsoft the education we invest into ourselves is not restricted to just Manifold. Learning about something like SQL Server or Oracle or ADO .NET will be useful knowledge in the many thousands of other applications that use such standard things.

Systems

Manifold is developed on Intel and AMD processors with average performance and not on unusually powerful machines. Most development is done using Windows Vista, Windows XP and Windows Server 2003. Faster machines are deliberately avoided to force a conservative perspective into development for core system functions. Given enough RAM for the task, required performance with almost any reasonably contemporary systems will be acceptable and performance on better than average systems will be stellar.

As processor speeds continue to improve performance on state-of-the-art machines will become dazzlingly fast. Machines with 4 to 8 gigabytes of RAM, many hundreds of gigabytes of disk drives, 64-bit quad core processors and CUDA-enabled nVidia graphics engines with over 100 stream processors have become very affordable. RAM, in particular, has become almost dirt cheap so there is no excuse for not equipping one's machine with gigabytes of RAM.

See the essay on Using RAM and other Machine Resources for the Manifold "spin" on this topic.

Very Large Jobs

No matter how fast Manifold can operate it will always be possible to ask Manifold to perform a task that will take a very long time to accomplish. Some jobs can seem to take an unreasonably long time if, without realizing it, we have suddenly increased the amount of data involved as compared to previous tasks.

Raster data and images can involve dramatically greater work for what appears to be a small increase in image size because the data in images increases as the square of any increase in height and width of the image. An image that's 40% larger takes 100% more work to manipulate. An image that is twice as big will take four times the work. For raster data such as DEMs it is easy to forget that scaling up from a 100 x 100 image to a 1000 x 1000 image will end up requiring 100 times as much work. What used to take one second can take 100 seconds with the larger image.

Many networking and geometric problems involve geometric growth in computation requirements. What appears to be a small increase in the problem can increase computation time from seconds to days. For example, growing the size of a road network so that it covers an entire state instead of a few counties could lead to a tenfold increase in the height or width of the map. The additional area of the map can easily add 100 times as many road nodes and links and thus lead to a million-fold increase in computation time for certain complex network tasks.

When performing computations that grow geometrically it is important to increase the size of the task in small steps. Begin by verifying your procedure with a very small subset of the data and then increase the size of the problem in small steps so you can see where asymptotic growth in computation requirements begins. Note also that increasing the size of jobs will likely also place greater demands on system RAM.

As memory requirements increase, at some point a machine will run out of available RAM and begin paging to disk. At that point processing will become profoundly slower. See the Using RAM and other Machine Resources essay for why. To avoid this effect, use a machine that is better scaled to the task at hand or reduce the size of the job to what will fit into the machine being used. It is unrealistic to expect a machine with 128MB of RAM to be able to process large tasks as efficiently as one equipped with 8GB of RAM.

Some specialized tasks with large maps will take days to accomplish. People worldwide launch such jobs every day with Manifold using a spare machine that is left "cooking" for a few days to accomplish a desired task. They're happy because such things used to require weeks with older software or hardware. Run some experiments before launching such large jobs so you know what to expect.

Memory and Large Files

When Manifold opens a very large .map file not all of the contents of the .map are brought into memory at once. Components will be brought into memory as needed from the .map file stored on disk. Once they are in memory accesses will occur faster than in the initial usage since, of course, RAM memory is thousands of times faster than hard disk.

The advantage of having lots of RAM in a computer is that Windows editions such as Windows XP or Windows 2000 will leave items in memory until the memory is needed for something else by Windows. If we have ample RAM, as we work with a project the various components will end up in RAM and will stay there. This effect is especially pronounced with x64 Windows editions, which can actually work with large amounts of RAM effectively.

If you do not have enough RAM to run projects in memory, no matter how fast a processor you have you will have much less performance if the system begins paging to disk. That is why RAM is more important than processor speed until you have enough RAM so you never have to page to disk.

Notes

By mentioning certain brands we don't mean to imply anything negative about other brands. At this writing, the current generation of machines being installed at manifold.net use nVidia graphics engines and mostly Intel processors. nVidia has done a super job of writing good code for Windows and creating innovations like CUDA, which currently provides the fastest general-purpose processing around.

However, Intel and AMD's ATI divisions continue to be a major power in processors and no doubt will respond to nVidia's competitive stimulus with advances of their own. Likewise, we expect that NVIDIA's masterful driver work will inspire other processor vendors to pay better attention to writing effective drivers. Leadership positions in CPUs and graphics engines can change overnight, and often have.

From a software perspective, we love seeing the processor and graphics vendors work harder to offer more performance at lower price!