ArcSWAT Bug – Index was outside the bounds of the array.: IN, mWriteInputFiles.sol

Description

  • ArcSWAT 2012.10.15
  • SSURGO Soil Database is used
  • Following error message is given when running Create Tables

Reason

When generation sol table, ArcSWAT would do following work.

  1. Get all unique soil IDs from hrus table in project database.
  2. Create a new table (tbSoilList) in SSURGO database (which is usually located in C:\Swat\ArcSWAT\Databases\SWAT_US_SSURGO_Soils.mdb) and copy all soil IDs into this table.
  3. Find soil parameter for each soil ID from table SSURGO_Soils.
  4. Write the soil parameter into sol table.

The error message comes from Step 4. ArcSWAT doesn’t check the case when the soil ID couldn’t be found in SSURGO_Soils table. In that case, the returned soil parameter will be an empty array. When trying to read the first element of this array, it would give an error message shown above.

Solution

Check soil shapefile/grid to make sure all soil IDs have been defined in SSURGO_Soils table. And redo the Land Use/Soil/Slope Definition.

Note: This should also work for other soil options (user soil lookup table or STATSGO) and other ArcSWAT version.

Advertisement

Interesting Test – Install ArcSWAT for ArcGIS 10.0/10.1 on ArcGIS 9.3

Interesting Test – Install ArcSWAT for ArcGIS 10.0/10.1 on ArcGIS 9.3

Method in post “Manually Install ArcSWAT” was used to install ArcSWAT for ArcGIS 10.0/10.1 on ArcGIS 9.3. The installation is successful but there are some problem in stream network generation (no problem in flow direction and accumulation calculation). The reason is the new version manager system in ArcGIS 10 (new file ArcGISVersion.dll located in C:\Program Files (x86)\Common Files\ArcGIS\bin). However, it would be enough for an existing model.

Manually Install ArcSWAT

Quick Download Link

Extract Installation File

ArcGIS 9.3: Register, Unregister

ArcGIS 10.0 and higher: Register, Unregister

Note:

  1. The bat file needs to be saved in the same folder as installation file or .NET assemblies.
  2. The register and unregister bat file needs to be run as administrator.
  3. Since there will be some errors, the method in this post could also be used to install ArcSWAT for ArcGIS 10.0/10.1 on ArcGIS 9.3. 

Contents

  • Why bother to install ArcSWAT manually?
  • Important Facts
  • Extract ArcSWAT program files from installation file
  • Copy new files to existing ArcSWAT installation folder
  • Re-register ArcSWAT .NET assemblies
  • Unregister previous ArcSWAT if necessary

Why bother to install ArcSWAT manually?

  1. ArcSWAT is updated frequently and must be re-installed to use the new functions. Uninstalling the previous version and installing the new version would need around ten minutes.
  2. You may experience problems when trying to uninstall and install ArcSWAT. These problems are usually difficult to solve. Having seen some these problems in ArcSWAT user group. In some cases, it couldn’t be solved until restore the entire Windows system.

Important Facts

  1. There are usually two steps involved in program installation: 1) copy files to specific folder and 2) register some file in the system if necessary.
  2. The core of ArcSWAT is six .NET assemblies and SWAT executable file. When a new version comes out, it means one or more of these assemblies are updated. Other files are usually kept same between different versions. Of course, they would be updated once a while.

So, most of the time, upgrading ArcSWAT only means updating these six .NET assemblies and SWAT executable file. Knowing this, it’s possible to upgrade ArcSWAT without normal installation. It could be done use following steps:

  1. Get the core files
  2. Replace old files
  3. Re-register the six .NET assemblies

Extract ArcSWAT program files from installation file

There is a MSI file, namely SWAT_Install.msi, in the installation package downloaded from SWAT website. It contains all the files which will be copied to the installation folder. We will extract all the files from it.

Execute following command in cmd window.

msiexec /a [SWAT_Install.msi_path] /qb TARGETDIR=[extraction_folder_path]

Where

SWAT_Install.msi_path is the path to SWAT_install.msi

extraction_folder_path is the path to the extraction folder where the extracted files will be saved.

Following window will show up to display the progress.

Wait it to finish and all the files will be extracted to the given folder. The extracted files are around 567 MB.

A bat file could be used to do this work. Create a text file in the folder, copy following commands to the file and save it as a bat file in the same folder as the installation file. Then double-click to run the commands. This bat file also could be downloaded here.

msiexec /a SWAT_Install.msi /qb TARGETDIR=%CD%\extract

Copy new files to existing ArcSWAT installation folder

If a previous version is installed in the system, copy following files into the installation folder to replace the existing files.

  • SWAT_PM2.dll
  • SWAT_WD.dll
  • SWAT_HRU.dll
  • SWAT_InputFileGen.dll
  • SWAT_InputFileEdit.dll
  • SWAT_RunSWAT.dll
  • SWAT_32debug.exe
  • SWAT_32rel.exe
  • SWAT_64debug.exe
  • SWAT_64rel.exe
  • swat2012.exe

Please note that the name of SWAT model executable files may be different for different version. There usually four versions: 32-bit debug, 32-bit release, 64-bit debug, 64-bit release.

Re-register ArcSWAT .NET assemblies

As mentioned in my previous post ArcSWAT Structure – Developer view, the ArcSWAT extension, toolbar, menus and windows are defined in six .NET assemblies. To use the new function in the new version, they must be registered in the system.

The register process is based on the version of ArcGIS.

  • ArcGIS 9.3

Run cmd as administrator and navigate to .NET folder (which is usually located at c:\Windows\Microsoft.NET\Framework\v2.0.50727). Execute following command for each of the six ArcSWAT .NET assemblies. It’s done when you see “Type registered successfully”.

regasm [assembly_path] /codebase

where

assembly_path is the path to the ArcSWAT .NET assembly, e.g. c:\swat\ArcSWAT\SWAT_PM2.dll

The same as extracting files, a bat file could be created using following commands to do the work. An existing bat file could be downloaded here. Please not that the bat file needs to be in the same folder and run as administrator.

SET DLL_DIR=%~dp0

CD %SystemRoot%\Microsoft.NET\Framework\v2.0.50727

regasm %DLL_DIR%SWAT_HRUs.dll /codebase

regasm %DLL_DIR%SWAT_InputFileEdit.dll /codebase

regasm %DLL_DIR%SWAT_InputFileGen.dll /codebase

regasm %DLL_DIR%SWAT_PM2.dll /codebase

regasm %DLL_DIR%SWAT_RunSwat.dll /codebase

regasm %DLL_DIR%SWAT_WD.dll /codebase

  • ArcGIS 10.0 and higher

Run cmd as administrator and navigate to ArcGIS bin folder (which is usually located at C:\Program Files (x86)\Common Files\ArcGIS\bin). Execute following command for each of the six ArcSWAT .NET assemblies. It’s done when you see a window saying “Registration succeeded”.

esriregasm /p:desktop
[assembly_path]

where

assembly_path is the path to the ArcSWAT .NET assembly, e.g. c:\swat\ArcSWAT\SWAT_PM2.dll

Similar as ArcGIS 9.3, a bat file could be utilized to do the work on just a click. Please note the location of esriregasm.exe may be different on your computer. For 32-bit system, it should be C:\Program Files\Common Files\ArcGIS\bin. Check the file location and modify this file before use if necessary.

SET DLL_DIR=%~dp0

CD C:\Program Files (x86)\Common Files\ArcGIS\bin

esriregasm /p:desktop %DLL_DIR%SWAT_HRUs.dll

esriregasm /p:desktop %DLL_DIR%SWAT_InputFileEdit.dll

esriregasm /p:desktop %DLL_DIR%SWAT_InputFileGen.dll

esriregasm /p:desktop %DLL_DIR%SWAT_PM2.dll

esriregasm /p:desktop %DLL_DIR%SWAT_RunSwat.dll

esriregasm /p:desktop %DLL_DIR%SWAT_WD.dll

Unregister previous ArcSWAT if necessary

If there is a previous version installed, it needs to be unregistered first. This process also depends on ArcGIS version.

  • ArcGIS 9.3

Run cmd as administrator and navigate to .NET folder (which is usually located at c:\Windows\Microsoft.NET\Framework\v2.0.50727). Execute following command for each of the six ArcSWAT .NET assemblies. It’s done when you see “Type un-registered successfully”.

regasm [assembly_path] /unregister

where

assembly_path is the path to the ArcSWAT .NET assembly, e.g. c:\swat\ArcSWAT\SWAT_PM2.dll

A bat file is also available here with following commands, which should also be executed as administrator.

SET DLL_DIR=%~dp0

CD %SystemRoot%\Microsoft.NET\Framework\v2.0.50727

regasm %DLL_DIR%\SWAT_HRUs.dll /unregister

regasm %DLL_DIR%\SWAT_InputFileEdit.dll /unregister

regasm %DLL_DIR%\SWAT_InputFileGen.dll /unregister

regasm %DLL_DIR%\SWAT_PM2.dll /unregister

regasm %DLL_DIR%\SWAT_RunSwat.dll /unregister

regasm %DLL_DIR%\SWAT_WD.dll /unregister

  • ArcGIS 10.0 and higher

Run cmd as administrator and navigate to ArcGIS bin folder (which is usually located at C:\Program Files (x86)\Common Files\ArcGIS\bin). Execute following command for each of the six ArcSWAT .NET assemblies. It’s done when you see a window saying “Unregistration succeeded”.

esriregasm /p:desktop /u
[assembly_path]

where

assembly_path is the path to the ArcSWAT .NET assembly, e.g. c:\swat\ArcSWAT\SWAT_PM2.dll

Similarly, a bat file could be used.

SET DLL_DIR=%~dp0

CD C:\Program Files (x86)\Common Files\ArcGIS\bin

esriregasm /p:desktop /u %DLL_DIR%SWAT_HRUs.dll

esriregasm /p:desktop /u %DLL_DIR%SWAT_InputFileEdit.dll

esriregasm /p:desktop /u %DLL_DIR%SWAT_InputFileGen.dll

esriregasm /p:desktop /u %DLL_DIR%SWAT_PM2.dll

esriregasm /p:desktop /u %DLL_DIR%SWAT_RunSwat.dll

esriregasm /p:desktop /u %DLL_DIR%SWAT_WD.dll

ArcSWAT Structure – Developer View

ArcSWAT is the standard SWAT interface which is used by most of the modelers. Knowing its structure would help understand what’s going on under each command and locate the possible exceptions more easily, which in turn would help improve model itself.

Key facts about ArcSWAT

  • ArcSWAT is ArcGIS Extension
  • ArcSWAT defines toolbar and menus
  • ArcSWAT is developed with .NET framework
  • ArcSWAT needs to be registered in system before use

ArcSWAT is ArcGIS Extension

An extension is a class which implements interface IExtension and IExtensionConfig.

ArcSWAT defines toolbar and menus

A toolbar is a class which implements interface IToolBarDef.

A menu is a class which implements interface IMenuDef.

A menu item is a class which implements interface ICommand. The method OnClick will be executed when the menu item is clicked.

ArcSWAT is developed with .NET framework

There are six .NET assemblies in ArcSWAT. They are corresponding to six menus in ArcSWAT toolbar and located at ArcSWAT installation folder.

  • SWAT_PM2.dll, project management, corresponding to “SWAT Project Setup” menu in ArcSWAT toolbar.
  • SWAT_WD.dll, watershed delineation, corresponding to “Watershed Delineator” menu in ArcSWAT toolbar
  • SWAT_HRU.dll, HRU definition, corresponding to “HRU Analysis” menu in ArcSWAT toolbar
  • SWAT_InputFileGen.dll, input generation, corresponding to “Write Input Tables” menu in ArcSWAT toolbar
  • SWAT_InputFileEdit.dll, input edit, corresponding to “Edit SWAT Input” menu in ArcSWAT toolbar
  • SWAT_RunSWAT.dll, swat simulation, corresponding to “SWAT Simulation” menu in ArcSWAT toolbar

ArcSWAT needs to be registered in system
before use

To use ArcSWAT, all six assemblies must be register in system, which is automatically done during installation.

The registration process could also be used to manually install ArcSWAT or fix problem of ArcSWAT installation.

Find Possible Solution for ArcSWAT Errors with ildasm.exe

For more details about ArcSWAT structure, please check out my another post: ArcSWAT Structure – Developer View.

Please Note:

  • SWATEditor and SWAT Check are also .NET programs. The method post here could be also used on them.
  • All the analysis and screenshots are based on ArcGIS 2009.10.1. It’s similar to other versions.
  • MSIL code given by ildasm.exe is not easy to read. For more readable solution, please consider to use free .NET decompilers out there. They could decompile .NET in C# which is more easier to read. I’m using dotPeek from JetBrains. The only problem is some methods couldn’t be decompiled in these tools and could only be analyzed in ildasm.exe.
  • Welcome to any discussions on this subject. Please send me email on hawklorry@gmail.com.

Contents

  • ildasm.exe
  • Where are the ArcSWAT assemblies?
  • Which assembly should be looked into?
  • Find codes corresponding to one ArcSWAT menu item
  • Find out what happened when one ArcSWAT menu item is clicked
  • What if error happens in a window pop-up by ArcSWAT?
  • Two kinds of errors

ArcSWAT is the most popular SWAT interface. It’s free-to-use but not open source. When something unexpected happened (like the one shown below), it’s not possible to look into source codes like SWAT model itself (please refer to my another post: Compile and Run SWAT with Photran + MinGW – the easiest way to remove mathematical errors from SWAT). So, how to find possible solutions?

The answer is decompiling/disassembling.

ArcSWAT consists of six .NET assemblies (shown below), which are corresponding to six menus in ArcSWAT toolbar (shown below). Decompile/Disassemble would show background codes in C# or Microsoft intermediate language (MSIL) code format, from which the background processes under all ArcSWAT menu could be analyzed and possible solutions could be reached.

ildasm.exe

ildasm.exe is a tool in Visual Studio and Windows SDK. It could parse any .NET Framework .exe or .dll assembly, and shows the information in human-readable format. A simple tutorial could be found here: http://msdn.microsoft.com/en-us/library/aa309387(v=vs.71).aspx. If it’s impossible to install Visual Studio, please get Windows SDK at http://www.microsoft.com/en-ca/download/details.aspx?id=8279 for Windows 7 and .NET 4.0. It’s free-to-use.

Where are the ArcSWAT assemblies?

They are located in ArcSWAT installation folder, which is usually c:\swat\ArcSWAT.

Which assembly should be looked into?

From ArcSWAT toolbar, the analysis assembly could be determined.

  • SWAT_PM2.dll, project management, corresponding to “SWAT Project Setup” menu in ArcSWAT toolbar.
  • SWAT_WD.dll, watershed delineation, corresponding to “Watershed Delineator” menu in ArcSWAT toolbar
  • SWAT_HRU.dll, HRU definition, corresponding to “HRU Analysis” menu in ArcSWAT toolbar
  • SWAT_InputFileGen.dll, input generation, corresponding to “Write Input Tables” menu in ArcSWAT toolbar
  • SWAT_InputFileEdit.dll, input edit, corresponding to “Edit SWAT Input” menu in ArcSWAT toolbar
  • SWAT_RunSWAT.dll, swat simulation, corresponding to “SWAT Simulation” menu in ArcSWAT toolbar

Find codes corresponding to one ArcSWAT menu item

One ArcSWAT menu item is corresponding to one class in one of the six assemblies. The name of these class would usually has a name starting with “cmb” or “com”. For example below, in SWAT_PM2.dll, there is class named cmbHelp. It’s corresponding to menu “ArcSWAT Help…”. To confirm this, expand class cmbHelp in ildasm.exe and double click on method get_Name to open it.

Considering name convention, it’s easy to find these classes just from the menu name.

Find out what happened when one ArcSWAT menu item is clicked

Method onClick is implemented in all classes corresponding to ArcSWAT menu items. The codes in this method will be executed when corresponding menu item is clicked. Double-click this method node in ildasm could open the codes. This is where should be looked into.

What if error happens in a window pop-up by ArcSWAT?

Most of the ArcSWAT menu item would pop-up a window and then further operation could be done in this window. A very good example is the “Watershed Delineation” window shown below.

In this case, the class corresponding to the window should be found first and then find the method corresponding to a specific button.

For “Watershed Delineation” window, follow steps below.

  1. Open SWAT_WD.dll with ildasm.exe.
  2. Find the class for menu “Automatic Watershed Delineation” which is “ComAutoDelin”.
  3. Expand class ComAutoDeline and double-click method “ICommand_OnClick” to open the codes window.
  4. Find the class name for the window In the codes window, which is frmMainForm. The name of classes corresponding to windows usually starts with “frm”.
  5. Find class frmMainForm in ildasm.exe and expand to check the methods. The name of methods corresponding to button clicks usually like “[button name]_click”, where button name is the name of the button. These methods are the codes corresponding to the buttons.

Two kinds of errors

Once the method body is found, possible solutions could be figured out through analysis of the codes. There would be two kinds of error.

  1. Data Problem ArcSWAT has special requirements on input data. Giving data in a different format or value would cause ArcSWAT give errors. Analysis on the codes given by ildasm.exe could help locate the problematic data.
  2. Software Problem ArcSWAT continues to be improved. There may be still some bugs there and would stuck you someday. Analysis on the codes could help locate the bug and report to the develop team.

Environment Canada Climate Data Reader

Please follow my blog to get the latest updates on ECReader as soon as they are available.

Please download the program package here and the quick guide is here. The .NET source codes is host in Google Codes.

Questions, bugs, more formats and functions, please send me email. I couldn’t be more happy to hear from you:)

Updates

Jan 10, 2014 Save and Load Defined Stations in ECReader1.1

Jan 6, 2014 File Name Convention in ECReader1.1

Jan 2, 2014  Version 1.1, Modify how to define query stations

Dec 28, 2013  New function to retrieve all EC climate stations and save as CSV file, based on which shapefile and kmz file could be also generated to help select climate stations in modelling.

Dec 20, 2013  Fix the uncompleted data bug.

What We Have

In Canada, Environment Canada is the major source of the climate data (precipitation and temperature). The climate data could be downloaded from Environment Canada website (http://climate.weather.gc.ca) for each year in CSV format which contains 27 columns data. To be used in ArcSWAT or SWAT, these files must be processed in following way:

  1. Extract min and max temperature and total precipitation from each file;
  2. Combine the data of different years from the same station into one dbf or txt file in a format required by ArcSWAT.

Let’s take a look. If 5 climate stations are located in/around the study watershed and the simulation period is 20 years, total 5*20 = 100 CSV files need to be downloaded first and then be processed into 5 dbf/txt files. It may take one or two days to finish this work. No modeler wants this.

Besides time consuming, this process is also error-prone. Large amount of files will be process in a same way. The brain and eye is easy to become tired. Wrong file may be processed or the same file may be processed twice. And the most important is that it’s difficult to find these errors. Modeler usually never check the climate data again after they are there, which leave it impossible to figure out climate data related problems in the model.

What We Could Have

Since the climate data preparation process involves many repeated work, a program is the natural solution. Select stations, click a button and then go to get a coffee. All the files are ready when you get back. Ideally? Yes, that’s the “Environment Canada Climate Data Reader” is trying to do.

Environment Canada Climate Data Reader, or just call it reader, is a .NET windows program designed to download climate data from Environment Canada and generate result files in a specific format (of course, it supports ArcSWAT dbf and txt format). With it, you could enjoy your coffee when doing workJ

Quick Guide

   1. Give Stations – Which stations will be used?

Station ID is the unique ID used by Environment Canada to identify each climate station. Before using reader, these IDs must be retrieved based on station name. If you don’t how to get the station id, just hit and a short tutorial will show you how to do that.

Already got station IDs? Let’s give the ID(s) to the reader first.

  • Only got one station? Good, just select option and input the station id in the box. Want to know the information of this station? Hit to check basic information of this station (to make sure correction station is used).

  • Got more than on stations? That’s normal. First prepare a csv file in Excel or any test editor, in which the first column is the station name and the second column is station id. It may look like this:

name,id

deerwood,29886

CSV file is ready? Select option and hit to load the file.

   2. Give Time Range – What’s your simulation period?

The simulation period may or may not be decided yet. No problem, just make it longer. ArcSWAT and SWAT would read the right data based on date.

   3. Select an Output Folder – Where do you want to save the result files?

The default output folder is the my document folder. Hit if you don’t like this. will open the given folder, which will allow you check the result files.

   4. Select an Output Format and Data Fields – What format do you prefer and Which data do you like?

Want to just use climate data in ArcSWAT? No problem, just select or and you are good to go (only min and max temperature and total precipitation will be exported).

Want some free air and do some analysis on the data? No problem, select or and select any climate data you want in the list.

Want the final SWAT format? Yes, for people who don’t use ArcSWAT, just select (only min and max temperature and total precipitation will be exported).

   5. Start Download and Drink Coffee

Ready? Hit to start the work. Progress message will be given on the bottom.

When you see , it’s done.

   6. Check Result Files

Curious about the results? Hit to open the output folder and open the result files. You may want to know the name convention for different format.

  1. P[Station ID].dbf for precipitation and T[Station ID].dbf for temperature, e.g. P29886.dbf and T29886.dbf.
  2. P[Station ID].txt for precipitation and T[Station ID].txt for temperature, e.g. P29886.txt and T29886.txt.
  3. [Station ID].txt, e.g. 29886.txt.
  4. [Station ID].csv, e.g. 29886.csv.
  5. pcp1.pcp for precipitation and tmp1.tmp for temperature.