Makefile – Compile SWAT using gfortran without modification

Note: The makefile here is an older version. For the latest version, please visit another post SWAT Makefile Updated – Stop running when overflow happens.

Quick Download Link

Makefile Generator, Debug Makefile, Release Makefile

Next post will be: Use Makefile in Photran to Debug SWAT

Please note that the modparm.f doesn’t need to be compiled as parm.mod any more. That process is already included in the Makefile.

Contents

  • Why some files fail to be compiled in Photran?
  • Use additional gfortran command options to compile problematic files
  • Makefile
  • Generate Makefile
  • The only modification
  • Use Makefile

In my previous post Compile and Run SWAT with Photran + MinGW – the easiest way to remove mathematical errors from SWAT, several swat source code files need to be modified to be able to compilable within Photran. It’s nice, but it’s even better if the source codes could be compiled without any modification.

Why some files fail to be compiled in Photran?

Compare the original source codes and the modified source codes through Google codes, and find that only following files need to be modified.

  • biozone.f
  • bmpinit.f
  • carbon_zhang2.f90
  • main.f
  • modparm.f
  • ovr_sed.f
  • percmain.f
  • rthsed.f

Except for main.f, these files have a common problem: one or more lines exceed the default line length set for Fotran fix format file (*.f) and free format file (*.f90). The default limitation of the line length for *.f is 72 and for *.f90 is 132. The default Fortran compile command (shown below) in Photran doesn’t consider this situable. If the length of one line exceed the limitation, the compiler (gfortran) would give errors.

  • Debug version: gfortran -funderscoring -O0 -g -Wall -c -fmessage-length=0
  • Release version: gfortran -funderscoring -O3 -Wall -c -fmessage-length=0

Use additional gfortran command options to compile problematic files

To solve this problem, these files are modified to make sure they meet the requirement on line length in previous post. Besides this, there is also another solution: change the line length limitation using gfortran command options.

  • Fixed format (*.f): add -ffixed-line-length-132 option to set the line length limitation of fixed format Fortran file to 132
  • Freeformat (*.f): add -ffree-line-length-200
    option to set the line length limitation of free format Fortran file to 200

For example, the gfortran command for debug version of biozone.f would be

gfortran -funderscoring -O0 -g -Wall -c -fmessage-length=0 -ffixed-line-length-132

And the gfortran command for release version of carbon_zhang2.f90 would be

gfortran -funderscoring -O0 -g -Wall -c -fmessage-length=0 -ffree-line-length-200

These files could be compiled successfully with the new command.

Makefile

In default Photran settings, it’s impossible to use different command options for different source files. Applying -ffixed-line-length-132 to other *.f files would also generate compile errors. How to solve this? Using Makefile.

Makefile defines how to compile and link a program. With it, it’s possible to define different commands for different source files and compile SWAT source codes without modification. For more information, please refer to GNU make document.

Generate Makefile

There are more than 300 Fortran source files in SWAT. It’s time-consuming work to write the Makefile manually. We already know the gfortran commands and the files which need to re-set the line length limitation. A program, which would get all *.f and *.f90 in the SWAT source codes folder and generate the compile commands in the makefile, is a quite straightforward and simple way to do the work.

Based on this, a .NET console application is created (source codes, executable). It’s fairly easy to use. Copy the program file to the source codes folder and double-click.

A prompt window would pop-up to ask which version should be the Makefile generated for, debug or release. Input d or debug for debug version, r or release for release version. Then the program would start to work and generate the corresponding Makefile in debug or release folder. The debug/release folder would be created if it doesn’t exist.

The two version of Makefile for rev 615 are also available to be download directly (debug version, release version). These two files could be also used for also version as long as the number and name of source codes files is same. Create the debug/release folder in the source codes folder and put the file into the corresponding folder.

The only modification

The only modification is to comment the first line of main.f (include modparm.f). There should be some command option to ignore this line but haven’t found it:)

Use Makefile

Makefile is fairly easy to use. Just use the GNU make command which comes along with MinGW. There are two version of make in MinGW: mingw32-make and make. Either one is ok.

Open cmd.exe and navigate to the Makefile folder. Execute mingw32-make or make. It’s done!

Advertisement

Eclipse Compatible SWAT Updated to Rev 614 – Fix ET > PET bug

SWAT model was update to rev 614 on Jan 7, 2014. Based on the difference between rev 613 and rev 614, the Eclipse compatible version is also updated. The source codes are here: https://code.google.com/p/swat-eclipse/.

List of Changes (click the file name to see detailed changes)

  • main.f The version number is updated.
  • etact.f Line 131 and Line 172-182, to make sure the actual ET doesn’t exceed PET. This would have impact on ET and surface runoff for some cases.

It’s highly recommended to update to this version.

Compile and Run SWAT with Photran + MinGW – the easiest way to remove mathematical errors from SWAT

Eclipse-friendly SWAT rev 613 codes: https://code.google.com/p/swat-eclipse/. TortoiseSVN is recommended to check out the codes.

Quick Download Link

Photran/PTP/Eclipse, MinGW

Or you could send me (hawklorry@gmail.com) your error message. I couldn’t be more happy to help.

Contents

  • Compiler and IDE
  • Create a Fortan Project
  • Download SWAT Source Codes and Add into Fortran Project
  • Compile modparm.f
  • Modify Source Codes to Compile
  • Debug/Release Version
  • Set Breakpoint and Run
  • Remove Static Library Dependency
  • How to Locate the Errors

Have seen some very basic mathematical errors (e.g. overflow) given by SWAT in SWAT user group, like this one here.

After 30 years development, SWAT has been a very complicated system. The latest version of SWAT (rev. 613) consist of 302 Fortran files. It’s very easy to import some small errors when adding new algorithms. And the developing team couldn’t test SWAT on any possible watersheds. After all it’s a model for research not for commercial.

When these small errors come out in your model, the first thing we need to do is to check the model data first. After that, if the errors still persist, while waiting for answer from SWAT developing team to release another version, we could also go into the codes based on the error information and add some conditions around the mathematical equation to avoid these errors. It could save you from frustration and gain you some satisfaction. You also could also submit your codes and contribute to SWAT development.

Compiler and IDE

The first thing need to do is to get a Fortran compiler and IDE. Not sure what compiler is used by the SWAT developing team. There are usually two options here.

    1. Intel Visual Fortran
    2. Photran which is a component of the Eclipse Parallel Tools Platform (PTP). Please note that Photran is not a separated software. To use it, please download PTP and extract it to any folder. You will find eclipse.exe in the folder, which is the Eclipse main program. Please go to http://eclipse.org/downloads/ to download this software. In this page, choose “Eclipse for Parallel Application Developer”.

Compared to Intel Visual Fortran, Photran is free and cross-platform. It’s chosen as the main environment to compile and run SWAT.

Photran doesn’t come along with any Fortran compiler. GFortran is chosen as the compiler working together with Photran. In Windows, the easiest way to use GFortran is to install MinGW. Please refer to my another post MinGW Installation Guide for SWAT Debugging.

So, Photran + MinGW is the main development environment.

Create a Fortan Project

In Eclipse or PTP, File -> New -> Fortran Project

Give a project name and select project type as “Executable (Gnu Fortran on Windows) and toolchains as GCC Fortan. (I’m working on a Windows PC.) You need to remember the project location to be used laster.

Then click Finish button to create the project. You will see this project in left Fortran Projects window.

Download SWAT Source Codes and Add into Fortran Project

Download the lastest version of SWAT from official SWAT website: http://swat.tamu.edu/software/swat-model/.

Please note that the official source codes are not ready to use. Some necessary modification is needed to be able to compile with gfortran, which will be illustrated below. I have put the modified version in Google Codes: https://code.google.com/p/swat-eclipse/, which is ready to use in Eclipse.

Extract source codes into the project folder specified when creating the fortran project.

In Fortran Projects window, right-click the project you create and select Refresh. Eclipse will automatically add all the source codes into the project.

Compile modparm.f

All SWAT global variables are defined in modparm.f, which need to be compiled as parm.mod before the source codes could be successfully compiled.

  1. Open cmd (Windows Command Processor) and go into the project folder with CD command.
  2. Compile modparm.f with gfortran: gfortran –c modparm.f

Modify Source Codes to Compile

As mentioned before, the official source codes are not ready to compile with gfortran. You would get some errors most of which are related to format. That may come from the difference of compilers or settings.

To successfully compile SWAT with gfortan, the source codes are modified. The modified version could be found here: https://code.google.com/p/swat-eclipse/. It’s based on the latest ver 613. I will update once the official SWAT is updated.

Compile Source Codes

Select the project in the Fortan Projects window and use “Project -> Build All” to start compile and build the final exe file.

You would see the progress messages in Console window.

The first time you compile SWAT, it will need several minutes to compile all fortran source code file (*.f and *.f90) into *.o files and link them together. The second time will be much faster because only modified source file will be compiled and linked.

Debug/Release Version

The same as most IDE, you could build debug and release version of the fortran program. Usually debug version is used to debug and runs slow and release version is used to distribute to users and runs faster.

As we want to find where the mathematical errors are and fix it, the debug version should be used first.

Set Breakpoint and Run

Set a breakpoint is farely easy. Just double-click the left edge of the codes editor and you will see a blue point will be added. Double-click again to remove the breakpoint.

SWAT assume all the input files are in the same folder of the program. To run SWAT successfully, we need to set the model folder as the working directory to tell SWAT where the input files are.

To do this, right-click your project and select “Properties”. In the project properties window, chose “Run/Debug Settings” and select one of the configurations in the right box. On the right, click “Edit…” button to edit the select configuration.

In Edit Configuration window, select Arguments table and edit the working directory as the model input folder, which is usually located in [ArcSWAT Project]\Scenarios\Default\TxtInOut.

Now, it’s ready to run. Click the bebug button to run the debug version of SWAT.

Remove Static Library Dependency

When the errors are removed, you may want to send a release copy to someone. If gfortran environment is not available in their computer, the program will fail to run. That’s because two library files are required to run the program compiled with gfortan.

To avoid this, go to the project properties window and select Fortran Build -> Settings in the left panel and select GNU Fortran Liker -> General in Tolol Settings on the right. Check “No shared libraries” to force linker to include the static libraries into the final exe file.

Please note that this could be done for debug and release version separately. Make sure you chose the right version.

How to Locate the Errors

It’s easy to locate the errors from the error message, which gives the line number and the Fortran file name. This is the call stack information and would usually show all the running routines/functions. The routine/function shown in the first line is usually the Fortran file which should be looked into.

For example, from the below error message, the error comes from routine virtual and the location is Line 423, which is sub_gwq_d(sb) = sub_gwq_d(sb) / subfr_nowtr(sb). The problem probably is in the right part of the equation. Modify it and debug on the same model until the problem has gone.

In some cases, the first line is .*c file rather than *.f file, like the one shown below. This usually happens when errors comes from native Fortran (mathematical) functions, which is Exp in this case.

Environment Canada Climate Data Reader

Please follow my blog to get the latest updates on ECReader as soon as they are available.

Please download the program package here and the quick guide is here. The .NET source codes is host in Google Codes.

Questions, bugs, more formats and functions, please send me email. I couldn’t be more happy to hear from you:)

Updates

Jan 10, 2014 Save and Load Defined Stations in ECReader1.1

Jan 6, 2014 File Name Convention in ECReader1.1

Jan 2, 2014  Version 1.1, Modify how to define query stations

Dec 28, 2013  New function to retrieve all EC climate stations and save as CSV file, based on which shapefile and kmz file could be also generated to help select climate stations in modelling.

Dec 20, 2013  Fix the uncompleted data bug.

What We Have

In Canada, Environment Canada is the major source of the climate data (precipitation and temperature). The climate data could be downloaded from Environment Canada website (http://climate.weather.gc.ca) for each year in CSV format which contains 27 columns data. To be used in ArcSWAT or SWAT, these files must be processed in following way:

  1. Extract min and max temperature and total precipitation from each file;
  2. Combine the data of different years from the same station into one dbf or txt file in a format required by ArcSWAT.

Let’s take a look. If 5 climate stations are located in/around the study watershed and the simulation period is 20 years, total 5*20 = 100 CSV files need to be downloaded first and then be processed into 5 dbf/txt files. It may take one or two days to finish this work. No modeler wants this.

Besides time consuming, this process is also error-prone. Large amount of files will be process in a same way. The brain and eye is easy to become tired. Wrong file may be processed or the same file may be processed twice. And the most important is that it’s difficult to find these errors. Modeler usually never check the climate data again after they are there, which leave it impossible to figure out climate data related problems in the model.

What We Could Have

Since the climate data preparation process involves many repeated work, a program is the natural solution. Select stations, click a button and then go to get a coffee. All the files are ready when you get back. Ideally? Yes, that’s the “Environment Canada Climate Data Reader” is trying to do.

Environment Canada Climate Data Reader, or just call it reader, is a .NET windows program designed to download climate data from Environment Canada and generate result files in a specific format (of course, it supports ArcSWAT dbf and txt format). With it, you could enjoy your coffee when doing workJ

Quick Guide

   1. Give Stations – Which stations will be used?

Station ID is the unique ID used by Environment Canada to identify each climate station. Before using reader, these IDs must be retrieved based on station name. If you don’t how to get the station id, just hit and a short tutorial will show you how to do that.

Already got station IDs? Let’s give the ID(s) to the reader first.

  • Only got one station? Good, just select option and input the station id in the box. Want to know the information of this station? Hit to check basic information of this station (to make sure correction station is used).

  • Got more than on stations? That’s normal. First prepare a csv file in Excel or any test editor, in which the first column is the station name and the second column is station id. It may look like this:

name,id

deerwood,29886

CSV file is ready? Select option and hit to load the file.

   2. Give Time Range – What’s your simulation period?

The simulation period may or may not be decided yet. No problem, just make it longer. ArcSWAT and SWAT would read the right data based on date.

   3. Select an Output Folder – Where do you want to save the result files?

The default output folder is the my document folder. Hit if you don’t like this. will open the given folder, which will allow you check the result files.

   4. Select an Output Format and Data Fields – What format do you prefer and Which data do you like?

Want to just use climate data in ArcSWAT? No problem, just select or and you are good to go (only min and max temperature and total precipitation will be exported).

Want some free air and do some analysis on the data? No problem, select or and select any climate data you want in the list.

Want the final SWAT format? Yes, for people who don’t use ArcSWAT, just select (only min and max temperature and total precipitation will be exported).

   5. Start Download and Drink Coffee

Ready? Hit to start the work. Progress message will be given on the bottom.

When you see , it’s done.

   6. Check Result Files

Curious about the results? Hit to open the output folder and open the result files. You may want to know the name convention for different format.

  1. P[Station ID].dbf for precipitation and T[Station ID].dbf for temperature, e.g. P29886.dbf and T29886.dbf.
  2. P[Station ID].txt for precipitation and T[Station ID].txt for temperature, e.g. P29886.txt and T29886.txt.
  3. [Station ID].txt, e.g. 29886.txt.
  4. [Station ID].csv, e.g. 29886.csv.
  5. pcp1.pcp for precipitation and tmp1.tmp for temperature.