Web-GIS for Water Distribution System Using ArcGIS for Server, .NET, Flex and SQL Server

Copyright: I have got the permission from project partners to publish these information. The copyrights of the data and related software are hold by all project partners. All the information here is for demonstration purpose only. It can’t be used for any other purpose. For any concerns, please don’t hesitate to contact me.

This is an ongoing project I have been working on for 7 years staring from 2007 when I was a PhD student in Tianjin University, Tianjin, China. The project kept evolving along with the development of web technologies and ArcGIS for Server. From Web ADF for .NET to ArcGIS API for Flex, we are trying to make the web GIS powerful and easy to use.

PROJECT OVERVIEW

When: 2007-present

Where: A provincial capital city in Northern China

Team: School of Environmental Science and Engineering, Tianjin University, China

Objective: Create an online GIS system for staffs located in various offices to manage water distribution system data and real-time monitoring data

Phases: Phase I: 2007~2009, Phase II: 2010~2012, Phase III: 2013~present

ROLES AND RESPONSIBILITIES

  1. Team leader, system designer and developer
  2. Communicate with client for detailed user requirement analysis
  3. Design the web GIS application architecture and develop most of the system with .NET and Flex
  4. Purchase, assemble and setup the IBM server to host map services and web services
  5. Design Geodatabase and configure SQL Server and ArcSDE
  6. Create and manage map services with ArcGIS for Server
  7. Deploy and test the system
  8. Customer support and system maintenance

BACKGROUND

Reclaimed water, as the less expensive alternative to portable water, has been introduced to many cities in China. Unlike the water distribution system for tap water built 100 years ago, the water distribution system for reclaimed water is constructed in digital era and its information (geospatial and non-spatial) could be established when it’s being constructed. The city recognized the advantage and needs of a distributed GIS system when they are building the water distribution system and started the project from 2007.

TECHNOLOGIES USED AND THREE PHASES

As staffs are located in different offices around the city, a distributed web-GIS system is more suitable for data management. ArcGIS for Server has been around for a year that time and was chosen as the main platform for the system.

The Geodatabase was created first to meet the data management requirements. Geometric network was utilized to simulate the network connection and help further analysis. GPS units were purchased to help field data collection. All data were stored in a SQL Server database through ArcSDE. Map services were created using ArcGIS for Server and a customized application was developed as the main interface to the GIS data for all staffs.

The project has gone through three phases. Web ADF for .NET was the framework used in phase I and was replaced with ArcGIS API for Flex in phase II. It’s going to the third phase from 2013 and more analysis functions would be added. With Web ADF for .NET, most of the functions were implemented through customized tasks using ASP.NET. After transferring to ArcGIS API for Flex, all previous functions were re-wrote as ASP.NET web services. Note that the client application was built with Flex(actionscript) directly rather than utilizing ArcGIS Viewer for Flex. It took more efforts but would give more flexibility on interface design and functionalities.

The system architecture is shown below. The client application was built with Flex(actionscript). Some basic GIS functions were implemented with ArcGIS API for Flex, e.g. identify and query. Other more complicated analysis requests would be given the ASP.NET web service in the service side, where ArcObjects was used to manipulate geospatial data residing in map services provided by ArcGIS for Server. There are two SQL Server databases for geospatial data and real-time monitoring data respectively. With limited budgets, the web service and map service was in one IBM server but could be distributed in different servers in the future.

SCREENSHOTS

Main Interface

The main interface includes menus, toolbar, table of content, express location list, status bar and the map area.

Valve Search (Network Analysis)

Valve search interface is shown below with just two buttons: the first one is used to choose the location where the burst happens and the second one is the submit button to start analysis.

The network analysis is utilized in this process to make sure no water could flow into the affected area when the chosen valves are closed.

It would give all the values that needs to be closed for emergency maintenance and all the residences affected to be notified through phone call or text message. The valve list would be sent to field workers.

The affected area is also shown with highlight color to be printed out and taken to field.

Real-time Data (Customized Graphics)

It will display real-time monitoring data on map and/or in a movable window with customizable setting.

To display the real-time data on top of the feature, a dedicated graphic layer is added and customized graphics
are added for each feature where the real-time monitoring data generates. The background and text color, refresh frequency could be configured through the setting window and the config file.

Click the graphic on map or list item to open following operation window to see real-time and historical data in table/chart, checkout warning information and perform some analysis.

Checkout historical data in chart and table

Query with Spatial and Attribute Condition

The main interface for query is shown below to define the spatial and attribute condition.

Define a rectangle area (green) as the search area

Define the attribute condition: Pipe Diameter = 200mm

Display the results in a table with the ability to export to Excel file

Summarize Features on Attributes

This functions summarizes features on attributes, which is similar as the summarize function in ArcMap (shown below) but with more flexibility.

The main interface for summarize function is shown below, where user has chance to choose layer, more than one summary fields, field group method (one value one group, range group and free group for numerical fields) and summary statistics option (count or summation of a numerical field). In following example, choose pipe layer to analyze and select two fields to summarize (diameter and material). One value one group method is used for pipe material field as it’s a text field, i.e. one material will be one group.

Three groups are defined for pipe diameter: 0~500mm, 500~1000mm and 1000~2000mm.

All the pipes would be grouped by diameter and material and the total length of pipes in each group would be calculated and given in the following results window, where summary information would be given in table and different types of chart (line, column and pie).

Clipe and Export

This function would generate an output file for given part of the map with given scale. The main interface is shown below, where user has the chance to define a rectangle area, the scale and the output file format. The file could be downloaded as soon as it’s generated successfully.

Measure

The measure function would give the total length of a user-defined polyline and the area of a user-defined polygon.

Advertisements

An Interface for Watershed Evaluating of Agricultural BMPs Using SWAT, C#, DotSpatial and SQLite

Copyright: I have got the permission from project partners to publish these information. The copyrights of the data and related software are hold by all project partners. All the information here is for demonstration purpose only. It can’t be used for any other purpose. For any concerns, please don’t hesitate to contact me.

I have been working on various projects and have developed several interesting products. I guess they could be part of my “portfolio”. From today, I would try to summarize some of them and post here. The very first one is an interface developed in 2012 when I was a post-doctoral in University of Guelph, Ontario, Canada.

Click here to download a PPT for this project

OVERVIEW

When: 2012

Where: South Tobacco Creek watershed near Miami, MB

Team: Watershed Evaluation Group, University of Guelph

Project: Watershed Evaluation of Beneficial Management Practices (WEBs), AAFC

Objective: Develop an interface to create and evaluate what-if agricultural BMP scenarios to help conservation districts/authorities gain understanding on water quantity and quality effects and allocate limited resources for BMP implementation

BACKGROUND

Agricultural best management practices (BMPs) has been applied in Canada to help reduce nitrogen and phosphorus coming from farm fields. With limited budget and clear water quality goal, conservation districts/authorities need to choose the most cost-effective BMPs and their locations. Managers needs a decision support system (DSS) to help them understand the impact of various available BMPs on hydrology and economy, i.e. what-if scenarios, and then make proper decisions.

The Watershed Evaluation of Beneficial Management Practices (WEBs) program was a successful nine-year Government of Canada initiative to determine the economic and water quality impacts of selected agricultural beneficial management practices (BMPs) at nine watershed sites across Canada. The interface was developed as part of the program for the South Tobacco Creek watershed located near Miami, Manitoba.

FUNCTIONS

  1. Create scenarios to add, remove and configure various BMPs
  2. Run SWAT and economic model for each scenario
  3. Display SWAT, economic and integrated results on field, farm, subbasin and watershed level with map and chart

TECHNOLOGIES USED

The interface is a Windows form application developed using C#. The main part of the interface is the map area implemented with DotSpatial map control. Microsoft chart controls are used for all charts.

In model side, to enable fast access to SWAT results, SQLite is added to SWAT source codes to write all results into a SQLite database with fixed structure including tables, views and indexes. A scenario definition file is also generated as the bridge between the interface and SWAT model to exchange data. Mix-language compilation (C and Fortran) is required to compile the modified SWAT codes.

The system architecture is show below.

ROLES AND RESPONSIBILITIES

  1. Design the main architecture and choose required technologies
  2. Design the GUI and code with C#
  3. Design the interface between GUI and SWAT with the scenario definition file
  4. Modify SWAT sourced codes to support SQLite results and scenario definition file

SCREENSHOTS

Project tree on the left to navigate different views, including project, scenario, BMPs and results

Project View

A project represents a watershed. It includes the geospatial and model information.

Scenario View

A scenario is a combination of various BMPs including small dams, holding ponds, grazing, tillage and forage conversion. SWAT and economic model could be run for each scenario. Results could be viewed after the simulation is done.

Small Dam View

Select/deselect small dam on map or from the list on the right to enable/disable them in model

Tillage View, Field Level

Select/deselect fields on map or from the list on the right to apply tillage. Farm level and subbasin level is also supported.

Result View, SWAT Results

  • Display results with different colors for polygon and different sizes for point
  • Select a feature to view time series in chart at the bottom
  • Control result display using the right panel

Result View, Economic Results

Result View, Integrated Results

  • Cost-effectiveness, i.e. the amount of reduction (on water, sediment, N or P) per $1000 between two scenarios.
  • Most valuable for conservation districts and authorities

Result View, Off-site Integrated Results

Off-side is for the result at the watershed outlet and is good for whole watershed evaluation.

ECReader – Environment Canada Climate Data Reader

ECReader (Environment Canada Climate Data Reader) is a .NET tool to download Canadian climate data from Environment Canada website. It comes from the SWAT modelling project I’m working on. I’m tired of downloading the data from EC website one station after another station, year by year and then processing them into the file format ArcSWAT wants. It’s no fun at all. As a programmer, I rather spend the same time to make a tool to do all these work in just one click. It would also help others too. So, please leave a comment or send me email (hawklorry@gmail.com) if you needs more functions. I would be happy to add it for you. Also please follow me on wordpress or facebook to get notified for any further development.

Program Package
https://drive.google.com/file/d/0B16YhFB_9MejSjE4RlQ2VjZLVlk/view?usp=sharing

Source Code

https://code.google.com/p/environment-canada-climate-data-reader/

Screenshot

The main features are list below.

  • Download climate data for a bunch of stations with just one click
  • A build-in station definition window to help locate stations of interest by name or location.
  • Support different output formats including the SWAT-ready dbf/txt format which could be used in ArcSWAT without any post-process (only for precipitation and temperature).

  • Support daily and hourly data
  • Give a map of all climate stations, which could be used outside the tool. The attribute table gives the station ID, name and the data availability of hourly, daily and monthly data. It’s a good dataset for any research required climate data. It’s available right in the tool and in csv, shapefile and kmz format for different environment.

Several posts have been published on this topic (list below). New functions would be added shortly to help fill the data gaps using nearby stations using IDW (Inverse Distance Weight) method. Lapse rate would also be considered for precipitation and temperature in this process.

December 12, 2013 Environment Canada Climate Data Reader

December 18, 2013 How to Get Environment Canada Climate Station ID From Name

December 28, 2013 Uncompleted Data Bug Fixed, Please Update

December 28, 2013 Environment Canada Climate Stations (Shapefile and KMZ)

December 31, 2013 6 More Columns Added to Environment Canada Climate Stations (Shapefile) to Help Check Data Availability

January 2, 2014 ECRearder 1.1 – New Climate Station Definition Window – No longer need to look up station IDs yourself

January 6, 2014 File Name Convention in ECReader1.1

January 9, 2014 Save and Load Defined Stations in ECReader1.1

 

 

ArcGIS Desktop Developer Associate 10 (EDDA10)

I was certified by ESRI as ArcGIS Desktop Developer Associate 10 on 2012. A brief description of this certification is given below and more detailed information please check out ESRI website.

Esri Certified ArcGIS Desktop Developer Associates are familiar with ArcGIS for Desktop and/or engine application development. The candidate is a developer who is sufficiently familiar with ArcGIS technologies to use, configure and set up the software and build desktop GIS applications.

This certification is for ArcGIS Desktop developers who have a good context for ArcGIS Desktop application development through familiarity with ArcGIS end-user tasks such as data analysis, data editing, data creation and data quality, map visualization, and map output.