Other Gallery Entries
Line Cleaner – BC Ministry of Forests
Abstract: The BC Ministry of Forests needed a tool to merge multiple road network layers into a single consistent, connected network. The Ministry contracted with Refractions Research to build a merging algorithm and deliver a user-friendly application to run the algorithm using uDig as the delivery platform.
The BC Ministry of Forests (MOF) uses several road data-sets when building triangulated irregular networks (TIN). TINs help MOF to plan and/or permit forest road building, as MOF prefers that timber cut-blocks be accessed from existing roads rather than from forest roads newly built for the purpose.
Because of close but not exact feature replication among and within the road data-sets, TINs tend to be complicated by nests where features do not meet as they should. These areas are time-consuming to fix or remove.
Refractions had the necessary expertise to combine data sets and remove all redundancies, a process known as “conflation-based data integrations.” However, since MOF's data-sets contain several million features, conflation on this scale would exceed MOF's allotted time-frame for this project. (Full-scale conflation demands dividing data-sets into manageably sized sub-sets, and a significant amount of validation and un-grouping post-conflation.)
Refractions considered brute-force manual processing, but discounted it because of the sheer number of features not only to process, but also on which to maintain proper topology and enforce consistent visitation and treatment. Eventually, the time and effort required for manual processing would approach those required for full-scale conflation.
Building the Application
With the constraints of conflation, and with their original goal and intended purpose in mind, MOF allowed Refractions to work under the proviso of being quite severe with the data. The solution is Line Cleaner, a “light conflation” application that works in two phases:
- Cleanse – makes candidate networks as similar as possible, and
- Clean – considers cleansed features automatically for spatial similarity and removes any redundancies.
Line Cleaner cleanses networks by simplifying complex, cyclical, very short and zero-length geometries, and removing pseudo-nodes and insignificant vertexes. Most significantly, in the cleansing phase, it is able to ensure that feature matches can be considered automatically.
The rigourous cleansing phase allows data cleaning to avoid both complex networking and pathing. Feature-by-feature, equivalents are considered, and either match others or do not. Since connectivity between data sets has been rationalized to common points, simple precedence tests can be conducted to determine which of the N close features to keep, and throw away all others. Connectivity to other features is also not an issue, having co-located all nodes.
Line Cleaner was developed as a plug-in to the uDig spatial application framework. While developed specifically for MOF, Line Cleaner will work on virtually any linear data-set(s). Both uDig and Line Cleaner are distributed free of charge under the Lesser General Public License.
- An out-of-the-box application, with the ability to view, edit and print files, database (Oracle, SDE, PostGIS) and Web Feature Servers (WFS).
- An industry-standard extension framework (RCP) for adding new functionality to the application.
- A rich toolkit of Java functionality, ready for developers to build on.
- Native support for Windows, Mac OS/X and Linux.
uDig is developed by a world-wide community of contributors, with an open process and transparent decision making.
Refractions Research is a consulting and systems integration company, with industry-leading exertise in enterprise geospatial applications. Refractions initiated and now hosts the uDig and, PostGIS projects, and continues to provide core development, training and consulting on the projects.