-
Notifications
You must be signed in to change notification settings - Fork 310
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Split polygons at dateline #580
base: develop
Are you sure you want to change the base?
Split polygons at dateline #580
Conversation
Added a routine to split coordinates into geometry that is split by the dateline. Added to NetTopologySuite.Samples.Technique.SplitPolygonAtDateline
Enhanced logic to determine geometry that crosses the dateline.
Added trim gap functionality such that if a polygon is split by the dateline then a part either on the western or eastern hemisphere can have a trim gap. This means it will not exactly touch the dateline.
Added more checks and debugs. Added comments and links for feature class tolerance.
added another test case
small tweaks
Perez seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Tweak for zero trim gap
Small tweaks
Small tweaks Added checks at start of routine Modified block that closes a list of coords to create a polygon
@@ -31,7 +31,6 @@ | |||
</ItemGroup> | |||
|
|||
<ItemGroup Condition=" '$(EnableApiCompat)' == 'true' "> | |||
<PackageReference Include="Microsoft.DotNet.ApiCompat" Version="6.0.0-beta.21159.11" PrivateAssets="All" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see a potential problem here, because EnableApiCompat == True
but the reference to the package was removed.
Honestly I don't even know how this package is used, but I think it should be restored
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wrote an article on my issue here #577
I didn't know how to fix it. It was suggested I remove a package and recompile. That seemed to work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My doubt is about pushing the package removal, I suggest to re-add the reference and then investigate the problem in #577
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has to go away. If it does not work in your environment, you have to fix your environment.
If you want to turn it off temporarily, change EnableApiCompat
to false (see 1st PropertyGroup)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes but I don't know how to go about fixing my environment. I'm not even sure where to start. I did a google search and not much came up. I'm just unfamiliar with ApiCompat and how it works.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sindizzy looks that - thanks to @airbreather - we have a fix for this!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we have a fix for this
Not sure about that one, since the NuGet.config file does actually exist in this project... the question is what's causing it to get ignored? (edit: assuming that is, in fact, what's causing the issues).
It's possible that we could try requiring the released .NET 6 SDK, since I think I remember reading something that the API compat feature was going to be more built-in in that version of the SDK.
public class SplitPolygonAtDateline | ||
{ | ||
[STAThread] | ||
public static void main() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe this code can be moved to a unit test (along with other sample code, possibly?) I know that other "Technique" items uses this parrern (inherited from JTS code) but basically this means that this code isn't tested.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't done any unit testing in any of my projects so am a complete rookie at that. As you mentioned I just followed the pattern of testing in other routines in the Technique area.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will help you with that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sindizzy added the unit test code, hope to not have done a mess.
This is just a sample, please feel free to add your own asserts and other checks.
// ======================================================================================== | ||
// inputs | ||
// ======================================================================================== | ||
string wktPrj = "GEOGCS['WGS 84',DATUM['WGS_1984',SPHEROID['WGS 84',6378137,298.257223563,AUTHORITY['EPSG','7030']],AUTHORITY['EPSG','6326']]," + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I understand, the projection string is used just ot check if the data is in a GEOGCS format. To me this check is useless, because we are not checking the geometry but a separate parameters. Maybe a check of the geometry values (inside the valid range of values) can be slower but useful?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suppose I can do that but say if someone has coordinates in UTM that go from 50 to 170 meters. That will pass the geographic check as we are only checking values and not units. My thought was that I didn't want users to just hand over their coordinates hoping it would work with any projection. The easiest way, for me at least, was to force them to provide a projection string.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see your point, but maybe it's something related to the documentation of the method/api. my2cents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Absolutely. I strived to add quite a bit of a description buttttttt there will be many that don't read it and just dive in. :)
/// </remarks> | ||
public static Geometry ToPolyExOp(List<Coordinate> inpCoords, string inpProj4Wkt, OgcGeometryType outType, double outTrimGap, double outDensifyResolution, bool isAttFixOutInvPolygons) | ||
{ | ||
Debug.Print("$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
these "debug.print" should be removed from the main methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, thank you. Per my main write-up "The PR is also heavy on the debugs and I can remove those once the NTS devs can take a look at the initial proposal."
// ################################################################################################################### | ||
if (inpCoords == null) | ||
{ | ||
throw new Exception("Null input coordinates."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please use "ArgomentNullException" instead of generic "Exception"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do.
{ | ||
case OgcGeometryType.LineString: | ||
if (inpCoords.Count < 2) | ||
throw new Exception("Input list has fewer than 2 coordinates. To create a polyline, the minimum is 2 coordinates."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ArgumentOutOfRange (of maybe InvalidOperationException?) here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I agree with your assessment here and will change my code. However, on my work related code I like to tell the user exactly what is wrong. At times I've encountered the ArgumentOutOfRange exception and wondered "ok what is the range then?".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you can leave the message. I see your point and ArgumentException can be used too.
I must admit that, once you don't use the base Exception
class, the proper exception to use can be arguable , so feel free to use the exception class you want. Often I use InvalidOperationException
, but not sure I can be taken as an example :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure thing. I am learning on the go.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I can use something like:
throw new ArgumentOutOfRangeException(nameof(count), count, "Must be non-negative.");
throw new Exception("Input list has fewer than 3 coordinates. To create a polygon, the minimum is 3 coordinates."); | ||
break; | ||
default: | ||
throw new Exception("Unsupported output geometry type."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NotSupportedException if the operation can not be implemented, or NotImplementedException if you plan to extend the code to support these operations in the future
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
got it. thanks.
/// https://desktop.arcgis.com/en/arcmap/10.3/manage-data/editing-fundamentals/creating-and-editing-multipart-polygons.htm | ||
/// https://desktop.arcgis.com/en/arcmap/10.3/guide-books/map-projections/what-happens-to-features-at-180-dateline-.htm | ||
/// </remarks> | ||
public static Geometry ToPolyExOp(List<Coordinate> inpCoords, string inpProj4Wkt, OgcGeometryType outType, double outTrimGap, double outDensifyResolution, bool isAttFixOutInvPolygons) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IList
can be used, I think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How different is using ListOf vs IList? I'm curious as to the pros and cons.
Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using IList
, the caller can use as input any implementation if the IList interface, like arrays.
The only code that needs to be changed is the Exists
checks at lines 197-198, then you can get rid of all the ToList and use directly the Coordinate array from the Geometry object itself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd prefer inpCoordinates
to be of type CoordinateSequence
. Maybe this whole function should be encapsulated in a class implementing IEntireCoordinateSequenceFilter
invoked by Geometry.Apply(filter)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or have an overload of the function taking a Geometry
as an argument. In that case IGeometryFilter
would be an option.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok let me take a look at the NTS codebase and see how CoordinateSequence is used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So how would the Geometry as an input work?
- The assumption that any geometry can be broken down to a coordinate list and then the user selects what kind of output comes from those coordinates? -OR-
- The assumption that if I provide a Polygon then I want output as polygon. If I provide a polyline then I want polyline for output? What if the user inputs other non-supported geometry types?
What makes the most sense?
Debug.Print("cutterPgon isValid = {0}", valOp2.IsValid); | ||
|
||
// perform an Erase to split the polygon with a polygon. this consists of an Intersection and then a Difference. | ||
//var intx = pPolygonShifted.Boundary.Intersection(cutterPgon); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Old commented code that can be removed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep. will remove after the review is done. I'm an over-debugger I have to confess. I wanted to double check every output so have tons of debugs. :)
Debug.Print("resGeoms were fixed"); | ||
|
||
// method 2 | ||
//var pm = new PrecisionModel(10000000000.0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Old code that can be removed? or it can be handled using a specific methofd parameter?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
well no. the routine to fix invalid geometries resulted from some research and eventually this article: #124
From that discussion it was noted that Buffer does not always work. So I added the second method in case I wanted to add as an option later. I could probably make the input an Enum rather than a Boolean. So I would have
public enum attFixOutInvPolygons: int
{
FixNone= 0,
FixViaBuffer=1,
FixViaPrecision=2,
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the enum idea
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Im adding it as my next commit.
/// https://desktop.arcgis.com/en/arcmap/10.3/manage-data/editing-fundamentals/creating-and-editing-multipart-polygons.htm | ||
/// https://desktop.arcgis.com/en/arcmap/10.3/guide-books/map-projections/what-happens-to-features-at-180-dateline-.htm | ||
/// </remarks> | ||
public static Geometry ToPolyExOp(List<Coordinate> inpCoords, string inpProj4Wkt, OgcGeometryType outType, double outTrimGap, double outDensifyResolution, bool isAttFixOutInvPolygons) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"out" parameters (all the parameters following "inpProj4Wkt") can be moved to a single "parameter class", just for the sake of the readability
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question, is there a number of parameters where you move to a parameter class? Obviously if you have 30 params you don't want those all in the function call. If I have 3 or 4, I typically leave it in the function call as-is. Looking at the NTS core code there are various calls with 5 even 6 params right in the function call. So I'm wondering if this change is needed? I'm open to it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Simple a suggestion, just because there are several "out" parameters, that maybe can be grouped for the sake of the readability.
Totally unrelated, see something like JsonConvert.Deserialize(jsonString, jsonDeserializationParamsClass).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok I will take a look. Thanks.
I just added some "formal" comments about the code, I don't know so much about the algorithm but I think you can add few unit test methods to show how to use the class to obtain different kind of results. |
@sindizzy This looks fantastic and fingers crossed it works well. One question: We have many polygons crossing the 180 meridian. With this change will we be able to intersect with those polygons? I will try and test this out myself, just wondering if you know already |
I believe with the polygons split into two parts you can use them like any other geometry. |
@euangordon can you suggest some more unit tests? |
Thanks I will spend some time this week to clean up the code and fix the commit authoring. As for the tests, I added 3 tests on how to use the routine. I followed other routines in the same area ie test\NetTopologySuite.Samples.Console\Technique. |
/// A Geometry class that contains the result of converting the coordinates into a Polyline or Polygon. The result could be multiple | ||
/// geometries. | ||
/// </returns> | ||
private static Geometry ToPolyExOp(List<Coordinate> coordsInp, string proj4WktInp) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should be made public, I suppose
with the minimum changes required to the original class, just to allow the unit test to be compiled
What am I looking for? |
Is that the final routine? |
@sindizzy, that is how I would do it, does it work for you? |
Let me test it. Can I give you some feedback later in the week? |
Ok I have taken a look and here are my comments: |
Also on line 87 I am not certain I understand the need to union the results. if we have a multi part polygon and we split it along the dateline then the output may be something we don't expect after the union. I would have to give this some more thought. |
Another thing I would definitely like to see added is the trim gap. As I noted in my routine, after some research, it is apparent to me that one major GIS vendor takes land masses the cross the dateline (like Fiji) and creates a single feature with multiple parts. However the rule for single feature with multi-part polygons is that they cannot share an edge. So by inspection these land masses have one part where a small gap is introduced so as to have it spatially not share an edge with the other part. What I have seen is one part has the edge at 180.000000 and the other has and edge at -179.99999999. Just a small enough gap so that the two parts can be a single feature. As an option I have that in my routine and would use it quite often. |
@sindizzy the example posted was not meant to replace your solution, it was meant to demonstrate how to work with |
Ahhhh gotcha. Ok let me use that as a base for my logic. |
Based on suggestion from @ FObermaier at https://gist.github.com/FObermaier/ba25a5b6ccc98cb86efbbd485d17bc01
@FObermaier Ok based on your suggestion I added a DatelineUtility class to my PR. I think I understand your preferred workflow now. I did change the main filter to This is because if a user passes a multi-part polygon then each polygon part could exist wholly in one hemisphere. If we use the CoordinateSequence then that could give us a false positive. Before I continue with this code I wanted to see if I am on the right track here. The filter cycles through each part in the geometry and then through the Coordinate array. I don't use the CoordinateSequence so wondering if that will pose a problem? Edit: I just thought about it and I think I will add another filter class for CoordinateSequence and that will be called from the main IsCrossesDatelineFilter. I think that's the best way forward. |
Added CoordinateSequence filter to the DatelineUtility. It is called from the main Geometry filter in case the main geometry has multiple parts like a multi-part polygon.
Alright, I added a CoordinateSequence filter that is called from the Geometry filter. This should account for multi-part geometries as the input. If we just do a CoordinateSequence then the coordinate sweep could jump from one part to another and we could get a false positive. |
/// <summary> | ||
/// Method to use to fix invalid geometry. | ||
/// </summary> | ||
public enum InvalidGeomFixMethod : int |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't need this. As NTS v2.4 we have NetTopologySuite.Geometries.Utilities.GeometryFixer
to fix invalid geometries
var valOp = new IsValidOp(resGeoms); | ||
if (!valOp.IsValid) | ||
{ | ||
if (outParams.InvGeomFixMethod == InvalidGeomFixMethod.FixViaBuffer) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This block should read:
resGeoms = NetTopologySuite.Geometries.Utilities.GeometryFixer.Fix(resGeoms);
test/NetTopologySuite.Samples.Console/Technique/DatelineUtility.cs
Outdated
Show resolved
Hide resolved
test/NetTopologySuite.Samples.Console/Technique/DatelineUtility.cs
Outdated
Show resolved
Hide resolved
You don't need to split up into seperate filters. Note: Only |
Undo the two prior filters and setup only the IEntireCoordinateSequenceFilter filter. This filter will cycle through all parts in a geometry as long as Done=false. Our logic sets Done=true when any part crosses the dateline so this should work with any type of input geometry.
Updated comments.
@FObermaier Ahhhhh I think I got it now. Thanks for taking time to teach me how this works. I went back to only the IEntireCoordinateSequenceFilter and did a commit. I think this is what you were talking about so when you get a chance please take a look. |
I see you updated a new routine so I took that and tweaked it just a little bit. Added my logic and the geographic CRS list. I will need to go through and study some parts more in detail but the baseline is there.
I noticed you updated your routine so I started over with DatelineUtility2. Added my logic and the geographic CRS list. I will need to study it further to understand certain parts. |
So, I haven't really been following this closely at all, but I do want to at least point out that the term "dateline" probably shouldn't show up in the code. I don't mind referring to it that way informally, but the term "antimeridian" is a more accurate term to use when referring to the thing that causes problems for NTS, since the International Date Line deviates from the true meridian in several places in order to avoid chopping countries in half. |
Yes I agree on the definition of the international dateline. However, I would like to point out that most users, even major GIS companies, refer to the antemeridian as the "dateline". When I search for articles and info on this challenge it's always using the "dateline" moniker. I'm good either way though. |
Hi All, what is the status of this? Is there any chance it will get merged in? |
I was working on it and I lost steam. I'll pick it up again soon. |
Thanks @sindizzy we did try and use your fork but didn't manage to get it working. Hopeing that once it is in the release build we will manage to get it working. As a workaround, I am working on splitting shapes that cross the dateline, but it painful. |
In the meantime you can try https://gist.github.com/FObermaier/ba25a5b6ccc98cb86efbbd485d17bc01 |
Yes I know how painful it is. You can try the algorithm here #496 That's where the discussion started. |
I had this as a side project and am going to see if I can get back to refactoring it. I've spent the last couple of years transitioning from VB to C#. I've also upgraded my own algorithm slightly. |
Prerequisites
Description
Presents a technique for converting coordinates to a polygon. With the coordinates all in one hemisphere it is a
fairly simple process. However, when the geometry crosses the dateline it becomes a more challenging problem.
This routine attempts to do both. That is to say, it converts coordinates to a poly geometry regardless if the
geometry traverses the dateline or not. If the geometry does cross the dateline then it is split up into multiple
parts. This benefits the user in two ways: first, any GIS will be able to display the geometry in a proper way regardless of map projection and two, the GIS will be able to perform geoprocessing functions on the geometry. Note that feature classes do
not support features with multiple parts that share the same edge. Thus, it is left up to the user on how to
commit the geometries to a feature class. Suggestions are to save the multiple geometries as individual features
or to slightly alter the edges of each geometry so as to save all parts as a single multi-part feature. For the
latter option, the algorithm provides an input to trim edges of parts so that they can be used in a single feature.
The algorithm arose from attempting to convert worldwide airspaces (which can be found in any online aviation db) to polygons. Some of the coordinates result in polygons traversing the dateline and the result is ugly. That is because the GIS is not inherently aware that the polygons cross the dateline. This is what a standard conversion will look like:
The goal is to split those special polygons so that the airspaces are properly represented:
The PR includes a sample input with three processing options. If the item crosses the dateline then you will always get back multiple geometries no matter what options you use. The difference is what the dev intends to do with the geometries. The sample input will produce the results discussed below. For easy viewing the map projection has been set to Plate Carrée with the central meridian at 180 degrees (the dateline). The PR is also heavy on the debugs and I can remove those once the NTS devs can take a look at the initial proposal.
If the dev intends to add them as a single part per feature then this is what you will get with the standard options:
If the dev intends to add them as multiple parts per feature then its a bit more complicated. Through research and observation it was noted that existing datasets representing worldwide polygons already deal with the multi-part issue by trimming one of the polygons ever so slightly such that the two parts do not mathematically share an edge. From my research this trim gap appears to be connected to the feature class tolerance. The trim gap is a variable that the user must provide but should be made as small as possible. For demonstration purposes the trim gap has been made quite large and this is what it would look like with advanced options:
Abel