Hajk 3.8 Release Candidate 1

[jacobwod]

212 commits and changes in 130 files since the 3.7 release – here's Hajk 3.8! This release was brought to you by @Hallbergs and @jacobwod . Please see the list of commits for all contributors.

Changelog

Major new features

Added FME Server plugin. #750

This opens up new possibilities for integration with Safe's FME Server application.

Search plugin respects different response types. #882

Search plugin now respects different info formats (GML 2/3/3.2 and GeoJSON). Makes Hajk work with QGIS Server's WFS implementation.

Improvements in Edit. #884

More data types are allowed in the plugin.

Improve Hajk's accessibility. #877

It's now possible to navigate the map using keyboard.

Make Client UI user-aware. #899

The added UI Control is just a proof-of-concept, with virtually no functionality other than displaying the user's name in the UI. However, this opens up a lot of possibilities for plugins to optionally read provided information from Active Directory.

New features

• Major improvements for QGIS Server users, including better support for GetLegendGraphics (Highlight of geometry when using infoclick not working for QGIS Server #879) and FeatureInfo (GetLegendGraphic not working for QGIS Server #880) requests.
• TLS Support in Backend makes it possible to authenticate via LDAPS. TLS support for LDAP (Active Directory) in Backend #893
• Allow Admin to enable Home button Allow Admin to enable Home button #876
• Optional info is now shown also for background layers. Feature/764 background layer info #808
• Make it possible to disable View rotation, among other default OpenLayers interactions Make it possible to disable View rotation  #875
• Location plugin: make it more like other map apps Location plugin: make it more like other map apps #863
• Better EDP integration support EDP - Added functionality missing in Hajk3 from Hajk2 #861
• Added endpoint /fmeproxy as a new controller Added endpoint /fmeproxy as a new controller #900

Bug fixes and minor features

• Edit crashes if a draw operation is cancelled #869, Control buttons might get rendered twice #864, Bad contrast on abort and close buttons in Dialog component #854, Render <blockquote> nicely in FeatureInfo #860, Maplinks in the preset-control ignores background layers. #859, Fix log4js setup in Backend #894, Make sure that all references to layer are removed when layer is deleted #898, Make sure that all versions of Hajk has similar configuration #897, Layers configured with < a /> tags without proper url-values in infoBox causes errors. #901, Don't zoom in when drawing a search polygon #902.

Checklist for admins

• Verify that search works with different kinds of sources. Previous to this release, search always used GeoJSON, but as of this release the setting from Admin UI is respected. Since the default setting in Admin UI has always been GML2, that's what will be used by default from this release. If you want to stay with GeoJSON: make sure to change the response type for your search sources.
• Verify that FeatureInfo ("info click") works, for multiple layer types.
• Ensure that the Edit plugin works with your current configuration. The plugin has seen substantial update in this release.
• If you use the Active Directory authentication in NodeJS backend, you now have the possibility to supply certificates and use LDAPS. Check out details in .env.
• If you're planning to use the new FME Server plugin, make sure that the url_base key is set in Admin UI's config.json. (This has been added to config.json in this release.)
• If you'd like to use the more compact version of the Location plugin, ensure to set the target properly to control in Admin UI for that plugin.

---

Downloads

There are multiple ways to get the latest Hajk up and running, including Docker container.

Pre-compiled releases

This time we have three different version of the pre-compiled solution (scroll down for download links):

1. Hajk 3.8.0-rc.1 NodeJS edition
2. Hajk 3.8.0-rc.1 Simple edition
3. Hajk 3.8.0-rc.1 .NET edition

Docker container

The container is available on Docker Hub: https://hub.docker.com/r/hallbergs/hajk

Installation

Refer to our recently updated installation guide.

---

This discussion was created from the release Hajk 3.8 Release Candidate 1.

[perosoderstrom]

Problem with "Utför sökning" in search plugin.

This works as expected:

Install Hajk 3.8 Release Candidate 1, nodejs backend, and configure .env, client/appConfig.json and admin/config.json. Hajk starts OK.

If you do a search, like "storgatan", you get a selection of addresses in Gothenburg, as expected, and if you click on "Utför sökning" you get the rest, as expected.

But if I clone and install from source I get problem:

Git clone, git checkout develop, no changes in config files, npm install, npm run dev/npm start: Hajk starts OK.

If you do a search, like "storgatan" you get a selection of addresses in Gothenburg, as expected, and if you click on "Utför sökning" you get all, as expected.

If I change "mapserviceBase": "http://localhost:3002/api/v1", in appConfig.json and try the same, then "Utför sökning" says "Sökningen gav inget resultat".

[Hallbergs]

Hmm. This sounds really strange. I've seen that the wfs used in the standard configurations (Addresses in Gothenburg) have been returning a connection error sometimes. Could it be that the wfs just happened to return a error? Could you check the response in the network tab when performing a search so we can track down the culprit?

[jacobwod]

This one turned out to be a local issue where @perosoderstrom had matchCase active, right? I assume it's "resolved".

[perosoderstrom]

Yes, this is resolved.

[perosoderstrom]

I get OK response (Status 200) from the fetch, but no features:

More odd things:

1. In admin -"Söktjänster", if I click on the original layer "Adresser Göteborg", I get an error. After the click i can still see the form, and I see that the layer name differs in "Lagerlista" and "Valt lager", and the layer in "Lagerlista" is not selected.
   
   

2. If I delete "Adresser Göteborg" from search services in admin, the layer is still left in map_1.json.

[jacobwod]

What does the the fetch request look like in detail? You should be able to "Copy as cURL" in your browser's developer tools. Here's what mine when Hajk does an autocomplete lookup for the term storg:

curl 'https://opengeodata.goteborg.se/services/adresser/wfs/v1' \
-X 'POST' \
-H 'Origin: https://hajk.jwod.se' \
-H 'Accept: */*' \
-H 'Content-Type: text/xml' \
-H 'Pragma: no-cache' \
-H 'Cache-Control: no-cache' \
-H 'Accept-Language: sv-se' \
-H 'Host: opengeodata.goteborg.se' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.1 Safari/605.1.15' \
-H 'Content-Length: 552' \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'Connection: keep-alive' \
--data-binary '<GetFeature xmlns="http://www.opengis.net/wfs" service="WFS" version="1.1.0" outputFormat="GML2" maxFeatures="100" xsi:schemaLocation="http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.1.0/wfs.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Query typeName="adresser.wfs.v1:adresser" srsName="EPSG:3006"><Filter xmlns="http://www.opengis.net/ogc"><PropertyIsLike wildCard="*" singleChar="." escapeChar="!" matchCase="false"><PropertyName>adress</PropertyName><Literal>storg*</Literal></PropertyIsLike></Filter></Query></GetFeature>'

I'm running directly from the Docker build, there's an Admin UI over here if you want to compare the search service's setup.

[Hallbergs]

I just had a meeting with Per. The issue was that matchCase was saved as true in localStorage, leading to no hits :)

The admin part we must investigate!

[jacobwod]

We have multiple problems with the Edit plugin. It does not work with our previous configurations, and we haven't yet managed to configure it "correctly" in the new Admin UI either.

Has anyone had any success with saving from WFS in 3.8 yet? @perosoderstrom @hitomin21 @Hallbergs @eri-blan @WCarlsson

Edit: I've managed to save at least one of our GeoServer-served WFS services: a PostGIS table with Point geometries. Still extensive testing is needed though, but it's good to see that some existing configuration seems to work!

[hitomin21]

@WCarlsson It seems like changing the format to 2021-10-25 00:00:00 will make postgis discard the time part of the datetime so we either need a setting for whether Oracle is used (which I prefer not to implement to keep the code as database agnostic as possible) or allow to store datetimes in string columns instead.

I think putting them in string columns ( is better, that should work across all database servers). What do you think? It means that you would specify "Datum & Tid" under Typ for a string field (same as we allow dates for that).

[hitomin21]

If that sounds OK, I will add it

[jacobwod]

I can't find any reference (if anyone else can point out the correct authoritative reference, feel free to paste the link!) at the moment. But if I recall correctly, the WFS-T will use the xsd:date format as defined in the XML specification.

I imagine that if the WFS-T requests sets the date value as a correct xsd:date (rather than xsd:string), the WFS server should take care of the rest. We are only interested in sending the data correctly to the WFS service (following the standard's specification) – whatever happens next (whether it's writing to PostGIS, Oracle or anything else) is the responsibility of the WFS server.

Can you check the POST to WFS-T @WCarlsson and verify that the date being sent is in xsd:date?

@hitomin21 I'd rather see that we don't recommend storing datetimes in string fields. Let's try to follow the standard: a date is a date, not a string. Let's await @WCarlsson's response first. 👍

[hitomin21]

@WCarlsson : Just a friendly reminder to get back about the WFS-T response.

[WCarlsson]

@hitomin21 Thanks for the reminder, this slipped my mind!
It seems that QGIS always treats oracle date format as dateTime per default and with dateTime there seems to be some problem, I will try to investigate this further to see if I can get dateTime to work or more info on why it doesn´t.

If we make sure to specify the attribute as a date attribute when setting up the service in QGIS instead of letting it assume the datatype by itself it is recognised as a date and then the edit plugin works as expected. Thanks for the tip to verify this @jacobwod

[Hallbergs]

Just pushed a fix to master which fixes a bug in the XLSX-export. (We were trying to get the keys from the features with .properties, but on ol-features we must use .getProperties()).

[perosoderstrom]

Very good @Hallbergs! This solves en old issue ( #411)

[Hallbergs]

Wonderful!

[jacobwod]

Great find and fix @Hallbergs

[Hallbergs]

We might still miss keys... I'll investigate further. @jacobwod

[jacobwod]

I've added an issue as I think it's outside of scope for this thread for now. #916.

[perosoderstrom]

Problem saving layer search properties in admin with .Net Backend

If you configure search properties, e.g. search URL, search field etc., for a WMS layer the properties are not saved in layers.json.

This works OK in Node Backend.

[Hallbergs]

Nice catch! Will definitely look into that!

[Hallbergs]

0668b4b should have sorted this. (Just pushed to master)

[perosoderstrom]

Just tested this and it seems to work ok. Thanks, @Hallbergs!

[perosoderstrom]

Noticed another problem (not having to do with this) when testing this.
In /mapservice/App_Data/map_1.json, map zoom is set to 1.33 ("zoom": 1.33,), and dotnet backend requires an integer. Setting this to "zoom: 2" will solve it.

[Hallbergs]

It's taken care of now.

[WCarlsson]

We have found a couple of problems.
Baselayers (bakgrundslager) made from multiple layers from a wms-service doesn´t work anymore, react errors show in console.

In edit a zoom occurs when closing with a "fast" doubleclick, with a slower doubleclick this effect is avoided. Seems similar to #902

When adding a search or edit service from a QGIS server we have to add ?SERVICE=WFS to the URL in order to be able to show attributes in admin. If we add the service parameter the service seems to work but calls from the edit service adds a second service parameter. In order to have the same format of URL for layer, search and edit it would be nice to not have to add the service parameter.

[jacobwod]

Baselayers (bakgrundslager) made from multiple layers from a wms-service doesn´t work anymore, react errors show in console.

This could be related to the recent addition of "Show more info" to BackgroundSwitcher. @hitomin21 or @jesade-vbg, can you take a look?

When adding a search or edit service from a QGIS server we have to add ?SERVICE=WFS to the URL in order to be able to show attributes in admin. If we add the service parameter the service seems to work but calls from the edit service adds a second service parameter. In order to have the same format of URL for layer, search and edit it would be nice to not have to add the service parameter.

Yeah, this needs a quick fix in Admin UI so get rid of this once and for all. Will do!

[Hallbergs]

I think all issues discussed in this thread has been taken care of except for the baseLayers one. @hitomin21 @jesade-vbg were you planning to look at this or should i take care of it?

[Hallbergs]

Got an email from @jesade-vbg, they won't be able to fix this any time soon! I've got a fix coming up!

[Hallbergs]

The base-layers issue should be fixed via fb5d83f (just pushed to master).

If anyone notices strange behavior regarding layers, please let me know :)

[Hallbergs]

All issues discussed here should be resolved!

[WCarlsson]

When adding a new menulink in documenthandler 2.0 in a map the name doesn´t save, the menulink is created and other settings seem to save as intended.

[Hallbergs]

This is resolved! See #920

[Hallbergs]

Just fixed another issue. See: #921

[jacobwod]

Some more fixes, #920, #923 and #932. Feel free to checkout, especially @WCarlsson should like the #932 (givet that it does what I think it does…).

All fixes are now in master.

[Hallbergs]

Wonderful!!

[WCarlsson]

Awesome!

[jacobwod]

The #847 changes led to another huge bug: all of our Hajk group layers stopped working because of this: #940. I've fixed it immediately but this is another example of the importance of testing.

[emmafrygne-vbg]

VARBERG
Thank you for providing the checklist for admins. I have used it, and we are using the .Net version. We have started testing the FME plugin, but are still testing it and will do an evaluation on monday (13/12) so I let you know after that.

[maan002]

FME Server Plugin

Some small issues/features to the FME Server plugin requested by users in Varberg:

• Add previous selection to the UI, i.e. output the selected group and product in later steps.
• Be able to configure the word for "group", i.e. in Varberg they would like to use "produktkategori".
• Be able to open the product description directly after the product is selected in step 2. Maybe a button labeled "Information"?
• Be able to configure the color and fill of the selected area polygon.

[Hallbergs]

Add previous selection to the UI, i.e. output the selected group and product in later steps.
Not sure what you mean here. Feel free to implement though.
Be able to configure the word for "group", i.e. in Varberg they would like to use "produktkategori".
Done.
Be able to open the product description directly after the product is selected in step 2. Maybe a button labeled "Information"?
Done.
Be able to configure the color and fill of the selected area polygon.
Done.

And when i say done, i mean they will be in 3.8rc2 (If you want to test already, checkout #955).

[Hallbergs]

Release candidate 2 is now released! Any further discussions regarding 3.8 should take place in the discussion thread for rc2.