Skip to content

Commit

Permalink
Version 3.0.
Browse files Browse the repository at this point in the history
  • Loading branch information
dracos committed Mar 4, 2020
1 parent 3221050 commit d0c432d
Show file tree
Hide file tree
Showing 42 changed files with 59,074 additions and 43,231 deletions.
2 changes: 1 addition & 1 deletion .tx/config
Expand Up @@ -2,7 +2,7 @@
host = https://www.transifex.com
type = PO

[fixmystreet.v2-6]
[fixmystreet.v3-0]
file_filter = locale/<lang>.UTF-8/LC_MESSAGES/FixMyStreet.po
source_file = locale/FixMyStreet.po
source_lang = en_GB
Expand Down
85 changes: 50 additions & 35 deletions CHANGELOG.md
@@ -1,11 +1,13 @@
## Releases

* Unreleased

* v3.0 (4th March 2020)
- Security:
- Fix XSS vulnerability in pagination page number.
- Rotate session ID after successful login.
- Switch to auto-escaping of all template variables (see below).
- Scrub admin description fields.
- Switch to auto-escaping of all template variables (see below). #2772
- Scrub admin description fields. #2791
- Front end improvements:
- Improved 403 message, especially for private reports. #2511
- Mobile users can now filter the pins on the `/around` map view. #2366
Expand All @@ -16,40 +18,51 @@
- Improve map JavaScript defensiveness.
- Upgrade jquery-validation plugin. #2540
- Pass ‘filter_category’ param to front page to pre-filter map.
- Remove on-map Permalink.
- Darken front page step numbers, and improve nested heading structure.
- Set report title autocomplete to off to prevent email autocompleting
- Remove on-map Permalink. #2631
- Darken front page step numbers, and improve nested heading structure. #2631
- Set report title autocomplete to off to prevent email autocompleting. #2518
- Add map filter debouncing to reduce server requests. #2675
- Add XSL to RSS feeds so they look nicer in browsers.
- Add XSL to RSS feeds so they look nicer in browsers. #2736
- Add per-report OpenGraph images. #2394
- Display GPS marker on /around map. #2359
- Use nicer default photo upload message. #2358
- Remove pan control from mobile widths. #2865
- Use category groups whenever category lists are shown. #2702
- Display map inline with duplicate suggestions on mobile. #2668
- Improved try again process on mobile. #2863
- Improve messaging/display of private reports.
- Improve messaging/display of private reports. #2884
- Add a web manifest and service worker. #2220
- Also check filter_category for category choice. #2893
- Reduce duplicate Permalink.updateLink calls when zooming map. #2824
- Hide ‘provide extra information’ preamble when no visible fields are present. #2811
- Improve user flow when JavaScript is not available. #2619
- Change ‘locate me automatically’ to ‘use my location’. #2615
- Include ‘submit’ button at very bottom of report form when signing in during report
- Provide ARIA roles for message controller box.
- Admin improvements:
- Add new roles system, to group permissions and apply to users. #2483
- Contact form emails now include user admin links.
- Contact form emails now include user admin links. #2608
- Allow categories/Open311 questions to disable the reporting form. #2599
- Improve category edit form. #2469
- Allow editing of category name. #1398
- Allow non-superuser staff to use 2FA, and optional enforcement of 2FA.
- Add optional enforced password expiry.
- Store a moderation history on admin report edit.
- Add user admin log page.
- Allow report as another user with only name.
- Allow staff users to sign other people up for alerts.
- Allow non-superuser staff to use 2FA, and optional enforcement of 2FA. #2701
- Add optional enforced password expiry. #2705
- Store a moderation history on admin report edit. #2722
- Add user admin log page. #2722
- Allow report as another user with only name. #2781
- Allow staff users to sign other people up for alerts. #2783
- Group categories on body page. #2850
- Add admin UI for managing web manifest themes. #2792
- Add a new "staff" contact state.
- Add a new "staff" contact state. #2891
- Store staff user when staff make anonymous report. #2802
- Record first time fixed/closed update sent to reporter in email.
- Pre-filter ‘all reports’ by area for inspectors
- show open311 failure details in admin report edit page. #2468
- New features:
- Categories can be listed under more than one group #2475
- OpenID Connect login support. #2523
- Heatmap dashboard. #2675
- Allow anonymous submission by a button, optionally per-category.
- Bugfixes:
- Prevent creation of two templates with same title. #2471
- Fix bug going between report/new pages client side. #2484
Expand All @@ -58,33 +71,35 @@
- Fix front-end testing script when run with Vagrant. #2514
- Handle missing category when sending open311 reports #2502
- Fix label associations with category groups. #2541
- Hide category extras when duplicate suggestions shown.
- Hide duplicate suggestions when signing in during reporting.
- Retain extra data if signing in during reporting.
- Have duplicate suggestion and assets coexist better.
- Don't include lat/lon of private reports in ‘Report another problem
here’ link.
- Allow contact send method to be unset always.
- Hide category extras when duplicate suggestions shown. #2588
- Hide duplicate suggestions when signing in during reporting. #2588
- Retain extra data if signing in during reporting. #2588
- Have duplicate suggestion and assets coexist better. #2589
- Don't include lat/lon of private reports in ‘Report another problem here’ link. #2605
- Allow contact send method to be unset always. #2622
- Fix z-index stacking bug that was causing unclickable RSS icons on /alert page. #2624
- Fix issue with inspector duplication workflow.
- Fix issue with inspector duplication workflow. #2678
- Fix removal of cached photos on moderation. #2696
- Checking of cached front page details against database. #2696
- Inconsistent display of mark private checkbox for staff users
- Clear user categories when staff access is removed. #2815
- Only trigger one change event on initial popstate.
- Only trigger one change event on initial popstate. #2862
- Fix error when hiding a user's updates with no confirmed updates. #2898
- Sort reporting categories in display order. #2704
- Do not clear asset attributes on category change.
- Development improvements:
- Upgrade the underlying framework and a number of other packages. #2473
- Add feature cobrand helper function.
- Add front-end testing support for WSL. #2514
- Allow cobrands to disable admin resending.
- Sass variables for default link colour and decoration.
- Allow cobrands to disable admin resending. #2553
- Sass variables for default link colour and decoration. #2538
- Make contact edit note optional on staging sites.
- Store email addresses report sent to on the report.
- Add configuration for setting Content-Security-Policy header.
- Add banner on staging website/emails, and STAGING_FLAGS option to hide it.
- Do not hard code site name in database fixture.
- Ensure OS dependencies are kept updated in development environments.
- Store email addresses report sent to on the report. #2730
- Add configuration for setting Content-Security-Policy header. #2759
- Add banner on staging website/emails, and STAGING_FLAGS option to hide it. #2784 #2820
- Do not hard code site name in database fixture. #2794
- Ensure OS dependencies are kept updated in development environments. #2886
- Enhance inactive scripts to act per-cobrand, or full deletion. #2827
- Open311 improvements:
- Support use of 'private' service definition <keywords> to mark
reports made in that category private. #2488
Expand All @@ -93,13 +108,13 @@
- Add new upload_files flag which sends files/photos as part of the
POST service request. #2495
- Allow description in email template with placeholder. #2470
- Do not store display-only extra fields on new reports.
- Support receiving updates from external source.
- Do not store display-only extra fields on new reports. #2560
- Support receiving updates from external source. #2521
- Improve JSON output of controller.
- unset external_status_code if blank in update
- unset external_status_code if blank in update. #2573
- Add support for account_id parameter to POST Service Request calls.
- Do not overwrite/remove protected meta data. #2598
- Spot multiple groups inside a <groups> element.
- Spot multiple groups inside a <groups> element. #2641
- Always update problem state from first comment #2832
- Backwards incompatible changes:
- The FixMyStreet templating code will now escape all variables by
Expand Down
2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -14,7 +14,7 @@ RSS alerts of problems in their area.

It was created in 2007 by [mySociety](https://www.mysociety.org/) for reporting
problems to UK councils and has been copied around the world. The FixMyStreet
Platform is now at version 2.6; see CHANGELOG.md for a version history.
Platform is now at version 3.0; see CHANGELOG.md for a version history.

## Installation

Expand Down
2 changes: 1 addition & 1 deletion bin/site-specific-install.sh
@@ -1,7 +1,7 @@
#!/bin/sh

# Set this to the version we want to check out
VERSION=${VERSION_OVERRIDE:-v2.6}
VERSION=${VERSION_OVERRIDE:-v3.0}

PARENT_SCRIPT_URL=https://github.com/mysociety/commonlib/blob/master/bin/install-site.sh

Expand Down
147 changes: 147 additions & 0 deletions docs/_posts/2020-03-04-v3.0.md
@@ -0,0 +1,147 @@
---
layout: post
title: Version 3.0
author: matthew
---

<div class="r" align="right">
<a data-flickr-embed="true" href="https://www.flickr.com/photos/iqremix/23232339432/" title="Follow the Yellow Line"><img src="https://live.staticflickr.com/750/23232339432_66b398ac1c.jpg" width="500" height="333" alt="Follow the Yellow Line"></a><script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script>
</div>

It has been quite a while since the last release, apologies, but today
we are happy to be releasing **version 3.0** of the FixMyStreet Platform,
which has a number of improvements.

### Front end improvements

* FixMyStreet can now be installed as a progressive web app. This means we've
added a web manifest (and an admin UI for managing this) and a basic service
worker that shows a page if you're offline, and continues the functionality of
allowing staff users to store and view their shortlisted reports offline.

If you serve your site over HTTPS, you will be able to add the website to
your homescreen (browsers may prompt the user) and have it work like an app.
This provides us with a solid base on which to continue improving this in
future, including hopefully adding functionality such as offline report drafting
through the web site.

<div class="r" style="height:217px;overflow:hidden">
<a href="https://user-images.githubusercontent.com/739624/58807589-406d1480-8610-11e9-8208-ca71e3e0424f.png">
<img style="margin-top:-450px" alt="Screenshot of mobile filters" src="https://user-images.githubusercontent.com/739624/58807589-406d1480-8610-11e9-8208-ca71e3e0424f.png">
</a>
</div>

* Various improvements have been made to the site on mobile – the "try again" process
is clearer, duplicate suggestions show an inline map, the photo upload message is
better, and map filters can now be accessed.

* Category groups are now used wherever a category list is shown – admin pages,
map filters, and so on; and you can pass a `filter_category` or `filter_group`
parameter to the front page or around page to pre-select that option, which
makes it easier to deep link to FixMyStreet from a page or form on another site.

* <img src="https://user-images.githubusercontent.com/4776/71002776-e4636c80-20d7-11ea-995b-72cfec673f73.png"
align="right" style="max-width:25%" hspace="8" alt="Screenshot of map geolocation blue dot">

If you use geolocation, your location will now be displayed on the map, as
shown in the screenshot.

* As asked for a few times on our mailing list, we now use a report's image as
its OpenGraph image on an individual report page when shared.

* We've added XSL to our RSS feeds which means browsers no longer display them
as raw XML but as a nice simple web page that explains its purpose. Before
and after shots below:

<div style="height:250px; overflow:hidden; text-align: center">
<img alt="RSS feed before changes, raw XML" hspace="8" align="top" style="max-width:40%" src="https://user-images.githubusercontent.com/154364/68796646-24ec4800-064b-11ea-8c21-607fc8198fb9.png">
<img alt="RSS feed after changes, looks much nicer" hspace="8" align="top" style="max-width:40%" src="https://user-images.githubusercontent.com/739624/68948832-a9a4a680-07b0-11ea-9e7b-4d53e329a35a.png">
</div>

### Security

All template variables are now automatically escaped by default, to help
protect against any future XSS vulnerabilities. We also rotate the user's
session ID after successful login, and scrub the admin description fields.

If any of your own templates outputs a variable that contains HTML that you
wish to continue to allow to display as HTML, you will need to alter your
template to escape the variable with the `safe` filter, e.g. `[% some_html |
safe %]`.

### Admin improvements

* FixMyStreet now has a new roles system, allowing you to create groups of
permissions and apply those roles to users.

<div class="r" style="height:280px;overflow:auto"><a href="https://user-images.githubusercontent.com/739624/65964516-0e03e480-e455-11e9-8c56-f4bb78096918.png"><img alt="Category edit form screenshot" src="https://user-images.githubusercontent.com/739624/65964516-0e03e480-e455-11e9-8c56-f4bb78096918.png"></a></div>

* The category edit form has been drastically improved; category names can now be
edited, categories can be listed under more than one group, and categories or
particular extra questions can disable the reporting form (for e.g. emergency
"please call" categories or questions).

* Two-factor authentication can be used by any staff member, and you can choose
to optionally enforce it for all staff.

* The admin report edit page now stores moderation history, like the front end,
and you can now view a user's admin log history.

* <img alt="Heatmap web page" src="https://user-images.githubusercontent.com/4776/72752804-0eb6b700-3bbb-11ea-915b-5afde5235014.png"
align="right" style="max-width:50%" hspace="8">

We've added a heatmap dashboard for staff users, which can show hotspots.
To enable this, you will need to add `heatmap: { yourcobrand: 1 }` to your
`COBRAND_FEATURES` configuration.

* There's a new "staff only" contact state, for categories that can only be
used by staff.

* Staff users can report as other users even if they only have a name,
and can sign other people up to alerts.

### Bugfixes

Of course there have been a lot of bugfixes as well. One I remember is
when going back to the initial state with popstate, a change event was
being triggered on every single option of the filter selects. This led
to a lot of change events running on the category/status multi-selects
which then needlessly repeated the same activities over and over. This
locked up the browser for seconds in locations with many categories.
Below is a chart showing browser performance before and after:

![Performance chart before bugfix, 12 seconds locked browser](https://user-images.githubusercontent.com/154364/73260403-bcc8ef00-41c1-11ea-87be-96a135f89453.png)
![Performance chart after bugfix, 0.2 seconds](https://user-images.githubusercontent.com/154364/73260623-32cd5600-41c2-11ea-9fa4-8122e9710440.png)

### Development improvements

We've upgraded the underlying framework and other packages, added a banner to
the staging website/emails to make it obvious when you're in development, added
configuration for admin resending, a Content-Security-Policy header, and
stopped hard coding the site name in the database fixture.

### Open311 improvements

* It is now possible for an external Open311 service to POST updates on a report
to FixMyStreet, rather than have FixMyStreet poll an external service for updates.

* Email templates can include a placeholder to include the description fetched
from the Open311 server in the update.

* Private reports are supported, in that an Open311 server can mark a category
as private which will then automatically mark all reports sent and received
in that category as private.

* Meta questions added in the admin can be marked as protected so that they
won't be overridden by data fetched from an Open311 server. This is useful
for e.g. an "emergency" question that the Open311 server does not care about.

### Upgrading

As mentioned above, but it is worth repeating, if any of your own templates
outputs a variable that contains HTML that you wish to continue to allow to
display as HTML, you will need to alter your template to escape the variable
with the `safe` filter, e.g. `[% some_html | safe %]`.

A full list of changes can be seen in the
[changelog](https://github.com/mysociety/fixmystreet/releases/tag/v3.0) as usual.

0 comments on commit d0c432d

Please sign in to comment.