Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Policy quality rating treats 'Unclear' rating for Measurable as 'Yes' instead of 'No' (requires update for 1000 Cities Challenge) #393

Closed
carlhiggs opened this issue Mar 15, 2024 · 0 comments
Assignees

Comments

@carlhiggs
Copy link
Member

Describe the bug
When rating policy for our previous 25 city study, we treated 'unclear' as evidence taking a 'benefit of the doubt' approach to evaluating measurability. We decided to revise that approach in the 1000 Cities challenge (if 'Unclear', then it is not measurable; which is more intuitive). However, when validating manual quality scoring conducted by @mdlowe it was found that the old method was still being used.

In addition, the policy "Transport and planning combined in one government department" was counted towards quality score, but should be omitted from this.

To Reproduce
Using completed policy checklists, the scoring can be checked:
from subprocesses._utils import get_policy_presence_quality_score_dictionary

xlsx = './data/policy_review/2024-02-28  - Completed policy checklists/manual_gohsc-policy-indicator-checklist_Final (1).xlsx'
get_policy_presence_quality_score_dictionary(xlsx)
# {'presence': {'numerator': 26, 'denominator': 32}, 'quality': {'numerator': 20.5, 'denominator': 62}}
## This is similar to score calculated independently by Melanie shared on 28 Feb; in the manual version, quality scored 19.

By making the above changes, the policy score is corrected

>>> get_policy_presence_quality_score_dictionary(xlsx)
{'presence': {'numerator': 26, 'denominator': 32}, 'quality': {'numerator': 19.0, 'denominator': 62}}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant