Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Someone please fix banyule_vic_gov_au #1964

Open
tkilik11 opened this issue Apr 11, 2024 · 4 comments
Open

[Feature]: Someone please fix banyule_vic_gov_au #1964

tkilik11 opened this issue Apr 11, 2024 · 4 comments
Labels
source defect Defect of a source reported

Comments

@tkilik11
Copy link

I propose a feature for:

Sources

Describe your wanted feature

Cant get this council to work

@tkilik11 tkilik11 added the enhancement New feature or request (excluding source request) label Apr 11, 2024
@5ila5 5ila5 added source defect Defect of a source reported and removed enhancement New feature or request (excluding source request) labels Apr 12, 2024
@5ila5
Copy link
Collaborator

5ila5 commented Apr 12, 2024

They use incapsular to prevent scraping, not sure if there is anything we can do, Could not fix it (will try later again maybe my ip is blocked now)

@ReneNulschDE
Copy link
Contributor

@5ila5 : I checked this too and you are right, the address to geolocation api is protected. I found a way (add http headers) to use at least the
geolocation_id: 4f7ebfca-1526-4363-8b87-df3103a10a87 option. Should I prepare a PR for this or do you want to keep it as source defect?

@5ila5
Copy link
Collaborator

5ila5 commented Apr 20, 2024

Making the source usable again should be a priority. So fixing the geolocation_id part and adding a comment in the md file would certainly help.

Just noticed this was already discussed here: #302

@ReneNulschDE
Copy link
Contributor

I have tried this in different ways and it is working for one geolocation-id only. I'll give up and will not create a PR. For the records, with this change I got it working for:

    - name: banyule_vic_gov_au
      args:
        geolocation_id: "4f7ebfca-1526-4363-8b87-df3103a10a87"

Change fetch(self) method from:

        calendar_request = calendar_session.get(
            self.OC_CALENDAR_URL,
            params={"geolocationid": self.geolocation_id, "ocsvclang": "en-AU"},
        )

to:

        calendar_request = calendar_session.get(
            self.OC_CALENDAR_URL,
            params={"geolocationid": self.geolocation_id, "ocsvclang": "en-AU"},
            headers={
                "user-agent": "Mozilla/5.0",
                "accept": "text/plain, */*; q=0.01",
                "Referer": "https://www.banyule.vic.gov.au",
            },
        )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
source defect Defect of a source reported
Projects
None yet
Development

No branches or pull requests

3 participants