fetch_url always caches resolved URLs

[chreke]

The util.url.fetch_url and util.url.fetch_url_text functions are supposed to have an optional cache ("Mapping cache: An optional cache. If the URL can be found in the cache, return the cache contents.", as per the documentation)

However, in the current implementation there will always be a cache, which I believe was not the intention.

The root cause is using {} as default for the cache argument; rather than re-instantiating a new dictionary every time the function is called, this re-uses the same dictionary between function calls.

See e.g. https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments for details

A simple fix would be:

def fetch_url(url, cache = None, encoding = None):
  cache = cache or {}

Expected Behaviour

URL contents should not be cached unless the cache argument is provided.

Minimal Example Spec

Actual Behaviour

URL contents get cached if no cache argument is provided.

Steps to Reproduce

Given a file test.yaml with the following contents:

openapi: 3.0.2
info:
  title: Test
  version: '0.0.1'

and the following program:

import os
from prance.util.url import absurl
from prance.util.fs import abspath
from prance.util.url import fetch_url

url = absurl("test.yaml", abspath(os.getcwd()))
fetch_url(url)

we get the following output:

{'openapi': '3.0.2', 'info': {'title': 'Test', 'version': '0.0.1'}}

Change the file to e.g.

openapi: 3.0.2
info:
  title: Foobar
  version: '0.0.1'

and then re-run the fetch_url(url) part. I would expect the output to change, but you still get:

{'openapi': '3.0.2', 'info': {'title': 'Test', 'version': '0.0.1'}}

Environment

• OS: MacOS 10.15.7
• Python version: 3.8
• Swagger/OpenAPI version used: 3.02
• Backend: openapi-spec-validator

@jfinkhaeuser

[jfinkhaeuser]

You're absolutely right! That's one of the things in Python I keep forgetting about. There will be similar issues; I'll go through the code to find them.