Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError when editing ressources #90

Open
amadzar opened this issue Sep 28, 2023 · 3 comments
Open

TypeError when editing ressources #90

amadzar opened this issue Sep 28, 2023 · 3 comments

Comments

@amadzar
Copy link

amadzar commented Sep 28, 2023

I am Using Ckan 2.10.1 and just installed ckanext-validation. I have linked the required presets and the required presets
scheming.dataset_schemas = ckanext.validation.examples:ckan_default_schema.json scheming.presets = ckanext.scheming:presets.json Ckanext.validation:presets.json
Now when I click on an existing resource and even when I am not changing anything but clicking on "Update resource" I get this error:
TypeError: str cannot be used as validator because it is not a user-defined function. Why is that and how can I fix this? I am quite new to the ckan world and it's also the first time working for me with ckanext-validation so maybe I am just missing something I don't know. Any help is much appreciated, thanks in advance!
@ThrawnCA
Copy link
Contributor

ThrawnCA commented Sep 30, 2023 •
edited

What is your ckanext-scheming version? Older versions may cause this error.

@Gutelvam
Copy link

I'm encountering the same error even after a clean installation. I've attempted this with CKAN version 2.10.1 and Python versions 3.10, 3.8.13, and 3.9.18, and in all cases, I encountered the same error when attempting to upload a file. Unfortunately, I don't have any additional information to provide about the error at this time. Here's the content of my ckan.ini file:

#
# CKAN configuration
#
# These are some of the configuration options available for your CKAN
# instance. Check the documentation in 'doc/configuration.rst' or at the
# following URL for a description of what they do and the full list of
# available options:
#
# http://docs.ckan.org/en/latest/maintaining/configuration.html
#
# The %(here)s variable will be replaced with the parent directory of this file
#
[DEFAULT]

## Default settings ############################################################
## This enables the `Flask-DebugToolbar
## <https://flask-debugtoolbar.readthedocs.io/>`_ in the web interface, makes
## Webassets serve unminified JS and CSS files, and enables CKAN templates'
## debugging features.
## 
## You will need to ensure the ``Flask-DebugToolbar`` python package is installed,
## by activating your ckan virtual environment and then running::
## 
##     pip install -r /usr/lib/ckan/default/src/ckan/dev-requirements.txt
## 
## If you are running CKAN on Apache, you must change the WSGI
## configuration to run a single process of CKAN. Otherwise
## the execution will fail with: ``AssertionError: The EvalException
## middleware is not usable in a multi-process environment``. Eg. change::
## 
##   WSGIDaemonProcess ckan_default display-name=ckan_default processes=2 threads=15
##   to
##   WSGIDaemonProcess ckan_default display-name=ckan_default threads=15
## 
## .. warning:: This option should be set to ``False`` for a public site.
##    With debug mode enabled, a visitor to your site could execute malicious
##    commands.
debug = false

[app:main]

## General settings ############################################################
use = egg:ckan
ckan.legacy_route_mappings = {}
config.mode = default

## Development settings ########################################################
ckan.devserver.host = localhost
ckan.devserver.port = 5000
ckan.devserver.threaded = false
ckan.devserver.multiprocess = 1
ckan.devserver.watch_patterns = 
ckan.devserver.ssl_cert = 
ckan.devserver.ssl_key = 

## Session settings ############################################################
ckan.user.last_active_interval = 600
cache_dir = /tmp/%(ckan.site_id)s
beaker.session.key = ckan
beaker.session.secret = OcRYZS4yk-N2rknfMu7-CdipiI8
beaker.session.auto = false
beaker.session.cookie_expires = false
# beaker.session.cookie_domain = .example.com
beaker.session.save_accessed_time = true
beaker.session.secure = false
beaker.session.timeout = 600

## Database settings ###########################################################
sqlalchemy.url = postgresql://ckan_default:12345@localhost/ckan_default
sqlalchemy.pool_pre_ping = true
# sqlalchemy.<OPTION> = 

## Site Settings ###############################################################
ckan.site_url = http://127.0.0.1:5000
apikey_header_name = X-CKAN-API-Key
ckan.cache_expires = 0
ckan.cache_enabled = false
ckan.mimetype_guess = file_ext
ckan.static_max_age = 3600
ckan.tracking_enabled = false
ckan.valid_url_schemes = http https ftp
ckan.requests.timeout = 5
ckan.hide_version = false
ckan.redirect_to_login_if_not_authorized = true

## Authorization Settings ######################################################
ckan.auth.anon_create_dataset = false
ckan.auth.create_unowned_dataset = false
ckan.auth.create_dataset_if_not_in_organization = true
ckan.auth.user_create_groups = true
ckan.auth.user_create_organizations = true
ckan.auth.user_delete_groups = true
ckan.auth.user_delete_organizations = true
ckan.auth.create_user_via_api = false
ckan.auth.create_user_via_web = true
ckan.auth.roles_that_cascade_to_sub_groups = admin
ckan.auth.public_user_details = true
ckan.auth.public_activity_stream_detail = false
ckan.auth.allow_dataset_collaborators = false
ckan.auth.allow_admin_collaborators = false
ckan.auth.allow_collaborators_to_change_owner_org = false
ckan.auth.create_default_api_keys = false
ckan.auth.login_view = user.login
ckan.auth.reveal_private_datasets = false
ckan.auth.enable_cookie_auth_in_api = true
ckan.auth.route_after_login = dashboard.datasets

## CSRF Protection #############################################################
WTF_CSRF_ENABLED = true
WTF_CSRF_CHECK_DEFAULT = true
WTF_CSRF_SECRET_KEY = AM3LDx_t1OKaABxkFbh_4mEPKxrAeRg3Ou_LxquSBX0
WTF_CSRF_METHODS = POST PUT PATCH DELETE
WTF_CSRF_FIELD_NAME = _csrf_token
WTF_CSRF_HEADERS = X-CSRFToken X-CSRF-Token
WTF_CSRF_TIME_LIMIT = 3600
WTF_CSRF_SSL_STRICT = true
WTF_I18N_ENABLED = true
ckan.csrf_protection.ignore_extensions = true

## Flask-Login Remember me cookie settings #####################################
REMEMBER_COOKIE_NAME = remember_token
REMEMBER_COOKIE_DURATION = 31536000
# REMEMBER_COOKIE_DOMAIN = .example.com
REMEMBER_COOKIE_PATH = /
REMEMBER_COOKIE_SECURE = false
REMEMBER_COOKIE_HTTPONLY = true
REMEMBER_COOKIE_REFRESH_EACH_REQUEST = false
REMEMBER_COOKIE_SAMESITE = None

## API Token Settings ##########################################################
api_token.nbytes = 32
api_token.jwt.encode.secret = string:%(beaker.session.secret)s
api_token.jwt.decode.secret = string:%(beaker.session.secret)s
api_token.jwt.algorithm = HS256

## Search Settings #############################################################
ckan.site_id = default
solr_url = http://127.0.0.1:8983/solr/ckan
solr_user = 
solr_password = 
ckan.search.remove_deleted_packages = true
ckan.search.solr_commit = true
ckan.search.show_all_types = dataset
ckan.search.default_include_private = true
ckan.search.default_package_sort = score desc, metadata_modified desc
search.facets = organization groups tags res_format license_id
search.facets.limit = 50
search.facets.default = 10
ckan.extra_resource_fields = 
ckan.search.rows_max = 1000
ckan.group_and_organization_list_max = 1000
ckan.group_and_organization_list_all_fields_max = 25
solr_timeout = 60

## Redis Settings ##############################################################
ckan.redis.url = redis://localhost:6379/0

## CORS Settings ###############################################################
ckan.cors.origin_allow_all = false
ckan.cors.origin_whitelist = 

################ ______DATASTORE SETTINGS_____#############
ckan.datastore.write_url = postgresql://ckan_default:12345@localhost/datastore_default
ckan.datastore.read_url = postgresql://datastore_default:12345@localhost/datastore_default

###############______XLOADER SETTINGS______##############
ckanext.xloader.api_token = eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJqdGkiOiI4TFdWQjF6cFFnOGxvMmdxU19mU2czUGZfbF85ZUMwSWRGX2JBRjlaREE0IiwiaWF0IjoxNjk2OTcxNTYxfQ.F-myzVPpq0TMrZe-Scd1HbElSbt_D6OKoCdha1LWCqc
ckanext.xloader.jobs_db.uri = postgresql://ckan_default:12345@localhost/ckan_default

#############_______VALIDATION SETTINGS______##############
scheming.dataset_schemas = ckanext.validation.examples:ckan_default_schema.json
scheming.presets = ckanext.scheming:presets.json
	               ckanext.validation:presets.json
ckanext.validation.run_on_create_async = True
ckanext.validation.run_on_update_async = True
ckanext.validation.formats = csv xlsx
ckanext.validation.show_badges_in_listings = True
ckanext.validation.default_validation_options={
    "skip_errors": ["blank-row", "duplicate-label"],
	}

## Plugins Settings ############################################################
ckan.plugins = activity datastore xloader validation scheming_datasets
ckan.resource_proxy.timeout = 5

## Front-End Settings ##########################################################
ckan.site_title = CKAN
ckan.site_description = 
ckan.site_intro_text = 
ckan.site_logo = /base/images/ckan-logo.png
ckan.site_about = 
ckan.theme = css/main
ckan.favicon = /base/images/ckan.ico
ckan.datasets_per_page = 20
package_hide_extras = 
ckan.dumps_url = 
ckan.dumps_format = 
ckan.recaptcha.publickey = 
ckan.recaptcha.privatekey = 
ckan.featured_groups = 
ckan.featured_orgs = 
ckan.default_group_sort = title
ckan.gravatar_default = identicon
ckan.debug_supress_header = false
ckan.homepage_style = 1
ckan.site_custom_css = 

## Resource Views Settings #####################################################
ckan.views.default_views = image_view datatables_view

## Theming Settings ############################################################
ckan.template_head_end = 
ckan.template_footer_end = 
ckan.template_title_delimiter = -
extra_template_paths = 
extra_public_paths = 
ckan.base_public_folder = public
ckan.base_templates_folder = templates
ckan.default.package_type = dataset
ckan.default.group_type = group
ckan.default.organization_type = organization
ckan.admin_tabs = {}

## Storage Settings ############################################################
ckan.storage_path = /var/lib/ckan/default
ckan.max_resource_size = 10
ckan.max_image_size = 2

## Uploader Settings ###########################################################
ckan.upload.user.types = 
ckan.upload.user.mimetypes = 
ckan.upload.group.types = 
ckan.upload.group.mimetypes = 

## Webassets Settings ##########################################################
ckan.webassets.path = 
ckan.webassets.use_x_sendfile = false

## User Settings ###############################################################
ckan.user_list_limit = 20
ckan.user_reset_landing_page = home.index

## Activity Streams Settings ###################################################
ckan.activity_streams_enabled = true
ckan.activity_streams_email_notifications = false
ckan.activity_list_limit = 31
ckan.activity_list_limit_max = 100
ckan.email_notifications_since = 2 days
ckan.hide_activity_from_users = %(ckan.site_id)s

## Feeds Settings ##############################################################
ckan.feeds.author_name = 
ckan.feeds.author_link = 
ckan.feeds.authority_name = 
ckan.feeds.date = 
ckan.feeds.limit = 20

## Internationalisation Settings ###############################################
ckan.locale_default = en
ckan.locales_offered = 
ckan.locales_filtered_out = 
ckan.locale_order = 
ckan.i18n_directory = 
ckan.i18n.extra_directory = 
ckan.i18n.extra_gettext_domain = 
ckan.i18n.extra_locales = 
ckan.i18n.rtl_languages = he ar fa_IR
ckan.i18n.rtl_theme = css/main-rtl
ckan.display_timezone = UTC
ckan.root_path = 
ckan.resource_formats = /usr/lib/ckan/default/src/ckan/ckan/config/resource_formats.json

## Form Settings ###############################################################
ckan.dataset.create_on_ui_requires_resources = true
package_new_return_url = 
package_edit_return_url = 
licenses_group_url = 

## Email settings ##############################################################
smtp.server = localhost
smtp.starttls = false
smtp.user = 
smtp.password = 
smtp.mail_from = 
smtp.reply_to = 
email_to = 
error_email_from = 

## Background Job Settings #####################################################
ckan.jobs.timeout = 180

## Logging configuration
[loggers]
keys = root, ckan, ckanext, werkzeug

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = WARNING
handlers = console

[logger_werkzeug]
level = WARNING
handlers = console
qualname = werkzeug
propagate = 0

[logger_ckan]
level = INFO
handlers = console
qualname = ckan
propagate = 0

[logger_ckanext]
level = DEBUG
handlers = console
qualname = ckanext
propagate = 0

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s] %(message)s

I've double-checked the version of ckanext-scheming, and it appears to be up to date at version 3.0.0. Here is a list of the libraries installed in the environment:

alembic==1.8.1
attrs==23.1.0
Babel==2.10.3
Beaker==1.11.0
bleach==5.0.1
blinker==1.5
boto3==1.28.62
botocore==1.31.62
cached-property==1.5.2
certifi==2021.10.8
chardet==5.2.0
charset-normalizer==2.0.12
-e git+https://github.com/ckan/ckan.git@9915ba0022b9a74a65e61c097b2fee584b044087#egg=ckan
ckanapi==4.7
-e git+https://github.com/ckan/ckanext-scheming.git@8646a9dce79aa0b5a46274271ca6330dc0870b92#egg=ckanext_scheming
-e git+https://github.com/frictionlessdata/ckanext-validation.git@1073c80dace453a404df3d5f1ac1ae86a88b5029#egg=ckanext_validation
ckanext-xloader==0.12.2
ckantoolkit==0.0.7
click==8.1.3
colorama==0.4.6
Deprecated==1.2.13
docopt==0.6.2
dominate==2.7.0
et-xmlfile==1.1.0
feedgen==0.9.0
Flask==2.0.3
Flask-Babel==1.0.0
Flask-Login==0.6.1
flask-multistatic==1.0
Flask-WTF==1.0.1
frictionless==5.0.0b9
greenlet==1.1.2
humanize==4.8.0
idna==3.3
ijson==3.2.3
importlib-metadata==4.11.3
isodate==0.6.1
itsdangerous==2.1.1
Jinja2==3.1.2
jmespath==1.0.1
jsonlines==4.0.0
jsonschema==4.19.1
jsonschema-specifications==2023.7.1
linear-tsv==1.1.0
lxml==4.9.1
Mako==1.2.2
Markdown==3.4.1
markdown-it-py==3.0.0
marko==2.0.0
MarkupSafe==2.0.1
mdurl==0.1.2
mypy==0.971
mypy-extensions==0.4.3
nose==1.3.7
openpyxl==3.1.2
packaging==21.3
passlib==1.7.4
petl==1.7.14
polib==1.1.1
psycopg2==2.9.3
Pygments==2.16.1
PyJWT==2.4.0
pyparsing==3.0.7
pysolr==3.9.0
python-dateutil==2.8.2
python-magic==0.4.27
python-slugify==8.0.1
pytz==2021.3
pytz-deprecation-shim==0.1.0.post0
PyUtilib==6.0.0
PyYAML==6.0
redis==4.1.4
referencing==0.30.2
requests==2.31.0
rfc3986==2.0.0
rich==13.6.0
rpds-py==0.10.4
rq==1.11.0
s3transfer==0.7.0
shellingham==1.5.3
simpleeval==0.9.13
simplejson==3.17.6
six==1.16.0
SQLAlchemy==1.4.41
sqlalchemy2-stubs==0.0.2a27
sqlparse==0.4.2
stringcase==1.2.0
tableschema==1.20.2
tabulate==0.9.0
tabulator==1.53.5
text-unidecode==1.3
tomli==2.0.1
typer==0.9.0
typing_extensions==4.3.0
tzdata==2022.1
tzlocal==4.2
unicodecsv==0.14.1
Unidecode==1.0.22
urllib3==1.26.9
validators==0.22.0
watchdog==2.1.6
webassets==2.0
webencodings==0.5.1
Werkzeug==2.0.3
wrapt==1.14.0
WTForms==3.0.1
xlrd==2.0.1
zipp==3.7.0
zope.interface==5.4.0

and for information, here is the error mensage when i try to upload something:

**2023-10-10 17:11:54,461 INFO  [ckan.config.middleware.flask_app]  304 /base/vendor/select2/select2.png render time 0.005 seconds
2023-10-10 17:12:03,877 ERROR [ckan.config.middleware.flask_app] str cannot be used as validator because it is not a user-defined function
Traceback (most recent call last):
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/navl/dictization_functions.py", line 246, in convert
    nargs = converter._code_.co_argcount
AttributeError: type object 'str' has no attribute '_code_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/ckan/default/lib/python3.10/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/ckan/default/lib/python3.10/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/usr/lib/ckan/default/lib/python3.10/site-packages/flask/views.py", line 84, in view
    return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs)
  File "/usr/lib/ckan/default/lib/python3.10/site-packages/flask/views.py", line 158, in dispatch_request
    return current_app.ensure_sync(meth)(*args, **kwargs)
  File "/usr/lib/ckan/default/src/ckan/ckan/config/middleware/../../views/resource.py", line 377, in post
    get_action(u'resource_update')(context, data)
  File "/usr/lib/ckan/default/src/ckan/ckan/logic/_init_.py", line 551, in wrapped
    result = _action(context, data_dict, **kw)
  File "/usr/lib/ckan/default/src/ckanext-validation/ckanext/validation/logic.py", line 544, in resource_update
    return up_func(context, data_dict)
  File "/usr/lib/ckan/default/src/ckan/ckan/logic/action/update.py", line 111, in resource_update
    updated_pkg_dict = _get_action('package_update')(context, pkg_dict)
  File "/usr/lib/ckan/default/src/ckan/ckan/logic/_init_.py", line 551, in wrapped
    result = _action(context, data_dict, **kw)
  File "/usr/lib/ckan/default/src/ckan/ckan/logic/action/update.py", line 311, in package_update
    data, errors = lib_plugins.plugin_validate(
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/plugins.py", line 327, in plugin_validate
    result = plugin.validate(context, data_dict, schema, action)
  File "/usr/lib/ckan/default/src/ckanext-scheming/ckanext/scheming/plugins.py", line 312, in validate
    return navl_validate(data_dict, schema, context)
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/navl/dictization_functions.py", line 305, in validate
    flat_data, errors = _validate(flattened, schema, validators_context)
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/navl/dictization_functions.py", line 356, in _validate
    convert(converter, key, converted_data, errors, context)
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/navl/dictization_functions.py", line 248, in convert
    raise TypeError(
TypeError: str cannot be used as validator because it is not a user-defined function**

Given this situation, I would greatly appreciate any assistance in resolving this issue. Thank you very much.

@Gutelvam Gutelvam mentioned this issue Oct 11, 2023
Open
@ThrawnCA
Copy link
Contributor

The example schema references the unicode validator when it should be unicode_safe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ThrawnCA @Gutelvam @amadzar