Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

authentication failed error on mongo using docker containers #311

Open
rooi opened this issue Sep 20, 2021 · 1 comment
Open

authentication failed error on mongo using docker containers #311

rooi opened this issue Sep 20, 2021 · 1 comment

Comments

@rooi
Copy link

rooi commented Sep 20, 2021

Hi,
Thanks for this repo. I'm running into an authentication failed error on mongo using the docker containers on macOS. Am I doing something wrong or could you, please, give pointers on how to solve this?
Thank you very much!

Software and Hardware Versions

Latest version with cpu

Problem description

ModelHub Connection Error
I traced it to a authentication failed on pymongo (see log below)

Steps to Reproduce the Problem

Docker on MacOS
Follow installation step using CPU (docker pull, docker compose ... up -d)

Expected Behavior

No connection error and being able to register a model

Other Information

The following log is obtained from the model-backend:

2021-09-20 18:25:20.974212: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-09-20 18:25:20.974785: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO: 172.28.0.1:57290 - "GET /api/v1/model?results=10&page=1&pagination%5Bcurrent%5D=1&pagination%5BpageSize%5D=10 HTTP/1.1" 307 Temporary Redirect
INFO: 172.28.0.1:57290 - "GET /api/v1/model HTTP/1.1" 307 Temporary Redirect
INFO: 172.28.0.1:57290 - "GET /api/v1/model/?results=10&page=1&pagination%5Bcurrent%5D=1&pagination%5BpageSize%5D=10 HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/venv/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 396, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/venv/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in call
return await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/fastapi/applications.py", line 179, in call
await super().call(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/applications.py", line 111, in call
await self.middleware_stack(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 181, in call
raise exc from None
File "/venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 159, in call
await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/middleware/cors.py", line 78, in call
await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/exceptions.py", line 82, in call
raise exc from None
File "/venv/lib/python3.7/site-packages/starlette/exceptions.py", line 71, in call
await self.app(scope, receive, sender)
File "/venv/lib/python3.7/site-packages/starlette/routing.py", line 566, in call
await route.handle(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/routing.py", line 227, in handle
await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/routing.py", line 41, in app
response = await func(request)
File "/venv/lib/python3.7/site-packages/fastapi/routing.py", line 183, in app
dependant=dependant, values=values, is_coroutine=is_coroutine
File "/venv/lib/python3.7/site-packages/fastapi/routing.py", line 135, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
File "/venv/lib/python3.7/site-packages/starlette/concurrency.py", line 34, in run_in_threadpool
return await loop.run_in_executor(None, func, *args)
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/venv/lib/python3.7/site-packages/modelci/app/v1/endpoints/model.py", line 30, in get_all_models
models = get_models(architecture=architecture, framework=framework, engine=engine, task=task, version=version)
File "/venv/lib/python3.7/site-packages/modelci/persistence/service.py", line 88, in get_models
return list(map(MLModel.parse_obj, models))
File "/venv/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
if len(self.__data) or self._refresh():
File "/venv/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
self.__send_message(q)
File "/venv/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
address=self.__address)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
exhaust=exhaust)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1465, in _retryable_read
exhaust=exhaust) as (sock_info,
File "/usr/lib/python3.7/contextlib.py", line 112, in enter
return next(self.gen)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1309, in _slaveok_for_server
with self._get_socket(server, session, exhaust=exhaust) as sock_info:
File "/usr/lib/python3.7/contextlib.py", line 112, in enter
return next(self.gen)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1247, in _get_socket
self.__all_credentials, checkout=exhaust) as sock_info:
File "/usr/lib/python3.7/contextlib.py", line 112, in enter
return next(self.gen)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 1231, in get_socket
sock_info = self._get_socket(all_credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 1281, in _get_socket
sock_info = self.connect(all_credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 1197, in connect
sock_info.check_auth(all_credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 793, in check_auth
self.authenticate(credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 810, in authenticate
auth.authenticate(credentials, self)
File "/venv/lib/python3.7/site-packages/pymongo/auth.py", line 673, in authenticate
auth_func(credentials, sock_info)
File "/venv/lib/python3.7/site-packages/pymongo/auth.py", line 295, in _authenticate_scram
res = sock_info.command(source, cmd)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 694, in command
exhaust_allowed=exhaust_allowed)
File "/venv/lib/python3.7/site-packages/pymongo/network.py", line 161, in command
parse_write_concern_error=parse_write_concern_error)
File "/venv/lib/python3.7/site-packages/pymongo/helpers.py", line 160, in _check_command_response
raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: Authentication failed., full error: {'ok': 0.0, 'errmsg': 'Authentication failed.', 'code': 18, 'codeName': 'AuthenticationFailed'}
INFO: 172.28.0.1:57294 - "GET /api/v1/model/ HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/venv/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 396, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/venv/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in call
return await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/fastapi/applications.py", line 179, in call
await super().call(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/applications.py", line 111, in call
await self.middleware_stack(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 181, in call
raise exc from None
File "/venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 159, in call
await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/middleware/cors.py", line 78, in call
await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/exceptions.py", line 82, in call
raise exc from None
File "/venv/lib/python3.7/site-packages/starlette/exceptions.py", line 71, in call
await self.app(scope, receive, sender)
File "/venv/lib/python3.7/site-packages/starlette/routing.py", line 566, in call
await route.handle(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/routing.py", line 227, in handle
await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/routing.py", line 41, in app
response = await func(request)
File "/venv/lib/python3.7/site-packages/fastapi/routing.py", line 183, in app
dependant=dependant, values=values, is_coroutine=is_coroutine
File "/venv/lib/python3.7/site-packages/fastapi/routing.py", line 135, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
File "/venv/lib/python3.7/site-packages/starlette/concurrency.py", line 34, in run_in_threadpool
return await loop.run_in_executor(None, func, *args)
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/venv/lib/python3.7/site-packages/modelci/app/v1/endpoints/model.py", line 30, in get_all_models
models = get_models(architecture=architecture, framework=framework, engine=engine, task=task, version=version)
File "/venv/lib/python3.7/site-packages/modelci/persistence/service.py", line 88, in get_models
return list(map(MLModel.parse_obj, models))
File "/venv/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
if len(self.__data) or self._refresh():
File "/venv/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
self.__send_message(q)
File "/venv/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
address=self.__address)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
exhaust=exhaust)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1465, in _retryable_read
exhaust=exhaust) as (sock_info,
File "/usr/lib/python3.7/contextlib.py", line 112, in enter
return next(self.gen)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1309, in _slaveok_for_server
with self._get_socket(server, session, exhaust=exhaust) as sock_info:
File "/usr/lib/python3.7/contextlib.py", line 112, in enter
return next(self.gen)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1247, in _get_socket
self.__all_credentials, checkout=exhaust) as sock_info:
File "/usr/lib/python3.7/contextlib.py", line 112, in enter
return next(self.gen)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 1231, in get_socket
sock_info = self._get_socket(all_credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 1281, in _get_socket
sock_info = self.connect(all_credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 1197, in connect
sock_info.check_auth(all_credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 793, in check_auth
self.authenticate(credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 810, in authenticate
auth.authenticate(credentials, self)
File "/venv/lib/python3.7/site-packages/pymongo/auth.py", line 673, in authenticate
auth_func(credentials, sock_info)
File "/venv/lib/python3.7/site-packages/pymongo/auth.py", line 295, in _authenticate_scram
res = sock_info.command(source, cmd)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 694, in command
exhaust_allowed=exhaust_allowed)
File "/venv/lib/python3.7/site-packages/pymongo/network.py", line 161, in command
parse_write_concern_error=parse_write_concern_error)
File "/venv/lib/python3.7/site-packages/pymongo/helpers.py", line 160, in _check_command_response
raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: Authentication failed., full error: {'ok': 0.0, 'errmsg': 'Authentication failed.', 'code': 18, 'codeName': 'AuthenticationFailed'}
INFO: 172.28.0.1:57302 - "GET /api/v1/ HTTP/1.1" 404 Not Found
INFO: 172.28.0.1:57302 - "GET /api/v1/model/ HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/venv/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 396, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/venv/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in call
return await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/fastapi/applications.py", line 179, in call
await super().call(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/applications.py", line 111, in call
await self.middleware_stack(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 181, in call
raise exc from None
File "/venv/lib/python3.7/site-packages/starlette/middleware/errors.py", line 159, in call
await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/middleware/cors.py", line 78, in call
await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/exceptions.py", line 82, in call
raise exc from None
File "/venv/lib/python3.7/site-packages/starlette/exceptions.py", line 71, in call
await self.app(scope, receive, sender)
File "/venv/lib/python3.7/site-packages/starlette/routing.py", line 566, in call
await route.handle(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/routing.py", line 227, in handle
await self.app(scope, receive, send)
File "/venv/lib/python3.7/site-packages/starlette/routing.py", line 41, in app
response = await func(request)
File "/venv/lib/python3.7/site-packages/fastapi/routing.py", line 183, in app
dependant=dependant, values=values, is_coroutine=is_coroutine
File "/venv/lib/python3.7/site-packages/fastapi/routing.py", line 135, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
File "/venv/lib/python3.7/site-packages/starlette/concurrency.py", line 34, in run_in_threadpool
return await loop.run_in_executor(None, func, *args)
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/venv/lib/python3.7/site-packages/modelci/app/v1/endpoints/model.py", line 30, in get_all_models
models = get_models(architecture=architecture, framework=framework, engine=engine, task=task, version=version)
File "/venv/lib/python3.7/site-packages/modelci/persistence/service.py", line 88, in get_models
return list(map(MLModel.parse_obj, models))
File "/venv/lib/python3.7/site-packages/pymongo/cursor.py", line 1207, in next
if len(self.__data) or self._refresh():
File "/venv/lib/python3.7/site-packages/pymongo/cursor.py", line 1124, in _refresh
self.__send_message(q)
File "/venv/lib/python3.7/site-packages/pymongo/cursor.py", line 1001, in __send_message
address=self.__address)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
exhaust=exhaust)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1465, in _retryable_read
exhaust=exhaust) as (sock_info,
File "/usr/lib/python3.7/contextlib.py", line 112, in enter
return next(self.gen)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1309, in _slaveok_for_server
with self._get_socket(server, session, exhaust=exhaust) as sock_info:
File "/usr/lib/python3.7/contextlib.py", line 112, in enter
return next(self.gen)
File "/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1247, in _get_socket
self.__all_credentials, checkout=exhaust) as sock_info:
File "/usr/lib/python3.7/contextlib.py", line 112, in enter
return next(self.gen)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 1231, in get_socket
sock_info = self._get_socket(all_credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 1281, in _get_socket
sock_info = self.connect(all_credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 1197, in connect
sock_info.check_auth(all_credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 793, in check_auth
self.authenticate(credentials)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 810, in authenticate
auth.authenticate(credentials, self)
File "/venv/lib/python3.7/site-packages/pymongo/auth.py", line 673, in authenticate
auth_func(credentials, sock_info)
File "/venv/lib/python3.7/site-packages/pymongo/auth.py", line 295, in _authenticate_scram
res = sock_info.command(source, cmd)
File "/venv/lib/python3.7/site-packages/pymongo/pool.py", line 694, in command
exhaust_allowed=exhaust_allowed)
File "/venv/lib/python3.7/site-packages/pymongo/network.py", line 161, in command
parse_write_concern_error=parse_write_concern_error)
File "/venv/lib/python3.7/site-packages/pymongo/helpers.py", line 160, in _check_command_response
raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: Authentication failed., full error: {'ok': 0.0, 'errmsg': 'Authentication failed.', 'code': 18, 'codeName': 'AuthenticationFailed'}
@rooi
Copy link
Author

rooi commented Sep 21, 2021

It seems creating the mongo user using the init_db.js file fails. Setting it manually using the following commands solves it

docker exec -it mongodb mongo

db = db.getSiblingDB('modelci');
db.createUser({user: "modelci", pwd: "modelci@2020", roles: ["readWrite"]});

I did not find a structural fix, but there are some mentions of using a chown to the docker volumes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@rooi