[Bug]: MLRun on Docker fails to load artifacts

[patrickfleith]

Installation check

• I have read the installation guide.

Installation OS

Mac

Installation Method

Docker

Kubernetes Cluster Type

N/A - Docker

MLRun CE Version

mlrun/jupyter:1.3.0 mlrun/mlrun-ui:1.3.0

Issue Description

Couple of issue after installation that are probably related. I'll explain all things here so it can help.

Primary issue:

• I cannot access MLRun API (in http://localhost:8080/). When I open the link on the browser I get {"detail":"Not Found"}

other suspicious behaviours
Only 3 containers are running out of the 4.

Loading artifact lead to 500 error

When I click on the download icon, I get

But when I click on retry, then I obtain the confusion matrix as expected in the tutorial

I have the same behaviour for all artifact.

I paste below the details of the 500 error

{
  "data": {
    "detail": "AttributeError(\"'K8sHelper' object has no attribute 'v1api'\")"
  },
  "status": 500,
  "statusText": "Internal Server Error",
  "headers": {
    "connection": "keep-alive",
    "content-length": "76",
    "content-type": "application/json",
    "date": "Tue, 25 Apr 2023 12:20:00 GMT",
    "server": "nginx/1.21.6"
  },
  "config": {
    "url": "projects/quick-tutorial-jovyan/files",
    "method": "get",
    "headers": {
      "Accept": "application/json, text/plain, */*"
    },
    "params": {
      "path": "/home/jovyan/data/quick-tutorial-jovyan/auto-trainer-train/0/feature-importance.html"
    },
    "baseURL": "/mlrun/api/v1",
    "transformRequest": [
      null
    ],
    "transformResponse": [
      null
    ],
    "timeout": 0,
    "xsrfCookieName": "XSRF-TOKEN",
    "xsrfHeaderName": "X-XSRF-TOKEN",
    "maxContentLength": -1,
    "maxBodyLength": -1,
    "transitional": {
      "silentJSONParsing": true,
      "forcedJSONParsing": true,
      "clarifyTimeoutError": false
    }
  },
  "request": {}
}

Note that I can execute the entire quickstart notebook without issues.

Installation Logs

Replace this line with the installation logs.

Additional Information

No response

[alonmr]

Hi @patrickfleith, thank you for submitting the issue.
Indeed the mlrun-api service is missing from the compose.with-jupyter.yaml file.
I will update here once fixed.

[alonmr]

@patrickfleith, it seems I misunderstood and the API is actually running inside the jupyter container.
If you check http://localhost:8080/api/healthz it should work.
The failure here is related to the API trying to access the k8s helper even though it is not running inside Kubernetes.
Will update with more information once I find the root cause.

[patrickfleith]

Indeed, at http://localhost:8080/api/healthz get an answer:

{"version":"1.3.0","namespace":"","docker_registry":"","remote_host":"","mpijob_crd_version":"v1alpha1","ui_url":"http://localhost:8060","artifact_path":"/home/jovyan/data","feature_store_data_prefixes":null,"spark_app_image":"","spark_app_image_tag":"","spark_history_server_path":"","spark_operator_version":"spark-3","kfp_image":"mlrun/mlrun:1.3.0","kfp_url":null,"dask_kfp_image":"mlrun/ml-base:1.3.0","api_url":"","nuclio_version":"1.11.17","ui_projects_prefix":null,"scrape_metrics":null,"hub_url":null,"default_function_node_selector":null,"igz_version":null,"auto_mount_type":null,"auto_mount_params":null,"default_function_priority_class_name":null,"valid_function_priority_class_names":null,"default_tensorboard_logs_path":null,"default_function_pod_resources":null,"preemptible_nodes_node_selector":null,"preemptible_nodes_tolerations":null,"default_preemption_mode":null,"force_run_local":null,"function":null,"redis_url":"","redis_type":"standalone","sql_url":"","ce_mode":"","ce":{"mode":"","release":""},"calculate_artifact_hash":"True","generate_artifact_target_path_from_artifact_hash":null,"logs":null}

[alonmr]

Hi @patrickfleith, Thank you for submitting the issue. MLRun main focus is with Kubernetes there you can unlock more value and features. I suggest using MLRun Community Edition instead which can be installed on your local machine with Minikube, Kubernetes for docker desktop, etc. The bug you found will receive more attention once we have better availability for it.