A simple example of building a pipeline with a custom dockerfile.

[decadance-dance]

Contact Details [Optional]

kllisre14@gmail.com

System Information

ZENML_LOCAL_VERSION: 0.54.1
ZENML_SERVER_VERSION: 0.54.1
ZENML_SERVER_DATABASE: mysql
ZENML_SERVER_DEPLOYMENT_TYPE: other
ZENML_CONFIG_DIR: /home/kllis/.config/zenml
ZENML_LOCAL_STORE_DIR: /home/kllis/.config/zenml/local_stores
ZENML_SERVER_URL: http://localhost:8080/
ZENML_ACTIVE_REPOSITORY_ROOT: /home/kllis/font_recognition_mlops
PYTHON_VERSION: 3.10.13
ENVIRONMENT: wsl
SYSTEM_INFO: {'os': 'linux', 'linux_distro': 'ubuntu', 'linux_distro_like': 'debian', 'linux_distro_version': '22.04'}
ACTIVE_WORKSPACE: default
ACTIVE_STACK: 123
ACTIVE_USER: admin
TELEMETRY_STATUS: enabled
ANALYTICS_CLIENT_ID: 623c517e-e34b-47b9-9f42-268a2cbcfbc2
ANALYTICS_USER_ID: b9298ae9-a19a-4f96-88f0-423dd282ed0d
ANALYTICS_SERVER_ID: 2c0f6083-1226-49a9-b54e-a531ef2143c6
INTEGRATIONS: ['aws', 'evidently', 'kaniko', 'kubeflow', 'kubernetes', 'mlflow', 'pillow', 'scipy', 'seldon', 'sklearn', 'slack']
PACKAGES: {'brotli': '1.1.0', 'deprecated': '1.2.14', 'gitpython': '3.1.41', 'jinja2': '3.1.3', 'mako': '1.3.0', 'markdown': '3.5.2',
'markupsafe': '2.1.3', 'pyjwt': '2.7.0', 'pymysql': '1.0.3', 'pyyaml': '6.0.1', 'sqlalchemy': '1.4.41', 'sqlalchemy-utils': '0.38.3',
'absl-py': '1.4.0', 'aiofiles': '23.2.1', 'aiohttp': '3.9.1', 'aiokafka': '0.10.0', 'aiosignal': '1.3.1', 'alembic': '1.8.1', 'anyio':
'4.2.0', 'argparse': '1.4.0', 'arrow': '1.3.0', 'asttokens': '2.4.1', 'async-timeout': '4.0.3', 'attrs': '22.2.0', 'aws-profile-manager':
'0.7.3', 'azure-common': '1.1.28', 'azure-core': '1.29.7', 'azure-mgmt-core': '1.4.0', 'azure-mgmt-resource': '23.0.1', 'bcrypt': '4.0.1',
'blinker': '1.7.0', 'boto3': '1.34.22', 'botocore': '1.34.22', 'cachetools': '5.3.2', 'certifi': '2023.11.17', 'cffi': '1.16.0',
'charset-normalizer': '3.3.2', 'click': '8.1.3', 'click-params': '0.3.0', 'cloudpickle': '2.2.1', 'colorama': '0.4.6', 'comm': '0.2.1',
'configparser': '6.0.0', 'contextlib2': '21.6.0', 'contourpy': '1.2.0', 'copier': '8.1.0', 'cryptography': '41.0.7', 'cycler': '0.12.1',
'databricks-cli': '0.18.0', 'decorator': '5.1.1', 'dill': '0.3.7', 'distro': '1.9.0', 'docker': '6.1.3', 'docstring-parser': '0.15',
'dunamai': '1.19.0', 'entrypoints': '0.4', 'evidently': '0.3.3', 'exceptiongroup': '1.2.0', 'executing': '2.0.1', 'fastapi': '0.89.1',
'fastapi-utils': '0.2.1', 'fire': '0.5.0', 'flask': '3.0.1', 'fonttools': '4.47.2', 'frozenlist': '1.4.1', 'funcy': '2.0', 'gevent': '23.9.1',
'geventhttpclient': '2.0.2', 'gitdb': '4.0.11', 'google-api-core': '2.15.0', 'google-api-python-client': '1.12.11', 'google-auth': '2.26.2',
'google-auth-httplib2': '0.2.0', 'google-cloud-core': '2.4.1', 'google-cloud-storage': '2.14.0', 'google-crc32c': '1.5.0', 'google-pasta':
'0.2.0', 'google-resumable-media': '2.7.0', 'googleapis-common-protos': '1.62.0', 'greenlet': '3.0.3', 'grpcio': '1.60.0', 'gunicorn':
'21.2.0', 'h11': '0.14.0', 'httplib2': '0.19.1', 'httptools': '0.6.1', 'idna': '3.6', 'importlib-metadata': '4.13.0', 'importlib-resources':
'6.1.1', 'ipinfo': '5.0.0', 'ipython': '8.18.0', 'ipywidgets': '8.1.1', 'isodate': '0.6.1', 'itsdangerous': '2.1.2', 'jedi': '0.19.1',
'jinja2-ansible-filters': '1.3.2', 'jinja2-time': '0.2.0', 'jmespath': '1.0.1', 'joblib': '1.3.2', 'jsonschema': '4.21.0',
'jsonschema-specifications': '2023.12.1', 'jupyterlab-widgets': '3.0.9', 'kfp': '1.8.22', 'kfp-pipeline-spec': '0.1.16', 'kfp-server-api':
'1.8.5', 'kiwisolver': '1.4.5', 'kubernetes': '25.3.0', 'llvmlite': '0.41.1', 'markdown-it-py': '3.0.0', 'matplotlib': '3.8.2',
'matplotlib-inline': '0.1.6', 'mdurl': '0.1.2', 'mlflow': '2.9.2', 'mlserver': '1.3.5', 'mlserver-mlflow': '1.3.5', 'multidict': '6.0.4',
'multiprocess': '0.70.15', 'nltk': '3.8.1', 'numba': '0.58.1', 'numpy': '1.26.3', 'oauthlib': '3.2.2', 'orjson': '3.8.14', 'packaging':
'23.2', 'pandas': '2.1.4', 'parso': '0.8.3', 'passlib': '1.7.4', 'pathos': '0.3.1', 'pathspec': '0.12.1', 'patsy': '0.5.6', 'pexpect':
'4.9.0', 'pillow': '10.2.0', 'pip': '23.3.1', 'plotly': '5.18.0', 'plumbum': '1.8.2', 'pox': '0.3.3', 'ppft': '1.7.6.7', 'prometheus-client':
'0.19.0', 'prompt-toolkit': '3.0.36', 'protobuf': '3.20.3', 'protobuf3-to-dict': '0.1.5', 'psutil': '5.9.7', 'ptyprocess': '0.7.0',
'pure-eval': '0.2.2', 'py-grpc-prometheus': '0.7.0', 'pyarrow': '14.0.2', 'pyasn1': '0.5.1', 'pyasn1-modules': '0.3.0', 'pycparser': '2.21',
'pydantic': '1.10.14', 'pygments': '2.17.2', 'pynndescent': '0.5.11', 'pyparsing': '2.4.7', 'python-dateutil': '2.8.2', 'python-dotenv':
'1.0.0', 'python-multipart': '0.0.6', 'python-rapidjson': '1.14', 'pytz': '2023.3.post1', 'pyyaml-include': '1.3.2', 'querystring-parser':
'1.2.4', 'questionary': '2.0.1', 'referencing': '0.32.1', 'regex': '2023.12.25', 'requests': '2.31.0', 'requests-oauthlib': '1.3.1',
'requests-toolbelt': '0.10.1', 'rich': '13.7.0', 'rpds-py': '0.17.1', 'rsa': '4.9', 'ruff': '0.1.7', 's3transfer': '0.10.0', 'sagemaker':
'2.117.0', 'schema': '0.7.5', 'scikit-learn': '1.4.0', 'scipy': '1.11.4', 'setuptools': '68.2.2', 'six': '1.16.0', 'slack-sdk': '3.26.2',
'smdebug-rulesconfig': '1.0.1', 'smmap': '5.0.1', 'sniffio': '1.3.0', 'sqlalchemy2-stubs': '0.0.2a38', 'sqlmodel': '0.0.8', 'sqlparse':
'0.4.4', 'stack-data': '0.6.3', 'starlette': '0.22.0', 'starlette-exporter': '0.17.1', 'statsmodels': '0.14.1', 'strip-hints': '0.1.10',
'tabulate': '0.9.0', 'tenacity': '8.2.3', 'termcolor': '2.4.0', 'threadpoolctl': '3.2.0', 'tqdm': '4.66.1', 'traitlets': '5.14.1',
'tritonclient': '2.41.1', 'typer': '0.9.0', 'types-python-dateutil': '2.8.19.20240106', 'typing-extensions': '4.9.0', 'tzdata': '2023.4',
'umap-learn': '0.5.5', 'uritemplate': '3.0.1', 'urllib3': '1.26.18', 'uvicorn': '0.26.0', 'uvloop': '0.19.0', 'validators': '0.18.2',
'watchfiles': '0.21.0', 'wcwidth': '0.2.13', 'websocket-client': '1.7.0', 'websockets': '12.0', 'werkzeug': '3.0.1', 'wheel': '0.41.2',
'widgetsnbextension': '4.0.9', 'wrapt': '1.16.0', 'yarl': '1.9.4', 'zenml': '0.54.1', 'zipp': '3.17.0', 'zope.event': '5.0', 'zope.interface':
'6.1'}

CURRENT STACK

Name: 123
ID: 981e04f7-211d-49b9-9190-88872854c5c7
User: admin / b9298ae9-a19a-4f96-88f0-423dd282ed0d
Workspace: default / d5402bc8-e31a-4104-b2f8-3ddf1a9e179d

ORCHESTRATOR: docker

Name: docker
ID: f2ae61b1-86a4-421e-b22b-ac6be7063e05
Type: orchestrator
Flavor: local_docker
Configuration: {'run_args': {}}
User: admin / b9298ae9-a19a-4f96-88f0-423dd282ed0d
Workspace: default / d5402bc8-e31a-4104-b2f8-3ddf1a9e179d

ARTIFACT_STORE: default

Name: default
ID: 87f1bffd-5117-43b6-ad69-c1565083a075
Type: artifact_store
Flavor: local
Configuration: {'path': ''}
Workspace: default / d5402bc8-e31a-4104-b2f8-3ddf1a9e179d

IMAGE_BUILDER: docker-builder

Name: docker-builder
ID: 2cb96498-ab0c-47a1-9d50-d6874e2c4257
Type: image_builder
Flavor: local
Configuration: {}
User: admin / b9298ae9-a19a-4f96-88f0-423dd282ed0d
Workspace: default / d5402bc8-e31a-4104-b2f8-3ddf1a9e179d

What happened?

It would be great if someone could share a working example of building a pipeline using a custom Dockerfile. Perhaps this would be worth adding to the official documentation as well. Thanks.

Reproduction steps

No response

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[htahir1]

Good idea - Ill add it to our list of internal docs tickets