Model Container Build Service Failing
Summary
When containerizing a model, the result is always FAILED.
Steps to reproduce
- Using Mantik GUI navigate to a successful run which produced an artifact.
- Register the run as a model
- Containerize the model
Software versions
- Device model:
- OS version:
- Software versions:
- Browser version:
What is the current bug behavior?
The CodeBuild always fails. See the logs section.
What is the expected correct behavior?
Codebuild should succeed and models should be cotianerized.
Relevant logs and/or screenshots
LOGS
2024-03-28T10:23:34.068+01:00 #7 54.88 Processing triggers for libc-bin (2.31-0ubuntu9.14) ...
2024-03-28T10:23:34.068+01:00 #7 54.90 Errors were encountered while processing:
2024-03-28T10:23:34.068+01:00 #7 54.90 openjdk-11-jre-headless:amd64
2024-03-28T10:23:34.068+01:00 #7 54.93 E: Sub-process /usr/bin/dpkg returned an error code (1)
2024-03-28T10:23:34.068+01:00 #7 ERROR: executor failed running [/bin/sh -c DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends wget curl nginx ca-certificates bzip2 build-essential cmake openjdk-8-jdk git-core maven && rm -rf /var/lib/apt/lists/*]: exit code: 100
2024-03-28T10:23:34.068+01:00 ------
2024-03-28T10:23:34.068+01:00 > [ 3/21] RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends wget curl nginx ca-certificates bzip2 build-essential cmake openjdk-8-jdk git-core maven && rm -rf /var/lib/apt/lists/*:
2024-03-28T10:23:34.068+01:00 54.83 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsgen to provide /usr/bin/wsgen (wsgen) in auto mode
2024-03-28T10:23:34.068+01:00 54.83 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jcmd to provide /usr/bin/jcmd (jcmd) in auto mode
2024-03-28T10:23:34.068+01:00 54.84 Setting up openjdk-8-jdk:amd64 (8u402-ga-2ubuntu1~20.04) ...
2024-03-28T10:23:34.068+01:00 54.85 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer to provide /usr/bin/appletviewer (appletviewer) in auto mode
2024-03-28T10:23:34.068+01:00 54.85 update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jconsole to provide /usr/bin/jconsole (jconsole) in auto mode
2024-03-28T10:23:34.068+01:00 54.86 Processing triggers for libgdk-pixbuf2.0-0:amd64 (2.40.0+dfsg-3ubuntu0.4) ...
2024-03-28T10:23:34.068+01:00 54.88 Processing triggers for libc-bin (2.31-0ubuntu9.14) ...
2024-03-28T10:23:34.068+01:00 54.90 Errors were encountered while processing:
2024-03-28T10:23:34.068+01:00 54.90 openjdk-11-jre-headless:amd64
2024-03-28T10:23:34.068+01:00 54.93 E: Sub-process /usr/bin/dpkg returned an error code (1)
2024-03-28T10:23:34.068+01:00 ------
2024-03-28T10:23:34.068+01:00 ERROR: failed to solve: executor failed running [/bin/sh -c DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y --no-install-recommends wget curl nginx ca-certificates bzip2 build-essential cmake openjdk-8-jdk git-core maven && rm -rf /var/lib/apt/lists/*]: exit code: 100
2024-03-28T10:23:34.068+01:00 Traceback (most recent call last):
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/bin/mlflow", line 8, in <module>
2024-03-28T10:23:34.068+01:00 sys.exit(cli())
2024-03-28T10:23:34.068+01:00 ^^^^^
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
2024-03-28T10:23:34.068+01:00 return self.main(*args, **kwargs)
2024-03-28T10:23:34.068+01:00 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/click/core.py", line 1078, in main
2024-03-28T10:23:34.068+01:00 rv = self.invoke(ctx)
2024-03-28T10:23:34.068+01:00 ^^^^^^^^^^^^^^^^
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
2024-03-28T10:23:34.068+01:00 return _process_result(sub_ctx.command.invoke(sub_ctx))
2024-03-28T10:23:34.068+01:00 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
2024-03-28T10:23:34.068+01:00 return _process_result(sub_ctx.command.invoke(sub_ctx))
2024-03-28T10:23:34.068+01:00 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
2024-03-28T10:23:34.068+01:00 return ctx.invoke(self.callback, **ctx.params)
2024-03-28T10:23:34.068+01:00 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/click/core.py", line 783, in invoke
2024-03-28T10:23:34.068+01:00 return __callback(*args, **kwargs)
2024-03-28T10:23:34.068+01:00 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/mlflow/models/cli.py", line 270, in build_docker
2024-03-28T10:23:34.068+01:00 build_docker_api(
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/mlflow/models/__init__.py", line 70, in build_docker
2024-03-28T10:23:34.068+01:00 get_flavor_backend(model_uri, docker_build=True, env_manager=env_manager).build_image(
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/mlflow/pyfunc/backend.py", line 347, in build_image
2024-03-28T10:23:34.068+01:00 _build_image(
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/mlflow/models/docker_utils.py", line 224, in _build_image
2024-03-28T10:23:34.068+01:00 _build_image_from_context(context_dir=cwd, image_name=image_name)
2024-03-28T10:23:34.068+01:00 File "/root/.pyenv/versions/3.11.6/lib/python3.11/site-packages/mlflow/models/docker_utils.py", line 250, in _build_image_from_context
2024-03-28T10:23:34.068+01:00 raise RuntimeError("Docker build failed.")
2024-03-28T10:23:34.068+01:00 RuntimeError: Docker build failed.
2024-03-28T10:23:34.068+01:00
2024-03-28T10:23:34.068+01:00 [Container] 2024/03/28 09:23:33.962369 Command did not exit successfully mlflow models build-docker --model-uri ${MODEL_URI} --name ${IMAGE_NAME} --env-manager local exit status 1
2024-03-28T10:23:34.068+01:00 [Container] 2024/03/28 09:23:33.974444 Phase complete: BUILD State: FAILED
Possible fixes
/cc @rico.berner /cc @fabian.emmerich