我试图在Azure ML服务上部署使用PySpark训练的ALS模型。我提供了一个score.py文件,该文件使用ALSModel.load()函数加载训练模型。以下是我score.py文件的代码。
import osfrom azureml.core.model import Modelfrom pyspark.ml.recommendation import ALS, ALSModelfrom pyspark.sql.types import StructType, StructFieldfrom pyspark.sql.types import DoubleType, StringTypefrom pyspark.sql import SQLContextfrom pyspark import SparkContextsc = SparkContext.getOrCreate()sqlContext = SQLContext(sc)spark = sqlContext.sparkSessioninput_schema = StructType([StructField("UserId", StringType())])reader = spark.readreader.schema(input_schema)def init(): global model # 注意这里的"iris.model"是在工作区下注册的模型名称 # 此调用应返回本地磁盘上model.pkl文件的路径。 model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), "recommendation-model") # 将模型文件加载回LogisticRegression模型 model = ALSModel.load(model_path) def run(data): try: input_df = reader.json(sc.parallelize([data])) input_df = indexer.transform(input_df) res = model.recommendForUserSubset(input_df[['UserId_index']], 10) # 只要是JSON可序列化的数据类型,您都可以返回 return result.collect()[0]['recommendations'] except Exception as e: traceback.print_exc() error = str(e) return error
当我使用Azure ML服务中的Model.deploy函数将其部署为LocalWebService时,我得到以下错误
Generating Docker build context.Package creation SucceededLogging into Docker registry viennaglobal.azurecr.ioLogging into Docker registry viennaglobal.azurecr.ioBuilding Docker image from Dockerfile...Step 1/5 : FROM viennaglobal.azurecr.io/azureml/azureml_43542b56c5ec3e8d0f68e1556558411f ---> 5b3bb174ca5fStep 2/5 : COPY azureml-app /var/azureml-app ---> 8e540c0746f7Step 3/5 : RUN mkdir -p '/var/azureml-app' && echo eyJhY2NvdW50Q29udGV4dCI6eyJzdWJzY3JpcHRpb25JZCI6IjNkN2M1ZjM4LTI1ODEtNGUxNi05NTdhLWEzOTU1OGI1ZjBiMyIsInJlc291cmNlR3JvdXBOYW1lIjoiZGV2LW9tbmljeC10ZnMtYWkiLCJhY2NvdW50TmFtZSI6ImRldi10ZnMtYWktd29ya3NwYWNlIiwid29ya3NwYWNlSWQiOiI1NjkzNGMzNC1iZmYzLTQ3OWUtODRkMy01OGI4YTc3ZTI4ZjEifSwibW9kZWxzIjp7fSwibW9kZWxzSW5mbyI6e319 | base64 --decode > /var/azureml-app/model_config_map.json ---> Running in 502ad8edf91e ---> a1bc5e0283d0Step 4/5 : RUN mv '/var/azureml-app/tmpvxhomyin.py' /var/azureml-app/main.py ---> Running in eb4ec1a0b702 ---> 6a3296fe6420Step 5/5 : CMD ["runsvdir","/var/runit"] ---> Running in 834fd746afef ---> 5b9f8be538c0Successfully built 5b9f8be538c0Successfully tagged recommend-service:latestContainer (name:musing_borg, id:0f3163692f5119685eee5ed59c8e00aa96cd472f765e7db67653f1a6ce852e83) cannot be killed.Container has been successfully cleaned up.Image sha256:0f146f4752878bbbc0e876f4477cc2877ff12a366fca18c986f9a9c2949d028b successfully removed.Starting Docker container...Docker container running.Checking container health...ERROR - Error: Container has crashed. Did your init method fail?Container Logs:/bin/bash: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libtinfo.so.5: no version information available (required by /bin/bash)/bin/bash: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libtinfo.so.5: no version information available (required by /bin/bash)/bin/bash: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libtinfo.so.5: no version information available (required by /bin/bash)/bin/bash: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libtinfo.so.5: no version information available (required by /bin/bash)2020-07-30T11:57:00,312735664+00:00 - rsyslog/run 2020-07-30T11:57:00,312768364+00:00 - gunicorn/run 2020-07-30T11:57:00,313017966+00:00 - iot-server/run bash: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libtinfo.so.5: no version information available (required by bash)2020-07-30T11:57:00,313969073+00:00 - nginx/run /usr/sbin/nginx: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libcrypto.so.1.0.0: no version information available (required by /usr/sbin/nginx)/usr/sbin/nginx: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libcrypto.so.1.0.0: no version information available (required by /usr/sbin/nginx)/usr/sbin/nginx: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libssl.so.1.0.0: no version information available (required by /usr/sbin/nginx)/usr/sbin/nginx: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libssl.so.1.0.0: no version information available (required by /usr/sbin/nginx)/usr/sbin/nginx: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libssl.so.1.0.0: no version information available (required by /usr/sbin/nginx)EdgeHubConnectionString and IOTEDGE_IOTHUBHOSTNAME are not set. Exiting.../bin/bash: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libtinfo.so.5: no version information available (required by /bin/bash)2020-07-30T11:57:00,597835804+00:00 - iot-server/finish 1 02020-07-30T11:57:00,598826211+00:00 - Exit code 1 is normal. Not restarting iot-server.Starting gunicorn 19.9.0Listening at: http://127.0.0.1:31311 (10)Using worker: syncworker timeout is set to 300Booting worker with pid: 41bash: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libtinfo.so.5: no version information available (required by bash)bash: /azureml-envs/azureml_7fbe163ce1d4208cd897650a64b7a54d/lib/libtinfo.so.5: no version information available (required by bash)
然而,在Jupyter笔记本中执行时,ALSModel.load()可以正常工作。
回答:
需要检查的几件事: