from pyspark.ml.regression import RandomForestRegressionModelrf = RandomForestRegressor(labelCol="label",featuresCol="features", numTrees=5, maxDepth=10, seed=42)rf_model = rf.fit(train_df)rf_model_path = "./hdfsData/" + "rfr_model"rf_model.save(rf_model_path)
当我第一次尝试保存模型时,这些代码是有效的。但是当我再次尝试将模型保存到相同的路径时,出现了以下错误:
Py4JJavaError: An error occurred while calling o1695.save. : java.io.IOException: Path ./hdfsData/rfr_model already exists. Please use write.overwrite().save(path) to overwrite it.
然后我尝试了:
rf_model.write.overwrite().save(rf_model_path)
结果出现了:
AttributeError: ‘function’ object has no attribute ‘overwrite’
看起来pyspark.mllib
模块提供了overwrite功能,但pyspark.ml
模块没有。如果我想用新模型覆盖旧模型,有人知道如何解决这个问题吗?谢谢。
回答:
你看到的消息是Java错误消息,而不是Python的。你应该先调用write
方法:
rf_model.write().overwrite().save(rf_model_path)