针对以下代码遇到此错误:
stage_string = [StringIndexer(inputCol=c, outputCol=c + "_string_encoded") for c in categorical_columns]stage_one_hot = [OneHotEncoder(inputCol=c + "_string_encoded", outputCol=c + "_one_hot") for c in categorical_columns]assembler = VectorAssembler(inputCols=feature_list, outputCol="features")rf = RandomForestClassifier(labelCol="output", featuresCol="features")pipeline = Pipeline(stages=[stage_string,stage_one_hot,assembler, rf]) pipeline.fit(df)
Cannot recognize a pipeline stage of type <class 'list'>.Traceback (most recent call last): File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/base.py", line 132, in fit return self._fit(dataset) File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/ml/pipeline.py", line 97, in _fit "Cannot recognize a pipeline stage of type %s." % type(stage))TypeError: Cannot recognize a pipeline stage of type <class 'list'>.
回答:
问题出在这一语句上pipeline = Pipeline(stages=[stage_string,stage_one_hot,assembler, rf])
,其中stage_string
和stage_one_hot
是PipelineStage
的列表,而assembler
和rf
是单个的管道阶段。
请按以下方式修改您的语句:
stages = stage_string + stage_one_hot + [assembler, rf]pipeline = Pipeline(stages=stages)