我正在使用AWS SageMaker的随机切割森林算法来检测异常。

import boto3import sagemakercontainers = {    'us-west-2': '174872318107.dkr.ecr.us-west-2.amazonaws.com/randomcutforest:latest',    'us-east-1': '382416733822.dkr.ecr.us-east-1.amazonaws.com/randomcutforest:latest',    'us-east-2': '404615174143.dkr.ecr.us-east-2.amazonaws.com/randomcutforest:latest',    'eu-west-1': '438346466558.dkr.ecr.eu-west-1.amazonaws.com/randomcutforest:latest',    'ap-southeast-1':'475088953585.dkr.ecr.ap-southeast-1.amazonaws.com/randomcutforest:latest'    }region_name = boto3.Session().region_namecontainer = containers[region_name]session = sagemaker.Session()rcf = sagemaker.estimator.Estimator(    container,    sagemaker.get_execution_role(),    output_path='s3://{}/{}/output'.format(bucket, prefix),    train_instance_count=1,    train_instance_type='ml.c5.xlarge',    sagemaker_session=session)rcf.set_hyperparameters(    num_samples_per_tree=200,    num_trees=250,    feature_dim=1,    eval_metrics =["accuracy", "precision_recall_fscore"])s3_train_input = sagemaker.session.s3_input(    s3_train_data,    distribution='ShardedByS3Key',    content_type='application/x-recordio-protobuf')rcf.fit({'train': s3_train_input})

(参考自 –> https://aws.amazon.com/blogs/machine-learning/use-the-built-in-amazon-sagemaker-random-cut-forest-algorithm-for-anomaly-detection/)

使用上述代码训练了模型，但没有找到评估模型的方法。如何在部署模型后获取准确率和F分数？

回答：

为了获取评估指标，您需要在训练过程中提供一个额外的名为“test”的通道。测试通道必须包含带标签的数据。这在官方文档中有所解释，https://docs.aws.amazon.com/sagemaker/latest/dg/randomcutforest.html ：

Amazon SageMaker的随机切割森林支持训练和测试数据通道。可选的测试通道用于在带标签的数据上计算准确率、精确率、召回率和F1分数指标。训练和测试数据的内容类型可以是application/x-recordio-protobuf或text/csv格式。对于使用text/csv格式的测试数据，内容必须指定为text/csv;label_size=1，其中每行的第一列代表异常标签：“1”表示异常数据点，“0”表示正常数据点。您可以使用文件模式或管道模式在格式为recordIO-wrapped-protobuf或CSV的数据上训练RCF模型

另外请注意…测试通道仅支持S3DataDistributionType=FullyReplicated

谢谢，

Julio