我在使用sklearn
中的test_train_split
模块来生成用于训练和测试的数据集的随机组合。我有一份指向原始数据集的文件路径列表。我还想知道数据是如何被打乱的,或者以同样的方式打乱文件路径列表,以便在打乱/分割的数据集可用后能够追踪文件路径?
回答:
如果你在test_train_split
和shuffle
中指定相同的random_state
,你将得到相同的顺序。
请看下面的代码片段进行演示。
from sklearn.utils import shufflefrom sklearn.model_selection import train_test_splitimport randomX = list()y = list()files = list()random_state = 42test_size = 0.9n = 100for i in range(n): X.append(i) y.append(i + random.random()) files.append('file_{0:02d}.csv'.format(i))X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=test_size, random_state=random_state)X_shuffle = shuffle(X, random_state=random_state)y_shuffle = shuffle(y, random_state=random_state)file_shuffle = shuffle(files, random_state=random_state)print(X_train)print(X_shuffle[int(n * test_size):])print(y_shuffle[int(n * test_size):])print(file_shuffle[int(n * test_size):])print(X_train == X_shuffle[int(n * test_size):])
[91, 74, 86, 82, 20, 60, 71, 14, 92, 51][91, 74, 86, 82, 20, 60, 71, 14, 92, 51][91.64119581793204, 74.77493553783724, 86.62410189510936, 82.40452263996107, 20.22784747831378, 60.913989700418675, 71.1940538438253, 14.644282494118647, 92.97808337955185, 51.289858815186356]['file_91.csv', 'file_74.csv', 'file_86.csv', 'file_82.csv', 'file_20.csv', 'file_60.csv', 'file_71.csv', 'file_14.csv', 'file_92.csv', 'file_51.csv']True