是否有便捷的方法将数据集分割成训练集和测试集，同时保持属于同一组的记录在一起？

例如，有一张表记录了每个person_id的独立变量和因变量，每个人可能有一条或多条记录：

import pandas as pdtbl = pd.DataFrame(dict(        person_id=list('aaabbcccdeeefffhiijj'),        random_variable=np.linspace(0, 1, 20),        dependent_variable=np.arange(20)    ))

现在，我想将数据分割成训练集和测试集，并确保同一人的记录在同一数据集中。显然，使用sklearn.cross_validation.train_test_split无法完成这项任务。我知道有sklearn.cross_validation.LeavePLabelOut，但它会创建所有可能的组合，而不是单一的分割，这并不是我目前想要的。

另一种方法是基于person_id字段计算一个哈希值，并用它进行抽样：

import numpy as npsalt = str(np.random.rand()) # 随机源 hash_values = tbl['person_id'].apply(lambda p: hash(salt + p) % 100)# 50/50 分割sel_training = hash_values < 50training_set = tbl.loc[sel_training]testing_set = tbl.loc[-sel_training]

有没有更优雅的方法来实现这个任务？

回答：

我最终编写了自己的交叉验证类来实现你所描述的功能。以下是代码（抱歉代码不是很整洁）。

class StratifiedKFold_ByColumn( object ):    def __init__( self, n_folds, X, y, colname ):        groupable = pd.concat( [X[colname], y], axis=1 )        grouped = groupable.groupby( [colname] ).aggregate( max )        self.column = X[colname]        self.colname = colname        # import pdb; pdb.set_trace()        self.folds = [            (train,val) for (train,val) in            sklearn.cross_validation.StratifiedKFold( y=grouped.values[:,0], n_folds=n_folds, shuffle=True )            ]        self.n_folds = n_folds        self.i = 0        self.y=y        # self.test()    def __len__(self):        return self.n_folds    def __iter__( self ):        self.i = 0        return self    def test( self ):        for train,val in self.folds:            train_mask = self.column.isin( train )            val_mask = self.column.isin( val )            print 'train:',self.y[train_mask].sum(), (1-self.y[train_mask]).sum()            print 'val:',self.y[val_mask].sum(), (1-self.y[val_mask]).sum()    def next( self ):        if self.i < self.n_folds:            train,val = self.folds[self.i]            self.i += 1            # import pdb; pdb.set_trace()            train_mask = self.column.isin( train )            val_mask = self.column.isin( val )            y_train = self.y[train_mask]            X_train = self.column[train_mask]            n_tr_1 = (y_train!=0).sum()            n_tr_0 = (y_train==0).sum()            # import pdb; pdb.set_trace()            assert n_tr_1 < n_tr_0            stride = n_tr_0/n_tr_1            X_train_1 = X_train[y_train!=0]            y_train_1 = y_train[y_train!=0]            X_train_0 = X_train[y_train==0]            y_train_0 = y_train[y_train==0]            train_idxs = []            for i_1 in range(0,n_tr_1):                train_idxs.append( X_train_1[i_1:(i_1+1)].index )                train_idxs.append( X_train_0[i_1*stride:(i_1+1)*stride].index )            train_idxs = flatten(train_idxs)            val_idxs = val_mask[val_mask].index            return np.array(train_idxs), np.array(val_idxs)        else:            raise StopIteration()