6.2 Converting specific benchmark objects to resample objects

As a BenchmarkResult object is basically a collection of multiple ResampleResult objects, we can extract specific ResampleResult objects using the stored hashes:

tab = bmr$aggregated(objects = FALSE)[task_id == "spam" & learner_id == "classif.rpart"]
print(tab)
##                hash resampling_id task_id
## 1: 7f572971e3439abb            cv    spam
##       learner_id classif.acc classif.auc
## 1: classif.rpart      0.8946      0.8971
rr = bmr$resample_result(tab$hash)
print(rr)
## <ResampleResult> of learner 'pima' on task 'classif.featureless' with 10 iterations
##      Measure   Min. 1st Qu. Median   Mean 3rd Qu.
##  classif.acc 0.5584  0.6104 0.6599 0.6511  0.7003
##  classif.auc 0.5000  0.5000 0.5000 0.5000  0.5000
##    Max.     Sd
##  0.7403 0.0657
##  0.5000 0.0000

We can now investigate this resampling and even single experiments using the previously introduced API:

rr$aggregated
## classif.acc classif.auc 
##      0.6511      0.5000
# get the iteration with worst AUC
worst = as.data.table(rr)[which.min(classif.auc), c("iteration", "classif.auc")]
print(worst)
##    iteration classif.auc
## 1:         1         0.5
# get the corresponding experiment
e = rr$experiment(worst$iteration)
print(e)
## <Experiment> [scored]:
##  + Task: pima
##  + Learner: classif.featureless
##  + Model: [featureless]
##  + Predictions: [PredictionClassif]
##  + Performance: classif.acc=0.5584, classif.auc=0.5000