5.3 Execution

With a Task, a Learner and Resampling object we can call resample() and create a ResampleResult object.

rr = resample(task, learner, resampling)
print(rr)
## <ResampleResult> of learner 'iris' on task 'classif.rpart' with 3 iterations
##     Measure Min. 1st Qu. Median Mean 3rd Qu. Max. Sd
##  classif.ce 0.06    0.06   0.06 0.06    0.06 0.06  0

Before we go into more detail, let’s change the resampling to a “3-fold cross-validation” to better illustrate what operations are possible with a ResampleResult.

resampling = mlr_resamplings$get("cv", param_vals = list(folds = 3L))
rr = resample(task, learner, resampling)
print(rr)
## <ResampleResult> of learner 'iris' on task 'classif.rpart' with 3 iterations
##     Measure Min. 1st Qu. Median    Mean 3rd Qu. Max.      Sd
##  classif.ce 0.04    0.06   0.08 0.06667    0.08 0.08 0.02309

The following operations are supported with ResampleResult objects:

  • Extract the performance for the individual resampling iterations:

    rr$performance("classif.ce")
    ## [1] 0.04 0.08 0.08
  • Extract and inspect the resampling splits:

    rr$resampling
    ## <ResamplingCV> with 3 iterations
    ## Instantiated: TRUE
    ## Parameters: folds=3
    ## 
    ## Public: clone, duplicated_ids, format, hash, id, instance,
    ##   instantiate, is_instantiated, iters, param_set, task_hash, test_set,
    ##   train_set
    rr$resampling$iters
    ## [1] 3
    rr$resampling$test_set(1)
    ##  [1]   4  10  21  23  27  28  29  31  32  33  34  35  36  37  40  42  43  44  47
    ## [20]  49  51  52  58  64  75  76  79  80  81  87  92 103 105 106 107 109 111 113
    ## [39] 114 117 119 126 128 129 131 133 139 147 148 149
    rr$resampling$train_set(3)
    ##   [1]   4  10  21  23  27  28  29  31  32  33  34  35  36  37  40  42  43  44
    ##  [19]  47  49  51  52  58  64  75  76  79  80  81  87  92 103 105 106 107 109
    ##  [37] 111 113 114 117 119 126 128 129 131 133 139 147 148 149   1   2   3   5
    ##  [55]   6   9  11  12  13  14  15  17  20  30  38  39  45  48  54  55  57  59
    ##  [73]  61  65  66  69  71  72  77  82  84  88  94  95  97  98 102 104 112 115
    ##  [91] 118 120 123 124 130 132 135 136 138 143
  • Retrieve the experiment of a specific iteration and inspect it:

    e = rr$experiment(iter = 1)
    e$model
    ## NULL