7.2 Encapsulation

The learner execution can be encapsulated, so that exceptions do not stop the program flow and output is logged to the experiment instead of just printed to the console. One way to encapsulate the execution is provided by the package evaluate. The encapsulation can be enabled via mlr_control():

task = mlr_tasks$get("spam")
learner = mlr_learners$get("classif.debug")
learner$param_set$values = list(warning_train = TRUE, error_train = TRUE)

ctrl = mlr_control(encapsulate_train = "evaluate")
e = Experiment$new(task, learner, ctrl = ctrl)
e$train()
## INFO [mlr3] Training learner 'classif.debug' on task 'spam' ...
## <Experiment> [trained]:
##  + Task: spam
##  + Learner: classif.debug
##  - Model: [missing]
##  - Predictions: [missing]
##  - Performance: [missing]
e$has_errors # any errors recorded?
## [1] TRUE
e$log("train") # print train log
## <Log> with 2 messages:
## [WRN] train: Warning from classif.debug->train()
## [ERR] train: Error in learner$train(task): Error
##    from classif.debug->train()
e$log("train")$warnings # get all the warnings
## [1] "Warning from classif.debug->train()"
e$log("train")$errors # get all the errors
## [1] "Error in learner$train(task): Error from classif.debug->train()\n"

You can also enable the encapsulation for the predict step of an experiment by setting encapsulate_predict in mlr_control().

Another possibility to encapsulate is execution via package the callr. callr spawns a new R process, and thus even guards the session from segfaults. On the downside, starting new processes comes with a computational overhead.

ctrl = mlr_control(encapsulate_train = "callr")
task = mlr_tasks$get("spam")
learner = mlr_learners$get("classif.debug")
learner$param_set$values = list(segfault_train = TRUE)
e = Experiment$new(task, learner)
e$train(ctrl = ctrl)
## INFO [mlr3] Training learner 'classif.debug' on task 'spam' ...
## <Experiment> [trained]:
##  + Task: spam
##  + Learner: classif.debug
##  - Model: [missing]
##  - Predictions: [missing]
##  - Performance: [missing]
e$has_errors
## [1] TRUE
e$log("train")$errors
## [1] "callr exited with status -11"

Note that it is still impossible to perform the predict step without a fitted model:

e$predict()
## INFO [mlr3] Predicting with model of learner 'classif.debug' on task 'spam' ...
## Error: No trained model available