Maintainers: Leslie Pérez Cáceres, Manuel López-Ibáñez
Creators: Pablo Oñate Marín, Leslie Pérez Cáceres, Manuel López-Ibáñez
Contact: https://groups.google.com/d/forum/irace-package
The iraceplot package provides different plots to visualize the data generated by the irace software for automatic algorithm configuration (hyper-parameter optimization).
This package provides visualizations of:
parallel_coord()
.boxplot_training()
and boxplot_test()
.sampling_frequency()
.plot_experiments_matrix()
.For more details about these functions, please check the user guide of the package and the documentation of the functions implemented in the package.
The package also provides an HTML report, using report()
, summarizing relevant information obtained during an execution of irace.
The aim of this package is to provide support for the analysis of the best parameter settings found, the assessment of the parameter space explored by irace and the overall performance of the configuration process. Such analysis might lead to insights about the role of algorithmic components their interactions, or to improve the configuration process itself.
Keywords: automatic configuration, offline tuning, parameter tuning, parameter visualization, irace.
A user guide comes with the package. The following is a quick-start guide. The user guide gives more detailed instructions.
The official instructions are available at https://cran.r-project.org/doc/manuals/r-release/R-admin.html. We give below a quick R installation guide that will work in most cases.
You should install R from your package manager. On a Debian/Ubuntu system it will be something like:
$ sudo apt-get install r-base
Once R is installed, you can launch R from the Terminal and from the R prompt install the iraceplot package. See instructions below.
You can install R directly from a CRAN mirror (https://cran.r-project.org/bin/macosx/).
Alternatively, if you use homebrew, you can just do
$ brew install --cask r
(Using brew install r
is not recommended because that will build R from source and you will not be able to use any CRAN binary, possibly resulting in annoying build failures).
Once R is installed, you can launch R from the Terminal (or from your Applications), and from the R prompt install the iraceplot package. See instructions below.
You can install R from a CRAN mirror (https://cran.r-project.org/bin/windows/). Once R is installed, you can launch the R console and install the iraceplot package from it. See instructions below.
For installing the stable version from CRAN, launch R or Rstudio and evaluate:
install.packages("iraceplot")
Or you may wish to try the development version from GitHub
install.packages("devtools")
devtools::install_github("auto-optimization/iraceplot")
The above commands will install many packages so they may take a while.
Load the package in the R console:
Load the log file generated by irace (irace.Rdata
or the filename given by the option logFile
of irace) for example, replace the path to your irace.Rdata
file in the following line:
iraceResults <- read_logfile("~/path/example/irace.Rdata")
For example you can plot the training performance with:
boxplot_training(iraceResults)
Check the documentation and the User Guide to find the plot most suited to your needs or generate a general-purpose report with:
report(iraceResults, "path/to/my_report")