---
title: "LiblineaR 2.50-1: New Features, Correctness, and Performance"
author: "Thibault Helleputte"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{LiblineaR 2.50-1: New Features, Correctness, and Performance}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

```{r setup}
library(LiblineaR)
```

This version of `LiblineaR` updates its bundled copy of the LIBLINEAR
C/C++ library from v2.1 (2015) to v2.50 (2024) -- the first upgrade of the
bundled library in a decade, since `LiblineaR` 2.10-0 in July 2016. This vignette covers two things: the new capabilities this brings
at the R level, and evidence that the upgrade is safe to rely on, along
with what it changes in practice.

## New in this release

Three new capabilities are available, on top of everything `LiblineaR`
already supported.

### One-class SVM: outlier and novelty detection

`type=21` fits a one-class SVM. Unlike every other `type`, it is
**unsupervised** -- it learns what "normal" data looks like from `data`
alone. `target` is still required as an argument (its length must match
the number of rows of `data`) but its *values* are ignored. `bias` must
be `<= 0` (this solver has no bias-term concept), and `nu` (a number in
(0, 1), default 0.5) is an upper bound on the fraction of training points
that may be treated as outliers.

`predict()` returns `1` for inliers and `-1` for outliers;
`decisionValues=TRUE` additionally returns the raw signed distance to the
decision boundary (positive means inlier -- useful if you want to rank
points by how anomalous they are, rather than just a yes/no call).

```{r oneclass}
x <- iris[, 1:4]
y <- iris[, 5]

m_oc <- LiblineaR(x, y, type = 21, bias = -1, nu = 0.1)
p_oc <- predict(m_oc, newx = x, decisionValues = TRUE)
table(p_oc$predictions)
head(p_oc$decisionValues)
```

### `regularizeBias`: excluding the intercept from regularization

By default, `LiblineaR`'s bias term (the intercept) is penalized by the
L1/L2 regularization exactly like every other coefficient. Setting
`regularizeBias=FALSE` excludes it, so it is fit freely. This is only
valid combined with `bias=1` exactly, and only for `type` 0, 2, 5, 6 or
11; other combinations raise a clear error. Most users should leave this
at its default (`TRUE`) -- an unregularized intercept is a specialized
choice, useful mainly when you have strong reasons to believe the
regularization penalty should apply only to the feature weights, not to
the overall offset.

```{r regularizebias}
m_default <- LiblineaR(x, y, type = 0, bias = 1)
m_unreg   <- LiblineaR(x, y, type = 0, bias = 1, regularizeBias = FALSE)
```

### `findC` now supports SVR (`type=11`)

For perspective: `heuristicC(data)` and `findC=TRUE` both help pick a good
`cost`, but they work very differently. `heuristicC()` is a fast,
closed-form formula -- it inspects `data` once and returns a single
suggested `cost`, with no training involved at all. `findC=TRUE` is an
actual empirical search -- it repeatedly trains and cross-validates across
a doubling sequence of `cost` values and returns whichever one empirically
scored best. `heuristicC()` is a cheap starting guess; `findC` is
expensive but data-driven tuning, and the two compose naturally:
`useInitC=TRUE` (the default) uses the `cost` you pass in -- often
`heuristicC()`'s own suggestion -- as `findC`'s starting point for the
search.

`findC=TRUE` previously only searched for a good `cost` with `type=0` and
`type=2`. It now also supports `type=11` (L2-regularized L2-loss SVR),
additionally searching `svr_eps` at the same time. For this `type` only,
`findC` returns a named list `list(C=..., svr_eps=...)` instead of a
plain number, since there are now two values to report; `type` 0 and 2
are unaffected and still return a plain number.

```{r findc-svr}
xr <- as.matrix(iris[, 1:3])
yr <- iris[, 4]

best <- LiblineaR(xr, yr, type = 11, findC = TRUE, cross = 5)
best

m_svr <- LiblineaR(xr, yr, type = 11, cost = best$C, svr_eps = best$svr_eps)
```

## Is the upgrade safe to rely on?

Beyond `LiblineaR`'s own regression test suite, this section reports an
independent check: models were trained with the **pristine, unmodified
LIBLINEAR command-line tools** (`train`/`predict`, compiled directly from
upstream source, no R or `LiblineaR` code involved at all) and compared
against `LiblineaR`'s own output, at both the old (v2.10) and new (v2.50)
LIBLINEAR version.

### Method

Four engines were compared: the LIBLINEAR command-line tools at v2.10 and
v2.50, and the `LiblineaR` R package before and after this upgrade. All
four were trained on **byte-identical data** (the same train/test split
of the same matrix) with matching hyperparameters (`cost`, `bias`,
`epsilon`, `svr_eps`). Four datasets were used: two synthetic
classification problems (2-class and 3-class, 600 rows each), one
regression problem (600 rows), and one larger problem (8000 rows, 300
columns) reserved for timing.

**A methodological caveat, confirmed empirically before trusting any of
the numbers below:** LIBLINEAR's dual coordinate-descent solvers (and the
L1-regularized solvers) shuffle their working set using a random number
generator. `LiblineaR` routes this through R's own RNG
(`set.seed()`-controlled), while the plain command-line tools use C's own
unseeded `rand()`. **For these solver types, coefficients are not
expected to match between `LiblineaR` and the command-line tools, even at
the identical LIBLINEAR version** -- they are two different, independent,
equally valid solutions to the same problem. What *is* expected, and
checked below, is that predictions agree almost exactly. Primal,
non-randomized solvers (logistic regression and L2-loss SVM/SVR, `type`
0, 2, 11) use no randomness at all, so their coefficients are expected to
match the command-line tools almost to floating-point precision -- and
are the real fidelity check.

### Result 1: does `LiblineaR` still correctly wrap LIBLINEAR?

Maximum absolute coefficient difference between `LiblineaR` and the
pristine command-line tools, same LIBLINEAR version, across the 3
correctness datasets (28 train/predict comparisons per version):

```{r fidelity-table}
fidelity <- data.frame(
  type = c(0L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 11L, 12L, 13L),
  solver = c("L2R_LR (primal)", "L2R_L2LOSS_SVC_DUAL", "L2R_L2LOSS_SVC (primal)",
             "L2R_L1LOSS_SVC_DUAL", "MCSVM_CS", "L1R_L2LOSS_SVC", "L1R_LR",
             "L2R_LR_DUAL", "L2R_L2LOSS_SVR (primal)", "L2R_L2LOSS_SVR_DUAL",
             "L2R_L1LOSS_SVR_DUAL"),
  max_abs_diff_v210 = c(1.72e-10, 9.65e-03, 1.98e-10, 3.60e-02, 1.38e-02,
                         3.93e-01, 6.82e-01, 7.32e-03, 1.84e-10, 7.93e-02, 1.71e-01),
  max_abs_diff_v250 = c(1.75e-10, 9.65e-03, 1.93e-10, 3.60e-02, 1.38e-02,
                         3.93e-01, 6.82e-01, 7.32e-03, 1.59e-10, 7.93e-02, 1.71e-01)
)
knitr::kable(fidelity, digits = 4,
  col.names = c("type", "solver", "max|diff| vs CLI (2.10)", "max|diff| vs CLI (2.50)"))
```

Primal types (0, 2, 11) match the pristine command-line tools to
**1e-10**, at both versions -- direct evidence that `LiblineaR` correctly
passes data through to the same underlying solver, unaffected by the
upgrade. Every dual/L1 type shows a larger difference, exactly as the
caveat above predicts -- and across all 54 such comparisons (both
versions, all 3 datasets), **minimum prediction agreement was 98.75%,
with most exactly 100%**, confirming these are two valid solutions to the
same problem, not a correctness gap.

### Result 2: what changed for an ordinary user

Maximum absolute coefficient difference between `LiblineaR` before and
after the upgrade, same 3 datasets:

```{r upgrade-table}
upgrade <- data.frame(
  type = c(0L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 11L, 12L, 13L),
  solver = fidelity$solver,
  max_abs_diff = c(0.0789, 0, 0.2281, 0, 0, 0, 0, 0, 0.0021, 0, 0)
)
knitr::kable(upgrade, digits = 4, col.names = c("type", "solver", "max|diff| (2.10-26 vs. 2.50-1)"))
```

Only the 3 primal Newton-solved types (0, 2, 11) show any drift at all:
LIBLINEAR replaced its trust-region Newton solver with a faster-converging
line-search variant partway through this version range, refined further
by later releases. Every dual/L1 type was **bit-for-bit identical** on
these ordinary, well-conditioned datasets -- LIBLINEAR 2.50's lower
iteration cap on dual coordinate descent (see Result 3) was never
actually reached on data this well-behaved. Predictive performance
(accuracy for classification, mean squared error for regression) matched
to 4 decimal places for every type on every dataset.

### Result 3: what actually changed under harder conditions

LIBLINEAR 2.50 caps its dual coordinate-descent solvers at 300 iterations
(down from 1000 in earlier versions) and, for three solver types,
automatically switches to the primal solver instead of running out the
clock if that cap is reached. Result 2 shows this is invisible on
easy-to-fit data. It stops being invisible on a harder problem. On a
larger, more weakly-separated dataset (8000 rows, 300 columns):

```{r large-table}
large <- data.frame(
  type = c(0L, 1L, 7L, 11L, 12L),
  solver = c("L2R_LR (primal)", "L2R_L2LOSS_SVC_DUAL", "L2R_LR_DUAL",
             "L2R_L2LOSS_SVR (primal)", "L2R_L2LOSS_SVR_DUAL"),
  metric_old = c(0.9188, 0.9219, 0.9200, 1.0234, 1.0502),
  metric_new = c(0.9181, 0.9188, 0.9200, 1.0235, 1.0234),
  time_old_s = c(0.133, 0.632, 1.035, 0.075, 3.400),
  time_new_s = c(0.147, 0.288, 0.998, 0.086, 1.201),
  speedup = c(0.90, 2.19, 1.04, 0.87, 2.83)
)
knitr::kable(large, digits = 4,
  col.names = c("type", "solver", "accuracy/MSE (2.10-26)", "accuracy/MSE (2.50-1)",
                "time 2.10-26 (s)", "time 2.50-1 (s)", "speedup"))
```

Two solver types where the iteration cap is actually reached on this
dataset (`type` 1 and `type` 12, both dual) show a **2.2x and 2.8x
speedup** respectively -- and `type` 12's test-set mean squared error also
*improved* (1.050 &rarr; 1.023), because the previous solver spent its
full 1000-iteration budget grinding toward a barely-better optimum on a
genuinely slow-converging problem, while the current one recognizes the
cap, switches solver, and reaches a comparably good (here, slightly
better) answer in a fraction of the time. The primal Newton types (0, 11)
are essentially a wash in either direction (0.87-0.90x) -- expected,
since that solver's algorithm changed but not its overall cost.

## Summary

- **New capabilities**: one-class SVM for outlier/novelty detection,
  optional unregularized intercept, and `findC` support for SVR.
- **Correctness**: `LiblineaR` continues to faithfully wrap LIBLINEAR,
  confirmed against the pristine command-line tools directly, not just
  against its own prior output. Coefficient-level agreement for every
  non-randomized solver is at floating-point precision.
- **Impact on existing code**: minimal. Only 3 of the 11 previously
  available solver types show any coefficient drift at all, and it does
  not measurably affect predictive performance in this study.
- **Performance**: dual coordinate-descent solvers on harder-to-converge
  problems can now be more than twice as fast, sometimes with *better*
  held-out performance, because LIBLINEAR 2.50 recognizes slow
  convergence and switches strategy instead of running a fixed iteration
  budget to completion.

**Reproducibility note.** These figures were measured on
`r R.version.string`, Linux, using LIBLINEAR upstream source at tags
`v210` and `v250` (github.com/cjlin1/liblinear), compiled standalone with
no changes. `set.seed()` was fixed identically across `LiblineaR` calls
for a given comparison; the command-line tools have no equivalent seed
control (see the caveat above). Datasets are synthetic and fixed by seed;
absolute numbers will differ on other data, but the qualitative pattern
(primal solvers: no timing benefit, but a different algorithm; dual
solvers: identical on easy data, faster and possibly more accurate on
hard data) is expected to hold generally, since it follows directly from
the documented behavior of the underlying LIBLINEAR change, not from
anything data-specific.

**Authorship note.** The scope and content of this report -- what to
test, which comparisons matter, and how to interpret the results -- were
designed and directed by the package maintainer. Drafting of this
vignette and the accompanying comparison scripts was AI-assisted (Claude,
Anthropic). Every figure was produced by actually running the real
LIBLINEAR and LiblineaR software, not generated by the AI; all code,
results, and conclusions were reviewed and validated by the maintainer
before publication.