Bdtjtk

Here's a small example reproducing the issue:

model <- glmnet(matrix(rnorm(3*100), 100, 3), rbernoulli(100))

preds <- predict(model, matrix(rnorm(3*100), 100, 3))

dim(preds)

# 100 60

But since the predicted variable is bernoulli, I'd expect the output to be either 1 or 2 dimensional (probability of 1, or probability of each class).

I've looked at the documentation for glmnet and for predict but I can't find anything that describes this behavior. What I'm looking for is to simply fit a model to some training data, and then compute class probabilities so that I can compute AUC.

I'm mainly asking about this behavior because this does not happen if for example I use the rpart package together with predict, that is for example

df <- data.frame(cbind(matrix(rnorm(3*100), 100, 3), rbernoulli(100)))

model <- rpart(X4 ~ ., df)

length(predict(model, data.frame(matrix(rnorm(3*100), 100, 3))))

# 100, as expected

Coming from Python, I'm finding a lot of this confusing, since the predict function seems to be general, though it obviously behaves differently for two binary classifiers.

In R, you would find many examples in which you get an output based on dimension/ class etc. of your input to a function.

For glmnet, by default you supply a range of lambda's:

lambda (i.e. shrinkage factor) is a hyper parameter for regularized regression model (glmnet).

set.seed(1)

model <- glmnet(matrix(rnorm(3*100), 100, 3), purrr::rbernoulli(100))

preds <- predict(model, matrix(rnorm(3*100), 100, 3))



dim(preds)

#[1] 100  61



length(model$lambda)

[1] 61

You need to tune it based on a desired performance measure, to find the optimal/ best value for your model. Once you have it, you can use it to get final predictions. Something like:

model <- glmnet(matrix(rnorm(3*100), 100, 3), purrr::rbernoulli(100), 

                lambda = 0.19)        # assuming its an optimal value

preds <- predict(model, matrix(rnorm(3*100), 100, 3))



dim(preds)

# [1] 100   1

while rpart doesn't require a hyper parameter by default as it fits a complete tree without pruning which is equivalent of supplying a single hyper parameter value that corresponds to fitting data till the leaf nodes. Therefore you obtain a single set of predictions. The downside of using this current classifier being that its not generalized.

Therefore if you are shifting from python to R for a applied machine learning tasks its best to utilize the caret package its a homogeneous framework combining multiple statistical models under a unified modeling approach.