pytensor_ml.activations.ReLU

class pytensor_ml.activations.ReLU

Rectified Linear Unit.

Compute the positive part of the input:

\[\mathrm{ReLU}(x) = \max(0, x).\]

Examples

Drop it into a stack wherever a nonlinearity belongs, usually straight after a linear layer:

from pytensor_ml.activations import ReLU
from pytensor_ml.layers import Input, Linear, Sequential

X = Input("X", shape=(None, 4))
network = Sequential(
    Linear("fc", n_in=4, n_out=8),
    ReLU(),
)
activations = network(X)

Methods