Encrypted ReLU

Definition

The rectified linear unit applied element by element to a secret vector. For a real vector $x = (x_0, \ldots, x_{N-1})$, the result $r$ has

Each output element depends only on the corresponding input element. There is no interaction between positions and no reduction across the vector.

Mathematical Domain

ReLU is piecewise linear with a kink at the origin: zero for negative inputs, the identity for positive ones, and not differentiable at zero. Arithmetic homomorphic encryption schemes evaluate only polynomials — additions and multiplications — so the corner cannot be expressed exactly. It is approximated by a polynomial over a bounded input range, and an implementation trades closeness to the kink against multiplicative depth, which is the budget the rest of a network needs.

The kernel fixes neither the arithmetic scheme, the input range, the vector length, nor the packing; these are defined by individual specifications.

Parameters Defined by Specifications

A specification narrows the general ReLU operation into a concrete problem by pinning:

• the element domain and encryption scheme (for example real values under CKKS)
• the input range on which the approximation must hold
• the vector length $N$
• the ciphertext packing and layout
• the correctness or accuracy rule and how it is scored
• which stages of the computation are timed

Applications

ReLU is the dominant activation in deep learning. It appears at the end of most layers in convolutional and fully connected networks, so every layer of a private-inference pipeline over such a model needs it. It is also the building block from which max-pooling and other rectifier-based operations are composed.