Encrypted Softmax

Definition

Softmax maps a vector of real scores to a probability distribution over its positions. Given

the result is the vector $y$ with

Every output is in $[0, 1]$ and the outputs sum to one, so $y$ lies on the probability simplex. Larger inputs receive larger shares, and the mapping is invariant to adding a constant to every input: $\operatorname{softmax}(x + c\mathbf{1}) = \operatorname{softmax}(x)$.

Mathematical Domain

Softmax is defined over real vectors. It is a non-polynomial function: it combines the transcendental exponential with a normalisation (a division by the sum of exponentials). Over encrypted data neither the exponential nor the reciprocal is a native operation, so both must be realised by approximation, and the approximation is only meaningful over a bounded input range.

The kernel does not fix the cryptographic scheme, the length N, the input range over which softmax is approximated, or the accuracy tolerance. These are defined by individual specifications.

Parameters Defined by Specifications

A specification narrows the general softmax into a concrete computational problem by defining, for example:

• the cryptographic scheme and element domain;
• the length N;
• the input range over which the exponential and normalisation are approximated;
• the input and output representation;
• the accuracy tolerance used to score closeness to the exact softmax.

Applications

Softmax is the output activation of classifiers and the core of the attention mechanism in transformers. Evaluating it over encrypted data enables privacy-preserving inference — confidential classification and language-model attention — and confidential probabilistic scoring and ranking.