vector.GeluVariant enum¶
GELU activation formula family.
Cases¶
| Keyword | Value | Description |
|---|---|---|
erf |
0 | Exact GELU formula using erf(x/sqrt(2)). |
tanh |
1 | Tanh-polynomial GELU approximation. |
logistic |
2 | Scaled logistic GELU approximation. |
Used by¶
vector.geluf.variant