Sigmoid function

From Wikipedia, the free encyclopedia

Jump to: navigation, search

Many natural processes and complex system learning curves display a history dependent progression from small beginnings that accelerates and approaches a climax over time. For lack of complex descriptions a sigmoid function is often used. A sigmoid curve is produced by a mathematical function having an "S" shape. Often, sigmoid function refers to the special case of the logistic function shown at right and defined by the formula

P(t) = \frac{1}{1 + e^{-t}}.

Another example is the Gompertz curve. It is used in modeling systems that saturate at large values of t.


[edit] Properties

In general, a sigmoid function is real-valued and differentiable, having either a non-negative or non-positive first derivative and exactly one inflection point. There are also two asymptotes, t \rightarrow \pm \infty.

The general case 1 / (1 + e-x) is particularly useful, used in Artificial Neural Networks partly because it has a simple derivative: if s(x) is the sigmoid function, then s'(x) = s(x) * (1 - s(x)).[1]

[edit] Examples

Besides the logistic function, sigmoid functions include the ordinary arc-tangent, the hyperbolic tangent, and the error function, but also the Gompertz function, the generalised logistic function, and algebraic functions like f(x)=\tfrac x\sqrt{1+x^2}.

The integral of any smooth, positive, "bump-shaped" function will be sigmoidal, thus the cumulative distribution functions for many common probability distributions are sigmoidal.

[edit] See also

[edit] References

  • Tom M. Mitchell, Machine Learning, WCB-McGraw-Hill, 1997, ISBN 0-07-042807-7. In particular see "Chapter 4: Artificial Neural Networks" (in particular p. 96-97) where Mitchel uses the word "logistic function" and the "sigmoid function" synonymously -- this function he also calls the "squashing function" -- and the sigmoid (aka logistic) function is used to compress the outputs of the "neurons" in multi-layer neural nets.

[edit] External links

Personal tools