A Journey into Crafting a Balanced Decaying Sigmoid Function

Introduction

The Sigmoid function is a widely used mathematical function in machine learning, especially in neural networks, for the purpose of introducing non-linearity into the model. However, there are scenarios where a decaying Sigmoid function, a flipped version of the standard Sigmoid, might be required. Furthermore, we may need to tune this function to fit specific ranges for the input and output.

In this blog, we will explore how to create a balanced Decaying Sigmoid function that meets specific criteria. Specifically, the function should have a smooth curve that decays within a specified range for $x$ and $y$ .

The Standard Sigmoid Function

The standard Sigmoid function is defined as:

It maps any given input $x$ into a value between 0 and 1 .

The Decaying Sigmoid Function

The Decaying Sigmoid function is essentially

and it maps $x$ to values between 0 and 1, but in a decaying manner. The mathematical representation is:

Customizing the Decaying Sigmoid Function

Our challenge is to modify this function to operate within specific bounds:

• $x$ should be in the range $[10000, 100000]$
• $y$ should be in the open range $(0.01, 0.49)$

The general form of the modified Decaying Sigmoid function can be given as:

where $a,b,c,d$ are parameters that control the shape, position, and bounds of the curve.

Understanding the Parameters

In our customized Decaying Sigmoid function, the parameters $a,b,c,d$ play crucial roles in shaping the curve and placing it within the desired bounds. Here's what each parameter does:

• $a$ This is the steepness parameter. It controls how quickly the curve rises or falls. A higher value of $a$ will make the curve steeper, while a lower value will make it more gradual.
• $b$ This is the horizontal shift parameter. It moves the curve along the $x$ -axis. By adjusting $b$ , you can control where the curve starts to decay.
• $c$ This is the upper bound for $y$ . It specifies the maximum value that $y$ can take, which, in our case, is set to 0.49.
• $d$ This is the lower bound for $y$ . It specifies the minimum value that $y$ can take. For our function, $d$ is set to 0.01.

After some experimentation and fine-tuning, we arrived at:

• $a = 5 \times 10^{-5}$
• $b=10000$
• $c = 0.49$
• $d = 0.01$

Thus, our final function becomes:

Conclusion

Crafting a customized Decaying Sigmoid function requires an understanding of how its parameters influence its shape and boundaries. We successfully managed to create a function that is neither too steep nor too flat and operates within the specified ranges for $x$ and $y$. This exercise demonstrates the flexibility and utility of mathematical functions in meeting specific application requirements.