Metadata-Version: 2.4
Name: actix
Version: 0.2.0
Summary: A collection of novel and experimental activation functions for TensorFlow and PyTorch
Author-email: Islam Tlupov <tlupovislam@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/EmotionEngineer/actix
Project-URL: Repository, https://github.com/EmotionEngineer/actix
Keywords: deep learning,tensorflow,keras,pytorch,activation functions,neural networks,machine learning,actix
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.19
Requires-Dist: matplotlib>=3.3
Provides-Extra: tf
Requires-Dist: tensorflow>=2.4; extra == "tf"
Provides-Extra: torch
Requires-Dist: torch>=1.8; extra == "torch"
Provides-Extra: all
Requires-Dist: actix[tf,torch]; extra == "all"
Dynamic: license-file

# Actix Functions

[![PyPI version](https://badge.fury.io/py/actix.svg)](https://badge.fury.io/py/actix)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
<!-- Add other badges if needed (CI/CD, test coverage, etc.) -->

`actix` is a Python package providing a collection of novel and experimental activation functions for deep learning models, implemented for both TensorFlow/Keras and PyTorch. Many of these are custom-designed and aim to offer improved performance or interesting properties compared to standard activations.

## Features

*   **Dual Framework Support:** Seamless integration with TensorFlow (Keras) and PyTorch.
*   **Parametric Activations:** Functions with trainable parameters that adapt during training.
*   **Static Activations:** Novel non-parametric functions.
*   **Easy to Use:** Simple API to get and use activation functions.

## Implemented Activation Functions

Below is a list of some of the activation functions included in this package. For the full list and their mathematical formulas, please refer to the [`actix/activations_tf.py`](actix/activations_tf.py)

**Parametric:**
*   `OptimA`
*   `ParametricPolyTanh`
*   `AdaptiveRationalSoftsign`
*   `OptimXTemporal`
*   `ParametricGaussianActivation`
*   `LearnableFourierActivation`
*   `A_ELuC` (Adaptive Exponential Linear Combination)
*   `ParametricSmoothStep`
*   `AdaptiveBiHyperbolic`
*   `ParametricLogish` (Parametric Swish/SiLU)
*   `AdaptSigmoidReLU`
*   `ParametricLambertWActivation`
*   `AdaptiveHyperbolicLogarithm`
*   `ParametricGeneralizedGompertzActivation`
*   `ComplexHarmonicActivation`
*   `WeibullSoftplusActivation`
*   `AdaptiveErfSwish`
*   `ParametricBetaSoftsign`
*   `ParametricArcSinhGate`
*   `GeneralizedAlphaSigmoid`
*   `EllipticGaussianActivation`

**Static:**
*   `SinhGate`
*   `SoftRBF`
*   `ATanSigmoid`
*   `ExpoSoft`
*   `HarmonicTanh`
*   `RationalSoftplus`
*   `UnifiedSineExp`
*   `SigmoidErf`
*   `LogCoshGate`
*   `TanhArc`
*   `RiemannianSoftsignActivation`
*   `QuantumTanhActivation`
*   `LogExponentialActivation`
*   `BipolarGaussianArctanActivation`
*   `ExpArcTanHarmonicActivation`
*   `LogisticWActivation`

## Performance Highlights

The `actix` activation functions have been benchmarked on standard datasets against common activations. Below are the highlights from the provided notebook runs. Note that performance can vary based on model architecture, hyperparameters, and random seeds. These results are intended to showcase the potential of `actix` functions.

The benchmarks were run using a CNN architecture for CIFAR-10 and a simple MLP for the regression tasks (Diabetes, California Housing).

**Key:**
*   **Acc (↑):** Mean Test Accuracy (higher is better)
*   **MSE (↓):** Mean Test Loss (lower is better)

| Dataset | Metric | LR | Top `actix` Activation(s) | `actix` Mean Score | Best Standard Activation | Standard Mean Score |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
| **CIFAR-10** | Acc (↑) | `1e-3` | `ParametricLogish` | 0.7730 | `gelu` | 0.7703 |
| | | | `OptimA` | 0.7684 | `swish` | 0.7697 |
| **California Housing** | MSE (↓) | `1e-2` | `OptimA` | 0.2143 | `sigmoid` | 0.2235 |
| | | | `OptimXTemporal` | 0.2183 | `relu` | 0.2399 |
| **Diabetes** | MSE (↓) | `1e-2` | `A_ELuC` | 0.4345 | `swish` | 0.4408 |
| | | | `ParametricBetaSoftsign` | 0.4349 | `mish` | 0.4523 |

**Observations:**

*   On **CIFAR-10**, `ParametricLogish` slightly outperformed the best standard activation (`gelu`). Several other `actix` functions like `OptimA` and `A_ELuC` were also highly competitive, performing on par with or better than `relu`.
*   On the **California Housing** regression task, `actix` functions showed a notable advantage. `OptimA` and `OptimXTemporal` achieved a significantly lower (better) MSE than the best-performing standard activation (`sigmoid`).
*   On the **Diabetes** regression task, the top `actix` functions (`A_ELuC`, `ParametricBetaSoftsign`) again edged out the best standard activation (`swish`), demonstrating their potential in tabular regression settings.

It's important to note that some `actix` functions (e.g., `ParametricBetaSoftsign` on CIFAR-10, or `ParametricGeneralizedGompertzActivation` on California Housing) did not converge well or resulted in errors (NaNs) under these specific test conditions. Many other experimental activations also showed poor performance, highlighting their sensitivity and the importance of hyperparameter tuning (especially the learning rate) when exploring new activation functions.

For more detailed results, please refer to the benchmark notebooks: [`cifar.ipynb`](/benchmark/cifar.ipynb) and [`regression.ipynb`](/benchmark/regression.ipynb).

## Installation

You can install `actix` via pip:

```bash
pip install actix
```

The package will automatically detect if TensorFlow or PyTorch (or both) are installed. The corresponding activation functions will be made available. To install with specific framework support:
```bash
pip install actix[tf]    # For TensorFlow only
pip install actix[torch] # For PyTorch only
pip install actix[tf,torch] # For both
```


## Usage

### TensorFlow / Keras

```python
import tensorflow as tf
# Import directly (recommended for custom layers)
from actix import OptimA, ATanSigmoid
# Or use the getter function
from actix import get_activation

# Option 1: Direct class instantiation
model_tf_direct = tf.keras.Sequential([
    tf.keras.layers.Dense(128, input_shape=(784,)),
    OptimA(), # Custom Keras Layer
    tf.keras.layers.Dense(64),
    ATanSigmoid(), # Custom Keras Layer for static activation
    tf.keras.layers.Dense(10, activation='softmax')
])
model_tf_direct.summary()

# Option 2: Using the get_activation function
optima_layer_tf = get_activation('OptimA', framework='tensorflow')
atan_sigmoid_layer_tf = get_activation('ATanSigmoid', framework='tf')
# For standard Keras activations through the getter:
# relu_tf = get_activation('relu', framework='tf')

model_tf_getter = tf.keras.Sequential([
    tf.keras.layers.Dense(128, input_shape=(784,)),
    optima_layer_tf,
    tf.keras.layers.Dense(64),
    atan_sigmoid_layer_tf,
    tf.keras.layers.Dense(10, activation='softmax')
])
model_tf_getter.summary()

# Compile and train (example)
# (x_train, y_train), _ = tf.keras.datasets.mnist.load_data()
# x_train = x_train.reshape(-1, 784).astype('float32') / 255.0
# y_train = tf.keras.utils.to_categorical(y_train, num_classes=10)
# model_tf_direct.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
# model_tf_direct.fit(x_train[:100], y_train[:100], epochs=1)
```

### PyTorch

```python
import torch
import torch.nn as nn
# Import directly (recommended for custom modules)
# Note: PyTorch versions might have 'Torch' suffix if TensorFlow version with same name exists
from actix import OptimATorch, ATanSigmoidTorch
# Or use the getter function
from actix import get_activation

# Option 1: Direct class instantiation
class MyModelTorchDirect(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(784, 128)
        self.act1 = OptimATorch() # Custom nn.Module
        self.fc2 = nn.Linear(128, 64)
        self.act2 = ATanSigmoidTorch() # Custom nn.Module for static activation
        self.fc3 = nn.Linear(64, 10)
    def forward(self, x):
        x = self.act1(self.fc1(x.view(-1, 784)))
        x = self.act2(self.fc2(x))
        return self.fc3(x)

model_torch_direct = MyModelTorchDirect()
print(model_torch_direct)

# Option 2: Using the get_activation function
optima_module_torch = get_activation('OptimA', framework='pytorch')
atan_sigmoid_module_torch = get_activation('ATanSigmoid', framework='torch')
# For standard PyTorch activations through the getter:
# relu_torch = get_activation('ReLU', framework='pytorch') # or 'relu'

class MyModelTorchGetter(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(784, 128)
        self.act1 = optima_module_torch
        self.fc2 = nn.Linear(128, 64)
        self.act2 = atan_sigmoid_module_torch
        self.fc3 = nn.Linear(64, 10)
    def forward(self, x):
        x = self.act1(self.fc1(x.view(-1, 784)))
        x = self.act2(self.fc2(x))
        return self.fc3(x)

model_torch_getter = MyModelTorchGetter()
print(model_torch_getter)

# Compile and train (example)
# criterion = nn.CrossEntropyLoss()
# optimizer = torch.optim.Adam(model_torch_direct.parameters())
# dummy_input = torch.randn(2, 784)
# labels = torch.randint(0, 10, (2,))
# output = model_torch_direct(dummy_input)
# loss = criterion(output, labels)
# loss.backward()
# optimizer.step()
```

### Utility Functions (Plotting)

`actix` also includes helper utilities to quickly visualize any activation function and its derivative. This is useful for understanding the behavior of a function before using it in a model. The `plot_activation` and `plot_derivative` functions work for both custom and standard activations from either framework.

```python
import actix

print("--- Plotting multiple functions ---")

# Example 1: A new function for TensorFlow
if actix._TF_AVAILABLE:
    print("\nPlot for GeneralizedAlphaSigmoid (TensorFlow)")
    actix.plot_activation('GeneralizedAlphaSigmoid', framework='tf')
    actix.plot_derivative('GeneralizedAlphaSigmoid', framework='tf')

# Example 2: A new function for PyTorch
if actix._TORCH_AVAILABLE:
    print("\nPlot for GeneralizedAlphaSigmoid (PyTorch)")
    actix.plot_activation('GeneralizedAlphaSigmoid', framework='torch')
    actix.plot_derivative('GeneralizedAlphaSigmoid', framework='torch')

# Example 3: A standard function for comparison
# Note: The plotting utility can also plot standard activations
if actix._TORCH_AVAILABLE:
    print("\nPlot for standard 'sigmoid' (PyTorch)")
    actix.plot_activation('sigmoid', framework='torch')
    actix.plot_derivative('sigmoid', framework='torch')
```

## Dependencies
*   Python 3.7+
*   NumPy (>=1.19)
*   Matplotlib (>=3.3)
*   TensorFlow (>=2.4, optional, for TF activations)
*   PyTorch (>=1.8, optional, for PyTorch activations)

## Contributing
Contributions are welcome! If you have an idea for a new activation function, find a bug, or want to improve documentation, please feel free to:
1.  Fork the repository.
2.  Create a new branch (`git checkout -b feature/your-feature-name`).
3.  Make your changes.
4.  Add tests for your changes.
5.  Ensure all tests pass (`pytest`).
6.  Commit your changes (`git commit -m 'Add some feature'`).
7.  Push to the branch (`git push origin feature/your-feature-name`).
8.  Open a Pull Request.

Please ensure your code adheres to common Python coding standards (e.g., PEP8/Black, Flake8).

## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
