Neurons 🧠

Neurons 🧠#

HONU#

class ghonn_models_pytorch.core.HONU(in_features, polynomial_order, *, activation='identity', **kwargs)[source]#

Higher-Order Neural Units (HONU) model for polynomial regression.

This model computes polynomial feature combinations of the input data and applies trainable weights to produce the output. It supports configurable polynomial orders and optional bias terms.

__init__()[source]#

Initializes the HONU model with the specified parameters.

__repr__()[source]#

Returns a string representation of the HONU model.

forward()[source]#

Performs a forward pass through the HONU model.

_validate_setup()[source]#

Validates the configuration of the model.

_initialize_weights()[source]#

Initializes the trainable weights of the model.

_get_combinations()[source]#

Precomputes index combinations for polynomial features.

_get_colx()[source]#

Computes the polynomial feature map for the input batch.

Initialize the Higher-Order Neural Units model.

Parameters:

  • in_features (int) – Number of input features.

  • polynomial_order (int) – Order of the HONU model.

  • activation (str, optional) – Activation function to be used, by default “identity”.

  • **kwargs –

    Additional keyword arguments:

    • weight_divisor (int or float, optional): Divisor for the randomly initialized weights, by default 100.0.

    • bias (bool, optional): Whether to include a bias term, by default True.

order#

Polynomial order of the model.

Type:

int

in_features#

Number of input features.

Type:

int

_weight_divisor#

Divisor used to scale the randomly initialized weights.

Type:

float

_weight_init_mode#

Method for initializing weights, can be “random”, “zeros”, “ones”, “xavier”, “kaiming_normal”, or “kaiming_uniform”.

Type:

str

_activation#

Activation function to be used.

Type:

str

_activation_function#

The actual activation function to apply.

Type:

callable

_bias#

Indicates whether a bias term is included in the model.

Type:

bool

weight#

Trainable weights of the model.

Type:

nn.Parameter

_num_combinations#

Number of polynomial feature combinations.

Type:

int

_comb_idx#

Precomputed index combinations for polynomial features.

Type:

Tensor

GHONU#

class ghonn_models_pytorch.core.GHONU(in_features, predictor_order, gate_order, *, predictor_activation='identity', gate_activation='sigmoid', **kwargs)[source]#

GHONU (Gated Higher-Order Neural Unit) model.

This model combines two Higher-Order Neural Units (HONUs): a predictor HONU and a gate HONU. The gate HONU modulates the output of the predictor HONU using a specified activation function.

__repr__()[source]#

Returns a string representation of the GHONU model.

forward()[source]#

Performs the forward pass of the GHONU model.

Initialize the GHONU (Gated Higher-Order Neural Unit) model.

Parameters:

  • in_features (int) – The number of input features for the model.

  • predictor_order (int) – The order of the predictor HONU.

  • gate_order (int) – The order of the gate HONU.

  • predictor_activation (str, optional) – The activation function to use for the predictor. Defaults to “identity”. Must be a valid function in torch.nn.functional.

  • gate_activation (str, optional) – The activation function to use for the gate. Defaults to “sigmoid”. Must be a valid activation function in torch.nn.functional.

  • **kwargs – Additional keyword arguments passed to the HONU modules (e.g., weight_divisor, bias).

in_features#

The number of input features for the model.

Type:

int

predictor_order#

The order of the predictor HONU.

Type:

int

gate_order#

The order of the gate HONU.

Type:

int

_gate_activation#

The activation function used for the gate.

Type:

str

_predictor_activation#

The activation function used for the predictor.

Type:

str

predictor#

The predictor HONU instance.

Type:

HONU

gate#

The gate HONU instance.

Type:

HONU

Contents