Spec Objects

Intro to Spec objects

All parameters in a leabra7 simulation are contained in “spec objects.” A layer’s parameters are contained in a LayerSpec object, a projection’s parameters are contained in a ProjnSpec object, and so on. This allows you to easily create many objects with the same bundle of parameters. For example, to create three layers with the inhibition_type parameter set to k-winner-take-all inhibition, you could use the following code:

import leabra7 as lb

net = lb.Net()

layer_spec = lb.LayerSpec(inhibition_type="kwta")
net.new_layer(name="layer1", spec=layer_spec)
net.new_layer(name="layer2", spec=layer_spec)
net.new_layer(name="layer3", spec=layer_spec)

If you don’t provide a value for a parameter in the spec’s constructor, it will be set to its default value. Here, we create a layer spec object with all parameters set to their default values:

>>> default_layer_spec = lb.LayerSpec()

Since parameters are stored as class attributes, we can access them using dot notation:

>>> default_layer_spec.inhibition_type
"fffb"

We see that by default, the layer spec’s inhibition_type parameter is set to "fffb", which stands for feedforward/feedback inhibition. To override the value, we would simply provide the desired value for inhibition_type as a keyword argument in the constructor:

>>> lb.LayerSpec(inhibition_type="kwta").inhibition_type
"kwta"

If you try to set a parameter that doesn’t exist, an exception will be raised

>>> lb.LayerSpec(nonexistent_param=3)
ValueError: nonexistent_param is not a valid parameter name for this spec.

Below is a list of all the specs in leabra7, along with their parameters and default values:

LayerSpec

class leabra7.LayerSpec

Contains parameters for network layers.

inhibition_type

The type of inhibition. Can be "fffb" for standard feedforward/feedback inhibition, "kwta" for k-winner-take-all inhibition, or "kwta_avg" for average-based k-winner-take-all inhibition. Defaults to "fffb".

KWTA inhibition sets the inhibition so that only k units have a "v_m_eq" value above their firing threshold.

KWTA-average inhibition is slightly more complicated: for each unit in a layer of size \(n\), it calculates the inhibition \(gi_{thr}\) that will put it at its firing threshold. Then it calculates two averages:

  1. The average of the top \(k\) values of \(gi_{thr}\), called gi_k.
  2. The average of the bottom \(n - k\) values of \(gi_{thr}\), called gi_nk

The final inhibition is a convex combination of these two values: p * gi_k + (1 - p) * gi_nk. The value of p is determined by kwta_pt.

kwta_pct

The proportion of winners for KWTA inhibition. Defaults to 0.1.

kwta_pt

In KWTA-average inhibition, the convex combination parameter p. See inhibition_type. Defaults to 0.5. Valid values are any float in the range \([0, 1]\).

ff

The feedforward inhibition multiplier for feedforward/feedback inhibition. Controls the relative strength of feedforward inhibition. Defaults to 1.0. Valid values are any float in the range \([0, \infty)\).

ff0

The feedforward inhibition offset. A layer with average activation below this level will not trigger feedforward inhibition. Defaults to 0.1. Valid values are any float in the range \([0, \infty]\).

fb

The feedback inhibition multiplier for feedforward/feedback inhibition. Controls the relative strength of feedback inhibition. Defaults to 1.0. Valid values are any float in the range \([0, \infty)\).

fb_dt

The feedback inhibition integration time constant. Defaults to 1 / 1.4. Valid values are any float in the range \([0, \infty]\).

gi

The global inhibition multiplier. Controls the relative strength of total (feedforward and feedback) inhibition. Defaults to 1.8. Valid values are any float in the range \([0, \infty]\).

avg_dt

The integration constant for the cos_diff_avg error metric (between plus and minus phase activations), which can be used to modulate the proportion of Hebbian learning and the learning rate. Defaults to 0.01. Valid values are any float in the range \([0, \infty]\).

clamp_max

Typically, units in input and output layers are clamped to binary values (0 or 1). But units cannot support an activation of 1. Thus any value above clamp_max will be reduced to clamp_max prior to clamping. Defaults to 0.95. Valid values are any float in the range \([0, 1)\).

unit_spec

The UnitSpec object containing parameters for the units in the layer. Defaults to a UnitSpec object with all default parameter values.

log_on_cycle
log_on_trial
log_on_epoch
log_on_batch

A tuple of strings specifying which layer attributes to log at the respective frequency. Defaults to an empty tuple, (). Valid members of the tuple are:

  • avg_act, the average layer activation.
  • avg_net, the average layer net input.
  • cos_diff_avg, the cosine difference between the trial plus-phase activation and the minus-phase activation.
  • fbi, the layer feedback inhibition.
  • unit_act, the activation of each unit.
  • unit_adapt, the adaption current of each unit.
  • unit_gc_i, the inhibition current in each unit.
  • unit_i_net, the net current in each unit.
  • unit_net_raw, the unintegrated (raw) net input to each unit.
  • unit_net, the integrated net input to each unit.
  • unit_spike, the spike status of each unit.
  • unit_v_m_eq, the equilibrium membrane potential of each unit (like v_m but does not reset when a spike happens).
  • unit_v_m, the membrane potential of each unit.

ProjnSpec

class leabra7.ProjnSpec

Contains parameters for network projections (bundles of unit-to-unit connections.)

dist

The probability distrubtion from which connection weights will be drawn. Defaults to lb.Scalar(0.5), but can be any distribution object listed in Connection Weight Distributions.’

pre_mask

An iterable of booleans that selects which pre layer units will be included in the projection. If the length is less than the number of units in the pre layer, it will be tiled. If the length is greater, it will be truncated. Defaults to (True, ), which includes all pre layer units.

post_mask

An iterable of booleans that selects which post layer units will be included in the projection. If the length is less than the number of units in the post layer, it will be tiled. If the length is greater, it will be truncated. Defaults to (True, ), which includes all post layer units.

sparsity

Sets the sparsity of the connection. If this is less than 1.0, then 1.0 - sparsity percent of the connections will be randomly disabled. Defaults to 1.0. Valid values are any float in the range \([0, 1]\).

projn_type

Sets the type, or “pattern”, of the projection. Defaults to "full", which connects every unit in the pre layer to every unit in the post layer. Can also be set to "one_to_one", which connects every \(i_{th}\) unit in the pre layer to every \(i_{th}\) unit in the post layer.

wt_scale_abs

The absolute net input scaling weight. Simply multiplies net input, without taking into account other projections terminating in the post layer. Defaults to 1.0. Valid values are any float in the range \([0, \infty)\).

wt_scale_rel

The relative net input scaling weight. Multiplies net input, but is normalized by the sum of relative net input scaling weights across other projections terminating in the same post layer. Defaults to 1.0. Valid values are any float in the range \([0, \infty)\).

lrate

The learning rate for the projection. Defaults to 0.02. Valid values are any float in the range \([0, \infty)\).

thr_l_mix

Mixing constant determining the proportion of Hebbian learning. A value of 0 denotes no Hebbian learning (so the learning will be completely error-driven), and a value of 1 denotes only Hebbian learning. Defaults to 0.1. Valid values are any float in the range \([0, 1]\).

cos_diff_thr_l_mix

Boolean flag controlling whether thr_l_mix is modulated by the post layer’s cos_diff_avg error metric. Defaults to False.

cos_diff_lrate

Boolean flag controlling whether lrate is modulated by the post layer’s cos_diff_avg error metric. Defaults to False.

sig_gain

The gain for the sigmoid function that is used to enhance weight contrast before sending net input. Defaults to 6. Valid values are any float in \([0, \infty)\).

sig_offset

The offset for the sigmoid function that is used to enhance weight contrast before sending net input. Defaults to 1. Valid values are any float.

log_on_cycle
log_on_trial
log_on_epoch
log_on_batch

A tuple of strings specifying which layer attributes to log at the respective frequency. Defaults to an empty tuple, (). Valid members of the tuple are:

  • "conn_wt", the sigmoid contrast-enhanced connection weights.
  • "conn_fwt", the non-contrast-enhanced connection weights.

UnitSpec

class leabra7.UnitSpec

Contains parameters for individual units.

e_rev_e

The excitation (net input) reversal potential. Defaults to 1. Valid values are any float.

e_rev_i

The inhibitory reversal potential. Defaults to 0.25. Valid values are any float.

e_rev_l

The leak reversal potential. Defaults to 0.3. Valid values are any float.

gc_l

The leak current, which is always constant. Defaults to 0.1. Valid values are any float.

spk_thr

The potential threshold value at which the unit spikes. Defaults to 0.5. Valid values are any float.

v_m_r

The potential reset value afte ra spike. Defaults to 0.3. Should be less than spk_thr.

vm_gain

The adaption current gain from membrane potential. Defaults to 0.04. Valid values are any float in \([0, \infty)\).

spike_gain

The adaption current gain from discrete spikes. Defaults to 0.00805. Valid values are any float in \([0, \infty)\).

net_dt

The net input integration time constant. Defaults to 1 / 1.4. Valid values are any float in \([0, \infty)\).

vm_dt

The membrane potential integration time constant. Defaults to 1 / 3.3. Valid values are any float in \([0, \infty)\).

adapt_dt

The adaption current integration time constant. Defaults to 1 / 144. Valid values are any float in \([0, \infty]\).

syn_tr

The synaptic transmission efficiency. Defaults to 1. Valid values are any float in \([0, \infty)\).

act_gain

The potential gain from clamping (higher values mean a lower potential value.) Defaults to 100.0. Valid values are any float in \((0, \infty)\).

ss_dt

The supershot learning average integration time constant. Defaults to 0.5. Valid values are any float in \([0, \infty)\).

s_dt

The short learning average integration time constant. Defaults to 0.5. Valid values are any float in \([0, \infty)\).

m_dt

The medium learning average integration time constnat. Defaults to 0.1. Valid values are any float in \([0, \infty)\).

l_dn_dt

The long learning average integration time constant, when it is decreasing. Defaults to 2.5. Valid values are any float in \([0, \infty)\).

l_up_inc

The increasing long learning average increment multiplier. Defaults to 0.2. Valid values are any float in \([0, \infty)\).

class leabra7.ValidationError

Exception raised when a spec contains an invalid parameter value.