ENCODERS & DECODERS

category

Encoder/decoder for a fixed number of categories.

Details and properties

Typically used for supervised learning.
Similar to codec, but generates non-overlapping encodings.
Encoder plugin for the input component.
Decoder plugin for the output component.
Encodes integers 0 to K-1 as non-overlapping SHRs.
The default number of categories is taken to be the ratio of hyperparameters N/P.
Divides the dimension N into K disjoint blocks of N/K elements, one block per category.
Each encoding draws a random sample of P elements from the category’s block. Repeated encodings of the same category therefore overlap but are not identical.
With the default K = floor(N/P), each block holds about P elements, and the encoding is fixed. With a smaller K, the blocks are larger than P, and the encoding varies from call to call.
Mappings with identical parameters [N, P, K] are globally shared, enabling encode-decode roundtrips.

Property	Default	Description
`categories`	floor(N/P)	number of categories

Encoding and decoding chain for categories

The following circuit encodes integers 0 to 19 as non-overlapping SHRs with population 10, each drawn from a reserved block of 50 elements, and subsequently decodes the internal representation, mirroring the input.

{ "hyperparameters": {"default": [1000, 10]},
  "dataflow": [
    {"component": "input", "plugin": "category", 
        "send": [1], "categories": 20},
    {"component": "output", "plugin": "category", 
        "receive": [1], "categories": 20}
  ]}

Source code

def category(config):
    hyperparameters = config.get("hyperparameters", [0, 0])
    n, p = hyperparameters[0], hyperparameters[1]

    K = config.get("categories", n // p if p > 0 else 0)
    if K * p > n:
        raise ValueError(
            f"Encoder requires dimension {K * p} or greater.")

    partitionsize = n // K if K > 0 else 0

    lex_key = (n, p, K)

    if lex_key not in circuit_category_lexicon:
        circuit_category_lexicon[lex_key] = {None: []}

    lex = circuit_category_lexicon[lex_key]
    T = matching_threshold((n, p), (n, p))
    
    def f(expr):
        if config.get("component") == "input":
            if not isinstance(expr, int) or expr < 0 or expr >= K:
                return []

            if expr not in lex:
                # 1-based indexing for native arrays
                start = partitionsize * expr + 1
                end = start + partitionsize
                lex[expr] = list(range(start, end))

            pool = lex[expr]
            sample_size = min(p, len(pool))
            return sorted(
                rng.choice(
                    pool,
                    size=sample_size,
                    replace=False).tolist())

        if not expr:
            return None

        overlaps = {k: len(set(v).intersection(expr))
                    for k, v in lex.items() if k is not None}

        result = [k for k, overlap in overlaps.items() if overlap >= T]

        if len(result) == 0:
            return None
        elif len(result) == 1:
            return result[0]
        else:
            return sorted(result)

    def clear():
        lex.clear()
        lex[None] = []

    return {
        "label": "CAT",
        "function": f,
        "clear": clear
    }