Skip to content

Conversation

@kolkov
Copy link
Contributor

@kolkov kolkov commented Dec 27, 2025

Summary

Add functional options pattern to NewLinear for optional bias configuration.

  • Add LinearOption type and WithBias(bool) function
  • Add HasBias() method for introspection
  • Update SwiGLUFFN to use public API (remove internal newLinearNoBias)
  • Export WithBias in public nn package

Usage

// With bias (default, backwards compatible)
layer := nn.NewLinear(784, 128, backend)

// Without bias (for LLaMA-style models, LM head, etc.)
lmHead := nn.NewLinear(hiddenSize, vocabSize, backend, nn.WithBias(false))

Motivation

Many neural network architectures require Linear layers without bias:

  • LM Head in language models (GPT, LLaMA, HRM)
  • Attention projections (some architectures)
  • SwiGLU FFN (already used internally)

Test plan

  • TestLinear_WithBias - tests default, explicit true, and false cases
  • TestLinear_NoBias_Forward - tests forward pass without bias
  • TestLinear_NoBias_StateDict - tests state dict without bias
  • All existing tests pass
  • golangci-lint: 0 issues
  • Pre-release check passed

Add functional options pattern to NewLinear for optional bias:
- Add LinearOption type and WithBias(bool) function
- Add HasBias() method for introspection
- Update SwiGLUFFN to use public API
- Export WithBias in public nn package

Usage:
  nn.NewLinear(in, out, backend, nn.WithBias(false))

Closes feature request for Linear without bias support.
@codecov
Copy link

codecov bot commented Dec 27, 2025

Codecov Report

❌ Patch coverage is 84.00000% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
nn/nn.go 0.00% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

@kolkov kolkov merged commit cbb6dfc into main Dec 27, 2025
9 checks passed
@kolkov kolkov deleted the feat/linear-without-bias branch December 27, 2025 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants