Skip to content

Make step activation inclusive at the threshold#3694

Merged
zcbenz merged 1 commit into
ml-explore:mainfrom
Pablosinyores:step-activation-threshold
Jun 17, 2026
Merged

Make step activation inclusive at the threshold#3694
zcbenz merged 1 commit into
ml-explore:mainfrom
Pablosinyores:step-activation-threshold

Conversation

@Pablosinyores

Copy link
Copy Markdown
Contributor

The step activation documents its boundary as inclusive:

step(x) = 0  if x < threshold
          1  if x >= threshold

Both nn.step and the nn.Step module render this x >= threshold form. The implementation, however, used a strict comparison:

return mx.where(x > threshold, 1, 0)

so an input exactly equal to the threshold returned 0 instead of 1. With the default threshold=0 this means step(0.0) evaluates to 0, which contradicts the documented behavior.

Switched to x >= threshold so the value at the boundary is included, matching the docstring. Added a test_step case covering the boundary for both the functional and module forms.

The documented behavior of `step` is to output 1 when `x >= threshold`,
but the implementation used a strict `x > threshold`, so an input equal to
the threshold (including the default threshold of 0) returned 0. Use a
non-strict comparison to match the documented boundary and add a test.

@angeloskath angeloskath left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@zcbenz zcbenz merged commit 9d6fb48 into ml-explore:main Jun 17, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants