Skip to content

feat: Add FLOPs tracking for nemotronh#1971

Open
tejasprabhune wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
tejasprabhune:tejas/nemotron-flops
Open

feat: Add FLOPs tracking for nemotronh#1971
tejasprabhune wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
tejasprabhune:tejas/nemotron-flops

Conversation

@tejasprabhune

Copy link
Copy Markdown

What does this PR do ?

This PR adds FLOPs tracking support for Nemotron-H models (e.g. Nemotron Nano V3).

Issues

List issues that this PR closes (syntax):

n/a

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this
nemotron_flops_config, nemotronh = convert_config_to_flops_config(nemotron_pretrained_config)

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • gated_linear_unit is added to FLOPSConfig as it is needed in _mlp_layer_flops in flops_formulas.py.

@tejasprabhune
tejasprabhune requested a review from a team as a code owner February 17, 2026 07:44

@terrykong terrykong left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guyueh1 to review

@tejasprabhune

Copy link
Copy Markdown
Author

Hey! Just wanted to bump this if @guyueh1 you have a chance to review :)

@guyueh1 guyueh1 changed the title Add FLOPs tracking for nemotronh feat: Add FLOPs tracking for nemotronh May 11, 2026
@guyueh1

guyueh1 commented May 11, 2026

Copy link
Copy Markdown
Contributor

Added "feat:" to the title to pass test

@svcnvidia-nemo-ci svcnvidia-nemo-ci added the waiting-on-customer Waiting on the original author to respond label May 11, 2026
@guyueh1

guyueh1 commented May 11, 2026

Copy link
Copy Markdown
Contributor

This PR doesn't consider nemotron_h MoE models which makes me concerned. Is it possible to add moe support?

@svcnvidia-nemo-ci svcnvidia-nemo-ci added waiting-on-maintainers Waiting on maintainers to respond waiting-on-customer Waiting on the original author to respond and removed waiting-on-customer Waiting on the original author to respond waiting-on-maintainers Waiting on maintainers to respond labels Jun 1, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added waiting-on-maintainers Waiting on maintainers to respond waiting-on-customer Waiting on the original author to respond and removed waiting-on-customer Waiting on the original author to respond waiting-on-maintainers Waiting on maintainers to respond labels Jun 14, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added waiting-on-maintainers Waiting on maintainers to respond and removed waiting-on-customer Waiting on the original author to respond labels Jul 3, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added waiting-on-customer Waiting on the original author to respond and removed waiting-on-maintainers Waiting on maintainers to respond labels Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-request waiting-on-customer Waiting on the original author to respond

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants