Skills
NVIDIA Model Optimizer Evaluation
by NVIDIA
Evaluate the accuracy of quantized or full LLMs with NVIDIA NeMo Evaluator, from config to benchmark scores.
Our verdict
We have not reviewed NVIDIA Model Optimizer Evaluation yet
No one at Enterprise DNA has checked this listing against its sources. The description comes from public listings gathered automatically. Any fact below shows where it came from and when it was last checked.
From the listing
About NVIDIA Model Optimizer Evaluation
An evaluation skill from NVIDIA's Model Optimizer plugin, based on the NeMo Evaluator Launcher assistant. It guides an agent through writing NeMo Evaluator Launcher YAML configs, deploying the model, running benchmarks such as MMLU, monitoring progress and validating completed runs before reporting scores. Quantizing, serving and comparing runs are handled by sibling skills.
Best for
ML engineers optimising and evaluating LLMs on NVIDIA hardware
Use cases
- Check how much accuracy a quantized model lost
- Run MMLU or other benchmarks on a model endpoint
- Validate a completed evaluation run before sharing scores
Notes
Why it matters
Quantization saves cost but can quietly cut accuracy. This skill makes running and validating the benchmark part of the same workflow, so the trade-off is measured.
Source
- Skill file: plugins/modelopt/skills/evaluation/SKILL.md in NVIDIA/Model-Optimizer.
- Repository: 5,428 GitHub stars, Apache-2.0 licence. Checked 24 Oct 2026 through the GitHub API.
- Last change to the skill: 22 Oct 2026.
- This page was written from the skill's own SKILL.md. It is not an Enterprise DNA source review and the skill has not been installed or tested by us.
Pros
- Official NVIDIA skill
- Validation step before scores are reported
- Clear split from the quantize, deploy and compare skills
Cons
- Needs NeMo Evaluator Launcher and GPU access
- Narrow to the NVIDIA toolchain
Key facts
What we have checked
- Days since the last commit
- 0
- GitHub forks
- 806
- GitHub stars
- 5,604
- Days to a first reply on issues
- 20.1
- Open issues
- 133
- Releases in the last 90 days
- 12
Each value shows when we last confirmed it and where it came from. Values we have not confirmed are left out, never guessed.
Stars and usage
How NVIDIA Model Optimizer Evaluation is growing
5,487 to 5,604
Health and security
Is NVIDIA Model Optimizer Evaluation looked after, and what do its setup files show
Health score
100 out of 100
| Part | Points | What we saw |
|---|---|---|
| Recent activity | 30 of 30 | Last push 0 days ago |
| Releases | 15 of 15 | 12 releases in 90 days |
| Issue replies | Not counted | The latest check has no issue response reading (fewer than 3 measured issues, or issues switched off) |
| Maintainer | 15 of 15 | Owned by an organisation |
| Official source | Not counted | MCP servers only |
Security check static checks
94 out of 100
- Low Downloads code and runs it straight away (a well-known installer), host astral.sh: plugins/modelopt/skills/evaluation/scripts/nel-next.sh, line 132.
- Low Sends data to a host outside the well-known APIs (to an address set at run time): plugins/modelopt/skills/evaluation/references/doom_loops.md, line 61.
- Note No dependency list or install command was found in the skill, so there were no packages to look up on OSV.
Alternatives
Alternatives to NVIDIA Model Optimizer Evaluation
| Record | Type | How we know | Checked |
|---|---|---|---|
| evaluating-llms-harnessThis is an alternative to it | Skill | Matched from the listing text |
Compare
Compare NVIDIA Model Optimizer Evaluation side by side
Sources
Where this comes from
- GitHubRegistry
A source shows what was published when we checked it. It does not certify security, performance or a working installation. Read our method.
Changelog
What changed
No changes recorded yet. Releases, price changes and deprecations appear here, with their dates, once we track them for this record.
Open-source & AI alternatives
Swap-in tools that solve the same job. Weigh the trade-offs before you commit.
Pairs with
Other entries in the index that connect to this one. Click through to see the chain.
eval-engineering
langchain-ai
Inspect an agent repository and optional traces, interview the user, write reviewed Task Specs, build and audit Harbor tasks, and bootstrap reusable project World Knowledge…
nemo-evaluator-sdk
Orchestra-Research
Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent wi
Get the free Developer’s Field Guide
A 27-page field guide to the AI coding workflow with Claude. Claude Code, MCP servers, the prompt patterns that work, and what to delegate. Free.
Enter your work email. We send it straight over, plus a few short notes worth knowing. Unsubscribe any time.