---
name: trtllm-flashinfer-upgrade-b2907a865df6
description: >-
  Upgrade flashinfer-python version in TensorRT-LLM. Fetches the latest releases
  from GitHub (stable and nightly), compares with the current pinned version,
  lets the user pick a target version, and updates all version references across
  the repo. Use when the user wants to bump or upgrade flashinfer.
license: Apache-2.0
metadata:
  author: NVIDIA Corporation
---

# FlashInfer Version Upgrade Skill

Automates upgrading the `flashinfer-python` package version across TensorRT-LLM.

## When to Use

- User asks to upgrade / bump / update flashinfer
- Routine dependency update duty for flashinfer-python

## Prerequisites

### Step 0a: Determine GitHub Username

Query `gh` for the authenticated user's login:
```bash
GITHUB_USERNAME=$(gh api user --jq .login)
echo "$GITHUB_USERNAME"
```

If this fails, `gh` is not authenticated — resolve Step 0c first, then retry.
As a fallback, derive the username from the fork remote:
```bash
GITHUB_USERNAME=$(git remote -v | grep -E 'github\.com/[^/]+/TensorRT-LLM' \
  | head -1 | sed -E 's|.*github\.com[:/]([^/]+)/TensorRT-LLM.*|\1|')
```
If neither works, ask the user via `AskUserQuestion`.

### Step 0b: Verify Fork Remote

Check that a git remote pointing to the user's fork of TensorRT-LLM exists:

```bash
git remote -v | grep -E 'github\.com/${GITHUB_USERNAME}/TensorRT-LLM'
```

If **no fork remote** is found, stop and notify the user:

> No GitHub fork remote detected. A fork of `NVIDIA/TensorRT-LLM` is required
> to push branches and create PRs.
>
> 1. Fork the repo at https://github.com/NVIDIA/TensorRT-LLM/fork
> 2. Add it as a git remote:
>    ```bash
>    git remote add fork https://github.com/<GITHUB_USERNAME>/TensorRT-LLM.git
>    ```
> 3. Re-run this skill.

### Step 0c: Verify `gh` CLI Is Authenticated

This skill uses the GitHub CLI (`gh`) to push branches and open PRs. Confirm it is
installed and authenticated:

```bash
gh auth status
```

Expected: `Logged in to github.com` with at least the `repo` scope. `repo` covers
pushing to the user's fork and opening PRs on `NVIDIA/TensorRT-LLM`, so no
separate fine-grained PATs are needed.

If `gh` reports "not logged in", instruct the user:

> ```bash
> gh auth login
> ```
>
> Choose: GitHub.com → HTTPS → authenticate with a web browser (or paste a PAT
> with `repo` scope).

**Note on `GH_CONFIG_DIR`:** If the user keeps multiple `gh` accounts (e.g. a
personal account and a separate account for `NVIDIA/TensorRT-LLM` work), they may
point `gh` at a non-default config directory. Check `CLAUDE.local.md` /
`AGENTS.md` or the environment for `GH_CONFIG_DIR`; if unclear, ask the user.
When set, prefix every `gh` invocation: `GH_CONFIG_DIR=<path> gh ...`.

Do **not** proceed with the upgrade workflow until `gh auth status` is clean and
the fork remote (Step 0b) is confirmed.

## Workflow

Execute these steps **in order**. Use `AskUserQuestion` for user choices and
`WebFetch` / GitHub API for release data.

### Step 1: Fetch Available Releases from GitHub

Fetch the release list from `https://github.com/flashinfer-ai/flashinfer/releases`.

Use `WebFetch` with the URL `https://github.com/flashinfer-ai/flashinfer/releases`
and extract all release tag names and dates. Collect both stable releases
(e.g., `v0.6.7`) and pre-release / nightly tags (e.g., `v0.7.0.dev20260401`).

Alternatively, use the GitHub API via curl:
```bash
curl -s "https://api.github.com/repos/flashinfer-ai/flashinfer/releases?per_page=30" \
  | python3 -c "
import json, sys
releases = json.load(sys.stdin)
for r in releases:
    tag = r['tag_name']
    pre = ' (pre-release)' if r['prerelease'] else ' (stable)'
    date = r['published_at'][:10]
    print(f'{tag}  {date}{pre}')
"
```

### Step 2: Check Current Version

Read the current pinned version from `requirements.txt`:
```bash
grep flashinfer-python requirements.txt
```
Expected format: `flashinfer-python==X.Y.Z`

### Step 3: Ask User Preferences

Ask the user **three questions** using `AskUserQuestion`:

1. **"Prefer a latest nightly release version?"**
   - Options: "Yes, show nightly/dev releases" | "No, stable releases only (Recommended)"
   - This filters the release list shown in the next question.

2. **"Which flashinfer-python version do you want to upgrade to?"**
   - Present up to 4 versions newer than the current version (filtered by
     the nightly preference above), with the latest as the recommended option.
   - If the current version is already the latest, inform the user and stop.

3. **"Also update `security_scanning/poetry.lock`?"**
   - Options: "No, skip the lockfile (Recommended)" | "Yes, update version + hashes"
   - Default: **No**. The lockfile is typically regenerated by maintainers
     separately; editing it here can produce spurious hash diffs and stale
     `metadata.content-hash` values.
   - If the user answers **Yes**, follow the "Updating `security_scanning/poetry.lock`
     hashes" subsection below; otherwise skip it entirely (do not touch
     `security_scanning/poetry.lock`).

### Step 4: Update All Version References

After the user selects a target version, update these files:

| File | What to change | Always |
|------|----------------|--------|
| `requirements.txt` | `flashinfer-python==OLD` → `flashinfer-python==NEW` | Yes |
| `security_scanning/pyproject.toml` | `"flashinfer-python (==OLD)"` → `"flashinfer-python (==NEW)"` | Yes |
| `ATTRIBUTIONS-Python.md` | `## flashinfer-python (OLD)` → `## flashinfer-python (NEW)` | Yes |
| `security_scanning/poetry.lock` | Update `version = "OLD"` → `version = "NEW"` under `[[package]] name = "flashinfer-python"`, and update the `files` list with new hashes | Only if user opted in at Step 3 question 3 |

#### Updating `security_scanning/poetry.lock` hashes

> Only perform this subsection if the user answered **Yes** to question 3 in
> Step 3. Otherwise skip it entirely.

The poetry.lock file contains SHA256 hashes for the wheel and sdist. Fetch them
from PyPI:
```bash
curl -s "https://pypi.org/pypi/flashinfer-python/NEW_VERSION/json" \
  | python3 -c "
import json, sys
data = json.load(sys.stdin)
for f in data['urls']:
    print(f'{f[\"filename\"]}  sha256:{f[\"digests\"][\"sha256\"]}')
"
```

Replace the old `files = [...]` block under `[[package]] name = "flashinfer-python"`
with the new filenames and hashes. Also update the `[package.dependencies]` section
if the new version has different dependencies (check PyPI JSON `requires_dist`).

**Important**: After manually editing both `security_scanning/pyproject.toml` and
`security_scanning/poetry.lock`, the lockfile's `metadata.content-hash` becomes stale.
Regenerate it by running:

```bash
cd security_scanning && poetry lock --no-update && cd ..
```

This refreshes the hash without changing any other package versions. If `poetry` is
available, you can alternatively use `poetry add flashinfer-python@NEW_VERSION` in the
`security_scanning/` directory to update both `pyproject.toml` and `poetry.lock`
automatically (including the content-hash).

#### Nightly / dev version special handling

If the user selects a nightly/dev version (e.g., `0.7.0.dev20260401`):
- The PyPI package may not exist — check first with `curl -s "https://pypi.org/pypi/flashinfer-python/VERSION/json"`.
- If not on PyPI, the `security_scanning/poetry.lock` hashes cannot be updated.
  Warn the user and leave a `# TODO: update hashes when published to PyPI` comment.
- The `requirements.txt` can pin to a git install instead:
  `flashinfer-python @ git+https://github.com/flashinfer-ai/flashinfer.git@TAG#egg=flashinfer-python`
  Ask the user which approach they prefer (PyPI pin vs git pin).

### Step 5: Verify Version Compatibility

After updating, check if any code has version-gated logic that needs adjusting:

```bash
grep -rn 'flashinfer.*__version__\|flashinfer.*version' \
  tensorrt_llm/ --include="*.py"
```

Known locations with version checks:
- `tensorrt_llm/_torch/speculative/interface.py` — `flashinfer.__version__ >= "0.6.4"`

If the new version is still >= the gated version, no changes needed. Otherwise, flag
to the user.

### Step 6: Summary

Print a summary of all changes made:
- Old version → New version
- Files modified (with line numbers)
- Any warnings (e.g., poetry.lock hashes couldn't be updated for nightly)
- Remind user to run `pip install -r requirements.txt` to test locally
- Remind user to run relevant unit tests:
  ```bash
  pytest tests/unittest/_torch/flashinfer/ -v
  pytest tests/unittest/_torch/attention/test_flashinfer_attention.py -v
  ```

### Step 7: Commit, Push, and Create PR

After all files are updated and verified:

> If the user opted **out** of the `poetry.lock` update at Step 3 question 3,
> drop `security_scanning/poetry.lock` from the `git stash`, `git add`, and
> commit message in the snippets below.

#### 7a. Create a new branch from upstream main

```bash
# Drop security_scanning/poetry.lock from this list if the user opted out.
git stash push -m "flashinfer-upgrade-wip" -- requirements.txt security_scanning/pyproject.toml security_scanning/poetry.lock ATTRIBUTIONS-Python.md
git checkout main
git pull --rebase https://github.com/NVIDIA/TensorRT-LLM.git main
git checkout -b ${GITHUB_USERNAME}/update_flashinfer_${NEW_VERSION}
git stash pop
```

Where `GITHUB_USERNAME` comes from the fork remote (e.g., `yihwang-nv`) and
`NEW_VERSION` is the selected version (e.g., `0.6.7.post3`).

#### 7b. Commit with DCO sign-off

```bash
# Drop security_scanning/poetry.lock from the `git add` list and the commit
# body if the user opted out.
git add requirements.txt security_scanning/pyproject.toml security_scanning/poetry.lock ATTRIBUTIONS-Python.md
git commit -s -m "[None][chore] Update flashinfer-python from OLD to NEW

Bump flashinfer-python dependency to the latest stable release.
Updated version pins in requirements.txt, security_scanning/pyproject.toml,
security_scanning/poetry.lock (if updated), and ATTRIBUTIONS-Python.md."
```

#### 7c. Push the branch to the user's fork

Identify the fork remote (from Step 0b — commonly named `fork`), then push:

```bash
FORK_REMOTE=fork   # adjust if the user named their fork remote differently
BRANCH="${GITHUB_USERNAME}/update_flashinfer_${NEW_VERSION}"
git push -u "${FORK_REMOTE}" "${BRANCH}"
```

If the push is rejected for auth reasons, confirm `gh auth status` shows `repo`
scope — `gh` installs a git credential helper that reuses its token for HTTPS
pushes. Users on a non-default config dir must export `GH_CONFIG_DIR` in the
same shell.

#### 7d. Open the PR on `NVIDIA/TensorRT-LLM`

```bash
gh pr create \
  --repo NVIDIA/TensorRT-LLM \
  --base main \
  --head "${GITHUB_USERNAME}:${BRANCH}" \
  --title "[None][chore] Update flashinfer-python from ${OLD_VERSION} to ${NEW_VERSION}" \
  --body "$(cat <<EOF
## Summary
- Bump flashinfer-python from ${OLD_VERSION} to ${NEW_VERSION} (latest stable)
- Updated version pins in requirements.txt, security_scanning/pyproject.toml, and ATTRIBUTIONS-Python.md (and security_scanning/poetry.lock if the user opted in)

## Test plan
- [ ] pip install -r requirements.txt installs successfully
- [ ] pytest tests/unittest/_torch/flashinfer/ -v
- [ ] pytest tests/unittest/_torch/attention/test_flashinfer_attention.py -v
- [ ] CI pre-merge passes
EOF
)"
```

`gh pr create` prints the new PR URL on success. Report it back to the user.

## Files Reference

All files that contain flashinfer-python version pins:

| File | Pattern |
|------|---------|
| `requirements.txt` | `flashinfer-python==X.Y.Z` |
| `security_scanning/pyproject.toml` | `"flashinfer-python (==X.Y.Z)"` |
| `security_scanning/poetry.lock` | `name = "flashinfer-python"` block with version + hashes |
| `ATTRIBUTIONS-Python.md` | `## flashinfer-python (X.Y.Z)` |

## Notes

- The `setup.py` has a comment about git+https install URLs — no version pin to update there.
- The `.pre-commit-config.yaml` and `pyproject.toml` reference flashinfer source files, not versions — no changes needed.
- The `flashinfer/` submodule (if present) is separate from the `flashinfer-python` PyPI package.


---
Source attribution: NVIDIA Corporation
Original project: https://github.com/yo-steven/skills-exploration-20260522/blob/6e1127ffb7690aa8038b172d0ae7cc720754870d/skills/TensorRT-LLM/trtllm-flashinfer-upgrade/SKILL.md
License: Apache-2.0
License information: https://www.apache.org/licenses/LICENSE-2.0
Original upstream instructions; downloaded from Mega AI Hub.

## Upstream license notice

SPDX-License-Identifier: Apache-2.0 AND CC-BY-4.0

Copyright (c) 2026 NVIDIA Corporation

This project is licensed under a dual license:
- Apache License 2.0 (for code and configuration)
- Creative Commons Attribution 4.0 International (CC BY 4.0) (for documentation)

You may use the contents of this repository under the terms of either license,
or both, as applicable to the type of content.

================================================================================
                                 Apache License
                           Version 2.0, January 2004
                        http://www.apache.org/licenses/

   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

   1. Definitions.

      "License" shall mean the terms and conditions for use, reproduction,
      and distribution as defined by Sections 1 through 9 of this document.

      "Licensor" shall mean the copyright owner or entity authorized by
      the copyright owner that is granting the License.

      "Legal Entity" shall mean the union of the acting entity and all
      other entities that control, are controlled by, or are under common
      control with that entity. For the purposes of this definition,
      "control" means (i) the power, direct or indirect, to cause the
      direction or management of such entity, whether by contract or
      otherwise, or (ii) ownership of fifty percent (50%) or more of the
      outstanding shares, or (iii) beneficial ownership of such entity.

      "You" (or "Your") shall mean an individual or Legal Entity
      exercising permissions granted by this License.

      "Source" form shall mean the preferred form for making modifications,
      including but not limited to software source code, documentation
      source, and configuration files.

      "Object" form shall mean any form resulting from mechanical
      transformation or translation of a Source form, including but
      not limited to compiled object code, generated documentation,
      and conversions to other media types.

      "Work" shall mean the work of authorship, whether in Source or
      Object form, made available under the License, as indicated by a
      copyright notice that is included in or attached to the work
      (an example is provided in the Appendix below).

      "Derivative Works" shall mean any work, whether in Source or Object
      form, that is based on (or derived from) the Work and for which the
      editorial revisions, annotations, elaborations, or other modifications
      represent, as a whole, an original work of authorship. For the purposes
      of this License, Derivative Works shall not include works that remain
      separable from, or merely link (or bind by name) to the interfaces of,
      the Work and Derivative Works thereof.

      "Contribution" shall mean any work of authorship, including
      the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to the Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"
      means any form of electronic, verbal, or written communication sent
      to the Licensor or its representatives, including but not limited to
      communication on electronic mailing lists, source code control systems,
      and issue tracking systems that are managed by, or on behalf of, the
      Licensor for the purpose of discussing and improving the Work, but
      excluding communication that is conspicuously marked or otherwise
      designated in writing by the copyright owner as "Not a Contribution."

      "Contributor" shall mean Licensor and any individual or Legal Entity
      on behalf of whom a Contribution has been received by the Licensor and
      subsequently incorporated within the Work.

   2. Grant of Copyright License. Subject to the terms and conditions of
      this License, each Contributor hereby grants to You a perpetual,
      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
      copyright license to reproduce, prepare Derivative Works of,
      publicly display, publicly perform, sublicense, and distribute the
      Work and such Derivative Works in Source or Object form.

   3. Grant of Patent License. Subject to the terms and conditions of
      this License, each Contributor hereby grants to You a perpetual,
      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
      (except as stated in this section) patent license to make, have made,
      use, offer to sell, sell, import, and otherwise transfer the Work,
      where such license applies only to those patent claims licensable
      by such Contributor that are necessarily infringed by their
      Contribution(s) alone or by combination of their Contribution(s)
      with the Work to which such Contribution(s) was submitted. If You
      institute patent litigation against any entity (including a
      cross-claim or counterclaim in a lawsuit) alleging that the Work
      or a Contribution incorporated within the Work constitutes direct
      or contributory patent infringement, then any patent licenses
      granted to You under this License for that Work shall terminate
      as of the date such litigation is filed.

   4. Redistribution. You may reproduce and distribute copies of the
      Work or Derivative Works thereof in any medium, with or without
      modifications, and in Source or Object form, provided that You
      meet the following conditions:

      (a) You must give any other recipients of the Work or
          Derivative Works a copy of this License; and

      (b) You must cause any modified files to carry prominent notices
          stating that You changed the files; and

      (c) You must retain, in the Source form of any Derivative Works
          that You distribute, all copyright, patent, trademark, and
          attribution notices from the Source form of the Work,
          excluding those notices that do not pertain to any part of
          the Derivative Works; and

      (d) If the Work includes a "NOTICE" text file as part of its
          distribution, then any Derivative Works that You distribute must
          include a readable copy of the attribution notices contained
          within such NOTICE file, excluding any notices that do not
          pertain to any part of the Derivative Works, in at least one
          of the following places: within a NOTICE text file distributed
          as part of the Derivative Works; within the Source form or
          documentation, if provided along with the Derivative Works; or,
          within a display generated by the Derivative Works, if and
          wherever such third-party notices normally appear. The contents
          of the NOTICE file are for informational purposes only and
          do not modify the License. You may add Your own attribution
          notices within Derivative Works that You distribute, alongside
          or as an addendum to the NOTICE text from the Work, provided
          that such additional attribution notices cannot be construed
          as modifying the License.

      You may add Your own copyright statement to Your modifications and
      may provide additional or different license terms and conditions
      for use, reproduction, or distribution of Your modifications, or
      for any such Derivative Works as a whole, provided Your use,
      reproduction, and distribution of the Work otherwise complies with
      the conditions stated in this License.

   5. Submission of Contributions. Unless You explicitly state otherwise,
      any Contribution intentionally submitted for inclusion in the Work
      by You to the Licensor shall be under the terms and conditions of
      this License, without any additional terms or conditions.
      Notwithstanding the above, nothing herein shall supersede or modify
      the terms of any separate license agreement you may have executed
      with Licensor regarding such Contributions.

   6. Trademarks. This License does not grant permission to use the trade
      names, trademarks, service marks, or product names of the Licensor,
      except as required for reasonable and customary use in describing the
      origin of the Work and reproducing the content of the NOTICE file.

   7. Disclaimer of Warranty. Unless required by applicable law or
      agreed to in writing, Licensor provides the Work (and each
      Contributor provides its Contributions) on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
      implied, including, without limitation, any warranties or conditions
      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
      PARTICULAR PURPOSE. You are solely responsible for determining the
      appropriateness of using or redistributing the Work and assume any
      risks associated with Your exercise of permissions under this License.

   8. Limitation of Liability. In no event and under no legal theory,
      whether in tort (including negligence), contract, or otherwise,
      unless required by applicable law (such as deliberate and grossly
      negligent acts) or agreed to in writing, shall any Contributor be
      liable to You for damages, including any direct, indirect, special,
      incidental, or consequential damages of any character arising as a
      result of this License or out of the use or inability to use the
      Work (including but not limited to damages for loss of goodwill,
      work stoppage, computer failure or malfunction, or any and all
      other commercial damages or losses), even if such Contributor
      has been advised of the possibility of such damages.

   9. Accepting Warranty or Additional Liability. While redistributing
      the Work or Derivative Works thereof, You may choose to offer,
      and charge a fee for, acceptance of support, warranty, indemnity,
      or other liability obligations and/or rights consistent with this
      License. However, in accepting such obligations, You may act only
      on Your own behalf and on Your sole responsibility, not on behalf
      of any other Contributor, and only if You agree to indemnify,
      defend, and hold each Contributor harmless for any liability
      incurred by, or claims asserted against, such Contributor by reason
      of your accepting any such warranty or additional liability.

   END OF TERMS AND CONDITIONS

================================================================================
              Creative Commons Attribution 4.0 International License
                              (CC BY 4.0)

                https://creativecommons.org/licenses/by/4.0/

You are free to:

  Share — copy and redistribute the material in any medium or format for any
  purpose, even commercially.

  Adapt — remix, transform, and build upon the material for any purpose, even
  commercially.

  The licensor cannot revoke these freedoms as long as you follow the license
  terms.

Under the following terms:

  Attribution — You must give appropriate credit, provide a link to the license,
  and indicate if changes were made. You may do so in any reasonable manner, but
  not in any way that suggests the licensor endorses you or your use.

  No additional restrictions — You may not apply legal terms or technological
  measures that legally restrict others from doing anything the license permits.

Notices:

  You do not have to comply with the license for elements of the material in the
  public domain or where your use is permitted by an applicable exception or
  limitation.

  No warranties are given. The license may not give you all of the permissions
  necessary for your intended use. For example, other rights such as publicity,
  privacy, or moral rights may limit how you use the material.

For the full legal text, see:
https://creativecommons.org/licenses/by/4.0/legalcode



# Mega AI Hub resource metadata

Creator / publisher: NVIDIA Corporation
Original project: https://github.com/yo-steven/skills-exploration-20260522/blob/6e1127ffb7690aa8038b172d0ae7cc720754870d/skills/TensorRT-LLM/trtllm-flashinfer-upgrade/SKILL.md
Attribution basis: Source repository record

Fields of work: Development & Engineering
Roles: ML / AI Engineer, Backend Developer, Data Engineer
Specialties: Backend, Git
Tools: GitHub
Country applicability: Country not specified
Classification basis: Source category, role and specialization labels
Instruction SHA-256: 2d1694e7e15850e91801618a52c47669657db6a681436fe75301cc1fecdf1021