Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 18 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,20 @@ jobs:
run: ./gradlew --stacktrace --no-problems-report native-lib:nodeTest
shell: bash

# Derive lowercase OS + runtime arch tokens for artifact names.
# OS comes from matrix.script_name (linux/windows); arch from uname -m
# (x86_64 today; stays correct if an arm64 runner is ever added).
- name: Derive platform tokens
run: echo "ARCH=$(uname -m)" >> "$GITHUB_ENV"
shell: bash

# archive:false makes the artifact name equal the uploaded file name.
# Copy the Gradle-named distro zip to the convention name so the CLI
# artifact reads dw-cli-<ver>-<os>-<arch>.zip.
- name: Stage renamed CLI distro
run: cp "native-cli/build/distributions/native-cli-${{env.NATIVE_VERSION}}-native-distro-${{ matrix.script_name }}.zip" "native-cli/build/distributions/dw-cli-${{env.NATIVE_VERSION}}-${{ matrix.script_name }}-${{ env.ARCH }}.zip"
shell: bash

# Upload the artifact file
- name: Upload generated script
uses: actions/upload-artifact@v7.0.1
Expand All @@ -79,7 +93,7 @@ jobs:
# .zip) and names the artifact after the file — which already carries
# ${matrix.script_name}, so the matrix legs don't collide. No `name:`:
# it would be ignored under archive:false.
path: native-cli/build/distributions/native-cli-${{env.NATIVE_VERSION}}-native-distro-${{ matrix.script_name }}.zip
path: native-cli/build/distributions/dw-cli-${{env.NATIVE_VERSION}}-${{ matrix.script_name }}-${{ env.ARCH }}.zip
archive: false

# Upload the Python wheel
Expand All @@ -98,14 +112,14 @@ jobs:
# `name:` input is ignored), so copy to an OS-qualified name first to keep
# the matrix legs from colliding.
- name: Stage OS-qualified Node package
run: cp native-lib/node/dataweave-native-0.0.1.tgz "native-lib/node/dataweave-native-0.0.1-${{ runner.os }}.tgz"
run: cp native-lib/node/dataweave-native-0.0.1.tgz "native-lib/node/dataweave-node-0.0.1-${{ matrix.script_name }}-${{ env.ARCH }}.tgz"
shell: bash

# Upload the Node.js package
- name: Upload Node package
uses: actions/upload-artifact@v7.0.1
with:
path: native-lib/node/dataweave-native-0.0.1-${{ runner.os }}.tgz
path: native-lib/node/dataweave-node-0.0.1-${{ matrix.script_name }}-${{ env.ARCH }}.tgz
# Single .tgz (already gzip-compressed); skip the redundant outer zip
# (v7+ feature). archive:false ignores `name:` and uses the file name,
# which the copy above made OS-unique.
Expand All @@ -117,7 +131,7 @@ jobs:
- name: Upload native shared library
uses: actions/upload-artifact@v7.0.1
with:
name: dwlib-${{env.NATIVE_VERSION}}-${{runner.os}}
name: dwlib-${{env.NATIVE_VERSION}}-${{ matrix.script_name }}-${{ env.ARCH }}
path: |
native-lib/python/src/dataweave/native/dwlib.dylib
native-lib/python/src/dataweave/native/dwlib.so
Expand Down
22 changes: 18 additions & 4 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,20 @@ jobs:
cd native-lib/node && npm run test:tck
shell: bash

# Derive lowercase OS + runtime arch tokens for artifact names.
# OS comes from matrix.script_name (linux/windows); arch from uname -m
# (x86_64 today; stays correct if an arm64 runner is ever added).
- name: Derive platform tokens
run: echo "ARCH=$(uname -m)" >> "$GITHUB_ENV"
shell: bash

# archive:false makes the artifact name equal the uploaded file name.
# Copy the Gradle-named distro zip to the convention name so the CLI
# artifact reads dw-cli-<ver>-<os>-<arch>.zip.
- name: Stage renamed CLI distro
run: cp "native-cli/build/distributions/native-cli-${{env.NATIVE_VERSION}}-native-distro-${{ matrix.script_name }}.zip" "native-cli/build/distributions/dw-cli-${{env.NATIVE_VERSION}}-${{ matrix.script_name }}-${{ env.ARCH }}.zip"
shell: bash

# Upload the artifact file
- name: Upload generated script
uses: actions/upload-artifact@v7.0.1
Expand All @@ -116,7 +130,7 @@ jobs:
# .zip) and names the artifact after the file — which already carries
# ${matrix.script_name}, so the matrix legs don't collide. No `name:`:
# it would be ignored under archive:false.
path: native-cli/build/distributions/native-cli-${{env.NATIVE_VERSION}}-native-distro-${{ matrix.script_name }}.zip
path: native-cli/build/distributions/dw-cli-${{env.NATIVE_VERSION}}-${{ matrix.script_name }}-${{ env.ARCH }}.zip
archive: false

# Upload the Python wheel
Expand All @@ -135,14 +149,14 @@ jobs:
# `name:` input is ignored), so copy to an OS-qualified name first to keep
# the matrix legs from colliding.
- name: Stage OS-qualified Node package
run: cp native-lib/node/dataweave-native-0.0.1.tgz "native-lib/node/dataweave-native-0.0.1-${{ runner.os }}.tgz"
run: cp native-lib/node/dataweave-native-0.0.1.tgz "native-lib/node/dataweave-node-0.0.1-${{ matrix.script_name }}-${{ env.ARCH }}.tgz"
shell: bash

# Upload the Node.js package
- name: Upload Node package
uses: actions/upload-artifact@v7.0.1
with:
path: native-lib/node/dataweave-native-0.0.1-${{ runner.os }}.tgz
path: native-lib/node/dataweave-node-0.0.1-${{ matrix.script_name }}-${{ env.ARCH }}.tgz
# Single .tgz (already gzip-compressed); skip the redundant outer zip
# (v7+ feature). archive:false ignores `name:` and uses the file name,
# which the copy above made OS-unique.
Expand All @@ -154,7 +168,7 @@ jobs:
- name: Upload native shared library
uses: actions/upload-artifact@v7.0.1
with:
name: dwlib-${{env.NATIVE_VERSION}}-${{runner.os}}
name: dwlib-${{env.NATIVE_VERSION}}-${{ matrix.script_name }}-${{ env.ARCH }}
path: |
native-lib/python/src/dataweave/native/dwlib.dylib
native-lib/python/src/dataweave/native/dwlib.so
Expand Down
13 changes: 7 additions & 6 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ jobs:
- name: Guess Extension Version
run: |
echo "NATIVE_VERSION=$(echo '${{github.ref}}' | sed -e 's,.*/v\(.*\),\1,')" >> $GITHUB_ENV
echo "ARCH=$(uname -m)" >> $GITHUB_ENV
shell: bash

# Runs a single command using the runners shell
Expand Down Expand Up @@ -83,7 +84,7 @@ jobs:
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: native-cli/build/distributions/native-cli-${{env.NATIVE_VERSION}}-native-distro-${{ matrix.script_name }}.zip
asset_name: dw-${{env.NATIVE_VERSION}}-${{runner.os}}
asset_name: dw-cli-${{env.NATIVE_VERSION}}-${{ matrix.script_name }}-${{ env.ARCH }}.zip
tag: ${{ github.ref }}
overwrite: true

Expand All @@ -92,8 +93,8 @@ jobs:
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: native-lib/python/dist/dataweave_native-0.0.1-py3-none-any.whl
asset_name: dw-python-wheel-${{env.NATIVE_VERSION}}-${{runner.os}}.whl
file: native-lib/python/dist/dataweave_native-0.0.1-py3-none-*.whl
file_glob: true
tag: ${{ github.ref }}
overwrite: true

Expand All @@ -103,7 +104,7 @@ jobs:
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: native-lib/node/dataweave-native-0.0.1.tgz
asset_name: dw-node-package-${{env.NATIVE_VERSION}}-${{runner.os}}.tgz
asset_name: dataweave-node-0.0.1-${{ matrix.script_name }}-${{ env.ARCH }}.tgz
tag: ${{ github.ref }}
overwrite: true

Expand All @@ -114,7 +115,7 @@ jobs:
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: native-lib/python/src/dataweave/native/dwlib.so
asset_name: dwlib-${{env.NATIVE_VERSION}}-${{runner.os}}.so
asset_name: dwlib-${{env.NATIVE_VERSION}}-${{ matrix.script_name }}-${{ env.ARCH }}.so
tag: ${{ github.ref }}
overwrite: true

Expand All @@ -124,7 +125,7 @@ jobs:
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: native-lib/python/src/dataweave/native/dwlib.dll
asset_name: dwlib-${{env.NATIVE_VERSION}}-${{runner.os}}.dll
asset_name: dwlib-${{env.NATIVE_VERSION}}-${{ matrix.script_name }}-${{ env.ARCH }}.dll
tag: ${{ github.ref }}
overwrite: true

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
# CI Artifact Naming Convention — Design

**Status:** Draft (brainstorming output, pending review)
**Date:** 2026-07-28
**Scope:** Naming/labeling only — no new build targets.

## Goal

Give the customer-downloadable artifacts — both the CI-run artifacts
(`main.yml`/`ci.yml`) and the tagged release assets (`release.yml`) — a single,
human-readable naming convention that is consistent across all families and
honestly reflects the **current** build state (Linux + Windows, x86_64 only).

## Problem

The artifacts from a master CI run
([run 30393123015](https://github.com/mulesoft/data-weave-cli/actions/runs/30393123015))
are named three different ways:

| Artifact (today) | OS token | Arch? | Issues |
|---|---|---|---|
| `native-cli-100.100.100-native-distro-linux.zip` | `linux` (lower) | ❌ | "native" appears twice; jargony |
| `native-cli-100.100.100-native-distro-windows.zip` | `windows` (lower) | ❌ | same |
| `dataweave-native-0.0.1-Linux.tgz` | `Linux` (upper) | ❌ | never says "node"; ambiguous vs Python |
| `dataweave-native-0.0.1-Windows.tgz` | `Windows` (upper) | ❌ | same |
| `dwlib-100.100.100-Linux` | `Linux` (upper) | ❌ | no arch |
| `dwlib-100.100.100-Windows` | `Windows` (upper) | ❌ | no arch |
| `dataweave_native-0.0.1-py3-none-manylinux2014_x86_64.whl` | in platform tag | ✅ `x86_64` | fine (auto-generated) |
| `dataweave_native-0.0.1-py3-none-win_amd64.whl` | in platform tag | ✅ `amd64` | fine (auto-generated) |

Three distinct inconsistencies:

1. **OS casing** — `Linux`/`Windows` (from `${{ runner.os }}`) vs
`linux`/`windows` (from `${{ matrix.script_name }}`), in the same run.
2. **Architecture** — present only on the Python wheels (setuptools generates
the platform tag automatically); absent everywhere else, even though every
artifact today is x86_64.
3. **Base-name clarity** — `native-cli-…-native-distro` repeats "native"; the
Node `.tgz` gives no signal it's the Node package (vs the Python one);
`dwlib` is opaque outside its FFI audience.

## Current build state (ground truth)

- **OSes:** Linux + Windows. **No macOS build** despite the code producing a
`.dylib` (the CI matrix is `mulesoft-ubuntu` + `mulesoft-windows` only).
- **Architecture:** x86_64 only, on both OSes (confirmed by the wheel platform
tags `manylinux2014_x86_64` / `win_amd64`). No arm64 / Apple Silicon.

So every artifact today is effectively `linux-x86_64` or `windows-x86_64` — the
arch simply isn't in most of the names.

## Artifact families and their purpose

| Base name (proposed) | What it is | Consumer |
|---|---|---|
| `dw-cli` | the `dw` command-line executable distribution (zip) | end users running `dw` in a terminal |
| `dataweave-node` | the Node.js binding, an npm package (`.tgz`) | Node developers (`npm install`) |
| `dwlib` | the raw shared library (`.so`/`.dll`/`.dylib`) + `.h` header | C/FFI integrators; underpins the bindings |
| `dataweave_native` (wheel) | the Python binding wheel | Python developers (`pip install`) |

## The convention

Every artifact follows one grammar:

```
<base-name>-<version>-<os>-<arch>.<ext>
```

- **`<os>`** — lowercase: `linux` | `windows` (future: `macos`)
- **`<arch>`** — lowercase, derived at runtime, not hardcoded: `x86_64` today
(future: `arm64`)
- **`<version>`** — unchanged per family: `100.100.100` for CLI/dwlib, `0.0.1`
for the Node package

### Base-name renames

| Today | Proposed | Rationale |
|---|---|---|
| `native-cli-…-native-distro-<os>.zip` | `dw-cli-100.100.100-<os>-<arch>.zip` | drops the double "native"; `dw-cli` names the actual `dw` executable |
| `dataweave-native-0.0.1-<Os>.tgz` | `dataweave-node-0.0.1-<os>-<arch>.tgz` | "node" disambiguates from the Python package |
| `dwlib-100.100.100-<Os>` | `dwlib-100.100.100-<os>-<arch>` | base kept for its FFI audience; casing + arch fixed |
| `dataweave_native-…-<platform>.whl` | **unchanged** | see exception below |

### Python wheel — deliberate exception

The wheel name (`dataweave_native-0.0.1-py3-none-manylinux2014_x86_64.whl`) is
generated by setuptools and **must** stay PEP 427-compliant
(`{name}-{version}-{python}-{abi}-{platform}.whl`) or `pip install` and platform
resolution break. It already carries the architecture in its platform tag, so it
needs no change. It is documented here as the one family that intentionally does
not follow the `<os>-<arch>` grammar.

### Result — 8 artifacts

```
dw-cli-100.100.100-linux-x86_64.zip
dw-cli-100.100.100-windows-x86_64.zip
dataweave-node-0.0.1-linux-x86_64.tgz
dataweave-node-0.0.1-windows-x86_64.tgz
dwlib-100.100.100-linux-x86_64
dwlib-100.100.100-windows-x86_64
dataweave_native-0.0.1-py3-none-manylinux2014_x86_64.whl (exception)
dataweave_native-0.0.1-py3-none-win_amd64.whl (exception)
```

## Implementation notes (where each rename actually happens)

A prior change set three of the four upload steps to
`actions/upload-artifact` with `archive: false`. Under `archive: false` the
`name:` input is **ignored** — the artifact name is the uploaded **file's own
name**. So renaming those three is not a CI-label tweak; it means renaming the
produced *file*.

- **`dw-cli` (CLI distro)** — rename the zip the Gradle `native-cli:distro`
task produces (the file basename currently
`native-cli-<ver>-native-distro-<script_name>.zip`). This is the file whose
name the upload step points at.
- **`dataweave-node` (Node tgz)** — the "Stage OS-qualified Node package" step
copies the `npm pack` output to an OS-qualified name; change that copy target
to `dataweave-node-0.0.1-<os>-<arch>.tgz`. `npm pack` still emits its own
fixed name; the staging copy is what gets uploaded.
- **`dwlib`** — this step still uses `name:` (it uploads multiple files —
`.so`/`.dll`/`.dylib` + `.h` — so it stays archived, and under `archive: true`
`name:` is honored). Its label is changed directly on the `name:` input; no
file rename needed.
- **Python wheel** — no change.

### Deriving the tokens

- **OS:** use a lowercase token. Prefer the existing `matrix.script_name`
(`linux`/`windows`) so casing is consistent by construction, rather than
`${{ runner.os }}` (`Linux`/`Windows`).
- **Arch:** derive at runtime from `uname -m` (all steps already run under
`shell: bash`), normalizing to `x86_64`. Deriving rather than hardcoding keeps
the names correct automatically if an arm64 runner is ever added. On the
current x86_64 runners this resolves to `x86_64` on Linux; Windows is
normalized to `x86_64`.

## `release.yml` alignment (in scope)

`release.yml` publishes the same four families as GitHub **release assets** on
`v*` tags, via `svenstaro/upload-release-action`. That action honors
`asset_name:` and does no zipping, so aligning it is **cheaper than the CI
side**: every rename is a single `asset_name:` string edit — no Gradle changes,
no file renames, and none of the `archive: false` subtlety.

Same convention applies: **OS lowercase** (switch `${{ runner.os }}` →
`${{ matrix.script_name }}`) and **arch runtime-derived** (add one
`echo "ARCH=$(uname -m)" >> $GITHUB_ENV` step, reference `${{ env.ARCH }}`).

### Asset renames

| Asset | Current `asset_name` | Spec-aligned |
|---|---|---|
| CLI distro | `dw-${VER}-${runner.os}` (missing `.zip`) | `dw-cli-${VER}-${os}-${arch}.zip` |
| Node tgz | `dw-node-package-${VER}-${runner.os}.tgz` | `dataweave-node-0.0.1-${os}-${arch}.tgz` |
| dwlib `.so` | `dwlib-${VER}-${runner.os}.so` | `dwlib-${VER}-${os}-${arch}.so` |
| dwlib `.dll` | `dwlib-${VER}-${runner.os}.dll` | `dwlib-${VER}-${os}-${arch}.dll` |
| dwlib `.h` | `dwlib-${VER}.h` | **unchanged** (header is os/arch-independent) |
| Python wheel | `dw-python-wheel-${VER}-${runner.os}.whl` | keep the wheel's own PEP 427 name (drop `asset_name:`) |

### Two deliberate divergences from the CI naming

- **dwlib ships as separate release assets** (`.so`/`.dll`/`.h`, each its own
asset), whereas CI bundles them into one archived `dwlib-…` artifact. Release
dwlib assets therefore carry their file extension — correct for individual
assets, not an inconsistency.
- **The wheel is the same PEP 427 exception, for a sharper reason here.** Today
the asset is renamed to `dw-python-wheel-….whl`; if a user runs `pip install`
on that downloaded file, pip validates the filename against PEP 427 and the
non-compliant name breaks the install / platform resolution. The asset must
keep the wheel's real name (`dataweave_native-0.0.1-py3-none-<platform>.whl`)
— simplest is to drop `asset_name:` and let the action use the file's own
name.

### Pre-existing bug to fix in the same pass

- **Wheel file path (line ~95): `…-py3-none-any.whl`.** The build produces a
*platform-tagged* wheel (`manylinux2014_x86_64` / `win_amd64`), not
`-any.whl`, so this `file:` path does not match the produced file and the
upload step fails to locate it. Change the glob to match the real platform tag
(e.g. `…-py3-none-*.whl`). (The stray `I` typo on the CLI-distro `tag:` line
has already been fixed separately.)

## Blast radius

- **CI artifact labels change** in the Actions UI and in the downloaded
filenames. Anyone with a hardcoded reference to an old name (scripts, docs)
must update.
- **Release asset names change** on the next `v*` tag. Same caveat — hardcoded
references to old release-asset names must update.
- **No runtime/API/binding-code impact** — this is packaging/labeling only.

## Out of scope (noted as future work)

- **macOS build** — the code produces a `.dylib`, but no macOS runner is in the
CI matrix, so no macOS artifact ships. Adding one is a separate effort.
- **arm64 / Apple Silicon builds** — no arm64 runner today. The convention is
arm64-ready (arch is derived, not hardcoded), but no such artifact is
produced yet.

## Success criteria

- All CI artifacts (except the documented wheel exception) match
`<base-name>-<version>-<os>-<arch>.<ext>`.
- OS token is lowercase everywhere.
- Architecture appears in every artifact name.
- Base names read clearly to a customer: `dw-cli`, `dataweave-node`, `dwlib`.
- `pip install` of the wheel and `npm install` of the tgz still work unchanged.
- **Release assets** (`release.yml`) follow the same convention, with the
documented divergences (per-file dwlib assets, PEP 427 wheel name).
- The `release.yml` wheel `file:` path matches the produced platform-tagged
wheel (no more `-any.whl` mismatch).
Loading
Loading