#1 Implement V1

[NoelDeMartin]

First version of the toolkit including the following:

• CI config to publish dev versions (prereleases).
• Vite plugin and build config for Solid Panes, extracting build set up and dev mode (replacing the old dev.ts sandbox).
• Exports some vite helpers that we could probably reuse in solid-ui as well, such as removing the duplication of configuring Lit components.

[timea-solid]

I would need to see it in action in a pane to understand fully. SO I approve it.
What I would like to know already is:

• explain usage of pnpm
• I see peerDependencies on major versions... 1) why? 2) I am not sure if we are actually going away from peer dependencies or what.

[NoelDeMartin]

I would need to see it in action in a pane to understand fully.

I just opened a PR in contacts-pane using it, check it out: SolidOS/contacts-pane#290

explain usage of pnpm

pnpm is almost the same as npm, but it has better performance and security features. You can learn more about it in the official documentation, but pretty much is a drop-in replacement for npm. Most commands are the same, you just have to run "pnpm" instead of "npm"... Although this repository uses Vite+, so actually you need to run vp 😅. Maybe we can set up a call so that I explain the repository and tools I'm using. Ideally, I would love if we start using this in all our repos, but I'm aware it's too much of a change so I'm not suggesting to do it now.

I see peerDependencies on major versions... 1) why? 2) I am not sure if we are actually going away from peer dependencies or what.

Yeah, this should probably be changed. It doesn't matter too much until we do a release, though (not a prerelease).

But no, we shouldn't go away from peerDependencies. We'll have to talk with @bourgeoa to see why his CI scripts are removing them.

[NoelDeMartin]

Here's some additional documentation about this PR, and some comments before tomorrow's meeting:

Prereleases

Prereleases are published automatically with any new commits in main. It doesn't matter how these commits get there, though with strict branch protection rules they'll only be pushed to main from a PR.

The prerelease tags are {currentVersion}-dev.{commitHash}. There are a few reasons for that:

• With automatic prereleases, it's difficult to follow a sequential prerelease number. Additionally, this format allows to easily see which commit the prerelease comes from (given that we're not tagging prereleases in github, or we shouldn't because it creates too much noise).
• Technically, this approach is not correct because it indicates a prerelease for a version that has already been released. In my own repositories, I use the -next.{hash} suffix instead to indicate that it's a version of the upcoming release. Even if this is not right, I still prefer it because it doesn't force me to decide which will be the next version. For example, if the last released version of a package was v0.3.1, what will be the next version? v0.3.2 or v0.4.0? maybe even v1.0.0? That's something I don't want to decide until I actually have to release. An additional benefit of this approach is that the version in package.json only changes for releases.

See the publish job in ci.yml for details on how all of this is automated.

Releases

This isn't configured in the repository yet, but releases should be published automatically to npm every time a release is tagged in github. In my projects, I also have a dedicated workflow that I use to publish versions manually, but I rarely use it.

peerDependencies

It is important to declare these for a couple of reasons. First of all, strict package managers (see Tooling) can throw errors if they aren't declared. And build tools like Vite can also use them to detect whether a dependency is external or not.

However, it's not straightforward to declare them when they depend on prereleases. I thought the * specifier matched everything, but apparently it doesn't match prereleases. The solution I've come up with is the following:

"peerDependencies": {
	"solid-ui": "^3.1.2 || ^3.1.3-0",
},

This will match 3.X releases and any prerelease from 3.1.3. Unfortunately, I don't think there is a way to make it match any prerelease, so every time we release a dependency, we'll have to update it (we would probably be updating the package.json anyways).

Tooling

The repository uses the pnpm package manager. For the most part, it's a drop-in replacement for npm, and uses the same registry, etc. But it has better performance, some additional features, and improved security.

It also uses Vite+, which is a unified toolchain to orchestrate pnpm, Vite, and other tooling. The added benefit is that everything is configured in vite.config.ts, including linting and formatting (which use oxc and oxfmt, more performant replacements for ESLint and Pretter).

Updating dependencies

In my projects, I never update any dependency automatically. Here's my reasons for each type.

SolidOS dependencies

If we need to use a new version of a package across the stack, we should update it manually. Otherwise, merging a broken package will cripple the CIs for all its dependants. In fact, we can see this happening in CI right now. rdflib recently released version v2.4.0, but solid-panes (and maybe others) depend exactly on 2.3.9. Because of that, now all CIs are broken because they try to update rdflib, but this causes a conflict with solid-panes. Here's one such example: https://github.com/SolidOS/solid-ui/actions/runs/28109692851/job/83233398913

But this is just one example. Imagine I introduce a breaking change in solid-ui. If I also update all the tests and the package itself is fine, the CI will work and will let me merge it. However, all other packages in the solidos stack will now fail, because they're going to be forced to update the dependency without necessarily changing their code. I think a good rule of thumb is that a project's CI should only fail for changes made in that same repository. If the CI depends on external packages (or scripts that update things automatically), CIs can start failing across the stack from changes unrelated to their PRs. Which creates a lot of headaches.

Instead of updating dependencies automatically, if we want this we'll probably need an ecosystem-ci.

The real solution, though, would be to use a monorepo. But as we've mentioned before, that's a lot more work.

npm audit / dependabot

These seem very important, but in practice they're rarely so. I could write more about this, but Dan Abramov already did so check out his blog post: npm audit: Broken by Design. TLDR: most of these vulnerability reports may not apply to our project, because they only see if a dependency is installed in our project, not how it's used or even if it's a dev dependency.

Other dependency updates

It may seem a good idea to update dependencies just to "stay up to date", but actually this has been one of the main security issues recently. WIth the advent of AI, one of the most common attacks are Supply chain attacks. Like the infamous Shai-Hulud 2.0.

Because of that, I think we should only upgrade dependencies at regular intervals (every few months), and consiciously looking at the impact of doing so. Some upgrades do introduce breaking changes, even if they are technically not breaking according Semver (not all packages follow it perfectly, and some can also introduce breaking changes by mistake).

Some package managers, like pnpm, have some built-in mechanisms to avoid some of these. For example, they don't run postinstall scripts automatically, and ignore updates that don't have a minimum release age.

Global workflows & scripts

I noticed we now have some global workflows that are reused in different repositories. I don't think this is a good idea for the rule of thumb I mentioned previously, CIs shouldn't be broken by external changes in a different repository. Of course, things can break if a dependency changes; but if dependency updates are not automated, this will not break the rule of thumb.

Instead, if we want to reuse scripts or workflow configurations, we should keep them in a package. They will still be reused, but updates won't break other packages if we don't have automatic updates. I also do this for my own projects, in @noeldemartin/scripts, and here we could encapsulate all of this inside of solidos-toolkit.