js-dependency-mining
Read-only mining codemod: scans package.json manifests (single-package repos and monorepos) together with package-lock.json, pnpm-lock.yaml, or yarn.lock, classifies each dependency, and emits js-dependency-mining metrics. It does not modify source files.
No fs capability required — no Node fs or path imports. The workflow only targets **/package.json. Lockfiles are loaded with jssgTransform and small JSON/YAML transforms that capture root.root().text() (same mechanism as cross-file edits in the JSSG docs).
Classifications
Each finding is keyed by owner package (workspace-relative path), package name, and edge type (direct | transitive). Metric dimensions include:
| Field | Values |
|---|---|
packageName | npm package name |
declaredVersion | Range or spec from the manifest (empty for transitive-only rows) |
resolvedVersion | Version from the workspace lockfile when resolvable (empty if missing or unknown) |
lockfileKind | npm, pnpm, yarn, or none when no lockfile was loaded |
overridePin | Best-effort pins from npm overrides, Yarn resolutions, or pnpm pnpm.overrides for this dependency in the current manifest (empty for transitive rows and when absent) |
requiredBy | Transitive only: immediate parent package name(s) in the lockfile graph that depend on this package (sorted, joined with |). Empty for direct rows or when unknown. |
ownerPackage | Workspace-relative owning package path |
manifestPath | Path to package.json (relative to workspace root) |
edgeType | direct, transitive |
purpose | runtime, dev, test, build, optional, unknown |
origin | registry, vcs, local, workspace, vendored, internal, unknown |
versionPolicy | pinned, ranged, prerelease, unknown |
Workspace root
While processing a manifest, the codemod walks up the directory path and uses jssgTransform to probe for package-lock.json (language json), pnpm-lock.yaml / yarn.lock / pnpm-workspace.yaml (language yaml), and reads sibling package.json files to detect npm workspaces. The first directory that qualifies as a workspace root wins; the manifest is then resolved relative to that root.
Pipeline
- Workspace root — Walk upward from the manifest; at each level, try lockfiles and workspace markers via
jssgTransformuntil a root is found. - Lockfile — From that root, load the preferred lockfile in order npm → pnpm → yarn using
jssgTransform+ capture transforms (jsonforpackage-lock.json,yamlfor pnpm/yarn). Parse withJSON.parse, tree-sitter YAML AST helpers for pnpm (with a text fallback for very large files), and the built-in Yarn Classic line parser. The loaded lockfile for a given workspace root is cached for the rest of the run so each manifest does not re-read the same file. - Metrics — For each dependency block in the manifest, emit direct rows; subtract directs from the lockfile transitive closure and emit transitive rows. While walking the lockfile graph, record immediate parent → child edges so transitive rows can list who pulled that package in (
requiredBy).
Workspace package names (for origin=workspace heuristics) are accumulated in a module-level registry as each package.json in the run is seen — no codemod:workflow state.
Configuration
Parameters are declared in workflow.yaml (params.schema) and merge with built-in defaults (extra test/build lists, default vendored fragments, etc.).
Where to edit them
- Codemod Insight UI — When you run this workflow in Insight, each parameter appears as a labeled field. You can change values there without editing YAML; that is the easiest way to tune classification for a run or saved job.
- CLI — Pass
--param key=valuetocodemod workflow run(repeat for multiple keys), e.g.--param internal_scopes=@acme --param test_packages=playwright.
Parameters and examples (values are comma-separated unless noted):
| Parameter | Example | What it does |
|---|---|---|
internal_scopes | @acme,@calcom | Scope prefixes for origin=internal. Packages whose names start with these (e.g. @acme/foo) are treated as internal when not better classified as workspace/VCS/local. |
test_packages | storybook,@storybook/react | Extra names merged into the test-tool list. If listed in devDependencies, purpose can be test (instead of generic dev). |
build_packages | turbo,tsc-alias | Extra names merged into the build-tool list. If listed in devDependencies, purpose can be build. |
vendored_path_fragments | patches/,/vendor/ | Extra path substrings. For file: / link: dependency specs, if the path contains one of these (in addition to defaults like vendor/), origin can be vendored. |
Implementation notes
- No
fs/pathimports. Path operations use pure-string helpers.jssgTransformreads lockfiles under the workflow target; paths must stay within that target (per JSSG rules). jssgTransforminjssg testis a no-op — lockfiles are not loaded in unit tests, so snapshots typically show direct dependencies only. Real runs (codemod workflow run/ Insight) load lockfiles and emit transitive rows.- Yarn classic lockfiles are read with language
yamlfor parsing; the custom line parser runs on the captured text. If ast-grep cannot parse a particularyarn.lock, that lock may be skipped (catch and continue). Berry-style dependency lines (name: "npm:…") can make naive key captures include a trailing colon; those are stripped when building package-name metrics. - Transitive dependencies are derived from the lockfile closure minus direct declarations (npm and pnpm v9 snapshots supported; yarn classic is best-effort).
- Conservative: ambiguous cases use
unknownwhere classification would be a guess (e.g. transitivepurpose). - pnpm
pnpm-lock.yamlis deserialized from the tree-sitter YAML AST (inline helpers inscripts/codemod.ts) inside ajssgTransformcallback — no third-party YAML library, so Insight bundling stays simple. - Overrides —
overridePinreflects override/resolution fields on the manifest being scanned; root-level overrides in anotherpackage.jsonare not merged in (scan the workspace root manifest for a full picture).