Written by Jamie Tanna
on 
CC-BY-NC-SA-4.0 Apache-2.0
10 mins

Properly patching packages: persistently producing patches for published projects, particularly practically prevented by patch-package policy

Featured image for sharing metadata for article

One thing I've been building for a few years is renovate-graph, a lightweight shim on top of Renovate (aka Mend Renovate CLI) to extract package data from repositories, primarily for use with dependency-management-data.

For the most part, you can use vanilla Renovate for this functionality, but renovate-graph gives you some better defaults if you only want to see what Renovate can detect in a given repository.

As I've worked with Renovate and renovate-graph, I've learned how you could utilise Renovate's local platform to make renovate-graph more efficient, which has known limitations such as being unable to resolve local> presets.

However, when running under renovate-graph, we do have that information, as it's part of the Repo Key that is then used to attribute the local repository to, so we may as well try and resolve the local> presets.

This does, however, require that we tweak some internal Renovate code to make this work, which isn't something that would ever make sense to contribute upstream to Renovate, as it's very specific to renovate-graph.

In this case, we have a few options on how to do this - one very painful option would be to maintain a separate fork of Renovate, which I didn't want to do.

I settled on - in the short term - vendoring a copy of the code that we were modifying, which was very painful to upgrade, and was recently removed as I'd hoped that pnpm patch would make this more maintainable.

It did, but unfortunately only for folks locally hacking on the renovate-graph codebase, as the patching capabilities are only applied when someone is using pnpm locally, and the patches (somewhat reasonably) do not translate to the final published library, who may not be using pnpm.

patch-package doesn't support patching dependencies, when distributing a library/CLI

It was at this point I turned to the patch-package package, which was slightly less nice than pnpm patch, but gave me the functionality + maintainability I needed.

This worked well, until it was needed by someone who was running i.e. npx @jamietanna/renovate-graph@latest, where the postinstall script does not run.

Through some digging through ~a dozen issues on the issue tracker, I found a few promising cases of others hitting this issue, but unfortunately with no fix:

I'd also found as a potential StackOverflow answer, and a possible workaround but they didn't seem to work for me.

Should I do this?

Through this digging, I spotted this reasonable concern from the maintainer:

Hi! 👋 It is not safe to publish npm packages with patched dependencies (except devDependencies) so we don't support it.

I also noticed this comment:

Sorry, patch-package doesn't support publishing libraries. This has been asked before a number of times but it's not safe and therefore not something i want to encourage. If you really want to do this for some reason I'm gonna leave it up to you to figure out how to write a postinstall script that would work ;)

Now, this one really piqued my interest and got me thinking - there must be a way to do it, so how could I go about doing it, given I didn't want to have to go back to the vendored code approach? 👀

I ended up doing some significant digging, and finally managed to work out how to do this yesterday 🚀

But it did also give me some pause, wondering if I should be doing this, and moreso documenting that this is how I did it 😅

As the maintainer of patch-package puts it, you probably shouldn't be looking to use this functionality if you're publishing packages, as it can break functionality that other packages may be expecting of the package you're patching, but could also lead to you being able to perform attacks on other packages and i.e. inject them with code that could allow you to do malicious things with executing code.

I'd strongly discourage the use of this workaround for all usecases other than "I have a command-line tool I'm distributing, and I need to apply a patch to a dependency I have", as that's what I'm using it for, and feel that this is the least worst means for doing this.

Example project

Note that given the project I've been doing the testing on is a fork of renovate-graph, which itself has a lot of code from Renovate, they're all released under the GNU Affero General Public License (AGPL-3.0), which doesn't do well to copy code snippets from.

I very much like the AGPL and am very happy using it, but I feel that for folks wanting to learn how I ended up being able to distribute patches for patch-package alongside executable code, they may not want to "infect" themselves with AGPL, so I'm sharing the steps in a "clean room" implementation of the code.

Solution

So how do we manage to hack patch-package to always install, regardless of whether postinstall is executed?

The first thing is that we should remove the postinstall script. Not only do we not need it any more, but it'll allow us to avoid risking broken CI builds if patching fails, which may not be intended.

Next, you may already be making sure that your patches directory is included in your final NPM package - if so, great. If not, make sure you add it:

{
  "files": [
    "dist/*",
    "patches/*"
  ]
}

Finally, the real bulk of the work, is making sure that we run patch-package when the executable starts.

This moves the installation of our patches from installation time, to runtime, which won't always work, but from my testing seems to be a bit more resilient.

Note that this does run the risk of the node_modules in the resulting execution environment not being writeable (for reasonable hardening reasons).

I found that there are two key cases that we need to handle to apply these patches:

The package you want to patch is installed in your node_modules folder

In my testing, I found that this happened when installing via npm i -g @jamietanna/renovate-graph, we would see the install path contain a node_modules with the relevant NPM packages for that executable.

Therefore, to execute the patching, you would have to write a Javascript/Typescript function which does something like:

# i.e. we're currently executing as the executable script in /usr/local/lib/node_modules/@jamietanna/renovate-graph/dist/executable.js
# so we need to make sure we go up to the module root, which then has the `node_modules`
cd ..

# this then means that Renovate is in the `node_modules` i.e. /usr/local/lib/node_modules/@jamietanna/renovate-graph/node_modules/renovate/package.json
ls node_modules/renovate/package.json

# then, we need to trigger patching
node_modules/.bin/patch-package

This is the same sort of flow that you'd see when using the postinstall.

The package you want to patch is in the same node_modules to the package performing the patching

In my testing, I found that there are a couple of cases where i.e. both renovate-graph and renovate were installed in the same node_modules folder:

  • npm i @jamietanna/renovate-graph
  • npx @jamietanna/renovate-graph@latest

Therefore, to execute the patching, you would have to write a Javascript/Typescript function which does something like:

# i.e. we're currently executing as the executable script in /root/.npm/_npx/4ad3a3b298976db6/node_modules/@jamietanna/renovate-graph/dist/executable.js
# so we need to make sure we go up to the parent folder for the `node_modules`
cd ../../../..

# this then means that Renovate is in the same `node_modules` than the installed `renovate-graph` i.e. /root/.npm/_npx/4ad3a3b298976db6/node_modules/renovate/package.json
ls node_modules/renovate/package.json

# then, we need to trigger patching
node_modules/.bin/patch-package --patch-dir node_modules/@jamietanna/renovate-graph/patches

This is slightly more complicated, and requires we be explicit about the relative path to the --patch-dir.

Testing

(These were tested against both Node 20 and Node 22, but I've not shared all the runs here)

Globally installing with npm (npm install -g @jamietanna/renovate-graph)

% docker run -v $PWD:/app -ti node:22 bash
# npm i -g @jamietanna/renovate-graph@0.28.2
# env LOG_LEVEL=debug renovate-graph
DEBUG: Attempting to patch Renovate
       "functionName": "attemptToPatchLocalNodeModulesFolder"
DEBUG: Attempting to run `node_modules/.bin/patch-package --error-on-fail --patch-dir patches` to patch `renovate`
       "functionName": "attemptToPatchLocalNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/usr/local/lib/node_modules/@jamietanna/renovate-graph",
       "commandAndArgs": "node_modules/.bin/patch-package --error-on-fail --patch-dir patches"
DEBUG: Executing command
       "command": "node_modules/.bin/patch-package --error-on-fail --patch-dir patches"
DEBUG: exec completed
       "durationMs": 251,
       "stdout": "patch-package 8.0.0\nApplying patches...\nrenovate@39.191.4 ✔\n",
       "stderr": ""
DEBUG: Successfully ran `node_modules/.bin/patch-package --error-on-fail --patch-dir patches`, with stdout:
       patch-package 8.0.0
       Applying patches...
       renovate@39.191.4 ✔

       "functionName": "attemptToPatchLocalNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/usr/local/lib/node_modules/@jamietanna/renovate-graph",
       "commandAndArgs": "node_modules/.bin/patch-package --error-on-fail --patch-dir patches",
       "stdout": "patch-package 8.0.0\nApplying patches...\nrenovate@39.191.4 ✔\n"
 INFO: Successfully patched Renovate
       "functionName": "attemptToPatchLocalNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/usr/local/lib/node_modules/@jamietanna/renovate-graph",
       "commandAndArgs": "node_modules/.bin/patch-package --error-on-fail --patch-dir patches"

Locally installing with npm (npm install @jamietanna/renovate-graph)

% docker run -v $PWD:/app -ti node:22 bash
# mkdir /project
# cd /project
# npm i @jamietanna/renovate-graph@0.28.2
# env LOG_LEVEL=debug npm exec renovate-graph
DEBUG: Attempting to patch Renovate
       "functionName": "attemptToPatchLocalNodeModulesFolder"
DEBUG: The command, `node_modules/.bin/patch-package`, was found but was not a file - skipping patching attempt
       "functionName": "attemptToPatchLocalNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/project/node_modules/@jamietanna/renovate-graph",
       "e": {
         "errno": -2,
         "code": "ENOENT",
         "syscall": "stat",
         "path": "/project/node_modules/@jamietanna/renovate-graph/node_modules/.bin/patch-package",
         "message": "ENOENT: no such file or directory, stat '/project/node_modules/@jamietanna/renovate-graph/node_modules/.bin/patch-package'",
         "stack": "Error: ENOENT: no such file or directory, stat '/project/node_modules/@jamietanna/renovate-graph/node_modules/.bin/patch-package'\n    at Object.statSync (node:fs:1740:25)\n    at attemptToPatchLocalNodeModulesFolder (/project/node_modules/@jamietanna/renovate-graph/dist/check-if-patched.js:165:25)\n    at async attemptToPatchRenovate (/project/node_modules/@jamietanna/renovate-graph/dist/check-if-patched.js:134:9)\n    at async ensureRenovateDependencyIsPatched (/project/node_modules/@jamietanna/renovate-graph/dist/check-if-patched.js:78:5)\n    at async /project/node_modules/@jamietanna/renovate-graph/dist/executable.js:44:5"
       }
DEBUG: Did not patch Renovate, trying again
       "functionName": "attemptToPatchLocalNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/project/node_modules/@jamietanna/renovate-graph"
DEBUG: Attempting to patch Renovate
       "functionName": "attemptToPatchParentNodeModulesFolder"
DEBUG: Attempting to run `node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches` to patch `renovate`
       "functionName": "attemptToPatchParentNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/project",
       "commandAndArgs": "node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches"
DEBUG: Executing command
       "command": "node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches"
DEBUG: exec completed
       "durationMs": 177,
       "stdout": "patch-package 8.0.0\nApplying patches...\nrenovate@39.191.4 ✔\n",
       "stderr": ""
DEBUG: Successfully ran `node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches`, with stdout:
       patch-package 8.0.0
       Applying patches...
       renovate@39.191.4 ✔

       "functionName": "attemptToPatchParentNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/project",
       "commandAndArgs": "node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches",
       "stdout": "patch-package 8.0.0\nApplying patches...\nrenovate@39.191.4 ✔\n"
 INFO: Successfully patched Renovate
       "functionName": "attemptToPatchParentNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/project",
       "commandAndArgs": "node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches"

Installing via npx (npx @jamietanna/renovate-graph@latest)

% docker run -v $PWD:/app -ti node:22 bash
# env LOG_LEVEL=debug npx @jamietanna/renovate-graph@0.28.2
DEBUG: Attempting to patch Renovate
       "functionName": "attemptToPatchLocalNodeModulesFolder"
DEBUG: The command, `node_modules/.bin/patch-package`, was found but was not a file - skipping patching attempt
       "functionName": "attemptToPatchLocalNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/root/.npm/_npx/6e5c757a7e3d9deb/node_modules/@jamietanna/renovate-graph",
       "e": {
         "errno": -2,
         "code": "ENOENT",
         "syscall": "stat",
         "path": "/root/.npm/_npx/6e5c757a7e3d9deb/node_modules/@jamietanna/renovate-graph/node_modules/.bin/patch-package",
         "message": "ENOENT: no such file or directory, stat '/root/.npm/_npx/6e5c757a7e3d9deb/node_modules/@jamietanna/renovate-graph/node_modules/.bin/patch-package'",
         "stack": "Error: ENOENT: no such file or directory, stat '/root/.npm/_npx/6e5c757a7e3d9deb/node_modules/@jamietanna/renovate-graph/node_modules/.bin/patch-package'\n    at Object.statSync (node:fs:1740:25)\n    at attemptToPatchLocalNodeModulesFolder (/root/.npm/_npx/6e5c757a7e3d9deb/node_modules/@jamietanna/renovate-graph/dist/check-if-patched.js:165:25)\n    at async attemptToPatchRenovate (/root/.npm/_npx/6e5c757a7e3d9deb/node_modules/@jamietanna/renovate-graph/dist/check-if-patched.js:134:9)\n    at async ensureRenovateDependencyIsPatched (/root/.npm/_npx/6e5c757a7e3d9deb/node_modules/@jamietanna/renovate-graph/dist/check-if-patched.js:78:5)\n    at async /root/.npm/_npx/6e5c757a7e3d9deb/node_modules/@jamietanna/renovate-graph/dist/executable.js:44:5"
       }
DEBUG: Did not patch Renovate, trying again
       "functionName": "attemptToPatchLocalNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/root/.npm/_npx/6e5c757a7e3d9deb/node_modules/@jamietanna/renovate-graph"
DEBUG: Attempting to patch Renovate
       "functionName": "attemptToPatchParentNodeModulesFolder"
DEBUG: Attempting to run `node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches` to patch `renovate`
       "functionName": "attemptToPatchParentNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/root/.npm/_npx/6e5c757a7e3d9deb",
       "commandAndArgs": "node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches"
DEBUG: Executing command
       "command": "node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches"
DEBUG: exec completed
       "durationMs": 176,
       "stdout": "patch-package 8.0.0\nApplying patches...\nrenovate@39.191.4 ✔\n",
       "stderr": ""
DEBUG: Successfully ran `node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches`, with stdout:
       patch-package 8.0.0
       Applying patches...
       renovate@39.191.4 ✔

       "functionName": "attemptToPatchParentNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/root/.npm/_npx/6e5c757a7e3d9deb",
       "commandAndArgs": "node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches",
       "stdout": "patch-package 8.0.0\nApplying patches...\nrenovate@39.191.4 ✔\n"
 INFO: Successfully patched Renovate
       "functionName": "attemptToPatchParentNodeModulesFolder",
       "command": "node_modules/.bin/patch-package",
       "cwd": "/root/.npm/_npx/6e5c757a7e3d9deb",
       "commandAndArgs": "node_modules/.bin/patch-package --error-on-fail --patch-dir node_modules/@jamietanna/renovate-graph/patches"

pnpx @jamietanna/renovate-graph@latest

This does not work as I've not yet added support for it, as pnpm works slightly differently.

Aside: What's that long title about?

So this was a slightly convoluted title that wasn't as concise as it maybe would've been, as I thought it would be fun to try and play around with options.

I tried to think of a title that would be a little bit of a fun tongue twister:

Patching NPM packages with patch-package patches of dependencies you distribute, when the patch-package package maintainer doesn't want you to patch those packages

Then I asked my local Ollama setup, with qwen2-5-coder:32b, to try and reword this to include more words beginning with p for a good amount of plosive alliteration, and slightly stolen from a recent episode of Fallthrough I was on, which resulted in a few different variants:

Parsing Patched Packages: Persisting Patches per Project's Preferences

Properly Patching Packages: Persistently Producing Patches for Published Projects, Particularly Practically Prevented by Patch-Package Policy

Prudent Patching: Persistently Pondering Package Proposals, Particularly Patch-Package's Parameters, Perplexing Practitioners' Preferences, Promptly Providing Practical Possibilities.

Not bad, albeit more verbose!

Written by Jamie Tanna's profile image Jamie Tanna on , and last updated on .

Content for this article is shared under the terms of the Creative Commons Attribution Non Commercial Share Alike 4.0 International, and code is shared under the Apache License 2.0.

#blogumentation #nodejs #npm #dependency-management-data.

This post was filed under articles.

Interactions with this post

Interactions with this post

Below you can find the interactions that this page has had using WebMention.

Have you written a response to this post? Let me know the URL:

Do you not have a website set up with WebMention capabilities? You can use Comment Parade.