[edit: after posting this comment i found out Marak might not be exactly a good person (you can check their Twitter if you feel like). my personal opinion on the consequences hasn't really...
[edit: after posting this comment i found out Marak might not be exactly a good person (you can check their Twitter if you feel like). my personal opinion on the consequences hasn't really changed, though I now realize that Marak's reasons on why they did this might not be the best. The rest of this comment has no major edits since I first posted it]
This article feels a bit, (i can't find the right word, but) selfish.
Calling Marak (the dev that pulled their packages) "sadistic", "problematic", etc.
Why did Marak pull his packages?
[Marak] will no longer be supporting the big corporations with his "free work" and that commercial entities should consider either forking the projects or compensating the dev with a yearly "six figure" salary.
I am 100% behind his decision, they have (or should have) all the right to at least inconvenience (I don't think you can "take back" permissions from open source licenses), and I feel like I might've done the same thing if I had something of the same "importance".
If you don't want your software to be used by someone, why contribute to FOSS at all?
Who is "someone"? I would want my software to be used by individuals and small non-profits, because at least they are more likely to be ethical, have similar values as me, and they probably pay anybody they hire a living wage.
Why should I want (or be forced to want, as the author here seems to want) to my software to be used by, idk, the hate speech breeding ground Facebook? Amazon's basically slave labor fulfillment centers? The military industrial complex? Anybody else who has millions of dollars to bribe governments but can't give to the people they benefit from?
(At least give me a huge paycheck so I can try to forget the consequences of my actions /joke)
A developer with this mindset should obviously go full proprietary and start licensing their software, because that is what businesses who want to make money do (including the so called "big corporations"!).
Because maybe this isn't how this developer feels like? There are many reasons to make software open source that doesn't involve "make Facebook profit even more".
Developers like Marak don't have that vision, they are confused about what open source is all about, and their role in it.
"Open source" is legalese. It's legalese about making software open for everyone to use. Legalese does not include ethics. Marak's ethical framework was being violated, and they took action.
This isn't Marak's fault. It's the ecosystem's fault.
"Open source" should be "mutual cooperation", not "free work for Amazon".
That's incorrect. Releasing code as open source is giving some of your rights away without compensation. Both as a matter of law and custom, your users are not going to pay you. Expecting to get...
That's incorrect. Releasing code as open source is giving some of your rights away without compensation. Both as a matter of law and custom, your users are not going to pay you. Expecting to get paid is misunderstanding the whole idea of it. It's a gift to the world.
It sounds like he chose the wrong license and is now blaming others for his mistake.
The writing is definitely trash but the point the author makes is correct. It’s a story as old as … well not that old but it happens a lot. Developer writes open source software. Software gets...
The writing is definitely trash but the point the author makes is correct.
It’s a story as old as … well not that old but it happens a lot. Developer writes open source software. Software gets noticed. Developer sees companies with programmers paid 300k/y use his library. Developer feels jealous and regrets decision to release software for free.
Of course, nobody would have used it if it weren’t free, and they wouldn’t have gotten the attention. There is some legitimately super high quality open source software out there that deserves to be well funded — this one is probably not it to be honest.
Honestly I think the problem comes back to a lack of fair returns. Google, Amazon, Apple etc do contribute to open source… just far far far less than they should. They also pay taxes … just, not as much as they should.
Meanwhile, smaller companies tend to contribute to open source quite well as a whole. Some don’t, some do far more than they need to, but as a whole the situation is so much better than even just five years ago.
I think we need to find a way to push specifically large corps to contribute their share. I’m thinking specifically of Apple and Amazon which have been awful with the FOSS community despite benefiting massively.
Apple hired the LLVM developers and made this project into essential infrastructure that many programming languages rely on now. The improvements they made to WebKit (which later ended up in...
Apple hired the LLVM developers and made this project into essential infrastructure that many programming languages rely on now. The improvements they made to WebKit (which later ended up in Chromium and its forks) are pretty important too.
Apple is one of the worst offenders not because of its lack of contributions, but because of how explicitly hostile it is to open source in general, and how it tries to make life difficult for...
Apple is one of the worst offenders not because of its lack of contributions, but because of how explicitly hostile it is to open source in general, and how it tries to make life difficult for free software.
This goes from including extremely old versions of software (such as Bash) because it's hostile to certain FOSS licenses, to forbidding eg. non-system web engines on iOS (and having the system one suck absolute ass), creating an extremely closed ecosystem for its protocols such as airdrop, Find My, non-standard bluetooth, not supporting certain standards for cross-platform compatibility such as RFCOMM, not giving out-of-ecosystem developers good API access to the Apple ecosystem (eg. requiring an Apple device in order to sign in to an Apple account (!!!), not supporting TOTP, …), and so on, so forth.
I knew Apple was bad, but having had to go back into their ecosystem the past year, holy shit, they're not just bad, they're HORRIBLE.
Only in code, though, not monetary compensation. It’s very possible for a company to use GPL code and monetize it as a service without providing any financial assistance to the author, which would...
Only in code, though, not monetary compensation. It’s very possible for a company to use GPL code and monetize it as a service without providing any financial assistance to the author, which would seem to leave the author of the faker library in the same dilemma.
Really, it seems he’d actually rather keep the library closed source in terms of licensing, that way he can grant small hobbyist a license for free and large corporations a license for money - fairly common practice, to be fair. But he got bit by the strong culture of open source altruism and licensed it inappropriately.
I can't talk about what happens at every company, but I have worked for a company where they took this quite seriously, including having automated checks on licences of new libraries that a new...
I can't talk about what happens at every company, but I have worked for a company where they took this quite seriously, including having automated checks on licences of new libraries that a new commit would introduce to the codebase. The company's legal team also had a hand in things, and were very much on the side of "don't let your engineers include libraries with these licences" (as opposed to being lax and not caring).
I think the article overstates the damage a bit, or assumes some context that might not be obvious to the reader. Applications that use the libraries aren't immediately broken. NPM by default pins...
I think the article overstates the damage a bit, or assumes some context that might not be obvious to the reader. Applications that use the libraries aren't immediately broken. NPM by default pins the exact versions of libraries when you first add them to a project. Everyone that uses the application later will still get the pinned versions of the dependencies unless they specifically try to change the dependency versions. The only people who were affected are people who haven't heard the news, try to add the library as a dependency now, and don't know to use the specific last good version. And it looks like NPM blocked the developer's account and overrode the latest tag on the popular colors library to point to the last good version, so no one should run into surprise issues from it now.
The use of version ranges is very common though. IIRC, the affected versions were downloaded something like 14,000 times, but I may be off here. We can't check anymore :\
NPM by default pins the exact versions of libraries when you first add them to a project.
The use of version ranges is very common though. IIRC, the affected versions were downloaded something like 14,000 times, but I may be off here. We can't check anymore :\
I'm interested: for any Javascript developers out there, what stops the Javascript ecosystem from having a standard library à la "use strict"? So many NPM disasters have come from small modules...
I'm interested: for any Javascript developers out there, what stops the Javascript ecosystem from having a standard library à la "use strict"? So many NPM disasters have come from small modules like left-pad or colors here that are very reasonably a part of the standard library of many other languages.
Javascript did standardize left-pad's functionality as the String padStart method in its standard library over 5 years ago. The functionality of colors isn't something that would be standardized...
Javascript did standardize left-pad's functionality as the String padStart method in its standard library over 5 years ago. The functionality of colors isn't something that would be standardized in the Javascript language itself as it's very platform-specific, though it wouldn't be unreasonable for Node.js to standardize it in its own standard library since it chooses to handle some terminal specific stuff. (The functionality of colors is not part of the standard library of most languages I've checked, like Python, Java, and C++.)
But in general it's true the JS/Node ecosystem does try to encourage lots of little modules (well, usually larger than these two examples) instead of standardizing everything. "[The Standard Library is] Where modules go to die" is a good article about some reasons for this attitude. Things in the standard library can't ever be removed or have backwards incompatible changes made, and imperfect things in the standard library unfairly compete against possibly better alternative libraries. Having to ban a developer and freeze a package version on very rare occasion seems like a small cost to pay for a very lively library ecosystem.
A big part may be due to the one-standard, many-implementations nature of Javascript engines. New features added to the spec need to be implemented by multiple engines in a way that behaves...
A big part may be due to the one-standard, many-implementations nature of Javascript engines. New features added to the spec need to be implemented by multiple engines in a way that behaves identically across all of them, whereas 3rd party libraries only need to be written once and can work in all standards-compliant engines. So by preferring to leave funtionality out of the spec, you're reducing the amount of duplicate work and the potential surface area for engines to implement things in a slightly different way from one another which would be problematic.
Well, you can't pull everything into the standard library. A strategy like that would just be shifting the problem a little, but not fully dealing with it. And, even standard library maintainers...
Well, you can't pull everything into the standard library. A strategy like that would just be shifting the problem a little, but not fully dealing with it. And, even standard library maintainers are imperfect, and may let something slip here and there. The onus is (should be) on the downstream lib users to be prudent and careful when employing third-party code.
The thing is that we all depend on the skills and talents of millions of unnamed people every day without even realizing. Right from the Uber driver on who's driving skills you depend on to the...
The thing is that we all depend on the skills and talents of millions of unnamed people every day without even realizing. Right from the Uber driver on who's driving skills you depend on to the anonymous pilot dude who safely shuttles your flight to that anon chef who prepares your dinner, you don't know any of their names. They can all turn out to be Maraks some day by doing a strike and causing "immediate effect on your business"! The question is are they ethically justified in doing so?
A similar maintainer of Log4J Java package recently brought down the NHS systems. Does the problem lie with NHS in this case? The ethical problem with these devs is not coming off clean with their intentions. Contributing to the society was clearly never the intention of these guys, it was going after these corporations for money. This was never the ethos of open source at all, why not they form their own corporations or even dual license their code to keep commercial entities away (like JetBrains does)?
That’s more on people being lazy. There’s no reason you can’t do versioned dependencies with npm or yarn, but many people are just too lazy to manually update the versioning. After the left pad...
That’s more on people being lazy. There’s no reason you can’t do versioned dependencies with npm or yarn, but many people are just too lazy to manually update the versioning.
After the left pad debacle npm no longer allows you to delete prior versions of libraries so if you had used a versioned dependency nothing would break.
While I mostly agree with you, I do believe the NPM ecosystem makes it particularly hard to stay on top of things, given how dependency graphs explode there. It's a cultural thing IMO.
That’s more on people being lazy. There’s no reason you can’t do versioned dependencies with npm or yarn, but many people are just too lazy to manually update the versioning.
While I mostly agree with you, I do believe the NPM ecosystem makes it particularly hard to stay on top of things, given how dependency graphs explode there. It's a cultural thing IMO.
What's wrong with the Log4j maintainer? The library had a vulnerability and they patched it. Free software is explicitly provided without warranty, so they didn't do anything wrong unless I missed...
A similar maintainer of Log4J Java package recently brought down the NHS systems. Does the problem lie with NHS in this case? The ethical problem with these devs is not coming off clean with their intentions. Contributing to the society was clearly never the intention of these guys, it was going after these corporations for money. This was never the ethos of open source at all, why not they form their own corporations or even dual license their code to keep commercial entities away (like JetBrains does)?
What's wrong with the Log4j maintainer? The library had a vulnerability and they patched it. Free software is explicitly provided without warranty, so they didn't do anything wrong unless I missed something.
Correct me if I'm wrong but unlike other ecosystems like Github and NPM (where all contributors work more or less individually), the Apache Software Foundation (ASF) works as one unit. Since Log4J...
Correct me if I'm wrong but unlike other ecosystems like Github and NPM (where all contributors work more or less individually), the Apache Software Foundation (ASF) works as one unit. Since Log4J is maintained and published as an Apache software, they must have some responsibility too?
All open source software comes with no warranty. It doesn't matter where you get it. Also, the Apache Foundation provides a legal structure but the projects themselves are run by the volunteers...
All open source software comes with no warranty. It doesn't matter where you get it.
Also, the Apache Foundation provides a legal structure but the projects themselves are run by the volunteers that maintain them. It's not really different from other open source projects in that way.
Anyone running modern production systems knows about testing followed by gradual or staged rollouts, in which changes to a running system are deployed gradually over a long period of time, to reduce the possibility of accidentally taking down everything at once. For example, the last time I needed to make a change to Google’s core DNS zone files, the change was tested against many many regression tests and then deployed to each of Google’s four name servers, one at a time, over a period of 24 hours. Regression testing checks that the change does not appear to affect answers it shouldn’t have, and then the gradual rollout gives plenty of time for both automated systems and reliability engineers to notice unexpected problems and stop the rollout.
NPM’s design choice is the exact opposite. The latest version of colors was promoted to use in all its dependents before any of them had a chance to test it and without any kind of gradual rollout. Users can disable this behavior today, by pinning the exact versions of all their dependencies. For example here is the fix to aws-cdk. That’s not a good answer, but at least it’s possible.
The right path forward for NPM and package managers like it is to stop preferring the latest possible version of all dependencies when installing a new package. Instead, they should prefer to use the dependency versions that the package was actually tested with, or versions as close as possible to those. I call that a high-fidelity build. In contrast, the people who installed aws-cdk and other packages over the weekend got low-fidelity builds: NPM inserted a new version of colors that the developers of these other packages had never tested against. Users got to test that brand new configuration themselves over the weekend, and the test failed.
I don't think the suggestion is really tenable on a npm-wide level. Who exactly is doing these tests? In most cases in open source, it'd be the maintainers... but they're also the threat model, in...
I don't think the suggestion is really tenable on a npm-wide level. Who exactly is doing these tests? In most cases in open source, it'd be the maintainers... but they're also the threat model, in this case.
Hundreds of new packages are uploaded every day to NPM, and tens of thousands get updates at least. From where in the name of god are we getting third party... volunteers?... to review all these updates?
Yeah, you can have that level of scrutiny when deploying things at Google, that's because Google's engineers are paid, there is a clear hierarchical organization and people know their responsibilities.
The closest thing that would be reasonable is like a community run "stable" packages collection, of common and highly used packages. This special artifact repository is designed to only update its artifacts when community members review updates. Only by heavily limiting the scope is it anywhere near reasonable.
And it's also expected that when you get packages from the wild wild west, you either do versioning or prepare for the consequences of your convenience.
Russ Cox is basically describing how Go's new package manager works and suggesting that NPM could do something similar. (He's biased since he designed the new package manager. Of course he thinks...
Russ Cox is basically describing how Go's new package manager works and suggesting that NPM could do something similar. (He's biased since he designed the new package manager. Of course he thinks his own ideas are good!)
Here's the idea: Let's say you write an application that depends on package A, which depends on package B. When the maintainer of package A does a new release, they test it using a particular version of B (possibly upgrading it), and the version they used is recorded in the module file included in the release.
Now when you sync to new version of A then you also get the version of B that A's maintainer tested against. So A's maintainer's release testing (however much or little they did) lets you know that A and B work together, which is a partial defense against B doing a bad release. You only get relatively stable releases of indirect dependencies, since someone else has tested them besides the original author.
(This doesn't quite work for diamond dependencies, because they will do releases at different times, possibly testing against different versions of B. Then the rule is to choose the newest version of B that one of your dependencies tested.)
You could think of this as a form of social proof, where new versions gradually get proven good by downstream dependencies publishing their version numbers. It does slow down updates, which might be a problem in the case of a security fix. However, sometimes security fixes are bad too, so it's probably better that they get tested?
This is a core point, and one I think really needs more discussion as a way forward. I've discussed this in a couple of other forums since the colors/faker incident broke, but we need to embrace...
Hundreds of new packages are uploaded every day to NPM, and tens of thousands get updates at least. From where in the name of god are we getting third party... volunteers?... to review all these updates?
This is a core point, and one I think really needs more discussion as a way forward. I've discussed this in a couple of other forums since the colors/faker incident broke, but we need to embrace something like https://github.com/crev-dev/crev across ecosystems. IMO such efforts could scale if companies got involved with their audits, specifically if we begin with packages that fit certain parameters like downloaded X times per week, low number of maintainers (bus factor / Nebraska problem highlighted in that one XKCD we see for all dependency issues) and so on.
Yeah, you can have that level of scrutiny when deploying things at Google, that's because Google's engineers are paid, there is a clear hierarchical organization and people know their responsibilities.
I'm curious about how much audit actually happens version-to-version of dependencies.
Wait wait wait. This is entirely in the power of the users of the package manager to manage within their own projects. Yes, you can say "just install any version", but you can also pin a version...
NPM’s design choice is the exact opposite.
The right path forward for NPM and package managers like it is to stop preferring the latest possible version of all dependencies when installing a new package
Wait wait wait. This is entirely in the power of the users of the package manager to manage within their own projects. Yes, you can say "just install any version", but you can also pin a version in your project's package.json. To suggest "npm is designed badly" or "what npm should change is ___" is not accurately describing the current state and design of npm (nor other package managers like Ruby's rubygem system, to which my statements apply just as well).
But many dependencies are indirect and reviewing everything yourself often isn't feasible. Maybe a community of library maintainers should have some orderly process for collectively deciding...
But many dependencies are indirect and reviewing everything yourself often isn't feasible. Maybe a community of library maintainers should have some orderly process for collectively deciding whether new versions of a library are good? That's basically what he's proposing. See my other reply for more about it.
[edit: after posting this comment i found out Marak might not be exactly a good person (you can check their Twitter if you feel like). my personal opinion on the consequences hasn't really changed, though I now realize that Marak's reasons on why they did this might not be the best. The rest of this comment has no major edits since I first posted it]
This article feels a bit, (i can't find the right word, but) selfish.
Calling Marak (the dev that pulled their packages) "sadistic", "problematic", etc.
Why did Marak pull his packages?
I am 100% behind his decision, they have (or should have) all the right to at least inconvenience (I don't think you can "take back" permissions from open source licenses), and I feel like I might've done the same thing if I had something of the same "importance".
Who is "someone"? I would want my software to be used by individuals and small non-profits, because at least they are more likely to be ethical, have similar values as me, and they probably pay anybody they hire a living wage.
Why should I want (or be forced to want, as the author here seems to want) to my software to be used by, idk, the hate speech breeding ground Facebook? Amazon's basically slave labor fulfillment centers? The military industrial complex? Anybody else who has millions of dollars to bribe governments but can't give to the people they benefit from?
(At least give me a huge paycheck so I can try to forget the consequences of my actions /joke)
Because maybe this isn't how this developer feels like? There are many reasons to make software open source that doesn't involve "make Facebook profit even more".
"Open source" is legalese. It's legalese about making software open for everyone to use. Legalese does not include ethics. Marak's ethical framework was being violated, and they took action.
This isn't Marak's fault. It's the ecosystem's fault.
"Open source" should be "mutual cooperation", not "free work for Amazon".
That's incorrect. Releasing code as open source is giving some of your rights away without compensation. Both as a matter of law and custom, your users are not going to pay you. Expecting to get paid is misunderstanding the whole idea of it. It's a gift to the world.
It sounds like he chose the wrong license and is now blaming others for his mistake.
The writing is definitely trash but the point the author makes is correct.
It’s a story as old as … well not that old but it happens a lot. Developer writes open source software. Software gets noticed. Developer sees companies with programmers paid 300k/y use his library. Developer feels jealous and regrets decision to release software for free.
Of course, nobody would have used it if it weren’t free, and they wouldn’t have gotten the attention. There is some legitimately super high quality open source software out there that deserves to be well funded — this one is probably not it to be honest.
Honestly I think the problem comes back to a lack of fair returns. Google, Amazon, Apple etc do contribute to open source… just far far far less than they should. They also pay taxes … just, not as much as they should.
Meanwhile, smaller companies tend to contribute to open source quite well as a whole. Some don’t, some do far more than they need to, but as a whole the situation is so much better than even just five years ago.
I think we need to find a way to push specifically large corps to contribute their share. I’m thinking specifically of Apple and Amazon which have been awful with the FOSS community despite benefiting massively.
Apple hired the LLVM developers and made this project into essential infrastructure that many programming languages rely on now. The improvements they made to WebKit (which later ended up in Chromium and its forks) are pretty important too.
Apple is one of the worst offenders not because of its lack of contributions, but because of how explicitly hostile it is to open source in general, and how it tries to make life difficult for free software.
This goes from including extremely old versions of software (such as Bash) because it's hostile to certain FOSS licenses, to forbidding eg. non-system web engines on iOS (and having the system one suck absolute ass), creating an extremely closed ecosystem for its protocols such as airdrop, Find My, non-standard bluetooth, not supporting certain standards for cross-platform compatibility such as RFCOMM, not giving out-of-ecosystem developers good API access to the Apple ecosystem (eg. requiring an Apple device in order to sign in to an Apple account (!!!), not supporting TOTP, …), and so on, so forth.
I knew Apple was bad, but having had to go back into their ecosystem the past year, holy shit, they're not just bad, they're HORRIBLE.
Is that really a fix though? What if the "big bad corporation" simply agrees to comply with GPL terms ;-)
Only in code, though, not monetary compensation. It’s very possible for a company to use GPL code and monetize it as a service without providing any financial assistance to the author, which would seem to leave the author of the faker library in the same dilemma.
Really, it seems he’d actually rather keep the library closed source in terms of licensing, that way he can grant small hobbyist a license for free and large corporations a license for money - fairly common practice, to be fair. But he got bit by the strong culture of open source altruism and licensed it inappropriately.
I can't talk about what happens at every company, but I have worked for a company where they took this quite seriously, including having automated checks on licences of new libraries that a new commit would introduce to the codebase. The company's legal team also had a hand in things, and were very much on the side of "don't let your engineers include libraries with these licences" (as opposed to being lax and not caring).
I think the article overstates the damage a bit, or assumes some context that might not be obvious to the reader. Applications that use the libraries aren't immediately broken. NPM by default pins the exact versions of libraries when you first add them to a project. Everyone that uses the application later will still get the pinned versions of the dependencies unless they specifically try to change the dependency versions. The only people who were affected are people who haven't heard the news, try to add the library as a dependency now, and don't know to use the specific last good version. And it looks like NPM blocked the developer's account and overrode the latest tag on the popular
colorslibrary to point to the last good version, so no one should run into surprise issues from it now.The use of version ranges is very common though. IIRC, the affected versions were downloaded something like 14,000 times, but I may be off here. We can't check anymore :\
I'm interested: for any Javascript developers out there, what stops the Javascript ecosystem from having a standard library à la
"use strict"? So many NPM disasters have come from small modules likeleft-padorcolorshere that are very reasonably a part of the standard library of many other languages.Javascript did standardize
left-pad's functionality as the String padStart method in its standard library over 5 years ago. The functionality ofcolorsisn't something that would be standardized in the Javascript language itself as it's very platform-specific, though it wouldn't be unreasonable for Node.js to standardize it in its own standard library since it chooses to handle some terminal specific stuff. (The functionality ofcolorsis not part of the standard library of most languages I've checked, like Python, Java, and C++.)But in general it's true the JS/Node ecosystem does try to encourage lots of little modules (well, usually larger than these two examples) instead of standardizing everything. "[The Standard Library is] Where modules go to die" is a good article about some reasons for this attitude. Things in the standard library can't ever be removed or have backwards incompatible changes made, and imperfect things in the standard library unfairly compete against possibly better alternative libraries. Having to ban a developer and freeze a package version on very rare occasion seems like a small cost to pay for a very lively library ecosystem.
A big part may be due to the one-standard, many-implementations nature of Javascript engines. New features added to the spec need to be implemented by multiple engines in a way that behaves identically across all of them, whereas 3rd party libraries only need to be written once and can work in all standards-compliant engines. So by preferring to leave funtionality out of the spec, you're reducing the amount of duplicate work and the potential surface area for engines to implement things in a slightly different way from one another which would be problematic.
Well, you can't pull everything into the standard library. A strategy like that would just be shifting the problem a little, but not fully dealing with it. And, even standard library maintainers are imperfect, and may let something slip here and there. The onus is (should be) on the downstream lib users to be prudent and careful when employing third-party code.
The thing is that we all depend on the skills and talents of millions of unnamed people every day without even realizing. Right from the Uber driver on who's driving skills you depend on to the anonymous pilot dude who safely shuttles your flight to that anon chef who prepares your dinner, you don't know any of their names. They can all turn out to be Maraks some day by doing a strike and causing "immediate effect on your business"! The question is are they ethically justified in doing so?
A similar maintainer of Log4J Java package recently brought down the NHS systems. Does the problem lie with NHS in this case? The ethical problem with these devs is not coming off clean with their intentions. Contributing to the society was clearly never the intention of these guys, it was going after these corporations for money. This was never the ethos of open source at all, why not they form their own corporations or even dual license their code to keep commercial entities away (like JetBrains does)?
That’s more on people being lazy. There’s no reason you can’t do versioned dependencies with npm or yarn, but many people are just too lazy to manually update the versioning.
After the left pad debacle npm no longer allows you to delete prior versions of libraries so if you had used a versioned dependency nothing would break.
While I mostly agree with you, I do believe the NPM ecosystem makes it particularly hard to stay on top of things, given how dependency graphs explode there. It's a cultural thing IMO.
What's wrong with the Log4j maintainer? The library had a vulnerability and they patched it. Free software is explicitly provided without warranty, so they didn't do anything wrong unless I missed something.
Correct me if I'm wrong but unlike other ecosystems like Github and NPM (where all contributors work more or less individually), the Apache Software Foundation (ASF) works as one unit. Since Log4J is maintained and published as an Apache software, they must have some responsibility too?
All open source software comes with no warranty. It doesn't matter where you get it.
Also, the Apache Foundation provides a legal structure but the projects themselves are run by the volunteers that maintain them. It's not really different from other open source projects in that way.
Pretty much all software comes with (close to) no warranty. Just pick a random EULA and search for “warrant[y]” and “liab[ility]” ;)
What NPM Should Do Today To Stop A New Colors Attack Tomorrow (Russ Cox)
I don't think the suggestion is really tenable on a npm-wide level. Who exactly is doing these tests? In most cases in open source, it'd be the maintainers... but they're also the threat model, in this case.
Hundreds of new packages are uploaded every day to NPM, and tens of thousands get updates at least. From where in the name of god are we getting third party... volunteers?... to review all these updates?
Yeah, you can have that level of scrutiny when deploying things at Google, that's because Google's engineers are paid, there is a clear hierarchical organization and people know their responsibilities.
The closest thing that would be reasonable is like a community run "stable" packages collection, of common and highly used packages. This special artifact repository is designed to only update its artifacts when community members review updates. Only by heavily limiting the scope is it anywhere near reasonable.
And it's also expected that when you get packages from the wild wild west, you either do versioning or prepare for the consequences of your convenience.
Russ Cox is basically describing how Go's new package manager works and suggesting that NPM could do something similar. (He's biased since he designed the new package manager. Of course he thinks his own ideas are good!)
Here's the idea: Let's say you write an application that depends on package A, which depends on package B. When the maintainer of package A does a new release, they test it using a particular version of B (possibly upgrading it), and the version they used is recorded in the module file included in the release.
Now when you sync to new version of A then you also get the version of B that A's maintainer tested against. So A's maintainer's release testing (however much or little they did) lets you know that A and B work together, which is a partial defense against B doing a bad release. You only get relatively stable releases of indirect dependencies, since someone else has tested them besides the original author.
(This doesn't quite work for diamond dependencies, because they will do releases at different times, possibly testing against different versions of B. Then the rule is to choose the newest version of B that one of your dependencies tested.)
You could think of this as a form of social proof, where new versions gradually get proven good by downstream dependencies publishing their version numbers. It does slow down updates, which might be a problem in the case of a security fix. However, sometimes security fixes are bad too, so it's probably better that they get tested?
This is a core point, and one I think really needs more discussion as a way forward. I've discussed this in a couple of other forums since the colors/faker incident broke, but we need to embrace something like https://github.com/crev-dev/crev across ecosystems. IMO such efforts could scale if companies got involved with their audits, specifically if we begin with packages that fit certain parameters like downloaded X times per week, low number of maintainers (bus factor / Nebraska problem highlighted in that one XKCD we see for all dependency issues) and so on.
I'm curious about how much audit actually happens version-to-version of dependencies.
Wait wait wait. This is entirely in the power of the users of the package manager to manage within their own projects. Yes, you can say "just install any version", but you can also pin a version in your project's
package.json. To suggest "npm is designed badly" or "what npm should change is ___" is not accurately describing the current state and design of npm (nor other package managers like Ruby's rubygem system, to which my statements apply just as well).But many dependencies are indirect and reviewing everything yourself often isn't feasible. Maybe a community of library maintainers should have some orderly process for collectively deciding whether new versions of a library are good? That's basically what he's proposing. See my other reply for more about it.