jsfbp that doesn't use fibers?

Fibers are neat, but the majority of node.js developers don't and won't use them, and they don't work in browsers at all. It would be nice if we could provide a "classical FBP" system that works using standard JavaScript APIs.
Actually, after reading http://www.jpaulmorrison.com/fbp/noflo.html I'm pretty confused about how JSFBP is considered more "classical" than NoFlo. That page suggests the key difference is that NoFlow is single-threaded (because JavaScript is single-threaded) and thus not suitable for CPU-intensive programs, but then admits JSFBP is also single-threaded. So is the main difference actually the synchronous vs asynchronous style of programming?
Selective reading of ports and backpressure can both be accomplished in an asynchronous style. For example, Node-style Streams have pause and resume methods for handling backpressure. The port could buffer up to a fixed number of IPs then call pause on the stream when the buffer fills up. The port could have a read method that takes a callback (or returns a promise) which gets called when the next IP is available, and calls resume on the stream if the buffer is no longer.
To take advantage of multiple cores you could use something like WebWorkers, though I think a generic system based on a standard IPC mechanism (e.x. ZeroMQ) and serialization protocol (e.x. Protocol Buffers) would be even cooler, allowing you to write components in any language, or even distribute processes across machines.
But anyway, there are several options for sync and async styles in JavaScript:
Asynchronous style:

EventEmitters or Streams (see above)
raw callbacks: probably Node-style callback(err, result) callbacks. This is my least favorite option, but is still the most popular style in Node.js. I probably wouldn't bother with this.
promises: promises are gaining a lot of traction in the JavaScript world lately, but they're still a very asynchronous style

Synchronous style:

generators (ES6, can be transpiled to ES5): not available in browsers except via transpiling
await (ES7, can be transpiled to ES5): lets you write synchronous style code that awaits on promises
some other synchronous-style transpiler: e.x. streamline.js

Also this may be useful relevant reading:  https://github.com/kriskowal/gtor
EDIT: task.js is another possibility: http://taskjs.org/

Also, I would argue that this is incorrect: "JSFBP only requires the addition of a small package on top of node.js, node-fibers, which is easy to install. Even though it does not use pure JavaScript, neither does node.js, but the latter is becoming more and more accepted." (http://www.jpaulmorrison.com/fbp/software.html#JSFBP)
A full Node.js environment could be emulated in the browser (and indeed "browserify" attempts to do this), but node-fibers can't be easily emulated in the browser because it introduces "blocking" semantics (something similar can be implemented once ES6 generators are available, though: https://stackoverflow.com/questions/18293563/can-node-fibers-be-implemented-using-es6-generators)

Honestly I can't see how an API that looks like JavaFBP and other classical implementations could be made with traditional Javascript. If you can find a way please tell.
This kind of api:
while( (a = read("in")) !== null ){
   send( "out", a.value + 5 )
}

Blocking is not a problem but a feature, and I'm not really convinced that generators can emulate this behavior. I hope it does though, I simply couldn't find a reasonable way.

Here's a simple example (using the Bluebird promise library and ES6 generators, available in iojs and recent versions of node with the "--harmony" flag)
(obviously read and send are fake versions of the real thing, but it should give you an idea of what's possible)
var Promise = require("bluebird");

var messages = [1, 2, 3];
function read(port) {
  return Promise.delay(1000).then(function() {
    if (messages.length > 0) {
      return { value: messages.shift() }
    } else {
      return null;
    }
  });
}

function send(port, value) {
  console.log("sending to port '" + port + "': " + value)
}

add5 = Promise.coroutine(function *() {
  var a;
  while( (a = yield read("in")) !== null ){
    send( "out", a.value + 5 )
  }
});

add5()

Hi @tlrobinson, I wrote  http://www.jpaulmorrison.com/fbp/noflo.html
before I got the idea of developing JSFBP, so it's not surprising that it
doesn't reflect my thinking about JSFBP.  At the time I didn't think it was
possible to build a "classical" FBP system using JS, and NoFlo is certainly
very different from classical FBP.    It seems to have a definite niche,
but it's a different one from that of classical FBP.
As you may be aware, I have been working with classical FBP for 40+ years,
and much of that time it was being used to run most of the batch code for a
major bank (using a green thread "classical" FBP implementation).  So I
believe I understand it quite well, and I have a lot of faith in it.
Around Christmas, my son suggested I look at return and yield, and, while
researching that, I came across a reference to Marcel Laverdet's
node-fibers in @bjouhier's excellent blog, Bruno's Ramblings  -
https://bjouhier.wordpress.com/ .  In it he compares a number of JS
approaches to getting rid of "callback hell", and the thing that stood out
to me about node-fibers is that it supports multiple stacks, which in my
experience is key to building a "classical" FBP implementation.
I did not see another approach that was as promising, and in fact, using
node-fibers, it took me no more than a few days to come up with a very
promising proof of concept, which I have been developing since.  Your
feedback and @comfreek's have been very useful as my JavaScript skills are
pretty rudimentary!
I agree that node-fibers is a bit further away from plain JS than nodejs,
say, but to me it still looks like the most promising technology for taking
advantage of JS.  We have discussed worker threads, and the consensus is
that the overhead is too high and communication between worker threads too
"narrow-band".   I have also been warned to avoid Promises!
If people are willing to add nodejs to basic Javascript, I fail to see why
they would be so unwilling to add one more  component - namely
node-fibers.  You raise a good point about not being able to browserify
node-fibers, but maybe there are other solutions to getting JSFBP in the
browser - however, I see no immediate urgency to solve this problem.  I
will be quite happy if JSFBP can supports servers, providing good
performance and good maintainability.  This at least gives users a single
language on both client and server - even if different mindsets!
We have used back pressure, selective receive, and information packets with
defined lifetimes and ownership as sort of a litmus test for classical FBP

if you can provide these functions with the tools you mention, and decent
performance and maintainability, I would be very interested in seeing your
solutions.  A number of people have come up with FBP-like approaches over
the last few decades, and I'm looking forward to seeing how these all work
out, as we gain a deeper understanding of this technology.   It might be
timely to try to come up with a checklist of success factors, so we can
measure the different offerings against it.  I can tell you one thing,
though: once you have made the paradigm shift described in my book, you
can't go back! And I would venture to say that, if you can go back, you
haven't made the paradigm shift!  :-)  That's 40+ years of experience
talking!

Best regards, and many thanks,
Paul M.
On Feb 11, 2015 5:40 PM, "Tom Robinson"<EMAIL_ADDRESS>wrote:

Fibers are neat, but the majority of node.js developers don't and won't
use them, and they don't work in browsers at all. It would be nice if we
could provide a "classical FBP" system that works using standard JavaScript
APIs.
Actually, after reading http://www.jpaulmorrison.com/fbp/noflo.html I'm
pretty confused about how JSFBP is considered more "classical" than NoFlo.
That page suggests the key difference is that NoFlow is single-threaded
(because JavaScript is single-threaded) and thus not suitable for
CPU-intensive programs, but then admits JSFBP is also single-threaded. So
is the main difference actually the synchronous vs asynchronous style of
programming?
Selective reading of ports and backpressure can both be accomplished in an
asynchronous style. For example, Node-style Streams have pause and resume
methods for handling backpressure. The port could buffer up to a fixed
number of IPs then call pause on the stream when the buffer fills up. The
port could have a read method that takes a callback (or returns a
promise) which gets called when the next IP is available, and calls resume
on the stream if the buffer is no longer.
To take advantage of multiple cores you could use something like
WebWorkers, though I think a generic system based on a standard IPC
mechanism (e.x. ZeroMQ) and serialization protocol (e.x. Protocol Buffers)
would be even cooler, allowing you to write components in any language, or
even distribute processes across machines.
But anyway, there are several options for sync and async styles in
JavaScript:
Asynchronous style:

EventEmitters or Streams (see above)
raw callbacks: probably Node-style callback(err, result) callbacks.
This is my least favorite option, but is still the most popular style in
Node.js. I probably wouldn't bother with this.
promises: promises are gaining a lot of traction in the JavaScript
world lately, but they're still a very asynchronous style

Synchronous style:

generators (ES6, can be transpiled to ES5): not available in
browsers except via transpiling
await (ES7, can be transpiled to ES5): lets you write synchronous
style code that awaits on promises
some other synchronous-style transpiler: e.x. streamline.js

Also this may be useful relevant reading:
https://github.com/kriskowal/gtor
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14.

That's why I suggested to use an abstraction layer in issue #13. Such an abstraction layer would ideally allow us to switch between different implementations, e.g. one for Node.js and another one for browsers. Another big and related advantage is that the implementation for browsers could only emulate Fibers, so that jsfbp is ready to run in browsers even though not with full performance.

If people are willing to add nodejs to basic Javascript, I fail to see why
they would be so unwilling to add one more  component - namely
node-fibers

This is simply not going to happen. AFAIK there has been zero serious discussion of adding fibers to the browser, and even if that started happening it would be many years before they were implemented and widely available.
Compare that to ES6's generators, which are built into current versions of Node.js, in the next version of JavaScript, so likely to start showing up in browsers soon. As you can see from my example above, you can get you something pretty similar to fibers. It's possible there's some reason why you can't implement classical FBP with generators instead of fibers, but I don't know yet (note that Bruno also mentioned generators and preprocessors like Streamline.js as a solution to callback hell)

Could you try making a prototype that achieves a functionality equivalent
to Collate ?
2015-02-12 13:26 GMT-03:00 Tom Robinson<EMAIL_ADDRESS>
If people are willing to add nodejs to basic Javascript, I fail to see why
they would be so unwilling to add one more component - namely
node-fibers
This is simply not going to happen. AFAIK there has been zero serious
discussion of adding fibers to the browser, and even if that started
happening it would be many years before they were implemented and widely
available.
Compare that to ES6's generators, which are built into current versions of
Node.js, in the next version of JavaScript, so likely to start showing up
in browsers soon. As you can see from my example above, you can get you
something pretty similar to fibers. It's possible there's some reason why
you can't implement classical FBP with generators instead of fibers, but I
don't know yet (note that Bruno also mentioned generators and preprocessors
like Streamline.js as a solution to callback hell)
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-74101216.

If you can find a way to simulate multiple stacks in JS, then we should be
able to do something!
On Feb 12, 2015 11:59 AM, "alfa256"<EMAIL_ADDRESS>wrote:

Could you try making a prototype that achieves a functionality equivalent
to Collate ?
2015-02-12 13:26 GMT-03:00 Tom Robinson<EMAIL_ADDRESS>
If people are willing to add nodejs to basic Javascript, I fail to see
why
they would be so unwilling to add one more component - namely
node-fibers
This is simply not going to happen. AFAIK there has been zero serious
discussion of adding fibers to the browser, and even if that started
happening it would be many years before they were implemented and widely
available.
Compare that to ES6's generators, which are built into current versions
of
Node.js, in the next version of JavaScript, so likely to start showing up
in browsers soon. As you can see from my example above, you can get you
something pretty similar to fibers. It's possible there's some reason why
you can't implement classical FBP with generators instead of fibers, but
I
don't know yet (note that Bruno also mentioned generators and
preprocessors
like Streamline.js as a solution to callback hell)
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-74101216.

—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-74108180.

Can Collate be written in the current version of JSFBP (using fibers)?

Absolutely!  The structure and even coding would be very similar to https://github.com/jpaulm/jsfbp/blob/master/components/rrmerge.js - you just have to add the "low tag" logic (output the IP with the lowest tag value, and then do a 'receive' from the input array port element that that IP came from).
Historically, Collate also adds open and close brackets to the output stream - this function could be packaged as a separate component, but it uses the same control fields as Collate, so it seemed practical to combine the two functions.

In my quest to understand collate I implemented a version without brackets here: https://github.com/jpaulm/jsfbp/pull/17
I'll work on proof-of-concept with generators/promises/streams next.

@jpaulm @alfa256 Here's a proof-of-concept using Streams, Promises, and ES6 Generators via Blubird's coroutine method: https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400
It may be difficult to understand if you're not very familiar with JavaScript and "promises" or "streams", but the relevant thing to note is I've implemented collate without using Fibers. Note that collate1 uses Generator functions, which will be available in ECMAScript 6, and I believe can be supported in older versions using a "transpiler". collate2 only uses Promises and should run in all browsers today, without any transpiling.
collate1 is a line-by-line port of "collate" that I implemented in JSFBP (https://github.com/jpaulm/jsfbp/blob/master/components/collate.js), the only changes are the addition of the yield statement in front of asynchronous operations (i.e. receive) and the "function *" syntax to make it a "generator function"
collate2 is admittedly less clear because it's written in an asynchronous style using Promises directly, but it proves "collate" can be implemented in JavaScript without fibers or even generators: https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400#file-collate-js-L98-L131
Note that I'm using Streams in "paused mode" (http://nodejs.org/api/stream.html)

Readable streams have two "modes": a flowing mode and a paused mode. When in flowing mode, data is read from the underlying system and provided to your program as fast as possible. In paused mode, you must explicitly call stream.read() to get chunks of data out. Streams start out in paused mode.

I've added a receive method to readable streams which returns a promise. You can use the promise directly (as I do in collate2) or yield it within a Bluebird coroutine if you prefer a more synchronous style (as I do in collate1).
I think basing a FBP system on Streams would be powerful because you can implement a component in whichever style makes the most sense. From the outside it doesn't matter. For example, copier could be implemented as InputPort.openInputPort('IN').pipe(InputPort.openInputPort('OUT'));
Streams also support "backpressure" so they can communicate to their upstream processes when they do or don't want more data: http://nodejs.org/api/stream.html#stream_event_drain
Also note that I'm using streams in "object mode" http://nodejs.org/api/stream.html#stream_object_mode but normal mode might be interesting in some cases.

So from what I can see the api changes would be adding a yield to
receive()s ?
If that's the case then It'd be great, also I wonder if the yield could be
even wrapped in the receive() ?
In any case great job, this could be great for the project.
2015-02-17 16:47 GMT-03:00 Tom Robinson<EMAIL_ADDRESS>
@jpaulm https://github.com/jpaulm @alfa256 https://github.com/alfa256
Here's a proof-of-concept using Streams, Promises, and ES6 Generators via
Blubird's coroutine method:
https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400
It may be difficult to understand if you're not very familiar with
JavaScript and "promises" or "streams", but the relevant thing to note is
I've implemented collate without using Fibers. Note that collate1 uses
Generator functions, which will be available in ECMAScript 6, and I believe
can be supported in older versions using a "transpiler". collate2 only
uses Promises and should run in all browsers today, without any transpiling.
collate1
https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400#file-collate-js-L62-L101
is a line-by-line port of "collate" that I implemented in JSFBP (
https://github.com/jpaulm/jsfbp/blob/master/components/collate.js), the
only changes are the addition of the yield statement in front of
asynchronous operations (i.e. receive) and the "function *" syntax to
make it a "generator function"
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/function*
collate2
https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400#file-collate-js-L104-L151
is admittedly less clear because it's written in an asynchronous style
using Promises directly, but it proves "collate" can be implemented in
JavaScript without fibers or even generators:
https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400#file-collate-js-L98-L131
Note that I'm using Streams in "paused mode" (
http://nodejs.org/api/stream.html)
Readable streams have two "modes": a flowing mode and a paused mode. When
in flowing mode, data is read from the underlying system and provided to
your program as fast as possible. In paused mode, you must explicitly call
stream.read() to get chunks of data out. Streams start out in paused mode.
I've added a receive
https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400#file-collate-js-L32-L48
method to readable streams which returns a promise. You can use the promise
directly (as I do in collate2) or yield it within a Bluebird coroutine if
you prefer a more synchronous style (as I do in collate1).
I think basing a FBP system on Streams would be powerful because you can
implement a component in whichever style makes the most sense. From the
outside it doesn't matter. For example, copier could be implemented as
InputPort.openInputPort('IN').pipe(InputPort.openInputPort('OUT'));
Streams also support "backpressure" so they can communicate to their
upstream processes when they do or don't want more data:
http://nodejs.org/api/stream.html#stream_event_drain
Also note that I'm using streams in "object mode"
http://nodejs.org/api/stream.html#stream_object_mode but normal mode
might be interesting in some cases.
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-74738438.

@alfa256 I don't believe yield can be moved inside of receive because it's a special keyword for use in generator functions (function* whatever(){}), but I don't think that's desirable anyway. Ideally the FBP kernel/framework should work without generator functions, but would support them as an extra layer of syntactic sugar in components.
Note that the only code in my proof of concept that's aware of generator functions or yield is the component itself. I think it should stay that way, it's up for debate.

Hi! Interested bystander here. I just finished reading the fbp book, and this jsfbp project excites me.
For what it's worth, I would like to second the motion for using streams. Streams have a large "following" (node, gulp, etc) and are relatively well understood even by pedestrian js developers like myself.
I share @tlrobinson's concern that node-fibers are off-putting. Streams, on the other hand, I can rally behind. Plus they are available in the browser today with a choice of packages, and hopefully native support soon.
Of course, there may exist very good reasons against using streams here, but my current (admittedly limited) understanding does not see any.
https://highlandjs.org/
https://github.com/substack/stream-handbook
https://streams.spec.whatwg.org

The proof of concepts look promising!
Note that Node.js 12.0 doesn't support generator functions (unless compiled with --harmony), so I switched to io.js in order to run your sample collate.js. It is essentially a Node.js fork with faster release cycles.
I have finished my changes which separate all Network-related thing from those related to (Fiber-)Runtime.
I would be interested in whether streams or promises can be easily integrated, given the new separated Network/Runtime architecture. A Runtime (stored in core/runtimes/[name]/index.js) is a class which must implement these functions:

pushToQueue(process): Pushes a process to the Queue
run(processes, options, callback)
setCallbackPending(callback)
getCurrentProc()

Admittedly, the last two methods seem to still heavily rely on Fibers. I will look into their use cases and investigate.
I have read that Promise implementations might tend to create memory leaks (just search for "Promises memory leak" or "Promises garbage collection"). This is especially related to unresolved promises and/or errors. There is a very long discussion titled Promises/A+ implementations can lead to resource leaks in Node.

I've expanded my proof-of-concept into a fully functional prototype: https://github.com/tlrobinson/sbp
Like the PoC, it uses Streams, as well as a combination of Promises, Generator functions, and Bluebird's coroutine (or Q's async) method to achieve a synchronous style similar to existing cFBP (classical FBP) implementations, including jsfbp.
I'd be curious to hear from the experts whether they'd consider this true "classical-FBP", and if not, why not?

@jpaulm @alfa256 @ComFreek @brodavi @kenhkan Any thoughts on this?

Collate1 looks interesting - it seems strange that you can't "hide" the
yield, but I guess we can just hold our noses!
What I am curious about is whether you ran a complete test case, e.g.
fbptest01 with only the two changes you mention wrt collate1, and if so how
did it perform relative to the version on JSFBP (
https://github.com/jpaulm/jsfbp/ ).
Collate2 looks much more alien to me - it even seems to have a recursive
loop in it - or did I miscount the brackets?!
@brodavi  You said,  " I share @tlrobinson's concern that node-fibers are
off-putting" .  From a component or network builder's point of view, the
only sign of fibers is one "require" - I don't think that should be
"off-putting". The lack of support for fibers in browsers is apparently
much more of a concern, but this will probably be remedied before long if
and when fibers catches on!
Regards,
Paul M.
@jpaulm https://github.com/jpaulm @alfa256 https://github.com/alfa256
Here's a proof-of-concept using Streams, Promises, and ES6 Generators via
Blubird's coroutine method:
https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400
It may be difficult to understand if you're not very familiar with
JavaScript and "promises" or "streams", but the relevant thing to note is
I've implemented collate without using Fibers. Note that collate1 uses
Generator functions, which will be available in ECMAScript 6, and I believe
can be supported in older versions using a "transpiler". collate2 only uses
Promises and should run in all browsers today, without any transpiling.
collate1
https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400#file-collate-js-L62-L101
is a line-by-line port of "collate" that I implemented in JSFBP (
https://github.com/jpaulm/jsfbp/blob/master/components/collate.js), the
only changes are the addition of the yield statement in front of
asynchronous operations (i.e. receive) and the "function *" syntax to make
it a "generator function"
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/function*
collate2
https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400#file-collate-js-L104-L151
is admittedly less clear because it's written in an asynchronous style
using Promises directly, but it proves "collate" can be implemented in
JavaScript without fibers or even generators:
https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400#file-collate-js-L98-L131
Note that I'm using Streams in "paused mode" (
http://nodejs.org/api/stream.html)

Readable streams have two "modes": a flowing mode and a paused mode. When
in flowing mode, data is read from the underlying system and provided to
your program as fast as possible. In paused mode, you must explicitly call
stream.read() to get chunks of data out. Streams start out in paused mode.

I've added a receive
https://gist.github.com/tlrobinson/c8407e0e3f31a0da5400#file-collate-js-L32-L48
method to readable streams which returns a promise. You can use the promise
directly (as I do in collate2) or yield it within a Bluebird coroutine if
you prefer a more synchronous style (as I do in collate1).
I think basing a FBP system on Streams would be powerful because you can
implement a component in whichever style makes the most sense. From the
outside it doesn't matter. For example, copier could be implemented as
InputPort.openInputPort('IN').pipe(InputPort.openInputPort('OUT'));
Streams also support "backpressure" so they can communicate to their
upstream processes when they do or don't want more data:
http://nodejs.org/api/stream.html#stream_event_drain
Also note that I'm using streams in "object mode"
http://nodejs.org/api/stream.html#stream_object_mode but normal mode might
be interesting in some cases.
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-74738438.

The lack of support for fibers in browsers is apparently
much more of a concern, but this will probably be remedied before long if
and when fibers catches on!

@jpaulm It's highly unlikely fibers will be added to browsers. If you don't believe me try asking a few other JavaScript experts.

@tlrobinson  And Streams and Promises are better than Fibers... how?!
Because some JavaScript guru has blessed them...?  Just curious!

@jpaulm The main thing is with fibers you can't tell if a function call is going to block execution, whereas with generators you explicitly yield within a generator function (that's why the yield can't be hidden) Here's a description of the difference between generators and fibers: https://gist.github.com/creationix/6416266
But the important thing is generators are already in ES6 (the next version of JavaScript), and fibers would be mostly redundant, so they're unlikely to ever make it into the JavaScript spec and therefore browsers.

I just wrote the following to Comfreek:

send suspends when the connection is full, and receive suspends when the
connection is empty.  I don't see this as a leaky abstraction - the
application designer does not even think in these terms - she just sees
processes running asynchronously,  with data chunks flowing across
unidirectional connections - with back pressure.

That's why I keep saying it's a paradigm shift!

I don't want to know whether  a function call is going to block execution

it should not affect the logic!  Go back and read Dijkstra on sequential
processes!  It's not rocket science!
On Mar 7, 2015 7:32 PM, "Tom Robinson"<EMAIL_ADDRESS>wrote:

@jpaulm https://github.com/jpaulm The main thing is with fibers you
can't tell if a function call is going to block execution, whereas with
generators you explicitly yield within a generator function (that's why the
yield can't be hidden) Here's a description of the difference between
generators and fibers: https://gist.github.com/creationix/6416266
But the important thing is generators are already in ES6 (the next
version of JavaScript), and fibers would be mostly redundant, so they're
unlikely to ever make it into the JavaScript spec and therefore browsers.
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-77716351.

In JavaScript it has always been safe to assume a function will run to completion before another event fires which could change the state that the function is using. Fibers break that assumption, which is unacceptable to a lot of people, including the people in charge of defining future versions of JavaScript.
Generators also break that assumption, but you have to explicitly opt-in by declaring a generator function (with the "function*" syntax) and use the "yield" keyword, so there's no chance of someone else's code deep in some library you're using yielding execution without your knowledge.
BTW this article expands on the gist I linked to in my last comment.
We can argue about which way is better, but it's not going to change the fact that you're unlikely to ever get fibers in the browser.

This discussion is interesting,  but I feel it should be on the Group,  so
it gets a wider audience.   Could one of you, @tlrobinson or @comfreek
start a topic on the Group,  maybe with the last few posts. ..?
On Mar 8, 2015 6:01 AM, "Tom Robinson"<EMAIL_ADDRESS>wrote:

In JavaScript it has always been safe to assume a function will run to
completion before another event fires which could change the state that the
function is using. Fibers break that assumption, which is unacceptable to a
lot of people, including the people in charge of defining future versions
of JavaScript.

"Change the state that the function is using."
That's really the whole point of FBP : two functions cannot use the same
"state ".   It just seems odd to raise that objection in the context of
FBP!  Maybe an education problem?!

It seems to me that the appropriate place for much of this discussion is
the JSFBP topic on the Group.  Could we move at least the important posts
there?  Thanks.
PS I would do it, but it would be very difficult using my little tablet!
:-)
On Mar 8, 2015 8:21 AM, "Paul Morrison"<EMAIL_ADDRESS>wrote:

This discussion is interesting,  but I feel it should be on the Group,  so
it gets a wider audience.   Could one of you, @tlrobinson or @comfreek
start a topic on the Group,  maybe with the last few posts. ..?
On Mar 8, 2015 6:01 AM, "Tom Robinson"<EMAIL_ADDRESS>wrote:

In JavaScript it has always been safe to assume a function will run to
completion before another event fires which could change the state that the
function is using. Fibers break that assumption, which is unacceptable to a
lot of people, including the people in charge of defining future versions
of JavaScript.

"Change the state that the function is using."
That's really the whole point of FBP : two functions cannot use the same
"state ".   It just seems odd to raise that objection in the context of
FBP!  Maybe an education problem?!

Hi ComFreek,
Nice to hear from you - and nice to be back!
I am starting to look at JSFBP's derivatives - your streams-impl, and Tom's sbp (streampunk?).   BTW Will you be updating your fork's readme to include the instructions you sent me?  I think it would be useful.
What I find so strange is that these (to my way of thinking) contorted forms are said to seem more natural to a JS user (at least you and Tom seem to be saying this) than the nice simple syntax of the current JSFBP version (much improved by you, I admit!).  In the former case, the user has to learn and get comfortable with Promises, Streams, etc., plus FBP, while in the case of basic JSFBP they just have to learn FBP!
I also don't see why 'yield' and those asterisks have to be visible - surely making them visible means that it could be meaningful to drop them - what happens if one or both of these are dropped by mistake?
Basically, it seems to come down to the fact that you and Tom don't believe that the JS community will accept node-fibers as readily as they have Promises etc.  Surely that's just a matter of building a constituency of people who are more interested in getting the job done than in playing with abstruse concepts which probably don't have a well-defined use case anyway!  Marcel @laverdet, are you getting any feedback on this?
ComFreek and Tom, I think I know where you stand on the issue of node-fibers vs. Promises, etc. - Ken, Alex, Ged, @bjouhier, @jeanhuguesrobert, @laverdet,  I would be interested in hearing your views.  I am double posting this note here, as I don't have email addresses for some of the people I would appreciate feedback from.
FYI The repositories I am looking at are:
https://github.com/jpaulm/jsfbp
https://github.com/ComFreek/jsfbp/tree/streams-impl
https://github.com/tlrobinson/streampunk.js
Best regards to all,
Paul M.

Correct, my position is that it is very unlikely that fibers have a future
in the JavaScript language standard (ECMAScript), and they can't even be
reasonably shimmed into existing versions of JavaScript via "transpiling"
(unlike generators or async/await). Therefore fibers will never work in
browser JavaScript (the largest deployed programming environment, by far),
and should not be used as the foundation of a general purpose programming
framework if there is a suitable alternative.
On Sun, Mar 29, 2015 at 12:01 PM jpaulm<EMAIL_ADDRESS>wrote:

Hi ComFreek,
Nice to hear from you - and nice to be back!
I am starting to look at JSFBP's derivatives - your streams-impl, and
Tom's sbp (streampunk?). BTW Will you be updating your fork's readme to
include the instructions you sent me? I think it would be useful.
What I find so strange is that these (to my way of thinking) contorted
forms are said to seem more natural to a JS user (at least you and Tom
seem to be saying this) than the nice simple syntax of the current JSFBP
version (much improved by you, I admit!). In the former case, the user has
to learn and get comfortable with Promises, Streams, etc., plus FBP,
while in the case of basic JSFBP they just have to learn FBP!
I also don't see why 'yield' and those asterisks have to be visible -
surely making them visible means that it could be meaningful to drop them -
what happens if one or both of these are dropped by mistake?
Basically, it seems to come down to the fact that you and Tom don't
believe that the JS community will accept node-fibers as readily as they
have Promises etc. Surely that's just a matter of building a constituency
of people who are more interested in getting the job done than in playing
with abstruse concepts which probably don't have a well-defined use case
anyway! Marcel @laverdet https://github.com/laverdet, are you getting
any feedback on this?
ComFreek and Tom, I think I know where you stand on the issue of
node-fibers vs. Promises, etc. - Ken, Alex, Ged, @bjouhier
https://github.com/bjouhier, @jeanhuguesrobert
https://github.com/jeanhuguesrobert, @laverdet
https://github.com/laverdet, I would be interested in hearing your
views. I am double posting this note here, as I don't have email addresses
for some of the people I would appreciate feedback from.
FYI The repositories I am looking at are:
https://github.com/jpaulm/jsfbp
https://github.com/ComFreek/jsfbp/tree/streams-impl
https://github.com/tlrobinson/streampunk.js
Best regards to all,
Paul M.
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-87455434.

Hi Paul,
The lack of threads, actually the lack of "continuations", aka callcc(), is definitely an issue that puts JavaScript in the unfortunate situation where C was before threads where introduced, a while ago, 25 years ago I guess.
I tend to think that the current situation will prevail for the next 5 years at least, forever maybe. The reasons for that are beyond my understanding (and too long to explain) but they are strong.
This is Gravity.
Consequently the solution is not to rely on threads.
Which means either A/ async everywhere or B/ transpiling somehow.
For obvious performance reasons, A is probably the solution for anything "core". For less critical code or "user land", some B might work but I would rather not be in position to select which B solution to use, better let that decision to the application developer.
If I were you I would build low level efficient abstractions to get my job done and I would provide "bridges" to interface with callback(err,..)/Promise/Generator/Worker/ServiceWorker/WhatEverNext application code.

@jpaulm Hi Paul,
I've been monitoring the fibers/promises/generators/async-await debates for the last 5 years and I concur with the general opinion that fibers, despite all its qualities, is very unlikely to get a very wide acceptance.
Fibers had a window of opportunity in 2011-1013 and it managed to gather some interesting momentum, especially with meteor. There were lots of passionate debates about it at the time, and many node.js top guns advocated strongly against it. At the time, fibers did not have many contenders but the landscape is changing: generators and promises are now baked into V8 (ES6) and async/await is lined up for ES7. Once JS gets promises + async/await, the main pain points of async programming (callback hell) will be addressed by vanilla JS and fibers will be harder to sell.
From a language design standpoint, there is a fundamental difference between fibers and the mechanisms that are being added to ES: fibers provide true (deep) continuations while generators and async/await only provide shallow (single frame) continuations. The shallow model forces all the yielding points to be explicit, which is an advantage when reasoning about concurrency issues (you don't have to dive into sub-functions to find out where a function may yield). The designers of the language are obviously strongly attached to this model and I don't foresee this changing any time soon.

Here is an example of 4800 'green' threads running in the browser via clojurescript and core.async. The Core.Async talk linked there is quite interesting: http://programming-enchiladas.destructuring-bind.org/rm-hull/8262502

Thanks, Forrest, maybe Clojure is the way to go!  Interesting that he talks
about addressing "callback hell" as well - this seems to be the most common
criticism of JavaScript!
On Fri, Apr 3, 2015 at 11:08 AM, Forrest Oliphant<EMAIL_ADDRESS>wrote:

Here is an example of 4800 'green' threads running in the browser via
clojurescript and core.async. The Core.Async talk linked there is quite
interesting:
http://programming-enchiladas.destructuring-bind.org/rm-hull/8262502
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-89314837.

Merci, Jean-Hugues et Bruno!
I assume all these mechanisms (Promises, Streams, etc.) are essentially green-thread-based.  Given that more and more devices have multiple cores, perhaps green thread solutions are no longer as relevant as they were...?  What approaches are there to having JavaScript play with multiple cores?
Also, Bruno, you say, "The shallow model forces all the yielding points to be explicit, which is an advantage when reasoning about concurrency issues (you don't have to dive into sub-functions to find out where a function may yield). "  From the point of view of FBP, that's exactly what you don't want!  A function can only yield on a suspended send or a suspended receive. Period.  All data is either in an IP or in a function's local (stack) storage.  So no fancy reasoning involved!
I'm curious how Promises work with stacks - I assume a Promise uses the next available space in the stack but has access to data above it in the stack.  It sounds like Clojure has figured out how to manage Promises with multiple stacks...

Promises and streams don't use green threads. They are usually (*) implemented as vanilla JS libraries. Promise libraries can be a bit hairy because the Promise specs impose a rich API but you can capture the essence of a promise (a future aka thunk) in a few lines of JS. I blogged about it a while ago (https://bjouhier.wordpress.com/2011/04/04/currying-the-callback-or-the-essence-of-futures/).
(*) promises are now part of the language itself (ES6) so they can be implemented in C and provided directly by the JS engine. But this does not change the threading model.
With promises, all JS code executes on the main stack. The promise (or future) encapsulates a callback and memorizes the result or error when the callback completes.
Streams are similar; they execute on the main stack and they encapsulate various events (node.js style streams) or a simple callback (my ez-streams library).
Generators go one step further. They still execute on the main stack but they capture one stack frame when the generator is suspended (with yield). That stack frame is pushed again on the stack when the generator is resumed (with run). So there is still only one thread, no green threads. The single frame continuation rule imposes that you yield only at top level in your generator function; you cannot yield from a subfunction. This way there is only one frame to save to capture the state of the generator.
Fibers are different. Each fiber has its own stack and yield may be called at any depth in a fiber's stack. Fibers are green threads.
Node uses more than one thread but all JS code is executed by a single thread with an event loop. C++ add-ons may use additional threads to execute operations while the JS thread is running but they always report their completion by posting a callback to the main loop which is run by the main JS thread.
There had been attempts to provide true (preemptive) JS threading in node.js. Threads-a-gogo was an early attempt. This has been revived recently by workers in io.js: https://github.com/iojs/io.js/pull/1159.
The model behind workers is different from the model behind threads in Java or .NET: each worker runs in its own isolate. You can pass messages between the main JS thread and worker threads but you cannot share mutable state between threads. Worker threads don't block; they perform their I/O asynchnously and they each have their own event loop to handle their I/O callbacks. This share nothing model is, IMO, much easier to work with than classical threading with shared state and monitors.

One more word on the shallow model and FBP: of course you would be on the safe side with FBP because the yielding points are imposed and hidden. You are also on the safe side if you use Marcel's futures library with the right discipline (the yielding point are wait() calls that surface at all levels). But the JS language designers have to cater for every situation, not just FBP or a specific futures library. This is why they are so attached to the single frame continuation rule.
Maybe you should take a look at Go. Green threads may be pariahs in JS but they are first-class citizens in Go; they are called goroutines.

Hi @bjouhier, thanks for the info on Promises - that helps quite a bit!  I
can of course see the *single frame continuation rule - *in fact, I have
long suspected that one could build a simpler FBP engine by restricting
sends and receives to the top level of a process.  I might play with that!  I
am not really keen, though, on leaking implementation restrictions into the
mental model - but I can see that it might be expedient!
You might be interested in looking at Vladimir Sibirov's Goflow -
https://github.com/trustmaster/goflow .
You mentioned Threads à gogo - doesn't that also require a native module -
which I thought was one of the problems with JSFBP being dependent on
Marcel's node-fibers...?    I suppose you could run
JavaScript on one or more Boost threads...?
Regards,
Paul
On Sat, Apr 4, 2015 at 4:59 PM, Bruno Jouhier<EMAIL_ADDRESS>wrote:

One more word on the shallow model and FBP: of course you would be on the
safe side with FBP because the yielding points are imposed and hidden. You
are also on the safe side if you use Marcel's futures library with the
right discipline (the yielding point are wait() calls that surface at all
levels). But the JS language designers have to cater for every situation,
not just FBP or a specific futures library. This is why they are so
attached to the single frame continuation rule.
Maybe you should take a look at Go. Green threads may be pariahs in JS but
they are first-class citizens in Go; they are called goroutines.
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-89666204.

Threads-a-go-go (and specifically this fork:
https://github.com/audreyt/node-webworker-threads ) provides an API similar
to "WebWorkers" which are in most browsers.
Unfortunately node-webworker-threads seemed very unstable last time I tried
it (a few weeks ago)
As an aside, I've got a toy FBP implementation in ~100 lines of Go here:
https://github.com/tlrobinson/streampunk.go
On Sat, Apr 4, 2015 at 4:18 PM jpaulm<EMAIL_ADDRESS>wrote:

Hi @bjouhier, thanks for the info on Promises - that helps quite a bit! I
can of course see the *single frame continuation rule - *in fact, I have
long suspected that one could build a simpler FBP engine by restricting
sends and receives to the top level of a process. I might play with that! I
am not really keen, though, on leaking implementation restrictions into the
mental model - but I can see that it might be expedient!
You might be interested in looking at Vladimir Sibirov's Goflow -
https://github.com/trustmaster/goflow .
You mentioned Threads à gogo - doesn't that also require a native module -
which I thought was one of the problems with JSFBP being dependent on
Marcel's node-fibers...?  I suppose you could run
JavaScript on one or more Boost threads...?
Regards,
Paul
On Sat, Apr 4, 2015 at 4:59 PM, Bruno Jouhier<EMAIL_ADDRESS>wrote:

One more word on the shallow model and FBP: of course you would be on the
safe side with FBP because the yielding points are imposed and hidden.
You
are also on the safe side if you use Marcel's futures library with the
right discipline (the yielding point are wait() calls that surface at all
levels). But the JS language designers have to cater for every situation,
not just FBP or a specific futures library. This is why they are so
attached to the single frame continuation rule.
Maybe you should take a look at Go. Green threads may be pariahs in JS
but
they are first-class citizens in Go; they are called goroutines.
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-89666204.

—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-89682442.

Node-webworker-threads looks interesting!  However I think I'm missing something important - https://github.com/audreyt/node-webworker-threads/blob/master/README.md - Why Threads (4) says "The threads of a process share exactly the same address space, that of the process they belong to. Every thread can access every memory address within the process' address space.", but @bjouhier above said  "You can pass messages between the main JS thread and worker threads but you cannot share mutable state between threads."  Are we talking about different entities?  Or is immutability enforced somehow?   I doubt that it applies to JS code run by node-webworker-threads. With V8 if you try to have two threads that run in the same isolate and that both modify a shared JS object, you will just get a crash. The V8 engine is not designed for that.

How would the machine know to crash?!  Von Neumann machines aren't usually smart enough...
@audreyt would you care to comment?  I read the passage about threads in https://github.com/audreyt/node-webworker-threads/blob/master/README.md as being very specifically about worker threads...?  TIA

What I meant is that you will get an assertion error or a SEGV or another kind of error and your process will terminate. Simply because you are calling an API from more than one thread and that API is not designed for that.

Ah, I see!  Thanks, @bjouhier!  Back to square one!
On Mon, Apr 6, 2015 at 2:33 AM, Bruno Jouhier<EMAIL_ADDRESS>wrote:

What I meant is that you will get an assertion error or a SEGV or another
kind of error and your process will terminate. Simply because you are
calling an API from more than one thread and that API is not designed for
that.
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-89945006.

@jpaulm
The more I re-read some of the comments above, the more I share the conclusion that a FBP implementation allowing its graph to span across different networks, operating systems and programming languages would definitely be superior to any FBP implementation restricted to a single programming language. As you can see, JSFBP will not be able to use real threads in near future as JS (and V8 as the runtime engine for Node.js) are simply not designed for that use case.
So, I'd like to throw in an idea, which has probably already been discussed in the FBP group or elsewhere:

Create a FBP library providing a client and server interface for receiving and sending packages, respectively, to the outer world. The connection shall be treated as general as possible, thereby allowing a wide range of connection types, starting with simple memory buffers storing the IPs and ranging up to mobile data connections as used in current smart phones.

@ComFreek https://github.com/the-grid/msgflo

I've also wondered if it would be possible to do this with a FBP wrapper around celery and its canvas primitives.

node-webworker-threads works on Node 0.10.x and is known to be unstable with 0.12.x and io.js, which I hope to fix in the coming weeks.
It is true that node-webworker-threads cannot share mutable structure across worker threads and with the main thread; the same principle applies to WebWorker in browsers, too.

@comfreek As always, my questions would be: what language(s) and what operating system(s)?  Given the existence of languages which don't play together too well, such as Java and C#, sockets would seem a promising approach...  JavaScript seems to be able to talk to C[++], so this suggests that it might be interesting to work with Boost...  What did you have in mind?

fwiw I want to echo what @tlrobinson & @bjouhier said about fibers never making it into browsers or a standard. I started working on fibers in early 2011 when early versions of generators were still being discussed for standardization. We all knew yield was coming to JS and that it would likely be single frame, but we didn't know the exact API or syntax. And it definitely wouldn't be ready soon. I wanted yield right then and there and full continuations were the easiest way to build it.

@jonnor Thanks for the link, the readme seems very elaborate! Is there any quickstart guide available? I would like to see it in action (e.g. having component A running on my PC and component B on my notebook, both connected to my router).
@jpaulm As it is the very idea of the "FBP library", I would target some a wide range of operating systems and programming languages by employing low-level languages, such as C and C++, and common network protocols.
I'll have a look at celery, thanks for the link @chadrik.

@ComFreek we're using it production a couple of places, but still have not landed on final API - so usage docs are not the priority yet. Will probably tag & announce first stable in a couple of weeks, when we've migrated everything in the production pipeline to it. For now, see: https://github.com/the-grid/msgflo/blob/master/spec/fixtures/participants.coffee https://github.com/jonnor/imgflo-server/blob/master/src/worker.coffee

Merci, @laverdet!  I think it is becoming obvious that what we really need
is software that takes advantage of multiple cores.  AFAIK none of the
planned extensions of JS provide this.   As @comfreek says, it would be
even better if it allows multiple languages and infrastructures to
communicate in a coordinated way.  Does msgflo (or any other systems that
people are aware of) address these requirements?
On Apr 8, 2015 1:49 PM, "Marcel Laverdet"<EMAIL_ADDRESS>wrote:

fwiw I want to echo what @tlrobinson https://github.com/tlrobinson &
@bjouhier https://github.com/bjouhier said about fibers never making it
into browsers or a standard. I started working on fibers in early 2011 when
early versions of generators were still being discussed for
standardization. We all knew yield was coming to JS and that it would
likely be single frame, but we didn't know the exact API or syntax. And it
definitely wouldn't be ready soon. I wanted yield right then and there
and full continuations were the easiest way to build it.
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-90986679.

Cluster and workers both take advantage of cores

Do you have a link for clusters?  Couldn't find anything on Google!
On Apr 9, 2015 3:24 PM, "Marcel Laverdet"<EMAIL_ADDRESS>wrote:

Cluster and workers both take advantage of cores
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-91333576.

First result in Google for "node cluster"

@jpaulm I encourage you to also take a look at https://github.com/iojs/io.js/pull/1159. This is work in progress on io.js but it looks very promising. A pure JS implementation based on a worker library which is blessed by io.js (and hopefully bundled with it some day soon) could be a winner.

Thanks @bjouhier - sounds interesting!
On Apr 9, 2015 4:39 PM, "Bruno Jouhier"<EMAIL_ADDRESS>wrote:

@jpaulm https://github.com/jpaulm I encourage you to also take a look
at iojs/io.js#1159 https://github.com/iojs/io.js/pull/1159. This is
work in progress on io.js but it looks very promising. A pure JS
implementation based on a worker library which is blessed by io.js (and
hopefully bundled with it some day soon) could be a winner.
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-91363571.

Here is a quote from http://www.jpaulmorrison.com/fbp/noflo.html :

... basic FBP business functions such as "Collate" require a process to be specific about which port it wants to receive from, and to be able to suspend until data arrives at that port ... A related architectural concept in FBP ... is what we call "back pressure", where an upstream process will be suspended if the connection it feeds into becomes full. Finally, ... the concept of information packet (IP) "lifetimes", by which an IP is tracked from creation to destruction and can only be "owned" by a single process at a time, or be in transit between processes.

How would these be approached in a fully isolated environment such as that described in https://github.com/iojs/io.js/pull/1159 ?

The same approaches you would use to distribute FBP processes across OS
processes or hosts, right?
On Sat, Apr 11, 2015 at 1:45 PM jpaulm<EMAIL_ADDRESS>wrote:

Here is a quote from http://www.jpaulmorrison.com/fbp/noflo.html :
... basic FBP business functions such as "Collate" require a process to be
specific about which port it wants to receive from, and to be able to
suspend until data arrives at that port ... A related architectural concept
in FBP ... is what we call "back pressure", where an upstream process will
be suspended if the connection it feeds into becomes full. Finally, ... the
concept of information packet (IP) "lifetimes", by which an IP is tracked
from creation to destruction and can only be "owned" by a single process at
a time, or be in transit between processes.
How would these be approached in a fully isolated environment such as that
described in iojs/io.js#1159 https://github.com/iojs/io.js/pull/1159 ?
—
Reply to this email directly or view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-91923133.

I asked first!  :-)  And no, I don't know how to do this across OS processes or hosts either.  IIRC Matt Lai has something he calls the "traffic cop" which can manage things like back pressure, but I'm not sure how that generalizes across multiple OS's.

Matt Lai gave a pretty complete description of his approach using Tcl/Tk in his March 19, 2014, post in https://groups.google.com/forum/#!searchin/flow-based-programming/appkata/flow-based-programming/hnst0E6AcUI/6al-uVAKL4oJ .

Go back and read Dijkstra on sequential
processes!  It's not rocket science!

Do you mean Hoare? http://spinroot.com/courses/summer/Papers/hoare_1978.pdf

No, Bradley, it is Dijkstra! See
http://www.cs.utexas.edu/users/EWD/ewd01xx/EWD123.PDF
I can't read it on my tablet but Google found it!  I believe this
predates Hoare's CSP.
Regards,
Paul

I have just posted a web page about the Concat (Concatenate) component in "classical" FBP and in NoFlo - http://www.jpaulmorrison.com/fbp/concat.html .  Comments would be appreciated.
Note: we are not sure if the cited Concat.coffee program is up to date.

What's the consensus on using Node Fibers? My experience with JavaScript and web app's, or hybrid mobile i.e. Cordova/PhoneGap, React Native... which all use JavaScript is that 70-90% of the app's code is client side. So for me JSFBP needs to run in the browser?
Does NoFlo run in the browser? What do they do differently from JSFBP to make that happen?

Electron utilize chromium multi process architecture http://dev.chromium.org/developers/design-documents/multi-process-architecture by rpc, can this be helpful to use multiple processes? http://electron.atom.io/blog/2016/07/28/electron-internals-node-integration

@tlrobinson Thanks for the update. Maybe async generator functions are also of interest for you:
// inside a "async function *myFunc(...) {"
while ((ip = await array[i].receive()) !== null) {
  yield ip;
}

One disadvantage is that you definitely give up your "handle of computation", whereas with the send method, you are free to await it or to perform other actions and then, let's say, collectively await all sent IPs.
Incidentally, I've recently written an AsyncLimitedQueue implementation with async functions and generators in TypeScript, where pushing can block after a certain amount of elements (aka. backpressure): https://github.com/ComFreek/async-playground/blob/21d4f9ec73db732f8fc4dfc54616aeae490cc700/queue/AsyncLimitedQueue.ts#L167

Hi Tom, this certainly does seem much closer to the spirit of classical
FBP!  I guess one has to specify "await", but it feels very "classical"!  I
assume JavaScript still can't handle multiple cores though - do you know if
this is likely to change any time soon...?
Regards,
Paul
On Fri, Apr 13, 2018 at 2:43 PM, Tom Robinson<EMAIL_ADDRESS>wrote:

Not sure if anyone is still interested in JavaScript implementations of
FBP, but I dusted off my streampunk.js
https://github.com/tlrobinson/streampunk.js project and updated it to
use async/await, which is included in ES2017 and supported natively in
node.js and most browsers. AFAIK fibers haven't gained any traction in the
last few years.
I also implemented the concat component: https://github.com/tlrobinson/
streampunk.js/blob/master/components/concat.js As you can see it's about
the same length as jsfbp's concat.
streampunk implements selective receive and back pressure:
https://github.com/tlrobinson/streampunk.js/blob/master/
test/backpressure.js
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jpaulm/jsfbp/issues/14#issuecomment-381226724, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AATGJ8Lah3N90tsCqKafJGNoB1zArqU6ks5toPG5gaJpZM4DfOIf
.

I have been reviewing this very interesting correspondence, and I see that there was a long hiatus after @giantelk 's 2015 query (which apparently wasn't answered), until @emptist 's suggestion in 2017, and @tlrobinson 's update a few days ago.  I imagine quite a lot must have changed in the JS world since 2015!
Also, since that time I have refined the locking logic in JavaFBP and C#FBP... and it turned out to be a bit more complex than I had thought (I think those two implementations are working now - anyone want to check them?!), so this logic would have to be replicated in a truly asynchronous, multi-core implementation using JS.
@giantelk, I confess I didn't quite understand your query, but my guess is that client side doesn't really need multiple cores, although single-core FBP would improve maintainability, so jsfbp could apply here (or I guess @tlrobinson 's latest, since it seems more viable than jsfbp), but server side would definitely benefit from multi-core support - so why not use JavaFBP or C#FBP...  which would in addition provide better performance?!  And we wouldn't have to twist JS into a pretzel!
@tlrobinson are you considering providing tutorial material for streampunk...?  Or a successor...?
Regards,
Paul

@tlrobinson  I looked at the WHATWG link you gave and I think I saw "chunks".  Do I get the impression that Streams has reinvented Objects?!  Or did I misunderstand?  I can't imagine why JS dropped Objects - unless JS just followed a different evolutionary path from something like C?

@tlrobinson  Thanks for the feedback on streampunk, etc.  Please keep us posted on how this area of JS evolves, and to what extent it is converging with classical FBP.
Regards,  Paul
FYI @laverdet @ComFreek @bjouhier


Allow setting an api_key for st2chatops instead of user/pass

Whenever I setup chatops, I prefer giving chatops an api_key instead of a username/password combo.
This PR allows us to add a previously generated st2 api_key as st2chatops.api_key in values.yaml (make sure to add the api_key object in st2.apikeys as well). Defining this api_key triggers replacing the ST2_AUTH_USERNAME and ST2_AUTH_PASSWORD environment vars with ST2_API_KEY but only for the st2chatops deployment.

OK. I think I found a clean enough way to make st2chatops use an api_key. What do you think?


(blender) Migrated to AU

Added update.ps1
Updated chocolateyInstall.ps1
Updated chocolateyUninstall.ps1 to use 'chocolatey-uninstall.extension'
Updated license url
Added project source url
Added bug tracker url
Added mailing list url
Updated release notes url
Added chocolatey-uninstall.extension dependency

NOTE: latest version (2.78) is in moderation, and needs to be updated.

@gep13
I vote that we move all those different extensions that are used the most to core. This one is perfect example.

Can you recheck, this doesn't look OK
C:\Users\appveyor\AppData\Local\Temp\1\chocolatey\blender\2.78\blender-2.78a-windows32.msi is of content type text/plain; charset=UTF-8
C:\Users\appveyor\AppData\Local\Temp\1\chocolatey\blender\2.78\blender-2.78a-windows64.msi is of content type text/plain; charset=UTF-8
   blender is updated to 2.78 and pushed

@majkinetor I believe this is just because the server returned an incorrect content type.
I tried to find a workaround for it, but couldn't. It still installs correctly and runs as it should.
I've seen these warnings before on other packages, which also installed correctly.

OK, thx. Just put a note on it next time so that I don't trip out.

C:\WINDOWS\system32> cinst blender --version 2.78 -y
Chocolatey v0.10.3
Installing the following packages:
blender
By installing you accept licenses for the packages.

blender v2.78
blender package files install completed. Performing other installation s
Downloading blender 64 bit
  from 'http://ftp.nluug.nl/pub/graphics/blender/release/Blender2.78/ble
WARNING: D:\Cache\Chocolatey\blender\2.78\blender-2.78a-windows64.msi is
Progress: 100% - Completed download of D:\Cache\Chocolatey\blender\2.78\
Download of blender-2.78a-windows64.msi (84.01 MB) completed.
Hashes match.
Installing blender...
blender has been installed.
 The install of blender was successful.
  Software installed to 'C:\Program Files\Blender Foundation\Blender\'

@majkinetor ok, I'll remember to that the next time.

Heh, manual cinst you should really try new Test-Package with my fork of chocolatey-test-environment if you want to be xtra efficient.

@majkinetor doesn't that only work with local nupkg files?
I had already removed the nupkg file, so it was easier to just use cinst this time.

No it doesn't, it now works with everything, it works with install, uninstall, parameters, everything you might need. And its way faster, I changed a lot of things.


[Bug] Tooltip trigger background color does not update when switching from dark to light theme

Version
1.6.3
Link to Minimal Reproduction
https://www.visactor.io/vchart/demo/theme/theme-style
Steps to Reproduce

Be sure not to trigger the tooltip, select the dark option, then mouseover to trigger the tooltip
Switch to the light option, notice the tooltip retains the dark theme color

https://github.com/VisActor/VChart/assets/42089082/6caf065b-d5d1-44e7-a786-f47ee9623218
Current Behavior
Tooltip trigger color will remains the last theme color when change the theme.
Expected Behavior
When change the theme, the tooltip color should be the theme color.
Environment
- OS: Windows 11
- Browser: Edge 119.0.2151.58
- Framework: Website exmamples

Any additional comments?
No response

1.8.0 已解决


ControllerManager 'TagChanged' event anomaly

When monitoring a UDT with BOOL members, the first member's value unexpectedly changes if there is a subsequent BOOL member in the UDT.
Current Behavior

Value of the first member of the UDT unexpectedly changes. Could create unpredictable behavior of the control system.
Expected Behavior

Monitor UDT's with BOOL members without effecting the first member.
Possible Solution (Optional)

Context

This use case only needs to read values, never write.
Steps to Reproduce (for bugs only)

Create a UDT (UDT_1) in the PLC with two members. The first as a DINT, the second a BOOL.
Run the code below and monitor the value of the DINT member (RSLogix).
Change the value of the DINT and it will change back to something else (depending on the value of the BOOL).
Change the value of the BOOL and the DINT value will unexpectedly change.

const {ControllerManager} = require('st-ethernet-ip')

const cm = new ControllerManager();

//addController(ipAddress, slot = 0, rpi = 100, connected = true, retrySP = 3000, opts = {})
const cont = cm.addController('<IP_ADDRESS>');

cont.connect();

//addTag(tagname, program = null, arrayDims = 0, arraySize = 0x01)
cont.addTag('UDT_1')

cont.on('TagChanged', (tag, prevValue) => {
  console.log(tag.name, ' changed from ', prevValue, ' => ', tag.value)
})

cont.on('error', (e) => {
  console.log(e)
})

Your Environment

Package version (Use npm list - e.g. 1.0.6): 2.5.0
Node Version (Use node --version - e.g. 9.8.0): 16.14.2
Operating System and version: OSX 10.15.7
Controller Type (eg 1756-L83E/B): 1769-L24ER-QB1B CompactLogix 5370
Controller Firmware (eg 30.11): 30.011

I was able to reproduce your error.  I will need to dig into this.

Found the issue and have a fix.  Testing it out and should be pushed out by tonight.

Thank you. Also see something similar with the writeTag() function.

WriteTag function should be fixed too.  The code that converted the JSON tag value object from BOOL to buffer data was completely wrong. 2c30c15ff67b49af1f23b57b0454d87031cc1e8c

Working as expected now. Thank you!
In my sample code above, are there any writes being performed behind the scenes by the ControllerManager object? The fact that tag values changed without commanding a write makes me nervous.

So what happened was that the tag structure object was incorrectly converted to buffer data. When the Nodejs program saw that the buffer data didn't match the buffer data being read from the PLC, the program 'thought' that there was updated data that needed to be written to the PLC controller.


Adding LGPE move names

Info taken from Bulbapedia:
https://bulbapedia.bulbagarden.net/wiki/List_of_moves_in_other_languages

Hi @AndreArrebola,
I noted that on Bulbapedia there are moves up to ID 796. I'm just wondering why you stopped at 742

Hi! Due to time constraints I only could add the LGPE new moves yesterday. I might add the Sword/Shield moves when I have time, though!
Do you think this can be merged already and then I add them in a future request, or do you prefer to wait until all moves are added?

Did you add them by hand?
I thought you used a script 👍
Then no worries at all!

Yes, I did it by hand! it's how I can help for now
I'm using this as an opportunity to learn pull requests with GitHub
So, just to confirm, this will be merged and then when I(or other contributor) are able to contribute with the SwSh movenames this will be added in another pull request, right?

Hi, I was thinking to merge your branch in staging to check if it deploys correctly, then I'll merge it in master.

Feel free to push to this branch up until it has been merged.

Hello, during the day I adapted the stats script to fetch the move languages and informations from bulbapedia using the API. It worked out very well, good job on making the original one!
I also added a way to generate a csv out of it, but it only works right with the move languages. This happens because the move table uses IDs for infos like types.
Do you think making a json associating these infos to their IDs(and then replacing them as they're fetched) would work?
For example, if it gets "Normal" from Bulbapedia, the script could search for it in the json file and get the respective ID
(I have job tomorrow so I might not be able to work on it before thursday!)

Hi, I've seen that you generated a new newmoves.json file. That's great! And no need to add automatically all these new moves to the moves.csv file, we can copy and paste them. Anyway, I've seen that your CSV is a bit different from the original one:
Original moves.csv
id,identifier,generation_id,type_id,power,pp,accuracy,priority,target_id,damage_class_id,effect_id,effect_chance,contest_type_id,contest_effect_id,super_contest_effect_id
728,clangorous-soulblaze,7,16,185,1,,0,11,3,419,100,,,

Your newmoves.csv looks like below:
729,VII,Electric,Zippy_Zap,50,15,100,+2,anyadjacent,Physical

But it should be:  (as you did manually)
729,zippy-zap,7,13,50,15,100,2,10,2,289,,,,

So basically we need the very same format. If you need to get the IDs of some resources you can look them up in pokeapi.co.

Forgot to change the order, I'll do it as soon as I have time, be it today or tomorrow!

Thanks a lot, and please take your time. Don't rush :P

Hello again! I tweaked the script so it generated the csv in the proper format :) I added the type/generation/damage type ids in json files since they are pretty much constant
I still didn't add the new lines to the csv, I just want you to take a look on it first. The results look promising though! I might add a way to generate the flags too

Excellent @AndreArrebola, thanks so much! Feel free to update all the official CSV files in /pokedex/data/csv/ so we can merge. You can add the new lines manually, no need to have the script doing it.

Commited the changes! Sorry for the wait.


shim/rollkit: downgrade go-da to 0.2.0

the version that was introduced in #260
(997046d926170d0bcf5e2824d5670c0a5fb85a2f) was using
the latest version of go-da (0.4.0).
However, the latest version of rollkit on which the gm
demo was based off, was using 0.2.0. That was causing
occasional problems.
So I went ahead and downgraded it. Now, it seems to
workfine (with exception of #259).
Closes #258.

#270
#269  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @pepyakin and the rest of your teammates on  Graphite


Error Message: An error occured during the install. You can now close this window.

Already have miniconda installed.
When I click the Install(recommended) button, I got the following message which is really unfriendly:
Downloading, installing and setting up a Python environment for TorchStudio.
This may take up to 15 minutes depending on your download speed, and up to 16GB.

Downloading Python installer (Miniconda3)...
Download complete: /home/peng/TorchStudio/python-miniconda3.sh

Installing Python in /home/peng/TorchStudio/python...
An error occured during the install. You can now close this window.

Could you elaborate more in the message?

@tribbloid  Indeed that doesn't say a lot, error logging will be more detailed in the next update.
Meanwhile you can do the following:
go to /home/peng/TorchStudio and delete the python folder (if any).
Then type the following in a terminal:
/home/peng/TorchStudio/python-miniconda3.sh -b -f -p /home/peng/TorchStudio/python
Please let me know how it goes. If the command above succeed, you can just relaunch TorchStudio and proceed with the rest of the install.

No it doesn't, something else was wrong:
$ sudo ~/TorchStudio/python-miniconda3.sh -b -f -p /home/peng/TorchStudio/python
sudo: /home/peng/TorchStudio/python-miniconda3.sh: command not found

One extra thing I discovered was that my OS (Pop OS 21.10 LTS) hasn't upgrade to CUDA 11.3. So it recommends torch 1.8 with CUDA 11.1 at the moment. This is below the dependency requirement of torchstudio

sorry I forgot to add bash before:
sudo bash ~/TorchStudio/python-miniconda3.sh -b -f -p /home/peng/TorchStudio/python
But indeed PyTorch 1.8 won't meet the minimum requirements for TorchStudio, PyTorch 1.9 minimum is required.

command successful but still didn't solve the problem

Well I suppose the problem at this point is more Pop OS not supporting PyTorch 1.9 ?

Noooo, I de-selected "Local GPU support" option, it should download PyTorch 1.9 with CPU/MKL support only.
But it doesn't, and very little is logged

Please copy paste the content of ~/TorchStudio/pythoninstall.log here.

@tribbloid Please install and download TorchStudio 0.9.6 and try again. If it fail, please upload the log files located in ~/TorchStudio/logs


Make subsetter.Options non-global

I admit that I am not very familiar with Python idioms, but the following truly surprised me:
>>> from fontTools.subset import Options
>>> x = Options()
>>> 'dlig' in x.layout_features
False
>>> y = Options()
>>> 'dlig' in y.layout_features
False
>>> x.layout_features.append('dlig')
>>> 'dlig' in y.layout_features
True

I would expect x and y to be independent instances of Options.

That's definitely was not intended.  I'll fix it up.

Thanks!


Fix/external links

Description:
Fix external links
Related issue:
#66

@domsbytes sure, no problem :)


[feature request] support server dart application

DESCRIPTION
it would be great to be able to use this package in backend,
but at the time of writing rHTTP depends on flutter
so it can't be used
as far as I know flutter_rust_bridge is "flutter" only by name
and can be used in pure dart implementation
PROPOSAL
support the creation of dart server based on rHTTP by provide a "pure dart" implementation

Yes, it seems to be possible according to https://cjycode.com/flutter_rust_bridge/guides/miscellaneous/pure-dart
I am wondering if it still works on Flutter because https://github.com/flutter/flutter/issues/129757 is not resolved and https://github.com/dart-lang/sdk/issues/50565 is only available via experimental flag.
If we are lucky, then adding a build.yaml file (https://github.com/fzyzcjy/flutter_rust_bridge/blob/master/frb_example/pure_dart/build.dart) is enough to enable experimental Dart support.

I tried to add build.dart into the project but flutter_rust_bridge_utils is not published on pub.dev yet.
Ref: https://github.com/fzyzcjy/flutter_rust_bridge/issues/2274


Support showing Discord profile on about page

Is your feature request related to a problem? Please describe.
I use Discord a lot for loose professional mingling. It would be ideal for me to link my Discord profile on my 'about' page, similar to how many other social media links are already integrated.
Describe the solution you'd like
I would like some quick pointers on how I can set up my about page to show my Discord link, preferably with the Discord icon.

I figured out how to do this by reading the conversation here: https://github.com/alshedivat/al-folio/issues/313
We just need to add a flag in _config.yaml and add a corresponding line in _includes/social.html.


COCO performance

Has anyone trained and tested this ssd on coco dataset? I got a MAP=0.213 which is 0.02 less than the original paper (0.232). So I wonder what's you guys performance? Thanks!

Has anyone trained and tested this ssd on coco dataset? I got a MAP=0.213 which is 0.02 less than the original paper (0.232). So I wonder what's you guys performance? Thanks!

Hi，Could you provide the code to generate the result file.Thanks a lot! My email is<EMAIL_ADDRESS>

Has anyone trained and tested this ssd on coco dataset? I got a MAP=0.213 which is 0.02 less than the original paper (0.232). So I wonder what's you guys performance? Thanks!

Hello, Ziyi Wu, Happy New Year! I got a map = 0.04, I think it may be that I use the wrong code to generate the result file. Could you send me the code to generate the result file? This question has puzzled me for a long time. Thank you very much!

Has anyone trained and tested this ssd on coco dataset? I got a MAP=0.213 which is 0.02 less than the original paper (0.232). So I wonder what's you guys performance? Thanks!

Hello, Ziyi Wu, Happy New Year! I got a map = 0.04, I think it may be that I use the wrong code to generate the result file. Could you send me the code to generate the result file? This question has puzzled me for a long time. Thank you very much!My email is<EMAIL_ADDRESS>

@lesonly Hi! Sorry for replying late. Actually this repo doesn't contain the code for testing on COCO as you can see. I adopt the code from another SSD repo (https://github.com/lufficc/SSD) and modified it to suit the input and output. I don't think you can use my code directly after I send it to you... Really sorry about that.
Honestly speaking, I recommend you to just shift to that repo if you really want a COCO result. That repo is also well written, and contains all the coco-evaluation codes you need. When I modified the code from this repo to be suitable for coco testing, it just took me a lot of time to debug.

Thank you for your enthusiasm reply!I am watching the  SSD repo (https://github.com/lufficc/SSD),but i can't find the code which to generate the result file.
------------------ 原始邮件 ------------------
发件人: "Ziyi<EMAIL_ADDRESS>发送时间: 2020年1月1日(星期三) 晚上7:48
收件人<EMAIL_ADDRESS>抄送: "朱"<23006592<EMAIL_ADDRESS>主题: Re: [amdegroot/ssd.pytorch] COCO performance (#422)
@lesonly Hi! Sorry for replying late. Actually this repo doesn't contain the code for testing on COCO as you can see. I adopt the code from another SSD repo (https://github.com/lufficc/SSD) and modified it to suit the input and output. I don't think you can use my code directly after I send it to you... Really sorry about that.
Honestly speaking, I recommend you to just shift to that repo if you really want a COCO result. That repo is also well written, and contains all the coco-evaluation codes you need. When I modified the code from this repo to be suitable for coco testing, it just took me a lot of time to debug.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

@lesonly Hi! Sorry for replying late. Actually this repo doesn't contain the code for testing on COCO as you can see. I adopt the code from another SSD repo (https://github.com/lufficc/SSD) and modified it to suit the input and output. I don't think you can use my code directly after I send it to you... Really sorry about that.
Honestly speaking, I recommend you to just shift to that repo if you really want a COCO result. That repo is also well written, and contains all the coco-evaluation codes you need. When I modified the code from this repo to be suitable for coco testing, it just took me a lot of time to debug.

Thank you for your enthusiasm reply!I am watching the SSD repo (https://github.com/lufficc/SSD),but i can't find the code which to generate the result file.

It's actually in SSD/ssd/data/datasets/evaluation/coco/init.py

It's actually in SSD/ssd/data/datasets/evaluation/coco/init.py

Thank you  for your warm reply and help again. I will study it carefully

@Wuziyi616 Hi, I

@Wuziyi616 Hi, I trained this ssd on COCO train2017 and tested the model on val2017，and I got the mAP 0.240，which is a little higher than the original paper (0.232). My result is as follow:
Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.240
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.411
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.248
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.059
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.259
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.405
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.235
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.355
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.384
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.131
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.425
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.588

@Wuziyi616 Hi, I trained this ssd on COCO train2017 and tested the model on val2017，and I got the mAP 0.240，which is a little higher than the original paper (0.232). My result is as follow:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.240
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.411
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.248
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.059
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.259
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.405
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.235
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.355
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.384
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.131
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.425
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.588

You're great!Could you send me the code to generate the result file? This question has puzzled me for a long time. Thank you very much!My email is<EMAIL_ADDRESS>

@Wuziyi616 Hi, I trained this ssd on COCO train2017 and tested the model on val2017，and I got the mAP 0.240，which is a little higher than the original paper (0.232). My result is as follow:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.240
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.411
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.248
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.059
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.259
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.405
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.235
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.355
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.384
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.131
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.425
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.588

@JiHaoqina You're great! I got a map = 21.8, which is same to Wuziyi616.I think it may be that I use the wrong code to generate the result file. Could you send me the code to generate the result file? or exchange it with each other. Thank you very much!My email is<EMAIL_ADDRESS>

can anyone share the  evaluation on coco dataset please

@Wuziyi616 Hi, I trained this ssd on COCO train2017 and tested the model on val2017，and I got the mAP 0.240，which is a little higher than the original paper (0.232). My result is as follow:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.240
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.411
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.248
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.059
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.259
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.405
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.235
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.355
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.384
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.131
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.425
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.588

can you please share your evaluated code since only training is available so it could be really helpful to share your testing code
thank you

Hello everyone! I've overcome this issue recently and here is the evaluation code and coco dataset class I used for this. There are two things I want to notice you:

The code is used for COCO 2017! So please download the 2017 Train, Val, Train/Val Annotations from http://cocodataset.org/#download. COCO 2014 is not supported by my dataset

If you want to train with the dataset provided in coco2017.py, just use:
train_set = COCODetection(root='path/to/data', image_sets=[('2017', 'train')], preproc=your_data_preprocessing_method, target_transform=COCOAnnotationTransform)

Hope this will help you!

Hello everyone! I've overcome this issue recently and here is the evaluation code and coco dataset class I used for this. There are two things I want to notice you:

The code is used for COCO 2017! So please download the 2017 Train, Val, Train/Val Annotations from http://cocodataset.org/#download. COCO 2014 is not supported by my dataset
If you want to train with the dataset provided in coco2017.py, just use:
train_set = COCODetection(root='path/to/data', image_sets=[('2017', 'train')], preproc=your_data_preprocessing_method, target_transform=COCOAnnotationTransform)

Hope this will help you!

Hello everyone! I've overcome this issue recently and here is the evaluation code and coco dataset class I used for this. There are two things I want to notice you:

The code is used for COCO 2017! So please download the 2017 Train, Val, Train/Val Annotations from http://cocodataset.org/#download. COCO 2014 is not supported by my dataset
If you want to train with the dataset provided in coco2017.py, just use:
train_set = COCODetection(root='path/to/data', image_sets=[('2017', 'train')], preproc=your_data_preprocessing_method, target_transform=COCOAnnotationTransform)

Hope this will help you!

can i modify your code to fit in COCO 2014 val dataset?


Wrong converting of md to html code

So, here is original .md file http://paste.ubuntu.com/25163892/
Here is code result : http://paste.ubuntu.com/25163897/
Using Fico flat cms, which uses parsedown to convers md to html as class. I thouht, maybe this is Pico issue, and changed parsedown.php class from github. But nothing changed.
What is going on?
Thank you.

I think this is a bug in the CMS.
Try putting the text into http://parsedown.org/, if it renders correctly then the problem is unlikely with Parsedown. Looks like the markdown is being processed correctly and then html encoded by something afterward going by those links provided.

It's very unlikely that this is an issue with Pico 😉 The most likely cause is an issue with @refeaime's custom theme. @refeaime, join us on #picocms on Freenode so that we can solve your problem.

I shall take your word for it! That does sound reasonable though, I'd be very surprised if this was happening in the default install 😉

thanks guys! =)


Create staircase.py

Right-aligned staircase with '#' and spaces.
Height = width = 'n'

This is not a Algorithm


Validator suggestions

I just published a dataset and the resulting DCAT is valid :+1:. The validator does give some warnings/suggestions though:

I'll list them here. Some of those could be implemented, others might be valid but not be recognized by the validator.
Catalog

Language: could be set to English
Themes: we do have dcat:themeTaxonomy and skos:ConceptScheme, but that is apparently not recognized
License: we have dct:rights, but should probably use dct:license?
Homepage: should we add one?

Dataset

Language: an IPT resource has a metadata language property. Would be useful to use.
Contact point: we have adms:contactPoint. Not sure why it is not recognized.

A foaf:homepage for Catalog is actually mentioned in the mapping documentation.

foaf:homepage described in separate issue: #48

Catalog

I'll just put english
This is an error in the validator
I changed it to license, this makes more sense

Dataset

I'll set the laguage
This is fixed

Great! I almost can't find any more questions/bugs/enhancements that I haven't reported already. :smile

Catalog
2. Will you report the error to the DCAT validator team?
Dataset

Make sure to take the metadata language (there is also a data language, but that doesn't apply here).

Reported and fixed the validator mistake
Do I need to use Eml#Language or Eml#MetaLanguage?

Eml#MetaLanguage

All warnings are fixed

Tested in v1.2: all warnings are now gone, except:

Validator still does not recognize theme (a validator issue)
Validator now returns an error: The property: contactPoint can't have multiple objects (reported in #54)

This issue can remain closed.

Did you open issues in the validator repo? If not, would you find the time to do so and reference them here?

They are aware of it and we test it with the "local validator" and every thing is fine. No more warnings or errors

@Amenai what's the "local validator"?

The not online version of the validator with all bug fixed. They will push it online in a second on "www.dcat.be/validator"


metronome class

Created metronome class wrapper similar to marathon

thanks @kensipe. Can we share methods with the Marathon class? At least the RpcClient?
This code is not being used right now. Should be integrated with the job main class: cli/dcoscli/job/main.py

@tamarrow tests locally... ready to go (after CI tests)

@tamarrow looks like CCM failed... otherwise good

rebased

@tamarrow rebased

@tamarrow rebased again, reverted toml file (sorry about that) and pep8 and integration checked.

thanks @kensipe! Will merge once tests pass


Java error on launch: Unsupported major.minor version 51.0

Been using Dbeaver extensively for several months and loving it, then had to wipe my machine clean and start over for reasons unrelated. After reinstalling Dbeaver, I now face the following error:
See the log file /Applications/Dbeaver.app/Contents/Eclipse/configuration/XXXXXXXXXX.log
Opening said log file tells me:
!SESSION Tue Apr 03 12:02:52 GMT-06:00 2018 ------------------------------------
!ENTRY org.eclipse.equinox.launcher 4 0 2018-04-03 12:02:52.828
!MESSAGE Exception launching the Eclipse Platform:
!STACK
java.lang.UnsupportedClassVersionError: org/eclipse/core/runtime/adaptor/EclipseStarter : Unsupported major.minor version 51.0
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClassCond(ClassLoader.java:637)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
	at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:650)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:590)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1499)

I have installed legacy Java from here: https://support.apple.com/kb/DL1572, as suggested during Dbeaver install.
Also installed the latest version of Java from here just in case: https://java.com/en/download/
Working on a 2017 MacBook Pro w/ High Sierra 10.13.4.
Any thoughts? Thanks!

Same problem here! What is meant by "using PKG installer". I downloaded JRE and just double clicked on .dmg and then on the .pkg ... Still got same issue..

Instead of downloading and installing java from java's website, go to Dbeaver's download page and grab the PKG installer + JRE from there instead.
https://dbeaver.jkiss.org/download/

Works! Thanks!

Alternatively, brew cask install java will provide an up-to-date java version on mac.


introduced constant multiple read only safe switch flag

Constant Multiple Readonly safety switch
NOTE to test readonly flag CM feature flag should also be on
Changes 👷♀
-introduced constantMultipleReadOnly flag
How to test 🧪

Both constant multiple and constant multiple read only off user shouldn't see any CM features|

all CM and BS flags off user is informed that it's not possible to access optimization at the moment

BasicBS on but Constant Multiple Read Only Enabled, user has no trigger - constant multiple remains hidden

BasicBS is on , CM is in readonly but user has trigger - message shall be displayed, user allowed to remove trigger

https://goerli.etherscan.io/tx/0xe842c7c2ae1c9dc680cec02a1fca99700ec42b78117b733befa13a2aacc4a17f

Summary should be the same as when read only flag is not enabled

Combination of read only flags (BS and CM) should should show banner in optimization tab only if there are no active optimization. If there is active either Auto-Buy or Constant-Multiple we need to give chance to the user to remove existing trigger.

Why on all screens also Notifications are true and Exchange ? are those dependencies ? If no (and I guess no) they should be false

Summary should be the same as when read only flag is not enabled

Gotcha

Copy of following banner (when 0 debt in vault) needs to be updated to include CM as well

If I reduce my debt to zero when CM is enabled I'm not able to remove it.


Add Finite Difference schemes

Add some of the core finite difference operations. The full slicing version can be used in demos then then we can use the proper packages like serket and kernex.

Key Packages

serket - has different FD operators
kernex - has the convolution version for the FD operator

Core Functionality

[x] Gradient
[x] Laplacian
[x] Determinant Jacobian
[x] Divergence
[x] Curl
[x] kernex functionality

https://github.com/Unidata/MetPy/blob/main/src/metpy/calc/tools.py
https://github.com/gbogopolsky/fdm-operators/tree/master/fdmoperators/numpy_operators
https://github.com/gbogopolsky/fdm-operators/tree/master/fdmoperators/pytorch_operators
https://github.com/seismo-live/seismo_live_build/blob/e4e8e59d9bf1b020e13ac91c0707eb907b05b34f/notebooks/Computational Seismology/The Finite-Difference Method/fd_taylor_operators.ipynb
https://github.com/seismo-live/seismo_live_build/blob/master/notebooks/Computational Seismology/The Finite-Difference Method/fd_taylor_operators_advanced.ipynb
https://seismo-live.github.io/tree/index.html
https://github.com/apaloczy/ap_tools/blob/master/dyn.py
https://github.com/zwicker-group/py-pde
https://github.com/PyLops/pylops
https://inst.eecs.berkeley.edu/~cs194-26/fa17/Lectures/ConvEdgesTemplate.pdf
https://www.crcv.ucf.edu/courses/cap5415-fall-2021/schedule-cap5415-fall-2021/

Closing due to #9 and issue at FiniteDiffX


Refactor: GitHub Actions Workflows for Issue Locking and Greetings ⚒✨

Description:

We are seeking to enhance the efficiency and cohesiveness of our GitHub Actions workflows for issue locking and greetings in the Imagine AI Repository. Currently, we have two separate workflows named lock-issues.yml and issues.yml (greetings script).
The lock script locks the issues before greetings script greets the user 🙅🏻
To ensure better organization and prevent potential conflicts, we propose a refactoring of these workflows into a single, unified workflow that seamlessly handles both operations.

The main objectives of this refactoring are as follows:

Seamless Integration: By combining the issue locking and greetings functionality into a single workflow, we can avoid potential conflicts between the two processes. This will ensure that the locking process occurs only after the greeting comment has been added to the newly opened issue.

Improved Workflow Management: A unified workflow will result in cleaner and more manageable code. It will be easier to track changes, make modifications, and ensure consistency throughout the workflow.

Simplified Configuration: With a single workflow, we can reduce redundancy and streamline the configuration process, leading to a more straightforward and maintainable setup.

Dependency Management: By setting dependencies between the locking and greetings steps, we can guarantee that the issue locking process will wait for the greetings comment to be added before taking action. This will prevent issues from being locked prematurely.

Enhanced User Experience: Merging the greeting and locking actions into one workflow will provide a more seamless and user-friendly experience for contributors. They will receive a warm welcome comment and be informed about the locking process in a coherent and organized manner.

Code Repository Cleanup: The merging of the workflows will result in a cleaner and more organized GitHub Actions section in the repository, leading to easier maintenance and better collaboration among contributors.

Screenshot

ISSUE IS LOCKED BEFORE GREETING - HAPPENING 90%

ISSUE IS LOCKED AFTER GREETING - HAPPENING 10%

Hey there @SurajPratap10 👋🏻
Assign this issue to me to refactor those workflows to perform more efficient and user-friendly GitHub Actions workflow for issue locking and greetings in this Imagine AI Repository!