Bundling with Parcel/npm

I just committed a branch that makes transcrypt work with Parcel Bundler. Parcel is a zero-config bundler that is gaining traction. The branch is at:
https://github.com/QQuick/Transcrypt/tree/parcel-bundler
The example run is at transcrypt/parcel-plugin-python/. See the README.md file for installation instructions.
When the test server is run, the parcel bundler starts with index.html and index.js, which contains:
import {main} from "./some.py";
main();

The .py extension tells parcel to run the file through transcrypt.

I've said this before, but IMO transcrypt must work seamlessly with bundlers for it to see wide use. Modern JS projects include many coding languages (JS, CoffeeScript, TypeScript, Rust). The role of the bundler is to transpile them all to regular JS and then bundle them together.
Transcrypt is fundamentally different from bundler-based transpilers because it acts both as a transpiler AND a bundler.  It doesn't just transpile single files, but it also includes everything those file import. For example, if some.py contains "import mylib", transcrypt compiles BOTH some.py and mylib.py. A single run of transcrypt usually results in a number of .js files, which are placed together in __target__.
Regular transpilers generally stick to single files. When asked to compile some.py, they compile ONLY that file and don't follow any import statements. The import statements continue to keep the extensions of the files they import (transcrypt switches everything to .js). It's the job of the bundler to follow the imports and call the right transpiler.
So while this branch makes transcrypt work with Parcel, it is a bit of hack because transcrypt isn't meant to work this way. It also breaks the normal transcrypt process because it keeps import file extensions (in revisit* methods). See the diff in compile.py for where this happens.
The recent change to ES6 and units is a HUGE step toward doing things the bundler way, but a few things remain:

Transcrypt needs to keep import extensions (e.g. not change them). Like this:

Python Source:
from calculator import addit

JS Output:
import {addit} from './calculator.py';

This way the bundler knows to run transcrypt again for calculator.py.

Since transcrypt will only transpile a single .js file (the main one), there's no need to copy other files to the __target__ directory. This means the __runtime__, imported local files, and built-in modules (re, time, etc.) DON'T need to exist as siblings. Instead, these should be put into the npm repository just like any other imports. In fact, since only a single file export is needed, we can do away with __target__ entirely.

An example import from the built-ins:
Source:
import re

JS Output:
import * as __module_re__ from 'transcrypt/re/__init__.py';

Note that the JS import is not relative (./), so the bundler will look in node_modules/transcrypt/re/__init__.py rather than in the local directory. It will expect the re module to be in the npm repository. Importing the __runtime__ should be the same way.
I know JdeH wants to keep the current transpiler+bundler abilities of transcrypt, and I think that's fine. But I'd really like to see transcrypt become a first-order transpiler that works with bundlers out of the box. Right now it's not quite set up right to be one, but I don't think it needs a lot of changes to get there.

There isn't a doubt in my mind that this should be in somehow!
Can not yet oversee what this means for sourcemaps etc.
Seems we'll need the target directory for those or ...?
And I'd like to retain the possibility to run Transcrypt stand-alone in a multi-module situation.
But that's a matter of adding a switch to enable cooperation with bundlers.
Any ideas?

Awesome! I am encouraged by your response and questions. I also can appreciate the need for backwards compatibility.
There are straightforward answers to your questions, but I can't write the response right now. I'll answer more in depth tomorrow. Things like the sourcemaps are done by the other languages (usually streamed, I think). The namespacing has a good solution too. Look for my response tomorrow.

FYI, here's the current npm stats on the bundlers. webpack is still the goliath and is going strong. Their last release supposedly did a lot to alleviate the "config hell" it required. My impression is the crazy configs of webpack put off a lot of python devs -- the entire thing was pretty orthogonal to python culture (e.g. one way to do things, focus on simplicity and clarity, etc.). Webpack requires a lot of wrangling in my experience. I guess the most recent release fixes much of that, but my personal work with it showed it was still a pain (although please factor in my bias against the wild west of JS culture, despite me living in the actual wild west :).
Parcel was released in Dec, 2017, so it's about a year old. It's an up-and-coming bundler, but I doubt it could ever overtake webpack (which has significant momentum right now). It advertises zero config and is a reaction against the webpack mess. My experiences with it over the last couple weeks is it is MUCH simpler than webpack, but it's still not the "clear and simple" that python devs would want. I do think it's the closest thing to it at this point. My opinion (which could probably easily be swayed) is that Parcel is currently the bundler that python devs would be most comfortable with.
But hopefully if we can get a bundler like Parcel working seamlessly, it will prepare transcrypt to be easily integrated into the other bundlers as well. Parcel plugins are pretty easy to code, although I've had to get into the actual parcel source code A LOT. Like most things JS, the documentation is less than stellar.
So again, here's the stats on the bundlers. You can see in the picture that Parcel doesn't really register. However, it's large number of stars is encouraging.

How to proceed with sourcemaps and the location to store them?

Let me try to summarize how the TypeScript parcel plugin works. It's a good example of how all the transpiler plugins work.

Parcel sees an import statement in a JS file: import * as bar from './foo.ts'. This triggers the plugin to go to work.
The TS plugin loads the source from foo.ts into a variable and does the transpile.
The plugin needs to return a JS object containing the following to Parcel:

transpiled.outputText - A string containing the output JS
transpiled.sourceMapText - A string containing the source map in JSON format. This is converted
In some plugins, the result might also include the actual AST tree using Babel AST objects (a widely-used library).

Note that the TS plugin doesn't have to use the filesystem at all and doesn't have to stream anything between processes because it's written in JS. It runs right in the Parcel process and returns the objects.
Transcrypt can't do this because it needs to shell a Python process out of the plugin. That's not hard from JS/Node, but we need to figure out a good way to transfer back 1) the transpiled JS and 2) the sourcemap JSON.
Remember that we adjust transcrypt to work like the other transpilers, it is transpiling the imported file ONLY. Not other .py files that the file might import. So we really only have TWO things to send back, NOT two things * the number of files.
It seems to me that streaming the result back to the plugin is the right way to do it so we don't clash with files on the filesystem. The stream could be JSON that contains the source + sourcemap. Alternatively, we could use two files placed in the __target__ directory like transcrypt currently does. The plugin is doing it this way right now -- it deletes the __target__ directory after reading the two files.
There are only two issues I saw with transpiling a single file rather than the entire an entire import tree. I mentioned them in my original post but I'll say them again because they apply to this point:

The import statements in the output JS source need to keep the file extensions of imported files. See my next post for the issue of resolution.
The python statement from bar import * requires information from bar.py because the JS doesn't have this concept.  The JS concept of import * as {name} from 'foo.js' is instead related to  python's import foo.  That means that the line from bar import * has to be transpiled as import { e1, e2, e3, ... } from './foo.js. We have to name each of the exported items in the imported file one by one, and that means we have to parse the imported file. This is doable when the imported file is Python, but it's a problem if the imported file is typescript or coffeescript or javascript.

One easy solution is to simply disallow the use of from bar import *. It's generally considered to be bad python anyway. This probably isn't the right thing to do long term, but you could punt on this one for the short term. Feel free to flame me on this one because I feel dirty even suggesting it. :)

I get your point about modules having to reside in the npm repository, but what exactly does that mean for e.g. the re module? It won't be named __init__.py on npm, would it, since that name would occur many times. What, in your view, would the e.g re module look like on npm?

Hopefully the following gets the idea across -- it will take experimenting with examples to get the "final" layout, so this isn't an actual proposal. I'll do the re module since others are just siblings. I'm also thinking clean slate without regard to backwards compatibility at this point (so it will need adapting before going forward):
node_modules/
    transcrypt/
        src/
            modules/
                re/
                    __init__.py :: `import translate`
                    translate.py
        lib/
            modules/
                re.js :: "import {translate} from './re/translate.py';"
                re/
                    translate.js

My reading of npm best practices says both the original language (for reference) and the transpiled JS (for actual use) get published to npm. The package.json file of the package tells npm that the source to use for imports is in transcrypt/lib/modules. In other words, the JS always gets used during bundling. Other languages like TypeScript can import the modules without having to install Python. I realize there's no real use case for importing transcrypt modules into TypeScript, but doing it this way keeps us in line with typical npm expectations.
Before uploading an update to npm, we run transcrypt on the modules/src/ directory so the two are in sync.

Having modules like re, math and iterator on npm is bound to lead to name conflicts. But I'd like to keep the original Python module names and import syntax. However, a Python statement like import math simply doesn't contain enough info to prevent a name conflict if all such modules are stored on npm in "worldwide namespace". Transcrypt could add a prefix, but this cannot be expected from Javacript modules importing packages written in Transcrypt.

Since transcrypt does the resolution of imports in .py files, we have full control over this. The only issue I see is when python standard modules shadow over npm modules. I'll discuss that at the end.
Here's an example of various cases:
Python source: (pother is a local sibling python file, jother is a local sibling js file, re is a Python/transcript standard module, moment is a js library in npm repo):
import pother
import jother
import re
import moment

As transcrypt parses through the file, it decides how to resolve each one. The resulting JS import line determines where the bundler finds each file.

import pother: transcrypt finds pother.py in the source directory. It generates import * as pother from './pother.py'. (Parcel will call transcrypt later to transpile pother.py)
import jother: transcrypt doesn't find pother.py but finds pother.js in the source directory. It generates import * as jother from './jother.js'.
import re: transcrypt doesn't find re.py or re.js in the source directory. So it goes to the npm repo, first looking in the transcrypt lib. It finds node_modules/transcrypt/lib/modules/re.js. It generates import * as re from 'transcrypt/lib/modules/re.js'.
import moment: transcrypt doesn't find re.py or re.js in the source directory. It doesn't find re.js in the transcrypt node_modules repo. It finds node_modules/moment/, a normal npm module. It generates import * as moment from 'moment'.

JS output:
import * as pother from './pother.py';
import * as jother from './jother.js';
import * as re from 'transcrypt/lib/modules/re.js';
import * as moment from 'moment';

So looking at the above, I think everything works beautifully with the one possible exception of a npm module being shadowed. Looking on npm, there is a module named re. That package can never be imported from python source code because transcrypt will always resolve to the standard python re.js (as above) and never to node_modules/re/.
I would argue that this is an edge case, so perhaps the solution is as simple as __pragma__ (‘js’, …)?

Feel free to PM me if you want to set up a F2F call to discuss any of this. Also be sure to check the diff on the actual transcrypt source code (master branch vs. parcel-bundler branch). The changes are pretty minimal but were just a proof of concept rather than finished code. I think that diff shows how close transcrypt is to making this happen, but I'm leaving the decisions to you on how/what/where to modify the transcrypt code if we go forward with this kind of thing.

FYI I'm implementing some of these things today (parcel-bundler branch, not in master).

Status report: The changes to transcrypt to allow direct plugin use are almost done. I'll probably finish it tomorrow.

@doconix
I'd like to evaluate the changes you made, but have trouble getting an overview since lots of whitespace changes are also marked as differences. I've tried the ?w=0 option in the diff view but it doesn't solve the problem. Would it be possible for you to have only relevant (non-whitespace) etc. changes on GitHub, enabling comparison to the master branch. I've tried an external file compare tool but if failed on the  job, ignoring relevant differences. Sorry I have to ask this, but since the work on Transcrypt really takes up lots of my time now I feel forced to streamline. Know that I greatly appreciate all the effort you put in!
@PythonLinks @doconix
It's agreed that Transcrypt has to stay as simple as possible. I must say I find it impossibly hard to get any kind of grip upon the explosion of half-compatible, half-contemplated approaches in the JavaScript world. The prime reason Python is becoming so popular is its simplicity, although part of that recently is getting lost in overstressing facilities like iterators, decorators, metaclasses, static typing etc.. I hope Python is spared from getting as complex as e.g.  C++, although I really like that language and have used it most of my working life. Especially with Guido no longer at the wheel I am worried that the "the more the merrier" approach takes over. To be honest it isn't the low threshold common sense scripting language anymore that I once bumped into.
From the docs of Transcrypt:
http://www.transcrypt.org/docs/html/what_why.html#what-is-transcrypt

One of the main goals of Transcrypt is to keep things simple, preventing it from disolution into an ocean of feature creep and conflicting demands. At its core it will stay lean and mean.

Wanting, or rather needing, this simplicity for me was the reason to abandon JS development of browsr apps and start the  Transcrypt project.
Transcrypt is primarily meant for Pythonista's who want to make browser applications in Python while using JS libs. Still the possibility of tight integration with the JS world is valuable. In this JS world an ever changing diversity of bundlers seems to be a fact of life. I don't expect these to converge as many JS developers don't seem to be particularly interested in that kind of thing. While popularity may shift, I don't expect it to ever really stabilize unless the ECMA committee proclaims a standard.
To support use of Transcrypt in combination with those, I propose the following criteria:

The current simple way of working, including use of the flat target directoryholding everything including sourcemaps, should be preserved as default.
Changes aimed at making Transcrypt fit in the bundler world should not complicate TS's sourcecode too much, as it will all have to be maintained.
Changes in Transcrypt itself should not be tied to one particular bundler. They should comply with emerging ECMA standards (still very virtual in this area), as does the module mechanism.

At the same time it would be very nice if Pythonista's could write e.g. datascience/AI related stuff in Python that would be available as JS compatible libraries that can be drawn into JavaScript apps the way JS programmers are used to, so using popular bundlers. This would enable Pythonista's to enter the enormous JS and Web Client world.
I want to be very clear: my heart is with Python, not with JavaScript. But since it's a massive stream, Pythonista's should be able to tap into it, even if I hope their main focus will be making algorithmically interesting server applications, rather than fancy front-ends whose main purpose is to look good.

@JdeH Thanks for the feedback on this. I've been at this the last couple days, and I'm pretty sure it will be done by today sometime. I'm almost there, but as you know, bugs can be a long tail.
The changes to transcrypt are actually minimal, and I've been able to keep full backwards compatability. The bundle way of doing things is just an option.
Some of the ideas I proposed didn't work as expected, so don't waste too much time trying to go through it. I think the finished code will be clear.
The implementation is very much in line with your 3 criteria. Well, except one. :) I was able to implement a deeper target structure rather than flat. I think it's more clean this way because the target directory matches the source directory. BUT, you da boss, and I can step back to flat if you want. But perhaps you can wait until you see the running code to make a final decision...
I'll post again when done.

@JdeH Status update: It's working as of about an hour ago but I still need to do tests and ensure things are working. It will be another day or two. I have to step away from it for a bit.

@JdeH I am now working in the parcel-bundler2 branch. I restarted from scratch (and copied relevant code from the other branch) so I could be back to a pristine copy of transcrypt.  I'm still putting the finishing touches on it, but I want to put out a few questions to you:

Right now I have the code in the transcrypt/parcel/ folder. Do you want this code in the transcrypt codebase, or do you want me to put it on a separate project? The plugin gets posted to npm, so it really doesn't matter where the source lives. It is really just a matter of preference on whether you want to keep things together or split them up.

I'd like to write a page in docs/ on using the parcel bundler, but again I want to ask if that is ok. Separate place or integrated with transcrypt docs?

Despite me making lots of changes to transcrypt's code in various tries, the end result is almost no changes!  The one change is in the constructor of Program.__init__:

The plugin needs to know what .py files transcrypt found and what .js files transcrypt created. I'm putting these in the .options file you create in Program.__init__.  The need for this is so the plugin can autoreload during development (recreate bundle anytime a .py file is changed). This is the ONLY thing that I had to change in the codebase, and all of it is in compiler.py.
The file now uses json format (instead of pickle) so my JS plugin can read it, and the name is changed: <main>.options -> <main>.transcrypt.json. There's no real need for that name change; I changed it simply to be closer to its new purpose. My reading of the code says that all the pickleableOptions are ok types for json.
Let me know if it's OK to expand the options file like this.

No changes to command line arguments.

The plugin is using the flat structure. No need for deep structure. I did have that code working if you ever want to explore it, but flat works fine for the plugin.

I tested the built-in modules (__runtime__, re, etc.) on npm, but it actually worked better to keep them within the __target__ directory. So no links to npm repo in the plugin.

The plugin is not tied directly into transcrypt, so other plugins and other bundlers hopefully can use (improve) the strategy I used for webpack, etc. No favoritism to this plugin. After a break, I can probably adapt it for webpack pretty easily. The current webpack plugin does work, but it's not seamless (imo).

Let me know about this code living inside transcrypt's github vs. separate project. And let me know about the docs. Regardless of your decisions on those two, I hope we can reference the docs where people can find them. I want to make it as easy as possible for people to use transcrypt in bigger projects.

@JdeH Just a ping on this one.

@doconix
I am attempting to run the examples, but at the first example I get:
Cannot find module 'parcel-bundler/src/Asset'
and
Cannot find module 'parcel-bundler/src/Logger'
I am using npm 6.5.0, parel-bundler 1.11.0, parcel-plugin-python 1.0.18, transcrypt 3.7.12
Any idea how to proceed? I'd love to find out how it all fits together...

@JdeH Sorry I've been out of touch on this. Thanks for spending time on it. I don't know why those files would be missing in the parcel-bundler package....
Anyway, agreed that we should discuss offline. Perhaps a screen share would be useful so I could show you what it's doing, and we can walk through each part.
I'd also like to mirror the ideas directly into webpack so transcrypt can work with both parcel and webpack (and the two plugins are similar to develop/maintain).

@JdeH I just posted a new update to the parcel-bundler2 branch. This does the following:

Moved the testing files for Parcel into the manual_tests directory. Made adjustments to code and docs as needed to match.
Created a Webpack loader that mirrors the way the Parcel plugin works.

One cool thing about the webpack loader is it is programmed in Transcrypt rather than JS. I tried switching the Parcel plugin to Transcrypt, but that has to extend the Parcel Asset JS class, so I reverted back to JS for it.
I still need to do the testing files for the webpack loader.

Just curious, is it safe to write a tutorial showing this Parcel-based approach?

As followup, here's a tweet from @jviide (with a tweet before it) showing VDOM and rendering in Python, as well as browser via Transcrypt. For folks interested in Python and SSR, it's an important step.

Logger.js
emoji.js
prettyError.js

Is there any plan to fix the missing file issue?  Other than that, this Parcel plugin has been working great.

Following up, there was a change in the Parcel library where the logging module was moved.  It can be fixed by changing the reference to the logging module import in asset.js as I mentioned here.

Now that Parcel v2 is out and the current Parcel plug-in for Transcrypt is incompatible with the new version, it might be worth revisiting this to create a parcel-plugin-transcrypt-v2 that deprecates the old one that is broken anyway.

I just published a Python transformer for Parcel V2 that uses Transcrypt.  I'm still working on the docs and it needs way more testing in different environments than I've done so far, but I figured I'd put it out there early and get some feedback.
Feel free to open up a GitHub issue if you run into any problems with it.


Acryldata master

Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable)

@MathieuTheerensKlarrio What is this PR for? This looks like an internal thing for your organisation


20150224 ie change style to ng style

Fixes an issue where dropdown content was not rendering in IE10.

?


Django Issue Checker error?

What happened?
Was looking to use asyncio httpx instead of requests, but I found the issue checker is running into errors?
https://github.com/cookiecutter/cookiecutter-django/actions/runs/5397595858/jobs/9802447698

Yes it failed this morning for some reasons. I just retried it and it worked. Might have been a flaky failure.

Well, it ran, but it re-opened the issue despite the fact that we are now on Django 4.2... So maybe there is something to fix after all 😄


Cannot show LAMMPS commands properly

For example, I tried the following text:
units 		metal
boundary	p	p	p
atom_style	atomic

They are typical LAMMPS words, but they did not highlighted in my case after installing this package. Is there anything that I need to add at the very beginning?

Hi @CDmaterial ,
Sorry about getting to you so late.
I have download Atom and installed it, for me it works perfectly fine. Have  you checked that Atom is recognizing the file as a LAMMPS file? It needs to have the .lmp extension, otherwise you have to choose the format manually in the bottom right

Hi,
Thank you for the reply. I am wondering why it need a .lmp extension? I thought this code is to show the lammps commands for input files.
Thanks,
Cong

On Nov 29, 2019, at 7:04 AM, Bruno Durán<EMAIL_ADDRESS>wrote:
Hi @CDmaterial https://github.com/CDmaterial ,
Sorry about getting to you so late.
I have download Atom and installed it, for me it works perfectly fine. Have you checked that Atom is recognizing the file as a LAMMPS file? It needs to have the .lmp extension, otherwise you have to choose the format manually in the bottom right
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/brunoduran/language-lammps/issues/2?email_source=notifications&email_token=AK42L7CQS4AJ66AGAHF2P3TQWEALZA5CNFSM4JHNB3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFOWIYI#issuecomment-559768673, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK42L7BF32JWG5T6YIAAHITQWEALZANCNFSM4JHNB3LA.

Hi Cong,

I am wondering why it need a .lmp extension?

Atom uses the extension of the file to decide which kind of file is. So files ending in .py are Python, .js JavaScript etc.
LAMMPS scripts does not have an official extension, but some users (like this one) adopted .lmp, .lmps or starts the filename with in..
When I made the plugin I chose to highlight automatically .lmp, .lmps and .lammps.

Hi Bruno,
Thank you for the explanation, and it works now!
Cheers,
Cong

On Dec 3, 2019, at 8:22 AM, Bruno Durán<EMAIL_ADDRESS>wrote:
Hi Cong,
I am wondering why it need a .lmp extension?
Atom uses the extension of the file to decide which kind of file is. So files ending in .py are Python, .js JavaScript etc.
LAMMPS scripts does not have an official extension, but some users (like this one https://github.com/lammps/lammps/blob/master/examples/USER/fep/CC-CO/bar01/in.bar01.lmp) adopted .lmp, .lmps or starts the filename with in..
When I made the plugin I chose to highlight automatically .lmp, .lmps and .lammps.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/brunoduran/language-lammps/issues/2?email_source=notifications&email_token=AK42L7GXI7I46H42RRWRAQDQWZMP3A5CNFSM4JHNB3LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFZK5XI#issuecomment-561163997, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK42L7BLYHPFZBQA7SCQX5LQWZMP3ANCNFSM4JHNB3LA.


Update install-behind-a-firewall-or-proxy-server.md

Learn Build status updates of commit caf294a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mac/install-behind-a-firewall-or-proxy-server.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

#sign-off


ctr: add commands: list and inspect

Example
$ sudo ctr list
ID      STATUS  PROCS   BUNDLE
foo     RUNNING 1       /home/suda/tmp/ocibundle/busybox

$ sudo ctr inspect foo
(*execution.GetContainerResponse)({
        Container: (*execution.Container)({
                ID: (string) (len=3) "foo",
                BundlePath: (string) (len=32) "/home/suda/tmp/ocibundle/busybox",
                Status: (execution.Status) 1
        })
})
(*execution.ListProcessesResponse)({
        Processes: ([]*execution.Process) (len=1 cap=1) {
                (*execution.Process)({
                        ID: (string) (len=4) "init",
                        Pid: (int64) 11922,
                        Args: ([]string) <nil>,
                        Env: ([]string) <nil>,
                        User: (*execution.User)(<nil>),
                        Cwd: (string) "",
                        Terminal: (bool) false,
                        ExitStatus: (uint32) 0
                })
        }
})

The output is not pretty, but I think it is enough as a dev tool.
Signed-off-by: Akihiro Suda<EMAIL_ADDRESS>

For inspect you can use the json.Indent to "prettify" the output (see here)

we could use https://github.com/davecgh/go-spew instead.
But meanwhile, LGTM

done @mlaventure @crosbymichael

Oups, didn't meant to approve the review, missed click.
@AkihiroSuda could you add the 2 config I requested for spew please?
It LGTM otherwise.
Thanks

@mlaventure
Updated PR.
(It also avoids printing pointer address, and contaminating the global spew.Config)


Fit persistence

Hi, this project looks incredible. Every time I compare sklearn with JS ML libraries, the python equivalents are much faster and more robust. I am looking to integrate sklearn with Saltcorn, an open-source no-code builder I wrote.
Is there a way to persist model fits? in python i can pickle a fit, save to database or file, and pickle.loads when it is time to make a prediction. Example:
# fit
clf = Pipeline(steps=[('preprocessor', ...),
                      ('classifier', LinearRegression())])

clf.fit(df_x, df_y)

str1=pickle.dumps(clf)

# load and predict
clf2 = pickle.loads(str1)
pred=clf2.predict(pandas.DataFrame.from_dict(...))

Is there a way to get the underlying python object as a blob or as JSON ? and to construct the JS object again?

This should be possible, but we need a good example of it.


 This is not  related to b/74395663, it's probably due to differences in the Python environment. I'll look into it but I don't think it's something anyone else needs to worry about (so disabling LGTM).

Thanks Allen!


Add license

See here for some options, but it would be cool to open-source this if we get some nice results.

Looks like this is fixed :)

Thanks!


Use copy instead of hand-rolled for for Lob copying

I've mentioned mmap earlier, but anonymous mmap is the same is malloc, so does not help here.
The streaming API needs some thinking: how would it retain the underlying loblocator and cursor handle till usage?

I would leave it up to you. I don't have the time at the moment.


Update with publishing of cppwinrt.vsix

@stevewhims : PR passed acceptance checklist. Please evaluate the proposed changes further and merge if appropriate. Thanks!

We chatted about this over email, the push to the "live" branch happened after this PR was created, so the live topic is now actually updated and correct. I'll go ahead and close this. Future comms can be done internally via email on this matter or any others. Thanks! -Steve


Orientation to API tasks

Describe the task
@ianliuwk1019 to help @NickSaglioni get familiar with the FAM API
Acceptance Criteria

[ ] Nick and Ian have shared knowledge
[ ] Nick can begin to take on some API tasks

Additional context

Also consider doing some pair programming sessions

Gave overview to Nick and did some troubleshooting on his environment together on Dec 23rd.


The GPG keys listed for the "google-chrome" repository are already installed but they are not correct for this package.

we have been using this action till yesterday. setup-python-amazon-linux is throwing below error now.
---> Package google-chrome-stable.x86_64 0:126.0.6478.126-1 will be updated
actions/runs/9968800181/job/27544637807#step:3:21)---> Package google-chrome-stable.x86_64 0:126.0.6478.182-1 will be an update
/9968800181/job/27544637807#step:3:22)--> Finished Dependency Resolution
actions/runs/9968800181/job/27544637807#step:3:26)================================================================================
actions/runs/9968800181/job/27544637807#step:3:27) Package                 Arch      Version               Repository        Size
[runs/9968800181/job/27544637807#step:3:28)================================================================================
/9968800181/job/27544637807#step:3:29)Updating:
actions/runs/9968800181/job/27544637807#step:3:30) google-chrome-stable    x86_64    126.0.6478.182-1      google-chrome    106 M
actions/runs/9968800181/job/27544637807#step:3:31)Transaction Summary
/actions/runs/9968800181/job/27544637807#step:3:32)================================================================================
actions/runs/9968800181/job/27544637807#step:3:33)Upgrade  1 Package
actions/runs/9968800181/job/27544637807#step:3:34)warning: /var/cache/yum/x86_64/2/google-chrome/packages/google-chrome-stable-126.0.6478.182-1.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 9b30acf2: NOKEY
actions/runs/9968800181/job/27544637807#step:3:35)Total size: 106 M
actions/runs/9968800181/job/27544637807#step:3:36)Downloading packages:
actions/runs/9968800181/job/27544637807#step:3:37)Retrieving key from https://dl.google.com/linux/linux_signing_key.pub
actions/runs/9968800181/job/27544637807#step:3:38)The GPG keys listed for the "google-chrome" repository are already installed but they are not correct for this package.
actions/runs/9968800181/job/27544637807#step:3:39)Check that the correct key URLs are configured for this repository.
actions/runs/9968800181/job/27544637807#step:3:40) Failing package is: google-chrome-stable-126.0.6478.182-1.x86_64
runs/9968800181/job/27544637807#step:3:41) GPG Keys are configured as: https://dl.google.com/linux/linux_signing_key.pub

Which Amazon Linux version are you running this action in?

Amazon Linux 2

Are you reusing self hosted runners for multiple workflows or is a new one getting spun up everytime ?

Google-chrome 126.0.6478.126-1 was installed in the runner, and when there was an update to the Google-chrome package, there was a conflict in gpg verification. By installing the latest version of Google Chrome, the issue was resolved.


[cli] Cannot map host directories if deploying from Windows command line

Rancher Versions:
Server: 1.5.3
healthcheck: 0.2.3
ipsec: 0.0.5
network-services: 0.1.1
scheduler: 0.5.1
rancher cli: 0.5.1
Docker Version:
17.03.0-ce
OS and where are the hosts located? (cloud, bare metal, etc):
Ubuntu 16.10 bare metal
Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB)
Single Rancher server, internal DB with a data directory mapped to a host
Environment Type: (Cattle/Kubernetes/Swarm/Mesos)
Cattle
Steps to Reproduce:

Set up Rancher server
Add Linux host to Rancher server
Set up Rancher CLI on Windows 10 machine to connect a Rancher server above
Unpack the attached docker-compose.zip into a separate directory on Windows machine
Open Windows command prompt and go to the directory where the attached file was unpacked
Issue the following command: rancher up --stack test

Results:
The stack creation is error out with the following message:
Activating (Expected state running but got error: Error response from daemon: create C: volume name is too short, names should be at least two alphanumeric characters)

The original file docker-compose.yml is
cadvisor:
  labels:
    io.rancher.scheduler.global: 'true'
  image: google/cadvisor:latest
  volumes:
    - "/:/rootfs:ro"
    - "/var/run:/var/run:rw"
    - "/sys:/sys:ro"
    - "/var/lib/docker/:/var/lib/docker:ro"

but when I open View Config in Rancher UI, I see the following config:
version: '2'
volumes:
  C:
    external: true
    driver: 'null'
services:
  cadvisor:
    image: google/cadvisor:latest
    volumes:
    - C:/rootfs:ro
    - C:/var/run:rw
    - C:/sys:ro
    - C:/var/lib/docker:ro
    labels:
      io.rancher.scheduler.global: 'true'

I'm having exactly the same problem.

Same error here:
Server: 1.5.6
Docker: 17.03.0-ce
rancher cli Windows: 0.5.2

It's the same with the Rancher CLI v0.6.0-rc1.
That makes the cli practically useless from a Windows environment. We would like to migrate from rancher-compose to rancher cli but thats right now it is not possible due this huge bug.

Having this same problem on rancher-windows-386-v0.6.5, Server: v1.6.11.

Thanks for your report!
With the release of Rancher 2.0, development on v1.6 is only limited to critical bug fixes and security patches.
If you think we should keep this issue open, please let me know.


G1ParScanThreadState:: error

Please provide a brief summary of the bug
A fatal error has been detected by the Java Runtime Environment:

SIGSEGV (0xb) at pc=0x00007fb1531d2d55, pid=9928, tid=0x00007fb151afc700

JRE version: OpenJDK Runtime Environment (8.0_292-b10) (build 1.8.0_292-b10)
Java VM: OpenJDK 64-Bit Server VM (25.292-b10 mixed mode linux-amd64 compressed oops)
Problematic frame:
V  [libjvm.so+0x5b3d55]  G1ParScanThreadState::copy_to_survivor_space(InCSetState, oopDesc*, markOopDesc*)+0x35

Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
hs_err_pid9928.log

An error report file with more information is saved as:
/tmp/hs_err_pid9928.log

If you would like to submit a bug report, please visit:
https://github.com/AdoptOpenJDK/openjdk-support/issues
Please provide steps to reproduce where possible
We have many runtime environments runnning the same versions of tomcat and  java - have only seen this one instance
Expected Results
it doesn't crash
Actual Results
it crashed
What Java Version are you using?
build 1.8.0_292-b10
What is your operating system and platform?
cento7 7  3.10.0-1160.66.1.el7.x86_64
How did you install Java?
archive
Did it work before?
yes - it was a runtime crash

Did you test with other Java versions?
we know we should upgrade but have not sen any other issues.

Relevant log output
log was attached above

@mlevine-active please update to the adoptium.net build 1.8.0_342 and reopen if this still exists.


Adding workflow to publish SqlLite nuget package

adding CI wf for SqlLiteRepo package
update Readme to include release history and future goals
main publish wf for sqllite

Rolling the dice on the nuget key. 🤞🏻


TIMESYNC-NTP got kernel upgrade

I see the test report in Summary page. It shows the kernel upgrade. why do we need its upgrade? The target kernel should be default not upgraded one.

It is not related wit test case, we specify the -CustomKernel during we run testing, since we hit it in the pipeline and repro it in pipeline.


abort compilation of regex-based pattern matching when the nfa to dfa…

… translation degenerates into huge dfa states

Nice and straightforward!  This should be a good stopgap while we work on reducing these DFA sizes.  :)


Fixup testcase (forgot to amend my last commit before pushing it)

@swift-ci test and merge

@dcci turns out I had unstaged changes in my source tree ...


Implemented 3rd argument for useMachine: onStop

I modelled a process for subscribing and unsubscribing from a long-running service as a state machine.
However, when integrating the machine with a React component, the unmount lifecycle was problematic with @xstate/react 's useMachine`:
A warning appeared on the console notifying me that an event has been sent to a service that isn't running, and that I need to service.start() first before I can send the event. Basically the service was already stopped by useMachine's React.useEffect implementation.
I tried to solve that in user-space by using React.useEffect (see Attempt) but I noticed that the event is not being sent to the service.
I added an example to the README that shows a similar scenario for unsubscribing from an interval event.
This is the easiest way I could think of to implement cleanup of the subscription.
That said, it's not an elegant solution in my opinion.
It's also possible to add an onStart argument (for completeness?), a function that would be called after the service has started.
Please share your thoughts on this! :hand:

Love the documentation, but as far as implementation, we should reduce the number of arbitrary arguments we pass into useMachine(...) (I like 1 - 2 arguments for simplicity).
Thankfully, the unsubscribe pattern you're trying to implement is already doable with useMachine() by referencing the service:

const [current, send, service] = useMachine(someMachine);

// Register stop listener once
useEffect(() => {
  service.onStop(() => sendFirestore('UNSUBSCRIBE'));
}, []);

Might be helpful to instead add the above ☝ to the documentation.


[svdconv] Wrong error message: *** ERROR M382:

Describe the bug
Wrong error message "M382" for register with 64 bit size
*** WARNING M382: RESET_MASK.svd (Line 51)   Register 'ABS': Reset Mask '0xFFFFFFFFFFFFFFFF' does not fit into Register width: 64 Bit.
To Reproduce
Run the below command on attached file
svdconv.exe RESET_MASK.svd
Expected behavior
M382 error shall not be shown
Environment (please complete the following information):

[tool or library]
svdconv version 3.3.46
OS: windows

RESET_MASK.zip

I get the same warning for the registers of size 64 bit.


Added prefix.dev as a startup

Hey I've added the startup I've been working for the past months to the list.
You've already seen some PR's and issues by @baszalmstra. We have been enjoying sea-orm and sea-query so far :)!

Awesome! I really liked the graphics style


[FEA] Supporting multi bands (multi-channel) images for geospatial (remote sensing) and microscopy

Is your feature request related to a problem? Please describe.
cuCIM is currently supporting only jpeg/deflate-compressed RGB image (which is prevalent on Digital Pathology image) only and doesn't support multi-channel TIFF images yet.
Supporting multi-channel would help to load geospatial(remote sensing. multi-band) or microscopy (multi-channel) images.
Describe the solution you'd like
Supporting multi bands(channels) is feasible to implement in cuCIM as existing code can handle IFD(Image File Directory)s of TIFF format (each band image is stored in an IFD) and, AFAIK, GeoTIFF is a TIFF format with domain-specific metadata.
It would be a significant effort to implement the full feature of GeoTIFF format. However, we can start providing pixel array data and some important metadata to be used in DeepLearning use cases.
Describe alternatives you've considered
For parsing/recognizing metadata of the GeoTIFF image, we may want to look at GDAL's implementation that is also based on libtiff library, or libgeotiff for a reference?
Additional context

GDAL - GTIFF format

Looks like we may need to handle other metadata such as Color Profile in GeoTIFF format.

Hello Gigony, thanks for putting this together.
We would love to see a GPU solution to load a multiband geotiff file (CUCIM), convert to a CUPY array so we can use CUML for processing, and back to GEOTIFF so we can present it on a map base. @jkatagi did a great job here but he was limited by CPU technology. Still a great piece of code.
I've tested CUCIM with a 700mb-235 bands GEOTTIF on a state if art MacBook pro, taking 5 minutes to load the file against 300us on a GTX 1080 equipped Linux machine. Clear way to go. We are stuck on converting the DNs into a cupy array to do our CUML stuff. Building a constellation of Hyperspectral imaging satellites, we need GPUs manipulating the data from end to end.
https://gist.github.com/jkatagi/a1207eee32463efd06fb57676dcf86c8

Hi @Terra4m, I wonder if there is a sample GEOTIFF file available public that can represent images you are handling. Would like to use it as a reference image to work on.
Looks like what your team is focusing on is loading(tif2array()) part of the code. Is it correct?
I wonder if there are other GEOTIFF-specific metadata that you would like cuCIM to handle.
def tif2array(input_file, calc_gain=True):
    """
    read GeoTiff and convert to numpy.ndarray.
    Inputs:
        input_file (str) : the name of input GeoTiff file.
        calc_gain (bool) : wheter calc GAIN to DN  or not (defaul:True).
    return:
        image(np.array) : image for each bands
        dataset : for gdal's data drive.
    """
    dataset = gdal.Open(input_file, gdal.GA_ReadOnly)
    # Allocate our array using the first band's datatype
    image_datatype = dataset.GetRasterBand(1).DataType
    image = np.zeros((dataset.RasterYSize, dataset.RasterXSize, dataset.RasterCount),
                     dtype=float)

    if calc_gain == True:
        # get gain
        gain = get_gain_band(input_file)

    # Loop over all bands in dataset
    for b in range(dataset.RasterCount):
        # Remember, GDAL index is on 1, but Python is on 0 -- so we add 1 for our GDAL calls
        band = dataset.GetRasterBand(b + 1)
        # Read in the band's data into the third dimension of our array
        if calc_gain == True:
            # calc gain value for each bands
            image[:, :, b] = band.ReadAsArray() * gain[b]
        else:
            image[:, :, b] = band.ReadAsArray()
    return image, dataset

Hello @gigony, to your questions:
1- Yes. Please find here a Hyperspectral Geotiff from Hyperion. 242 bands.
2- Yes, although we are looking for a complete GPU end-to-end pipeline, for now, we are trying to figure out the red arrow A in the attached reference pipeline. To be clear, we need a solution to read the geotiff file in memory into a cupy array through GPU.

3- I will follow up with the metadata soon. That's another beast!

Hi @Terra4m
I checked the file you provided. and it has a single image plane and 242 samples per pixel with no compression used.

Image Width: 1341 Image Length: 3461
Bits/Sample: 16
Sample Format: signed integer
Compression Scheme: None
Photometric Interpretation: min-is-black
Samples/Pixel: 242
Rows/Strip: 1
Planar Configuration: single image plane

TIFF data with no compression is what cuFile (GPUDirectStorage) can help if the image file is in a GDS-supported storage device (NVMe SSD, etc.).
Even with GDS, its performance would not that much. You can expect 15%~30% gain to directly load the 3d array into GPU memory (CuPy Array)
Could you please share your use cases regarding loading GeoTIFF image?
For example.

Reading whole image into the memory and process the image   (usually inference use case when image size fit to the system/GPU  memory)
Reading image patches from random files and image locations into the memory and feed into AI model (usually Training use case)

Looks like the part of the code fits to case 1.
If it is the case of 1), there is also another solution you can exploit -- using tifffile (github). It can load basic GeoTIFF into numpy array and can read basic metadata from GeoTIFF.
Using tifffile and CuPy, I was able to get 10x gain compared with the existing solution (to execute tif2array method)

4.41s (including 1.1s to load/convert whole image to a numpy array. the rest time is for converting to CuPy array and applying gain) vs 48.1s on my local machine[AMD Ryzen 5 3600XT 6-Core Processor 3.80 GHz, GTX 3090] on NVMe SSD )

And I would like to mention that, cuCIM team can schedule to implement the feature that loads GeoTIFF images with multi-samples per pixel or multi bands(channels) and that supports both use cases (reading whole image and part of images). However, the team needs to discuss when we start implement the feature because there are other tasks scheduled (we will publish roadmap once determined). I think you can expect ~15% improvements with GDS for this sample image and ~10% additional gain by using cuCIM instead of tifffile because there is not much optimized regarding loading RAW/uncompressed image data.
Let me show you some example of using tifffile and CuPy for def tif2array(input_file, calc_gain=True).
Hope this helps and please give us your feedback on the use cases and features needed.
from tifffile import TiffFile

tif = TiffFile('hyperion_allbands.tif')

tif.pages[0].dtype
#     dtype('int16')

tif.geotiff_metadata

#    {'KeyDirectoryVersion': 1,
#     'KeyRevision': 1,
#     'KeyRevisionMinor': 0,
#     'GTModelTypeGeoKey': <ModelType.Projected: 1>,
#     'GTRasterTypeGeoKey': <RasterPixel.IsArea: 1>,
#     'GTCitationGeoKey': 'WGS 84 / UTM zone 13N',
#     'GeogCitationGeoKey': 'WGS 84',
#     'GeogAngularUnitsGeoKey': <Angular.Degree: 9102>,
#     'ProjectedCSTypeGeoKey': <PCS.WGS84_UTM_zone_13N: 32613>,
#     'ProjLinearUnitsGeoKey': <Linear.Meter: 9001>,
#     'ModelPixelScale': [29.977628635346758, 29.991331984975442, 0.0],
#     'ModelTiepoint': [0.0, 0.0, 0.0, 405600.0, 6018600.0, 0.0]}

Using CuPy + tifffile
from tifffile import TiffFile
import cupy as cp

def get_gain_band_cupy(input_file):
    """
    Dummy method
    """
    return cp.ones((242,)) * 1.15

def tif2array_cupy(input_file, calc_gain=True):
    """
    read GeoTiff and convert to numpy.ndarray.
    Inputs:
        input_file (str) : the name of input GeoTiff file.
        calc_gain (bool) : wheter calc GAIN to DN  or not (defaul:True).
    return:
        image(np.array) : image for each bands
        dataset : for gdal's data drive.
    """
    tif = TiffFile(input_file)
    tif_image = tif.asarray() # 'maxworkers' parameter is None which means 'up to half the CPU cores are used.'

    image = cp.asarray(tif_image)
    raster_count = image.shape[2] # (Y, X, S)

    if calc_gain == True:
        # get gain
        gain = get_gain_band_cupy(input_file)

    image = image * gain
    return image, tif

%%timeit
out_image, tif = tif2array_cupy('hyperion_allbands.tif')

del out_image
tif.close()

4.41 s ± 289 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Using existing code using GDAL
import numpy as np
from osgeo import gdal
from osgeo import gdal_array
from osgeo import osr

def get_gain_band(input_file):
    """
    Dummy method
    """
    return [1.15] * 242

def tif2array(input_file, calc_gain=True):
    """
    read GeoTiff and convert to numpy.ndarray.
    Inputs:
        input_file (str) : the name of input GeoTiff file.
        calc_gain (bool) : wheter calc GAIN to DN  or not (defaul:True).
    return:
        image(np.array) : image for each bands
        dataset : for gdal's data drive.
    """
    dataset = gdal.Open(input_file, gdal.GA_ReadOnly)
    # Allocate our array using the first band's datatype
    image_datatype = dataset.GetRasterBand(1).DataType
    image = np.zeros((dataset.RasterYSize, dataset.RasterXSize, dataset.RasterCount),
                     dtype=float)

    if calc_gain == True:
        # get gain
        gain = get_gain_band(input_file)

    # Loop over all bands in dataset
    for b in range(dataset.RasterCount):
        # Remember, GDAL index is on 1, but Python is on 0 -- so we add 1 for our GDAL calls
        band = dataset.GetRasterBand(b + 1)
        # Read in the band's data into the third dimension of our array
        if calc_gain == True:
            # calc gain value for each bands
            image[:, :, b] = band.ReadAsArray() * gain[b]
        else:
            image[:, :, b] = band.ReadAsArray()
    return image, dataset

%%timeit

out_image, dataset = tif2array('hyperion_allbands.tif')
del out_image
del dataset

48.1 s ± 561 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)


Wifi Passwords List is broken with v2.2.0 Pre-Release

Request for this issue to be closed. @Cryptiiiic


Modify Markdown cell while code cell is running

Issue Type: Bug
Is this possible to do? While one code cell is running that I modify the markdown cell.
Azure Data Studio version: azuredatastudio-insiders 1.9.0-insider (449cd9ea27d49b66667bda043d467adb9377b734, 2019-06-17T07:46:28.633Z)
OS version: Windows_NT x64 10.0.17763

System Info

Item
Value

CPUs
Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz (8 x 3591)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwarerasterization: enabledsurface_synchronization: enabled_onvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
31.93GB (21.43GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (2)

Extension
Author (truncated)
Version

sql-vnext
Mic
0.13.2

azdata-sanddance
msr
1.2.0

@ronychatterjee what happens if you try to do this today?
Also, a customer scenario would be fantastic 😄

@chlafreniere it stops me from doing anything as the cell is running.
Also when a cell is running for 30 mins you might want to quickly change something in the markdown cells. Perfect use case.

Closing it as it seems to be running fine on Windows.


Skip batch tests if API credentials are not available.

We want to make sure tests are runnable locally even if someone doesn't have access to indico API credentials.

Merged in with #10


HSEARCH-2253 Remove ID sort workaround using "_uid" in Elasticsearch

This PR is based on #1192 (HSEARCH-2396), which will have to be merged first.
Relevant JIRA ticket: https://hibernate.atlassian.net/browse/HSEARCH-2253

Nice! Good to go once the pending one is merged.


icon can`t be added on the map

Describe the bug
i cant add icon layer;i cant modify points layer with image
To Reproduce
<template>

</template>

Expected behavior
it does`t work ,please resolve it,please

You'll need to provide a better description than just "it doesn't work". What does not work? Do you get any errors on the console? Ideally, provide a minimal application that exposes the issue.

You'll need to provide a better description than just "it doesn't work". What does not work? Do you get any errors on the console? Ideally, provide a minimal application that exposes the issue.

I hope I can add image annotation to point elements, but after using setstyle, all the previous points disappear

You'll need to provide a better description than just "it doesn't work". What does not work? Do you get any errors on the console? Ideally, provide a minimal application that exposes the issue.

And there was no mistake

Without seeing your code, or better a minimal example that exposes the problem, we cannot help you.

Without seeing your code, or better a minimal example that exposes the problem, we cannot help you.

I've given the full code，thank you

<template>

</template>

I replaced ol with openlayers, but it still didn't work

You are using the deprecated openlayers package. Use the ol package instead.

You are using the deprecated openlayers package. Use the ol package instead.

I just want to show that openlayers can't do either. I've tried ol, but it can't. The example of ol is at the top

You still owe us a better description than just "it does not work". And the code you provided is not executable without the rest of your project.

package.json
{
  "name": "vue-openlayers",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "serve": "vue-cli-service serve",
    "build": "vue-cli-service build",
    "lint": "vue-cli-service lint"
  },
  "dependencies": {
    "core-js": "^3.6.4",
    "ol": "^6.5.0",
    "openlayers": "^4.6.5",
    "vue": "^2.6.11"
  },
  "devDependencies": {
    "@vue/cli-plugin-babel": "~4.3.0",
    "@vue/cli-plugin-eslint": "~4.3.0",
    "@vue/cli-service": "~4.3.0",
    "babel-eslint": "^10.1.0",
    "eslint": "^6.7.2",
    "eslint-plugin-vue": "^6.2.2",
    "vue-template-compiler": "^2.6.11"
  },
  "eslintConfig": {
    "root": true,
    "env": {
      "node": true
    },
    "extends": [
      "plugin:vue/essential",
      "eslint:recommended"
    ],
    "parserOptions": {
      "parser": "babel-eslint"
    },
    "rules": {}
  },
  "browserslist": [
    "> 1%",
    "last 2 versions",
    "not dead"
  ]
}

And what exactly does not work? What were you expecting to happen, what did happen instead? You're saying "all previous points disappear", but your code only creates one point.


Consider replacing EthAddress with address from Viem.

Due to Viem we now have 2 different ethereum address types. This leads to unnecessary conversions. Consider dropping our custom EthAddress type.

Impractical because all of our code expects the type to have toBuffer and fromBuffer methods.


add logic for  jdk-version specific suites

https://github.com/adoptium/aqa-tests/pull/5553#issuecomment-2340815163 have reminded me old issue, where some external tests, eg jacoco, works fine with 8-17 but do not work with jdk21
Unluckily, current logic to clone and checkout is in https://github.com/adoptium/aqa-tests/blob/master/external/jacoco/test.properties
So I see several  options with many pros and cons:

narrow the checkout in test.sh ( I do not like this)
have duplicated testfolder (so jacoco target would be limited for jdk21 onwards, and will have updated tag, and new jacoco17 wold be limited to 17 and older)
create flexible checkout logic in properties file similar to what I did for os-based  properties (I'm not sure I like this)

tag_version -> tag(7|8)_version  and  tag(21|22)_version ....
this is less expendable to other non-external  test

This will be quite a precedent for other - at least external - tests
Many others maybe have to be simply disabled for older/newer jdks, but that is easily done in playlists?

Thanks for raising this issue @judovana.  Before any option is chosen, it will be good to create a table of external apps, their versions and which ones work with which Java version.  That will be useful to visualize and help inform the best way forward.

Thanks for raising this issue @judovana. Before any option is chosen, it will be good to create a table of external apps, their versions and which ones work with which Java version. That will be useful to visualize and help inform the best way forward.

thats terrible task.... I would much rather go one by one:(


Google Cloud Shell: since @google/clasp v2.4.0 "clasp push -w" crashes

DOES WORK:
npm i<EMAIL_ADDRESS>-g
clasp push -w
DOES NOT WORK:
npm i @google/clasp -g
clasp push -w
ERROR
/usr/local/nvm/versions/node/v12.14.1/lib/node_modules/@google/clasp/build/src/index.js:2
import why from 'wtfnode'; // should be your first require
^^^^^^
SyntaxError: Cannot use import statement outside a module
at Module._compile (internal/modules/cjs/loader.js:891:18)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
at Module.load (internal/modules/cjs/loader.js:811:32)
at Function.Module._load (internal/modules/cjs/loader.js:723:14)
at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)
at internal/main/run_main_module.js:17:11
SPECIFICATIONS
Google Cloud Shell, bash terminal
NPM v8.5.0
Node v12.14.1

Same issue any update?

Updating to the lastested node version solved the issue.
I updated from 10 to 14.
Google docs says its wotk from node version 4.
Please update

My cloud shell currently uses node v16.4.0 which seems to be the latest LTS version.
Now there is a new error when using the following commands:
npm i @google/clasp -g
clasp push -w

Error:
drive.google.com ✖
Error: Looks like you are offline.

Seems to be fixed in the meantime.


Support latest monaco version

Fixes #23

Anmy change to get this merged?

@china-bin,
@tanvesh01, @Ethan-Arrowood, @yarsky-tgz, @MartinBuessemeyer, @zlyyyy, @mattcroat, @tarnishablec
Can anyone please merge this PR and make a release?
Or may be someone knows how to contact with the maintainer?

@antfu, @Scrum - the maintainers of https://github.com/vitejs/awesome-vite where listed all popular plugins for vite (including this one)
Can you please help us to contact the maintainers?
Or may be you know another ways... :(


Content-Type application/x-www-form-urlencoded by default

Hi all!
I updated from 1.14.0 to 1.15.0 and had some issues with all my requests getting broken.
Before I was just doing very basic JSON PUT requests, like this
val (request, response, result) = "http://myserver.local/api/v1/test".httpPut()
                .header("Content-Type" to "application/json")
                .body("{"id":"test","somemorefields":"andvalues", ...")
                .response()

with the latest Update to 1.15, this doesn't work anymore. The Content-Type being sent is then "application/x-www-form-urlencoded; application/json" and my server rejects this.
Is this a nice default behaviour? I wouldn't expect a default header to be set and my explicitly defined headers sent on the second position only (which makes everything fail now). I guess this whole implementation changed with https://github.com/kittinunf/Fuel/pull/393...
Sidenote for anyone with the same problem: I figured out that the headers can be reset by wrapping them in a map
... .header(mapOf("Content-Type" to "application/json"))...

Kind Regards
Matthias

Also just scratched my head for a good while today on this why our back end was sending 500's to the app.

Definitely confusing

@ftomassetti @RingoMckraken @mmuth I have just merged in a method that sets the header
when released
"http://myserver.local/api/v1/test".httpPut()
                .header("Content-Type" to "application/json")
                .body("{"id":"test","somemorefields":"andvalues", ...")
                .response()

becomes
"http://myserver.local/api/v1/test".httpPut()
                .jsonBody("{"id":"test","somemorefields":"andvalues", ...")
                .response()

Right on, I'll give it a spin, thanks!

Looks like Request.jsonBody is documented on the README.md, but this is not yet released correct?

True


Bug: Running Tools in parallel within LangGraph returns a "does not match existing tool call" error

I am running into an issue with parallel executions of tools within LangGraph.
You can easily reproduce the issue by running the following graph:
https://github.com/langchain-ai/langchain-academy/tree/main/module-6/deployment
Using this graph, when sending a prompt like "My name is John," it will send you to the update_profile node, which uses Trustcall to update a JSON payload in its memory to store some information about the user. Trustcall runs a Tool called PatchDoc, but the update_profile also runs a Profile Tool.
Doing so raises this error:
Error: Tool call name call_NAnT466ldLHgmgBdeA3xoS6E Profile does not match existing tool call PatchDoc

The issue seems to be within the joinExternalMessages function on the file runtimes/external-store/external-message-converter.tsx that does not expect to receive messages from 2 different tools before one is completed.

Thank you for reporting this! I am able to reproduce this issue and am working on a fix


Azure DevOps Additon

Is your feature request related to a problem? Please describe.
We need to investigate the addition of support for Azure DevOps Git repositories.
Describe the solution you'd like
Access for teams/users to be able to utilize Azure DevOps repo's in their workflow, especially for larger teams over different locations.
@ivan-burazin as requested.

/bounty $200

Because this issue is of a much broader scope than the usual bounties in our repository. We might assign collaborators that we feel should tackle solving it.
That being said, to get assigned, we would require that the collaborator shows us they understand the scope of the issue and a plan on how to implement certain Git Provider methods using the Azure DevOps API.
It is not a requirement to get assigned to the issue to start working on it, but if we end up assigning someone, we expect collaborators to respect that.

/attempt #460
Hey @Tpuljak, I'm planning to use the Azure DevOps Go SDK for this. I've already implemented two methods and will complete the remaining methods soon.

@Tpuljak  adding myself in queue..


Update index.js

Level of documentation SidebarLink is fixed

Hello @rad10wave thank you for working on this but you need to first comment on the issue to be assigned, as someone else did and now they're assigned. This helps avoiding confusion and making sure your work is merged to the repository. Thank you.

oh I am sorry I am new to this. I will take care from next time.
Thank you !
On Mon, Mar 22, 2021 at 5:28 PM Shahed Nasser @.***>
wrote:

Hello @rad10wave https://github.com/rad10wave thank you for working on
this but you need to first comment on the issue to be assigned, as someone
else did and now they're assigned. This helps avoiding confusion and making
sure your work is merged to the repository. Thank you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sButtons/sbuttons/pull/1362#issuecomment-804006857,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJ7IV3S5IFAZYMOJKVSYNGTTE4WEVANCNFSM4ZS4TPKQ
.


Constant refreshing of server in Desktop App v3.7 after changing settings

I confirm (by marking "x" in the [ ] below):

[x ] This is not a troubleshooting question. Troubleshooting questions go here: http://www.mattermost.org/troubleshoot/.
[x ] This doesn't reproduce on web browsers (such as in Chrome). If it does, issue reports go to the Mattermost Server repository.
[x ] I have read contributing guidelines.

Summary
Constant refreshing of server in Desktop App v3.7 after changing settings
Steps to reproduce

Click on "File" > "Settings"
Scroll down and change any setting (which is auto-saved)
Click the "X" at the top right of the screen to close the settings menu

Expected behavior

Be taken back to the server tab I was on and continue working.

Observed behavior

Server constantly refreshes (I will DM the screen capture to you - .mp4 not supported on GitHub) with errors in the dev tools (see attached screenshot)

Possible fixes
Sorry, no idea...

The message is transferred from webview. So it seems that the problem happens in webapp itself.
Would you try clearing caches(Ctrl+Shift+R)? Somehow webapp failed to communicate with the server. But it looks strange because getPostsPage returns status 404. I worry that there is mismatch between the webapp and the server.

Sorry @yuya-oc,
Forgot to include in the original report that I did clear cache and refresh and the issue still persisted. Tried a few times with the repro steps and it was the same each time.
It does not repro on the browser...

Yuya added a workaround for it in the troubleshooting section: https://github.com/mattermost/docs/pull/1163
We can increase priority of this issue if we hear more reports about it, but assigning to Unscheduled for now.

Thanks Jason, I'll close off this issue for now as we have a workaround in the docs.  If we hear more reports, we can reopen.


Clarify license

jquery-tmpl was dual-licensed via the MIT and GPL2 licenses. I'm guessing that your code is also, but I think it would be good to clarify this point. Will you add a license to the code?

The info has now been added...


one bundle per view, named entry points?

Hey,
I'm trying to serve one bundle per view file. My layout file yields to :javascript_includes, and in each view I call
content_for :javascript_includes do
   <%= javascript_include_tag 'bundleName-bundle' %>
end

and of course:
react_component("BundleNameComponent", props: @props)

For my webpack config, I have changed entry to an object, like so:
  entry: {
    bundleName: './app/bundles/bundleName/component.jsx',
  }, /...

(which means I've removed the defaults - not entirely sure if this may be the problem.. How should I reintroduce them?)
so that I can then set my output entry like:
  output: {
    filename: '[name]-bundle.js',
    path: '../app/assets/webpack',
  },

The components being set as entry points contain JSX, and at the end register with ReactOnRails:
ReactOnRails.register({ BundleComponentName });

and serve separate bundles. This works great in Chrome, but seems to be flaky in Firefox. If I make irrelevant trivial changes to the view, I get unhelpful React error messages that may or may not relate to Context. I can expand on this if needed, but have a gut feeling it's to do with my entry points.
Were the default entry points needed for Firefox to behave itself? How can I reintroduce them and serve multiple bundles like this? Am I way off track? The error is really frustrating as it seems to go away under certain circumstances - reinstalling NPM modules, restarting the development machine etc. Any guidance would be much appreciated.

Ah, maybe my gut feeling is wrong on this. I have included a named bundle for the shims, like:
 shims: [
      'es5-shim/es5-shim',
      'es5-shim/es5-sham',
      'babel-polyfill'
    ]

and have included this in my layout file. No change in behaviour in Firefox.

Do you still have an issue, @ttrmw ?

Yes I do, but it's hard for me to reproduce and may be specifically to do with the NPM module I'm including, and not react_on_rails. I'm gonna close this for now.


[ENH-343] Raised up baseline to 1.1.0

enhancement #343

Sorry, I didn't see your comment before starting the review.

Baseline should be correct now.


New consuming segment becomes visible to queries too late

While debugging this issue: https://github.com/apache/pinot/issues/11948, we found that the server could start the new consuming segment before the broker put the new segment into routing table, so queries continued to access the older segments whose records were already invalidated by new data ingested by new consuming segment, so returned smaller values for count(*) queries.
e.g. logs from a test setup with the RealtimeQuickStart but modified to create segments more frequently and send queries in a tight loop to expose the issue. Watch out the timelines:

server started to consume data, so it started to invalidate records in older segments
count(*) queries were routed to the server
broker was told to update routing table for dips__0__5__20231107T2314Z, but too late. As the new records in new segment were not visible to queries yet, the count(*) queries returned smaller values than expected.

...

2023/11/07 15:14:04.247 INFO [RealtimeSegmentDataManager_dips__0__5__20231107T2314Z] [dips__0__5__20231107T2314Z] Starting consumption loop start offset 5000, finalOffset null

...

2023/11/07 15:14:04.366 INFO [ServerQueryLogger] [pqr-4] Processed requestId=72826337000000804,table=dips_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value
)=5/5/2/-1/0/0/0/0/0,schedulerWaitMs=0,reqDeserMs=1,totalExecMs=0,resSerMs=0,totalTimeMs=1,minConsumingFreshnessMs=1699398843946,broker=Broker_<IP_ADDRESS>_8000,numDocsScanned=2000,scanInFilter=0,scanPostFilter=0,sched=FC
FS,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0

...

2023/11/07 15:14:04.483 INFO [PinotHelixResourceManager] [grizzly-http-server-24] Sent 1 segment refresh messages to brokers for segment: dips__0__5__20231107T2314Z of table: dips_REALTIME

...

the support of 'optional segments' would help solve this issue. PR landed, and closing this issue for now.


How can I switch active tabs based on window width?

I can't quite grasp how to switch from one tab to another based on window width.
ie,
<tabset>
    <tab>
        <tab-heading>One</tab-heading>
        ONE
    </tab>
    <tab>
        <tab-heading>TWO</tab-heading>
        TWO
    </tab>
</tabset>

When the window width is at least 900px, it should auto-switch to the first tab.
When the window width is less than 900px, it should auto-switch to the second tab.
But once the switch happens, the user should still be able to access the other tab (it should not lock a user into a particular tab).
I tried wrapping the <tabset> in a directive, and then manipulating the active class on the elements but angular-ui does not register this change, so it ends up having tabs with multiple active classes.

The issue tracker is for bugs and issues, not implementation questions. Those can be asked on StackOverflow.


Vulnerability roundup 111: slic3r-1.3.0: 2 advisories [5.5]

search, files

[ ] CVE-2021-45846 CVSSv3=5.5 (nixos-21.11, nixos-unstable)
[ ] CVE-2021-45847 CVSSv3=5.5 (nixos-21.11, nixos-unstable)

CVE details
CVE-2021-45846
A flaw in the AMF parser of Slic3r libslic3r 1.3.0 allows an attacker to cause an application crash using a crafted AMF document, where a metadata tag lacks a "type" attribute.
CVE-2021-45847
Several missing input validations in the 3MF parser component of Slic3r libslic3r 1.3.0 can each allow an attacker to cause an application crash using a crafted 3MF input file.

Scanned versions: nixos-21.11: 2128d0aa28e; nixos-unstable: 5aaed40d22f.
Cc @bjornfor

Still unfixed upstream
https://github.com/slic3r/Slic3r/issues/5117
https://github.com/slic3r/Slic3r/issues/5118
https://github.com/slic3r/Slic3r/issues/5119
https://github.com/slic3r/Slic3r/issues/5120

I think we should remove Slic3r and use PrusaSlicer instead: https://github.com/slic3r/Slic3r/issues/5175


Write documentation

[ ] How to setup OAuth token
[ ] How to develop locally
[ ] How to deploy internally
[ ] Link to public site
[ ] License and privacy policy
[ ] Attribute external resources

Won't fix as the project is deprecated


Dots in parameters get converted to underscores because of parse_str

I've banged my head against the wall over this for the past hour until I looked at curl's debug output and really paid attention and noticed that https://api.cloudflare.com/client/v4/zones?account.id=... becomes https://api.cloudflare.com/client/v4/zones?account_id=..., so Cloudflare is correct in ignoring my filtering and I'm lucky I haven't provided some very, very constructive feedback on their API documentation yet.
In the trait ManagesPsrRequests there's
        parse_str($uri->getQuery(), $existingQuery);
        return $uri->withQuery(
            http_build_query(array_merge($existingQuery, $this->query()->all()))
        );

and the docs confirm:

Because variables in PHP can't have dots and spaces in their names, those are converted to underscores. Same applies to naming of respective key names in case of using this function with result parameter.

I've found that I can do
    public function query() : ArrayStoreContract
    {
        $query = new ArrayStore();
        $query->set([ "account.id" => $this->accountId ]);

        return $query;
    }

in the Request as a workaround, but that feels a lot more clunky than just having
    public function resolveEndpoint() : string
    {
        return "/zones?account.id={$this->accountId}";
    }

Guzzle does some explode-based parsing (GuzzleHttp\Psr7\Uri::getFilteredQueryString) which looks a little sketchy, but it should work just as well. parse_str doesn't support ; as a delimiter either, so I assume it should be fine to switch to something like that unless there's a conscious decision behind using parse_str that I'm not aware of.

Hey @jhard
Thank you for making me aware of this issue and I'm sorry for your frustration that this caused you! Does this still happen when you use the defaultQuery method on the connector/request?
This is the recommended way to add query parameters:
class YourRequest extends Request
{
    // ...

    protected function defaultQuery(): array
    {
        return [
             'account.id' => $this->accountId,
        ];
    }
}

However, I do support putting query parameters in the URL - so I'm going to look at a much better query string parser! Should have a PR up soon.

Do you mind having a quick look at this PR please @jhard? https://github.com/saloonphp/saloon/pull/370 Just want someone to double-check the new implementation. I think it's a bit better than the explode-based passing as if someone wrote a query parameter like foo==bar it will still be parsed as foo=bar.

Hey  @Sammyjo20 thank you! No worries, it hasn't been the first time I've bashed my head against the wall yesterday, and I'm sure it won't be the last this week. I have a special wall-finding talent and I suck at finding openings in them.
I can confirm that it works with defaultQuery.
#370 looks good in general, but it needs to decode the names and values, i.e.
            $name  = urldecode((string) strtok($parameter, '='));
            $value = urldecode((string) strtok('='));

so url-encoded characters in won't be double-escaped (i.e. %40 => %2540) when http_build_query reassembles the query string. I've added string casting in case strtok returns false (e.g. on ?value=).
I'd also add
            if ( str_starts_with($parameter, "=") || ! $name ) {
                continue;
            }

to handle cases like ?=abc and ?a=d& like parse_str would, i.e. no empty names allowed. Currently, it will turn =abc into ["abc" => ""] and a=b& into ["a" => "b", "" => ""]. The latter shouldn't really cause a problem unless someone on the other receiving end parses it similarly and trips over it, but it's probably best to not send it in the first place.

No, I understand that's equivalent. They are meant to be key-value pairs, and having only a key is shorthand for having a key with an empty value, and they are always string values, there's no way to differentiate between "" and null, except not sending the key at all for null.
Someone might do some custom parsing, but I believe pretty much all frameworks / languages these day are aligned on how to understand them. RFC6570 on the URI Template suggests that it's how you assign an empty value, but mentions ;empty for the semicolon-separated version (which are deprecated and not supported by http_build_query) in Level 3 templates:

{;x,y}                ;x=1024;y=768
{;x,y,empty}          ;x=1024;y=768;empty

{?x,y}                ?x=1024&y=768
{?x,y,empty}          ?x=1024&y=768&empty=

The issue should be fixed in the next release - let me know if not!


ServiceProvider does not register automatically with Laravel 5.5 and up

Readme is not correct currently. Need to adapt composer file.

Fixed in release 1.0.7.


Advanced Peripherals stuff

I've set up tentative initial versions of the Advanced Peripherals' stuff. I've pretty much exhausted my creativity for now, but I do think it can be further enhanced! A few comments:

When I couldn't directly reference something in Create, I tried going for a steampunk aesthetic. See the Computer Scientist (which now has goggle-like glasses and leather+brass trimming to their overcoat) and the Memory Card (which was inspired by those nixie tubes containing LEDs), for example.
I'm not entirely satisfied with everything. On the subject of the Memory Card, for one, even though I DO think it looks neat, it's not "on brand", as it were. It doesn't look conceptually like the awesome floppy disks, for example.
I intended to turn the AR glasses into goggles, but... They've been deactivated in 1.19. No way of testing them for now.

Well, that's all for now. Hope this can be at least a starting point for these items in the project!

I tested the pack and the edits are great!
I really like the look of the memory card, you did a great job here!
I like the fact that you keep the color and feel of the default textures, which is a plus for users who knows the mod :+1:
I was looking after adding detailed models, but I think it could become laggy really quickly...

I think we can do something like an engine for the automa core.
For the end automa core, we could add a bottle with a liquid looking like ender pearl.
For the husbandry automa core, I was thinking about a lead or something like that...

Hey, I'm glad you liked it!! I was really proud about the memory card, just a bit bummed out that I couldn't figure out a way to make it... more dynamic, I think? Meh, I'm too critic of myself sometimes 😅
So, on to your points:

I like the fact that you keep the color and feel of the default textures, which is a plus for users who knows the mod 👍

Yeah, I thought that was pretty important for consistency sake! Curiously, though, the one you used for the bas-relief test was the only one I DID change. It didn't make much sense to me that something interacting with redstone was... Yellow. But we already had a red component, so I took a page from the redstone repeater and made it white/gray and red 😅 It's not consistent with AP itself, but it's consistent with the base game, which I think is a plus!

I was looking after adding detailed models, but I think it could become laggy really quickly...

Hmmmm, I might be entirely off-base here, but take a look at Valhelsia and MrCrayfish's furniture mods. They are comparatively "high-poly", and don't, as far as I can tell, make the game too much laggier. And I'm playing on a glorified potato, so I should know 🤣 I don't think AP contraptions will need as many blocks as a well-decorated living hall, so I'm pretty sure you'd be safe here, especially if you just want to give stuff some bas-relief, as show above. Besides, they look neat! If you want to err on the side of caution, you could offer two versions of the resourcepack - one with the more detailed models, and the other with just the new textures applied to base models.

I think we can do something like an engine for the automa core.

Hey, that's not a bad idea. I must confess, I'm a little burnt out for now (seriously, I was able to do stuff this fast because I was having an anxiety crisis and had to focus on something 😅), but I'll try thinking of something! If you have something on your mind, don't wait out on me, though.

Also for the wrench we could do something looking like the create wrench in 3D, with a feeling of precision, I don't know if this make sens.

It makes a lot of sense! I just don't know if it's worth the hassle for now, though, the computer tool is currently useless!

For the detailed models, I'll try and test the difference with and without the pack for the lag issue - but I think that my PC can win any potato PC competition.
If the model don't add 10 fps of lag for something like 100 or 1000 blocks I'll go for it.
Is it an issue for you if I commit directly in your branch?
It seems like you used the Github interface to edit the pack :thinking:

Go for it! Yep, I used the interface, tbh I don't know any other way to do it 😅


Put an array of all the color names in ColorUtils

So we (and extension developers) can reuse them whereever we want.
The actual regex is unchanged, it's just getting generated now.

I like this method better than stuffing them in the regex, but I'll need to check what changes I need to make to the Inline Color Editor to use this. :+1:

They are still in the regex, so you don't have to change anything.

This looks good. I agree that rebeccapurple should be added to the list.

@redmunds I will close #10252 in favor of this.

I think we need #9596 first for rebeccapurple to work properly, right.
I don't know what happens right now if you try to open an color inline editor on rebeccapurple, though.

Added support for rebeccapurple (and changed comments in order to match the color names included).
Still, the Color Inline Editor doesn't work too well with rebeccapurple (see https://github.com/adobe/brackets/issues/10250#issuecomment-67748764) (Quick View works perfectly fine, btw), but imho we can still merge it as there won't be many out there wanting to edit rebeccapurple anyway, I guess.
Sooner or later, we should, of course, merge #9596 for full support.

@MarcelGerber I am going to try to finish up #9596 today.

@redmunds While you're online, can you look into merging this too so we can finish up rebeccapurple support?

Looks good. Merging.


Windows native client has bad anti-aliasing

Steps to reproduce
Open Windows native client and see
Expected behavior
Fonts have a good quality
Observed behavior (that appears unintentional)

Left: Firefox has good anti-aliasing.
Right: Windows native client has bad anti-aliasing at the same system.
Font: Open Sans.
Seems like there are different anti-aliasing algorithms are enforced. At least.
Left is Firefox.
Right is Mattermost Windows Native client.

Possible fixes
CSS anti-aliasing settings?

What version are you using? We know v3.6.0 had a rendering issue, and believe that it's improved in v3.7.0.
If you still feel it's not good, would you compare to Chrome? The rendering engine is same. So if they have different result, we might be able to consider something.

I'm using v.3.7.0

Chrome looks better.
Left: Chome
Right: Windows native app

Result of comparison is the same as in above comments.

Confirmed on Windows 10 with 100% of DPI scaling. But it looks fine in other DPIs like 125%.
It happens when using a traditional lower DPI and looks Electron's limitation.
https://github.com/electron/electron/issues/8708

@yuya-oc, Mybe it's better that in previous version, but I have 125% DPI scaling by default, and I find it blurry as well as for 100% DPI, especially when you watch it right after browser version.
FYI, Visual Studio Code is on Electron and it has a good anti-aliasing with algorithm like in my examples at the left side. There are lot of streams in github about their struggling with anti-aliasing: maybe soem information will be helpful for you.

https://github.com/electron/electron/issues/8708 is solved, whether the latest version of Electron is used in your desktop client?

P.S.: When I enable display mode to show only one display, restart Mattermost, but it's still grayscale anti-aliased in Windows.

@fadeevab The next version, v3.8.0 uses newer Electron that supports different DPI. However it would not resolve the original problem because their reasons are different.

Font rendering: DirectWrite which is used in Chrome internal.
Different DPI: Missing Per monitor DPI awareness in application manifest. (see also #357)

For reference, would you tell me the issue you saw of VSCode?

Let me check wait a minute

The nuance is that VS Code doesn't have an issue with anti-aliasing at the same Windows at the same time.

@yuya-oc Does this still appear to be an issue with Electron?

@jasonblais I couldn't find other open issues. But as far as looking the conclusion of electron/electron#8708, I felt they think it's not fixable.

@yuya-oc, How do you explain, that VSCode has proper subpixel anti-aliasing, not grayscale (my latest screenshot)?

@fadeevab I'm worried because I'm not sure why the difference appears. As you said "struggling", it seems not easy.
I tried adding CSS like https://github.com/Microsoft/vscode/issues/1896#issuecomment-315463143 by using DevTools console. But I couldn't get better anti-aliasing.
* {
  -webkit-font-smoothing: antialiased !important;
  text-rendering: optimizeLegibility !important;
}

Left: Chrome, Right: Mattermost Desktop

The link below is a minimum implementation of mattermost desktop which uses the latest Electron 1.7.4 (the master branch uses 1.6.10) and CSS insertion like above.
https://www.dropbox.com/s/jcan9895e5bqap1/mattermost-minimum-win32-ia32.zip?dl=0
Wondering if you could test this application and try modifying CSS by using DevTools(Ctrl+Shift+I).

I did everything and more with CSS, nothing help.
I tried to play with basic Electron Demo CSS by toggling developer tools, and I can't make to force subpixel-antialiasing. Electron Demo has grayscale anti-aliasing from the scratch (at least in Windows).
But if to download chromium and open index.html with <body>test</body> there is a subpixel antialiasing.
So I suppose VSCode team fought with Electron and found a workaround.
More info why anti-aliasing could be broken: https://github.com/electron/electron/issues/6344#issuecomment-230773130.
Anti-aliasing could be turned into grayscale by a couple of reasons. One of them is a 3d transform. I just saw in VSCode lines like:
    if (canUseTranslate3d() && !isWindows /* Windows: translate3d breaks subpixel-antialias (ClearType) unless a background is defined */) {
	const transform = `translate3d(0px, -${renderTop}px, 0px)`;
	this.rowsContainer.style.transform = transform;
	this.rowsContainer.style.webkitTransform = transform;
    } else {
        this.rowsContainer.style.top = `-${renderTop}px`;
    }

See https://github.com/Microsoft/vscode/blob/fcd9a8996aca402aa186ad18e5d356ddf0c53e2f/src/vs/base/browser/ui/list/listView.ts.
So, they care about 3d transfrom to not break subpixel-antialiasing.
I'm not sure about overall Mattermost architecture, but I saw some 3d transform in Platform. Again, I'm not really sure still.
To continue need to force subpixel-antialiasing in the simplest Electron demo, and need to be sure there is no side-effects from the code. Not

(I sometimes press this ridiculous "Close and comment" button)

@fadeevab Thanks for more detailed feedback! It looks the problem is grayscale anti-aliasing described in electron/electron#6344. When I set backgroundColor for Electron's BrowserWindow, I got good anti-aliasing as well as Chrome.
Actually our app uses <webview> to show mattermost/platform, so we need to set backgroundColor as well. But it's not exposed in Electron. So I added a issue. electron/electron#10025

Great! I saw this point too, but I set just basic background in CSS and it didn't help. Great you managed to deal with it.

Will you add workaround to mattermost or wait for a fix in electron?

Unfortunately there is no workaround we can do in our code. Only Electron's native part can fix this.

I got it. Let's wait than, thank you :)

It caused by setting the   transparent:true !!!!  I have a greate font appr now

Great!

@YangShengHaHa I think transparent:true is not used in the app though, how did you apply your fix? I'm wondering if you can make a PR.

I'm also interested in a fix for this as well. For me, I avoid using the Windows client because of the lack of focus when I view the text. Is there a solution for this issue? If not, do you know if Electron is actively working to fix it? Thank you!

Hi @6a7070, thanks for your interest. As I wrote at electron/electron#10025, we can know why bad anti-aliasing happens.
To solve this, we need to improve Electron source code. But I haven't ever heard Electron team is working for this.

On a side note, @YangShengHaHa we'd be curious to hear how you resolved (where did you set transparent:true)? That might benefit others in the community as well if you were able to find a fix for the problem.

@jasonblais @yuya-oc when you create your BrowserWindow , set transparent:false in then option of then BrowserWindow constructor.

Hi @YangShengHaHa, unfortunately I could not find difference even when using transparent: false at the current master branch. In the first place, the default value of transparent is false, though.
I think we need to set transparent: false to <webview> not only for the main BrowserWindow because their webContents are independent if transparent: false truly solves the problem.
I'm wondering if you could show difference as screenshots.

Hey guys, AFAIK, to subpixel (good) anti-aliasing need to set backgroundColor to white in some native code, not transparent: "BrowserWindow shows different rendering when using backgroundColor: '#FFF' in the constructor."

@yuya-oc I can not take picture for our project code because of the security policy, you can write a simple demo and you will get the answer. I consider wherever this is one the cause of font anti...  We have the best practice WeLink for electron from Huawei inc. Setting the BrowserWindow transparent to false in main.js resolves our font problem in WeLink ^_^

Thanks for all the feedback. I'm still unclear how this might be fixed. Is this something I would need to configure in my own compiled version of the desktop client? I find that I don't use the desktop client because of this issue. Is there anything I can do to help test?

In normal Electron apps, @YangShengHaHa's approach would be good to fix the problem. It's to create non-transparent BrowserWindow in main.js. However it's not enough for us.
To show Mattermost webapp, we are using <webview> tag. It has its own renderer process (including background color) apart from the main window. So we need to set non-transparent background for each <webview>, but there are no such APIs in <webview> or its webContents object.
So in order to solve the problem, we need to make a improvement for Electron APIs.

Ticket for tracking: https://mattermost.atlassian.net/browse/MM-14133

Closing old issue, looks like this is working fine. Happy to re-open if we think this is still a problem.


AB#9267 Update readme

@arsforza I don't know if you've come across the issue where you needed to rebuild node-sass?
But maybe you can also include that in the list of "known issues/fixes/workarounds":
So it could help to run npm rebuild node-sass (from the directory of an interface), to fix some node-sass-errors.
(But I haven't been able to reproduce the situation on my machine)
Also, I would recommend not to repeat commands to do something in that README-section, but rather link to the related steps in the "Getting Started"-section. That way we only have to keep 1 place up-to-date.


Queue bulk subscribe read

This PR refactors the way queue subscriptions are handled by servers and clients to make them more efficient.
Previously, clients (RTSs) had to individually subscribe to notifications from remote queues, for each of the actor's queue they handled. Particularly, this had 2 inconveniences:

Whenever an RTS notices it becomes responsible for an actor (via gossip views installed from servers and running consistent hashing), it needs to individually subscribe to the input queues of each actor it became responsible for, and also un-subscribe to queues of actors it is not anymore responsible for. This adds un-needed complexity in the RTS to handle the distributed RTS case, as well as un-needed extra RPCs between RTSs and servers when the RTS membership changes
Whenever there are messages in the remote queues for actor queues, the RTS(s) responsible for that actor receive independent notifications for each queue, which is inefficient

The changes in the current PR are the following:

We introduce queue groups. Essentially, this decouples client subscriptions from queue membership to a queue group on the server side. In the new API, the clients (RTSs) subscribe to a queue group rather than individual queues. In the current implementation, this happens exactly once (when the RTS comes up), and there is a single default group for each RTS.
Queue membership to groups is automatically updated on the server side from installed gossip views using consistent hashing mapping, the same way that actor membership gets updated on the RTS side. This means that, as RTSs die or re-join the system, actor input queues automatically get assigned to the right group on the server side.
To make things efficient on the notification path, current group membership of queues (as well as client subscriptions to groups and corresponding per-client metadata, such as read heads) is cached within each queue's data in the backend (so that inefficient consistent hashing lookups do not need to occur at each enqueue).
RTSs receive a single notification per-group whenever there is data in any of the queues it is responsible for, rather than multiple notifications per-actor queue
The code base related to this area has been re-factored to some degree. For instance, the consistent hashing implementation has been pulled out into a separate API, that is used on both the RTS/client side (to compute actor-to-RTS mapping) and on the DB server side (to compute queue-to-group mapping).

Puh, I've read through quite a bit of the code, skimmed some parts. I can't really say much on the new queue group stuff - too much to take in for me. Looks nice overall. Changes in RTS look OK.
For me it's much better to try the stuff out and rely on testing and empirical use in order to verify correctness.
Had some nitpicks on log messages which I think have gotten back a \n which we have previously removed (likely merge artifact).

Thanks! Correct, the '\n' at the end of some logging messages was re-introduced after the merge with main (in my branch these were printf's which needed a '\n'). Thanks for noticing, will fix the remaining ones.

And to clarify, I haven't done any testing or tried to use this. Seeing how testing fails in CI on Linux I think that needs to be fixed first though :)

And to clarify, I haven't done any testing or tried to use this. Seeing how testing fails in CI on Linux I think that needs to be fixed first though :)

Yes, I believe that CI failure is also an issue also related to the merge with the changes in the main branch, will fix it before you can test it.

@aagapi ignore the mac build failures, they are not related to your branch. We just haven't had any builds on main or anywhere for a time... something has changed with GHC so that it now fails, I've seen it in other CI runs, see #1139.

I've merged #1145 which is a temporary workaround for the macos build issues. Please rebase this branch on main and you'll get that fix.


Component: Image with Caption:- On mobile and tablet image is displayed in oval shape

Issue Description:- In image with caption component image is getting displayed in oval shape.  This fix must be included in the first patch for the May v1.7.0 regression testing.
Steps to reproduce the issue

Enter url in the mobile browser.
Select 'Image with Caption' component.
Notice the shape of image.

OS:- iOS 12, 13, Android 8,9
Devices:- Mobile and Tablet
URL:- https://ibmdotcom-react-staging.mybluemix.net/?path=/story/components-imagewithcaption--default

@RobertaJHahn @annawen1 : Checked today Image in 'Image with Caption' is getting displayed properly in mobile and tablet.
Attached screen shot.


revert(vald): revert optimism finality changes

Description

optimism block finality can be queried via the finalized tag in the eth_getBlockByNumber query now

Todos

[ ] Unit tests
[ ] Manual tests
[ ] Documentation
[ ] Connect epics/issues
[ ] Tag type of change
[ ] Upgrade handler

Steps to Test
Expected Behaviour
Other Notes

testing failure?


chore(schema): change name and description on json schema

Feature or Problem
Manifest name shows up in IDE's as "Manifest" which isn't very helpful.
Related Issues
nada

@lachieh just need a DCO from ya

thanks, done


pin yarn version for volta and delete migrations

After the migration to nx@16, was receiving the following error
Volta error: No Yarn version found in this project.

Use `volta pin yarn` to select a version (see `volta help pin` for more info).

Addressed by adding a pinned version of yarn, though we're not sure why volta is complaining at all...

tested by running the eda ✅