ConnectionError of osf clone

Hi:
When I tried to clone an large size public project.
The following error occurred with the opportunity:
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

How do I continue to download the file from the disconnected part instead of redownloading the entire project?
thanks

Complete error record
Traceback (most recent call last):
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/home/weir/anaconda3/lib/python3.7/http/client.py", line 1344, in getresponse
    response.begin()
  File "/home/weir/anaconda3/lib/python3.7/http/client.py", line 306, in begin
    version, status, reason = self._read_status()
  File "/home/weir/anaconda3/lib/python3.7/http/client.py", line 267, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/home/weir/anaconda3/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 318, in recv_into
    raise SocketError(str(e))
OSError: (104, 'ECONNRESET')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/weir/anaconda3/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/home/weir/anaconda3/lib/python3.7/http/client.py", line 1344, in getresponse
    response.begin()
  File "/home/weir/anaconda3/lib/python3.7/http/client.py", line 306, in begin
    version, status, reason = self._read_status()
  File "/home/weir/anaconda3/lib/python3.7/http/client.py", line 267, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/home/weir/anaconda3/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 318, in recv_into
    raise SocketError(str(e))
urllib3.exceptions.ProtocolError: ('Connection aborted.', OSError("(104, 'ECONNRESET')"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/weir/anaconda3/bin/osf", line 8, in <module>
    sys.exit(main())
  File "/home/weir/anaconda3/lib/python3.7/site-packages/osfclient/__main__.py", line 104, in main
    exit_code = args.func(args)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/osfclient/cli.py", line 91, in wrapper
    return_value = f(cli_args)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/osfclient/cli.py", line 167, in clone    file_.write_to(f)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/osfclient/models/file.py", line 53, in write_to
    response = self._get(self._download_url, stream=True)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/osfclient/models/core.py", line 23, in _get
    return self.session.get(url, *args, **kwargs)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/osfclient/models/session.py", line 44, in get
    response = super(OSFSession, self).get(url, *args, **kwargs)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/home/weir/anaconda3/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', OSError("(104, 'ECONNRESET')"))

Hey @weir12,
When you say a "large project", do you mean a project containing a large file or a project containing many small/medium-sized files?  And could you you give me an order-of-magnitude estimate on the file size or number of files?  Thanks!
Cheers,
@felliott

I encountered the same error as @weir12 did.
This error occur when I was trying to download from a fMRI/EEG dataset, which include a large number of files, and some files are also large.

This error occurs consistently when trying to clone https://osf.io/3cwf9/, which also can't be successfully downloaded as a zip

How do I continue to download the file from the disconnected part instead of redownloading the entire project?
thanks

For anyone else who ends up here, you can skip redownloading existing files using the --update flag:
osf clone --update

Have you ever solved this problem?

How do I continue to download the file from the disconnected part instead of redownloading the entire project?
thanks

For anyone else who ends up here, you can skip redownloading existing files using the --update flag:
osf clone --update

This is also not working for me. It again downloads all the files.


weeklycounter.js is not on the compiled flipclock.js

Hi!
I see the commit for the weekly face counter here: https://github.com/objectivehtml/FlipClock/blob/ae18a3c0cb563e10dd79963238f357e1ce95a887/src/flipclock/js/faces/weeklycounter.js
Also the getWeekCounter is missing on the flipclock.js check here:
https://github.com/objectivehtml/FlipClock/blob/ae18a3c0cb563e10dd79963238f357e1ce95a887/src/flipclock/js/libs/time.js
But the weekly counter face is not on the compiled version of the flipclock, check here:
https://github.com/objectivehtml/FlipClock/blob/master/compiled/flipclock.js
This can be fixed? With that we can use that clock face without modify our website code.
Regards.

There is a new branch js-es6. The library was rewritten and this should be fixed.


Control pushing dev images with a dedicated flag

Up until now pushing the dev images depended on the name of the
deployment.
But this task was failing for deployments which were not done on a
cluster started with 'oc cluster up', but with 'minishift', for example.
Instead of relying on the name of the deployment to decide whether to
push the dev images, create a dedicated flag, to allow disabling this
task even when the deployment is done on an environment called 'dev' but
not on a cluster started with 'oc cluster up'.
Signed-off-by: Hunor Csomortáni<EMAIL_ADDRESS>

Looks OK.
I'm just wondering how do you push images to 'minishift' then?
Or how do you make it work on 'minishift' without pushing them?

I usually don't build images for 'minishift' but use the stage ones. When I need building I use the docker env from the minishift VM and build and push the images from there. This task could be modified to acomodate this approach, but atm I don't see it worthwhile to spend the effort.


Disentangle code

This PR separates various concerns into their own modules. It defines a clean interface between modules. I'd like to use the type system better, get logic out of overkill monads, use understandable names, and document heavily.
This is a work in progress.

This is looking really exciting 😊

I think my goal for this PR is to do one big thing: finish the Intent module and make App.hs use it for parsing requests. Continued refactors can go into future PRs.

So far so good, waiting for this module to touch App code :)

The good news thus far: memory leak is fixed for bulk insertion of JSON/CSV!

It has some quirks though, like if I insert [{good}, {good2}, {bad}, {good3}] then only good, good2, good3 get added to the db and we get no error from the bad one. Relatedly it won't choose a default value for omitted columns, so inserting to a table with auto-incrementing primary key fails.
I commented out the PATCH verb stuff entirely so that I could focus on getting POST to work.

Posting big arrays is very slow, like half a minute for 100,000 objects.
I was worried it is caused by my checking that all keys in the objects match... but that doesn't appear to be the slow part. When I comment out the logic in ensureUniform the post is still slow. confusing profiler output Could it be the parsing itself? Was it always this slow? We used to have the memory problem which prevented benchmarking at this size of data.

I confirmed that posting 100,000 CSV objects is very fast, despite being converted into an Aeson Value for submission to postgresql. It's gotta be the json parsing. I'll continue working on "the trick" of selecting the right values from json_populate_recordset to get the insert test passing for columns with default values.


Parsing XML for replica data fails on certain GDBs

On a certain enterprise geodatabase registrant report_gdb_as_html fails with the following message:

Traceback (most recent call last):
File "C:/Users/140318/Desktop/Python/Scripts/GDB Summary/SummarizeGDB.py", line 23, in
registrant.report_gdb_as_html(gdb_origin, summary)
File "C:\Python27\ArcGISx6410.3\lib\site-packages\registrant-0.3-py2.7.egg\registrant_reporter.py", line 105, in report_gdb_as_html
replicas = gdb.get_replicas()
File "C:\Python27\ArcGISx6410.3\lib\site-packages\registrant-0.3-py2.7.egg\registrant_geodatabase.py", line 83, in get_replicas
xml = ET.fromstring(data)
File "C:\Python27\ArcGISx6410.3\lib\xml\etree\ElementTree.py", line 1301, in XML
return parser.close()
File "C:\Python27\ArcGISx6410.3\lib\xml\etree\ElementTree.py", line 1654, in close
self._raiseerror(v)
File "C:\Python27\ArcGISx6410.3\lib\xml\etree\ElementTree.py", line 1506, in _raiseerror
raise err
xml.etree.ElementTree.ParseError: no element found: line 2, column 0

This only occurs on one geodatabase, the others work as expected. Attached is the xml replica summary with connection data removed.
replicaxml.zip

@tom-montgomery , thanks for reporting this. Interesting. The XML file you've attached is not the file that would be created when ExportReplicaSchema_management runs, right? The reason I say that is because the .xml file created by this GP tool doesn't have any formatting such as new lines whereas yours has.
What I think you could do is to run the code in debugging mode and set a breakpoint on line
https://github.com/AlexArcPy/registrant/blob/master/registrant/_geodatabase.py#L77. Check the .xml file created and see what is loaded into the data variable. I think it loads just a few lines instead of the whole .xml file contents. Play with the ET.fromstring(data) and try to identify what's wrong with the .xml file created.

Yea its the export from ExportReplicaSchema_management... I will try debugging. Thanks.

I think its because these databases are still version 9.3. Not sure if its fixable

Ah, interesting. If I remember right, the schema of geodatabase (including replica) has changed in 10.x, so I am afraid I won't be able to parse the 9.3 schema's XML. I guess most users are on 10.0+, so I don't think it would be worth trying to support the 9.3 version. @tom-montgomery , are you OK with that?

Definitely man all good. I just realized why because my replica2mxd was failing on the same databases. We really need to upgrade for a lot of reasons...we are oldschool like that haha. =)

The failure of parsing of replica's XML schema was caused by having the geodatabase in v 9.3. Only 10.0+ are supported by registrant.


In jnigen, Java objects seem to be auto managed by Dart heap, is it allowed, or shall we call dispose manually for each object?

Cross-post from https://github.com/dart-lang/sdk/issues/54233, since if we should not do this, the jnigen may need some refactor. (I hope we are allowed to do such auto management!)

By default Java objects (classes extending JObject) are attached to a NativeFinalizer, so once they're unreachable and GC runs, DeleteGlobalRef is called.
Alternatively you can delete the global reference manually by calling .release():
final o = someJObject();
// do stuff with o
o.release();

or using an Arena:
using((arena) {
  final o = someJObject()..releasedBy(arena);
});
// o is released here.

release internally detaches the native finalizer, and calls the DeleteGlobalRef immediately.
This doesn't mean that the object is completely GC'd though. Only this one global reference to the object is released. Once the object is unreachable completely (no references in either Java or global references in Dart), JVM will eventually GC it.

Thank you for the info, I know it is done by native finalizer, and my main worry was described in https://github.com/dart-lang/sdk/issues/54233.

Thank you for the info, I know it is done by native finalizer, and my main worry was described in dart-lang/sdk#54233.

JNIgen solves this by giving the option of manually releasing the references to the user. Small number of small objects? Fine to let it be cleaned up by the GC. Large number of objects? Maybe use an arena to explicitly clean up the references (and still rely on JVM to clean up the object in its own GC cycle – since there isn't really an explicit dispose in Java).
Java is also different from Rust with the memory ownership rules and not having a GC. So the exact same approach might not be applicable to your package.

Java is also different from Rust with the memory ownership rules and not having a GC. So the exact same approach might not be applicable to your package.

+1 https://github.com/dart-lang/native/issues/849#issuecomment-1840736268

JNIgen solves this by giving the option of manually releasing the references to the user. Small number of small objects? Fine to let it be cleaned up by the GC. Large number of objects? Maybe use an arena to explicitly clean up the references (and still rely on JVM to clean up the object in its own GC cycle – since there isn't really an explicit dispose in Java).

If users of JNIgen are in a context where there is already a dispose cycle (e.g. some Flutter code), you can hook into that.
Organizing flutter_rust_bridge in the same way (e.g. attaching a native finalizer, and cancelling the native finalizer on a manual dispose call) gives your users the option based on usage pattern.


Add to nvim-treesitter

Please open a PR to add this to nvim-treesitter. Please be aware of their conventions for .scm files (such as highlights.scm)

Thanks @liamwh. I've opened a PR here: https://github.com/nvim-treesitter/nvim-treesitter/pull/6856


Continue iOS 11 zip builder fix

Continue #7983 fixing to handle an iOS 11 minimum version pod
This extends the hack a bit more and my test case now successfully builds.
Most of this code should delete soon when we move Carthage to xcframework support
This PR successfully builds the zip and the structure diff looks reasonable. See comments.
The quickstarts are failing and that looks related to the new Analytics structure introduced in 7.11.0

~/zip-m95  $ diff -r ../zip-m93/Firebase Firebase | grep "Only in"
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/ios-arm64_armv7/GoogleUtilities.framework/Headers: GULHeartbeatDateStorable.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/ios-arm64_armv7/GoogleUtilities.framework/Headers: GULHeartbeatDateStorageUserDefaults.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/ios-arm64_i386_x86_64-simulator/GoogleUtilities.framework/Headers: GULHeartbeatDateStorable.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/ios-arm64_i386_x86_64-simulator/GoogleUtilities.framework/Headers: GULHeartbeatDateStorageUserDefaults.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/ios-arm64_x86_64-maccatalyst/GoogleUtilities.framework/Headers: GULHeartbeatDateStorable.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/ios-arm64_x86_64-maccatalyst/GoogleUtilities.framework/Headers: GULHeartbeatDateStorageUserDefaults.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/macos-arm64_x86_64/GoogleUtilities.framework/Headers: GULHeartbeatDateStorable.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/macos-arm64_x86_64/GoogleUtilities.framework/Headers: GULHeartbeatDateStorageUserDefaults.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/tvos-arm64/GoogleUtilities.framework/Headers: GULHeartbeatDateStorable.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/tvos-arm64/GoogleUtilities.framework/Headers: GULHeartbeatDateStorageUserDefaults.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/tvos-arm64_x86_64-simulator/GoogleUtilities.framework/Headers: GULHeartbeatDateStorable.h
Only in Firebase/FirebaseAnalytics/GoogleUtilities.xcframework/tvos-arm64_x86_64-simulator/GoogleUtilities.framework/Headers: GULHeartbeatDateStorageUserDefaults.h
diff: ../zip-m93/Firebase/FirebaseAnalytics/PromisesObjC.xcframework/macos-arm64_x86_64/PromisesObjC.framework/Headers/Headers: No such file or directory
diff: Firebase/FirebaseAnalytics/PromisesObjC.xcframework/macos-arm64_x86_64/PromisesObjC.framework/Headers/Headers: No such file or directory
Only in Firebase: FirebaseAppCheck
Only in ../zip-m93/Firebase/FirebaseAuth/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/FirebaseAuth/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/FirebaseAuth/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
Only in ../zip-m93/Firebase/FirebaseAuth/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/FirebaseAuth/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/FirebaseAuth/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
Only in ../zip-m93/Firebase/FirebaseAuth/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/FirebaseAuth/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/FirebaseAuth/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
diff: ../zip-m93/Firebase/FirebaseFirestore/BoringSSL-GRPC.xcframework/macos-arm64_x86_64/BoringSSL-GRPC.framework/Headers/Headers: No such file or directory
diff: Firebase/FirebaseFirestore/BoringSSL-GRPC.xcframework/macos-arm64_x86_64/BoringSSL-GRPC.framework/Headers/Headers: No such file or directory
diff: ../zip-m93/Firebase/FirebaseFirestore/abseil.xcframework/macos-arm64_x86_64/abseil.framework/Headers/Headers: No such file or directory
diff: Firebase/FirebaseFirestore/abseil.xcframework/macos-arm64_x86_64/abseil.framework/Headers/Headers: No such file or directory
diff: ../zip-m93/Firebase/FirebaseFirestore/gRPC-C++.xcframework/macos-arm64_x86_64/gRPC-C++.framework/Headers/Headers: No such file or directory
diff: Firebase/FirebaseFirestore/gRPC-C++.xcframework/macos-arm64_x86_64/gRPC-C++.framework/Headers/Headers: No such file or directory
diff: ../zip-m93/Firebase/FirebaseFirestore/gRPC-Core.xcframework/macos-arm64_x86_64/gRPC-Core.framework/Headers/Headers: No such file or directory
diff: Firebase/FirebaseFirestore/gRPC-Core.xcframework/macos-arm64_x86_64/gRPC-Core.framework/Headers/Headers: No such file or directory
Only in ../zip-m93/Firebase/FirebaseFunctions/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/FirebaseFunctions/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/FirebaseFunctions/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
Only in ../zip-m93/Firebase/FirebaseFunctions/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/FirebaseFunctions/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/FirebaseFunctions/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
Only in ../zip-m93/Firebase/FirebaseFunctions/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/FirebaseFunctions/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/FirebaseFunctions/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
Only in ../zip-m93/Firebase/FirebaseMessaging: FirebaseInstanceID.xcframework
Only in ../zip-m93/Firebase/FirebaseStorage/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/FirebaseStorage/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/FirebaseStorage/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
Only in ../zip-m93/Firebase/FirebaseStorage/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/FirebaseStorage/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/FirebaseStorage/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
Only in ../zip-m93/Firebase/FirebaseStorage/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/FirebaseStorage/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/FirebaseStorage/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
Only in ../zip-m93/Firebase/GoogleSignIn/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/GoogleSignIn/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/GoogleSignIn/GTMSessionFetcher.xcframework/ios-arm64_armv7/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
Only in ../zip-m93/Firebase/GoogleSignIn/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/GoogleSignIn/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/GoogleSignIn/GTMSessionFetcher.xcframework/ios-arm64_i386_x86_64-simulator/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
Only in ../zip-m93/Firebase/GoogleSignIn/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMGatherInputStream.h
Only in ../zip-m93/Firebase/GoogleSignIn/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMMIMEDocument.h
Only in ../zip-m93/Firebase/GoogleSignIn/GTMSessionFetcher.xcframework/ios-arm64_x86_64-maccatalyst/GTMSessionFetcher.framework/Headers: GTMReadMonitorInputStream.h
~/zip-m95  $

The quickstart tests should be fixed by #8002 and a temporary AdMob 8.4.1 in SpecStaging (equivalent to 8.4.0) except for its GoogleAppMeasurement dependency.
I'll restart the tests, revert the nightly undo, and merge to run the real nightlies.


external(autocad): improve dimension conversions

This is a master issue for the Autocad / Civil3d Dimension conversion improvement task.
The goal is to improve current dimension conversion support in Autocad <-> Autocad workflows, with the following sequence of tasks for each time of Autocad Dimension:

Create a test file for the Dimension sub type, to cover all reasonable and common cases of the dimension. For example, an ordinate dimension test file should contain various forms of both horizontal and vertical ordinate dimensions with different styles applied.
Send the test file to Speckle from Autocad, and then receive it. If the received version is not identical to the sent version, fix the conversion.

Build from the most current release branch: release/2.8

Current conversion state
Conversion results from the Autocad Standard test file (provided)

Known issues:

Radial and Diametric dimensions have incorrect center point and leader styling
Anglular dimension value is incorrect
Rotated dimension text point is incorrect

Progress and estimated timeline: 1-3 hours per dimension
As progress is made on each dimension, please comment on this issue with:
(1) a link to the created test file for review
(2) the results of sending and receiving the initial test file in autocad
(3) any conversion issues observed and a rough estimate for how long it would take to fix
(4) check each box below as it is completed
AlignedDimension:

[ ] Create test file
[ ] Test and fix test file conversion issues

RotatedDimension:

[ ] Create test file
[ ] Test and fix test file conversion issues

OrdinateDimension

[ ] Create test file
[ ] Test and fix test file conversion issues

RadialDimension

[ ] Create test file
[ ] Test and fix test file conversion issues

DiametricDimension

[ ] Create test file
[ ] Test and fix test file conversion issues

ArcDimension

[ ] Create test file
[ ] Test and fix test file conversion issues

LineAngularDimension2

[ ] Create test file
[ ] Test and fix test file conversion issues

Point3AngularDimension

[ ] Create test file
[ ] Test and fix test file conversion issues

SampleDimension
Test file link: [ ]
Initial results: [ ]

Issues:

[ ] Sample issue (eg text point is incorrect)

Estimated time to fix:

AlignedDimension

[x] Create test file
[x] Test and fix test file conversion issues

(1)  dwg has been posted in discord
(2)  screen shot has been posted in discord
(3)  No conversion issues found
RotatedDimension

[x] Create test file
[x] Test and fix test file conversion issues

(1)  dwg has been posted in discord
(2)  screen shot has been posted in discord
(3)  There is a text location issue in RotatedDimension (Text has moved to another location but not received correctly).
OrdinateDimension

[ ] Create test file
[ ] Test and fix test file conversion issues

RadialDimension

[ ] Create test file
[ ] Test and fix test file conversion issues

DiametricDimension

[ ] Create test file
[ ] Test and fix test file conversion issues

ArcDimension

[ ] Create test file
[ ] Test and fix test file conversion issues

LineAngularDimension2

[ ] Create test file
[ ] Test and fix test file conversion issues

Point3AngularDimension

[ ] Create test file
[ ] Test and fix test file conversion issues

Description
Dimensions sent from Autocad can be received back but with a catch, they are not the old dimensions anymore 🙂. Take a look at the screenshots below.
Expected Behaviour
Receiving dimensions back as we sent them.
Steps to reproduce

Open Autocad test file.
Select dimensions to Speckle.
Open a new drawing and receive what you just sent.
Compare the two drawings.

Streams/Commits Used
https://latest.speckle.dev/streams/3ed8357f29/commits/46905429f6
Files Used
Autocad Test File
https://drive.google.com/open?id=1-Ci9fX0xfOgtMy0phTIlqzIZeqMV72pM&authuser=bilal%40speckle.systems&usp=drive_fs
Screenshots
Sent:

Received:

Related Issues
#1625

Closing this ticket as we are moving to Jira to reduce noise. You can access it there.


Documentation: Update link for forms documentation

Update the link to the forms documentation to reflect 3.0. The code example in this file was already updated in #1302
Have used 3.0 instead of current to allow for possible future BC changes.

see #1387


Bump to 0.9.2

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Thanks for the PR, @druvus! Could you please update the title of your PR from 1.9.2 to 0.9.2?
I'll fix the errors on Travis (HTTP fail just needs a restart) and AppVeyor (needs rerendered to handle changes introduced in conda 4.4).
Something strange is going on though. The binaries built in the PR are being uploaded directly to Anaconda Cloud:
https://travis-ci.org/conda-forge/r-cowplot-feedstock/jobs/339223016#L589
https://anaconda.org/conda-forge/r-cowplot/files?version=0.9.2
@bgruening Is this an error or is this new behavior?

@conda-forge-admin, please rerender

Thanks @druvus!

@bgruening Did you see my question above? The conda packages were uploaded to Anaconda Cloud before the PR was merged. Is this new behavior? If yes, would you be able to point me to where the relevant discussion took place?

OK. I think something went wrong here. The travis-ci/push and appveyor/branch builds were started during the PR, but these should only be started after merging. This other recent PR behaved as I expected: https://github.com/conda-forge/r-assertive.types-feedstock/pull/1

Urgs, no have not seen this, sorry.
I don't think this is normal. Lets keep an eye on this and see if we can find a second case.

@bgruening Here's another PR that travis-ci/push and appveyor/branch: https://github.com/conda-forge/r-assertive.numbers-feedstock/pull/1


Uninstall fails

On Chocolatey v2.2.2, getting this when I try to uninstall:
Uninstalling the following packages:
nerd-fonts-Cousine

nerd-fonts-Cousine v3.1.1
ERROR: The term 'Uninstall-ChocolateyFont' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
nerd-fonts-Cousine uninstall not successful.
Error while running 'C:\ProgramData\chocolatey\lib\nerd-fonts-Cousine\tools\chocolateyUninstall.ps1'.
 See log for details.
nerd-fonts-Cousine not uninstalled. An error occurred during uninstall:
 nerd-fonts-Cousine uninstall not successful.

Chocolatey uninstalled 0/1 packages. 1 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Failures
 - nerd-fonts-Cousine (exited -1) - Error while running 'C:\ProgramData\chocolatey\lib\nerd-fonts-Cousine\tools\chocolateyUninstall.ps1'.

@mica Sorry for missing this issue.  Are you still experiencing this?  'Uninstall-ChocolateyFont' is built into chocolatey itself and was added in a specific version that these packages should have a dependency on.


Implement Business Fields

The default business form field shall be a single field. When text is typed into it the field will reference the Places API and begin suggesting businesses. If the user is registered within the Places API, then they should be able to select their business and have its information fill the form.
The backup field should not be visible by default. It should consist of Address, City, State, and Zip Code fields.
Near the field there should be some element that can toggle between the default and the backup business field. When clicked, the backup field should replace the default field if the default field is showing, or the reverse if the secondary field is showing.

Resolved in #19.


add connection-pool

Codecov Report

Merging #21 into master will decrease coverage by 2.7%.
The diff coverage is 60%.

@@            Coverage Diff             @@
##           master      #21      +/-   ##
==========================================
- Coverage   90.12%   87.41%   -2.71%
==========================================
  Files          11       12       +1
  Lines         962     1057      +95
==========================================
+ Hits          867      924      +57
- Misses         95      133      +38

Impacted Files
Coverage Δ

lib/src/connection.dart
92.1% <ø> (ø)
:arrow_up:

lib/src/connection_pool.dart
60% <60%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42278fe...b23563a. Read the comment docs.

Thanks for the PR. I'll have to reject it as it stands right now and provide some feedback:

The Connection interface is confusing and duplicates behavior from the execution context. As far as I can tell, it is interjecting isClosed checks on many methods. Why not have the pool not return connections that are closed?

Two implementations of the connection - pooled and normal - are unnecessary. It will create a future problem as code will have to be replicated across two implementations - as the codebase evolves, that always means the implementations start to diverge.

Avoid using while and for with await. Instead use Future.wait or Future.forEach to keep the call stack reasonable.

Marking a method as async means it will automatically be added to the micro task queue when invoked, so no need to open an async method by scheduling a microtask.

As far as I can tell, the only way to return a connection back to the pool is to close it? Connections should stay up as long as possible to avoid the cost of setting the up and tearing them down; ideally, a connection in a pool would never close.

Hi @joeconwaystk,
I add class Connection, because I want create abstraction on connection and hide inner implementation (pooled or not pooled) connection.
PooledConnection is not second implementation, this is lightweight proxy and supporting his not must be hard.
Pooled connection of course not must closed is my mistake. I fix this.
Thanks.

I'll have a chance to review much later today,  but two questions:

Why does there need to be a proxy?
I don't see a way for the user to return the connection.

When the user closes the proxy connection, it returns to the pool.
User not must known how was created connection.

User getting connection;
Execute some work;
Closing connection;

  Future _sameWorks(Connection connection) async {
    await connection.execute("select 1");
    await connection.close();  //if this _PooledConnection, it return in pool
  }

We can not use the released connections, the proxy protects this case.

OK, the direction makes sense. Here's how I'd like the interface to look (omitting implementation):
class PostgreSQLConnectionPool {
  PostgreSQLConnectionPool(int size, String host, int port, String databaseName,
   {String username,
   String password,
   int timeoutInSeconds: 30,
   String timeZone: "UTC",
   bool useSSL: false});

  PostgresSQLConnection get connection;

  Future open();
  Future close();
}

A consumer of the pool doesn't have to worry about closing connections or anything, it just does this:
Future doThing(PostgreSQLConnectionPool pool) async {
   await pool.connection.query("SELECT * FROM x");
}

connection always returns the 'best' PostgreSQLConnection. Each PostgreSQLConnection has its own internal queue; the best connection is the one with the smallest queue. Since there is already a timeout mechanism for the query, if the queue gets backed up, we still get the same error as we would if we had failed to acquire a connection.
There is one important note to make here: pool.connection does not return a Future. The reason being is that the snippet above is a lot more empathetic than:
Future doThing(PostgreSQLConnectionPool pool) async {
   await (await pool.connection).query("SELECT * FROM x");
}

However, to avoid the above, there are some guarantees and some slight behavioral changes to PostgreSQLConnection.

First, the current behavior of PostgreSQLConnection is to reject any query/execute/transaction when it is closed. However, if it is closed and has yet to be opened, it needs to instead queue that operation instead of reject it. When it opens, it starts processing its queue. One issue that arises in doing this is if the connection is never opened, the queue will stall - however, but the timeout mechanism on query/execute will prevent this from happening.

Second, the pool must monitor when a PostgreSQLConnection becomes disconnected. Therefore, PostgreSQLConnection must have a Future done property. When the connection is closed, this future will complete. When the pool is first opened, it adds a listener for each connection.done it creates. The pool replaces (and opens) any connection in that listener.

Does that make sense?

It's good idea.


Model toggler functions

model.toggle_f( 'attr', value), returning bound function for boolean property, as for collections (might be useful for simple radio buttons)

model.toggler( 'attr' )


Replace hardcoded Inno Setup 5 path

Inno Setup 6 is the newest version for starters. Although this may become unneeded when #14 is implemented

The last version of Inno Setup 5 is 5.6.1: https://files.jrsoftware.org/is/5/ for those wanting to use this feature for now


Dates on rolls are rendering minutes in the month's location

It looks like dates for rolls are rendering the minutes of the date in the month's location:

For a11y, these are showing:

26/29/2024 10:29:53 and 26/30/2024 10:30:57, respectively.

But they should be showing:

26/10/2024 10:29:53 and 26/10/2024 10:30:57, respectively (since it's October).

It looks like the issue might be that the wrong patterns are used in the format function:
https://github.com/fultimator/fultimator/blob/70b9272b66036ac43cc6603e8f1b1eab040b09ec/src/components/roller/Roll.js#L51
https://github.com/fultimator/fultimator/blob/70b9272b66036ac43cc6603e8f1b1eab040b09ec/src/components/roller/Roll.js#L238
It seems like these should be using MM instead of mm for the month. Like this:
            {format(roll.timestamp.toDate(), "dd/MM/yyyy hh:mm:ss")}

and
                {format(attempt.timestamp.toDate(), "dd/MM/yyyy hh:mm:ss")}

respectively.
I don't have firebase setup, so I can't follow the full contribution guide. But if this can be run and validated locally, I'll toss up a PR.

I managed to work around the hard dependency on firebase. Lemme see if I can toss up a PR.


Running uv pip compile with --find-links pointing towards a local wheel does not produce the expected output

Hello!
Unsure whether this is a bug, or I could be misusing uv!
I'm trying to migrate over from pip-tools to using UV instead. I've got a local wheel that I'd like to install, that lives within my repository as a .whl file. In pip-compile world, I'd run pip-compile --find-links=./local_wheels --output-file=requirements.txt pyproject.toml and this generated a requirements.txt file. The resultant requirement.txt file would contain --find-links ./local_wheels at the top of the file. This would mean that when I run pip-sync, it would make use of wheels found within the /local_wheels directory
I expected uv pip compile --find-links ./local_wheels --output-file=requirements.txt pyproject.toml to return a similar output to requirements.txt, but it doesn't have the expected --find-links ./local_wheels in the autogenerated output
Am I missing something or is this a bug?

I think you can pass --emit-find-links to include them (we just don't do it by default).

It's mentioned here for index URLs, but I should probably add a reference to --emit-find-links to help with search: https://docs.astral.sh/uv/pip/compatibility/#pip-compile-defaults

It's mentioned here for index URLs, but I should probably add a reference to --emit-find-links to help with search: https://docs.astral.sh/uv/pip/compatibility/#pip-compile-defaults

Thank you @charliermarsh that was what I was looking for! Thank you for such a speedy response.
Out of interest, I have another issue, where for some reason setuptools is being added to requirements.txt, whereas it wasn't before. Is there another flag I'm missing that will make uv behave similar to how pip-tools was?

You could add --no-emit-package setuptools. I think pip-compile may omit it by default right now? But IIRC that's being removed in future versions of pip-tools.

You should be able to set these things in your pyproject.toml if you want:
[tool.uv.pip]
emit-find-links = true
no-emit-package = ["setuptools"]

(Something like that -- didn't test it before writing :))

A big thank you @charliermarsh, that is exactly what I needed! Thank you for solving both of my issues, I had started going around in circles, so much appreciated <3

No prob! I can add some things to the compatibility guide so it's easier to reference in the future.


Problem to run on linux but not on windows

System Info
see: https://github.com/hwchase17/langchain/discussions/1533
Who can help?
@hwchase17
Information

[ ] The official example notebooks/scripts
[ ] My own modified scripts

Related Components

[ ] LLMs/Chat Models
[X] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction
use FAISS on windows and next try to reuse those embeddings on lunix
Expected behavior
I have a problem, when I want to run a project with langchain on windows, everything works perfectly, and with the same conditions on linux (libraries, python version, etc.) it doesn't work and throws this error, does anyone know what it could be?

2023-03-08 15:13:24 Failed to run listener function (error: search() missing 3 required positional arguments: 'k', 'distances', and 'labels')
2023-03-08 15:13:24 Traceback (most recent call last):
2023-03-08 15:13:24 File "/usr/local/lib/python3.9/site-packages/slack_bolt/listener/thread_runner.py", line 120, in run_ack_function_asynchronously
2023-03-08 15:13:24 listener.run_ack_function(request=request, response=response)
2023-03-08 15:13:24 File "/usr/local/lib/python3.9/site-packages/slack_bolt/listener/custom_listener.py", line 50, in run_ack_function
2023-03-08 15:13:24 return self.ack_function(
2023-03-08 15:13:24 File "//./main.py", line 28, in question
2023-03-08 15:13:24 response = processQuestion(query)
2023-03-08 15:13:24 File "/api.py", line 42, in processQuestion
2023-03-08 15:13:24 sources = doSimilaritySearch(index, query)
2023-03-08 15:13:24 File "/utils.py", line 87, in doSimilaritySearch
2023-03-08 15:13:24 docs = indexFaiss.similarity_search(query, k=5)
2023-03-08 15:13:24 File "/usr/local/lib/python3.9/site-packages/langchain/vectorstores/faiss.py", line 166, in similarity_search
2023-03-08 15:13:24 docs_and_scores = self.similarity_search_with_score(query, k)
2023-03-08 15:13:24 File "/usr/local/lib/python3.9/site-packages/langchain/vectorstores/faiss.py", line 136, in similarity_search_with_score
2023-03-08 15:13:24 docs = self.similarity_search_with_score_by_vector(embedding, k)
2023-03-08 15:13:24 File "/usr/local/lib/python3.9/site-packages/langchain/vectorstores/faiss.py", line 110, in similarity_search_with_score_by_vector
2023-03-08 15:13:24 scores, indices = self.index.search(np.array([embedding], dtype=np.float32), k)
2023-03-08 15:13:24 TypeError: search() missing 3 required positional arguments: 'k', 'distances', and 'labels'

I am using faiss_cpu 1.7.4 and I still run into the issue mentioned above. I tried the solution suggested above

Answer generated by a 🤖
Answer
The error you're encountering is related to the FAISS library and not directly to LangChain. It seems that the FAISS library behaves differently on Windows and Linux. To fix the issue, you can try modifying the similarity_search_with_score_by_vector function in the langchain/vectorstores/faiss.py file.
Replace this line:
scores, indices = self.index.search(np.array([embedding], dtype=np.float32), k)

with:
embedding_array = np.array([embedding], dtype=np.float32)
distances = np.empty((embedding_array.shape[0], k), dtype=np.float32)
labels = np.empty((embedding_array.shape[0], k), dtype=np.int64)
self.index.search(embedding_array, k, distances, labels)
scores, indices = distances, labels

This does not work however. The error I get is that I'm missing one positional argument now. When I look into the documentation there is this parameter n that is not mentioned in the above solution. This error occurs only on Linux and not on Windows.


EBP: Improve styles for selection session item summary page

Checklist

[x] the contributor license agreement is signed
[x] commit message follows commit guidelines
[x] screenshots are included showing significant UI changes

Description of change

Made alignment of the item summary panel consistent across browsers (width: fit-content was used before, needed to add width: -moz-fit-content to make them work the same)
Fixed the alignment of the navbar (some of the text and part of the logo were cut off before)
Fixed alignment of the Select Summary Page button to be to the right of the Item title.
The summary/details tab bar divider now goes across the full length rather than roughly halfway
Realigned the help text within the help button (before when that was shown it would cut off the first word of every line)
Fixed faint line above Save button
Made the Select Summary Page button round and the same size as the others
Aligned the right side of the comment text box to the rest of the content in the panel
Hidden a divider on the Selection Panel between the buttons and the destination tree
Aligned the selection panel vertically to the resource summary panel rather than the tabs (only applied to the resource summary selection session page)

Before:

After:

Thanks Sam! Looks much nicer!
I guess these changes only apply to the ItemSummary page in Selection session so the normal ItemSummary page still looks the same ?

Yup, that's correct. I've only made changes where we can be specific to a selection session.

Woooah, thats a LOT of failed tests.
Looks like there is an error in legacy.scss and it is unable to generate. Looking into now.

Looks like the failure to compile was a mistyped variable name. Should work now.


Improve the error messages in the dataset viewer

The error messages are shown in the dataset viewer. We should try to improve them:

help the user understand what is going on
help the user fix the issue
show that we understand what is going on, and that it's not a bug.

See https://huggingface.slack.com/archives/C04L6P8KNQ5/p1675171833108199 (internal) and https://github.com/huggingface/datasets/issues/4886.
Also:

https://github.com/huggingface/datasets-server/issues/731#issuecomment-1410297107
https://github.com/huggingface/datasets-server/issues/718

Is the issue solved by #754, @albertvillanova?


Add crates.io badges and categories

Fixes #26.

r? @fitzgen

Coverage remained the same at 0.0% when pulling 3a5f5b4756296408d351e44b232d29245bf41b06 on jonhoo:crates-io-metadata into dcc595b31569c794f8329b60c1c2aae1f6253ca3 on gimli-rs:master.


Can I get rid of auto horizontal scrolling?

Issue Type: Bug
Hi,
I am a fan of VS Code, but I just realized that there is something annoying function which scrolls automatically when lines are long.
This blocks me coding?
Is there any way for me to get rid of this?
Thanks
VS Code version: Code 1.41.1 (26076a4de974ead31f97692a0d32f90d735645c0, 2019-12-18T14:58:56.166Z)
OS version: Windows_NT x64 10.0.17134

/needsMoreInfo can you illustrate in which cases the horizontal scrolling is annoying and blocks your from coding?
One way to disable horizontal scrolling is to enable Toggle Wordwrap (View>Toggle Wordwrap)


how I improve the panel while keeping the cryptominer data

The endpoint.php is what accepts the miner data, it then inserts it into the database. The database can then be read however you want and manipulated in any way freely.

where is the database

The file unamwebpanel.db.

thx


Make text mining work

The frontend API prototype for text mining is finished, we need to hook the fronted up with the backend - Fenominal.
Definition of done

[ ] - Fenominal has a stable release
[x] - create an backend endpoint for text IO
[ ] - implement the backend

f45445f added the backend endpoint for text IO


Dynamically Controlling Components

My application contains several components (overhead map, guidance panel) that are known apriori by the App object that renders the <DockviewReact>.  I create it like so:
<DockviewReact
    onReady={onReady}
    components={components}
    className={'dockview-theme-light'}
/>

where {components} is a list of two:
const components = {
    default: (props: IDockviewPanelProps) => {
        return <MainComponent />;
    },
    guidance_panel: (props: IDockviewPanelProps) => {
        return <GuidancePanelComponent />;
    },
};

What I am interested in is having the ability to add new components arbitrarily.   I want to be able to press a button and call something like:
dockviewApi.addPanel({
            id: 'camera',
            component: 'camera',
            renderer: 'always',
            title: 'Camera',
            floating: {
                position: {left: 10, bottom: 10},
                width: 400,
                height: 300,
            },
        });

The catch is I don't know apriori to the <DockviewApi> object creation that this camera component needs to exist.
I'm not sure if I'm just missing some understanding of what the components are in <DockviewReact> or if it's just not possible.  Any help would be appreciated.

In the case where you need dynamic components the approach I would recommend is to register a single component with dockview and have that component call into a dynamic collection of components that you maintain. You can pass that dynamic components name into the panel at the time of creation using the params property.
Anything in params is persisted with the dockview state.
Let me know if that's useful?
const dynamicComponents: Record<string, React.FC> = {
    component_1: () => {
        return <div>{'component_1'}</div>;
    },
};

const components = {
    default: (
        props: IDockviewPanelProps<{ arbitraryComponentName: string }>
    ) => {
        const name = props.params.arbitraryComponentName;

        const Component = dynamicComponents[name];

        return Component ? <Component /> : null;
    },
};

const RootComponent = () => {
    const onReady = (event: DockviewReadyEvent) => {
        event.api.addPanel({
            id: 'panel_1',
            component: 'default',
            params: { arbitraryComponentName: 'component_1' },
        });
    };

    return <DockviewReact components={components} onReady={onReady} />;
};

This worked for me.  Appreciate the help.


get.obs_df and get.var_df produce an error when there is only 1 key

When trying to get the obs_df or the var_df, the function throws an error when there is a single key. In this case it iterates through the letters in the given key and causes a KeyError.

pbmc = sc.datasets.pbmc68k_reduced()
sc.get.obs_df(pbmc, keys=('HES4'))
sc.get.var_df(pbmc, keys=('AAAGCCTGGCTAAC-1'))

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-60-663347265b80> in <module>
----> 1 sc.get.var_df(pbmc, keys=('AAAGCCTGGCTAAC-1'))

~/.conda/envs/scvelo_updated/lib/python3.8/site-packages/scanpy/get.py in var_df(adata, keys, varm_keys, layer)
    223             not_found.append(key)
    224     if len(not_found) > 0:
--> 225         raise KeyError(
    226             f"Could not find keys '{not_found}' in columns of `adata.var` or"
    227             " in `adata.obs_names`."

KeyError: "Could not find keys '['A', 'A', 'A', 'G', 'C', 'C', 'T', 'G', 'G', 'C', 'T', 'A', 'A', 'C', '-', '1']' in columns of `adata.var` or in `adata.obs_names`."

Versions:

scanpy==1.5.1 anndata==0.7 umap==0.4.4 numpy==1.19.0 scipy==1.5.0 pandas==1.0.5 scikit-learn==0.23.1 statsmodels==0.11.1 python-igraph==0.8.2 louvain==0.7.0

In python ('HES4') parses as a string, not as a tuple. You'd have to put ('HES4',) to get a tuple, while you could write ['HES4'] to get a list.
The behavior here could be more straight-forward, but you should be able to get what you want by passing an iterable of strings.

that makes sense, thank you for the prompt response!


Circuit Terminaison with 2 sides

Hi,
Working for an ISP, a circuit has 2 sides ofr us. Side A, which represent le customer end (CPE router), and side B which represent the ISP side (PE Collect router).
Is it possible to add 2 terminaisons but not necessarily ?
Thanks in advance.
Thomas V.

Duplicate of #49


Flatlist keyboardShouldPersistTaps not working

Please provide all the information requested. Issues that do not follow this format are likely to stall.
Description
I have the following structure of views in my app. I have several input dialogues being presented individually using a Flatlist and Smartview. I need to click  <TouchableNativeFeedback/> when the <InputItem/> is focused and keyboard is presented. Currently it requires two clicks for the button (one dismisses keyboard and another executes action)
structure  of views:
<View>     <Dialog>     <View >     <KeyboardAvoidingView>     <FlatList     renderItem={({ item }) => (     <SmartView //structure of view per input  <SafeAreaView> <View>    <InputItem/>  <TouchableNativeFeedback/> //button that must be clicked </View>  </SafeAreaView>  />     )}>     </KeyboardAvoidingView>     </View>     </Dialog>     </View >
I have tried to add keyboardShouldPersistTaps='always' / 'handled' in the properties of the Flatlist, but nothing did change in terms of behavior.
React Native version:
Using real device iPad 5 (iPad OS 14.4.2)
Run react-native info in your terminal and copy the results here.
System:     OS: macOS 11.2.3     CPU: (4) x64 Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz     Memory: 1.27 GB / 32.00 GB     Shell: 5.8 - /bin/zsh   Binaries:     Node: 15.5.1 - /usr/local/bin/node     Yarn: 1.22.10 - /usr/local/bin/yarn     npm: 7.3.0 - /usr/local/bin/npm     Watchman: 4.9.0 - /usr/local/bin/watchman   Managers:     CocoaPods: 1.9.3 - /usr/local/bin/pod   SDKs:     iOS SDK:       Platforms: iOS 14.4, DriverKit 20.2, macOS 11.1, tvOS 14.3, watchOS 7.2     Android SDK:       API Levels: 23, 29, 30       Build Tools: 28.0.3, 29.0.2, 29.0.3, 30.0.3, 31.0.0       System Images: android-29 | Intel x86 Atom_64, android-29 | Google APIs Intel x86 Atom, android-30 | Google APIs Intel x86 Atom       Android NDK: Not Found   IDEs:     Android Studio: 4.1 AI-201.87<IP_ADDRESS>99119     Xcode: 12.4/12D4e - /usr/bin/xcodebuild   Languages:     Java: 1.8.0_275 - /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/bin/javac     Python: 2.7.16 - /usr/bin/python   npmPackages:     @react-native-community/cli: Not Found     react: 16.13.1 => 16.13.1      react-native: ~0.63.4 => 0.63.4      react-native-macos: Not Found   npmGlobalPackages:     *react-native*: Not Found
Steps To Reproduce
Provide a detailed list of steps that reproduce the issue.

Click button to prompt dialogue
Dialogue has an input field with auto focus and submit button next to it
Enter details in the input and click on submit button 'without dismissing keyboard'
After click, keyboard gets dismissed and action of button is NOT called.
Press again to call action.

Expected Results
Describe what you expected to happen.

Action of button must be called on first click even if keyboard is prompted.

Fixed issue by adding  keyboardShouldPersistTaps={"always"} to component  <Dialog>


⚠ Bofrost.lu has degraded performance

In 697b0c0, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1224 ms

Resolved: Bofrost.lu performance has improved in 2657140 after 23 minutes.


Remove or delete module feature

Thanks for amazing lib.
Has a plan to a feature remove module?
like
 php artisan remove:module  {module-name}

This could be labelled as good first issue I think.


Javascript is disabled on your browser AND Some characters could not be decoded, and were replaced with REPLACEMENT CHARACTER.

Hi
when I use Library mode in python sometimes get these errors. How can fix them?
url= "https://www.varzesh3.com/news/1403838/%D9%88%D8%A7%D9%84%D9%86%D8%B3%DB%8C%D8%A7-%D8%AD%D8%B3-%D9%85%DB%8C%E2%80%8C-%DA%A9%D9%86%D9%85-21-%D8%B3%D8%A7%D9%84-%D8%AF%D8%A7%D8%B1%D9%85"
article = NewsPlease.from_url(url)
print(article.maintext)
Javascript is disabled on your browser, to have the best experience of using varzesh3.com, enable it.
AND
url='https://www.tabnak.ir/fa/news/129424/%D9%BE%D8%B1%D8%AF%D8%B1%D8%A2%D9%85%D8%AF%D8%AA%D8%B1%DB%8C%D9%86-%D9%85%D8%B4%D8%A7%D8%BA%D9%84-it-%DA%A9%D8%AF%D8%A7%D9%85%D9%86%D8%AF'
article = NewsPlease.from_url(url)
Some characters could not be decoded, and were replaced with REPLACEMENT CHARACTER.
best regards

Can you please reopen this bug report using the corresponding template? Thanks.


Users need more examples on resdocks proxy configuration

This has come up in a few support threads (anecdotally, will start linking to this ticket as they come) and it has also came up from user feedback (e.g. see )
On a relevant note, it might make sense to make the new supervisor api endpoint that can be used to configure redsocks more visible in the docs - https://github.com/resin-io/resin-supervisor/blob/master/docs/API.md#patch-v1devicehost-config (e.g. when/how one can find it useful?)

Should the issue title be renamed from resdocks to redsocks? :wink:

We should also document that login and password fields do not need quotes (as seen in https://github.com/darkk/redsocks/blob/5df6a30e5f8e75455e851f2ef87c647eb3b08dff/tests/regw/redsocks.conf)


Pauli measurement

Reviving #4024.
Fixes https://github.com/Qiskit/qiskit-terra/issues/3967
The conversation of #4024 dealt with the addition of gates after measurement. When does the removal of such gates, after final measurements, occur? The output of transpile still contains the gates after the final measurement.
To do items:

[x] Store the basis in the instruction parameters, instead of the instruction name.
[ ] Allow also measurement in the I basis (no measurement).
[x] Verify that works well when measuring multiple qubits.

Need help in identifying:

Handlings of measurement instructions, which need to handle also Pauli measurement instructions.
Handlings of both measurement and Pauli instructions, which need to handle only measurement instructions.
Is there an assumption that measurement is only for one qubit.

Ready for review.  @Cryoris @chriseclectic

Is there any update on this PR? I think this is very useful.

A few comments:

I didn't add support to measure_pauli in qasm.
I touched only a little the transpiler. I don't know which node is created in the DAG, in the presence of the new measure_pauli instruction, and what's its type (see for example _process_node and _process_measure in qiskit/converters/ast_to_dag.py). Checks whether something is a measurement appear all over, and I mostly didn't change them, I don't know if a change is required there, and if yes then how: in gate translations; in decisions related to optimization that are based on the number of measurements; and in passes: RemoveDiagonalGatesBeforeMeasure, AlignMeasures, BarrierBeforeFinalMeasurements, RemoveFinalMeasurements, OptimizeSwapBeforeMeasure.
I touched only a little the visualization. I tried my best with matplotlib, but didn't change text and latex. Note that, unlike measure, measure_pauli can take multiple qubits.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

53 of 67 (79.1%)  changed or added relevant lines in 8 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.006%) to 82.565%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit/circuit/measure.py
30
32
93.75%

qiskit/visualization/text.py
12
14
85.71%

qiskit/visualization/matplotlib.py
2
12
16.67%

Totals

Change from base Build<PHONE_NUMBER>:
0.006%

Covered Lines:
49969

Relevant Lines:
60521

💛  - Coveralls

Is there any reason for creating MeasurePauli instead of extending Measure with a basis parameter?

Is there any reason for creating MeasurePauli instead of extending Measure with a basis parameter?

@1ucian0 I'm not sure, but to get the answer it's important to emphasize:

MeasurePauli can have multiple qubits, whereas Measure takes only one qubit. Therefore doing something like this would also mean extension of Measure to any number of qubits.
Eventually, the goal of this PR is that for a simulator backend the transpiler will leave the instruction as-is, and will not convert it to a sequence of single-qubit measurements in the standard basis. We want to exploit the fact that multi-qubit Pauli measurement can be simulated more efficiently.

I found this PR is very helpful to cleanly write many calibration sequences in the experiments. I think now post rotations are okey since most backends support gate after measurement.

I'd like to draw the attention of the reviewers to my comments https://github.com/Qiskit/qiskit-terra/pull/5311#issuecomment-932893153.

Is there any reason for creating MeasurePauli instead of extending Measure with a basis parameter?

Actually, this is a a bad idea. Each basis is, in essence, a different instructions with different decompositions and backends (such as IBM Quantum provider) and plugins (such as QASM2) might support a subset of them. Taking this suggestion back.

Eventually, the goal of this PR is that for a simulator backend the transpiler will leave the instruction as-is, and will not convert it to a sequence of single-qubit measurements in the standard basis. We want to exploit the fact that multi-qubit Pauli measurement can be simulated more efficiently.

The multi-qubits feature makes this PR a bit tricky. My main red flag is that multi-qubits, in this context, implies multi-clbit
and that breaks some assumptions. For example, there is no support for drawing writing-into-multiple-clbits. This might sounds like a detail, but it might have deeper implications that are not so obvious.
Considering that and since it seems kinda trivial for a backend to detect a sequence of measures, is this feature worthy? If we are okey with removing it, then we just need to add the instructions MeasureX and MeasureY (MeasureZ could be some sort of alias of the current Measure) with their own decomposition in terms of Measure.

Considering that and since it seems kinda trivial for a backend to detect a sequence of measures, is this feature worthy? If we are okey with removing it, then we just need to add the instructions MeasureX and MeasureY

Sounds fine for me.
@nkanazawa1989 you've been involved
@chriseclectic this work started following a discussion with you
Do you (Chris & Naoki) agree?

Single qubit measure is okey with me --  as current measurement instruction does. User can write a macro for mutiqubit measure if they really want. The important point for me is pauli measure instruction will add more context to the circuit and they make the code and visualized circuit more readable.

As I said, I agree with the move to single-qubit. However I recommend to read a bit of discussion about it in #4024, which remained without conclusions.
What's everybody's opinion:

From user's perspective, a single instruction measure_pauli (or measure_pauli with parameters (as done in #4024) or separate measure_x, measure_y, measure_z instructions (as suggested in #3967) ?
Same question from internal implementation perspective - a single class PauliMeasure (or MeasurePauli) with parameters or separate classes MeasureX, MeasureY (in any case they will all inherit directly from Instruction) ?

Sorry to cut in, what about an idea in the middle? That is introducing measure_pauli accepting multi-qubit pauli measurement while it internally produces single-qubit PauliMeasures for now to avoid the drawer issue for mult-clbits. The direct creation of multi-qubit pauli measurement, e.g. PauliMeasure(basis="XYZ"), should raise "NotImplementedError" for now. In this way, once the drawer issue is fixed, we can introduce true multi-qubit pauli measurement in the future without a big change. Am I missing something?

closing in favor of https://github.com/Qiskit/qiskit-terra/pull/7716

I thought some qiskit-application developers may want a general PauliMeasure interface (but I myself had no concrete use case), while there is a clear need for single-qubit Measures from qiskit-experiments. So it makes sense to introduce single-qubit Measures at this moment. Provided we introduce single-qubit Measures, I think we should not introduce measure_pauli now (it should be added only in the case we introduce PauliMeasure in future)

Multiqubit Pauli measurements are essential for lattice surgery fault-tolerant QC. I hope that feature is integrated into Qiskit soon.
-A random person.

I thought some qiskit-application developers may want a general PauliMeasure interface (but I myself had no concrete use case), while there is a clear need for single-qubit Measures from qiskit-experiments. So it makes sense to introduce single-qubit Measures at this moment. Provided we introduce single-qubit Measures, I think we should not introduce measure_pauli now (it should be added only in the case we introduce PauliMeasure in future)

I thought some qiskit-application developers may want a general PauliMeasure interface (but I myself had no concrete use case), while there is a clear need for single-qubit Measures from qiskit-experiments. So it makes sense to introduce single-qubit Measures at this moment. Provided we introduce single-qubit Measures, I think we should not introduce measure_pauli now (it should be added only in the case we introduce PauliMeasure in future)

Multi-qubit Pauli measurements are essential for lattice surgery fault-tolerant QC (for example https://arxiv.org/pdf/1808.02892). While it is not relevant for current hardwares, Qiskit supporting multi-qubit Pauli measurements would be nice for those who are interested in making a compiler for FTQC. I hope this feature is integrated into Qiskit soon.
-A random person.


Chart does not reload properly

https://user-images.githubusercontent.com/44088388/181748221-77046722-74cc-432a-87a3-296fb5588a01.mov

Hi @bhesaniyakrunal,
can you post your code?

is there any fix for this issue ?


fix: delete the correct dashboard cache key

SUMMARY
This PR fixes a bug from #11234 where cache keys for dashboard bootstrap data are incorrectly deleted.
We noticed this when some dashboard failed to update cache when updating charts.
delete_memoized from Flask caching treats instance method and class method differently:
        Delete memoized is also smart about instance methods vs class methods.
        When passing a instancemethod, it will only clear the cache related
        to that instance of that object. (object uniqueness can be overridden
        by defining the __repr__ method, such as user id).
        When passing a classmethod, it will clear all caches related across
        all instances of that class.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A
TEST PLAN

Create a dashboard and add a chart
Change chart title in Dashboard edit mode
Explore the chart, make updates and save
Return to dashboard and refresh. In base, you may see the chart is not updated. After the fix, the chart should update.

ADDITIONAL INFORMATION

[x] Has associated issue:
[x] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Updated the cache key nonetheless!

Codecov Report

Merging #11273 into master will decrease coverage by 5.88%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master   #11273      +/-   ##
==========================================
- Coverage   65.60%   59.72%   -5.89%
==========================================
  Files         834      393     -441
  Lines       39566    24825   -14741
  Branches     3615        0    -3615
==========================================
- Hits        25959    14826   -11133
+ Misses      13498     9999    -3499
+ Partials      109        0     -109

Flag
Coverage Δ

#cypress
?

#javascript
?

#python
59.72% <0.00%> (-1.05%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

superset/models/dashboard.py
80.63% <0.00%> (ø)

superset/db_engines/hive.py
0.00% <0.00%> (-85.72%)
:arrow_down:

superset/databases/commands/create.py
31.91% <0.00%> (-59.58%)
:arrow_down:

superset/db_engine_specs/hive.py
53.90% <0.00%> (-30.08%)
:arrow_down:

superset/views/database/mixins.py
59.64% <0.00%> (-22.81%)
:arrow_down:

superset/db_engine_specs/presto.py
69.76% <0.00%> (-12.32%)
:arrow_down:

superset/db_engine_specs/mysql.py
79.59% <0.00%> (-12.25%)
:arrow_down:

superset/databases/commands/update.py
85.71% <0.00%> (-8.17%)
:arrow_down:

superset/databases/api.py
81.38% <0.00%> (-7.98%)
:arrow_down:

superset/databases/dao.py
94.11% <0.00%> (-5.89%)
:arrow_down:

... and 458 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9caf875...ed9a7a0. Read the comment docs.


Change copyright to "prometheus authors" or some inclusive group

Now that this project is in the prometheus-community org, I think that it would be good for the community to own the project and for the copyright to change from @slrtbtfs to e.g. The Prometheus Authors.
WDYT @slrtbtfs?

What would be the exact legal implications of this?
Since this project contains code from multiple contributors, would they have to agree for changing the copyright notice of existing files?


JS: Tokenize : as an operator

This resolves #2072.
This will make JS and JSON consistent in how they tokenize :.

To make this broader: I'm not in love with changing all : from punctuation -> operator because of ternaries. Their usage in object literals is far more common than in ternaries, so punctuation is more likely to be correct.

I am for changing : to operator because of consistency. JSON also uses operator and it ternaries look a little strange.
There is also the point that other highlighters also use something akin to operator, e.g. GitHub:
const value = { hello: 'world', foo: (bar ? 1 : 0) };

It's true that the JSON component already has it tagged as an operator for some reason.
I'm only worried that in the end, someone will open another issue asking for the exact opposite of this PR.

I agree that : should not be an operator when it is punctuation.
I'm not sure how PrismJS is tokenizing things, but,  I believe that inorder to fix this, you would have to not broadly assume : is punctuation, but take in prior tokenized items as context and apply the the correct token type "punctuation" or "operator"

Well, IMO the underlying issue here is that there is no clear-cut definition of what differentiates an operator from mere punctuation.
I always felt that this divide is kind of artificial but it's quite useful for highlighting.
So my question: What defines an operator vs punctuation?
The JS language spec doesn't.

It's true that the JSON component already has it tagged as an operator for some reason.
I'm only worried that in the end, someone will open another issue asking for the exact opposite of this PR.

I am afraid of this happening too.
The reason why I've brought this up in an issue is because our UX designer is looking at our code snippets on our site and not seeing parity github (we are using github's theme) - the first thing they pointed out to us was that our colons were not matching.  Since they are going over this with a fine-toothed comb, I'm sure if it does happen, it will happen much sooner rather than later lol.

@RunDevelopment The JS spec does define it, I believe. The colon is part of the Conditional Operator (12.14) which is an operator as stated, but also part of the PropertyName of an ObjectLiteral (12.2.6) in which case nothing says it's an operator.
But the spec also defines a comma operator, which we do not handle. x')
@MattMcFarland The thing is Prism is regex-based, it has no notion of context. Absolute correctness is not a goal for this project, as Lea once said.
If we can handle some cases of colons used as operator, without adding too much complexity, that's good. But we most likely won't be able to handle the generic case.
If your UX designer expects your code snippets to be highlighted with absolute correctness, then I'm afraid Prism is not the right tool for the job.

Well, IMO the underlying issue here is that there is no clear-cut definition of what differentiates an operator from mere punctuation.
I always felt that this divide is kind of artificial but it's quite useful for highlighting.
So my question: What defines an operator vs punctuation?
The JS language spec doesn't.

Having not looking at the spec myself, I believe you.  I think you have a very convincing argument for broadly assuming : is an operator.   Even if someone comes back saying they think it should be punctuation, it looks like : is used as an operator for github's syntax highlighting, if you look at this snippet:
const foo = bar ? 0 : 1
const baz = {
  stuff: things,
  things: 'stuff',
  doStuff: () => MyThing.doStuff()
}

@Golmote Please.. Say it ain't so!  I'd much rather stick with prism if I can.  :) I do think that if there are no negative consequences for switching : to operator from punctuation, then you should do it.  It does look like it is treated like operator when looking at github snippets at least.

I also agree that we shouldn't change our tokenization to fit a specific theme.
I implemented this because of consistency.

@Golmote
The spec even has this punctuator section.

@Golmote @mAAdhaTTah
Do we have some guidelines to distinguish between punctuation and operator?
Because when I create a language, I just choose whatever I think looks better in Prism's themes. Even though this kinda goes against 'we shouldn't change our tokenization to fit a specific theme'.

I agree you shouldnt change tokenization to fit a specific theme as well.
When I bring up github's theme and am looking for parity, it's only to show my use case.
As for the issue, I made it a point to leave theme's out of it and just stick with the facts, where an operator was being misread as punctuation.  That's what I was hoping a fix for :)
As for parity with Github, that's something my UX designer is using as a benchmark.  If they can give us some leeway that would be great, if not, it would just create more work on our board that we would probably have to defer.

Consistency is a good enough argument to me, honestly. So I won't fight against this PR.
@RunDevelopment No we don't have guidelines for this. I personnally tried to stick to the specs, whenever possible, but I might have made aesthetic choices too, from time to time. Again, given Prism is no linter, that's perfectly fine.

I'm not sure that spec reference is that helpful, cuz it shows some tokens that are definitely operators, like spread .... Broadly, what is an operator is defined the spec. It sounds like punctuation is... made up? for the benefit of syntax highlighting?
@RunDevelopment So no, I don't think we have guidelines for "punctuation".

It does look like it is treated like operator when looking at github snippets at least.

It looks this way because the GitHub theme highlights both with the same color. The ? is also highlighted in that color, so hypothetically, if whatever highlighter GH uses highlighted them as different things but applied the same color, the result would be the same while "treating" them as different.

If you need to satisfy your designer, you could easily just change the color of punctuation & operator to be the same in your theme. No conflict 😄

@mAAdhaTTah I tried that but it didn't work.  It changes commas and dots to red (github treats them differently than operator)

@MattMcFarland Maybe we could add a bit functionality similar to Highlight keywords for general tokens to satisfy your UX designer? I'm thinking about the ability to modify the classes of tokens on a per-language level.
Anyway, we shouldn't discuss this here.

@Golmote

I personally tried to stick to the specs, whenever possible

Me too but most specs just call everything an operator that's not very applicable. Also, if present, what is punctuation vs operators varies from spec to spec.

Ah so maybe GH's highlighter is treating : as an operator.

Ah so maybe GH's highlighter is treating : as an operator.

Seems like it. It's highlighted like = but differently from (){}.

Inconsistency between JS & JSON is a bit weird. We should probably bring them into alignment. The issue for me is it seems weird to apply ": is an operator" to JSON because it's definitely not an operator there. I know this flies in the face of the original request, but my inclination is to make them both punctuation.

Funnily enough, the Rouge syntax highlighter switched from having ? and : tagged as operators to having them both tagged as punctuation in 2012.

lol there's your consistency: make ? punctuation!

make ? punctuation!

We can also do that.
But with that, I have another question: What about optional chaining and nullish coalescing?
?? is an operator for sure but ?. ?

sorry sorry i was kidding 😄

Seems like I didn't get the joke... Well, happens.
Anyway, so what do want to do?
About ?? and ?.: I'm planning to add both with either this PR or another one, so I just wanted some input because the result of this will determine their fate (as tokens).

To me, ?? is an operator while ?. is punctuation. But I'll admit I have not much to back it up.
Regarding :, consistency with JSON seems relevant. We can either change the JSON one to punctuation or the JS one to operator. I'm fine with both solutions, but I think @mAAdhaTTah has a point saying that it's semantically not an operator in JSON at all.

Also, @RunDevelopment I like your idea that this could be solved using some smart plugin, that would give all the liberty they want to the users.
Another solution could be to add two classes on just the colon symbol, since we consider it special. Like punctuation colon. This is a bit far-fetched, but it would allow users to override only its style.

I think @mAAdhaTTah has a point saying that it's semantically not an operator in JSON at all.

? and : and being punctuation is JS and JSON it is?
Just want make sure. I want them to be consistent with both JSON and each other.

No, I meant only change : to a punctuation in JSON. No need to mess with ? at all.

Seems like I won't get my consistent ?:.
I'll cry in a corner then. 😢

both JSON and each other

Oh, I missed that part of your sentence indeed, sorry about that.
@mAAdhaTTah Is it that bad to have : tagged as an operator solely for aesthetic purpose?

Seems like the discussion isn't over. 😄
But on the point of : being an operator: When we then consider other C style languages such as C/C++ where :: is an op but : is punctuation even though : can only be part of ?: to my knowledge.
And regarding JSON: In Yaml, a superset of JSON, : is being highlighted as punctuation.
Point is: If we really want to make it consistent, we have a looot of languages to go through where we can have the same discussion. Maybe we should create guidelines so we can make all languages roughly consistent with each other and not just JS and JSON.

In C/C++, the colon isn't only used in ternary. (see https://stackoverflow.com/questions/1711990/what-is-this-weird-colon-member-syntax-in-the-constructor for example)
Guidelines would be nice, but what would they be based of? What kind of generic property can be used to definitively make a choice here (and one that hopefully doesn't require a real deep knowledge of each language)?

Seems like the discussion isn't over. 😄

Hah yeah I didn't mean to imply I was making a definitive answer here.

Part of me was thinking guidelines would be helpful here, but I actually don't know we could come up with guidelines that apply broadly enough to be useful.

@mAAdhaTTah Is it that bad to have : tagged as an operator solely for aesthetic purpose?

No. The goal of Prism has never been correctness, so I don't really mind doing it for that reason either. A goal of consistency leads me to suggest punctuation for :, but if the goal is aesthetics then operator may be the way to go here.
Y'all are the regex experts, but I'm assuming it's impossible to highlight the ternary's colon as an operator separately from an object literal's colon as punctuation, correct?

TIL YAML is a superset of JSON.

Y'all are the regex experts, but I'm assuming it's impossible to highlight the ternary's colon as an operator separately from an object literal's colon as punctuation, correct?

We can certainly handle simple cases of ternary, like literals. But we most likely won't be able to handle every complex expression, especially expressions containing other colons. Furthermore ternaries can be infinitely nested, which we can't handle either.

Suff like
a ? {b: function() { label: return 1; }, c: /d:/ ? e : "f:g"} : h;

Ugh.

I'm assuming it's impossible to highlight the ternary's colon as an operator separately from an object literal's colon as punctuation, correct?

The problem, like @Golmote said, is that there can be any valid JS expression between  ? and :.
The infinite nesting problem can be solved by just stopping after a fixed level of nesting.
I actually wrote a little function which could transform a CF grammar into a RE with that idea. The pattern of a JS expression with a max nesting level 3 was >10kB if I remember correctly. Nesting with multiple branches means that our pattern grows exponentially. It's not fun and it becomes even worse because when I tested this with Chrome, it just wouldn't work if patterns were too long.
Point is: While possible, please don't.

A goal of consistency leads me to suggest punctuation for :

@mAAdhaTTah Why is punctuation consistent for :? Do you mean consistent with other languages? Because most C style languages have the same problem.
This is why I said that making ? punctuation isn't such a bad idea. And we could even change this in C like to apply it to all C style languages.
It's just a little... strange. Because ?: is definitely an operator.

Regarding guidelines:
I don't think we can do much better than trivial stuff like ()[]{},; is punctuation.
But then there are languages like EBNF where , is the concatenation operator... (I tokenized it as punctuation despite that.)
The only thing we might be able to do is to go for aesthetics and utility (aka not too ugly and not all the same token type).

I think I've got a principle we can apply that gets us : as an operator:

Don't make any token "punctuation" if it fits into another token type. In other words, "punctuation" is a token of last resort.

If we accept this principle, we have a guideline for deciding what "punctuation" is in the future while giving us the outcome we want on JS & JSON (this should stay as an operator to be consistent).

I like that principle @mAAdhaTTah

@mAAdhaTTah Your principle sounds good.
I'll make an issue, so we can collect these guidelines and put them on the website once we have enough.

If we have docs on making a new language, that's where they should live. If not, then... well, we need that lol.

We only have https://prismjs.com/extending.html AFAIK.
@mAAdhaTTah's idea for the guidelines sounds good to me. This way we have the best of all worlds: this issue can be fixed, we have JS/JSON consistency and a rule to follow that "makes sense".
I still think a plugin that extends the possibilities for the users would be a nice addition.

Speaking of guidelines, @RunDevelopment what rule are you following to order the operator regexp parts, if any?

what rule are you following to order the operator regexp parts, if any?

To order? You mean why a|b instead of b|a?
Well, above all: correctness. E.g. -|-- doesn't match --, so it has to be --|-. After correctness is ensured, I generally order them to group together as many as possible to make the regex as small as possible. E.g. [!=]=|- instead of !=|-|==.
After that is done, nothing really. I just choose whatever order as long as it's correct and fairly small.

@Golmote

I still think a plugin that extends the possibilities for the users would be a nice addition.

#2075.

I'll merge this now and will make another PR for ?? and ?..
We can continue the guideline discussion in #2083.

To order? You mean why a|b instead of b|a?
Well, above all: correctness. E.g. -|-- doesn't match --, so it has to be --|-. After correctness is ensured, I generally order them to group together as many as possible to make the regex as small as possible. E.g. [!=]=|- instead of !=|-|==.
After that is done, nothing really. I just choose whatever order as long as it's correct and fairly small.

Fair enough. At one point in time, I tried to group them by their starting character(s), because that would theoretically be more optimized (least amount of backtracking needed when testing each alternative).

#2075.

You're far too quick and I'm far too blind! :joy:

that would theoretically be more optimized (least amount of backtracking needed when testing each alternative).

I would really like to know how they optimize this under the hood. Most regexes are fairly simple and could easily be transformed into a DFA.
Time to look into the V8 source I guess.

You're far too quick

Gotta go fast! blue hedgehog noises

I would really like to know how they optimize this under the hood. Most regexes are fairly simple and could easily be transformed into a DFA.
Time to look into the V8 source I guess.

From "Mastering Regular Expressions (2nd edition)", by Jeffrey E. F. Friedl:

[...]

I don't know how true are those statements but this is definitely interesting.

I now know where to find regex ninjas ;)

@MattMcFarland Yeah dude, these two are so good at this. I'm mostly just here to make sure things move along.
If you wanna learn regex, look over any of the PRs with extensive comments. Lotta good insights.

@Golmote
I did some perf tests.
sample        | func          | comp          | avg        | dev        | min        | max        | samples
------------- | ------------- | ------------- | ---------- | ---------- | ---------- | ---------- | ----------
prism-core.js | old operator  | 100% +-21%    | 0.07933    | 0.01674    | 0.04493    | 0.5169     | 1318
              | new operator  | 102% +-13%    | 0.08099    | 0.009933   | 0.04720    | 0.2069     | 1335
              | flat operator | 101% +-15%    | 0.08043    | 0.01168    | 0.04720    | 0.2379     | 1329
prism.js      | old operator  | 100% +-9%     | 3.470      | 0.3251     | 3.247      | 5.950      | 396
              | new operator  | 101% +-9%     | 3.503      | 0.3267     | 3.248      | 5.839      | 394
              | flat operator | 101% +-9%     | 3.495      | 0.3012     | 3.274      | 5.523      | 394

avg (average), dev (standard deviation), min, and max are all in ms. comp is the relative average and standard deviation compared to the lowest average.
old operator is the operator pattern before this PR, new operator is the operator pattern after this PR sans :, and flat operator is just all operators as a keyword list of sorts.
prism-core.js is what it sounds like from master, and prism.js is the current Prism version from the download page will all languages included.

Testing code
const fs = require("fs");
const { performance } = require("perf_hooks");
// force the lazy init to happen
performance.now();

const samples = fs.readdirSync('./samples').filter(f => !/^_/.test(f)).map(f => {
	return {
		title: f,
		value: fs.readFileSync('./samples/' + f, 'utf8')
	}
});

benchmark(samples, [
	toTestCase(
		"old operator",
		/-[-=]?|\+[+=]?|!=?=?|<<?=?|>>?>?=?|=(?:==?|>)?|&[&=]?|\|[|=]?|\*\*?=?|\/=?|~|\^=?|%=?|\?|\.{3}/g
	),
	toTestCase(
		"new operator",
		/--|\+\+|\*\*=?|=>|&&|\|\||[!=]==|<<=?|>>>?=?|[-+*/%&|^!=<>]=?|\.{3}|[~?]/g
	),
	toTestCase(
		"flat operator",
		/--|-=|-|\+\+|\+=|\+|!==|!=|!|<<=|<<|<=|<|>>>=|>>>|>>=|>>|>=|>|===|==|=>|=|&&|&=|&|\|\||\|=|\||\*\*=|\*\*|\*=|\*|\/=|\/|~|\^=|\^|%=|%|\?|\.\.\./g
	),
]);

/**
 *
 * @param {string} title
 * @param {RegExp} regex
 */
function toTestCase(title, regex) {
	return {
		title,
		value: text => allMatches(regex, text),
	};
}

/**
 *
 * @param {{ title: string; value: T }[]} testSamples
 * @param {{ title: string; value: (value: T) => void}[]} testFunctions
 * @template T
 */
function benchmark(testSamples, testFunctions) {
	const warmupIterations = 1_000;
	const warmupMaxTime = 200;

	const warmupSample = testSamples[0].value;
	testFunctions.forEach(({ value }) => {
		const start = performance.now();
		for (let i = 0; i < warmupIterations; i++) {
			if (performance.now() - start > warmupMaxTime) {
				break;
			}
			value(warmupSample);
		}
	});

	const maxSamples = 10_000; // samples
	const maxTimePerFunc = 2_000; // ms

	const columnWidth = [
		Math.max(...testSamples.map(s => s.title.length)),
		Math.max(...testFunctions.map(s => s.title.length)),
	];

	function printLine(...cells) {
		let s = "";
		for (let i = 0; i < cells.length; i++) {
			let cell = String(cells[i]);
			if (i < columnWidth.length) {
				cell = cell.padEnd(columnWidth[i], " ");
			}
			if (s) {
				s += " | "
			}
			s += cell;
		}
		console.log(s);
	}

	let headerPrinted = false;

	/**
	 * @param {{ title: string; value: T }} sample
	 * @param {{ title: string; value: (value: T) => void}} func
	 * @param {Object<string, any>} result
	 */
	function printResult(sample, func, result) {
		if (!headerPrinted) {
			columnWidth.push(...Object.keys(result).map(k =>
				Math.max(10, k.length, Math.ceil(String(result[k]).length * 1.25))
			));
			printLine("sample", "func", ...Object.keys(result));
			printLine(...columnWidth.map(i => "-".repeat(i)));
			headerPrinted = true;
		}
		const cells = [
			func === testFunctions[0] ? sample.title : "",
			func.title
		];

		for (const key in result) {
			cells.push(result[key]);
		}
		printLine(...cells);
	}

	/**
	 * @param {Object<string, any>[]} results
	 */
	function printResults(sample, results) {
		const min = Math.min(...results.map(r => r.comp[0]));
		results.forEach(r => {
			const [avg, dev] = r.comp;
			r.comp = `${Math.round(100 * avg / min)}% +-${Math.round(100 * dev / min)}%`;
		});
		results.forEach((r, i) => {
			printResult(sample, testFunctions[i], r);
		})
	}

	for (const sample of testSamples) {
		const results = [];
		for (const func of testFunctions) {
			/** @type {number[]} */
			let samples = [];
			const samplingStart = performance.now();

			for (let i = 0; i < maxSamples; i++) {
				if (performance.now() - samplingStart > maxTimePerFunc) {
					break;
				}
				global.gc && global.gc();

				let start = performance.now();
				func.value(sample.value);
				samples.push(performance.now() - start);
			}

			const min = Math.min(...samples);
			const max = Math.max(...samples);
			const avg = samples.reduce((x, y) => x + y) / samples.length;
			const dev = Math.sqrt(samples.reduce((s, x) => s + (x - avg) ** 2) / (samples.length - 1))

			const prec = (n = 0) => n.toPrecision(4);

			results.push({
				comp: [avg, dev],
				avg: prec(avg),
				dev: prec(dev),
				min: prec(min),
				max: prec(max),
				samples: samples.length,
			});
		}
		printResults(sample, results);
	}
}

function allMatches(re, text) {
	if (!re.global) {
		throw new Error(`RegExp has to be global! ${re}`);
	}
	re.lastIndex = 0;
	while (re.exec(text)) { }
}

Run using Node.js v12.11.0 with the following command: node --expose-gc bench.js.
I take control over the GC (as much as I can) to avoid minor GCs during testing.
bench.zip

Backtracking optimization or not doesn't seem to matter.
While your pattern generally seems to be just a tat faster than the other ones (like 1% faster), it's hard to tell because of the noise.
To reduce this noise, I experimented a little with v8's GC and got it down by ~10% but there's still too much to conclusively say that your pattern is faster.

@RunDevelopment Nice! :open_mouth:
The results do not confirm my theory that much. I guess the engine optimizes those patterns indeed!
The conclusion seems to be the same in Firefox too.


Windows bot does not provide correct preview links

The links provided by the new Windows bot when creating a preview do not actually link to the generated preview, but result in a 404. Example: https://github.com/mozilla/pdf.js/pull/8012#issuecomment-276801814

Looks like zip is missing on the bot. I'll try to figure it out tomorrow and why the error didn't show up in the log.

Should be fixed now.


Copy and paste doesn't just work in Eclipse IDE

Environment
Windows build number: Microsoft Windows [versão 10.0.21390.2025]
Your Distribution version: Ubuntu 20.04
Your WSLg version: 1.0.22

Steps to reproduce
Copying and pasting works with Gedit, Intellij and any other app. But with Eclipse IDE it doesn't work.
https://user-images.githubusercontent.com/235259/123024773-fbaba600-d3af-11eb-893e-bfe21e0302e5.mp4
Follow the feedback hub link
https://aka.ms/AAczzv8
WSL logs:
pulseaudio.log
weston.log
versions.txt
Expected behavior
Copy and paste text from anywhere into the Eclipse IDE.
Actual behavior
It is only possible to copy and paste text from within the text editor windows of the Eclipse IDE itself. If text is copied from elsewhere, it is not possible to paste into the Eclipse IDE. For example, if you copy text from any webpage, it is not possible to paste it into the Eclipse IDE.

Plenty of copypaste scenarios were fixed for me in version 1.0.24. You can install it from https://github.com/microsoft/wslg/releases/download/v1.0.24/wsl_graphics_support_x64.msi

@rodrigopizzi, thanks for reporting the issue, I confirmed this doesn't work with the latest WSLg release and I will look into. Thanks for trying out WSLg!