"io.UnsupportedOperation: read" when processing log statement for s3 multipart upload

Problem description

What are you trying to achieve?
I use airflow MsSQLToS3Operator which in turn depends on smart_open to save extracted SQL Server datasets to s3.

What is the expected result?
The expected result is that the dataset that I extract will be uploaded to s3

What are you seeing instead?
I get the following stack-trace

[2021-06-09 09:43:17,500] {mssql_to_s3_operator.py:276} INFO - Uploading!
[2021-06-09 09:43:17,657] {taskinstance.py:1481} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/airflow/models/taskinstance.py", line 1137, in _run_raw_task
    self._prepare_and_execute_task_with_callbacks(context, task)
  File "/usr/local/lib/python3.6/dist-packages/airflow/models/taskinstance.py", line 1311, in _prepare_and_execute_task_with_callbacks
    result = self._execute_task(context, task_copy)
  File "/usr/local/lib/python3.6/dist-packages/airflow/models/taskinstance.py", line 1341, in _execute_task
    result = task_copy.execute(context=context)
  File "/home/developer/airflow/dags/modified_operators/mssql_to_s3_operator_mod.py", line 100, in execute
    self.build_fetch_query(hook)
  File "/home/developer/airflow/dags/modified_operators/mssql_to_s3_operator_mod.py", line 169, in build_fetch_query
    self.get_records_batch(hook, query_filter)
  File "/home/developer/airflow/dags/modified_operators/mssql_to_s3_operator_mod.py", line 277, in get_records_batch
    fout.write(results)
  File "/home/developer/.local/lib/python3.6/site-packages/smart_open/s3.py", line 839, in write
    self._upload_next_part()
  File "/home/developer/.local/lib/python3.6/site-packages/smart_open/s3.py", line 870, in _upload_next_part
    self._total_bytes / 1024.0 ** 3,
  File "/usr/lib/python3.6/logging/__init__.py", line 1308, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/lib/python3.6/logging/__init__.py", line 1444, in _log
    self.handle(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1454, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1516, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 861, in handle
    rv = self.filter(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 720, in filter
    result = f.filter(record)
  File "/usr/local/lib/python3.6/dist-packages/airflow/utils/log/secrets_masker.py", line 157, in filter
    record.__dict__[k] = self.redact(v)
  File "/usr/local/lib/python3.6/dist-packages/airflow/utils/log/secrets_masker.py", line 203, in redact
    return tuple(self.redact(subval) for subval in item)
  File "/usr/local/lib/python3.6/dist-packages/airflow/utils/log/secrets_masker.py", line 203, in <genexpr>
    return tuple(self.redact(subval) for subval in item)
  File "/usr/local/lib/python3.6/dist-packages/airflow/utils/log/secrets_masker.py", line 205, in redact
    return list(self.redact(subval) for subval in item)
  File "/usr/local/lib/python3.6/dist-packages/airflow/utils/log/secrets_masker.py", line 205, in <genexpr>
    return list(self.redact(subval) for subval in item)
io.UnsupportedOperation: read

Analysis performed
I can see in the code for mssql operator, this call.
    with smart_open.smart_open(url, "wb") as fout:

    # In s3.py open()
        if mode not in constants.BINARY_MODES:
        raise NotImplementedError('bad mode: %r expected one of %r' % (mode, constants.BINARY_MODES))

I have tried to change "wb" to "wb+" which might fix the problem but it doesn't allow me further down in the smart_open call-stack.
The error message is coming from the following snippet in s3.py
    #
    # Internal methods.
    #
    def _upload_next_part(self):
        part_num = self._total_parts + 1
        logger.info(
            "%s: uploading part_num: %i, %i bytes (total %.3fGB)",
            self,
            part_num,
            self._buf.tell(),
            self._total_bytes / 1024.0 ** 3,
        )

I think that the bug might be exposed by the secrets_masker.py class from airflow, however I'm unsure if the bug is there or if it is from smart_open itself?
I should mention that this worked fine in Airflow 2.0.1 and 2.1.0 has exposed this problem. I have tried in a docker container with python 3.6, 3.7 and 3.8 but the same problem regardless.
Versions
Linux-5.10.25-linuxkit-x86_64-with-glibc2.2.5
Python 3.8.10 (default, May 12 2021, 15:56:47)
[GCC 8.3.0]
smart_open 5.1.0

Thank you for the detailed report. I'm still unsure about what's happening here. The biggest mystery for me is:

You're uploading a file, so smart_open should be writing, but
The error is "io.UnsupportedOperation: read", so something is trying to read from a write-only stream

It's possible that the newer version of Airflow is expecting the log stream to be read-writeable (why?), but smart_open doesn't support this for S3 yet. Can you investigate and let me know if this hypothesis is correct? Depending on the answer, we have several options:

Treat this as a problem in Airflow, and see if it's possible for them to adjust their logging
Treat this as a problem in smart_open, and make read-write streams work for S3
Treat this as a problem in your application, and write logs to e.g. local disk before uploading to S3

We’ll try to fix this on Airflow’s side.


Dynamic Content

I frequently find myself wanting to be able to modify the DOM content of a strapdown document on the fly. For instance, when creating a SIP, I don't want to create a nearly identical document for each environment. I'd rather create a master document, with placeholders, and have a way to select the environment (say with a select box) which triggers replacing a placeholder with its appropriate value.
Has anyone figured out a decent way to do this?

@shawncplus IMHO being able to load markdown from another location is a very nice feature for strapdown.js ... guys, could be it possible to merge this into strapdown?

Ain't strapdown incredibly sexy because of it's feature-less-ness?

It is always a complex discussion wether to add, or not, features to any technology... I think it is a very useful and simple addition to strapdown... but is just my opinion :smile_cat:

I might try out the Bootmark fork (which includes external sources) and requesting features there, since this repo isn't getting any love lately.

i'll leave this here
it's pretty useful
bootmark


Index templates: support 'ensure => latest' to force (re)installation

Hi,
I've encountered a scenario where I deployed new index templates and due to the 'ensure => present' default, they weren't loaded to ES.
Appreciate comments/opinions.
Best,
Yarden

The custom type should be ensuring that the content in elasticsearch and the content you've defined in the template should be synced, so it should be the same either way.
If you'd like some help, I'd need some example puppet manifests/logging output/template content from Elasticsearch/etc.


Add missing space around content on subscriber page

Issue Reference

Description

All other similar admin edit pages have a mandatory space around its content.
How To Test This?
Try to open a page with editing any newsletter subscriber:

Before
After

Documentation

[ ] My pull request requires an update on the documentation repository.

@lex111
This is working fine, please find the screenshot Here
Thanks for your contribution.


ERROR Error: Uncaught (in promise): Firebase analytics is not initialized. Make sure initializeFirebase() is called once

Describe the bug
Hi there!
I'm using this plugin in my Capacitor app, it works great but every time I start the app y get the following message on console
ERROR Error: Uncaught (in promise): Firebase analytics is not initialized. Make sure initializeFirebase() is called once
To Reproduce
Steps to reproduce the behavior:

Open the app
See the console logs

Expected behavior
Don't show any error message on the console
Screenshots

Desktop (please complete the following information):

OS: Mac OS 14.4.1 (23E224)
Browser Chrome
Version 124.0.6367.119

Additional context
Plugin version: 4.0.0
Capacitor version: 4.0.0

If you're running it in the browser while developing it then are running it on the "web" platform so you need to call this.
Alternatively you can wrap you firebase calls to confirm that this is running on device.
For my code, I don't need it on the web, its only for on device builds so I made a service that wraps my firebase calls, and skips it if its not the right platform:
import { Injectable } from '@angular/core';
import { FirebaseAnalytics } from '@capacitor-community/firebase-analytics';
import { Platform } from '@ionic/angular';

@Injectable({
  providedIn: 'root',
})
export class FirebaseAnalyticsLoggerService {
  constructor(private platform: Platform) {}

  async logEvent(name: FirebaseAnalyticsEventTypes, value: any) {
    await this.platform.ready();

    if (!this.platform.is('hybrid')) {
      console.log('Firebase Analytics aborted: not running on a device.');
      return;
    }

    try {
      const result = await FirebaseAnalytics.logEvent({ name, params: value });
      console.log('Firebase Analytics Result: ', result);
    } catch (error) {
      console.error('Firebase Analytics Error: ', error);
    }
  }
}

This is for an ionic / angular app, if you're working in something else then I believe capacitor has its own platform class.


Feature Request: Remove usage of react-helmet

Hi!
With Gatsby 4.19 a Head API was released: https://www.gatsbyjs.com/docs/reference/release-notes/v4.19/#gatsby-head-api
I'm slowly removing react-helmet from all my projects and replace it with the Head API. To be able to completely remove that react-helmet dependency this plugin would need to not rely on react-helmet.
While you can't use Gatsby Head API for your plugin, you can use the normal setHeadComponents API. Or did you run into any problems when using that and opted for react-helmet? :)
Happy to help with guidance on how to remove that peerDep.
Thanks!

Hi Lennart. Thanks for the heads up. I've haven't been following Gatsby's development in the past few months, but I'm happy to improve the plugin if needed. I'm away on holiday for the next 10 days, hopefully I'll be able to pick this up once I'm back. I'll let you know if I run into any issues.

Awesome, enjoy your vacation :)

Hi @codeAdrian, I'm seeing if you have had any chance to look at this.  I've been a super happy user of this plugin, and I'm migrating all of my code away from React Helmet and moving towards the native Gatsby head approach and I'm hoping to see this move in the more modern direction as well ;)
Cheers! Thanks for a great product

Hi, same here!

Another +1 - using Gatsby v5 and would love to remove react-helmet 👍🏼

another plus 1


Cut a new release?

Would it be possible for a new release to get cut for intellij/pushed? Some of the changes on master I'd like to see pushed out if possible.
Thanks

Sorry for the delay replying. Our external release schedule is tied to our internal releases. The next release will be late next week, but thereafter should be more regular again, once every two weeks.


Fully functional rapid building for Android on Windows!

Now all that's left is getting the Windows template project to compile.

Closing this one so I can submit the pull request with the ability to compile Windows projects.


Investigate python dependencies on a fresh machine

On my new fresh workstation I hit quite a few issues when trying to run grunt.

I hit the following error in requirments.py which is a built in package:

File "/usr/local/lib/python2.7/dist-packages/packaging/requirements.py", line 59, in <module>
    MARKER_EXPR = originalTextFor(MARKER_EXPR())("marker")

Fixed it by adding "" to MARKER_EXPR() in requirements.py as suggested in https://github.com/nylas/sync-engine/issues/432#issuecomment-275931718

There was a version conflict with the six package. Fixed it by uninstalling and installing again.

I hit some other issues as well but those might have been a result of me trying to installing sub packages manually which we should not do.

Closing due to age.


Search results not working in Safari Browser (14.0.3)

Hi,
really like your theme, but the search result links are not working in Safari. Am I missing something?
Anyhow, thanks for your effort.

Thanks for your report. Sadly it's hard to debug for me as I don't have an Apple device. Anything in the browser console? Does the hover work and is the target of the link displayed? Or how does it look like?

Can't see anything in the console. The hover (dotted underline) is displayed correctly, but upon click the dropdown closes and nothing happens.
It used to work in the beginning that is why I am using 0.7.3.

@LukasHirt Maybe you have a few minutes to look into this? Any idea?

Maybe you have a few minutes to look into this? Any idea?

@xoxys Sure, I'll take a look :wink:

Ok I was able to reproduce the issue on my machine using Gnome Web Browser which is also using WebKit. Looks like it's a general WebKit rendering issue...

@reisixx CI is preparing v0.10.1 release now. Would be good if you could test it.

CI is preparing v0.10.1 release now. Would be good if you could test it.

You were fast. I did some research as well and it seems that with safari tabindex needs to be supplied to the children so that it works correctly. But if it works fine with your solution then great!

 Setting a tabindex to the a elements seems to be working as well. Using tab index it doesn't even matter if it is set correctly or not. Adding tabindex="0" to every link is enough to make it clickable.... Do you have a source with some explanations why this works or is necessary?

Do you have a source with some explanations why this works or is necessary?

@xoxys Not sure TBH. I have found this suggestion in this stackoverflow https://stackoverflow.com/a/66035686 Somehow it seems that the focus has not been captured on the links which resulted in the dropdown being hidden. Setting the tabindex explicitly specifies that the item is focusable.

Really strange :smile: Anyway, thanks for your research! As .has-hits:active seems to have the same effect (keep the focus on the overlay search result list active until the link was really clicked) I'll keep this way for now.


Add CI badges and categories for crates.io

Fixes #177
r? @philipc

Coverage increased (+0.02%) to 93.946% when pulling afa38aca1f0ea37ff19701ff69f1d6a376ff8214 on fitzgen:badges-and-categories into c681e64a89e9bcefd6624a3c165bb14f7206a325 on gimli-rs:master.

Coverage decreased (-0.003%) to 94.135% when pulling f678abdd760badda1fa51d1c19abff282a582c5e on fitzgen:badges-and-categories into 6ece0f0c4690cae545c67c75559435d3495a1b9c on gimli-rs:master.


some problem in ros2 humble

i try to run the node in ros2 humble, but when i colcon build it, it shows some error:
`
CMake Error at CMakeLists.txt:28 (find_package):
By not providing "Findmpi_cmake_modules.cmake" in CMAKE_MODULE_PATH this
project has asked CMake to find a package configuration file provided by
"mpi_cmake_modules", but CMake did not find one.
Could not find a package configuration file provided by "mpi_cmake_modules"
with any of the following names:
mpi_cmake_modulesConfig.cmake
mpi_cmake_modules-config.cmake

Add the installation prefix of "mpi_cmake_modules" to CMAKE_PREFIX_PATH or
set "mpi_cmake_modules_DIR" to a directory containing one of the above
files.  If "mpi_cmake_modules" provides a separate development package or
SDK, be sure it has been installed.
`
do you know how can I fix it ?

I think your host is missing the mpi lib installed. Please install it and try again


Restrict number of used cpu-cores

Hi,
is it possible to restrict the number of used cpu-cores for parallel processing like it is possible with RSTAN?
See for example: https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started' and within there the option:
options(mc.cores = 80)
Thank you very much for your help!

Hi @PToTheK , I believe if you start Julia with julia -p auto it will create 4 threads on the Julia side (if you run 4 chains). There may be some additional threads if you compiled cmdstan multi-threaded.
If you start with julia -p 2 and look in the e.g. bernoulli_run.log you can see that when cmdstan is pulling draws it seems to use only 2 threads (instead of 4 earlier).
That is the (course) level of granularity control possible. I must admit, I wrote this Julia code over 10 years ago and haven't changed it much since then while a lot of work had been done on map and threads.

Thank you very much for the fast answer!
I would like to ask you a followup question.
Does your code run each chain on a different core or thread?
Some context:
I am analysing a multimodal model.
Therefore I want to use 100 chains, wich should be evaluated in parallel on a condor-cluster.

HI @PToTheK Yes, each chain is a thread and allocated to cores as available.
Over time I've helped out in a few cluster setups. Best thing is to have a small test framework to check when your cluster needs specific things.

Hi @goedman, thank you very much once again for your quick and precise answer.
This answer helps a lot!
Thank you very much for this great interface to STAN!


[internal-tools] treemirror adjusts the sources array in *.map files to be relative to their new location

This relates to issue #1880.
Goal is for treemirror to adjust the sources array in the *.map files it's copying to be relative to their destination location.
/root
  /development
    /index.d.ts.map
    /folder/index.d.ts.map
  /src
    /index.ts
    /folder/index.ts

After running in root treemirror development . \"**/*.d.ts.map"
/root
  /index.d.ts.map
  /folder/index.d.ts.map

sources array in /root/index.d.ts.map should have changed from: ['../src/index.ts'] to: ['./src/index.ts']
sources array in /root/folder/index.d.ts.map should have changed from: ['../../src/folder/index.ts'] to: ['../src/folder/index.ts']
Appears to be working for me on both windows and mac but would definitely appreciate another pair of eyes on it to make sure its working as expected.

This is old and probably irrelevant now so closing


Save state of expert modus in snippet module

1. Why is this change necessary?
I guess everyone hate it, that the expert mode in snippet module does not save
2. What does this change do, exactly?
Saves the state of snippet module expert mode
3. Describe each step to reproduce the issue or behaviour.
4. Please link to the relevant issues (if any).
5. Which documentation changes (if any) need to be made because of this PR?
6. Checklist

[ ] I have written tests and verified that they fail without my change
[ ] I have squashed any insignificant commits
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[X] I have read the contribution requirements and fulfil them.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/SW-20598
Please use this issue to track the state of your pull request.

Thanks @shyim! 💙


Debug message not full

Hi,
On the version 0.4.3 the debug messages contains way less informations that in the previous version.
Moreover, the FFMpeg command given by the debug log isn't complete.
Please see the difference on the attached picture. Version 0.4.3 on the left, 0.4.2 on the right.

Fixed in 464443af0d9e124ca6bbd75064ab18f17c3cf1d3


Workflow is causing Alfred to run multiple times

It seems that your awesome workflow is causing Alfred to run multiple times. Which results in the text expender to past 3 times the output I need.
It is exactly the same at described here: https://www.alfredforum.com/topic/11957-alfred-pastes-the-same-snippet-multiple-times/

Hi @patrickloonstra 👋,
I'm super happy that someone (except me) is using this, however, I'm sorry there might be such a bug. Although I'm not using anything special.
Could you please provide steps how to reproduce it? 🙏

I'm getting it just by running Alfred 3 and using emoj repeatedly.

@garthk Emoj? This extension is using e as a keyword.

Interesting - heads up that I had the same symptoms and came here from the same alfredforum post (https://www.alfredforum.com/topic/11957-alfred-pastes-the-same-snippet-multiple-times/?tab=comments#comment-63483) but am actually seeing this issue triggered in another alfred-emoji workflow (this one: https://github.com/janclarin/emoji-alfred-workflow), not this one.
(So consider this a recommendation that folks double check that this is the right repo for the workflow you're using!)

Came here to say, I'm using another workflow that was built off of Alfy, and the same thing is happening to me. I can even get it up to 4 instances sometimes before I notice.
This has to be something in the Alfy integration that needs to be fixed.

Here's the cause of the issue: https://github.com/SamVerschueren/alfred-notifier/issues/8

Thanks so much, @deanishe!


Bump redox_users; Disable redox_users default features

I just released a new version of redox_users with argon behind a feature flag (enabled by default). There had been some buzz a while back about the size of the dependency tree for this crate and dirs earlier, so I've finally gotten around to doing my bit.

Thanks! Here's the tracking issue for this: https://gitlab.redox-os.org/redox-os/users/-/issues/26

Ah thanks!
Kinda weird that this crypto-dude-wearing-sunglasses-on-profile-pic linked in the Redox thread behaves exactly like one would expect from a crypto-dude-wearing-sunglasses-on-profile-pic. I think his behavior caught my attention multiple times already.


Spikes

A lot of messages arrive at the same time despite being send in constant interval (20 per second).
Number to the left is a tick number - number to the right is the count of the messages received in this very animation frame.
This is not the case when switching back to websocket/ws - the result is that despite huge CPU gain (idle for 70% of time) the game behaves like if it was updated by the number of accumulated frames times slower.

Below - same screenshot from websocket/ws

This is according to design. If you receive 10 frames, you will get 10 messages emitted. If you have problems with this you need to solve it somehow, this shouldn't affect your networking code.

If you expect to get one message per "tick" then you need to call process.nextTick in your callback and queue up a closure with the copy of the message for later traversal. This is ofc massively coslty in performance but it should even out these "spikes".

@alexhultman I think you may have misunderstood. What @rezoner means by tick is the rendering frame number.
Using websocket/ws messages are being received usually 3 rendering frames apart (this is ~20 times a second) which is the rate they are being sent at.
When using uWebSockets, even though the messages are being sent at a 1/20th of a second rate they are getting grouped up so multiple packets get received by the client at the same time with a larger gap between them (like maybe uWebSocket is grouping them for performance instead of sending them immediately?)
I've only had a brief look through, but I don't see anywhere in the code enabling TCP_NODELAY, which may cause this?
ref https://en.wikipedia.org/wiki/Nagle's_algorithm

Ah.
You have any code to reproduce it separate from the game? You could probably display real time at every message event and have a simple timer send at equal periods.
Tell me more about what side is sending etc, or just provide some code

@alexhultman
Live (open console):
WS - http://rezoner.net/temp/ws/
UWS - http://rezoner.net/temp/uws/
Source:
https://gist.github.com/rezoner/c28239f4932fe6ab46b666b5a4a229d9
Note: You will not have the spikes when running both server and client on localhost.

Nice code. So if I set this up in a local network it should trigger?

I guess you need at least some distance between client and server so the latency is greater than 0.
Running that on localhost will not give you spikes.

I don't know what ws does and I haven't run the code but my best bet would be for you to turn on TCP_NODELAY on this socket if you need one send to be received as one package. When you call socket.send this is pretty much a wrapper of Unix send straight up and I don't control the behavior of the TCP stack.

There is really no guarantee of how and when packages will be delivered, the TCP protocol is just a stream in which you put stuff and it comes out the other end in any possible segmentation / delay. So yeah, TCP_NODELAY would probably solve your problem. I could add the ability to set this on and off.

https://nodejs.org/api/net.html#net_socket_setnodelay_nodelay
Does ws use this function?

https://github.com/websockets/ws/search?utf8=✓&q=setnodelay
They seem to set noDelay yes, so this is most probably the problem.

@rezoner could you try and modify uws.js to setNoDelay(true) on the socket in handleUpgrade? That should translate all the way into the lib.

@alexhultman
Jackpot

And with that I am able to at least double the number of players per server.

So maybe we could make this optional with a server setting noDelay: boolean? Do you want to PR this?

@alexhultman I am letting you to decide whether it should default to true or be configurable - I am happy either way. It's enabled by default in WS - but you have to think of CPU consequences. Imho it should be true by default as it won't be a matter for 90% of projects - and if someone is in the latter 10% he will surely be in need for a very specific performance tricks and gonna dig that up in docs.

Do you want to PR this?

Nah, I've done nothing just copy-pasted your own findings.

I added an option noDelay that is false by default (TCP default, I don't think one should mess with TCP defaults by default). Set this to true in cases such as games, but not in cases such as file transfer, chat, etc.

I am curious... how exactly does uWebSockets push out more than double the performance compared to WS? What is it exactly doing behind the scenes that makes it so much faster? I want to switch too... lol

Ws is one pile of garbage, there is no competition with garbage.

@Dillybob92 Okay sorry for me previous post. But really, in all honestly, ws is truly terrible. If you compare ws against WebSocket++ or libwebsockets you will see that ws is in fact really, really, really slow. Most users report about 2x performance increase when switching from ws, but you should theoretically be able to achieve a larger difference if WebSockets is your bottleneck. Most projects have other bottlenecks kicking in though.

@alexhultman That's fine! I am glad I found this thread though, because I am using ws and spent a lot of time within it, but if the performance here is drastically better.. I am switching too. Is it most likely because this is a native binding solution? Or is it just designed differently?

@Dillybob92 Make sure to report any issues you find if you swap, there are quite a few functions missing right now.


Option "mail to:" missing.

Hi,
Is it possible to extend the capability of the APK with the option in the mail showing the name of the person the mail is directed to.
So besides the "mail from:" to have possibility to display "mail to:".  On PC with Thunderbird for example there is such option but for Android I did not find any mail APK having such option. I have one account with copies of the mail where I would like to see to who the mail is sent rather then from who.

K-9 Mail already displays the To address when the message was sent from one of your configured identities.


Improve commit messages to include change detail.

When there is an issue creating the git commit message, it is using an ambiguous commit message and discarding the error.

This PR updates it to better handle (i.e. report) the error so any potential causes can be addressed.

The existing git commit message was ridiculously ambiguous, lacking any useful detail. Multiple commits to the same repo would simply yield the same commit message each time.

This PR updates the git commit message to include more detail including the 'from' (curr) and 'to' (next) tags, so we can see from the commit history what changed, without having to drill into the content of each patch.

#68


[Q] cc errors after tsp process

Hello,
I change a mpeg ts with tsp (injecting/replacing 2 tables, merging 1 service). Before the tsp process the ts has 0 cc errors, after the tsp process about 1000 per day. The ts stream has enough null bits (>500 kbit/s). The load of the server is about 9.
Even if I just change name als provider in the stream, cc errors occur.
How can I improve the output signal?

From that description, we haven't the slightest idea of what you are doing. Please provide the exact commands you use and the analysis of the symptoms.
If you use UDP output, the common cause is the burst effect which has been described many times here, with possible solutions.

It may be the internal buffering which creates latencies and bursts. Try tsp --max-flushed-packets 7 --max-input-packets 7 --max-output-packets 7 in the two commands, the main one and the one in the merge plugin. This forces the transmission from one plugin to the next one every 7 packets (the output granularity). By default, it waits for more packets. Transmitting packets more often eats more CPU but it reduces the latency.


Release Notes or Changelog Addition

Hi, concerning possible improvements, I saw a new version 0.2.0 was released and I wanted to quickly check the changes. It would be great to have something like release notes.

I did include a CHANGELOG, however I'll now also tag them as releases and include the changes as notes there https://github.com/cashubtc/cdk/releases/tag/v0.2.0. To make it clear.  Ill do my best to keep these as complete and accurate as possible.

Thanks for the prompt fix! The release description is very helpful.


Support for TCP

I am unsure of the level of difficulty or even the possibility of allowing transport over TCP, but our type of infrastructure does not allow for any open UDP ports for security reasons. Additionally, our stats agent lives in a separate host container from our applications so we're unable to directly leverage UDS.
We may be an edge-case, but I'm happy to look into this / contribute if this functionality being available is reasonable.

Hi, thanks!  The DogStatsD server only supports UDP and UDS (or named pipes on Windows), not TCP -- so that would be a server-side change.
I'm not sure what you mean by "host container", but if the agent and applications are in different containers on the same host, then you could potentially mount the UDS socket in both containers.  We don't recommend running the agent and app on different hosts, as the metrics are tagged with the host running the agent.

Yup - realized after I posted here would need to implement a TCP listener in the host agent. Yeah our infrastructure is a bit different - we don't directly control our deploys of our images so we can't guarantee the host agent is on the same host. I know it sounds weird 😁
Sorry "host containers" was just poor articulation. The IaaS we use deploys our apps and services into multiple containers on AWS enclave.

Hm, that's a tricky spot, then.  I can imagine some workarounds, such as forwarding the UDS socket with something like socat, but those might not be great choices, either.  Please contact our support who can dig into the particulars of the situation and determine the best solution.


Add deprecation notice to readme and archive the repository

We don't use this internally anymore so we should make it clear that we don't intend on maintaining or supporting it.

Done in https://github.com/kensho-technologies/orama/pull/165.


add xpu.edu.cn

Xi’an Polytechnic University 西安工程大学
School official website: https://en.xpu.edu.cn/
The website of school of computer Science: https://jsjxy.xpu.edu.cn/English.htm
The website of the mail system: https://mail.xpu.edu.cn/

@buxudor lib/domains/stoplist.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.


Fix: center on node feature works with collapsible disabled

Fixes https://github.com/bkrem/react-d3-tree/issues/386.
If you disable collapsible, the "center on node" feature (via dimensions prop) ceases to work. My understanding is this due to the fact that a click on the node wouldn't trigger an update of the component. This PR adds the wasClicked state to the shouldNotTransform criteria. Tested this locally and it seems to work, as shown in attached clip.
https://user-images.githubusercontent.com/128327117/231506155-501ec223-9bcf-4d9f-97ff-025faca10c7c.mov

Released as https://github.com/bkrem/react-d3-tree/releases/tag/v3.5.2


Publish image to Dockerhub

This is a request to publish the image to dockerhub so that I don't need to build it locally.

Hi @ghostsquad, you don't need to build it locally, you can use the Github package:
https://github.com/grafana/k6-operator/pkgs/container/operator
The latest there usually contains the latest commit on main branch.

Oh thank you, I didn't realize that this was using using github packages to publish the images.


Support for emitting modules into a subdirectory

Was: Linker crashes when emitting modules into a subdirectory
If exporting a method/value under a module name that contains a /, the linker crashes because it attempts to pass the entire name directly to createTempFile. Having a slash in the module name is useful when integrating with frameworks such as Next.js, which use directories to map components to application routes.
For example, running fastLinkJS with a project containing the following export:
object Next {
  @JSExportTopLevel(name = "component", moduleID = "foo/bar")
  def component(): js.Any = ...
}

results in the following crash
[error] java.lang.IllegalArgumentException: Invalid prefix or suffix
[error]         at java.base/java.nio.file.TempFileHelper.generatePath(TempFileHelper.java:60)
[error]         at java.base/java.nio.file.TempFileHelper.create(TempFileHelper.java:125)
[error]         at java.base/java.nio.file.TempFileHelper.createTempFile(TempFileHelper.java:159)
[error]         at java.base/java.nio.file.Files.createTempFile(Files.java:878)
[error]         at org.scalajs.linker.PathOutputDirectory$Impl.$anonfun$writeAtomic$2(PathOutputDirectory.scala:71)
[error]         ...

Uh oh, that was never meant to be allowed. The infrastructure of OutputDirectory is only meant to represent a single, flat directory. We should disallow this at compile time instead of crashing at link time.

Having a slash in the module name is useful when integrating with frameworks such as Next.js, which use directories to map components to application routes.

I sympathize, but IMO this falls under the category of "frameworks that want a specific shape for the output code rather than semantics". We don't guarantee that we will support every possible framework's idea of how the target .js files should "look like".
Have you considered defining the structure separately, as a hierarchy of .js files that import the relevant files of the Scala.js output?

My 2cts:

My original idea was to not restrict module IDs in the IR specification, very similar to how ESModules do not define what the module string means.
Per consequence, how modules are mapped to files is specific to the linker implementation.
In the current linker implementation the mapping from module ID to filename is defined via OutputPatterns (and arguably very crude).

I could see the value of extending OutputPatterns to something more general. Challenges:

OutputPatterns needs to be serializable, so it's the usual flexibility / complexity tradeoff (we can't just make it a function).
The full output universe needs to be understood, otherwise the incremental linker cannot remove unnecessary files anymore.

@armanbilge asked me to reference my own usecase for this - a while back I attempted to do the same for Cloudflare Pages. Quoting my Discord message:

Cloudflare pages can work it's magic on files like /admin/api/get_[[[id]].js but iirc I get errors trying to use / in the name of the module when using ES modules

Current thinking is to output filenames in admin__api__get_[[id]]  format and externally rewrite to paths - perhaps there's a better way?

There's one more challenge I forgot: How to synthesize internal module names. If they need to go into special directories, things might get more complicated (OTOH, a simple prefix config might be enough).

@shadaj @keynmol in your use cases, how would you expect internal modules (modules with shared code between the modules you declare) to be named / structured?

For Next.js, we would probably want to dump all internal modules into a components folder, but don't have any naming restrictions on the actual modules.

For cloudflare, I believe only modules that explicitly export a function with a certain name (e.g. onRequest or onRequestGet) are considered by CF for routing.
Therefore other files and their names (intermediate modules) are likely to be ignored (*), so I would personally not expect any structure from them - they can all be dumped top level for all I care.
If I have time, I will test the (*) assumption to make sure it works as I described.

For cloudflare, I believe only modules that explicitly export a function with a certain name (e.g. onRequest or onRequestGet) are considered by CF for routing.
Therefore other files and their names (intermediate modules) are likely to be ignored (*), so I would personally not expect any structure from them - they can all be dumped top level for all I care.

FWIW, this is a problem: We cannot guarantee that an internal module does not export a function with a name that is special to cloudflare.

Just stumbled into this again. I understand that ESM-style npm packages are typically organized into subdirectories e.g.
import<EMAIL_ADDRESS>
So it would be nice to be able to make such things with Scala.js as well.

Maybe, but do we have examples of where this is actually useful? For shoelace:
import<EMAIL_ADDRESS>        --------------- -------- --------------- ------ ------
        scope           package  build artifacts module redundant

Even what the doc lists as utilities (e.g. include) is under the components "namespace".

I'd like to add another voice for supporting structured directories. It's pretty tedious to manually setup the file structure for Next.js or Remix by hand.
Is this something where a PR would be considered? Alternatively, is there a mechanism that would allow extending the plugin to have this functionality within build.sbt?

For cloudflare, I believe only modules that explicitly export a function with a certain name (e.g. onRequest or onRequestGet) are considered by CF for routing.
Therefore other files and their names (intermediate modules) are likely to be ignored (*), so I would personally not expect any structure from them - they can all be dumped top level for all I care.

FWIW, this is a problem: We cannot guarantee that an internal module does not export a function with a name that is special to cloudflare.

I don't think it's important to make this guarantee for CloudFlare or any other tooling. Assuming that generated modules can have some user-defined directory structure + internal modules live somewhere else, I would expect it to be easy to post-process this into whatever structure the user needs/wants.

Is this something where a PR would be considered?

IMO, yes, for sure. However, I'd urge you to first propose and discuss the exact changes you'd like to do (in this issue). Especially for internal modules, it is not clear to me how the interface could look like.

Alternatively, is there a mechanism that would allow extending the plugin to have this functionality within build.sbt?

You could move the modules around after they are built by Scala.js. But that would also involve adjusting the import paths (since they are relative). So, yes, a mechanism could be built, but to the best of my knowledge, there isn't any at the moment.

A couple of thoughts I had:
The following might get us quite far:

Interpret / in module IDs as paths
Add a setting internalModuleIDPrefix
(optional) specify that generated internal module IDs never contain a slash (IIUC already the case)

Unclear with this approach is how to deal with module ID equality: Right now, the following are considered different module IDs: a/b, a//b, a/./b, a/c/../b`.

Unclear with this approach is how to deal with module ID equality: Right now, the following are considered different module IDs: a/b, a//b, a/./b, a/c/../b`.

Given that any slash today results in a crash, and that a single . or .. likely does as well, I think we can forbid leading, trailing and double slashes, as well as path elements that are . or ...

Unclear with this approach is how to deal with module ID equality: Right now, the following are considered different module IDs: a/b, a//b, a/./b, a/c/../b`.

Given that any slash today results in a crash, and that a single . or .. likely does as well, I think we can forbid leading, trailing and double slashes, as well as path elements that are . or ...

Wouldn't it be simpler to just use something like java.nio.Path.normalize instead of trying to be clever about it?

Our linker cross-compiles to JS. In Scala.js we don't have access to Path, so we cannot do that.

Also FWIW: Paths are platform specific (at least if we're using the default FileSystem). So we need to be careful: We do not want the linker's output to depend on the platform it is running on.

Apologies that I don't have a better understanding of how the plugin works internally - I've not had a lot of time to do some digging just yet. I've also never done anything with cross compilation, so maybe my understanding is all garbage!

Our linker cross-compiles to JS. In Scala.js we don't have access to Path, so we cannot do that.
At some point it needs to execute platform-specific code though, no? ie. there is some write(path: String, contents: ???) function somewhere along the way, which has a different implementation in JS to JVM? Given this assumption:

Could modify this function to create intermediate directories if required
Create a new normalize function using the same convention; ie. dispatching to separate implementations based on the platform being executed.

There are two notions of "platform" at play here. Usually we mean JVM v JS, indeed. But here we're talking about the OS and its filesystem. We want the linker to produce identical outputs regardless of the OS and of the filesystem(s) being used. Therefore, we cannot rely on normalize (whether on the JVM or JS).

I'm not sure that there's anything that we need to do to support various OS here. The underlying runtime can deal with this for us.

As far as actually writing the modules to the filesystem is concerned, this is correct. However, we also need to consider the usage sites of the modules (i.e. the imports):
When the linker splits off some code into an internal module, it will need to import that code. Currently, it will emit:
import * as <internal name> from "./<moduleID>.<ext>"

Where the specifics are determined by the OutputPatterns.
So far, the assumption was that moduleID does not contain any special characters for the underlying filesystem (i.e. no / or \). If we start supporting subdirectories, we must change/break this assumption.
At this point, we need to take care, that:

The code we generate is the same on all platforms/operating systems (e.g. it is not OK if you get back slashes in imports when you build on windows but you get forward slashes when you build on *nix).
The code we generate works on all JS platforms. Since (at least last time I checked), what a module identifier means is not specified in the ESM spec, this is necessarily trade-offy. Given that we already assume that ./ denotes a relative import, it is IMO OK to assume further slashes mean subdirectories.

You might correctly observe at this point, that (apart from the changes you're already working on in your branch), there is nothing to do for the success path. However, we need to make sure that we are not generating broken code (e.g. `import ... from './a/../b\foo.js'). This is what @sjrd and I have been discussing about.
HTH

When the linker splits off some code into an internal module, it will need to import that code. Currently, it will emit:

This makes sense. It seems that I didn't do enough testing; I can replicate this once I start writing non-trivial code. Are you able to point to where this code is emitted from the linker?

it is not OK if you get back slashes in imports when you build on windows but you get forward slashes when you build on *nix

I don't understand why this is important, but a simple solution here is to use only / as the path separator. If a path contains only /, Windows will understand it correctly (I did an experiment to test this, some sample code here).
But more importantly I think upholding this assumption will end up with broken code if any form of directory nesting is allowed, regardless of the separator used. For example:

Assuming that / is used as the path separator character and have some modules emitted to <build root>/util/foo.

// If I'm working on a *nix system, I'm likely to have an export that looks like
@JSExportTopLevel("baz", moduleID = "app/bar")
// This would import modules from `foo` using `import <names> from "../util/foo"`; which would work correctly

// If I'm working on a Windows system, I'm likely to have an export that looks like
@JSExportTopLevel("baz", moduleID = "app\\bar")
// This would import modules from `foo` using `import <names> from "../util\\foo"`; which I believe _would break_ because of the mixed characters - Though I have not tested this assumption.

A mirror of this situation occurs if \ is used as the path separator: code generated on Windows will be fine, but anything generated on *nix will die.

Note that in both of these cases, the break occurs because the user followed the conventions of their OS but scala-js uses only a single character for the separator. One might argue that the user could be educated to follow an alternative pattern (eg. always use *nix paths, even on Windows), but that is a situation that I expect would be ripe for confusion.
So if this is a hard requirement; I don't think that there's any simple way to both support any OS and support emitting into subdirectories.

We definitely have to standardize on / on all systems, even Windows. We will need to forbid any other special characters of file systems, notably \  and :. The rationale for Windows users is that those are URI relative paths. Not file system ones.

We definitely have to standardize on / on all systems, even Windows. We will need to forbid any other special characters of file systems, notably \  and :. The rationale for Windows users is that those are URI relative paths. Not file system ones.

Just to be clear; you're saying that the user must always use something that looks like @JSExportTopLevel("baz", moduleID = "app/bar"), right?
This seems reasonable to me. I'm worried that it's a little bit awkward for Windows users, but it looks like it's the easiest to implement. Perhaps also improves consistency for anyone collaborating between Windows/*nix :shrug:
Assuming you're happy with the direction I've started looking at earlier (the link here), I can start figuring out how to get the linker to emit import statements with the correct relative paths.
I believe that org.scalajs.linker.standard.ModuleSet.Module is at the center of this, but I haven't figured out how the paths are being populated yet - it would be much appreciated if anyone has any pointers

Just to be clear; you're saying that the user must always use something that looks like @JSExportTopLevel("baz", moduleID = "app/bar"), right?

Yes.

Perhaps also improves consistency for anyone collaborating between Windows/*nix

Yes. In fact, this is a requirement: The same input with the same options must produce the same output on all systems/OSs.

Assuming you're happy with the direction I've started looking at earlier (the link here), I can start figuring out how to get the linker to emit import statements with the correct relative paths.

That looks like a good start to me.

I believe that org.scalajs.linker.standard.ModuleSet.Module is at the center of this, but I haven't figured out how the paths are being populated yet - it would be much appreciated if anyone has any pointers

The linker frontend does not deal with paths, but with ModuleIDs.: an abstract representation of the identity of the module

For external modules, this is what the user provides in JSImport.
For public modules, this is what the user provides in JSExportTopLevel
For internal modules, this is generated automatically
Definitions of types of modules

The linker backend (aka Emitter) then translates these into filenames to put them after from.

Code that generates imports for internal modules
internalModulePattern config
internalModulePattern is set via OutputPatterns

Default patterns

HTH

Nikita's comment on Discord just made me realize that when building this, we need to be extremely careful to keep relative path imports meaningful.

When I have some time I'll aim to come back to this; however for now I've had some life/work stuff get in the way


runtime error in VdCnn.cs

Just playing with your VdCnn sample, which throws excpetion 'Unable to find an entry point namend 'AddControlInput' in DLL tensorflow. Happens on that line:
                   // batch normalization
                    conv = tf.layers.batch_normalization(conv, training: is_training);

The references are SciSharp.TensorFlow.Redist v1.14.0 and TensorFlow.NET v0.10.3
Thanks for your help!

Have to replace tensorflow with this one: https://github.com/SciSharp/TensorFlow.NET/tree/master/tensorflowlib/runtimes/win-x64/native

Hi! I just upgraded to your latest TensorFlow.NET 0.10.10 and used the dll you provided.
Now I get 'can't get graident function through get_gradient_function TopKV2' on this line:
           tf_with(tf.control_dependencies(update_ops.Select(x => (Operation)x).ToArray()), delegate
            {
                var adam = tf.train.AdamOptimizer(learning_rate);
                adam.minimize(loss, global_step: global_step);
            });

Stack trace:
System.NotImplementedException: can't get graident function through get_gradient_function TopKV2
at Tensorflow.ops.get_gradient_function(Operation op)
at Tensorflow.gradients_util.<>c__DisplayClass0_0.<_GradientsHelper>b__0(NameScope scope)
at Tensorflow.Python.tf_with[T](T py, Action`1 action)


Refresh kube-component reference docs

This PR updates the generated reference docs for kube-components.
The source used if from the 'release-1.10' branch of
kubernetes/kubernetes.
Close: #8280

/assign
cc @steveperry-53
@tengqm This PR makes sense, except ... it removes the doc for a deprecated flag that isn't yet fully removed from the code/apiserver (--admission-control). We've now got --enable-admission-plugins properly documented (per issue #8280), but per discussion in PR #7887 we've lost the deprecated flag. And I don't understand the code base well enough to be able to figure out what's going on there. It looks to me as though on master, at https://github.com/kubernetes/kubernetes/blob/master/pkg/kubeapiserver/options/admission.go#L67ff, we should still get doc for --admission-control -- but that's not what's in the release-1.10 branch (code).
I won't hold up this PR for very long, but I'd like to find someone stateside who can explain the code to me, and maybe we can find some not terrible time that works for both of us to chat, too. I'm guessing these aren't the only flags that aren't showing up properly in the doc, but I'm using them as an indicator of the larger problem. Thanks for your patience!

@Bradamant3 I understand the concern for not showing the "hidden" flags. Those flags are not shown on the command line when you do, for example, kube-apiserver --help. Do you really think it makes senses to list an option in the reference documentation when that option disappeared from the command line? The whole idea of making flags "hidden" is stop advertising them to users. They are not completely removed simply because there are existing users who are still using them. This is my understanding why kubelet has hidden so many flags and kube-apiserver has also hidden a few of them. In my opinion, the "larger" problem you want to solve is at the upstream kubernetes/kubernetes project.

@tengqm absolutely agreed that the larger problem needs to be solved upstream. But if you look at the master branch upstream, these lines are added (the second one NOT in release-1.10 branch), to show the doc for (at this this one) deprecated flag (https://github.com/kubernetes/kubernetes/blob/master/pkg/kubeapiserver/options/admission.go#L76 but also pasted here):
    fs.MarkDeprecated("admission-control", "Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version.")
    fs.Lookup("admission-control").Hidden = false

/assign @heckj
/assign @chenopis

@Bradamant3 I don't buy in the idea of "hiding" flags at all.  The components is doing things *undocumented". By "undocumented", I mean there are flags not shown on man pages, not meant to show as help messages, not meant to show as markdown docs, but these flags do exist secretly and they work! This design is somehow breaking the convention how things are deprecated (aka. deprecation policy) across the community.

I think some form of these should be picked to 1.10.x to ensure we don't hide the flags that are currently usable:

https://github.com/kubernetes/kubernetes/pull/62505
https://github.com/kubernetes/kubernetes/pull/62621

/approve


Indent guides do not appear after restart until window loses focus or file is reopened

Issue Report Checklist

[x] Searched the issues page for similar reports
[x] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
[x] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
[ ] Could not reproduce inside jupyter qtconsole (if console-related)
[x] Tried basic troubleshooting (if a bug/error)

[x] Restarted Spyder
[x] Reset preferences with spyder --reset
[ ] Reinstalled the latest version of Anaconda
[x] Tried the other applicable steps from the Troubleshooting Guide

[x] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description
Spyder stops showing indent guides after restart. This was reported by a Debian user, reported at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1028637 and I have successfully reproduced the behaviour.
What steps reproduce the problem?
Our user wrote:

Start with a clean Spyder profile, that is, remove the /home/<user>/.config/spyder-py3 dir, if  it exists
Start Spyder
Open the following file /usr/lib/python3.10/subprocess.py
Goto Menu > Source and enable the following:
[x] Show indent guides
[x] Show selector for classes and functions  (this is actually irrelevant)
[x] Underline errors and warnings  (this is actually irrelevant)
Indent guides are properly shown
Shut down Spyder
Restart Spyder, go to tab with /usr/lib/python3.10/subprocess.py: the indent guides are not shown.

The Debian user added: "If I close the tab with "/usr/lib/python3.10/subprocess.py" file an then reopen the file again in a new tab then the indent guides will be shown again properly (until Spyder is restarted again, then they will disappear again)."
I further discovered that if the Spyder window loses focus and then regains the focus, the indent guides are then shown.

Hey @juliangilbey, thanks for reporting. Indent guides have not been very well tested (that's why they are not enabled by default), so I'm not surprised by this behavior.
We'll try to take a look at this in a future release (can't say when exactly, sorry).

Any update? While not essential, it would make it very convenient.

I did some improvements to folding and indent guides that will land in Spyder 6. But more work still remains to be done, so I'm marking this for 6.1.


Neo4j AuraDS on GCP: Cannot run long queries

We use Neo4j AuraDS on GCP: https://pantheon.corp.google.com/marketplace/product/endpoints/prod.n4gcp.neo4j.io
We have a script to regularly update data in Neo4j DB.
Previously this script worked (on Neo4j AuraSB), but does not seem to work anymore (our data has grown slightly).
The script now throws Failed to read from defunct connection error on queries that take more time (it still works on smaller queries).
Can you please help us run queries on managed Neo4j again?
Neo4j Version: 4.4.0 Enterprise
Neo4j Mode: Neo4j AuraDS on GCP
Driver version: Python driver 4.4.5
Steps to reproduce
Code:
import neo4j
neo4j_driver = neo4j.GraphDatabase.driver(
    _NEO4J_URI, auth=(_NEO4J_USER, _NEO4J_PASSWORD))
with neo4j_driver.session() as session:
    session.run(query='''
//:auto
MATCH (r:Request)
WHERE r.obscured_consumer_project_number IS NOT NULL
// Use hash to stabilize and randomize the sequence ordering for same request_time and user_id
// It's also desirable to make the ordering the same for different relationships (e.g. NEXT_PROJECT_REQUEST and NEXT_USER_REQUEST)
// Use the row id for hashing or an explicit object like `apoc.hashing.fingerprint({method: r.method, request_time: r.request_time, ...})`
WITH *, apoc.hashing.fingerprint(r.id) AS hash
WITH * ORDER BY r.request_time, hash
WITH r.obscured_consumer_project_number AS obscured_consumer_project_number, collect(r) AS requests
UNWIND apoc.coll.pairs(requests) AS request_pair
WITH request_pair[0] AS r1, request_pair[1] AS r2
WHERE r2 IS NOT NULL
CALL {
  WITH r1, r2
  MERGE (r1)-[:NEXT_PROJECT_REQUEST]->(r2)
} IN TRANSACTIONS OF 1000 ROWS
;
'''
)

Actual behavior
Applying query wipeout.cypher
Applying query NEXT_PROJECT_REQUEST_relationships.cypher
Failed to read from defunct connection IPv4Address(('7XXX.databases.neo4j.io', 7687)) (IPv4Address(('YYY', 7687)))
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/neo4j/io/_common.py in _yield_messages(self, sock)
     56                     # Determine the chunk size and skip noop
---> 57                     buffer.receive(sock, 2)
     58                     chunk_size = buffer.pop_u16()

/opt/conda/lib/python3.7/site-packages/neo4j/packstream.py in receive(self, sock, n_bytes)
    483             if n == 0:
--> 484                 raise OSError("No data")
    485             self.used += n

OSError: No data

The above exception was the direct cause of the following exception:

SessionExpired                            Traceback (most recent call last)
/tmp/ipykernel_2514/2341552839.py in <module>
----> 1 main([])

/tmp/ipykernel_2514/2512792290.py in main(***failed resolving arguments***)
     17       bq_client=bq_client,
     18       neo4j_driver=neo4j_driver,
---> 19       batch_size=1000,
     20   )

/tmp/ipykernel_2514/2580852736.py in ingest_all_data(bq_client, neo4j_driver, batch_size)
    113     print(f"Applying query {query_file}")
    114     with neo4j_driver.session() as session:
--> 115       session.run(query=pathlib.Path(query_file).read_text())

/opt/conda/lib/python3.7/site-packages/neo4j/work/simple.py in run(self, query, parameters, **kwparameters)
    216             query, parameters, self._config.database,
    217             self._config.impersonated_user, self._config.default_access_mode,
--> 218             self._bookmarks, **kwparameters
    219         )
    220

/opt/conda/lib/python3.7/site-packages/neo4j/work/result.py in _run(self, query, parameters, db, imp_user, access_mode, bookmarks, **kwparameters)
    115         self._pull()
    116         self._connection.send_all()
--> 117         self._attach()
    118
    119     def _pull(self):

/opt/conda/lib/python3.7/site-packages/neo4j/work/result.py in _attach(self)
    204         if self._closed is False:
    205             while self._attached is False:
--> 206                 self._connection.fetch_message()
    207
    208     def _buffer(self, n=None):

/opt/conda/lib/python3.7/site-packages/neo4j/io/_common.py in inner(*args, **kwargs)
    186             def inner(*args, **kwargs):
    187                 try:
--> 188                     func(*args, **kwargs)
    189                 except (Neo4jError, ServiceUnavailable, SessionExpired) as exc:
    190                     self.__on_error(exc)

/opt/conda/lib/python3.7/site-packages/neo4j/io/_bolt4.py in fetch_message(self)
    281
    282         # Receive exactly one message
--> 283         details, summary_signature, summary_metadata = next(self.inbox)
    284
    285         if details:

/opt/conda/lib/python3.7/site-packages/neo4j/io/_common.py in __next__(self)
     81
     82     def __next__(self):
---> 83         tag, fields = self.pop()
     84         if tag == b"\x71":
     85             return fields, None, None

/opt/conda/lib/python3.7/site-packages/neo4j/io/_common.py in pop(self)
     75
     76     def pop(self):
---> 77         return next(self._messages)
     78
     79

/opt/conda/lib/python3.7/site-packages/neo4j/io/_common.py in _yield_messages(self, sock)
     72
     73         except (OSError, socket.timeout, SocketDeadlineExceeded) as error:
---> 74             self.on_error(error)
     75
     76     def pop(self):

/opt/conda/lib/python3.7/site-packages/neo4j/io/__init__.py in _set_defunct_read(self, error, silent)
    556             self.unresolved_address, self.server_info.address
    557         )
--> 558         self._set_defunct(message, error=error, silent=silent)
    559
    560     def _set_defunct_write(self, error=None, silent=False):

/opt/conda/lib/python3.7/site-packages/neo4j/io/__init__.py in _set_defunct(self, message, error, silent)
    601         else:
    602             if error:
--> 603                 raise SessionExpired(message) from error
    604             else:
    605                 raise SessionExpired(message)

SessionExpired: Failed to read from defunct connection IPv4Address(('XXX.databases.neo4j.io', 7687)) (IPv4Address(('YYY', 7687)))

P.S. The DB was taking less than 30% of instance space and there were no OOM errors.
Using Ne04j browser I got "SessionExpired" errors when trying to run the query.
After resizing the DB from {8, 16, 2} to {24, 48, 5}, the query finishes in the browser:
Created 8655786 relationships, completed after 430174 ms
The DB now takes <14% of storage space.

Hi and thanks for reaching out.
First, it'd be helpful to get some driver debug logs. Please run see the API docs on how to enable it. Feel free to replace the IP addresses, but I'd like to see the full log of that one query including the driver start-up.

@Ark-kun cheers for reaching out and sorry to hear you're having trouble.
Just because I'm going to need to ask for some personal details, would you mind filing a support ticket so we can have a private conversation about the instance in question?
The easiest way is in the console navbar -> get help -> support or directly here: https://aura.support.neo4j.com/hc/en-us
Once that's done, we can start exploring :) At this point, it's really hard to know if it's the instance, our infra, the network or the driver - but we'll figure it out!
Thank you!
LG

I'll close this issue here then. Should it boil down to a driver issue, please feel free to re-open this a open a new issue.


How to make it work

Hi,
Any hint on how to glue all the stuff would be greatly appreciated :)
I tried to call AdminServiceFactory.Create(settings) in the OWIN Startup class, but I got an exception (Method not found: 'MongoDB.Driver.MongoServer MongoDB.Driver.MongoClient.GetServer()).

please?

Did anyone find out how this works? Whats the setup procedure and how do you call the methods?


Images Postgres Alpine, optimization

I just test it on my test server and my production server, it works perfect :)
But I'm not using Wal-e and I think it is important. If you can test it, that's great. If not we'll try to find out someone for this ^^
I also add @xcompass as reviewer. I like to have his wise feedback on change like this.

@pichouk I also do not have the opportunity to check how Wal-e works (

TODO/reminder before validating this PR:

Remove lines mentioned in this discussion : https://github.com/mattermost/mattermost-docker/pull/207#discussion_r152430875
Review from @xcompass
(Optionnal) Test Wal-e feature. Should work but if someone can do this extra test, could be interesting.

test WALE_S3, not work
25.11.2017 19:59:10LOG:  archive command failed with exit code 127
25.11.2017 19:59:10DETAIL:  The failed archive command was: envdir /etc/wal-e.d/env /usr/local/bin/wal-e wal-push pg_xlog/00000001000000000000000C
25.11.2017 19:59:11sh: envdir: not found
25.11.2017 19:59:11LOG:  archive command failed with exit code 127
25.11.2017 19:59:11DETAIL:  The failed archive command was: envdir /etc/wal-e.d/env /usr/local/bin/wal-e wal-push pg_xlog/00000001000000000000000C
25.11.2017 19:59:12sh: envdir: not found
25.11.2017 19:59:12LOG:  archive command failed with exit code 127
25.11.2017 19:59:12DETAIL:  The failed archive command was: envdir /etc/wal-e.d/env /usr/local/bin/wal-e wal-push pg_xlog/00000001000000000000000C
25.11.2017 19:59:12WARNING:  archiving transaction log file "00000001000000000000000C" failed too many times, will try again later```

Hmm seems that it is an issue with envdir. Maybe it is just a missing command in Alpine.

WAL-E not support Alpine :(

Arf... What made you say that ?

WAL-E, not work to alpine(
error:
03.12.2017 2:10:50LOG:  archive command failed with exit code 127
03.12.2017 2:10:50DETAIL:  The failed archive command was: envdir /etc/wal-e.d/env /usr/local/bin/wal-e wal-push pg_xlog/000000010000000000000002
03.12.2017 2:10:50WARNING:  archiving transaction log file "000000010000000000000002" failed too many times, will try again later

@pichouk @xcompass  can make a backup a separate container?

About the WAL-E support, I really don't know what to do. I don't use it and I don't really know why it didn't work on Alpine.
I would like to investigate on this, but I have no time for now. I'll see in few days/weeks if I can achieve to something with WAL-E and Alpine.

I don't think we can use a separate container. I think postgres is actually calling WAL-E script directly. Re: WAL-E error, could you enable WAL-E logging: https://github.com/wal-e/wal-e#logging to see where it fails? Might be a configuration thing.

I squashed all commits and tested the image. Seems to works with existing and new databases :)
Looks good to me ! I'll like an approval from @xcompass too but apart from that I'm ok to merge.

Are you including Bash in the alpine package? Or just Sh? It seems Mattermost needs/prefers bash?
Maybe worth seeing if it fixes this issue: https://github.com/mattermost/mattermost-docker/issues/234
Just add the following to the dockerfile for the app:
RUN apk add --no-cache bash and change the entrypoint to be bin\bash

@ionbasa I don't understand, this PR is about PostgreSQL image, not App image. I guess you speak about #208 right ?

@pichouk Yes, I posted in the wrong PR, sorry.

I guess we can merge now. Thanks a lot @andruwa13 :)


Removed Jade from html group

Jade should not be included under html because it serves a very different purpose, while it does compile to html it doesn't look like html and is a language for templating  rather than just markup.
Fixes #2977

GitHub, in its sole discretion, has the right to suspend or terminate your account and refuse any and all current or future use of the Service, or any other GitHub service, for any reason at any time. Such termination of the Service will result in the deactivation or deletion of your Account or your access to your Account, and the forfeiture and relinquishment of all Content in your Account. GitHub reserves the right to refuse service to anyone for any reason at any time.................!

Hi @zacps, thanks for opening this PR.
While I believe I understand your motivations for making this change, we've chosen to implement language groups in Linguist as we believe they represent higher-level logical groupings for collections of languages. For example the HTML group includes:

Jade
HTML+Django
HTML+ERB
HAML
Handlebars
...and many more

Quoting from http://jade-lang.com/reference/

Jade is a terse language for writing HTML templates.

Which I think makes Jade a good candidate for the HTML group as currently defined.

Jade should not be included under html because it serves a very different purpose, while it does compile to html it doesn't look like html and is a language for templating rather than just markup.

This argument can be made for pretty much every example in the list above too.
My question to you is: is there a usability problem that you're facing here? One thing I do think needs fixing on GitHub is what happens if you click the language bar for a repository and view the search results.
Currently, clicking the language bar on https://github.com/zacps/go-wiki shows you that a bunch of the files are in HTML, when this is clicked though...

you get sent to this page:

which I think is pretty confusing for all. My suggested fix is that we work out how to link these groups properly to the languages in the group (in this case we would show the Jade results).
The alternative is that we simply remove the groups from Linguist but I think they serve a useful purpose right now.

Yeah that's a fair point, I can't say I would know how to change that behaviour though. I guess wherever the handler for the bar is it's looking at the group name rather than the name itself.

There are languages like CoffeeScript/LiveScript, which are shown separately from JavaScript.
I think this is very similar to how Jade/Pug, HAML and others are different from HTML, so I'd also like to see them be separated from HTML group.


Merge the JupyterLab and Jupyter Notebook subprojects?

Problem
Jupyter Notebook 7 final has now been available for a couple of months (released in July 2023). There is a migration guide for Classic Notebook (v6) users, but now most of the development work is focused on Notebook 7.
As Notebook 7 is built on top of JupyterLab, there seems to be more overlap between the JupyterLab and Jupyter Notebook subprojects, in terms of scope, community, features, code base.
So the question is: do we still need to keep the JupyterLab and Jupyter Notebook subprojects separate? Or could the two subprojects be merged into a single "Jupyter Frontends" subproject? As a way to make it even more visible that JupyterLab and Jupyter Notebook are the two official frontends being developed by Project Jupyter.
Proposed Solution
Notebook 7 is pretty much JupyterLab re-assembled in a different way, so folks familiar with JupyterLab development would also be comfortable working on Notebook 7.  For JupyterLab and Jupyter Notebook developers, this could also help catch potential issues from decisions and technical choices happening upstream in JupyterLab, that can have an impact downstream in Notebook 7.
Having Notebook 7 on the radar could help keep the Notebook 7 use case in mind when designing features or making changes upstream in JupyterLab. This would benefit end users as developing the two frontends more closely could help improve UX and general design decisions.
Summary of the proposal:

Merge the JupyterLab and Jupyter Notebook meetings: https://github.com/jupyter/notebook-team-compass/issues/30
Merge the subprojects, and decide on a name. Maybe "Jupyter Frontends" would be a good candidate?
Merge the council members into a single council

Additional context
Opening this issue here in the JupyterLab team compass repo to get an idea of what people think, before opening an issue or PR in https://github.com/jupyter/governance.
This was also briefly discussed in https://github.com/jupyter/notebook-team-compass/issues/30.
For reference two other Jupyter subprojects were merged previously: https://github.com/jupyter/governance/pull/174

this makes a lot of sense to me judging by the contents of the meetings. this change would free up space for some other efforts potentially, or room for the docs working group to grow. the notebook team did an amazing keeping those efforts through all the difficult releases. merging meetings seem to signal positive growth.
i think this change could be really good for reinforcing accessibility changes. right now, a lot of changes are being made to jupyterlab components because it is our flagship. notebook should be the accessible flagship and i think merging the meeting might allow more folks to contribute to accessibility improvement notebooks when it shares equity with jupyterlab in meetings.

+1 to this idea, thanks for posting here, Jeremy!
As with the recent Foundations/Standards subproject merge, the process is:

Get consensus between the two subprojects that they'd like to merge
Open an PR in the governance repo, like https://github.com/jupyter/governance/pull/174
The SSC and EC vote on it (as it is a governance change)
Presuming it passes the vote in step 3, the PR is merged and the new combined subproject is official. Congratulations!

This makes a lot of sense to me. Thanks, all.

I am +1 on this. Thank you for opening the issue @jtpio.

I support this change, with the minor caveat that if the new name was "Jupyter Frontends", we would be in a place where qtconsole is a frontend that is not in the "Jupyter Frontends" .
"Jupyter Web Frontends" might be too much of a mouthful. Another approach would be to explicitly point to qtconsole as another frontend that's not under the purview of this new lab+notebook unified subproject.

Thx for opening this @jtpio.
Would that new "Jupyter (Web) Frontends" meeting/council also cover the Notebook 6 line, or just focus on the JupyterLab stack.
It is hard to know the numbers, but there is still a significant portion of users on Notebook 6 and wonder where they would be supported?

Not sure atm if the github notebook council team is updated, so I have also sent a mail to the Notebook council members via google groups to give visibility.

Jupyter Web Frontends

Then jupyterlab-server does contain non-frontend code though. JupyterLab does too for that matter (e.g. extension manager, all the lab-specific CLI stuff). Either Jupyter Frontends or Jupyter Web Frontends would feel like a misnomer. Some other options to explore would be Jupyter Clients or Jupyter User Interfaces (CLI is a user interface too). Or just Jupyter Notebook & Lab.

Or just Jupyter Notebook & Lab

That has the merit to be explicit. If we want more explicit (more is better in this case), this could be Jupyter Notebook>7 & Lab

CLI is a user interface too

100% onboard with that, and it looks like this proposal does not cover CLI stories if I am not mistaken.

Then jupyterlab-server does contain non-frontend code though. JupyterLab does too for that matter (e.g. extension manager, all the lab-specific CLI stuff). Either Jupyter Frontends or Jupyter Web Frontends would feel like a misnomer.

I would be in favor of a "Jupyter Frontends" project, with the condition that frontends are well separated from the backend. For instance, JupyterLab should be backend-agnostic, but it currently requires jupyter-server. This is an issue for users who want to use Jupyverse, because it brings additional installation constraints (see this issue).

For reference and as a follow-up to the JupyterLab meeting yesterday (https://github.com/jupyterlab/team-compass/issues/229#issuecomment-1896983165) we are current drafting the proposal in https://github.com/jupyter/governance/pull/200.
Will report here once https://github.com/jupyter/governance/pull/200 is ready for review.

Will report here once jupyter/governance#200 is ready for review.

https://github.com/jupyter/governance/pull/200 should now be ready for review.

https://github.com/jupyter/governance/pull/200 has been merged.
So it looks like some of the next steps could be:

merge the council members from the Notebook council (listed in https://github.com/jupyter/notebook-team-compass) here, and ask council members if they want to be part of the merged council
document the merge in https://github.com/jupyter/notebook-team-compass, and archive https://github.com/jupyter/notebook-team-compass
continue with the ongoing SSC representative election

Should we combine the team compasses into a new one
frontends-team-compass?
On Fri, Feb 2, 2024 at 8:26 AM Jeremy Tuloup @.***>
wrote:

jupyter/governance#200 https://github.com/jupyter/governance/pull/200
has been merged.
So it looks like some of the next steps could be:

merge the council members from the Notebook council (listed in
https://github.com/jupyter/notebook-team-compass) here, and ask
council members if they want to be part of the merged council
document the merge in
https://github.com/jupyter/notebook-team-compass, and archive
https://github.com/jupyter/notebook-team-compass
continue with the ongoing SSC representative election

—
Reply to this email directly, view it on GitHub
https://github.com/jupyterlab/team-compass/issues/230#issuecomment-1924212263,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAGXUBD6JRAKLDXYNKX5VLYRUHUZAVCNFSM6AAAAABBLN32GWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRUGIYTEMRWGM
.

On Leave - Professor of Physics and Data Science, Cal Poly
@ellisonbg on GitHub

Should we combine the team compasses into a new one frontends-team-compass?

Good question. https://github.com/jupyter/governance/pull/200 picked the JupyterLab team compass for the merge, mostly for convenience. Regarding the name maybe we could indeed consider renaming it to frontends-team-compass? Although it would probably be easier if the repo stays under the jupyterlab organization on GitHub (for using teams, also most of the development is happening in the jupyterlab org).

I think the rename sounds good (and keeping it in the JupyterLab org).

Closing as the repo has now been renamed (in https://github.com/jupyterlab/frontends-team-compass/pull/246).
Thanks all!


feat(card): add UPDATE_DATA action handler to card

Closes #
Summary

Allow custom cards to update their data outside of the normal Dashboard refresh cycle by firing an UPDATE_DATA action

Change List (commits, features, bugs, etc)

fix(Card): make all the actions a constant
feat(CardRenderer): listen to a new UPDATE_DATA action and update the card values accordingly based on the payload

Acceptance Test (how to verify the PR)

Verify no storybook changes

:tada: This PR is included in version 2.32.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Restore support for use wake word switch in m5stack atom echo and port to raspiaudio-muse-luxe

Addressing https://github.com/esphome/wake-word-voice-assistants/issues/11
Before merging, I'd recommend to have some more users test the configurations to make sure everything is working as expected as I have not been able to test the raspiaudio-muse-luxe.factory.yaml and I'm not sure if the improv_serial and esp32_improv components should be in the factory config or the standard config. I created the config based on m5stack atom echo 's factory config.

Also please split adding the muse luxe into its own PR


Create Files Signed by Rare Certificates.csl

I didn't have a very large scale environment to test this one in.  I simplified the query to remove the pivot on how many distinct machines had the file by SHA1 and added a Timestamp filter.


Consistent ws message format

In case of the ws message I'd rather have a consistent JSON structure, or even better I'd handle it a forgiving way. Here I think the best would be to go like this:
objectiveReactionId = message.get(constants.OBJECTIVE)
if objectiveReactionId is not None:
    self.model.objective = self.model.reactions.get_by_id(objectiveReactionId)

It keeps the ws hanging at the moment.

Closing as this is resolved in the caffeine branch


Cells sometimes rendered wrong on scrolling

Hi,
I've used this awesome project in one of mine, but when I scroll, there is a small chance the corners of the cell are not rounded and the inset is not applied. Check out the video below for clarification on what I mean:
https://www.youtube.com/watch?v=WHP0k6V5Mb4
This also happens in the example project compiled to my phone.

Thanks for opening the issue @MarcoStuurman. If I get time I’ll look in to it. Alternatively, if you come across a solution, it would be great if you could open a PR.

Hi @superpeteblaze I think I’m not really finding my way around in Swift well enough yet to find the problem, but I’ll try to look for it soon

any news for this?

No news yet @SaezChristopher. I have not found the time to look at this, so at the moment, hoping for some help from the community.

Hi, I think I might discovered the problem based on this comment: https://stackoverflow.com/a/13604032
Actualy, you can refactor the enterly math based in the solution that was proposed, or a alternative way is to:
1: Call the "collection.reloadData()"
2: Manualy Invalidates the current layout of the cell, forcing them to redraw everything with calling the "cell.setNeedsLayout()" at the end of the "cellForItemAt indexPath: IndexPath" function.
I hope I have helped you with something, it worked for me, 👍🏻.

Awesome! Thanks for letting us know @MacMichas 🙌🏼.
CC @MarcoStuurman @SaezChristopher if you try this an it work's, I'll close this issue.

didnt work....

Hi aviv..
Can you post your code? I ran tests now and it worked the way I instructed it in the post above.

@MacMichas Cool, that solves the issue. Awesome.


Adjust Divider UX

Hi, nice project! Datalog/prolog-like facts+rules is a great way to work with data, web-based notebooks are an ideal experience for exploring and sharing it, and I've been interested in trying out svelte + tailwindcss, so your project is surprisingly synchronous for me. Thank you for sharing it!
While exploring the project I noticed a couple small things so I fixed them and put them in separate commits for you to review:
Smaller one first: I factored the divider buttons to generate from a list. This should make it easier to add/remove buttons for new types of cells in the future and I think it's a tiny bit easier to read (also maybe an opportunity to generate from centrally registered cell types in some future version?). This is my first time using Svelte so I wanted to poke some (very!) minor low-hanging fruit first. 😸  Note that I moved the
 to the bottom of the div to reproduce identical button classes with :not(:first-child); I believe the result is the same.
Second, I noticed that you had to be unusually precise to trigger the opacity hover in order to see the divider buttons, in fact I was confused for a few minutes trying to figure out how to add new cells before I discovered that you just have to hover in the right place. So I adjusted the styles to make the divider hover target easier to activate, specifically:

Remove CellDivider h-2 so the div takes up the whole space between cells
Remove margin on Cells
Use negative margin on divider to slightly overlap neighboring cells'
padding. Keeps both compact spacing and easy hover target.
Increase divider z-index so the overlap works correctly

With some added borders to visualize, before:

Here you can see that you have to hover the buttons directly or that tiny sliver around the
 for the buttons to become visible.
After:

Now, hovering anywhere on the overlapped area triggers the hover effect of the divider which makes it much easier to find. I checked and none of the regular cells use this area so this change shouldn't impede their functionality. This does have a small effect on the vertical spacing of neighboring cells (3% taller based on the screenshots), but it is not very noticeable and worth the UX imo.
I'm sorry for writing so many words for such a small PR but I don't have time to make it shorter, please forgive me! 😅🙏 What do you think?

Thanks for the invite! I made the changes you mentioned.


How can a similar policy be created manually (for example to add some exceptions)?

I am unable to find these services in the cloud app list:
Azure portal
Azure PowerShell
Azure CLI.
is it all covered by the cloud app "Microsoft Azure Management"?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0327017d-eaaf-2f62-88b5-09f5bb142987
Version Independent ID: f08f88b4-bea1-e47f-e978-feae569b32d5
Content: Baseline policy Require MFA for service management (preview) - Azure Active Directory
Content Source: articles/active-directory/conditional-access/howto-baseline-protect-azure.md
Service: active-directory
Sub-service: conditional-access
GitHub Login: @MicrosoftGuyJFlo
Microsoft Alias: joflore

@rafalfitt Thanks for the feedback! We are currently investigating and will update you shortly.

@rafalfitt yes those are covered by "Microsoft Azure Management"

#please-close

@rafalfitt We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.

So please add these 3 words "Microsoft Azure Management" to the web page ;-)


preserve transparency

I've just been looking everywhere for ways to keep the transparent background of my PNGs, maybe adding a line in the doc could be useful for others, in the meantime, you have to do it like that:
$transparent = true;
Image::open($imageSourcePath)
             ->zoomCrop($width, $height, $transparent ? 'transparent' : $hexaColor, 'center', 'center')
             ->save($imageTargetPath, $extension, 85)
        ;

Or u can do simply like that too
 $image->open($imageSourcePath)                 ->zoomCrop($width, $height,'rgba(0,0,0,0)', 'center', 'center')                 ->save($imageTargetPath, 100);


RFC: nf-core inspiration for openlab

users install nf-core simply using
pip3 install nf-core
we will have a similar experience with
npm install openlab
Home Screen

openlab app list

interactive job definition when calling openlab job submit without a JSON file as argument
command line
inspiration from nf-core launch rnaseq

web configuration
https://nf-co.re/launch?id=1650885294_494afc2d85e5

The complete website, including the configuration builder, is open source via MIT license here: https://github.com/nf-core/nf-co.re

automatic detection of the parameter.json

https://www.loom.com/share/8e1f51db9176459896d7deded1ac0b96

The web configuration builder is an awesome idea! I also really, really like the commandline UI.


Fix for install-certificate workflow premature timeout.

Accepted in D1404 - https://secure.phabricator.com/D1404

Hm. Not sure why two commits got generated. I think I messed something up with arc merge (there was a change and an --amend done locally).

Thanks! Cherry-picked as 7e63e232ba6c310ccbf3648a031c370b71956107.


Allow custom jinja filters and tests

Jinja purposely limits the amount of logic and 'code' you can put into your templates, which makes sense, but they also then allow injecting custom filters and tests so you can expose more complicated helpers that the templates use. get_jinja_env allows passing in filters, but the call sites don't pass through any. It would be nice if there was a way to define custom filters in my template_src that are available to me when rendering files.

This might be a dupe of a the suggestion https://github.com/copier-org/copier/issues/208#issuecomment-699608983. I did find that file when looking for something like this issue, but thought it was only about adding one filter until I read all the way to the bottom.

Yes, it's duplicate. Thanks for the report anyway :blush:


[CI] Revive replayer test

Explain your changes:
Enable Replayer tests for berkeley
Explain how you tested your changes:
Job is enabled and green
Checklist:

[ ] Dependency versions are unchanged

Notify Velocity team if dependencies must change in CI

[ ] Modified the current draft of release notes with details on what is completed or incomplete within this project
[ ] Document code purpose, how to use it

Mention expected invariants, implicit constraints

[ ] Tests were added for the new behavior

Document test purpose, significance of failures
Test names should reflect their purpose

[ ] All tests pass (CI will check this if you didn't)
[ ] Serialized types are in stable-versioned modules
[ ] Does this close issues? List them

Closes #13596

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me


Tools included in the web site, but not showing up in TSV/JSON

There are at least several tools which nicely show up on the web site, but are neither included in all_tools.json nor all_tools.tsv. Examples of those tools include, but are not limited to:

2d_simple_filter
2d_auto_threshold
2d_feature_extraction
bfconvert
rfove

Thanks for spotting, that is certainly odd, I will look into it ...

So there are two issues here.

For some reason, 2d_simple_filter is not fetched anymore, the reason is not clear to me atm. Need to investigate.
The other issue is, that actually the interactive table for all tools is not created anymore, this is a bug in the code: https://github.com/galaxyproject/galaxy_codex/actions/runs/9658380463/job/26646826987 I actually didn't spot that, since our CI does not fail if python code fails, which is not good ....I'll make another issue for that.

If I may suggest, to prevent similar issues in the future, it might be useful to include the date when the table was built last into the footer :)

The tool that broke the CI was this here: https://github.com/BMCV/galaxy-image-analysis/tree/98c731e30143679febc9c479c4fdf01abda2857b/tools/points2binaryimage
It tricked our parser with this link leading nowhere: https://github.com/BMCV/galaxy-image-analysis/blob/98c731e30143679febc9c479c4fdf01abda2857b/macros/creators/bmcv.xml
Thanks for spotting, now we can fix that logic.

If I may suggest, to prevent similar issues in the future, it might be useful to include the date when the table was built last into the footer :)

The creation time can be seen from the github commit. I actually found that problem directly that way. I just didn't look at the complete interactive table for a long time ...

The tool that broke the CI was this here: https://github.com/BMCV/galaxy-image-analysis/tree/98c731e30143679febc9c479c4fdf01abda2857b/tools/points2binaryimage It tricked our parser with this link leading nowhere: https://github.com/BMCV/galaxy-image-analysis/blob/98c731e30143679febc9c479c4fdf01abda2857b/macros/creators/bmcv.xml Thanks for spotting, now we can fix that logic.

Thanks for spotting the broken link!


slow or unresponsive typing

writing in the chat box slows down and is unresponsive for a few instants periodically after a few minutes of using (using the wayland session)
issue fixed on X11 buuuut
Operating System: Bazzite 40
KDE Plasma Version: 6.0.5
KDE Frameworks Version: 6.3.0
Qt Version: 6.7.1
Kernel Version: 6.9.4-201.fsync.fc40.x86_64 (64-bit)
Graphics Platform: Wayland
Processors: 16 × AMD Ryzen 7 7800X3D 8-Core Processor
Memory: 30.6 GiB of RAM
Graphics Processor: NVIDIA GeForce RTX 4070/PCIe/SSE2
Manufacturer: ASUS

Thanks for reporting. Unfortunately, I don't use wayland as my daily driver yet, so it will take me some time to debug it.


mock build list requests to avoid potential s3 issues

Fixes #753.

@johnnyman727 Does appveyor need to be re-run? https://ci.appveyor.com/project/rwaldron/t2-cli/build/958

@HipsterBrown perhaps. Sent up a new commit sha so it should restart (I don't know of any other way to do it).

@HipsterBrown I'm not sure what's up with AppVeyer. It's failing with Failed to provision build worker virtual machine in a cloud. Please try restarting build later. Maybe someone with more AppVeyer experience can comment?

@johnnyman727 Yea I noticed that. I've been looking for a way to re-run the build without pushing new commits (like Travis allows) but I can't find anything in the docs. Looking through the support forums as well.

@johnnyman727 Looks to be an issue for other appveyor users today -> http://help.appveyor.com/discussions/problems/4750-failed-to-provision-build-worker-virtual-machine-in-a-cloud-please-try-restarting-build-later

LGTM once Appveyor passes. fingers-crossed && knocks on wood


Solution Architecture Diagram for Real-time Twitter sentiment analysis in Azure Stream Analytics

I have seen too many technical sales experts, specialists, and solution architects reading the article published here and missing the middleman client application required to get the solution up and running in Azure. Most assume Twitter and Event Hubs can be connected by simply registering a Twitter Application. I hope adding a solution diagram can help to make it visually clear that there is a TwitterCoreClient application in the picture.

@daronyondem : Thanks for your contribution! The author(s) have been notified to review your proposed change. @enkrumah

@daronyondem : Thanks for your contribution! The author(s) have been notified to review your proposed change. @enkrumah

Hi @JasonWHowell - I can't assign this pull request to author @enkrumah because their GitHub account isn't in the Microsoft Docs organization (the process to join the org is described in the internal contributor guide); I'll assign it to you for now. Could you coordinate review as needed? Thanks.

Hi @JasonWHowell - I can't assign this pull request to author @enkrumah because their GitHub account isn't in the Microsoft Docs organization (the process to join the org is described in the internal contributor guide); I'll assign it to you for now. Could you coordinate review as needed? Thanks.

CC: @enkrumah
#reassign:enkrumah

CC: @enkrumah
#reassign:enkrumah

Thanks for the feedback @JasonWHowell Let me know how you want me to change the PR. I can remove the visio link from the document, but keep the file, or delete both.

Thanks for the feedback @JasonWHowell Let me know how you want me to change the PR. I can remove the visio link from the document, but keep the file, or delete both.

Update: It looks like this commit was moved to the private repository, and it was merged to master from there. 👍🏻

Update: It looks like this commit was moved to the private repository, and it was merged to master from there. 👍🏻


Working bad since last updates

Hi there!
Since the last big update (when it requires to install an extra plugin etc) this plugin is not working as well as it did before.
Sometimes it doesn't stop on breakpoints. The PHP Debug panel opens randomly when it's closed. It opens a lot of random panels... in summary. it doesn't work well.
I'm wondering if it is just me or other users are experiencing the same weird behaviour.
Cheers!

Duplicate of #291
This should be fixed in the next release (sometime this week). There is a flag that's sent to xdebug but to disable multiple connections but it looks like xdebug is ignoring this in certain situations.

Should be Fixed in v0.3.5. If not please reopen

Hey, thanks for the update. I've to 0.3.5 but issues are still there.
For example, it stops on the breakpoint even if I don't have the PHP Debug console open.
It also opens randomly PHP Console when debugging
Cheers!

@jvlobo Have to unchecked the setting to allow multiple connections?

Ok, I've unchecked that one.
But I had unchecked the Continue to listen for debug session even if the debugger windows are all closed and it stop on breakpoint with the debugger windows closed...
Thanks!

@jvlobo Have to unchecked the setting to allow multiple connections?

@cchamplin If I uncheck this checkbox, it won't catch any break point at all.
If I leave this setting, it opens many tabs like PHP Debug(<number>) and when I try to 'step over' it writes 'Session active: BREAK' and all the navigation buttons are disabled. The only option, that I have in such case — Detach or Stop.

As of now if I uncheck that box it may or may not catch the initial breakpoint, but stepping through will break almost immediately as new requests/connections are made throughout the run of that php process so it just gets hung.  If I allow multiple connections, I'm just flooded with new phpdebug/console tabs.  Can we just have it behave as it did before and accept multiple connections and use the same two tabs?

Unchecking accept multiple connections should result in the same behavior as the prior version (0.2.x) where new connections are rejected if a connection is already active. Can you enable protocol debugging and provide a paste of the output from the atom debug console?

It just seems to be very inconsistent- if I disallow multiple connections, my cli spawned php doesn't hit any breakpoints, I just see a session start/stop, the first of which is my web request to apache which then spawns off a cli php- but no breakpoints are hit.  If I enable multiple connections, the breakpoints get hit, but any web requests which may be going on spawn off a bunch of other debug tabs.  With xdebug output enabled I still only see this:
Session Initialized: default
Session Terminated: default
Session Initialized: default
Session Terminated: default

Not sure if it's related, but I found that I had to quit Atom and relaunch to get debugging to work consistently. Then I changed from port 9000 to 8881 and I haven't had the problem since. It was related to committing code to GitHub (Shift Command A, then P). Can't imagine why that was, but changing the port fixed it.

As of now my main problem seems to be that it doesn't listen to the "Continue to listen for debug sessions..." option, so merely closing the pane doesn't work to stop it listening so I'm forced to just close it out to stop it from catching things when I want to let php just run through without hitting all of the breakpoints I've set.

Any news on all the issues coming from last update?
Cheers!

@cchamplin sorry, I'm use PHP debugger really rare, because I'm frontend developer :)
I have currently tested with disabled "Allow multiple debug sessions at once" and it seems to be ok. I can't reproduce the behaviour I have written before. With and without protocol debug option.