Proxy randomly closes during connection

Bug Description
After a properly established connection, sometimes I notice the following error pops out of nowhere:
Proxy error, connectivity issue or a bug.
                    Please report it to us on https://github.com/metalbear-co/mirrord/issues/new?assignees=&labels=bug&projects=&template=bug_report.yml
                    You can find the `mirrord-intproxy` logs in /tmp/mirrord.log.
                    connection closed

 /tmp/mirrod.logs:
  2024-11-06T18:35:27.624023Z  INFO mirrord_intproxy::layer_initializer: new session, process_info: ProcessInfo { pid: 73153, name: "node", cmdline: ["/Users/tyler/.volta/tools/image/node/20.11.1/bin/node", "--enable-source-maps", "--inspect", "/Users/tyler/voiceflow/platform/apps/runtime/build/main"], loaded: true }
    at mirrord/intproxy/src/layer_initializer.rs:63

  2024-11-06T18:42:14.387598Z  WARN mirrord_kube::api::kubernetes::portforwarder: finished retry strategy, closing connection, connect_info: AgentKubernetesConnectInfo { pod_name: "mirrord-agent-wk5eyjv6hc-vllzt", agent_port: 44128, namespace: Some("dev-tyler"), agent_version: Some("3.122.1") }
    at mirrord/kube/src/api/kubernetes/portforwarder.rs:172

  2024-11-06T18:42:14.395494Z ERROR mirrord_intproxy::agent_conn: failed to receive message from the agent, inner task down
    at mirrord/intproxy/src/agent_conn.rs:226

  2024-11-06T18:42:14.396023Z ERROR mirrord_intproxy: task AGENT_CONNECTION failed: agent unexpectedly closed connection
    at mirrord/intproxy/src/lib.rs:243

  2024-11-06T18:42:14.396986Z ERROR mirrord::internal_proxy: Internal proxy encountered an error, exiting, error: Main internal proxy logic failed: agent unexpectedly closed connection
    at mirrord/cli/src/internal_proxy.rs:146

Steps to Reproduce
N/A hard to reproduce but happens randomly
Backtrace
No response
mirrord layer logs
No response
mirrord intproxy logs
No response
mirrord agent logs
No response
mirrord config
No response
mirrord CLI version
mirrord 3.122.1
mirrord-agent version
ghcr.io/metalbear-co/mirrord:3.122.1
mirrord-operator version (if relevant)
No response
plugin kind and version (if relevant)
No response
Your operating system and version
OSx Sonoma 14.4.1
Local process
nodeJS
Local process version
nodeJS v20.11.1
Additional Info
No response

Hi,
Looking at the error this seems to be an issue with kubectl port-forward (on the API level)
  2024-11-06T18:42:14.387598Z  WARN mirrord_kube::api::kubernetes::portforwarder: finished retry strategy, closing connection, connect_info: AgentKubernetesConnectInfo { pod_name: "mirrord-agent-wk5eyjv6hc-vllzt", agent_port: 44128, namespace: Some("dev-tyler"), agent_version: Some("3.122.1") }
    at mirrord/kube/src/api/kubernetes/portforwarder.rs:172

We know that kubectl port-forward tends to be flaky in some cases, but it's kinda out of our control (we even do retries as seen in the logs to handle it).
You can check your kube-apiserver if it restarted/had any error but other then that I think it's not something we can solve from our side.
P.S - The operator doesn't use kubectl port-forward (it implements its own API) and we didn't see a report of such issue for users of it - so perhaps that can be a solution.

It seems like the agent just dies when this happens. Retried again but with mirrord 3.124.0
Same thing in mirrord-intproxy:
  2024-11-06T19:27:15.175003Z  WARN mirrord_kube::api::kubernetes::portforwarder: finished retry strategy, closing connection, connect_info: AgentKubernetesConnectInfo { pod_name: "mirrord-agent-nxavbqhb4z-6strh", agent_port: 32170, namespace: Some("dev-tyler"), agent_version: Some("3.122.1") }
    at mirrord/kube/src/api/kubernetes/portforwarder.rs:172

  2024-11-06T19:27:15.190863Z ERROR mirrord_intproxy::agent_conn: failed to receive message from the agent, inner task down
    at mirrord/intproxy/src/agent_conn.rs:226

  2024-11-06T19:27:15.191048Z ERROR mirrord_intproxy: task AGENT_CONNECTION failed: agent unexpectedly closed connection
    at mirrord/intproxy/src/lib.rs:243

  2024-11-06T19:27:15.193633Z ERROR mirrord::internal_proxy: Internal proxy encountered an error, exiting, error: Main internal proxy logic failed: agent unexpectedly closed connection
    at mirrord/cli/src/internal_proxy.rs:146

Agent dying when connection ends (because port forward died) makes sense. If agent crashed I would've expected it to send "bye" message. If you'd like to make sure it's not agent dying for some reason I'd tail the logs of the agent and see if you get any error besides "no connections alive, shutting down"

Yup after investigating more port forward is definitely the culprit. Is there any good way around this at all?
https://github.com/kubernetes/kubectl/issues/1525#issuecomment-2196452123
We can look into the operator, since it's not using port-forward

Yup after investigating more port forward is definitely the culprit. Is there any good way around this at all? kubernetes/kubectl#1525 (comment)
We can look into the operator, since it's not using port-forward

Reading the issue you mentioned I don't really understand what's going on since the crate we use for kube uses websocket already, but maybe k8s does websocket -> spdy -> in the webserver ?
in any case it looks like a longer term thing (we need kube crate to support the new connection if needed to change) and it might be using it already tbh

@aviramha I am seeing this in our kube-apiserver logs:

Failed calling webhook, failing open agent-injector-ambassador.getambassador.io: failed calling webhook "agent-injector-ambassador.getambassador.io": failed to call webhook: Post "https://agent-injector.ambassador.svc:443/traffic-agent?timeout=5s": no endpoints available for service "agent-injector"

This is still happening quite frequently on our end.
Before on Kubernetes 1.27 it was not noticeable, but after upgrading to Kubernetes 1.30 it's happening quite often and has been distruptive.
I've enabled logs for the Kubernetes apiserver via AWS EKS, but nothing significant has appeared there. I can send them over to you via a private channel if you want.
Is there anything in particular that you are looking for in the kube-apiserver logs?

We found out that user used calico CNI - need to retest.


update Calibration

Calibration: split timing margins, reorder output to be the same as in html node
html: adjust default values, add [ms] note to timing boxes

@tsknightstorm,
Thanks again for the testwork and improvements!!
Bart


Crash when I interact with "Experience Obelisk"

I play in my modpack with 200+ mods, the minecraft crash when I interact with "Experience Obelisk"
Crash report

EnderIO:EnderIO-1.7.10-<IP_ADDRESS>9_beta
EnderCore:EnderCore-1.7.10-<IP_ADDRESS>_beta
Minecraft:1.7.10
Forge:forge-1.7.10-<IP_ADDRESS>8-1.7.10-universal
SpongeForge:No
Optifine:Yes
Single Player

Something is disabling the exp liquid

Is a mod conflict ?

sorry though i responded here. Yes some mod is disabling the exp liquid that the experience obelisk needs

Ok


File naming error in sample.py

Environment
Rotary end_repair commit 9a4678b
Running on a Linux (Ubuntu) server
Problem description
With the following files in my input folder:
S18_long.fastq.gz  S18_R1.fastq.gz  S18_R2.fastq.gz

When I run rotary init -o output_dir -d rotary_db_dir -i input, I get the following error:
Traceback (most recent call last):
  File "/Analysis/jmtsuji/miniconda3_me23/envs/rotary_tmp/bin/rotary", line 8, in <module>
    sys.exit(main())
  File "/Analysis/jmtsuji/projects/tool_development/2022_08_01_rotary_testing/rotary/rotary/rotary.py", line 122, in main
    init(args, config, output_dir_path)
  File "/Analysis/jmtsuji/projects/tool_development/2022_08_01_rotary_testing/rotary/rotary/rotary.py", line 202, in init
    file = SequencingFile(file_path=(os.path.join(input_path, file)))
  File "/Analysis/jmtsuji/projects/tool_development/2022_08_01_rotary_testing/rotary/rotary/sample.py", line 26, in __init__
    raise ValueError(f'{self.name} is not a fastq file.')
ValueError: S18_R2.fastq.gz is not a fastq file.

Possible cause
I think this is because the .gz is not being handled properly by os.path.splitext() in sample.py, class SequencingFile
    def __init__(self, file_path):
        self.path = file_path
        self.name = os.path.basename(self.path)

        file_extension = os.path.splitext(self.name)[1]

        if 'fastq' not in file_extension and 'fq' not in file_extension:
            raise ValueError(f'{self.name} is not a fastq file.')

        ...

Perhaps just the .gz is being saved as not the .fastq?
Proposed solution
Maybe the easiest workaround would be to add a test for if the file has a .gz extension and then handle it differently.

Addressed in https://github.com/jmtsuji/rotary/pull/64

Thanks -- your proposed change looks good to me.


Master branch is for 0.10

Thanks for your pull request.  Please review the following guidelines.

[x] Title includes issue id.
[x] Description of the change added.
[x] Commits are squashed.
[ ] Tests added.
[ ] Documentation added/updated.
[x] Also please review CONTRIBUTING.md.

retest this please


⚠ Roblox Devforum has degraded performance

In 0658c7d, Roblox Devforum (https://devforum.roblox.com) experienced degraded performance:

HTTP code: 200
Response time: 4682 ms

Resolved: Roblox Devforum performance has improved in 30adfae.


Flutter run results in Error connecting to the service protocol: failed to connect to http://<IP_ADDRESS>

The issue:
As of yesterday my flutter run commands have resulted in error connecting to service protocol. I've gotten flutter run to work a few times, by running flutter clean and flutter pub get then flutter run.
However, as of today, I can't get 90% of my builds to run whatsoever. During the times that it does work, after a few reloads,  I run into the same issue.
Currently, I'm working off of an apple developer certificate, connected via a usb to my iphone 10 and running the xcode build before I do any terminal commands(flutter clean, flutter pub get, flutter run). My xcode builds always succeed, so i don't believe it's an issue with my build configs/certificate issue
flutter doctor -v

flutter run -v (logs where build running fails)
        ] Error connecting to the service protocol: failed to connect to http://<IP_ADDRESS>:1030/fYlyN59L88I=/
[   +2 ms] "flutter run" took 41,584ms.

#0      throwToolExit (package:flutter_tools/src/base/common.dart:28:3)
#1      RunCommand.runCommand (package:flutter_tools/src/commands/run.dart:509:7)
<asynchronous suspension>
#2      FlutterCommand.verifyThenRunCommand (package:flutter_tools/src/runner/flutter_command.dart:615:18)
#3      _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:73:64)
#4      _rootRunUnary (dart:async/zone.dart:1134:38)
#5      _CustomZone.runUnary (dart:async/zone.dart:1031:19)
#6      _FutureListener.handleValue (dart:async/future_impl.dart:139:18)
#7      Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:680:45)
#8      Future._propagateToListeners (dart:async/future_impl.dart:709:32)
#9      Future._completeWithValue (dart:async/future_impl.dart:524:5)
#10     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:32:15)
#11     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:290:13)
#12     RunCommand.usageValues (package:flutter_tools/src/commands/run.dart)
#13     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:73:64)
#14     _rootRunUnary (dart:async/zone.dart:1134:38)
#15     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
#16     _FutureListener.handleValue (dart:async/future_impl.dart:139:18)
#17     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:680:45)
#18     Future._propagateToListeners (dart:async/future_impl.dart:709:32)
#19     Future._completeWithValue (dart:async/future_impl.dart:524:5)
#20     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:32:15)
#21     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:290:13)
#22     IosProject.isSwift (package:flutter_tools/src/project.dart)
#23     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:73:64)
#24     _rootRunUnary (dart:async/zone.dart:1134:38)
#25     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
#26     _FutureListener.handleValue (dart:async/future_impl.dart:139:18)
#27     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:680:45)
#28     Future._propagateToListeners (dart:async/future_impl.dart:709:32)
#29     Future._completeWithValue (dart:async/future_impl.dart:524:5)
#30     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:32:15)
#31     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:290:13)
#32     IosProject.buildSettings (package:flutter_tools/src/project.dart)
#33     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:73:64)
#34     _rootRunUnary (dart:async/zone.dart:1134:38)
#35     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
#36     _FutureListener.handleValue (dart:async/future_impl.dart:139:18)
#37     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:680:45)
#38     Future._propagateToListeners (dart:async/future_impl.dart:709:32)
#39     Future._completeWithValue (dart:async/future_impl.dart:524:5)
#40     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:32:15)
#41     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:290:13)
#42     XcodeProjectInterpreter.getBuildSettings (package:flutter_tools/src/ios/xcodeproj.dart)
#43     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:73:64)
#44     _rootRunUnary (dart:async/zone.dart:1134:38)
#45     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
#46     _FutureListener.handleValue (dart:async/future_impl.dart:139:18)
#47     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:680:45)
#48     Future._propagateToListeners (dart:async/future_impl.dart:709:32)
#49     Future._completeWithValue (dart:async/future_impl.dart:524:5)
#50     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:32:15)
#51     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:290:13)
#52     _DefaultProcessUtils.run (package:flutter_tools/src/base/process.dart)
#53     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:73:64)
#54     _rootRunUnary (dart:async/zone.dart:1134:38)
#55     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
#56     _FutureListener.handleValue (dart:async/future_impl.dart:139:18)
#57     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:680:45)
#58     Future._propagateToListeners (dart:async/future_impl.dart:709:32)
#59     Future._completeWithValue (dart:async/future_impl.dart:524:5)
#60     Future.wait.<anonymous closure> (dart:async/future.dart:400:22)
#61     _rootRunUnary (dart:async/zone.dart:1134:38)
#62     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
#63     _FutureListener.handleValue (dart:async/future_impl.dart:139:18)
#64     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:680:45)
#65     Future._propagateToListeners (dart:async/future_impl.dart:709:32)
#66     Future._complete (dart:async/future_impl.dart:514:7)
#67     _BufferingStreamSubscription.asFuture.<anonymous closure> (dart:async/stream_impl.dart:206:14)
#68     _rootRun (dart:async/zone.dart:1122:38)
#69     _CustomZone.run (dart:async/zone.dart:1023:19)
#70     _CustomZone.runGuarded (dart:async/zone.dart:925:7)
#71     _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:391:13)
#72     _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:401:15)
#73     _BufferingStreamSubscription._close (dart:async/stream_impl.dart:285:7)
#74     _SinkTransformerStreamSubscription._close (dart:async/stream_transformers.dart:98:11)
#75     _EventSinkWrapper.close (dart:async/stream_transformers.dart:25:11)
#76     _StringAdapterSink.close (dart:convert/string_conversion.dart:251:11)
#77     _Utf8ConversionSink.close (dart:convert/string_conversion.dart:302:20)
#78     _ConverterStreamEventSink.close (dart:convert/chunked_conversion.dart:82:18)
#79     _SinkTransformerStreamSubscription._handleDone (dart:async/stream_transformers.dart:143:24)
#80     _rootRun (dart:async/zone.dart:1122:38)
#81     _CustomZone.run (dart:async/zone.dart:1023:19)
#82     _CustomZone.runGuarded (dart:async/zone.dart:925:7)
#83     _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:391:13)
#84     _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:401:15)
#85     _BufferingStreamSubscription._close (dart:async/stream_impl.dart:285:7)
#86     _SyncStreamControllerDispatch._sendDone (dart:async/stream_controller.dart:774:19)
#87     _StreamController._closeUnchecked (dart:async/stream_controller.dart:631:7)
#88     _StreamController.close (dart:async/stream_controller.dart:624:5)
#89     _Socket._onData (dart:io-patch/socket_patch.dart:1838:21)
#90     _rootRunUnary (dart:async/zone.dart:1138:13)
#91     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
#92     _CustomZone.runUnaryGuarded (dart:async/zone.dart:933:7)
#93     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:338:11)
#94     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:265:7)
#95     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:766:19)
#96     _StreamController._add (dart:async/stream_controller.dart:642:7)
#97     _StreamController.add (dart:async/stream_controller.dart:588:5)
#98     new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1386:35)
#99     _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:892:18)
#100    _microtaskLoop (dart:async/schedule_microtask.dart:43:21)
#101    _startMicrotaskLoop (dart:async/schedule_microtask.dart:52:5)
#102    _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
#103    _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:175:5)

Hi @kendall-lu
I see there's an open issue addressing the case you described.
Please follow up on that issue,
I'm closing the current one as duplicate.
If you disagree please write in the comments
and I will reopen it.
Thank you


[QUESTION] Multi-language

Does var filter = require('leo-profanity'); supports multiple language by default or should I create 2 var and load two dictionary?

No, it only support only english language by default.
If you like to filter all languages available, you can do something like this.

// load the module
const filter = require('leo-profanity');

// in this state,
// the "filter" only use "english" dictionary by default
filter.list()

// remove all bad words from the filter
// now the filter can't filter anything cause there are no bad words
filter.clearList()

// adding word (from builtin dictionary) into the filter
filter.add(filter.getDictionary('en'))
filter.add(filter.getDictionary('fr'))
filter.add(filter.getDictionary('ru'))

// now in this state
// the filter includes all bad words across 3 languages available
filter.list()

Hi. Sorry for disturbing. May I know is there a way to setup a custom dictionary or adding dictionaries?

There is no way to add dictionary for now, but you can add list of words. Please see an example below.
// load the module
const filter = require('leo-profanity');

// in this state,
// the "filter" only use "english" dictionary by default
filter.list()

// remove all bad words from the filter
// now the filter can't filter anything cause there are no bad words
filter.clearList()

// add custom list of bad words
filter.add(['b00b', 'b@@b']);


feat(StringList): implement str_len

#131
The method behaves like:
>>> import ulist as ul
>>> strs = ul.from_seq(["hello", "world!"], dtype='string')
>>> strs.str_len()
UltraFastList([5, 6])

But I don't know how to add a test case for this implementation.
First PR for ulist ^_^

#131 The method behaves like:
>>> import ulist as ul
>>> strs = ul.from_seq(["hello", "world!"], dtype='string')
>>> strs.str_len()
UltraFastList([5, 6])

But I don't know how to add a test case for this implementation. First PR for ulist ^_^

Hey, @yingmanwumen. Thanks for the PR.
We use pytest to test the new functions and methods. For this issue, the unit tests are put in ulist/tests/test_string.py. After adding your test cases, please run cd /you_path/../tests/test_string.py and pytest  test_string.py locally. The PR can also trigger the Github action to run the unit tests in the cloud.


add missing word

Here's what this PR contains:
Add missing word in Profiler documentation

Thank you!


Revert "Merge pull request #1 from Brandon85/Wheels85-patch-1"

Reverts Brandon85/upgraded-waddle#2


Add Razorpay Gateway for checkout integrated with website and customer details gets stored

I am online to accept the PR just do it ASAP

Do you actually want to use Razorpay gateway for payment purposes or a sample model?
If you actually want to use payment services then please provide your api key and id of razorpay

@RaviAnand59 just Say there #api key and i will fill it and follow Code of Conduct by Star the repo


⚠ Bofrost.es has degraded performance

In 44ad85c, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1635 ms

Resolved: Bofrost.es performance has improved in 54e0a0f after 13 minutes.


Feature: Add optional timeout variable for network connections

I needed to shorten the timeout for printers to return to the user more quickly.
I'm not familiar with merges etc sorry. Thought I would share maybe make it into dev branch
printer.py line 155
def __init__(self, host, port=9100, timeout=60, *args, **kwargs):
    """

    :param host : Printer's hostname or IP address
    :param port : Port to write to
    """
    Escpos.__init__(self, *args, **kwargs)
    self.host = host
    self.port = port
    self.timeout = timeout
    self.open()

def open(self):
    """ Open TCP socket with ``socket``-library and set it as escpos device """
    self.device = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    self.device.settimeout(self.timeout)
    self.device.connect((self.host, self.port))

    if self.device is None:
        print("Could not open socket for {0}".format(self.host))

Thank you for your contribution!
If you want to you can clone the repository and the create a pull request against dev-branch. If you do so please add something like param timeout: timeout in ?ms? for the socket-library to the docstring of the __init__-method.
Otherwise I could simply add this commit by myself, if it is ok for you. If you want to be credited in the commit you could send me the mail-adress that you use for github to patrick.kanzler{at}fablab.fau.de


[UI] Add delay to the user-dropdown

Rancher Versions:
Server: 1.4.0
Steps to Reproduce:

Attempt to change the stack view from grouped to dots.
Overshoot the pointer and accidentally log out.

Results:

You are logged out.

Expected

The user drop-down should wait a fraction of a second before popping-out.
You stay logged in!

The drop-down is not positioned the same in 2.0, so it won't be as easy to logout


Thank you for your contribution to the WebPower package!

Hi Conor,
Thank you so much for your commits to the WebPower R package! I'm one of the contributors of this package and just want to let you know that we are updating the WebPower package on cran and are adding you as one of the contributors. Have a nice day!

Thank you @iasnobmatsu!


[Feature Request/Idea] New Power Type "Action on Game Event"

Hello!
I just thought it would be cool if there was a way to make origins with sound-based abilities, using the new 1.19 sound feature.
An idea I had was to have a power type that would activate whenever the player detected a game event. I'm not sure how easy or hard this would be, so it's completely up to you if you want to add it or not.
Some ideas for the power type would be to have a field for the sound strength, a field for a bi-entity action/entity action, or a bi-entity condition/entity condition.
If you do decide to make it, thanks in advance.

This is now in Origins 1.11.x in the form of a power type named game_event_listener


fix: added favicon #86

fixed #86

Added Favicon

Can you share the screenshot? @priyankarpal

Can you share the screenshot? @priyankarpal

I just created PR from github directly not from the Local machine. 😢

I think there are som error when we try to run


zstash check deleting local tar file

I thought we fixed this, but maybe not completely or it came back?
I'm running zstash on compy (without HPSS). zstash create worked fine, but zstash check deletes the files after checking is done. Here is what I did:
module load anaconda3/2019.03
source /share/apps/anaconda3/2019.03/etc/profile.d/conda.sh
conda activate zstash_env
cd /compyfs/gola749/E3SM_simulations/20191216.alpha20.piControl.ne30_r05_oECv3_ICG.compy
mkdir zstash
zstash create --hpss=none  --maxsize 128 . 2>&1 | tee zstash/zstash_create_20200224.log
zstash check --workers=2 2>&1 | tee zstash/zstash_check_20200226.log

I wonder whether it has to do with the fact that I'm not specifying --hpss=none on the command line the second time around, instead relying on what is stored in the database. We need to fix this urgently and make sure we have tests catch this.

@golaz can you specify which files are being deleted?
I created a directory and ran the following from there:


[OCSI] add new osci author kshitijalwadhi

Description
Added author kshitijalwadhi for OCSI as per the guide.
Check List

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

@kshitijalwadhi once you address Josh's comments we should be good to go. Thank you!

Closing as still missing DCO and we are past deadline


why not download to Mega?

why not download to Mega like this https://github.com/jpillora/node-torrent-cloud? and how can i install it on vps as a web page like it work on heroku? thanks

Because It is not the purpose of this application. It store files into the server in order to manage the file from the web interface.
If you want to you Mega anyway see Mega-Torrent.
If you want to install it on a VPS see Lunik-Torrent installation.

i tried install Lunik-Torrent on coud9's ubuntu workspace, it show lots of error ater "npm run forever-start",
`><EMAIL_ADDRESS>forever-start /home/ubuntu/workspace/Lunik-Torrent

forever -a start src/index.js

sh: 1: forever: not found
npm ERR! Linux 4.2.0-c9
npm ERR! argv "/home/ubuntu/.nvm/versions/node/v4.4.3/bin/node" "/home/ubuntu/.nvm/versions/node/v4.4.3/bin/npm" "run" "forever-start"
npm ERR! node v4.4.3
npm ERR! npm  v2.15.1
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR<EMAIL_ADDRESS>forever-start: forever -a start src/index.js
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>forever-start script 'forever -a start src/index.js'.
npm ERR! This is most likely a problem with the lunik-torrent package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     forever -a start src/index.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs lunik-torrent
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!
npm ERR!     npm owner ls lunik-torrent
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     /home/ubuntu/workspace/Lunik-Torrent/npm-debug.log`
and c9 has installed npm, so i has no use "npm install".

In order to use forever-start, you need to install forever.
npm install forever -g

i tried again,

but still failed

i tried to run index.js , it show that

You forget:
npm install
see Lunik-Torrent installation.

thank so much, i will try  Mega-Torrent later

You'r welcome ;)


merge package template from Kronos-Tools/npm-package-template

.travis.yml

chore(travis): merge from template .travis.yml

Codecov Report

Merging #119 into master will not change coverage.
The diff coverage is n/a.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3033456...2767af7. Read the comment docs.


allow CLI/SSH based configuration from a file referenced in topo

Currently containerlab allows to mount full config files for some kinds (like srl).
But it is not yet possible to specify a file with CLI commands that would have been applied on the boot of the system.
A proposal is to allow referencing a file that contains CLI commands that should be executed on nodes SSH availability.
As an MVP the code should

distinct if the file mounted in config statement contains full config or CLI commands. The distinction can be made by the file extension. Maybe *.cli will be a reasonable  extension to notify containerlab that the file contains CLI commands, rather than full config.
once it is known that the config contains CLI commands the code should wait till SSH service is available on the default port
when ssh becomes available, the code should send the CLI commands over SSH channel
the username/password will be default ones as known to containerlab. If custom user/pwd needed, they should be handled via ENV vars (TBD)

/cc @jbemmel

this can be closed as we how config engine in place
additional work will be done in different purpose-built PRs


[improve][txn]Add log when can't send TcClientConnectRequest to broker.

Main Issue: https://github.com/apache/pulsar/issues/20851
Motivation
When the Proto version does not allow us to send TcClientConnectRequest to the broker, we should add a log to debug it.
Modifications
Add a waining log.
Verifying this change

[ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

[ ] Dependencies (add or upgrade a dependency)
[ ] The public API
[ ] The schema
[ ] The default values of configurations
[ ] The threading model
[ ] The binary protocol
[ ] The REST endpoints
[ ] The admin CLI options
[ ] The metrics
[ ] Anything that affects deployment

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository:

Codecov Report

Merging #20880 (8ebfd77) into master (69d7a2b) will decrease coverage by 0.09%.
Report is 10 commits behind head on master.
The diff coverage is 81.08%.

@@             Coverage Diff              @@
##             master   #20880      +/-   ##
============================================
- Coverage     73.17%   73.08%   -0.09%
+ Complexity    32231    32230       -1
============================================
  Files          1874     1874
  Lines        139302   139379      +77
  Branches      15322    15328       +6
============================================
- Hits         101934   101865      -69
- Misses        29311    29462     +151
+ Partials       8057     8052       -5

Flag
Coverage Δ

inttests
24.15% <50.00%> (-0.27%)
:arrow_down:

systests
25.06% <50.00%> (-0.14%)
:arrow_down:

unittests
72.38% <81.08%> (-0.08%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed
Coverage Δ

...he/pulsar/broker/resources/NamespaceResources.java
80.71% <ø> (-0.72%)
:arrow_down:

...broker/service/persistent/DispatchRateLimiter.java
81.48% <ø> (ø)

...roker/service/persistent/SubscribeRateLimiter.java
58.76% <ø> (ø)

...ulsar/client/impl/TransactionMetaStoreHandler.java
68.12% <0.00%> (-0.09%)
:arrow_down:

...rg/apache/pulsar/broker/service/AbstractTopic.java
87.03% <66.66%> (-1.70%)
:arrow_down:

...roker/loadbalance/impl/ModularLoadManagerImpl.java
81.60% <76.00%> (-0.20%)
:arrow_down:

...rg/apache/pulsar/broker/service/BrokerService.java
80.96% <78.57%> (-0.95%)
:arrow_down:

...va/org/apache/pulsar/broker/service/ServerCnx.java
71.28% <93.02%> (+0.14%)
:arrow_up:

...oker/service/nonpersistent/NonPersistentTopic.java
73.27% <100.00%> (-1.02%)
:arrow_down:

...sar/broker/service/persistent/PersistentTopic.java
79.39% <100.00%> (-0.16%)
:arrow_down:

... and 61 files with indirect coverage changes


Wrong microScript transpiler output of while statement in some cases

I've tested this online at https://microstudio.dev/ .
When the project uses "microScript v1 - transpiled" language, the following piece of code is transpiled incorrectly. The update statement is evaluated incorrectly which leads to an infinite loop execution. It manifests as a Timeout error.
  ox=0
  while (ox+=1)<=3
    y=0
  end

Thanks for the perfect minimal test case btw, it helped a lot! :-)


Breakdown test.py into multiple files

This would help readability of tests and would be valuable for future work.  We should use test.py as a loader to run other tests (including doctests in README.md).  @travs @slarson @mwatts15

Closed by #112


migrate changing_relation_type test

Adapter PR for Core PR: https://github.com/dbt-labs/dbt-core/pull/6054

Description

Checklist

[ ] I have read the contributing guide and understand what's expected of me
[ ] I have signed the CLA
[ ] I have run this code in development and it appears to resolve the stated issue
[ ] This PR includes tests, or tests are not required/relevant for this PR
[ ] I have opened an issue to add/update docs, or docs changes are not required/relevant for this PR
[ ] I have run changie new to create a changelog entry

might need to have a standalone bigquery test I think you have to do a full refresh run to go from a table to a view if i'm not mistaken

Added extra args option in dbt-core to support doing a full-refresh


[HOLD for payment 2023-05-26] [Navigation Reboot] Remove the X from modal screens

Tracking issue https://github.com/Expensify/App/issues/11768
Created a new branch navigation-refactor to which we will merge all the PRs of this refactor.
Remove X from the modal screens
Bookmark in the doc
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01453b9cb7d461b061
Upwork Job ID:<PHONE_NUMBER>517261312
Last Price Increase: 2023-03-12

Going to implement this

Job added to Upwork: https://www.upwork.com/jobs/~01453b9cb7d461b061

Triggered auto assignment to Contributor Plus for review of internal employee PR - @parasharrajat (Internal)

Unassigning CPlus for now, I will ask in the channel once the PR will be ready

the main pr will be merged soon so this will be worked on shortly

this one should be simple, lets knock it out of the way fast ⚡

Initially, I thought that we will create one PR for this issue and the UP issue but I think we can do it independently. I will take a look at that tomorrow

Yes I think lets just make this separate PRs for simplicity

@staszekscp can you comment on this one so I can assign you too

Triggered auto assignment to @anmurali (NewFeature), see https://stackoverflowteams.com/c/expensify/questions/14418#:~:text=BugZero process steps for feature requests for more details.

Nothiing to do here right now, PR is in a review but we will have to pay the C+ once we merge

@anmurali, @mountiny, @adamgrzybowski, @0xmiroslav Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

This is not overdue, pr is in a review

This Pr is implementing this too https://github.com/Expensify/App/pull/18402
We will hopefully merge that this week and do the final round of testing

PR done, flat review rate for @0xmiroslav at $1000 once the payment is due, thank you!

@anmurali once you are around there is just payment to be handled here and we can close this issue, thanks!

Actually I think Anu is still ooo so reassigning

hired @mountiny here - https://www.upwork.com/jobs/~01327e1b6e9a42d905
waiting for him to accept because the OG job link expired

paid.


It's stable, bump to 1.0.0

Since Fractal is stable, consider releasing v1.0.0 to express it's maturity.

I generally agree, but I think there are many issues & PRs that should be fixed / merged before, as the json api specs seem to not be fully supported.

I agree with @lukasoppermann, also there's some things which I'm going to merge and refactor before releasing 1.0.0, which will be in the very near future!

Hello guys ! I'm just wondering if v1.0.0 is going one soon ? I'm wondering because this issue was raised 2 years ago.


Refresh token support for OAuth/OIDC public clients

Hello Misagh,
Third and last part of my work on the improvement of the OAuth2.0/OIDC support for public clients in CAS.
This PR adds support for refresh token for this kind of clients. Here the changelog:

Add OAuth20RefreshToken Authenticator for OAuth/OIDC public clients
only.

Removed public clients support in the OAuth20ClientIDClientSecret
Authenticator (worked with client_secret="" which is not specified in OAuth2.0/OIDC).

Add tests for the new OAuth20RefreshToken Authenticator.

Add test for the existing OAuth20ClientIdClientSecret Authenticator.

With this last PR public clients will be fully supported (PKCE, refresh token & revocation).
Please let me know if you need any further information.
Regards,
Julien

Just added a few tests to improve coverage...

Excellent work. Thank you very much for the contribution. Keep 'em coming!

Thank you for the nice comment !
I still have a few fix for oauth/oidc in mind and maybe try this conformance test for oidc : https://openid.net/certification/testing/.
Could be nice to have CAS in the list of certified OIDC OP. :)
Regards,
Julien

That work is in progress; we have a test instance here: https://op.certification.openid.net:61314/ running tests against a CAS instance that is deployed on Heroku, and about 75% of tests do pass. Unfortunately, due to lack of funding and commitment, the work has paused.

Will take a look at it ! Thanks.


Refactor Reservation Scopes

Let's refactor the ReservationScopes module to use Concerns; this will hopefully help address #1097.

Ok, this is done and appears not to break anything. I'll check and see if that helped with the CodeClimate score and open a PR.

Damn that didn't help... one option I've seen is to refactor scopes into multiple files (see here), but that seems overly complex. I'll do a bit more research to see if there are any other ways we can fix the scope clutter...

I don't know if this helps us, necessarily, but this is a great article with some fairly advanced techniques for improving Models.

Yup, here's some more stuff that might be useful:
1
2

+1 to query objects! :D

If it would help, I'm happy to talk through the architecture of this :)

Ok, back on this issue. I wrote my first query object but am having trouble with loading it (since I put them in app/queries and as far as I can tell app/models loads first, so when it tries to find the query object it fails. I'll get back to this later.

Nope, it was just a typo in my class name :disappointed:

Alright, it looks like this improves our CodeClimate score for the reservation_scopes file, so I'm moving them all over :smile:

I've discovered that our upcoming checkin e-mail was potentially including reservations that had not been checked out (e.g. a 1-day reservation) and or reservations that had already been checked in... looks like we messed up our scope at some point. I'll fix that in this issue, increasing priority.

@esoterik: looks like we broke this in 766e9ab9 - any idea why we removed the not_returned scope? Seems like this might have potentially broken a bunch of stuff...

Seems like our tests are insufficient if we broke this... we need to add some edge cases to our mailer / rake task specs (or add specs for the query objects themselves)? I'm opening this up a separate issue and I'll move over the current scopes as they are.

Added some sweet inheritance for our Query Objects and some namespacing - CodeClimate says things are improving! I'll try to finish this up soon :smile:

:tada: :clap:

Thanks @caseywatts! I might recruit you for some code review to make sure I did this all properly once it's done :stuck_out_tongue:

deal :)

We removed not_returned because it was the same as active; I think that the current active does the same thing as the old not_returned? It gets all reservations that are either reserved or checked out.

Huh, but if you look at the pull request for #462 it doesn't look like we replaced it with active in a lot of places. Am I missing something?

The scope for reserved_on_date includes active, and reserved_in_date_range includes reserved -- those are the only places where we don't replace it with active.
The reserved_in_date_range calls might be the culprit?

To update here - the due_soon scope accidentally left out a checked_out status query; as that's a minor fix I'm just going to bundle it in here. I'm also likely going to rename the scope to due_today since that's actually what it does.

Also note that we'll be removing the delegate hackery and add #new calls to each of the Query Objects for clarity (so it's obvious we're passing an instance of the object, and not the class itself).

@caseywatts: let's shift discussion of this issue over here (re: your Slack comments). Based on what you wrote I'm going to move multipurpose scopes to the model file and then use Query Objects for single-use scopes. I'll also be writing tests for our scopes as well - where/how does one generally test Query Objects (since it seems somewhat coupled to the resource / model itself)? Do you just rely on the tests for the use case itself (kinda like an integration test)?

Started working on getting rid of the delegation stuff, might have broken some stuff. Also read a bunch about how people generally handle scopes, will keep working on cleaning this up this week.

More cleanup... still not 100% sure on how best to structure these scopes and where to add specs.

After starting to rearrange the scopes (and moving them back into the Reservation model for clarity), we're understandably seeing a decrease in the score for the Reservation model but benefiting from the removal of the separate scopes module. I'll still keep complex scopes in separate Query Objects but any scope based on a single parameter seems as though it belongs inside the model rather than creating a separate PORO for it.
Hoping to wrap this up today and making sure that we haven't broken anything :smile:.

Gotta run to a meeting, but it seems as though changing the future scope to a query object broke things since it's not chaining properly. I'll look more into this later (there's one other thing broken in the specs that I haven't had time to look into).

Fixed up the rubocop errors, now trying to debug the failing spec.

Weird, according to this post we should be getting chainable queries with Query Objects due to the way that Rails handles things. That doesn't seem to be the case, so I'll have to investigate further.

all specs pass, now it's just time to figure out why chaining broke...

Ok, I think the chaining issue is because we didn't end up originally keeping the delegate call in our Query Objects. My guess is because the scopes were defined as Reservations::ScopeQuery.new, and since we weren't passing in a relation it was defaulting to Reservation.all for all queries. The only way we can chain scopes is if we use the delegation method as suggested in this article, which seems like it ensures that each time we send call to the query object, it delegates it to a new instance of that query object.
... a few hours of reading later ...
Ok, after investigating further I think I figured this out. If we define the scope as follows:

class FooQuery
  def initialize(relation=default)
    # stuff
  end

  def call
    # more stuff
  end
end

...

scope :foo, FooQuery.new

then Rails instantiates a copy of the FooQuery class when initializing Rails, and since no relation is passed in as a parameter all calls of that scope use the default relation as the base. However, if we use delegation in the query, so that we have:
class FooQuery
  class << self
    delegate :call, to: :new
  end

  def initialize(relation=default)
    # stuff
  end

  def call
    # more stuff
  end
end

...

scope :foo, FooQuery

then we're instantiating a new copy of FooQuery each time we call the scope, and Rails is appropriately passing a chained relation as a parameter to FooQuery.new so everything works. If we wanted to use the first definition for Query Objects, we'd have to manually instantiate a new copy of the object, also manually passing the chained relation to new.
@caseywatts can you review this and make sure it makes sense / I haven't missed anything? In the meantime, I'm going to set this branch up with the latter formulation since that makes more sense to me at the moment. Thanks!


Create bonhoeffer.txt

Bonhoeffercollege Castricum
<EMAIL_ADDRESS>

@StephanMeijer Could you please rename lib/domains/nl/bonhoeffer to lib/domains/nl/bonhoeffer.txt to reflect this repository rules? Thank you!

@StephanMeijer You might miss the previous message to you. Could you please update the file as we asked you in the comment above?

Yes, I will, I am sorry. I did not see your message.

See #2120


[Affected functionality] Clear and concise summary, e.g. "[Create news] 'Publish' button is not displayed on the 'Preview' page"

Environment: Version 129.0.6668.71 (Official Build) (64-bit)
Reproducible: always
Build found: last commit from https://github.com/ita-social-projects/GreenCityClient/tree/dev
Preconditions

https://www.greencity.cx.ua/#/greenCity is opened.
'My habits' page is opened.
User clicked 'Add New habit' button on 'My habits' page.
'All habits' page is opened.
Any habit is chosen and 'More' button is clicked.
'Invite friends' button on Habit's description page is clicked.

Steps to reproduce

Enter to the text field 'Search friend'  a part of the correct name of a friend which contains special characters, e.g. "" (quotes).

Actual result
Search by special characters (e.g. quotes) in the name doesn`t produce result.
Expected result
According to the user story user can add 3 – 15 symbols ( letters, numbers, special characters) to the field.
User story and test case links
User story https://github.com/ita-social-projects/GreenCity/issues/4772
Test case https://1242.atlassian.net/projects/SCRUM?selectedItem=com.atlassian.plugins.atlassian-connect-plugin:com.kanoah.test-manager__main-project-page#!/v2/testCase/SCRUM-T95
Labels to be added
"Bug", Priority ("priority: medium"), Severity ("severity: medium"), Type ("Functional")

@KharytonovaYuliya while i`m searchilg with special characters ( - ; ' ) i get the result

in this issue allowed special characters are described
https://github.com/ita-social-projects/GreenCity/issues/4277

Works fine


Create NFT

Can I create NFT through this?

The API support submitting a serialised transaction to the local node which can contain minted tokens. What behaviour are you looking for here?

@rhyslbw we plan to create a website that enables users to mint their own tokens. we want to take advantage of this using graphql

Have you had a read of https://docs.cardano.org/en/latest/native-tokens/learn-about-native-tokens.html ? You can submit transactions with the API, but creating policies and minting is a client-side operation, or at least sits within the application scope.


Allow extra env from agent.env

Some are using corporate proxy, this allow to pass http_proxy and so on during container creation providing values:
agent:
  env:
    http_proxy: http://ip:port
    https_proxy: http://ip:port

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: bigbrozer sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Refactor & Finalize LiquibaseCommand API

Description
Currently, liquibase.Liquibase is the primary "porcelain" façade for working with Liquibase (https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Porcelain). However, as Liquibase's functionality has grown over the years, we have outgrown a single facade class.
A few years ago, we started the liquibase.command package as the replacement for the liquibase.Liquibase class and have been experimenting using it for a few commands. Now is the time to take what we have learned to finalize the design and apply it to all our existing commands.
Backwards Compatibility
The liquibase.Liquibase facade methods should be backwards compatible, even as their implementations are replaced with calls to the new LiquibaseCommand classes.
The calls to the existing LiquibaseCommand classes will not necessarily remain backwards.
Requirements / Changes
NOTE: This section will evolve as we do the actual implementation and see the actual API changes in action. Follow along with the PR for the full details.

LiquibaseCommands can be grouped by zero or more nested parent names
LiquibaseCommand objects are pluggable by command group + name
Method to execute a command will be moved from LiquibaseCommand to CommandFactory
LiquibaseCommand instances provide metadata of name, description, available parameters, required parameters, etc. which can be used to populate CLI etc. help.
LiquibaseCommand parameters are exposed as getter/setters on the command that can be set programatically
Unconfigured LiquibaseCommand parameters will be set with values the LiquibaseConfiguration system as part of the execute() function
LiquibaseCommands return an object containing output data, allowing commands to call others programmatically.

Questions
Replace or enhance?
Is the simple "replacement" strategy above what we want?
An extension can replace the standard "update" command with a different version that either enhances or replaces the functionality, but only one extension can do that. The extension can create instances of the UpdateCommand class and call it if they like, but they don't have to and how they do that is completely up to them.
An alternative would be to not replace the commands but to more register as a "chain" like ServletFllters or register pre/post hooks. That would allow an extension to not replace the standard UpdateCommand logic, but to surround it with additional logic.
┆Issue is synchronized with this Jira Bug by Unito
┆Fix Versions: Liquibase 4.4.0

Replace or Enhance discussion
There have been a couple requests for a hook-style/enhance approach vs. the replace one.
My main worry is that it adds more complexity than we need.
More complex:

Documentation etc. is more difficult, since it's not a simple "command X is class com.example.X, so help outputs its description of what it does"

Thoughts:

We need a way to compose commands, like updateSQL contains the logic of Update but more. That is a new command, not a replacement or extension.

Some requests for "I want the diff command to support diffing against alternate system Y" is really more of a "extend the snapshot plugin system" answer than a "enhance diff" answer.

If we did an all-in "extend" system, we'd maybe not even want to have a "real" command in the center? Just a series of hooks like validate, preRun, postRun, etc. and the CommandFactory.execute() collects up all the implementations for the command by name, and then sorts them and then calls the validate functions in order, then the preRun functions, then the postRun etc. ?

We'll want to have commands call each other, and so I'd been imagining things like "the update command will call register = new RegistrChangelogCommand(); register.setProjectName(...); ... type logic, which has some nice type-safety to it.
But, that also is hard-coding in the RegisterChangeLog implementation. Even if we replace that "new" with a CommandFactory.getCommand("registerChangeLog") we have to cast it to what we expect that implementation to be in order to do the setProjectName etc. things.

Maybe something like we're doing with the ConfigurationDefintions could work? Where we are using generics etc. to make them type-safe and easy to use but still more composable than you can do with regular classes?

I can see places where extending could be useful. Like validate maybe wants the ability to keep glomming on things to validate? Can do that with some sort of a separate validation subsystem too, though.

A hook-based system could make some cross-cutting command logic much nicer to implement. For example, hub related or logging or auditing logic could add itself to a set of commands

Summary
The cross-cutting opportunities and forcing the command callers to not expect specific implementations are maybe big enough reasons to consider the hook-based implementation. I'm going to take a stab at implementing that and see what the complexity ends up like in real life.

The PR is currently pointed to LB-1222 (#1691 ) because it requires/includes the changes in there. That is just a temporary thing to help with the PR review process.
The other ticket should be completed and merged to the default branch first, and then this PR will be updated to point to that branch as well.

We had an internal code review meeting to discuss the changes #1708 and we were happy with that direction.
The "hook"/pipeline style turned out to not add unneeded complexity, keeps the programmatic interactions cleaner/safer, and allows for some useful functionality such as cross-cutting logic.
I'll update the description of this ticket to not have questions on the pipeline vs. replace, and the next steps wrapping up of #1708 and then moving on to converting code to use it.


Searching a public site

Details

Project URL: https://readthedocs.org/projects/ugo-py-doc/
Build URL (if applicable):
Read the Docs username (if applicable): ugoproto

Expected Result
A description of what you wanted to happen
I would like my site to be searchable. It's the main reason why I created the docs. To be able to search and retrieve info because the webpages are indexed.
Actual Result
A description of what actually happened
Among the 3 projects I created
http://ugodoc.readthedocs.io/en/latest/
http://ugo-py-doc.readthedocs.io/en/latest/
http://ugo-r-doc.readthedocs.io/en/latest/
only the first one is searchable.  r and py have the same settings though? They don't return search results. They are 'public'.

These sites are using mkdocs. There are a few issues open around mkdocs search. Closing this as a duplicate.


Memory leak in SwiftUI

There is a memory leak in the SwiftUI library, that is unlike the one in UIKit.

I will see if removing the bidirectional coupling will solve this.

I will see if removing the bidirectional coupling will solve this.

Instruments still indicate memory leaks, but they're deep into the SwiftUI runtime.
Also, memory consumption no longer seems to go up when I switch keyboard or keyboard type.

Instruments still indicate memory leaks, but they're deep into the SwiftUI runtime.
Also, memory consumption no longer seems to go up when I switch keyboard or keyboard type.

This can be tested in v4.0.

This can be tested in v4.0.


Add a workaround to fix CAF OpenSSL options

In the actor_system_config, some settings are not like the others. While most modules read their options directly from the content object, the SSL configuration is stored in member variables of the main config class. The CAF command line and config file parsers handle this correctly, but settings from vast.yaml files or those that are passed in as environment variables are injected through a different code path.
This commit adds assignments to set the aforementioned class members explicitly.
I considered the alternative solution of translating non CAF-native option formats to native ones and let the parse function handle it, but that approach would be a large amount of work rather little benefit.

Should we ask upstream why the SSL options are treated this way rather than being read from the content settings object?

I asked about it in the chat.


Improve error message when rescript/bs-platform version is too old

When depending on<EMAIL_ADDRESS>

I'm not sure where "the directory null" comes from
Maybe we could look for bs-platform in node_modules if rescript can't be found and suggest updating?

I think this is being done already.
CC @fhammerschmidt

If you mean #529, that fixes the "null" in the error message but still only looks inside node_modules/rescript and does nothing with node_modules/bs-platform.
Now that I think of it, maybe we could also detect when npm install is not run yet and suggest doing that?

How about just rephrasing the message? Not sure If we really should check for bs-platform recursively just for the error message.
Something like:
Can't find ReScript binary in <working-directory>/node_modules/.bin or parent directories. Did you install it? It's required to use "rescript" > 9.1

Good idea 👍

@cristianoc fine for you?

Note bs-platform is not supported.
That's also specified in the release notes.
So the title of this seems confusing.

@cristianoc How about this?

Sounds good. The only thing to check is that the message is compatible with the search strategy (which does not look into a single directory).


feat(agent): add new agent derivation path

Checklist:

[x] My PR follows the contribution guidelines of this project
[x] My PR is free of third-party dependencies that don't comply with the Allowlist
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have checked the PR title to follow the conventional commit specification

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

105 of 138 (76.09%)  changed or added relevant lines in 13 files are covered.
4 unchanged lines in 3 files lost coverage.
Overall coverage increased (+0.3%) to 41.835%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

EdgeAgentSDK/Apollo/Sources/Model/Ed25519Key.swift
5
6
83.33%

EdgeAgentSDK/Apollo/Sources/Model/Secp256k1Key.swift
1
2
50.0%

EdgeAgentSDK/Apollo/Sources/Model/X25519Key.swift
5
6
83.33%

EdgeAgentSDK/Apollo/Sources/ApolloImpl+Public.swift
0
3
0.0%

EdgeAgentSDK/Apollo/Sources/Operations/CreateEd25519KeyPairOperation.swift
3
7
42.86%

EdgeAgentSDK/Apollo/Sources/ApolloImpl+KeyRestoration.swift
24
30
80.0%

EdgeAgentSDK/Domain/Sources/Models/KeyManagement/DerivableKey.swift
32
49
65.31%

Files with Coverage Reduction
New Missed Lines
%

EdgeAgentSDK/Apollo/Sources/ApolloImpl+Public.swift
1
68.83%

EdgeAgentSDK/Apollo/Sources/Operations/CreateEd25519KeyPairOperation.swift
1
48.0%

EdgeAgentSDK/Apollo/Sources/ApolloImpl+KeyRestoration.swift
2
52.5%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
4883

Relevant Lines:
11672

💛  - Coveralls


Be able to specify the fee aside the amount to be transferred in Xtokens

This is a usecase we encountered in Moonbeam, where we need to specify the fee (the amount to be provided in BuyExecution) as a separate argument to the amount being transferred, with the goal of better controlling the amount received in the destination side. We are in the process of adding this to Xtokens in our own fork, would you be interested on having such feature too? I could do a  PR with my changes if you are interested

Just FYI, I am making this changes as an additional functionality (by adding two new extrinsics), I am not changing any of the existing functionality in Xtokens

Happy to accept PR as long as it is transaction version compatible.

This is a first drafter, let me know if this is something you would like to add https://github.com/open-web3-stack/open-runtime-module-library/pull/658


New Explicit Resource Management docs and improvements

After the discussion with @n1ru4l ;
Some changes are done in @whatwg-node/server, so in order to configure the new feature (including the new hook);

Allow to configure server adapter from Yoga options such as disposeOnProcessTerminate
Allow to use the new hook
Document dispose method, onDispose hook, ctx.waitUntil and everything related to the things related to the resource management and graceful shutdown

what are the most common things people would use this for in Yoga?

https://github.com/dotansimha/graphql-yoga/pull/3567#issuecomment-2551370368
@Urigo For example in Hive Client Plugin, it creates a queue of background HTTP calls sending the reports to Hive Console. And in case of shutdown, we need to flush the queue and make sure all reports are sent.
This is very similar in OTEL, Sentry and Prom.

#3567 (comment)
@Urigo For example in Hive Client Plugin, it creates a queue of background HTTP calls sending the reports to Hive Console. And in case of shutdown, we need to flush the queue and make sure all reports are sent. This is very similar in OTEL, Sentry and Prom. Ideally they all should use ctx.waitUntil. In that case, CF Workers can handle it by themselves and in Node.js we handle it with .dispose method in our queue logic. But that usually doesn't happen when plugins use 3rd party SDKs like OTEL, Hive Client etc. But it is still ok when you use onDispose.

so its not actually for gateway developers but just for plugin developers?


[BUG] VOID type supported by Delta, but is not part of the Delta log protocol specification

Bug
Describe the problem
Running
CREATE TABLE default.void_table (y STRING, x VOID) USING DELTA

will create a table with a column of type VOID. However VOID is not among the primitive types listed in the protocol specification. Not all readers of Delta tables understands the VOID type, since it is not part of the specification, and will be unable to read such tables.
Steps to reproduce
CREATE TABLE default.void_table (y STRING, x VOID) USING DELTA

Observed results
Table is created.
Expected results
Table is not created, or VOID is among the primitive types of the protocol specification.
Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

[ ] Yes. I can contribute a fix for this bug independently.
[ ] Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
[ ] No. I cannot contribute a bug fix at this time.

What environment are you running this in?

Per Trino thread: https://github.com/trinodb/trino/issues/15193#issuecomment-1327914522 - this appears to be happening in Databricks environment.

@allisonport-db we may need to consider a way to fix existing tables.
If one creates a table containing void columns (they may not notice it, since printSchema won't return the column) and later tries to merge, even if the source column is correctly typed now, Integer for example, and autoMerge is set, they will receive an error like:
AnalysisException: The schema of your Delta table has changed in an incompatible way since your DataFrame or
DeltaTable object was created. Please redefine your DataFrame or DeltaTable object.


[5.8.0] [Analytics] AverageSessionDuration Aggregation is not working with Oracle

Describe the issue:
In Identity Analytics 5.8.0 pack, AverageSessionDuration stats is not populated.
Corresponding Siddhi query to get the average session duration is:
from SessionInformationTable select userStore as username, avg(duration)/1000 as duration GROUP BY username having (startTimestamp >= 0 and startTimestamp <= {{to}}L ) and ((endTimestamp >= {{from}}L and endTimestamp <= {{now}}L) or isActive == true ) order by duration desc

And the generated SQL is given below:
SELECT SessionInformationTable.userStore AS username, (avg(SessionInformationTable.duration ) / 1000 ) AS duration FROM SessionInformationTable WHERE ((((SessionInformationTable.startTimestamp
>= 0 ) AND (SessionInformationTable.startTimestamp <=<PHONE_NUMBER>542 ))AND (((SessionInformationTable.endTimestamp >=<PHONE_NUMBER>541 )AND (SessionInformationTable.endTimestamp <=<PHONE_NUMBER>204 ))OR (SessionInformationTable.isActive = 1 )))AND (SessionInformationTable.meta_tenantId = -1234 )) GROUP BY SessionInformationTable.username ORDER BY SessionInformationTable.duration DESC

When executing this script, it throws the following exception:
ORA-00979: not a GROUP BY expression

Updating the Siddhi query in IsAnalyticsAverageSessionDuration widget as follow fixes the issue
from SessionInformationTable
select userStore as username, avg(duration)/1000 as duration
GROUP BY userStore , duration, meta_tenantId,sessionId, startTime, terminateTime, endTime, isActive, username, userstoreDomain, remoteIp, region, tenantDomain, serviceProvider, identityProviders, rememberMeFlag, userAgent, currentTime, startTimestamp, renewTimestamp, terminationTimestamp, endTimestamp, timestamp
having (startTimestamp >= 0 and startTimestamp <= {{to}}L ) and ((endTimestamp >= {{from}}L and endTimestamp <= {{now}}L) or isActive == true )
order by duration desc

Environment information

Product Version: Identity Analytics 5.8.0
Database: Oracle 12c , 19c

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Please use rubocop --debug when pasting rubocop output as it contains additional information.
$ cat f.rb
class MoneyController < ApplicationController
  def index
    render :json => (ExchangeRate.supported_rates.map do |k, v|
      [k, v.select { |k, v| [:iso_code, :name, :symbol, :html_entity].include?(k) }.to_h]
    end.to_h.camelize)
  end
end
$ ./rubocop -a f.rb
f.rb:1:1: C: Style/Documentation: Missing top-level documentation comment for class MoneyController.
class MoneyController < ApplicationController
^^^^^^^^^^^^^^^^^^^^^^^^^^
f.rb:1:1: C: [Correctable] Style/FrozenStringLiteralComment: Missing frozen string literal comment.
class MoneyController < ApplicationController
^
f.rb:3:18: C: [Corrected] Style/RedundantParentheses: Don't use parentheses around a method call.
    render json: (ExchangeRate.supported_rates.map do |k, v| ...
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
f.rb:4:23: W: Lint/ShadowingOuterLocalVariable: Shadowing outer local variable - k.
      [k, v.select { |k, _v| %i[iso_code name symbol html_entity].include?(k) }.to_h]
                      ^
f.rb:4:30: C: Performance/CollectionLiteralInLoop: Avoid immutable Array literals in loops. It is better to extract it into a local variable or a constant.
      [k, v.select { |k, _v| %i[iso_code name symbol html_entity].include?(k) }.to_h]
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 1/1 file |======================= 100 =======================>| Time: 00:00:01

1 file inspected, 5 offenses detected, 1 offense corrected, 1 more offense can be corrected with `rubocop -A`
$ cat f.rb
class MoneyController < ApplicationController
  def index
    render json: ExchangeRate.supported_rates.map do |k, v|
      [k, v.select { |k, _v| %i[iso_code name symbol html_entity].include?(k) }.to_h]
    end.to_h.camelize
  end
end

i.e. evaluation order becomes
(render json: ExchangeRate.supported_rates.map do |k, v|
  [k, v.select { |k, _v| %i[iso_code name symbol html_entity].include?(k) }.to_h]
end).to_h

And ofc it is hard to "coerce" a String to_h
     # --- Caused by: ---
     # NoMethodError:
     #   undefined method `to_h' for #<String:0xbe6ff68>
     #   Did you mean?  to_c
     #                  to_b
     #                  to_s
     #                  to_r
     #                  to_d
     #                  to_f
     #                  to_i
     #   ./f.rb:5:in `index'

Steps to reproduce the problem
This is extremely important! Providing us with a reliable way to reproduce
a problem will expedite its solution.
RuboCop version
Include the output of rubocop -V or bundle exec rubocop -V if using Bundler.
If you see extension cop versions (e.g. rubocop-performance, rubocop-rspec, and others)
output by rubocop -V, include them as well. Here's an example:
$ [bundle exec] rubocop -V
1.67.0 (using Parser <IP_ADDRESS>, rubocop-ast 1.32.3, analyzing as Ruby 3.3, running on ruby 3.3.5) [x86_64-linux]
  - rubocop-performance 1.22.1
  - rubocop-rspec 3.1.0

Related to https://github.com/rubocop/rubocop/issues/13299; precedence for complex expressions with multiline blocks is weird.


LDAP without e-mail leads to ArrayIndexOutOfBoundsException

Hi,
If you configure UAA to delegate authentication to an LDAP server and that server does not return an e-mail field, then the ExternalLoginAuthenticationManager  class throws an ArrayIndexOutOfBoundsException exception.
The problem occurs in the following piece of code:
https://github.com/cloudfoundry/uaa/blob/master/common/src/main/java/org/cloudfoundry/identity/uaa/authentication/manager/ExternalLoginAuthenticationManager.java#L209
I believe it was introduced with this change (problem did not occur on CF 222): https://github.com/cloudfoundry/uaa/commit/7adc0b61dce6c284c46cc1453fbc46e9dd61a46b
The workaround for now is to configure the uaa.ldap.mailSubstitute  and uaa.ldap.mailSubstituteOverridesLdap properties explicitly.
P.S. It's important to note that login happens with user names that are not e-mails.

Upon inspection we concluded that, when familyName and givenName are not mapped, there was little to no actual value in creating values from the email so we simply removed this code.  We believe this should resolve your issue as well.


[main] Upgrade to latest dependencies

Cron -knative-prow-robot
/cc knative-sandbox/eventing-writers
/assign knative-sandbox/eventing-writers
Produced by: knative-sandbox/knobots/actions/update-deps
Details:
bumping knative.dev/serving 951c48c...dbd3fc3:
  > dbd3fc3 Update net-certmanager nightly (# 13218)
  > 15ee48d Update net-contour nightly (# 13220)
  > 0474e35 Update net-istio nightly (# 13219)
bumping knative.dev/pkg d02dcd0...f78a006:
  > f78a006 upgrade to latest dependencies (# 2573)
bumping knative.dev/eventing ab981d6...cb3901f:
  > cb3901f Increase max namespace count for e2e test runner (# 6486)
  > 1d00055 upgrade to latest dependencies (# 6485)

/lgtm
/approve


Revert "New SIL instructions to support tail-allocated arrays in SIL."

Reverts apple/swift#4722
Checking if it broke LLDB tests: https://ci.swift.org/job/oss-lldb-incremental-linux-ubuntu-14_04/5353/

@swift-ci Please smoke test

LLDB tests are currently failing on Linux on the master branch, but passed after this revert.  Merging.


WebFinger should ignore stored users with status != None

Still seeing several accounts on bluesky who follows ap.brid.gy, and gets followed by the account, but no mirror gets created on the fediverse. Latest a fellow Swede on bluesky bassesbasta.bsky.social who have no bridged account and doesn't really find a reason why. Anything they can do to try to make it work?

Hmm! Have they checked everything on https://fed.brid.gy/docs#troubleshooting ?

If so, the next thing they can try is unfollow and re-follow ap.brid.gy.

Thanks! Yes they tried to re-follow but still no effect. Is it still that "hide account from logged out view" makes the bridge not working? If so, that might be the case.

Interesting. That account used to have a different DID, https://web.plc.directory/did/did:plc:nxgbz3zl3dfad7ghyvwroiwi , which no longer has an ATProto repo. They switched to a new one, https://web.plc.directory/did/did:plc:onlmd532kfzrtqb3w2r4mfre , which is serving fine, and working ok in BF, https://fed.brid.gy/bsky/did:plc:onlmd532kfzrtqb3w2r4mfre , but our WebFinger is still trying to use the old one. Hopefully an easy fix. Thanks for reporting!

Oh yeah that's right! He deleted his old account a while back, but created a new account recently, with the same username since it was available again.

^ Fix should be deployed in ~10m.


move sdk/rust and sdk/go out of spin repo

The rust and go sdks were some of the first SDKs written for spin and have lived in the Spin repo since they were first written. There isn't much of a reason for them to live in the spin repo while the rest of the SDKs currently available live outside of the Spin repo. Let's move these SDKs under the fermyon/ org into their respective repos with the following steps (preserving as much history as possible along the way):

[ ] Move WIT files to a separate github repo and re-add to the Spin repo as a git subtree (reasoning below)
[ ] Move sdk/rust into spin-rust-sdk repo. Add wit files to new repo using git subtree.
[ ] Update all references to the rust sdk in the Spin repo to point to the new repo
[ ] Move sdk/go into spin-go-sdk. Add wit files to new repo using git subtree.
[ ] Update all references to the go sdk in the Spin repo to point to the new repo

The rust and go sdks, several crates in Spin, some examples, and some Makefiles all rely on the wit files being available in the repository (for the wit-bindgen tool to use to generate bindings). If we move the current wit directory into a separate repo and add into the Spin repo as a git subtree, it would make moving the rust and go sdks to the spin-rust-sdk and spin-go-sdk repos much easier. We wouldn't have to keep a separate copy of the wit files.
As part of this issue, I created a new repo, spin-wit, which houses the WIT files used across the spin project. This will allow us to add the wit files in a directory in any github repo with the git subtree add command. See example below:
git remote add witfiles git@github.com:fermyon/spin-wit.git
git subtree add —-prefix=wit/ witfiles main

This is a good primer on git subtrees: https://medium.com/@v/git-subtrees-a-tutorial-6ff568381844
As part of this issue, we'll also need to build tooling or process around how to update WIT files. If WIT files are updated in a commit on the Spin repo, we'll want to use CI to also add those changes to the spin-wit repo or we can require that any changes to WIT files be made in the spin-wit repo and then be pulled into the Spin repo using the git subtree pull command. See example below:
git subtree pull —prefix=wit witfiles main

Would it be worth doing a SIP first so we can get a sense for how all this is going to work?  I'd like to understand how this is going to affect versioning and developer experience and all that (if the answer is 'not at all' then great but the multipage tutorial on git subtrees was a bit alarming grin).  Thanks!

Absolutely I can put together a SIP for this work.

SIP provided here: https://github.com/fermyon/spin/pull/1067

This is done :shipit:


Update accounts daemon with revised procedure.

ssh-keys in instance metadata will store all SSH keys.
block-project-ssh-keys will prevent use of SSH keys in project metadata.

LGTM


Update naming of bloXroute

Update naming of bloXroute

The first column is the Operator. We might want to rename that column to Operator/Relay Name or something like that if we want to include some relay name.

Done

@remyroy - do you think it is relevant context for ethstaker community to know if relay operators are known to be operating in multiple ecosystems (i.e. on ethereum-native rollups, sidechains, other L1 ecosystems etc)? just struck me as somewhat good to know. For example bloXroute I believe has been active across Solana and I think some other EVM chains (not sure if they have their services available on rollups yet)

@remyroy - do you think it is relevant context for ethstaker community to know if relay operators are known to be operating in multiple ecosystems (i.e. on ethereum-native rollups, sidechains, other L1 ecosystems etc)? just struck me as somewhat good to know. For example bloXroute I believe has been active across Solana and I think some other EVM chains (not sure if they have their services available on rollups yet)

I don't think it's relevant. We already have a link to the operator's website and users can do their own search from there.

cool thanks
From: Rémy Roy @.>
Sent: Thursday, January 12, 2023 9:33 AM
To: eth-educators/ethstaker-guides @.>
Cc: Eric Siu @.>; Comment @.>
Subject: Re: [eth-educators/ethstaker-guides] Update naming of bloXroute (PR #65)
@remyroyhttps://github.com/remyroy - do you think it is relevant context for ethstaker community to know if relay operators are known to be operating in multiple ecosystems (i.e. on ethereum-native rollups, sidechains, other L1 ecosystems etc)? just struck me as somewhat good to know. For example bloXroute I believe has been active across Solana and I think some other EVM chains (not sure if they have their services available on rollups yet)
I don't think it's relevant. We already have a link to the operator's website and users can do their own search from there.

Reply to this email directly, view it on GitHubhttps://github.com/eth-educators/ethstaker-guides/pull/65#issuecomment-1380449409, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWTTSXMGXM5DCQ2TYWBOZADWSAIYLANCNFSM6AAAAAATFZLCEA.
You are receiving this because you commented.Message ID<EMAIL_ADDRESS>

Seems good. Thanks for the PR.


Fix Remove Duplicates README

Currently the we are using the same function example for the two versions.
DeltaHelpers.removeDuplicateRecords(
  deltaTable = deltaTable,
  primaryKey = "id",
  duplicateColumns = Seq("firstname","lastname")
)

@brayanjuls - let's pair soon and take another pass at the README.  It's one of the most important ways for us to grow this project.  We need to make sure to have an absolutely beautiful README to attract new users.

This was fixed.


Why AL0603 for assigning integers to an enum?

Bug or feature?
Why is warning AL0603 raised when directly assigning an integer to MyEnum, and why not when assigning it indirectly in foreach?
procedure TestAL0603()
var
    MyEnum: Enum MyEnum;
    Integers: List of [Integer];
begin
    // both raise warning AL0603:
    MyEnum := 1;
    MyEnum := Integers.Get(1);

    // both raise no warning
    foreach MyEnum in MyEnum.Ordinals() do ...;
    foreach MyEnum in Integers do ...;
end;

Don't get me wrong - I prefer it without the warning. So much cleaner ... But then I see no reason why we should get AL0603 at direct assignment.
Versions:

AL Language: 13.0

We've looked at this on and off again, and the problem is that the rule has value in direct assignment - but also if we were to enforce it in foreach it will end up with the overhead mentioned above. So I'm in favour of just closing this as "yes, we know about this and we'll keep it inconsistent for now" to avoid sparking a larger debate if we were to make it consistent.


Sema: Clean up handling of protocol operators with concrete operands

More peeling off refactorings from https://github.com/apple/swift/pull/28698.
In this case we would "devirtualize" the protocol requirement call
by building the AST to model a direct reference to the witness.
Previously this was done by recursively calling typeCheckExpression(),
but the only thing this did was recover the correct substitutions
for the call.
Instead, we can just build the right SubstitutionMap directly.
Unfortunately, while we serialize enough information in the AST
to devirtualize calls at the SIL level, we do not for AST Exprs.
This is because SIL devirtualization builds a reference to the
witness thunk signature, which is an intermediate step between
the protocol requirement and the witness. I get around this by
deriving the substitutions from walking in parallel over the
interface type of the witness, together with the inferred type
of the call expression.

@DougGregor @xedin This eliminates the second-to-last typeCheckExpression() call from inside CSApply. The last one is in the handling of EditorPlaceholderExpr.

@swift-ci Please smoke test

@swift-ci Please test source compatibility

@swift-ci Please smoke test macOS

@swift-ci Please test source compatibility

@swift-ci Please test source compatibility Release

@swift-ci Please benchmark

@swift-ci Please test source compatibility Release

Performance: -O
Code size: -O
Performance: -Osize

Regression
OLD
NEW
DELTA
RATIO

FlattenListLoop
2654
3136
+18.2%
0.85x (?)

Code size: -Osize
Performance: -Onone
Code size: -swiftlibs

How to read the data
The tables contain differences in performance which are larger than 8% and
differences in code size which are larger than 1%.
If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.
Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 64 GB


Add options object to iop.open call

Some image processors need to know the type of the image in order to open them as a buffer. This is the case with lwip.
To make this possible, we could pass the filename (or just the extension) to smartcrop as part of the options object, and then make it available to iop.open as a second argument. That way, the iop can decide how best to open the image when open() is called.
(Does this make sense? 😅 )

For reference, here's a first version of the adapter module for lwip. At the moment I'm hardcoding "jpg" as the image type, but this PR would make it possible to always use the correct type.

Hello Eduardo,
Thanks for your PR. What you are proposing sounds good. I suggest two changes to your PR:

Pass options.input to iop.open rather than all of options. That will make things a bit more contained
Add a test for it

Sure, that makes sense.
I pushed another commit where I'm passing options_ instead of options to iop.open (I'm assuming this is what you meant with options.input?)
I don't see any tests involving the iop adapters, so not sure how you want to approach this. Happy to put together a test if you point me in the direction you want to go.
Thanks!

I actually meant pass options.input with pass options.input. See the commit.
I really don't like having another way for passing in images.
Just handle this use case in your iop.open by checking if it already is and image and then returning that.
I implemented this feature in 1376a54b3262188020b44359a5ff4b78b262281e.Passing the whole options object is IMO a violation of separation of concern. Can you let me know if this solves your use case?

Understood.
I believe that commit does what I need. Thank you.
Closing.


Add documentation on how to leverage the helm charts

Right now, the helm charts live off in their own space, but have no reference on our docs page. Add a section under User Guides, similar to docker and kubernetes, on how to get started using helm.

https://github.com/deps-cloud/deploy
https://deps.cloud/docs/deployment/helm/


Fix a noise shape of StyleMelGANGenerator to export ONNX model

Hi! Thank you for creating this awesome project. In this PR, I suggest modifying StyleMelGANGenerator to export the ONNX model.
Problem details
I bumped the following issue when I tried to export StyleMelGANGenerator.
/path/to/venv/lib/python3.8/site-packages/parallel_wavegan/models/style_melgan.py:222: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  math.ceil(c.size(2) / self.noise_upsample_factor),

This warning shows that math.ceil() operation cannot be traced by torch.onnx.export() function. So that the shape of noise will be treated as constant values.

After applying this patch, we can export ONNX model.

Thank you for your modification! It looks fine.
I slightly modified to pass the CI.


Better multiprocessing dataloader support for large datasets

problem statement
I have a large corpus for training, and ideally I don't want to load them all at once due to limited CPU memory. Currently, the DatasetStream API serves this purpose by returning one Dataset at a time.
However, the DatasetStream.__iter__ API limits the data pipeline performance if the data preprocessing is complicated and computational heavy. For instance, for BERT training, the raw dataset is split into multiple small chunks. and for each of these chunks, we

read file in memory, perform tokenization, vocabulary lookup
group sentences into documents according to document boundaries
for each sentence in a document, generated random next sentence pairs and masks for language modeling
prepare labels for next sentence prediction and masked language model
Then we have a dataset for samples suitable for BERT training.

Note that all the steps above happens before a dataset is formed (i.e. transform() cannot be used here). Performing all these steps with a single python process is slow.
Now, the DatasetStream.__iter__ API returns one dataset at a time, which is a problem is used in combination with PrefetcherIter(worker_type='process'), as the process-based prefetcher does not allow creation of child processes.
walk-arounds

Use PrefetcherIter(worker_type='thread'). It causes the training loop to slowdown because of the time sharing between the dataset creation thread and the thread for training the model.
Use PrefetcherIter(worker_type='process_no_daemon'). It launches a process without daemon mode, where children processes can be created. This solves the speed issue, but at the cost of letting users maintain all these processes. In another word, users have call process.join() before exiting the program, otherwise the program hangs.

better solutions?
I think GluonNLP needs to provide a data loader for a SimpleDatasetStream that supports multiprocessing for dataset creation. I picked SimpleDatasetStream because it has a complete view of what data url to load, where DatasetStream does not provide that visibility.
For implementation, the dataloading would happen in hierarchy using two worker pools. The first worker pool is responsible for loading and forming the dataset. The formed dataset is then passed to the second worker pool, where workers generate data batches that user wants.
@leezu @szha

Also adding data loader expert @zhreshold

@eric-haibin-lin The best workaround IMO is to ditch the PrefetchIter, instead, let SimpleDatasetStream be self complete with option to prefetch dataset.
In this case, SimpleDatasetStream will have a daemon IO process (mapped to dataset list queue?, data queue, label queue, transform function queue) which do all the pre-processing tasks(reading, splitting...).
It will also have a process pool with each worker mapped to queues(data queue, label queue, transform function queue, output queue). Once IO process pushes splitted data/label/transform function to theses queues, they get process by all the workers in process pool. Main thread will grab the results from output queue.
Does it make sense? I know I am not fully aware of the whole pipeline, please correctly me if I am wrong.

So if I understand correctly the issue is that @eric-haibin-lin would like to spawn processes within the environment executing DatasetStream.__iter__.
Now this normally would be the standard python main process and there wouldn't be any issue; however, PrefetcherIter allows wrapping DatasetStream.__iter__ into a separate (daemonic) process. Daemonic processes cannot spawn further processes in Python.
@zhreshold I'm not sure if extending SimpleDatasetStream with an option to prefetch dataset would solve the issue: It doesn't matter if SimpleDatasetStream or PrefetchIter is responsible for creating the separate (daemonic) process within which DatasetStream.__iter__ is executed. In either case it is executed inside a daemonic process and cannot spawn further processes. Please let me know if I misunderstand your suggestion.
To avoid switching from daemonic to non-daemonic processes (for which the user must manually take care that they are correctly terminated on program termination), we may expose the already existing process pool to the user? (I'm not yet sure if this would lead to other issues) Alternatively we may provide an API where users can register their intent to use a process-pool and gluon-nlp may create the pool within the main process and pass the handle to the daemonic subprocess running DatasetStream.__iter__. Finally there may be an easy way to work around any issues caused by switching to non-daemonic processes.

My point is that daemon process can't have child process to accelerate tasks, however, it can have sibling processes which they will communicate by queues.
The bad news is, it's not safe and elegent to expose queues in different classes, that's why I suggest to DatasetStream should be capable of prefetching in itself and all resources are encapsulated.
Users will only touch DatasetStream.__iter__ then.

Thanks for the clarification @zhreshold. In that case your proposed MultiProcessingReadyDatasetStream would expose an interface that would allow users to submit tasks to sibling processes during the execution of DatasetStream.__iter__?
I'd also like to add an orthogonal perspective: Besides using multiprocessing inside the DatasetStream.__iter__ we may parallelise multiple calls to DatasetStream.__iter__. Currently we can only prefetch a single element from the DatasetStream in a single worker process (or thread). If multiple elements can be prefetched in parallel (possibly in different processes) we may achieve the same improved user-experience (lower latency).
This is currently not possible as DatasetStream is a Stream / python iterator and we can only access the next upcoming element. To retrieve multiple elements, eg. for distributing them to multiple processes for further processing, we have to call next() multiple times. But as DatasetStream.__iter__ returns Dataset objects, each call to next() is expensive and in @eric-haibin-lin's case too slow.
Instead of directly constructing a Dataset on each call to next(), it may be sensible to work with Futures. Since Python 2 is in the process of being deprecated (though no concrete plan is made for GluonNLP yet), we may consider building on https://docs.python.org/3/library/concurrent.futures.html or https://docs.python.org/3/library/asyncio-task.html when addressing this problem. This may allow using existing standard library code and keeping our API simple. Thus, a call to next() would create a task and submit it to a pool of processes, but would not block calling next() again to begin computation for the following element.

@leezu thanks for the suggestion on Futures. Do we still have to change the DatasetStream interface if it is used?


Custom Logger

I think we should make it compatible with the custom logger modules created in the applications. In this way, job status can be monitored in autonomous state. I can write directly, if that's okay.

After reviewing you PRs and spending some time thinking about the root of initiative, I think we should implement a generic solution where we're not forcing zerolog or uber/zap but allow an interface for BYOL (bring your own logger)


Superset URL - connection refused

Hi, first time poster so please forgive if I miss out anything obvious that should be included.
We have a two main instances of Superset installed, used to visualise business data. Version 1.0.1, in Ubuntu using venv. Python version is 3.8. They are port forwarded through our firewall and DNS is attached from an external provider. Each instance is held on a separate server, and located in separate buildings.
We have a third version, setup the same as above but used only for testing and training and this is also on a separate server in a separate building.
Also, we run an older version of Superset from previous installations, which is version 0.36.
Yesterday (June 30, 2021), all instances of Version 1.0.1 were working as expected and we were able to use each separate URL to view and edit charts and dashboards.
Today however (July 1), each separate URL of the V1.0.1 instance is inaccessible via external browsers and also the internal IP addresses. The older V0.36 version still works.
The browser error is pointing to a network issue but we have ruled out internal network issues as these V1.0.1 instances are on 3 separate sites with 3 separate networks. We have also isolated a Superset version on a home network and found the same issues between older versions and newer versions.
Could there be an issue with any external library files that Superset connects to?
We have tried upgrading one of the v1.0.1 instances to v1.1.0 but this action has had no effect.
Screenshots below of Chrome and Firefox.
Thanks in advance for any advice or help you can give.

This seems specific to your installation of superset or networking set up

This seems specific to your installation of superset or networking set up

Hi @nytai , you think so, even though nothing in our setup or network has changed overnight? All fine one day, broken the next.

just an update for anybody that views this.
This wasn't a network issue at all, as we kind of new from the beginning.
We have run some upgrades today and discovered a new version of Superset has been released, v1.2.0 which fixes this problem. The issue itself is better explained by another user who posted today, here: https://github.com/apache/superset/issues/15524
Release announcement here: https://github.com/apache/superset/releases


Sink connector stops sending alerts to slack & discord, Need a way to troubleshoot

The connector reads from a topic with the following data:
{"forks":137,"stars":1763}
{"forks":137,"stars":1763}
{"forks":137,"stars":1763}
{"forks":138,"stars":1763}
{"forks":138,"stars":1763}
{"forks":138,"stars":1764}
{"forks":138,"stars":1764}
{"forks":138,"stars":1764}

I am using the following sink connector:
meta:
  version: 0.1.0
  name: discord-stars-sink
  type: http-sink
  topic: github-stars
http:
  endpoint: "https://discord.com/api/webhooks/<key>"
  headers:
    - "Content-Type: application/json"
transforms:
  - uses<EMAIL_ADDRESS>  - uses<EMAIL_ADDRESS>    with:
      spec:
        - operation: shift
          spec:
            "result": "content"

The connector stopped sending stars to slack.
I have a 2nd connector sending stars to discord, and this is failing too.
I need a way to troubleshoot this issue, and prevent it from reoccurring.
Note: This is an issue in my production environment, and it needs immediate attention.

Logging should help.


Deflate/gunzip

It would be nice to see support for deflating/gunzipping requests. Have you thought about this?

Looks like http://nodeca.github.io/pako/ does exactly this. Perhaps it would be worth figuring out if the two can work together so as to keep as much as possible outside of axios core.

@jimthedev thanks for the tip. I'll take a look at it.

@jimthedev @OliverJAsh I think I would recommend keeping this functionality out of axios. I want to keep the bloat down as much as possible from the core. Minified pako is 44k vs axios minified is 15k. Should be able to easily create an interceptor to inflate/deflate requests as needed.

Sounds good.
On Sun, Jan 25, 2015 at 3:21 PM, Matt Zabriskie<EMAIL_ADDRESS>wrote:

@jimthedev https://github.com/jimthedev @OliverJAsh
https://github.com/OliverJAsh I think I would recommend keeping this
functionality out of axios. I want to keep the bloat down as much as
possible from the core. Minified pako is 44k vs axios minified is 15k.
Should be able to easily create an interceptor to inflate/deflate requests
as needed.
—
Reply to this email directly or view it on GitHub
https://github.com/mzabriskie/axios/issues/23#issuecomment-71394224.


Hotfix: bolt button's incorrect display in IE11

Jira
http://vjira2:8080/browse/BDS-1525
http://vjira2:8080/browse/BDS-1529
Summary
Fixes an issue with the bolt-button's display renders incorrectly in IE11.
Details
There's an issue with the bolt-button's CSS where the :host selector was used, and the CSS was written incorrect using scss extend. This removes that extend and rewrite the CSS manually. When writing :host selector CSS, it cannot be combined with other selectors.
How to test
Run the branch locally and check the following:

[x] 1. Given I am viewing the product landing page in IE11.
[x] 2. When I navigate near the bottom of the page, I should see a section with a blockquote, that comes right after the section with cards.
[x] 3. Then I look at the call to action button below the blockquote. This button should have auto width and is centered aligned.

@remydenton @danielamorse I added a test for aligning inline buttons.