Performance of download

I am not sure if I am doing something wrong but the download speed is way slower than it could be. I am trying to fetch a 1GiB file from my local vm as a testcase. The vm is just a freshly setup debian 12 with only openssh installed.
This is what I am doing:
use async_trait::async_trait;
use log::{info, LevelFilter};
use russh::*;
use russh_keys::*;
use russh_sftp::client::SftpSession;
use std::sync::Arc;
use tokio::io::{BufReader, BufWriter};

struct Client;

#[async_trait]
impl client::Handler for Client {
    type Error = anyhow::Error;

    async fn check_server_key(
        &mut self,
        server_public_key: &key::PublicKey,
    ) -> Result<bool, Self::Error> {
        info!("check_server_key: {:?}", server_public_key);
        Ok(true)
    }
}

#[tokio::main]
async fn main() {
    env_logger::builder().filter_level(LevelFilter::Warn).init();

    let file_size = 1024 * 1024 * 1024;

    let config = russh::client::Config {
        limits: Limits::default(),
        window_size: file_size as u32,
        ..Default::default()
    };
    let mut session = russh::client::connect(Arc::new(config), ("<IP_ADDRESS>", 22), Client)
        .await
        .unwrap();

    if session.authenticate_password("mop", "mop").await.unwrap() {
        let channel = session.channel_open_session().await.unwrap();
        channel.request_subsystem(false, "sftp").await.unwrap();
        let sftp = SftpSession::new(channel.into_stream()).await.unwrap();

        // interaction with i/o
        let start = std::time::Instant::now();
        let filename = "test";
        let file = sftp.open(filename).await.unwrap();

        let mut buf = Vec::with_capacity(file_size);
        let mut reader = BufReader::with_capacity(file_size, file);
        // let mut writer = BufWriter::with_capacity(1024 * 1024 * 100, buf);
        // let mut reader = BufReader::new(file);
        let mut writer = BufWriter::new(&mut buf);

        tokio::io::copy_buf(&mut reader, &mut writer).await.unwrap();

        println!("read in {:?}", start.elapsed());
    }
}

After tweaking the values a bit I get roughly 60-70 MBps:
[mop@udoo-juergens russh-sftp-client]$ target/release/russh-sftp-client
read in 15.670531058s

without doing anything I get only 10 MBps :face_with_spiral_eyes: :
read in<PHONE_NUMBER>68s

A quick test with the sftp command line:
[mop@udoo-juergens russh-sftp-client]$ sftp<EMAIL_ADDRESS>Connected to <IP_ADDRESS>.
sftp> get test
Fetching /home/mop/test to test
test                                                                                100% 1024MB 270.1MB/s   00:03
sftp>

Is this the expected performance? Or did I do something wrong? :thinking:
I also did a quick cross check using a go library and this one is very close to what the sftp command line is doing:
package main

import (
	"bufio"
	"fmt"
	"log"
	"os"
	"time"

	"github.com/pkg/sftp"
	"golang.org/x/crypto/ssh"
)

func main() {
	start := time.Now()
	// var hostKey ssh.PublicKey
	config := &ssh.ClientConfig{
		User: "mop",
		Auth: []ssh.AuthMethod{
			ssh.Password("mop"),
		},
		HostKeyCallback: ssh.InsecureIgnoreHostKey(),
	}
	sclient, err := ssh.Dial("tcp", "<IP_ADDRESS>:22", config)
	if err != nil {
		log.Fatal("Failed to dial: ", err)
	}
	defer sclient.Close()

	// open an SFTP session over an existing ssh connection.
	client, err := sftp.NewClient(sclient)
	if err != nil {
		log.Fatal(err)
	}
	defer client.Close()

	f, err := client.Open("test")
	if err != nil {
		log.Fatal(err)
	}

	localFile, err := os.Create("test")
	if err != nil {
		log.Fatal(err)
	}
	defer localFile.Close()

	writer := bufio.NewWriter(localFile)

	f.WriteTo(writer)
	f.Close()

	writer.Flush()
	fmt.Printf("File downloaded %v\n", time.Since(start))
}

Your example runs in about 40-50 seconds for me, which is still pretty slow. I noticed that after sending the packet, it takes anywhere from 2-4ms to get the first response data, but sometimes it spikes up to 42ms, which seems suspiciously slow. I'll dive into more debugging as soon as I get some more time.

I digged a bit deeper and can reproduce that it sometime takes longer than 40ms to read().
The 40ms are happening whenever it renegotiates the window. By setting the window_size to 0xfffffff I was able to come around that. Still: I am stuck at 70MB/s whereas it should be 200MB+


Deduplicate dependency management entries in POMs

Fix a regression where previously we would check that we did not add duplicate entries to the dependency-management section of the POM. We still properly de-duplicate regular dependencies.
Fixes: https://github.com/gradle/gradle/issues/26996
Before merging the PR, comments starting with

❌ ❓must be fixed
🤔 💅 should be fixed
💭 may be fixed
🎉 celebrate happy things

I've triggered the following builds for you. Click here to see all build failures.

PullRequestFeedback build

@bot-gradle test rfr

I've triggered the following builds for you. Click here to see all build failures.

ReadyForRelease build

@bot-gradle test and merge

I've triggered a build for you. Click here to see all failures if there's any.


Treat _intake_wheel_speed as class variable

Also in general you shouldn't use __init__ for components. Use setup


WORK IN PROGRESS: AVRO 2404: Fix insecure urls.

I don't think that we should replace each http to https, especially not those which are in license header. We should only use secure URLs for downloading artifacts no?

@nandorKollar We can do it that way.
Yet what I see is that in more and more ways security is becoming and issue.
If we generate a documentation website and people click on a non-secure link it is still a way of injecting malicious code.
So that I why I chose to go 'all the way'.
I see either way as 'valid' yet I prefer to get rid of everything 'non-safe' as much as possible.

@nielsbasjes I see, though I'm not strongly agains replacing everything to secure URL, my concern in this case is that it can cause unwanted merge conflicts (especially during backporting). Does Rat check pass with the changed license header too?
I'm not sure how did you replace the protocol to https, but maybe not every URL has a corresponding secure version, pointing to an invalid url after changing (in case you used script for this, then probably this is not a valid concern :) ). In addition, it looks like there are some URLs which still use http, but looks like have a secure endpoint too, for example http://www.exolab.org

The rat check passes because I added the config to allow the https link.
I also submitted a patch for Rat to support these out of the box: https://issues.apache.org/jira/browse/RAT-212  (which is allowed according to legal)

@nielsbasjes any update on this? Great work so far, but I think we should get this in ASAP.

Cleaned up the changes.

Fixed the issue id in the commit messages (I made a mistake)
Skipped the white space changes (shouldn't be in here).
The https license URL has been checked with legal and has been committed in Apache Rat as "valid" (will be in rat 0.14) https://issues.apache.org/jira/browse/RAT-212

I'm manually checking the documentation links.
Reasons for me to leave urls unchanged:

It is something like a ns or dtd name in an XML
It is a test url in a unit test (linking to localhost)
Sites does not respond on https or give a certficate error.
Domain is completely down (or even for sale) like  grepcode.com, codehaus.org,

To quickly find a usable list of remaining urls I used this bash command:
find . -type f | fgrep -v .gem | xargs fgrep http:/ | egrep -v '(http://localhost|http://<IP_ADDRESS>|xmlns|xsi|\.dtd|test|NOTICE|LICENSE|INSTALL|README|See http)'

The only remaining this at this point is the same for both py and py3:
nbasjes@ubuntu-vm:~/workspace/Apache/avro$ fgrep -B5 -A5 http:// ./lang/py3/avro/txipc.py ./lang/py/src/avro/txipc.py
./lang/py3/avro/txipc.py-
./lang/py3/avro/txipc.py-class TwistedHTTPTransceiver(object):
./lang/py3/avro/txipc.py-  """This transceiver uses the Agent class present in Twisted.web >= 9.0
./lang/py3/avro/txipc.py-     for issuing requests to the remote endpoint."""
./lang/py3/avro/txipc.py-  def __init__(self, host, port, remote_name=None, reactor=None):
./lang/py3/avro/txipc.py:    self.url = "http://%s:%d/" % (host, port)
./lang/py3/avro/txipc.py-
./lang/py3/avro/txipc.py-    if remote_name is None:
./lang/py3/avro/txipc.py-      # There's no easy way to get this peer's remote address
./lang/py3/avro/txipc.py-      # in Twisted so I use a random UUID to identify ourselves
./lang/py3/avro/txipc.py-      import uuid
--
./lang/py/src/avro/txipc.py-
./lang/py/src/avro/txipc.py-class TwistedHTTPTransceiver(object):
./lang/py/src/avro/txipc.py-  """This transceiver uses the Agent class present in Twisted.web >= 9.0
./lang/py/src/avro/txipc.py-     for issuing requests to the remote endpoint."""
./lang/py/src/avro/txipc.py-  def __init__(self, host, port, remote_name=None, reactor=None):
./lang/py/src/avro/txipc.py:    self.url = "http://%s:%d/" % (host, port)
./lang/py/src/avro/txipc.py-
./lang/py/src/avro/txipc.py-    if remote_name is None:
./lang/py/src/avro/txipc.py-      # There's no easy way to get this peer's remote address
./lang/py/src/avro/txipc.py-      # in Twisted so I use a random UUID to identify ourselves
./lang/py/src/avro/txipc.py-      import uuid

My python knowledge is to limited to know if this needs to be fixed or not.
At least with this set of changes the entire build uses secured transport and we are now able to quickly find all insecure usecases more quickly.
Please review

Thanks Niels for taking care of this! 👏

@Fokko : This has been comitted on master ONLY.
Should this be on other branches too? Perhaps the 1.9 branch?

Yes, we should cherry-pick -x this onto the 1.9 branch 👍

I'm checking the cherry picked branch-1.9 edition against CI by simply creating a new pull request: https://github.com/apache/avro/pull/564


Kafka plugin, multiple consumer issue, UI/spanId related.

Prerequisites

[x]  I have checked the FAQ, and issues and found no answer.

What version of pinpoint are you using?
2.0.4
Describe the bug
When one app send a kafka message triggered by a http request, multiple apps consume the same message and process the message, then these apps make another http request to a http service. The whole thing could be described as the following server map (two outbounds).

issue 1:
The main server map could not show correctly(the consumer node could be omitted or linked) when one outbound is chosen as the screenshot below.

issue 2:
The app consumer-verify1, consumer-verify2 and consumer-verify3 each called verify-http-server-dev. They all should be linked to the verify-http-server-dev node and the call counts should be 1 for each.

The server map for this transaction should look like the following:

The request headers:
verify1 -> http-server
GET /api/verify1?seq=msg_1599102848086 HTTP/1.0
Host: verify.example.com
Connection: close
User-Agent: Apache-HttpClient/4.3.6 (java 1.5)
Accept-Encoding: gzip,deflate
Pinpoint-TraceID: pub-loc1^1599102239212^2
Pinpoint-SpanID:<PHONE_NUMBER>85031189
Pinpoint-pSpanID:<PHONE_NUMBER>584694206
Pinpoint-Flags: 0
Pinpoint-pAppName: consumer-verify1
Pinpoint-pAppType: 1210
Pinpoint-Host: verify.example.com

verify2 -> http-server
GET /api/verify2?seq=msg_1599102848086 HTTP/1.0
Host: verify.example.com
Connection: close
User-Agent: Apache-HttpClient/4.3.6 (java 1.5)
Accept-Encoding: gzip,deflate
Pinpoint-TraceID: pub-loc1^1599102239212^2
Pinpoint-SpanID:<PHONE_NUMBER>945348215
Pinpoint-pSpanID:<PHONE_NUMBER>584694206
Pinpoint-Flags: 0
Pinpoint-pAppName: consumer-verify2
Pinpoint-pAppType: 1210
Pinpoint-Host: verify.example.com

verify3 -> http-server
GET /api/verify3?seq=msg_1599102848086 HTTP/1.0
Host: verify.example.com
Connection: close
User-Agent: Apache-HttpClient/4.3.6 (java 1.5)
Accept-Encoding: gzip,deflate
Pinpoint-TraceID: pub-loc1^1599102239212^2
Pinpoint-SpanID: -1471803627387986967
Pinpoint-pSpanID:<PHONE_NUMBER>584694206
Pinpoint-Flags: 0
Pinpoint-pAppName: consumer-verify3
Pinpoint-pAppType: 1210
Pinpoint-Host: verify.example.com

They all had the same Pinpoint-pSpanID but got different Pinpoint-pAppName. The Pinpoint-pSpanIDs should be different.

The main server map got this issue when there is only one consumer.

The web side got these logs.

2020-09-03 13:29:40 WARN  one to N replaced. node:verify-pub(STAND_ALONE:1000)->host:kafka.example.com:9092 accept:[AcceptApplication{host='kafka.example.com:9092', application=consumer-verify1(SPRING_BOOT:1210)}, AcceptApplication{host='kafka.example.com:9092', application:9092=kafka.example.com(KAFKA_CLIENT:8660)}]
2020-09-03 13:29:40 ERROR targetLinkData not found findLinkKey:LinkKey{fromApplication='verify-pub', fromServiceType=STAND_ALONE, toApplication='consumer-verify1', toServiceType=SPRING_BOOT, hash=870137464}

The config for the kafka plugin:

profiler.kafka.producer.enable=true
profiler.kafka.consumer.enable=true
profiler.springkafka.consumer.enable=true
profiler.kafka.consumer.entryPoint=

hi @yjqg6666
In my guessing, this is probably due to that Queue is a virtual node.
When draws Servermap,  it is expected that the application after the queue will be included, because Queue is a virtual node, it is not included as depth,
In this case, since the line is limited by the depth, it is assumed that it is not drawn together and only the nodes that has no connection are expressed .
This is my guess, and the exact problem seems to need analysis.
thanks :)

@koo-taejin thanks for the quick reply. How about the second one? The pSpanID may came from the kafka header and used for three different applications. I think it's a hard-to-solve issue. How about replace with new spanIds on the web side?

@yjqg6666
The queue type is a little different from the original design, and there are a little problem in processing the queue type.
I will also consider it.
Thanks for your reporting

@koo-taejin Should i keep this issue open or close it.

@yjqg6666
Please keep this issue with open state
thanks :)

I just got a workaround.  In the kafka plugin when it consume a record, I tried to save the Pinpoint-SpanID in a ConsumerRecord header as an annotation and generate a new span id for the consumer. In the web when construct a call tree and if the service type of a span is queue(i only check if it's kafka), i restore the span id from the previous-set annotation value. It's working in the call tree and servermap.  It's not a good way but hopefully it may have some help.
Hopefully there would be a better official solution for the queue type(one-to-many).

@yjqg6666
Thank you so much for giving me a great idea.
If it is possible, could you send me your work to PR?
Let me test it and see if there are any something that I have to care about.
thanks :)

@koo-taejin I just sent a PR.

In the PR,  the web side contains some specific code for Kafka-only, it's not a good solution. There should be a better /general solution for the queues.

@yjqg6666
I let you know progression of this issue.
I am looking to solve problem 1.
The cause seems to be the wrong EndPoint.
Kafka client acts as a server, but it is a client.
Besides, Kafka client kept everything in isolation, making it difficult to obtain this information.
So it has difficult to get the EndPoint.
There seems to be a problem with trying to handle it well without EndPoint, because it was too difficult to get the correct EndPoint with Stably.
I have been trying to dig in Kafka structure to solve it.
I am thinking of getting EndPoint at Kafka Selector.
So, If I find out good solution, then I will commit it.
thanks :)

@yjqg6666
I expected to have solved issue 1 via #7283
Could you check whether this issue has solved or not?
thanks :)

@koo-taejin I am building the master now and will check and update soon.

@koo-taejin Checked and have confirmed that the first issue is fixed. Thumb up!
The topic-publish app:

The topic-consume app:

The topic-publish app(inbound-1 and outbound-2):

@yjqg6666
I am glad to hear that.
I try to be thinking of solve issue 2 from now.
If I have any progression, then I am going to share with you.
thanks :)

@yjqg6666
Can you check the issue 2 using the master branch?
As a result of analysis, it is expected that the information of callerHost has been changed due to the changed endPoint, so it will come out normally.
If the results come out, please share
thanks :)

@yjqg6666
Can you check the issue 2 using the master branch?
As a result of analysis, it is expected that the information of callerHost has been changed due to the changed endPoint, so it will come out normally.
If the results come out, please share
thanks :)

checking.

@koo-taejin not working.
It should be:

Now still get this one using master:

@yjqg6666
Does the above appear in the main server map instead of the server map in CallStack?
I thought the Main server map would have been solved because it draws the server map base on caller and callee tables.

@yjqg6666
It is judged that it works well if I make reproducing situation.
Is there any doubt that it is different from the part that I made?

servermap in main

servermap in callstack

@koo-taejin It's fixed in the main server map. Now the issue only exist in the server map for a single transaction and only if the consumer apps both/all then all the same other app.
In you case tj-testweb2 and tj-testweb3 both call tj-testweb4.

This is the main server map:

In your case tj-testweb2 do not call tj-testweb4 and can not see the issue. If called the call num between tj-testweb2/tj-testweb3 and tj-testweb4 should be 2 instead of 1.

@yjqg6666
I have understood that what you mean.
Thanks for your detailed explanation.

@yjqg6666
I let you know progression of this issue 2.
I have found out reason of this issue and the way I solve this issue.
I am working on finding, if this solution has another problem.
If I am sure that it has not any problem, then I am going to commit it.
thanks :)

@yjqg6666
I expected to have solved issue 2 via #7290
Could you check whether this issue has solved or not?
thanks :)

I expected to have solved issue 2 via #7290
Could you check whether this issue has solved or not?

checking.

@koo-taejin Now it works like a charm.

@yjqg6666
I am glad to hear that.
Please close this issue, If you do not have any problem.
thanks :)


Subscriptions are getting updated when saving without selecting any tiers for APIs in CREATED state.

Description:
Allows to save without selecting any subscription plan for APIs in CREATED state. But it automatically sets the previously selected one.
Steps to reproduce:

Create a API
In Subscription uncheck all the selected ones.
Save

Affected Product Version:
3.0.0
Environment details (with versions):

OS:
Client:
Env (Docker/K8s):

Optional Fields
Related Issues:

Suggested Labels:

Suggested Assignees:

Fixed with https://github.com/wso2/carbon-apimgt/pull/8247


check async functions like normal functions

Fixes #74

They do generate errors, but those are disabled by default. I think that's because pytype historically used this syntax, and when I first wrote the check we still had some legacy examples. But we can probably turn those checks on by default now, or just fold them into Y010.
Thanks for the merge!


Enforcing Empty Lines Around Class Structures

This probably isn't an issue, but I've been utterly unable to accomplish this. From: https://github.com/rubocop-hq/ruby-style-guide#consistent-classes - you see the order we should generally structure our classes. The thing I am trying to enforce is the use of empty lines around each grouping. I feel like I can't be the only person who wants this, but have been completely unable to locate the correct cop, and my google-fu has been too weak to find a solution. Could someone please point me in the right direction?
Thank you!

Hi, @allentsai!
We have a Layout/ClassStructure cop that tries to deal with the ordering of concepts within your class, although it is very much a work in progress, and so isn't enabled by default. You can enable it in your configuration:
Layout/ClassStructure:
  Enabled: true

I also don't think it deals with whitespace, so I will mark this as a feature request for now.=


No module named 'optim_orders'

Hi, I'm having a bit of a problem trying to reproduce your code. The module optim_orders has not been found, it would be very helpful if you could provide instructions on getting the missing files.Thanks again for making your code public.

Thanks for pointing it out. We have fixed it. Try it now~

Hi~ We have updated the environment configuration and conducted code testing with the 4090 and A100 GPUs.
Please refer to the readme for step-by-step guidance on reproducing our setup and running the code successfully.


[core] Fix configuration (simpler)

val app = Javalin.create()

app.configure {
    it.addSinglePageRoot("/", "/public/html.html")
    it.addSinglePageRoot("/", "src/test/resources/public/html.html", Location.EXTERNAL)
    it.addStaticFiles("/public")
    it.addStaticFiles("src/test/resources/public", Location.EXTERNAL)
    it.contextPath = "/"
    it.defaultContentType = "text/plain"
    it.dynamicGzip = true
    it.enableWebjars()
    it.enableCorsForOrigins("*", "my-origin")
    it.prefer405over404 = false
    it.autogenerateEtags = true
    it.requestCacheSize = 8192L
    it.requestLogger { ctx, executionTimeMs ->  }
    it.accessManager { handler, ctx, permittedRoles ->  }
    it.sessionHandler { SessionHandler() }
    it.wsContextPath = "/"
    it.wsFactoryConfig {  }
    it.wsLogger {  }
    it.server {
        Server()
    }
}

@ernestas2k tried a different approach here. Fixed a few of the issues so that it's more of a pure config file now.

Great work! I like it a lot, config is now static holder and cors situation is handled explicitly in configure(cfg) {} which shows there's something dynamic going when applying configuration.


Add a 'disable' option to the event-type jeditable field, fix width of organiser input field

Assigning to @Coenego for review

Coverage remained the same at 100.0% when pulling d314ca8d629caa3959a6f4e78aaa9a54a5d8932d on bp323:disableSelect into ca0598dc60661531ca553774281f52b9673794fc on CUL-DigitalServices:master.


Decimal separator are not unified

All Wallets
. is used as the separator

Wallet Details
, is used as the separator

Transactions
, is used as the separator

Expected
All have the same Decimal separator

Duplicate of https://github.com/threefoldtech/threefold_wallet/issues/145


flash does not work when CPU is in sleep mode

In general, the st-flash utility has been working fairly well for me. Thank you for your excellent work, guys. I believe the issues I've had, except for the one I am creating this ticket for, have open tickets already.
The problem I'm having is that I'm planing on creating an interrupt driven design. So I have written an experimental program that sets up an interrupt (connected to the user button) and then puts the CPU to sleep by running the __WFI() function. When I flash a program that runs that function, subsequent flashing with st-flash does not work. To get back in touch with the chip, I have to boot into Windows and use the STM-32 ST-LINK Utility from ST to erase the flash. A clue to what is wrong is that the STM-32 ST-LINK Utility does not work either, unless "Connect under Reset" is enabled in Settings.
After flashing the program that runs the __WFI() function, any attempt to flash causes the following error:
dahl@ubuntu:~/Desktop/3/stlink/example/32l_lcd$ make write
../../flash/flash write lcd.bin 0x08000000
2012-02-09T23:25:54 INFO src/stlink-common.c: Loading device parameters....
2012-02-09T23:25:54 WARN src/stlink-common.c: unknown chip id! 0
stlink_sram_flash() == -1
make: *** [write] Error 255
Any attempt to erase causes (note how the program displays a different chip id):
make: *** [write] Error 255
dahl@ubuntu:~/Desktop/3/stlink/example/32l_lcd$ ../../flash/flash erase
2012-02-09T23:28:01 INFO src/stlink-common.c: Loading device parameters....
2012-02-09T23:28:01 WARN src/stlink-common.c: unknown chip id! 0xe0042000
Mass erasing
If I hold the Reset button while flashing:
ahl@ubuntu:~/Desktop/3/stlink/example/32l_lcd$ ../../flash/flash erase
2012-02-09T23:28:28 INFO src/stlink-common.c: Loading device parameters....
2012-02-09T23:28:28 INFO src/stlink-common.c: Device connected is: L1 Med-density device, id 0x10386416
2012-02-09T23:28:28 INFO src/stlink-common.c: SRAM size: 0x4000 bytes (16 KiB), Flash: 0 bytes (0 KiB) in pages of 256 bytes
Even after I erase the flash in Windows, I get the following. This is fixed by pushing the Reset button on the card:
dahl@ubuntu:~/Desktop/3/stlink/example/32l_lcd$ make write
../../flash/flash write lcd.bin 0x08000000
2012-02-10T14:13:21 INFO src/stlink-common.c: Loading device parameters....
2012-02-10T14:13:21 INFO src/stlink-common.c: Device connected is: L1 Med-density device, id 0x10386416
2012-02-10T14:13:21 INFO src/stlink-common.c: SRAM size: 0x4000 bytes (16 KiB), Flash: 0x20000 bytes (128 KiB) in pages of 256 bytes
2012-02-10T14:13:21 INFO src/stlink-common.c: Attempting to write 7332 (0x1ca4) bytes to stm32 address: 134217728 (0x8000000)
2012-02-10T14:13:21 WARN src/stlink-common.c: pecr.pelock not clear (0x7)
2012-02-10T14:13:21 WARN src/stlink-common.c: Failed to erase_flash_page(0x8000000) == -1
stlink_fwrite_flash() == -1
make: *** [write] Error 255

What version do you use? The name recently changes to st-flash. Try with recent git.

Pulled down a new copy today and the issue has been fixed. Thank you!

@MartinNowak: Does this work now or are we still facing a bug? Please retry with Release v1.6.0.

When the STM32 is in STOP2 mode, we need to assert the reset line with the ST-LINK in order to connect to it. ST-LINK Utility on windows calls this 'connect under reset'. The st-flash utility does call stlink_jtag_reset(), but I've found that this doesn't work because its either happening at the wrong time, or the second parameter is incorrect (or both). I did some testing with wireshark to see what the correct parameters are to assert and deassert the reset line and they are 0 and 1 as you would expect. st-flash is using 2 which is maybe toggle? In my fork, I implemented a fix that works by adding a connect_under_reset parameter to stlink_open_usb(...) and asserting the reset line before calling stlink_enter_swd_mode(), which matches what I saw ST-LINK Utility doing with the connect under reset option. I can work on getting a cleaner version of that into a PR.

Also refer to #36.

As it appears nobody seems to work actively on this topic.
Are there any volunteers with specific know how who can contribute?

There have been several approaches to implement a "connect-under-reset-feature" in the codebase.
Also various reset issues related to this topic still appear frequently.
Against this background a clean-up along with a proper implementation and testing is necessary.
The most recent approach was implemented by @geoffreymbrown.
This issue is related to: #220, #254, #261, #382, #532, #533, #692
I have opened a separate branch connect-under-reset which shall focus on this somehow essential refactoring.
Volunteers with advanced skills are strongly encouraged to step in if they are able to contribute.


UnicodeEncodeError: 'ascii' codec can't encode character '\u2019' in position 15: ordinal not in range(128)

Traceback (most recent call last):
File "/opt/anaconda3/envs/Hyper-Table-Recognition/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/anaconda3/envs/Hyper-Table-Recognition/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/Users/chouroukhelaoui/PycharmProjects/image-table-ocr/table_ocr/demo/main.py", line 51, in
csv_output = main(sys.argv[1])
File "/Users/chouroukhelaoui/PycharmProjects/image-table-ocr/table_ocr/demo/main.py", line 34, in main
for cell in cells
File "/Users/chouroukhelaoui/PycharmProjects/image-table-ocr/table_ocr/demo/main.py", line 34, in
for cell in cells
File "/Users/chouroukhelaoui/PycharmProjects/image-table-ocr/table_ocr/ocr_image/init.py", line 33, in main
txt_file.write(txt)
UnicodeEncodeError: 'ascii' codec can't encode character '\u2019' in position 15: ordinal not in range(128)
in some cases, we get this issue it can't be fixed by adding this line of code in  "/image-table-ocr/table_ocr/ocr_image /init.py"  line 32 :
txt = txt.encode('ascii', 'ignore').decode('ascii')

你好！邮件已收到，谢谢！

I think this might be because open uses your locale's encoding when you don't specify one as a kwarg in the open call and your locale's encoding is ASCII.
https://docs.python.org/3/library/functions.html#open
https://docs.python.org/3/library/locale.html#locale.getencoding
If I'm right, I don't think this requires a code change since it's adjustable at the environment-level.


mean shape landmark visualized in viewer #2 should the deformed shape's

When a 3D model and corresponding set of landmarks are selected to visualize 3D deformations, and the toggle mean shape landmarks is enabled, the mean landmarks shown in viewer #2 (deformed model) should the deformed mean shape one. Otherwise they just interfere with the visualization and make it harder.


store/chrome,firefox: Support List() (via NSS)

Issue: https://github.com/adamdecaf/cert-manage/issues/10

I'm going to merge this and fix tests on another PR.


Why does this plugin default to "Display Sprites" false?

Seems weird, that's the whole point of the plugin.
cool plugin though, I wished for something like this before but assumed it didn't exist.

Yea that's a good point 🙂, ill definitely change that in an update, along with a couple other things I'd like to change such as the way you clear sprites and making the scaling of mini-map sprites clearer


I didn't received deposit

I take withdrawal kucoin and successful complete but i can't receive deposit my bitget wallet and issue is kcc network not support bitget exchange and i speak support help people they told kcc network not support bitget and not help and kucoin also same told not help ( please help me if not support bitget exchange then please back my amount transfer kucoin

TXID 0xda5f83d819a1a97457ab6efcac6cd2933b8e33494afa27a1e6b592190dd6a610
This all secrenshot
Please help me now its 8 day gone i can't receiving my deposit help me please

Same thing happened to me but thanks to Blackhathacker111 who helped me recover all my funds,message him on telegram his telegram username is  https://t.me/Blackhathacker111


Host upstream packages.

For the long term, we should make sure we have full contol over pythia dependencies.
This mean, keeping copies of things like libffi on bin.chevah.com and have chevahbs script download them from bin.chevah.com
Originally posted by @adiroiban in https://github.com/chevah/pythia/pull/32#discussion_r857625420

An interesting thing I've noticed today… While trying to generate a Linux arm64 package for the 3b1a8ba revision of Pythia currently used in our server repo, the build failed because of a checksum error for zlib version 1.2.11.
Turns out that particular version was pulled out from the upstream server because of vulnerabilities, and curl was downloading an HTML file instead.
If it were hosted on our side, we wouldn't have known. I say +1 for still downloading packages from upstream, they know best when something should definitely not be used any longer.

Yes... the problem is for legacy version, we might need even old versions
In terms of library versions, we should have separate dedicated checks like we have for python.
The thing is that we are in business and customers are paying our salary to make sure we are able to help them with all their needs, including legacy systems.
We have many customers using SFTPPlus over separate leased private fibers and  with everything over VPN... so no public access.
I think this that it's very important to make sure we have everything on our servers


Last row in a TableComponent is hidden below the bottom of the table's container

When using a TableComponent with a lot of rows, the last row gets hidden below the container. I can scroll slightly past the bottom of the container, and inspecting the html shows a row hidden below that. Seems like this may be an issue where the column header height isn't being subtracted from the height of the table grid.
Here's a sample image where the 30th row is hidden.

CC @MattIrv @anthonydresser

@corivera It seems you can workaround the issue by calling loading=false before table.updateProperties(...). meanwhile I will investigate the root cause and work on the fix.

Closing older engineering-focused issues.  Please activate a bug for the specific user-facing feature that is impacted by this behavior.  Thanks!


include or exclude files feature

getfilenamesoftype

https://github.com/microsoft/vscode/wiki/Adopting-Multi-Root-Workspace-APIs
this helps with file filtering


performance optimisation when stage has been disposed of

I realised that there were some functions still being called recursively at every frame even after removing the scene.
In hope of getting rid of as much as I could, I was removing everything by doing:
stage.removeAllComponents();
stage.dispose();
stage.viewer.container.innerHTML = "";
stage = null;

This might not be too much of a problem in some pages, but if you happen to add and remove stages a lot (popup/modal with ngl viewer, or single-page applications) then this can build up.

After adding and removing a stage 20 times (using a modified version of /examples/embedded.html) This is what I would get on the performance tab of Chrome.
Every single frame I would have 20 calls to mouseObserver._listen() and 20 to viewer.animate() that would amount to about 5ms of useless use of the main thread, and memory usage growing (even if it does get caught by the garbage collector eventually).
Compare to this screenshot using the code in this PR

Here with have absolutely nothing happening after doing the same steps, which is way better regarding resource usage.

OK, so I had installed this LGTM analysis thing on my forked repo and force-pushed to retrigger a check but I don't think it's something that needs to be done from my side. Let me know if I have to do something in order to pass the check

This looks basically OK, though I'm not expert on this code, but perhaps you should protect against accidentally calling window.cancelAnimationFrame with an undefined value, and set the request back to undefined once the request is canceled. (And type frameRequest to include undefined.)

but perhaps you should protect against accidentally calling window.cancelAnimationFrame

This is not strictly necessary. cancelAnimationFrame does nothing when called with undefined or any invalid value.

Reviewed and tested after a rebase. This works as advertised. Thanks @aurel-l  for your contribution!

Reviewed and tested after a rebase. This works as advertised. Thanks @aurel-l  for your contribution!


Revert "pre-deploy: remove unnecessary kernel upgrade"

Reverts blueboxgroup/ursula#1574
This was the last change to pass CI.  We're getting a lot of these errors now:
TASK: [wait for instance to boot] *********************************************
Monday 15 February 2016  17:55:20 +0000 (0:00:18.050)       0:01:54.992 *******
skipping: [<IP_ADDRESS>] => (item={u'skipped': True, u'changed': False})
failed: [<IP_ADDRESS>] => (item={'invocation': {'module_name': u'os_floating_ip', 'module_args': ''}, 'item': {'OS-EXT-IPS-MAC:mac_addr': 'fa:16:3e:b8:9e:84', 'version': 4, 'addr': '<IP_ADDRESS>', 'OS-EXT-IPS:type': 'fixed'}, 'changed': True, 'floating_ip': {'status': 'DOWN', 'fixed_ip_address': '<IP_ADDRESS>', 'floating_ip_address': '<IP_ADDRESS>', 'network': '503e6577-3831-4e71-9812-7d080e9c104c', 'attached': True, 'id': '133409e0-5e65-4012-8e21-24a06fc45308'}}) => {"elapsed": 180, "failed": true, "item": {"changed": true, "floating_ip": {"attached": true, "fixed_ip_address": "<IP_ADDRESS>", "floating_ip_address": "<IP_ADDRESS>", "id": "133409e0-5e65-4012-8e21-24a06fc45308", "network": "503e6577-3831-4e71-9812-7d080e9c104c", "status": "DOWN"}, "invocation": {"module_args": "", "module_name": "os_floating_ip"}, "item": {"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:b8:9e:84", "OS-EXT-IPS:type": "fixed", "addr": "<IP_ADDRESS>", "version": 4}}}
msg: Timeout when waiting for <IP_ADDRESS>:22

FATAL: all hosts have already failed -- aborting

Looking to see if reverting this change fixes this particular issue.  It may be that we need to keep the "restart" handler and just discard the upgrade bit?

This doesn't seem to have been our problem. Closing.


Update to v2.10.8

Checklist

[x] Used a fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge/core I need some help with this update... Is  there any place I can go with questions (the google group?). The current status of this build:

The windows builds fail because of 1) syntax errors for py27 2) unsatisfiable dependencies for py36 and py37, which is weird because everything should be available.
The Mac builds fail because of a segfault during the building of the python bindings, but that can be expected because I think the paths passed to configure.py may be erroneous; If I build the package locally on my mac using the same script as build.sh in a manually created conda env, then there's no problem and the build script completes just fine.

Is this still correct for MacOS, pathwise?
CONDA_BIN=$PREFIX/bin
QT_MAJOR_VER=`qmake -v | sed -n 's/.*Qt version \([0-9])*\).*/\1/p'`
BUILD_SPEC=macx-clang
...
$PYTHON configure.py --pyqt=PyQt${QT_MAJOR_VER} --sip=${CONDA_BIN}/sip --qsci-incdir=${PREFIX}/include/qt --qsci-libdir=${PREFIX}/lib --spec=${BUILD_SPEC} --no-qsci-api

Still waiting for the Linux builds, although they first quit on the build number being < 1000, while the checklist of this PR states the build number should be reset to 0 for new versions ;)

You can always go to the gitter with questions: https://gitter.im/conda-forge/conda-forge.github.io
The number should not be reset to 0, the 1000 is there to allow the gcc7 packages to take precedence when we merge the labels.
I'm not certain why those are unavailable, they are up on anaconda.org.
https://anaconda.org/conda-forge/pyqt/files
https://anaconda.org/conda-forge/sip/files

@conda-forge/core this package need to built against the Qt versions on pkgs/main or anaconda channels. Qt and sip are present with higher versions there (5.9.7 and 4.19) compared to conda-forge (5.6.2 and 4.18). QScintilla 2.10 requires a sip version higher than 4.18, thus can't be built with the packages present on conda-forge.
How can I proceed? Is there any chance that the Qt packages (en related) on conda-forge will also be updated to newer versions in the very near future?

@conda-forge/core. Any updates on the timeline with which newer PyQt versions will become available on conda-forge? I'm curious to why it has been stuck at 5.6 for such a long time, while the main repo has 5.9 already. I'd really love to update this package.

Would encourage you to engage with the pyqt maintainers directly and see if there is anyways to move that effort forward.

@jakirkham thanks for the reply. I've checked the qt and pyqt repositories, and namely the first shows that a lot of work is being done to get qt to 5.12, but that it's very challenging. It's just a matter of patience then, so I'll keep in eye on the progress they make and will work on this after they release the newer qt versions.

@jakirkham I did read something about setting a strict flag, which would allow conda-forge builds to fetch libraries from anaconda, but I guess that won't do anything here, since the qt packages are also available on conda-forge and thus will have precedence. Or will pinning a version that is not available on conda-forge allow the qt packages to be fetched from anaconda? I just want to check if the build succeeds or needs some more work.

Hi! This is the friendly automated conda-forge-linting service.
I was trying to look for recipes to lint for you, but it appears we have a merge conflict.
Please try to merge or rebase with the base branch to resolve this conflict.
Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

@conda-forge/core @ocefpaf I'm trying to upgrade this package, and appear to be succeeding for all platforms except python 2.7 on Windows. There are unsatisfiable dependencies there:
conda.exceptions.UnsatisfiableError: The following specifications were found to be in conflict:
  - qt=5.9 -> vc[version='>=14.1,<15.0a0']
  - vc=9

I do not explicitly state to use vc9, so I guess conda build does this implicitly. Could you help me out with this? I also remember reading the pyqt maintainers to have trouble building their package for Windows python 2.7 and that they were advised to just skip it, but I can't find the reference anymore. Do I remember reading this correctly and does this also apply for this package?

@conda-forge/core The builds now pass,  but I do have a couple of questions before merging:

Like the pyqt maintainers, I skipped builds for Windows Python 2.7. Is this the correct decision? I guess these are no longer possible since Qt does not support VS2008 any longer?
I had to add a hacky section, just like pyqt to let Linux find g++. It was not automatically found by the spec or the environment. Please let me know if this is the right way to go (for now), or if this should be solved differently.


python3.8

Python 3.8 is quite outdated but seems to be the version that is installed via apt get python3.

[ ] Check compatibility of mhubio and segdb with newer python versions.
[ ] Bump python version to python 3.10 or higher.
[ ] Test existing model implementations.
[ ] Report findings.
[ ] Publish new base image with newer python version.

@denbonte :)


Errors

Fixes #27

Would it be better to check if initial_data.type != "connect" and then send back some response. This would mean something wasn't handled correctly on the frontend, or is it fine as is?


Fix the URL for the "Getting Started" page in README.md

Problem
The link to "Getting Started" in README.me goes to a 404. The docs must have been reorganized recently.
Solution
Updated the URL from https://twitter.github.io/finatra/user-guide/getting-started/index.html to https://twitter.github.io/finatra/user-guide/index.html#getting-started

Hi @ggrossman, thanks so much for your contribution! It's been merged in 156a540


Any plan to add more options ?

Also , can you use Streams instead of valueNotifiers
it will be much easier to use with streams

I will evaluate the use of Streams. I wanted to implement sending images and files when I have some time.

well i wrote almost whole library ( for sending Messages only ) , and then i found this one ,just before publishing that  😅
so i dropped that plan ,
maybe i can help with the Streams implementation , or infact , we don't really require Streams of ValueListeners
according to me , logical thing will be to add , all these things as a Callback
so a user can add callback into the SendMessage method , to get update of the events ,
and what i did in my implementation was ,
i had two methods , one for initialising ,
and one for Sending message
initialize will make sure that web is opened and we are logged in
and once we are initialized , then we can use sendMessage method , as many times we want
this will save lots of time as well

@patricknicolosi my repo for reference
https://github.com/rohitsangwan01/whatsapp_bot_flutter


TypeError: '<=' not supported between instances of 'NoneType' and 'tuple'

When I try to run it I get this:
searedvandal-> python3.7 ~/eac_logsigner/eac.py --verify "White Noise - An Electric Storm.log"
Traceback (most recent call last):
  File "/home/searedvandal/eac_logsigner/eac.py", line 167, in <module>
    elif version <= CHECKSUM_MIN_VERSION:
TypeError: '<=' not supported between instances of 'NoneType' and 'tuple'

Any idea on how to fix?

Can you attach (not paste) the log file?

White Noise - An Electric Storm.log

It's now fixed:
$ python3 eac.py verify White.Noise.-.An.Electric.Storm.log
White.Noise.-.An.Electric.Storm.log:  OK

Thanks a lot!


Error within ErrorBoundary fallback causes endless loop.

Hello everyone :)
Have been playing around with solid a lot lately and encountered a "weird" behavior. I'm not sure if it is according to spec or a bug:
Describe the bug
When the fallback render function throws an error, the playground gets stuck. My guess is, that an error in an error boundaries fallback ends up in that very same error boundary and not its parent ErrorBoundary, causing some sort of endless loop.
To Reproduce
import { render } from "solid-js/web";
import { ErrorBoundary, For } from "solid-js";

function App() {
  return (
    <>

Welcome

      <ErrorBoundary fallback={Outer error}>
        <ErrorBoundary
          fallback={(err, reset) => {
            // Change this to err2 to cause an error
            console.log(err);
            return <button onClick={reset}>Retry</button>;
          }}
        >
          {(() => {
            throw new Error("BOOM");
          })()}
        </ErrorBoundary>
      </ErrorBoundary>
    </>
  );
}

render(() => <App />, document.getElementById("app"));

Reproduction in playground:

Open the playground
Change the err variable in the inner ErrorBoundary to err2
Page gets stuck.

Expected behavior
Intuitively, I guessed that the outer error boundary would catch the error and act as another fallback layer.

Yeah that is probably right assumption. Hadn't hit this case yet but sounds like we need to change the behavior a bit to get this working right.

Wow, that was quick...
Thanks a lot :)


Conditional example_gan.py

Hi,
I am trying to do a version of example_gan.py in which both generator and discriminator admit a label that tells which number to generate/identify. This requires the the label to be untouched when the input is passed through eliminate_z. I thought of the following modification:
def eliminate_z(gan, latent_sampling, label_dim):
    """
    Eliminate z from GAN using latent_sampling
    :param gan: model with 2 inputs: z, x
    :param latent_sampling: layer that samples z with same batch size as x
    :return: Model x -> gan(latent_sampling(x), x)
    """
    x = gan.inputs[1]
    l = x[:,-label_dim:]
    z = concatenate([latent_sampling(x), l])
    model = Model(inputs=x, outputs=fix_names(gan([z, x]), gan.output_names), name=gan.name)
    return model

However, when running the following example (which is in essence example_gan.py with the modifications needed to do a conditional GAN)
import numpy as np
import os
from keras.layers import Reshape, Flatten, LeakyReLU, Activation, concatenate, Input, Lambda, Concatenate
from keras.models import Sequential, Model
from keras.optimizers import Adam, SGD
from keras.callbacks import TensorBoard
from keras.utils import to_categorical
from keras_adversarial.image_grid_callback import ImageGridCallback
from keras_adversarial import AdversarialModel, simple_gan, gan_targets
from keras_adversarial import normal_latent_sampling, AdversarialOptimizerSimultaneous
from keras_adversarial.legacy import l1l2, Dense, fit
import keras.backend as K
from keras.datasets import mnist

def model_generator(latent_dim, label_dim, image_shape, hidden_dim=1024, reg=lambda: l1l2(1e-5, 1e-5)):
    inputs = (Input(shape=(latent_dim + label_dim,), name='generator_input'))
    y = (Lambda(lambda x: x[:,-label_dim:], output_shape=(label_dim,)))(inputs)
    T = (Dense(int(hidden_dim / 4), name="generator_h1", W_regularizer=reg()))(inputs)
    T = (LeakyReLU(0.2))(T)
    T = (Dense(int(hidden_dim / 2), name="generator_h2", W_regularizer=reg()))(T)
    T = (LeakyReLU(0.2))(T)
    T = (Dense(hidden_dim, name="generator_h3", W_regularizer=reg()))(T)
    T = (LeakyReLU(0.2))(T)
    T = (Dense(np.prod(image_shape), name="generator_x_flat", W_regularizer=reg()))(T)
    T = (Activation('sigmoid'))(T)
    T = concatenate([T, y])
    model = Model(inputs=inputs, outputs=T, name="generator")
    return model

def model_discriminator(imagepluslabel_shape, hidden_dim=1024, reg=lambda: l1l2(1e-5, 1e-5), output_activation="sigmoid"):
    inputs = (Input(shape=imagepluslabel_shape, name='discriminator_input'))
    T = (Dense(hidden_dim, name="discriminator_h1", W_regularizer=reg()))(inputs)
    T = (LeakyReLU(0.2))(T)
    T = (Dense(int(hidden_dim / 2), name="discriminator_h2", W_regularizer=reg()))(T)
    T = (LeakyReLU(0.2))(T)
    T = (Dense(int(hidden_dim / 4), name="discriminator_h3", W_regularizer=reg()))(T)
    T = (LeakyReLU(0.2))(T)
    T = (Dense(1, name="discriminator_y", W_regularizer=reg()))(T)
    T = (Activation(output_activation))(T)
    model = Model(inputs=inputs, outputs=T, name="discriminator")
    return model

def eliminate_z(gan, latent_sampling, label_dim):
    """
    Eliminate z from GAN using latent_sampling
    :param gan: model with 2 inputs: z, x
    :param latent_sampling: layer that samples z with same batch size as x
    :return: Model x -> gan(latent_sampling(x), x)
    """
    x = gan.inputs[1]
    l = x[:,-label_dim:]
    z = concatenate([latent_sampling(x), l])
    model = Model(inputs=x, outputs=fix_names(gan([z, x]), gan.output_names), name=gan.name)
    return model

def simple_gan(generator, discriminator, latent_sampling, label_dim):
    # build basic gan
    gan = build_gan(generator, discriminator)
    # generate z on gpu, eliminate one input
    if latent_sampling is None:
        return gan
    else:
        return eliminate_z(gan, latent_sampling, label_dim)

# z \in R^100
latent_dim = 100
# x \in R^{28x28}
image_shape = (int(28 * 28),)
# label in R^10
label_dim = 10

imagepluslabel_shape = (int(28 * 28 + label_dim),)
# generator ((z,l) -> (x,l))
generator = model_generator(latent_dim, label_dim, image_shape)
# discriminator ((x,l) -> y)
discriminator = model_discriminator(imagepluslabel_shape)
simple_gan(generator, discriminator, normal_latent_sampling((latent_dim,)), label_dim)

I get the following error:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-40-432f035f7981> in <module>()
----> 1 simple_gan(generator, discriminator, normal_latent_sampling((latent_dim,)), label_dim)

<ipython-input-38-20b3c0a977c6> in simple_gan(generator, discriminator, latent_sampling, label_dim)
     34         return gan
     35     else:
---> 36         return eliminate_z(gan, latent_sampling, label_dim)
     37
     38

<ipython-input-38-20b3c0a977c6> in eliminate_z(gan, latent_sampling, label_dim)
     23     l = x[:,-label_dim:]
     24     z = concatenate([latent_sampling(x), l])
---> 25     model = Model(inputs=x, outputs=fix_names(gan([z, x]), gan.output_names), name=gan.name)
     26     return model
     27

c:\python\python35\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
     86                 warnings.warn('Update your `' + object_name +
     87                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 88             return func(*args, **kwargs)
     89         wrapper._legacy_support_signature = inspect.getargspec(func)
     90         return wrapper

c:\python\python35\lib\site-packages\keras\engine\topology.py in __init__(self, inputs, outputs, name)
   1652         for x in self.outputs:
   1653             seen_nodes = set()
-> 1654             build_map_of_graph(x, seen_nodes, depth=0)
   1655
   1656         # Build a dict {depth: list of nodes with this depth}

c:\python\python35\lib\site-packages\keras\engine\topology.py in build_map_of_graph(tensor, seen_nodes, depth, layer, node_index, tensor_index)
   1648                 if node_marker not in seen_nodes:
   1649                     build_map_of_graph(x, seen_nodes, current_depth + 1,
-> 1650                                        layer, node_index, tensor_index)
   1651
   1652         for x in self.outputs:

c:\python\python35\lib\site-packages\keras\engine\topology.py in build_map_of_graph(tensor, seen_nodes, depth, layer, node_index, tensor_index)
   1648                 if node_marker not in seen_nodes:
   1649                     build_map_of_graph(x, seen_nodes, current_depth + 1,
-> 1650                                        layer, node_index, tensor_index)
   1651
   1652         for x in self.outputs:

c:\python\python35\lib\site-packages\keras\engine\topology.py in build_map_of_graph(tensor, seen_nodes, depth, layer, node_index, tensor_index)
   1643                 node_index = node.node_indices[i]
   1644                 tensor_index = node.tensor_indices[i]
-> 1645                 next_node = layer.inbound_nodes[node_index]
   1646                 # use node_marker to prevent cycles
   1647                 node_marker = make_node_marker(next_node, current_depth + 1)

AttributeError: 'NoneType' object has no attribute 'inbound_nodes'

As far as I understand it is the Model function that is failing, but I cannot still manage to know why. Help would be very much appreciated.

@apozas sorry for not getting to this question earlier.
You should build the generator and discriminator submodels with multiple inputs if you are making a CGAN. If you're doing something custom, you don't really need elimate_z.
Something like this should work for you:
xreal = Input(...)
label = Input(...)
z = normal_latent_sampling((latent_dim,))(label)
# concatenate z and label or do whatever
# calculate xgen=G(Z,L) and yreal=D(X)
# create G and D submodels
g = Model([z,label], xgen)
d = Model([xreal,label], yreal)
# create combined model
yfake = d([xgen, label])
model = Model([z, label, xreal], [yreal, yfake])
# create adversarial model
amodel = AdversarialModel(player_weights=[g.weights, d.weights], ...)

I'll try to put an example along these lines up but it is going to be a busy semester.
Cheers

Hi @bstriner, thank you very much for your assistance. Now I have no problems with creating the model nor compiling it, but an error pops out when I try to fit it. It seems some kind of compatibility clash with the latest versions of keras (I am using keras 2.0.8). The code that triggers the error and the error itself are the following.
import numpy as np
import os
from keras.layers import Reshape, Flatten, LeakyReLU, Activation, concatenate, Input, Lambda
from keras.models import Sequential, Model
from keras.optimizers import Adam, SGD
from keras.callbacks import TensorBoard
from keras.utils import to_categorical
from keras_adversarial.image_grid_callback import ImageGridCallback
from keras_adversarial import AdversarialModel, simple_gan, gan_targets
from keras_adversarial import normal_latent_sampling, AdversarialOptimizerSimultaneous
from keras_adversarial.legacy import l1l2, Dense, fit
import keras.backend as K
from keras.datasets import mnist

latent_dim = 100
label_dim = 10
image_shape = (int(28 * 28),)
hidden_dim = 1024
reg=lambda: l1l2(1e-5, 1e-5)
output_activation='sigmoid'

label = (Input(shape=(label_dim,), name='label'))
z = normal_latent_sampling((latent_dim,))(label)
xreal = (Input(shape=image_shape, name='discriminator_input'))

# Generator
inputs_g = (concatenate([z, label], name='input_concatenation'))
T = (Dense(int(hidden_dim / 4), name="generator_h1", W_regularizer=reg()))(inputs_g)
T = (LeakyReLU(0.2))(T)
T = (Dense(int(hidden_dim / 2), name="generator_h2", W_regularizer=reg()))(T)
T = (LeakyReLU(0.2))(T)
T = (Dense(hidden_dim, name="generator_h3", W_regularizer=reg()))(T)
T = (LeakyReLU(0.2))(T)
T = (Dense(np.prod(image_shape), name="generator_x_flat", W_regularizer=reg()))(T)
T = (Activation('sigmoid'))(T)
generator = Model(inputs=label, outputs=[T, label], name="generator")

# Discriminator
inputs_d = (concatenate([xreal, label], name='input_concatenation'))
T = (Dense(hidden_dim, name="discriminator_h1", W_regularizer=reg()))(inputs_d)
T = (LeakyReLU(0.2))(T)
T = (Dense(int(hidden_dim / 2), name="discriminator_h2", W_regularizer=reg()))(T)
T = (LeakyReLU(0.2))(T)
T = (Dense(int(hidden_dim / 4), name="discriminator_h3", W_regularizer=reg()))(T)
T = (LeakyReLU(0.2))(T)
T = (Dense(1, name="discriminator_y", W_regularizer=reg()))(T)
T = (Activation(output_activation))(T)
discriminator = Model(inputs=[xreal, label], outputs=T, name="discriminator")

yreal = Activation("linear", name="yreal")(discriminator(discriminator.inputs))
yfake = Activation("linear", name="yfake")(discriminator(generator(generator.inputs)))
gan = Model(inputs=discriminator.inputs, outputs=[yreal, yfake])
amodel = AdversarialModel(base_model=gan,
                      player_params=[generator.trainable_weights, discriminator.trainable_weights],
                      player_names=["generator", "discriminator"])
amodel.adversarial_compile(adversarial_optimizer=AdversarialOptimizerSimultaneous,
                                              player_optimizers=['SGD', 'SGD'],
                                              loss='binary_crossentropy')

 (x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape[0], 784)
x_train = x_train / 255
y_train = to_categorical(y_train)
y = gan_targets(x_train.shape[0])
fit(amodel, x=[x_train, y_train], y=y)

And the error spit is
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-7-c4767d075ffa> in <module>()
----> 1 fit(amodel, x=[x_train, y_train], y=y)

c:\python\python35\lib\site-packages\keras_adversarial-0.0.3-py3.5.egg\keras_adversarial\legacy.py in
fit(model, x, y, nb_epoch, *args, **kwargs)
     16 def fit(model, x, y, nb_epoch=10, *args, **kwargs):
     17     if keras_2:
 ---> 18         return model.fit(x, y, *args, epochs=nb_epoch, **kwargs)
     19     else:
     20         return model.fit(x, y, *args, nb_epoch=nb_epoch, **kwargs)

c:\python\python35\lib\site-packages\keras\engine\training.py in fit(self, x, y, batch_size, epochs,
verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch,
steps_per_epoch, validation_steps, **kwargs)
   1573         else:
   1574             ins = x + y + sample_weights
-> 1575         self._make_train_function()
   1576         f = self.train_function
   1577

c:\python\python35\lib\site-packages\keras_adversarial-0.0.3-
py3.5.egg\keras_adversarial\adversarial_model.py in _make_train_function(self)
    159                                                                                self.optimizers,
    160                                                                                [model.constraints for model in
--> 161                                                                                self.layers],
    162                                                                                self.updates,
    163                                                                                self._function_kwargs)

c:\python\python35\lib\site-packages\keras_adversarial-0.0.3-
py3.5.egg\keras_adversarial\adversarial_model.py in <listcomp>(.0)
    158                                                                                self.player_params,
    159                                                                                self.optimizers,
--> 160                                                                                [model.constraints for model in
    161                                                                                self.layers],
    162                                                                                self.updates,

AttributeError: 'Model' object has no attribute 'constraints'

@bstriner any update to resolve this issue?

A complete example of AC-GAN using keras-adversarial would be nice !
if possible :-)


feat(js): setup React Native

Signed-off-by: blu3beri<EMAIL_ADDRESS>

It now uses jq to fetch the version from the Cargo.toml.

Is jq installed by default?

Yeah that's the one thing I was not sure about. I will test it quickly on a random project.

@TimoGlastra yes jq is by default installed on GH runners :).


DRAFT: C#-Performance:  Use a custom static lookup to handle InvokeGodotClassMethod and HasGodotClassMethod

Preface
This is in an early draft state and by no means complete. I am presenting my working proof of concept to discuss if and how to progress with the main idea of this PR. There are lots of open TODOs, questions and polishing to be done. Please focus on the idea instead of the rough sketch shown here (there is also some debug output which will obviously be removed later on).
Be warned: this is a long one to give readers enough context! If you want a TL;DR scroll to the bottom for the performance numbers.
Introduction
Everytime Godot wants to call a C# script method from C++, the auto-generated InvokeGodotClassMethod is invoked.
The current C# bridge implementation of InvokeGodotClassMethod approximately works like this for a C# script which overrides _Ready and _Process:
protected override bool InvokeGodotClassMethod(in godot_string_name method, NativeVariantPtrArgs args, out godot_variant ret)
{
    if (method == MethodName._Ready && args.Count == 0) {
        _Ready();
        ret = default;
        return true;
    }
    if (method == MethodName._Process && args.Count == 1) {
        _Process(global::Godot.NativeInterop.VariantUtils.ConvertTo<double>(args[0]));
        ret = default;
        return true;
    }
    return base.InvokeGodotClassMethod(method, args, out ret);
}

While this seems pretty o.k. there is a small detail which makes InvokeGodotClassMethod more complicated than it seems: the engine is passing the string "_process" as the method name while the user script InvokeGodotClassMethod checks for "_Process". This case is handled by the C# glue of the builtin Godot classes (e.g. Node2D or Node3D) which all user scripts are inherting from (either directly or indirectly via other user scripts):
// excerpt from the generated Node.cs
// MethodProxyName__* are the PascalCase variants, MethodName._Process is "_process" here
if ((method == MethodProxyName__process || method == MethodName._Process) && args.Count == 1 && HasGodotClassMethod((godot_string_name)MethodProxyName__process.NativeValue))
{
    _Process(VariantUtils.ConvertTo<double>(args[0]));
    ret = default;
    return true;
}
if ((method == MethodProxyName__ready || method == MethodName._Ready) && args.Count == 0 && HasGodotClassMethod((godot_string_name)MethodProxyName__ready.NativeValue))
{
    _Ready();
    ret = default;
    return true;
}

And then there is HasGodotClassMethod here - what does it do? It checks if a method is actually implemented as e.g. Node has to supply an empty stub implementation for _Process(double delta) to make this code compile. But why not just call the stub implementation? Because the C++ -> C# function parameter marshalling done via VariantUtils could be costly, especially for more complex types like arrays - so why marshall if an empty stub is called? This also has to be implemented this way because what happens if a user actually implements both _Process(double) and _process(double) in their script? We can't just auto-proxy "_process" to "_Process" because of this.
At this point we traversed the class hierarchy for a bit and jump back up to the original C# class to call HasGodotClassMethod.
This really isn't cheap and there is another part to this: the engine calls our C# script with "_process", "_enter_tree" etc. no matter if we implement it or not and calling HasGodotClassMethod with an "unimplemented" method name traverses the whole class hierarchy and depending on what class the current C# script is based on, this can be quite expensive - especially for _PhysicsProcess for Nodes which don't override it in C#.
Additionally, every* public C# script method gets added into the if-chain of InvokeClassMethod and HasGodotClassMethod,  which indirectly makes all Godot base class (Node etc) method calls slower because of how the snake_case to PascalCase handling is implemented (see above) - this hurts, especially for _Process and _PhysicsProcess.
This makes the current implementation:

somewhat slow (see performance numbers later)
hard to reason about as a human
hard to reason about for the branch predictor

An alternative implementation
I will try to keep this section high level:
Instead of a bunch of ifs we will use a Dictionary<(ParameterCount, MethodName), Delegate> to resolve functions. Adding all the C# bridge details to this yields a wrapper class (called ScriptMethodRegistry for now) and the following snippet for a user script called MainScene which implements _Ready and _Process in C#:
public new static readonly ScriptMethodRegistry<MainScene> MethodRegistry = new ScriptMethodRegistry<MainScene>()
    .Register(MethodName._Ready, 0, (MainScene scriptInstance, NativeVariantPtrArgs args, out godot_variant ret) =>
    {
        scriptInstance._Ready();
        ret = default;
    })
    .Register(MethodName._Process, 1, (MainScene scriptInstance, NativeVariantPtrArgs args, out godot_variant ret) =>
    {
        scriptInstance._Process(global::Godot.NativeInterop.VariantUtils.ConvertTo<double>(args[0]));
        ret = default;
    })
    .Compile();

This still misses the  "_process" -> "_Process" mapping for the engine calls which is done via aliases, here is an excerpt from Node.cs:
 public static readonly ScriptMethodRegistry<Node> MethodRegistry = new ScriptMethodRegistry<Node>()
    // remember MethodProxyName__process is the PascalCaseName here
    .AddAlias(Node.MethodProxyName__process, 0, Node.MethodName._Process)

These aliases are then imported which gives us:
public new static readonly MethodRegistry<MainScene> MethodRegistry = new MethodRegistry<MainScene>()
    // this is new, we register our parents registry (aliases and function registrations)
    // our parent registry did the same with Node, which did the same with GodotObject ...
    .Register(global::Godot.Node3D.MethodRegistry)
    .Register(MethodName._Ready, 0, (MainScene scriptInstance, NativeVariantPtrArgs args, out godot_variant ret) =>
    {
        scriptInstance._Ready();
        ret = default;
    })
    .Register(MethodName._Process, 1, (MainScene scriptInstance, NativeVariantPtrArgs args, out godot_variant ret) =>
    {
        scriptInstance._Process(global::Godot.NativeInterop.VariantUtils.ConvertTo<double>(args[0]));
        ret = default;
    })
    .Compile();

This concept fixes all* evil issues:

if a script overrides a function that its parent already implemented, it gets overriden because the registry of the parent is added first and then potentially overriden by local .Register definitions
if Godot users actually implement _process(double) and _Process(double), the correct method is invoked when the engine passes "_process" as the method name

The .Compile() step is needed because we need to apply the alias logic somewhere.
And thats about it, invoking a function is easy and fast now, a negative response is just as fast which is a big boon (see introduction):
protected override bool InvokeGodotClassMethod(in godot_string_name method, NativeVariantPtrArgs args, out godot_variant ret)
{
    if (MethodRegistry.TryGetMethod(in method, args.Count, out var scriptMethod))
    {
         scriptMethod(this, args, out ret);
         return true;
    }

    ret = new godot_variant();
    return false;
}

HasGodotClassMethod
I went ahead and changed the implementation for HasGodotClassMethod as well because it kind of uses the same pattern and the ScriptMethodRegistry could fulfill its requirements with minor adjustments. We will have to think about other places like InvokeGodotClassStaticMethod, should we apply this pattern there too?
Downsides and caveats
Here is an unordered list of downsides / caveats I can think of:

I pointed out that the current implementation is hard to reason about, the proposed changes are not improving this in a significant way and to some people may even seem more complex, which is understandable
We have to be careful not to do too much work in a static context as it would negatively impact users (stutters, hanging because of load time initialization); this is especially annoying for Godot users because its hard to manually control static initialization (I think RuntimeHelpers.RunClassConstructor etc. should be avoided in Godot user code). Speaking from personal experience, we would have to do significantly more work for this to become noticable
We need one static dictionary per C# script type (no matter if its generated C++ glue classes or user scripts), this increases Godot C#'s memory footprint which we should measure at least once

Known issues

Example: A user-script which inherits from Node which provides _Process(int) will erroneously be called as a substitute for _Process(double) because the current aliasing implementation does no type checks. This will be difficult to fix / address. The master version of InvokeGodotClassMethod would only "erroneously" call the stub _Process(double) in Node  (because HasGodotClassMethod does no type checks and thus returns true). How would this work in GDScript, what is the expected or acceptable behaviour?

This is already complicated in current master if a user defines _Process(int) before defining _Process(double) in their C# script. Overloads are deduplicated in ScriptMethodGenerator by name and argument count and only one of them is emitted into InvokeGodotClassMerthod - _Process(int) in this case. The correct method would still be called when the engine passes "_process" to InvokeGodotMethod though
We need a clear stance on method overloading with equal argument counts here, shouldn't we forbid it in general? Right now it feels like undefined behaviour (e.g. it dependent on method declaration order in C#)

master handles overloading for methods with the same parameter count by using the first occurence, this PR uses the last occurence

Personal opinion on downsides, caveats, issues and how to deal with them
This is just my personal opinion on how to deal with certain issues, feel free to skip it.
I think we need a clear definition of what the C# integration supports and what is outright illegal, leading to compilation issues (the same as missing the partial identifier in user scripts).
I propose the following new constraints:

same-parameter method overloading for public methods is disallowed
public methods are disallowed to start with _[a-z] (in words: underscore with a lowercase character) as this heavily conflicts with the snake_case to PascalCase resolution which even breaks master at the moment (_process(int) would be called by the engine "_process" call)

Users affected by 1. are already somewhat affected, as master only registers the first overload for signal bindings etc.
Users affected by 2. are on the wrong path anyways, using _ as a prefix to non-engine supplied functions is suspicious on its own, going for lowercase characters afterwards is even more weird.
I think very few users will be affected by these constraints. Right now its hard to gauge what input (user scripts) the C# integration has to accomodate, which is also shown by the lack of extensive tests or documentation on what edge-cases are legal or illegal (the C# integration even silently discards all but one same parameter method overload).
Godots C# integration has good error reporting in place, we should use it more.
Method overloading was also requested in GDScript, but declined for now because of performance concerns.
Performance measurements
To stress test this I made a test project which spawns 100.000 Nodes with an attached C# script. Said C# script overrides _Ready and _Process. I used it in another PR (https://github.com/godotengine/godot/pull/89807) if you wonder why its name is different from this PR:
VariantOptimizationTestProject.zip
On my machine (Win11, NET.Core 7.0.2, CPU: 5800x3D), I get the following results:

Implementation
Run type
Process Time (ms)

master this PR is based off of
Editor "Run project"
238 - 245ms

This PR (across multiple runs)
Editor "Run project"
40 - 45ms

This PR + https://github.com/godotengine/godot/pull/89807
Editor "Run project"
25 - 28 ms

Godot 4.3 Dev 5
Official Windows export template
Run 1: min 16,97ms - Run 2: min 17,33 ms

This PR
Windows LTO=full export template
Run 1: min 14,70ms - Run 2: min 15,07ms

This PR + https://github.com/godotengine/godot/pull/89807
Windows LTO=full export template
Run 1: min 13,90 ms - Run 2: min 14,01 ms

The gains for in Editor running are significant, in the area of one order of magnitude (10x).
The relatively small gap between the exported templates caught me by surprise, having the editor be closer to the release template is a goodie though. I opted for the min observed frame time for the release template because it shows the least externally influenced frame.
This stark difference has me pondering if such a big change is worth ~15% release performance for such an overblown test project.
TODOs

[x] Make the PR build in CI (code style, code gen tests, spell checking)
[x] Rename FunctionRegistry to ScriptMethodRegistryand its accompanying classes to *Method* to be in line with other Godot wordings
[x] Cleanup TODOs meant for myself
[ ] Talk to C# binding code owners about open questions, issues and how to solve them (see remaining code TODOs)
[ ] Check with the C# binding code owners if its ok to make godot_string_name._data visible with internal as everything else breaks apart if this change is declined
[ ] Discuss if same argument count method overloads in C# scripts should be allowed, they don't work correctly anyways. Emulating the master overload handling (discarding all but one overload, by definition order) would be complicated in this PR
[ ] Clarify why the HasGodotClassMethod is used in CSharpInstanceBridge._Get  without parameter count checks, a getter with parameters makes no sense to me

TODOs if we proceed with this PRs idea

[ ] What about InvokeGodotClassStaticMethod?
[ ] Check the assembly size, it must have shrunk a bit
[ ] Add unit tests for ScriptMethodRegistry as the whole "inheritance" and aliasing part of it is pretty easily testable

Thank you to @paulloz and @raulsntos for helping me in Rocket.Chat / Discord on how to go on with my idea!

I could use some help with fixing the pipeline because the diff isn't really helping me:

I checked for LF / CLRF issues but the whole file is LF like other places.

I expect this is a bom issue, double-check the file encoding between utf8 and utf8-bom.

Thanks, I was lazy it seems :D I fixed the BOM issue and will move on to the other broken pipeline steps. May I get a workflow approval for future pipeline runs? Is this approval manual for every push I do for draft MRs?

I gave this one more round of polish, names are more in line with standard Godot wording an I refactored the ScriptMethod delegate to not the return bool as that should be handled by InvokeGodotClassMethod - that seemed to have gained me 2-3ms (~5%) locally, but I won't edit the main post because I feel like that could be noise from less stuff running in parallel on my machine. The remaining TODOs are about discussions to be had.
P.S: If someone with some spare time and an actual Godot C# project stumbles upon this PR, I would love to hear if this PR breaks or improves stuff for you!

It seems like I miscalculated how much the release template performance would compare to the "Editor -> Run Project": The gains in the release template are small, around 10% with this PR, nearly 20% when adding the other referenced PR.
I added the new numbers to the main post, heres the updates test project which shows the process time in a Label
VariantOptimizationTestProject_New.zip

And now I wonder if my test project gets kind of dead code eliminated in release builds because _Process is a nop, need to rework it.


Securing open source

Add tab and content for Securing open source to Program page.

Thanks! Will review.


typo: cognitiveservices/data-plane/QnAMaker

knowlegebases -> knowledgebases

Automation for azure-sdk-for-js

Automation for azure-sdk-for-python

Automation for azure-sdk-for-node

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-java

Can one of the admins verify this patch?

Automation for azure-sdk-for-go


[Issue]:  not supported for japanese language..

Is there an existing issue for this?

[X] I have searched the existing issues
[ ] I have checked #657 to validate if my issue is covered by community support

Describe the issue
I am testing the graphrag for japanese text to create a graph and query the text file using local and global method.
issue1- when the community report are generated , many part of the report the result are "title": "\u5343\u6210\u5de5\u696d\u682a\u5f0f\u4f1a\u793e like unicode format
issue2: when i query using the local method, it don't give the output from the text that i feeded at the time of indexing process. the llm is giving it own knowledge instead of searching from the lancedb or from community report.
model use llama3 using ollama service

Please see #696 for non-English content, and #657 for non-OpenAI models. There may be commentary from the community that helps resolve your issue.

This is caused by not doing ensure_ascii=False when doing json.dumps.
I solved this by changing the class FileWorkflowCallbacks in file_workflow_callbacks.py as follows
class FileWorkflowCallbacks(NoopWorkflowCallbacks):
    """A reporter that writes to a file."""

    _out_stream: TextIOWrapper

    def __init__(self, directory: str):
        """Create a new file-based workflow reporter."""
        Path(directory).mkdir(parents=True, exist_ok=True)
        self._out_stream = open(  # noqa: PTH123, SIM115
            Path(directory) / "logs.json", "a", encoding="utf-8", errors="strict"
        )

    def on_error(
        self,
        message: str,
        cause: BaseException | None = None,
        stack: str | None = None,
        details: dict | None = None,
    ):
        """Handle when an error occurs."""
        self._out_stream.write(
            json.dumps({
                "type": "error",
                "data": message,
                "stack": stack,
                "source": str(cause),
                "details": details,
            }, ensure_ascii=False)
            + "\n"
        )
        message = f"{message} details={details}"
        log.info(message)

    def on_warning(self, message: str, details: dict | None = None):
        """Handle when a warning occurs."""
        self._out_stream.write(
            json.dumps({"type": "warning", "data": message, "details": details}, ensure_ascii=False) + "\n"
        )
        _print_warning(message)

    def on_log(self, message: str, details: dict | None = None):
        """Handle when a log message is produced."""
        self._out_stream.write(
            json.dumps({"type": "log", "data": message, "details": details}, ensure_ascii=False) + "\n"
        )

        message = f"{message} details={details}"
        log.info(message)

def _print_warning(skk):
    log.warning(skk)


strconv: ParseFloat returns out of range error on invalid suffix

What version of Go are you using (go version)?

$ go version
go version go1.16.4 darwin/arm64

Does this issue reproduce with the latest release?
Yes, behavior inconsistency against the document.
This is an undocumented breaking change between 1.14 and 1.15.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/itchyny/Library/Caches/go-build"
GOENV="/Users/itchyny/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/itchyny/.share/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/itchyny/.share/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/homebrew/Cellar/go/1.16.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/homebrew/Cellar/go/1.16.4/libexec/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.16.4"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ph/slcnxxv955n3zk1fzk50rmvr0000gn/T/go-build4154388589=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?
Execute the following code.
package main

import (
	"fmt"
	"strconv"
)

func main() {
	fmt.Println(strconv.ParseFloat("1e100xxx", 64))
	fmt.Println(strconv.ParseFloat("1e1000xxx", 64))
}

What did you expect to see?
Just as Go <= 1.14,
0 strconv.ParseFloat: parsing "1e100xxx": invalid syntax
0 strconv.ParseFloat: parsing "1e1000xxx": invalid syntax

The strconv.ParseFloat document says

If s is not syntactically well-formed, ParseFloat returns err.Err = ErrSyntax.
If s is syntactically well-formed but is more than 1/2 ULP away from the largest floating point number of the given size, ParseFloat returns f = ±Inf, err.Err = ErrRange.

The input 1e1000xxx here satisfies the first if statement just like 1e100xxx does, so the err.Err should be ErrSyntax, not ErrRange. They are not syntactically well-formed.
The change seems to be the side effect of https://github.com/golang/go/commit/1d31f9b1e05d3766ed2132b5856d364d00c5fdf9, which changed the error on unconsumed characters.
What did you see instead?
0 strconv.ParseFloat: parsing "1e100xxx": invalid syntax
+Inf strconv.ParseFloat: parsing "1e1000xxx": value out of range

cc @griesemer @ianlancetaylor


Release notes for 0.16.1

Please make suggestions for any changes!

Does anyone know why RTD is failing? RTD is often inscrutable to me...

the relevant output of sphinx is:
/home/docs/checkouts/readthedocs.org/user_builds/xray/checkouts/4435/doc/whats-new.rst:55: WARNING: Bullet list ends without a blank line; unexpected unindent.
/home/docs/checkouts/readthedocs.org/user_builds/xray/checkouts/4435/doc/whats-new.rst:57: WARNING: Definition list ends without a blank line; unexpected unindent.

Thanks @keewis . I was looking at the bottom of the page

for warnings that don't print a stacktrace (sphinx warnings) you typically have to look at the output between waiting for workers... (directly below the reading sources... lines) and pickling environment


buffer in a tmpfile

hashpipe needs to buffer all the data as the hash isn't decidable until the last bit. we could buffer on disk with a tmpfile instead of memory.

18:39 <Luke> jbenet: if I use hashpipe, it has to read the whole input into memory, right?
18:40 <jbenet> Luke: yeah, for now. https://github.com/jbenet/hashpipe/issues/1
18:40 <Luke> jbenet: yeah but either way that has to live somewhere. swapping might also take care of it
18:41 <jbenet> Luke: yeah. ideally it would be an option with a specified filename, in case you need to control it.

<Tv`> jbenet: you could use an unlinked tempfile

thanks @tv42!

buffering is a tricky issue and truly breaks the advantages of a pipeline. in practice a solid script would always check the exit code of hashpipe, regardless of the output.  perhaps an option to hashpipe?

It should be an option at the very least
—
Sent from Mailbox
On Fri, Apr 3, 2015 at 8:40 PM, jmscott<EMAIL_ADDRESS>wrote:

buffering is a tricky issue and truly breaks the advantages of a pipeline. in practice a solid script would always check the exit code of hashpipe, regardless of the output.  perhaps an option to hashpipe?
Reply to this email directly or view it on GitHub:
https://github.com/jbenet/hashpipe/issues/1#issuecomment-89493798

buffering the blob still does not remove the need to check the exit code in a pipeline of any consequence, so why buffer?  for example, hypothtically the write of the buffered data on the pipeline could fail with a partial write, as well. if hashpipe fails to digest properly simply abort with a burp to stderr and and a non-zero exit code and let the caller worry about how to harden the environment.  in other words, the exit code indicates correctness as much as the output.

if hashpipe fails to digest properly simply abort with a burp to stderr and and a non-zero exit code

yep, this is what it already does.
the buffering is to avoid nuking the machine's memory. swap may work fine, but it's still unideal.

so hashpipe buffers (to temp file) the whole blob before writing to output?
if so then why?  the exit code would indicate correctness, not the output.
a caller will have to check the exit code anyway in a strict environment,
so it's not clear to be what advantage buffering gives.
-j
On Sat, Apr 4, 2015 at 6:01 PM, Juan Batiz-Benet<EMAIL_ADDRESS>wrote:

if hashpipe fails to digest properly simply abort with a burp to stderr
and and a non-zero exit code
yep, this is what it already does.
the buffering is to avoid nuking the machine's memory. swap may work fine,
but it's still unideal.
—
Reply to this email directly or view it on GitHub
https://github.com/jbenet/hashpipe/issues/1#issuecomment-89679224.

it does not currently buffer to a temp file. im saying there should be an option to.

the buffering is to avoid nuking the machine's memory.

suppose the file being input is 1TB.

@jmscott assuming ./a | hashpipe $hash | ./b, ./b has no access to hashpipes exit code, but will start processing as soon as it gets data on stdin.
@jbenet you could chunk the file, calculate a hash for each chunk and then hash the list of hashes. On start, it reads and verifies the list of hashes with argv[1] and is then able to verify and write each chunk to stdout.

no, ./b should NOT receive any data until the whole hash is verified. this is a well known attack vector. attackers can cut the download mid-stream and leave things in inconsistent states.

@jbenet I think you misread my comment. ./b starts processing data as soon as it gets any, which must not happen until the hash is verified. This is the reason why it can't just write to stdout and expect the program to wait for an exit node.

Never mind, noticed you've been referring to the 2nd part.


How do you access $ and _ ?

I have XML output like this:
var value = content[2]['myvalues'][0]['values'][0];
console.log(value);

// Output
{
  _: 'This is some content',
  '$': {
    'attribute-id': 'test233',
    'xml:lang': 'en_US'
  }
}

I can't do content[2]['myvalues'][0]['values'][0]['_'] or content[2]['myvalues'][0]['values'][0]['$']. They don't return any content. What am I missing here?

Oh wow I can just do ._ and .$. Wild. I had no idea that was going to be valid syntax... Nevermind :)


Stabilize stream consumption before routing queries to servers

We noticed that in some cases when a pinot-server hosting segments in CONSUMING state is restarted it showed high query latencies. The high latency correlated with high consumption rate from streams (a restart causes a consuming segment to consume from the starting offset until latest event).
A few different options to address this problem:

Save state before shutdown. In this case it will mean saving all the events consumed so far (and the last offset consumed) before shutdown. Completing the segment is one way of doing that, but then it not only takes a while to complete the segment, but the next replica to restart (in a rolling restart) will go through the same motion, producing a very small segment in the process. So, saving state locally is better, but is a complex matter. Could be easier with mmaped consuming segments, but we still have inverted indices on heap.

Re-construct state after startup, which is what we do now (in terms of consuming from the starting offset).

Stop routing query requests while consuming segments are busy catching up.

The last option seems to be simple.
There are a few different ways of implementing that. We currently have a way to stop routing queries by not re-setting the isShuttingdown flag until segments are loaded. We can extend the criteria to include that "loading" a consuming segment effectively means that the cosnuming segment has "caught up".
It seems best to include this as a part of ServiceStatus.
It is a little hard to determine if the consuming segments have caught up. The best way is to query the stream to get the current offset or timestamp and decide if it has. Other ways could be to add up the rates of consumption of different consuming segments and as long as the total consumption rate is below a certain threshold, announce that we are ready. Some streams may also throttle consumption.
For now, we should start by integrating ServiceStatus to also query consuming segments (perhaps through the tabledatamanager). Perhaps we can start off with a (configured) time out, and then figure out better ways to determine if consuming segments have caught up.

https://github.com/apache/pinot/pull/7267 adds a mechanism to get the latest offset for each consuming segment and uses that as the basis for ending the consumption catch-up period.

@sajjad-moradi can this issue be closed now?


Add option to always show number of PRs next to menubar icons

Currently, PullBar shows number of PRs next to the menubar icon only if one type of PRs is selected. If more than one type is selected, then there is no number.
It would be nice to add add an option to still show the number of preferred types of PRs, say, you want to see Created and Review Requested PRs in the menu, and next to the menubar icon show number of Review Requested PRs.

Closed by https://github.com/menubar-apps/PullBar/pull/25


Support secrets in app model environment settings.

At the moment in the app model we do the following methods to populate environment variables:
WithEnviroment(string, string)
WithEnvironment(string, Func<string>)
WithEnvironment(Action<Dictionary<string, string>)

I think that we'll probably end up having to add:
WithEnvironment(string, string, EnvironmentHandling) // enum EnvironmenHandling { Secret, NotSecret }
WithEnvironment(string, Func<string>, EnvironmentHandling)
WithEnvironment(Action<EnvironmentContext>)

Usage on that last one:
WithEnvironment((env) => {
  env.AddVariable("FOO", "bar");
  env.AddSecret("BAR", "foo");
});

Most orchestrators have this concept of something that is mapped as an environment variable but is otherwise handled as a secret throughout the platform.

Closing in vavor of #770


Directory structure not easy to grasp

I wonder if the structure is compulsory for us. Otherwise, I propose to switch from the existing structure of the scheme
{experiment name}/00#_{processing step}/{UseCase name}

to the following
usecases/{UseCase name}/00#_{processing step}

Why?

Strictly hierarchical, easy to understand
Each UseCase is calculated separately during development and so the directories should be isolated as well. Data for each UseCase is grouped and can be copied or deleted easily without hopping from folder to folder.
Path operations in the code are easier to understand.
The structure follows a simple principle: Python scripts that can be run and that manipulate data write files only into directories down the hierarchy, never on the same level or above (for reading it's different)

Additionally, consider:

that the name parts "data" and "results" are not relevant. It puts a difference in the user's mind which I don't see. We could omit that without loosing anything. It will make the path statements in the code more readable.
that the content of both 004_results_template and 001_data_raw seem identical to me across all the experiments and UseCases. We better move this (kind of) input data to a folder outside the results context
that UseCase names include the experiment name even now. Why not leave the structure flat by using a general experiments or UseCases or scenarios usecases folder?
that UseCase names in the run scripts file names won't be necessary anymore because of the isolation of the UseCases (easier in the code) (already done)
#96
The comparison folder is the actual result of all oemoflex activities. Also, data from other models come into play (in contrast to all the other folders). Putting it on a top hierarchy level would reflect both facts. This is analog to data: whereas data is the input from the project partners, comparison is the output to them.

This leads us to the following structure:
[usecases]
    - [FlexMex_1_1]
        - [001_preprocessed]
	- [002_optimization]
	- [003_postprocessed]
	- infer.py
	- preprocessing.py
	- optimization.py
	- postprocessing.py
	- runall.py
    - [FlexMex_1_2]
        - [001_preprocessed]
        - ...
    ...
    - [FlexMex_2_1]
    - [FlexMex_2_2]
    ...

[meta-data]                (see #96)
    - mapping-input-scalars.yaml
    - mapping-input-timeseries.yaml
    - mapping-output-scalars.yaml
    - mapping-output-timeseries.yaml
    - [output_template]    ('004_results_template' before)
        - Scalars.csv
        - (dir structure for the timeseries)

[data]                     ('001_data_raw' before)
    - [In]
    - [Out]

[comparison]     ('004_comparison' before)
    - compare_scalars.py
    - join_results.py

[oemoflex]
    - oemof-tabular-settings.yml

EDIT 2020/10/16: New names for some dirs. Comparison moved. #96 added.
EDIT 2020/11/16: Move oemoflex-tabular-settings.yml.

Thanks for these ideas. If a restructuring helps getting an overview, we should do it. Let us however first finish the remaining use cases and make the ideas here more precise before we change something.

With one script file per use case, the 'usecases' dir hierarchy becomes even easier:
[usecases]
    - [FlexMex_1_1]
        - [001_preprocessed]
	- [002_optimization]
	- [003_postprocessed]
    - [FlexMex_1_2]
        - [001_preprocessed]
        - ...
    ...
    - [FlexMex_2_1]
    - [FlexMex_2_2]
    ...
    - FlexMex_1_1.py
    - FlexMex_1_2.py
    ...
    - FlexMex_2_1.py
    - FlexMex_2_2.py

To delete a run result, we just have to simply delete one [FlexMex_x_y] dir, without touching the script files. The same is true for keeping a copy of a run. We just have to rename one folder and it won't be touched again by later runs.

We included the thoughts discussed here in #122, therefore we close this issue.

We included the thoughts discussed here in #122, therefore we close this issue.


Use ToLowerInvariant instead of ToLower when forming registrationUrl

Fixes #3329

Thx
herdemir<EMAIL_ADDRESS>schrieb am So., 5. Aug. 2018, 21:12:

Fixes #3329 https://github.com/fsprojects/Paket/issues/3329
You can view, comment on, or merge this pull request online at:
https://github.com/fsprojects/Paket/pull/3330
Commit Summary

Use ToLowerInvariant instead of ToLower when forming registrationUrl

File Changes

M src/Paket.Core/Dependencies/NuGetV3.fs
https://github.com/fsprojects/Paket/pull/3330/files#diff-0 (2)

Patch Links:

https://github.com/fsprojects/Paket/pull/3330.patch
https://github.com/fsprojects/Paket/pull/3330.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/fsprojects/Paket/pull/3330, or mute the thread
https://github.com/notifications/unsubscribe-auth/AADgNF4kq-C4hr8TO4uiN-vPyZTgJAXrks5uN0OXgaJpZM4Vvf3n
.


AED - support turning off/on 3p dungeons

^aed set toggle3p
default is enabled
Either exclude or include the string Multiplayer depending on this setting
Also remember to add this status to ^aed show

Hi I'm maybe doing this


Comment out a debug log line

Whoops! Sorry about that! Fixed in v2.0.3.
Thanks for the help!

My pleasure!


Creating android session

Is it possible to create android session (tgnet.dat) if I have telethon session?

logically it is, however, for now converter does not support such a functionality, maybe in future it will

any update?

No updates for now, however, it might be implemented in future as it sounds as a great feature.

I am happy to tell you that I start to develop this feature now

I am happy to tell you that I start to develop this feature now

Thank you very much. I am looking forward to you completing this task

Hi, this functionality has been implemented in newer version of AndroidTelePorter, I suggest you reading documentation.


[FLINK-2493] Simplify names of example program JARs

The names of the example JARs under build-target/examples have some confusion, the version number might stop users from submitting examples to incompatible versions of Flink
Propose to name the file examples/ConnectedComponents.jar rather than examples/flink-java-examples-0.10-SNAPSHOT-ConnectedComponents.jar.

The CI failure be reported from flink-yarn-tests .but in my local machine, i did many times checking with "mvn clean verify" in flink-yarn-tests, all are successful.
Tests run: 12, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 146.192 sec - in org.apache.flink.yarn.YARNSessionFIFOITCase
Results :
Tests run: 14, Failures: 0, Errors: 0, Skipped: 2
[INFO]
[INFO] --- maven-failsafe-plugin:2.17:verify (default) @ flink-yarn-tests ---
[INFO] Failsafe report directory: /Users/chenliang/flink/flink-yarn-tests/target/failsafe-reports
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:08 min
[INFO] Finished at: 2015-09-04T21:52:47+08:00
[INFO] Final Memory: 36M/758M
[INFO] ------------------------------------------------------------------------


Use an rclpy context manager.

This makes the code cleaner and ensures we cleanup no matter how we exit.

CI:

Linux
Linux-aarch64
Linux-rhel
Windows