s3fs error

I try the example and get the following error:
[ec2-user@ip-172-31-15-95 CommonCrawlJob-master]$ python examples/GoogleAnalytics.py -r local <(tail -n 1 data/some_file.txt)
No configs found; falling back on auto-configuration
Creating temp directory /tmp/GoogleAnalytics.ec2-user.20170726.211320.330067
Running step 1 of 1...
Traceback (most recent call last):
  File "GoogleAnalytics.py", line 12, in <module>
    GoogleAnalytics.run()
  File "/tmp/GoogleAnalytics.ec2-user.20170726.211320.330067/job_local_dir/0/mapper/1/mrjob.zip/mrjob/job.py", line 439, in run
  File "/tmp/GoogleAnalytics.ec2-user.20170726.211320.330067/job_local_dir/0/mapper/1/mrjob.zip/mrjob/job.py", line 448, in execute
  File "/tmp/GoogleAnalytics.ec2-user.20170726.211320.330067/job_local_dir/0/mapper/1/mrjob.zip/mrjob/job.py", line 526, in run_mapper
  File "/usr/local/lib/python2.7/site-packages/ccjob/commoncrawl.py", line 83, in mapper
    for record in self.read_warc(line.strip()):
  File "/usr/local/lib/python2.7/site-packages/ccjob/commoncrawl.py", line 75, in read_warc
    with self.s3.open(keypath, 'rb') as fp:
  File "/usr/local/lib/python2.7/site-packages/s3fs/core.py", line 293, in open
    fill_cache=fill_cache, s3_additional_kwargs=kwargs)
  File "/usr/local/lib/python2.7/site-packages/s3fs/core.py", line 931, in __init__
    self.size = self.info()['Size']
  File "/usr/local/lib/python2.7/site-packages/s3fs/core.py", line 940, in info
    return self.s3.info(self.path, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/s3fs/core.py", line 402, in info
    raise FileNotFoundError(path)
s3fs.core.FileNotFoundError: aws-publicdatasets/common-crawl/crawl-data/CC-MAIN-2017-26/segments/1498128319265.41/warc/CC-MAIN-20170622114718-20170622134718-00000.warc.gz
Step 1 of 1 failed: Command '['sh', '-ex', 'setup-wrapper.sh', '/usr/bin/python', 'GoogleAnalytics.py', '--step-num=0', '--mapper']' returned non-zero exit status 1

Do you have any info on this please?

Hey @2803
So it looks like Common Crawl has updated their S3 bucket from
s3://aws-publicdatasets/common-crawl/ -> s3://common-crawl/ in US East Region
Update your data/some_file.txt so that it points to the actual WARC file and let me know if that fixed the problem.

thanks i will change that!


Update to SDK v3

Opening PR as requested https://github.com/vandium-io/aws-param-store/issues/31#issuecomment-1864222052
Issues that need resolving still:

[ ] the -sdkv3 suffix from the fork needs removing
[ ] what to do about version numbers (keep on v4, or change?)
[x] behaviour of getParameter for a nonexistent parameter name needs to be resolved

merge this

please

I've tested getParameter against a nonexistent parameter in aws, and both the sdk-v2 and the sdk-v3 versions throw a "ParameterNotFound" error, so I don't think that should block merging this. The identity of the thrown error type is obviously different, and it has different details, and that might justify releasing this as a new major version.
(The plural getParameters version also returns a response with both .Parameters and .InvalidParameters in both cases).


Update the quickstart guide

@rlamana @pchico83 are you ok if we do the voting app rename (and if to move it to here or it's own repo) on a different PR?


Configuring default value for traefik.docker.network

Do you want to request a feature or report a bug?
Feature
What did you expect to see?
I was told from the Slack channel that it is not possible to define a default value for traefik.docker.network via toml configuration file.
I request you to add a new configuration parameter in the toml file for setting the default value for the docker network.

Seems to be solved by #3471.
I close the issue.


User documentation

It would be nice to provide an example of user code in the README. If you give me the appropriate pointers into the code, I could come up with an example and make a PR.

@fleimgruber Thank you. Here is the example for Falcon: https://github.com/grktsh/falcon-oas/blob/380921e82a50b565b3df6e494b06cc9dba961db7/src/falcon_oas/middlewares.py#L106
In summary, first, create a Spec instance with oas.create_spec_from_dict() and for each request:

[L107] Create an instance of the subclass of oas.Request.
[L109] Find Operation Object for the request instance. If the media type of the request body is unmet, oas.exceptions.UndocumentedMediaType is raised.
[L115] Create oas.SchemaUnmarshaler instance.
[L122] Unmarshal the request and obtain unmarshaled paramters and request body. If unmarshaling fails, oas.exceptions.UnmarshalError is raised.

@fleimgruber I added the example section to README. If there are unclear points, feel free to reopen the issue and/or make a PR.

@grktsh Thanks for putting this together and sorry for not getting back to this earlier. Will have a look and let you know in case!


make the queue fully nogc and optimize it with a freelist

Since this queue is implemented as a singly linked list, the nodes that are popped can be directly used as a free list.

This isn't really my area of expertise, but I'm interested in seeing some benchmarks nonetheless

For this benchmark:
void main()
{
    import std.datetime.stopwatch;
    import std.stdio;

    void test()
    {
        auto queue = Queue!int();
        foreach(i; 0 .. 1000000)
        {
            foreach(j; 0 .. 100) queue.push(i); // after: for each i > 0 we alloc 90 and recycle 10
            foreach(j; 0 .. 10) queue.pop;
        }
    }
    benchmark!test(1)[0].writeln;
}

before: 8 secs, 616 ms, 333 μs, and 2 hnsecs
after: 6 secs, 847 ms, 188 μs, and 1 hnsec

Now and TBH, i dont know if this is a realistic scenario in D:YAML.

Actually this should be tested with a D-YAML type... int does not lead to add a GC range.

I think it's quite safe to merge this ?

module is dedicated to this queue ( i.e easily reversible)
test suite passes
there will be beta for 0.7.0 where it will tested in actual projects.

I have no objections


Add a path command to CLI

I frequently need to get the path of a selected photo in Photos (path to original) for various testing.  I currently do this by hopping into the osxphotos repl and doing selected[0].path but it would be convenient to have an osxphotos path that just prints out the path to the original image (and maybe a --edited to get the edited path or --preview to get the preview path.   Bonus points if this doesn't need to load the entire database first, just does a query for the selected uuid.

See also #868. Could use colors or dim to show missing path vs path on disk.

By default use selected. But also provide --uuid, --uuid-from-file
Show all paths by default provide --json
Can limit with --path-edited, --path-live-phot etc


Improvement for k8s.io/docs/reference/using-api/api-concepts/

@NicoMarksman Nice catch!

Thanks for the pull request, but the existing text is correct for all APIs contained in a named API group
/close

@liggitt this MR was intended to solve https://github.com/kubernetes/website/issues/35381
but I think that @NicoMarksman misunderstood the problem. I have added more clarification.
This documentation should explicitly state that core resources use /api instead of /apis.


API-less devices aren't findable

device_get_binding(NAME) can't find devices without API structs defined because of
/** @brief Determine whether a device has been successfully initialized.
 *
 * @param dev pointer to the device in question.
 *
 * @return true if and only if the device is available for use.
 */
static inline bool z_device_ready(const struct device *dev)
{
	return dev->driver_api != NULL;
}

This seems like a pretty cheap hack, since all devices do not have a driver API but should still be findable (for instance because they have a power management API). Successfully initialized is NOT equivalent to driver_api != NULL.
I currently have no alternative solution to checking if the device was inited properly, but the current solution is not good.

Ah, I see this has been modified in latest upstream...

Also #28873 is relevant, but I believe this specific concern isn't an issue now.  Re-open if necessary.


legends co work in progress station

station in progress

lets discuss next week about adding it on map load order for Alpha sector

I am happy to let you know that Legends Co Station has been merged during a commit with Invictus https://github.com/Pax-Astralis-Dev/pax-astralis-station/commit/47759777d816d425df9aaeb48c4fa857a17f39ad


⚠ Bofrost.be has degraded performance

In 3e79f78, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 958 ms

Resolved: Bofrost.be performance has improved in 6751b20 after 6 minutes.


V2 : arguments are not passed to the executable

Hello,
It seems that arguments are not passed when using winsw 2.
Everything works well with version 1.
test.zip
I attached an example to reproduce (unzip in C:)
Thanks in advance

Will check it ASAP

Created the PR

Thx for bug fix.
When do you plan to do a new release ?

@sparqueur Done. Release is rather click a button process now, so there is no problem to deliver fixes

Thx, everything seems to work well :)


improve image build times by increasing build machine size

we need to be able to build this quickly and even with a relatively cheap build the default cloudbuild machine type is single core.
custom sizes take longer to start but run faster.

/approve
/lgtm
/hold for Terry

much better, image push step done in ~2m https://prow.k8s.io/view/gs/kubernetes-jenkins/logs/post-oci-proxy-push-images/1643332044014489600


update legacy_zeropoints for DR8

Opening this PR now to track progress, but the goal is to get all the calibration and zeropoint files in working order for DR8, with a data model that includes PLVER, EXPNUM, and PROCDATE for all files, and to address #30, #31, and #35, among other issues.

Just to spell out what I think "all files" means:

CCDs files (survey, annotated; though we should really combine these)
splinesky (merged, unmerged; though I'd prefer to live in a world where the unmerged files did not exist)
psfex (merged, unmerged)
astrom, photom (though we should really combine these, and maybe have already)

Then as calibs and images are read in they should be checked against the CCDs file.  The photom / astrom files should be checked against the CCDs file as their information is incorporated into the CCDs file.

Have tested this version of the code on all three cameras and it seems pretty stable.  Will open a separate PR for any additional corner cases.
Note that this depends on https://github.com/legacysurvey/legacypipe/pull/246 which I'll merge next.


While trying to use InfoWindow getting error 'React.Children.only expected to receive a single React element child'

Hello, I am trying to use the InfoWindow with Marker as mentioned in the documentation but still getting the error. using 9.0.2 version currently?
Can you please tell me what is the problem here? And how do I solve it?
<Marker position={props.latlng} icon={{url:marker,anchor:{x:20,y:20}}} animation={true} onClick={()=>props._onChange}>
      <InfoWindow> hello world </InfoWindow>
</Marker>

Anything ?? @tomchentw

right：

blabla

error:

blabla

right:
<InfoWindow>

blabla

</InfoWindow>

error:
<InfoWindow>
  blabla
</InfoWindow>


⚠ Jenkins Plugins Artifactory has degraded performance

In e8f6bbb, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1108 ms

Resolved: Jenkins Plugins Artifactory performance has improved in dd0ad7b after 27 minutes.


bin: change arbiter init to call arbiter bind upon creation

This is a small quality of life improvement that will eliminate requiring users to to generate bindings upon project initialization. This should touch the code in the binary. Specifically it should add a call to this function
https://github.com/primitivefinance/arbiter/blob/f10fee12b215a80968affa893f438953f7e5d890/bin/bind.rs#L19-L43
To the end of the initialization function here:
https://github.com/primitivefinance/arbiter/blob/f10fee12b215a80968affa893f438953f7e5d890/bin/init.rs#L29-L64

Reminder to edit the instructions in the read me when this change is made

I'm not entirely sure this would be a good change. It's unlikely that as a user I will want the bindings for e.g. Counter.sol to remain in the project once my development environment is setup. With that said, I think we should add an optional flag -t or --template at some point (similar to forge) to allow users to set up their project with different templates depending on use case.

I'm not entirely sure this would be a good change. It's unlikely that as a user I will want the bindings for e.g. Counter.sol to remain in the project once my development environment is setup. With that said, I think we should add an optional flag -t or --template at some point (similar to forge) to allow users to set up their project with different templates depending on use case.

@Autoparallel, What are your thoughts? You suggested this feature.

Couldn't they then just remove the contracts they don't want and arbiter bind again and be on with it?


Update upgrade doc for upgrading from v0.6.2

Currently, kfctl of v0.6.2 does not support -f flag, following exiting documentation in https://www.kubeflow.org/docs/upgrading/upgrade/
Step 2 :
...
kfctl delete -V -f ${CONFIG_FILE}
will throw an error.
The documentation should be modified to download the kfctl of v0.7.0 first.

This change is

/assign @jinchihe

Hey @janeman98 why not just remove -f option in the step 2? if user uses old kfctl client, should also can remove the kubeflow cluster. Thanks.

@jinchihe Thanks for the review!
From upgrading from v0.6.2: remove -f option in the step 2 should work
However, for upgrading from v0.7.0 to future version (say v1.0) : remove -f option in the step 2 will NOT work (since -f is required -- see below)
(v.0.7.0 kfctl)
kfctl delete -V
Error: Must pass in -f configFile
Usage:
kfctl delete [flags]
Flags:
--delete_storage   Set if you want to delete app's storage cluster used for mlpipeline.
-f, --file string      The local config file of KfDef.
-h, --help             help for delete
-V, --verbose          verbose output default is false
Must pass in -f configFile

@jinchihe
Since v1.0 release is not there yet, can we remove -f option in the step 2 for now? When we have v1.0, we can document how to upgrade from v0.6.2 to v1.0 and v0.7.0 to v1.0.
What do you think?

@jinchihe
Since v1.0 release is not there yet, I believe we can remove -f option in the step 2 for now. When we have v1.0, we can document how to upgrade from v0.6.2 to v1.0 and v0.7.0 to v1.0.
What do you think?

I think the upgrading feature will be enhanced and the docs will be rewritten if 0.7 -> 1.0. Since this is hard to say this is upgrading... personally, we just need to consider old -> 0.7 case. Agree?

@jinchihe
Since v1.0 release is not there yet, I believe we can remove -f option in the step 2 for now. When we have v1.0, we can document how to upgrade from v0.6.2 to v1.0 and v0.7.0 to v1.0.
What do you think?

I think the upgrading feature will be enhanced and the docs will be rewritten if 0.7 -> 1.0. Since this is hard to say this is upgrading... personally, we just need to consider old -> 0.7 case. Agree?

Agree. I will remove -f option for now.

@jinchihe
I have updated the file as per your review feedback (+ move description on CONFIG_FILE to step#5). Please review/approve and let me know if further modification is needed.

/lgtm
/approve
Thanks @janeman98


[Docs] Documentation changes related to the feature of resolving env-parameters at server-side while importing an API project

Purpose
With the new changes stated in the following doc [1] the API Controller documentation and examples need to be changed. These changes will be reflected in doc space with this PR.
[1]. https://docs.google.com/document/d/1_pvfEUzI86Fa4280iRtS16pfhEB_3FqznNbvCJKXqlo/edit
Goals
Fixes https://github.com/wso2/product-apim-tooling/issues/540
Approach
Changes in configuring-environment-specific-parameters.md page

Changes in configuring-different-endpoint-types

Related PRs

https://github.com/wso2/carbon-apimgt/pull/9420
https://github.com/wso2/product-apim-tooling/pull/535

Test environment
OS - Ubuntu 20.04 LTS
Java - JDK 1.8_252
APIM - Latest Master branch version

@Chamindu36 , can you please get this PR technically reviewed, so that there after the TW review can be done.


Datadog API

Test with DataDog API. It exposed a problem with ambiguous columns in request and response of the same endpoint. It also uses multiple API keys for authentication.

Requires #25 and #26


Transparent encryption: improve dynamic key management

This improves the handling of rolling updates solving two outstanding issues. First tunnel keys were not being updated causing traffic that had a miss in the ipcache but had a node destination to use the old keys. Fix by ensuring tunnel updates also push updates into datapath when keys change, previously keys were ignored when comparing if old and new configs were the same.
Next add a watcher to the key secret file to allow avoiding Cilium restart in this case. Now the new keys will be consumed on secret update.

This change is

test-me-please

I'm going to drop the inotify part and push just the bugfix for key updates via restart. It resolves a flake I see with IPsec in CI and don't want to hold it up while I resolve the issues with inotify.

test-me-please

Coverage increased (+0.03%) to 42.591% when pulling 172259cf99cb66d09c973e4ccaf2882acdc64521 on rolling-updates-improve into 4b45f062f18e59162fce2871a8f03ea056029589 on master.

test-me-please

Previous failure looks related to DNS,
cmd: "kubectl exec -n default testclient-ngbvg -- curl -s -D /dev/stderr --fail --connect-timeout 5 --max-time 8 http://<IP_ADDRESS>:10080 -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'"" exitCode: 28 duration: 5.137094109s stdout:
time-> DNS: '0.000018()', Connect: '0.000000',Transfer '0.000000', total '5.000879'
stderr:
command terminated with exit code 28


Automatically update docker image

Fixed by updating the syntax used for Verilator waivers to the syntax needed by a newer version of verilator.


Slow performance, BTRFS timeouts

Operating system
Harvester 1.3.1
Description
I ran container in Kubernetes with 2 discs: SSD and HDD via PVC.
The host node is Dell Precision T7820, 128GB RAM, 32 cores Intel Xeon Gold 5218.
vDSM has 4GB RAM, 4 cores.
I decided to copy about ~400GB to HDD. It's very slow. in the logs I see these warnings and errors - see logs.
Overall, the vDSM works very slow.
Is there anything I can do troubleshoot the performance? And idea what the issue may be?
Kubernetes manifest
apiVersion: v1
kind: Pod
metadata:
  name: dsm
  labels:
    name: dsm
  namespace: dsm
  annotations:
    # https://github.com/k8snetworkplumbingwg/multus-cni/blob/master/docs/how-to-use.md#launch-pod-with-json-annotation
    k8s.v1.cni.cncf.io/networks: '[ { "name": "untagged","namespace": "default","mac": "aa:fa:1b:fb:1a:54","interface": "net2" } ]'
spec:
  terminationGracePeriodSeconds: 120 # the Kubernetes default is 30 seconds and it may be not enough
  containers:
    - name: dsm
      image: vdsm/virtual-dsm
      resources:
        limits:
          devices.kubevirt.io/vhost-net: 1
          memory: "5Gi"
          cpu: "4"
        requests:
          cpu: 200m
          memory: "1Gi"
      securityContext:
        privileged: true
        capabilities:
          add: ["NET_ADMIN"]
      env:
        - name: RAM_SIZE
          value: 4G
        - name: CPU_CORES
          value: "4"
        - name: DISK_SIZE
          value: "380G"
        - name: DISK2_SIZE
          value: "2000G"
        - name: DISK_FMT
          value: "qcow2" # qcow2 does not allocate the total disk size, but grows with the data
        - name: DHCP
          value: "Y"
        - name: VM_NET_DEV
          value: "net2"
      volumeMounts:
        - mountPath: /storage
          name: dsm-ssd
        - mountPath: /storage2
          name: dsm-hdd
        - mountPath: /dev/kvm
          name: dev-kvm
  volumes:
    - name: dsm-ssd
      persistentVolumeClaim:
        claimName: dsm-ssd-pvc
    - name: dsm-hdd
      persistentVolumeClaim:
        claimName: dsm-hdd-pvc
    - name: dev-kvm
      hostPath:
        path: /dev/kvm

Docker log
[10681.689074] INFO: task btrfs-transacti:9502 blocked for more than 120 seconds.
[10681.691936]       Tainted: P           O    4.4.302+ #69057
[10681.694104] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[10681.697137] btrfs-transacti D ffff880079dc3998     0  9502      2 0x00000000
[10681.699975]  ffff880079dc3998 ffff880079dc0000 ffff880179290380 ffff88007282e440
[10681.703032]  ffff880079dc4000 ffff88017fd1a4c0 7fffffffffffffff ffff88007282e440
[10681.705415]  7fffffffffffffff ffff880079dc39b0 ffffffff815479c0 7fffffffffffffff
[10681.707545] Call Trace:
[10681.708247]  [<ffffffff815479c0>] schedule+0x30/0x80
[10681.709488]  [<ffffffff81549c22>] schedule_timeout+0xb2/0x140
[10681.711293]  [<ffffffff8104397e>] ? kvm_clock_get_cycles+0x1e/0x20
[10681.713023]  [<ffffffff810b55b4>] ? ktime_get+0x34/0xa0
[10681.714389]  [<ffffffff8154715b>] io_schedule_timeout+0x9b/0x100
[10681.715498]  [<ffffffff815470c0>] ? pci_mmcfg_check_reserved+0xa0/0xa0
[10681.716634]  [<ffffffff815483b0>] __wait_for_common+0xa0/0x140
[10681.717648]  [<ffffffff8107fc70>] ? wake_up_q+0x60/0x60
[10681.718569]  [<ffffffff8154857f>] wait_for_completion_io+0x1f/0x30
[10681.719641]  [<ffffffff812c0179>] blkdev_issue_discard+0x1d9/0x220
[10681.720735]  [<ffffffffa0b907fe>] btrfs_issue_discard+0x17e/0x250 [btrfs]
[10681.722212]  [<ffffffffa0b98cb6>] btrfs_discard_extent+0xf6/0x190 [btrfs]
[10681.723604]  [<ffffffffa0b9c6ab>] btrfs_finish_extent_commit+0xfb/0x250 [btrfs]
[10681.725095]  [<ffffffffa0bbcc38>] btrfs_commit_transaction+0xc08/0xf00 [btrfs]
[10681.726335]  [<ffffffffa0bb9151>] transaction_kthread+0x211/0x290 [btrfs]
[10681.727494]  [<ffffffffa0bb8f40>] ? open_ctree+0x3500/0x3500 [btrfs]
[10681.728571]  [<ffffffff81075297>] kthread+0xc7/0xe0
[10681.729388]  [<ffffffff810751d0>] ? kthread_parkme+0x20/0x20
[10681.730335]  [<ffffffff8154ae1f>] ret_from_fork+0x3f/0x80
[10681.731217]  [<ffffffff810751d0>] ? kthread_parkme+0x20/0x20

[25974.225814] BTRFS warning (device sdc1): commit trans:
[25974.225814] total_time: 598309, meta-read[miss/total]:[371/169041], meta-write[count/size]:[29/10096 K]
[25974.225814] prepare phase: time: 1, refs[before/process/after]:[244/123/0]
[25974.225814] wait prev trans completed: time: 0
[25974.225814] pre-run delayed item phase: time: 1, inodes/items:[59/97]
[25974.225814] wait join end trans: time: 0
[25974.225814] run data refs for usrquota: time: 0, refs:[0]
[25974.225814] create snpashot: time: 0, inodes/items:[0/0], refs:[0]
[25974.225814] delayed item phase: time: 0, inodes/items:[0/0]
[25974.225814] delayed refs phase: time: 0, refs:[2]
[25974.225814] commit roots phase: time: 0
[25974.225814] writeback phase: time: 598308

Screenshots (optional)
No response

Im actually having the same issue and I'm not sure why.

Volume 1 is on a ZFS nvme mirror and I only use this for DSM, apps, databases and indexing.

Volume 2 is a 30TB is on a ZFS HDD pool. I tried mirroring, RaidZ, and RaidZ2, all with and without L2ARC. I also tried both lz4 and zstd compression, disabling atime, changing the block size to 4k to better match the btrfs vdisk.

No matter what I try I end up with numerous btrfs errors in the logs. Speeds of large transfers will initially be ok and then very quickly drop down to about 50MB/s and it's only a matter of time before the container locks up, or the VM inside the container crashes. This often leaves me in a position where I cant kill the container process and have to reboot the entire host.

Relevant blog post about this error:
https://vivani.net/2020/02/08/task-blocked-for-more-than-120-seconds/

To solve this problem, we have to increase the ‘dirty_background_bytes‘ kernel setting  to higher values to be able to accommodate the throughput.

Relevant blog post about this error:
vivani.net/2020/02/08/task-blocked-for-more-than-120-seconds

To solve this problem, we have to increase the ‘dirty_background_bytes‘ kernel setting  to higher values to be able to accommodate the throughput.

Would this all be set on the host? I'm running on Unraid and thoes options are available in the GUI


Structure validation error.

Schema
"responses": {
                    "200": {
                        "description": "",
                        "schema": {
                            "type": "object",
                            "properties": {
                                "booking_events": {
                                    "$ref": "#/definitions/booking_event"
                                },
                                "event_creators": {
                                    "$ref": "#/definitions/event_creator"
                                },
                                "venues": {
                                    "$ref": "#/definitions/venue"
                                },
                                "booking_event_details": {
                                    "$ref": "#/definitions/booking_event_details"
                                },
                                "booking_event_packages": {
                                    "type": "array",
                                    "items": {
                                        "type": "object"
                                    }
                                },
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>      <------------ this section
                                "downloads": {
                                    "type": "array",
                                    "items": {
                                        "$ref": "#/definitions/download_file"
                                    }
                                },
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
                                "highlights": {
                                    "type": "array",
                                    "items": {
                                        "type": "string"
                                    }
                                },
                                "to_brings": {
                                    "type": "array",
                                    "items": {
                                        "type": "string"
                                    }
                                },
                                "to_wears": {
                                    "type": "array",
                                    "items": {
                                        "type": "string"
                                    }
                                },
                                "pricings": {
                                    "type": "object",
                                    "properties": {
                                        "low": {
                                            "type": "boolean"
                                        },
                                        "high": {
                                            "type": "boolean"
                                        },
                                        "total": {
                                            "type": [
                                                "integer",
                                                "string"
                                            ]
                                        }
                                    }
                                }
                            }
                        }
                    },
                    "201": {
                        "$ref": "#/responses/trait:success:201"
                    },
                    "401": {
                        "$ref": "#/responses/trait:success:401"
                    }
                }
........................................
"definitions": {
...........................
        "download_file": {
            "type": "object",
            "properties": {
                "id": {
                    "type": "integer"
                },
                "owner_id": {
                    "type": "integer"
                },
                "active": {
                    "type": "integer"
                },
                "type": {
                    "type": "string"
                },
                "key": {
                    "type": "string"
                },
                "is_main": {
                    "type": "string"
                },
                "name": {
                    "type": "string"
                },
                "filename": {
                    "type": "string"
                },
                "created": {
                    "type": "string",
                    "pattern": "^(2[0-9]{3})-(0[1-9]|1[012])-(0[1-9]|[12][0-9]|3[01]) (0[0-9]|1[0-9]|2[0123])\\:([012345][0-9])\\:([012345][0-9])$"
                },
                "modified": {
                    "type": "string",
                    "pattern": "^(2[0-9]{3})-(0[1-9]|1[012])-(0[1-9]|[12][0-9]|3[01]) (0[0-9]|1[0-9]|2[0123])\\:([012345][0-9])\\:([012345][0-9])$"
                },
                "encrypted_key": {
                    "type": "string"
                },
                "filepath": {
                    "type": "string"
                },
                "path": {
                    "type": "string"
                },
                "url": {
                    "type": "string"
                },
                "exists": {
                    "type": "boolean"
                },
                "size": {
                    "type": "integer"
                },
                "size_string": {
                    "type": "string"
                },
                "extension": {
                    "type": "string"
                }
            }
        },
.........................................
}

Response
{
    "success": true,
   ...........................................................................
    "booking_event_packages": [],
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>    <---------- this section
    "downloads": [
        {
            "Doc": {
                "id": "123",
                "owner_id": "100",
                "active": "1",
                "type": "doc",
                "key": "dg46bv1i36uekrjq",
                "is_main": "0",
                "name": "bookings endpoint",
                "filename": "aus39y0rqvrzxsj8u423.txt",
                "created": "2016-10-17 15:50:50",
                "modified": "2016-10-17 15:50:50",
                "encrypted_key": "S9vLirwnAZB2WoSuVsnfOksSJ96zUdI9EuN31zQ_t1M5dA_W8F4GWOUGDEwGLzSho74ZaSL_9KrpJWM5r1e5osFBlwOrYB6Vyin-F2X4qAVGY57oV8wEPitQ25dGckTt",
                "filepath": "https:\/\/s3-ap-southeast-2.amazonaws.com\/emc-dev\/e0a64-docs\/100\/aus39y0rqvrzxsj8u423.txt",
                "path": "s3:\/\/emc-dev\/e0a64-docs\/100\/aus39y0rqvrzxsj8u423.txt",
                "url": "https:\/\/s3-ap-southeast-2.amazonaws.com\/emc-dev\/e0a64-docs\/100\/aus39y0rqvrzxsj8u423.txt",
                "exists": false,
                "size": 0,
                "size_string": "0 bytes",
                "extension": "txt"
            }
        }
    ],
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
    "highlights": [
        "one",
        "two"
    ],
    "to_brings": [
        "hello",
        "goodbye"
    ],
    "to_wears": [
        "apples",
        "pears"
    ],
    "pricings": {
        "low": false,
        "high": false,
        "total": 0
    }
}

This is the code to validate the above schema and response.
$validator = new Validator
$validator->reset();
$validator->check($jsonResponse, $expectedSchema);
$validator->isValid();

As you can see above, the download section's structure is definately different. In the schema, downloads have an array of object while the response have an object with key name Doc. Thus, I believe that json-schema library should detect this error. But it seems to only detect the data type errors.
How can I detect this type of error? or is this type of error impossible to detect in json-schema library?
Please give me a small clue to resolve this issue.
Thank you.

If you want it to catch the error, make this change to your definition:
"download_file": {
            "type": "object",
            "additionalProperties":false,
            "properties": {

@ygpark2 in an attempt to cleanup this repo we are trying to filter the issues and see which ones might be closed. Is it safe to assume this is a rather old issue, which sadly was left unanswered, and can be closed? Feel free to close it yourself with some comments if helpful.


CMake CI doesn't enable assertions

So it turns out that none of our CMake CI appears to enable assertions. We default to RelWithDebInfo and only set NDEBUG if the build type id Debug: https://github.com/google/iree/blob/914651328acaa0ea508b7cf70b68c3aa764d8321/CMakeLists.txt#L200-L246
First off it seems weird that we're opinionated about the default build type in both Bazel (see https://github.com/google/iree/issues/6140) and CMake, but have different opinions (CMake doesn't have asserts on).
But also, regardless of the default, I'm pretty sure we want assertions for the CI builds.
There's some weirdness here with interactions with LLVM (see https://github.com/google/iree/issues/5750) and the LLVM hides some inappropriate things behind NDEBUG such that unsetting it actually can cause a substantial performance hit, so we'll need to investigate before just enabling everywhere.

Basically fixed by #7482. I'd love to do this (and build modes in general) in a more principled way, but I'm calling this issue fixed.


Remove duplicate 'isStdLib' function

cmd/dep has a function isStdLib which duplicates paths.IsStandardImportPath.
This change moves package paths up to the top level internal package, in order to allow use from cmd/dep, and removes isStdLib (after borrowing the more verbose comment).

I'm struggling to see the difference. Can you expand on what "as sealed-up as possible" means regarding package structure?
Has there been discussion about the logistics of re-exporting? Do you expect to move the whole gps subtree at once? Or just individual members?

Another option would be to drop package paths and just put the function into package gps.

I'm struggling to see the difference. Can you expand on what "as sealed-up as possible" means regarding package structure?

When gps was at github.com/sdboyer/gps, it necessarily had to function as a standalone system, without any "reverse" dependencies on dep. One of the ways that I keep responsibilities straight in my head is by maintaining that line, even after its relocation.
So - functionally, there's no difference. But for now, I think that keeping the subtree sealed - that is, not importing anything from dep outside its own subtree - imposes a discipline on the project that ultimately serves us well. Maybe we change that down the line, but I'd like that to be an intentional plan, not something we back our way into incrementally.

Has there been discussion about the logistics of re-exporting?

No actual discussion yet, but #575 is where it'll be.

Do you expect to move the whole gps subtree at once? Or just individual members?

My expectation is the whole subtree.

Another option would be to drop package paths and just put the function into package gps.

Ah, yes! This would probably be even better. The only reason to protect the identifier via internal the first place was when it was a var, rather than an actual function. The other refactoring you did relieved us of that requirement.

The only reason to protect the identifier via internal the first place was when it was a var, rather than an actual function.

Ah, that was the missing piece for me. I was thinking we still wanted to hide it one way or another.
package gps/pkgtree actually imports paths, so just dropping the function into gps would introduce a cycle. However, there is only a single usage...
func (rm ReachMap) FlattenOmitStdLib() []string {
	return rm.FlattenFn(paths.IsStandardImportPath)
}

...and now that the callers will have access to the function, they could just make the inline call themselves. This would also reduce the surface of the irregular Flatten* api, and generally decouple the two concepts.

Putting the function in gps actually introduces a cycle in pkgtree_test.go which can't easily be resolved via the usual package *_test due to accessing unexported package members. So we're back to gps/paths, but I still thought the Flatten change was sensible.

I think just the small conflict fix, then this should be ready.

Sorry, another conflict :/


Build(deps): Bump @patternfly/react-core from 4.276.11 to 4.276.12 (#5114)

Pull request opened by the merge tool on behalf of #5114

Merged in 3d551271184f457d7b56a84be503bf3ec297a952


Mainnet bootstraps

We currently have bootstrap nodes from:

Boar
Staked
Figment

Waiting for Bison and we'll be ready.

OK, I am going to remove them for now and confirm with Figment.

Waiting for green and will merge.

🔥

Figment seed nodes fixed.


GO:0051990 associated with conflicting (and wrong?) EC numbers

http://amigo.geneontology.org/amigo/term/GO:0051990
(R)-2-hydroxyglutarate dehydrogenase activity
Def line includes "EC:<IP_ADDRESS>" (= "L-2-hydroxyglutarate dehydrogenase") as a source.
Xref line has "1.1.1" (= "Oxidoreductases -> Acting on the CH-OH group of donors -> With NAD(+) or NADP(+) as acceptor")
I think both of these are wrong - I think the correct EC xref should be:
EC <IP_ADDRESS> (= D-2-hydroxyglutarate dehydrogenase)

Tagging myself @hattrill

yes, EC <IP_ADDRESS>
(R)-2-hydroxyglutarate + acceptor <=> 2-oxoglutarate + reduced acceptor
<IP_ADDRESS> is the wrong substrate (L-2 hydroxyglutarate.
And "The enzyme has no activity with NAD(+) or NADP(+), and was assayed in vitro using artificial electron acceptors. Thus, 1.1.1 is not correct either.
Any annotations existing?  The bulk all seem to be IEA types

The TAIR annotations are to EC <IP_ADDRESS>.

I can quickly-ish look at the PAINT family and see all the source annotations made to that term, but  the only way to know whether the source annotation intended to use the text def or the EC # is for each source annotation to be checked. The PAINT tool only knows about the GO term ID, not any corresponding EC #'s.
Once the source annotations have been examined, let me know if you need me to change the PAINT annotation.

D2hgdh | D-2-hydroxyglutaric acid dehydrogenase Flybase @hattrill  can you check
2HGDH | D-2-hydroxyglutarate dehydrogenase, mitochondrial EXP Reactome @deustp01 can you check if made for text def rather than the EC number

Tair already checked; if fly and reactome can check, I will finish this one off. Thanks!

I'll take a look.

Was actually a couple more:
SGD to DLD2
Reactome IDH1
HGNC D2hgdh (Rat Gene) (@slaulederkind  do you pick these up from HGNC?)

Yes, the source is HGNC.

@hdrabkin the FlyBase annotation is to a hydroxyglutarate dehydrogenase activity, matches EC:<IP_ADDRESS>.

The HGNC annotations also match EC:<IP_ADDRESS>

2HGDH | D-2-hydroxyglutarate dehydrogenase, mitochondrial EXP Reactome - made based on this text: "(R)-2-hydroxyglutarate + acceptor = 2-oxoglutarate + reduced acceptor."
@hdrabkin , what's the complaint about IDH1? I see from above that there is one but can't / am too lazy to? figure out what the problem is.

HI @deustp01 ; the complaint was that the term should be xrefs/reed to EC<IP_ADDRESS>, not <IP_ADDRESS> , so wanted to check that the annotations were to the def, which matches  EC<IP_ADDRESS>; I I need to do then is to change the EC to the proper one.

replaced EC:<IP_ADDRESS> with EC:<IP_ADDRESS> in def and xref.


Use d_-> not auto& d = *d_;

auto& d = *d_ in composed operations is a needless complexity whose original intent was to save typing. Instead, it just confuses people who want to study the implementation. Therefore it should be removed. Replace d. with d_->.

Hmmm, this is not so straightforward as I thought. Consider:
d_->s.get_io_service().post(bind_handler(std::move(*this), ec, 0));

This expression will crash since we move *this and then access d_->. Maybe its fine the way it is.

We could rename it, e.g. auto& self = *d_ or auto& self = *data_.

This can't work, it was written that way for a reason.


feat(module-tools): add resolve.alias, to support redirect module id

Summary
When we use alias: { "react": "react-native"} and want to redirect the module id, it failed.
Now we use a new config, which names "resolve.alias" to support it.
In this config, we remove the default alias, and make it easier to understand.
Related Links

Checklist

[x] I have added changeset via pnpm run change.
[ ] I have updated the documentation.
[ ] I have added tests to cover my changes.

If it have be approved, I will add a document to introduce it.


Add DualSense trigger effects

This PR adds DualSense trigger effects to simulate a "click" when mapped to a digital button.
Currently, we reference the DS4Windows code's way to find the corresponding values to send to the controller from the perc_threshold setting for the trigger buttons. We make the trigger click when it reaches the configured threshold, simulating a digital button at some point in the middle of the travel.
The trigger effect is turned on when rumble is turned on. Here is the logic:

If rumble is turned off, don't enable the haptic triggers
Otherwise, look through the adapter mappings for the PS5's PAD_RM and PAD_LM (R2 and L2):

If we find a mapping to an analog axis, skip
If we find a mapping to a digital button, use its threshold to set the trigger's haptic point. If there are multiple mappings to digital buttons, use the highest threshold found.
If there is only one mapping overall for the trigger, don't enable haptics.

The above logic works out nicely, and will feel pretty authentic in most cases.

For consoles with just analog L/R, the triggers won't click in most peoples' configurations.
For consoles with just digital L/R, the triggers won't click by default if there's just one mapping. The triggers can be set up to click by adding an additional digital mapping.
For GameCube, which has both analog and digital L/R on the same button, they will click at the digital threshold point.

Demo video: https://www.youtube.com/watch?v=nUx-Em1jNR8
Fixes #483

My appetite for config format changes is extremely low TBH.
If rumble is enabled just enable this aswell.
If done well this feature shouldn't bother people who aren't bothered with rumble to start with.

Ok it looks all good from my ESP32 board. I'd like to try it on my GameCube internal board, but having a little trouble; I can't get the web firmware to recognize my build as hw2.
I checked the binary with a hex editor and I think the problem is the header doesn't contain hw2.
I'm building with configs/hw2/gamecube.

Nvm,
for file in configs/hw2/*; do echo "$(git describe --always --tags --dirty) hw2 $(basename $file)" | cut -c -31 > version.txt; cat version.txt; cp configs/hw2/$(basename $file) sdkconfig; idf.py build; mv build/BlueRetro.bin build/BlueRetro_$(basename $file).bin; done
Looks like it does the trick.

Still can't figure out how to get the proper threshold for "LM/RM mapped to digital trigger" from ps.c. Will keep looking tomorrow

@darthcloud Okay I think I finally got it.
The logic is as follows. When the controller init:

If rumble is turned off, don't enable the haptic triggers
Otherwise, look through the adapter mappings for the PS5's PAD_RM and PAD_LM:
If we find a mapping to an analog axis, skip
If we find a mapping to a digital axis, use its threshold to set the trigger haptic point

First of thanks a lot for this PR.
I tried it and it's quite cool.
I made a few change, can you please review them?
I retested and it lot fine.

Everything looks good to me!
Looking forward to this!

Actually I just tried it on my GameCube, I have 1 comment

I'm away for weekend, will build and retest when I'm back tomorrow night. But look good. Thanks!

Thanks again!!


Serialize and Deserialize

Basically you just need a way to build up a graph of objects and then output them to some structured storage format (JSON, XML, YAML, whatever).

http://stackoverflow.com/questions/234724/is-it-possible-to-serialize-and-deserialize-a-class-in-c
http://stackoverflow.com/questions/11415850/c-how-serialize-deserialize-objects-without-any-library
http://www.boost.org/doc/libs/1_36_0/libs/serialization/doc/index.html

class Serializable
{
public:
    Serializable(){}
    virtual ~Serializable(){}

    virtual void serialize(std::ostream& stream) = 0;
    virtual void deserialize(std::istream& stream) = 0;
};

struct PersonInfo : public Serializable // Yes! It's possible
{
    unsigned int age_;
    string name_;
    enum { undef, man, woman } sex_;

    virtual void serialize(std::ostream& stream)
    {
        // Serialization code
        stream << age_ << name_ << sex_;
    }

    virtual void deserialize(std::istream& stream)
    {
        // Deserialization code
        stream >> age_ >> name_ >> sex_;
    }
};


Make better use of pinned memory pool

Currently the RapidsHostMemoryStore only knows about the size of the unpinned memory pool, and puts pinned and unpinned memory buffers at the same spill priority. This likely causes pinned memory buffers to stick around longer than necessary when spilling.
This change allows the store to keep total buffer size under the combined size of the pinned and unpinned memory pools, and try to spill pinned buffers first (lower priorities).
Testing on my desktop with UCX enabled on q50, performance increases with a larger pinned memory pool (up to ~60% with 64GB pinned memory), and performs slightly better with the common configuration (8GB pinned, 32GB unpinned).
As far as I can tell this doesn't have any effect when UCX is turned off.
Signed-off-by: Rong Ou<EMAIL_ADDRESS>

build

build

The code looks good to me from a code perspective.  My main concern is with UCX and something like the TPC-DS power benchmark.
For normal operations the Spilled data should be short lived, so using pinned memory before the others, and spilling/freeing it first sounds fine.
For UCX on the TPC-DS power benchmark we will try to run lots of queries one right after the other.  Spark does not quickly release the shuffle data once a query finishes. It relies on the RDDs being garbage collected on the driver. That means with this change we are likely to have all of the pinned memory pool and all of the spill pool holding old stale shuffle data. There will likely be no memory left for operations like a buffer for reading input data. This is likely to slow down those queries because they have to get non-pinned memory allocated directly from the OS. That in and of itself would probably be okay.
But, if we are running under cgroups (on YARN or Kubernetes) then we have now increased memory pressure, and made it more likely that we will be shot for using too much memory, and also made it difficult for a user to predict how much memory we really will use. I think we already have this problem today with UCX because we allocate the entire non-pinned pool and then try to use as much pinned memory as possible first. But because the OS does not actually give you pages until they are used, it is more likely to not allocate some of them from the non-pinned pool at all...
I am probably just blowing this out of proportion, especially with UCX being off by default.

Agree with @revans2. If I understand the change correctly, given that now target size is larger (by including both the pinned and pageable memory) this makes host memory spill less aggressive. If there existed a trigger to spill host pinned memory (no OOM like in the device case) in case where pinned memory is running low that would be different, but we don't have such a trigger, just this limit that we are now raising.

So there are two behavioral changes in this PR:

With the first spilling, instead of trying to get under the size of the pageable memory pool, we now try to stay below the combined size of pinned and pageable memory pools.
Pinned memory buffers now have lower priorities, so will be spilled first.

2 doesn't introduce more pressure on the pinned memory pool, it should lessen it. 1 may possibly lead to us holding onto pinned memory buffers longer since we are spilling less. On the other hand, the previous code spills buffers randomly between pinned and pageable, and the new code is more focused on spilling pinned buffers, so the overall effect may be less pressure on pinned memory.

so the overall effect may be less pressure on pinned memory.

Except in some cases like after a shuffle and before an expensive scan possibly from executing queries one after the other (the power mode of execution in TPCDS is a good example). In this case the store could be full from a prior expensive query, and everything is held since the RDD hasn't been GC'ed, but we are asking for pinned memory when scanning data for the next input, which won't be there, especially now because our target size for the host store is now larger.

Okay, so I think my main concern with this really comes down to
"Do we care about the maximum memory usage and making it predictable?"  For me I do care about this. But UCX is not really something that is on in production right now, and we already have this same problem, just not perhaps to the same degree. I would be fine if we had a separate follow on issue to look at how do we keep the worst case host memory usage within a predictable bound, and if we even care about it in practice.
For the other concern I can see cases on both sides. At this point it really is a matter of tuning and what do we want to be the default tuning to be. I would be happy if we just had a config for the maximum amount of memory that the spill pool could use both pinned and unpinned, or what was the maximum amount of pinned memory it could use. The default can be all of it. The idea is just to have some way to tune it in the future as we try to play around with this. That config could even be hidden to start out with too so users don't need to think about it unless they run into problems.
But to answer the question of how it should be tuned by default we probably need to come up with some suite of benchmarks that we care about so we can decide which is better and which is not for these specific use cases. Without that we are just speculating.  This is probably something we need to do for all of our performance work.

We do already have control over the sizes of both pinned memory pool and pageable memory pool:

spark.rapids.memory.host.spillStorageSize: size of the pageable memory pool, default 1GB
spark.rapids.memory.pinnedPool.size: size of the pinned memory pool, default 0

We don't have a way to fully limit host memory usage though since buffers larger than the size of the pool are allocated directly.

Except in some cases like after a shuffle and before an expensive scan possibly from executing queries one after the other (the power mode of execution in TPCDS is a good example). In this case the store could be full from a prior expensive query, and everything is held since the RDD hasn't been GC'ed, but we are asking for pinned memory when scanning data for the next input, which won't be there, especially now because our target size for the host store is now larger.

The current code doesn't handle this well since the spilling could be mostly on pageable buffers.

If we add the config that @revans2 suggested, I think it would be fine to go in. It is a max for the pool that is smaller than pageable + pinned, but default it to pageable + pinned for now. It gives us the flexibility to override the behavior, while still prioritizing the pinned buffer spilling. Thoughts?

If we add the config that @revans2 suggested, I think it would be fine to go in. It is a max for the pool that is smaller than pageable + pinned, but default it to pageable + pinned for now. It gives us the flexibility to override the behavior, while still prioritizing the pinned buffer spilling. Thoughts?

Maybe add a new spark.rapids.memory.pageablePool.size for the pageable memory pool

I would prefer if this was spark.rapids.memory.host.pageablePool.size since all configs specific to host memory should be under the spark.rapids.memory.host. prefix.

+1 on the config!

+1 for the new config idea

build

build


Fix for titleString & descriptionString in the SliderPage

Looks like a broke two fields of SliderPage when I converted it to Kotlin 😅
both titleString and descriptionString should not have a backing field.
We want those properties to behave as getters and do not have an associated value.
The current implementation was causing several titles to disappear:

Okay, thanks!


Google Drive forces download with RetroAchievements turned on

I have my ROMs in my Google Drive and uses File Stream to mount them in my PC. When I have RetroAchievements turned on, it always forces the download of any game I click to see it's info. Is the search for RetroAchievements being done by hash? If yes, would it be feasible to have an option to just search by game name? Thanks in advance!

Yes, the search is done with a hash before doing it with the name of the game.
I did not add an option but there will be no systematic new hash except during a manual update.


Need documentation for brocade driver

Now that #563 is in, we need to document the brocade driver in docs/network-drivers.md.

Closing.


trying to set permission for camera

Hi I am trying to set permissions for camera but when use to print the response of request is giving me undefined.

@vishalTechnoFreek - please provide more info. Is this ios or android? which versions of RN and this library are you using?


TypeError: Cannot read property 'name' of undefined

What in the world am I doing wrong to cause this to be undefined? https://github.com/form-data/form-data/blob/03444d21961a7a44cdc2eae11ee3630f6969023d/lib/form_data.js#L217
I'm assuming it is referring to value

I think you need to use form instead of formData as the object key. I ran into this using the request-promise module.

@chrisabrams Do you have code snippet that triggers the error? And would be nice to see the error too. Thanks.

@alexindigo I'm also running into the issue. I am also using request-promise and I'm trying to send a local file.
          var formData = {
            apiKey: apiKey,
            version: revisionKey,
            minifiedUrl: jsFilePath,
            sourceMap: fs.createReadStream(mapFilePath),
          };
          var promise = request({
            uri: 'https://upload.bugsnag.com/',
            method: 'POST',
            formData: formData,
          });

I used a debugger and can confirm that the file is being ready properly using fs.statSync(mapFilePath).
@MarkHerhold I still think you need to use formData - all of the other examples I see out there use it.

What I noticed is that the issue comes up if any of the properties that you are setting on the form data has a value of undefined. Guarding against that fixed the issue for me.

Hello, I've ran into this issue as well and after seeing @jordpo 's answer, I checked to see if I was leaving any undefines. Turns out I'm only appending two keys and values and pretty sure it's not undefined. So I'm still wondering why I get this error. I'm sending an ID string and a file:
        var requestLib = require('request');
        var FormData = require('form-data');
	var fs = require('fs');
	var form = new FormData();

	form.append('pid', params.pid);
	form.append('deliveryAttachment', fs.createReadStream(params.deliveryAttachment[0].path));

	requestLib({
		url: url,
		method: "POST",
		headers: {
		     'Content-Type': 'multipart/form-data'
		},
		formData: form
	},function (error, response, body){
              ...
        });

Quick update, I was able to fix my issue by just directly creating an object instead of using the FormData module. I just followed @jordpo 's example and the file was uploaded.
        var data = {
		pid: params.pid,
		deliveryAttachment: fs.createReadStream(params.deliveryAttachment[0].path)
	};
	requestLib({
		url: url,
		method: "POST",
		headers: {
			'Content-Type': 'multipart/form-data'
		},
		jar: getJar(),
		formData: data
	},function (error, response, body){
              ...
        });

@ronaldpabilonia awesome. Yeah from the examples I've played around with, when using the form-data module directly, it's best to call submit on it vs adding it as a value in your request.
formData.submit(url, function(error, result) {
      if (error) {
        reject(error);
      }
      result.resume();

      result.on('end', function() {
        resolve();
      });
    });

I wonder if the request library is expecting an object vs a FormData instance and when it tries to instantiate another instance the key value pairs are undefined.

Also happened to me with NULL values, not just undefined.

null and undefined should probably be treated as empty strings.

Closing as duplicate of https://github.com/form-data/form-data/issues/137
I hope to tackle this soon.


Add support for creating and placing files in folders

This is a known limitation that is pretty serious for things like Android projects which require specific folder structures, but I'm guessing for any app that has more than a handful of files.
A few things that should happen for this support to be first-class:
1 - Enable Add > New Folder command just like any other VS project type (NOTE: this currently DOES show up if you right click on a manually-created folder via the .fsproj!)
2 - When files are added to folder, ensure they are serialized in the right order in the .fsproj (or alternatively remove that weird limitation??)

@KevinRansom Closed by #2692


Add an eventlet based handler

r+ pending CHANGES update

Moving to a new PR so that the change history is clean.


Outsource plotting of ggplots into separate functions

Pull request: https://github.com/miniatureseal/dsci-310-group-10-GCC/pull/70

Merged!


Final code for section 9 does not match the description

Section 9: Trigger network updates
In the final code for SearchRepositoriesActivity private fun initSearch(query: String) is different than the one explained in the tutorial some lines above.
lifecycleScope.launch {
            @OptIn(ExperimentalPagingApi::class)
            adapter.dataRefreshFlow.collect {
                binding.list.scrollToPosition(0)
            }
        }

instead of
lifecycleScope.launch {
            adapter.loadStateFlow
                   // Only emit when REFRESH LoadState changes.
                   .distinctUntilChangedBy { it.refresh }
                   // Only react to cases where REFRESH completes i.e., NotLoading.
                   .filter { it.refresh is LoadState.NotLoading }
                   .collect { binding.list.scrollToPosition(0) }
        }

Also in the final code for the SearchRepositoriesActivity, there is a method included that is not used anymore i.e. private fun showEmptyList(show: Boolean)

Thank you for reporting this!
showEmptyList is only removed a bit later so keeping it there for now.


Review recent changes to V8 public API

It mostly mean to review https://github.com/v8/v8/commits/master/include/v8.h commits and figure our which one are relevant and what we have to implement and create issues for each specific feature or group of features.

At the moment we lack of modules and dynamic import support which recently went stable and also some minor APIs.


has_static_member_data_context<> template is missing

https://github.com/NilFoundation/crypto3-pubkey/blob/88d0a13745815185256eefffc378e5ee40e84352/include/nil/crypto3/pubkey/type_traits.hpp#L44
this template must have existed in the past?

Found the answer:
there is no static member named "context" so BOOST doesn't generate the has_static_member_data_context<> template.
BOOST_TTI_HAS_STATIC_MEMBER_DATA(context)
https://www.boost.org/doc/libs/1_73_0/libs/tti/doc/html/BOOST_TT_idm46513446631776.html
this is working correctly as long as the BOOST macro remains directly above the template:
        BOOST_TTI_HAS_STATIC_MEMBER_DATA(context)
        template<typename T>
        struct is_eddsa_params {
            static constexpr bool value = has_static_member_data_context<T, const typename T::context_type>::value;
            typedef T type;
        };

suggestion for style improvement in crypto3: when using BOOST macros like this one, do not leave a blank line between the macro and the template that depends on the macro.


Suggestion

Support MKV files. Please!

Hello,
Unfortunately this is not possible. MKV container is not intended for online streaming and may contain unsupported codecs (H.254/AAC).
Thank you.


Dont throw error on empty arguments for inputs.boolean

Thanks for the awesome library. Its super useful and well written!
I am having an issue with the following situation:
User makes a request to: GET /api/person?isMale=&name=john
The argument is defined on the req_parser as follows:
req_parser.add_argument('isMale',  type=inputs.boolean)
In this situation the reqparse throws an exception. I am thinking it would be better if it just returned None for isMale.
My reasoning is as follows. If the argument is blank it implies that the person can be either a Male, or a Female and thus the value should not be True or False but it also should not throw an error, as leaving an argument empty is a common practice.
If you agree with this change I will be happy to create a pull request ASAP
Thanks!

I have also added the ability for inputs.boolean to accept integer values 1 or 0

See #471


Loom fails to rename LootTable.pools on forge

When using forge and official mappings, the field pools in LootTable isn't renamed.
This is most likely due to this patch that changes the type of the field.
The main consequence is that forge's ObfuscationReflectionHelper may be broken (breaks with Quark-r2.4-308).

To add to this, there are more fields that match this description. Like KeyMapping.MAP


build: support for asan/tsan/msan/ubsan sanitizers

Modern versions of gcc and clang support various runtime sanitizers that
can be used to find nasty bugs at runtime. Note that not all of these
sanitizers may be active at the same time. However, these tools are aimed
at developers and likely require at least some level of reference during
testing while enabled -- which discouraged me from doing a matrix of
conditionals in autohell.

Well now I'm really confused. This builds for me when I pass no arguments to configure.

Fails here too. This is triggering a dependency problem somehow or a bug.

Issues with autohell fixed in #90. Thanks @gquintard!


Integrated Terminal Scrolls 14 lines

Scrolling in the integrated terminal jumps 14 lines. This happens on the following integrated terminals; cmd, powershell and powershell core.
This is very similar to Scrolling via mousewheel in the terminal scrolls by a lot on Windows (~12 lines) #35111. However, that reports that the jump is only 12 lines and it was fixed over a year ago. It was also referenced here, Integrated terminal scrolls too fast using mouse wheel #47376. However, that was also closed.
I came across the "editor.mouseWheelScrollSensitivity": 0.5 setting, but that does not appear to affect the terminal.
Apologies if I missed any information out as this is my first time posting.
VSCode Version: 1.35.1
OS Version: 1903
Steps to Reproduce:

Open VSCode
Open the integrated terminal. Make sure it is a powershell terminal.
Run the command for ($i=1; $i -le 100; $i++){echo Hello;}
Scroll

Does this issue occur when all extensions are disabled?: Yes

I forgot to mention this is with an external mouse. My trackpad allows line by line scrolling so it is not an issue with the trackpad.

This is the section in the terminal lib that handles the wheel event, AFAICT it's doing the right thing:
https://github.com/xtermjs/xterm.js/blob/c50a6ef41539e6be1317c602e52b490878fdfac8/src/Viewport.ts#L165-L189

Yeah I guessed that editor setting wouldn't do anything but thought it was worth a shot. Looking at the code,
else if (ev.deltaMode === WheelEvent.DOM_DELTA_PAGE) {
    amount *= this._currentRowHeight * this._terminal.rows;
}

It appears that it does intend to jump the whole page in the terminal. But is this normal? In a terminal such as powershell or Windows command prompt, when I scroll normally, I would not expect it to jump a whole page. Normally I would expect it to jump about three lines. Is there any way I could configure it so the scroll jumps are not as big?

@DanJBower but that's only when the mouse wheel is configured set to jump a full page:
ev.deltaMode === WheelEvent.DOM_DELTA_PAGE

https://developer.mozilla.org/en-US/docs/Web/API/WheelEvent/deltaMode

@Tyriar Thanks for the response. I looked up that setting and after a google search, I found this could be changed on my computer settings. However, when I opened the settings, I discovered my mouse was set to jump 3 lines at once. Which explains why CMD and Powershell normally jump three lines for me I guess.

Does this mean that the integrated terminal in VSCode should also be jumping three lines for me?

Yes VS Code's terminal should be jumping 3 lines if those are you settings

But it isn't...?

Upstream issue: https://github.com/xtermjs/xterm.js/issues/1805

@Tyriar Thanks for the help

/duplicate https://github.com/microsoft/vscode/issues/65351


No Conversion Occuring

I've set base rate to USD and trying to convert AED to USD. It seems to not be picking up AED however I believe it should be supported?

@jeppestaerk has updated the code but he didn't updated the release.
Installing the index.js (unzip it) I've attached below by folloing this step would fix your problem.
Alfred Preferences > Workflows > right click on Currency Converter > Open in Finder > copy& paste the file here
index.js.zip

Or you could just install the workflow below which i've added some new features. Read more in Pull Request.
alfred-currency-conversion.zip


Better handle scroll-to-zoom in 3D views

Problem: when radius shrinks, scrolling no longer does much. This makes the user think something is broken.
Solution: switch to dolly when radius is small enough. That way scrolling will always move you closer, one way or another.
Closes https://github.com/rerun-io/rerun/issues/965 (but does not address os-specific scroll/zoom speed problems)
Checklist

[x] I have read and agree to Contributor Guide and the Code of Conduct

Blender just stops zooming, giving you that "I feel stuck" feel.

Well before we let it rest eternally as a draft we might as well merge it if we think it doesn't make things worse 🤷


Update .gitignore

http://stackoverflow.com/questions/27361337/the-project-is-using-an-unsupported-version-of-gradle
ignoring gradle file will leave the cloner with no default gradle.wrapper which will not tell the android studio which gradle version to use
which will end up pointing to false gradle version (unsupported)

@majedev what do you think? what is the best way to do this? Shall we put the gradle-wrapper into git?

@mahdihijazi as per my comment on #3, this should be fine.
@samerzmd thanks!


Npm script fixes

improved gitmod script
fixes for commiting, echoing cmd before git commands, and doing a release PR

fixes #15


Enderdragons purge comand

Pretty self-explanatory, the purge and "set monsters" commands have not been updated to affect enderdragons.
So I'm going to try and fix that :)

Fixed Via pull

Ah forgot about this issue, thanks for the close.


add links to other std.container.* modules

LGTM but we need to think about the horizontal overflow (#798 or #793).
Will we need to manually update this list every time the Phobos module set changes? Perhaps there could be a POSIX Make rule which updates a versioned file (as a workaround to win32.mak being underpowered)..

how did I miss'em?


Question regarding safe search free with label detection

So on the pricing page for google vision there's this section;

Which states that safe search detection is free with label detection, however I cannot see any way using the SDK to make a request for both labels and safe search in one. Doing them separately at the moment, and in the console the stats are saying that there are 2 calls (which presumably we'll get charged for eventually).
Am I missing something here, is there actually a way to do this with the SDK.

You'll need to make a single request like this:
const vision = require('@google-cloud/vision');
const client = new vision.v1.ImageAnnotatorClient();

const request = {
  image: {
    ...
  },
  features: [
    {
      type: client.features.LABEL_DETECTION,
    },
    {
      type: client.features.SAFE_SEARCH_DETECTION,
    }
  ]
}

client.annotateImage(request)
  .then(...

This is super useful @kinwa91, thank you. I highly recommend adding this somewhere in the documentation :)


Windows Support

It would be lovely if I could use this library on Windows, as my driver only comes in dll form.
I'm comfortable with the WinAPI and C, but cython is doing my head in.
Would a potential "easy" fix be using ctypes.dll.LoadLibrary, since that's cross-platform? It seems like there might be some boilerplate behind the scenes to cast function types.
From what I can tell, the current method loads the symbols globally, which makes it "just work". I can't see any reference to the dll itself past the load method.

Hi, yes, at the moment it's globally linked, and was easier from the POV of Cython; and while it does make things like supporting multiple drivers more difficult, there's already other multiplexers in existence, in P11Kit, etc. I did consider just using P11Kit's loader, which would remove all need for custom dlopen code, but it adds another dependency into a thing that people are going to try and run on vintage enterprise Linux.
You're right it loads the functions into the global symbol space. I don't know if there an equivalent for LoadLibrary, we could completely refactor it to use a handle, or we could port to using the P11Kit loader.
ctypes is a different beast, so that's not going to work, but you could simply call LoadLibrary directly. There's a good chance it has a built in wrapper as dlopen does, otherwise it's easy enough to write one.

https://github.com/dlfcn-win32/dlfcn-win32 supports RTLD_GLOBAL and probably could help avoid rewriting _loader altogether.
Otherwise what I think we should do is rewrite loader to produce a struct with all of the functions we need iterated onto it, which could easily use either dlopen/dlsym or LoadLibrary/GetProcAddress (and perhaps support multiple drivers in the future).

I think a full rewrite may be unnecessary, as dlfcn-win32 looks fully featured enough to be a dropin replacement (I didn't know it existed). Making a struct of functions seems closer to re-implementing P11Kit anyway.

Are you comfortable enough to give it a go for the Windows Cython build? I've not tried to build Cython on Windows before and I don't have a Windows development env to try it on.

The build itself "works", just missing dlfcn.h ;) I'll see what I can do - might get some ugly pxds out of it, but it should work.

Turns out, no dice. I got about as far as my own (separate) pure C implementation, only to be hit by the linker wondering where all the functions were actually implemented. I got around that in pure C by generating a .lib for my driver dll, but that kills cross compatibility, as expected.
It seems the only way to be crossplatform is to use some sort of LoadLibrary call or similar.
For the moment I'll move back to PyKCS11 - it has a bug that is preventing me from upgrading to Python 3, but 2 isn't so bad :) Thanks for the assistance!
I'll close this, since "fixing" the import situation is more trouble than it's worth.

I'd like to keep it open, since it's a big missing part, and this is a useful record of what we've tried so far. The first thing I would try is linker flags to delay resolving the symbols. Then we can change to accessing the symbols through a handle.
For reference: https://joscor.com/blog/load-pkcs11-library-dll-in-c-c/

Ok, sounds good! It may be worth noting that the PKCS library has its own function pointer table (at least it seemed that way) which might make grabbing all the functions simpler.

You can create a build package for windows using dlfcn-win32, without need to include as dependency with something like python setup.py bdist_wininst.  Windows system is required for create a build package, but can solve the windows distribution.

Hi, I continue trying to support windows, so I include dlfcn-win32 and try to compile and works to build _loader.pyx, but I have other problem building 'pkcs11._pkcs11' extension
My changes are here https://github.com/danni/python-pkcs11/compare/master...luisza:winbuild
building 'pkcs11._pkcs11' extension
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DCK_PTR=* "-DCK_DEFINE_FUNCTION(returnType, name)=returnType name" "-DCK_DECLARE_FUNCTION(returnType, name)=returnType name" "-DCK_DECLARE_FUNCTION_POINTER(returnType, name)=returnType (* name)" "-DCK_CALLBACK_FUNCTION(returnType, name)=returnType (* name)" -IC:\Users\usuario\AppData\Local\Programs\Python\Python36-32\include -IC:\Users\usuario\AppData\Local\Programs\Python\Python36-32\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" /Tcpkcs11\_pkcs11.c /Fobuild\temp.win32-3.6\Release\pkcs11\_pkcs11.obj
_pkcs11.c
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1163): error C2081: 'CK_NOTIFY': name in formal parameter list illegal
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1167): error C2091: function returns function
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1184): error C2059: syntax error: '<parameter-list>'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1192): error C2059: syntax error: '<parameter-list>'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1198): error C2059: syntax error: '<parameter-list>'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1206): error C2059: syntax error: '<parameter-list>'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1215): error C2061: syntax error: identifier 'CK_CREATEMUTEX'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1216): error C2061: syntax error: identifier 'DestroyMutex'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1216): error C2059: syntax error: ';'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1217): error C2061: syntax error: identifier 'LockMutex'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1217): error C2059: syntax error: ';'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1218): error C2061: syntax error: identifier 'UnlockMutex'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1218): error C2059: syntax error: ';'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1221): error C2059: syntax error: '}'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11t.h(1229): error C2143: syntax error: missing '{' before '*'
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11f.h(24): error C2081: 'C_Initialize': name in formal parameter list illegal
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11f.h(29): error C2091: function returns function
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11f.h(38): error C2081: 'C_Finalize': name in formal parameter list illegal
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11f.h(40): error C2091: function returns function
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11f.h(47): error C2081: 'C_GetInfo': name in formal parameter list illegal
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11f.h(49): error C2091: function returns function
c:\users\usuario\desktop\desarrollo\python-pkcs11\extern\pkcs11f.h(56): error C2081: 'C_GetFunctionList': name in formal parameter list illegal

Looking about this error  I saw that some errors are for MS Visual Studio 14.0 (python 3.6 is build with this )
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-errors-1/compiler-error-c2081
Any idea how to solve this ?

My guess is the problem is here: https://github.com/danni/python-pkcs11/compare/master...luisza:winbuild#diff-2eeaed663bd0d25b7e608891384b7298L26
There's a bunch of #defines that are passed into pkcs11[tf].h, which have different representations on different platforms. One of those files should contain the appropriate representations for Visual Studio.

Fixed in #33

Code which is using python-pkcs11 module is throwing error in windows. Appreciate if anyone can help me here
_pkcs11.pyx
tree = Parsing.p_module(s, pxd, full_module_name)
_loader.c
pkcs11_loader.c(788): fatal error C1083: Cannot open include file: 'dlfcn.h': No such file or directory

@jankiraoupada dlfcn should only be imported on Posix. Are you trying to build in Cygwin? That will lead to whacky type sizes and likely a broken result.

@jankiraoupada dlfcn should only be imported on Posix. Are you trying to build in Cygwin? That will lead to whacky type sizes and likely a broken result.

@jankiraoupada dlfcn should only be imported on Posix. Are you trying to build in Cygwin? That will lead to whacky type sizes and likely a broken result.

I am trying to install a python project (which uses python-pkcs11  -- 0.5.0 ) in my windows 10, which is developed and tested in OSX and linux. I am simply using python virtual environment to setup execution environment, however getting the above error while installing packages.

0.5.0 does not support Windows. You will need to upgrade.
On 17 Nov 2020, 10:47 PM +1100, jankiraoupada<EMAIL_ADDRESS>wrote:

@jankiraoupada dlfcn should only be imported on Posix. Are you trying to build in Cygwin? That will lead to whacky type sizes and likely a broken result.
I am trying to install a python project (which uses python-pkcs11 -- 0.5.0 ) in my windows 10, which is developed and tested in OSX and linux. I am simply using python virtual environment to setup execution environment, however getting the above error while installing packages.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or unsubscribe.

ws. You will need to upgrade.

Thanks Danny, worked after upgrade.


Adding Indicators hides the currentPrice tag

Because SVG z-index is based on the render order, adding the indicators makes the current price tag fall behind the y axis labels.

#538 may handle this with the redesign.


feat: Credential Attribute Label Translation

Summary of Changes
This PR is for the beginning of the integration of the OCA into the Bifold. It allows the functioning of the open PR #577 in the BC wallet.
Related Issues
n/a
Pull Request Checklist
Tick all boxes below to demonstrate that you have completed the respective task. If the item does not apply to your this PR check it anyway to make it apparent that there's nothing to do.

[x] All commits contain a DCO Signed-off-by line (we use the DCO GitHub app to enforce this);
[x] Updated LICENSE-3RD-PARTY.md for any added dependencies or vendored components;
[x] Updated documentation as needed for changed code and new or modified features;
[x] Added sufficient tests so that overall code coverage is not reduced.

If you have any questions to any of the points above, just submit and ask! This checklist is here to help you, not to deter you from contributing!
Pro Tip 🤓

Read our contribution guide at least once; it will save you a few review cycles!
Your PR will likely not be reviewed until all the above boxes are checked and all automated tests have passed.

PR template adapted from the Python attrs project.

You will need to rebase your changes.


Update Cassette Name Check Condition

This PR is to fix the deprecation warning in https://github.com/php-vcr/phpunit-testlistener-vcr/issues/39

I'm struggling to figure out why this would be a problem - there is an empty check right above that code.

I'm struggling to figure out why this would be a problem - there is an empty check right above that code.

I've cleaned up the code a bit to also check for null now.
PHP8.1 shows a warning when the $cassetteName is null which the check couldn't catch.


jdbcUrl for postgres

The MSSQL service provides a jdbcUrl property to the secret (e.g. here, which is missing from the MySQL/postgres services.
Is there a reason for this inconsistency, or would a PR for adding JDBC URLs to the other DBs be useful?

PR would be welcome!

Solved by https://github.com/Azure/open-service-broker-azure/pull/698.


Fix getName() returning null

Removed $commands, as not used
Store the Collector name into $data['name'] as it will be serialized/deserialized too

Thanks @ChristianRiesen for a first in-office review!
Hope @Oliboy50 will have a look like they did for your MR ;-)

I agree with @b-viguier, this MR could break other's code. But the latest version is still not working due to the "name" returned as null.
Should I re-add the property and mark it as "deprecated"?

@thePanz true, but I think v3.0.0 is not broken (if you use it with < Symfony3.4), only v3.0.1 and v3.0.2 are...
anyway, there's a negligible chance that someone is overriding our DataCollector (what would be the point of using a third party bundle which provides dev tools if you override its dev tools?), I think we could dump a v3.0.3 without worrying too much... and it would be less scary for users to update from v2.x to v3.x than to v4.x :)


Python 2.5 compatibility

misc.py in the pipeline module uses the index method on tuples. This method didn't exist until Python 2.6. Would it be all right to use this homemade function instead, to stay compatible with 2.5?

Sure, it's definitely more readable.
I'm kind of new to this git stuff. If you merge this, I guess you'll get my reverted commit into the master branch which seems unnecessary. Is there any way for me to fix this or should I just make a new branch and a new pull request?


CommonFixture.MatmulLargeBlock incorrect output on multichip, N300

Re-enable test (grep for tag #7912)
WH_ARCH_YAML=wormhole_b0_80_arch_eth_dispatch.yaml ./build/test/tt_metal/unit_tests_fast_dispatch --gtest_filter=CommonFixture.MatmulLargeBlock
Result is bad output from test, see: https://github.com/tenstorrent/tt-metal/actions/runs/8856069066/job/24322007219

@tt-dma run eth and non eth on same machine, see if hangs are consistent

Confirmed that this test passes with FD-on-tensix, and fails on FD-on-eth. Nore that this test doesn't hang, just has incorrect output.

what is the command line to run for multichip?

There is a bug in cq_dispatcher that misses sending some bytes at end of page when wrapping to the top of cb buffer.
This is causing the mismatch in large matmul when reading back with ERB. The reason we don’t see it in Tensix dispatch is that tensix buffer is large enough that we do not wrap. On Ethernet, we have to wrap and that exposes the issue.
I have tried a hack, that gets the large matmul to pass with ethernet dispatch.
@pgkeller will have to see what a proper fix should be. It's to do with extra accounting we do in relay_to_next_cb in cq_dispatch.cpp. When packetizing, we count 16 bytes of packet header that has to be added to dispatch payload. This means that of the 4KB dispatch page, we only send 4080 bytes, since 16 are taken by packet header. Later on if we need to wrap, there are still 16 bytes payload remaining in current page. But dispatcher does not send those. It rewinds to start of dispatch cb buffer and ultimately ends up sending 16 garbage trailing bytes in the very last page/transfer. (edited)
After my hack to get to get the test to pass, there was another bug in host code where I was not setting the buffer size correctly for prefetch_h->Mux. The size depends on dispatch type (Tensix Vs Eth). But the code was always setting the buffer size to be 256 KB which is the total size of eth L1. For tests sending enough prefetcher traffic to cause it to wrap, it would wrap eth L1 and overwrite eth fw and cause it to crash resulting in tests hang.
I will push a fix for mux buffer sizing. I’ll try my hack for dispatch_d on ci and see how it goes.
The branch is here: ucheema/issue-7912-debug

Fixes pushed to fd-2/main.
CommonFixture.MatmulLargeBlock re-enabled for multi chip.


Implements the accept/close logic for vsock

Looks like I forgot to add this in the original commit. This now will avoid logging messages about unknown connections on unknown families and give better output.
Ex: "AF_VSOCK connection received from cid: -1 port: 3389"

-1 for clarity:
#define VMADDR_CID_ANY -1U

Hi @jterry75
How can we test xrdp over vsock ?
Thx.

@speidy - Simple VSOCK setup for QEMU
I found this link that might be helpful. The scenario here would be to directly remote to a linux VM hosted in QEMU that might not have networking. IE: The XRDP channel to the guest can be done over VSOCK between the host/guest. The easiest way to do this is to create a VM with QEMU and enable VSOCK as your RDP client destination (using socat or others) and then set use_vsock=true in the xrdp.ini file in the guest. (To change the default port=3389 you can also update the xrdp.ini). Now you can start connecting to XRDP over the host/guest channel and send/receive connections.

+1


QAT callback doesn't work with multi-input models or forward kwargs

🐛 Bug
The QuantizationAwareTraining can only handle models whose forward signature looks like forward(data). Adding additional positional or keyword arguments breaks the forward propagation.
Please reproduce using the BoringModel
The BoringModel doesn't have the right forward signature to produce errors, but I included an equivalently simple model below
To Reproduce
import functools
import torch
import torch.nn as nn

class Simple(nn.Module):
    def __init__(self):
        super(Simple, self).__init__()
        self.conv = nn.Conv2d(3, 16, kernel_size=3)
        self.softmax = nn.Softmax(dim=3)

    def forward(self, a, b, softmax=False):
        """Use 2 positional arguments and 1 keyword argument"""
        feat_b = self.conv(b)
        feat_a = self.conv(a)
        combined = feat_a + feat_b
        if softmax:
            return self.softmax(combined)
        else:
            return combined

    @property
    def example_input_array(self):
        return torch.randn(1, 3, 28, 28), torch.randn(1, 3, 28, 28)

def wrap_quantize_forward_context(model, func):
    """From QuantizationAwareTraining callback"""
    @functools.wraps(func)
    def wrapper(data):
        data = model.quant(data)
        data = func(data)
        data = model.dequant(data)
        return data
    return wrapper

# Works fine without quantization
model = Simple()
input_sample = model.example_input_array
_ = model(*input_sample)
_ = model(*input_sample, softmax=True)

# Simulate the `on_fit_start` hook behavior
model.quant = torch.quantization.QuantStub()
model.dequant = torch.quantization.DeQuantStub()
model.forward = wrap_quantize_forward_context(model, model.forward)
_ = model(*input_sample)
_ = model(*input_sample, softmax=True)

The last two lines will raise the following exceptions
TypeError: wrapper() takes 1 positional argument but 2 were given
TypeError: wrapper() got an unexpected keyword argument 'softmax'

Expected behavior
Forward prop should work fine for both the quantized and unquantized model
Environment

CUDA:
- GPU: GeForce GTX 1060 with Max-Q Design
- available:         True
- version:           10.2
Packages:
- numpy:             1.19.2
- pyTorch_debug:     False
- pyTorch_version:   1.8.1+cu102
- pytorch-lightning: 1.3.0
- tqdm:              4.59.0
System:
- OS:                Linux
- architecture: 64bit
- processor:         x86_64
- python:            3.6.13
- version:           #148-Ubuntu SMP Sat May 8 02:33:43 UTC 2021

Additional context
The solution to the minimal example is to make the decoration wrapper more generic
def wrap_quantize_forward_context(model, func):
    """From QuantizationAwareTraining callback"""
    @functools.wraps(func)
    def wrapper(*data, **kwargs):
        data = model.quant(data)
        data = func(*data, **kwargs)
        data = model.dequant(data)
        return data
    return wrapper

However, I notice that callbacks/quantization.py also includes a second decorator wrap_qat_forward_context, and I am not exactly clear on the intent behind having two separate decorators. I see that wrap_quantize_forward_context is only used in the on_fit_end hook. Any insight here @Borda?
I can make a PR for the above solution, but first I tested it in an actual training by patching my local install
def wrap_qat_forward_context(
    quant_cb,
    model: 'pl.LightningModule',
    func: Callable,
    trigger_condition: Optional[Union[Callable, int]] = None
) -> Callable:
    @functools.wraps(func)
    def wrapper(*data, **kwargs) -> Any:  # <-- Changes to decoration signature

        _is_func_true = isinstance(trigger_condition, Callable) and trigger_condition(model.trainer)
        _is_count_true = isinstance(trigger_condition, int) and quant_cb._forward_calls < trigger_condition
        _quant_run = trigger_condition is None or _is_func_true or _is_count_true
        if _quant_run:
            quant_cb._forward_calls += 1
            data = model.quant(data)  # <-- Triggers AttributeError 'tuple' object has no attribute 'detach'
        data = func(*data, **kwargs)
        if _quant_run:
            data = model.dequant(data)
        return data

    return wrapper

So I tried also expanding the data before passing to model.quant but this goes back to a TypeError
def wrap_qat_forward_context(
    quant_cb,
    model: 'pl.LightningModule',
    func: Callable,
    trigger_condition: Optional[Union[Callable, int]] = None
) -> Callable:
    @functools.wraps(func)
    def wrapper(*data, **kwargs) -> Any:

        _is_func_true = isinstance(trigger_condition, Callable) and trigger_condition(model.trainer)
        _is_count_true = isinstance(trigger_condition, int) and quant_cb._forward_calls < trigger_condition
        _quant_run = trigger_condition is None or _is_func_true or _is_count_true
        if _quant_run:
            quant_cb._forward_calls += 1
            data = model.quant(*data)  # <-- Attempted tuple unpacking, get TypeError: forward() takes 2 positional arguments but 3 were given
        data = func(*data, **kwargs)
        if _quant_run:
            data = model.dequant(*data)
        return data

    return wrapper

@addisonklinke do you suggest integrating it in PL?

Yes, I think it would be reasonable to support multi-inputs and forward kwargs. Many of the models I work with require at least the former. I've been playing around locally with something like this
def on_fit_start(self, trainer, pl_module):
    if pl_module.example_input_array is None:
        raise MisconfigurationException('Example input needed to determine number of QuantSubs for QAT')
    pl_module.quants = [torch.quantization.QuantStub() for _ in range(len(pl_module.example_input_array))]

@functools.wraps(func)
def wrapper(*data, **kwargs) -> Any:
    data = [q(d) for q, d in zip(model.quants, data)]
    data = func(*data, **kwargs)
    data = model.dequant(data)
    return data

I'll start a draft PR soon once I have it more finalized.
Even with the above changes, there is still an error after the fit completes. If I try running a forward pass, the quants attribute is no longer quantizing the input tensors, so then I wind up passing a CPU tensor to a quantized kernel which raises the runtime error described in the PyTorch docs. Could you provide a more detailed explanation of the input_compatible flag? I think that might help me sort out the expected behavior post training - currently I wonder if I'm running inference incorrectly and that the runtime error could be avoided if I took a different approach

Dear @addisonklinke,
Any progress on the draft PR ?
Best,
T.C

Hoping to have it up today or tomorrow

Closing this stale issue because we removed the QAT callback in #16750. As explained in the linked PR:
The QAT callback can no longer be maintained by us. It has many issues that make the callback uneffective and these can't be fixed at the moment.
Users who rely on this callback: You can stay on the Lightning 1.9.x version which gets long-term support (LTS) OR you can copy the callback code and maintain it yourself.
If someone from the community is interested in fixing and maintaining this callback, please let us know. If important issues come up that affect several users, we will prioritize fixes accordingly for the 1.9.x LTS.


Click Event

Hi,
I am using ng2-select and its working very fine. But the thing is I need to use click event. How can I use that. Before Selected event I want to fire Click event. Please help me in this way.
Thanks

I've asked for this too, here: https://github.com/valor-software/ng2-select/issues/555

@Hasnain-Bukhari you can use the (mousedown) event, the problem with (click) is that it requires both (mousedown) and (mouseup) and (mouseup) never has the chance to fire since the element being clicked is replaced on (mousedown).
I'm using (mousedown) to track whether or not the select has been opened, and https://www.npmjs.com/package/angular2-click-outside to tell when it's been closed (referring back to (mousedown) so that I don't get false positives on the close.

That said, it'd be pretty nice if this library could simply emit a custom click event. :)

Hi, I got the same prob but for the time being I have done a quick hack to solve my problem.
Just use
[disabled]="isDisabledTrue"
(focusout)="removeData(item)"
in your element
where ,
removeData() {
const self = this;
self.isDisabledTrue = true;
setTimeout(() => {
self.isDisabledTrue = false;
}, 10);
}
This will handle both outside click situation and multiple ngSelect dropdown issue as well


bug ??

GORM Playground Link

module gorm-test
go 1.19
require (
github.com/go-sql-driver/mysql v1.7.0 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
gorm.io/driver/mysql v1.4.4 // indirect
gorm.io/gorm v1.24.2 // indirect
)
Description

In Mysql id =3, id=5 "submit_at" is null , But the query shows that there is a value and it is the previous one

It is probably that embedded value bugs.
This bugs was fixed by below.
https://github.com/go-gorm/gorm/pull/5901


Question : What's the consequence of not migrating Items to the v5 structure ?

In the v5.0.0-a06 changelog, you explained how Item has to be refactored :
https://github.com/mikepenz/FastAdapter/releases/tag/v5.0.0-a06
Since the library maintains retrocompatibility, nothing prevents me from going on using my v4 items.
I wonder what's the actual consequence of not migrating them ?

@RobbWatershed there are no functional consequences as far as I can think of. the main difference is that you wouldn't have split the element into its logical pieces, which is very important for some people.
So if things work for you, you can keep it as it is :)

That's fine. Thanks for the quick answer !


Optimization of VC UTBot Quickstart wizard

Optimization of  Quickstart wizard to

minimize the actions from user side
extend diagnostic if possible
simplify the procedure for Windows client + WSL2 server case.

close by #302