IRGen: Ensure collocation of relative pointers in resilient witness tables

Ensure collocation by recording the dependence between witness table and
witness before functions are processed. Debug info of inlined function
scopes can reference the witness and will cause the wrong IRGenModule to
be associated before lazy witness tables are processed.
No, I am not sure that this is the only instance of this but the same
solution can apply to other instances if we find them.
rdar://39116991

@swift-ci Please test


chore(docker): create Docker file and dependencies

also add readme explanation for docker build process.

closed by #26


Create GraphConfig class that contains graphId and graphHooks

When instantiating a graph there are multiple things to configure: graphId, graphHooks, graphLibrary, graph view, schema, store properties.
The GraphId, GraphHooks, GraphLibrary and graph view should be bundled inside a GraphConfig class. This should make instantiating a graph a bit clearer. This will result in just requiring 3 things to instantiate a graph:

GraphConfig
Schema
StoreProperties

Merged into develop-1.0.0.


avm.importKey    returns - 404 page not found

Describe the bug
curl --location --request POST '<IP_ADDRESS>:9650/ext/bc/X'
--header 'Content-Type: application/json'
--data-raw '{
"jsonrpc":"2.0",
"id"     :3,
"method" :"avm.importKey",
"params" :{
"username":"YOUR USERNAME HERE",
"password":"YOUR PASSWORD HERE",
"privateKey":"ewoqjP7PxY4yr3iLTpLisriqt94hdyDFNgchSxGGztUrTXtNN"
}
}'
returns - 404 page not found
To Reproduce
The sequence of calls is -
// create user

ret = create_usr(fct_id,fct_pw);

// createAddress
ret = mk_adr(fct_id,fct_pw);
//    printf("\n faucet adr - %s", fct_adr);     fflush(stdout);
// get balance in faucet addr

ret = get_balnc("AVA", fct_adr);
// if no funds - import key and send to
ret = imprt_key(fct_key, fct_id,fct_pw);     <<<--- 404 page not found

Expected behavior
return json ok string
Screenshots
...
Operating System
ubuntu 18.04 ,20.04
Additional context
I  do testing in a local network of nodes when the ava daemon is strated as -
./ava ... --network-id=local ...
By submitting this issue I agree to the Terms and Conditions of the Developer Accelerator Program.

I was also seeing this the first time I launched, but it worked after restarting the daemon. In my case it was for avm.createAddress:
    "jsonrpc":"2.0",
    "id"     :2,
    "method" :"avm.createAddress",
    "params" :{
        "username": "x",
        "password": "x!"
    }
}' -H 'content-type:application/json;' <IP_ADDRESS>:9650/ext/bc/X

Hello Thomas,I did try importing key multiple times - again and again it is failing.
The sequence of commands include creating a user , using pre-created and/or newly created address, getting balance and then import key...
The same behavior is on - ava testnet and ava -local daemon. I use the binary of the 05/31 'denali' release on ubuntu 20.04 .
Paul.
On Friday, June 5, 2020, 8:27:27 PM PDT, Thomas Zarebczan<EMAIL_ADDRESS>wrote:

I was also seeing this the first time I launched, but it worked after restarting the daemon. In my case it was for avm.createAddress:
"jsonrpc":"2.0",
"id"     :2,
"method" :"avm.createAddress",
"params" :{
"username": "x",
"password": "x!"
}
}' -H 'content-type:application/json;' <IP_ADDRESS>:9650/ext/bc/X
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

@nlpvan-paul @tzarebczan
Hi, this is probably happening because your node's X-Chain is not yet bootstrapped.
When it is done bootstrapping (in the newest release) it will print:
INFO [06-09|13:26:09] <X Chain> /snow/engine/avalanche/transitive.go#80: bootstrapping finished with 1 vertices in the accepted frontier
Try running a node with the latest version (0.5.4 at the moment), waiting until it's bootstrapped (< 30 minutes) and trying again to issue API calls.
Let me know how that goes.
Thanks for reporting and for your patience! :)

hi,
the node did actually bootstrap but I use 0.5.0 release.please keep notifying us on new releases.I will download and check 0.5.4 release.
paul.
On Tuesday, June 9, 2020, 10:31:15 AM PDT, danlaine<EMAIL_ADDRESS>wrote:

@nlpvan-paul @tzarebczan
Hi, this is probably happening because your node's X-Chain is not yet bootstrapped.
When it is done bootstrapping (in the newest release) it will print:
INFO [06-09|13:26:09]  /snow/engine/avalanche/transitive.go#80: bootstrapping finished with 1 vertices in the accepted frontier
Try running a node with the latest version (0.5.4 at the moment), waiting until it's bootstrapped (< 30 minutes) and trying again to issue API calls.
Let me know how that goes.
Thanks for reporting and for your patience! :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hi @nlpvan-paul are you still facing this issue after bootstrapping when running the most recent release?

Hi,
I did switch to other projects. I just did download 0.5.7 release and will find time to test. If issues still present I will report it.Paul.
On Wednesday, June 24, 2020, 12:30:34 PM PDT, danlaine<EMAIL_ADDRESS>wrote:

Hi @nlpvan-paul are you still facing this issue after bootstrapping when running the most recent release?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Ok, going to close this for now. Let me know if you have any issues going forward


The first run of SriovKernel2Noop test is unstable

Description
For some reason, it takes a very long time to set up the datapath for the first run of the SriovKernel2Noop.
Requests are working fine.
After the Request, we run a livelinesschecker that pings the endpoint. But since the datapath is not ready, this leads to healing, which now constantly calls Close/Request. Because of this, we cannot create a connection at all.
Turned off this check for a while to look at ci - https://github.com/networkservicemesh/deployments-k8s/pull/8788

Current state
How it worked in previous releases:
- establishing a connection for SriovKernelNoop also took a long time, but then forwarder-vpp turned on by mistake and Sriov mechanisms were not involved. So, it worked as Kernel2Vxlan2Kernel
This bug was fixed in 1.8.0
Checked:

Running the test in the current form - does not work (this is what you need to figure out)
Disable LivelinessChecker - ping starts working after some time. This is a workaround for now.
Restart the test: SriovKernelNoop after SriovKernelNoop - does not work
Restart the test: SriovKernelNoop after Vfio2Noop - works without delay
Deploy the ponger first, and then the client - works without delay
If, after the ping starts working, we introduce violations on the side of the ponger (set the interface down) - the ping does not start working again even after set the interface up.
Manual configuration. The first server has a virtual function that uses the kernel driver. The second is configured as a ponger - works without delay

At the moment, this problem looks like that after an attempt to heal (close) we somehow incorrectly return (restore) the virtual function to the server side.

Manual configuration. Inject a VF to the pod manually. The second is configured as a ponger - works without delay
a. dmesg logs in case of simple Close() (no healing):

[10761.672271] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[10761.672402] IPv6: ADDRCONF(NETDEV_CHANGE): calib67fef27028: link becomes ready
[10768.941673] iavf 0000:8a:19.0: enabling device (0000 -> 0002)
[10769.012997] iavf 0000:8a:19.0: Invalid MAC address 00:00:00:00:00:00, using random
[10769.013261] iavf 0000:8a:19.0: Multiqueue Enabled: Queue pair count = 4
[10769.013667] iavf 0000:8a:19.0: MAC address: 86:ee:3b:26:b6:97
[10769.013671] iavf 0000:8a:19.0: GRO is enabled
[10769.017860] iavf 0000:8a:19.0 ens6f3v0: renamed from eth0
[10769.110078] iavf 0000:8a:19.0 nsm-1: renamed from ens6f3v0
[10769.229324] iavf 0000:8a:19.0 nsm-1: NIC Link is Up Speed is 25 Gbps Full Duplex
[10769.229347] IPv6: ADDRCONF(NETDEV_CHANGE): nsm-1: link becomes ready
[10791.747276] iavf 0000:8a:19.0 ens6f3v0: renamed from nsm-1

b. dmesg logs in case of healing:

[ 8020.309562] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 8020.309668] IPv6: ADDRCONF(NETDEV_CHANGE): calia3bb69036f3: link becomes ready
[ 8028.422319] iavf 0000:8a:19.0: enabling device (0000 -> 0002)
[ 8028.492292] iavf 0000:8a:19.0: Invalid MAC address 00:00:00:00:00:00, using random
[ 8028.492603] iavf 0000:8a:19.0: Multiqueue Enabled: Queue pair count = 4
[ 8028.493129] iavf 0000:8a:19.0: MAC address: 96:c7:84:bf:a9:27
[ 8028.493134] iavf 0000:8a:19.0: GRO is enabled
[ 8028.499188] iavf 0000:8a:19.0 ens6f3v0: renamed from eth0
[ 8028.589131] iavf 0000:8a:19.0 nsm-1: renamed from ens6f3v0
[ 8028.720561] iavf 0000:8a:19.0 nsm-1: NIC Link is Up Speed is 25 Gbps Full Duplex
[ 8028.720593] IPv6: ADDRCONF(NETDEV_CHANGE): nsm-1: link becomes ready
[ 8030.539703] iavf 0000:8a:19.0 ens6f3v0: renamed from nsm-1
[ 8030.553035] ice 0000:8a:00.3: MAC 33:33:ff:de:c6:45 does not exist for VF 0
[ 8030.857268] iavf 0000:8a:19.0 nsm-1: renamed from ens6f3v0
[ 8030.976677] iavf 0000:8a:19.0 nsm-1: NIC Link is Up Speed is 25 Gbps Full Duplex
[ 8030.976705] IPv6: ADDRCONF(NETDEV_CHANGE): nsm-1: link becomes ready
[ 8032.752744] iavf 0000:8a:19.0 ens6f3v0: renamed from nsm-1
[ 8032.760535] ice 0000:8a:00.3: MAC 76:38:5b:de:c6:45 does not exist for VF 0
[ 8032.769228] ice 0000:8a:00.3: MAC 33:33:ff:de:c6:45 does not exist for VF 0

Closed by https://github.com/networkservicemesh/deployments-k8s/pull/8919


prod-preview.openshift.io isn't loading (except incognito)

prod-preview.openshift.io isn't loading  (except incognito)
Gist link : https://gist.github.com/naina-verma/2fe9df96fc9a5f9109b854d5b9ecadd4

Looks fine to me. Did you try to clean the local storage and refresh?

I saw this today too - had to clean the local storage and refresh.
But why is this required ? seems to happen when we done an update - we can't expect users to know how to go clean local storage.

We already have 2 issues tracking the need to fix the logout, #260 and #594, going to close this a duplicate.
@maxandersen to answer your question is has to do with token timeouts and when they expire or don't. Without a functional logout, we can't now clear the tokens. So you must do it manually.


Studies on staging not released to public FTP - GCP000165

study id '6086a786d805de0001cf9125'
Embargo was removed on Thursday 26th October.
Study GCST90016665 is on staging and [edit: not] in curation DB unpublished studies table, but has not been released to public.

@earlEBI this is an Old submission whose Status is not in proper state, For Body of work submission the status should be 'DEPOSITION_COMPLETE' but the status is 'SUBMITTED', the study is not in Unpublished_study table , will manually change the status to 'DEPOSITION_COMPLETE' & it should be picked up by Depo Sync today , worth checking the status of old submissions for Body of work submission while changing the embargo date

Hi @earlEBI could you please help confirm if this is good to be closed ? Thanks

@sprintell looks good, thanks


[Question] Actor updates Q function mean vs min

Hi,
in https://github.com/ikostrikov/walk_in_the_park/blob/40321ecb3561f7be98d73a2c12337a878b0c18e1/rl/agents/sac/sac_learner.py#L127 you update the actor based on the mean over all Q functions.
In the SB3 implementation of SAC, the minimum over all Q functions is used https://github.com/DLR-RM/stable-baselines3/blob/5ef10c8e69b52e1376e6c2c636737d6dd528dda1/stable_baselines3/sac/sac.py#L265
Was this a design decision or are both methods viable?
Thanks,
Jakob

Hi Jakob,
It depends on a specific task and setup. In some cases, the mean is better while in the others the min is better.


Add "unused" prompt type for marathon

I don't think this is necessary anymore with cmty submitted prompts and marathon archives


Typography Link design

Is your feature request related to a problem? Please describe.
The underline for the link is too close to the text
Describe the solution you'd like
Increase margin to underline
Describe alternatives you've considered
As is:

As should be:

Code for the first one:
<Typography link >LINK TEXT</Typography >

Thanks for bringing this to our attention! We’re currently working on the next iteration of EDS and typography is one of the areas where we intend to make some improvements, so I’ll add this issue to the epic.

Some resources for the person who grabs this issue:

https://www.harbortype.com/the-state-of-underlines-and-strikethroughs/
https://developer.mozilla.org/en-US/docs/Web/CSS/text-underline-offset
https://developer.mozilla.org/en-US/docs/Web/CSS/text-decoration-thickness


profiling go programs

suggested via slack by @mholt

+1

:+1:

:+1:

:+1:

:+1:

+1

:100: !!


Warn LXD users with an IPv6-enabed bridge

It took me some time to find out why conjure-up was only giving me the option to choose the docker0 bridge.
By default, when running lxd init, IPv6 NAT will be configured on the the bridge, and conjure-up simply ignore IPv6-enabled bridges.
The solution is here : https://docs.conjure-up.io/stable/en/troubleshoot#common-problems
These two commands:
lxc network set lxdbr0 ipv6.nat false
lxc network set lxdbr0 ipv6.address none

I only added the link in the doc though.
I assume a lot of people will run into this issue, so it's worth adding this in the page.

/assign @kbarnard10

(I signed the CLA)

Hey there! @angristan, looks like you haven't signed the CLA yet. Could I please have you do that? https://github.com/kubernetes/community/blob/master/CLA.md

Should be ok now!

/lgtm
/approve


User mutual playlist

I want to be able to see playlists that have a certain number of songs in common with the one I create so I can find people with similar interests.

Duplicate issue


40ba142b-dffe-4dec-9450-6f99aae81432

this is for automated testing

178dd77c-a10d-4c86-977a-d3d657615f5d


os: os.CopyFS - Infinite loop when referencing to itself

Go version
1.23.3
Output of go env in your module/workspace:

Output of `go env`
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\**REDACTED**\AppData\Local\go-build
set GOENV=C:\Users\**REDACTED**\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\**REDACTED**\go\pkg\mod
set GONOPROXY=**REDACTED**
set GONOSUMDB=**REDACTED**
set GOOS=windows
set GOPATH=C:\Users\**REDACTED**\go
set GOPRIVATE=**REDACTED**
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:/Users/**REDACTED**/go/go1.23.3
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLCHAIN=auto
set GOTOOLDIR=C:\Users\**REDACTED**\go\go1.23.3\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.23.3
set GODEBUG=
set GOTELEMETRY=local
set GOTELEMETRYDIR=C:\Users\**REDACTED**\AppData\Roaming\go\telemetry
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=**REDACTED**
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=C:\Users\**REDACTED**\AppData\Local\Temp\go-build2571252335=/tmp/go-build -gno-record-gcc-switches

What did you do?
Minimal reproductible example:
Create a folder structure like that:
- main.go
- test/
  - loop/
  - file1.txt

package main

import (
	"log"
	"os"
	"time"
)

func main() {
	go func() {
		if err := os.CopyFS("test/loop/", os.DirFS("test/")); err != nil {
			log.Fatalln(err)
		}
	}()

	time.Sleep(time.Millisecond * 200)
	os.Exit(0)
}

What did you see happen?
After running the minimal reproductible example, (I volontarily limitated it to 200ms in a goroutine, because it happen to be an infinite loop), approximately 200 files and folder are created, because the folder loop is copied in itself in an infinite loop.
What did you expect to see?
On windows, if we try to copy a directory in itself, we have a Warning dialog and the other files/folders are still copied.
I should expect that os.CopyFS do the copy but ignore if it is itself. Or return an error, but it could be annoying as we don't know if the rest of the files have been copied.

Reference issue of os.CopyFS: https://github.com/golang/go/issues/62484

Related Issues

os: CopyFS overwrites existing file in destination. #68895 (closed)
os: SameFile should not follow symlinks on windows #21854 (closed)

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

I think this is working as intended, CopyFS won't have the ability to peek into what the given abstract filesystem is, especially if the given filesystem is wrapped.

I understand that, so, definitly the developer has to do a check before.
But this behaviour is not documentated (at least as I saw).
And since it do an infinite loop and will stuck the program, it can also take down the whole server. And for me, this should not happen.

Do you want to send a patch for the documentation?

You can open a pull request on GitHub, or you can use Gerrit.  See https://go.dev/doc/contribute.  Thanks.
I don't think we want to copy to a temporary directory.  That penalizes every caller for a rare mistake.
I think it would be OK for os.CopyFS to do a sanity check if the fs.FS is a os.dirFS.  We could add something like
    if osDirFS, ok := fsys.(dirFS); ok {
        // check for nesting between osDirFS and dir
    }

seems it would be quite a fragile check if someone were to use say fs.Sub

I agree that would be fragile.  I haven't been able to think of any loop detector that would be reliable without preventing valid uses.


fix the handling of PartialResultSet.resume_token

We currently assume that the service only adds a resume_token when the PartialResultSet ends with a completed row (except when the PartialResultSet also completes the stream, in which case the resume_token remains empty).
While this appears to be true in practice (and we create a "PartialResultSetSource reader produced a resume token that is not on a row boundary" error when it is not), apparently we should allow for a resume_token in an incomplete row, and even in the middle of a chunked_value.
See #9359 for more context on the current state.  Internally, see go/cloud-spanner-handling-resume-token.

Will address this when we encounter issues.


[FLINK-14305] Transfer ownership of JobManagerMetricGroup to Dispatcher

What is the purpose of the change
This PR is based on #9827
With this commit the Dispatcher is now responsible for managing the lifecycle of
the JobManagerMetricGroup.
cc @zentol
Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable)

Merged.


Phone number displayed instead of invited user’s name on “Reclaim Payment” screen

Frequency: 100%
**Repro on build version:**Android internal build V1.14.0<PHONE_NUMBER>) , Test Flight build V1.14.0(58)
Repro on devices: iPhone SE (14.2), Redmi Note 8 (9.0), iPhone 6+(12.4.5)
Testing Account:<PHONE_NUMBER> / Backup Key
Pre-condition: Users must have been invited to user B who’s contact sould saved in Device.
Repro Steps:

Launch the application & login with the test account.
Go to “Pending Invitation” In-app CTA & observe user B’s profile name should be displayed.
Tap on the “Reclaim” button.
Now user should on “Reclaim Payment” screen
Observed that,
Current Behavior: Phone number displayed twice on  “Reclaim Payment” screen
Expected Behavior:  Invited user’s name should be displayed instead of the phone number twice.
Impact: Bad user experience.
Investigation: Also it is observed that, in previous builds “Unknown” text displayed as profile name.
Attachment: NameNotDisplayedInReclaimPaymentScreen.mp4

https://valora.testquality.com defect D16.

https://valora.testquality.com defect D41.

Hey @MuckT we have verified the above issue in latest android internal release build and latest iOS testflight build V:1.29.2 and observed that issue is still reproducible
Device : Google Pixel 2XL(11.0) , iPhone 12(14.7.1)
thanks..!!


Wrong version of nlogger in go.mod

Using version v0.4.0 of konfig in a go module enabled project fails with undefined: nlogger.Provider
Checking version 0.4.0's go.mod I can see that version v.0.0.2 of nlogger is stated there but the Provider type was not introduced till v0.0.4.
I manually fixed it by specifying v0.0.4 in my go.mod file but Konfig's go.mod will also need to be updated, I will trying submitting a PR for that.
Thanks for the awesome package.

Should be fixed by #34 . Thank you!


Use .tbd files for pure Darwin builds even against impure runtimes

This will likely be a sweeping change but should make some impure things a lot cleaner and less questionably legal.

Now that TAPI is open source, this is a lot more feasible

still important to me

I believe this issue can be closed? (Seems like GitHub didn't pick up the 'closes x' instruction from the PR description in #98541.)
There's already a separate issue for the bootstrap: #104580


Fix benchmarks since iterate memory pages is expensive

I got some strange results by benchmarking promises with those benchmarks. Performances seemed to be worst after Node.js 10.
I reported the issue on Node and one of the developers finds out: https://github.com/nodejs/node/issues/33384
It's because default page size has been divided by 2 after Node.js 10 so iterate over memory pages with process.memoryUsage() to get the rss is at least 2 times slower.
I fixed the benchmark so they don't take the pages iteration time into account.
I also updated the benchmark results.

Should we merge this one so subsequent benchmarks will be correct?

Should we merge this one so subsequent benchmarks will be correct?

@petkaantonov ?

@petkaantonov ?


Bug in Nav component

Bug is found. :bug:
Link to learn in Header has green color instead of red.
(It's not a bug, it's a new feature :smile:)
Going to catch it.

closed with #94


Revert "Remove unused type variables."

Revert "Remove unused type variables."
This reverts commit 156e38b7ae0e4ea7941dc1d2dbc9b4aeb515f1f6.
This change was not compatible with GHC 8.6.5:
src/Cardano/Ledger/Era.hs:92:25: error:
    Unexpected type ‘_’
    In the default declaration for ‘TranslationError’
    A default declaration should have form
      default TranslationError a b = ...
   |
92 |   type TranslationError _ _ = Void

I don't think we (read: all downstream repos) are ready yet to burn that bridge,
at least not for something so trivial as this.

bors merge


Cant deploy

... presumably due to emoji in note names, I like symbols can we make this work?
Error occurred prerendering page "/💠 KKROMA, AIWARE, & AINIMA/🔑 Key Emergent AI architecture/Key Emergent AI architecture". Read more: https://nextjs.org/docs/messages/prerender-error
Error: [next-mdx-remote] error compiling MDX:
Unexpected character - (U+002D) before name, expected a character that can start a name, such as a letter, $, or _

@Cinemacloud yes probably using emojis will be problematic in the url.

FIXED.


Support spatie/nova-translatable

How to use this with Laravel Nova?

Hi,
I don't use Laravel Nova, so I don't really know how it works under the hood.
But this package works just like any other validation rule.

I think when we use spatie/laravel-translatable, with Nova we have to use /spatie/nova-translatable .
We should be able to use laravel-unique-translation as follow :
Text::make(__('Slug'), 'slug')
->creationRules('unique_translation:posts,slug')
->updateRules('unique_translation:posts,slug,{{resourceId}}'),

But right now it only works with the App locale because the function getAttributeNameAndLocale should explode differently like :
        $parts = explode('_', $attribute);
        $name = $parts[1];
        $locale = $parts[2] ?? App::getLocale();

because of this line  in the nova-translatable package

@pa-bouly  Thanks for the insight.
It seems that the locale won't always be at index 2 tho, if the original attribute name contains underscores. So we need to strip off the first part and use the very last part as the locale.
Would any project use both the underscore and dot notation?
We can either...

check for a valid locale with both strategies and use the first one found
add an option to enable the Nova strategy (project wide)

I think I like the first approach the best, giving the dot notation priority and using the underscore as "fallback".
Any thoughts?

Yes, i like the first approach too, easier to implement and doesn't need one more option to configure.
And this way, a project would be able to use both the underscore and dot notation.
Btw the setMissingErrorMessages will have to change a bit as well I think to be able to have a key like
"translations_{$name}_{$locale}.{$rule}",
Thanks for your quick response ;)

Yes, that would be one additional message to add.
Good catch :)
To wrap it up:
We would require the locale to be either...

the second value when using the dot notation,
or the last value when using the underscore notation (and translations_ would be stripped off).

Some limitations apply when using the underscore notation...

the locale itself cannot contain an underscore, because it's just impossible to know (we can't compare with supported locales).
you can not use dot (array) and underscore notation at the same time; when a dot is in the attribute name, the second attribute part is assumed to be the locale...

Do you think this is acceptable?
At first glance, I don't see any other complications.
I have no way to test this with Nova, so I am relying on your feedback to make this work :)

Yep it looks good to me !
No problem @ivanvermeyen  , i would be happy to help and test it !

@pa-bouly  Just pushed a commit to dev-master.
If you can verify everything works as expected, I'll tag it :)

@pa-bouly I've also updated the readme with a small Nova section.
Is it clear enough? Feel free to add some more info if it's not. :)

I just tested on one of my projects and it works well! :+1:

And the readme looks good to me too, thank you @ivanvermeyen :)

Cool!
Tagged in release 3.2.0 👍


Should I expect object detection on iPhone to be as good as using Turi Create on Mac?

I've gone through the Object Detection tutorial here:
https://github.com/apple/turicreate/blob/master/userguide/object_detection/introduction.md
...and created my own inventory of images (100+ jpgs) to train to recognize a particular object. After a lot of struggling with docs, python learning (including switching between multiple Python environments), etc., I finally was able to generate several object detection predictions that I can visualize in the Turi Create visualizer.
I was very surprised that this worked at all (especially given the object I was trying to recognize).  I got several predictions with very good confidence levels.  This is awesome stuff!!!
However, after exporting the model to CoreML and embedding in a sample app using the same test images, I don't seem to get prediction results with the same high quality - either on the iOS Simulator or an actual iPhone X. The prediction results are OK, but definitely not as good as when run on the Mac. The prediction bounding boxes are not matching the location as well as on the Mac.
I'm using the Swift code in the 'Advanced Usage' tutorial here:
https://github.com/apple/turicreate/blob/master/userguide/object_detection/advanced-usage.md
...to calculate the "best prediction" using the non-maximum suppression example function, etc.
So, without going deep on what exactly could be wrong in my particular setup, I have some basic questions:
Should I expect similar results when run in iOS? Do I need to increase my training image set to a larger number, run more iterations, something else?
thanks in advance
Any info is appreciated.

@dmcgloin Really sorry about the many issues that you've been having. #146 remains a very high priority for us, but the fix needs to go into macOS as opposed to Turi Create, so it makes the process slower.
Perhaps you could provide a Turi Create model and an image, and I can run it both inside Turi Create and using Core ML and we can look at what I get? I apologize for the slow response!

Thanks @gustavla Can I send the model and image(s) privately? My email is in my profile.

@dmcgloin Really sorry for letting this issue be unaddressed for so long. I have received the xcode project that you sent me and tried it out. It didn't seem to trigger any predictions at all. Even if I had seen predictions, I would need to calibrate my expectation by seeing some good results that you described seeing on macOS.
Your question was why are predictions different between macOS and iOS (which they should not be, so we definitely need to investigate that). To test this, did you ever try doing predictions on a set of already taken images and then compared the results? It's hard to do a perfectly fair comparison between macOS and iOS on a live app, especially since the sensors are different. That difference in sensors can also be what is causing a degradation in performance. If you want to target both macOS and iOS with a single model, I would recommend using training data that is a mix of data taken from MacBooks and iPhones.
Since this is such an old issue, please let me know if this investigation is still relevant too! I apologize again for not addressing it sooner.

Hey @gustavla - I very much appreciate you taking the time to investigate and respond. It's been a while - and I honestly haven't had time to revisit this. But I see from one of the WWDC presentations that there are improvements in this area (I believe I saw a new API for getting the prediction bounding box - avoiding some of the intermediary calculations) - and I see your team has put out a few new releases. And it looks like
https://github.com/apple/turicreate/issues/146 is resolved in macOS 10.14 - although I can't yet update my Mac with a beta.
In any case, this is exciting good stuff!
For me, when I get the time, I hope to start again with Xcode 10 and iOS 12 - and properly dive in again.
As such, I think it makes sense to close this issue. I apologize if I've wasted your time on this topic.

@dmcgloin Definitely no wasted time! We're always happy about issues coming in. Sorry it just took me so long to get to it!

@andrewgleave seems that I have the same issue you had (#1016 ).
How you solved the issue in detail? Do you have any hints for me please?

Not really other than what I added above.
The aspect ratio of the external webcam was not the same as the ratio of the other windowed layers which was causing some skewing of the positioning. Looking at #1016 I think it would be useful to have a canonical MacOS example.

@andrewgleave , ok. In my case it's just about iOS (not MacOS app involved). Then probably your solution won't help. Thank you anyway.


PowerShell script for AppVeyor/MSYS2

I am thinking it would be nice to set up AppVeyor to automatically build and test the library with MSYS2 in Windows whenever the martinh master branch is updated.  We have to tell AppVeyor what commands to run, and those commands can either be in the form of an MsBuild project, a PowerShell script, or a Batch script.  This pull request adds an appropriate PowerShell script.
This PowerShell script doesn't necessarily have to be in the repository; you could choose to just copy and paste it into the web interface of AppVeyor instead.  However, putting it in the repository allows us to keep track of its version history and makes it easier for others to use it.
AppVeyor actually has MSYS2 installed by default, so we don't need to do any work to install MSYS2.
Here is an example of the build output that this script would generate:
https://ci.appveyor.com/project/DavidEGrayson/libconfuse/build/1.1.35

Thanks!

De nada. Thank you! :smiley:


OpenCV 3.1.0 sfm module build error

I've run the cmake command to build the opencv_contrib modules with
cmake -D CMAKE_BUILD_TYPE=RELEASE
-D CMAKE_INSTALL_PREFIX=/usr/local
-D INSTALL_C_EXAMPLES=OFF
-D INSTALL_PYTHON_EXAMPLES=ON
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules
-D BUILD_EXAMPLES=ON ..
When I run make -j4, I get errors on building sfm module examples.
**[ 82%] Building CXX object modules/sfm/CMakeFiles/example_sfm_scene_reconstruction.dir/samples/scene_reconstruction.cpp.o
[ 82%] Building CXX object modules/sfm/CMakeFiles/example_sfm_import_reconstruction.dir/samples/import_reconstruction.cpp.o
[ 82%] Linking CXX executable ../../bin/opencv_test_sfm
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `GOMP_parallel_start'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `GOMP_loop_dynamic_start'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `omp_get_max_threads'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `GOMP_loop_end_nowait'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `omp_get_thread_num'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `GOMP_loop_dynamic_next'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `GOMP_parallel_end'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `omp_get_num_threads'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `omp_get_wtime'
collect2: error: ld returned 1 exit status
make[2]: *** [bin/opencv_test_sfm] Error 1
make[1]: *** [modules/sfm/CMakeFiles/opencv_test_sfm.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
/home/shriya/opencv_contrib/modules/sfm/samples/import_reconstruction.cpp: In function ‘int main(int, char**)’:
/home/shriya/opencv_contrib/modules/sfm/samples/import_reconstruction.cpp:36:55: error: ‘SFM_IO_BUNDLER’ was not declared in this scope
   importReconstruction(argv[1], Rs, Ts, Ks, points3d, SFM_IO_BUNDLER);
                                                       ^
/home/shriya/opencv_contrib/modules/sfm/samples/import_reconstruction.cpp:36:69: error: ‘importReconstruction’ was not declared in this scope
   importReconstruction(argv[1], Rs, Ts, Ks, points3d, SFM_IO_BUNDLER);
                                                                     ^
make[2]: *** [modules/sfm/CMakeFiles/example_sfm_import_reconstruction.dir/samples/import_reconstruction.cpp.o] Error 1
make[1]: *** [modules/sfm/CMakeFiles/example_sfm_import_reconstruction.dir/all] Error 2
[ 82%] Linking CXX executable ../../bin/example_sfm_scene_reconstruction
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `GOMP_parallel_start'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `GOMP_loop_dynamic_start'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `omp_get_max_threads'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `GOMP_loop_end_nowait'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `GOMP_parallel_end'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `omp_get_num_threads'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `omp_get_wtime'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `omp_get_thread_num'
../../lib/libopencv_sfm.so.3.1.0: undefined reference to `GOMP_loop_dynamic_next'
collect2: error: ld returned 1 exit status
make[2]: *** [bin/example_sfm_scene_reconstruction] Error 1
make[1]: *** [modules/sfm/CMakeFiles/example_sfm_scene_reconstruction.dir/all] Error 2

BUILD SUCCESSFUL
Total time: 3 seconds
[ 82%] Built target opencv_java
make: *** [all] Error 2
**

Hi,
The same occurred to me. To solve it I had to recompile and reinstall the ceres library. Hope it helps.


add documentation for DPRNG (and user-controllable runtime parameters?)

@neboat @DorothyCurtis @ailiop @timkaler @cleiserson I am curious what you think about creating an entry in our reference documentation for user-controllable runtime parameters, as a place where we would include documentation for deterministic random number generation. More below:
Issues
OpenCilk 2.0 supports deterministic pseudo-random number generation, but the website does not document how to use it.
Improvements
Let's add documentation, perhaps based on this explanation from @neboat

OpenCilk 2.0 supports pedigrees and DPRNG.  Simply link the Cilk program with the additional library -lopencilk-pedigrees to enable pedigrees and DPRNG.  No need to recompile. (PS: Include the header file <cilk/cilk_api.h> to get declarations for __cilkrts_get_dprand(), __cilkrts_dprand_set_seed(), and __cilkrts_get_pedigree(), as well as the __cilkrts_pedigree structure type.)
There are three options for DPRNG using OpenCilk 2.0.

Use the OpenCilk runtime's builtin DPRNG.  To generate a pseudorandom number, call __cilkrts_get_dprand().  A __cilkrts_get_dprand() call returns a 64-bit unsigned integer, although the return value will be one of 2^64-59 possible results.  (You can also reseed that pseudorandom number generator by calling __cilkrts_dprand_set_seed(seed) at the beginning of the program, where seed is a 64-bit unsigned integer.)  The __cilkrts_get_dprand() function is easy to use and fast, but it offers only limited control over the DPRNG.
Use an existing DPRNG library that uses pedigrees.  The only existing library I know of is the DotMix implementation from CilkPub.  We have a copy of the CilkPub code among the OpenCilk repositories, but CilkPub is implemented entirely in C++, so I doubt that's helpful to you right now.
Write your own DPRNG library that uses pedigrees.  The library can read the pedigree of the current strand by calling __cilkrts_get_pedigree(), which returns the pedigree as a singly-linked list.  (Specifically, it returns a __cilkrts_pedigree structure, which contains a rank and a pointer to a parent __cilkrts_pedigree structure.)  This option allows one to flexibly implement a new DPRNG in C or C++, but doing so may be more complicated than you're looking for.

Additional discussion
Where does this documentation fit into the website? It looks to me like reference documentation (not tutorial, user's guide, or discussion). Are there similar items? I propose to wrap this documentation task within a larger task to create a reference page for user-controllable runtime parameters.

Here is a preliminary rough draft:
https://deploy-preview-195--sage-licorice-6da44d.netlify.app/doc/tutorials/tutorial-for-dprng/


Fixed: different results using fill() method with same data

Hello, guys! While I was writing tests for #347, I noticed that fill() method produces different results depending on how we pass data to it. Here what I mean (example from readme.md):
// First, using the locale as array key.
$greece = $country->fill([
    'en'  => ['name' => 'Greece'],
    'fr'  => ['name' => 'Grèce'],
]);

// The second way is to use the following syntax.
$greece = $country->fill([
    'name:en' => 'Greece',
    'name:fr' => 'Grèce',
]);

The difference appears when one (or many) of passed languages is not exists in translatable.locales config array. Unknown languages must be ignored, and they are ignored in example 1, but not in second example. There are some tests to illustrate this issue.

Thank you @askello!


Publish a Docker Image to Public Registry

The current usage instructions require the user to clone this repository then execute a docker build, which builds the operator itself as well as copies in the helm chart. For my own experimentation, I executed the build to create a reusable docker base image. For my operators, I start with that base image, set environment variables, then copy in the chart. Like this:
FROM quay.io/cpitman/helm-app-operator
ENV API_VERSION cop.redhat.com/v1alpha1
ENV KIND SpringBootApp
ENV HELM_CHART /basic-spring-boot
ADD /basic-spring-boot/ /basic-spring-boot

This feels like a much easier way to work with the operator, since many build systems (like Quay) don't support setting build arguments.
Could one of the maintainers setup an official base image, with the required hooks to keep it up to date?

@cpitman I definitely agree. We're planning to migrate this operator into the operator-framework/operator-sdk repository so that new helm operators can be created with the operator-sdk binary. Part of that migration will very likely include what you've described.
Run this command with the latest operator-sdk binary to get a feel for how it currently works for the ansible operator:
operator-sdk new my-spring-boot --type=ansible --api-version=example.org/v1alpha1 --kind=SpringBoot

Awesome, I am trying out the Ansible operator as well. I'll close this for now.


#4134 Complete the Query DSL doc

Description
Complete the Query DSL doc to make sure it starts by query
How was this PR tested?
Describe how you tested this PR.
The markdown table is formatted by VSCode.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thank you @tuziben


Add backend parameters to native annotation

[@chochos] The native annotation is incomplete; there is a lot of stuff in the language module annotated native which can be compiled to js, so it's really only annotated for the jvm compiler.
So the annotation should have a parameter which is the list of backends that the declaration is native for; no args means it's native for all backends.
Question is: should the backends be objects, or simply strings?
i.e.
native("jvm") class Bla(){}
native(jvm) class Ble(){}

we can define a Backend or Platform class with two cases, jvm and javascript. It can even be used in the process object, which I think already has a platform property.
[Migrated from ceylon/ceylon.language#252]

I guess we can consider this done :)


Exclude packages that don't work in the OSS build.

Allows the //iree/... tests to work.

How this this approach compare to using .bazelignore? I just added https://github.com/google/iree/blob/master/.bazelignore for hal/dawn (but didn't test if that actually works)

Appears to be two ways to basically say the same thing. I'll switch to .bazelignore (which is working)


i18n internationalization

Is your proposal related to a problem?
There is currently no support for internationalization in Jackson
Describe the solution you'd like

Support i18n using best practices for React and next.js.
Figure out a good solution for supporting translations of string used in the app.

hey i like to take on this issue.


rxjs 7 - ideas to be taken from the fastest and smallest alternative

I was just acquainted with the Rxjs 7 design document. I was wondering if you knew about this library which is the equivalent to Rxjs of what Preact is to React, i.e. same API but faster and smaller : https://github.com/langhuihui/rx4rx/tree/master/stdlib
Last time I checked it, the library was 4k. There is also a speed-optimized version based on classes (and that one happens to have a README in english) : https://github.com/langhuihui/rx4rx/tree/master/highlib. Speed-wise it beats anything.
I am bringing to your knowledge under the hypothesis that you may find there some extra ideas (once you get past the chinese language - the code reads as usual) to reduce size and increase speed of rxjs.

Thanks for pointing this out. I've seen this one before. It's really not great that this person decided to take the "rx" name, honestly. It implies some sort of affiliation.
Myself (and I'm sure most of the core team) have rolled tons of small toy version of reactive libraries featuring things like observables. You can make them really, really small. But there are a TON of trade-offs. The library in question here has no handling of errors in user-provided functions, no guaranteed resource tear down (memory leaks if you mess up), no accounting for producer interference on shared sources, etc. RxJS is trying to provide a more comprehensive, production-ready set of functionality.
That said, I do love looking at these things (and rolling my own, over and over again). RxJS, by version 8, from all experiments, should be substantially smaller (~50-60% of the size) while retaining all it's current safety and features.

Thanks for a quick answer Ben. Yes, I totally believe that rxjs has a set of requirements that contrains it. I was brought to send you this after reading the v7 document because, with this library:

it is damn easy to add new operators
this shows classes (in highlib) can get you faster (at least with V8) implementation than a functional approach (in stdlib). Most.js also use classes among other things to get its superlative speed

I could not figure out however if schedulers could be added on top rather than within the observable design. Just like most.js this library does not seem to do much in the sense of scheduling.
Anyways, you already looked at it, and came to some conclusions so all is good.


WebSocket Connection Error with Gunicorn and Multiple Workers

After mounting Chainlit to the FastAPI application, if the Gunicorn command is used to start the application and multiple processes are specified (using the -w option), an error will occur. The startup command is: gunicorn -w 2 -k uvicorn.workers.UvicornWorker main:app --bind <IP_ADDRESS>:3010
bug description
If the method based on FastAPI provided on the Chainlit official website (https://docs.chainlit.io/deploy/api) is used to mount Chainlit, and then the application is started using the Gunicorn command with multiple processes specified, such as: gunicorn -w 2 -k uvicorn.workers.UvicornWorker main:app --bind <IP_ADDRESS>:3010, an error will be encountered on the web UI stating "Unable to connect to server".
Additionally, the console will display the following errors:
[2024-09-24 11:03:23 +0800] [10116] [INFO] connection closed
[2024-09-24 11:03:24 +0800] [10116] [INFO] ('<IP_ADDRESS>', 63124) - "WebSocket /chat/ws/socket.io/?EI0=4&transport=websocket&sid=TVH1xbI1HXjNDocwAAAK" 403
[2024-09-24 11:03:24 +0800] [10116] [INFO] connection rejected (403 Forbidden)
[2024-09-24 11:03:24 +0800] [10116] [INFO] connection closed
[2024-09-24 11:03:25 +0800] [10116] [INFO] ('<IP_ADDRESS>', 63129) - "WebSocket /chat/ws/socket.io/?EI0=4&transport=websocket&sid=i_R_YcNkA9SIqMBuAAAc" 403
[2024-09-24 11:03:25 +0800] [10116] [INFO] connection rejected (403 Forbidden)
[2024-09-24 11:03:25 +0800] [10116] [INFO] connection closed
Chainlit mounting code:
app = FastAPI()
@app.get("/")
def read_root():
    return {"message": "Hello, World!"}

mount_chainlit(app=app, target="src/chainlit_app.py", path="/chat").

software information

chainlit：1.1.402
python：3.8.5
Browser：google chrome
OS：windows or linux

@dosu Could you suggest a more concise title for the issue?
@huoxiao-C Do let us know whether the bot helped you out!

hello dokterbob，the bot method is not ok，we still cannot specify multiworker with gunicorn

@huoxiao-C Sorry it took a while.
Could you perhaps increase logging to provide a full traceback for the 403? Also, make sure you're on the latest version (we improved some things around error handling).
We need to know the exact code generating this error.
Should work roughly like this: https://search.brave.com/search?q=fastapi+logging+of+tracebacks&source=web&summary=1&conversation=7c7357e15d8a0fc3f88da7&summary_og=bb464324cb55a1e106371f
Could you also double check (if you haven't already) that -w 1 doesn't generate the error?

@dokterbob hello dokterbob, thanks for your reply
1）There is no problem with setting the number of Gunicorn workers to 1.
2）I tried the method you mentioned, the message is as flows. The web UI still show that connection to the server failed，

and the frontend server shows

。
This is my testing code
main.py
import traceback
from chainlit.utils import mount_chainlit
from fastapi import FastAPI, HTTPException
from fastapi.exceptions import RequestValidationError
from fastapi.responses import PlainTextResponse
from starlette.exceptions import HTTPException as StarletteHTTPException

app = FastAPI()

@app.exception_handler(StarletteHTTPException)
async def http_exception_handler(request, exc):
    print(f"Exception: {exc}, Traceback: {exc.__traceback__}")
    print(f"new Traceback：", ''.join(traceback.format_tb(exc.__traceback__)))
    return PlainTextResponse(str(exc), status_code=exc.status_code)

mount_chainlit(app=app, target="cl_test.py", path="/chat")

cl_test.py
import chainlit as cl
@cl.on_chat_start
async def on_chat_start():
    print("on_chat_start调用")
    guide_text = """Hi，我是DataAgent\n"""
    await cl.Message(guide_text).send()

app run command
gunicorn -w 2 -k uvicorn.workers.UvicornWorker main:app --bind <IP_ADDRESS>:3010
software information
os：linux
python：3.9.20
chainlit：1.3.2
gunicorn：23.0.0

@huoxiao-C Sorry for the delay. Having learned more about the code, there is no persistence in sessions (ref #1513). Whatever chainlit stores about the session is thread-local, in memory.
That means that the only way to scale is to use sticky sessions supplied by the load balancer.

Would love feedback on #1575, please see if it can resolve your issue. We're also open to a suggested approach on cookie based sticky sessions.


:seedling: add verify-crd-compatibility make target and GHA

using the crd-diff tool with the same configuration as recently merged into operator-controller

/hold

v1.0.0 has been released
/hold cancel


Commitment refactoring

@jelysn the software upgrade test is throwing this error:
[alice] panic: unknown field "total_committed" in types.Params

see here https://github.com/elys-network/elys/actions/runs/8576567644/job/23507748365#step:6:67


Sound problem

Platform (Hardware/OS): Raspberry Pi 3 B
Node version: v10.4.0
MagicMirror version: 2.4.0
Module version: 1.1.0
Description of the issue: I enable the sound output by hdmi but the alarm sound is not working. I can see the alarm on screen. I configure the Fadetimer to 1sec (my sound is 6sec). But, no sound comes out of the speaker.
Any idea ?

Did you try to play other sounds on your pi, to see if it's a general issue with your setup or this specific module?

Hi. I think I found the problem…
I use the text to speech from Google. In my browser, i work pretty well. But with the module, I've got an error (can't get the audio file).
cf. http://translate.google.com/translate_tts?tl=en&client=tw-ob&q=Your text here
If I download the audio file and use it with "my_file.mp3", it works pretty well too!

If you inspect this page it is a video player, that's probably the issue. Try a different service or the local file.


App window does not account for windows scale

Image clips out of the window

Resetting zoom without restarting the app results in the window continuing to be magnified


I pmed you on reddit. can you help me

HI your themes and stuff are awesome and your awesomewm themes awesome. I need some help configuring stuff on your setup if you can help me. I pmed you on reddit with the stuff i need help with getting to work. your anonymous-theme is awesome and your config

Hi, yes, i'm not all times on reddit (only few hours when i post a new theme :p) and thanks to report this issue.
So for the configuration, you have to edit some scripts.
Mails
You need to edit the files ~/.config/awesome/widgets/email.sh, the script use offlineimap to retrieve new mails all the 5min.
And execute a ls <mail_directory> | wc -l to count the new mails if any.
After If you have an other way to get your mail, you can edit the script and comment all the part concerning offlineimap.
# if [ -x $offlineimap_bin ] ; then
#    $offlineimap_bin 2>/dev/null
#    if [ $? -ne 0 ] ; then
#      echo "$(date) - failed to get email" >> $log_file
#      exit 1
#    fi
#  else
#    echo "$(date) - $offlineimap_bin no found" >> $log_file
#    exit 1
#  fi

Battery
The battery use the script ~/.config/awesome/widgets/battery.sh and look at /sys/class/power_supply/BAT0/uevent. You have to edit the script to correct the good path.
Volume
The volume use ~/.config/awesome/widgets/volume.lua with ALSA, you have to edit the variable local volume_script = [[ ]] to match with your need.
I use aplay -l to get the card name:
$ aplay -l
card 2: Pro [SB X-Fi Surround 5.1 Pro], device 0: USB Audio [USB Audio]
Subdevices: 0/1
Subdevice #0: subdevice #0

So here, my card is card 2: Pro, and the matching variable volume_script = [[ bash -c "amixer -c Pro ...
Also, you can get the volume with mpc but i don't have the same result.
TOR

Also i do not use tor so how can switch it with firefox

The widget don't use the tor-browser but directly tor with iptables, the widget only search if my ip address use tor so i can't replace it with any web browser.
You can disable the widget by commentting 2 lines in the file ~/.config/awesome/bar_themes/anonymous.lua with --, just change:
12    local tor = require("widgets.tor")
178   tor_widget,

To:
12   --local tor = require("widgets.tor")
178  --tor_widget,

Restart awesome and the tor widget is disable.
Tell me if something is not clear or any suggestions.

Thanks for getting back to me with configuring your setup its awesome :p.
I still need help with the mail script. i don't use offlineimap i usually use web based email. Gmail how can i set it up I'm lost on what to do.

I'm unable to find out the battery. i run the desktop  i added this to it. /sys/class/power_supply/AC/uevent

You can check here,
with only the .offlineimaprc  and at offlineimap doc but you have to configure gmail to allow imap on other email clients first.

I will enhance the widget for the volume and the battery (try to detect more paths) but the widget for the mail will be hard, a manual configuration is required.

the widgets for volume and battery should work now if you can confirm.

WOW you are awesome Thank you very much you ROCK!
The powersupply is good now it shows plug which is true. The volume still says 0%

another small thing by any chance did you change the terminal colors. I thought they where purple before. now there just white and back

i found my sound with the command you gave me. I'm just lost what do i do next add it where.
card 0: I82801AAICH [Intel 82801AA-ICH], device 0: Intel ICH [Intel 82801AA-ICH]

the sound should work now, i have missed to add the card 0 :), else your card name is I82801AAICH.
About the color more purple, I have test but i do not see any difference (and i didn't touch the colorscheme) so it's weird.

so do i have to re add something or update something

If the volume and the battery works, it's good for this issue

I just added like this
28 ..widgets/audio.sh volume I82801AAICH", 4,
you said you fixed it by. the sound should work now, i have missed to add the card 0 :), do i have to re add something when you did that run a command

so this is good, thanks and don't hesitate to post an issue if you find another problem.

you said you fixed it by. the sound should work now, i have missed to add the card 0 :), do i have to re add something when you did that run a command

just reload awesome

This is what i have.
https://imgur.com/5fwBKEX
I'm missing the last box for the music.

have you installed mpc?

yeah i installed mpc, mpd , ncmpcpp. but the symbols play, next pause do not show

Ok, can you post the output of:
$ mpc status

volume: n/a   repeat: off   random: off   single: off   consume: off

Ok nice, i have the same error when my playlist is void, i work on

how do i get the buttens up play,pause next

wait a bit :p, i prepare a commit to solve this

oh ok Thanks so much you have been very helpful your awesome. Do you have any blogs or websites to follow you on. :). I have a youtube channel and i'm going to give you best shoutout ever :)

It should be solved now with 30fe8b1, thanks.
Yes i have a little blog at https://szorfein.github.io/ , but i have to enhance the css for large screen when i have the time :)

yep i see them now. Thanks.
can mpc ,mpd play .m4a itunes files

of course :)

for some reason i'm having trouble playing music. its not going

you have to use ncmpcpp to create the playlist

how do i create it in playlist

First, you have to configure mpd to read your music directory, look the manual too man mpd and man ncmpcpp, all is explain.
When you see your musics with ncmpcpp, just add them with enter or space.

i already configured mpd its just added them to playlist. and not sure if right its not playing... bash bash

i added them there still not playing hmm

i'm getting this now
ERROR: Failed to open "Headphones" (alsa); Failed to open ALSA device "plug:bs2b": No such file or directory

you can comment these lines from the mpd.conf
# name		   "Headphones"
# device		 "plug:bs2b"

OK. i figred it out. i had to change the audio output in mpd.conf what you had to pulseaudio. it plays now
two things wrong the puease symbol isn't coming up and the playingtrack in ncpmpp how there supposed to be dots thet squears.
https://imgur.com/UA2E8gV

these squear in ncmpcpp should come from your locale, you probably have to use utf8, what you use?
$ env | grep -i lang

Also, i don't know why you have theses symbol near the tasklist, maybe a wrong awesome version, i use awesome 4.3

Also, i don't know why you have theses symbol near the tasklist (the alert icon), maybe a wrong awesome version, i use awesome 4.3

what do you mean? these symbol near tasklist..
Mines, en_US.utf8
i will check my awesome version.
well as i was doing this i put en_US.utf8 in the mpd.conf and the play and puase buttens are working now.
I just still see those sqeaurs in place of dots...

I'm running aweosme 4.3 also

I just still see those sqeaurs in place of dots...

Check your locale:

https://bbs.archlinux.org/viewtopic.php?id=166531
https://wiki.archlinux.org/index.php/Locale
https://superuser.com/questions/509950/why-are-unicode-characters-not-rendering-correctly
https://wiki.gentoo.org/wiki/UTF-8

And either on using the right font: nerd font iosevka for the theme-anonymous.

yeah my locale is this en_US.utf8. I'm using the right fonts. i installed. i checked with gucharmap and its saying that symbol is DejaVu Sans?
Also the volume isn't showing up anymore

Also the volume isn't showing up anymore
Your card is always card 0: I82801AAICH [Intel 82801AA-ICH], device 0: Intel ICH [Intel 82801AA-ICH] ?

Also, can you post the output of:

$ locale
$ amixer -c 0?

i need more informations to help us.

yes i'm using pulseaudio for mpd if thats what you mean? other then that thats my sound i gave you.
➜ locale         LANG=en_US.utf8 LC_CTYPE=en_US.utf8 LC_NUMERIC=en_US.UTF-8 LC_TIME=en_US.UTF-8 LC_COLLATE="en_US.utf8" LC_MONETARY=en_US.UTF-8 LC_MESSAGES="en_US.utf8" LC_PAPER=en_US.UTF-8 LC_NAME=en_US.UTF-8 LC_ADDRESS=en_US.UTF-8 LC_TELEPHONE=en_US.UTF-8 LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=en_US.UTF-8 LC_ALL=
➜ amixer -c 0 Simple mixer control 'Master',0   Capabilities: pvolume pswitch   Playback channels: Front Left - Front Right   Limits: Playback 0 - 31   Mono:   Front Left: Playback 11 [35%] [-30.00dB] [on]   Front Right: Playback 11 [35%] [-30.00dB] [on] Simple mixer control 'Master Mono',0   Capabilities: pvolume pswitch   Playback channels: Front Left - Front Right   Limits: Playback 0 - 63   Mono:   Front Left: Playback 0 [0%] [-94.50dB] [off]   Front Right: Playback 0 [0%] [-94.50dB] [off] Simple mixer control 'Headphone Jack Sense',0   Capabilities: pswitch pswitch-joined   Playback channels: Mono   Mono: Playback [off] Simple mixer control '3D Control - Switch',0   Capabilities: pswitch pswitch-joined   Playback channels: Mono   Mono: Playback [off] Simple mixer control 'PCM',0   Capabilities: pvolume pswitch   Playback channels: Front Left - Front Right   Limits: Playback 0 - 31   Mono:   Front Left: Playback 31 [100%] [0.00dB] [on]   Front Right: Playback 31 [100%] [0.00dB] [on] Simple mixer control 'PCM Out Path & Mute',0   Capabilities: enum   Items: 'pre 3D' 'post 3D'   Item0: 'pre 3D' Simple mixer control 'Surround',0   Capabilities: pvolume pswitch   Playback channels: Front Left - Front Right   Limits: Playback 0 - 31   Mono:   Front Left: Playback 31 [100%] [0.00dB] [on]   Front Right: Playback 31 [100%] [0.00dB] [on] Simple mixer control 'Surround Jack Mode',0   Capabilities: enum   Items: 'Shared' 'Independent'   Item0: 'Shared' Simple mixer control 'Center',0   Capabilities: pvolume pvolume-joined pswitch pswitch-joined   Playback channels: Mono   Limits: Playback 0 - 15   Mono: Playback 15 [100%] [-24.00dB] [on] Simple mixer control 'LFE',0   Capabilities: pvolume pvolume-joined pswitch pswitch-joined   Playback channels: Mono   Limits: Playback 0 - 15   Mono: Playback 15 [100%] [-24.00dB] [on] Simple mixer control 'Line',0   Capabilities: pvolume pswitch cswitch cswitch-exclusive   Capture exclusive group: 0   Playback channels: Front Left - Front Right   Capture channels: Front Left - Front Right   Limits: Playback 0 - 63   Front Left: Playback 0 [0%] [-34.50dB] [off] Capture [off]   Front Right: Playback 0 [0%] [-34.50dB] [off] Capture [off] Simple mixer control 'Line Jack Sense',0   Capabilities: pswitch pswitch-joined   Playback channels: Mono   Mono: Playback [off] Simple mixer control 'CD',0   Capabilities: pvolume pswitch cswitch cswitch-exclusive   Capture exclusive group: 0   Playback channels: Front Left - Front Right   Capture channels: Front Left - Front Right   Limits: Playback 0 - 63   Front Left: Playback 0 [0%] [-34.50dB] [off] Capture [off]   Front Right: Playback 0 [0%] [-34.50dB] [off] Capture [off] Simple mixer control 'Mic',0   Capabilities: pvolume pswitch cswitch cswitch-exclusive   Capture exclusive group: 0   Playback channels: Front Left - Front Right   Capture channels: Front Left - Front Right   Limits: Playback 0 - 63   Front Left: Playback 0 [0%] [-34.50dB] [off] Capture [on]   Front Right: Playback 0 [0%] [-34.50dB] [off] Capture [on] Simple mixer control 'Mic Boost (+20dB)',0   Capabilities: pswitch pswitch-joined   Playback channels: Mono   Mono: Playback [off] Simple mixer control 'Mic Select',0   Capabilities: enum   Items: 'Mic1' 'Mic2'   Item0: 'Mic1' Simple mixer control 'Video',0   Capabilities: pvolume pswitch cswitch cswitch-exclusive   Capture exclusive group: 0   Playback channels: Front Left - Front Right   Capture channels: Front Left - Front Right   Limits: Playback 0 - 63   Front Left: Playback 0 [0%] [-34.50dB] [off] Capture [off]   Front Right: Playback 0 [0%] [-34.50dB] [off] Capture [off] Simple mixer control 'Phone',0   Capabilities: pvolume pswitch cswitch cswitch-exclusive   Capture exclusive group: 0   Playback channels: Front Left - Front Right   Capture channels: Front Left - Front Right   Limits: Playback 0 - 63   Front Left: Playback 0 [0%] [-34.50dB] [off] Capture [off]   Front Right: Playback 0 [0%] [-34.50dB] [off] Capture [off] Simple mixer control 'Beep',0   Capabilities: pvolume pvolume-joined pswitch pswitch-joined   Playback channels: Mono   Limits: Playback 0 - 15   Mono: Playback 0 [0%] [-45.00dB] [off] Simple mixer control 'Aux',0   Capabilities: pvolume pswitch cswitch cswitch-exclusive   Capture exclusive group: 0   Playback channels: Front Left - Front Right   Capture channels: Front Left - Front Right   Limits: Playback 0 - 63   Front Left: Playback 0 [0%] [-34.50dB] [off] Capture [off]   Front Right: Playback 0 [0%] [-34.50dB] [off] Capture [off] Simple mixer control 'Mono Output Select',0   Capabilities: enum   Items: 'Mix' 'Mic'   Item0: 'Mix' Simple mixer control 'Capture',0   Capabilities: cvolume cswitch   Capture channels: Front Left - Front Right   Limits: Capture 0 - 15   Front Left: Capture 8 [53%] [12.00dB] [on]   Front Right: Capture 8 [53%] [12.00dB] [on] Simple mixer control 'Mix',0   Capabilities: cswitch cswitch-exclusive   Capture exclusive group: 0   Capture channels: Front Left - Front Right   Front Left: Capture [off]   Front Right: Capture [off] Simple mixer control 'Mix Mono',0   Capabilities: cswitch cswitch-exclusive   Capture exclusive group: 0   Capture channels: Front Left - Front Right   Front Left: Capture [off]   Front Right: Capture [off] Simple mixer control 'Channel Mode',0   Capabilities: enum   Items: '2ch' '4ch' '6ch'   Item0: '2ch' Simple mixer control 'Downmix',0   Capabilities: enum   Items: 'Off' '6 -> 4' '6 -> 2'   Item0: 'Off' Simple mixer control 'Exchange Front/Surround',0   Capabilities: pswitch pswitch-joined   Playback channels: Mono   Mono: Playback [off] Simple mixer control 'External Amplifier',0   Capabilities: pswitch pswitch-joined   Playback channels: Mono   Mono: Playback [on] Simple mixer control 'High Pass Filter Enable',0   Capabilities: pswitch pswitch-joined   Playback channels: Mono   Mono: Playback [on] Simple mixer control 'Spread Front to Surround and Center/LFE',0   Capabilities: pswitch pswitch-joined   Playback channels: Mono   Mono: Playback [off] Simple mixer control 'Stereo Mic',0   Capabilities: pswitch pswitch-joined   Playback channels: Mono   Mono: Playback [off] Simple mixer control 'V_REFOUT Enable',0   Capabilities: pswitch pswitch-joined   Playback channels: Mono   Mono: Playback [on]

➜ pactl list sinks Sink #0 	State: SUSPENDED 	Name: alsa_output.pci-0000_00_05.0.analog-stereo 	Description: Built-in Audio Analog Stereo 	Driver: module-alsa-card.c 	Sample Specification: s16le 2ch 48000Hz 	Channel Map: front-left,front-right 	Owner Module: 6 	Mute: no 	Volume: front-left: 20613 /  31% / -30.14 dB,   front-right: 20613 /  31% / -30.14 dB 	        balance 0.00 	Base Volume: 65536 / 100% / 0.00 dB 	Monitor Source: alsa_output.pci-0000_00_05.0.analog-stereo.monitor 	Latency: 0 usec, configured 0 usec 	Flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY  	Properties: 		alsa.resolution_bits = "16" 		device.api = "alsa" 		device.class = "sound" 		alsa.class = "generic" 		alsa.subclass = "generic-mix" 		alsa.name = "Intel 82801AA-ICH" 		alsa.id = "Intel ICH" 		alsa.subdevice = "0" 		alsa.subdevice_name = "subdevice #0" 		alsa.device = "0" 		alsa.card = "0" 		alsa.card_name = "Intel 82801AA-ICH" 		alsa.long_card_name = "Intel 82801AA-ICH with AD1980 at irq 21" 		alsa.driver_name = "snd_intel8x0" 		device.bus_path = "pci-0000:00:05.0" 		sysfs.path = "/devices/pci0000:00/0000:00:05.0/sound/card0" 		device.bus = "pci" 		device.vendor.id = "8086" 		device.vendor.name = "Intel Corporation" 		device.product.id = "2415" 		device.product.name = "82801AA AC'97 Audio Controller" 		device.form_factor = "internal" 		device.string = "front:0" 		device.buffering.buffer_size = "19188" 		device.buffering.fragment_size = "6396" 		device.access_mode = "mmap" 		device.profile.name = "analog-stereo" 		device.profile.description = "Analog Stereo" 		device.description = "Built-in Audio Analog Stereo" 		alsa.mixer_name = "Analog Devices AD1980" 		alsa.components = "AC97a:41445370" 		module-udev-detect.discovered = "1" 		device.icon_name = "audio-card-pci" 	Ports: 		analog-output;output-amplifier-on: Analog Output / Amplifier (priority: 9910) 		analog-output;output-amplifier-off: Analog Output / No Amplifier (priority: 9900) 	Active Port: analog-output;output-amplifier-on 	Formats: 		pcm

ok thank, first your local https://wiki.archlinux.org/index.php/Locale#LC_COLLATE:_collation
you have to change LC_COLLATE to LC_COLLATE=C

OK how do i set it to LC_COLLATE=C i looked at the Arch linke you gave i'm not seeing how to do so

sudo localectl set-locale LC_COLLATE=C

The sound should work with 6ef9280

OK Thanks so Much @szorfein
I figured out what the issue is with the symbols its the Font whatever Font you have set in ncmpcpp terminal using i must not have installed. Are you using bitmap font in ncmpcpp terminal. What terminal is the ncmpcpp widget app using.

the locale will work after your next reboot, else i didn't change the terminal for the screenshot, it use xst with the nerd-font-iosevka :)

well i changed the font in xfce4-terminal is what i use. To Roboto Mono Light and the symbols showed up. So where is font used in ncmpcpp player located?

They come from the .Xresources:
73 st.font: Iosevka Term:pixelsize=14

You should make a $ xrdb -merge ~/.Xresources if you modify this file.

 No change in ncmpcpp. Do i have to do something else? or is font someplace else its getting from ncmpcpp app

st.font: Roboto Mono Light:pixelsize=14

Try with this:
st.font: Roboto Mono:style:Light:pixelsize=14

Or
st.font: Iosevka Term:style=Regular:pixelsize=14

Now, this makes sense. OMG. it's using xterm not termite
OK, so the fonts symbols work for termite and xfce4-terminal. terminal opening up on my end is xterm and the fonts are not changing. This what your using? So not sure why it's not changing must have to be changed somewhere else? How can i change it to use termite in ncmpcpp app
Also the volume still doesn't work? :( :(
see if you add this to it. He's stuff usually works for me
https://github.com/streetturtle/awesome-wm-widgets/tree/master/volume-widget

To change the terminal, i have alrealy wrote something about this... https://github.com/szorfein/dotfiles#environment .

What terminal do you use?

it's wrote here https://github.com/szorfein/dotfiles#environment lol

Oh yeah my bad. :) I'm confused I'm using your zshrc. so why didn't xst start?
where do i put this. export TERMINAL=/usr/bin/xst
Also i don't have /etc/profile.d/ vte-2.91.sh _vte_osc7 . Where do i get these

I put this at the top of .zshrc export TERMINAL=/usr/bin/xst if thats right? xterm still opens

I fixed it Terminal and symbols GOSH YES its OVER and done :) :) :)
OK now still volume isn't working? It did work before when you first updated it but after then i rebooted it didn't take for some reason. What commands would you like?

finally, i've tested the volume with 81986f6 on arch with pulseaudio, it should work.

The volume still not showing. for widget?

Ok got it working weird? so I recopied this into volume.lua
`
local wibox = require("wibox")
local awful = require("awful")
local beautiful = require("beautiful")
local naughty = require("naughty")
local widget = require("util.widgets")
local helpers = require("helpers")
-- beautiful vars
local volume_icon = beautiful.widget_volume_icon
local fg = beautiful.widget_volume_fg
local bg = beautiful.widget_volume_bg
local l = beautiful.widget_volume_layout or 'horizontal'
-- widget creation
local icon = widget.base_icon()
local text = widget.base_text()
local icon_margin = widget.icon(icon)
local text_margin = widget.text(text)
volume_widget = widget.box(l, icon_margin, text_margin)
local function update_widget(volume)
icon.markup = helpers.colorize_text(volume_icon, fg)
text.markup = helpers.colorize_text(volume, fg)
end
awful.widget.watch(
-- you can change  with your card name if non detected
os.getenv("HOME").."/.config/awesome/widgets/audio.sh volume Pro", 4,
function(widget, stdout, stderr, exitreason, exitcode)
local volume = stdout:match('%w+:%s?(%d+)') or 0
update_widget(volume.."%")
end
)`
which is weird to me because you said to put it like this with my audio device.  What does volume Pro mean anyway?

How do i unlink your files to your dotfiles so i don't get changes you make?

yep, it should lack the audio.sh, you have use stow to delete and recreate the symlink on new files:
$ cd dotfiles
$ stow -D theme-anonymous
$ stow theme-anonymous

Thanks, i see you updated the wiki on everything its NICE. I never used stow before or what it do still learning. I like it

Its not unlinking them...? I did stow -D dotfiles/ and then stow -D images/ but its still symlink


EmberProjectComponent: Add public as simple source folder

As a workaround for PHPStorm and WebStorm add public folder as simple source folder without type.
Related issue: #36

Could you add a check that this is only applied on webstorm and phpstorm?

http://www.jetbrains.org/intellij/sdk/docs/basics/getting_started/plugin_compatibility.html
I am not sure if theres a simple fix for this or not. We should some how check if java is supported by ide or not.
Simpler way would add condition if intellij then else always as source folder.

I dig more into this and let you know when this can be merged.

@Turbo87 I have added check based on ApplicationInstance.versionName, i am not sure if this is good enough?

@raido looks good on first glance. I'll need a bit of time to test this properly though.

Any update?

@raido sorry, was busy with other projects. thanks for your contribution. I'll will try to get a patch release out in the next few days.


memory leakage caused by logging_output in criterion "masked_lm"

🐛 Bug
I found memory (not GPU memory) leakage event when trying masked language modeling task.
The memory usage was increasing when the trainning proceeded.
The bug was caused by 'loss' item in logging_output in the file criterions/masked_lm.py.
The original code is,
64         logging_output = {
65             'loss': loss,
66             'ntokens': sample['ntokens'],
67             'nsentences': sample['nsentences'],
68             'sample_size': sample_size,
69         }
The bug could be fixed with the following code.
64         logging_output = {
65             'loss': loss.data,
66             'ntokens': sample['ntokens'],
67             'nsentences': sample['nsentences'],
68             'sample_size': sample_size,
69         }

Interesting, can you confirm that using .detach() instead of .data works as well? I believe detach() is the preferred way of doing this now.

@joshim5 has a fix for this

Landed in https://github.com/pytorch/fairseq/commit/8eb9123f560d32940f96a01369d61c1684dce085. @hury07 please reopen this issue if you still encounter problems.


support scripts using module

from mdn docs:
https://developer.mozilla.org/en-US/docs/Web/API/Document/currentScript

The Document.currentScript property returns the

Ok, import.meta isn't helpful here. It looks like there is an open issue on the spec for this:
https://github.com/whatwg/html/issues/1013

Thanks for looking into this. Would love to make it happen once it's in available.


Docker windows: invoke process using a file in a shared volume

I have this command that works perfectly on linux but fails on windows.
docker run --init --publish=4010:4010  -it -v $(pwd)/src/main/resources:/tmp -P stoplight/prism:3 mock -d -h <IP_ADDRESS> "/tmp/swagger.yaml"

On linux the prism process understands that the file I'm looking for is in /tmp but somehow on windows it does not see and returns the following error using the bash console as adminstrator:
[10:23:51 AM] › [CLI] ✖  fatal     ENOENT: no such file or directory, open 'C:/Users/leo/Temp/swagger.yaml'

when using the poser-shell command as adminstrator it returns the following error:
    docker run --init --publish=4010:4010 --rm -it -v C:\Users\leo\IdeaProjects\java-demo-rest\src\main\resources:/tmp -P stoplight/prism:3 mock -d -h <IP_ADDRESS> "/tmp/swagger.yaml"
[10:42:10 AM] › [CLI] …  awaiting  Starting Prism…
    [10:42:11 AM] › [CLI] ✖  fatal     ENOENT: no such file or directory, open '/tmp/swagger.yaml'

Environment :

Library version: [ 3.0.0]
OS: [Windows 10]

Hey,
this does not really seem to be a Prism specific error — but rather the file is being mounted somewhere else; can you ssh in the container and check out if the file is effectively there?

I'm closing this since it's not Prism related, but feel free to reopen in case you can provide more details.


python-cdk: Installation of Lambda Function Dependencies fails with DistutilsOptionError

Command run:
pip install requests -t .

Instructions: https://github.com/aws-samples/aws-modern-application-workshop/tree/python-cdk/module-5#use-pip-to-intall-lambda-function-dependencies
Error:
Collecting requests
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests)
  Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
  Using cached https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5 (from requests)
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Installing collected packages: urllib3, certifi, chardet, idna, requests
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 360, in run
    prefix=options.prefix_path,
  File "/usr/lib/python2.7/dist-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/usr/lib/python2.7/dist-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 247, in move_wheel_files
    prefix=prefix,
  File "/usr/lib/python2.7/dist-packages/pip/locations.py", line 153, in distutils_scheme
    i.finalize_options()
  File "/usr/lib/python2.7/distutils/command/install.py", line 289, in finalize_options
    raise DistutilsOptionError("can't combine user with prefix, "
DistutilsOptionError: can't combine user with prefix, exec_prefix/home, or install_(plat)base

@trivikr Could you confirm if you are using a Cloud9 environment or your local machine?

I'm using Cloud9 environment.
I ignored this error, and continued with other steps in module 5. I was able to get Lambda deployed.

Getting the same error output using Cloud9 environment, the lambda deploys successfully but when triggered there is a CloudWatch error when the lambda executes
Unable to import module 'streamProcessor': No module named 'requests'

update: after switching to root user, the command pip install requests -t . executed successfully. seems to be some permissions issue (based on the username maybe?)


[WIP] [HUDI-2593] Virtual keys support for metadata table

What is the purpose of the pull request
Adding virtual keys support for Metadata table
Brief change log

Metadata table today has virtual keys disabled, thereby populating the metafields
for each record written out and increasing the overall storage space used. Hereby
adding virtual keys support for metadata table so that metafields are disabled
for metadata table records.

HFile data blocks written out and read back assume the record key and partition
path metafields are always available. After turning on the virtual keys, since
these metafields are not available, added support for HFile/Avro data block
readers, scanners to look at the table set record key property instead of the
hardcoded metafield record key.

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

still need to spend more time.

@prashantwason can you please take a pass

@manojpec : can you also add whats pending (like tests, wiring in props etc, what ever is pending) in the description so that reviewer is aware of the status of the patch. for eg, I guess we have not wired in the record key properties in table config/write config yet.

@manojpec I did not get how the metafields will be removed from the base files in metadata table.

While writing the HFile base file the records would have been read from HfileDataBlock so wont have the metafields. Correct?
When reading records from the baseFile itself, where are the metafields populated? Can you please point me to this portion in code?

as already pointed out, watch out for compaction code path. Since partition is hard coded and payload does not have partition field as such, please walk through compaction code path and ensure things are ok.
@prashantwason : for base file code path, please refer to HoodieBackedTableMetadata.readFromBaseAndMergeWithLogRecords(). thats where we read the record from basefile  as GenericRecord and then generate HoodieRecord.

@vinothchandar

so the zip files are getting added for this PR specifically? the test ones? any reason we needed this? this seems bit orthogonal to delta streamer right

Table Zip files were part of the test resources already. These resources are used by the tests to prep/fillup the table. Problem is these tables come with old properties which doesn't contain all the new params needed for the metadata table. So, changed the metadata table properties alone in the old zip and re-zipped them. Now those tests using the zip resources are passing.

@nsivabalan
CI Passed
Integ test run succeeded - https://gist.github.com/manojpec/e6354223a8f7a0896e256fd5c9b2c09e

Table Properties from the integ test run
root@adhoc-2:/opt# hdfs dfs -cat hdfs://namenode:8020/user/hive/warehouse/hudi-integ-test-suite/output/.hoodie/hoodie.properties
#Properties saved on Fri Nov 19 22:26:56 UTC 2021
#Fri Nov 19 22:26:56 UTC 2021
hoodie.table.precombine.field=test_suite_source_ordering_field
hoodie.table.partition.fields=timestamp
hoodie.table.type=COPY_ON_WRITE
hoodie.archivelog.folder=archived
hoodie.populate.meta.fields=true
hoodie.timeline.layout.version=1
hoodie.table.version=3
hoodie.table.recordkey.fields=_row_key
hoodie.table.base.file.format=PARQUET
hoodie.table.keygenerator.class=org.apache.hudi.keygen.TimestampBasedKeyGenerator
hoodie.table.name=table1

root@adhoc-2:/opt# hdfs dfs -cat hdfs://namenode:8020/user/hive/warehouse/hudi-integ-test-suite/output/.hoodie/metadata/.hoodie/hoodie.properties
#Properties saved on Fri Nov 19 22:26:59 UTC 2021
#Fri Nov 19 22:26:59 UTC 2021
hoodie.table.type=MERGE_ON_READ
hoodie.archivelog.folder=archived
hoodie.populate.meta.fields=false
hoodie.compaction.payload.class=org.apache.hudi.metadata.HoodieMetadataPayload
hoodie.timeline.layout.version=1
hoodie.table.version=3
hoodie.table.recordkey.fields=key
hoodie.table.base.file.format=HFILE
hoodie.table.keygenerator.class=org.apache.hudi.metadata.HoodieTableMetadataKeyGenerator
hoodie.table.name=table1_metadata

Good job on the patch!


Improve logging for Azure WebJobs Extensions storage account resolution

Library name and version
Azure.Storage.Blobs 5.0.0
Describe the bug
To aid in CRI investigations we need to log the name of the storage account we resolve and use for functions. The Azure Functions programming model allows customers to specify binding expression patterns that allow them to separate config from code, with the storage account used being resolved at runtime from app settings. We get many CRIs where customers mess up their deployments such that the wrong environment variables are deployed and the functions are targeting the incorrect storage account leading to errors/confusion. I was just investigating a multi-day CRI where it ended up being the customer was deploying the wrong code/config. Having logs would make these cases much faster to resolve. As it stands, we have to get access to customer app settings as well as decompile binaries to figure it out.
For example, in the blob trigger case, the code here starts the listener using a resolved BlobContainerClient. We could simply log the AccountName along with the function name. We'd want to do a similar thing for QueueTrigger, as well as for input/output bindings, looking to see if we can somehow centralize this code to make it easier to apply in all the cases.
Other non-storage extensions would benefit from such logs as well, so once we establish a pattern here, we should do the same for other extensions.
Expected behavior
Logs should clearly show the storage account being used for function bindings.
Actual behavior
We don't currently log this information, making diagnosing problems difficult.
Reproduction Steps
See recent ICM https://portal.microsofticm.com/imp/v3/incidents/details/348978764.
Environment
All environments.

//cc: @seanmcc-msft, @amnguye