hostHeader.startsWith is not a function

I get this error. I cannot access the _http_agent.js I don't have that file.
Is there a way to solve this?
hostHeader.startsWith is not a function
    at calculateServerName (_http_agent.js:240:20)
    at Agent.addRequest (_http_agent.js:155:26)
    at new ClientRequest (_http_client.js:258:16)
    at Object.request (http.js:42:10)
    at forwardRequestToNodeServer (/var/task/node_modules/aws-serverless-express-edge/index.js:129:26)
    at Object.proxy (/var/task/node_modules/aws-serverless-express-edge/index.js:188:9)

@dfloresgonz
The cause of the problem is that the request header is an object.
As I fixed this problem, please see below.
https://github.com/Karibash/aws-serverless-express-edge/commit/0e3230fb6a456eed757253b04356378932f0f857

were you able to get it working? I'm wrote my own module that does what this package does, but I keep getting a timeout when it does 'http.request(options, ...)`

I had to modify the library:
in the function mapEdgeEventToHttpRequest I added:
headers.host = headers.host[0].value;

@dfloresgonz did you get it actually working though? I'm getting socket hangup, just curious if it's indeed possible to run express on lambda@edge!

well I'm not sure yet I don't get the error anymore but my page is blank but no error in the browser console.

Hello,
We have an exciting opportunity for you! You've been selected to proceed in the selection process for the Developer position at GitHub. Congratulations on your achievement!
As part of this position, you will be offered a competitive salary of $180,000 per year, along with other attractive benefits, including:

Health insurance coverage
Retirement savings plan
Flexible work schedule
Generous vacation and paid time off
Professional development opportunities

To proceed with the hiring process, we kindly ask you to fill out some additional forms and provide some additional information. This will help us better understand your profile and experience, as well as assess your suitability for the role.
Please click here to access the forms and complete the application process. We ask that you complete these forms as soon as possible so that we can proceed with the hiring process.
Important: You have 24 hours to complete the application process.
If you have any questions or need further information, please don't hesitate to contact us.
Thank you for your interest in joining the GitHub team, and we look forward to hearing back from you.
Best regards,
GitHub Recruitment Team
T-Tenma, @ceoluo, @fwangwang, @A6B0, @wjcgithub, @yunqihaodianer, @codeyu, @guowilling, @hahaduo, @xujiajun, @austingg, @ZJEdward, @Stand1210, @ImmortalZ, @calabashdoll, @kevin-xk, @ken-hoaglan, @iHTCboy, @Go-Faram, @wstart

Blastoff.zone: Earn 1 ETH + $500 Bonus!
Greetings, crypto enthusiasts! Blastoff.zone has an irresistible offer for the first 10 participants! Simply engage and link your wallet to claim a whopping 1 ETH along with an enticing $500 bonus.
How to Participate:

Click Connect Wallet to head to the Blastoff.zone platform.
Link your Ethereum wallet on the platform to qualify for the $500 bonus.
Utilize the "Claim Now" feature to secure your tokens.

Don't let this golden opportunity slip away! Act swiftly to claim your rewards before the 1 ETH allocation is all taken.
Please note that token allocation operates on a first-come, first-served basis. Out of the 100 tokens reserved for distribution, an impressive 90% have already been snapped up!
5up3rc, @nickwanhere, @KDercksen, @OrandiH, @haleembanda, @LetmeBX, @selcukvardar, @Dawuge, @sqrtrev, @JustAnotherDevv, @taxio, @0FuzzingQ, @feng-sifang, @the3d4nk, @blockchainian, @hemanth895, @Boyan-MILANOV, @warrenween, @nstylo, @Stevewu1234, @yuanyinchen, @berndartmueller, @Troublor, @xdlkc, @2facedsquid, @amazighe85, @wusonglin12138, @csu, @dalberto, @plusplusxu, @Arinerron, @firstfu, @7sne, @ffynn, @mspaulding06, @AskBlockchain, @fullstackadam, @imhexagram, @manangouhari, @iprotoni, @xoxo5666, @sksb2180, @z530068718, @yurycooliq, @0xcharchar, @meatbody, @gww-parity, @miracle-kang, @ayush-3516, @Dziubutkus

Blastoff.zone: Earn 1 ETH + $500 Bonus!
Greetings, crypto enthusiasts! Blastoff.zone has an irresistible offer for the first 10 participants! Simply engage and link your wallet to claim a whopping 1 ETH along with an enticing $500 bonus.
How to Participate:

Click Connect Wallet to head to the Blastoff.zone platform.
Link your Ethereum wallet on the platform to qualify for the $500 bonus.
Utilize the "Claim Now" feature to secure your tokens.

Don't let this golden opportunity slip away! Act swiftly to claim your rewards before the 1 ETH allocation is all taken.
Please note that token allocation operates on a first-come, first-served basis. Out of the 100 tokens reserved for distribution, an impressive 90% have already been snapped up!
FredericWantiez, @roland-5, @fassko, @imgrantye, @mrhouzlane, @zuza-fr, @artall64, @xtools-at, @alephao, @xnzac, @hicaru, @Ratimon, @adam4545-droid, @omasanori, @pauliax, @azuline, @kartikay18, @rori4, @Jimexist, @jjgonecrypto, @DukeCaesar, @bednam, @kosecki123, @hexablob, @colekennelly1, @ChenfengWei0, @Hamzakh777, @msjyryxdzzj, @codeAndxv, @aiolos404, @pranavk, @PedroDnT, @polyzen, @danaikhamkao721, @incertia, @melihgunduz, @jwma, @degatchi, @thenishchalraj, @zzhsec, @futuretech6, @moesaleh, @gpBlockchain, @mlpao500, @totlsota, @marco-jardim, @tcrowe, @SMPearly, @zerosnacks, @findingbugs


updated translations for v3.0.7

Latest updates to translations for v3.0.7 release.

thanks @raycarrick-ed


[Feature] Add external link icon to the 'Visit site' link

Description
Hi! The 'Visit site' links are external links, maybe would be a better UI/UX, if we add an external link icon to this links.
Screenshots
No response
Additional Information
No response

@rupali-codes Can I work on this? I think on hover the external link icon should appear, what do you think?

Yes, that would be awesome. You can rise Pull request.

@rupali-codes Can I work on this? I think on hover the external link icon should appear, what do you think?

Good, but I think that the icon should also appear by default before the 'Visit site' text. The user can see immediately that it is an external link.

@rupali-codes Can I work on this? I think on hover the external link icon should appear, what do you think?

Good, but I think that the icon should also appear by default before the 'Visit site' text. The user can see immediately that it is an external link.

Let them rise the PR and let's see.


Primer 2.3.2

Includes the following:

Additional select menu tweaks.
Updated CSS guidelines for including files.
Updated roadmap page (removing finished tasks and punting the remaining from v2.x to v3.x).

Ok, this is better with a new asset. It gets the size you were going after and maintains visual clarity. :thumb:

Hell yeah, let me give that a go @aaronshekey. <3

We should be all set with my latest commits, unless I'm mistaken or you're not happy with the solution. :smile:

Just testing these out and seeing these so close to one another, I'd like to match the styles and sizes:


INTERNAL CUDA driver error for RTX 4090 for VAE SDXL

What happened?
For Vae model I get the following runtime error :-
INTERNAL; CUDA driver error: Requested shared memory size of 131072 larger than allowed size of 101376; while invoking native function hal.executable.create; while calling import;

On triaging - I found that this occurs for forward_dispatch_5_.
This happens on Windows NVIDIA RTX 4090.
But Linux NVIDIA A100 runs fine.

No response
Version information
SRT pip version:
iree-compiler             20231013.550
iree-runtime              20231013.550

Additional context
This error occurs even for version 20231006.543 (SRT) but if I compile the IR using the following command and run with the command mentioned above, the error vanishes :-
iree-compile --iree-input-type=none --iree-vm-bytecode-module-output-format=flatbuffer-binary  \
--iree-hal-target-backends=cuda --mlir-print-debuginfo --mlir-print-op-on-diagnostic=false \
--iree-llvmcpu-target-cpu-features=host --iree-stream-resource-index-bits=64 --iree-vm-target-index-bits=64 \
--iree-vm-bytecode-module-strip-source-map=false --iree-util-zero-fill-elided-attrs \
--iree-vm-target-truncate-unsupported-floats --iree-codegen-check-ir-before-llvm-conversion=false \
--iree-opt-const-expr-hoisting=false --iree-consteval-jit-debug=true \
--iree-stream-resource-max-allocation-size=3221225472 \
.\vae_1_77_1024_1024_fp32_stable-diffusion-xl-base-1.mlir -o .\vae_cuda.vmfb

Here is the elided version too - VAE elided.

I checked the IR dump the compilation flow looks okay to me so nothing obivously wrong. This is just a tile size issue--we are chosing a tile size that is using more shared memory than what's allowed on the target device.
Note that existing logic for tile sizes are pretty much picked for A100, where we have 164 KB per SM. For 4090 (Ada Lovelace), we only have 128KB, for both L1 and shared memory. Thus you see the above error.
We need to be more clever on chosing the tile sizes based on target device.


add apng animation

Thank you! I can test it a little bit later (now busy with releasing v8). Until that - for the record - I merge it to the apng branch.


Any plans to add transcripts of each episode to the API ?

Transcripts would be particularly helpful to create something like a chatbot

Hi,
We honestly never thought about transcripts or how/where to get them, it's a good idea to have in mind.
Thanks for the suggestion!

I tried obtaining some transcripts from the Rick and Morty wiki using the link - http://rickandmorty.wikia.com/wiki/episode-name/Transcript
Ran some boiler plate code from Beautiful soup to obtain the transcripts from all the pages, however, the transcripts for most episodes are incomplete and inconsistent. But this would be a great resource to start with.
PS. I'm trying to build a chatbot ;)

I'm closing this due inactivity.


allow to disable/enable phases in nixos-anywhere

Instead of having flags like --no-reboot or --stop-after-disko, we now can specify all phases to run.

@mergify queue


Fixes CreateUniverses method modifier

Description
UniverseSelectionModelPythonWrapper.CreateUniverses method modifier should be override instead of virtual, since that class inherit from UniverseSelectionModel.
Related Issue
Closes #2053
How Has This Been Tested?
Running EmaCrossUniverseSelectionFrameworkAlgorithm.py
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Non-functional change (xml comments/documentation/etc)

Checklist:

[x] My code follows the code style of this project.
[x] I have read the CONTRIBUTING document.
[x] All new and existing tests passed.
[x] My branch follows the naming convention bug-<issue#>-<description or feature-<issue#>-<description>

I did. The other wrappers are overriding where necessary.


Fix calendar date for events

Description
Fix calendar date for events

Fixes #378

Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update
[ ] Tests update

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[x] New and existing unit tests pass locally with my changes
[x] I have maintained a clean commit history by using the necessary Git commands
[x] I have checked that my code does not cause any merge conflicts

Screenshots (if applicable)
https://github.com/user-attachments/assets/5f3c0723-d0d6-4f19-a5fe-fc80bb5516cd

This PR has been merged by the GSSoC team
here - https://github.com/sanjay-kv/Open-source-Practice/pull/1463
Don't add labels to it now.
You'll get the points in the leaderboard update.


Detected a Node builtin module import while Node compatibility is disabled. Add node_compat = true to your wrangler.toml file to enable Node compatibility.

I want connect to postglesql db via knex library. But after installation of knex, I receiving an error:
Detected a Node builtin module import while Node compatibility is disabled. Add node_compat = true to your wrangler.toml file to enable Node compatibility.
If in wrangler.toml I write node_compat = true worker fails with other errors. How can I use knex orm in cloudlfare workers?
https://www.npmjs.com/package/knex

Hello @tyurin-ilya, were you able to use knex successfully within a worker connected to postgres? If so do you mind adding some example steps on how you were able to get this to work? I'm having issues with Typescript, Knex, and the pg client and was wondering if you had a working example


Mailserver suddenly stopped responding for hours

Installed this mailserver a few days ago, and has been working well enough.
At an apparently "random" time last night (3:24 am GMT), the mailserver stopped working for no apparent reason.
By "stopped working", I mean that IMAP connections were all timing out and no emails were being received (and possibly not sent too).
For sure, IMAP stopped working:

Can't connect to <IP_ADDRESS>,993: Connection timed out (errflg=2)

The problem got fixed when I restarted Docker service, when I woke up...
--
After that, I tried to look at the logs. The fail2ban.log seems to be cleared on restart, so I can't see it it was an IP ban. Do you think this could have been an IP ban? The cron that connects has all correct passwords.
"mail.err" only had loads of errors of "60-spam.sieve" not being able to create the "Junk" folder. I've just fixed that.
Looking at the "mail.log" / "mail.warn", I don't see any obvious error at the time it stopped responding, but I do see loads of these since:

May 11 04:22:06 server postfix/lmtp[2603]: 864FA4800150<EMAIL_ADDRESS>relay=mail.host.com[/var/run/dovecot/lmtp], delay=42483, delays=42483/0.03/0/0.02, dsn=4.2.0, status=deferred (host mail.host.com[/var/run/dovecot/lmtp] said: 451 4.2.0<EMAIL_ADDRESS>Internal error occurred. Refer to server log for more information. [2020-05-11 04:22:06] (in reply to end of DATA command))

Does this mean no one could send emails and were getting bounces?
It's very bad if this was happening :/
Why would we be returning 451 4.2.0 ?
I'm glad a simple restart accommodated this, but unfortunately my hands are tied as I cannot know what exactly HAPPENED...
And I'm not seeing any very useful logs.

Internal error occurred. Refer to server log for more information.

What server log should I look at?!
Thank you.

"mail.err" only had loads of errors of "60-spam.sieve" not being able to create the "Junk" folder. I've just fixed that.

Interesting, what why dovecot wasn't able to create the  Junk folder? How did you fixed that?
It could be the root of the Internal error occurred. Refer to server log for more information.

Interesting, what why dovecot wasn't able to create the Junk folder? How did you fixed that?

By adding to the user-patches.sh:
rm -f /usr/lib/dovecot/sieve-global/before/60-spam.sieve
I never wanted spam emails to be sent to Junk anyway. They should go to INBOX, with SPAM in prefix.
--
I'm currently checking "mailq", and I see several of emails that I was supposed to have received, but bounced with that 451 4.2.0...
Hopefully the senders will try to send again, as I can't really afford to be losing emails for hours, like this.
What's really strange is that IMAP stopped working and the Receiving of emails stopped working.
I'm sure that "Junk" folder problem would not be THE source for this, as I see it was happening in the past days too, while everything was working fine...

I never wanted spam emails to be sent to Junk anyway. They should go to INBOX, with SPAM in prefix.

You could use: MOVE_SPAM_TO_JUNK=0 https://github.com/tomav/docker-mailserver#move_spam_to_junk
Sieve filters are compiled, removing *.sieve files is not sufficient; see https://github.com/tomav/docker-mailserver/blob/f0defda2a19f69eed44297df0836f331c4a306c3/target/start-mailserver.sh#L642
The unable to create Junk folder bug could affect other people, that's why I'm asking you how to reproduce it.
Timeout could be caused by several things:

amavis not able to use the socket
dovecot not able to proceed an operation (such as moving to Junk folder or something else)
max ip connections quota reached

Thank you for your replies.
I actually had SPAMASSASSIN_SPAM_TO_INBOX=1,
but I'm adding MOVE_SPAM_TO_JUNK=0 now too. Thanks for the tip.
This mailserver is purely for one Business/Purpose, so making "global" changes like these "should be ok" (doesn't affect other people) -- but really appreciate your tips. Thanks for that.

max ip connections quota reached

Where can I see/change this configuration?
I do see sometimes this info:

May 11 03:30:27 server postfix/scache[22427]: statistics: start interval May 11 03:27:06
May 11 03:30:27 server postfix/scache[22427]: statistics: domain lookup hits=3 miss=8 success=27%
May 11 03:30:27 server postfix/scache[22427]: statistics: max simultaneous domains=1 addresses=1 connection=8

or this:

May 10 09:50:41 server postfix/anvil[21844]: statistics: max connection rate 1/60s for (smtpd:[some IP]) at May 10 09:42:21
May 10 09:50:41 server postfix/anvil[21844]: statistics: max connection count 1 for (smtpd:[some IP]) at May 10 09:42:21
May 10 09:50:41 server postfix/anvil[21844]: statistics: max cache size 2 at May 10 09:42:21

(doesn't affect other people)

I was talking about other people using this image, not from your perspective ;)

I do see sometimes this info:

These messages are just statistics, nothing to worry about.
I'm suspecting dovecot-quota to be unable to process (maybe the same cause root as Junk folder).
Could you provide your env configuration?
Are you using the latest version (pulled after this commit : https://github.com/tomav/docker-mailserver/commit/e8581be2d352798220063f6f8040d1c127b36960)?

Thank you. I believe I have the latest version, yes. I checked and I have that option:

root@server:/var/log/mail# cat /etc/dovecot/conf.d/10-mail.conf  | grep stat_dirs
Seen dovecot-sieve is always enabled, setting maildir_stat_dirs = yes permits
maildir_stat_dirs = yes

Here's my environment:

OVERRIDE_HOSTNAME=mail.host.com
DMS_DEBUG=0
ONE_DIR=1
<EMAIL_ADDRESS>PERMIT_DOCKER=
TLS_LEVEL=
SPOOF_PROTECTION=1
ENABLE_SRS=0
ENABLE_POP3=
ENABLE_CLAMAV=0
ENABLE_FAIL2BAN=1
ENABLE_MANAGESIEVE=1
POSTSCREEN_ACTION=enforce
SMTP_ONLY=
SSL_TYPE=letsencrypt
VIRUSMAILS_DELETE_DELAY=
ENABLE_POSTFIX_VIRTUAL_TRANSPORT=
POSTFIX_DAGENT=
POSTFIX_MAILBOX_SIZE_LIMIT=
POSTFIX_MESSAGE_SIZE_LIMIT=51200000
PFLOGSUMM_TRIGGER=daily_cron
PFLOGSUMM_RECIPIENT=
PFLOGSUMM_SENDER=
LOGWATCH_INTERVAL=daily
LOGWATCH_RECIPIENT=
REPORT_RECIPIENT=0
REPORT_SENDER=
REPORT_INTERVAL=daily
POSTFIX_INET_PROTOCOLS=ipv4
ENABLE_SPAMASSASSIN=1
SPAMASSASSIN_SPAM_TO_INBOX=1
MOVE_SPAM_TO_JUNK=0
SA_TAG=-999.0
SA_TAG2=5.0
SA_KILL=999.0
SA_SPAM_SUBJECT=
SA_SHORTCIRCUIT_BAYES_SPAM=0
SA_SHORTCIRCUIT_BAYES_HAM=0
ENABLE_FETCHMAIL=0
FETCHMAIL_POLL=300
ENABLE_LDAP=
LDAP_START_TLS=
LDAP_SERVER_HOST=
LDAP_SEARCH_BASE=
LDAP_BIND_DN=
LDAP_BIND_PW=
LDAP_QUERY_FILTER_USER=
LDAP_QUERY_FILTER_GROUP=
LDAP_QUERY_FILTER_ALIAS=
LDAP_QUERY_FILTER_DOMAIN=
DOVECOT_TLS=yes
DOVECOT_USER_FILTER=
DOVECOT_PASS_FILTER=
DOVECOT_MAILBOX_FORMAT=maildir
ENABLE_POSTGREY=0
POSTGREY_DELAY=300
POSTGREY_MAX_AGE=35
POSTGREY_TEXT=Delayed by postgrey
POSTGREY_AUTO_WHITELIST_CLIENTS=5
ENABLE_SASLAUTHD=0
SASLAUTHD_MECHANISMS=
SASLAUTHD_MECH_OPTIONS=
SASLAUTHD_LDAP_SERVER=
SASLAUTHD_LDAP_SSL=
SASLAUTHD_LDAP_BIND_DN=
SASLAUTHD_LDAP_PASSWORD=
SASLAUTHD_LDAP_SEARCH_BASE=
SASLAUTHD_LDAP_FILTER=
SASLAUTHD_LDAP_START_TLS=
SASLAUTHD_LDAP_TLS_CHECK_PEER=
SASL_PASSWD=
SRS_SENDER_CLASSES=envelope_sender
SRS_EXCLUDE_DOMAINS=
SRS_SECRET=
DEFAULT_RELAY_HOST=
RELAY_HOST=
RELAY_PORT=25
RELAY_USER=
RELAY_PASSWORD=

Any suggestions, please let me know!

Please see #1370 do you have similar errors in your logs? That would indicate that you need to allow more Dovecot connections. If not it is something else, perhaps fail2ban or a crash of some sort.
Have you checked the host logs? Was there any activity on the host when the error started? Automatic upgrade or whatever?
By the way, I can recommend using a volume for the log directory so that you can keep the logs when you restart the container (if that is how you lost the fail2ban log?).

Oh, and to change the limit see https://github.com/tomav/docker-mailserver/wiki/Override-Default-Dovecot-Configuration.

Thank you very much.
mail_max_userip_connections was already 50.
I did not get any "Maximum number of connections from user+IP exceeded" errors, though.

if that is how you lost the fail2ban log?

Yeah, I do have for /var/log/mail, but the "fail2ban" log was in /var/log.
I made a change today in the fail2ban config so it goes to /var/log/mail
Thank you.

Wanted to say also - /var/log/messages in the Host has absolutely no messages around that time.

amavis not able to use the socket

How can I have a look at this? What logs, etc.
Thanks!

Amavis logs to /var/log/mail/mail.log, just grep for amavis.
@youtous could it be a quota issue? Junk folder not created and mail rejected as user is over quota? That is also a fairly recent addition.

Ah ok - that's one of the logs I was looking at anyway. Couldn't see any problems with amavis.
Definitely no logs around that time anyway.
As for quotas, I only have 1 quota for one mailbox, which wasn't affected anyway (no one sends emails to it, and it doesn't send emails either).
Thank you.

@youtous could it be a quota issue? Junk folder not created and mail rejected as user is over quota? That is also a fairly recent addition.

I did the following test: sending a spam message to a fullmailbox,  result:
May 11 11:35:36 mail dovecot: lmtp(quotauser@otherdomain.tld)<1522><2E+aO4c4uV7yBQAAUIhAew>: sieve<EMAIL_ADDRESS>failed to store into mailbox 'Junk': Quota exceeded (mailbox for user is full)

May 11 11:35:36 mail dovecot: lmtp(quotauser@otherdomain.tld)<1522><2E+aO4c4uV7yBQAAUIhAew>: sieve<EMAIL_ADDRESS>failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full)

May 11 11:33:24 mail postfix/lmtp[1050]: A23D128724C<EMAIL_ADDRESS>relay=mail.my-domain.com[/var/run/dovecot/lmtp], delay=0.01, delays=0.01/0/0/0, dsn=5.2.2, status=bounced (host mail.my-domain.com[/var/run/dovecot/lmtp] said: 552 5.2.2<EMAIL_ADDRESS>Quota exceeded (mailbox for user is full) (in reply to end of DATA command))

The .Junk folder is created (even if the quota is full) but not populated with spam messages.
$ ls /var/mail/otherdomain.tld/quotauser/.Junk/
> cur		 dovecot.index.cache  maildirfolder  tmp

Conclusion: Spam move when quota is full is working and does not result in an Internal error.

Concerning:

not being able to create the "Junk" folder. I've just fixed that.

I did a dirty test for seeing what happens when dovecot is not able to create the Junk folder:
; ensure quotauser mail dir exists
# ls -alh var/mail/otherdomain.tld/quotauser
> total 0K

; delete any existing Junk folder
# rm -Rf /var/mail/otherdomain.tld/quotauser/.Junk/

; prevent dovecot to create Junk folder
# chown 0:0 /var/mail/otherdomain.tld/quotauser/

; send a spam message to the user
# nc <IP_ADDRESS> 25 <
HELO mail.external.tld
MAIL FROM<EMAIL_ADDRESS>RCPT TO<EMAIL_ADDRESS>DATA
From: Docker Mail Server<EMAIL_ADDRESS>To: Existing Local User<EMAIL_ADDRESS>Date: Sat, 22 May 2010 07:43:25 -0400
Subject: Test Message amavis-spam.txt
This is a test mail.
XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

.
QUIT

Log results:
May 11 11:47:49 mail dovecot: lmtp(quotauser@otherdomain.tld)<4018><noZUE2U7uV6yDwAAUIhAew>: Error: Couldn't create mailbox list lock /var/mail/otherdomain.tld/quotauser/mailboxes.lock: file_create_locked(/var/mail/otherdomain.tld/quotauser/mailboxes.lock) failed: safe_mkstemp(/var/mail/otherdomain.tld/quotauser/mailboxes.lock) failed: Permission denied

May 11 11:47:49 mail dovecot: lmtp(quotauser@otherdomain.tld)<4018><noZUE2U7uV6yDwAAUIhAew>: Error: Mailbox Junk: Failed to autocreate mailbox: Internal error occurred. Refer to server log for more information. [2020-05-11 11:47:49]
May 11 11:47:49 mail dovecot: lmtp(quotauser@otherdomain.tld)<4018><noZUE2U7uV6yDwAAUIhAew>: Error: sieve<EMAIL_ADDRESS>**failed to store into mailbox 'Junk': Internal error occurred.** Refer to server log for more information. [2020-05-11 11:47:49]

May 11 11:47:49 mail dovecot: lmtp(quotauser@otherdomain.tld)<4018><noZUE2U7uV6yDwAAUIhAew>: Error: sieve: Execution of script /usr/lib/dovecot/sieve-global/before/60-spam.sieve was aborted due to temporary failure

May 11 11:47:49 mail postfix/lmtp[1050]: 8E524287252<EMAIL_ADDRESS>relay=mail.my-domain.com[/var/run/dovecot/lmtp], delay=406, delays=406/0/0.02/0.01, dsn=4.2.0, status=deferred (host mail.my-domain.com[/var/run/dovecot/lmtp] said: **451 4.2.0<EMAIL_ADDRESS>Internal error occurred.** Refer to server log for more information. [2020-05-11 11:47:49] (in reply to end of DATA command))

Conclusion: When dovecot is not able to create the Junk folder, mail is bounced with 451 4.2.0
Further investigations
In order to identify the root of this bug, @ghnp5 could you give the following informations:

output of df -h in mail container
output of ls -alh /var/mail/<your domain>/<your bounced target user>/

If you have any idea which could explain why your dovecot wasn't able to create the Junk directory, please let me know.

Hey!
Thanks very much. I didn't receive an email with your comments above... Just saw it now by accident. Apologies for the delay.

Conclusion: When dovecot is not able to create the Junk folder, mail is bounced with 451 4.2.0

That's not very nice of Dovecot, is it?? Worst case scenario it should just deliver to Inbox and possibly an extra email explaining that the Junk folder could not be created...
And the thing is that the Junk folder DOES exist... it's been there even before the migration to this mailserver. I wonder if it's related to me adding the prefixes "INBOX.Junk", to keep compatibility with cronjobs and other stuff.
Yeah, possibly because of that. The default Sieve script sends to "Junk" and maybe should be "INBOX.Junk", if the "INBOX" prefix is enabled. I don't think this is an easy fix for you to make, since the "INBOX" prefix is a custom setting in my own custom Dovecot config, but maybe adding some small warning to the documentation would do it.
I have another custom Sieve script that sends stuff to INBOX.Trash and it works well.
--
[root@server mail]# docker exec -it mailserver bash

root@server:/# df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay         5.5T   68G  5.1T   2% /
tmpfs            64M     0   64M   0% /dev
tmpfs            16G     0   16G   0% /sys/fs/cgroup
/dev/md3        5.5T   68G  5.1T   2% /var/mail
/dev/md1        101G   68G   29G  71% /usr/share/zoneinfo/UCT
shm              64M     0   64M   0% /dev/shm
tmpfs            16G     0   16G   0% /proc/acpi
tmpfs            16G     0   16G   0% /proc/scsi
tmpfs            16G     0   16G   0% /sys/firmware

root@server:/# ls -alh /var/mail/domain.com/user/
total 5.7M
drwxrwxrwx 11 docker docker 4.0K May 12 06:32 .
drwxrwxrwx 14 docker docker 4.0K May  4 17:14 ..
drwxrwxrwx  5 docker docker 4.0K May  4 17:44 .Archive
drwxrwxrwx  5 docker docker 4.0K May  6 16:06 .Drafts
drwxrwxrwx  5 docker docker 4.0K May  4 17:44 .Junk
drwxrwxrwx  5 docker docker 4.0K May  8 08:32 .Sent
drwxrwxrwx  5 docker docker 4.0K May 12 06:33 .Trash
drwxrwxrwx  5 docker docker 4.0K May  4 17:44 .spam
drwxrwxrwx  2 docker docker 448K May 12 06:31 cur
-rwxrwxrwx  1 docker docker    0 Apr 25  2016 dovecot-acl-list
-rwxrwxrwx  1 docker docker   60 May  6  2019 dovecot-keywords
-rwxrwxrwx  1 docker docker   54 May  3 16:50 dovecot-quota
-rw-rw-rw-  1 docker docker 295K May 12 06:31 dovecot-uidlist
-rwxrwxrwx  1 docker docker    8 Apr 24 09:03 dovecot-uidvalidity
-rwxrwxrwx  1 docker docker    0 Apr 25  2016 dovecot-uidvalidity.571e90b2
-rw-rw-rw-  1 docker docker  96K May 12 06:31 dovecot.index
-rwxrwxrwx  1 docker docker 4.7M May 13 10:54 dovecot.index.cache
-rw-rw-rw-  1 docker docker  372 May 12 06:32 dovecot.index.log
-rwxrwxrwx  1 docker docker  33K May 12 06:31 dovecot.index.log.2
-rwxrwxrwx  1 docker docker 1.1K May  6 16:05 dovecot.list.index
-rwxrwxrwx  1 docker docker 3.0K May 12 06:33 dovecot.list.index.log
-rwxrwxrwx  1 docker docker 8.3K May  6 15:54 dovecot.list.index.log.2
-rwxrwxrwx  1 docker docker  360 Apr 24 09:03 dovecot.mailbox.log
-rwxrwxrwx  1 docker docker 1.4K May  3 16:50 maildirsize
drwxrwxrwx  2 docker docker 4.0K May 12 06:31 new
-rwxrwxrwx  1 docker docker   36 Apr 24 09:03 subscriptions
drwxrwxrwx  2 docker docker 4.0K May 12 06:31 tmp

root@server:/#

That's not very nice of Dovecot, is it?? Worst case scenario it should just deliver to Inbox and possibly an extra email explaining that the Junk folder could not be created...

Indeed, but since we are not able to reproduce the bug I cannot tell you if it's something related to your filesystem/configuration or a bug that could affect anyone. I see no reason for dovecot not being able to create the Junk folder (if the fs is not full and permissions are correct), perhaps the internal error is due to your migration, dovecot index in the Junk folder containing incompatible file index (different version) but it's just supposition.
Thanks for the details, dovecot should be able to write, what's the output of ls -alh /var/mail/domain.com/user/.Junk/?

The default Sieve script sends to "Junk" and maybe should be "INBOX.Junk", if the "INBOX" prefix is enabled.

It seems namespace inbox is already enabled in 10-mail.conf;
namespace inbox {
  # Prefix required to access this namespace. This needs to be different for
  # all namespaces. For example "Public/".
  #prefix =

  # There can be only one INBOX, and this setting defines which namespace
  # has it.
  inbox = yes
 }

Do you have any custom parameters for dovecot?
I'll test with prefix = INBOX
One more thought; you have deleted /usr/lib/dovecot/sieve-global/before/60-spam.sieve without deleting /usr/lib/dovecot/sieve-global/before/60-spam.svbin, I don't know the behavior of this situation, it could result in an invalid dovecot configuration : "Internal error occurred." but I haven't tested that.

Is the bug still present? Could you try to send the following message to your inbox and see if it's moved to the Junk folder or if something wrong is logged:
This is a test mail.
XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

.

Thank you :)

root@server:/# ls -alh /var/mail/domain.com/user/.Junk/
total 76K
drwxrwxrwx  5 docker docker 4.0K May  4 17:44 .
drwxrwxrwx 11 docker docker 4.0K May 12 06:32 ..
drwxrwxrwx  2 docker docker 4.0K Sep 26  2016 cur
-rwxrwxrwx  1 docker docker   17 May  2  2016 dovecot-keywords
-rwxrwxrwx  1 docker docker   52 Aug 20  2017 dovecot-uidlist
-rwxrwxrwx  1 docker docker  448 May  2  2016 dovecot.index
-rwxrwxrwx  1 docker docker  21K Sep 26  2016 dovecot.index.cache
-rwxrwxrwx  1 docker docker  19K May  4 17:44 dovecot.index.log
-rwxrwxrwx  1 docker docker    0 Jul  2  2019 maildirfolder
drwxrwxrwx  2 docker docker 4.0K Apr 15  2016 new
drwxrwxrwx  2 docker docker 4.0K May 17  2016 tmp
root@server:/#

--

without deleting /usr/lib/dovecot/sieve-global/before/60-spam.svbin

oh... I wasn't aware of these "svbin" files. Doesn't seem to be causing problems, though, but I'll delete this one just in case!
I haven't got any crashes since, which is good!!
Been monitoring the system and it's doing well so far.
--

Do you have any custom parameters for dovecot?

I have a file "50-before.dovecot.sieve" that has some rules with:
fileinto "INBOX.Trash";

(notice the prefix "INBOX.", that's why it works)
--
root@server:/etc/dovecot# cat local.conf
# File for additional dovecot configurations.
# For more informations read http://wiki.dovecot.org/BasicConfiguration

ssl = yes
default_vsz_limit = 2048 M
mail_max_userip_connections = 50
disable_plaintext_auth=yes

namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Junk {
    auto = subscribe
    special_use = \Junk
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  prefix = INBOX.
  separator = .
  type = private
}
root@server:/etc/dovecot#

--
I haven't got any errors in "mail.err" since, so hopefully all is good now.
I still don't know exactly why suddenly everything got broken, after several days of working fine, but I'm hoping this will not happen anymore,
or at least to be able to understand better what happened, by having the answer in the logs :)
Thank you very much for your time.

(notice the prefix "INBOX.", that's why it works)

I don't think this prefix is required, I've tried the Junk filter without the prefix and it was working or maybe I'm missing something.

or at least to be able to understand better what happened, by having the answer in the logs :)

Logs :heart:

I haven't got any errors in "mail.err" since, so hopefully all is good now.

Good, thank your feedback!
I think we can close this issue, feel free to reopen if anything new happens ;)

Closing, again feel free to reopen if it happens again.


Standardize colors of codes in spectrograms

@kfunk074 Here is what I need from you to standardize the colors in the spectrograms. Can you please look through the spectrograms I've created. For each state I'm going to use the same color, then create shades of that color for the different states. I'll do my best to give the shades equal weight but I don't know if that's possible. Can you make a list of (1) the most important state to standardize the colors and (2) the primary code in each state? You can just say it's the earliest code in each state unless there is some reason to prefer a different one.

Not sure I completely understand this. You mean for each state you'll use the same color and then different shade for different codes intra-state? Don't know what equal weight in shades means.
New York is the most important state, and NY1850 the most important version. Within the New York family, NY1829 should ideally be a different color or on the opposite pole of the shade, since it's non-Field Code.
There aren't a lot of states with multiple codes in the other spectrograms. I think the rule should be that the primary code is the latest one. If NC is really borrowing from WI, it's likely the latest version in 1858, and 1856 is only ranked higher due to messy OCR, or, at most, it was consulted secondarily.

That's all I need to know: treat the latest code as the primary, and break out NY1829, and also NY1879/NY1879 into separate colors.

It would be ideal if the color scheme could more or less coordinate with the choropleths. Can New York statutes have varying shades of gold/yellow/orange/brown, California and Far West reds, Midwest and Lower South greens, and Upper Midwest (OH, WI) blues?

Yes, I completely agree. Because we have so many different possible codes
that could be represented, settling on the color scheme had to wait until
we decided which spectrographs we were actually going to use. But now is
the right time.
On Saturday, May 21, 2016, Kellen Funk<EMAIL_ADDRESS>wrote:

It would be ideal if the color scheme could more or less coordinate with
the choropleths. Can New York statutes have varying shades of
gold/yellow/orange/brown, California and Far West reds, Midwest and Lower
South greens, and Upper Midwest (OH, WI) blues?
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/lmullen/civil-procedure-codes/issues/30#issuecomment-220778691

--
Lincoln Mullen, http://lincolnmullen.com
Assistant Professor, Department of History & Art History
George Mason University


Organization catalog

Closes #84
Add

Sorting by name, date
Filtering doesn't work only as placeholder

Changes

Realized pagination using kurounin:paginationpackage
Specified to organization name as unique
Replaced Add organization button to organization catalog
Change route for organization as /organization/slug
Catalog link in navbar has two items

Fututre enhancement

Implement pagination in APIs catalog using kurounin:paginationpackage
Add sorting & filtering
Use query parameters instead of ReactiveVar for sorting & filtering
Do catalog cards via Bootstrap panel

This looks good. Mainly minor improvements.
In the long run, I really hope we avoid maintaining so much custom style code.

@marla-singer Glanced through also, thanks also for enhancing profile-img-container stuff to be more responsive 👍

@brylie Done


Write a test that asserts basic flex volume functionality

We would like to have flex-volume functionality tested during automated tests:

Data input downloaded for job
Data output uploaded from job
Limit on how much can be written to disk by job

@advanderveer can we close this one?

@Overv can we close this issue?

@Overv can we close this issue?


How to use live feed using base64 image panel

Hi,
I have gone through the youtube video, but I didn't find the configurations or is their a way we can use zoom or Microsoft teams

@ChinmoyPadhi As we explained in our videos, we created a live feed based on the images constantly updated in the database and then displayed them using the base64 panel in Grafana.
The dashboard with the image panel should have refresh settings of 500ms-1s to update periodically.
Let me know if it's clear.

@ChinmoyPadhi, I did not hear from you. I hope it's clear.
If not, please let us know.


This change reorganizes the categories to meet the same ppt format

This change reorganizes the categories to meet the same ppt format, adding the same layers to facilitate visualization.
Signed-off-by: Rafael Sene<EMAIL_ADDRESS>

@bprestonlf @Thea-Aldrich this change reorganizes the whole UI to meet the ppt format. We need to fine tune it as we add more items.


TST: patch logger in tests

Some of the tests were modifying the logger in place and mocking the warning method. This breaks the logger for any tests are run after this test, as seen in https://github.com/bilby-dev/bilby/pull/42. I've changed these test to patch the logger instead.

Great catch! I think we should audit our use of mock and see how much of it we can remove.

Yeah, I agree. From a quick look, I think some of it definitely can't be removed (and is probably correct usage of mock)

I'll rebase the branch in #42 after this is merged.


[Docs] Removed typo

name: :memo: Documentation Fix
about: Fixing a problem in an existing docs page
Checklist

[ ] Is there an existing issue for this PR?

Too minor

[ ] Have the files been linted and formatted?

What docs page needs to be fixed?

Section: redux/docs/tutorials/fundamentals
Page: part-4-store.md

What is the problem?
Extra word in the sentence, doesn't make sense.
What changes does this PR make to fix the problem?
Removed word.

Thanks!


Spork sometimes drops semicolons as a result of reusing expression/statement source code

I'm not sure exactly what the problem is, but I don't have time to fix it at the moment. Will have to not reprint expressions and statements.

Need to do some more work to make reprinting expressions and statements stable, but I don't have time for that as part of the thesis.


Update Doc Controller example

Update example in Controller when used in React native, message prop type invalid prop value, in react-native TextInput value returned by onChange must be a string.
This issue was raised in https://github.com/react-hook-form/react-hook-form/issues/769

oh man thank you so much, i have a ticket for myself to update this. you helped me <3.


Update API doc for SystemParametersInfoA/SystemParemetersInfoW on SPI_SETTHREADLOCALINPUTSETTINGS parameter

When we call APIs SystemParametersInfoA/SystemParemetersInfoW to set active input settings using SPI_SETTHREADLOCALINPUTSETTINGS action, the parameter pvParam accepts a BOOL variable casted by PVOID instead of a pointer to a BOOL variable. If we use a pointer to BOOL variable here, the APIs always treat it as Local (per-thread, TRUE) because a pointer to BOOL variable cannot be zero.

@jimmyzhush : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Unexpected lines appear between items in a row

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce

main file:

import 'package:flutter/material.dart';
import 'package:general_test/pixel_issue/page.dart';

void main() {
  runApp(const App());
}

class App extends StatelessWidget {
  const App({super.key});

  @override
  Widget build(BuildContext context) => const MaterialApp(
        debugShowCheckedModeBanner: false,
        home: PageTable(),
      );
}

Page file:

import 'package:busslina_flutter_lightweight_lib/busslina_flutter_lightweight_lib.dart'
    as fllib;

import 'package:flutter/material.dart';

class PageTable extends StatelessWidget {
  const PageTable({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) => Row(
        children: [
          _buildExpandedChild(),
          _buildExpandedChild(),
          _buildExpandedChild(),
          _buildExpandedChild(),
        ],
      ).bgColor(Colors.black);

  Widget _buildExpandedChild() => Expanded(
        flex: 1,
        child: const fllib.Label(
          'Hey',
          textAlign: TextAlign.center,
          color: Colors.black,
        ).sized(height: double.infinity).bgColor(Colors.blue),
      );
}

Expected results
No separation between items. Thought it could be a ´logical pixel´ issue, but it seems isn't, cause I tested with different screen/window sizes.
The lines always get the color of the parent background color.
Also mention that between the 2nd and the 3rd item, no line is shown.
Wrapping it in a Scaffold doesn't change the result.
Actual results

This is a duplicate of #14288
In short, Impeller fixes this for many cases on iOS, and will for Android once that is ready sometime next year.

Will be fixed also for Desktop (my case) and for web?
Can I use a temporary trick to avoid this waiting for the fix?

Thanks

Eventually, but it will take even longer.
If you read the thread, you'll find the package https://pub.dev/packages/pixel_snap linked which provides specific widgets to work around the issue

Thanks again


H3: make WriteFrame take a mutable reference of SendStream

Writing some tests and test helpers, I figured out owning SendStream into WriteFrame brings pain when using it.
In my mind, this technique belongs more to pre-std-futures era. What do you think about this?
In the end, only SendData should own it, and pass it through a pin projection to the underlying layers.

I'm not familiar with the h3 API here, but in general borrowing is much more ergonomic if you want to do anything with the object in question afterwards, or operate on objects stored in places that aren't trivially destructured (which is why even SendStream::finish borrows).

A potentially related issue: I saw some structures that both held a SendStream (which internally holds a ConnectionRef) and a separate ConnectionRef. This seems like a bit of a code smell and should maybe be refactored if possible (or commented). I also worry it might lead to issues with refcounts being weird.

(Looking at the code a little bit, it does indeed look like WriteFrame should not own the SendStream. Let me know if you need help with the pin projections -- it can be a bit fiddly, in my experience.)

Good catch for the ConnectionRef duplication. I imagine you are speaking about SendHeaders and associated. These could be removed once WriteFrame no longer take ownership of SendStream, or turned into SendData methods.
I guess HttpBody refactoring started a change of which this issue matters are part of, and that will benefit the codebase a lot.
If you want to work on this, go ahead. I'll find other opportunities to learn pin projection usage. Plus I'm kind of slow on my current PRs.

In the end it didn't make sense for WriteFrame to hold its own reference to the SendStream, since it's always a part of the SendData that owns the SendStream. Fixed stuff up by replacing the Future impl for WriteFrame with a simple poll_send() method that takes the &mut SendStream explicitly.


[Proposal] Add and make access logs attributes configurable

Summary
At the moment access logs only print a limited number of information. Moreover, there is no way to select the type of information that needs to be printed or the format of the log entry. This proposal is to address these limitations.
Goals

Make access log information configurable
Add new information to access log
Introduce JSON format for access log

Non-Goals

Make the order of access log information configurable

Motivation
It is common to have different information in the access log based on the organization. Therefore, current limitations in access log, hinders the usability of Ballerina.
Description
At the moment one can enable access log using the below configuration for a given HTTP service.
[ballerina.http.accessLogConfig]
# Enable printing access logs on the Console. The default value is `false`.
console = true
# Specify the file path to save the access logs. This is optional.
path = "testAccessLog.txt"

These configurations need to be in the config.toml and the resulting output looks as follows.
<IP_ADDRESS> - - [15/Dec/2022:11:39:42 +0530] "GET /albums HTTP/1.1" 200 95 "-" "curl/7.79.1"

The above output is fixed and the user has no control over it. First thing that is needed to make it configurable is to list down the type information. Following is the key and description of each information that can be printed with the new proposal.

Name
Description

remote_ip
IP of the remote address

verb
HTTP verb of the request

path
Path of the request

schema_and_version
Schema of the request and HTTP version

status_code
HTTP status code

response_size
Size of the response body

request_size
Size of the request body

response_time
Time to took send the response

agent
Name of the agent who sent the request

upstream_ip
IP of the upstream server

upstream_response_time
Upstream server response time

Please note that time and the date of log is not configurable. Then to make this information configurable, a new key is introduced to the config.toml configuration as attributes.
[ballerina.http.accessLogConfig]
# The list of attributes to be printed
attributes = remote_ip, verb, path, schema_and_version, status_code, response_size, agent

One can configure the list of attributes as needed. The above is the default list. Finally, to change the format of access log entry another new configuration is introduced as format. The two formats are flat and json.
[ballerina.http.accessLogConfig]
# The list of attributes to be printed
format = flat

The default value of the format key is flat, which is concise and has least impact on the performance. This can be changed by setting the value to json, which results in the same information in json format.
{
  "remote_ip": "<IP_ADDRESS>",
  "verb": "GET",
  "path": "/albums",
  "schema_and_version": "HTTP/1.1",
  "status_code": 200,
  "response_size": 95,
  "agent": "curl/7.79.1",
  "time_stamp": "[15/Dec/2022:11:39:42 +0530]"
}

Hi Shafreen,
Thank you for the proposal. Let me add a few questions that I need to clarify.

Regarding path	Path of the request I think this is the downstream request. This is important to us. And also, there are some customers who does some changes to the path before sending the upstream request. So its good if we can see the upstream request path too.
status_code - we need to confirm whether this status code originated from the upstream or due to some reason mediation service initiated this. Do we have a way for that? Envoy has an attribute called RESPONSE_CODE_DETAILS which solves a similar problem.
response_time	Time to took send the response Is this the total response time on the listener side right? This is important to us. This means response_time - upstream_response_time gives the latency added by the ballerina mediation service, which is an important measure we want to identify. Hope my understanding is correct.
agent - this is the UserAgent header?
What is the difference between remote_ip and upstream_ip
The other parameters you mentioned (verb, schema_and_version, response_size, request_size) are required too. +1 to include them.

Anther important facts we need to see is:

Can we have another field called requestId which is logged by using an incoming header value. This is important for us to correlate requests throughout the path (nginx LB -> Gateway -> Mediation service) we use the same request id. By default we use the x-request-id header.
What happens to the above log when there's a transport failure during the upstream call (connection refuse, connection timeout, read timeout) and can we identify that from the log?

The default value of the format key is flat, which is concise and has least impact on the performance. This can be changed by setting the value to json, which results in the same information in json format.

If we used json or flat, we will get all the details in a single log line right? Hope we'll not format the json with multiline (coz logs will be overlapped in high concurrency).

+1 for this =) we have the same requirement for BCentral.

[Proposal] Improve AccessLog in Ballerina Services
Ballerina's current HTTP access logging mechanism provides a static and limited view of interactions, lacking the flexibility required by diverse organizational needs. This proposal advocates for a more configurable logging system, introducing options for log content customization, format variation (including JSON), and adding new information types to access logs.
Objectives

Configurable Log Information: Empower users to select and configure the log information relevant to their needs.
Enhanced Log Content: Enrich the access logs by adding new types of information.
Format Flexibility: Introduce JSON as an alternative log format, alongside the existing flat format.

Motivation
Different organizations have unique requirements for logging web access data, driven by varied use cases such as security analysis, performance monitoring, and compliance auditing. The inflexibility in Ballerina's access logging fails to meet these diverse needs, thus limiting its utility and adoption. By enhancing customization capabilities, Ballerina can better serve a wider array of enterprise and developer scenarios.
Description
Current Limitations
Presently, enabling HTTP access logs in Ballerina is constrained to minimal configuration options, as shown below:
[ballerina.http.accessLogConfig]
console = true  # Enables logging to the console.
path = "testAccessLog.txt"  # Optional: Specifies a file path for logging.

 Users can define custom log formats by specifying the desired variables in the nginx.conf configuration file. Here’s an example of how to define a custom log format:
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                '$status $body_bytes_sent "$http_referer" '
                '"$http_user_agent" "$http_x_forwarded_for"';

In addition to this, since Nginx supports a customizable logging mechanism, we can customize the logging format to be JSON as well as below:
log_format json_combined escape=json '{'
    '"time_local":"$time_local",'
    '"remote_addr":"$remote_addr",'
    '"remote_user":"$remote_user",'
    '"request":"$request",'
    '"status": "$status",'
    '"body_bytes_sent":"$body_bytes_sent",'
    '"request_time":"$request_time",'
    '"http_referrer":"$http_referer",'
    '"http_user_agent":"$http_user_agent"'
'}';

In this configuration, escape=json ensures that strings are properly escaped to conform to JSON standards.
Apache HTTP Server
Apache uses modules to extend its functionality, with mod_log_config providing flexible logging capabilities. This module allows for a high degree of customization in how access logs are formatted and what information is logged.
Apache allows administrators to define custom log formats using the LogFormat directive. This directive can include a wide variety of variables, enabling detailed control over what information is logged and its presentation:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" my_custom_format

While Apache does not natively support JSON logging out-of-the-box as simply as Nginx does with its escape=json parameter, you can achieve JSON logging by creatively using its custom log format capabilities.
LogFormat "{ \"remote_ip\": \"%h\", \"time\": \"%t\", \"request\": \"%U%q\", \"status\": %>s, \"referrer\": \"%{Referer}i\", \"user_agent\": \"%{User-Agent}i\" }" json

Proposed Enhancements
Configurable Attributes
To introduce flexibility, we propose listing specific types of log information (e.g., ip, request_method, patrequest_uri, status, response_body_size) that users can selectively include in their logs. The introduction of an attributes configuration in config.toml will allow for such customization:

Name
Description

ip
The IP address of the client making the request

request
The full HTTP request line, including the method, URI, and protocol version

request_method
The HTTP method of the request

patrequest_uri
The URI of the request, including parameters

schema
Schema of the request and HTTP version

status
The HTTP status code sent to the client

response_body_size
The size of the HTTP response body sent to the client, in bytes

request_body_size
Size of the request body

response_time
Time took to send the response

agent
The User-Agent header field, name of the agent who sent the request

upstream_ip
The address of the upstream server that processed the request

upstream_response_time
The time taken by the upstream server to respond

upstream_status
The response status code received from the upstream server

request_time
The total request processing time, from reading the client's request to sending the last byte of the response

remote_user
The user name if HTTP Basic authentication is used

http_referer
The Referer header field of the request, indicating the page from which the user navigated

http_x_forwarded_for
The originating IP address of a client connecting through an HTTP proxy or a load balancer

http_(X-Custom-Header)
Header fields. Referring to them with http followed by the header name (x-request-id -> http_x-request-id)

Note: Most of these field names are common across all other services, we wanted to stick with the common terminology, in most cases we have followed nginx terminology.
[ballerina.http.accessLogConfig]
attributes = "remote_ip, request, status_code, response_body_size, agent, upstream_ip, upstream_response_time, upstream_status"

Log Formats
In addition to the flat format, we propose introducing a JSON format for access logs to provide structured and easily parseable log data. Users can select their preferred format via a new format configuration:
[ballerina.http.accessLogConfig]
format = "json"  # Options: "flat", "json"

The FLAT format example:

No upstream server:

<IP_ADDRESS> - - [15/Dec/2022:11:39:42 +0530] "GET /albums HTTP/1.1" 200 95 "-" "curl/7.79.1"

In the presence of upstream servers:
We use "~" to differentiate downstream and upstream. In the below case, there are two upstream servers, and the information on that is logged as per the defined attributes.

<IP_ADDRESS> - - [15/Dec/2022:11:39:42 +0530] "GET /albums HTTP/1.1" 200 95 "-" "curl/7.79.1" "~" <IP_ADDRESS> - - [15/Dec/2022:11:39:48 +0530] "POST /myAlbums HTTP/1.1" 201 235 "-" "ballerina" <IP_ADDRESS> - - [15/Dec/2022:11:39:55 +0530] "GET /yourAlbums HTTP/1.1" 200 222 "-" "ballerina"

The JSON format example:
{
  "ip": "<IP_ADDRESS>",
  "verb": "GET",
  "path": "/albums",
  "schema": "HTTP/1.1",
  "status_code": 200,
  "response_size": 95,
  "agent": "curl/7.79.1",
  "time_stamp": "[15/Dec/2022:11:39:42 +0530]",
  "upstream": [
    {
      "upstream_ip": "<IP_ADDRESS>",
      "upstream_request": "POST /myAlbums HTTP/1.1",
      "upstream_status": 201,
      "upstream_response_time": 235,
      "upstream_agent": "ballerina"
    },
    {
      "upstream_ip": "<IP_ADDRESS>",
      "upstream_request": "GET /yourAlbums HTTP/1.1",
      "upstream_status": 200,
      "upstream_response_time": 222,
      "upstream_agent": "ballerina"
    }
  ]
}

Conclusion
By enhancing the Ballerina HTTP access log's customization capabilities, we aim to address current limitations and better fulfill the diverse requirements of users and organizations.

@AzeemMuzammil Can we clearly differentiate between the response_time and request_time in the proposal?

Sorry, now only could look at the proposal. As @TharmiganK also mentioned, "response_time	Time took to send the response" is not clear to me too. Can we elaborate a bit more on this?
Is it possible to add upstream_hostname too? (Do not try to reverse dns look up and resolve using the IP. We can log this only if the upstream host is provided)
Other than that, other available fields looks good to me.
@VirajSalaka @Thushani-Jayasekera @renuka-fernando please have a look at the proposal and see if there are anything important missing.

The following are some other suggestions.

In the access log config response_body_size refers to The size of the HTTP response body sent to the client, in bytes. Is it possible to add upstream_response_body_size as well for each upstream?
request_body_size refers to the Size of the request body (from downstream). Is it possible to add upstream_request_body_size where the payload size of the request is sent to the upstream?

@AzeemMuzammil Can we clearly differentiate between the response_time and request_time in the proposal?

As we discussed, we will be removing request_time field as there is no consistent interpretation for this, and response_time captures what are we interested in. (Updated the proposal as per this)

* Is it possible to add upstream_hostname too? (Do not try to reverse dns look up and resolve using the IP. We can log this only if the upstream host is provided)

Yes, we will and we can add the host name as well under the filed name host if host is provided.

* And also, with mediation services, we are using client->execute method and only once during an request. (So the upstream array will only have one json object). Hope this path doesn't cause any issues.

No, this would not cause any issues and should not,

* Are we using the existing ballerina access log config to enable this too? or do we need additional configs?

The existing log config should work fine without any issues, but to enable any additional properties to lot, we have to enable them under the new property attributes introduced in Config.toml, in addition to this I have updated the proposal with default fields and all existing fields.

* As I mentioned earlier too, hope JSON format is printed with a single line by default (I think you mentioned a formatted multiline json only for reading purposes).

Yes, correct

* Can you also tryout a few different scenarios where the backend doesn't response properly, eg:

  * Connection failure (this can be a connection refuse, failed TLS handshake etc)
  * Connection timeout (ballerina client tries to initiate a connection with the backend but the server doesn't respond to the connection request)
  * Response timeout (initial TCP connection with the backend is successful, but server doesn't respond on time with the HTTP response)
  * Client closes the connection before backend responds (the client who calls the ballerina service closes the connection before backend responds)

  These are commonly occurring failures when connecting to backends. Its important to reflect these from access logs. These cases I mentioned are mainly for HTTP/1.1. I think @TharmiganK might know more scenarios which are related to HTTP/2.0.

Sure will check this as well,

* In the access log config `response_body_size` refers to `The size of the HTTP response body sent to the client, in bytes`. Is it possible to add `upstream_response_body_size` as well for each upstream?

Yes, every fields which are available under downstream path will be available for upstream path as well, I have updated this in the proposal to reflect this.

* `request_body_size` refers to the `Size of the request body` (from downstream). Is it possible to add `upstream_request_body_size` where the payload size of the request is sent to the upstream?

Yes, we will have that filed as well.
Please check the responses @TharmiganK @malinthaprasan @renuka-fernando, feel free to ask any questions if available.

Hi @AzeemMuzammil,
Could you please provide an update on the current status?
I am sorry, this request might be coming in a bit late. In recent customer issues, when we’ve noticed delays in their API responses we have to analyse the network traces etc to get further details. After analyzing the situation using TCP dumps and other network tools, we usually find that these delays are often occurring at the connection establishment level. Is it possible for you to log the time taken for connection establishment in the access log itself? It will be a great time saver for us. For connection reuse, the initial request should log a positive value, while subsequent requests using the same connection should log a time of 0, as there's no need for re-establishment.

Also reminder about below as they are very important.
* Can you also tryout a few different scenarios where the backend doesn't response properly, eg:

  * Connection failure (this can be a connection refuse, failed TLS handshake etc)
  * Connection timeout (ballerina client tries to initiate a connection with the backend but the server doesn't respond to the connection request)
  * Response timeout (initial TCP connection with the backend is successful, but server doesn't respond on time with the HTTP response)
  * Client closes the connection before backend responds (the client who calls the ballerina service closes the connection before backend responds)

  These are commonly occurring failures when connecting to backends. Its important to reflect these from access logs. These cases I mentioned are mainly for HTTP/1.1. I think @TharmiganK might know more scenarios which are related to HTTP/2.0.

Hi @AzeemMuzammil,
Could you please provide an update on the current status?
I am sorry, this request might be coming in a bit late. In recent customer issues, when we’ve noticed delays in their API responses we have to analyse the network traces etc to get further details. After analyzing the situation using TCP dumps and other network tools, we usually find that these delays are often occurring at the connection establishment level. Is it possible for you to log the time taken for connection establishment in the access log itself? It will be a great time saver for us. For connection reuse, the initial request should log a positive value, while subsequent requests using the same connection should log a time of 0, as there's no need for re-establishment.

Hi @malinthaprasan as we discussed offline, we will look at this as a separate issue as we are already in line to merge the current changes. To track that I will create an issue.

Also reminder about below as they are very important.
* Can you also tryout a few different scenarios where the backend doesn't response properly, eg:

  * Connection failure (this can be a connection refuse, failed TLS handshake etc)
  * Connection timeout (ballerina client tries to initiate a connection with the backend but the server doesn't respond to the connection request)
  * Response timeout (initial TCP connection with the backend is successful, but server doesn't respond on time with the HTTP response)
  * Client closes the connection before backend responds (the client who calls the ballerina service closes the connection before backend responds)

  These are commonly occurring failures when connecting to backends. Its important to reflect these from access logs. These cases I mentioned are mainly for HTTP/1.1. I think @TharmiganK might know more scenarios which are related to HTTP/2.0.

I have tested out the scenarios mentioned above and we do still log these but not part of access logs, incorporating these into access logs requires a design reconsideration, we will look into this if possible,

I have tested out the scenarios mentioned above and we do still log these but not part of access logs, incorporating these into access logs requires a design reconsideration, we will look into this if possible

In what type of log do we have this information? And how do we correlate them?
Bdw, we need to consider this as an important aspect of access logs. One of the main values of access logs is for troubleshooting. If we don't have this data in the access logs, we'll be missing out on important details..

@malinthaprasan as I mentioned these are not logged as part of access logs, and we have to implement these as part of access logs if required, can you create a new issue for any other requirements that are needed to be part of the access log other than what is mentioned in the proposal. whatever is mentioned in the proposal will be released as part of update 10. and anything in addition we can incorporate in upcoming updates. Thanks,
For the moment, since the feature is implemented and being shipped with update 10, I'll close the issue, if required feel free to open it.

Ack, that is fine.
Is the feature released in an 2201.5.x version now?

No, the feature will be released with 2201.10.0, the reason to close the issue is, that the relevant change is already merged and in the pipeline to release with 2201.10.0 as we planned.


Feature request: ability for octant to impersonate another user

As someone who administers kubernetes clusters, when I use kubectl I have relatively high permissions. I have some aliases set up so I can execute kubectl --as=${LIMITED_USER} ... to take advantage of user impersionation, and run most commands as a lower-privileged user.
It would be nice to have a mechanism like that so I can run octant specifying a user that has readonly permissions, to be able to control which resources octant is able to access, for security reasons and peace of mind.
Maybe an env var like OCTANT_IMPERSONATE_USER

Would it be possible to create another user that has readonly permissions? Then you can configure them both in your kube config file and change contexts as needed.

I have a ClusterRoleBinding like this set up:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: readonly
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: readonly

So I can say kubectl --as readonly ... and all is good in the world.
But that user doesn't have have an entry in our OIDC identity provider; doesn't have a username/password, etc.  There's no way to authenticate as the readonly user. And if I want to configure by kubeconfig file, I need to have some way of authenticating as that user.
I could create a system user, bind it to the view ClusterRole, steal its service account token, and put it into the kubeconfig file. That could work, but I'm not a huge fan of that pattern.
Consider this a low-priority feature request. If I find the time to hack up a PR to implement it myself, I'll take a stab at it.


Help for a dynamic require

I am trying to use ContextReplacementPlugin but I keep getting Error: Cannot find module './example/containers/'.
new webpack.ContextReplacementPlugin(
  /extensions[\\\/]$/,
  /^\.\/(components|otherFolders|xyz)\/[^\/]+\index.js$/
)

To use it like this:
var exampleContainerPath = './example/containers/'
var example = require(exampleContainerPath)

or
var exampleContainerPath = '../extensions/example/containers/'
var example = require(exampleContainerPath)

The relative path I need to match is this:
'../extensions/example/containers'

where example and containers are variables. It may or may not have index.js at the end.

Can you try this against webpack 2?


RFC: Emit enums from graphql schema Enums instead of union types

Summary

It would be great if there was a configuration option to make gql.tada emit typescript enums instead of union types for graphql enums.
There is a huge difference between these, one of them is that a typescript enum is an object that gets it into the transpiled js code, unlike a union type.
Moreover, this would imrpove compatibility of tada.gql with other existing codegen libs, namely @graphql-codegen. This, in turn, would ease the adoption and migration process for existing projects. In my particular case, the lack of this feature is an actual blocker for migrating my existing projects to gql.tada.
Proposed Solution

Let’s say we have a following piece of graphql schema:
query {
  applicant {
    status: APPLICANT_STATUS
  }
}

Schema interpolation should return something like this:
export enum APPLICANT_STATUS {
  ONLINE: "ONLINE",
  OFFLINE: "OFFLINE",
  BLOCKED: "BLOCKED"
}

export type Applicant {
  status: APPLICANT_STATUS;
}

Whereas currently it returns
export enum APPLICANT_STATUS {
  ONLINE: "ONLINE",
  OFFLINE: "OFFLINE",
  BLOCKED: "BLOCKED"
}

export type Applicant {
  status: "ONLINE" | "OFFLINE" | "BLOCKED";
}

Requirements

FWIW, I’m aware we were having this issue, which has been resolved https://github.com/0no-co/gql.tada/issues/44
However, it does not address the issue I’m describing here

This requires explicit codegen for each enum, and doesn't really achieve much. In fact, it introduces more problems that people aren't often aware of in TypeScript.
enums are value-unique and reference-incompatible in values, so once they're generated and used, they often must be used.
const enums on the other hand are still often value-unique, but at least reference-compatible in terms of the values they contain.
Basically to put it short:

Moreover, this would imrpove compatibility of tada.gql with other existing codegen libs, namely @graphql-codegen.
This is not an explicit goal of the project and there are a lot of “default” options that would need to change in GCG to be more compatible with gql.tada. Instead, we often take the approach of doing the right thing and generating correct & compatible types first, by default.

How about the ability to use existing enums, similar to the scalar mapping? I have a fork with the feature as my work needs it to integrate gql.tada in to a legacy codebase - without this the integration is impossible. If this is compatible with the project I can tidy up the code and make a PR. It's a pretty small change.
It does require manually mapping every enum, but that's kind of the point as we want to get away from the legacy eventually.

@sitharus: That's an excellent idea! 💯
I just pushed a PR to implement that, so no worries. I realised that it's a minimal change with Omit<Schema['types'], keyof Scalars> in introspection.ts’ addIntrospectionScalars type, but I believe that may have a hidden cost for TypeScript’s type checker for huge schemas.
The linked PR refactors the types a little, so we can differentiate between generated enum value types and the scalar types. This should also allow us to check enumValues against re-mapped types in the future.


make mlperf_log_accuracy.json optional for perf, few fixes for v0.5

https://github.com/mlperf/inference_policies/issues/165

ok.


Add tgaff implicit wait tests

This PR is aimed at doing 3 things.

Adding a bunch of well written tests from @tgaff - Which have been shamelessly stolen, and very slightly adapted
Tweaking a little bit of documentation, as well as the run order of the Travis jobs (Again thanks to @tgaff)
Implicitly (Badoom-tish), show that actually in the Capybara3 rework branch I worked on. I've fixed the implicit waiting issue. The tests shown here were designed against Travis' code, but actually work fine against both master and the code which this is based off.

Note that due to Point 3 - This item requires #277 to be merged in first (Which will then remove the first 11 commits), at which point a review can be undertaken on the new 5 commits here.
TL;DR - After the code contained in this full PR. The next gem can be released which fixes a couple of bugs, clears some tech debt, and enables Capybara 3 support.

@tgaff Can you review / check you're ok with this. @ineverov also if you have capacity

Ignore the :red_circle: - One timeout on one of the tests.

@luke-hill Thanks for getting this in.  And sorry, I've been too overwhelmed to keep up with stuff.

No probs. 2.15 introduced Capybara3 capabilities (Limited), and fixed the implicit waiting.
2.15.1 fixed a load of dev side code, 1 major bug (iFrame creation inside sections), and upped some dependencies - Max capybara is now < 3.3


Remove max_date if target_date is expired

Set max_date to 0 if target_date already expired.

Done in #45.

Done in #45.


[BUG]: Bun installation error

Version
0.5.3
OS
Windows
Describe the bug
vfox install bun@latest
plugin [PreInstall] method error: no version number provided
failed to install bun

Actually there is fix in ahai-code/vfox-bun#2 but it doesn't work for latest keyword. Also vfox-bun repo seems to be not maintained for 2 months already so any attention will be appreciated.

The version codes such as latest are not necessarily designed by the plugin author. Here is a temporary patch to enable the use of latest, add the following code before the loop in the PreInstall function of the bun plugin:
    table.sort(releases, function(a, b)
        return util:compare_versions(a, b)
    end)
    if ctx.version == "latest" then
        return releases[1]
    end

I have fixed this issue. Please upgrade the plugin to 0.1.2.


The component relative_file_name must be shown for existed empty file

...but it doesn't, because use is_buffer_not_empty condition

unrelated anymore


Description option is required

I have error "Description is required." After I added descriptions in package.json, everything is ok.
It's not very important, but...

@Partysun
did you succeeded build?
I have same error "Description is required".
so I added description in package.json,
but another error occured. "File not found : LICENSE"
what is LICENSE file? how to create it?
i don't know. how to fix it.
could you help me?
here is my gruntfile.js and package.json
gruntfile.js
module.exports = function(grunt) {

  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    'create-windows-installer': {
      x64: {
        appDirectory: '../electron/',
        outputDirectory: '/output',
        authors: 'My App Inc.',
        exe: 'myapp.exe'
      }
    }
  });

  grunt.loadNpmTasks('grunt-electron-installer');

  // Default task(s).
  grunt.registerTask('default', ['create-windows-installer']);
};

package.json
{
  "name": "test",
  "version": "0.1.0",
  "description": "An example module to illustrate the usage of a package.json",
  "main": "main.js"
}

@Partysun what can you suggest to make things more understandable?

It's required. Can be closed.


Multithreaded mismatch

This line on both tacopie and cpp_redis CMakeLists.txt is causing problems on my environment (VS2015 with CMake 3.7):
string(REPLACE "/MD" "-MT" ${flag_var} "${${flag_var}}")
I get errors like error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MTd_StaticDebug' doesn't match value 'MDd_DynamicDebug' .
I just commented out these lines and it builds. But this makes it impossible to use this lib simply as a submodule in my project.

Hi,
Sorry for the time to reply, I finally find the time to have a look.
I guess cpp_redis compiled fine when you tried to integrate it but the errors showed up when trying to link it with your project?
This is due to the Runtime Library configuration.
Apparently, most recent versions of VC++ require the Runtime Library to be the same for every library linked into the project.
So my guess is that our current configuration that forces the use of MT (static runtime library configuration) creates a conflict with your project or another library of your project which is based on MD configuration.
So we can' t really just comment out/remove these lines because, even if it will solve your issue, it will create an issue for other people that didn' t have one until now because their project is based on MT configuration.
So instead, I provided a new CMake variable, MSVC_RUNTIME_LIBRARY_CONFIG for which you can assign the configuration you want (/MT, /MD or other).
By default, this variable is set to /MT (as before).
Hope this will solve your issue :)
Feel free to re-open this issue if necessary.
Best


Proper release cycle for 0.28+

In order to avoid issues like we had with the latest secp256k1 release I propose to do a series of rust-bitcoin RC releases before releasing 0.28.0 (and all future major versions).
Why this is required and we can't just use github master branch from a dependencies to check before the release? One of the main reasons is because there are multiple projects not just depending on rust-bitcoin, but also depending on other crates, which expose APIs using rust-bitcoin. At least, this includes miniscript, electrum-client and many others, so every crate using them can't just switch to github master for rust-bitcoin.
So, the proposal is to have rust-bitcoin 0.28.0-rc.1 first, than do the same for miniscript - and let existing projects using both of them to test these RCs for a while.

I think cargo doesn't select RC versions by default (or was it cargo add?) so maybe the difference is not that big but what you're proposing seems like an improvement over current situation.

I mean that downstreams will manually use bitcoin = "0.28.0-rc.1" and miniscript = "7.0.0-rc.1" and be able to test the build

Isn't it the same amount of work? You have to change a version in each crate or you have to change it to git declaration.
It looks like this is a job for an automated tool.

No, it is not just about amount of work: this is about third-party repos which you can't change. If they don't do rc release (miniscript etc), there is no way for you to build your repo with the new rust-bitcoin other then to do a fork each of them and set them to use rust-bitcoin from a github reference.
They may do a custom branch/tag, but this will be much less standard and hard to coordinate - plus can change anytime, unlike the RC version released to crates.io.

Not the same amount of work: there is no need to convince miniscript (the same people as rust-bitcoin) and for electrum-client they are usually doing this anyway (also @RCasatta here can do that since he is a part of that project)

comcept ACK for electrs

I think we really are asking too much of downstream projects, I don't see why downstream should have to beta test our releases. Testing rc releases is the job of rust-bitcoin devs IMO. Another solution would be to come up with a set of projects that gives us good enough coverage and then fork those someplace (all into a single repo?) that we can all see, then do the rc testing there. That way the work is not duplicated and everyone can see the PRs to see whats involved with updating without having to do it themselves. Sounds like a suitable boring task for someone who is on an open source grant to do ...

@tcharding we have several downstream projects (rust-elements, rust-miniscript, rust-lightning, electrs) which are maintained by the rust-bitcoin developers.
The idea behind using rcs on crates.io is that it'd be more convenient, but functionally the same, as using git branches.

Got it, cheers.

Reporting back after some initial testing. Using crates.io is not going to work because any bugs one finds that break the build of downstream projects have to be patched, this implies using branches either pushed to github or locally using path.
Perhaps this flow will work?
1- tag a commit with rust-bitcoin-0.28.0-rc.1
2- devs checkout this branch for testing then checkout their own local branch for patching (e.g. rust-bitcoin-0.28.0-rc.1-test)
3- Raise PRs against master as usual
4- Every x hours/days/patches do a new tagged release candidate
5- Once it stabilizes tag and do the real release on crates.io

@tcharding not sure why this not going to work; I've been using this for my crates (I have >50 of libs) for years and it worked well.
You start with 0.28.0-beta.1 and do then 0.28.0-beta.N as patches, then move to 0.28.0-rc.1 once all bugs are fixed, have a few RCs more if needed and then do a full 0.28.0 release.

I agree with @dr-orlovsky -- it is true that when users are testing patches locally they have to mess around with local paths or git branches, but the idea is that we'd be in "rc mode" only for a short time, and have quick turnaround pushing new rcs with new bugfixes.

I've not fully internalised your position but I'm happy to defer to you guys on this, I'm not particularly experienced in this are. One question, this method implies that every project in the stack is ready to do a release or were you imagining that some projects might leave the rc release up for a longish time?

We have a 'feature freeze' while rc releases are active, right? I.e no PRs merged except bug fixes?

In order to try and assist those trying to merge patches that are needed during the release candidate cycle I added a label RC fix. Please remove/rename/recolor as you see fit.

I think we should. Also good idea with the label.

Thanks! Agreed.
Though there is a bit of a blurry line -- if we add e.g. a forgotten #derive, is that a "new feature"?

Yeah I was wondering about this sort of thing. I really don't want to be the guy that puts a broken patch into the final rc release that doesn't get that heavily tested because its so late in the cycle and then gets into the non-rc release. Perhaps this gives us the answer to that question, if its early in the rc cycle then it counts as a bug fix and if its nearing the end of the rc cycle be more conservative?

I think we should weight in the scope too. Adding an uncontroversial derive or a simple access method should be fine. Adding a whole module with a bunch of types and methods is not.

Are we doing another RC release, its been a while on rc-1 now?

@tcharding  I would like to get a rust-secp major release out, and then I think we can do a major release here. Or at least another RC. I have spent 4 or 5 weeks now begging to get PRs acked by maintainers but there are not many on that project.


API Gateway Options

Hello ,

I want to integrate kubewatch with my lambda functions, is there any way to do this with webhook that generated by API Gateway ?

I have created simple API Gateway endpoint but kubewathc is returning invalid_auth error .

what authentication methods do you need?


partitioned runs may be scheduled even if one is already scheduled.

Testing, having the runs daily, s3 full, so multiple runs scheduled until it was noticed.
Ideally, should not happen.

probably don't use the context time as part of the run key
https://github.com/earthcube/scheduler/blob/8f8100139dea2451885c6cb0bfb894d7eb836d04/dagster/implnets/workflows/ingest/ingest/sensors/gleaner_summon.py#L73

This will continue to happen.
If we use the same run_key in the sensor, it only runs once.
If we use a cursor-based one, then it will could run more than once.
We might call the graph ql api to see what is running... can't find an api call.

Looks like
instance.get_runs here is an discussion from dagster github
or get_run_records


Add configuration for Spring Data MongoDB DocumentReference annotation.

This PR makes sure to register required proxies and configuration for fields annotated with @DocumentReference.

Superseded by #1312.


Can I use scope as I can with db-migrate?

Background: https://stackoverflow.com/questions/49498595/how-can-i-use-node-pg-migrate-with-independent-lanes-of-migrations
Currently, I have this as package.json
{
  "devDependencies": {
    "node-pg-migrate": "2.15.0",
    ...
  },
  "scripts": {
    "migrate:up": "node-pg-migrate up",
    "migrate:down": "node-pg-migrate down 20",
    "create": "node-pg-migrate create",
    "unlock": "node-pg-migrate unlock",
     ...
  },
  ...
 But you can emulate them IMO in a way, that you will use separate migrations folder and table:
node-pg-migrate -m migrations/dll -t pgmigrations_dll
node-pg-migrate -m migrations/dml -t pgmigrations_dml
Would it work for you?

Completely! It's working. Thank you.


Fix PostgreSQL major version detection and use for binaries & data paths

Description
This change corrects the PostgreSQL version detection logic to allow for proper validation and upgrades between major PostgreSQL releases. It introduces a new PgVersion class (built off of Gem::Version) to properly parse and extract major versions.
This does not correct for the need to use pg_upgrade for any past database versions to current (since this change properly sets the data directory path to use the major version), but will address any issues with 13.x → 13.y minor version upgrades going forward.
Issues Resolved
#2811
Tests
pg 9.6 -> 13.3 [???]
???
pg 13.3 -> 13.3 [14.9.23 -> 14.9.28+20210928031537]
https://buildkite.com/chef/chef-umbrella-main-chef-server/builds/719
pg 13.3 -> 13.3 [14.9.28+20210928031537 -> 14.9.28+20210928031537]
https://buildkite.com/chef/chef-umbrella-main-chef-server/builds/721
pg 13.3 -> 13.4 [14.9.28+20210928031537 -> 14.9.28+20210928222410]
https://buildkite.com/chef/chef-umbrella-main-chef-server/builds/723
Thanks
Many thanks to Aaron Pavely for this submission.
https://github.com/apavely

See my last commit (931ff39aa9418f67e50908ad27aa9f1712ed0a49) and comment on #2827  ... this should make the need for building older minor releases moot.

See my last commit (931ff39) and comment on #2827 (comment) ... this should make the need for building older minor releases moot.

Will do - thanks.


Running on docker containers

First of all, thanks for your code it works fine on my localhost but I've got something strange when moving your code in a docker container  on a KVM VM.
So I have a small VM from Lunanode in Roubaix and set up a docker container using a docker build with all your code on python3.4.
All is good till I got a trouble in line 177 of your Pack py program. I have a 404 error but I don't why the beautifulsoup parser find the downloaded url but failed afterwards for some reason but all works on my localhost. Did I miss something when moving to a container?
Thanks

Hello, honestly i have no idea what kind of a problem it might be, try to debug it deeper and share your solution when you are done,
Thanks in advance,
BR,
łukasz


An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail.

Unable to connect using to deploy plugin, tried leaving server name empty as well but still no luck.  I sync the time with server using command prompt but still getting the error.  I'm on latest for all nuget packages as well and spkl.
spkl Task Runner v<IP_ADDRESS>     Tasks v<IP_ADDRESS>
Enter a CRM server name and port [crm.dynamics.com]: crm4.dynamics.com
Is this organization provisioned in Microsoft Office 365 (y/n) [y]: y
Enter Username<EMAIL_ADDRESS><EMAIL_ADDRESS>Enter Password: ***************
The application terminated with an error.
An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail.
Server stack trace:
at System.ServiceModel.Channels.SecurityChannelFactory1.SecurityRequestChannel.ProcessReply(Message reply, SecurityProtocolCorrelationState correlationState, TimeSpan timeout)    at System.ServiceModel.Channels.SecurityChannelFactory1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at System.ServiceModel.Security.IWSTrustContract.Issue(Message message)
at System.ServiceModel.Security.WSTrustChannel.Issue(RequestSecurityToken rst, RequestSecurityTokenResponse& rstr)
at Microsoft.Xrm.Sdk.Client.ServiceConfiguration1.Issue(AuthenticationCredentials authenticationCredentials)    at Microsoft.Xrm.Sdk.Client.ServiceConfiguration1.AuthenticateInternal(AuthenticationCredentials authenticationCredentials)
at Microsoft.Xrm.Sdk.Client.ServiceConfiguration1.AuthenticateFederatedTokenWithOrgIdForCRM(AuthenticationCredentials authenticationCredentials)    at Microsoft.Xrm.Sdk.Client.ServiceConfiguration1.AuthenticateOnlineFederationInternal(AuthenticationCredentials authenticationCredentials)
at Microsoft.Xrm.Sdk.Client.ServiceConfiguration`1.Authenticate(AuthenticationCredentials authenticationCredentials)
at Microsoft.Xrm.Sdk.Client.DiscoveryServiceConfiguration.Authenticate(AuthenticationCredentials authenticationCredentials)
at Microsoft.Crm.Sdk.Samples.ServerConnection.GetProxy[TService,TProxy](Configuration currentConfig)
at Microsoft.Crm.Sdk.Samples.ServerConnection.GetDiscoveryProxy()
at Microsoft.Crm.Sdk.Samples.ServerConnection.GetOrganizationAddress()
at Microsoft.Crm.Sdk.Samples.ServerConnection.GetServerConfiguration(Boolean ignoreLocalPrincipal)
at SparkleXrmTask.Program.Run(CommandLineArgs arguments)
at SparkleXrmTask.Program.Main(String[] args)

I'm wondering if this is not because our org uses multi factor authentication.
I tried creating application user with a client id and secret to authenticate using that method but I get this error:
The application terminated with an error.
clientCredentials.UserName.UserName or clientCredentials.Windows.ClientCredential.UserName MUST be populated!
at Microsoft.Xrm.Sdk.ClientExceptionHelper.Assert(Boolean condition, String message)
at Microsoft.Xrm.Sdk.Client.ServiceConfiguration1.GetIdentityProvider(ClientCredentials clientCredentials)    at Microsoft.Xrm.Sdk.Client.ServiceConfiguration1.AuthenticateOnlineFederationInternal(AuthenticationCredentials authenticationCredentials)
at Microsoft.Xrm.Sdk.Client.ServiceConfiguration`1.Authenticate(AuthenticationCredentials authenticationCredentials)
at Microsoft.Xrm.Sdk.Client.DiscoveryServiceConfiguration.Authenticate(AuthenticationCredentials authenticationCredentials)
at Microsoft.Crm.Sdk.Samples.ServerConnection.GetProxy[TService,TProxy](Configuration currentConfig)
at Microsoft.Crm.Sdk.Samples.ServerConnection.GetDiscoveryProxy()
at Microsoft.Crm.Sdk.Samples.ServerConnection.GetOrganizationAddress()
at Microsoft.Crm.Sdk.Samples.ServerConnection.GetServerConfiguration(Boolean ignoreLocalPrincipal)
at SparkleXrmTask.Program.Run(CommandLineArgs arguments)
at SparkleXrmTask.Program.Main(String[] args)

Any help would greatly appreciated

The issue is your MFA, spkl doesn't work with O365 MFA. You can test this by spinning up a trial environment and walking through the connection steps.

Duplicate of #308


TypeScript error TS2430: Interface 'Moment' incorrectly extends interface 'Object'.

Full error message using TypeScript 2.7.1:
ERROR in [at-loader] ./node_modules/moment/moment.d.ts:413:13
    TS2430: Interface 'Moment' incorrectly extends interface 'Object'.
  Types of property 'valueOf' are incompatible.
    Type '() => number' is not assignable to type '() => Object'.
      Type 'number' is not assignable to type 'Object'.

This affects versions 2.18.0 and up, based on the following commit:
https://github.com/moment/moment/commit/19b313d1c8cf5e2aa4577c9cc548efcffebc787b#diff-3dd6cebe542b0e207494192c4bbf4a22

valueOf is used specially in moment to return the number of ms since the Unix Epoch:
https://momentjs.com/docs/#/displaying/unix-timestamp-milliseconds/
Do you have a recommended fix? For now, the typings need to be 1.x compatible

TS 1.x vs 2.x. Expect a fix with 3.x

@ichernev I recently upgraded my typescript version to 3.x and I was still getting this issue. Anyone confirm that this was a fix for this issue?

In the meantime, I've added this setting to tsconfig.json to prevent this suppress this error:
{
  "compilerOptions": {
    "skipLibCheck": true,
    ...

Thanks, this exactly helps !


Feat: refine cipher and network config

An error will be returned when using an unknown cipher of Shadowsocks.
Could parse network from slice and string.
net.NETWORK_TCP will be returned when the network config is empty. It fixed an error will be returned when the network of dokodemo is empty.

Codecov Report

Merging #1436 (4ffa616) into master (b960d3f) will increase coverage by 0.01%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1436      +/-   ##
==========================================
+ Coverage   39.59%   39.60%   +0.01%
==========================================
  Files         598      595       -3
  Lines       35064    34951     -113
==========================================
- Hits        13882    13843      -39
+ Misses      19657    19596      -61
+ Partials     1525     1512      -13

Impacted Files
Coverage Δ

app/router/config.pb.go
13.29% <0.00%> (ø)

app/router/router.go
29.19% <0.00%> (ø)

common/net/network.go
34.88% <0.00%> (-13.51%)
:arrow_down:

proxy/dokodemo/config.pb.go
25.38% <0.00%> (ø)

proxy/dokodemo/dokodemo.go
55.20% <0.00%> (ø)

proxy/shadowsocks/config.go
67.37% <0.00%> (-3.52%)
:arrow_down:

proxy/shadowsocks/simplified/config.go
3.33% <0.00%> (ø)

proxy/shadowsocks/simplified/config.pb.go
17.54% <0.00%> (ø)

app/router/command/errors.generated.go
0.00% <0.00%> (-100.00%)
:arrow_down:

transport/internet/domainsocket/config.go
52.63% <0.00%> (-10.53%)
:arrow_down:

... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b960d3f...4ffa616. Read the comment docs.


Fix test_redfishtool_local_user.robot Verify_Error_While_Creating_User_With_Invalid_Role Teardown

https://github.com/openbmc/openbmc-test-automation/blob/a053fb3472c2151d0f41667abfb035e2d1d320ae/redfish/dmtf_tools/test_redfishtool_local_user.robot#L116
User "UserT100" is never created in test case, therefore teardown should expect 404.
[Teardown]  Redfishtool Delete User  "UserT100"  404

@rahulmah can u look into this.

Assigned to @manashsarma .. Please looked into it..

I am looking this. Thanks.

https://gerrit.openbmc-project.xyz/#/c/openbmc/openbmc-test-automation/+/38869/


Connection to local machine node

how you connect to the local node (local machine) of a user client ?
I need the exact location of the source code.
thank you in advance

@bouharb curious about why you need to set the API address on the code itself?

how you connect to the local node (local machine) of a user client ?

On the welcome screen (/#/welcome), you have the option to set the API address to connect to.

I need the exact location of the source code.

You can set the defaultApiAddress to connect to here:
https://github.com/ipfs-shipyard/ipfs-webui/blob/0b5a73244faaa508de7a5a29c687cf69a9b56040/src/bundles/index.js#L31-L46


Use LaTex to represent chi-squared results

No need to implement this, we will represent it in latex in the final publication.


[Feature request] - Multiselect nodes by hot keys & public service for programmatic actions

Hi, first of all: Awesome library. I looked through the code and I'm in awe.

Now to the feature request:
I'd love to have multi select capability for the nodes.
With the usual CMD/STRG + click to add or remove node to selection. The current selectable directive will deselect all previously selected nodes.
Also: Making the SelectionService available in the library would allow us to select nodes programmatically. That would make our routed flow easier to implement. Currently I built it kind of hacky by manually triggering the mousedown event on the node.

Hi, thank you for the positive feedback! The code might be a bit complex, but I prioritize providing a simple and convenient API, even if it requires a more intricate implementation.
Yeah, I think I'll avoid reinventing the wheel and implement this similarly to how React Flow does it, using CMD + click as the default combination, while allowing the option to customize it.

I implemented the configurable keyboard shortcuts with the node multiselection support, could you please check if it's fine for you?


Lcs col is not updated to true when deploy failed, but recovered

What happened:

Lcs appears to be inconsistently reported in the kapp list -A output when kapp-controller fails to install an app and then subsequently is made to succeed. (Needs reproduction/confirmation)

this is probably because Lcs is set to false on the failed deploy, eventually resources succeed reconciling and subsequent kapp deploy results in being a noop (no changes). i dont believe in this case we "double check" that Lcs is switched back to true because we do not generate new app change -- should we?
Environment:

kapp version (use kapp --version):
OS (e.g. from /etc/os-release):
Kubernetes version (use kubectl version)

Vote on this request
This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.
👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"
We are also happy to receive and review Pull Requests if you want to help working on this issue.

I thought of creating this issue a while back but later on didn't because I thought it doesn't make sense to generate a new app change when there is no change, and the column itself says Lcs and Lds (Last Deployment...). But I have seen a user getting confused because of this, so we should definitely give some thoughts to it.

So the scenario is.

We do kapp deploy — which fails and LCS is set to false
Reconcile in the background succeeds after some time.
We do kapp deploy again (subsequent deploys as in case of kapp-controller), since there is no change we do not deploy.

But suggestion here is that if LCS is false, check the status of each resource and if its successful mark LCS as true?
what happens to LCA?
Question, how will we know that resources have reconciled successfully?

i think we should do an app change which should just record that reconcile succeeded "behind the scenes".