Unable to update or remove Traffic Ops in Docker

I'm submitting a ...

[x] bug report
[ ] new feature / enhancement request
[ ] improvement request (usability, performance, tech debt, etc.)
[ ] other

Traffic Control components affected ...

[x] CDN in a Box
[ ] Documentation
[ ] Grove
[ ] Traffic Control Client
[ ] Traffic Monitor
[x] Traffic Ops
[ ] Traffic Ops ORT
[ ] Traffic Portal
[ ] Traffic Router
[ ] Traffic Stats
[ ] Traffic Vault
[ ] unknown

Current behavior:
When installing a Traffic Ops RPM, the RPM's %pre script checks to see if we are upgrading the package (which includes reinstalling). If so, it attempts to stop the traffic_ops systemd service. However, unprivileged docker containers cannot use systemd, so it is running in an unprivileged docker container, this causes the %pre script (and the whole upgrade) to fail.
This issue also affects removing Traffic Ops in unprivileged docker containers.

Expected / new behavior:

Traffic Ops should upgrade or remove uninhibited by the environment's ability to use systemctl.
Minimal reproduction of the problem with instructions:

Copy the RPM into a centos:7 docker container
yum -y install <the RPM>
yum -y reinstall <the RPM>

And:

Copy the RPM into a centos:7 docker container
yum -y install <the RPM>
yum -y remove <the RPM>

Anything else:
This is blocking #4247.
See console output below. The relevant error is Failed to get D-Bus connection: Operation not permitted.

Expand to see the console output for reinstalling.
[root@8d16eeb6ab0e /]# yum -y reinstall traffic_ops-4.0.0-10407.bfa61795.el7.x86_64.rpm
Loaded plugins: fastestmirror, ovl
Examining traffic_ops-4.0.0-10407.bfa61795.el7.x86_64.rpm: traffic_ops-4.0.0-10407.bfa61795.el7.x86_64
Resolving Dependencies
--> Running transaction check
---> Package traffic_ops.x86_64 0:4.0.0-10407.bfa61795.el7 will be reinstalled
--> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================================================================================================================================================================================================
 Package                                                 Arch                                               Version                                                               Repository                                                                                Size
=================================================================================================================================================================================================================================================================================
Reinstalling:
 traffic_ops                                             x86_64                                             4.0.0-10407.bfa61795.el7                                              /traffic_ops-4.0.0-10407.bfa61795.el7.x86_64                                              26 M

Transaction Summary
=================================================================================================================================================================================================================================================================================
Reinstall  1 Package

Total size: 26 M
Installed size: 26 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
trafops:x:997:
trafops:x:999:997::/opt/traffic_ops:/sbin/nologin

Backing up config files.

tar: app/local: Cannot stat: No such file or directory
tar: app/cpanfile.snapshot: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
Failed to get D-Bus connection: Operation not permitted
error: %pre(traffic_ops-4.0.0-10407.bfa61795.el7.x86_64) scriptlet failed, exit status 1
Error in PREIN scriptlet in rpm package traffic_ops-4.0.0-10407.bfa61795.el7.x86_64
  Verifying  : traffic_ops-4.0.0-10407.bfa61795.el7.x86_64                                                                                1/1

Failed:
  traffic_ops.x86_64 0:4.0.0-10407.bfa61795.el7

Complete!

Expand to see the console output for removing.
[root@8d16eeb6ab0e /]# yum -y remove traffic_ops
Loaded plugins: fastestmirror, ovl
Resolving Dependencies
--> Running transaction check
---> Package traffic_ops.x86_64 0:4.0.0-10407.bfa61795.el7 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================================================================================================================================================================================================
 Package                                                 Arch                                               Version                                                              Repository                                                                                Size
=================================================================================================================================================================================================================================================================================
Removing:
 traffic_ops                                             x86_64                                             4.0.0-10407.bfa61795.el7                                             @/traffic_ops-4.0.0-10407.bfa61795.el7.x86_64                                              26 M

Transaction Summary
=================================================================================================================================================================================================================================================================================
Remove  1 Package

Installed size: 26 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Failed to get D-Bus connection: Operation not permitted
error: %preun(traffic_ops-4.0.0-10407.bfa61795.el7.x86_64) scriptlet failed, exit status 1
Error in PREUN scriptlet in rpm package traffic_ops-4.0.0-10407.bfa61795.el7.x86_64
  Verifying  : traffic_ops-4.0.0-10407.bfa61795.el7.x86_64                                                                                1/1

Failed:
  traffic_ops.x86_64 0:4.0.0-10407.bfa61795.el7

Complete!

Should we change the base image of infrastructure/cdn-in-a-box/traffic_ops/Dockerfile from FROM centos:7 to FROM centos/systemd? I believe that would achieve what you want, as well as generally making systemd commands work on CiaB containers.

systemd is included already in centos:7. centos/systemd also requires the container to be privileged in order to work and fails with the same error if it is unprivileged. Do we want CiaB to run in privileged containers?

Ah, you're right, systemctl stop traffic_ops fails. I have an unprivileged centos/systemd container, and always use service traffic_ops stop, which gives the same DBUS message, but succeeds. Odd.
We could change the RPM to call service traffic_ops stop instead. But we'd also have to make it require the initscripts package. Which I have mixed feelings about. It's part of the Linux Standard Base, IMO supporting/requiring LSB isn't a bad thing. But I don't want to start a SystemD holy war.
It's also frustrating to have to deal with SystemD and Docker not playing well together. On one hand, I'm not sure we should be changing the Prod RPM for the CiaB. On the other, someone wanting to use Docker for their Prod system isn't unreasonable.

Hm, this looks promising: https://developers.redhat.com/blog/2016/09/13/running-systemd-in-a-non-privileged-container/

I personally am still fully in favor of not using RPMs at all in CiaB, which would solve this by not requiring a daemon manager at all.

If the goal is only to unblock #4247 the yum reinstall could be changed to something like:
yum deplist traffic_ops-4.0.0-10414.5ac65099.el7.x86_64.rpm |
    grep provider: |
    awk '{print $2}' |
    xargs yum -y install

That would avoid re-running the RPM's %pre script. Upgrades are are not as common in Docker anyway, in practice one would instead build a newer image.

No longer blocking #4247.

I personally am still fully in favor of not using RPMs at all in CiaB, which would solve this by not requiring a daemon manager at all.

That has advantages and disadvantages. One of the biggest disadvantages is making the CiaB less like Production environments. Which makes both development and testing more artificial, and more likely to miss something and introduce a bug, potentially even bringing down the Production deployment.

CiaB isn't really used for too much testing anyway, as I understand.


Client.getCurrentChatMessage() doesn't work

No errors but it also doesn't return anything. Setting the chat message works fine, just trying to get the string doesn't

Does setting the chat message work when the chat gui is already open?

Setting it works fine, even if the chat GUI isn't open

Fixed in #100


Support for multiple controllers or joysticks

Patch indicating how multiple controller support can be added - essentially, just passing the SDL numeric ID into "on_controller" to let the application to separate inputs per controller
https://gist.github.com/grrussel/6bf44e5b058af61865ecc4a70a2c0753

Yea, makes sense to me!

See pull request #40


PLEASE HELP VirtualHMD Ping Pong Ball tracking not getting to SteamVR

I have everything working except my VirtualHMD tracking in Steamvr.  In Stream Home I can move around by teleporting but I can't walk around in my play area.  I think I've found the error in the PSMove Log.  Both Moveservice and Freepie are running and VirtualHMD ID:0 is activated before launching Trinus and SteamVR.
2018-12-12 20:50:05.254]: ClientConnection::start - Starting client connection id 0
[2018-12-12 20:50:05.255]: ClientConnection::send_connection_info - Sending connection id to client 0
[2018-12-12 20:50:05.326]: ServerRequestHandler - Start controller(0) stream (pos=1,phys=1,raw_sens=0,cal_sens=0,trkr=0,roi=0)
[2018-12-12 20:50:05.331]: ServerRequestHandler - Start controller(1) stream (pos=1,phys=1,raw_sens=0,cal_sens=0,trkr=0,roi=0)
[2018-12-12 20:54:45.219]: ServerRequestHandler - Stop controller(1) stream
[2018-12-12 20:54:45.364]: ClientConnection::handle_tcp_read_request_header - Failed to read header on connection 0: An existing connection was forcibly closed by the remote host
[2018-12-12 20:54:45.369]: ClientConnection::stop - Stopping client connection id 0
[2018-12-12 20:54:45.371]: ServerNetworkManager::handle_udp_read_connection_id - Failed to receive UDP connection id: No connection could be made because the target machine actively refused it
I have some other logs but will have to post them later when I get back home.

Do you have driver4vr? You MUST have it installed to tracking data be sent to SteamVR.


Removes deprecated MINIMUM_STAKE_DELEGATION

Problem
MINIMUM_STAKE_DELEGATION was deprecated in v1.11 and can be removed.
Summary of Changes
Remove it.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81.9%. Comparing base (14d6c79) to head (da49980).
Report is 12 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##           master     #963     +/-   ##
=========================================
- Coverage    81.9%    81.9%   -0.1%
=========================================
  Files         853      853
  Lines      231797   231801      +4
=========================================
+ Hits       189845   189846      +1
- Misses      41952    41955      +3


Update sbt-site to 1.4.1

Updates com.typesafe.sbt:sbt-site from 1.4.0 to 1.4.1.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.typesafe.sbt", artifactId = "sbt-site" } ]

labels: sbt-plugin-update, semver-patch

Coverage remained the same at 89.873% when pulling 4ec7f29a8cd8531e10373780e13b947d0879b277 on scala-steward:update/sbt-site-1.4.1 into 0cef271050e52973eee79b37c91e3c1d91cf96f6 on sangria-graphql:master.

Coverage remained the same at 89.873% when pulling 4ec7f29a8cd8531e10373780e13b947d0879b277 on scala-steward:update/sbt-site-1.4.1 into 0cef271050e52973eee79b37c91e3c1d91cf96f6 on sangria-graphql:master.


Post request to API by endpoint "https://api.trader.xyz/orderbook/order" gives out an error related to cors

Unable to repro this anymore


Log error Globus events to stderr

A Parsl script may "hang" waiting for a Globus file transfer when in fact Globus is retrying to transfer a file that does not exist until a Globus transfer deadline is reached. Parsl should log error Globus events to stderr besides logging to runinfo/NNN/parsl.log.

It has been fixed and tested by @yadudoc .


[BUG] Illegal Argument exception when using AWSCLI v2 with base64 encoded data

Describe the bug
When trying to receive published, base64 encoded data using the AWS CLI v2, kinesis mock reports an Illegal base64 character 22
To Reproduce
Steps to reproduce the behavior:

Running kinesis mock with docker run --rm -p 4566:4566 --name kinesis-mock -e KINESIS_MOCK_PLAIN_PORT=4566 ghcr.io/etspaceman/kinesis-mock:0.4.4
Run the test script, and watch the logs:

#!/bin/bash
set -euxo pipefail
AWS=${AWS:-awslocal}
KINESIS_STREAM_NAME=test-stream

${AWS} kinesis create-stream --stream-name $KINESIS_STREAM_NAME --shard-count 1 --stream-mode-details StreamMode=PROVISIONED

base64_content=$(echo '{"test": "data"}' | base64)

${AWS} kinesis put-record \
  --stream-name $KINESIS_STREAM_NAME \
  --data  ${base64_content} --partition-key id

SHARD_ITERATOR=$(${AWS} kinesis get-shard-iterator --shard-id shardId-000000000000 --shard-iterator-type TRIM_HORIZON --stream-name $KINESIS_STREAM_NAME --query 'ShardIterator');

${AWS} kinesis get-records --shard-iterator $SHARD_ITERATOR

Using aws cli v2:
aws-cli/2.8.3 Python/3.9.11 Linux/6.6.6-arch1-1 exe/x86_64.arch prompt/off

The CLI documentation for the aws cli v2 specifies (https://awscli.amazonaws.com/v2/documentation/api/latest/reference/kinesis/put-record.html):

The formatting style to be used for binary blobs. The default format is base64. The base64 format expects binary blobs to be provided as a base64 encoded string.

I confirmed that the payload sent to kinesis is indeed base64 encoded once.
Also, I validated the case against AWS.
Expected behavior
Records get returned properly.
Error logs
[error] kinesis.mock.KinesisMockService$ 2023-12-12T16:56:24.557165Z contextId=fa937ab2-8e90-46c1-a35a-081f9b30f185 Error servicing request: POST / from <IP_ADDRESS>
java.lang.IllegalArgumentException: Illegal base64 character 22
  at oy.loa(/main.js:506:31)
  at mra(/main.js:507:249)
  at hqb(/main.js:3470:21)
  at {anonymous}()(/main.js:3320:72)
  at w.h(/main.js:2942:262)
  at z7(/main.js:4433:246)
  at e.Nk(/main.js:4461:328)
  at ZN.Nk(/main.js:1376:197)
  at {anonymous}()(/main.js:3605:53)
  at u.ra(/main.js:2942:45)

Additional context
This first occurred as a bug within LocalStack, but we verified it against aws-mock with the given instructions directly.
Happy to help with anything, however I would appreciate pointing me in the right direction, my scala is rather bad.

I'm a little confused on the use case here. Wouldn't this result in double base64 encoding? Typically when interacting with the aws cli, you just send the unencoded data.

With aws cli V2, the default Mode (without raw-in-base64-out) it should take the base64 and not double encode it. I will double-check the wire format, but if I remember correctly it is not double base64 encoded.

Ah I see the item in the documentation now. I can take a look into this.

I think the issue with your script is here:
SHARD_ITERATOR=$(${AWS} kinesis get-shard-iterator --shard-id shardId-000000000000 --shard-iterator-type TRIM_HORIZON --stream-name $KINESIS_STREAM_NAME --query 'ShardIterator');
What happens is that the result is encapsulated in double-quotes. E.g. "AAAAAAAAAAFKBrg2QSdJBHErnvUcZTJimAdal2SOXPJAZZ4U8lK1C2yqgf4Uio89mBPbYYUY6qX3/ug/cnn7YH0LLuZxyOFCh6hLtMj3YeO8a3O4mlMhM6riEOMpcIF/HZ4poJrs/qWd5w/vcFRu89hXcXriUJ8O9bJvIWm3VRhoy2GlKzwWW5HSnWlT60Dc3xZiKD5pgzY=". kinesis-mock doesn't like the double quotes because it cannot parse it correctly.
Without the double quotes, things work great on my end. Was able to retrieve the records.

Ah thanks! Will confirm this tomorrow, when I get back to my machine.
Still, it works with this script on aws. While I agree that this is only a minimal issue, we try to be as accepting as aws in that regard.
I can raise a quick PR which strips out the quotes from the shard iterator on the API side, if this is ok with you? Otherwise we can strip it out on the localstack side before it gets forwarded as well, if you prefer that.
Thanks for the quick confirmation and analysis!

Still, it works with this script on aws.

When you say this, do you mean on an actual AWS kinesis stream? Or against aws-mock? (As a side note, can you link me as to what aws-mock is?)

I am so sorry, I mean kinesis-mock. It seems I had a brain freeze when typing this. I just meant I tested it against this project without going through localstack as well, to ensure we do nothing wrong while proxying.
Yes, i tested it against aws and it worked. However, I am not sure if the return value was without quotes, or it accepts them with quotes. I will check right away!

Cool, thanks. Yeah if AWS accepts it, kinesis-mock should too. The update would need to be made here. We'd also need a test for it.

Just double checked against AWS to be sure:
It actually sends the shard iterator including the double quotes, and is fine with it. The body looks like this:
'body': b'{"ShardIterator": "\\"AAAAAAAAAAFyfvzLhrSct576P2qqlSiKkMj2x+2RW1T4ud3UeOURVmvU5hzkOWkUHdz6GXVWy9wUs/W41wOIe3we1M3mWfczfd8TvPkqvKosoDqzMxhjY6sqSImrRoM3q6+CjShyJ0PT3Q/0q1NsQHYjAJQhLhl/YgkEOqBbY1AHaXvb9viwvi4gv/+TwU7zHQ4+wHnWOSKOJcfmN0uN1Q7knSxPQbGbKvJ45h+R3vTCcXQgMXPDsQ==\\""}',

Including the escaped double quotes. Will try to get a quick PR open tomorrow!

Does the getRecords call succeed with that value passed as the ShardIterator?

Yes, exactly.


chore(nodebuilder/tests): Clean up sync_test tests

This PR is a part of the swamp cleanussy effort to clean up our integration tests and make them more deterministic + readable.
This PR focuses on sync_test.go file.
It also contains a change to fraud proof broadcasting test but I can happily split out into separate PR.

Codecov Report

Merging #2162 (09cb498) into main (4d31ad4) will decrease coverage by 0.23%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #2162      +/-   ##
==========================================
- Coverage   56.12%   55.89%   -0.23%
==========================================
  Files         213      215       +2
  Lines       13846    13918      +72
==========================================
+ Hits         7771     7780       +9
- Misses       5303     5359      +56
- Partials      772      779       +7

see 20 files with indirect coverage changes


An alternative integration with Mosby: AndroidAnnotations & Bolts

Hi sockeqwe, thanks for great MVP framework, I try to play for awhile with Mosby and find it suitable for my Android projects but just have a question regarding integration of Mosby with other libraries.
I can see Mosby is more friendly to be integrated with Butterknife & RxJava, but I personally is getting used with AndroidAnnotations for view binding and Bolts for asynchronous work. Can I have advice as if those two libraries can integrate with Mosby without any pain ?
Thanks,

Yes this should be possible, but I never tried both libraries before.
AndroidAnnotations should work. Just try to annotate your Activity extends MvpActivity with @EActivity etc.
Alernatively, if AndroidAnnotation for some strange reasons disallow to extend from MvpActivity you can try the following:
Mosby follows the composition over inheritance principle by providing a delegate.
See the docs http://hannesdorfmann.com/mosby/viewstate/ (scroll down until ViewState Delegate section). So basically you have to "forward"  all lifecycle event to Mosby's ActivityDelegate or FragmentDelegate, see https://github.com/sockeqwe/mosby/blob/master/mvp/src/main/java/com/hannesdorfmann/mosby/mvp/MvpActivity.java
In addition you also have to implement ActivityMvpDelegateCallbackor FragmentMvpDelegateCallback
Regarding Bolts library:
I can't see any reason why Bolts wont work with Mosby. Just keep in mind to cancel tasks in presenter.detachView(false) (parameter false means that the presenter should be destroyed permanently so some cleanup should be done here, like canceling tasks).


Contact txn ui

had to make a small contract change, I can redeploy the other contracts & sub those addresses in if this is approved

@avclarke @diogoperillo could you please review again when you have the chance?


Fix double evaluation and filter sensitive settings

Fixes #5609
Description
Fix the double evaluation and filter sensitive settings

Pull Request Test Coverage Report for Build 250326

0 of 0   changed or added relevant lines in 0 files are covered.
6 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.03%) to 78.852%

Files with Coverage Reduction
New Missed Lines
%

/libraries/AdaptiveExpressions/BuiltinFunctions/GetNextViableTime.cs
1
90.91%

/libraries/AdaptiveExpressions/BuiltinFunctions/GetPreviousViableTime.cs
2
90.91%

/libraries/Microsoft.Bot.Builder.Dialogs/Memory/Scopes/SettingsMemoryScope.cs
3
95.71%

Totals

Change from base Build 250298:
-0.03%

Covered Lines:
18151

Relevant Lines:
23019

💛  - Coveralls


changing sample_rate gives unexpected result

I know this is an error in my thinking rather than your code, which is excellent, but I cannot figure out how to produce a good sounding sample at higher sample rates. I first tried (source and target wavs are sample rate 48000):
convert.py -w downloaded_models/wav2vec_small.pt -v downloaded_models/vocoder.pt -c 'downloaded_models/fragmentvc.pt' source.wav target1.wav target2.wav --sample_rate 48000
but the pitch was 3x too high, so I downloaded the VCTK data, preprocess at 48000:
python preprocess.py ../../in/VCTK/ downloaded_models/wav2vec_small.pt features  --sample_rate 48000
and retrained:
python train.py features --batch_size 2 --save_dir ./ckpts
...but even using the resulting model, the pitch is 3x higher than expected. Can you point me in the right direction?
EDIT: I noticed the speed AND pitch are too high:
$ mediainfo '/mnt/data/ml/sound/work/FragmentVC/out/Tom03.wav'
General
Complete name                            : /mnt/data/ml/sound/work/FragmentVC/out/Tom03.wav
Format                                   : Wave
File size                                : 340 KiB
Duration                                 : 3 s 629 ms
Overall bit rate mode                    : Constant
Overall bit rate                         : 768 kb/s
Audio
Format                                   : PCM
Format settings                          : Little / Signed
Codec ID                                 : 1
Duration                                 : 3 s 629 ms
Bit rate mode                            : Constant
Bit rate                                 : 768 kb/s
Channel(s)                               : 1 channel
Sampling rate                            : 48.0 kHz
Bit depth                                : 16 bits
Stream size                              : 340 KiB (100%)
$ mediainfo '/mnt/data/ml/sound/in/Tom03.wav'
General
Complete name                            : /mnt/data/ml/sound/in/Tom03.wav
Format                                   : Wave
File size                                : 608 KiB
Duration                                 : 6 s 412 ms
Overall bit rate mode                    : Constant
Overall bit rate                         : 777 kb/s
Producer                                 : Pro Tools
Description                              : dUBITS=$17082011 / dSCENE=25-4 / dTAKE=2 / dTAPE=IBDM10 / dFRAMERATE=25.000ND / dSPEED=025.000-NDF / dTRK2=Boom 1
Encoded date                             : 2020-08-17 00:00:00
Producer_Reference                       : aaymNFNMQkQk
UMID                                     : 0x60A2B340101010501010F1013000000A2AA9A77E7318000169C8EC3650D1B1C
Audio
Format                                   : PCM
Format settings                          : Little / Signed
Codec ID                                 : 1
Duration                                 : 6 s 412 ms
Bit rate mode                            : Constant
Bit rate                                 : 768 kb/s
Channel(s)                               : 1 channel
Sampling rate                            : 48.0 kHz
Bit depth                                : 16 bits
Stream size                              : 601 KiB (99%)

If you preprocessed the training data, configured and retrained the model correctly, the model is then capable of generating Mel spectrogram that can possibly be inverted to 48KHz waveform by a vocoder.
However, the pretrained vocoder I provided can only generate 16KHz waveform. So if you wish to generate 48KHz audio, you'll have to retrain the vocoder as well.

Ah, I thought that might be the case. So now I've downloaded your universal-vocoder and I'm trying:
python preprocess.py /data/sound/in/VCTK preprocessed --sample_rate 48000
Yist, thank you for being so clear with your instructions and generous with your time. I'll let you know how I get on.

Update: I trained the data with
python train.py preprocessed
and ran a convert with
convert.py -w downloaded_models/wav2vec_small.pt -v /universal-vocoder-trained-at-48000/vocoder-ckpt-100000.pt -c 'downloaded_models/fragmentvc.pt' source.wav target1.wav target2.wav --sample_rate 48000
..the output is now the right pitch, but it is twice the speed of the source.wav.
I would be very grateful for your thoughts on how to proceed.

Update: I trained the data with
python train.py preprocessed
and ran a convert with
convert.py -w downloaded_models/wav2vec_small.pt -v /universal-vocoder-trained-at-48000/vocoder-ckpt-100000.pt -c 'ckpts/retriever-best-loss0dot6863.pt' source.wav target1.wav target2.wav --sample_rate 48000
..the output is now the right pitch, but it is twice the speed of the source.wav.
The vocoder model and checkpoint file 'ckpts/retriever-best-loss0dot6863.pt' were trained at 48000Hz.
I would be very grateful for your thoughts on how to proceed.

Hi! Can you share your results, please? For smaple_rate=48k and 16k. For example with sound cloud links

Here you go. I noticed the loss on vocoder training is quite high and doesn't reduce much after 100000 steps. It's around 2.3. Loss on the 'checkpoint path' model is .6863.

Here are the original and vocoded files at 48k. I noticed the loss on vocoder training is quite high and doesn't reduce much after 100000 steps. It's around 2.3. Loss on the 'checkpoint path' model is .6863.

Thank you! I think that it is possible to try to change other parameters like: nfft, hop_len, win_len, not only the sample rate.
Here you can find thre recommendations for these params:

I tried to do it, but without retraining vocoder, so the loss on fragmnentVC did not reduce lower than 1.76. Maybe u can get better result than me:)

I noticed the defaults for the FragmentVC and universal-vocoder models are different, so I'm currently retraining the vocoder model with the n_fft, hop_length and win_length the same as the FragmentVC defaults (except with both set to --sample_rate 48000. I'll let you know if it improves anything.

Setting the parameters when retraining the vocoder model to match the parameters in FragmentVC helped:;it's now the correct speed. However the quality of the output is still very low. I feel I should concentrate on training the vocoder, as the loss is so large (2.3). Is this what is causing the problem?  Can anyone help me?

@seaniedan Since we've never experimented on generating 48KHz results, I can't tell you whether FragmentVC is capable of generating sharp spectrogram in such high sampling rate.
As for the vocoder, it's not clear whether this WaveRNN-based vocoder can generate high quality 48KHz audio. As far as I know, the original paper only experimented on generating 24KHz audio.


readme installation guide

Done
Updated readme with Ubuntu installation guide

@carkod can you please take a look at this? thanks


Upstream update - Feb 2020

This PR brings our fork up to date with upstream GOGS up to the end of February 2020.  It includes 66 of the 220 commits in gogs/master since our last update.  I felt it was a good place to stop for a first review since it includes a rather big overhaul to the settings module (now called conf).
Tests have been adjusted for GIN changes.  I'll deploy this on the dev server for testing.
Biggest changes
These commits needed the most modifications because they changed code that includes GIN related changes and caused merge conflicts.  These need the most care when reviewing.
2d6080d5c90153dc5b846211c608a74343e0f9cc - bindata
19a6893324e39f22c2c00bb6380858b900a0c576 - logging library
484c071f6d4f6635e7b3052d117dbc8f8bf5b320 - server settings overhaul: touches almost every file because it changes the structure of the settings (now conf) module
c5c6f531460c1d95b4706466990e7b50b31b9b7c - email settings overhaul: small changes needed to accommodate our collaborator email invite functionality
08aab3fcd768b6df5c08af9ae49440e153dbdc76 - settings overhaul: finalisation of the settings overhaul; moved a block of the settings (now conf) struct to a different file and changed mapping error checking, which includes some of our added settings
Clean vs dirty commits
The is the output of git cherry comparing the upstream commits to all the commits in this branch.  Commits prefixed with - applied cleanly and the rest needed conflict resolution.  In most cases, conflicts were a matter of fixing import paths (changing "gogs.io" to "github/G-Node") or regenerating statically linked assets.  The 5 commits listed above are the ones that needed substantial conflict resolution.
+ 8843e1cf1438df1e2794618c71917fe4d0404675 Remove vendor directory
+ df04865ab07462b3a531eeb49912a675630469dd New file: gin-features.md
+ a1fa72a3913d2f252ec42c16d55b330364520998 repo: use editorconfig v2 (#5888)
- 5ddd21caaa2d0d8fd6c767a4ff72ee3d106e99c0 user/setting: check for IsErrEmailAlreadyUsed when updating user (#5900)
- fe3606800b569f9528da8b93091b8c8e87f13d39 docker: check "/app/gogs" separately on startup (#5895)
+ b794a2e2c50dad76afa3de8fd2a0ffc5501b4c79 repo: fix redirect after opening/closing milestone (#5903)
+ c612a1104ca761f95fa307c800999d180d50cc52 api: sanitize raw markdown content (#5907)
+ 09062a49250f4f629dba5d04d54d5f3f585ac69e api/misc/markdown: remove Mode option (#5910)
- cb8ec73067c83962cf5e226050995d024ee2be11 repo: able fill pull request title by template from md file (#5901)
- e3a49bf7bde46a008e0f690d125eecafd56e9ab0 conf: update instructions for installing go-bindata
+ 0572b0730efa7b474d219a15f33d0ab04b99759e conf: fix wrong default example value
+ 9813d2716858f655a436f7741347386f2c077a58 all: use lazyregexp (#5911)
- af572f7986a806508799918418c2d4e5c89bbd77 public/js: upgrade jQuery 1.11.3 to 1.12.4
- a8995481a9ab5f4427fc803695b0665bebbed5ca public/js: upgrade clipboard 1.5.9 to 2.0.4
- ba38ff7349939b16fdc7d41e0b226e16170ce43f public: upgrade Semantic UI 2.3.1 to 2.4.2
- 51d4c81e36f63d76071ca3894af52752d296bd31 public/js: upgrade jQuery 1.12.4 to 3.4.1
- 434a470123938256e31aeccfbe5013b0b9b15b0a public/plugins: upgrade Dropzone 4.2.0 to 5.5.0
- 168446a2787478c909b566a03be31b2a58749bb4 public/plugins: upgrade highlight.js 9.16.2 to 9.18.0
+ 311de7d0a3c6d5d0d6f681e59fd03f80359ee037 [ImgBot] Optimize images (#5913)
+ dcbb34a57785106c72e69adf0cf8e3d1a38a15a3 locale: sync from Crowdin
- 93284cf6b24bc29e1049bed6c67d39a77bef4d3b db/repo: fix Git version check to 1.8.3
+ 3660f457a94b0d636316a98edaf8a11cbf62d28e README: add new sponsor BitLaunch
+ 51b7dad183438283a236dc719a8ee4e4051962cd README: fix phonemic transcription (#5916)
+ 2d6080d5c90153dc5b846211c608a74343e0f9cc Use go-bindata to embed `public` and `templates` files into binary (#5920)
+ 78c679edc711cb887066d049a8728afce2b205a7 Add CHANGELOG and adopt new release workflow
+ 95533d6efe7a89a2addad504485949fbdb39b1fd go.mod: update github.com/go-macaron/i18n to v0.5.0
- 1c6f29a9e2e44c2bdd549ed88e283725d1f9208a mailer: fix template not found error
+ e016434fd9f3cef554ceec93c6f78cfbd9328756 README: update Features list
+ 85f788068bf43594c28b6b201ecc7c067b1f121e README: clean up
+ f5f03cd820c5ccfe24545410ab133c8034275363 docker: fix Docker Hub build failure
+ 19e7f37b7e4a64a626c2ec9ed8538de17644af1d docs: add local development
+ c1bf813ec83600b99e0c6768d838acba32bae6ce docs: improve set up guide
- 56b96a23543fe5929d8e7589264266cf34cc59e6 docs: add steps to install npm and lessc
- 11c5520d3cb38a8eab1a198c3834757d087de4c0 Makefile: generate Sourcemap for less files
+ 673bcd272fbc27e1f58407bf741af25e101bf215 admin: show all version in dashboard
-<PHONE_NUMBER>d6b5f3f04b2f298275bdc7a4c9b8a2 log: use trace level for some logs
- b7c9b23e521d35140fdefc499906c04f1da7d3b1 .travis: fix config warning
- 9675ada320d5e76abcad438d9f7d49cfc888ddf9 CHANGELOG: add missing entry for ea75f01ba2
+ 1e7ed6f64f70a22b24c48150e2260a9c5a9b555c osutil: add unit tests
+ 2941ed30a058e4d82e0ef045eac27061cd27affc repo/editor: clean up tree path
- 05fe82be400879e27a2d6a249d15daeb25d57e04 CHANGELOG: add missing entry for ce1ec81d6f
+ 9db1d17eafaeb37409bc432fe1dea2577e139942 conf: remove unused config option SHOW_FOOTER_VERSION
+ 19a6893324e39f22c2c00bb6380858b900a0c576 log: migrate to unknwon.dev/clog/v2 (#5927)
+ f09c705ccc3759b4f4eaed5862c693893c5a80df build: rename build tag miniwinsvc -> minwinsvc
- 45cadb51f976e7e3002aa3a25b28f85d969ac667 go.mod: update golang.org/x/crypto/ssh for security fixes
- 1c55c68a66a374c008f027732093d4d9fe90c971 docker: update link to Docker Hub and add info for gogs-rpi
+ 484c071f6d4f6635e7b3052d117dbc8f8bf5b320 conf: overhaul server settings (#5928)
+ 24d3f7a960a55ea55362957c86117f57d992fdf2 admin/config: show all server and SSH options
+ 760d040858d86abd205747b4bf6afef38163e7ad conf: overhaul repository settings (#5932)
+ 1e17763e907ac3b0ffc4b0884d9f58b39182c221 conf: overhaul database settings
+ 0305a0d21d7e7ca73f1f4a39c2e324c845a1a06f cmd: fix unable to find correct custom config
+ a4cdc7546a5cdedb5fa5b8c36456ce545d55b1f1 README: fix image and link
- a97ab61a6d16d0a4a7ee1cc533c270b3f7f711e9 docker: fix link to Docker Hub
+ bc074025d2d46baba90450fb1c730a563c2ca645 go.mod: update unknwon.dev/clog/v2 to v2.1.1
+ f4e032b745a124f04f007418a096cf223b8ec2b7 conf: overhaul security settings
+ f4d534bc5fb176def46c0691323b73d34a4ef552 css: add img margin-right
+ 10f700c294e3eb1a6ed2e5dbeab33cad8b5ba978 README: add emoji to headings
+ fb7c4873472a8d840b463c07ad11acba6b4a0078 cmd/web: fix error when Unix socket not exists
- 41b9e255cd568843983e1b82bdf2325530545815 cmd/web: fix wrong ExternalURL when specify port via CLI flag
+ c5c6f531460c1d95b4706466990e7b50b31b9b7c conf: overhaul email settings (#5940)
- 31ded134989b327697e1b93abdc179ba48c2cd3c docs: improve local development guide
- 203340c6bf60e65a8fc7754e468be9851482ad08 api: support getting repository Git tree (#5934) (#5937)
+ f67efda4e8c784005927c6628a7cdf289c54cb27 conf: overhaul auth and user settings (#5942)
- e41a9f3ec2a889816924fc37e8a89699effd583e public: fix jquery3.x syntax error (#5951)
- 2d358e198ce8b18ac0e887abe429b5b8e949bb64 build: requires Go 1.13
+ fd76121d3c1508e43bb10fbbd1bead329165e5ef conf: overhaul sessions settings (#5952)
+ 08aab3fcd768b6df5c08af9ae49440e153dbdc76 conf: overhaul settings (#5953)
+ 6cd36a6952c956ee0f142a3c948cf7496c5147b5 conf: add unit tests (#5954)
+ 4c792a1abdb936733d31723ae6b339e8b4b38739 css: fix /admin/config pre overflow
+ 117b25f4eac80b033821512c098e01bcc6a3dd0a js: fix jQuery error

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@4cbb0d4). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             master     #88   +/-   ##
========================================
  Coverage          ?   5.09%
========================================
  Files             ?      53
  Lines             ?   10204
  Branches          ?       0
========================================
  Hits              ?     520
  Misses            ?    9598
  Partials          ?      86


Add registration details

As far as I know, the private DNS zone registration mechanism (that it's being done by the DHCP side) isn't documented anywhere, which can cause some confusing behavior.  This change at least adds some FAQs around this.  There should likely be changes to the main private DNS zone documentation as well.

@MikeBazMSFT : Thanks for your contribution! The author(s) have been notified to review your proposed change.

OK, changes made.


Import as RLMRealm_Private.h as a module would cause issues when using Realm as a subdependency

Import as RLMRealm_Private.h as a module would cause issues when using Realm as a subdependency.
Because <Realm/RLMRealm_Private.h> is part of the Private Realm Module, there seems to be an issue when you use Realm as a sub-dependency with Cocopods which doesn't find the file within the headers associated to the module, even if we expose them as private headers.
Importing the file relative to the directory using quotation marks fixes the issue.

This is something we should probably have an installation test for if we currently don't have anything which breaks when we include the header incorrectly.

Added installation test using Realm as a subdependency with Cocoapods to test this


Doesn't work with apps with auth

This is a great middleware and it works.
But it stops apps with authentication from working, if an app has app.UseCookieAuthentication the login will fail.
Any ideas?

The updates didn't fix the problem.

I have added on example with cookie authentication. Can you try now?

Yesss, it works beautifully. This is without a doubt the best, simpliest and lightest HTML Minifier out there, thank you so much Anuraj.


Use unreleased version for jdbc starters

As title

/azp run java - spring - tests-weekly

API change check
API changes are not detected in this pull request.


Add possibility to use branches when using targetRepositoryName

Is your feature request related to a problem? Please describe.
Currently it is not possible to pass a branch when using the extension for more than one repository.
Thus, the extension always searches for the dependabot.yml in the default branch.
This is especially useful for staged projects where the config might differ from branch to branch
Describe the solution you'd like
It would be great if it was possible to pass the targetbranch the same way as the targetRepositoryName like
  - task: dependabot@1
    inputs:
      targetRepositoryName: myrepository
      targetRepositoryBranchName: mybranch

Describe alternatives you've considered
The alternative I'm forced to use is that I move all configs to the default branches which is cumbersome

Why would the dependabot.yml file need to be in another branch?
Also, you can specify which branch to check for updates and open PRs against in the dependabot.yml file.

In my situation, I'm trying to run dependabot from a pipeline configured from another repo (hence using targetRepositoryName). I have a branch open in the target repository to try out changes to how we configure access to our private feeds in dependabot.yml. I don't want to merge it until I test it out.


v10 Support

This is a pretty significant restructuring and rewrite-lite (relite?). Foundry
v10 adds proper hex borders so this is building on that.
So far the changes are less invasive than the v9 and earlier.

[ ] Add support for more hex sizes (core goes to 4)
[ ] Fix orientation of 2/4. They are currently mismatched. Flip one. (Or file a core bug?)
[ ] Add global option for default orientation
[ ] Add token level support for alternate orientation
[ ] Extend token config to allow changing orientation and repositioning art

[ ] Hotkeys for this as well

[ ] Ruler fixes? There's a core bug report scoped for the next stable release about this
[x] Set hitArea to match (File core bug over this as well?
[x] Color config (Now uses the canvas disposition color config, so should be more consistent)
[x] Border order stuff. (This is very different, but the fix was surprisingly concise. Default value was flipped to be consistent with core)
[x] Don't be weird on non-hex grids
[x] Border fill
[x] Border width
[ ] Localization support
[x] Bonus: Round tokens for gridless

I'm not sure how long it will take me to finish. As is, this does currently work, but I want to do some cleanup. Plus this is, to put it mildly a bit incomplete compared to the earlier versions.
I have the stubs of most of what needs to be messed with to get alternate borders and stuff working (tasks 1-4), the gist of which is:

getSnappedPosition needs to be tweaked to be more intelligent on which space it tries to align to based on border and orientation.
_adjustSnapForTokenSize should likewise be tweaked.
_adjustPositionForTokenSize should probably be tweaked and could possibly be more useful for getSnappedPosition.
Borders are grabbed from the grid, based only on token size. Alt borders would need an alternate method, possibly one that simply inverts the result from getBorderPolygon

Additionally ruler movement has a report, so waiting for a fix on that may be valid depending. Ref https://github.com/foundryvtt/foundryvtt/issues/8196
As it stands, the only two things I want to finish before opening this up are the token config extension and localization support.

Oh, also the roundy borders should probably have a settings toggle.

I'm going to descope fixing ruler movement from this since it's a core bug. All that's left is cleaning up obsolete files and doing some testing.


Watch for comment support

Just noticed that https://github.com/Microsoft/TypeScript/pull/5450 got opened which adds comment support to tsconfig.json. No clue if it will make it in, but if it does then tsconfig should also support.

Added. This will be supported in the next few days here too.

Ok, two simple solutions - use jju (which is the fallback to display errors for json-parse anyway, the errors are quite good) which supports comments or use strip-json-comments which works but may need a PR to replace with whitespace instead of stripping to preserve error positions.

jju sounds good :rose:

@basarat My concern for that is supporting invalid syntax compared to TypeScript. I think if the whitespace fix can be applied, we should strip and replace with whitespace then parse as usual which will give us correct error positions.

My concern for that is supporting invalid syntax compared to TypeScript

I'm okay with that. VSCode already supported more syntax than tsc did ¯\_(ツ)_/¯

I was thinking the same thing. It's OK for tsconfig to be more permissive with syntax than tsc, but not the other way around.

Opened both https://github.com/rlidwka/jju/issues/17 and https://github.com/sindresorhus/strip-json-comments/issues/24. Since not everyone uses tsc, I'd like to not accidentally support anything extra. The cleanest way for that to happen feels like stripping the comments (which is exactly what TypeScript is doing) instead of supporting JSON5.

Merged and released with 2.2.0 :+1:


Update service-fabric-reverseproxy.md

10.0.05 is not a valid IP address, <IP_ADDRESS> is

Hi @TimHess, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@TimHess : Thanks for your contribution to the Azure documentation! The author, @BharatNarasimman,vturecek, has been notified to review your proposed change.

#sign-off

Only the author of the article can use the sign off comment to add the ready-to-merge label.

#sign-off


Run tests 20 times

This is not meant to be merged, but just to show that current main contain the fix to the pointer problem raised in #2011 .
Closing as all tests are passing.


some styles of header definition (to add to a redirect) do not work as expected in netlify.toml (only when parsed by netlifyctl)

CONTEXT:
Unable to use the following in a netlify.toml file:
[[redirects]]
  from = "/map"
  to = "https://maps.google.com"
  status = 200
  force = true
  headers = {X-From = "netlify_live", x-api-key= "LIVE API KEY"}

This causes the following error when running netlifyctl deploy

"Near line 13 (last key parsed 'redirects.headers'): Expected value but found '{' instead."

WORKAROUND:
The following format, however, works:
[[redirects]]
  from = "/calculator/test"
  to = "https://futuregerald-requestbin.herokuapp.com/pzyoldpz"
  status = 200
  force = true
  [redirects.headers]
    X-From = "netlify_test"
    x-api-key= "TEST API KEY"

Did you determine if a single header defined this way works @depadiernos ?  Is the bug limited to multiple?

The bug is not limited to multiple. Even with a single header, the deploy will fail. The following will fail also:
[[redirects]]
  from = "/map"
  to = "https://maps.google.com"
  status = 200
  force = true
  headers = {X-From = "netlify_live"}

The TOML parser we use only supports version 0.2.0 of the TOML spec. Inline tables were added in 0.4.0.

Thanks @brycekahle !  That also explains https://github.com/netlify/bitballoon/issues/1469 !

I was referencing the TOML parser we have in netlifyctl. I'm not sure what the origin uses. We should make sure they match though.


Rename Github repo and docker image to docker-frontail

Request from @mthenw at https://github.com/mthenw/frontail/issues/45.

https://registry.hub.docker.com/u/servicerocket/docker-frontail/.
We will close the old one https://registry.hub.docker.com/u/servicerocket/docker-fronttail/, next sprint (next week).


ResourceNotFoundError in predocs phase

Please provide us with the following information:

This issue is for a: (mark with an x)
- [X] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

azd provision
(and all resources are created fine, i think)
Any log messages given by the failure

Packaging services (azd package)
(✓) Done: Packaging service backend

Package Output: /tmp/azure-search-openai-demo-backend-azddeploy-1712906840.zip

SUCCESS: Your application was packaged for Azure in 51 seconds.
Checking if authentication should be setup...
Loading azd .env file from current environment...
AZURE_USE_AUTHENTICATION is set, proceeding with authentication setup...
Creating Python virtual environment "app/backend/.venv"...
Installing dependencies from "requirements.txt" into virtual environment (in quiet mode)...
Not setting up authentication.
Provisioning Azure resources (azd provision)
Provisioning Azure resources can take some time.
Subscription: icdev (bb369bdb-6d2a-483a-ac31-fa61b10cacfa)
Location: East Asia
(-) Skipped: Didn't find new changes.
Loading azd .env file from current environment...
Creating Python virtual environment "app/backend/.venv"...
Installing dependencies from "requirements.txt" into virtual environment (in quiet mode)...
Not updating authentication.
Loading azd .env file from current environment...
Creating Python virtual environment "app/backend/.venv"...
Installing dependencies from "requirements.txt" into virtual environment (in quiet mode)...
Running "prepdocs.py"
Using local files: ./data/*
Ensuring search index gptkbindex exists
Search index gptkbindex already exists
Skipping ./data/employee_handbook.pdf, no changes detected.
Ingesting '2190.json'
Splitting '2190.json' into sections
Uploading blob for whole file -> 2190.json
Computed embeddings in batch. Batch size: 2, Token count: 303
Ingesting 'query.json'
Splitting 'query.json' into sections
Uploading blob for whole file -> query.json
Computed embeddings in batch. Batch size: 7, Token count: 2199
Ingesting '2192.json'
Splitting '2192.json' into sections
Uploading blob for whole file -> 2192.json
Computed embeddings in batch. Batch size: 2, Token count: 375
Ingesting '2191.json'
Splitting '2191.json' into sections
Uploading blob for whole file -> 2191.json
Computed embeddings in batch. Batch size: 2, Token count: 418
Ingesting '2189.json'
Splitting '2189.json' into sections
Uploading blob for whole file -> 2189.json
Computed embeddings in batch. Batch size: 1, Token count: 205
Ingesting 'Benefit_Options.pdf'
Extracting text from './data/Benefit_Options.pdf' using Azure Document Intelligence
Traceback (most recent call last):
File "/workspaces/azure-search-openai-demo/./app/backend/prepdocs.py", line 494, in
loop.run_until_complete(main(ingestion_strategy, setup_index=not args.remove and not args.removeall))
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/workspaces/azure-search-openai-demo/./app/backend/prepdocs.py", line 225, in main
await strategy.run()
File "/workspaces/azure-search-openai-demo/app/backend/prepdocslib/filestrategy.py", line 84, in run
sections = await parse_file(file, self.file_processors, self.category, self.image_embeddings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/azure-search-openai-demo/app/backend/prepdocslib/filestrategy.py", line 26, in parse_file
pages = [page async for page in processor.parser.parse(content=file.content)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/azure-search-openai-demo/app/backend/prepdocslib/filestrategy.py", line 26, in
pages = [page async for page in processor.parser.parse(content=file.content)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/azure-search-openai-demo/app/backend/prepdocslib/pdfparser.py", line 54, in parse
poller = await document_intelligence_client.begin_analyze_document(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/core/tracing/decorator_async.py", line 77, in wrapper_use_tracer
return await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/ai/documentintelligence/aio/_operations/_operations.py", line 3241, in begin_analyze_document
raw_result = await self._analyze_document_initial(  # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/ai/documentintelligence/aio/_operations/_operations.py", line 130, in _analyze_document_initial
map_error(status_code=response.status_code, response=response, error_map=error_map)
File "/workspaces/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/core/exceptions.py", line 164, in map_error
raise error
azure.core.exceptions.ResourceNotFoundError: (404) Resource not found
Code: 404
Message: Resource not found
ERROR: failed running post hooks: 'postprovision' hook failed with exit code: '1', Path: '/tmp/azd-postprovision-1942051034.sh'. : exit code: 1
ERROR: error executing step command 'provision': failed running post hooks: 'postprovision' hook failed with exit code: '1', Path: '/tmp/azd-postprovision-1942051034.sh'. : exit code: 1
Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)
Github codespaces with python 3.11

azd version?

run azd version and copy paste here.
azd version 1.8.0 (commit 8246323c2472148288be4b3cbc3c424bd046b985)

Versions

main branch as of April 12, 2024.
Mention any other details that might be useful

Thanks! We'll be in touch soon.

Make sure your document intelligence service in one of these regions : westus2, eastus, or westeurope

@jimmylevell You say you're using private endpoints? Did you add those manually in the Portal? We have a PR which adds support for private endpoints, but it's not yet in main. We're still testing that, so I don't know if we've seen issues with using Document Intelligence. FYI to @mattgotteiner who's working on that PR.

@pamelafox thank you for your fast reply.
We needed to deploy the solution based on internally policies manually within Azure. In this process all Azure resources have been configured with private endpoints. The solution is working as expected within our tenant (Azure Switzerland North). The only required change we needed to introduce was reverting to an older form recognizer dependency in the pdfparser.py.

- from azure.ai.documentintelligence.aio import DocumentIntelligenceClient
+ from azure.ai.formrecognizer.aio import DocumentAnalysisClient

The issue with the form recognizer can also be illustrated using the following demo-code:
Working Sample
Based on Document Intelligence Studio Sample
from azure.core.credentials import AzureKeyCredential
from azure.ai.formrecognizer import DocumentAnalysisClient

endpoint = "https://<private-endpoint-instance>.cognitiveservices.azure.com/"
key = ""

formUrl = "https://raw.githubusercontent.com/Azure-Samples/cognitive-services-REST-api-samples/master/curl/form-recognizer/sample-layout.pdf"

document_analysis_client = DocumentAnalysisClient(
        endpoint=endpoint, credential=AzureKeyCredential(key)
    )

poller = document_analysis_client.begin_analyze_document_from_url("prebuilt-document", formUrl)
result = poller.result()

for kv_pair in result.key_value_pairs:
    if kv_pair.key and kv_pair.value:
        print("Key '{}': Value: '{}'".format(kv_pair.key.content, kv_pair.value.content))
    else:
        print("Key '{}': Value:".format(kv_pair.key.content))

None-working Sample
Based on MS Docs Sample
from azure.core.credentials import AzureKeyCredential
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.ai.documentintelligence.models import AnalyzeResult

endpoint = "https://<same private endpoint instance>.cognitiveservices.azure.com/"
key = ""

document_intelligence_client = DocumentIntelligenceClient(endpoint=endpoint, credential=AzureKeyCredential(key))
with open("./data/OHB5336.pdf", "rb") as f:
    poller = document_intelligence_client.begin_analyze_document(
        "prebuilt-layout", analyze_request=f, content_type="application/octet-stream"
    )
result: AnalyzeResult = poller.result()

Therefore, I believe the issue is related with the newer document intelligence package.

I'm seeing this too, freshly downloaded repository.

Hm, so we've gotten private endpoints working in PR here: https://github.com/Azure-Samples/azure-search-openai-demo/pull/864/files
So I'm looking to see what Document Intelligence specific changes are in there that might be relevant.
There's this configuration of network bypass:
https://github.com/Azure-Samples/azure-search-openai-demo/pull/864/files#diff-8a64001dc63e4053382af7bbd6519e074e3a637a9e0a50b5b6e8ca136b4224ceR37
That's the only change I see specific to Document Intelligence.
We don't change our URL to Document Intelligence, did you change your URL? I don't believe that should be necessary.
(I am still learning Azure networking so I may be wrong)
cc @mattgotteiner in case he has insights

Just to clarify we have the main branch of the app running our Azure instructure. Each resource only uses private endpoints. As mentioned we have configured each resource manually, but no code changes were required (besides the mentioned form recognizer library). Let me know if any further information would help your PR.
What is weird is that that I am using the same configuration once with the form recognizer library and once with the document intelligence library. The later throwing the error ResourceNotFound. I am accessing the default form recognizer url provided in the Azure portal, which is pointing to the private endpoint IP.
(same here)

Had the same issue. Root-cause is due to deploying the Document Intelligence resource into the Australia East environment (need for client demonstration). Resolution was to implement the similar solution to what was recommended here by @jimmylevell :)
Replacing the Document Intelligence related components (i.e. DocumentIntelligenceClient and DocumentTable modules) with Form Recognizer components (i.e. DocumentAnalysisClient and FormTable) with the below code:
from azure.ai.formrecognizer.aio import DocumentAnalysisClient
from azure.ai.formrecognizer import FormTable

...

async with DocumentAnalysisClient(
            endpoint=self.endpoint, credential=self.credential
        ) as document_analysis_client:
            poller = await document_analysis_client.begin_analyze_document(
                model_id=self.model_id, document=content
            )
            form_recognizer_results = await poller.result()

            offset = 0
            for page_num, page in enumerate(form_recognizer_results.pages):
                tables_on_page = [
                    table
                    for table in (form_recognizer_results.tables or [])
                    if table.bounding_regions and table.bounding_regions[0].page_number == page_num + 1
                ]

                # mark all positions of the table spans in the page
                page_offset = page.spans[0].offset
                page_length = page.spans[0].length
                table_chars = [-1] * page_length
                for table_id, table in enumerate(tables_on_page):
                    for span in table.spans:
                        # replace all table spans with "table_id" in table_chars array
                        for i in range(span.length):
                            idx = span.offset - page_offset + i
                            if idx >= 0 and idx < page_length:
                                table_chars[idx] = table_id

                # build page text by replacing characters in table spans with table html
                page_text = ""
                added_tables = set()
                for idx, table_id in enumerate(table_chars):
                    if table_id == -1:
                        page_text += form_recognizer_results.content[page_offset + idx]
                    elif table_id not in added_tables:
                        page_text += DocumentAnalysisParser.table_to_html(tables_on_page[table_id])
                        added_tables.add(table_id)

                yield Page(page_num=page_num, offset=offset, text=page_text)
                offset += len(page_text)

    @classmethod
    def table_to_html(cls, table: FormTable):
        table_html = "
"
        rows = [
            sorted([cell for cell in table.cells if cell.row_index == i], key=lambda cell: cell.column_index)
            for i in range(table.row_count)
        ]
        for row_cells in rows:
            table_html += "
"
            for cell in row_cells:
                tag = "th" if (cell.kind == "columnHeader" or cell.kind == "rowHeader") else "td"
                cell_spans = ""
                if cell.column_span is not None and cell.column_span > 1:
                    cell_spans += f" colSpan={cell.column_span}"
                if cell.row_span is not None and cell.row_span > 1:
                    cell_spans += f" rowSpan={cell.row_span}"
                table_html += f"<{tag}{cell_spans}>{html.escape(cell.content)}</{tag}>"
            table_html += "
"
        table_html += "
"
        return table_html

Make sure your document intelligence service in one of these regions : westus2, eastus, or westeurope

I'm getting this error for a document intelligence serviced located in "eastus2". @Caden-Ertel could you help me to understand that what's the rational behind your statement?
And if it's not possible to locate the service based on above locations, what's the solution?


Select Field can appear without any selection when clicking near the arrow

Problem description
When clicking sveral times near the arrow in a selection field the field can appear empty.
Steps to reproduce

Go to http://www.material-ui.com/#/components/select-field
Click several times near the arrow (left to the arrow)- it might need a few tries but the text inside the selection fied will disappear(look at the image added below)
https://s3.amazonaws.com/devimg.crazylister.com/100/MaterialUI-SelectField.png

Versions

Material-UI: The one that appears on http://www.material-ui.com/#/components/select-field
React: The one that appears on http://www.material-ui.com/#/components/select-field
Browser: Chrome Version 53.0.2785.116 m (64-bit)

Getting this too, seems pretty bad

blocking the blur action also fixed for me! thank you @olslash


Why am I unable to rollback my migrations?

Environment
Knex version: 0.95.7
Database + version: PostgreSQL, 13.2
OS: macOS 10.15.7 (19H1217)
Bug
After my tech lead implemented a new role permissions system in our application, I've been unable to clear the database via rolling back migrations. The command I usually run is knex --env local migrate:rollback --all. It seems as though the new migration file (i.e., 12345_updatePermissions.js) my tech lead created is what's causing the issue, since in my terminal I get returned the following error:
migration file "12345_updatePermissions.js" failed
migration failed with error: alter table "User" alter column "role" type text check ("role" in ('Admin', 'SuperUser', 'User')) using ("role"::text check ("role" in ('Admin', 'SuperUser', 'User'))) - syntax error at or near "check"
error: alter table "User" alter column role" type text check ("role" in ('Admin', 'SuperUser', 'User')) using ("role"::text check ("role" in ('Admin', 'SuperUser', 'User'))) - syntax error at or near "check"

I tried looking for a "check" in 12345_updatePermissions.js; however, I was not able to find it. The following is the contents of 12345_updatePermissions.js:
const tableName = 'User';

exports.up = async (knex) => {
  await knex.schema.alterTable(tableName, (table) => {
    table.dropColumn('role');
  });
  await knex.schema.alterTable(tableName, (table) => {
    table.enu('role', ['Owner', 'Admin', 'Researcher', 'AdvancedResearcher', 'User']).defaultTo('User');
  });
};

exports.down = async (knex) => {
  await knex.schema.alterTable(tableName, (table) => {
    table.enu('role', ['Admin', 'SuperUser', 'User']).defaultTo('User').alter();
  });
};

I've tried removing the alter() method at the end of the exports.down function, but to no avail (I really didn't think this would help, but I was desperate to try something). I don't know what else I should change to resolve this. I'd really appreciate it if someone could help me make sense of the error, especially the "check", and explain how I could go about fixing it.

Documentation probably should list types which are supported by .alter(). With enums it doesn't seem to make very reasonable queries.

To alter check constraint type of your enum you first need to drop the old constraint and then create a new one. Also with down script you would need to decide what to do with rows, which has AdvancedResearcher role set.

Hi @elhigu, thanks for helping me out! Could you please explain what you mean by "drop the old constraint." What is the "old constraint" and how would I go about dropping it?

If you have created enum, it is usually done as a string column that has CHECK constraint, with values that are allowed to be set to that column. So you can drop that constraint with knex.schema.raw() and probably you will also need .raw for creating a new constraint with updated enum values.

I appreciate your taking the time to help me, but I still don't understand how to apply what you've explained. Furthermore, my issue isn't resolved yet, so I'm a bit confused as to why you closed this issue.

The issue was closed , because it doesn't seem to be a bug rather than missing documentation that does not mention about limitations of the .alter().
The rest of the question is about general  SQL knowledge which should be asked about in knex gitter or stackoverflow. Also if someone has time to explain further about enums that are implemented as string columns that has check constraint for making sure that invalid strings are not set to column it can be still be discussed here further.

@lilfrootyloops
I think the Drop the old constaint is something like this (Not exactly for ur case, but could be similar)
Just quick info, what I was doing is that I'm incrementally adding column to the table, and I wanted to have a default value for the column since it can't be null. Yet I do not want the constraint of the table to be there after the migration, hence I remove it using .raw()
exports.up = function(knex) {
  return (
    knex.schema
      .table('someTable', function(table) {
        table
          .string('user', 50)
          .notNull()
          // This creates the constraint
          .defaultTo('1', { constraintName: 'df_template_user_migrate' });
      })
      // To drop the default value constaint while updating table
      .raw(`ALTER TABLE someTable DROP CONSTRAINT IF EXISTS df_template_user_migrate;`)
  );
};

exports.down = function(knex) {
  return knex.schema.table('someTable', function(table) {
    table.dropColumn('user');
  });
};

To visualize what is the roadblock, you should try to look at the schema from ur DB and figure out which constraint you need to remove.

I ended up solving my issue after playing around with the script for a bit. All I had to do was swap out table.enu('role', ['Admin', 'SuperUser', 'User']).defaultTo('User').alter() in favor of table.dropColumn('role') in line 14. The following is the contents of the updated 12345_updatePermissions.js:
const tableName = 'User';

exports.up = async (knex) => {
  await knex.schema.alterTable(tableName, (table) => {
    table.dropColumn('role');
  });
  await knex.schema.alterTable(tableName, (table) => {
    table.enu('role', ['Owner', 'Admin', 'Researcher', 'AdvancedResearcher', 'User']).defaultTo('User');
  });
};

exports.down = async (knex) => {
  await knex.schema.alterTable(tableName, (table) => {
    table.dropColumn('role');
  });
};


README file is incomplete.

nice work Isaac on your README file. Note: There is an extra unused ./ in front of the exam folder link in your README file that is better to remove.


cross filter not working when including dashboard in iframe

Bug description
Hello,
I can integrate a dashboard in my UI via the iframe provided with the embedded sdk, but cross filters don't work, in particular I can't drill down and drill by in a pie chart.
I see a 404 error when calling
the call to the api /api/v1/dataset/1
whereas the same call works when browsing the same dashboard in the Apache Superset UI.

Screenshots/recordings
No response
Superset version
4.0.2
Python version
3.9
Node version
16
Browser
Chrome
Additional context
No response
Checklist

[ ] I have searched Superset docs and Slack and didn't find a solution to my problem.
[ ] I have searched the GitHub issue tracker and didn't find a similar bug report.
[ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

this seems indeed a problem with permissions, because the issue disappears after changing
GUEST_ROLE_NAME = 'Gamma'
to
GUEST_ROLE_NAME = 'Admin'
to the config.py file
but now I have a CSRF 400 issue.
Anyway, what are the missing permissions to make the role Gamma work, instead of switching to Admin?


Dimmed splash screen on Android

Tested versions
Reproducible on Godot 4.2.1 stable.
System information
Windows 11, JDK 17(Also tried JDK 22), Android SDK Build Tools 33
Issue description
Exporting to android shows a very dimmed splash screen.
Steps to reproduce
Create a project with mobile renderer.
Add your android preset in export.
Enable Import ETC2 ASTC in VRAM Compression.
Provide unique package name.
Export the project or run a remote debug.
Launch the app.
Expected Result: Bright or at least color accurate splash screen image would appear when you launch the app.
Actual Result: Very dim splash screen image
Minimal reproduction project (MRP)
I believe this isn't project dependent but just in case the project below should be dim.
Dim Splash Screen.zip

It is probably this function. I'm not sure yet whether it will be activated before the splash screen.

https://github.com/godotengine/godot/blob/7ebc866418b075df58cbe4e31fcf8b0c3acd70a1/platform/android/java/lib/src/org/godotengine/godot/Godot.kt#L479

I just tested on version 4.2.2, and the issue also exists there. Someone in the forum suggested changing to gl_compatibility, but it did not work for me. I wonder if there is a temporary workaround.

Can you also reproduce it if you override the splash screen? Use a different image file and display it for longer.  For example 3000 ms, does the splash screen remain dark for 3 seconds?

Enable Import ETC2 ASTC in VRAM Compression.

This may be the issue. I don't think VRAM-compressed splash screen images can be correctly displayed.
Try using lossless compression instead. The difference in memory usage is pretty low since it's just a single image, and it's only displayed during startup.

@Alex2782 So I tried what you suggested, but I extended the duration to 10 seconds just to exaggerate the result. Apparently, the splash image only appears for a split second. When I slowed down the recording, it was visible only until the app became full screen. By the way, it’s also dim during the time it appeared.
https://github.com/godotengine/godot/assets/45304248/5e9043f8-b80f-4e2a-a9ed-e05e27511021
I also discovered an inconsistency with the splash image size as well. However, since it has already been discussed here, I won’t delve into it any further.
@Calinou I tried both the default and custom splash images. Additionally, I ensured that their compression is lossless, but they still appear dim.

I get what's happening now, based on the video output. The splash screen is actually black all along. The image you're seeing for a split second when opening the app is Android's own placeholder image, which is a static image defined in the APK and not something drawn by Godot.


Enable automatic testing

Testing is to be included in CI/CD pipeline.
A simple test is already included. More tests should be added.

Partly fixed with #20


After some time seems we lose docker IPs

Just checking out the beta site and weirdly it's "down". But all the containers are up:
ubuntu@ip-172-30-1-29:~$ sudo docker ps
CONTAINER ID        IMAGE                                   COMMAND                  CREATED             STATUS              PORTS                      NAMES
a117a2471238        mattgodbolt/compiler-explorer:swift     "/run.sh"                7 days ago          Up 7 days           <IP_ADDRESS>:20483->20483/tcp   swift
c889787e9d8d        mattgodbolt/compiler-explorer:haskell   "/run.sh"                7 days ago          Up 7 days           <IP_ADDRESS>:20482->20482/tcp   haskell
77f01e26f3bc        mattgodbolt/compiler-explorer:ispc      "/run.sh"                7 days ago          Up 7 days           <IP_ADDRESS>:20481->20481/tcp   ispc
5c8c85e22283        mattgodbolt/compiler-explorer:cppx      "/run.sh"                7 days ago          Up 7 days           <IP_ADDRESS>:20480->20480/tcp   cppx
9dc62aac9867        mattgodbolt/compiler-explorer:rust      "/run.sh"                7 days ago          Up 7 days           <IP_ADDRESS>:10242->10242/tcp   rust
e18b5b235a81        mattgodbolt/compiler-explorer:d         "/run.sh"                7 days ago          Up 7 days           <IP_ADDRESS>:10241->10241/tcp   dx
075ddfdce5a1        gliderlabs/logspout                     "/bin/logspout sys..."   7 days ago          Up 7 days           80/tcp                     logspout

Just trying to fetch / using telnet localhost fails with a bad gateway, and in the logs:
2017/11/14 13:45:34 [error] 5#5: *54864 connect() failed (113: No route to host) while connecting to upstream, client: <IP_ADDRESS>, server: gcc.godbolt.org, request: "GET / HTTP/1.1", upstream: "http://<IP_ADDRESS>:10240/", host: "gcc.godbolt.org"

interfaces:
ubuntu@ip-172-30-1-29:~$ ifconfig | grep 'inet '
          inet addr:<IP_ADDRESS>  Bcast:<IP_ADDRESS>  Mask:<IP_ADDRESS>
          inet addr:<IP_ADDRESS>  Bcast:<IP_ADDRESS>  Mask:<IP_ADDRESS>
          inet addr:<IP_ADDRESS>  Mask:<IP_ADDRESS>

Seems we've lost the 172.17.0.* docker bridge interface IPs. There's a ton of docker bridge interfaces but none with IP addresses.
As we've seen issues on the main site, I wonder if something funny's going on here.

A service restart "fixed" it. Though I noted the docker containers' virtual interfaces are never assigned IPs, so the IP thing is a red herring.


Developers SBAT reference documentation about key DAppKit functions

Expected Behavior
Please describe the behavior you are expecting
Current Behavior
What is the current behavior?

Closing this because DappKit is deprecated. #8259 is tracking the deprecation work.


Decide what API calls and parameters we want in the app.

https://spoonacular.com/food-api/docs
This is the API docs for reference.

We need to select parameters we want the user to be able to select to refine their search.


Please update to FCM API v1 without click_action

FCM API v1 has not click_action in notification.
https://firebase.google.com/docs/cloud-messaging/send-message
SDK lib prevents other listeners from receiving the click event. I can not use a listener for notificationclick in SW.
https://github.com/firebase/firebase-js-sdk/blob/master/packages/messaging/src/controllers/sw-controller.ts#L155
How to handle properly click event?
Curl for send message using FCM API v1...
curl -X POST -H "Authorization: Bearer $ACCESS_TOKEN" -H "Content-Type: application/json" -d '{
"message":{
  "token": "???",
  "data": {
    "badge": "images/badge.png",
    "icon": "images/icon.png",
    "image": "images/image.png"
  },
  "webpush": {
    "headers": {
      "TTL": "3600",
      "Urgency": "high"
    }
  },
  "android": {
    "priority": "high",
    "ttl": "3600s"
  },
  "notification": {
    "title": "TEST",
    "body": "BODY"
  }
}
}' "https://fcm.googleapis.com/v1/projects/$PROJECT_ID/messages:send"

@JosefJezek  right, we can't listener for notificationclick at SW.
The solution from https://github.com/firebase/quickstart-js/issues/102#issuecomment-278584863
sound good but not work for me...
Thanks for the help!

click_action is supported in the web push notification payload.

Using link in fcmOptions is preferred: https://firebase.google.com/docs/reference/fcm/rest/v1/projects.messages#webpushfcmoptions


why does show open files show multiple entries for same file?

Because if a file on a specific server could only be downloaded once, the potential for it getting out of sync with the servers version is quite large. I didn't, and don't, want to deal with that.

There is a settings option titled "Clear open files on start" or something like that that allows you to purge the downloaded files each time atom is opened up.

I do have that setting in my setup. I was just curious why there were around 10 instances of the file in the open list. It’s not a big problem.
Thanks for a great package that helps out when working from home.


pulsar-admin doesn't respect pulsar.service.url set in the connector config

Describe the bug
Trying to run debezium-postgres connector from pod in kubernetes, pg and pulsar are also deployed as separate pods. Looks like connector doesn't take into account provided pulsar.service.url: "pulsar://pulsar:6650" but trying to connect to  localhost/<IP_ADDRESS>:6650
The run command for connector:
command: ["./bin/pulsar-admin", "sources", "localrun", "--source-config-file", "configs/pg-pulsar-config.yaml"]

Logs from connector:
06:29:50.783 [pulsar-client-io-1-1] WARN  org.apache.pulsar.client.impl.ConnectionPool - Failed to open connection to localhost:6650 : io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: localhost/<IP_ADDRESS>:6650
06:30:20.010 [function-timer-thread-6-1] ERROR org.apache.pulsar.functions.runtime.RuntimeSpawner - public/default/debezium-postgres-source-java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException: java.util.concurrent.CompletionException: io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: localhost/<IP_ADDRESS>:6650 Function Container is dead with exception.. restarting
06:30:20.011 [function-timer-thread-6-1] INFO  org.apache.pulsar.functions.runtime.thread.ThreadRuntime - ThreadContainer starting function with instance config InstanceConfig(instanceId=0, functionId=21d2e125-382f-4da7-ab79-36eccfe8966d, functionVersion=84fca34b-91ad-4075-abd8-5f44182d1250, functionDetails=tenant: "public"
namespace: "default"
name: "debezium-postgres-source"
className: "org.apache.pulsar.functions.api.utils.IdentityFunction"
autoAck: true
parallelism: 1
source {
  className: "org.apache.pulsar.io.debezium.postgres.DebeziumPostgresSource"
  configs: "{\"database.user\":\"postgres\",\"database.dbname\":\"postgres\",\"database.hostname\":\"pg\",\"database.password\":\"postgres\",\"database.server.name\":\"dbserver1\",\"database.port\":\"5432\",\"pulsar.service.url\":\"pulsar://pulsar:6650\"}"
  typeClassName: "org.apache.pulsar.common.schema.KeyValue"
}
sink {
  topic: "debezium-postgres-topic"
  typeClassName: "org.apache.pulsar.common.schema.KeyValue"
}
resources {
  cpu: 1.0
  ram:<PHONE_NUMBER>
  disk:<PHONE_NUMBER>0
}
componentType: SOURCE
, maxBufferedTuples=1024, functionAuthenticationSpec=null, port=33649, clusterName=local, maxPendingAsyncRequests=1000)
06:30:20.786 [pulsar-client-io-1-1] WARN  org.apache.pulsar.client.impl.ConnectionPool - Failed to open connection to localhost:6650 : io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: localhost/<IP_ADDRESS>:6650

Connector config:
tenant: "public"
namespace: "default"
name: "debezium-postgres-source"
topicName: "debezium-postgres-topic"
archive: "connectors/pulsar-io-debezium-postgres-2.6.2.nar"

parallelism: 1

configs:
  database.hostname: "pg"
  database.port: "5432"
  database.user: "postgres"
  database.password: "postgres"
  database.dbname: "postgres"
  database.server.name: "dbserver1"
  pulsar.service.url: "pulsar://pulsar:6650"

pulsar.service.url is used for the offset topic used in debezium. It is not the one used for connecting to the sink topic.
You can run bin/pulsar-admin --admin-url http://pulsar:8080 sources localrun --source-config-file configs/pg-pulsar-config.yaml to point to the right pulsar cluster.

@Jennifer88huang @Huanli-Meng Can you take a look at this thread to improve our documentation for debezium connector?
@wolfstudy I think we can improve debezium connector to use same pulsar url for both the history topic used by the debezium connector and the output topic.

Got it. will check the doc and update it accordingly.

ok, works now, I had to provide  --broker-service-url pulsar://pulsar:6650 as well.


Getting Exception in thread "main" org.apache.spark.SparkClassNotFoundException: [DATA_SOURCE_NOT_FOUND] Failed to find the data source: deltaSharing. Please find packages at https://spark.apache.org/third-party-projects.html. while trying to read table as dataframe from a share.

import io.delta.sharing.client
import org.apache.spark.sql.SparkSession
object ReadSharedData extends App {
val spark = SparkSession.builder()
.master("local[1]")
.appName("Read Shared Data")
.getOrCreate()
val profilePath = "/home/knoldus/Desktop/Delta Open Sharing/resources/config.share"
val sharedFiles = client.DeltaSharingRestClient(profilePath).listAllTables()
sharedFiles.foreach(println)   /// this works fine and lists all the tables in the share provided by data provider.
val popular_products_df = spark.read.format("deltaSharing").load("/home/knoldus/Desktop/Delta Open Sharing/resources/config.share#checkout_data_products.data_products.popular_products_data")
popular_products_df.show()

@mohika-knoldus did you resolve this? I'm having the same issue.

@mohika-knoldus
The solution was to install apache Hadoop.
If you add some config to your spark context it will download it automatically:
spark = (SparkSession
.builder
.config('spark.jars.packages', 'org.apache.hadoop:hadoop-azure:3.3.1,io.delta:delta-core_2.12:2.2.0,io.delta:delta-sharing-spark_2.12:0.6.2')
.config('spark.sql.extensions', 'io.delta.sql.DeltaSparkSessionExtension')
.config('spark.sql.catalog.spark_catalog', 'org.apache.spark.sql.delta.catalog.DeltaCatalog')
.getOrCreate()
)

Or you can download it from the website.
Then you can read the table in like this:
delta_sharing.load_as_spark(table_url).show()
or like this:
spark.read.format("deltasharing").load(table_url).limit(100)

so either there is a dependency on python library or apache hadoop at the end ?

Thank you for the solution. @oliverangelil

Hi,
Do you know what other problems could be causing this? I am trying to run the quickstart_spark.py script from the examples folder, and even after adding the line for Hadoop, I am still getting the same issue.
Thanks!


Update tutorials.md

Just adding a quick note into the docs, to avoid a head scratching bug I got lost in for a couple of hours!

Thanks!


New Feature: Scrolling vertically

Hi, is that feature available on this library? if not, do you've plan for it. Seems like you've used ViewPager. I think migrating to ViewPager2 will help you to achieve it. Such a nice library it is!
Thank you.

Not a new feature actually :smile: it was implemented years ago, see #141 #166 #162 and many others :-)
Closing this if you don't mind, as it's duplicated

I need to download it offline as library, right?

You mean the Pdf document?
yes, you need to store it somewhere in your sd card, then load it

You mean the Pdf document?
yes, you need to store it somewhere in your sd card, then load it

no, I mean the other branch's library.

Oh, sorry :-)
there is a separate release that was taken from this branch. it is 1.1.3 You can get the dependency from #141 :D

how about with RemotePDFViewPager?  I have use 1.1.3 and change to VerticalViewPager, but it still same.


Add SAVED result code to hmi_connection

PR Changes:

Add SAVED result code to hmi_connection

Related: APPLINK-27448

Please review ASAP: @anosach-luxoft , @LuxoftAKutsan , @orhan-mehmedov, @dtrunov

@OHerasym Reviewed

Reviewed

Reviewed

OHerasym Reviewed.


Delete List Using the Delete List Button

As a user
I want to be able to delete active list
So that I can use the common button for this
Given: ToDoit app is running
And: The active has chosen
When: The user press 'Delete List' button
Then: The active list has been deleted
Acceptance:

Remove delete images from list titles

Current design:

Expected design:

@grace000 I removed closing tag for img, unused div and spaces. I also remembered what Dave told us about naming the inputValue and actionFormValue, so I renamed them too.


fixed potential typo in tegra186-flash-helper.sh

It seems to me boardrev should be board_version

Thanks... missed that one when I renamed that variable.


Firebase deploy fails

Deploying Firebase Cloud Functions fails in testing environment:
yarn run v1.22.18
$ firebase deploy

=== Deploying to 'phase2-eval'...

i  deploying storage, firestore, functions
Running command: yarn --prefix "$RESOURCE_DIR" build
$ tsc
✔  functions: Finished running predeploy script.
i  firebase.storage: checking storage.rules for compilation errors...
✔  firebase.storage: rules file storage.rules compiled successfully
i  firestore: reading indexes from firestore.indexes.json...
i  cloud.firestore: checking firestore.rules for compilation errors...
✔  cloud.firestore: rules file firestore.rules compiled successfully
i  functions: ensuring required API cloudfunctions.googleapis.com is enabled...
i  functions: ensuring required API cloudbuild.googleapis.com is enabled...
✔  functions: required API cloudbuild.googleapis.com is enabled
✔  functions: required API cloudfunctions.googleapis.com is enabled
i  functions: preparing codebase default for deployment
i  functions: preparing . directory for uploading...
i  functions: packaged /Users/daehyun/workspace/mpc-phase2-suite/firebase (43.33 KB) for uploading
i  functions: ensuring required API cloudscheduler.googleapis.com is enabled...
✔  functions: required API cloudscheduler.googleapis.com is enabled
i  storage: latest version of storage.rules already up to date, skipping upload...
✔  firestore: deployed indexes in firestore.indexes.json successfully
i  firestore: latest version of firestore.rules already up to date, skipping upload...
✔  functions: . folder uploaded successfully
✔  storage: released rules storage.rules to firebase.storage
✔  firestore: released rules firestore.rules to cloud.firestore
i  functions: creating Node.js 16 function refreshParticipantAfterContributionVerification(us-central1)...
i  functions: updating Node.js 16 function checkAndRegisterParticipant(us-central1)...
i  functions: updating Node.js 16 function coordinateContributors(us-central1)...
i  functions: updating Node.js 16 function initEmptyWaitingQueueForCircuit(us-central1)...
i  functions: updating Node.js 16 function processSignUpWithCustomClaims(us-central1)...
i  functions: updating Node.js 16 function registerAuthUser(us-central1)...
i  functions: updating Node.js 16 function setParticipantReady(us-central1)...
i  functions: updating Node.js 16 function startCeremony(us-central1)...
i  functions: updating Node.js 16 function stopCeremony(us-central1)...
i  functions: updating Node.js 16 function verifyContribution(us-central1)...
Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.
Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.
Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.
Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.
Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.
Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.
Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.
Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.
Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.
Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.

Functions deploy had errors with the following functions:
        checkAndRegisterParticipant(us-central1)
        coordinateContributors(us-central1)
        initEmptyWaitingQueueForCircuit(us-central1)
        processSignUpWithCustomClaims(us-central1)
        refreshParticipantAfterContributionVerification(us-central1)
        registerAuthUser(us-central1)
        setParticipantReady(us-central1)
        startCeremony(us-central1)
        stopCeremony(us-central1)
        verifyContribution(us-central1)
i  functions: cleaning up build files...
Error: There was an error deploying functions:
- Error Failed to update function verifyContribution in region us-central1
- Error Failed to create function refreshParticipantAfterContributionVerification in region us-central1
- Error Failed to update function checkAndRegisterParticipant in region us-central1
- Error Failed to update function initEmptyWaitingQueueForCircuit in region us-central1
- Error Failed to update function startCeremony in region us-central1
- Error Failed to update function stopCeremony in region us-central1
- Error Failed to update function coordinateContributors in region us-central1
- Error Failed to update function registerAuthUser in region us-central1
- Error Failed to update function processSignUpWithCustomClaims in region us-central1
- Error Failed to update function setParticipantReady in region us-central1
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Reproduce
cd firebase
yarn install

# Remove `remoteconfig` filed in firebase.json

FIREBASE_TOKEN=<YOUR-FIREBASE-TOKEN> yarn firebase:deploy

I have found this in the Firebase Cloud Functions logs. Therefore, the error seems related to a missing package (dotenv).
Detailed stack trace: Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'dotenv' imported from /workspace/dist/src/auth.js
** other traces **
Could not load the function, shutting down.

Some minor improvements:

Removed unuseful remote.config.template here a95ebb4fe86e7f677c9eae296b6770c8d652d5f2
Automate deploy process with custom scripts for downloading custom firestore indexes locally before deploy e10f162b9dfea4da8ead7091e898249acd9237f1

Everything should work fine :) Thansk @gurrpi for pointing this out!


Rmarkdown preview should respect code chunk options

In particular, for something like
```{r, eval=FALSE}
# example code: do not execute
load("bigdataset.rdata")
rxNeuralNet(bigformula, data=bigdataset)
```

it should just copy the code to the preview pane.

(I'm referring to the green triangle button, "Run current code chunk and update preview pane".)

It does read chunk options, just doesn't do anything with eval. So should then just copy output. OK. With comment or just the code?

Yes, it should have the comment as well. You would generally have eval=FALSE if the chunk is meant as an example, so the code is meant for the reader.

https://github.com/Microsoft/RTVS/commit/54601da3a4e4f0e7315f768e601a1f70d2bee8a3


Recognizing more terminals

People seem to have all sorts of TERM variables (like screen-256color, tmux-256color...) many of which aren't recognized by tcell so they get terminal entry not found. But it seems like if they just change their TERM variable to xterm-256color everything works. Is there a way of doing this automatically?
Here is the issue in micro: zyedidia/micro#20.

Would be nice if the lib could just support termcap/terminfo database. Should be possible to parse those into whatever stuff tcell uses.
Beware of reddit storm as a consumer of this library (micro) was just announced in version 1.0 on /r/linux.

It occurs to me that these are all -256color variants… We can certainly
automate detection of the 256 color so that if we have a terminal type
without the -256 color we just assume the XTerm style 256 color sequences.
This would be fairly analogous to what we do for -truecolor.
That said, I’d like confirmation of this.  Note also that users can create
their own terminal database from these terminals, provided that suitable
definitions for them exist in terminfo.
On Tue, Aug 30, 2016 at 9:12 AM, saivert<EMAIL_ADDRESS>wrote:

Would be nice if the lib could just support termcap/terminfo database.
Should be possible to parse those into whatever stuff tcell uses.
Beware of reddit storm as a consumer of this library (micro) was just
announced in version 1.0 on /r/linux.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/gdamore/tcell/issues/118#issuecomment-243493238, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABPDfeA4UCQ_w70WtzcmIT2rwnUAIuxiks5qlFaFgaJpZM4Jws7V
.

Ok I see. Thanks for the tip about mkinfo.go -- I had missed that somehow in the readme. I think this might solve my problem by compiling mkinfo.go with micro and running it if I get a terminal entry not found error.
Is it possible to run mkinfo without needing ncurses to be installed? I'd like to provide micro binaries with no runtime dependencies -- maybe there is a way of embedding just the necessary functions inside the binary or something? I think this has more to do with CGO than tcell.

The idea is that end users should rarely, if ever, need to use mkinfo.  The reason its a separate program is because of the CGO and curses (or ncurses) dependency -- I didn't want to write a full terminfo database parser (and the terminfo database format is not necessarily standard or portable, although ncurses uses the same format everywhere).
What you can do is have your users send you the output from infocmp $TERM ... this can then be imported into your own terminfo database with tic, and you can then run mkinfo yourself.
Frankly if you send me the output from infocmp for those terminal types I'd be happy to add them to the database myself.
Alternatively, I can add the code to do the -256color magic, assuming that these do work reasonably like xterm-256color (I have no reason to believe they don't -- but I don't have them.)

So what I'm going to do for now is update the database with latest from debian, plus rxvt-unicode-256color, and dvtm.  This won't get everyone, but it should get pretty much most of Linux. :-)

That's 1626 terminal types btw.


Use a loader before displaying the documentation site's not-loaded table issue.

resolve

thanks @garimarajput748