Is it possible to train TTS for a new language?

Thank you for your work. I would like to inquire about the possibility of training for a new language. If this is feasible, could you please provide more details on the following:

How much data is required?
In what format should the data be?
What resources are approximately needed to achieve results comparable to those for the English language?

Your insights on this matter would be greatly appreciated. Thank you in advance for your assistance.

1. Required Data Volume

Data Volume: Approximately 95K hours of English and Chinese data were used to train the base model. To achieve a high-quality TTS system, a large amount of data is required. For stable performance, at least 10,000 hours of voice data for a certain language is needed.

2. Data Format

Audio Format: The audio files should be in mono WAV format, with a sampling rate of 24kHz, using 100-dimensional log Mel spectrogram features, and a frame hop length of 256.
Text Processing: English uses letters and symbols directly; Chinese characters are processed into complete pinyin through Jieba segmentation and pypinyin.

3. Required Resources

Computational Resources: Training of the base model was conducted on 8 NVIDIA A100 80G GPUs, lasting more than a week. This demonstrates the significant computational resources required to train a high-quality TTS model.
Model Configuration: The base model includes 22 layers, 16 attention heads, and embedding/feed-forward network (FFN) dimensions of 1024/2048. The configuration of the small model is detailed in Appendix B.1.

4. Results Comparable to English

Model Performance: To ensure high-quality TTS comparable to English, a large amount of training data and advanced model architecture were used. In addition, various test sets were used for evaluation, including LibriSpeech-PC test-clean, Seed-TTS test-en, and Seed-TTS test-zh, to ensure the model's wide applicability and fairness of comparison.

5. Training Details

Optimizer: The AdamW optimizer was used, with a peak learning rate of 7.5e-5, linear warm-up for the first 20K updates, followed by linear decay.
Regularization: Attention and FFN use a dropout rate of 0.1 to prevent overfitting.
Data Augmentation: Random masking of 70% to 100% of Mel spectrogram frames for fill-in-the-blank training tasks.

All training details is mentioned in our paper.
And you could simply train your own model for a new language:

Leverage Emilia Dataset (DE EN FR JA KO ZH), as we have include script for it (NOTE. download the mentioned version of Emilia in script, cuz it's currently updated to a WebDataset ver.)
or prepare your own data pairs if not covered, just tailor a Dataset Class in model/dataset.py to your need

For Base model (multilingual, ~300M), we use <50K hours for each language
For Small model (e.g. Chinese-only, ~150M), we have made it work with just 1K hours data, config. mentioned in our paper also
Just one thing, the training would take a long time, especially for E2 TTS (if you choose)
And be patient, 8RTX3090 small model for one week (200~400K updates to hear something reasonable)
8A100 for base model similarly.

@SWivid For each language a separate model has to be trained ? or a joint model can be trained for multiple additional languages?

Do you plan to add Spanish language in the near future?

Could you please provide specific hyperparameters for training a small model? For example, batch_size_per_gpu and grad_accumulation_steps ?

Could you please provide specific hyperparameters for training a small model? For example, batch_size_per_gpu and grad_accumulation_steps ?

check src/f5_tts/scripts/count_params_gflops.py and appendix in our paper


OverlayTutorialWidgetHint builder doesn't get called in OverlayTutorialCustomShapeEntry

Hi,
Thank you very much for your excellent work on this package, it's very useful. I'm currently using it for showing an introduction tutorial in my app, but I'm having a problem.
I have configured an OverlayTutorialHole with an OverlayTutorialCustomShapeEntry, and inside a OverlayTutorialWidgetHint showing a text. The overlay and the hole are being showed successfully. However, the builder in the OverlayTutorialWidgetHint is never being called, and so the hint isn't showed. My code is:
...
                   Align(
                        alignment: Alignment.bottomRight,
                        child: OverlayTutorialHole(
                          enabled:
                              true,
                          overlayTutorialEntry: OverlayTutorialCustomShapeEntry(
                            shapeBuilder: (rect, path) {
                              final animatedFABRect =
                                  mainFABKey.globalPaintBounds;

                              path = dart_ui.Path.combine(
                                PathOperation.difference,
                                path,
                                dart_ui.Path()
                                  ..addOval(
                                    animatedFABRect!,
                                  ),
                              );
                              return path;
                            },
                            overlayTutorialHints: <OverlayTutorialWidgetHint>[
                              OverlayTutorialWidgetHint(
                                  position: (rect) => Offset(0, rect.center.dy),
                                  builder: (context, rect, rRect) {
                                    return SizedBox(
                                      width: rRect.left,
                                      child: Padding(
                                        padding:
                                            const EdgeInsets.only(right: 8.0),
                                        child: Row(
                                          mainAxisAlignment:
                                              MainAxisAlignment.end,
                                          children: [
                                            Flexible(
                                              child: Text(
                                                'Click here to add counter',
                                              ),
                                            ),
                                          ],
                                        ),
                                      ),
                                    );
                                  })
                            ],
                          ),
                          child: AnimatedFloatingActionButton(...)

Thank you !

Hey @smunl, the fix is available in v2.2.0.


Feature Request: Allow editing for healing items

I would like to see changing options for the healing items, as example:
AFAK

allow configuration of stopping light and/or heavybleeding
allow configuration of curing a fracture
allow configuration of restoring blacked bodyparts

Why:
For beginners or entry level/just for fun players it removes the hazzle to remember which item needs to be used for which type of injury just making it easier.

Hi @ViP3R76,
Thank you for your suggestion! I’ve implemented the changes you requested in the latest update, v1.2.0.
Additionally, I've introduced a new stimulant called Exodrine.
Feel free to download the latest version and let me know if there's anything else you'd like to see improved!
Thank you for your feedback and support 🥇


[Proposal] Reward allocation design according to Stake ratio

TL;DR
Reward allocation algorithm in Random Sampling without Replacement (non-duplicate elections) with a compensatory model of opportunity loss.

When we elect n Voters from N Candidates without duplication, the problem is that the Voter who is selected in the k-th selection loses the opportunity to earn the rewards for the rest n-k times selection.
Let α to be the base reward for winning of one selection, s[I] to be the amount of Candidate i’s stake, S to be the total amount of stake for the set of Candidates, σ[r] be the total stake of the Candidates who were selected before the r-th selection. Then the i’s expected reward in r-th selection is αe[i,r]=α s[I] / (S−σ[r]).
Add this expected reward as compensation for the lost opportunity n-k selection to the reward for Candidate who was elected in the k-th selection, in addition to the base reward for winning.
This is, the k-th selected Candidate in the selection of n Voters eventually acquires the next reward:

This means that the reward up to the k-th selection are obtained by random simulations and the rewards that fail to participate in selections after the k-th selection are obtained in terms of probabilistic expectations.
Problem Setting
What we want to do: Creates a randomly selected subset from a set. In this case, the elements of the set have weights that represent the probability of being selected. Candidates who are selected will be rewarded. we want this reward to be proportional to their weight.
On this page, I'm considering a way to select exactly a given number n of elements (Voters) from a set (Candidates) with a probability corresponding to their weight (Stakes). And we also want the reward of each Candidate to be exactly proportional to its weight.

Sampling with replacement: An intuitive way to select exactly n elements randomly from a set of N is to repeat random sampling from the set until the selected subset becomes n. However, this way isn't guaranteed to finish in constant time because the same element may be repeatedly selected. This becomes a problem of liveness in distributed systems.
Sampling without replacement: It's possible to prevent selecting the same element repeatedly by excluding the selected element from the set. However, as the number of elements in the set decreases, the probability of winning the remaining elements increases. For example, in a case with a biased probability of winning, the element with a higher probability of winning is selected earlier, so the one that has a relatively lower probability of winning tends to be higher than the actual bias. This is a problem of fairness.

The purpose of this page is to lead a formula that solves the problem of fairness in 2.
The problem in 2 is that the total Stake in Candidate set becomes smaller as the Voter selection progress, so the probability of being elected by a Candidate who is unlikely to be elected, i.e., a Candidate with lower Stake, becomes a higher probability than the Stake it holding. In fact, as shown in Fig 1, in case we repeat the election that elects 25 Voters from a set of 100 Candidates with inverse bias in Stake, if all 25 Voters acquire the same reward in one election, then the Candidate with the higher Stake will acquire a lower rate of the overall reward distribution.

Fig 1. The ratio of Stake that each Candidate holding and the reward that each one earned in non-duplicate case. In the service requirements, the two should be aligned.
Perspectives and Formulas
I assumed that the underlying cause of the reduction of reward for Candidates who are more likely to be selected is the loss of opportunity due to their inability to participate in selections after selected in the round. In other words, adding the number of elections it has already been elected to and couldn't participate in, and the expected amount of its income, to its reward, would seem to be a fair distribution.
More precisely, the increase in the probability of winning is due to a decrease in the total Stake in the Candidate set as the result of the elected Voter drops out of the Candidate. Therefore, the chance of being elected is getting higher in later elections than in the first.
Let α be the base reward and si be the Candidate i's Stake holdings, The i's expectation reward for a single selection can be expressed as follows:

where S is the total Stake holdings of the candidates participating in that election.
Expected reward in duplicate case
(You can skip this section because this is an explanation as comparative.) I notate S=S=∑isi if all of Candidates participate in the all n selection (i.e., if there is a possibility of multiple winning for a selection) because the S is a fixed value for all selection. Thus, the expected reward for n selections from a Candidate set can be simply added up and expressed as follows:

Due to the nature of the multinomial distribution, repeating n=1 100 times, repeating n=25 4 times, and doing n=100 once all yield the same expected reward.
Although this case can work simple and fair like Fig.2, it doesn't always result in n Voters being selected n times due to duplicate elections. Thus, the Byzantine assumption may be weakened by a smaller number of Voters than assumed. And due to the possibility of consecutive duplicate selections, it's not possible to guarantee the end time even if the selection is repeated until a certain number of Voters are selected.

Fig 2. Reward distribution ratio in case of the duplicate winning scheme.
Expected reward in non-duplicate case
By excluding the selected Voters from the Candidates, exactly n Voters can be selected in n elections (if N≥n).
However, as the number of Candidates decreases, the total amount of Stake in Candidate set decreases, and the probability of being selected increases in later elections. This would result that each total reward amount will not proportional to the number of Stake holdings. The trend is that Candidates with more Stake holdings are selected earlier and those with fewer holdings are less likely to be selected so that Candidates with more Stake holdings are accounted for less compensation. This can be seen from the graph in Fig. 1.
There are two reasons why such a difference occurs.

If a Candidate is selected for the k-th selection, it will not receive a reward equivalent to the remaining n−k selections.
As the total Stake holdings S of the Candidate set decreases as the number of Candidates decreases, the expected reward α¯i for Candidate i fluctuates with the progress of the selection.

Introduce the concept of reward expectation
Assume a scratch lottery to win $1M for a particular person. If 100,000 people participate in this lottery, the expected reward per person is $1,000,000÷100,000 = $10 (eliminating the gambling nature as a pastime, it's worth buying the scratch lottery if you can buy it for $9, but it isn't if it's $11).
 So if you couldn't participate in the lottery for some unavoidable reason such that the numbers weren't written of print error, your opportunity loss is $10. However, in a very rough way, it could say that if you get the expected reward of $10 as compensatory of the opportunity loss, the distribution of worth with other participants will be fair.

Opportunity loss compensation model
The following example Fig 3 represents an election that selects four Voters, i,j,k,l, out of a set of N Candidate set.

Fig 3. An election to select four Voters from a set of N Candidates.

In the first selection, the winner i is given a reward α.
In the second selection, the winner j is also given a reward α. There are two important points here.

First, the total amount of Stake S2 of the second selection has decreased by si due to the departure of i in the first selection, and the winning probability px,2 of the remaining Candidates increase.
Second, the first winner i doesn't participate in the second selection. This means that i is losing the opportunity to get a reward of 2nd selection. If i were left in the Candidates, the expected reward for the second selection would have been α¯i,2=αsiS2. In other words, the amount of opportunity loss that i suffered due to not being able to participate in the 2nd selection can be said to be α¯i,2.

Similarly, in the third selection, the total amount of stake in the Candidate set is S3=S1−si−sj, the winner k is also given a reward α, and i and j suffer an opportunity loss in an amount equal to α¯i,3=αsiS3 and α¯j,3=αsjS3, respectively.
It is the same for the 4th selection, but note that the last selected l has all 4 selection opportunities, so it doesn't have any opportunity loss.

This model is based on the hypothesis that we can make it fair by compensating for lost opportunity rewards as expected reward that would have ben probabilistically obtained i it had participated din the selection.
In summary, a Candidate i who is selected for the k-th selection should be rewarded with the following:

where  Sˆr is the total stake of the Candidates selected from r=1 to k−1.
Result
Here, I show the reward distribution by the simulation. The amount of Stake held by each Candidate and the reward for them are expressed as an overall ratio.

Fig 4.1. Stake distribution with inverse bias.

Fig 4.2. Stake distribution with linear bias.

Fig 4.3. Stake distribution with a flat.

Fig 4.4. A case where Stake is largely ×100 more biased toward one Candidate.
Fig 4.x shows the result of simulation in which only the base reward for winning the election and the opportunity loss is compensated by the expected reward in addition to the base reward in an election in which 25 Voters are selected from 100 Candidates. For 10000 trials, The base reward-only allocation (red points) results in lower distributions for Candidates with high Stake ratios, and higher distributions with relatively low Stake ratios. It can be seen that the allocation compensated by the expected reward behaves according to the Stake ratio.
Note that in the case where Stakes are flat in Fig 4.3, this shows a fair distribution even without correcting. In other words, the unfairness of bias is an inherent problem of weighted random sampling.

Fig 5.1. The number of Candidates is very small and close to the number of Voters.
Fig 5 shows the case where the number of candidates is small and the number of candidates is close.
Consideration
This simulation uses floating-point arithmetic. When the unit of reward is an integer, it's necessary to ensure that the results are not biased by rounding.
The total amount of reward α×n+δ paid in a single election will vary from election to election. However, if  S1 is very large with respect to si, then δ will be a small value.
References

https://gist.github.com/torao/39b6cb2d4d95d9f497c6c28fcd4745a1
https://docs.google.com/spreadsheets/d/1jZYplc6YJemC8e6X8xmppFLvGnTRLCkEHVaz4VeiiVk/edit#gid=0

I've noticed this problem of inequality of opportunity due to random sampling without replacement is essentially the same as the Monty Hall problem, which is a famous example of the posterior probability of Bays Theorem. A more theoretical explanation may be possible if we consider this problem as a type of Bayesian statistics.


org.flatpak.Builder not on flathub

It looks like org.flatpak.Builder as been removed from flathub.  Are there any alternatives install methods you could detail?

Sorry for not replying to you earlier. Flatpak Builder should still be available on flathub (I can still install it from there). However, I've just added another install method: You can get the latest x86_64 build from Github Actions now. Just download & unzip the artifact, then install from the .flatpak file.

That helped immensely.  I got it going and it synced the time.  Is that the only complete feature at the moment?  I just want to be sure it built properly.

In addition to the time sync, it also forwards notifications to the watch and allows you to control any MPRIS-compatible audio/video player with wasp-os' music player app.


[StableHLO] Rename the stablehlo_experimental to stablehlo

Now that the stablehlo input conversion pipeline nearly reached the parity with the mhlo/xla pipelines, drop the experimental suffix and rename --iree-input-type=stablehlo_experimental to stablehlo.
Issue: https://github.com/openxla/iree/issues/12678

Woot! Amazing news 🎉


Support for .mdx files

Does uptriever not support .mdx files?
Seeing the following issue when I try to run on our docs:

Unsupported file type: /Users/pranay/code/signoz-web/data/docs/alerts-management/exceptions-based-alerts.mdx
/opt/homebrew/lib/ruby/gems/3.3.0/gems/uptriever-0.1.1/lib/uptriever/document.rb:26:in `to_html'
/opt/homebrew/lib/ruby/gems/3.3.0/gems/uptriever-0.1.1/lib/uptriever/document.rb:32:in `to_chunk_json'
/opt/homebrew/lib/ruby/gems/3.3.0/gems/uptriever-0.1.1/lib/uptriever/cli.rb:44:in `block in run'
/opt/homebrew/lib/ruby/gems/3.3.0/gems/uptriever-0.1.1/lib/uptriever/cli.rb:44:in `each'
/opt/homebrew/lib/ruby/gems/3.3.0/gems/uptriever-0.1.1/lib/uptriever/cli.rb:44:in `flat_map'
/opt/homebrew/lib/ruby/gems/3.3.0/gems/uptriever-0.1.1/lib/uptriever/cli.rb:44:in `run'
/opt/homebrew/lib/ruby/gems/3.3.0/gems/uptriever-0.1.1/bin/uptriever:7:in `<top (required)>'
/opt/homebrew/lib/ruby/gems/3.3.0/bin/uptriever:25:in `load'
/opt/homebrew/lib/ruby/gems/3.3.0/bin/uptriever:25:in `<main>'

Not yet (though that's sounds like an interesting and challenging task).
It doesn't seem that's converting MDX to HTML is a simple task even from Node (looking at this issue and another one).
I think, for now the quickest solution would be to write a custom JS script to convert .mdx files to .html and then run uptriever (and add **/*.mdx to the ignore list).
If you know how to write a generic mdx2html.js script, we would be happy to embed it into uptriever 🙂


Added a modal! When an element is clicked on, a modal with more infor…

Modal added! Huzzah

Nice looking modal, self


Fix pkeyutl inability to directly access keys on hardware tokens

Cleaned up version of #548, rebased on top of fresh master.  @mouse07410, please confirm that this looks like your stuff

+1 from me, if @mouse07410 confirms.

Yes it is a cleaner version of my stuff, thank you!  +1 from me.  Please commit/merge this!
One thing, we may have a problem with these (around line 109 in apps/pkeyutl.c):
    {"peerform", OPT_PEERFORM, 'F'},
    {"keyform", OPT_KEYFORM, 'f', "Private key format - default PEM"},

The problem is - both -peerformand (especially!)-keyformcan (or should be able to) take as argument one of:PEM, DER, engine. The current specification of these two seems to have dropped the engine` part.
Can you address it, replacing 'F' (and 'f') with whatever's appropriate to ensure all the three alternatives are enabled?

One thing, we may have a problem with these (around line 109 in apps/pkeyutl.c):
    {"peerform", OPT_PEERFORM, 'F'},
    {"keyform", OPT_KEYFORM, 'f', "Private key format - default PEM"},

The problem is - both -peerform and (especially!) -keyform can (or should be able to) take as argument one of: PEM, DER, **engine**. The current specification of these two seems to have dropped the engine part.

Both are 'E' in master, so...

Ok, that issue resolved, I'm merging

Aaaand merged!

Thank you!! !! !! ;)


Make scikit-learn-intelex an optional dependency

Report
In order to be able to install ehrapy on machines with MacOS and ARM architecture (M-chips), we need to make scikit-learn-intelex an optional dependency.
I had a quick look where the module sklearnex is used and it seems that one always checks if the module is importable anyways before actually using it. Hence, making it an optional dependency should not cause any problems.
Version information
No response

Done.


riot-rs-threads: WFI not working as expected on riscv/ esp

While testing #304 I noticed that the WFI (wait for interrupt) in riot_rs_threads::arch::riscv::sched (when the runqueue is empty) doesn't work as expected, at least not when using it together with embassy timers.
The expected behavior would be that when the scheduler wfi, an embassy timer that expires would cause an interrupt and the scheduler would wake up again.
It works for the cortex_m impl as expected, so it's most likely that:
a) we are using the WFI mechanism for riscv wrong and/or a missing something or
b) there is some issue related to the embassy timer implementation for esp
I'll look into it.

It works for the cortex_m impl as expected, so it's most likely that:

This is actually not correct. I ran into the same issue for the nrf52840dk when using the executor-thread from #304 and running this simple tasks:
// Will run in a thread when the `executor-thread` feature is enabled
#[riot_rs::task(autostart)]
async fn task() {
    riot_rs::debug::println!("[task] Hello");
    embassy_time::Timer::after(embassy_time::Duration::from_secs(5)).await;
    riot_rs::debug::println!("[task] Hello again");
}

The execution stalls after the timer.
Interestingly, the issue here is different than for esp.
For the esp, the WFI seems to block everything, so when the timer triggers, the __pender for the embassy executor doesn't get a chance to run in the first place.
For cortex-m on the other hand, the __pender function runs and it tries to wake up the thread, but then the PendSV exception doesn't go through. Also, this only seems to be an issue when testing on nrf52840dk; on the rp2040 the PendSV exception doesn't go through.
@kaspar030 any idea what the issue here could be? I will do further debugging, but maybe you already know what the problem is?

For cortex-m on the other hand, the timer triggers but then the PendSV exception that should wake up the thread doesn't go through. Also, this only seems to be an issue when testing on nrf52840dk; on the rp2040 the PendSV exception goes through.

I can reproduce this on nrf52840dk. Just adding a println!() in there makes the thread wake up again. weird.

Adding a fence in the cortex-m sched() fixes the issue on nrf52840dk. @elenaf9 can you check if adding such a barrier fixes the risc-v case?
btw, on cortex-m, the pendsv is intentionally set to have the lowest priority (0xff), so other ISRs can interrupt. is sth similar configured on riscv?

For esp, I noticed the issue when testing the esp-wifi thread (#300), because the systimer didn't trigger.
The WFI seems to block it. Increasing the priority of the systimer0 for the esp-wifi thread fixes it.

ah, re-read and saw this now. The interrupt that does the "wfi" dance needs to be preemptible, so probably set to the lowest priority possible. (does that even work on risc-v?)

For esp, I noticed the issue when testing the esp-wifi thread (#300), because the systimer didn't trigger.
The WFI seems to block it. Increasing the priority of the systimer0 for the esp-wifi thread fixes it.

ah, re-read and saw this now. The interrupt that does the "wfi" dance needs to be preemptible, so probably set to the lowest priority possible. (does that even work on risc-v?)

Yes, I have set the FROM_CPU_INTR1 interrupt (that triggers the scheduler) at lowest prio.
The problem was that the systimer interrupt in esp-wifi was also set to lowest prio. So I guess for esp it's not a bug, but rather me not thinking the WFI mechanism through :)
If the interrupts in embassy are all set to a higher prio, this won't be a problem.
I'd close this issue as soon as #312 is merged, and reopen/ reiterate if we run into an scenario where we wait for an (embassy?) interrupt that doesn't have a high enough prio.


Treat License text content as a different viewholder

split one row to header and content
on header click, insert content row and notifyItemRangeInserted
on second header click/content row click, remove content row and notifyItemRangeRemoved

https://github.com/bignerdranch/expandable-recycler-view


Easily set included schemes without file browser

I was confused at first because there weren't clear instructions on what to do to apply one of the included schemes. I didn't know that I needed to navigate to ~\Matlab\Add-Ons\Collections\MATLAB Schemer\scottclowe-matlab-schemer-2156bb9\schemes to find a file to load.
My suggestion would be to add support for quickly setting the included schemes via an API like schemer_import('monokai') and the function automatically loads the included file.

Also for a coworker the Add-Ons folder wasn't in his Matlab folder, but in AppData, so I suppose that location varies depending on the installation

Sorry to hear you had difficulties getting it to run.
The quick start instructions are to run schemer_import without any
parameters. If you do this it will open a GUI to select the colour scheme
.prf file. By default, the GUI should open at the directory containing the
schemes which ship with schemer. So it should open with the directory
containing monokai.prf showing so you can select it straight away.
What happens for you when you run schemer_import without any arguments?
On Tue, 26 May 2020, 21:27 Brett Ables<EMAIL_ADDRESS>wrote:

I was confused at first because there weren't clear instructions on what
to do to apply one of the included schemes. I didn't know that I needed to
navigate to ~\Matlab\Add-Ons\Collections\MATLAB
Schemer\scottclowe-matlab-schemer-2156bb9\schemes to find a file to load.
My suggestion would be to add support for quickly setting the included
schemes via an API like schemer_import('monokai') and the function
automatically loads the included file.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/scottclowe/matlab-schemer/issues/26, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAWCADZMFDV3LXXBOY2NCVDRTQQZRANCNFSM4NLCZC7Q
.

Thank you for mentioning the directory containing the schemes. I otherwise wouldn't have found it. I also have the GUI open to my home directory when running schemer_import without argument.

I got around this (and made switching between my favorite schemes on the fly much easier) by writing a small wrapper that uses which to find the proper location.
function switch_scheme(theme)
arguments
    theme (1,1) string = "default";
end

switch theme
    case "default"
        themefile = "default.prf";
    case "dark"
        themefile = "darkmate.prf";
    case "contrast"
        themefile = "vibrant.prf";
    otherwise
        themefile = "default.prf";
end

themefile = fullfile(fileparts(which("schemer_import")),"schemes",themefile);

schemer_import(char(themefile));

end

I got around this (and made switching between my favorite schemes on the fly much easier) by writing a small wrapper that uses which to find the proper location.
function switch_scheme(theme)
arguments
    theme (1,1) string = "default";
end

switch theme
    case "default"
        themefile = "default.prf";
    case "dark"
        themefile = "darkmate.prf";
    case "contrast"
        themefile = "vibrant.prf";
    otherwise
        themefile = "default.prf";
end

themefile = fullfile(fileparts(which("schemer_import")),"schemes",themefile);

schemer_import(char(themefile));

end


django.db.utils.DataError: value too long for type character varying(2) on manage.py migrate

What I'm trying to achieve
I upgraded saleor to the newest version from master in our testing environment. When trying to migrate the updated DB schemas, the command fails:
Operations to perform:
  Apply all migrations: account, auth, checkout, contenttypes, core, discount, django_prices_openexchangerates, django_prices_vatlayer, extensions, giftcard, menu, order, page, payment, product, shipping, site, sites, warehouse, webhook, wishlist
Running migrations:
  Applying checkout.0023_checkout_country...Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.StringDataRightTruncation: value too long for type character varying(2)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/migrate.py", line 231, in handle
    post_migrate_state = executor.migrate(
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/operations/fields.py", line 110, in database_forwards
    schema_editor.add_field(
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/schema.py", line 480, in add_field
    self.execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/schema.py", line 142, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.DataError: value too long for type character varying(2)

Steps to reproduce the problem
Upgrade version 2.9 with data in the DB to the newest version from master
Did anybody face this issue as well? Seems to be an issue from Postgres, but I can't allocate the root-cause.

We've seen something similar in another project, if you've created the database with COUNTRIES_ONLY configured set to a different value, then Django fails to detect the CountryField(multiple=True) change and properly migrate the database to the new field size.

thank you for that hint! We set the following environment variables:
DEFAULT_CURRENCY='EUR'
DEFAULT_COUNTRY='DE'

And that seems to be wrong! After removing the quotes it runs further.
But now I see the following:
  Applying product.0112_auto_20200129_0050...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/migrate.py", line 231, in handle
    post_migrate_state = executor.migrate(
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/operations/special.py", line 190, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/app/saleor/product/migrations/0112_auto_20200129_0050.py", line 28, in create_unique_slugs_for_producttypes
    slug_values.append(slug)
AttributeError: 'QuerySet' object has no attribute 'append'

Should I open a different issue or do you have a quick fix for this as well? 🙈

That looks like a bug in migration code, please open a separate issue for it.

By the way, you should be able to work around this by changing the code of the saleor/product/migrations/0112_auto_20200129_0050.py migration lines 22-24 to read:
            slug_values = list(ProductType.objects.filter(
                slug__istartswith=first_char
            ).values_list("slug", flat=True))

The only change is the list() call around the query.

thank you for the fast answer! When adding your fix I receive the following new issue:
Running migrations:
  Applying product.0112_auto_20200129_0050...Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NotNullViolation: column "slug" contains null values

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/migrate.py", line 231, in handle
    post_migrate_state = executor.migrate(
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/migration.py", line 124, in apply
# Generated by Django 2.2.9 on 2020-01-29 06:50
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python3.8/site-packages/django/db/migrations/operations/fields.py", line 249, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/schema.py", line 564, in alter_field
    self._alter_field(model, old_field, new_field, old_type, new_type,
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/schema.py", line 152, in _alter_field
    super()._alter_field(
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/schema.py", line 710, in _alter_field
    self.execute(
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/schema.py", line 142, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: column "slug" contains null values

Could you maybe just give me a hint, what might be the issue here? Could I modify something in the DB to make the migration run again?

I think the migration is buggy and fails under certain circumstances. Please open a separate issue for the migration failing so we can close this one.

Will do it, thanks

jannikzed .. did find solution .. i have same migration failling ?

the issue is tracked here: https://github.com/mirumee/saleor/issues/5391
Unfortunately, it is still failing.


Unclear in documentation if PostgreSQL search backend supports indexing extra fields

Issue Summary
Search indexing page warns that "Indexing extra fields is only supported with ElasticSearch as your backend." while PostgreSQL search backend page is described as "Supports all the search features available in Wagtail." The PostgeSQL backend in also recommended for "sites with less than a million pages"
It's unclear from the documentation if the PostgreSQL backend supports indexing extra fields.
Steps to Reproduce
The extra fields warning can be viewed here: http://docs.wagtail.io/en/v1.12.2/topics/search/indexing.html
The warning has remained unchanged since before the PostgreSQL backend was introduced, which can be seen here:
http://docs.wagtail.io/en/v1.9/topics/search/indexing.html#indexing-custom-models

Thanks for the report @sivictorallen! Yes, that sentence is no longer accurate since the PostgreSQL backend was added - updated now.


Add parameter --log-file

Tell us more about this new feature.
We are using mountpoint-s3 inside Docker containers running as ECS Tasks in regulated environment where very minimalistic solutions are possiblle.
All logs must go to STDOUT OR STDERR.
AWS documentation also recommends this.
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/container-considerations.html
Configure containerized applications to write logs to stdout and stderr.
Decoupling log handling from your application code gives you flexibility to adjust log handling at the infrastructure level. One example of this is to change your logging system. Instead of modifying your services, and building and deploying a new container image, you can adjust the settings.
Another documentation recommends this:
https://repost.aws/knowledge-center/ecs-eks-troubleshoot-container-logs
Build your container with application log files linked to STDOUT and STDERR. Or, configure your application to directly log to /proc/1/fd/1 (stdout) and /proc/1/fd/2 (stderr). For examples, see nginx and httpd official container images on the Docker Hub website.
The offical nginx docker image also uses this:
# forward request and error logs to docker log collector
    && ln -sf /dev/stdout /var/log/nginx/access.log \
    && ln -sf /dev/stderr /var/log/nginx/error.log \

Current situation for mountpoint-s3:
https://github.com/awslabs/mountpoint-s3/blob/main/doc/LOGGING.md
Logging to a file
You can redirect logs to a file instead of syslog by providing a destination directory using the -l, --log-directory command-line argument with mount-s3.
mount-s3  <MOUNT_PATH> --log-directory <LOG_DIRECTORY>
The directory will be created if it doesn't exist. A new log file will be created for each execution of mount-s3. Both the directory and log files are created with read/write access for the process owner and read access for the process owner's group. Log files are not automatically rotated or cleaned up.
Section "Logging to a file" should be rather renamed to "Logging to directory".
I need mountpoint-s3 to log all into STDOUT or STDERR.
It is not possible to redirect logging into one specific file.
Why you do not offer that basic option?
WE NEED parameter for logging to STDOUT or STDERR (as mentioned earlier in this post).
Give us just one simple parameter: --log-file

Hi, thanks for creating this issue.
In the logging documentation, it says that to log to stdout, you can run mount-s3 in foreground mode (the -f, --foreground command-line argument). If you need to, you can redirect stdout to stderr with mount-s3 -f ... 1>&2
Please let us know if that doesn't work for your use case.

Hello @muddyfish
this does not work for our use case.
We have Dockerfile where we call entrypoint.sh shell script.
I have tried mount-s3 with --foreground
sudo -E mount-s3 --foreground --debug --debug-crt --log-metrics --allow-other --allow-delete --allow-overwrite --dir-mode 0777 --file-mode 0666 --region eu-central-1 "${BUCKET_NAME}" /home/s3dir
The logging of mount-s3 is happening, but after mount-s3 we need to start java then and will NOT happen, because mount-s3 is in foreground. The main process in the foreground should be java. This is purpose of the container. The java is using files mounted from s3.
java -Xmx4000M -XX:+UseZGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=99 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom -jar "./${APP_NAME}.jar"
If i start mount-s3 in foreground, then java is not started and health-check for my java app will fail and container is then killed.
I need to tell mount-s3, that there is one log file and then i can redirect that log file to stdout.
Now you just offer me to run mount-s3 as main process in foreground and that is not our use case.
We can just define log directory. Many new files are then created inside docker container and that is very tricky to handle. We do not want to keep log files inside container.

In that case, as a workaround, I can suggest that you redirect stdout and run Mountpoint in the background with &.
mount-s3 -f ... > /dev/stdout &
java ...

I hope that's able to help you move forward. I'll leave this open to gather interest in the feature for full support (👍 the main issue).

In that case, as a workaround, I can suggest that you redirect stdout and run Mountpoint in the background with &.
mount-s3 -f ... > /dev/stdout &
java ...

In the workaround above, it does mean a failure to mount will not be detected as the container will no longer fail when the mount fails. (The main application may fail, of course, if it depends on the content of the mount.)

We'd probably recommend here that Mountpoint should be run inside another container, and that the mount is shared between the Mountpoint container and your application container. This is described in our documentation on docker under "Running as a service": https://github.com/awslabs/mountpoint-s3/tree/main/docker#running-as-a-service
As Simon said, we'll leave this issue open to gather interest in the feature.


FIX for PremProxy and FreeProxy

http_request_randomizer.requests.errors.ProxyListException.ProxyListException: list is empty
-> FreeProxyParser.py: html parsing is list error. (now solved.)
reqeusts.exceptions.InvalidURL: Failed to parse: http://(ip):(port) | FreeProxy
-> UnPack.py: headers add
-> PremProxyParser.py: headers add

Thanks for the PR @alsrua7222  !
Tested locally and fixes both PremProxyParser by adding headers on the requests as #74 and FreeProxyParser by updating the attributes.
Can you please fix test_FreeProxyParser so we can merge this? I believe a simple update on free_proxy_mock is needed.
@urlmatch(netloc=r'(.*\.)?free-proxy-list\.net$') def free_proxy_mock(url, request):     return """<table border="0" cellpadding="0" cellspacing="0" class="table table-striped table-bordered"

I didn't know if it was updated or not.
It's already been updated.
I wasted my time.
However, libraries help a lot with proxies. thank you.
After an error, when I traced back, the html parsing structure was changed.
Also, I wanted to point out that "headers" should be added to the library.
As you said, we simply updated the html table structure of free_proxy_list.net.
Is it correct to do this?


search() method isn't working: module called by Tweepy being passed a generator object where string was expected

Preface: I can't tell where this issue lies: my connection to and account with the API, my code, or an issue with Tweepy.
I'm trying to execute a search and receiving an error several layers deep. Here's a pastebin including the code I'm using and the full trace of the error:
http://pastebin.com/9he4Ddc0

More information:
I'm running tweepy in a dedicated virtualenv, and have selected 3.4 as the python version for the environment.
I've tried other methods in the API as well:

verify_credentials()
get_user()
followers_ids()
all throw the same error.

Interestingly, when I try to run trends(), I get an error saying that the method doesn't exist, despite it existing in the documentation.
Could there be something else going on?

One or more of your credentials that you're importing are generator objects instead of strings.
API.trends was deprecated and removed with https://github.com/tweepy/tweepy/commit/e323190a4f2642b3b91b05161e1700e72e0731f7 in Tweepy 1.12, and the documentation was removed with https://github.com/tweepy/tweepy/commit/cbcb62f251ff64626d940cb3d29fc5ea76494f5a in Tweepy v3.2.0.


[BUG] Services are excluded in case of NBR_RETRY exceed only in case of collision.

Which version the bug has been detected
Luos engine 2.5.2
Describe the bug
Luos_engine should exclude services in case of communication trouble. But in fact it seems to not exclude services that doesn't reply.
This is coming from this line : https://github.com/Luos-io/luos_engine/blob/main/network/robus/src/transmission.c#L131
How to reproduce the bug
Use a Gate and an other node, send data to the node from the gate then unplug the node. Gate should send you an alert due to services exclusion, but it's not...

Removing the if (ctx.tx.collision) test line 131 solves the issue.
I don't know about potential side effects though

I don't see any!
The system will just be less permissive...

We will remove this "if" statement.
In a future release, we should only excluded a node if no ACK and/or NACK has been received.
If there are collisions, the other node is not crashed in all cases: maybe it can answer later or the network was oveloaded.

This last comment is described here #392
Current issue 327 will be closed when 2.7.0 will be released

Done in Luos engine 2.7.0


Reset the mock spawn

I have tests that need to check a function spawning the right commands.
However, because multiple tests calling that function, it's hard to rely on mockSpawn.calls to know which call was the one used in one particular test.
Is there a way that I can reset the mock spawn in my setUp or tearDown?

For anyone else that needs this to work, here's a solution in jest:
...

jest.mock('child_process', () => ({
  ...jest.requireActual('child_process'),
  spawn: jest.fn()
}))
let spawnMock

...

describe('myTest', () => {
  beforeEach(() => {
    spawnMock = mockSpawn()
    spawn.mockImplementation(spawnMock)
  })

  ...
})

and then you just access spawnMock in each test when you need to set behaviour, check calls, etc.


Turning ESPNet into c++ cudastream problem

I want to turn your python code into C++/Cuda Code.
So I want to convert your model into pt / ONNX model.
I found this conversion problem:
/home/wjl/project/ESPNetv2/segmentation/cnn/Model.py:139: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if w2 == w1:
/home/wjl/project/ESPNetv2/segmentation/cnn/Model.py:91: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if expanded.size() == input.size():
It seems "if ==" not support in cudastream, how can I avoid it?

We opted to create residual connections on-the-fly and it seems like this issue is related to the if condition responsible for the same. You can comment out the if condition (but don't comment the residual connection part) and I think it should work fine.

Released onnx models here:
https://github.com/sacmehta/EdgeNets
So closing this issue


Update gems

Last pull request left minuteman in broken state. This fixes all dependencies, and sets them up to date. 1 test fails for me locally, but it was failing for quite a while, checked with a bunch of old commits.
@elcuervo it would be nice if you could release a new gem. :)

@elcuervo

Deprecated due to the future branch merge


Whitelist DAOs

Similar to github org whitelist
For DAO voters group:

Query top 100 DAOs ranked by membership count
Query all voters of the DAO, write to a db along with timestamp of the most recent vote
Create API to verify if a eth wallet has voted to any DAOs in the last 12 months

we will only perform step 1-2 periodiacally (once a month, manually)

Closed by #13 and #18

re opening
changing the API to be something like
GET /user/:username/belongs-to-voters-group


Feature/pt bdi support checkbox fields

Bring more responsible support for mapping to checkbox fields to BDI/BGE. Mapped field on
Data Import must be picklist with None/True/False or text field for full support, but current user processes that use checkbox-to-checkbox mappings will be unaffected by this change.
Critical Changes
Changes
Issues Closed
New Metadata
Deleted Metadata

Lurch: W-031084


Stp 831 refactor model in typescript

Description
Ticket STP-831 - Refactor existing Javascript code into Typescript

Break down model/src/values into separate classes, helpers and types.
Typescriptify.
Unit test everything.
Improve Error messages in AbstractCondition class.

Type of change

[X] New feature (non-breaking change which adds functionality)

How Has This Been Tested?

[X] I have added tests that prove my changes are effective.
[X] New and existing unit tests pass locally with my changes

Checklist:

[X] My changes do not introduce any new linting errors
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[N/A] I have made corresponding changes to the documentation and versioning
[X] My changes generate no new warnings
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[X] I have rebased onto master and there are no code conflicts

Canceling this PR because I have merged it into another PR I am about to submit.


MediaType Example doesn't serialize Json output as expected

I'm trying to write some default responses to my API, and I have a problem to show my json as MediaType Example on Swagger-UI.
In the codes bellow, the object MensagemRetorno is just a POJO, without any customization; and the ObjectMapper has customization of UpperCamelCase strategy as well.
I've tried two alternatives to configure the example:
1.  Writing the MediaType Example as plain string:
Schema<MensagemRetorno> mensagemRetornoSchema = new Schema<>();
mensagemRetornoSchema.setName("MensagemRetorno");
mensagemRetornoSchema.set$ref("#/components/schemas/MensagemRetorno");

Content content = new Content()
  .addMediaType(MediaType.APPLICATION_JSON_VALUE,
      new io.swagger.v3.oas.models.media.MediaType()
        .schema(mensagemRetornoSchema)
        .example(objectMapper.writeValueAsString(mensagemRetorno)));

ApiResponse response = new ApiResponse()
	.description("some description")
	.content(content));

In this case, the problem is that plain string is serialized as a json string escaped:

I've tried a lot of customizations on object mapper without any success.
2.  Writing the MediaType Example as POJO:
Content content = new Content()
  .addMediaType(MediaType.APPLICATION_JSON_VALUE,
      new io.swagger.v3.oas.models.media.MediaType()
        .schema(mensagemRetornoSchema)
        .example(mensagemRetorno));

In this case, the problem is that the serialized json doesn't respect the object mapper strategy UpperCamelCase:

The curious is, if I don't provide any MediaType Example, the json output example (default, auto generated), is serialized as expected:
Content content = new Content()
  .addMediaType(MediaType.APPLICATION_JSON_VALUE,
      new io.swagger.v3.oas.models.media.MediaType()
        .schema(mensagemRetornoSchema));
//     .example(mensagemRetorno));

The biggest issue in this last case is that the media type example itself isn't generated on /v3/api-docs page:

Any idea how to customize and provide a MediaType Example with a custom UpperCamelCase strategy?
I am using:
springdoc-openapi-ui:1.5.0
io.swagger.core.v3:swagger-* 2.1.5 (transitive)
org.webjars:swagger-ui 3.36.1 (transitive)

This is related to swagger-model POJO serialization, as swagger-core uses it's own customized ObjectMapper (configured here) for the whole specification, therefore including any "free form" example.
To define a naming strategy limited e.g. to the example contents (as we cannot apply e.g. an UpperCase strategy to the whole doc, as it would break the OpenAPI specifications for OpenAPI properties - e.g. schema and not 'Schema), you would need to use e.g a @JsonNaming(PropertyNamingStrategy.UpperCamelCaseStrategy.class) annotation at MensagemRetorno class level, or provide a custom naming strategy implementing a specific strategy e.g. based on package name, or use MixIns, JsonSerializer or other Jackson provided approaches for custom property naming/serialization.
If this affects multiple POJOs in your code base, possibly going for the custom naming strategy would be the best approach.
Please close ticket if this answers your question

Unfortunately I've already done a lot of tries to customize this output, including POJO personalization with @JsonNaming and Mixins (with my custom ObjectMapper.
Sounds impossible to writes a UpperCamelCase in Schema Examples, because no matter what I've set in io.swagger.v3.oas.models.media.MediaType().example(object)) my customizations doesn't reflect on output.
Do you know if this MediaType Example is wrote by the internal ObjectMapper, I couldn't catch this point.

The following seems to work ok with your example and @JsonNaming, but would also work with Mixins or strategy, the mapper to use is the Swagger provided one, or a similarly customized one.
...
...
        Schema<MensagemRetorno> mensagemRetornoSchema = new Schema<>();
        MensagemRetorno msg = new MensagemRetorno();
        msg.categoria = "ALERTA";
        msg.codigo = 400;
        msg.descricao = "Bad request";
        msg.detalhes = new String[]{"Descrição do(s) erro(s) de sistema, validacoes on mensagens informaivas relacionadas a 'Bad Request'"};
        mensagemRetornoSchema.setName("MensagemRetorno");
        mensagemRetornoSchema.set$ref("#/components/schemas/MensagemRetorno");

        Content content = new Content()
                .addMediaType(javax.ws.rs.core.MediaType.APPLICATION_JSON,
                        new io.swagger.v3.oas.models.media.MediaType()
                                .schema(mensagemRetornoSchema)
                                .example(msg));

        ApiResponse response = new ApiResponse()
                .description("some description")
                .content(content);
        ObjectMapper mapper = Yaml.mapper().copy();
        // you can use a custom naming strategy or mixins insted, e.g. mapper.setPropertyNamingStrategy(new UpperCaseNaming());
        System.err.println(mapper.writer(new DefaultPrettyPrinter()).writeValueAsString(response));

...
...
    @JsonNaming(PropertyNamingStrategy.UpperCamelCaseStrategy.class)
    static class MensagemRetorno {
        public String categoria;
        public Integer codigo;
        public String descricao;
        public String[] detalhes = new String[]{};
    }

@frantuma thanks for yout reply!
So, yes, in that case works in parts. I had similar configuration, but setting example as plain string, I have the issue said on: 1. Writing the MediaType Example as plain string:
And setting as object, the issue was said in 2. Writing the MediaType Example as POJO:.
In your test case, you've  tested the custom ObjectMapper as well, but I wonder that isn't the real test, since that the example isn't written by the custom ObjectMapper at all. As you seems to advice, sounds by written by internals and I am not sure, if that behavior is correct.

@frantuma thanks for yout reply!
So, yes, in that case works in parts. I had similar configuration, but setting example as plain string, I have the issue said on: 1. Writing the MediaType Example as plain string:
And setting as object, the issue was said in 2. Writing the MediaType Example as POJO:.
In your test case, you've tested the custom ObjectMapper as well, but I wonder that isn't the real test, since that the example isn't written by the custom ObjectMapper at all. As you seems to advice, sounds by written by internals and I am not sure, if that behavior is correct.

I am sorry, I just saw, the DefaultPrettyPrinter right now, I will run some tests with this detail and I came back here with results

@frantuma In fact, the configuration said in mapper.writer(new DefaultPrettyPrinter()).writeValueAsString(response) there is no new effect in output serialization in swagger output.
We can observe that the problem isn't related to custom object mapper at all, it's related about the impossibility the customize the output of mediaType example, based on custom POJO or custom String.
In short:

When io.swagger.v3.oas.models.media.MediaType().example(STRING) is set,  the plain string (as json) is serialized in escaped form;
When io.swagger.v3.oas.models.media.MediaType().example(POJO) is set, no matter how customizations was done by custom object mapper, mixins or jackson's annotations;

#Question this behavior is correct?
I meant, if I want a customization of examples (eg. which relies on naming strategy), shouldn't I be able to customize them as I wish?


Adding chatbot to multiple rooms?

Context:

We're building a financial data chatbot described by this grant: https://urbit.org/grants/cypherscope-bot
You can add it to your chat and query it for financial charts, like BTCUSD prices for the last week.
We expect this to be added to multiple chats and expect it to run on a single Moon
The bot is currently stateless BUT we expect to have a paid tier so we may need to store a mapping of chats to payment tier.

Issue:
Our bot's fn main() consists of the constructor:
Chatbot::new_with_local_config(respond_to_message, "~mitten-dapper", "chat-3973").run();
I guess I'm looking for the best way to implement this stuff:

Listen for when we're added to a new chat (and spawn a new chatbot)
Persist which chats contain our chatbot as a member, so nothing is lost if the Moon shuts down

I couldn't find anything resembling this in the examples.
Thank you!

Ah, great to see interest in using this framework for an interesting grant.
For your requirements I'd recommend subscribing to graph-store or hark-store for tracking when you're added to a new chat, and having a local data store (file or database) which you save the chat info to locally. Then on reboot, create a new chatbot struct for each of the locally saved chats.

Thanks!. Will get back to you shortly if I have more questions.


Use Generator pointers as methods receivers

Signed-off-by: Haiyan Meng<EMAIL_ADDRESS>

@mrunalp , @wking , PTAL.

$ git grep 'func.*Generator)' origin/pr/143
says you got them all, so this looks good to me :).

LGTM


integration test for odo list command with a devfile

Which area/kind this issue is related to?

/area integration-tests
Issue Description
Add integration test for odo list command with a devfile.

Resolved by https://github.com/devfile/integration-tests/pull/51


[BUG] Console appears not to be a terminal in background thread

I've set up a widget with a spinner as renderable. I instantiate using Live(spinner_widget) and then call start() on the Live instance. However, the renderable doesn't update, even though the thread is started and calls refresh() on the Live instance at intervals. It can be updated through e.g. an action, as per the code sample given below.
It appears that the proximate reason is that in this line
https://github.com/Textualize/rich/blob/5f4e93efb159af99ed51f1fbfd8b793bb36448d9/rich/live.py#L238
self.console.is_terminal is returning False. That seems to be because in this line
https://github.com/Textualize/rich/blob/5f4e93efb159af99ed51f1fbfd8b793bb36448d9/rich/console.py#L954
isatty is None when called in the live refresh thread.
In an attempt to understand what's happening, I inserted some logging statements. Just after the start() call, I see this in the log:
__main__       22 MainThread <_RefreshThread(Thread-3, started daemon<PHONE_NUMBER>93664)>
rich.console  956 MainThread isatty: <bound method _WriterThread.isatty of <_WriterThread(Thread-1, started daemon<PHONE_NUMBER>18976)>>: True
rich.console  956 MainThread isatty: <bound method _WriterThread.isatty of <_WriterThread(Thread-1, started daemon<PHONE_NUMBER>18976)>>: True
rich.console  956 MainThread isatty: <bound method _WriterThread.isatty of <_WriterThread(Thread-1, started daemon<PHONE_NUMBER>18976)>>: True
... (more duplicate lines elided)
rich.console  956 MainThread isatty: <bound method _WriterThread.isatty of <_WriterThread(Thread-1, started daemon<PHONE_NUMBER>18976)>>: True
rich.console  956 Thread-3   isatty: None: False
rich.console  956 Thread-3   isatty: None: False
rich.live     223 Thread-3   Console: <console width=220 None> is terminal? False

Since in the thread the console appears to be what you'd expect, it's not  immediately clear why isatty would be None. This might be a misunderstanding on my part rather than an actual bug, but I think I followed the documentation to set things up and so expected it to work.
Example Code
The code I used:
import logging
import time

from rich.live import Live
from rich.spinner import Spinner
from textual.app import App, ComposeResult
from textual.widgets import Static, Footer

logger = logging.getLogger(__name__)

class SpinnerStatic(Static):
    def __init__(self, spinner, **kwargs):
        super().__init__(**kwargs)
        self.spinner = spinner
        self.live = Live(spinner)

    def on_mount(self):
        self.update(self.spinner)
        self.live.start()
        time.sleep(0.1)
        logger.debug(self.live._refresh_thread)

class TestApp(App):
    BINDINGS = [("r", "refresh", "Refresh")]

    def compose(self) -> ComposeResult:
        spinner = Spinner('moon', speed=1.0)
        w = SpinnerStatic(spinner, id='spinner')
        self.spinner = w
        yield w
        yield Footer()

    def action_refresh(self):
        self.spinner.refresh()

logging.basicConfig(level=logging.DEBUG,
                    filename='spinner.log',
                    filemode='w',
                    format='%(name)-12s %(lineno)4d %(threadName)-10s %(message)s')
app = TestApp()
app.run()

Platform

Click to expand
Linux (Ubuntu 64-bit), standard terminal
I may ask you to copy and paste the output of the following commands. It may save some time if you do it now.
If you're using Rich in a terminal:
$ python -m rich.diagnose
╭───────────────────────── <class 'rich.console.Console'> ─────────────────────────╮
│ A high level console interface.                                                  │
│                                                                                │
│ ╭──────────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=220 ColorSystem.TRUECOLOR>                                    │ │
│ ╰──────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                │
│     color_system = 'truecolor'                                                   │
│         encoding = 'utf-8'                                                       │
│             file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'> │
│           height = 24                                                            │
│    is_alt_screen = False                                                         │
│ is_dumb_terminal = False                                                         │
│   is_interactive = True                                                          │
│       is_jupyter = False                                                         │
│      is_terminal = True                                                          │
│   legacy_windows = False                                                         │
│         no_color = False                                                         │
│          options = ConsoleOptions(                                               │
│                        size=ConsoleDimensions(width=220, height=24),             │
│                        legacy_windows=False,                                     │
│                        min_width=1,                                              │
│                        max_width=220,                                            │
│                        is_terminal=True,                                         │
│                        encoding='utf-8',                                         │
│                        max_height=24,                                            │
│                        justify=None,                                             │
│                        overflow=None,                                            │
│                        no_wrap=False,                                            │
│                        highlight=None,                                           │
│                        markup=None,                                              │
│                        height=None                                               │
│                    )                                                             │
│            quiet = False                                                         │
│           record = False                                                         │
│         safe_box = True                                                          │
│             size = ConsoleDimensions(width=220, height=24)                       │
│        soft_wrap = False                                                         │
│           stderr = False                                                         │
│            style = None                                                          │
│         tab_size = 8                                                             │
│            width = 220                                                           │
╰──────────────────────────────────────────────────────────────────────────────────╯
╭─── <class 'rich._windows.WindowsConsoleFeatures'> ────╮
│ Windows features available.                           │
│                                                       │
│ ╭───────────────────────────────────────────────────╮ │
│ │ WindowsConsoleFeatures(vt=False, truecolor=False) │ │
│ ╰───────────────────────────────────────────────────╯ │
│                                                       │
│ truecolor = False                                     │
│        vt = False                                     │
╰───────────────────────────────────────────────────────╯
╭────── Environment Variables ───────╮
│ {                                  │
│     'TERM': 'xterm-256color',      │
│     'COLORTERM': 'truecolor',      │
│     'CLICOLOR': None,              │
│     'NO_COLOR': None,              │
│     'TERM_PROGRAM': None,          │
│     'COLUMNS': None,               │
│     'LINES': None,                 │
│     'JUPYTER_COLUMNS': None,       │
│     'JUPYTER_LINES': None,         │
│     'JPY_PARENT_PID': None,        │
│     'VSCODE_VERBOSE_LOGGING': None │
│ }                                  │
╰────────────────────────────────────╯
platform="Linux"
$ pip freeze | grep rich
rich==12.6.0

Rich Live and Textual, are mutually incompatible.
Textual does not currently have a Spinner widget, but it will soon.

:disappointed: Sorry to hear that. But does this part of the Textual documentation

You can use any renderable defined in Rich or third party libraries.

hold true for everything else in Rich apart from Live? And regarding the mutual incompatibility - I'm just curious - is it to do with async vs. threading approaches?

Live is not a renderable. It’s displays renderables in a thread, which is not aware of Textual.

I noticed that even if I set up my own thread to refresh the spinner, it doesn't work as expected. Is there some reason why a widget.refresh() call will work in the main thread, but not in background threads? Modified example:
import logging
import threading
import time

from rich.spinner import Spinner
from textual.app import App, ComposeResult
from textual.widgets import Static, Footer

logger = logging.getLogger(__name__)

class SpinnerStatic(Static):
    def __init__(self, spinner, **kwargs):
        self.spinner = spinner
        super().__init__(**kwargs)

    def on_mount(self):
        self.update(self.spinner)

    def refresh(self, *args, **kwargs):
        logger.debug('Refreshing spinner')
        super().refresh(*args, **kwargs)

class TestApp(App):
    BINDINGS = [("r", "refresh", "Refresh")]

    def compose(self) -> ComposeResult:
        spinner = Spinner('moon', speed=1.0)
        w = SpinnerStatic(spinner, id='spinner')
        self.spinner = w
        yield w
        yield Footer()

    def update_on_thread(self):
        logger.debug('Thread starting')
        for i in range(10):
            time.sleep(0.25)
            self.spinner.refresh()
        logger.debug('Thread is done')

    def on_mount(self):
        t = threading.Thread(target=self.update_on_thread, daemon=True)
        t.start()

    def action_refresh(self):
        logger.debug('Refresh action called')
        self.spinner.refresh()

logging.basicConfig(level=logging.DEBUG,
                    filename='spinner.log',
                    filemode='w',
                    format='%(name)-12s %(lineno)4d %(threadName)-10s %(message)s')
app = TestApp()
app.run()

When I run this program, pressing the R key cycles the spinner, but the code in the background thread doesn't. Log shows:
asyncio        54 MainThread Using selector: EpollSelector
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       36 Thread-3 (update_on_thread) Thread starting
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 MainThread Refreshing spinner
__main__       22 Thread-3 (update_on_thread) Refreshing spinner
__main__       22 Thread-3 (update_on_thread) Refreshing spinner
__main__       22 Thread-3 (update_on_thread) Refreshing spinner
__main__       22 Thread-3 (update_on_thread) Refreshing spinner
__main__       22 Thread-3 (update_on_thread) Refreshing spinner
__main__       22 Thread-3 (update_on_thread) Refreshing spinner
__main__       22 Thread-3 (update_on_thread) Refreshing spinner
__main__       47 MainThread Refresh action called
__main__       22 MainThread Refreshing spinner
__main__       22 Thread-3 (update_on_thread) Refreshing spinner
__main__       22 Thread-3 (update_on_thread) Refreshing spinner
__main__       22 Thread-3 (update_on_thread) Refreshing spinner
__main__       40 Thread-3 (update_on_thread) Thread is done
__main__       47 MainThread Refresh action called
__main__       22 MainThread Refreshing spinner

Is there a fundamental problem with using threads like this, or have I just made some mistake with the way I've set things up?

@vsajip I can't answer your last explicit question (about threads) but I can share a piece of code that runs your Spinner, possibly good enough for you:
https://user-images.githubusercontent.com/5621605/203750456-85c3f530-0d87-40f3-bbbc-cd6177a7f547.mp4
The code (shown below) is a trivial modification of something similar I did for a progress bar.
from rich.spinner import Spinner

from textual.app import App, ComposeResult
from textual.widgets import Static

class SpinnerWidget(Static):
    def __init__(self):
        super().__init__("")
        self._spinner = Spinner("moon")

    def on_mount(self) -> None:
        self.update_render = self.set_interval(1 / 60, self.update_spinner)

    def update_spinner(self) -> None:
        self.update(self._spinner)

class MyApp(App[None]):
    def compose(self) -> ComposeResult:
        yield SpinnerWidget()

MyApp().run()

Oh, very nice! Thank you very much. As I'm just learning about this framework, any information like this is very useful for me.

Textual uses asyncio which is generally single threaded. So your thread approach is unlikely to work. Fortunately it is generally easier to implement such things with timers, as Rodrigo's examples shows!

@willmcgugan Any obvious place in the documentation where something like my two examples should go or can I just write a blog post with a couple of examples?

I think we will have builtin widgets for spinner / progress eventually. So probably not worth adding those to docs right now.
However, it would make an excellent blog post!

So your thread approach is unlikely to work.

Can I suggest that, if threads should not be used with this framework, that should be documented somewhere (or, if they can be used, any caveats/constraints surrounding their use)?


[Issue] Login seems successful but isn't

[X] Make sure you've installed the latest version using instructions in the wiki

Output from azd version
azd version 1.3.1 (commit b5030da0f29ffe98664c40450187fd8a4cb0d157)
Describe the bug
I'm trying to login with --tenant-id and it kept claiming to be successful, but then I added --debug and realized its failing. I think it should convey the failure in non-debug mode too.
vscode ➜ /workspaces/azure-search-openai-demo (appregbicep) $ azd auth login --tenant-id 2f32b846-1945-4cf2-a2ca-087cf844c482 --debug
2023/10/03 18:05:02 main.go:54: azd version: 1.3.1 (commit b5030da0f29ffe98664c40450187fd8a4cb0d157)
2023/10/03 18:05:02 main.go:208: using cached latest version: 1.3.1 (expires on: 2023-10-03T18:18:50Z)
2023/10/03 18:05:02 cobra_builder.go:142: Resolved action 'azd-auth-login-action'
2023/10/03 18:05:02 middleware.go:124: running middleware 'debug'
2023/10/03 18:05:02 middleware.go:124: running middleware 'experimentation'
2023/10/03 18:05:02 experimentation.go:42: assignment context: azdaacf:75644;initflow_t:78271;9936g960:76971;
2023/10/03 18:05:02 middleware.go:124: running middleware 'telemetry'
2023/10/03 18:05:02 telemetry.go:46: TraceID: d520e01b4900808f7db1564d07dd61fb
2023/10/03 18:05:03 command_runner.go:307: Run exec: '/vscode/vscode-server/bin/linux-arm64/6c3e3dba23e8fadc360aed75ce363ba185c49794/bin/helpers/browser.sh https://login.microsoftonline.com/2f32b846-1945-4cf2-a2ca-087cf844c482/oauth2/v2.0/authorize?client_id=04b07795-8ddb-461a-bbee-02f9e1bf7b46&code_challenge=vnGaTUJAkuBbPIAo19A7Kp3ihqR5XTTqPbbDMg9_Y6U&code_challenge_method=S256&prompt=select_account&redirect_uri=http%3A%2F%2Flocalhost%3A35275&response_type=code&scope=https%3A%2F%2Fmanagement.azure.com%2F%2F.default+openid+offline_access+profile&state=<redacted>' , exit code: 0
  |       | Retrieving subscriptions...2023/10/03 18:05:08 main.go:48: Retry: =====> Try=1
2023/10/03 18:05:08 main.go:48: Request: ==> OUTGOING REQUEST (Try=1)
   GET https://management.azure.com/tenants?api-version=2021-01-01
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armsubscriptions/v1.0.0 (go1.21.0 X:loopvar; linux),azdev/1.3.1 (Go go1.21.0 X:loopvar; linux/arm64)

  |==     | Retrieving subscriptions...2023/10/03 18:05:08 main.go:48: Response: ==> REQUEST/RESPONSE (Try=1/332.851875ms, OpTime=332.874667ms) -- RESPONSE RECEIVED
   GET https://management.azure.com/tenants?api-version=2021-01-01
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armsubscriptions/v1.0.0 (go1.21.0 X:loopvar; linux),azdev/1.3.1 (Go go1.21.0 X:loopvar; linux/arm64)
   --------------------------------------------------------------------------------
   RESPONSE Status: 200 OK
   Cache-Control: no-cache
   Content-Length: 7914
   Content-Type: application/json; charset=utf-8
   Date: Tue, 03 Oct 2023 18:05:08 GMT
   Expires: -1
   Pragma: no-cache
   Strict-Transport-Security: REDACTED
   X-Cache: REDACTED
   X-Content-Type-Options: REDACTED
   X-Ms-Correlation-Request-Id: REDACTED
   X-Ms-Ratelimit-Remaining-Tenant-Reads: REDACTED
   X-Ms-Request-Id: 762ea000-f985-40ab-8627-755d6036a3de
   X-Ms-Routing-Request-Id: REDACTED
   X-Msedge-Ref: REDACTED

2023/10/03 18:05:08 main.go:48: Retry: response 200
2023/10/03 18:05:08 main.go:48: Retry: exit due to non-retriable status code
2023/10/03 18:05:08 main.go:48: Retry: =====> Try=1
2023/10/03 18:05:08 main.go:48: Request: ==> OUTGOING REQUEST (Try=1)
   GET https://management.azure.com/subscriptions?api-version=2021-01-01
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armsubscriptions/v1.0.0 (go1.21.0 X:loopvar; linux),azdev/1.3.1 (Go go1.21.0 X:loopvar; linux/arm64)

2023/10/03 18:05:08 main.go:48: Response: ==> REQUEST/RESPONSE (Try=1/70.648125ms, OpTime=70.707041ms) -- RESPONSE RECEIVED
   GET https://management.azure.com/subscriptions?api-version=2021-01-01
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armsubscriptions/v1.0.0 (go1.21.0 X:loopvar; linux),azdev/1.3.1 (Go go1.21.0 X:loopvar; linux/arm64)
   --------------------------------------------------------------------------------
   RESPONSE Status: 200 OK
   Cache-Control: no-cache
   Content-Length: 47
   Content-Type: application/json; charset=utf-8
   Date: Tue, 03 Oct 2023 18:05:08 GMT
   Expires: -1
   Pragma: no-cache
   Strict-Transport-Security: REDACTED
   X-Cache: REDACTED
   X-Content-Type-Options: REDACTED
   X-Ms-Correlation-Request-Id: REDACTED
   X-Ms-Ratelimit-Remaining-Tenant-Reads: REDACTED
   X-Ms-Request-Id: 5bf3ed66-b702-44e8-ac0d-e309943ad5db
   X-Ms-Routing-Request-Id: REDACTED
   X-Msedge-Ref: REDACTED

2023/10/03 18:05:08 main.go:48: Retry: response 200
2023/10/03 18:05:08 main.go:48: Retry: exit due to non-retriable status code
2023/10/03 18:05:08 main.go:48: Retry: =====> Try=1
2023/10/03 18:05:08 main.go:48: Request: ==> OUTGOING REQUEST (Try=1)
   GET https://management.azure.com/subscriptions?api-version=2021-01-01
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armsubscriptions/v1.0.0 (go1.21.0 X:loopvar; linux),azdev/1.3.1 (Go go1.21.0 X:loopvar; linux/arm64)

  |===    | Retrieving subscriptions...2023/10/03 18:05:09 main.go:48: Response: ==> REQUEST/RESPONSE (Try=1/78.934459ms, OpTime=78.990876ms) -- RESPONSE RECEIVED
   GET https://management.azure.com/subscriptions?api-version=2021-01-01
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armsubscriptions/v1.0.0 (go1.21.0 X:loopvar; linux),azdev/1.3.1 (Go go1.21.0 X:loopvar; linux/arm64)
   --------------------------------------------------------------------------------
   RESPONSE Status: 200 OK
   Cache-Control: no-cache
   Content-Length: 47
   Content-Type: application/json; charset=utf-8
   Date: Tue, 03 Oct 2023 18:05:08 GMT
   Expires: -1
   Pragma: no-cache
   Strict-Transport-Security: REDACTED
   X-Cache: REDACTED
   X-Content-Type-Options: REDACTED
   X-Ms-Correlation-Request-Id: REDACTED
   X-Ms-Ratelimit-Remaining-Tenant-Reads: REDACTED
   X-Ms-Request-Id: 2a8cf7eb-f39f-473d-80d5-9672e91c6793
   X-Ms-Routing-Request-Id: REDACTED
   X-Msedge-Ref: REDACTED

2023/10/03 18:05:09 main.go:48: Retry: response 200
2023/10/03 18:05:09 main.go:48: Retry: exit due to non-retriable status code
2023/10/03 18:05:09 main.go:48: Retry: =====> Try=1
2023/10/03 18:05:09 main.go:48: Request: ==> OUTGOING REQUEST (Try=1)
   GET https://management.azure.com/subscriptions?api-version=2021-01-01
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armsubscriptions/v1.0.0 (go1.21.0 X:loopvar; linux),azdev/1.3.1 (Go go1.21.0 X:loopvar; linux/arm64)

  |====   | Retrieving subscriptions...2023/10/03 18:05:09 main.go:48: Response: ==> REQUEST/RESPONSE (Try=1/190.298833ms, OpTime=190.328625ms) -- RESPONSE RECEIVED
   GET https://management.azure.com/subscriptions?api-version=2021-01-01
   Accept: application/json
   Authorization: REDACTED
   User-Agent: azsdk-go-armsubscriptions/v1.0.0 (go1.21.0 X:loopvar; linux),azdev/1.3.1 (Go go1.21.0 X:loopvar; linux/arm64)
   --------------------------------------------------------------------------------
   RESPONSE Status: 200 OK
   Cache-Control: no-cache
   Content-Length: 42874
   Content-Type: application/json; charset=utf-8
   Date: Tue, 03 Oct 2023 18:05:09 GMT
   Expires: -1
   Pragma: no-cache
   Strict-Transport-Security: REDACTED
   X-Cache: REDACTED
   X-Content-Type-Options: REDACTED
   X-Ms-Correlation-Request-Id: REDACTED
   X-Ms-Ratelimit-Remaining-Tenant-Reads: REDACTED
   X-Ms-Request-Id: 2454327f-d453-4412-9e54-32e6960b8d5d
   X-Ms-Routing-Request-Id: REDACTED
   X-Msedge-Ref: REDACTED

2023/10/03 18:05:09 main.go:48: Retry: response 200
2023/10/03 18:05:09 main.go:48: Retry: exit due to non-retriable status code
  | ======| Retrieving subscriptions...2023/10/03 18:05:10 azd_credential.go:34: POST https://login.microsoftonline.com/12c2c747-3cbe-4a7d-9af6-a151055e24d4/oauth2/v2.0/token
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
--------------------------------------------------------------------------------
{
  "error": "invalid_grant",
  "error_description": "AADSTS50076: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access '797f4846-ba00-4fd7-ba43-dac1f8f63013'.\r\nTrace ID: 112a0ee8-2a41-48cf-860e-eda7a5e43100\r\nCorrelation ID: 025ecc64-43e4-4e18-897d-14bb1ae5fa0c\r\nTimestamp: 2023-10-03 18:05:10Z",
  "error_codes": [
    50076
  ],
  "timestamp": "2023-10-03 18:05:10Z",
  "trace_id": "112a0ee8-2a41-48cf-860e-eda7a5e43100",
  "correlation_id": "025ecc64-43e4-4e18-897d-14bb1ae5fa0c",
  "error_uri": "https://login.microsoftonline.com/error?code=50076",
  "suberror": "basic_action"
}
--------------------------------------------------------------------------------

2023/10/03 18:05:10 subscriptions_manager.go:275: failed to load subscriptions from tenant 'PythonB2C' : reauthentication required, run `azd auth login` to log in
Logged in to Azure.

Environment
Information on your environment:
* Language name and version Python 3.10
* IDE and version : VS Code, Dev Container
Version: 1.81.1 (Universal)
Commit: 6c3e3dba23e8fadc360aed75ce363ba185c49794
Date: 2023-08-09T22:20:33.924Z
Electron: 22.3.18
ElectronBuildId: 22689846
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 22.6.0

Also I could use some tips on getting my login with tenant-id to work. Also fails in terminal.

@pamelafox Will work with you on this one. We saw a similar issue before that we suspect that login prompt is not triggering the MFA prompt.

it kept claiming to be successful, but then I added --debug and realized its failing
What's happening is that the login is succeeding. But when we tried to load subscriptions for the tenant, the user wasn't found to be authorized and hence why the failure message shows up in --debug. When you run any azd command that loads the environment it should fail with the same error message. It's not meant to be an unhandled failure.

Ah okay I eventually realized after doing az login with that --tenant that the lack of subscriptions was the issue, so am working on that. So I think the remaining issue is that there should be an error message.

Can i justs use "az" to set auth context ..and then let azd reuse it ..?

@cforce Are you running into an issue? By default, azd loads all subscription contexts that you have. When you provision an azd environment, you select the subscription the environment should be tied to. The environment is then stamped with the subscription ID for reuse. There shouldn't be need for switching contexts -- it's all tied to the azd environment.

I ran into the same issue with error:  AADSTS50076. Somehow my Microsoft account already required to use 2FA to login, but it was not enabled on https://account.microsoft.com/ -> Security -> Advanced Security -> Two Factor authentication (names might differ a bit as I do not have my account in English, so I translated to English) I enable it and I authenticate again and my azd up passed.

@pamelafox @Modryz closing this one. Re-open if need be.


[RESOLVED] Interface eth0(): High error rate (>2 for 5m)

Problem started at 15:00:33 on 2023.06.14
Problem name: Interface eth0(): High error rate (>2 for 5m)
Host: S100-Primario
Severity: Warning
Operational data: errors in: 2, errors out: 0
Original problem ID: 28679
Event details in Zabbix: http://<IP_ADDRESS>/tr_events.php?triggerid=23483&eventid=28679

Problem has been resolved in 17m 56s at 15:18:29 on 2023.06.14
Problem name: Interface eth0(): High error rate (>2 for 5m)
Host: S100-Primario
Severity: Warning
Original problem ID: 28679


Get the list of available scripts from the database

In the course of working on #5, a select element (aka a combo box) was added to allow users to choose a "space" from which to load and change shared programs.  The list of spaces is currently hardcoded.  This should be changed such that the list is obtained from the DB.

The list is now obtained from the DB.  Closing.


feat: Update Avatar and Profile Page Loading State

Summary of Changes
Updates Avatar to have a shimmering circle while loading
Updates the header of the Profile page to have a shimmering loader while loading

Tests
Updated tests
CheckList
Make sure you have checked all steps below to ensure a timely review.

[x] PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"

In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

[x] PR includes a summary of changes, including screenshots of any UI changes.
[x] PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
[x] In case of new functionality, my PR adds documentation that describes how to use it.

All the public python functions and the classes in the PR contain docstrings that explain what it does

[x] PR passes all tests documented in the developer guide

Codecov Report

Merging #487 into master will decrease coverage by 4.79%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #487      +/-   ##
==========================================
- Coverage   80.83%   76.04%   -4.80%
==========================================
  Files         223       42     -181
  Lines        5510     1845    -3665
  Branches      640      182     -458
==========================================
- Hits         4454     1403    -3051
+ Misses        992      393     -599
+ Partials       64       49      -15

Impacted Files
Coverage Δ

amundsen_application/api/utils/search_utils.py
95.12% <0.00%> (-0.88%)
:arrow_down:

...pplication/static/js/ducks/notification/reducer.ts

...plication/static/js/ducks/announcements/reducer.ts

...on/static/js/ducks/tableMetadata/owners/reducer.ts

...ents/common/EntityCard/EntityCardSection/index.tsx

...js/components/TableDetail/ColumnListItem/index.tsx

...ic/js/components/TableDetail/TableIssues/index.tsx

...mponents/TableDetail/RequestMetadataForm/index.tsx

amundsen_application/static/js/ducks/log/api/v0.ts

...tic/js/components/TableDetail/SourceLink/index.tsx

... and 170 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e2c1399...e2ea4d6. Read the comment docs.

@Golodhros are you able to clean up the diff to remove changes that are already merged down to master?
I can also look at each relevant commit separately and see how that goes.

@Golodhros are you able to clean up the diff to remove changes that are already merged down to master?
I can also look at each relevant commit separately and see how that goes.

Done! I forgot to set the upstream to master.


Map getters

@MappableClass()
class OrderState with OrderStateMappable {
late Task currentTask;
late PointTask navigatedPointTask;
late Set selectedOrders = {};
bool get isActive => currentTask.pointId.toString() == drivingToPointId;
List get pointTasks => currentTask.pointTasks;
}
I have noticed that getters doesn't get put inside
  @override
  final Map<Symbol, Field<OrderState, dynamic>> fields = const {
    #currentTask: _f$currentTask,
    #navigatedPointTask: _f$navigatedPointTask,
    #selectedOrders: _f$selectedOrders,
  };

but when they do (if I generate parameter with same name and field is populated), then they do work as expected. How can I set globally to map all getters, if it is possible?
P.S.
Just tried adding manually, and now values are shown
  static bool _$isActiveTask(OrderSelectionState v) => v.isActiveTask; //Added
  static const Field<OrderSelectionState, bool> _f$isActiveTask = //Added
      Field('isActiveTask', _$isActiveTask, mode: FieldMode.member); //Added

    @override
  final Map<Symbol, Field<OrderSelectionState, dynamic>> fields = const {
    #currentTask: _f$currentTask,
    #navigatedPointTask: _f$navigatedPointTask,
    #selectedOrders: _f$selectedOrders,
    #isActiveTask: _f$isActiveTask, //Added
  };

But would like to have them genetared

Try annotating your getters with @MappableField

With no success

True this was only supported on fields. I added this as a new feature to release v3.2.0, so annotating getters is now supported.
Please update and check that it works now.


Adding customEndpoints usage to docs for source-wordpress

Updating the docs for source-wordpress to reflect customEndpoints option.

Thanks :)

That's all well and good, but I cannot find any reference to using customEndpoints with Gridsome. I can add that to my config, but there seems to not be any documentation on how to use it to display. It's a shame because such features will just get forgotten if people can't use them


E04 Octomap not publish

Hi, I am working on tutorials 4, and solve the issue of run.sh like mentioned in other issues. However, my Octomap doesn't work since it says noting to publish. I aleady edited the launch file as told in tutorial. Can you help me with this issue?

Still cant solve the issue. Any help please?


Change columns order for upgrade cashier

If you're upgrading from cashier v1 to v2, the columns amount_refunded and amount_charged_back are placed after the timestamps (end) of the table.
With this pull request they are added after total_due same as the initial migration for v2.

Thanks @Mul-tiMedia , good one!


reorder metaanalysis docs with sami's feedback

🚀 Deployed on https://66bea0bfc24898790b94752c--cozy-fox-0defec.netlify.app


Refactoring

Creating class PuzzleGame and adding function as it's member function and puzzleBoard as the data member to store the board information.

Looks Good


Consume ECS Metadata with Falco

Motivation
Falco can run in ECS Fargate now, can we look at improving our integration?

Feature
With the new workaround pdig and running falco in ECS Fargate, can we start exploring consuming ECS and Fargate metadata into Falco in the same way we consume container and Kubernetes metadata?
This is relevant because this can be used as our first input plugin that can be built outside of the core Falco engine and use the new inputs API.

Alternatives

Additional context

Hi, any update on this thread?
Thanks,

Hi folks, any suggestion on how this can be achieved?