Camera underwater effect seems to break when enabling directional light shadows

Describe the bug
The camera underwater effect seems to break
Screenshots / video
Shadow On : https://i.ibb.co/RS2HxmF/cctitre1.png
Shadow Off : https://i.ibb.co/QY7yKzk/cctitre2.png
Log
None
Versions
Open source master branch on Unity 2018.3.6f1
To Reproduce

Unzip project
Open main scene in crest exemples
In Ocean Renderer component of Ocean game object check "Create Shadow Data"
(optional; for better visibility) Open Ocean-Underwater Material and change X, Y and Z values of "Fog Density" to 0.02
In Environment > Directional light gameobject change "Shadow Type" to "Hard Shadow" or "Soft Shadows"
Press Play and go deeper in the ocean, some strange dark artifacts will start to appear

Platform

Editor
Platform Windows 10

Additional context
None

Thanks, i see it. I guess this is issues in the UnderwaterCurtain.shader. I won't have time to look at this in the short term, but if someone else has time to pick this up i'm happy to talk it through. Otherwise i'll post back when i've had a run at it.

This should be fixed on 60cdcdccda1de4217e9c4dc0ec0a88ad910b014f . We knocked out the shadowing factor for underwater. We are looking at rebuilding the underwater effect and will see if we can reinstate shadows.


[gluon] nomenclature for hybrid computation

I find that names like HybridBlock, HybridSequential, and hybridize() in gluon are not effectively conveying what their utilities and usages are. To me, 'HybridBlock' sounds like a mixture of 'Block' and something else, although it is actually an extension of a Block. Also, calling hybridize() sounds like imperative and symbolic program will be mixed in executions, although in reality, calling it makes an one-way transition of the block from an imperative program to a symbolic program. Lastly, 'hybrid' is an ambiguous word which meaning is unclear until a user takes a careful look at the hybrid computation tutorial.
Proposal: Why don't we call these CompilableBlock, CompilableSequential, and compile()? Then, it should be clear CompilableBlocks are extensions of Blocks, and executions of CompilableBlocks will be faster after calling compile() than they were before, because 'compile' is a concept most programmers are familiar with.
Furthermore, from the API design point of view, it might be clearer if compile() creates a new Block rather than changing the state of an exiting HybridBlock. Then it should be clear that CompilableBlock is still an imperative program, whereas CompilableBlock.compile() is a declarative program.
I am curious what others' thoughts on this are. Tagging @szha since I talked with him on this separately.

@piiswrong

This issue is closed due to lack of activity in the last 90 days. Feel free to ping me to reopen if this is still an active issue. Thanks!
Also, do please check out our forum (and Chinese version) for general "how-to" questions.


SCT-501-fix-restoring-view-states

--- JIRA LINKS ---
https://infullmobile.atlassian.net/browse/SCT-501

I love it


Home page card styling for mobile

On mobile the front page is showing the 3 cards side by side. In my opinion the should use full screen width to be more readable.

Thanks for reporting that @danielpomas!  They were literally added minutes before go-live last night.  Feel free to make a PR to correct it.
@allcontributors please add @danielpomas for bugs

@pardel sent me a Twitter DM on this same topic with a screenshot.

@allcontributors please add @pardel for bugs.

@allcontributors, please add @danielpomas for code.

@allcontributors please add @danielpomas for code


Why does Conformer add x twice for FFN?

During the forward pass:
https://github.com/flashlight/flashlight/blob/master/flashlight/fl/contrib/modules/Conformer.cpp#L175-L181
https://github.com/flashlight/flashlight/blob/master/flashlight/fl/contrib/modules/Conformer.cpp#L187-L193
ffn1 = x + ffn1(x)
x = x + 0.5 * ffn1
ffn2 = x + ffn2(x)
x = x + 0.5 * ffn2

This basically weighs x at 1.5 each ffn step instead of 1.0
The other conformer implementations I've found don't do this (they only scale down ffn, they don't double-add x)

https://github.com/sooftware/conformer/blob/main/conformer/encoder.py#L79
https://github.com/lucidrains/conformer/blob/0e91f0323089cc00fbde3e112a984d94d0af09b8/conformer/conformer.py#L190

I'm also confused about the pair of "pointwise conv" steps just being linear layers in flashlight:
https://github.com/flashlight/flashlight/blob/master/flashlight/fl/contrib/modules/Conformer.cpp#L40-L42

Hey!

This basically weighs x at 1.5 each ffn step instead of 1.0

Thanks for pointing! I am not sure what implementation is the right one. For our implementation I directly follows the paper https://arxiv.org/pdf/2005.08100.pdf. Probably we can remove this weight from 1.5 to 1. - if you have comparison between the two versions, let me know.

I'm also confused about the pair of "pointwise conv" steps just being linear layers in flashlight:

Pointwise convolutions is conv with kernel 1x1 which equivalent to apply linear layer on channel dim. So we can either call conv 1x1 or apply linear layer (we do the latest).

For our implementation I directly follows the paper

I don't see the 1.5x weight in the paper, it still seems to me like adding x twice may be a bug. Here's their Conformer Block diagram, which only adds x once at each step for a standard residual connection:

I don't have the resources to prioritize an experiment on this, intuitively it seems to me like it would hurt the network by over-amplifying the residual values. If my math is right, the residual path will end up as something like 431439x larger across 16 conformer blocks (via x * (1.5 ** 32)) for no reason.

Pointwise convolutions is conv with kernel 1x1 which equivalent to apply linear layer on channel dim

I didn't see it at first but I ran this experiment and now I see the equivalence of pointwise conv and linear in the channel dimension. Thanks!
import torch
import torch.nn as nn

with torch.no_grad():
    conv = nn.Conv1d(1, 2, kernel_size=1, bias=False)
    conv.weight = nn.Parameter(torch.tensor([1, 2], dtype=torch.float64).reshape(2, 1, 1))
    fc = nn.Linear(1, 2, bias=False)
    fc.weight = nn.Parameter(torch.tensor([[1], [2]], dtype=torch.float64))
    x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=torch.float64)
    x = x.reshape(3, 1, 3)
    print(x)
    print('conv')
    print(conv(x))
    x = x.reshape(3, 3, 1)
    print('fc')
    print(fc(x).transpose(1, 2))

I don't see the 1.5x weight in the paper, it still seems to me like adding x twice may be a bug. Here's their Conformer Block diagram, which only adds x once at each step for a standard residual connection:

Look at the figure 4 where ffn layer definition is given, so we are doing exactly it in the block https://github.com/flashlight/flashlight/blob/master/flashlight/fl/contrib/modules/Conformer.cpp#L187-L193.

They use that skip connection notation on every block diagram, see also figures 2 and 3.
The extra add ultimately makes no sense to me due the point of residual skip connections, which is to pass the input through unmodified (not to multiply the original value by 400,000). If you applied it for figures 2 and 3 as well you'd be multiplying the original x by 186 billion, which I think could cause precision issues even at fp32. (I think the 400k multiply could cause precision issues at fp16 which only has +-64k of precision).
I will also note that my 28M conformer that outperforms the RASR models was trained at full precision, and I haven't been able to reproduce it at half precision. This could be related?

Here's the bug as code to demonstrate the impact:
import math

def fn(x):
    return math.sin(x) + 1

def model1(x):
    print(f'model1({x})')
    for i in range(16):
        ffn1 = fn(x)
        x    = x + ffn1
        ffn2 = fn(x)
        x    = x + ffn2
        print(f'i={i+1:<2} x={x:.6f} ffn1={ffn1:.6f} ffn2={ffn2:.6f}')
    return x

def model2(x):
    print(f'model2({x})')
    for i in range(16):
        ffn1 = x + fn(x)
        x    = x + 0.5 * ffn1
        ffn2 = x + fn(x)
        x    = x + 0.5 * ffn2
        print(f'i={i+1:<2} x={x:>14.6f} ffn1={ffn1:>14.6f} ffn2={ffn2:>14.6f}')
    return x

model1(1)
model2(1)

Output:
model1(1)
i=1  x=4.137107 ffn1=1.841471 ffn2=1.295636
i=2  x=4.382678 ffn1=0.160961 ffn2=0.084610
i=3  x=4.474347 ffn1=0.053864 ffn2=0.037805
i=4  x=4.524482 ffn1=0.028198 ffn2=0.021936
i=5  x=4.556551 ffn1=0.017603 ffn2=0.014467
i=6  x=4.578979 ffn1=0.012118 ffn2=0.010310
i=7  x=4.595608 ffn1=0.008886 ffn2=0.007743
i=8  x=4.608460 ffn1=0.006811 ffn2=0.006041
i=9  x=4.618706 ffn1=0.005396 ffn2=0.004850
i=10 x=4.627076 ffn1=0.004385 ffn2=0.003984
i=11 x=4.634046 ffn1=0.003637 ffn2=0.003334
i=12 x=4.639945 ffn1=0.003067 ffn2=0.002832
i=13 x=4.645005 ffn1=0.002623 ffn2=0.002436
i=14 x=4.649393 ffn1=0.002269 ffn2=0.002119
i=15 x=4.653238 ffn1=0.001984 ffn2=0.001861
i=16 x=4.656634 ffn1=0.001749 ffn2=0.001647

model2(1)
i=1  x=      4.461118 ffn1=      2.841471 ffn2=      4.080764
i=2  x=     10.766859 ffn1=      4.492521 ffn2=      8.118963
i=3  x=     24.525078 ffn1=     10.792901 ffn2=     16.723536
i=4  x=     55.682264 ffn1=     24.954127 ffn2=     37.360244
i=5  x=    126.426009 ffn1=     55.920259 ffn2=     85.567231
i=6  x=    286.683423 ffn1=    128.116598 ffn2=    192.398230
i=7  x=    645.865051 ffn1=    286.967119 ffn2=    431.396139
i=8  x=   1454.189986 ffn1=    645.900686 ffn2=    970.749182
i=9  x=   3273.942748 ffn1=   1455.549188 ffn2=   2183.956337
i=10 x=   7367.433547 ffn1=   3275.335114 ffn2=   4911.646483
i=11 x=  16577.375035 ffn1=   7368.045258 ffn2=  11051.837717
i=12 x=  37300.417417 ffn1=  16579.098907 ffn2=  24866.985859
i=13 x=  83926.624290 ffn1=  37301.133534 ffn2=  55951.280212
i=14 x=<PHONE_NUMBER>17 ffn1=  83928.478199 ffn2=<PHONE_NUMBER>55
i=15 x=<PHONE_NUMBER>47 ffn1=<PHONE_NUMBER>36 ffn2=<PHONE_NUMBER>23
i=16 x=<PHONE_NUMBER>23 ffn1=<PHONE_NUMBER>50 ffn2=<PHONE_NUMBER>03

Yep, makes sense for me now. I know why I did this extra res connection - their 1/2 in formula (1). Now I think you are right, as for conv I interpreted formula (1) as the res connection from the plot (so no extra res connection here) but not for ffn as they have 1/2 in formula (1) but not on the figure plot. About the fp32 vs fp16 - could be, but I have a doubt, because there is layer norm in the block at the end, so this should normalize all things to the proper scales I guess. Let me discuss with the team how to elaborate this bug so that we can reuse our released conformer model and will send the fix then (or feel free to send PR as you found the bug :) )

All nice catches and results! Let me test this patch for rasr model, as will have models result will merge the patch. Thanks a lot for all work you did @lunixbochs @joazoa !

These are some results from all 3 patches: (baseline versus new)
loss:    0.25212 | 0.21177
train-TER:  0.90 | 0.77
train-WER:  3.84 | 3.18
DS1:    1.28120 | 1.10865
DS1-TER:  3.74 | 3.14
DS1-WER: 13.48 | 11.49
DS2-loss:    0.46202 | 0.47701
DS2-TER:  0.99 | 0.86
DS2-WER:  3.78 | 3.21
DS3-loss:    0.57905 | 0.47689
DS3-TER:  1.58 | 1.36
DS3-WER:  5.36 | 4.51
DS4-loss:    0.70196 | 0.56444
DS4-TER:  1.98 | 1.68
DS4-WER:  6.13 | 5.16
DS5-loss:    2.66550 | 2.30005
DS5-TER:  7.07 | 6.67
DS5-WER: 20.68 | 18.44

merged


README: better first glance idea

Fixes #347
Make the project's first-glance easier to digest.
Signed-off-by: Vincent Batts<EMAIL_ADDRESS>

FWIW, build error is a flake, couldn't talk to Fedora's registry.

LGTM with link fixed.

link fixed. PTAL

would like this merged before kubecon gets fully underway today :-)

@nalind OK to merge?  Build issues are not related.

@TomSweeneyRedHat Yeah, it reads fine to me.  I'm going to have to bypass the merge bot.


Adding requirement that localNetworkGatewayName have at least one cha…

…racter.
This checklist is used to make sure that common issues in a pull request are addressed. This will expedite the process of getting your pull request merged and avoid extra work on your part to fix issues discovered during the review process.
PR information

[x] The title of the PR is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
[x] Except for special cases involving multiple contributors, the PR is started from a fork of the main repository, not a branch.
[x] If applicable, the PR references the bug/issue that it fixes.
[x] Swagger files are correctly named (e.g. the api-version in the path should match the api-version in the spec).

Quality of Swagger

[x] I have read the contribution guidelines.
[x] My spec meets the review criteria:

[x] The spec conforms to the Swagger 2.0 specification.
[x] The spec follows the guidelines described in the Swagger checklist document.
[x] Validation tools were run on swagger spec(s) and have all been fixed in this PR.

@marstr,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

This PR is to address issue Azure/azure-sdk-for-go#576. The relevant Swagger Spec Section is 5.2.2.

@Nilambari do you have any comments?  Also is there a max length that should be added?

Ping @Nilambari

No modification for NodeJS

No modification for Python

 https://github.com/Azure/azure-sdk-for-ruby/pull/745

No modification for Ruby


Integrate with conformance cucumber tests

The cloudevents/conformance repository now has HTTP transport cucumber specifications. This SDK should integrate these into the tests.
Ref: https://github.com/cucumber/cucumber-js

Going to assign that to me and start playing with. In case things get delayed (until next week) I remove the assignment :+1:

removing the assignment. I can revisit if needed later, also short week for me here.


AnalysisException: Recursive view df detected (cycle: df -> df)

https://community.databricks.com/s/question/0D53f00001Qugr7CAB/upgrading-from-spark-24-to-32-recursive-view-errors-when-using
This appears to be a bug introduced in Spark 3.2.0.
For the moment we recommend using a different version of Spark.  We're not intending to fix this in Splink unless the Spark maintainers confirm this is a deliberate change (seems unlikely because it breaks backward compatibility)

Issue raised against main Spark project here: https://issues.apache.org/jira/browse/SPARK-37690

Added a comment against the issue above.  Copying here for infp:
Someone here has suggested this is an intentional breaking change introduced in Spark 3.1:
From Migration Guide: SQL, Datasets and DataFrame - Spark 3.1.1 Documentation (apache.org)

In Spark 3.1, the temporary view will have same behaviors with the permanent view, i.e. capture and store runtime SQL configs, SQL text, catalog and namespace. The capatured view properties will be applied during the parsing and analysis phases of the view resolution. To restore the behavior before Spark 3.1, you can set spark.sql.legacy.storeAnalyzedPlanForView to true.

Grateful if someone could clarify.  Worth noting that the example code works in Spark 3.1.2, just not 3.2.0.  It's not obvious to me the above quote implies createOrReplaceTempView would fail in the example code posted in the issue.

FYI -- changing all of the references to df across term_frequencies.py to something like df_no_tf fully resolves this issue in Spark 3.2.

Added a comment against the issue above. Copying here for info:
Someone here has suggested this is an intentional breaking change introduced in Spark 3.1:
From Migration Guide: SQL, Datasets and DataFrame - Spark 3.1.1 Documentation (apache.org)

In Spark 3.1, the temporary view will have same behaviors with the permanent view, i.e. capture and store runtime SQL configs, SQL text, catalog and namespace. The capatured view properties will be applied during the parsing and analysis phases of the view resolution. To restore the behavior before Spark 3.1, you can set spark.sql.legacy.storeAnalyzedPlanForView to true.

Unfortunately, setting spark.sql.legacy.storeAnalyzedPlanForView to true did NOT resolve the issue. I am still seeing 'AnalysisException: Recursive view Daily_ControlResults_DF detected'


Process Metrics unavailable to the MultiProcess CollectorRegistry

We are looking to get the process metrics such as process_resident_memory_bytes and process_cpu_seconds_total added to our FastAPI metrics endpoint.  We are using the MultiProcessCollector which invokes CollectorRegistry and it looks like this doesn't allow for adding the ProcessCollector which generates those metrics.  If for testing purposes I manually edit instrumentation.py and register PROCESSCOLLECTOR I can see the desired metrics in the output.
Is there some overriding reason that PROCESSCOLLECTOR is unavailable to the MultiProcessCollector?  Is there some other way to achieve including the process metrics in the output?  If not, we can investigate submitting a PR that allows optional additional collectors to be passed into the expose method.

Reading the documentation https://github.com/prometheus/client_python#multiprocess-mode-eg-gunicorn from my understanding ProcessCollector is not support with multi process mode.


Docker 1.12 support

Is docker 1.12 support being worked on at the moment at all and if not, would you all like some help implementing the new endpoints? Is it pretty straight forward to implement in the gem? Anything you can think of that may be a gotcha with 1.12 features?
Thanks.

Here is pull request related to this issue #462


Usage of Speechbrain functions breaks PyTorch computation graph

Hi,
I've been trying to obtain gradients from CRDNN loss with respect to the input data. I am using the CRDNN recipe for Librispeech with minor modifications.
            loss = model.compute_objectives((p_seq, wav_lens), x, stage=Stage.TRAIN) # Not sure about the length param
            print(loss)
            print(torch.autograd.grad(loss, wavs, retain_graph=True))

However, I receive an error:
speechbrain.core - Exception:
Traceback (most recent call last):
  File "uncertainty_baselines/pipeline_correlation_scores.py", line 169, in <module>
    pipeline_uncertainty(hparams, test=hparams["test_mode"])
  File "uncertainty_baselines/pipeline_correlation_scores.py", line 94, in pipeline_uncertainty
    _, _, confs, confs_raw = uncertainty_method.split_samples(dataloader, brain, conf_cnt=None)
  File "/home/kacper/Desktop/repos/uncertainty_baselines/uncertainty_baselines/baselines/baseline_methods.py", line 36, in split_samples
    confs, confs_raw = self.get_confidence_scores(data_loader, model)
  File "/home/kacper/Desktop/repos/uncertainty_baselines/uncertainty_baselines/baselines/baseline_methods.py", line 214, in get_confidence_scores
    print(torch.autograd.grad(loss, wav_lens, retain_graph=True))
  File "/home/kacper/miniconda3/envs/uncertain/lib/python3.8/site-packages/torch/autograd/__init__.py", line 234, in grad
    return Variable._execution_engine.run_backward(
RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.

This means that most likely the computation graph is broken somewhere.  This issue highlights this problem: https://discuss.pytorch.org/t/example-for-one-of-the-differentiated-tensors-appears-to-not-have-been-used-in-the-graph/58396/4
There might be the case where some feature engineering step converts the data to numpy or other non-PyTorch tensor which will break computation graph.
I know that it's like looking for a needle in a hay but are you aware feature extraction/engineering steps that might break the computation graph? I need to have gradients computed with respect to the input for my research.
Possible functions that I suspect are currently
compute_features: !new:speechbrain.lobes.features.Fbank
   sample_rate: !ref <sample_rate>
   n_fft: !ref <n_fft>
   n_mels: !ref <n_mels>

augmentation: !new:speechbrain.lobes.augment.TimeDomainSpecAugment
   sample_rate: !ref <sample_rate>
   speeds: [95, 100, 105]

normalize: !new:speechbrain.processing.features.InputNormalization
   norm_type: global

Forward loop for reference. I am using the CRDNN recipe for Librispeech with minor modifications.
    def compute_forward(self, batch, stage, wav_raw=None, wav_lens_raw=None, turn_off_augment=False):
        """Forward computations from the waveform batches to the output probabilities."""
        batch = batch.to(self.device)
        if wav_raw is not None and wav_lens_raw is not None:
            wavs, wav_lens = wav_raw, wav_lens_raw
        else:
            wavs, wav_lens = batch.signal

        print(wavs.requires_grad)
        tokens_bos, _ = batch.tokens_bos
        wavs, wav_lens = wavs.to(self.device), wav_lens.to(self.device)

        # Add augmentation if specified
        if stage == sb.Stage.TRAIN:
            if hasattr(self.modules, "env_corrupt"):
                print("in env corrupt")
                wavs_noise = self.modules.env_corrupt(wavs, wav_lens)
                wavs = torch.cat([wavs, wavs_noise], dim=0)
                wav_lens = torch.cat([wav_lens, wav_lens])
                tokens_bos = torch.cat([tokens_bos, tokens_bos], dim=0)
            if turn_off_augment is False:
                if hasattr(self.hparams, "augmentation"):
                    print("Before augmentation")
                    print(wavs.requires_grad)
                    wavs = self.hparams.augmentation(wavs, wav_lens)
                    print(wavs.requires_grad)
                    print("After augmentation")

        # Forward pass
        print(wavs.requires_grad)
        feats = self.hparams.compute_features(wavs)
        print(wavs.requires_grad)
        feats = self.modules.normalize(feats, wav_lens)
        print(wavs.requires_grad)
        x = self.modules.enc(feats.detach())
        e_in = self.modules.emb(tokens_bos)  # y_in bos + tokens
        h, _ = self.modules.dec(e_in, x, wav_lens)

        # Output layer for seq2seq log-probabilities
        logits = self.modules.seq_lin(h)
        p_seq = self.hparams.log_softmax(logits)
        print(wavs.requires_grad)
        # Compute outputs
        if stage == sb.Stage.TRAIN:
            current_epoch = self.hparams.epoch_counter.current
            if current_epoch <= self.hparams.number_of_ctc_epochs:
                # Output layer for ctc log-probabilities
                logits = self.modules.ctc_lin(x)
                p_ctc = self.hparams.log_softmax(logits)
                print(wavs.requires_grad)
                return p_ctc, p_seq, wav_lens
            else:
                return p_seq, wav_lens, logits
        else:
            if stage == sb.Stage.VALID:
                p_tokens, scores = self.hparams.valid_search(x, wav_lens)
            else:
                p_tokens, scores = self.hparams.test_search(x, wav_lens)
            return p_seq, wav_lens, p_tokens

Everywhere in this code, print(wavs.requires_grad) is equal to True when I disable augmentation of the input. This means that probably somewhere in Speechbrain function computation graph is broken due to the Numpy conversion (perhaps)

Hi I think I actually found the culprit:
        x = self.modules.enc(feats.detach())

This detaches the tensor from the computation graph. After removing the detach() option I can now compute the gradient w.r.t input.
Do you know perhaps why the tensor is detached at this stage? Are steps until this point considered as feature engineering/extraction?


Trying to migrate to this new version of the SDK

Installed the new version with:
pipenv install google-genai

Now trying to find the new imports for: ChatSession, GoogleAPIError, BlockedError, GenerationConfig, GenerativeModel, etc.
Where are they?

Installed the new version with:
pipenv install google-genai

Now trying to find the new imports for: ChatSession, GoogleAPIError, BlockedError, GenerationConfig, GenerativeModel, etc.
Where are they?


Docker documentation is contradictory regarding VolumesFrom

Here is what you get when you search in the docker repo for the string "VolumesFrom": https://github.com/docker/docker/search?utf8=✓&q=VolumesFrom
The issue is that most of the response formats ascribed to docker inspect say that the endpoint will return something like
 "VolumesFrom":""
This is untrue - as of docker 1.7 (and probably much earlier), docker returns either null or something like ["contrainerid"] as the value for that key.
The tricky part is that early docker (1.0) used to accept a string as input for VolumesFrom in the remote api, before a change (maybe around 1.4?) had it take an array instead. I'm not sure when both the input and output changed, but someone more knowledgeable should probably retroactively correct the documentation thereof.
I stumbled across this issue because docker 1.7's remote API started throwing errors when passed string values for VolumesFrom - it used to silently accept the empty string as a valid input. I don't count this as a bug, but it was certainly surprising.
As an aside there has GOT to be a better way to arrange all of these docs besides a series of increasingly large and self-contradictory markdown files.

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

By the way, that bot is obnoxious. At least shorten it to say something less off-putting like "Hi! If this is a bug report regarding the runtime, please follow this link to ensure that we have all the information we need! This message is being posted automatically because we don't see the minimum amount of information we need to investigate bugs in the engine."

Documentation was fixed. Thanks for report.


Document when to use BottomNavigationBarThemeData labelStyles

Steps to Reproduce

Execute flutter run on the code sample

Expected results:
I want to see the selectedLabel.color red and the unselectedLabel.color black
Actual results:
I see the selectedLabel.color blue (default primary color) and the unselectedLabel.color grey

Code sample

import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Test Bottom Theme',
      theme: ThemeData(
          primaryColor: Colors.blue.shade900,
          appBarTheme: AppBarTheme(backgroundColor: Colors.blue.shade900),
          bottomNavigationBarTheme: BottomNavigationBarThemeData(
              unselectedLabelStyle: const TextStyle(color: Colors.black),
              selectedLabelStyle: const TextStyle(color: Colors.red),
              selectedIconTheme: IconThemeData(color: Colors.blue.shade900))),
      home: const HomeScreen(),
    );
  }
}

class HomeScreen extends StatelessWidget {
  const HomeScreen({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(   ),
      body: Container(),
      bottomNavigationBar: BottomNavigationBar(
        items: const <BottomNavigationBarItem>[
          BottomNavigationBarItem(icon: Icon(Icons.home), label: "Home"),
          BottomNavigationBarItem(icon: Icon(Icons.bookmark), label: "Bookmarks"),
        ],
      ),
    );
  }
}

Logs

~flutter run --verbose

no issues

~flutter analyze

Analyzing components...
No issues found! (ran in 3.5s)

~flutter doctor -v

[!] Flutter (Channel unknown, 3.3.3, on Ubuntu 20.04.5 LTS 5.15.0-52-generic, locale en_US.UTF-8)
    ! Flutter version 3.3.3 on channel unknown at /home/bricekk/snap/flutter/common/flutter
    ! Upstream repository unknown
    • Framework revision 18a827f393 (5 weeks ago), 2022-09-28 10:03:14 -0700
    • Engine revision 5c984c26eb
    • Dart version 2.18.2
    • DevTools version 2.15.0

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /home/bricekk/Android/Sdk
    • Platform android-33, build-tools 33.0.0
    • Java binary at: /snap/android-studio/124/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at google-chrome

[✓] Linux toolchain - develop for Linux desktop
    • clang version 10.0.0-4ubuntu1
    • cmake version 3.16.3
    • ninja version 1.10.0
    • pkg-config version 0.29.1

[✓] Android Studio (version 2021.3)
    • Android Studio at /snap/android-studio/124/android-studio
    • Flutter plugin version 70.0.3
    • Dart plugin version 213.7371
    • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)

[✓] IntelliJ IDEA Community Edition (version 2022.2)
    • IntelliJ at /snap/intellij-idea-community/390
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code
    • VS Code at /snap/code/current
    • Flutter extension can be installed from:
      🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[✓] Connected device (3 available)
    • sdk gphone x86 (mobile) • emulator-5554 • android-x86    • Android 11 (API 30) (emulator)
    • Linux (desktop)         • linux         • linux-x64      • Ubuntu 20.04.5 LTS 5.15.0-52-generic
    • Chrome (web)            • chrome        • web-javascript • Google Chrome 106.0.5249.119

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

Not sure whether it is by intention or not but as a workaround you can use fixedColor property on BottomNavigationBar class which controls exactly the color of selected label.

The selectedColor unconditionally overrides the corresponding text style's color, even if the default is used. This is typical for components that allow one to specify a text style and a text color. It's definitely not explained clearly.
BottomNavigationBar's API doc doesn't appear to mention BottomAppBarTheme(Data), another flaw.

i don't think is related...

selectedLabelStyle and unselectedLabelStyle will be used by setting the useLegacyColorScheme (property of BottomNavigationBar) to false


Pokemon Optimizer limit on Eevees

Put a limit (12)  to the number of Eevee that can be kept when player is missing one evolution.
This will avoid inventory being full without any solution to empty it.

:


Add "false positive" and "remove" link flair to automod

https://www.reddit.com/message/messages/5c9ro0

Done


Fix msbuild failing when '@' is present in path (2nd attempt)

Fix for https://github.com/grpc/grpc/issues/30746 - "C# Protobuf msbuild task fails with glob pattern if "@" is present in the path"
After investigating there is a feature/bug with MSBuild: https://github.com/dotnet/msbuild/issues/8104
where setting the Source metadata from the Identity metadata needs unescaping if the original items come from
a wildcard match and the paths contain an '@' character. See https://github.com/dotnet/msbuild/issues/8104#issuecomment-1298309795 for details.
Also as a precaution make sure no duplicate files in the list of files passed to the CSC compiler.
Tested on Linux and Windows with MSBuild versions 15, 16 and 17

@jtattermusch new fix for https://github.com/grpc/grpc/issues/30746 which should also fix the distribution tests.
I'm adding tests to Grpc.Tools.Tests separately.


Feature Request: Allow consumer to control bucket ids

All timers are grouped into "buckets" that are identified by the ms parameter. Using the ms parameter as a bucket identifier can have undesired consequences.
My suggestion: add an extra id parameter and use it as bucket identifier instead of ms parameter.
Furthermore, I think this parameter should be required (it should not default to the value of ms).
Which would be a non-backwards-compatible change.

Since, as I understand, set-harmonic-interval was developed to be used by react-use, let me use react-use as an example.
Suppose you want to have 2 sets of components - numerous instances of <X/> and numerous instances of <Y/> - and:

you want both <X/> and <Y/> to update after the same interval (same ms parameter value)
but you don't necessarily want instances of X to update when instances of Y update

This is impossible with the current implementation, because buckets are created per ms parameter.
Hopefully the above explanation is sufficient but consider this example for completeness: https://codesandbox.io/s/tender-kepler-9r1j1?file=/src/App.js
The stoplights will harmonically change the light every 5 seconds.
The triggers will harmonically flash every 5 seconds as well.
But triggers will not start flashing until you press any of them.
And if it just so happens, that you push the trigger 100ms before the stoplights are about to change, the trigger indicators will be on only for that 100ms, while I'd expect them to be on for the first 5 seconds after triggering.

Yes, I see how having control over buckets would be useful, what I don't understand is

Furthermore, I think this parameter should be required (it should not default to the value of ms).

Why not make it default to ms with ability to overwrite?

@streamich

Why not make it default to ms with ability to overwrite?

I find it surprising to have an assumption that all intervals with the same ms param will be in the same bucket - I think it'd be better if the API required its' users to be aware and in control of how the grouping works to avoid surprises (i.e. bugs) as the code evolves.

I've recently encountered a bug which reminds me of this situation, and it could have been avoided if the "bucketing" was explicit from the start.
Let me try to visualize it.
Imagine a component like this:
import React, { Component } from 'react';

const cache = {};

class CollectionCache extends Component {
  constructor() {
    state = cache;
  }

  componentDidUpdate() {
    cache = state;
  }

  doSomething = (/* some params */) => {
    this.setState(/* do something to state */);
  };

  render() {
    const { children, ids } = this.props;

    return children({
      collection: this.state.map((id) => ids.includes(id)),
      doSomething: this.doSomething,
    });
  }
}

It's all fine as long as CollectionCache ids are globally unique. But if they're not, then there will be bugs if you want to use CollectionCache for 2 different collections: entities from one collection would appear in the other.
A solution is to have a cache key (bucket id):
import React, { Component } from 'react';

const cache = {};

class CollectionCache extends Component {
  constructor() {
    state = cache[this.props.cacheKey] || {};
  }

  componentDidUpdate() {
    cache[this.props.cacheKey] = state;
  }

  doSomething(/* some params */) {
    this.setState(/* do something to state */);
  }

  render() {
    const { children, ids } = this.props;

    return children({
      collection: this.state.map((id) => ids.includes(id)),
      doSomething: this.doSomething,
    });
  }
}

Should the cacheKey prop be required from the beginning, there would be no bug. But if the cacheKey prop had a default value, the programmer could easily omit it and the bug would be there. Especially when using TypeScript, I find I'm lazy to check the documentation - if it compiles and seems to work, then I don't want to pass any more props.
I hope you can see the similarity of this problem. If not, I'll be happy to provide a working example of an issue.


specify repo name when problem with repo

Right now all cosmos requests fail when there is a problem with one repo. We should at least specify which repo is having the problem.
It would help the UI team if we could specify the name as a separate property. cc @mlunoe

No activity in a while. Closing.

Uhmm.... Why are we closing this?
This is still very much not a great experience.


add setup.txt

The setup.txt file is a text file that describes the connection and user information necessary to get started with git.

This is a code owner review test.


Add developing section to README

Fixes https://github.com/gibiansky/IHaskell/issues/1316
Describes how to start working on IHaskell using nix develop. Right now this section is quite bare. Perhaps in the future it can be expanded with how to build the different versions of IHaskell, and how to run the tests. For now, this should be enough to get people on their way to start writing pull requests :wink:
I'm really enthusiastic about this project, by the way. This kernel allows for some powerful stuff. A big thanks to the developers for creating and maintaining it. :rocket: :rocket: :rocket:

This approach is not how I would recommend working on IHaskell, and I don't see myself merging it in its current state, see https://github.com/gibiansky/IHaskell/issues/1316#issuecomment-1013980103.

This approach is not how I would recommend working on IHaskell, and I don't see myself merging it in its current state, see #1316

Ah, I've added a link to the mentioned blog post as first recommendation. The flake method is now shown as alternative :+1:

Thanks!


move_all_by fails when the layout contains pins

move_all_by (shown below :point_down:)
https://github.com/BerkeleyPhotonicsGenerator/BPG/blob/8018c0daef7d3a041e4d187c289932f9636deb9e/BPG/photonic_core.py#L559-L564
assumes each object from the list in the chain has a move_by with dx, dy, and unit_mode parameter. Unfortunately, PhotonicPinInfo (PinInfo) doesn't use unit_mode:
    def move_by(self, dx=0, dy=0):

ref: https://github.com/BerkeleyPhotonicsGenerator/BAG_framework/blob/31334948d686a8241b064995af1ea824d78a5d1c/layout/objects.py#L1917

Thanks, I can add a unit_mode argument in there.

Actually I think it would be best to post this issue to the main BAG repository, since this affects them as well. I've forked BAG over into this organization with the intent to embed it within BPG for easier setup, but that task will likely not happen for a while.
The fastest way to resolve your issue would be to change it in the main repo, but regardless when I finish moving BAG over, I will make this change here.

Actually I think it would be best to post this issue to the main BAG repository, since this affects them as well.

:+1:, created https://github.com/ucb-art/BAG_framework/issues/8!
(I updated the issue title to explain that this issue could occur in more places.)


Chore/rs 4187 update rspec

Big shout out to Jeff for doing all the matcher updates to the spec files 🙏
https://github.com/user-interviews/rails-server/issues/4187

Gonna leave it to you and Jeff to figure out that config change -- otherwise this is looking great!


Abstract backend calling

It might be handy to support abstract backend calling in Flow in a way similar to what Keras backend does.
An hypothetical API could be as follows:
(in bash)
export FLOW_BACKEND = "libsumo"

(in python)
from flow import backend as K
K.addFull(...)
K.setSpeed(...)
...
K.step()
...
K.getPosition(...)
K.getAcceleration(...)

Hi Fangyu, could you explicate on what the advantages of this are over the current Vehicles class?

Sure! In fact, the proposed abstract backend API is a level lower than the current Vehicle class. We are replacing the traci calls inside of the Vehicle class, not the Vehicle class itself. For example, instead of calling traci.vehicle.setSpeed(...), we do something like:
import traci as K
K.vehicle.setSpeed(..)

This recommendation is assuming that, in the future, we want to replace traci with something better, i.e., libsumo. In that situation, all we need to modify is the backend variable. For example, an naive implementation could be:
import libsumo as K
K.vehicle.setSpeed(..)

Going into this direction one step further, if we want to use something other than SUMO, say, Aimsun, all we need to do is to change (kind of) import libsumo as K to import aimsun as K. This way, it makes comparing different simulation engines much more tractable and efficient.
This same concept could also apply to the DRL interface. Maybe, we want to be able to freely interchange between rllib, rllab, and perhaps google/dopamine. Abstract backend calling can make our life ε more easier in this regard.
Note that the way that Keras implements this is a little bit more complex than what I have shown, but the architectural principle holds the same.

Since this is a subset of the aimsun kernel, I’m gonna close it


Round up returns wrong number

I'm expecting the result of the roundUp to be 1.12, but it returns wrongly 1.13.
const Big = require('big.js');

const amountAvailable = 1.12;
const quantityArs = 1197.5384615384617;
const exchangeRate = 1069.2307692307693;

// Big
console.log(Big(amountAvailable).times(exchangeRate).toNumber()); // 1197.5376
console.log(Big(quantityArs).div(exchangeRate).toNumber()); // 1.12
console.log(Big(quantityArs).div(exchangeRate).round(2, Big.roundUp).toNumber()); // 1.13
console.log(Big(1.12).round(2, Big.roundUp).toNumber()); // 1.12

Look first at the value of Big(quantityArs).div(exchangeRate) and then ask again if you still think the rounding is incorrect.


Ubuntu Server fails

Hello,
I am very interested in trying this.
I am trying ubuntu-20.04.1-live-server-arm64.iso  and would like to perform a install. so I also created a DMG. But it does not seems to launch the installer ... do you have an idea what I am doing wrong ?
I use the following parameters
./vftool -k /Users/simonph/Desktop/Ubuntu/UbuntuServer/vmlinuz -i /Users/simonph/Desktop/Ubuntu/UbuntuServer/initrd -d /Users/simonph/Desktop/Ubuntu/VM.dmg -c /Users/simonph/Desktop/Ubuntu/UbuntuServer/ubuntu-20.04.1-live-server-arm64.iso

And here is the console

How did you obtain the kernel and initrd files ? I'm trying to find this since yesterday 😅

You can find vmlinuz and initrd inside the iso image. (I used Keka to mount it). For the vmlinuz, you have to copy it, then rename it as .gz and extract.

Did you do anything to the initrd file ?

no

if you type exit at the prompt, it may give you some insight at what is going on.

Probably missing some -a  but which ones

try with -a "console=hvc0 root=/dev/vda1" I think it needs other arguments to work well because right now the text is all messed up. But thats progress

I have tried that , but nothing appears on the /dev/ttys  now

I got this tty waiting message:
$HOME/Library/Developer/Xcode/DerivedData/vftool-gifyyerhcgkkioeipwnxidpgzatn/Build/Products/Debug/vftool \
  -k $HOME/develop/UbuntuServer/vmlinuz \
  -i $HOME/develop/UbuntuServer/initrd \
  -d $HOME/VM/VM-Ubuntu.dmg \
  -c $HOME/develop/UbuntuServer/ubuntu-20.04.1-live-server-arm64.iso \
  -a "console=hvc0 root=/dev/vda1"

2020-11-29 16:47:19.508 vftool[18625:1746529] vftool (v0.1 25/11/2020) starting
2020-11-29 16:47:19.508 vftool[18625:1746529] +++ kernel at file:///Users/peter/develop/UbuntuServer/vmlinuz, initrd at file:///Users/peter/develop/UbuntuServer/initrd, cmdline 'console=hvc0 root=/dev/vda1', 1 cpus, 512MB memory
2020-11-29 16:47:19.512 vftool[18625:1746529] +++ fd 3 connected to /dev/ttys002
2020-11-29 16:47:19.512 vftool[18625:1746529] +++ Waiting for connection to:  /dev/ttys002

Create an empty disc with the MacOS disk utility. The virtual machine failed to start!

Hi all (I'm late to the thread, real life intervened)  :)  I've just tried an ubuntu-server ISO again, and can re-create.  Two things to note:

The default RAM size (512MB) is too low for the ubuntu initrd -- passing -m 2048 solves the (initrd) prompt issue that @phsimons reported.
The ubuntu installer is unusable as @xiaolait and @sylvainfilteau note.  (Even hitting ^T for simple mode, it's getting random keypress inputs.)

Notes on installers:

Ubuntu-server boots, but the terminal is unusable.
Debian install kernel/initrd on the debian-10.6.0-arm64-netinst.iso doesn't appear to have virtio drivers, so is a no-go.  I installed Debian using debootstrap (see below).
Fedora-33 boots, but then seems to crash (it immediately terminates/init halts the system).
openSUSE-Tumbleweed-NET-aarch64-Current.iso (as of 28Nov20) installer boots nicely, is colourful and is usable, except has minor terminal issues: I keep having to hit tab/return to get the package download bargraphs to update.

However.... I don't believe running the installers is as easy as it sounds at this stage:  once the packages are installed to your disc.img, then you'll need to fish out a kernel/initrd from the installed image, back to macOS, to boot it.  (The initrd and likely the kernel will be different to those the installer uses.)
For this reason, enduring a chicken & egg situation and solving with another Linux box is IMO the easiest route currently.  For example, installing Ubuntu or Debian using debootstrap, something like:

Create a disc image, partitions, rootfs
Mount root on say /mnt
debootstrap --arch=arm64 --foreign stable /mnt as root
While that's cooking, build a mainline arm64 kernel (which won't need an initrd to boot debian, at least).  When debootstrap is finished, the kernel/disc.img can be booted in a VM, and debootstrap --second-stage used to finish the installation (plus some manual stuff, like creating /etc/fstab, /etc/hosts, tzdata/locales).  There are Debian/Ubuntu debootstrap directions on le net...

That doesn't help anyone that's desperate to use CentOS/Fedora etc.  :)  Anyone know if they have a debootstrap equivalent?
Finally:  legend has it that there will be an EFI-related boot method in future, which may pave the way to booting ISOs (or even OVMF/edk2) directly, and that removes all of this faffing around with finding kernels/initrds.  Hopefully a future macOS release will provide, here.

Create an empty disc with the MacOS disk utility. The virtual machine failed to start!

Oh dear -- I haven't seen this error before.  The disc simply needs to be a file that exists and is of the size you want to use.  For example, dd if=/dev/zero of=my_disc.img bs=1024k count=20480 makes a 20GB file of zeros.
But, see my previous message -- at this stage it is probably easiest to use Virtualization.framework/vftool with a disc image that is pre-installed elsewhere, unfortunately.  Someone suitably motivated could perhaps write a script to extract kernels/initrds from a distribution install ISO, run the installer and then re-extract kernels/initrd from the installed disc, perhaps.
I'll see if I can recreate that error, though, how odd.

Thanks for the extended explanations ...
indeed Ubuntu-server boots, but the terminal is unusable.

But at this point isn't there a way to ssh in the VM and continue the install ?

I've had good luck with the Ubuntu cloud images linked by a kind user on the MacRumors forums [1]:

kernel: https://cloud-images.ubuntu.com/releases/focal/release/unpacked/ubuntu-20.04-server-cloudimg-arm64-vmlinuz-generic
initrd: https://cloud-images.ubuntu.com/releases/focal/release/unpacked/ubuntu-20.04-server-cloudimg-arm64-initrd-generic
disk image: https://cloud-images.ubuntu.com/releases/focal/release/ubuntu-20.04-server-cloudimg-arm64.tar.gz

Let's assume you place all of these files in a linux directory under your home directory.  Prepare the kernel:
mv ubuntu-20.04-server-cloudimg-arm64-vmlinuz-generic vmlinux.gz
gunzip vmlinux.gz

The initrd (just renaming for convenience):
mv ubuntu-20.04-server-cloudimg-arm64-initrd-generic initrd

And finally, the disk image:
tar xvfz ubuntu-20.04-server-cloudimg-arm64.tar.gz
mv focal-server-cloudimg-arm64.img disk.img

With these being cloud images, I want to disable cloud-init, set the root password, generate some SSH host keys, and configure DHCP.  To work on the disk image, we'll boot the virtual machine without specifying the root to the Linux command line.
vftool -k vmlinux -i initrd -d disk.img -m 1024 -a "console=hvc0"

In another terminal, use screen to attach to the tty.  The VM will finish booting and drop to an (initramfs) prompt.  Then run:
mkdir /mnt
mount /dev/vda /mnt
chroot /mnt

touch /etc/cloud/cloud-init.disabled

echo 'root:root' | chpasswd

ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
ssh-keygen -f /etc/ssh/ssh_host_ed25519_key -N '' -t ed25519

cat <<EOF > /etc/netplan/01-dhcp.yaml
network:
    renderer: networkd
    ethernets:
        enp0s1:
            dhcp4: true
    version: 2
EOF

exit
umount /dev/vda

Kill the VM with CTRL+C in the terminal you ran vftool and then run it again, but this time specifying the root to the kernel:
vftool -k vmlinux -i initrd -d disk.img -m 1024 -a "console=hvc0 root=/dev/vda"

You can now connect with screen and login to the root user with root as the password.  You can also ssh into the VM if you configure your public key.  Show the IP address via ip addr show | grep "inet 192"

Great Job!
Works for me..... Congratulation....

Works well, thanks

Got to install Ubuntu server on a disk image but not able to boot into it. Tried passing command line arguments to set root directory, doesn't seem to work. Got any idea?

I use this to install the initramfs
mkdir /mnt
mount /dev/vda /mnt
chroot /mnt

touch /etc/cloud/cloud-init.disabled

echo 'root:roor' | chpasswd

echo "m1silicon" >/etc/hostname
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
ssh-keygen -f /etc/ssh/ssh_host_ed25519_key -N '' -t ed25519

cat <<EOF > /etc/netplan/01-dhcp.yaml
network:
    renderer: networkd
    ethernets:
        enp0s1:
            dhcp4: true
    version: 2
EOF

exit
umount /dev/vda

CTRL-C inside the VM
Then boot with disk:
$ vftool -k vmlinux -i initrd -d ubuntu-20.04-disk.img -m 1024 -a "console=hvc0 root=/dev/vda"

How I can expand the filesystem?
apt-get update
# crash with no space at filesystem...
df
Filesystem     1K-blocks    Used Available Use% Mounted on
udev              460708       0    460708   0% /dev
tmpfs              98816   10608     88208  11% /run
/dev/vda         1286424 1270040         0 100% /
tmpfs             494076       0    494076   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs             494076       0    494076   0% /sys/fs/cgroup
/dev/loop0         50048   50048         0 100% /snap/core18/1936
/dev/loop1         27520   27520         0 100% /snap/snapd/9730
/dev/loop2         61312   61312         0 100% /snap/lxd/18152
tmpfs              98812       0     98812   0% /run/user/0

I did this with qemu-img installed using brew
To increase the disk by 50Gb
qemu-img resize disk.img +50G
Then, you will need to resize the fs

You install the emu with rosetta 2 intel brew?
I got the installation error at ny arm brew installation?
==> /opt/homebrew/Cellar/sphinx-doc/3.3.1_1/libexec/bin/pip install -v --no-deps --no-binary :all: --ignore-installed /private/tmp/sphinx-doc--MarkupSafe-20201201-12738-1vhhiv8
Last 15 lines from /Users/peter/Library/Logs/Homebrew/sphinx-doc/15.pip:
    requirement.install(
  File "/opt/homebrew/Cellar/sphinx-doc/3.3.1_1/libexec/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 856, in install
    six.reraise(*exc.parent)
  File "/opt/homebrew/Cellar/sphinx-doc/3.3.1_1/libexec/lib/python3.9/site-packages/pip/_vendor/six.py", line 703, in reraise
    raise value
  File "/opt/homebrew/Cellar/sphinx-doc/3.3.1_1/libexec/lib/python3.9/site-packages/pip/_internal/operations/install/legacy.py", line 74, in install
    runner(
  File "/opt/homebrew/Cellar/sphinx-doc/3.3.1_1/libexec/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 273, in runner
    call_subprocess(
  File "/opt/homebrew/Cellar/sphinx-doc/3.3.1_1/libexec/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 242, in call_subprocess
    raise InstallationError(exc_msg)
pip._internal.exceptions.InstallationError: Command errored out with exit status 1: /opt/homebrew/Cellar/sphinx-doc/3.3.1_1/libexec/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/tmp/pip-req-build-ltp9y04v/setup.py'"'"'; __file__='"'"'/private/tmp/pip-req-build-ltp9y04v/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/tmp/pip-record-x3k5bx1n/install-record.txt --single-version-externally-managed --compile --install-headers /opt/homebrew/Cellar/sphinx-doc/3.3.1_1/libexec/include/site/python3.9/MarkupSafe Check the logs for full command output.
WARNING: You are using pip version 20.2.4; however, version 20.3 is available.
You should consider upgrading via the '/opt/homebrew/Cellar/sphinx-doc/3.3.1_1/libexec/bin/python -m pip install --upgrade pip' command.
Removed build tracker: '/private/tmp/pip-req-tracker-292f1dnh'

Do not report this issue to Homebrew/brew or Homebrew/core!

We do not provide support for this (yet).
Reinstall Homebrew under Rosetta 2 until we support it.
You will encounter build failures with some formulae.
Please create pull requests instead of asking for help on Homebrew's GitHub,
Twitter or any other official channels. You are responsible for resolving
any issues you experience while you are running this
unsupported configuration.

I install qemu with rosetta 2.
I resize the image:'
$ qemu-img resize ubuntu-20.04-disk.img +50G
WARNING: Image format was not specified for 'ubuntu-20.04-disk.img' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
Image resized.
$ ls -l
total 45735352
drwxr-xr-x   8 peter  staff          256  1 Dez 06:38 .
drwxr-xr-x  12 peter  staff          384  1 Dez 05:06 ..
-rw-r--r--   1 peter  staff          170 24 Nov 00:41 README
-rw-r--r--   1 peter  staff     27702569 24 Nov 00:39 initrd
-rw-r--r--   1 peter  staff<PHONE_NUMBER>0  1 Dez 18:59 ubuntu-20.04-disk.img
-rw-r--r--   1 peter  staff    471734209 24 Nov 00:42 ubuntu-20.04-server-cloudimg-arm64.tar.gz
-rw-r--r--   1 peter  staff<PHONE_NUMBER>0  1 Dez 06:38 vm_disc.img
-rw-r--r--   1 peter  staff     30382464 24 Nov 00:39 vmlinux

But after start inside the ubuntu vm the resizing isn't available:
df
Filesystem     1K-blocks    Used Available Use% Mounted on
udev              975012       0    975012   0% /dev
tmpfs             201676     736    200940   1% /run
/dev/vda         1286424 1262040      8000 100% /
tmpfs            1008376       0   1008376   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs            1008376       0   1008376   0% /sys/fs/cgroup
/dev/loop0         27520   27520         0 100% /snap/snapd/9730
/dev/loop1         50048   50048         0 100% /snap/core18/1936
/dev/loop2         61312   61312         0 100% /snap/lxd/18152
tmpfs             201672       0    201672   0% /run/user/0

The disk info show that the image can grow...
qemu-img info ubuntu-20.04-disk.img
image: ubuntu-20.04-disk.img
file format: raw
virtual size: 51.3 GiB<PHONE_NUMBER>0 bytes)
disk size: 1.31 GiB

Ahh!
I forgot the "resize2fs /dev/vda" inside the ubuntu fs... :)

Were you able to boot into the disk? I always end up with the live version?

Now docker is running at the vm.
Install the docker cli (Rosetta) and use an docker ssh context.
Docker Engine is now directly available via at my Visual Studio Code.
Container Hacking is possible at the M1....
Can I help to document the setup?

Were you able to boot into the disk? I always end up with the live version? If so, how did you manage to get it boot from the disk as the root! Any help is very much appreciated!

It is a two phase boot process.
Frist boot with RAM disk and create the /dev/vda manually without login
Second boot with disk (add the root=/dev/vda parameter), is a normal boot with login

Thanks a ton! But if I reboot won’t I lose the changes since we’re not writing the changes! Correct me if I’m wrong! Can you walk me through the process? That’d be of great help

a few days ago I created a turn-key ready bundle containing SimpleVM an Ubuntu. the image works also with vftool:
https://github.com/mibosshard/SimpleVM/releases/download/20201128/SimpleVM_Ubuntu.zip
https://github.com/mibosshard/SimpleVM/releases
I also made a binary of vftool:
https://github.com/mibosshard/vftool/releases/download/20201126/vftool.zip
https://github.com/mibosshard/vftool/releases

fwiw, it's not necessary to install qemu (and homebrew on rosetta).
Just appending data on the cloud image is enough.
I used truncate -s 20G focal-server-cloudimg-arm64.img (truncate needs to be installed, for instance via macports), but a plan >> should be enough

I managed to get things up and running following these instructions, so thanks so much to everyone who contributed!
For me, DNS resolution doesn't work out of the box. The default in /etc/resolv.conf is <IP_ADDRESS>, and even doing basic things like trying to install a package failed at the DNS resolution. For now, I've just replaced the symlink with a basic file that uses DNS servers I know to work, and it seems like there are some considerably more complex but probably more technically correct systemd approaches. (I'm coming from the FreeBSD world.)
Lastly, in case anyone else runs into this, if you're a tmux user, it seems like you have to start the VM from outside of tmux. If you start it inside tmux, it starts up okay, but as soon as you try to connect to tty using screen, it bails:
2020-12-03 14:30:09.666 vftool[8771:542552] +++ Configuration validated.
2020-12-03 14:30:09.667 vftool[8771:542552] +++ canStart = 1, vm state 0
2020-12-03 14:30:09.690 vftool[8771:542558] --- VM start error: Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start" UserInfo={NSDebugDescription=The virtual machine failed to start}

Starting the VM using the exact same command from outside of tmux works fine. (Attaching via screen from inside tmux works fine.)

Great work everyone!  Glad to see the various guides -- anyone wish to do a wiki page perhaps?
@pgib, that's a good find w.r.t. the startup failure (I hadn't been able to recreate it when it was reported before).  What I'm thinking regarding the console is to provide some other switch to fall back to using "stdio" for the console instead of creating a pty/requiring attaching to /dev/ttyblah.  This would give immediate startup and print output to the console, and might help this issue, but wouldn't really give a usable terminal -- so would be most useful once installed and after you can SSH in.  Or, as a fallback, acquire a pty but don't wait for a connection.
I'll also have a look at adding multiple discs, something I've wanted too.  May get some time at the weekend, hang in there.
Thanks for trying vftool and persisting, and helping each other.  👍
-M

So this all works and I can boot into it and make changes, but once I quit vftool everything is lost. I can still reboot into it, but any changes made aren't persisted. Is this intended? Any way to persist changes so I can turn this into an always-on daemon even after restart?

That’s because the initrd image that you’re using is hard coded to boot into the live version, to install, attach a block device, try it with Ubuntu server, extract the initrd image from the installation and use it, even then the Initramfs doesn’t support virtio block devices out of the box, you need to generate a new one to get it working.

@PraneetNeuro thanks for the tips, any guide / github / reference for doing that? I'm not super familiar with linux/virtio/intramfs at all.

Download and use the Ubuntu server image as it supports a non GUI based installation i.e command line installation
Install qemu-img through homebrew and create a disk image of desired size where you’d be installing the OS
You’ll have to modify the source to add a block storage device attachment which is off virtio type

Initrd is used to set up the initial RAM layout before the kernel is loaded onto the memory
Vmlinuz is the compressed version of the kernel which is initially loaded into the ram
Both of these images are available in the iso image of Ubuntu under the Casper directory but the initramfs provided with the iso doesn't support booting from a disk which has Ubuntu installed

Proceed with the given images of initrd and vmlinuz in the ISO for the installation, the installation will be successful

Mount the vdb2 partition of the disk and extract the initrd present at the root of the partition in any means of your choice, say SCP

This initrd will support booting from the disk but doesn’t have virtio driver support and won’t support the virtual disk that we have out of the box

Generate the initramfs with virtio block storage support from a usable Ubuntu server system or VM and use it as the initrd image

Make sure to pass in the argument root=/dev/vdb

I wrote an end-to-end guide on how to get vftool running with Docker on M1, which seems to be the goal of many in this thread. Here!

I wrote a blog post of my journey to a silicon container vm :)
Containerized the apple silicon

Check my repo out, definitely more user friendly and is a bit easier with regards to the disk management https://github.com/PraneetNeuro/Project-Mendacius

Hi,
I've started with the finestructure tutorial and ended up here. I also tried with the image, vmlinuz, and initrd posted in a previous comment and I am always getting a
--- VM start error: Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start" UserInfo={NSDebugDescription=The virtual machine failed to start}
My files:
focal-server-cloudimg-arm64.img: Linux rev 1.0 ext4 filesystem data, UUID=ede099f5-61df-4eb8-a811-c8aea1d942eb, volume name "cloudimg-rootfs" (extents) (64bit) (large files) (huge files)
initrd: LZ4 compressed data (v0.1-v0.9)
ubuntu-20.04-server-cloudimg-arm64.tar.gz: gzip compressed data, last modified: Tue Dec  1 16:59:03 2020, from Unix, original size modulo 2^32<PHONE_NUMBER>
vmlinuz: Linux kernel ARM64 boot executable Image, little-endian, 4K pages

Command:
./vftool -k vmlinuz -i initrd -d focal-server-cloudimg-arm64.img -m 2048 -a "console=hvc0"
Output:
2020-12-10 17:22:25.113 vftool[30069:402845] vftool (v0.1 25/11/2020) starting
2020-12-10 17:22:25.114 vftool[30069:402845] +++ kernel at vmlinuz -- file:///Users/rpvilao/Images/ubuntu/, initrd at initrd -- file:///Users/rpvilao/Images/ubuntu/, cmdline 'console=hvc0', 1 cpus, 1024MB memory
2020-12-10 17:22:25.114 vftool[30069:402845] +++ fd 3 connected to /dev/ttys002
2020-12-10 17:22:25.114 vftool[30069:402845] +++ Waiting for connection to:  /dev/ttys002
2020-12-10 17:22:40.058 vftool[30069:402845] +++ Attaching disc focal-server-cloudimg-arm64.img -- file:///Users/rpvilao/Images/ubuntu/
2020-12-10 17:22:40.058 vftool[30069:402845] +++ Configuration validated.
2020-12-10 17:22:40.058 vftool[30069:402845] +++ canStart = 1, vm state 0
2020-12-10 17:22:40.083 vftool[30069:402850] --- VM start error: Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start" UserInfo={NSDebugDescription=The virtual machine failed to start}

Any clues on what might be happening? Thanks!

@rpvilao Are you in a tmux session by chance?

@pgib thanks for replying. I am using a regular terminal session (zsh)

@pgib your question gave me an idea... I opened the terminal using rosetta (to make it work with brew), without rosetta it works thanks :)

I've enabled the "Discussions" GitHub feature which may be of use to capture some of the good info people have provided in this issue.  I've noticed a bunch of Linux setup questions posed as vftool issues -- strictly we should create issues only for vftool bugs/requests (otherwise bugs are likely to get lost).  I'm glad folks are helping each other out (thanks!), so the wiki/discussions should be a useful place to do this (and would make general Linux tips/info easier to find).

I'm unfortunate to run into
2020-12-12 14:20:53.002 vftool.bin[4205:81177] +++ fd 3 connected to /dev/ttys002
2020-12-12 14:20:53.002 vftool.bin[4205:81177] +++ Waiting for connection to:  /dev/ttys002
2020-12-12 14:20:56.184 vftool.bin[4205:81177] +++ Attaching disc vm/focal-server-cloudimg-arm64.img
2020-12-12 14:20:56.185 vftool.bin[4205:81177] +++ Configuration validated.
2020-12-12 14:20:56.185 vftool.bin[4205:81177] +++ canStart = 1, vm state 0
2020-12-12 14:20:56.211 vftool.bin[4205:81192] --- VM start error: Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start" UserInfo={NSDebugDescription=The virtual machine failed to start}

when attempting to connect with screen. no tmux or rosetta.
any hints to debugging this?

@shroomist iTerm? it uses tmux, if so ...

I have same errors with @shroomist. I tried the terminal app(default) without rosetta but still not running.

Any feels that the IO seed in the Ubuntu OS is too slow? It's shouldn't be related to the image.

Does anyone feel that the IO speed in the Ubuntu OS is too slow? It shouldn't be related to the image.

yep, definitely slower than it should be...
https://twitter.com/mibosshard/status/1337903596235616256

hey @rossbachp , Ive followed your guide - all works fine.
except, as soon as I try to use apt-get install I get
[code]
Setting up flash-kernel (3.98ubuntu12) ...
flash-kernel: deferring update (trigger activated)
Processing triggers for flash-kernel (3.98ubuntu12) ...
Unsupported platform.
dpkg: error processing package flash-kernel (--configure):
installed flash-kernel package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
flash-kernel
E: Sub-process /usr/bin/dpkg returned an error code (1)
[/code]
any thoughts?

Another way to resize image(without qemu):
cp ubuntu.img ubuntu2.img
cp ubuntu.img ubuntu3.img

truncate -s 40G ubuntu.img
attach ubuntu2.img as /dev/vda,  ubuntu3.img as /dev/vdb, and ubuntu.img as /dev/vdc
/dev/vdc is the 40GB img

vftool -k vmlinux -i initrd -d ubuntu2.img -d ubuntu3.img -d ubuntu.img -m 4096 -a "console=hvc0 root=/dev/vda init=/bin/bash"

#attach to the  tty using screen
...
mkfs.ext4 -L cloudimg-rootfs /dev/vdc
mount -o ro /dev/vdb /home
mount /dev/vdc /mnt
cp -ar /home/* /mnt/
umount /mnt /home

#shutdown the VM by killing the vtftool.

rm ubuntu2.img ubuntu3.img

Then we have the ubuntu.img in 40GB size.

I did it way simpler using dd to append 20 GB of zeros to the img file (truncate may work just as well), booted the VM, and used resize2fs to refisize the filesystem (live!).

I try the ubuntu server installer.

With screen attaching the pts, the terminal is unusable.
With arg "-t 0", if add cfmakeraw for STDIN_FILENO, the terminal works!

I try the ubuntu server installer.

With screen attaching the pts, the terminal is unusable.

With arg "-t 0", if add cfmakeraw for STDIN_FILENO, the terminal works!

Check out my repo project-mendacius

Following exaclty https://github.com/evansm7/vftool/issues/2#issuecomment-735455161 on Macbook Pro 2019 (Intel) using zsh. I get the error everyone else is getting -
➜  linux ../Projects/vftool/build/vftool -k vmlinux -i initrd -d disk.img -m 2048 -a "console=hvc0"

2020-12-25 22:41:09.230 vftool[8442:220049] vftool (v0.3 10/12/2020) starting
2020-12-25 22:41:09.231 vftool[8442:220049] +++ kernel at vmlinux, initrd at initrd, cmdline 'console=hvc0', 1 cpus, 2048MB memory
2020-12-25 22:41:09.231 vftool[8442:220049] +++ fd 3 connected to /dev/ttys001
2020-12-25 22:41:09.231 vftool[8442:220049] +++ Waiting for connection to:  /dev/ttys001
2020-12-25 22:41:11.551 vftool[8442:220049] +++ Attaching disc disk.img
2020-12-25 22:41:11.551 vftool[8442:220049] +++ Configuration validated.
2020-12-25 22:41:11.551 vftool[8442:220049] +++ canStart = 1, vm state 0
2020-12-25 22:41:11.586 vftool[8442:220073] --- VM start error: Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start" UserInfo={NSDebugDescription=The virtual machine failed to start}

Any help would be super appreciated!

I'm having the issue where the VM will not start as well.
I notice the following two sandbox errors just before vftool exits after I connect to the tty:
Sandbox: com.apple.Virtua(8285) deny(1) syscall-unix 220
Sandbox: com.apple.Virtua(8285) deny(1) syscall-unix 338
Could these be related?

I've found the solution for the problem with --- VM start error: Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start" UserInfo={NSDebugDescription=The virtual machine failed to start}.
I was trying to update the network configuration file sudo nano /Library/Preferences/SystemConfiguration/com.apple.vmnet.plist
The first vmnet had been <IP_ADDRESS>, after running vftool the IP address was increased to <IP_ADDRESS> and it started failing on running. When I've changed the IP to <IP_ADDRESS>, it started working again. Maybe you need to clean up the file cat /var/db/dhcpd_leases.
I suggest making backups before doing changes in those files.

I've found the solution for the problem with --- VM start error: Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start" UserInfo={NSDebugDescription=The virtual machine failed to start}.
I was trying to update the network configuration file sudo nano /Library/Preferences/SystemConfiguration/com.apple.vmnet.plist
The first vmnet had been <IP_ADDRESS>, after running vftool the IP address was increased to <IP_ADDRESS> and it started failing on running. When I've changed the IP to <IP_ADDRESS>, it started working again. Maybe you need to clean up the file cat /var/db/dhcpd_leases.
I suggest making backups before doing changes in those files.

I made a small makefile that does the bare minimum in downloading, extracting, and running vftool for ubuntu images, in case that's helpful.
Just a heads up - if your vm immediately exits, make sure you have downloaded the arm64 ubuntu image, not the amd64 ubuntu image if you are on an M1 mac. Spent an embarrassingly long time with that small issue.
https://gist.github.com/6fd8908936b30f19f8d1a025a40417ae

I made a small makefile that does the bare minimum in downloading, extracting, and running vftool for ubuntu images, in case that's helpful.
Just a heads up - if your vm immediately exits, make sure you have downloaded the arm64 ubuntu image, not the amd64 ubuntu image if you are on an M1 mac. Spent an embarrassingly long time with that small issue.
https://gist.github.com/6fd8908936b30f19f8d1a025a40417ae

Check out https://github.com/PraneetNeuro/Project-Mendacius allows you to download images based on the architecture and a user friendly gui based tool

Check out https://github.com/PraneetNeuro/Project-Mendacius allows you to download images based on the architecture and a user friendly gui based tool

Check out https://github.com/PraneetNeuro/Project-Mendacius allows you to download images based on the architecture and a user friendly gui based tool

Is there a way to make share folders between host and guest?

Check out https://github.com/PraneetNeuro/Project-Mendacius allows you to download images based on the architecture and a user friendly gui based tool

Is there a way to make share folders between host and guest?

For the people with the VM start error: Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start" UserInfo={NSDebugDescription=The virtual machine failed to start} error, It is possible that you have a problem with the files you provide, for example you forgot to gunzip vmlinuz
At least that was my problem :facepalm:

In case it helps anyone, as I was tearing my hair out a bit trying to get a consistent setup on Intel and ARM Macs, for our developers... it seems the intel vmlinuz file is compressed differently to the ARM one and does NOT need uncompressed before use.
If it helps anyone, here's the full steps for setting up an ubuntu 20.04 server on both intel and arm macs:
https://gist.github.com/mungler/c7f67bbb27ddbac01d30a1727c0346b5

This comment solved my issue
https://github.com/evansm7/vftool/issues/22#issuecomment-949458055

Hi, I am able to compile vftool and able to boot ubuntu-20.04.3-desktop-amd64.iso

I have met an error when I first start the VM by vftool, can you guys have some available solutions? THX!!
Error massge: " VM start error: Error Domain=VZErrorDomain Code=2 "


vscode extension can not parse the latest buf binary version

Reproduce

Install the latest buf binary (v1.0.0-rc1)
Install the buf-vscode extension
buf-vscode throws a error: Failed to get buf version: failed to parse version output: 1.0.0-rc1

Possible Fix
Why not use the standard semver regexp rules suggested by semver ?
Addressed here: https://regex101.com/r/vkijKf/1/

I can reproduce, we'll get this fixed soon.

We have just released a fix in v0.3.1.


Docs build failure with recently released nbsphinx 0.8.8

See https://dev.azure.com/LiberTEM/LiberTEM/_build/results?buildId=1542&view=logs&jobId=2e37e63e-629f-5d60-4506-5615d0d227d3&j=2e37e63e-629f-5d60-4506-5615d0d227d3&t=580c1cad-5a9c-5ef1-d5a7-d2e4b5865820. I can reproduce this locally.
A band-aid that blacklists nbsphinx 0.8.8 is contained in #1174.
For a proper fix we should create a minimal reproducer and fix our notebook or follow up with upstream.

Maybe we can postpone this until it (possibly) breaks again when the next nbsphinx release is out or we run into other issues with using 0.8.7 for now?

As a note, the concrete problem is fixed in https://github.com/spatialaudio/nbsphinx/commit/28184d2d952411922dbd6f157c99880b41f0f34b

As a note, the concrete problem is fixed in spatialaudio/nbsphinx@28184d2

Nice, so waiting out the next release is the correct strategy I guess? :-)

Yeah, blacklisting the broken release is the right move

Considered fixed, then! :-)


Dump CD-ROM Drive not compatible ?

Describe the bug
I can dump DVDs, DVD9s but not PS1 CD-ROMs. I guess my player is not compatible for CD-ROM.
TSSTcorp CDDVDW SH-224DB
DiscImageCreator.exe cd d MyGameTitle.bin 8 /c2 /nl
Log file
LOGDUMP.zip

I guess my player is not compatible for CD-ROM.

Yes, see README.md.


phoneNumber descriptions alignment to the other APIs

What type of PR is this?
Add one of the following kinds:

enhancement/feature

What this PR does / why we need it:
I checked the other APIs and '+' is optional but here it's required and it's better to align it so it's uniform.
Which issue(s) this PR fixes:
Fixes (https://github.com/camaraproject/NumberVerification/issues/42)
Special notes for reviewers:
@DT-DawidWroblewski @bigludo7 could you please review it.

@bigludo7 @DT-DawidWroblewski I also replaced the uml link with the one in main branch from this repo instead of the temporary one from my GIT repo. Please review.


[QUERY]Getting TokenCredentials to connect to azure resources for Desktop Applications

Hi All,
I am working on a .Net Core Desktop Application that needs to connect to Azure Blob Storage to download some files. To authenticate the requests to storage we need a token to be given during the blob service client creation and I am not sure what’s the correct way to get this token credential.
Using DefaultAzureCredential
Reading the documentation here it seems there is an option to use DefaultAzureCredential that looks for credentials in multiple places.. It is working fine for my local dev environment as it's getting the credential from VS Code/Azure CLI. But for prod, I assume that this approach is for applications running in Azure. However, my application is a desktop application. Can I still use this to acquire TokenCredential in the prod environment as well or is there anything else I need to set up to make sure it works fine for other users when distributed as a packaged application (I want to avoid user interaction as much as possible)?
Using ClientSecretCredential
Another approach to get the credentials is to use the ClientSecretCredential class by passing the teantId, clientId, and clientSecret for the registered App. Since clientSecret should not be stored in the source code, how can I get this value in my application? I thought of using a KeyVault for this but again the issue is how to get the TokenCredentials to connect to KeyVault?
or is there any other approach that I can use to get the TokenCredential to connect to Azure?
P.S: I am using Microsoft.Identity.Client to authenticate users with Microsoft account following the example here . Is there a way I can use the AccessToken retrieved here to get a TokenCredential for blob client?
code for reference
var blobClient = new BlobClient(new Uri(s_InstallerURI), DefaultAzureCredential);
                var blobServiceClient = blobClient.GetParentBlobContainerClient().GetParentBlobServiceClient();
                UserDelegationKey key = blobServiceClient.GetUserDelegationKey(DateTimeOffset.UtcNow, DateTimeOffset.UtcNow.AddHours(1));
                var sasBuilder = new BlobSasBuilder()
                {
                    BlobContainerName = blobClient.BlobContainerName,
                    BlobName = blobClient.Name,
                    Resource = "b",
                    StartsOn = DateTimeOffset.UtcNow,
                    ExpiresOn = DateTimeOffset.UtcNow.AddHours(1)
                };
                sasBuilder.SetPermissions(BlobSasPermissions.All);
                var blobUriBuilder = new BlobUriBuilder(blobClient.Uri)
                {
                    Sas = sasBuilder.ToSasQueryParameters(key, blobClient.AccountName)
                };
                var packageManager = new PackageManager();
                packageManager.AddPackageAsync(blobUriBuilder.ToUri(), null, DeploymentOptions.ForceApplicationShutdown).AsTask().Wait();

Any help is much appreciated. Thanks in advance.
Environment:

To connect to azure blob I am using  Azure.Storage.Blobs Version=12.9.1
Hosting platform - macOS and Windows using .Net5
IDE : Visual Studio 2019

Thank you for your feedback.  Tagging and routing to the team members best able to assist.

Hi @ShrutiJaiswal1494 - One clarification: Are you intending to connect to the storage account using a specific that you own identity rather than the identity of the user of the application? Because this is a desktop environment, I don't think there would be a way to protect your credential, no matter how it is acquired.
Have you considered using a SAS token instead? This would allow you to delegate constrained and timeboxed access to users of the desktop app.

Hey @christothes, Thanks for the reply. My application will be Microsoft internal. So I want everyone with Microsoft account to have access but I am not sure how to do it. For your suggestion of SAS token, I am already using a SAS with user delegation key to time-bound the access.
My only concern is how to get a TokenCredential to build a service client for different users when using the App. Otherwise, I will need to store the account name and key in the code which is not preferred.

If I build the service client based on the credentials which I own then what is the best pay to store those credentials in a desktop app?
If I build the client using their local credential like using DefaultAzureCredential how do I make sure that it works for them for cases when they are not logged into azure CLI/PowerShell or VS. ?
(I believe Environment variable and Managed identity is not an option for me as Managed identity requires the app to run on Azure and setting the env var on each user's desktop will disclose the secret. Correct me if I am wrong.)

If I build the service client based on the credentials which I own then what is the best pay to store those credentials in a desktop app?

There isn't a great way that doesn't compromise the credential, since it is a desktop app.

If I build the client using their local credential like using DefaultAzureCredential how do I make sure that it works for them for cases when they are not logged into azure CLI/PowerShell or VS. ?

You can't - that is a pre-requisite that you could handle in your app's error messages.

(I believe Environment variable and Managed identity is not an option for me as Managed identity requires the app to run on Azure and setting the env var on each user's desktop will disclose the secret. Correct me if I am wrong.)

Correct.

If I use the includeInteractiveCredentials with DefaultAzureCredential, will it still throw an error if the user is not logged in azure CLI/PowerShell or VS, or will it prompt the user to login?
I have verified all Microsoft users through Microsoft.Identity.Client. Is there a way to use the Access Token that I receive from Microsoft.Identity.Client to get a TokenCredential for Azure?

@ShrutiJaiswal1494 if your intent is to authenticate users in your app as you are doing directly with msal in the link you gave here. I would suggest using the InteractiveBrowserCredential, as it is the closest analogue to how you are authenticating with the PublicClientApplication. The InteractiveBrowserCredential will launch the system browser to authenticate the user in net core apps, or use the embedded browser to authenticate users in WPF or WinForms apps.
Also, if you wish to persist tokens across app executions you can enable this by using the TokenCachePeristenceOptions and AuthenticationRecord. Our Client Side User Authentiation sample walks through this scenario in detail.

I have verified all Microsoft users through Microsoft.Identity.Client. Is there a way to use the Access Token that I receive from Microsoft.Identity.Client to get a TokenCredential for Azure?

If you would rather continue using MSAL's PublicClientApplication to authenitcate users, it's fairly trivial to wrap this in a TokenCredential implementation. An example of how to do this can be found in our Authenticating with MSAL Directly sample. This sample details how to wrap the MSAL application object in a TokenCredential implementation. The sample shows wrapping the ConfidentialClientApplication, but wrapping PublicClientApplication should be fairly similar.

@ShrutiJaiswal1494 if your intent is to authenticate users in your app as you are doing directly with msal in the link you gave here. I would suggest using the InteractiveBrowserCredential, as it is the closest analogue to how you are authenticating with the PublicClientApplication. The InteractiveBrowserCredential will launch the system browser to authenticate the user in net core apps, or use the embedded browser to authenticate users in WPF or WinForms apps.
Also, if you wish to persist tokens across app executions you can enable this by using the TokenCachePeristenceOptions and AuthenticationRecord. Our Client Side User Authentiation sample walks through this scenario in detail.

Thanks for the reply @schaabs. I am thinking to use the InteractiveBrowserCredential approach to authenticate the users however I had some doubts. Since I want the blob storage to be accessed by everyone in Microsoft, will I need to give access to each and every MS employee to my blob storage? If yes, then is there a security group or something that I can give access to?


feat: support runtime configurations in workspace

What type of PR is this?

/kind feature
What this PR does / why we need it:
This PR adds supports for runtime configurations in workspace.
Which issue(s) this PR fixes:

Fixes #1192
Special notes for your reviewer:
Does this PR introduce a user-facing change?

Additional documentation e.g., design docs, usage docs, etc.:

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

42 of 140 (30.0%)  changed or added relevant lines in 11 files are covered.
11 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.2%) to 54.275%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/cmd/apply/apply.go
0
1
0.0%

pkg/engine/operation/graph/resource_node.go
1
2
50.0%

pkg/engine/runtime/init/init.go
0
3
0.0%

pkg/engine/runtime/terraform/terraform_runtime.go
3
6
50.0%

pkg/modules/generators/app_configurations_generator.go
6
27
22.22%

pkg/engine/runtime/terraform/tfops/workspace.go
27
55
49.09%

pkg/engine/runtime/kubernetes/kubernetes_runtime.go
0
41
0.0%

Files with Coverage Reduction
New Missed Lines
%

pkg/engine/runtime/terraform/tfops/workspace.go
1
58.39%

pkg/engine/operation/parser/parser.go
3
79.27%

pkg/engine/api/destroy.go
7
83.06%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
9369

Relevant Lines:
17262

💛  - Coveralls


Allow different nfs options for different mounts

Hey there,
It would be great if it was possible to use different NFS options for different shared folders.  For example, I want to share /Users with the standard "-alldirs -mapall=$(id -u):$(id -g)" setup, but I want to share another directory with "-alldirs -maproot=0".  This isn't currently possible, as all shared folders share the same options.  Any ideas?

Hey @rbayliss ,
I guess we could do something like this:
docker-machine-nfs test --shared-folder="/var/www,-alldirs -maproot=0" --nfs-config="-alldirs -mapall=\$(id -u):\$(id -g)"

The --nfs-config would be applied to all shared-folder on which a override config is not given. The override would be given in the --shared-folder config after a e.g. ',' separation ?

Yep, that would definitely do it.  Love that it doesn't break BC either.

Does this work currently?  I'm facing an issue a similar issue to @rbayliss where I need to share one directory with root permissions and others with normal permissions.


Application don't give error, but not showing, when i just added url_launcher: ^5.0.1 in pubspec.yaml file

C:\Windows\system32>flutter doctor -v
[√] Flutter (Channel stable, v1.0.0, on Microsoft Windows [Version 10.0.17763.316], locale tr-TR)
    • Flutter version 1.0.0 at C:\src\flutter
    • Framework revision 5391447fae (3 months ago), 2018-11-29 19:41:26 -0800
    • Engine revision 7375a0f414
    • Dart version 2.1.0 (build 2.1.0-dev.9.4 f9ebf21297)

[√] Android toolchain - develop for Android devices (Android SDK 28.0.3)
    • Android SDK at C:\Users\RT\AppData\Local\Android\sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
    • All Android licenses accepted.

[√] Android Studio (version 3.3)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin version 33.1.1
    • Dart plugin version 182.5215
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)

[√] VS Code (version 1.31.1)
    • VS Code at C:\Users\RT\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 2.23.1

[√] Connected device (1 available)
    • BLA L09 • GAB7N18423000073 • android-arm64 • Android 9 (API 28)

• No issues found!

Please consider asking support questions in one of the other channels listed at http://flutter.io/support .


missing Property test

Towards #477

Current coverage is 91.54%

Merging #816 into master will increase coverage by +0.02% as of b7d6591

@@            master    #816   diff @@
======================================
  Files           77      77
  Stmts         7815    7815
  Branches      1580    1580
  Methods          0       0
======================================
+ Hit           7153    7154     +1
+ Partial        202     201     -1
  Missed         460     460

Review entire Coverage Diff as of b7d6591

Powered by Codecov. Updated on successful CI builds.


[Digital Twin Kit] Release Checks

This issue tracks all relevant release checks for KITs.
Release Info
There won't be any normative changes to the DT domain in R24-03. That's why the kit is evolving, adding some more best practices, guidance how to interact with dataspace components and data provisioning.
Adoption View

[x] Vision and Mission statement present
[x] Business value statement present
[x] Semantic Model Standards submitted and published
[x] Business process documentation available
[x] Architecture documentation available
[x] Access and usage policies documented

Optional:

[x] Tutorials available
[x] White Paper available

Development View

[ ] API specification and protocols published on SwaggerHub
[x] Sample data available
[x] Reference implementation available
[x] Architecture documents available

Operation View

[x] Setup guide available

General Checks

[x] Developer journey approved
[ ] CHANGELOG maintained

Helpful Links

Catena-X standard library

Hi @arnoweiss, thank you for your ticket and PR. Looks great! Some clarifying questions / suggestions:
We could find information about the "Business Process" and "Architecture". Do you think we can highlight this sections clearer in the navigation?
Furthermore, we could not find any "Access and Usage Policies", "Whitepapers" (e.g. 101s), or "Tutorials", but the boxes are checked in this ticket. They are not mandatory for network services. So did we miss the links / documents or should we uncheck the boxes?

@arnoweiss
Friendly reminder - Please fix the requested changes by 04.03.2024 (EOB) so that we can perform the final checks and prepare the release communication. Thanks!

PR merged: https://github.com/eclipse-tractusx/eclipse-tractusx.github.io/pull/654


config: Removed deprecated variables for coverity module

This commit is to delete the deprecated variables that are used
for enabling the coverity module. And it is to append yellow and
red card instead of warning card for managing github repository
with a fine-grained approach.
Changelog

Added yellow and red card
Removed deprecated variables
Added annotation for config variables

Signed-off-by: Geunsik Lim<EMAIL_ADDRESS>
[Template] PR Description
In general, github system automatically copies your commit message for your convenience.
Please remove unused part of the template after writing your own PR description with this template.
$ git commit -s filename1 filename2 ... [enter]

Summarize changes in around 50 characters or less

More detailed explanatory text, if necessary. Wrap it to about 72
characters or so. In some contexts, the first line is treated as the
subject of the commit and the rest of the text as the body. The
blank line separating the summary from the body is critical;
various tools like `log`, `shortlog` and `rebase` can get confused
if you run the two together.

Further paragraphs come after blank lines.

**Changes proposed in this PR:**
- Bullet points are okay, too
- Typically a hyphen or asterisk is used for the bullet, preceded
  by a single space, with blank lines in between, but conventions vary here.

Resolves: #123
See also: #456, #789

**Self evaluation:**
1. Build test: [ ]Passed [ ]Failed [*]Skipped
2. Run test: [ ]Passed [ ]Failed [* ]Skipped

**How to evaluate:**
1. Describe how to evaluate in order to be reproduced by reviewer(s).

Add signed-off message automatically by running **$git commit -s ...** command.

$ git push origin <your_branch_name>

:memo: Version: 1.3.20190927. Thank you for submitting PR #576. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnsuite.mooo.com/.

INFO: You can read if there are syntax errors in the our shellcheck report. Please read http://nnsuite.mooo.com/TAOS-CI/ci/repo-workers/pr-checker/576-201911140819360.98327803611755-4aa4703b1e10d2c20defd9ac92dbc5c66b3a58cd/report/shellcheck_syntax_result.txt.


[Firefox] Slightly simplify how the generator is determined, when sending telemetry data from the viewer

When working on PR #11463 I couldn't help thinking that the Array.prototype.some callback function, used when determining the generator, was somewhat difficult to read with its partly unused and strangely named parameters.

/botio-linux preview

Thank you! I also had to re-read this a few times before I understood the old code; the new one is indeed easier to read.


Finetuning for onboarding darkmode

What

[ ] The next page button has too less opacity
[ ] The title is dark on dark on several pages
Note: you can reset onboarding quickly from the dev mode (see README to enable it)

Part of

#684
#561

Screenshot/Mockup/Before-After

I was looking the same bug and thought of raising an issue,I would be able to work on this
Assign me plz


Feature Request: Favorites

Is your feature request related to a problem? Please describe.
I would like to define some channels in an additional "favorites" group. Now, channels are sorted default by group. However, most of the times I am not interested in all the channels in the group. But, in some cases I want to view some due to specific content available in a channel I would usually not watch. It is annoying to always have to come back to the ui and enable a specific channel.
Describe the solution you'd like
Being able to map channels to a second group and/or have a checkbox to enable "favorite" channels.
Describe alternatives you've considered
I understand I can do this also in most clients but, I would like to have a solution that is more universal and doesn't depend on the client application in use.

I'm not understanding this. You can already add custom EPG categories and assign it to a channel. What else is being asked for?

You are totally right. Sorry!
Thanks anyways for all of your efforts!


Can't focus text fields in "Dialog > FULLSCREEN" demo in Gallery

Gallery, release build, Android, Pixel 2 XL.
Tap the Dialog demo item, then the FULLSCREEN button, then try to focus "Event name".
Right now, I get nothing. It's as if they're just Text fields.
cc @HansMuller who is working on text fields.

That widget is not an editable text field:
https://github.com/flutter/flutter/blob/master/examples/flutter_gallery/lib/demo/material/full_screen_dialog_demo.dart#L169

Haha, well that would certainly do it.

The "event name" field in the full-screen dialog demo is now editable and can be edited.


Added email login option

Default behavior is normal, from the app settings there are checkboxes to enable login via email/password, it can be set to username/password, email/password or both

Hi @laikmosh ! Thank you for your pull-request! I am sure this feature will help some other users. And I also decided to enable login via email in my current project. Many, many thanks!
I have added one commit in my develop-branch with a few bugfixes. Logging in via header & email did not work correctly.
I would be very happy if you could test out the branch again yourself. After that your pull request will end up in the next AppApi version 1.2.7!


Indicator width not rendered properly when the number of tabs are more

Hi
Thanks for the great view. However i ran into an issue when i had to increase to my tab count to 5. When the tab count is till say 3, the indicator(underline below the tab's text) works fine. But when the number of tabs are increased it still renders the width of the indicator equal to the width of the screen/number of tabs and not exactly covering the text of the tab header. Is there any props that can be passed for this functionality?

Do you have scrollEnabled on tab bar? And have you set width in tabStyle?

oops my mistake. Missed the scrollenabled on the tab bar. Thanks for your quick reply


[Event Request] Table 113 "Sales Invoice Line" - procedure CalcVATAmountLines()

    procedure CalcVATAmountLines(SalesInvHeader: Record "Sales Invoice Header"; var TempVATAmountLine: Record "VAT Amount Line" temporary)
    begin

----------------------- ADD EVENT ----------------------
        IsHandled := false;
        OnBeforeCalcVATAmountLines(Rec, SalesInvHeader, TempVATAmountLine, IsHandled);
        if IsHandled then
            exit;
----------------------------------------------------------

        TempVATAmountLine.DeleteAll();
        SetRange("Document No.", SalesInvHeader."No.");
        if Find('-') then
            repeat
                TempVATAmountLine.Init();
                TempVATAmountLine.CopyFromSalesInvLine(Rec);
                TempVATAmountLine.InsertLine;
            until Next = 0;
    end;
..............

[IntegrationEvent(false, false)]
local procedure OnBeforeCalcVATAmountLines(SalesInvLine: Record "Sales Invoice Line"; SalesInvHeader : Record "Sales Invoice Header"; var TempVATAmountLine : TEMPORARY Record "VAT Amount Line"; var IsHandled: Boolean)
begin
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.