Flex attention - gaps in profiler

Repost from the PyTorch forum
I have recently been playing with Flex attention, trying to replace some of my custom triton kernels. Unfortunately Flex attention ones were substantially (around 5x) slower than the custom one I tried. I use something similar to the Tri Dao flash attention kernel, but with bias. I started looking at the torch profile for a single forward pass on the complied version after the model warmup and it looks very strange.
import torch
from typing import Optional
from torch.nn.attention.flex_attention import (
    create_block_mask,
    flex_attention,
)
from torch.nn.attention.flex_attention import _mask_mod_signature
import random
from functools import partial
from triton.testing import do_bench
from torch.profiler import profile, ProfilerActivity, record_function

torch._dynamo.config.cache_size_limit = 1000
flex_attention = torch.compile(flex_attention, dynamic=False)

def flexattention_varlen_bias(
    query: torch.Tensor,  # shape (total_tokens_q, n_heads, head_dim)
    key: torch.Tensor,    # shape (total_tokens_k, n_heads, head_dim)
    value: torch.Tensor,  # shape (total_tokens_v, n_heads, head_dim)
    *,
    bias: Optional[torch.Tensor] = None,  # shape (total_tokens_q,)
    document_id: torch.Tensor,  # shape (total_tokens,)
    use_causal_mask: bool = False,
) -> torch.Tensor:
    device = query.device
    _, _, total_tokens, _ =  query.size()

    def additive_attention_fn(score, b, h, q_idx, kv_idx):
        return score + bias[kv_idx]

    def document_causal_mask(b, h, q_idx, kv_idx):
        causal_mask = q_idx >= kv_idx if use_causal_mask else True
        document_mask = document_id[q_idx] == document_id[kv_idx]
        return causal_mask & document_mask

    block_mask = create_block_mask(
        document_causal_mask,
        1,
        1,
        total_tokens,
        total_tokens,
        device=device,
    )

    return flex_attention(
        query,
        key,
        value,
        block_mask=block_mask,
        score_mod=additive_attention_fn if bias is not None else None
    )

if __name__ == "__main__":
    random.seed(0)
    device = "cuda"
    dtype = torch.float16
    causal = True
    max_seq_len, doc_count = 2048, 8
    nheads = 32
    HEAD_DIM = 128

    def generate_random_lengths(total_length, num_documents):
        lengths = [1] * num_documents
        remaining_length = total_length - num_documents
        for _ in range(remaining_length):
            index = random.randint(0, num_documents - 1)
            lengths[index] += 1
        return lengths

    lengths = generate_random_lengths(max_seq_len, doc_count)

    document_id = torch.zeros(max_seq_len, dtype=torch.int32, device=device)
    current_idx = 0
    for doc_id, length in enumerate(lengths):
        document_id[current_idx:current_idx+length] = doc_id
        current_idx += length

    query = torch.randn(1, max_seq_len, nheads, HEAD_DIM, device=device, dtype=dtype)
    key = torch.randn_like(query)
    value = torch.randn_like(query)
    bias = torch.rand(max_seq_len, device=device, dtype=dtype)

    flex_attention_fn = partial(flexattention_varlen_bias,
        query,
        key,
        value,
        bias=bias,
        document_id=document_id,
        use_causal_mask=causal
    )

    benchmark_time = do_bench(flex_attention_fn, warmup=25, rep=100)
    print(f"Benchmark time: {benchmark_time:.4f} ms")

    with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA], record_shapes=True) as prof:
        with record_function("flexattention_varlen_bias"):
            out = flex_attention_fn()
    prof.export_chrome_trace("flexattention_trace.json")

    print("Output shape:", out.size())
    # print("Document ID tensor:", document_id)
    print("Lengths of each document:", lengths)

The profile:

There are a lot of gaps, it looks as if the flex attention was not properly compiled. Do you have any ideas how to reduce these? I tried a bunch of different compilation modes but unfortunately to no avail.

can you try flex_attention = torch.compile(flex_attention, dynamic=False, fullgraph=True) which will raise warnings with reasons why it can't be compiled

it doesn't raise the warnings, and it is still super scattered

The problem is that create_block_mask is not being compiled
https://gist.github.com/drisspg/4cc2996497e34ce19d3a445dca2147e4
I think one problem with your setup though is that you are creating the block mask at every call to flex attention where this should typically be passed in so that you can amortize the block mask creation over the N transformer Blocks in a typical transformer.

When I do a token-by-token generation I will always need to generate a new block mask right for every call won't I? There seems to not be a good way around it. I see how it is quite wasteful, but we will need to modify the document_id for every call we do to the function call that we do, resulting in us needing to recompile it for every run. Or am I mistaken?

*I deleted the above comment cause it was wrong
Regarding your comment, since for every call we do different document_id tensor, it  is kind of inevitable that we will have to recompile it, right? I struggle to see an optimization we could do here

@drisspg one more question, do you have any idea why
cbm = torch.compile(create_block_mask, dynamic=False, fullgraph=True)
    block_mask = cbm(
        document_causal_mask,
        1,
        1,
        total_tokens,
        total_tokens,
        device=device,
    )

compiles fine on the pytorch experimental '2.6.0.dev20241015' but it does not compile on the stable '2.5.1+cu124', with an error. For both of them the pytorch cuda version is 12.4 (torch.version.cuda). Running on 4090.
File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 2150, in run_node
    raise RuntimeError(make_error_message(e)).with_traceback(
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_dynamo/utils.py", line 2132, in run_node
    return node.target(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch._dynamo.exc.TorchRuntimeError: Failed running call_function <built-in function getitem>(*(FakeTensor(..., device='cuda:0', size=(8192,), dtype=torch.int32), BatchedTensor(lvl=3, bdim=0, value=
    FakeTensor(..., device='cuda:0', size=(8192,), dtype=torch.int64)
)), **{}):
vmap: It looks like you're calling .item() on a Tensor. We don't support vmap over calling .item() on a Tensor, please try to rewrite what you're doing with other operations. If error is occurring somewhere inside PyTorch internals, please file a bug report.

from user code:
   File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/nn/attention/flex_attention.py", line 850, in create_block_mask
    partial_block_mask, full_block_mask = inner_func(
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/nn/attention/flex_attention.py", line 775, in _create_block_mask_inner
    mask_tensor = create_mask(mask_mod, B, H, Q_LEN, KV_LEN, device, _compile=True)
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/nn/attention/flex_attention.py", line 755, in create_mask
    mask = mask_mod(b, h, m, n)
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/apis.py", line 203, in wrapped
    return vmap_impl(
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 331, in vmap_impl
    return _flat_vmap(
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 479, in _flat_vmap
    batched_outputs = func(*batched_inputs, **kwargs)
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/apis.py", line 203, in wrapped
    return vmap_impl(
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 331, in vmap_impl
    return _flat_vmap(
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 479, in _flat_vmap
    batched_outputs = func(*batched_inputs, **kwargs)
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/apis.py", line 203, in wrapped
    return vmap_impl(
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 331, in vmap_impl
    return _flat_vmap(
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 479, in _flat_vmap
    batched_outputs = func(*batched_inputs, **kwargs)
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/apis.py", line 203, in wrapped
    return vmap_impl(
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 331, in vmap_impl
    return _flat_vmap(
  File "/home/piotr_mazurek/miniconda3/envs/luminous-inference-test/lib/python3.12/site-packages/torch/_functorch/vmap.py", line 479, in _flat_vmap
    batched_outputs = func(*batched_inputs, **kwargs)
  File "/home/piotr_mazurek/luminous-inference/src/luminous_inference/kernels/flex_attention_varlen_bias.py", line 33, in document_causal_mask
    document_mask = document_id[q_idx] == document_id[kv_idx]

You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True

I1114 15:55:28.448000 2178885 site-packages/torch/_dynamo/utils.py:399] TorchDynamo compilation metrics:
I1114 15:55:28.448000 2178885 site-packages/torch/_dynamo/utils.py:399] Function                  Runtimes (s)
I1114 15:55:28.448000 2178885 site-packages/torch/_dynamo/utils.py:399] ----------------------  --------------
I1114 15:55:28.448000 2178885 site-packages/torch/_dynamo/utils.py:399] _compile.compile_inner

ohh sorry we recently made updates (not landed in 2.5.1) to enable compile for create_block_mask without needing to call the _compile=True private func


Pass pixscale to ellipse_to_plot

ellipse_to_plot was never finished. Since we pass pixscale in everywhere else, it should be passed instead of wcs.

@e-koch don't deprecate older versions of numpy - we actually have to support np1.7 to support CASA  (!!!! =( !@#$!@#$)
(see #312)

Well crap. Alright.

@e-koch my fight with CASA will likely be to the death.  I think the changes you're making should be fine - what was going wrong with older versions of numpy?  Let's try to find a workaround here rather than wait on CASA to wake up

Ah, this shouldn't be here! https://github.com/radio-astro-tools/radio_beam/blob/master/radio_beam/tests/data/Makefile
There shouldn't be any conflicts without this.

No, the most important is numpy 1.7 + python 2.7, which is what CASA <= 5.0 runs on


singularity: 4.0.3 -> 4.1.0

Description of changes
Changelog: https://github.com/sylabs/singularity/releases/tag/v4.1.0
cc @jbedo @ShamrockLee
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

singularity.passtrhu.tests builds successfully on x86_64-linux.

Result of nixpkgs-review pr 283786 run on x86_64-linux 1

4 packages built:

singularity
singularity-overriden-nixos
udocker
udocker.dist

singularity.passtrhu.tests builds successfully on x86_64-linux.

Also successful on aarch64-linux.

Result of nixpkgs-review pr 283786 run on aarch64-linux 1

4 packages built:

singularity
singularity-overriden-nixos
udocker
udocker.dist


hybrid k8s federation with azure: cluster offline (dial tcp: i/o timeout)

Is this a request for help?: yes
I'm running a Kubernetes federation on-premise, using CoreDNS. The host cluster runs v1.7.1. I've got this container cluster in Azure, v1.6.6, and I'm trying to attach it to the federation. I could join the cluster, but it shows as Offline when I list the members of the federation.
I enabled RBAC in the Azure cluster, so I could create clusterroles and bindings.
I can get and create resources in the Azure cluster (using kubectl). I can access the API via curl. But it seems that my federation controller-manager cannot contact Azure cluster's API.
Logs from the Azure kube-apiserver show periodically:
I0727 07:58:05.787112       1 logs.go:41] http: TLS handshake error from <IP_ADDRESS>:52693: EOF
Logs from my Federation controller-manager show periodically:
E0727 08:33:06.464839       1 cluster_client.go:145] Failed to list nodes while getting zone names: Get https://myk8scluster-myresourcegroup-25ac51.eastus.cloudapp.azure.com/api/v1/nodes: dial tcp: i/o timeout
W0727 08:33:06.464884       1 clustercontroller.go:221] Failed to get zones and region for cluster azure: Get https://myk8scluster-myresourcegroup-25ac51.eastus.cloudapp.azure.com/api/v1/nodes: dial tcp: i/o timeout

And the cluster keeps Offline:
$ k get clusters --context=fed
NAME               STATUS    AGE
azure              Offline   16h
clenimar-kube-c1   Ready     20h
clenimar-kube-c3   Ready     20h
gke                Ready     20h

Any thoughts?
Thank you very much.

I believe there were multiple fixes for RBAC with cluster federation in 1.6.7. (https://github.com/kubernetes/kubernetes/pull/48077). It's probably worth deploying 1.6.7 and determining if your issue still remains

makes sense, I'll give it a go. @lachie83 is there an easy way to upgrade it? or specify the version while deploying a new cluster? searched a bit and it seems that Azure doesn't provide this functionality and I have to do the upgrade manually.


Support project references

Description
TypeScript supports project references, which is commonly used in monorepos to build all packages with a single command, and speeds up compilation in large projects. We should add support for this in TS Bridge as well.

Looks like this may be quite complicated to do in a performant way. TypeScript has a createSolutionBuilder API to create a program that builds all referenced projects, but:

There doesn't seem to be a way to reuse it for building both CommonJS and ESM, since it thinks it has built the project already and doesn't need to emit again. We can create separate solutions, but that means type checking twice.
It doesn't expose a type checker which we need to apply transformations, so we have to initialise a regular program separately, which will perform type checking in addition to the two solutions (so three times in total?).

For the initial implementation we could just manually loop through the referenced projects and build them like normal. Suggestions on how to use createSolutionBuilder would be appreciated too!

Does this mean parsing project references in tsconfig files manually, or were you envisioning something else?

TypeScript's parseJsonConfigFileContent function already does this for us. We need to loop through the projects and build each one individually. The only issue is that TypeScript assumes that the files are built as .js (assuming .ts source file), but we can work around this by modifying compilerOptions.paths to point .js to .cjs or .mjs. Not ideal, but I don't have a better solution right now.

Fixed by #30 and #32. There are some improvements we could make (like parallelisation for performance improvements), but at least project references are working.


Add schema to DLP and make DLP responsive

Fixes #163.
Should be merged after #173, as it's branched off from that. Until that PR is merged, the diff here will look large, because it contains those too. The only relevant changes here are in DandiMetaViewer.vue.
This also makes the DLP more responsive, so it should look better on smaller pages.

@satra can you QA this from the perspective of looking at the UI, and does it show what you want?
e.g., https://deploy-preview-175--gui-dandiarchive.netlify.com/#/dandiset-meta/5e6ff4140fe0ec33e6b3d5d2
We can skip code review for now because we are close to rewriting all this.

@mgrauer - this addresses the primary thing, which is to list the sections that have information. however, showing json in sections is quite suboptimal from a presentation standpoint, so hopefully there is an intermediate state between this and the full ui redesign.
i have some things to finish by sunday, but i can take a crack at it after i do.
so i'll merge this in so things are aligned and @bendichter  can start editing some of the metadata.


Update password policy to min length 16

We have raised an issue with AWS to follow (or rather, add features allowing us
to follow) the password guidance from NCSC:
https://www.ncsc.gov.uk/guidance/password-guidance-simplifying-your-approach
AWS doesn't communicate the password policy to users when changing password
which makes the complexity rules particularly unpleasant.
Until then we should follow what we can and remove the complexity requirements
and lower the minimum length.

If we're removing most of the constraints, should we remove them entirely from the policy rather than leaving them in as false?


middlewareAPI dispatch

https://github.com/reactjs/redux/blob/master/src/applyMiddleware.js#L27
hi, I don't understand here, what's the difference between function (action) => dispatch(action) and function dispatch. They both accept action and return whatever comes from dispatch, right? Can we just write as following? Thanks.
var middlewareAPI = {
  getState: store.getState,
  dispatch: dispatch
}

It's to establish a closure so that when dispatch gets swapped out later on, you're referencing the right function.


Cannot run example scene (Mac)

Upon installing the software through Brian Howell's tutorial (http://bhowell4.com/manic-install-tutorial-for-mac/) I ran the following code on Terminal to test run manim:
python -m manim example_scenes.py SquareToCircle -pl
but the Terminal on my Mac gives the following response:
/Documents/manim/venv/bin/python: No module named manim.main; 'manim' is a package and cannot be directly executed
I'm terribly unfamiliar with coding so would really appreciate help on this one x

The tutorial http://bhowell4.com/manic-install-tutorial-for-mac/ is dated from 2018. Then the syntax was: python3 extract_scene.py example_scenes.py SquareToCircle -pl.
Since, manim was reorganised and the syntax is python3 manim.py example_scenes.py SquareToCircle -pl.

And now the system says this, once I typed in the new code:
python3: can't open file 'manim.py': [Errno 2] No such file or directory
Before activating the virtual environment I've changed the path to the manim folder already. I can find the manim.py file in the manim folder, but can't get the video to play...any clue?


Allow timestamp with timezone on mssql databases

The SQL Server supports the following date / time data types:
date (Transact-SQL)
datetime (Transact-SQL)
datetime2 (Transact-SQL)
datetimeoffset (Transact-SQL)
smalldatetime (Transact-SQL)
time (Transact-SQL)
In my opinion, timestamp can be represented by (datetime, datetime2, datetimeoffset and smalldatetime).
Regarding the docs and integration tests, I want to wait for the reply about the incompatibility between dialects on the parameter for timestamp function on schema builder.

Code seems to be fine, this is so simple change that I don't require an integration test for it now... However documentation PR is still needed before this can be merged.

You mean the PR on knex/documentation repo?
I just did it.
https://github.com/knex/documentation/pull/128
Please check for me.
Thanks

Thanks!


Pluggable Providers: Automate Amazon

Move ServiceModels lib/miq_automation_engine/models to gems/manageiq-providers-amazon/lib/miq_automation_engine/models
Move datastore db/fixtures/ae_datastore/ManageIQ to gems/manageiq-providers-amazon/db/fixtures/ae_datastore/Amazon
Seed the Amazon datastore

@miq-bot assign durandom
@miq-bot add_label providers, providers/amazon
@mkanoor can we have a chat about this?

Had discussions with @durandom and we agreed to create a separate domain for Amazon and move the Automate Instances and Methods into that domain, which can be seeded and locked. The same would happen with the service models.

its all in manageiq-content now


(bug) Case sensitive in tab search

I've been using another workflow to search through Chrome and Safari tabs for quite a few years now and I've always hated how slow it was. This workflow is 10x faster which is insanely great!
Unfortunately, it's nearly useless to me without being case-insensitive.
It does seem to work sometimes, so I'm not sure if it's a bug or just partially case-insensitive or what, eg. maybe the URL is case insensitive but the tab title isn't? Regardless, I've found quite a few cases where it doesn't work case-insensitively, primarily YouTube which doesn't use stubs or otherwise have anything useful to search in their URLs, so search terms generally only match tab titles.
To reproduce:

Go to https://www.youtube.com/watch?v=0EqSXDwTq6U
Activate Alfred
Type "cht Charlie"
Notice "Charlie bit my finger! ORIGINAL..." appears as a result.
Backspace to change Alfred text to "cht charlie" - with lowercase c in charlie
Notice no results appear.

If this isn't 100% reproducible let me know and I'll provide system info, etc.
Thanks!

Thanks for your detailed report!
This bug is fixed now (on 0.2.8)

Thanks so much for the quick reply and fix!
I do have a thought here. It's a small bug most likely, but I can't test it since 0.2.8 seems to have broken ./alfred-chromium-workflow on my machine. I created a new issue for that: #9
Sorry for all the trouble!


Atari100k, DER, and OTRainbow

I just have not yet gotten the results for the OTRainbow.
Please review the PR, but don't merge it until the I make the results ready.

Based on my experiments (26 games all 5 seeds), we have:
DER
DER baseline, mean: 0.28543676671224993, median: 0.16136157795467734
DER RLHive, mean: 0.29438721585036576, median: 0.1904422041798461
** OTRainbow**
OTRainbow baseline, mean: 0.26407827599681083, median: 0.2036846538468431
OTRainbow RLHive, mean: 0.2385813492674032, median: 0.1378254499932332

I think this may have been a problem with how terminal states were being handled before. Should be fixed now? Could you try rerunning with latest code?


Conversion from Janet to 64bit integer don't match Janet logic

The TryFrom<Janet> implementation for i64 and u64 only tries to get value from the Janet value if the it is an abstract type.
While the internal Janet implementation can get if the Janet value is a number (f64/double), a JanetString or Janet Abstract

Fixed on ae5e3c39e350954fe20a140cf2ca2baabc5be5a8


test: add v1.15.0-beta.0 to the CI

Signed-off-by: André Martins<EMAIL_ADDRESS>

This change is

test-missed-k8s

Coverage increased (+0.01%) to 41.787% when pulling 96da427e2d8067511e893b3fb6ce87ada5699dc3 on pr/add-k8s-1.15 into 22970ea50bafe060f2fa032ccd58e3651e81f759 on master.

test-me-please

test-me-please

@aanm why does this need backporting?

@aanm why does this need backporting?

We decided that we need to support new releases of k8s versions on already cilium releases.


Fix Brazil subdivisions holidays

Proposed change

Fix Brazil subdivisions holidays (requesting unsupported categories should return an empty dict now).
This problem was raised in #1559 ("Strange behaviour 3.1").
Type of change

[ ] New country/market holidays support (thank you!)
[ ] Supported country/market holidays update (calendar discrepancy fix, localization)
[ ] Existing code/documentation/test/process quality improvement (best practice, cleanup, refactoring, optimization)
[ ] Dependency update (version deprecation/upgrade)
[x] Bugfix (non-breaking change which fixes an issue)
[ ] Breaking change (a code change causing existing functionality to break)
[ ] New feature (new python-holidays functionality in general)

Checklist

[x] I've followed the contributing guidelines
[x] I've added references to all holidays information sources used in this PR
[x] The code style looks good: make pre-commit command generates no changes
[x] All tests pass locally: make test, make tox (we strongly encourage adding tests to your code)

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

27 of 27 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 99.978%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
10275

Relevant Lines:
10275

💛  - Coveralls


Provide an option to crash the remultExpress server if ensureSchema fails

Is your feature request related to a problem? Please describe.
When deploying to GCP, pods can't be loaded with dependencies on other pods - so the server might load before the DB instance and ensureSchema will fail silently. This scenario is better treated with exit 1 so the pod will restart itself until success.
Describe the solution you'd like
Add an optional option to remultExpress (defaults to false) that if true, will crash the server when ensureScheme fails.

checkout<EMAIL_ADDRESS>

@sefininio can you check it out and let me know if it's ok?

Oh, I completely missed the previous comment (on the same freaking day! - amazing)
I'll try to bring up an env where this can be tested. Thanks!

Yes, that works!
I now get failed ensure schema of players  Error: connect ECONNREFUSED which is awesome , thanks man!

Great - closing the issue


fix(tools): update element internals polyfill

What I did

update the element internals polyfill to a version which fixes click-on-label
copy the polyfill to the web root when building 11ty
load the polyfill on all docs pages

fixes #2214
Testing instructions
load /components/switch in safari
Notes to reviewers
I'm loading the polyfill on all pages, this could be more carefully scoped by examining the page data in _head.njk, but that seemed excessively precious to me.

@bennypowers - I ended up doing the same thing when I was looking at it. pfe-switch could end up on any page so might as well load the polyfill on all of the pages.

Lollipop Goodbar Twizzler Gobstoppers 🍭

@heyMP - Dude... LGTG??? Looks good to go?

Lollipop Goodbar Twizzler Gobstoppers 🍭

@heyMP - Dude... LGTG??? Looks good to go?

M&Ms feel slighted

LOLOL... I told you I'm bad at these🤣


Error on EXOAcceptedDomain when type is "Internal Relay"

Details of the scenario you tried and the problem that is occurring
During création of MOF file, I have an error with Exo Accepted Domain of "Internal Relay" type
Verbose logs showing the problem

Suggested solution to the issue
Add Internal relay in Domain Type accepted
The DSC configuration that is used to reproduce the issue (as detailed as possible)
    EXOAcceptedDomain e4bcc237-1998-4a3a-8b5e-3e74ccdea6f0
    {
        DomainType           = "InternalRelay";
        Ensure               = "Present";
        GlobalAdminAccount   = $Credsglobaladmin;
        Identity             = "ttt.ttt";
        MatchSubDomains      = $False;
        OutboundOnly         = $False;
    }

The operating system the target node is running
Windows 10 and Windows Srever 2019

PR merged. Closing


Symphony Connector merger of two connectors

JIRA issue created by: rcook
Originally opened: 2012-09-05 11:49 AM
Accept Item will be most problematic service to merge.  Check in and Check out probably could be done by Adam but Adam has no way to test.  Lookup User is already done and Adam’s version covers more functionality than Michelle’s.
Michelle can evaluate what would be involved in a “merger” for Accept Item.  Michelle can perform her evaluation against a checked in code base branch in 1.0 version of Toolkit.  Before moving forward and coding though, she would need latest code from Adam.  This would be checked in as a branch in the 1.1 code base.   The first release candidate for this connector though would be in the 1.2 release.  Randy to create Jira issues for this.
Michelle would be willing to test the ILL type functions.
Notify to [~adam_constabaris]

JIRA User: rcook
JIRA Timestamp: 2012-10-24 12:10 PM
NC State/Lehigh merger - NC State on connector 1.0, version behind, so Michelle can't do a code merge
Per Adam - Need to install Perl part of code on dev Sirsi instance and get that part working, and then process inside the connector. The inside the connector work is for Adam (map the output) and the Perl test side is for Michelle.
Michelle to get a range of perl output to Adam so he can do the JSon sample cgi output mapping in the connector.
Notify [~adam_constabaris]

JIRA User: rcook
JIRA Timestamp: 2012-12-12 11:32 AM
Michelle is trying to install the NC State script.  Has questions for [~adam_constabaris].

JIRA User: adam_constabaris
JIRA Timestamp: 2012-12-18 01:17 PM
I suggest listing the questions here, because they might help with documentation.

JIRA User: rcook
JIRA Timestamp: 2013-04-16 11:33 AM
Based on discussion 4/16/2013, will not do this.


Adding MLIR cpp backend integration implementation

This is follow up to #635

This PR presents the cpp implementation for roadmap at https://github.com/ROCmSoftwarePlatform/llvm-project-private/issues/37. Implementation details are as follows:

Use Jenkinsfile to pull in MLIR as a source component

Added cmake logic to pull in libMLIRMIOpen

Implemented three solvers one each for a independent direction
Defined the interface needed for MLIR cpp backend
Implemented the infrastrcture need to build MLIR kernels
Added new test stage that specifically test mlir solvers

Added new test cases to test each solver

Status update:

Pipeline has run through 90% of stages without an issue. Only failure from MIOpenTensile stage is due to an irrelevant failure: Cannot contact MI100-5. I have triggered to rerun now.

Including the most important one: Hip Implicitgemm layouts, here.

Some fairly trivial solver registration conflict show up in solver.cpp because I also added new solvers. I will be addressing this conflict together with new review feedback, if there is any.

Status update:

Pipeline has run through 90% of stages without an issue. Only failure from MIOpenTensile stage is due to an irrelevant failure: Cannot contact MI100-5. I have triggered to rerun now.

Including the most important one: Hip Implicitgemm layouts, here.

Some fairly trivial solver registration conflict show up in solver.cpp because I also added new solvers. I will be addressing this conflict together with new review feedback, if there is any.

@jerryyin Looks like additional conflicts. Please fix, and we will prioritize.

@jerryyin Looks like additional conflicts. Please fix, and we will prioritize.

@daniellowell I did a merge instead of rebase to preserve the CI result at e41093d. As you can see, the merge conflict commit is fairly trivial. I believe this PR is ready for review now.

@daniellowell I did a merge instead of rebase to preserve the CI result at e41093d. As you can see, the merge conflict commit is fairly trivial. I believe this PR is ready for review now.

@jerryyin Let me make some changes in this branch. Please dont update it to avoid merge conflicts.

@atamazov Sounds good, thanks!

@jerryyin Redesign is still in progress. Please review my changes. I would like the rationale and design of all the changes I make to be crystal clear to you.

@jerryyin Indeed, these MLIR solvers and non-tunable (all the kernels generated during tuning are equivalent). Worse, these seem to fail if tuning is used, see attached log (ROCm 4.0, Radeon VII, build of commit ce75718e0). The reason is NOT clear to me. I guess it is important to understand why this happens, but please judge yourself.
To reproduce, checkout ce75718e0 (your original PR + fresh develop) and:
$ MIOPEN_FIND_ENFORCE=4 \
MIOPEN_LOG_LEVEL=6 \
MIOPEN_DEBUG_FIND_ONLY_SOLVER=ConvHipImplicitGemmMlirCppFwd \
./bin/MIOpenDriver conv -n 64 -c 1024 -H 14 -W 14 -k 1024 -y 1 -x 1 -p 0 -q 0 -F 1 -w 2 -t 1 -V 0

I will make the solvers non-tunable.
mlir-fwd-mlir-tuning-ce75718e0.log.txt

In the original HIP solver both grid size and HIP kernel depend on PerformanceConfig, but in the MLIR solver, the HIP kernel remains the same (PerformanceConfig does not change the kernel). This is the reason of the issue, I think.

@jerryyin

I would like the rationale and design of all the changes I make to be crystal clear to you.

I highly recommend reviewing commit-by-commit. Read commit descriptions. Do not hesitate to ask if something is unclear.

@jerryyin "Reverse" review is done. Now you own this branch.

@jerryyin Please review my changes, https://github.com/ROCmSoftwarePlatform/MIOpen/pull/676#issuecomment-782938022. As soon as you finish, I'll approve the PR.

@jerryyin

In theory, since they are just similar kernels if one of those can finish then rest of them should too.

In the original HIP solver both gridsize and kernel change during tuning, see https://github.com/ROCmSoftwarePlatform/MIOpen/pull/676#issuecomment-782928831. The same kernel run with too big grid leads to memory access fault (out-of buffer memory access).

The same kernel run with too big grid leads to memory access fault (out-of buffer memory access).

That's right. I think I find out what the real issue is, that MLIR cflags generation does not change with different tunable params, for example, blocksize, M/N/KperBlock, M/NperThread. The reason of that is because I never thought about it is necessary to take those Performance config parameters into MLIR (MLIR decide on the best parameter on its own, following the similar logic to EuristicInit(). No wonder every time it landed on the same tuning parameters, therefore causing the memory access fault.

There's a flat version number. I will bump the version number each time there is an API change

:+1:

I get rid of the redundant mlir namespace in the exported header

Hmm. I highly recommend resolving https://github.com/ROCmSoftwarePlatform/llvm-project-mlir/pull/61#pullrequestreview-602738329 and updating this PR.

Converted to draft to avoid spoiling docker image cache on CI. You can continue work as usual. But please change this back to "normal" PR only after ending of CI moratorium (see mail). Then first merge from develop. Push the [Ready for review] button only after that. Thanks.

@JehandadKhan is it allowed to go?

@jerryyin cmake 3.14.4 is required but our CI is running on 3.10.2. This PR should include installation of cmake >= 3.14.4 to the hip-clang.docker. Please consult with @pfultz2 and make necessary change.

@pfultz2 Is it possible to install the latest cmake following instructions from here:
Basically
# Add kite distribution to install latest cmake
RUN wget --no-check-certificate -qO - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | apt-key add -
RUN apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'

# cmake installation

This is how MLIR repo make sure we always install the latest cmake package.

@jerryyin

Is it possible to install the latest cmake

I do not think so. Also I recommend using mail for this discussion.

@atamazov One thing I have noticed that is that building MLIR repository can result in following:
virtual memory exhausted: Cannot allocate memory, here.
It has never occurred in our dev machine before. I'm wondering if:

This rocmtest13 node is not configured with SWAP file correctly, causing the compiler to OOM.
Should we limit the number of threads in compilation to like var1=$(nproc) var2=4 make -j( $var1 / $var2 ) in compiling the MLIR target?

virtual memory exhausted: Cannot allocate memory, here.

Yep, I've seen that too in your job.

* This `rocmtest13` node is not configured with SWAP file correctly, causing the compiler to OOM.

This is small SWP but 62GB of MEM installed which should be enough for machine that has nproc=32.

* Should we limit the number of threads in compilation to like `var1=$(nproc) var2=4 make -j( $var1 / $var2 )` in compiling the MLIR target?

Yes. However I've built MLIR on my local machine without any limitation, while it has nproc=32 and only 16+32GB mem+swp.

I gained access to ROCmTest13, tested the change and verified that the build went through

Good!

If you would like to reserve some cores, you can use something like -j `expr $(nproc) / 2` or -j `expr \`nproc --ignore 4\`` etc.

@atamazov @pfultz2 FYI the entire pipeline has run through: here.

@atamazov

Please merge your changes into Dockerfile

Done in previous commit, 5bdb3ca

Remove hip-clang-docker from this branch.

Somehow I missed this when resolving the conflict. Addressed in 6abdaf0

@jerryyin :tada:

@atamazov It's been a pleasure, I will try to shrink the PR size to make it more manageable in the future.


'Hello World' exercise is missing header file when downloaded via the CLI

When downloading via the CLI, the project is missing hello_world.h but CMakeLists.txt is still looking for it.
ref: https://github.com/exercism/cpp/blob/26352eb5dccf8553228add904ae6aca227e3b07f/exercises/practice/hello-world/CMakeLists.txt#L35
I think it could be due to changes to config.json in commit 7e75421
Completing the challenge via the online editor required a 1 one line change, but via the CLI I had to create a header file in order to build and run the tests.

I think you're completely correct, I've opened a PR to fix it. Looks like it was just a misunderstanding when that change was made.
Thanks for reporting. In the future you're welcome to setup the PR yourself as well :)


E0001 for a validate template in latest version but not 0.21.0

cfn-lint version: (0.21.1)
Description of issue.
With this cfn template:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  ServerLamdaRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
        Version: 2012-10-17
      Path: /
    Type: AWS::IAM::Role

We got the following error using latest version(0.21.1):
E0001 Error transforming template: datetime.date(2012, 10, 17) is not JSON serializable
cfn.yml:1:1

However, when using cfn-lint(0.21.0), this template is correctly validated:
{𝞿/RO-1092⁁ncy? ~/.xia/sha/sta/sam}cfn-lint --version && cfn-lint cfn.yml && echo "here"
cfn-lint 0.21.0
here

Tested the above with 0.21.2 as well, same E0001 error:
∇{𝞿/RO-1092⁁ncy? ~/.xia/sha/sta/sam}cfn-lint --version && cfn-lint cfn.yml && echo "here"
cfn-lint 0.21.2
E0001 Error transforming template: datetime.date(2012, 10, 17) is not JSON serializable
cfn.yml:1:1

Was not expecting another version after this mornings release so the original title is not totally up to date, sorry about that.


[Release] sdk/resourcemanager/education/armeducation/0.1.0

Fixes: https://github.com/Azure/sdk-release-request/issues/2732
New Service

/azp run prepare-pipelines

/azp run go - armeducation - ci


Add github.com/gibizer/operator-lint

To be able to add that in the same way as in other service-operators the existing operator-lint target is renamed to kube-linter and operator-lint target now run github.com/gibizer/operator-lint.
The kube-linter target extended to ensure that the kube-linter binary is installed locally.
The operator-lint need go.work setup to be able to check the ./api module properly so a gowork make target is added and go.work is ignored in .gitignore.
The go work sync found a conflict in dependencies between the main and the ./api module so github.com/kr/pretty is bumped to v0.2.1 to resolve that.
Then the operator-lint findings are fixed.

Thanks for the PR! Can you please run pre-commit locally so that the CI job does not fail?

Hm. I run it locally and only the gotest target fails that I haven't touched and that is passing in the github workflow.
$ pre-commit run -a
check for added large files..............................................Passed
fix utf-8 byte order marker..............................................Passed
check for case conflicts.................................................Passed
check that executables have shebangs.....................................Passed
check that scripts with shebangs are executable..........................Passed
check for merge conflicts................................................Passed
check for broken symlinks............................(no files to check)Skipped
detect destroyed symlinks................................................Passed
check yaml...............................................................Passed
check json...............................................................Passed
detect private key.......................................................Passed
fix end of files.........................................................Passed
don't commit to branch...................................................Passed
trim trailing whitespace.................................................Passed
make-gofmt...............................................................Passed
make-govet...............................................................Passed
make-gotest..............................................................Failed
- hook id: make-gotest
- exit code: 2

if [ -d  "/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/CI_TOOLS_REPO" ]; then \
	echo "Ci tools exists"; \
	pushd "/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/CI_TOOLS_REPO"; \
	git pull --rebase; \
	popd; \
else \
	git clone https://github.com/openstack-k8s-operators/openstack-k8s-operators-ci "/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/CI_TOOLS_REPO"; \
fi
Ci tools exists
~/upstream/git/openstack-k8s-operators/ceilometer-operator/CI_TOOLS_REPO ~/upstream/git/openstack-k8s-operators/ceilometer-operator
Already up to date.
~/upstream/git/openstack-k8s-operators/ceilometer-operator
/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/CI_TOOLS_REPO/test-runner/gotest.sh
+ GOWORK=off
++ dirname /home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/CI_TOOLS_REPO/test-runner/gotest.sh
+ BASE_DIR=/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/CI_TOOLS_REPO/test-runner
+ cd /home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/CI_TOOLS_REPO/test-runner/../..
+ MODULE_DIR=.
+ '[' -n '' ']'
+ pushd .
~/upstream/git/openstack-k8s-operators/ceilometer-operator ~/upstream/git/openstack-k8s-operators/ceilometer-operator
+ GOWORK=off
+ go test -mod=mod -v ./...
?   	github.com/openstack-k8s-operators/ceilometer-operator	[no test files]
=== RUN   TestAPIs
Running Suite: Controller Suite
===============================
Random Seed:<PHONE_NUMBER>
Will run 0 of 0 specs

STEP: bootstrapping test environment
2023-03-23T13:11:42+01:00	DEBUG	controller-runtime.test-env	starting control plane
2023-03-23T13:11:42+01:00	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 0, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory"}
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:329
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:259
github.com/openstack-k8s-operators/ceilometer-operator/controllers.glob..func1
	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:60
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:113
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:64
github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/suite_nodes.go:25
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runBeforeSuite
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:123
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:63
github.com/onsi/ginkgo/internal/suite.(*Suite).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/suite/suite.go:79
github.com/onsi/ginkgo.runSpecsWithCustomReporters
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:245
github.com/onsi/ginkgo.RunSpecs
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:220
github.com/openstack-k8s-operators/ceilometer-operator/controllers.TestAPIs
	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:46
testing.tRunner
	/usr/local/go/src/testing/testing.go:1446
2023-03-23T13:11:42+01:00	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 1, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory"}
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:329
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:259
github.com/openstack-k8s-operators/ceilometer-operator/controllers.glob..func1
	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:60
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:113
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:64
github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/suite_nodes.go:25
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runBeforeSuite
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:123
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:63
github.com/onsi/ginkgo/internal/suite.(*Suite).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/suite/suite.go:79
github.com/onsi/ginkgo.runSpecsWithCustomReporters
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:245
github.com/onsi/ginkgo.RunSpecs
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:220
github.com/openstack-k8s-operators/ceilometer-operator/controllers.TestAPIs
	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:46
testing.tRunner
	/usr/local/go/src/testing/testing.go:1446
2023-03-23T13:11:42+01:00	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 2, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory"}
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:329
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:259
github.com/openstack-k8s-operators/ceilometer-operator/controllers.glob..func1
	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:60
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:113
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:64
github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/suite_nodes.go:25
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runBeforeSuite
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:123
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:63
github.com/onsi/ginkgo/internal/suite.(*Suite).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/suite/suite.go:79
github.com/onsi/ginkgo.runSpecsWithCustomReporters
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:245
github.com/onsi/ginkgo.RunSpecs
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:220
github.com/openstack-k8s-operators/ceilometer-operator/controllers.TestAPIs
	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:46
testing.tRunner
	/usr/local/go/src/testing/testing.go:1446
2023-03-23T13:11:42+01:00	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 3, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory"}
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:329
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:259
github.com/openstack-k8s-operators/ceilometer-operator/controllers.glob..func1
	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:60
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:113
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:64
github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/suite_nodes.go:25
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runBeforeSuite
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:123
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:63
github.com/onsi/ginkgo/internal/suite.(*Suite).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/suite/suite.go:79
github.com/onsi/ginkgo.runSpecsWithCustomReporters
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:245
github.com/onsi/ginkgo.RunSpecs
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:220
github.com/openstack-k8s-operators/ceilometer-operator/controllers.TestAPIs
	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:46
testing.tRunner
	/usr/local/go/src/testing/testing.go:1446
2023-03-23T13:11:42+01:00	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 4, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory"}
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).startControlPlane
	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:329
sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Start
	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:259
github.com/openstack-k8s-operators/ceilometer-operator/controllers.glob..func1
	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:60
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:113
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:64
github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/suite_nodes.go:25
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runBeforeSuite
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:123
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:63
github.com/onsi/ginkgo/internal/suite.(*Suite).Run
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/suite/suite.go:79
github.com/onsi/ginkgo.runSpecsWithCustomReporters
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:245
github.com/onsi/ginkgo.RunSpecs
	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:220
github.com/openstack-k8s-operators/ceilometer-operator/controllers.TestAPIs
	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:46
testing.tRunner
	/usr/local/go/src/testing/testing.go:1446
Failure [0.001 seconds]
[BeforeSuite] BeforeSuite
/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:49

  Unexpected error:
      <*fmt.wrapError | 0xc0006127a0>: {
          msg: "unable to start control plane itself: failed to start the controlplane. retried 5 times: fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory",
          err: <*fmt.wrapError | 0xc000612780>{
              msg: "failed to start the controlplane. retried 5 times: fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory",
              err: <*fs.PathError | 0xc0002d1dd0>{
                  Op: "fork/exec",
                  Path: "/usr/local/kubebuilder/bin/etcd",
                  Err: <syscall.Errno>0x2,
              },
          },
      }
      unable to start control plane itself: failed to start the controlplane. retried 5 times: fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory
  occurred

  /home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:61
------------------------------
STEP: tearing down the test environment
Panic [0.000 seconds]
[AfterSuite] AfterSuite
/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:75

  Test Panicked
  runtime error: invalid memory address or nil pointer dereference
  /usr/local/go/src/runtime/panic.go:260

  Full Stack Trace
  sigs.k8s.io/controller-runtime/pkg/internal/testing/controlplane.(*APIServer).Stop(0x5c1e90?)
  	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/testing/controlplane/apiserver.go:424 +0x8f
  sigs.k8s.io/controller-runtime/pkg/internal/testing/controlplane.(*ControlPlane).Stop(0xc000364780)
  	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/testing/controlplane/plane.go:97 +0x49
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Stop(0xc000364780)
  	/home/gibi/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/envtest/server.go:193 +0x114
  github.com/openstack-k8s-operators/ceilometer-operator/controllers.glob..func2()
  	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:77 +0x38
  github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0xc000506380?)
  	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:113 +0xb1
  github.com/onsi/ginkgo/internal/leafnodes.(*runner).run(0x0?)
  	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/runner.go:64 +0x125
  github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run(0xc000155360, 0x43ea16?, 0x0?, {0xc00006ef00?, 0xc0000c3af0?})
  	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/leafnodes/suite_nodes.go:25 +0x86
  github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runAfterSuite(0xc000161340)
  	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:138 +0xce
  github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run(0xc000161340)
  	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/specrunner/spec_runner.go:71 +0xdd
  github.com/onsi/ginkgo/internal/suite.(*Suite).Run(0xc00037e1c0, {0x7efc943c8a00, 0xc0001029c0}, {0x1780ec7, 0x10}, {0xc0001629c0, 0x1, 0x1}, {0x19e2300, 0xc00030e480}, ...)
  	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/internal/suite/suite.go:79 +0x4e5
  github.com/onsi/ginkgo.runSpecsWithCustomReporters({0x19cc120?, 0xc0001029c0}, {0x1780ec7, 0x10}, {0xc000117718, 0x1, 0xf?})
  	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:245 +0x189
  github.com/onsi/ginkgo.RunSpecs({0x19cc120, 0xc0001029c0}, {0x1780ec7, 0x10})
  	/home/gibi/go/pkg/mod/github.com/onsi/ginkgo@v1.16.5/ginkgo_dsl.go:220 +0x14a
  github.com/openstack-k8s-operators/ceilometer-operator/controllers.TestAPIs(0x408519?)
  	/home/gibi/upstream/git/openstack-k8s-operators/ceilometer-operator/controllers/suite_test.go:46 +0x45
  testing.tRunner(0xc0001029c0, 0x1863220)
  	/usr/local/go/src/testing/testing.go:1446 +0x10b
  created by testing.(*T).Run
  	/usr/local/go/src/testing/testing.go:1493 +0x35f
------------------------------

Ran 0 of 0 Specs in 0.002 seconds
FAIL! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
--- FAIL: TestAPIs (0.00s)
FAIL
FAIL	github.com/openstack-k8s-operators/ceilometer-operator/controllers	0.011s
?   	github.com/openstack-k8s-operators/ceilometer-operator/pkg/ceilometer	[no test files]
FAIL
make: *** [Makefile:279: gotest] Error 1

make-golangci............................................................Passed
make-golint..............................................................Passed
make-generate............................................................Passed
make-kube-linter.........................................................Passed
make-operator-lint.......................................................Passed
bashate..................................................................Passed

Hm. I run it locally and only the gotest target fails that I haven't touched and that is passing in the github workflow.

"fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory" means that you don't have kubebuilder initialized. Running "source <(setup-envtest -p env)" should make it pass for you.
Hmm, the check here fails on your operator-lint job: https://github.com/openstack-k8s-operators/ceilometer-operator/actions/runs/4499305516/jobs/7917002491?pr=57#step:7:35
I've re-run it, so let's see if it passes now.

I've re-run it, so let's see if it passes now.

Failed again ... maybe you have some dependencies installed in your local environment, which GHA environment does not have?

Hm. I run it locally and only the gotest target fails that I haven't touched and that is passing in the github workflow.

"fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory" means that you don't have kubebuilder initialized. Running "source <(setup-envtest -p env)" should make it pass for you.

Yeah, I think that should be part of the make target. See example in
https://github.com/openstack-k8s-operators/nova-operator/blob/70b7e01ff148f10c490b3a20625714994caa2c08/Makefile#L202-L205
https://github.com/openstack-k8s-operators/nova-operator/blob/70b7e01ff148f10c490b3a20625714994caa2c08/Makefile#L127-L129

I've re-run it, so let's see if it passes now.

Failed again ... maybe you have some dependencies installed in your local environment, which GHA environment does not have?
" - files were modified by this hook"
also the above ... maybe you need to git commit --amend some changes?

would be nice to see which files :)

would be nice to see which files :)

I had issues recently with CI_TOOLS_REPO when files there were modified unnecessarily. Maybe deleting the directory before the operator-lint (it's not needed anyway for the lint).

Is this PR still valid and it is something that it should be merged? If that is the case, can someone resolve the merge conflicts?
Thank you!

Is this PR still valid and it is something that it should be merged? If that is the case, can someone resolve the merge conflicts?
Thank you!

The goal of the PR is still valid. But I was not able to solve the pre-commit CI issue with the change. I rebased the PR now

I have tested it and I see what you see Gibi: it works perfectly locally and it does not work on the CI environment. We might be lacking something on the CI. Trying to find out what that might be, I looked at the nova-operator to try to compare environments but I dont see the pre-commit workflow executing there.
Do you have a reference of an operator where this linter is working well in the CI?

Even more strange that it does not work in github action, but it works in the prow job properly https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openstack-k8s-operators_telemetry-operator/57/pull-ci-openstack-k8s-operators-telemetry-operator-main-precommit-check/1643613344659148800/artifacts/test/build-log.txt
I'm wondering, do you want to keep the github action now that prow runs pre-commit as well?

That's an excellent question, I will come back with an answer.

We have been discussing this, and we see value to have the operator-lint in the pre-commit action, that is mainly intended to be used by the dev in its local environment. However, it is broke, but it seems that @paramite as already figured out what the problem is and proposed a change: https://github.com/gibizer/ceilometer-operator/pull/1

We have been discussing this, and we see value to have the operator-lint in the pre-commit action, that is mainly intended to be used by the dev in its local environment. However, it is broke, but it seems that @paramite has already figured out what the problem is and proposed a change: gibizer#1

@gibizer ^

Thanks. Let's see if this helps.

We have been discussing this, and we see value to have the operator-lint in the pre-commit action, that is mainly intended to be used by the dev in its local environment. However, it is broke, but it seems that @paramite has already figured out what the problem is and proposed a change: gibizer#1

Sure, I'm on the side to run operator-lint in pre-commit. There is now a prow job called ci/prow/precommit-check that runs the operator-lint as part of pre-commit in CI. Also if you set up pre-commit locally in your repo with pre-commit install then operator-lint will be run by pre-commit locally for every git commit. So as far as I see both the local and the CI workflow is covered without the github action Workflow for pre-commit run / Run pre-commit jobs (pull_request). Honestly I run out of ideas why the same pre-commit run behaves differently in the github workflow compared to the local run or the CI run in prow. However I think that the github action is now simply a duplicate that could be removed since both the local and the CI testing is covered without it.

Thanks. Let's see if this helps.

No it does not.

Honestly I run out of ideas why the same pre-commit run behaves differently in the github workflow compared to the local run or the CI run in prow.

The CI tools exclusion made it show the reason why we have the job failing. IIUC CI environment preparation is changing go modules set [1] hence the pre-commit fails. I'm gonna clone your PR from scratch in VM and reproduce the issue to fix it.
[1] https://github.com/openstack-k8s-operators/telemetry-operator/actions/runs/4698121758/jobs/8330026476?pr=57

the custom pre-commit has been removed by https://github.com/openstack-k8s-operators/telemetry-operator/pull/82, so now it only executes in prow.

Cool thanks @jlarriba !

Sorry for late response, but I figured out the problem here yesterday (reproduced on different PR). The problem is multiple commits. Because pre-commit runs on each of them. If there is problem on first one, it fails and it passws for you locally, because of course you fixed those issues, but most probably amended to last commit like every other sane person ... That's a little process flaw I would say, but it is what it is. Just rebase, squash them and it will pass :).
I still think having this linter running in our pre-commit job (even though we removed it in this repo) is useful.

@gibizer @jlarriba ^

Thanks for the explanation @paramite


initial publisher script

The script only works for gcloud. Probably we should name it as publish_gcloud_storage.sh.

LGTM


Cannot find NvInfer.h with TensorRT install

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary):
TensorFlow version:1.12.0
Python version:3.5
Installed using virtualenv? pip? conda?:
Bazel version (if compiling from source):0.15
GCC/Compiler version (if compiling from source):GCC
CUDA/cuDNN version:9.0
GPU model and memory:7.1.4

Describe the problem
When I try to compile Tensorflow I met this issue. To solve this problem I first try to modify the TensorRT install path in perform ./configure, but I find path is not changed, for example:
default path:  /usr/lib/x86_64-linux-gnu
I set path is: /usr/local/TensorRT
This is useless, I don't know why,
So, I second try to copy NvInfer.h to /usr/lib/x86_64-linux-gnu, but this problem stall exist, and I find NvInfer.h is exist in /usr/lib/x86_64-linux-gnu, why cody can't find this file, I feel confused.
Provide the exact sequence of commands / steps that you executed before running into the problem
bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
ERROR: Skipping '//tensorflow/tools/pip_package:build_pip_package': error loading package 'tensorflow/tools/pip_package': Encountered error while reading extension file 'build_defs.bzl': no such package '@local_config_tensorrt//': Traceback (most recent call last):
File "/install_bar/tensorflow-1.12.0/third_party/tensorrt/tensorrt_configure.bzl", line 171
_trt_lib_version(repository_ctx, trt_install_path)
File "/install_bar/tensorflow-1.12.0/third_party/tensorrt/tensorrt_configure.bzl", line 81, in _trt_lib_version
_find_trt_header_dir(repository_ctx, trt_install_path)
File "/install_bar/tensorflow-1.12.0/third_party/tensorrt/tensorrt_configure.bzl", line 67, in _find_trt_header_dir
auto_configure_fail(("Cannot find NvInfer.h with Ten...))
File "/install_bar/tensorflow-1.12.0/third_party/gpus/cuda_configure.bzl", line 317, in auto_configure_fail
fail(("\n%sCuda Configuration Error:%...)))
Cuda Configuration Error: Cannot find NvInfer.h with TensorRT install path /usr/lib/x86_64-linux-gnu
WARNING: Target pattern parsing failed.
ERROR: error loading package 'tensorflow/tools/pip_package': Encountered error while reading extension file 'build_defs.bzl': no such package '@local_config_tensorrt//': Traceback (most recent call last):
File "/install_bar/tensorflow-1.12.0/third_party/tensorrt/tensorrt_configure.bzl", line 171
_trt_lib_version(repository_ctx, trt_install_path)
File "/install_bar/tensorflow-1.12.0/third_party/tensorrt/tensorrt_configure.bzl", line 81, in _trt_lib_version
_find_trt_header_dir(repository_ctx, trt_install_path)
File "/install_bar/tensorflow-1.12.0/third_party/tensorrt/tensorrt_configure.bzl", line 67, in _find_trt_header_dir
auto_configure_fail(("Cannot find NvInfer.h with Ten...))
File "/install_bar/tensorflow-1.12.0/third_party/gpus/cuda_configure.bzl", line 317, in auto_configure_fail
fail(("\n%sCuda Configuration Error:%...)))
Cuda Configuration Error: Cannot find NvInfer.h with TensorRT install path /usr/lib/x86_64-linux-gnu

@Julius-ZCJ ,
Can you please provide ./configure output ?thanks!

@oanush  Here is my ./configure set:
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.15.0 installed.
Please specify the location of python. [Default is /usr/bin/python]:
Found possible Python library paths:
/usr/lib/python3/dist-packages
/usr/local/lib/python3.5/dist-packages
Please input the desired Python library path to use.  Default is [/usr/lib/python3/dist-packages]
Do you wish to build TensorFlow with Apache Ignite support? [Y/n]: n
No Apache Ignite support will be enabled for TensorFlow.
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: n
No XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.
Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 9.0]:
Please specify the location where CUDA 9.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7]:
Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Do you wish to build TensorFlow with TensorRT support? [y/N]: y
TensorRT support will be enabled for TensorFlow.
Please specify the location where TensorRT is installed. [Default is /usr/lib/x86_64-linux-gnu]:
Please specify the NCCL version you want to use. If NCCL 2.2 is not installed, then you can use version 1.3 that can be fetched automatically but it may have worse performance with multiple GPUs. [Default is 2.2]: 1.3
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 6.0]:
Do you want to use clang as CUDA compiler? [y/N]: n
nvcc will be used as CUDA compiler.
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
Do you wish to build TensorFlow with MPI support? [y/N]: n
No MPI support will be enabled for TensorFlow.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See tools/bazel.rc for more details.
--config=mkl            # Build with MKL support.
--config=monolithic     # Config for mostly static monolithic build.
--config=gdr            # Build with GDR support.
--config=verbs          # Build with libverbs support.
--config=ngraph         # Build with Intel nGraph support.
Configuration finished

Thanks for your report. Can you please attach a complete log of ./configure and bazel build starting from a fresh environment on the master branch? I'm afraid we don't provide support for building older versions.

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 29 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 44 days with no activity and the awaiting response label was assigned. Is this still an issue?

We are closing this issue for now due to lack of activity. Please comment if this is still an issue for you. Thanks!


[BUG] VM still got IP Address when poweroff

Describe the bug
The VM still displays IP Address with the state Off.
The VM was powered off via executing command sudo poweroff inside the VM.
refer to #1882
To Reproduce
Steps to reproduce the behavior:

Create VM
Login to VM and execute sudo poweroff
VM's State become Off and the IP Address still available

Expected behavior
VM should not display IP Address.
Environment:

Harvester ISO version: master-03ee6213-head
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): qemu/KVM single node

Additional context
The VM still had IP Address

VM's Restart Button is Enabled

VM's Restart Button is Disabled when the VM was stopped via dashboard

Pod of VM s1 been removed, but VM s3b still exists

The ip will disappear after a few seconds.

Both ways of stopping the vm will have the problem that the ip does not disappear immediately.
The frontend is enhanced, when the vm stops, the UI will hide the IP information

I think there still have something different between execute poweroff and click stop.
The VM s1 execute poweroff to stop, and the VM s2 click stop button to stop.
and as the screenshots, s1's pod will remaining with completed status; in the dashboard, Restart Button will appear and clickable, showing scheduled Node.
I also tested when click Restart / Start VM s1, completed will be removed and create new pod for it. (the behavior same as VM s2 which click stop button to stop.)
I'm thinking should we just hide the IP Address and scheduled Node ? or we can terminate the completed VM to let behaviors consistent with other VM?
@guangbochen do you have some suggestions?
https://user-images.githubusercontent.com/5169694/163952478-b34bf53a-b4d4-45a7-8713-6e8933f4a979.mp4


Don't depend on en_US.UTF-8 locale

Instead, create locale with codecvt_utf8 facet on all platforms (even if
codecvt_utf8 is deprecated since C++17).  There is no guarantee that
"en_US.UTF-8" is a supported locale name, so the locale constructor might throw
a runtime_error.  (See the discussion in the comments to
https://gerrit.libreoffice.org/#/c/62508/ "Add check for en_US.utf8 locale"
for a real-live example of issues caused by that.)
(And the  and  headers apparently need to be included always,
regardless of NUMBERTEXT_BOOST.)

Many thanks for the fix! László


Add imports to stringify method

I'd like to be able to use the library as part of a protobuf file generation project I am working on but am running into an issue where imports are not preserved when calling parse to transform protobuf to JSON and then calling stringify to stringify the JSON back into the .proto format.
It seems like a solution to this would be pretty simple. Happy to create a PR if desired.

Sounds good! As long as it doesn’t add any io but just parses ut I’m +1

That was the idea, just parsing what is already there and adding to the string.


Fix Spark-DL notebooks for CI/CD and update to latest dependencies

Changes:
Version updates:

Updated all examples to run with latest Triton (24.08), Tensorflow (2.17.0), and Torch (2.4.1), and match updated APIs.
Notable changes:

Using new .keras model format in Tensorflow wherever applicable
Added use of Torch-TensorRT inference locally and on Spark to PyTorch notebooks

Environment separation:

Updated README for instructions on creating separate environments for Torch/Tensorflow to avoid CUDA conflicts.
Separated Huggingface examples into _torch/_tf versions for environment separation, and demonstrate model interoperability.

CI/CD:

Included local Spark Session initialization to work with CI/CD pipeline. Confirmed this will not affect preexisting Spark sessions if the notebook is already running on a cluster.
Verified that all notebooks run error-free with jupyter nbconvert.

@eordentlich @leewyang requesting a review - thanks!


Addition of Donation Banner

Related to

Fixes #1199 by @annatuma
Description

I have added a Donation Banner here.
Technical details

The donation Banner is visible unless user will close it.It user clicked on "Yes i 'll donate now",the user will be redirected to link "https://www.classy.org/give/297881/#!/donation/checkout".
##Demo:

Checklist

[ ] My pull request has a descriptive title (not a vague title like Update index.md).
[ ] My pull request targets the default branch of the repository (main or master).
[ ] My commit messages follow best practices.
[ ] My code follows the established code style of the repository.
[ ] I added tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[ ] I tried running the project locally and verified that there are no
visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Hi @neeraj-2, you wrote a lot of custom styles. CC Search is built on vocabulary which has a ton of built in classes and components to help you, so you don't have to hard code colors, spacing, etc. yourself. Please look at vocabulary and try switching some of your styles over!

Hi @neeraj-2, you wrote a lot of custom styles. CC Search is built on vocabulary which has a ton of built in classes and components to help you, so you don't have to hard code colors, spacing, etc. yourself. Please look at vocabulary and try switching some of your styles over!

ok,@zackkrida i will do that .

@zackkrida  now i did not use custom styles,can u please tell me why the tests are failing ?
:(

Yes @neeraj-2 i will let you know once I review!

@zackkrida  i have created a new draft pr for it #1219,that's why i closed it


Dockerize application

I work on dozens of projects, many of which go into production in dockerized environments. This PR sets up the project in a dockerized environment. I had to comment out the streamlit hosting lines (5-7) because there is no pysqlite3-binary package that is installable on arm64 (that I am aware of) and instead install sqlite3 as part of setting up the docker image. Maybe the imports could be inside a try/except block?

Thanks! Always good to dockerize @tedsecretsource

Maybe the imports could be inside a try/except block?
good point! Will do that


MSBuild Documentation example for Exclude in Item does not work and is clearly untested

[Enter feedback here]
I am struggling to make Exclude in MSBuild work.
The following does not exclude any of the items requested to be excluded. Why?
        <CommonDllsSDK
            Include="$(SignedBinaries)\CommonDlls\**\*.*"
            Exclude="$(SignedBinaries)\CommonDlls\**\*Res*.*;
                     $(SignedBinaries)\CommonDlls\**\Helper.exe;
                     $(SignedBinaries)\CommonDlls\**\Application*.*;
                     $(SignedBinaries)\CommonDlls\**\ErrorReporter*.*"
         />

Your documentation doesn't even provide a working example.
This is not conducive to an adoption of your technology.
Spot the error here:
<Project DefaultTargets="Compile"
    xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >

    <PropertyGroup>
        <builtdir>built</builtdir>
    </PropertyGroup>

    <ItemGroup>
        <CSFile Include="*.cs Exclude="Form2.cs"/>

        <Reference Include="System.dll"/>
        <Reference Include="System.Data.dll"/>
        <Reference Include="System.Drawing.dll"/>
        <Reference Include="System.Windows.Forms.dll"/>
        <Reference Include="System.XML.dll"/>
    </ItemGroup>

    <Target Name="PreBuild">
        <Exec Command="if not exist $(builtdir) md $(builtdir)"/>
    </Target>

    <Target Name="Compile" DependsOnTargets="PreBuild">
        <Csc Sources="@(CSFile)"
            References="@(Reference)"
            OutputAssembly="$(builtdir)\$(MSBuildProjectName).exe"
            TargetType="exe" />
    </Target>
</Project>

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 30273668-658c-d411-5b6d-7f58256bb0f3
Version Independent ID: c8812ae3-0fb1-a30f-02dd-ebc2d28b9dc8
Content: How to: Exclude Files from the Build - Visual Studio
Content Source: docs/msbuild/how-to-exclude-files-from-the-build.md
Product: visual-studio-windows
GitHub Login: @Mikejo5000
Microsoft Alias: mikejo

The example has been fixed in #3070

@rainersigwald Rainer, do you see a problem in the example cited by @ncook-vero ?

@Mikejo5000 it's in here (#3070 ?)
 <CSFile Include="*.cs Exclude="Form2.cs"/>

OK, that is a good idea. It's possible the error in the example came into the content later, as a change to the example that also included a typo.


Add own sound

I can't find this anywhere but it is said to be supported on https://chris.dziemborowicz.com/apps/hourglass/

There is an FAQ towards the bottom of the page at https://chris.dziemborowicz.com/apps/hourglass/.
Save the sound file to the folder where you installed Hourglass – typically, C:\Program Files (x86)\Hourglass.
Your sound file should then appear when you right-click on any empty space in the Hourglass window and select the Sound submenu.
The supported formats are .aac, .m4a, .mid, .midi, .mp3, .wav, and .wma. If your sound file is not in one of the supported formats, you will need to convert it first.

That can be weird. I can find nothing but Hourglass.exe in the installing folder,
It won't help even if I create it myself nor use repairing in the install pack.
PS I'm using Windows 10.14393.351

That's all that should be there. If you want to add your own custom sounds, you can just copy them into that folder so that they sit next to Hourglass.exe. The sounds will then appear in the "Sounds" right-click menu. (The built-in sounds are embedded in Hourglass.exe itself.)