C++ compilation of rule '@jemalloc//:jemalloc' failed:

Trying to install tensor flow on cluster from source. I have installed bazel
[bazel release 0.4.4- (@non-git)], and I am using python 2.7.13 with pyenv. Upon trying to build the tensorflow pip wheel I am getting a compilation error:
ERROR: ~/.cache/bazel/_bazel/e924d9c3ba75314415252c6f4f93bb86/external/jemalloc/BUILD:10:1: C++ compilation of rule '@jemalloc//:jemalloc' failed: gcc failed: error executing command /opt/apps/compilers/gcc/4.8.2/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -B/opt/apps/compilers/gcc/4.8.2/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object ... (remaining 38 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
Has anyone experienced this? Is this a consequence of the warning I get from bazel for being on an NFS:
WARNING: Output base '~/.cache/bazel/_bazel/e924d9c3ba75314415252c6f4f93bb86' is on NFS. This may lead to surprising failures and undetermined behaviour.
My gcc version is:
gcc (GCC) 4.8.2
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Ultimately I just disabled jemalloc when prompted by the tensorflow/configure file. Tensor flow seems to have compiled fine from there on.

cc @jhseu who added jemalloc. @Montmorency can you specify your linux/bazel versions?

@Montmorency Possibly unrelated, but I've met random problems using bazel build on NFS hosted home directory.
export TEST_TMPDIR="/tmp" # or some local directory that is not hosted on NFS

This might help.

@jhseu Can you comment?

@Montmorency I tested against gcc 4.8.4 and it works for me.
Can you copy the compilation error that you get before that error message? Also, can you try removing NFS as a potential issue by doing what @byronyi mentioned?

Thanks for response!
Bazel
 [bazel release 0.4.4- (@non-git)]

Distro
$cat /etc/issue
  Scientific Linux release 6.6 (Carbon)
  Kernel \r on an \m
$uname -r
  2.6.32-431.17.1.el6.x86_64
$uname -i
  x86_64
$gcc --version
gcc (GCC) 6.2.0
Copyright (C) 2016 Free Software Foundation, Inc.

Update:
I had a closer look and started using a more recent compiler. The NFS is not an issue. Closer inspection of the output pointed to failure to do with JEMALLOC_THP and some undefined variable(MADV_NOHUGEPAGE). I think the cluster has an older linux kernel. I manually removed these flags from the jemalloc/BUILD file and compilation completed without any further issues:
deleted:
"#undef JEMALLOC_THP": "#define JEMALLOC_THP",
"#undef JEMALLOC_HAVE_SECURE_GETENV": "#define JEMALLOC_HAVE_SECURE_GETENV",
I removed the second flag when the build failed at linking stage I think this is related to an older version of openssl on the cluster which configure didn't seem to detect (getenv rather than secure_getenv). Training the models is fine with this build but I do still a strange glibc error when running model.predict_proba():
*** glibc detected *** python: double free or corruption (!prev): 0x0000000001375da0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3168e75e66]
/lib64/libc.so.6[0x3168e789b3]
/lib64/ld-linux-x86-64.so.2(_dl_deallocate_tls+0x67)[0x31686112f7]
/lib64/libpthread.so.0[0x316920675d]
/lib64/libpthread.so.0[0x31692078ea]
/lib64/libpthread.so.0(pthread_join+0xd4)[0x31692081f4]
/opt/apps/compilers/gcc/6.2.0/lib64/libstdc++.so.6(_ZNSt6thread4joinEv+0x27)[0x7fbc4ea24a77]
/users/k1511981/.pyenv/versions/t2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(+0x248c310)[0x7fbc51186310]
/users/k1511981/.pyenv/versions/t2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(_ZN10tensorflow6thread10ThreadPool4ImplD0Ev+0x144)[0x7fbc51160594]
/users/k1511981/.pyenv/versions/t2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(_ZN10tensorflow10FileSystem16GetMatchingPathsERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPSt6vectorIS6_SaIS6_EE+0x5ce)[0x7fbc511830de]
/users/k1511981/.pyenv/versions/t2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(_ZN10tensorflow3Env16GetMatchingPathsERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPSt6vectorIS6_SaIS6_EE+0xab)[0x7fbc5117f7eb]
/users/k1511981/.pyenv/versions/t2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(+0x986bfb)[0x7fbc4f680bfb]
/users/k1511981/.pyenv/versions/t2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(+0x9877c4)[0x7fbc4f6817c4]

Ah yeah, that kernel is really old (from 2009!). I'm not sure we should try to support jemalloc with that kernel when there's a reasonable workaround.
The double free issue is unrelated because it happens even when disabling jemalloc (from your other comment on #6968). Also, it's in code that's unaffected by jemalloc. Looking at the stack trace, it's crashing in pthread_join in deallocating thread-local storage. Seems unlikely to be a TensorFlow issue (possibly also related to the old linux kernel?)
Closing the issue out as intended behavior.

I haven't tested, but some searching indicates that jemalloc should build as long as the Linux kernel version is >= 2.6.38, otherwise it needs to be disabled.

I removed bazel cache, did bazel clean, disabled jemalloc via configure and still get the following error
ERROR: /home/ebice/.cache/bazel/bazel_ebice/975e0509e630426b34ea61d02aa8b898/ex                                                                                ternal/jemalloc/BUILD:10:1: C++ compilation of rule '@jemalloc//:jemalloc' faile                                                                                d: gcc failed: error executing command /opt/rh/devtoolset-6/root/usr/bin/gcc -U                                                                                FORTIFY_SOURCE -fstack-protector -Wall -B/opt/rh/devtoolset-6/root/usr/bin -B/us                                                                                r/bin -Wunused-but-set-parameter -Wno-free-nonheap-object ... (remaining 38 argu                                                                                ment(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Pr                                                                                ocess exited with status 1.
external/jemalloc/src/pages.c: In function 'je_pages_huge':
external/jemalloc/src/pages.c:203:30: error: 'MADV_HUGEPAGE' undeclared (first u                                                                                se in this function)
return (madvise(addr, size, MADV_HUGEPAGE) != 0);

@edi-bice If you're on a Linux kernel older than 2.6.38, you'll have to disable jemalloc.

@edi-bice Unfortunately, I don't have access to any machine with such old Linux kernels to test. That shouldn't happen if jemalloc is really disabled as far as I can tell.

"really disabled" is the keyword. Apparently bazel clean did not really clean everything. In addition to .bazelrc the file .tf_configure.bazelrc remained even after a bazel clean and inside there jemalloc=true despite configure output stating "jemalloc disabled".

Did you rerun ./configure? That file is deleted and updated again here:
https://github.com/tensorflow/tensorflow/blob/master/configure#L167


config ignored by grunt

Running grunt always stores files in public_www, even when another directory is given in the config file

Fixed by https://github.com/C-Bouthoorn/Epsilon/commit/9df0fb93910d810ba7d84db84733030ca318772d


API: consider better decorator option

It's not an issue in py3.5, but in older versions this is annoying - because a decorator is used in the definition, the function signature on to_datetime is lost.  This is the big one I've run into, but there may be some other places in the API it comes up.

I saw this blog today - could use one of the strategies there.  It may make sense to vendor rather than adding another dependency?
https://hynek.me/articles/decorators/

it should propagate
I think it just needs wraps

actually @chris-b1 where is this from? ? in ipython and help work just fine. can you elaborate.

actually read that article. Ok that is nice.

I'm assuming you're looking on python3?  In python 2 the standard wraps doesn't propagate signatures.
In [1]: import functools

In [2]: def dec_stdlib(f):
   ...:     @functools.wraps(f)
   ...:     def inner(*args, **kwargs):
   ...:         f(*args, **kwargs)
   ...:     return inner

In [6]: import boltons.funcutils

In [7]: def dec_boltons(f):
   ...:     @boltons.funcutils.wraps(f)
   ...:     def inner(*args, **kwargs):
   ...:         f(*args, **kwargs)
   ...:     return inner

In [4]: @dec_stdlib
   ...: def f(a, b):
   ...:     return a + b

In [5]: f?
Signature: f(*args, **kwargs)
Docstring: <no docstring>
Type:      function

In [8]: @dec_boltons
   ...: def f(a, b):
   ...:     return a + b

In [9]: f?
Signature: f(a, b)
Type:      function

I attempted this using https://github.com/mahmoud/boltons/blob/master/boltons/funcutils.py - which has a wraps replacement, but the interaction of the magic broke a bunch of stuff.  I'm sure it would be possible to work through it, but I'm letting this one sit - I guess it's good self-motivation to update things to python 3.

I've looked into this a bit using python 3.7 and the getfullargspecs problem still stands in 3.7. I'll remove the 2/3 compat label, but let the issue stand open.
For the record, giving the drawbacks to wrapt and decorators.py, I like vendoring boltons.functools.wraps the best (assuming it can be pulled out painlessly - haven't checked that).
.

@chris-b1 since we're not py35+ is this closeable?

I would think so!  Although I am not following @topper-123 comment above on getfullargspecs so maybe some part of this that doesn't work?

My point was that inspect.getfullargspec still does not work 100 %. Quoting the the linked article:

It’s kind of cheating: the signatures are still broken. inspect.signature() is just smart enough to lie to you.

OTOH, if mypy accepts the "lie", all the better. Does anyone know? (PyCharm does not).
To demonstrate the difference between a decorated and non-decorated function:
>>> from inspect import getfullargspec
>>> from functools import wraps
>>> def my_func(a:int, b: int = 1) -> int:
    ...:     return a + b
>>> def deco(func):
...     @functools.wraps(func)
...     def inner_func(*args, **kwargs):
...         return func(*args, **kwargs)
...     return inner_func

>>> getfullargspec(my_func)
FullArgSpec(args=['a', 'b'], varargs=None, varkw=None, defaults=(1,), kwonlyargs=[], kwonlydefaults=None, annotations={'return': <class 'pandas.core.series.Series'>, 'a': <class 'pandas.core.series.Series'>, 'b': <class 'int'>})
>>> getfullargspec(deco(my_func))
FullArgSpec(args=[], varargs='args', varkw='kwargs', defaults=None, kwonlyargs=[], kwonlydefaults=None, annotations={'return': <class 'pandas.core.series.Series'>})

Notice the output is quite different.
inspect.signature gives the same output for both decorared and non-decorated, but that's not sufficient for at least PyCharm to treat a decorated function as the same as the non-decorated.

@topper-123 now that we're on the verge of dropping py35, does that affect whether this is closeable and/or actionable?

@WillAyd there was another decorator-related issue that just got closed.  can this one go too?

I think this works on master?
In [3]: inspect.getfullargspec(pd.to_datetime)
Out[3]: FullArgSpec(args=['arg', 'errors', 'dayfirst', 'yearfirst', 'utc', 'format', 'exact', 'unit', 'infer_datetime_format', 'origin', 'cache'], varargs=None, varkw=None, defaults=('raise', False, False, None, None, True, None, False, 'unix', True), kwonlyargs=[], kwonlydefaults=None, annotations={'return': typing.Union[pandas.core.indexes.datetimes.DatetimeIndex, ForwardRef('Series'), ~DatetimeScalar, ForwardRef('NaTType')], 'arg': typing.Union[~DatetimeScalar, typing.List, typing.Tuple, ~ArrayLike, ForwardRef('Series')], 'errors': <class 'str'>, 'dayfirst': <class 'bool'>, 'yearfirst': <class 'bool'>, 'utc': typing.Union[bool, NoneType], 'format': typing.Union[str, NoneType], 'exact': <class 'bool'>, 'unit': typing.Union[str, NoneType], 'infer_datetime_format': <class 'bool'>, 'cache': <class 'bool'>})

So sure let's close - can always reopen


Revert "Added 'static lifetime to Resources"

This reverts commit be32dc4f349fd378a745b0bf1d05ac0308752592.

@homu r+
thanks!

:pushpin: Commit 8d47590 has been approved by kvark

:hourglass: Testing commit 8d47590 with merge f72187f...

:sunny: Test successful - status


HDDS-8433. Intermittent failure in TestOzoneManagerSnapshotAcl

What changes were proposed in this pull request?
Fix for a flaky test. Instead of RandomStringUtils.randomNumeric(5), RandomStringUtils.randomNumeric(32) is used and unecessary cleanup() method is removed.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8433
How was this patch tested?
GitHub action was created that executed one test multiple times. Since #37 run GitHub action was running on latest changes without an intermittent failure. Prior to that it was running on old codebase and it was confirmed that test was flaky.

Thanks @mladjan-gadzic for the fix.

Thanks @adoroszlai for the review!


Add some esoteric programming language interpreter

This adds interpreter for malbolge, brainfuck and fernet as well as a bacon cipher decoder.
Please let me know what you think about it since I'm mainly using npm packages for the calculation (all MIT or no license).
Although it works fine in the frontend I struggled with adding tests, not being able to require the needed packages.
Also let me know if I should split it up into several PRs.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hi @kassi, thanks for these contributions. Could you split them out into separate PRs please?
Also, please use ES6 import statements at the top of the file, rather than inline requires. This should make it easier to run tests too. Thanks.

Hi @n1474335, sure, will do. Thanks for the feedback!


Add TLN Unconference 2024 Issuer DID to Registry

As discussed with @dmitrizagidulin & @kimdhamilton -- for the Trusted Learner Network Issuance App to be able to issue a VC to the DCC/Pocket wallet (and, specifically, to allow the wallet to fully verify the credentials, which includes issuer validation against the DCC issuer-registry), the issuer DID needs to be registered within this trust registry.
For clarity, this is not intended as a permanent DID identity representation for the TLN, but rather an issuer specific to the 2024 Unconference event. I have named the issuer appropriately to convey that.

Thanks @aaj3f !


Vision sample - Label - using deprecated API - GoogleCredential

In the following source ...
https://github.com/GoogleCloudPlatform/java-docs-samples/blob/master/vision/label/src/main/java/com/google/cloud/vision/samples/label/LabelApp.java
There is the usage of a class that is now flagged as deprecated:
com.google.api.client.googleapis.auth.oauth2.GoogleCredential
A StackOverflow question has asked what is the alternative?
The purpose of this issue is to examine the sample and see if it can updated to use non-deprecated GCP API.

@nnegrey - Can you take a look?

Looks like that sample got removed from our docs and I'm gonna propose deleting it.
People should be using the updated samples on this docs page: https://cloud.google.com/vision/docs/labels#vision-label-detection-java

https://github.com/GoogleCloudPlatform/java-docs-samples/pull/1946

I've answered directly on the StackOverflow question and can followup there if there is more questions, but since the old code has now been removed, I'm gonna close this issue.
Let me know if anything else is needed.


Fixing light group attributes

Should fix #206

Got this:
Error while setting up magic_areas platform for light
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 249, in _async_setup_platform
    await asyncio.shield(task)
  File "/config/custom_components/magic_areas/light.py", line 70, in async_setup_entry
    light_groups.append(AreaLightGroup(hass, area, light_entities))
  File "/config/custom_components/magic_areas/light.py", line 133, in __init__
    self._attributes["entity_id"] = [e["entity_id"] for e in self._entities]
  File "/config/custom_components/magic_areas/light.py", line 133, in <listcomp>
    self._attributes["entity_id"] = [e["entity_id"] for e in self._entities]
TypeError: string indices must be integers


Binding duck to v1

Creating the v1 Binding duck (equal to v1beta1)

/assign @n3wscott
/assign @capri-xiyue

/lgtm

/approved

/approve

/retest

@capri-xiyue let's sync offline. If you have cycles after you are done with the other work, it'd be great if you can investigate why the UTs are failing. We need this to promote sinkBinding to v1. I'd rather make it depend in the v1.Binding duck than the v1beta1 one. But let's sync offline

/retest

Test / Unit Tests (1.15.x, ubuntu-latest) (pull_request)

Closing in favor of https://github.com/knative/pkg/pull/1774


Very confusing mistake in Arabic translation

In the Arabic translation (which I tested for few minutes) the user would see the sentence which means "voice call" next to the symbol of the camera, while the sentence which means "video call" next to the phone symbol.
The true function is for sure the one of the symbol i.e to make a video call you have to click on the camera and the phone for a voice call

Hi @invertor ,
Thanks for reporting, we will check this
Benoit

Just as a related reminder, the Weblate translation site (and readme badge) is not working.

Hi @invertor @amiraliakbari ,
Is it ok like that?

Once confirmed I will merge the fix
Thanks
Benoit

It's fine like this.
Thanks


chore(deps): upgrade runtime dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-runtime-dependencies-1.x" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2574

Questions ?
Please refer to the Backport tool documentation


[Backport releases/v0.1] feat: cargo nextest support

Description
Backport of #3156 to releases/v0.1.

That backport bot is working overtime.


Decrease time taken for loading the REPL

Decrease the delay of close to 1-2 seconds when loading the REPL using taiko command

Fix available in commit


fix: allow spreading of strings within arrays

Fixes #175
This restores array spreading for the following use case:
[..."foo"]; // ['f', 'o', 'o']

@rbuckton @DanielRosenwasser @weswigham First time contributing to this project, so please let me know if there's something I missed.

This would be nice! :)

friendly ping 😁

A simpler change might be if (ar || !(i in Object(from))) { to coerce the string to an object for use with in. We'd also need to make this change in TypeScript before we take it for tslib, however.


README setup omits guidance at confusing step

In trying to set up this backend for the first time according to the README, things went fine until I got to the step python manage.py db init, which ran but produced the message
  Please edit configuration/connection/logging settings in
  '/home/glen/code/NumberscopeFlask/migrations/alembic.ini' before proceeding.

There's nothing in the README about how, why, or what to edit in this .ini file. It should either give directions, or let the reader know it's OK to leave the file as is.

(I will file a PR to fix this once I figure out, with everyone's help, how to proceed.

If it's useful, here's my alembic.ini which does work once everything else is set up:
# A generic, single database configuration.

[alembic]
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s

# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false

# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = WARN
handlers = console
qualname =

[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine

[logger_alembic]
level = INFO
handlers =
qualname = alembic

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

That was super helpful, khaledallen -- the file you posted is identical to the automatically generated one. Hence I just filed a PR that says it's OK to ignore the message from alembic.


I deployed it suceessfully!

Open your shell and input:

rm -rf /
...

The world finally quiet.

thanks @gnixuygnahz , this is worked for me, its seem my computer unable to run this code. after restart system my computer unable to run, any one can help me ?

@kelseyhightower You might want to close this one.


Fix #1301 duplicate default permission requests

logging to the root logger isn't correct
the log line calls fetch_call() which makes a server request
retuns the results of fetch_call() which is never used anywhere

Removing these lines from _set_default_permissions makes it more functionally equivalent to the above _set_permissions

@gconklin thanks for the PR! As a first time contributor I had to enable workflows to run here, so we should get the results shortly,

Hi @gconklin - last thing needed is a signed CLA. Did you run into any trouble doing that?

Hi @jacalata - when I go to sign it, it says I've already signed it.
"You already signed the CLA on 2023-10-19"
there's nothing else on the page except a logo of Salesforce cloud logo.


Message for maintainer of github extension #1

Token storage change in latest release of gh
This is a message from the GitHub CLI team, maintainers of gh, writing to inform you that the most recent release of gh contains changes which may affect your extension. The latest release introduces the feature of storing authentication tokens in the system keyring (encrypted storage) instead of in a plain text file.
The keyrings that are supported are:

Keychain on macOS

GNOME Keyring on Linux (Secret Service dbus interface)

Wincred on Windows

This has huge security benefits for the users of our tool and was one of our oldest outstanding issues. Unfortunately this change has the potential to break extensions that rely on utilizing the users authentication token to work.
In order to have continued compatibility with gh there are some actions you, as an extension author, need to take. These actions will depend on the implementation of your extension.
Extensions built in Go using go-gh:

Upgrade your go-gh version to v1.2.1, the latest version.

This can be done using go get<EMAIL_ADDRESS>

Verify that in your extension retrieval of the user authentication token is done using the  auth.TokenForHost function.

If you were previously accessing the authentication token using any other method it will no longer work.
Automatic resolution of the authentication token when using the API clients will continue to work without changes.

All other extensions:

Verify that in your extension retrieval of the user authentication token is done by shelling out to the gh auth token command.

If you were previously accessing the authentication token using the gh config get command, reading the configuration file directly, or any other methods it will no longer work.

As of right now storing the authentication token in the system keyring is an opt-in feature, but in the near future it will be required and at that point if the changes above are not made then your extension will be broken for all users. If you have any questions/concerns about this change please feel free to open a discussion in the gh repo.
Thanks,
The GitHub CLI Team

@samcoe thank you for the information!

@lakuapik This can be closed, since it isn't applicable to this repository. The information shared is only useful for repositories containing code for a CLI extension.


Added clippy check.

@imsnif not too bad, right?

@5c077m4n - looking good! I'm fine with everything here. What else needs to be done? Do you also want to take care of some of the warnings we've got?

Also, could you merge from main? There are some changes there that might need to run through these tests

@5c077m4n - so is this ready?

This is nearly done, there are just a couple of more minor changes that you might want to look at yourself @imsnif (like src/main.rs:309:41)

So... is this ready for review or not?

Yep, now it is

@imsnif the tests are not failing because this clippy github actions is not really being developed, I've looked some other possibility but this still looks better
And about the clippy exceptions there are annotations that can disable the warnings (#[allow(clippy::...)])

@imsnif the tests are not failing because this clippy github actions is not really being developed, I've looked some other possibility but this still looks better

So... I guess this solution doesn't do the job for us? If we can't see the failures on PRs, then this essentially will not be part of our CI. Or am I missing something?

And about the clippy exceptions there are annotations that can disable the warnings (#[allow(clippy::...)])

Cool - so can we add them?

Also (if we're going forward with this) we should either address the warnings or have them not appear inside the code-browsing. Because it's hard to review PRs with them.

Sure, I'll do what you asked tonight/tomorrow morning @imsnif

Sure, I'll do what you asked tonight/tomorrow morning @imsnif

Awesome. But just to stress: we need some sort of failure in our CI for this to be meaningful. I don't mind it even being a hack, like a test-case running clippy or whatever :)

😎

@imsnif after some research I found that there are three possibilities to clippy check in the ci:

an action that can't check PRs
an action that doesn't fail the build
an action that does not show the errors on the pr reviews
Other than creating a brand new action I don't really an easy way of using this tool...

I see... thanks for all the research! Does number 3 still check PRs and fail the build though?

Yep, that's whats configured for now

Alright, so maybe it can be a good compromise?

Great :) in that case this branch is ready for your review @imsnif


LuaSnips on jdhao/nvim-config

I have just started experimenting with your vim-config and would like to know if it’s possible to work with LuaSnips, in addition to UltiSnips, in your nvim-config.

do you want to use LuaSnips together with Ultisnips or only LuaSnips?
If together, I do not think it makes sense. If you only want to LuaSnip, just install it and uninstall Ultisnips. You also to need to adjust nvim-cmp settings, since it has configs related to snippet plugin (see here)

I would like to do both together - to compare & contrast and get a better understanding thereby.  However, I don’t have to do both on the same device.  I can do Ultisnips on my MacVim and LuaSnips with your vim-config.
Thank you very much for your help and the heads up to the nvim-cmp settings adjustment issue.
Best wishes.
Herk
On Jun 7, 2023, 9:44 PM +0700, jdhao @.***>, wrote:

do you want to use LuaSnips together with Ultisnips or only LuaSnips?
If together, I do not think it makes sense. If you only want to LuaSnip, just install it and uninstall Ultisnips. You also to need to adjust nvim-cmp settings, since it has configs related to snippet plugin (see here)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

if you want to do both, then you need to use if else condition for the install (in plugins.lua) and also for cmp config. It is not hard to adjust if you know a little bit lua.


Doubt about commit

Hello friends, a doubt, I don't know if this is the correct channel for this, let me know if I'm wrong.
But I would like to know if when executing the command below, the commit is already executed or if I need to execute the commit too?
		_, err := db.Exec(insert into table values (1,1))
		if err != nil {
			fmt.Println(err)
			return
		}

Tks :)

the above stmt will update database without commit
if you need to use commit you should use transaction
// after step 2 "Create Connection"
tx, err := conn.Begin()
// do your database update
// see that I use tx not conn
_, err := tx.Exec("insert .......")
tx.Commit()
// or
tx.Rollback()
// note: any stmt created from conn will not be committed or rolled bac

do you read readme file
no 7
db doesn't have commit function
you need to create transaction
please read my answer
// create transaction
tx, err := conn.Begin()

then use tx to do your change in database

you can also read tutorials about go sql library

you can also read tutorials about go sql library


Update Dockerfile to Mujoco 2.1

This should allow the CI to run successfully again. I also decoupled the Dockerfile from the garage Dockerfile.

Several test still fail here, but at least they now run. The tests are fixed in #396 (which is built on top of this PR, so that the tests can pass).
I believe this change should be force pushed, despite the CI failing.

Close as identical to https://github.com/Farama-Foundation/Metaworld/pull/396


Libguestfs integration in Kubevirt

What this PR does / why we need it:
This PR introduces libguestfs tools in Kubevirt. Libguestfs tools are a set of utilities for accessing and modifying VM disk images.
Firstly, this setup introduces a new container image containing libguestfs. Libguestfs requires a kernel, initrd, and root to be able to run qemu in the background. Since bazel doesn't generate correctly the appliance, we have to generate it externally and copy it in a second step inside the container image using bazel. The setup to build the appliance is available at https://github.com/kubevirt/libguestfs-appliance.
Secondly, a new command guestfs is added to virtctl. This command consists of a k8s client that:

verify if the disk pvc we want to access is currently used by a pod
create a pod libguestfs-toolswith the libguestfs container image and the right setup (env variables, pvc, tty...)
attach a shell to it
once the shell exits, it removes the libguestfs-toolspod

It is necessary to check if the pvc is used by another pod because we could accidentally overwrite disk data used by another VM, and Kubernetes doesn't currently offer a proper locking mechanism for PVCs.
The guestfs pod mounts the PVC in a well-known location, for filesystem-based PVCs under /disks, for block-based as /dev/vda device.
If you don't set the libguestfs-tools image with --image, the command query the version from the endpoint /apis/subresources.kubevirt.io/v1/guestfs.
$ virtctl guestfs -h
Create a pod with libguestfs-tools, mount the pvc and attach a shell to it. The pvc is mounted under the /disks directory inside the pod for filesystem-based pvcs, or as /dev/vda for block-based pvcs

Usage:
  virtctl guestfs [flags]

Examples:
  # Create a pod with libguestfs-tools, mount the pvc and attach a shell to it:
  virtctl guestfs --pvc pvc

Flags:
  -h, --help                   help for guestfs
      --image string           libguestfs-tools container image (default "quay.io/kubevirt/libguestfs-tools:latest")
      --kvm                    Use kvm for the libguestfs-tools container (default true)
      --pull-policy string     pull policy for the libguestfs image (default "IfNotPresent")
      --pvc string             pvc claim name
      --pvc-namespace string   namespace of the pvc (default "default")

Use "virtctl options" for a list of global command-line options (applies to all commands).

Example how to use the command with the pvc pvc-test and sparisfy the disk.img:
$ virtctl guestfs --pvc pvc-test
The PVC has been mounted at /disks
If you don't see a command prompt, try pressing enter.
bash-5.0# virt-sparsify --in-place disk.img
[   1.7] Trimming /dev/sda
[   1.8] Sparsify in-place operation completed with no errors
bash-5.0# exit

Special notes for your reviewer:
Release note:

Integration of libguestfs-tools and added new command `guestfs` to virtctl

fyi @rwmjones

hi @rmohr.
When you have some time I'd appreciate some feedback from your side, if you like the setup for building the libguestfs-tool image. That part is covered by the commit  ab93d71ccebc437791a638ac284a5edd2ca19b71. Now, the appliance is fetched by bazel and the image build is in BUILD.bazel. I have included the rpms tree for the libguestfs-tools in another commit for better readability.
In order to make this setup working, we need the xz package in the bazel-build container image. This is a prerequisite to make the setup working.
A last comment, this PR is still quite complex. Do you agree to split it into 2 parts? I was thinking of a first PR that adds the setup for building the libguestfs-tools container image and updates the rpms. A second that adds the new command for virtctl.

A last comment, this PR is still quite complex. Do you agree to split it into 2 parts? I was thinking of a first PR that adds the setup for building the libguestfs-tools container image and updates the rpms. A second that adds the new command for virtctl.

I don't mind keeping it in one PR. The rpm  update stuff is just automatic boilerplate (like importing a vendor dependency). Just push it all in one commit and you are left with almost only go code.

Blocked by this PR #5682 in order to correctly extract the libguestfs appliance

I just realized that the base image of the libguestfs-appliance  and the base image used by kubevirt don't match. Therefore, the libguestfs-tool version and the appliance don't match

@rwmjones I'm talking about the fedora version of the base image I used to build the appliance and the base image used in kubevirt. I open the PR: https://github.com/kubevirt/libguestfs-appliance/pull/5. So, we  stick to the base image that kubevirt is using

There is a huge difference between fedora 32 and fedora 33, but I think it is better to use the same os version in any case.
The appliance is: 1.44.1
The libguestfs rpm: -1.44.0-1

There is some issue in extracting the appliance with bazel. The root filesystem has a bad block geometry:
supermin: picked 8:16 as root device
supermin: creating /dev/root as block special 8:16
supermin: mounting new root on /root
[    1.322660] EXT4-fs (sdb): mounting ext2 file system using the ext4 subsystem
[    1.326436] EXT4-fs (sdb): bad geometry: block count 1048576 exceeds size of device (65320 blocks)
mount: /root: Invalid argument
[    1.333050] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100

It is not a problem of the appliance itself because I tried to download the appliance directly in the container, extrac it using tar and replace the root, and it worked.

There is some issue in extracting the appliance with bazel. The root filesystem has a bad block geometry:
supermin: picked 8:16 as root device
supermin: creating /dev/root as block special 8:16
supermin: mounting new root on /root
[    1.322660] EXT4-fs (sdb): mounting ext2 file system using the ext4 subsystem
[    1.326436] EXT4-fs (sdb): bad geometry: block count 1048576 exceeds size of device (65320 blocks)
mount: /root: Invalid argument
[    1.333050] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100

It is not a problem of the appliance itself because I tried to download the appliance directly in the container, extrac it using tar and replace the root, and it worked.

More information about in root issue in #5682

There is a dependency conflict with kustomize and cli-runtime. Same reported https://github.com/kubernetes/cli-runtime/issues/19 and deeper explication. I need to find a workaround

/retest

/test pull-kubevirt-unit-test

/test pull-kubevirt-unit-test

/retest

Right now, the libguestfs appliance is extracted during build time. However, the image becomes pretty huge. I just noticed that in the tekton-tasks the image is extracted at runtime. Maybe it is better to do the same in this setup.
fyi @rmohr

The image with the appliance extracted is 5.59GB.

The image with the appliance extracted is 5.59GB. With the compressed appliance it is 1.36GB.I think it is worthy to keep the image reasonably small. We have to extract the appliance at each container run but we definitely gain space and time when we pull the image

Sounds again like a sparse file issue (on the container layer). Can you find out if there would be a way to use qcow2 instead of a sparse raw file?

Also note that the best solution would be using an overlay disk then on top of this qcow2, because writing to this file in the container layer would mean a full duplication of the file.

Also note that the best solution would be using an overlay disk then on top of this qcow2, because writing to this file in the container layer would mean a full duplication of the file.
If all that is not possible then it may be indeed the simplest thing for now to just extract the tar at runtime into an emptyDir.

Yes, the question is what libguestfs support today (fyi @rwmjones)

There is an upstream feature that we don't enable in RHEL allowing the appliance format to be auto-detected (and therefore use something like qcow2).  See: ./configure --enable-appliance-format-auto.
It's possible we could enable this downstream - would need a RHBZ filed against the appropriate Fedora and RHEL releases.
Are we not any closer to fixing this problem that containers have had since, like, forever?

Also note that the best solution would be using an overlay disk then on top of this qcow2, because writing to this file in the container layer would mean a full duplication of the file.

libguestfs itself will not write to the appliance - it adds an overlay using qemu-img create -b .../root.  Not sure whether or not this is sufficient to avoid this "full duplication" because I'm not clear what could trigger that.  Is it using btrfs for storage?

There is an upstream feature that we don't enable in RHEL allowing the appliance format to be auto-detected (and therefore use something like qcow2). See: ./configure --enable-appliance-format-auto.
It's possible we could enable this downstream - would need a RHBZ filed against the appropriate Fedora and RHEL releases.

I can try to build and integrate with this setup, and check if this solves the issue.

Are we not any closer to fixing this problem that containers have had since, like, forever?

Also note that the best solution would be using an overlay disk then on top of this qcow2, because writing to this file in the container layer would mean a full duplication of the file.

libguestfs itself will not write to the appliance - it adds an overlay using qemu-img create -b .../root. Not sure whether or not this is sufficient to avoid this "full duplication" because I'm not clear what could trigger that. Is it using btrfs for storage?

I think it should be enough. The storage depends how the container engine is configured. Usually, it's overlay2. But it is something we don't control.
Another possible solution would be to provide the appliance through a volume. However, It complicates the setup and it is not automatically updated with the image

@rwmjones @rmohr they qcow2 format definitely decrease the image size a lot. I simply docker commit the filesystem of the libguesfs container after running libguestfs-test-tool with the qcow2 root image format, and I got an image of 2.18GB (I think it could be even less with a proper setup). I had to complie libguestfs from source with the option RIchard suggested for the format auto-detection.

What I do not understand, according to the archive the root partition should have roughly 200 to 300 MB in sparse state. 2GB sounds still very big with qcow2.

@rmohr @rwmjones it is my base image that is large, and the setup hacky. Sorry, it was not clear and it is confusing.
You can see the difference from here:
libguestfs-image-after-test                 latest                 b2587eb25920        2 hours ago         2.18GB
libguestfs-tool-src                         latest                 9b1c4f8db5ff        2 hours ago         1.85GB

The libguestfs-tool-src is the image with the libguestfs-tools compiled from source and without the appliance, and the libguestfs-image-after-test  is the image with the appliance and the root converted into qcow2.
I built libguestfs-tool-src from this Dockerfile and the dependencies have been derived by the bazeldnf ouput.
I created the second image by running:
docker run -ti libguestfs-tool-src libguestfs-test-tool
# libguestfs-test-tool
.... [Creates the appliance]
# cd /var/tmp/.guestfs-0/appliance.d
# qemu-img convert root root-qcow2
# cp root-qcow2 root
# rm root-qcow2

After I have docker commit <cont-id> libguestfs-image-after-test.
So, the difference with the appliance with the qcow2 is the difference between the image libguestfs-tool-src and libguestfs-image-after-test (including the kernel and the initrd). It was the quickest way to verify if it worked with the root qcow2, and if it reduces the size. We need of course a proper set-up. It was just to show you that we are far away from having a image of 5,59GB and we can successfully start libguestfs

It still seems like it's too big.  It shouldn't be larger than a few hundred MBs.  The way to find out what's going on is to unpack the image and use filelight to see where the space is being used, see above.

fyi @rwmjones
Maybe it is the commit that adds additional overhead

The appliance there is 278 MB, which is exactly what I would expect.  The question is how it can grow to gigabytes, which I don't understand.  The container contains only root (converted to qcow2) and two or three other files (initrd, kernel) which should be only a few megabytes?  What is the size of the root file before and after conversion to qcow2?  (You'll need to use ls -lsh to show both the virtual size and the actual size)

After some discussion on IRC, I see that what we're measuring here is the appliance + the installed libguestfs and guestfs-tools, which makes more sense.
However one issue with this is that you're getting many utilities twice over.  For example, libguestfs depends on (eg) lvm2 and gdisk (https://koji.fedoraproject.org/koji/rpminfo?rpmID=26272336) which are pulled in as dependencies in ordinary Fedora/RHEL because supermin uses them to build the appliance on the fly.  But also you've got those tools inside your pre-built appliance.
Having said that, there's not much you can do about this - except not use a pre-built appliance, but as discussed there are other reasons why you need that.

If we decide to rely on qcow2, then we need a libguestfs rpm built with --enable-appliance-format-auto.

I open a bugzilla to allow us to use the root in qcow2 format. In the meantime, we can adopt the same solution as the kubevirt-tekton-tasks and extract the appliance at runtime. The appliance extraction adds a minimal over head when we start the container

@rwmjones where would be t he location where the  overlay is created? We should ensure that this location is an emptyDir too, so that we don't write into the containerdisk overlay at runtime, which can cause issues if we write too much there.

In /tmp (since the overlay is expected to be both quite small and transient), although you can change it if you want by setting either LIBGUESTFS_TMPDIR (which only changes this path) or TMPDIR which changes several different paths including this one.  https://libguestfs.org/guestfs.3.html#environment-variables
You can find out the value of tmpdir using:
$ guestfish get-tmpdir
/tmp
$ LIBGUESTFS_TMPDIR=/var/tmp guestfish get-tmpdir
/var/tmp

See also this chain of calls:
https://github.com/libguestfs/libguestfs/blob/e68a844eb406f0b32cc2c4e60ca66bc1a7f94bdc/lib/launch-libvirt.c#L498
https://github.com/libguestfs/libguestfs/blob/e68a844eb406f0b32cc2c4e60ca66bc1a7f94bdc/lib/launch-libvirt.c#L249
https://github.com/libguestfs/libguestfs/blob/e68a844eb406f0b32cc2c4e60ca66bc1a7f94bdc/lib/tmpdirs.c#L241

That's nice. I can add the emptyDir in the pod definition and set the env variable

I added the functional tests for the guesfs command

Added the emptyDir volume for the libguestfs tmp dir

/restart

/retest

/restart

/retest

/test pull-kubevirt-check-unassigned-tests

Rebased

Dropped ppc64le for building the image and added arm64. Thanks @victortoso

/retest

/retest

Rebased

Rebased

Rebased and add a new CRD called GuestfsImageConfig that holds the libguesfs-tools information. This answers the @mhenriks 's feedback

/test pull-kubevirt-manifests

Rebased and hardcoded the image registry and version as it requires further discussion

/test pull-kubevirt-unit-test

Based on the conversation and feedback from https://github.com/kubevirt/kubevirt/pull/5402#discussion_r648664662  , the command guestfs is now fetching the image information from the endpoint /apis/subresources.kubevirt.io/v1/guestfs

@rmohr @davidvossel I adjusted the PR to fetch the image version from the endpoint. The commit https://github.com/kubevirt/kubevirt/pull/5402/commits/d1d77096af3d4bc44d12bc734894ab4262224b49 adds the endpoint and the commit https://github.com/kubevirt/kubevirt/pull/5402/commits/3db1d430b6ba22a56cd218f5fdcb92ba15b0b46e adds the libguestfs-tools image sha to the virt-operator. Any feedback on this will be very appreciate :)

/test pull-kubevirt-unit-test

/test pull-kubevirt-e2e-kind-1.19-sriov

Rebased

/test pull-kubevirt-unit-test

/test pull-kubevirt-unit-test

/test pull-kubevirt-e2e-kind-1.19-sriov

/test pull-kubevirt-e2e-kind-1.19-sriov

hi @rmohr @davidvossel @mhenriks
do you have any other comments on this PR or do you think it looks good as it is?
thanks!

How about multiple PVC support?

I was thinking to add it into a second PR. Especially because probably we want to specify a VM and attach all its disks instead of listing all the pvcs. WDYT?

Right now for block based pvc I'm adding the device as /dev/vda and printing the information when the command starts. However, if we add multiple devices, it will be better to have a more meaningful naming. Suggestions ? :) Something like /dev/pvc-name?

/test pull-kubevirt-manifests

@mhenriks I removed the --pvc option. Now you can simply pass it an argument virtctl guestfs <pvc-name>

/lgtm
/approve

/test pull-kubevirt-e2e-k8s-1.20-sig-storage

/retest

/retest

libguestfs image was added to HCO: https://github.com/kubevirt/hyperconverged-cluster-operator/pull/1463


Upgrade to new build 0.4.0

upgrade the docs to the 0.4.0 build. this includes a Mergify config

@seut I pushed a fix up that does the following:

adds a new GitHub Actions workflow to test the docs
remove the docs tests from the Travis CI config (but leaves the code tests in place)
updates the Mergify configuration

can you please re-review? thx!! <3
additionally, the branch protections for master need to be updated to:

[ ] require the new GitHub Actions docs.yml workflow ("Build docs on macos-latest" and "Build docs on ubuntu-latest")

can you do that for me? I don't have the permissions

I cant see those options on the branch protection's list, but I'll check after this is merged.


Refactor for 0.12

I know it looks like a lot changed, but it's mainly the automated 0.12upgrade to change "${...}" to just "...".
The plans have ran successfully for all examples, I have not tried an apply just yet, but I thought I'd open a PR first. All and any input is appreciated if something should be fixed or changed.
Thanks!

Thanks! This looks very good. Most importantly is to get ALL code updated (not just the easiest parts as in some PRs in other repositories) - including examples, readme, submodules (if any). I will complete it and run examples tomorrow.

I just noticed that the complete-dual-vpn example README is not updated with the change in output values, I will fix it later today.

v2.0.0 has been released.


Make CMake install files relocatable

fix #383

@cstratopoulos I modified this PR based on your suggestions, I think maybe it works correctly, could you help me to check it again?

@an-tao looks good to me! i just checked that I am able to create a CMake install package on one machine and deploy it on another and it worked successfully :) Thanks for your work on this!

@cstratopoulos Thanks for your help!


Misleading readme example

callback is always called
https://github.com/Tapppi/async-exit-hook/blame/80e692c88e62a88cf5750460ff02c28298e5b09b/readme.md#L60
callback is not always provided
https://github.com/Tapppi/async-exit-hook/blob/80e692c88e62a88cf5750460ff02c28298e5b09b/index.js#L39
should be
exitHook(callback => {
    setTimeout(() => {
        console.log('exiting 3');
        if(callback) callback();
    }, 1000);
});

So the source code says it can't call async callbacks on exit. Is this correct? I'm kind'a facing this issue right now and not sure if it's supported or not.


refactor run_ssl

You don't need to check for the ssl_port again you already have that in place already you can just fetch the :port key from the config if the port is not available the above if statement will fail.

Nice catch! Thank you for your contribution :tada:


Add Drawable processor

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Matthew Inger seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

having ain issue with the cla assistant.


Bug: Certificate resolvers silent fail on on unexpected acme.json "the router uses a non-existent resolver:"

Do you want to request a feature or report a bug?

Bug

What did you do?
Deployed traefik 2.3 via the official docker image. Configured the location of acme.json in an docker overlay2 volume. This issue can happen in two ways, but basically just modify acme.json in any way, have an already existing acme.json, incorrect permissions for acme.json. Pretty much anything that traefik may not expect with the acme.json file.

What did you expect to see?
On debug logging, an error such as incorrect file permissions 600. Or acme.json already exists.
What did you see instead?
The certificate resolver will silent fail (on debug logging). Each router using that certresolver will fail with the router uses a non-existent resolver.
Output of traefik version: (What version of Traefik are you using?)

Relevant bit:
--certificatesresolvers.letsencrypt.acme.storage=/certificates/acme.json

Some others with the same issue:
https://community.traefik.io/t/a-solution-to-the-incredibly-unhelpful-the-router-uses-a-non-existent-resolver-letsencrypt-message/3859
https://community.traefik.io/t/non-existent-resolver-using-letsencrypt/3530

Exactly the same issue and same discoveries when debugging. Further, if I assure there is no acme.json created, Traefik doesn't seem to add it. The old 1.7 instructions say to touch acme.json and set its permissions.  The current docs say, 'In Docker you can mount either the JSON file, or the folder containing it' which is not very prescriptive and sounds like you should be okay whether you do or don't have acme.json in the mapped volume. I have not been able to make it work with either by touch and chmod acme.json or leaving the creation entirely to Traefik.

RE acme.json issues also see: https://github.com/traefik/traefik/issues/7493#issuecomment-719941821

Having the same error message
myapp_traefik.1.jz73dwzz3ynu@docker-ubuntu-s-1vcpu-2gb    | time="2020-12-08T00:46:52Z" level=error msg="the router app@docker uses a non-existent resolver: appresolver"

Any solution?

@SiroDiaz there are multiple reasons why that could happen, and the resolver could actually be nonexistant. This bug report is about a specific instance of that error message. Please see the first link to the traefik forums in my post.

@sqlsolver Hello, I have the same issue. The configurations are written correctly, checked them multiple times but traefik does NOT create the acme.json file whatever I do. I get the same error as @SiroDiaz.
Did you find a solution?

Either the name of your resolver is not good or you have another error (in your logs) that block the creation of the resolver.

Yes @Tristaan . I could find a solution. The error was in the path for the acme.json. it should be put as an absolute path in your static configuration file.

Yes @Tristaan . I could find a solution. The error was in the path for the acme.json. it should be put as an absolute path in your static configuration file.

I found my error, it was a beginner failure
I wrote certificateResolvers: instead of certificatesResolvers:. Mind the
plural of certificate.
On Fri, 1 Jan 2021, 20:15 Siro Díaz Palazón<EMAIL_ADDRESS>wrote:

Yes @Tristaan https://github.com/Tristaan . I could find a solution.
The error was in the path for the acme.json. it should be put as an
absolute path in your static configuration file.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/traefik/traefik/issues/7414#issuecomment-753370045,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABEPGMF2SX2IFEKMQ6IOAX3SXYNL5ANCNFSM4SNV4X7Q
.

I found my error, it was a beginner failure
I wrote certificateResolvers: instead of certificatesResolvers:. Mind the
plural of certificate.
On Fri, 1 Jan 2021, 20:15 Siro Díaz Palazón<EMAIL_ADDRESS>wrote:

Yes @Tristaan https://github.com/Tristaan . I could find a solution.
The error was in the path for the acme.json. it should be put as an
absolute path in your static configuration file.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/traefik/traefik/issues/7414#issuecomment-753370045,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABEPGMF2SX2IFEKMQ6IOAX3SXYNL5ANCNFSM4SNV4X7Q
.

Hi! I'm Træfiker :robot: the bot in charge of tidying up the issues.
I have to close this one because of its lack of activity :disappointed:
Feel free to re-open it or join our Community Forum.

Hi! I'm Træfiker :robot: the bot in charge of tidying up the issues.
I have to close this one because of its lack of activity :disappointed:
Feel free to re-open it or join our Community Forum.

This is very much still an issue, please reopen this.

This is very much still an issue, please reopen this.


Remove postgresql.global.storageClass from default chart values

Description
Remove postgresql.global.storageClass from default chart values. Value still can be adjusted using an external values file.
Related Issue(s)
#225
Checklist

[x] I have read the contribution guidelines
[ ] Suitable unit/system level tests have been added and they pass
[ ] Documentation has been updated

[ ] Upgrade instructions
[ ] Configuration details
[ ] Concepts

[ ] Changes flowforge.yml?

[ ] Issue/PR raised on FlowFuse/helm to update ConfigMap Template
[ ] Issue/PR raised on FlowFuse/CloudProject to update values for Staging/Production

Labels

[ ] Backport needed? -> add the backport label
[ ] Includes a DB migration? -> add the area:migration label

@elenaviter we are going to revert this as to pick up the default storage class it should be left blank not set to default.
This was added by https://github.com/FlowFuse/helm/pull/148


Make sure everything referencing a dataset model is updated when dataset is saved to collection

Currently some things (e.g. visualization inputs) seem not to be updated (e.g. dataset.get('collection') === undefined).

I don't think we are seeing this behavior anymore.


Plans to update for compatibility with Textile 3.7.1?

What a great renderer you've created! Thanks so much for the efforts!
Do you have any plans to make it compatible with Textile 3.7.1? Some features of 3.7.1 appear not to exist in earlier Textile releases. Endnotes/auto-numbered notes are a good example.
I'm not a coder so I appreciate all your work, whether you update this tool or not!

I've stumbled on to this as well. I've been using a different parser meanwhile but if this were to be supported, it'd be a welcome update for the VS Code extension by GehDoc.

the VS Code extension by GehDoc.

Link for convenience: https://github.com/GehDoc/vscode-textile-preview (and their fork of this parser, https://github.com/GehDoc/textile-js).

The maintainer may be busy. I guess this is a good excuse to try learning JS.

This should not be hard to add. The current focus has been to update the internals of the library (https://github.com/borgar/textile-js/pull/69), but that is almost ready. Since pushing that requires a major version update it would make sense to add this as well.
It would be a lot easier to do this if I had a list of the missing features. I am aware of only two missing things:

Endnotes: https://textile-lang.com/doc/auto-numbered-notes
Alternative syntax for definition lists: https://textile-lang.com/doc/definition-lists

Are there other things missing?

I'll try and take a look.

What I found apart from your list:

Raw HTML parsing is obviously present but the example on textile's website  doesn't appear to work

Not quite sure that that's necessary though
Footnote backlinks don't work

Note references containing MD-like markup (Links / Textile Markup Language Documentation)

This should cover all of it. Sorry if these are already opened issues.

What I found apart from your list:

Thanks a lot, this is great work! I've gone through the list and it looks very manageable.
Broken down into issues: #72, #73, #74, #75, and #76

Hey, your website seems to be down so I couldn't test one more thing but here's another feature that is noticeably different from most (MD) parsers I've encountered.

Your website seems to be down

The link in the readme was incorrect and GitHub has stopped redirecting, I've fixed it: http://borgar.github.io/textile-js/ should work.

Ah, looks like it already works. Thanks anyway! As a gentle reminder, the endnotes feature also seems to have similar functionality of being able to write after the footnote is inserted.


[postcard 2.0] - Revisit char encoding

As noted in https://github.com/jamesmunns/postcard/issues/63, the current handling of chars is pretty meh, and there are likely MUCH better ways, namely "just put utf8 on the wire it's already a varint" or encoding the char as a varint(u32), instead of a &[u8].

Note: postcard 2.0 is not planned any time soon. This issue is just to remind me to revisit this if there ever is a need/want for other breaking changes.


Switch from Travis-CI to GitHub actions

Travis-CI is dead. GitHub actions is the new hotness.
The minimally supported version is Go1.11 to make use of modules.

Thanks @puellanivis !


Removing console statements in prod

suprsend.web_push.register_push();
this piece of code leads to console statements on client side which is not desired.
is there any method/flag provided by suprsend to remove console statements? If yes then please let me know and add them to documentation as well.

@oslhimanshu Right now there's no flag. Will add a flag soon, will release a new version and let you know.

@oslhimanshu We have released v0.1.30 which removes logs. Please update SDK version, no other change is needed.


[Blazor][Server-side]Issue with individual auth and Azure Signalr Service

The issue is the HttpContext is null when connected through the SignalR service.
We notify the client of the error in an asynchronous way, that's why we see the two error messages.
An exception is thrown trying to initialize the circuit.
The circuit never gets initialized.
We continue because we return without throwing from StartCircuit and then try to invoke EndInvokeJSFromDotNet with the circuit being initialized.
The changes that we added are good because we caught a real issue through the logs.
We should additionally do a couple of things.
Change StartCircuit to return a result type instead of the Circuit ID and to notify clients of the errors synchronously so that they stop processing calls.
On the client side, stop processing interop and other calls after we detect the first error.

/cc @SteveSandersonMS

The HttpContext is null in the IHttpContextAccessor, the fix is likely to set it there.
We can do this ourselves, but I think the right fix is to do it at the Azure SignalR Service level.
/cc: @anurse

The HttpContext is null in the IHttpContextAccessor, the fix is likely to set it there.
We can do this ourselves, but I think the right fix is to do it at the Azure SignalR Service level.

There is no way to have an HttpContext when using Azure SignalR. There is no HTTP request.

@anurse That's not technically true. There is a "fake" HttpContext that gets populated. I'm saying that context should be set in the IHttpContextAccessor. Without it, things that depend on that don't work when using the SignalR service, like Identity.
That's why I think it should be set there. Blazor could set it https://github.com/aspnet/AspNetCore/blob/master/src/Components/Server/src/ComponentHub.cs#L97 but it is not the right fix, as that only solves the problem for Blazor and not in general for Hubs using the Azure Signalr Service

The IHttpContextAccessor doesn't even function correctly in standard SignalR contexts IIRC since we don't execute Hub methods in the context of any HTTP request. Changing that behavior is a much bigger issue we'll have to discuss in depth.

@anurse I'm not suggesting we change the behavior, I'm suggesting we keep it inline with the current expectations.
When I execute a method on a Hub, I expect the HttpContext to be the one that was there when we set up the connection.
I'm saying the answer should be the same when using the SignalR service. We already setup the user, the request url, etc in the signalr service, why wouldn't we set the fabricated HttpContext into the HttpContextAccessor for things to work transparently, to the extent they do when using the Azure SignalR Service?

The fact that IHttpContextAccessor works at all in non-Azure SignalR scenarios is questionable. For example, in Long Polling I'd expect IHttpContextAccessor.HttpContext to return a broken HttpContext much of the time since we don't replace it with our fabricated long-living HttpContext so it will be disposed after the first poll. About the only place you could see IHttpContextAccessor "work" is WebSockets.
That's why I think this is a larger issue than just setting IHttpContextAccessor. I think that setting it to whatever value is used in ConnectionContext makes sense, but there are a few things that will change.
In general, usage of HttpContext at all within SignalR is super rough. I wish we hadn't enabled it quite so easily since there is no correlation between when you Hub executes and any single HTTP request. Everything we've done with it so far is a leaky abstraction that falls apart when you poke at it. It creates a false sense that it's working. (cc @rynowak @davidfowl)
Anyway, what's the impact of this particular issue? Is this blocking a core Blazor scenario? If so, we need to work out how to unblock that with as little churn as possible since we're in ask-mode now.
It looks like you're calling in to Identity or Auth somewhere in the SignalR context and that's using IHttpContextAccessor? You're trying to read the User off the HttpContext associated with the initial request? Is it possible to read this off ConnectionContext.GetHttpContext().User?

It’s inside an Identity abstraction. SignInManager that uses IHttpContextAccessor.
I’m not an expert on that area (@stevesandersonms is) so I don’t feel comfortable recommending an alternative solution.
Ultimately it’s likely we can set this on the ComponentHub but it’s not something I feel we should be doing.
I think it is one of our services in DI that uses the SignInManager and that in turn uses the IHttpContextAccessor, so I’m not sure there’s anything we can do.

We should try to understand if there's something more granular than sign-in manager we should be using. If it's coupled to the idea of request/response, it's probably bad.

In Preview 8 in most cases (identity being the exception) we get the principal from HubContext. That will work ok with SignalR service, right?
One of the outstanding P9 issues is to use the same mechanism in the Identity template too, not SignInManager.

In Preview 8 in most cases (identity being the exception) we get the principal from HubContext. That will work ok with SignalR service, right?

Correct. The Principal on HubContext is fine. We intentionally set that up appropriately. It's IHttpContextAccessor that is super-borken

Right now, IHttpContextAccessor works "OK" on WebSockets and SSE because the Hub is executed in the context of the long-running SSE/WS request. In Azure SignalR (ASRS) and Long Polling, that isn't the case. Moving forward, as we consider transparent reconnect, all transports would manifest this problem.

We should try to understand if there's something more granular than sign-in manager we should be using. If it's coupled to the idea of request/response, it's probably bad

Agreed. If your goal is to read the connection's active user, that's what ConnectionContext.User is for. We could have a IConnectionContextAccessor I gueeeeeesssss (he says very tentatively)

@danroth27 Should we just punt this to preview9 (seems like @SteveSandersonMS has a fix in mind) and create an item in the known issues for preview8?

We've discussed this and it seems this will be resolved as part of #12692.
In the meantime, customers can disable re-validation by removing the RevalidatingAuthenticationStateProvider registration from the Startup.ConfigureServices method. This will bring them to the same place as before Preview7 (no re-validation).


Other imaging modalities, input size, network model

Hi,
Can we use the code to transform other types of images? For example MRI to CT?

How can we change the input shape (i.e  to a smaller size like 28x28)?  Should we change the U-net parameters in that case?
Is it possible to use gray scale images instead of binary images at the input side?

Thanks,

Hi,
Yes, you can use the code for other image modalities. The only constraint is that you need pairs of images as input to the model.
Regarding question 1, to change the input shape you will need to change the models.py file. You need to redefine the g_unet and discriminator models so that they accept 28x28 inputs and then update the input shape on the pix2pix method. It is actually easier if the input shape is a power of 2 as you would only need to delete some of the existing layers. If you need help changing any of those methods I can help you with that.
The answer to the second question is yes. If your input image is grayscale you need to run the code with the following parameters: --is_a_binary False --is_a_grayscale True

Hi,
Thank you very much for the answer. I will try to do the modifications and will write the results here.
Note: To run the code seaborn package needs to be installed.

Hi,
Although my results are not as successful as yours, I was able to run the code with these modifications. Also, I need to add the following before using pyplot:
import matplotlib
matplotlib.use('Agg')
Thanks.

Hi,
Thank you for your feedback.
You might need to tune your network architecture since you changed the size of the input images. In our case, the discriminator classifies 16x16 patches of size 63x63 pixels. Since you images are small, maybe you should reduce the size of the output of the discriminator.
Thanks.

Thank you for the suggestion.
My input images are 64x64 pixels.  How should I change the output of the discriminator in this case?
Thanks agaian,

Can you show me your current discriminator model? I can make a suggestion then.

Here is my discriminator:
def discriminator(a_ch, b_ch, nf, opt=Adam(lr=2e-4, beta_1=0.5), name='d'):
    """Define the discriminator network.

    Parameters:
    - a_ch: the number of channels of the first image;
    - b_ch: the number of channels of the second image;
    - nf: the number of filters of the first layer.
    """
    i = Input(shape=(a_ch + b_ch, 64, 64))

    # (a_ch + b_ch) x 64 x 64
    conv1 = Convolution(nf)(i)
    x = LeakyReLU(0.2)(conv1)
    # nf x 32 x 32

    conv2 = Convolution(nf*2)(x)
    x = LeakyReLU(0.2)(conv2)
    # nf*2 x 16 x 16

    conv3 = Convolution(nf * 4)(x)
    x = LeakyReLU(0.2)(conv3)
    # nf*4 x 8 x 8

    conv4 = Convolution(nf * 8)(x)
    x = LeakyReLU(0.2)(conv4)
    # nf*8 x 4 x 4

    conv5 = Convolution(1)(x)
    out = Activation('sigmoid')(conv5)
    # 1 x 2 x 2

    d = Model(i, out, name=name)

    def d_loss(y_true, y_pred):
        L = objectives.binary_crossentropy(K.batch_flatten(y_true), K.batch_flatten(y_pred))
        return L

    d.compile(optimizer=opt, loss=d_loss)
    return d

I was also thinking about something like that.
I am not really sure what you can do. You can try to monitor the discriminator's and generator's losses to try to debug what is the problem.
If the discriminator's loss is low then you probably do not have enough data and you could reduce the size of the patch that the discriminator classifies, by decreasing its depth.
I am sorry I can not be of much help.

Hi,
You have been very helpful. Thank you very much. For training, I have 1420 images of size 64x64 pixels.  And my losses are like this:

Do you think the model overfits very quickly?

Yes, the discriminator's loss goes down rather quickly and the generator's loss is decreasing too slowly. You could remove one more layer of the discriminator and try again.

I removed three layers from the discrminator (conv2, conv3 and conv4). The losses are as follows but the visual results are not satisfying.

Do you have any other suggestions? Thank you very much for your helps and time.

I am sorry for the delay in the response.
Now it looks like that you have the opposite problem. You discriminator is not being able to distinguish between real and fake pairs. You should add some layers.
I hope that you were able to solve your problem.

Thank you very much for your comment. Unfortunately adding or removing just one layer of the discriminator makes this difference. If I add one more layer then the discriminator's loss goes down rather quickly and the generator's loss is decreasing too slowly as before.  Do you have any other suggestion?
Second thing is, generated images are not smooth. I can see borders of the patches clearly (below). What should I do to solve this and have a smooth image?

Again many thanks for your help.

That is strange.
You could try changing the alpha parameter. If you increase alpha, the model will give more weight to the L1 reconstruction loss. By default alpha is set to 100. Try increasing it and see what happens.

I will try this as soon as possible. Thanks a lot!

Hi,
Increasing alpha did not improve the results (Losses were not stable). Then, I removed just one layer from the discriminator and kept the others.  Now I have four convolutional layers. I have selected alpha=50. Here is the results:

Results are slightly better. But generated images are still see pixelated.

Hi!
I am sorry I did not respond sooner. Actually, those plots do not look that bad. You just need to let the model run for more epochs and the results will eventually get better. You can see that there's a slight trend upwards in the discriminator's loss and a downward trend in the pix2pix loss.
Did you manage to make it work?

Hi!
Thanks a lot! I will try to run for more epochs. My results are exactly at the same situation where I left. I could not find a better way to workaround the problems.

Hi!
You can change the plot_loss method in util/util.py to save the losses to a file. I am sorry, I do not have time to make that change in the code at the moment :(

Hi,
Can we use the code to transform other types of images? For example MRI to CT?

How can we change the input shape (i.e  to a smaller size like 28x28)?  Should we change the U-net parameters in that case?
Is it possible to use gray scale images instead of binary images at the input side?

Thanks,

Hi, could you share with the requirement.txt file so that we can run the test demo? thanks a lot!


chore(modules): update flavor and accent defaults

Title.

Maybe we should also alias flavor to flavour.

Maybe we should also alias flavor to flavour.

I've mentioned that before in the Discord and I think that is planned but not for this PR.


Optimizing rankratioviz' performance

This isn't a problem yet, but as we start trying larger datasets here this will become an issue.
Action items:

Disable Altair's "MaxRows check" -- this is more of a temporary measure than anything, but it'll prevent Altair from giving up immediately as soon as more than 5k features or 5k samples are in a dataset.

Store datasets in their own JSON files in the data directory. Our use of column mapping in the sample plot means that the costs of storing data in memory/etc aren't that severe, but it is still probably a good idea to separate datasets from the Vega-Lite JSON specs generated by Altair. This should also remove our need to disable the MaxRows check (if there are more than 5k features, would the column mapping stuff trigger a MaxRowsError? -- I should look into this, and consider storing that stuff in its own JSON file as well although that sounds a bit excessive).

NOTE that this will probably (?) require using cross-origin requests again, which will essentially nullify the benefits of #72 (i.e. users will again have to use view.qiime2.org or python3 -m http.server to view a visualization). It should be doable to make this an option—i.e. if you want to generate a "serverless" visualization then that should be a default, and if you want to make a viz that uses CORS but potentially can tolerate more data then you can specify a --use-external-data flag or something for the python script.

This is probably a pretty far off thing, but worth it to start thinking about it now.

Reduce the number of samples shown to the user (idea c/o @mortonjt). IMO we should approach this carefully, giving as much control to the user as possible (e.g. making the user very aware of how many samples have been excluded).

Also worth using a sparse DF for the table instead of dense DF, as is currently done. (Note that this would need to either 1) convert the BIOM DF to a sparse DF, since until biocore/biom-format#808 the default biom.Table.to_dataframe() produced a dense DF, or 2) would need to require an up-to-date version of biom.) Although I'm still having trouble with generating a sparse DF in a reasonable amount of time—maybe a scipy CSR matrix would be a better option, if we can make it work in the code.

Another thing: as mentioned in 041aaaf, it'd be cool if possible to load count data in a web worker or something to minimize strain on the browser. That's the main scaling problem IMO—it's basically just the contents of a BIOM table.
One silly option: could we just... not include samples that have 0 counts for a given feature? (or vice versa, doesn't really matter, just depends on the way the counts JSON is laid out.) As long as we know that we have a list of all sample IDs, we can infer in the JS code which samples have "0" for a given feature (or vice versa). Actually, we could maybe even spin off the sparse BIOM table thing into its own JS library??? That could be super useful.

Another good idea: improving DataFrame iteration. I know iterrows() is used in process_input(), and there are probably at least a few other places where something faster (e.g. vectorized operations) could be done.

Draft algorithm for how this would work:
# to be added to the end of gen_sample_plot()
counts_dict = table.to_dict()
sparse_counts_dict = {}
for feature_id, sample_counts in counts_dict.items():
    fdict = {}
    for sample_id, ct in sample_counts.items():
        if ct != 0:
            fdict[sample_id] = ct
    if len(fdict.keys()) > 0:
        sparse_counts_dict[feature_id] = fdict

Now, sparse_counts_dict only contains data for nonzero counts. In the JS, we'll need to somehow figure out a list of sample IDs (probs can just embed it in the sample plot JSON analogously to how feature lists are embedded in the rank plot JSON) independent of the count JSON.
Also, note how we might not even add fdict into sparse_counts_dict (this would happen if all samples have 0 for a given feature, which is ostensibly possible in Qurro at present). This would mean that that feature is effectively useless in the Qurro visualization. I think we can get around this by just only passing nonzero features to the visualization -- see #171 for how we'd address this.


Add context and protocol options to SocketClient connector

Add an optionnal $context option on SocketClient/Connector
It accepts a valid stream_context_create ressource
This way it's easier to add binding or local certificate on tls
Add also optionnal protocol on create, default is still tcp, but you can use others like udp or tls
Doc updated

This PR seems like a good starting point, thanks @Vinceveve!
This PR has been WIP for quite some time now and things have changed considerably since then :-)
React now consists of individual components that are maintained individually. Supporting context options is now related to the react/socket-client component.
@elazar has already filed ticket https://github.com/reactphp/socket-client/issues/4 to keep track of this. This ticket also links to a few outstanding PRs, so I suppose it makes sense to focus our context options related efforts on this component.


Update egui to 0.19

Checklist

[x] I have read the Contributor Guide
[x] I have read and agree to the Code of Conduct
[x] I have added a description of my changes and why I'd like them included in the section below

Description of Changes
I've updated egui to 0.19 to make puffin_egui compatible with that version.
Related Issues
N/A

Thanks! Can be merged once the linting errors are resolved.

Ah, I think the problem is that I only updated puffin_egui and not the other packages. I'll try to update them as soon as possible.

Sorry for making this way harder than it should be.

No worries @sidit77  - have you tried check.sh?

Oh, that's actually pretty useful. I'm running it right now to make sure that the last update will really really really be the last.

Thank you very much!

A new egui-macroquad 0.12 with egui 0.19 support has beend has been released: https://github.com/optozorax/egui-macroquad/commit/6973d471de936935a22162e403f748cfcdd0bdff

I already reenabled the demo: https://github.com/EmbarkStudios/puffin/pull/93/commits/ac8745dd3d6984118c127e4a464064ed076f502c


Unable to connect to MongoDB using TLS

I have a setup in which a golang  app is producing records to a Kafka cluster and kafka-connect-mongodb is acting as a the sink to a local MongoDB cluster.
The issue I am facing is I am unable to connect to MongodB once TLS is enabled.
MongoDB startup:
mongod -v --port 27017 --dbpath /var/lib/mongodb/ --replSet rs0 --auth  --sslMode requireSSL --sslPEMKeyFile /path/to/server.pem --sslCAFile /path/to/ca.pem
Connector :
curl -X POST \
  http://localhost:8083/connectors \
  -H 'Accept: */*' \
  -H 'Cache-Control: no-cache' \
  -H 'Connection: keep-alive' \
  -H 'Content-Type: application/json' \
  -H 'Host: localhost:8083' \
  -H 'Postman-Token: efde734d-a14b-4604-9c9a-5d9694a83b34,6552f42b-f277-4c14-8377-5abaf0b68385' \
  -H 'User-Agent: PostmanRuntime/7.15.0' \
  -H 'accept-encoding: gzip, deflate' \
  -H 'cache-control: no-cache' \
  -H 'content-length: 773' \
  -d '{
    "name": "mongo-sink",
    "config": {
        "connector.class": "com.mongodb.kafka.connect.MongoSinkConnector",
        "tasks.max": "1",
        "topics": "myTopic",
        "connection.uri": "mongodb://user:pass@<IP_ADDRESS>:27017/myDB?ssl=true&replicaSet=rs0",
        "database": "myDB",
        "collection": "collection1",
        "value.converter": "org.apache.kafka.connect.json.JsonConverter",
        "value.converter.schemas.enable": "false",
    }
}

When kafka-connect-mongodb tries to write to MongoDB, I get the following error:
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
	at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
	at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
	at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)

So, I imported the certificates to jre keystore (/usr/lib/jvm/java-8-oracle/jre/lib/security/cacerts) and I get the following error:
INFO Exception in monitor thread while connecting to server <IP_ADDRESS>:27017 (org.mongodb.driver.cluster:76)
com.mongodb.MongoSocketReadException: Prematurely reached end of stream
	at com.mongodb.internal.connection.SocketStream.read(SocketStream.java:112)

Next, I tried setting the keystore and truststore jks location path in connect-distributed.properties file and got the same error.
ssl.truststore.location=/path/to/truststore.jks
ssl.truststore.password=pass
ssl.keystore.type=JKS
ssl.keystore.location=/path/to/client.jks
ssl.keystore.password=pass

Finally as a sort of hack, I added the following options in KAFKA_JMX_OPTS in kafka-run-class.sh and I was able to successfully connect to Mongo DB
-Djavax.net.ssl.trustStore=/path/to/truststore.jks
-Djavax.net.ssl.trustStorePassword=pass
-Djavax.net.ssl.keyStore=/path/to/client.jks
-Djavax.net.ssl.keyStorePassword=pass

Can you tell me where I am going wrong? Where do I set the JVM ssl properties?  Is setting it in KAFKA_JMX_OPTS the right approach?

I figured out that we can export the KAFKA OPTS env variable. closing the issue.

@hazarika thx for your clarification on this. as always I'm happy to learn more about your use case - highly appreciated if you would be willing to share this :)


add ability to publish range of ports

Closes #8899
Signed-off-by: Srini Brahmaroutu<EMAIL_ADDRESS>

np, I am happy to rebase. Please see that this gets merged.

LGTM

LGTM

:fireworks:

\o/


feat(ci-cd): semantic-release

GH-14
Description
Adding semantic release for automatic release process
Fixes #14
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Intermediate change (work in progress)

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Test A
[ ] Test B

Checklist:

[ ] Performed a self-review of my own code
[ ] npm test passes on your machine
[ ] New tests added or existing tests modified to cover all changes
[ ] Code conforms with the style guide
[ ] API Documentation in code was updated
[ ] Any dependent changes have been merged and published in downstream modules

:tada: This PR is included in version 3.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


error in npm package name

it is not npm install gitbook-mermaid-plugin but npm install gitbook-plugin-mermaid

Thanks, I have fixed it in README file

Damn, I did miss package.json. Fixed.


Increment versions for tables releases

Increment package versions for tables releases

API change check
API changes are not detected in this pull request.


Code for creating standard CDK files across instruments

Summary
We determined that we will use XArray to generate CDF files, so we need to create a standard piece of code which contains shared requirements from ISTP that instruments can use.
Requirements
The more specific the better.

[ ] Create a standardized way to create metadata which aligns with ISTP standards for CDF files
[ ] Create any common attributes, variables, or utility methods which would aid in CDF file writing

Related tickets
Solves spike #71

CDF?

CDF?

Thanks, let me update this 😆


CogitoUIButton crashes project when using disabling button

Cogito and Godot Engine Version:
Godot 4.2.stable
Cogito beta 202403.25
Description:
When making a cogitoUIbutton disabled, Invalid get index '3' (on base: 'Dictionary'). error
Reproduction steps:
Make button disabled, run project
error appears at "var target = styleboxes[current_state] as StyleBoxFlat"
fixed with adding lines:
styleboxes[BaseButton.DRAW_DISABLED] = get_theme_stylebox('disabled').duplicate()
add_theme_stylebox_override('disabled', tween_stylebox)
Expected behavior:
Project not to crash

Hey, thanks for flagging this!
Could you give https://github.com/Phazorknight/Cogito/commit/195b5cc12166d34d57daead4ee6d5e49d5102cc5 a try and see if that works better?
I've implemented the lines you added.

Yes it works on my end. Also, I am using your project for a game I am making (Producer Tycoon), and I have some people that play it on steamdeck. So far they haven't had any major issues with it.

@DaQueenGames awesome, that's great to hear. Closing this issue.
If you feel up for showing off your game, feel free to post in the Discussions tab! Would love to see it.


fix: browse badges paddding

Summary of Changes
Adds padding for badges when the list spans more than one line.
Tests
n/a
Documentation
n/a
CheckList
Make sure you have checked all steps below to ensure a timely review.

[x] PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"

In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

[x] PR includes a summary of changes.
[x] PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
[x] In case of new functionality, my PR adds documentation that describes how to use it.

All the public functions and the classes in the PR contain docstrings that explain what it does

sure, I've added the change to use value from variables @Golodhros


add support for version

Hi,
With this you can check the version with command:
bazel run //src:djinni -- --version

or:
bazel run //generator:generator -- --version

If you are using the generated binary direct:
./djinni --version

It is a resource from scopt library.
It let you check what version you are using.

I can think in a way to send the target version to be replaced there. What you think?

I do not like this solution because of how it modifies source files in a github action.  May I ask why it is so important to embed the version string in the generator tool?  IMO it's enough to verify the hash of the downloaded package to ensure the right version is used.

I made a solution for my project. I save a file called .version when i download the binary from djinni, and use the data inside this file, instead of execute --version from binary.


[Bugfix] Free cross attention block table for preempted-for-recompute sequence group.

This PR fixes the error "AssertionError: block table already exists" for encoder-decoder models, when the scheduler attempts to reschedule a sequence that was previously preempted for recomputation.
The error occurs at the following location: https://github.com/vllm-project/vllm/blob/8f0a9ca890a125f2b0fef49ba042ecf5b37830a8/vllm/core/block_manager.py#L190
Error logs look like:
Engine background task failed
Traceback (most recent call last):
   File "/workspace/vllm/vllm/engine/async_llm_engine.py", line 57, in _log_task_completion
     return_value = task.result()
   File "/workspace/vllm/vllm/engine/async_llm_engine.py", line 868, in run_engine_loop
     result = task.result()
   File "/workspace/vllm/vllm/engine/async_llm_engine.py", line 791, in engine_step
     request_outputs = await self.engine.step_async(virtual_engine)
   File "/workspace/vllm/vllm/engine/async_llm_engine.py", line 311, in step_async
     ) = self.scheduler[virtual_engine].schedule()
   File "/workspace/vllm/vllm/core/scheduler.py", line 1226, in schedule
     scheduler_outputs: SchedulerOutputs = self._schedule()
   File "/workspace/vllm/vllm/core/scheduler.py", line 1185, in _schedule
     return self._schedule_default()
   File "/workspace/vllm/vllm/core/scheduler.py", line 1020, in _schedule_default
     prefills = self._schedule_prefills(budget,
   File "/workspace/vllm/vllm/core/scheduler.py", line 956, in _schedule_prefills
     self._allocate_and_set_running(seq_group)
   File "/workspace/vllm/vllm/core/scheduler.py", line 1420, in _allocate_and_set_running
     self.block_manager.allocate(seq_group)
   File "/workspace/vllm/vllm/core/block_manager.py", line 190, in allocate
     assert (request_id
 AssertionError: block table already exists

The root cause appears to be that the current implementation doesn't free cross-attention blocks during preemption.
BEFORE SUBMITTING, PLEASE READ THE CHECKLIST BELOW AND FILL IN THE DESCRIPTION ABOVE

 PR Checklist (Click to Expand)
Thank you for your contribution to vLLM! Before submitting the pull request, please ensure the PR meets the following criteria. This helps vLLM maintain the code quality and improve the efficiency of the review process.
PR Title and Classification
Only specific types of PRs will be reviewed. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

[Bugfix] for bug fixes.
[CI/Build] for build or continuous integration improvements.
[Doc] for documentation fixes and improvements.
[Model] for adding a new model or improving an existing model. Model name should appear in the title.
[Frontend] For changes on the vLLM frontend (e.g., OpenAI API server, LLM class, etc.)
[Kernel] for changes affecting CUDA kernels or other compute kernels.
[Core] for changes in the core vLLM logic (e.g., LLMEngine, AsyncLLMEngine, Scheduler, etc.)
[Hardware][Vendor] for hardware-specific changes. Vendor name should appear in the prefix (e.g., [Hardware][AMD]).
[Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.
Code Quality
The PR need to meet the following code quality standards:

We adhere to Google Python style guide and Google C++ style guide.
Pass all linter checks. Please use format.sh to format your code.
The code need to be well-documented to ensure future contributors can easily understand the code.
Include sufficient tests to ensure the project to stay correct and robust. This includes both unit tests and integration tests.
Please add documentation to docs/source/ if the PR modifies the user-facing behaviors of vLLM. It helps vLLM user understand and utilize the new features or changes.

Adding or changing kernels
Each custom kernel needs a schema and one or more implementations to be registered with PyTorch.

Make sure custom ops are registered following PyTorch guidelines: Custom C++ and CUDA Operators and The Custom Operators Manual
Custom operations that return Tensors require meta-functions. Meta-functions should be implemented and registered in python so that dynamic dims can be handled automatically. See above documents for a description of meta-functions.
Use torch.libary.opcheck() to test the function registration and meta-function for any registered ops.  See tests/kernels for examples.
When changing the C++ signature of an existing op, the schema must be updated to reflect the changes.
If a new custom type is needed, see the following document: Custom Class Support in PT2.

Notes for Large Changes
Please keep the changes as concise as possible. For major architectural changes (>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue (RFC) discussing the technical design and justification. Otherwise, we will tag it with rfc-required and might not go through the PR.
What to Expect for the Reviews
The goal of the vLLM team is to be a transparent reviewing machine. We would like to make the review process transparent and efficient and make sure no contributor feel confused or frustrated. However, the vLLM team is small, so we need to prioritize some PRs over others. Here is what you can expect from the review process:

 After the PR is submitted, the PR will be assigned to a reviewer. Every reviewer will pick up the PRs based on their expertise and availability.
 After the PR is assigned, the reviewer will provide status update every 2-3 days. If the PR is not reviewed within 7 days, please feel free to ping the reviewer or the vLLM team.
 After the review, the reviewer will put an  action-required label on the PR if there are changes required. The contributor should address the comments and ping the reviewer to re-review the PR.
 Please respond to all comments within a reasonable time frame. If a comment isn't clear or you disagree with a suggestion, feel free to ask for clarification or discuss the suggestion.

Thank You
 Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM. Your contributions make vLLM a great tool for everyone!

cc @afeldman-nm ?

Do we plan to proceed with this fix? I have been seeing the same error?

@kathyyu-google Can you rebase and we'll re-run the tests? IMO they are unrelated.

Thanks @andoorve! I have updated the PR.


[BUG] Enhancing Navbar

Description
Navbar for mobile & tablet devices is not appealing and I would like to add animation like this is my project
I would like to work on this issue.
Screenshots

Any additional information?
No response
What browser are you seeing the problem on?
No response
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I have read the discussion tab thoroughly and got the project idea
[X] I am willing to work on this issue (optional)
[X] I am a GSSoC'23 contributor

@vaibhav-xt go for it make it more appealing


A lot of fixing

I don't think you have totally understand the design of negtive cache and the implementation code of iterate upstream lookup and maybe you lack enough comments to illustrate your modifications.
For the updates to modification file, please keep the changes info your personal fork repo.

OK!
Thank you for the great project.


Fix ordering of bullets in install guide.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

articles/quantum-InstallConfig.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

I just found that one too, thanks for being a few steps ahead of me, Chris :)

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

articles/quantum-InstallConfig.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


AboutModal with scrolling content doesn't have the scroll fade-out

See https://carbon-for-ibm-products.netlify.app/?path=/story/carbon-for-ibm-products-patterns-about-modal-aboutmodal--fully-loaded

i spent some time looking into this and i'm not 100% what to make of it. this is the css in question that's being used-
background-image: linear-gradient(to bottom, rgba(var(--cds-ui-01, #f4f4f4), 0), var(--cds-ui-01, #f4f4f4));

compared to the css that's being used on a standard carbon modal that I took from sandbox
background-image: linear-gradient(to bottom, rgba(244,244,244,0), #f4f4f4);

based on this stackoverflow it would appear that functions can't be passed to rgba. so i'm guessing that this is build problem on our end where the variables aren't just being evaluated to their normal values and it's causing issues with rgba.
thoughts?