Make cross-colo listening optional within Helix ClusterManager

Introduce listenCrossColo config in ClusterMapConfig that defaults
to true. It can be set to false when making sensitive changes to
the cluster information in ZK so that a bad change does not get
propagated across colos. This helps in safely and selectively
introducing new changes.

Codecov Report

Merging #1084 into master will decrease coverage by 0.11%.
The diff coverage is 100%.

@@             Coverage Diff              @@
##             master    #1084      +/-   ##
============================================
- Coverage     68.65%   68.53%   -0.12%
+ Complexity     4833     4826       -7
============================================
  Files           382      382
  Lines         30512    30495      -17
  Branches       3890     3890
============================================
- Hits          20948    20900      -48
- Misses         8520     8540      +20
- Partials       1044     1055      +11

Impacted Files
Coverage Δ
Complexity Δ

...m.github.ambry.clustermap/HelixClusterManager.java
92.87% <100%> (-0.65%)
34 <0> (+1)

...java/com.github.ambry/config/ClusterMapConfig.java
100% <100%> (ø)
1 <0> (ø)
:arrow_down:

...rc/main/java/com.github.ambry.store/BlobStore.java
82.35% <0%> (-3.07%)
74% <0%> (-4%)

...github.ambry.rest/AsyncRequestResponseHandler.java
88.59% <0%> (-2.29%)
23% <0%> (ø)

...in/java/com.github.ambry.store/BlobStoreStats.java
70.34% <0%> (-1.09%)
100% <0%> (-1%)

...ain/java/com.github.ambry.router/PutOperation.java
89.8% <0%> (-0.97%)
81% <0%> (-2%)

...src/main/java/com.github.ambry.commons/BlobId.java
86.97% <0%> (-0.43%)
59% <0%> (-1%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7299f84...2e542e5. Read the comment docs.

@lightningrob @jsjtzyy addressed comments.


Share user secrets in Dev-Teams

The current mechanism for user screts in development environment (https://docs.microsoft.com/en-us/aspnet/core/security/app-secrets?view=aspnetcore-3.0&tabs=windows) stores the secrets in a json file which is stored on the local machine for the current user.
But for development teams, it would be nice to share the secrets, so that not every team member must configure the same passwords.
The current tool dotnet user-secrets and the extension for user secrets  stores the secrets in a folder depending on the following environment variables (https://github.com/aspnet/Extensions/blob/master/src/Configuration/Config.UserSecrets/src/PathHelper.cs):

APPDATA (Windows)
HOME (Linux)
Environment.SpecialFolder.ApplicationData
Environment.SpecialFolder.UserProfile
DOTNET_USER_SECRETS_FALLBACK_DIR

Simple Solution
A simple solution to share secrets in a Team would be to let configure a network folder as storage location for the secret files.
e.g. The PathHelper could make an additional lookup as first step with the environment variable
DOTNET_USER_SECRETS_DIR
So each team member can set this variable one time to a shared folder to share the secrets (https://github.com/aspnet/Extensions/pull/2726).
Flexible Solution
Another possiblity would be, to store a 'directory-json' file in the root folder of the secrets which contains the location for project secrets files on project Level.
e.g. the user excute:
dotnet user-secrets setlocation "\\SERVER\Projects\MyProject\UserSecrets" --project "C:\apps\WebApp1\src\WebApp1"

This would create a directory json in the default root directory of user secrets storage location:
[
   {   "userSecretsId": "79a3edd0-2092-40a2-a04d-dcb46d5ca9ed",   "location:": "\\\\SERVER\\Projects\\MyProject\\UserSecrets" }
]

All future request to the secret json files will be redirected to the project specifc location.

cc @Tratcher @davidfowl

But for development teams, it would be nice to share the secrets, so that not every team member must configure the same passwords.

My initial concern here is that user secrets was very intentionally designed around not sharing secrets. It's the whole reason it exists. Generally we've suggested using other systems to share secrets, even for development scenarios (things like Azure Key Vault, or custom config providers).
Just expanding this to allow a custom directory isn't a huge deal, but it does expand our security model for this.
Using an environment variable is very risky, since updating this env var could end up messing with other projects. A "redirector" file in the default location could work but feels overly complicated.
Just adding the ability for the dotnet user-secrets command to target another file doesn't seem too scary. It could be a command-line argument and you could easily have some scripts that set up the correct arguments and proxy the rest of the arguments through (.\my-project-secrets set 'Foo' 'Bar' which just calls dotnet user-secrets --location \\SERVER\... set 'Foo' 'Bar')
When adding something like this to the product a big question we need to consider is "Does this add value to a large group of users?". I'm not so sure that value is clear right now. The user-secrets tool has a relatively significant maintenance burden today (flaky infrastructure, VS tooling integration code, etc.). This is the first report I've heard of asking for this (though I may be missing something). It seems to me like this is a better place for a third-party tool and provider (based on the existing code) to explore, and we could investigate a first-class option when there is more data to show the value to existing customers.

Another option: Have you tried creating a symbolic link to place the shared file in the normal location? https://winaero.com/blog/create-symbolic-link-windows-10-powershell/
If there are bugs in the handling of symlinks, we could look at fixing those.

Yes, the symbolic link could work for the team problem. But a build in feature would support more scenarios. E.g. The user secrets can currently only used in Dev-Environments, with a build in feature where the location can be specified, it could also be used in PROD-Environments to separate the secrets from the appsettings. For this case, the simple solution with the environment variable would be enough, the variable could be specified in the web.config, so it must not be machine wide.

Using an environment variable is very risky, since updating this env var could end up messing with other projects. A "redirector" file in the default location could work but feels overly complicated

This is true, but the indention was to use this on dev-machines with a single owner. There exist currently already the same "risky" solution with the .net core environment where a variable can be used to the define the environment name which will be used for the selection of the corresponding application settings files. This also could break other applications on the same machine, so the developer have to know what he do. My solution with the environment variable for the store location is very similar, so it used the same concept, is fully downward compatible, easy to implement and offers a lot of new scenarios:

Dev-Team secret sharing
Usage on PROD machines to separate appsettings and secrets in the same way
Easy usable in docker environments to place the secrets on a mount point

This is the first report I've heard of asking for this
One is always the first. But the discussion is open now. Maybe other people find this usefully but have not yet thought about this.

One is always the first. But the discussion is open now. Maybe other people find this usefully but have not yet thought about this.

Yep, we just don't have enough data to consider this worth the high cost of integrating it into the main product at this.

Closing as we don't believe this is something the core user secrets component should provide. We can always review this as we hear more from users.


luci-app-statistics: fix collectd config generation

Fix config generation for the following sections:

curl
exec
network
iptables

This was added with the following commit.
https://github.com/openwrt/luci/commit/c1380ab36797dc4a9cba5a6aa738019452178d79
Signed-off-by: Florian Eckert<EMAIL_ADDRESS>

Please add Fixes: c1380ab ("make luci-app-statistics more extensible") before the signed-off-by. Otherwise LGTM

@feckert , sorry for breaking this, and thanks for the fix.
However wouldn't it be simpler to just change the code from plugins[name] = plugin.legend to
if plugin.legend then
  plugins[name] = plugin.legend
end

Then you don't need a long cascade of if statements…

@mberndt123 on problem I quickly found a solution to solve this issue.

However wouldn't it be simpler to just change the code from plugins[name] = plugin.legend to

feel free to open another PR


Preserve keyword case

Some users prefer that the original case of SQL keywords be preserved, for example:
System.out.println(CCJSqlParserUtil.parseStatements("select * from sysdatabases").toString());
Desired behavior:
Optionally (or by some other means) preserve keyword case, for example print:
select * from sysdatabases
System:
JSqlParser version 3.1 (latest)
Java 8

That would some work to do. JavaCC only accepts keyword tokens. Therefore the used case for an input statement is not preserved.

Thanks, I have filed https://github.com/javacc/javacc/issues/120 (in the interim, time permitting I might experiment with case-sensitive matches since I don't expect my users to type keywords like FroM).

I don't think, the request you filed at javacc will have any influence on this. The grammar of JSqlParser itself uses this tokens and build an object hierarchy from this. The case of object names is preserved, but when you use a toString or a deparser, the keywords do not come from the accepted tokens, but from string constants like "SELECT".
One solution here would be to write or adapt the deparsers to be able to use upper case or lower case keywords.
You could file a PR.

So may we close this one?
Additionally, you have the original SQL. There the case is not attached.
As I said, the deparsers could be changed in a way, to make some case changes on used keywords.

Sorry, yes I will close this.  I did put forth some effort but am not ready to change the deparsers in the near future.

So maybe you could share your efforts so far ...

9f8ca07 is how far I got before being tasked with a different project to work on.


C functions with UInt8[x]  arguments aren't compatible with Slice(UInt8).

Generated function from libgen:
fun crypto_secretstream_xchacha20poly1305_init_push(state : CryptoSecretstreamXchacha20poly1305State*,
  header : UInt8[24],
  k : UInt8[32]
) : LibC::Int

Error:
 > LibSodium.crypto_secretstream_xchacha20poly1305_init_push(self, head, k.to_slice)
                                                                         ^-------
Error: argument 'k' of 'Sodium::LibSodium#crypto_secretstream_xchacha20poly1305_init_push'
must be StaticArray(UInt8, 32), not Slice(UInt8) (nor Pointer(UInt8)
returned by 'Slice(UInt8)#to_unsafe')

I could copy to a StaticArray but there are several reasons why it's inappropriate.

k is a key held in nonswappable auto clearing memory.  Copying to a StaticArray may leave key material in swap.  It also needs manual clearing and is easy to forget or mess up.
Memory (in my shards) is allocated based on c function return values to determine the size.  If  StaticArray is used  I need more code to support macro generated classes because the size functions aren't available in macros or header values aren't available at all.  There is greater risk of these values getting out of sync with new versions because I need to update and check them manually.
It's more work (both programmer and program) and seems unnecessary.

Crystal 0.34.

Well, you could always pretend one thing is the other to_unsafe.as(StaticArray(UInt8, 24)*).value. Is this safe? I'm not sure, aren't static arrays passed by value? Is that even possible for a heap allocated object like Slice? That's also why I'm not sure Crystal could even do what you propose. It doesn't feel safe, at least not generally.

Is static array a thing in C? As far as I know you should use pointer of UInt8 instead of static array in the binding, because C arrays don't know their length.

Header definition
void crypto_stream_xchacha20_keygen(unsigned char k[crypto_stream_xchacha20_KEYBYTES])

@didactic-drunk Take this c file:
// c.c
#define crypto_stream_xchacha20_KEYBYTES 32U

void crypto_stream_xchacha20_keygen(unsigned char k[crypto_stream_xchacha20_KEYBYTES])
            __attribute__ ((nonnull)) {
}

Compile it with clang and output llvm ir:
🍏 clang -S -emit-llvm c.c
c.c:15:29: warning: GCC does not allow 'nonnull' attribute in this position on a
      function definition [-Wgcc-compat]
            __attribute__ ((nonnull)) {
                            ^
1 warning generated.

Look at the resulting LLVM IR:
; ModuleID = 'c.c'
source_filename = "c.c"
target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx10.15.0"

; Function Attrs: noinline nounwind optnone ssp uwtable
define void @crypto_stream_xchacha20_keygen(i8* nonnull %0) #0 {
  %2 = alloca i8*, align 8
  store i8* %0, i8** %2, align 8
  ret void
}

attributes #0 = { noinline nounwind optnone ssp uwtable "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "frame-pointer"="all" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="penryn" "target-features"="+cx16,+cx8,+fxsr,+mmx,+sahf,+sse,+sse2,+sse3,+sse4.1,+ssse3,+x87" "unsafe-fp-math"="false" "use-soft-float"="false" }

!llvm.module.flags = !{!0, !1}
!llvm.ident = !{!2}

!0 = !{i32 1, !"wchar_size", i32 4}
!1 = !{i32 7, !"PIC Level", i32 2}
!2 = !{!"clang version 10.0.0 "}

See define void @crypto_stream_xchacha20_keygen(i8* nonnull %0)? That a pointer to UInt8.
Static arrays don't exist at runtime in C, only at compile-time. The C compiler can probably yell if you exceed the bounds at compile time, but at runtime "static arrays" are just pointers. At least in functions. In structs they are still probably arrays, but their size is used for the struct layout.
libgen is wrong and it should generate pointers in that case.

Alternatively, we could allow specifying static arrays in C bindings, and codegen them as pointers, but allowing only to pass static arrays. But the workaround of passing a pointer is good. Do you know how this works in, say, Rust? If they handle static arrays like what I say here, we might consider doing it. Super low priority, though, not gonna happen before 1.0

but allowing only to pass static arrays.

I was hoping for length verified Slices.  With this library too large and it's ignored.  Too small may overwrite memory.

we could allow specifying static arrays in C bindings, and codegen them as pointers, but allowing only to pass static arrays

That's definitievely not the correct behavior and not what's required of a c program.  unsigned char k[crypto_stream_xchacha20_KEYBYTES] helps the compiler verify the length when it can but also accepts unsigned char *.
Also see points 1-3.

I was hoping for length verified Slices.

I don't think that's feasible. It would require injecting custom validation code for calling external functions.
That's a lot to ask for a realistically niche use case. As @asterite already mentioned, even in C there's no such runtime validation of the call arg.
So it's a compile-time only validation. At runtime, you can just pass any pointer.
Such a compile-time validation can't be done in Crystal. The only container type with a fixed size at compile time is StaticArray and that is not really useful for this, as argued above.
IMO there is nothing actionable here. In order to validate the size of the arguments, it needs to happen at runtime to support collections with dynamic size (like Slice). And that should just be implemented in the code that calls the external functions.


fix: remove restrictions on read

Remove artificial restrictions on read.
Are these necessary? I could see having one on objectType perhaps, but the other one not so much.
Description

References

Review Checklist

[ ] I have added documentation for new/changed functionality in this PR or in openfga.dev
[ ] The correct base branch is being used, if not main

Codecov Report

Merging #149 (5ce9f62) into main (78045ed) will decrease coverage by 0.40%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #149      +/-   ##
==========================================
- Coverage   75.44%   75.03%   -0.41%
==========================================
  Files          76       76
  Lines        8372     8244     -128
==========================================
- Hits         6316     6186     -130
- Misses       1673     1674       +1
- Partials      383      384       +1

Impacted Files
Coverage Δ

server/test/read.go
95.40% <ø> (-0.98%)
:arrow_down:

server/commands/read.go
82.45% <100.00%> (-1.42%)
:arrow_down:

server/commands/check.go
86.27% <0.00%> (-0.57%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Let's do this when we update Read.


Hit illegal abi signature in PublicABIGenerationPass when there are non 32 bit types

I hit the issue when compiling ASR models. I think we don't have well support for returning some types like ui64, i16, i64.
To repro:
func @foo() -> (tensor<i64>, tensor<i32>, tensor<512xi16>, tensor<4x128xf32>, tensor<3x512xi16>, tensor<3xui64>) attributes {iree.module.export} {
  %0 = mhlo.constant dense<0> : tensor<i64>
  %1 = mhlo.constant dense<0> : tensor<i32>
  %2 = mhlo.constant dense<2> : tensor<3xi32>
  %3 = mhlo.constant dense<0> : tensor<512xi16>
  %4 = mhlo.constant dense<0.000000e+00> : tensor<4x128xf32>
  %5 = mhlo.constant dense<0> : tensor<3x512xi16>
  %6 = mhlo.constant dense<0> : tensor<3xui64>
  return %0, %1, %3, %4, %5, %6 : tensor<i64>, tensor<i32>, tensor<512xi16>, tensor<4x128xf32>, tensor<3x512xi16>, tensor<3xui64>
}

Run iree-translate --iree-mlir-to-vm-bytecode-module --iree-hal-target-backends=dylib-llvm-aot ~/abi.mlir
You will see the errors:
/usr/local/google/home/hanchung/abi.mlir:1:1: error: illegal abi signature ('I1!R43!B3!t7B3!t6B7!t5d512B7!d4d128B9!t5d3d512U1!'): Unrecognized raw tag
func @foo() -> (tensor<i64>, tensor<i32>, tensor<512xi16>, tensor<4x128xf32>, tensor<3x512xi16>, tensor<3xui64>) attributes {iree.module.export} {

It looks like ui64 is the root cause. I replace it with i32, then everything looks good.

Seems like a dupe of #6131. I think handling unsigned types is a meta-issue and should be fixed before we try to play whack-a-mole.


Update to new Scalar API

This removes all uses of Scalar::[from_bits, from_bits_clamped}. Some comments inline.
I'll update the git dependency once https://github.com/dalek-cryptography/curve25519-dalek/pull/519 drops.

@tarcieri Ok one last look. I changed all the constructors to not clamp the inputs (this is what age does). Clamping is always done when doing scalar-point multiplication.


RuntimeError: CUDA error: invalid device function

Compiled successfully。
Error when running training and testing：
RuntimeError: CUDA error: invalid device function (launch_kernel at /opt/conda/conda-bld/pytorch_1579022060824/work/aten/src/ATen/native/cuda/Loops.cuh:103)
ubuntu16.04   cuda10.0.130    pytorch1.4.0   python3.7.4

Please use the Error template. This might be because the Pytorch version is incompatible with your CUDA version.

Compiled successfully。
Error when running training and testing：
RuntimeError: CUDA error: invalid device function (launch_kernel at /opt/conda/conda-bld/pytorch_1579022060824/work/aten/src/ATen/native/cuda/Loops.cuh:103)
ubuntu16.04 cuda10.0.130 pytorch1.4.0 python3.7.4

Hello, I have the same problem, and how did you solve it?

This issue is closed because there is no valid information to debug the issue. Please raise a new issue using the error template  if you meet similar problem and cannot solve it.

Compiled successfully。
Error when running training and testing：
RuntimeError: CUDA error: invalid device function (launch_kernel at /opt/conda/conda-bld/pytorch_1579022060824/work/aten/src/ATen/native/cuda/Loops.cuh:103)
ubuntu16.04 cuda10.0.130 pytorch1.4.0 python3.7.4

Hello, I have the same problem, and how did you solve it?

I have the problem too, have you solved that?


OCPBUGS-30113: 4.13: Add feature annotations to CSV

Add all required feature annotations to the operator.
This is a manual cherry-pick of https://github.com/openshift/aws-efs-csi-driver-operator/pull/127 to 4.13.
In 4.13, we do not support "tokenized authentication with AWS APIs via AWS Secure Token Service (STS) by using the Cloud Credential Operator (CCO)", that was added as 4.14 feature https://issues.redhat.com/browse/STOR-1347
@openshift/storage

/jira cherrypick OCPBUGS-30110

/label backport-risk-assessed

/label qe-approved
/label cherry-pick-approved

/lgtm

/jira refresh


Not fully display tags

the text on the label are cutting off, they not fully showing.
Is there a fix for this that i could use?

Can you attach a screenshot?

I have the same problem

I have this issue on iPhone 5 screen size, but everything is ok (probably) on iPhone 6 sized screen. After some scrolling issue disappears.

This is probably because you're using a bigger font size than 12. A quick hack is to change the multiplier in the intrinsicContentSize func from size.width += paddingX * 2 to size.width += paddingX * 4

I had the same problem, that tags did not fully display the text. In my case, I manually set the font size for my TagListView and it worked. BUT, set the fontSize only after you added all the tags otherwise it will not work.

It worries me too, please could you make some auto font scale to fit size?


Anerdw: Watch ps290q.com

Anerdw requests the watch of the watch_keyword ps290q\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

ps290q\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by VLAZ in Charcoal HQ


Adding Web Framework support

Still working on general code organization, but please comment on it as well.

adds automatic support for a standard approach to:

Web API
MVC

When the new world order of collapsed MVC / Web API hits, this two packages would collapse into one.

Rather than specific startup interfaces, what if you used a generic:
public interface IStartupStep<TContext>
{
    void Configure(TContext ctx);
}

You could then have a generic bootstrapper:
public class StartupBootstrapper
{
    // ...

    public static void Bootstrap<TContext>(IApplication app, TContext ctx)
    {
        foreach (var v in app.Resolve<IEnumerable<IStartupStep<TContext>>>())
        {
            v.Boot(ctx);
        }
    }
}

Usage:
public static class WebApiConfig
{
    public static void Register(HttpConfiguration config)
    {
        StartupBootstrapper.Bootstrap(/* app from where? */, config);
    }
}

Rather than specific startup interfaces,

Nice, I like it mikey. The one thing that I don't have solved yet is that WebAPI is very picky about when you run its start up steps. MVC seems to not care and the generic one doesn't either. So we just need to solve the Web Api one.

Rather than specific startup interfaces,

Nice, I like it mikey. The one thing that I don't have solved yet is that WebAPI is very picky about when you run its start up steps. MVC seems to not care and the generic one doesn't either. So we just need to solve the Web Api one.

In the System.Web.HttpApplication world, you're looking at either a standard Chassis call in Global.asax.cs or maybe hooking into an equivalent spot in the lifecycle with an HttpModule via IStartupStep<HttpApplication>:
public class WebApplication : System.Web.HttpApplication
{
    protected void Application_Start()
    {
        var app = GetChassisApp();
        StartupBootstrapper.Bootstrap(app, this);
    }
}

And then you would have a ConfigureWebApi : IStartupStep<HttpApplication> that makes the GlobalConfiguration.Configure(...) call which then bootstraps stuff that depends on HttpConfiguration. (Strictly speaking, this last should be in a separate Chassis.Fairing.WebApi.WebHost package with the additional dependency on Microsoft.AspNet.WebApi.WebHost).

In an OWIN world, you simply do something like this in your Startup:
var app = GetChassisApp();
var config = new HttpConfiguration();
StartupBootstrapper.Bootstrap(app, config);
app.UseWebApi(config);

Rather than specific startup interfaces, what if you used a generic:

I'm going to do this in a separate PR so it can be very focused there.

Rather than specific startup interfaces, what if you used a generic:

I'm going to do this in a separate PR so it can be very focused there.

I don't really see anything else here that can't be fiddled with later. :shipit;?

You could then have a generic bootstrapper:

Well if its typed you can't really have a generic 'bootstrapper' because it has to know the T.

We would have to have a bootstrapper per context object.


[Question] What is olympusnews/index.txt used for?

I was looking at the workflows for this website and was curious about why the news is indexed when the actual website itself doesn't use the articles at all - is this being used in another repository?

Wrong repository :sweat_smile:


adds support for asynchronous reading of data from ServletInputStream

jet changes PR inspired from this jetty proxy example.
Changes proposed in this PR

adds support for asynchronous reading of data from ServletInputStream

Why are we making these changes?
We prefer async reading of data from the input streams to avoid blocking async channels.

Will reopen when I work on this again.


Remove redundant readMoreEntries call in PersistentDispatcherMultiple…

Motivation
Remove redundant readMoreEntries call in PersistentDispatcherMultipleConsumers and PersistentStickyKeyDispatcherMultipleConsumers
Currently, readMoreEntries() will be called in readEntriesComplete(), may be some redundancy here. It's better to call readMoreEntries() in the sendMessagesToConsumers().
For shared subscription, no need to wait send messages to consumers complete, so call readMoreEntries() in the end of sendMessagesToConsumers().
For key_shared subscription, need to wait send messages to consumers complete, so call readMoreEntries() while all consumers writeAndFlush() completely.

run java8 tests
run Integration Tests


Fix UT (broken snapshot)

Current UT are broken by the verify snapshot (still not sure exactly why). My first guess was to force a .posix.normalize on file path but it didn't resolve the issue.

I'll take a look on it.

Opened PR


ds_train = tfds.load("celeb_a", split="train") goes wrong

version info:
Tensorflow2.1
tensorflow_addons==0.9.1
when I run:
ds_train = tfds.load("celeb_a", split="train")
it goes wrong, and the error info like below:
and I use Tensorflow2.4 with compatible tensorflow_addons, it also make the same question.
some friends online say, it caused by tensorflow_addons's  version, and i try to fixed it but failed as he provide info
All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "Not found: Could not locate the credentials file.". Retrieving token from GCE failed with "Aborted: All 10 retry attempts failed. The last failure: Unavailable: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Couldn't resolve host 'metadata'".
2021-10-22 04:48:16.978059: E tensorflow/core/platform/cloud/curl_http_request.cc:596] The transmission  of request 0x55c106c4f8d0 (URI: https://www.googleapis.com/storage/v1/b/tfds-data/o/dataset_info%2Fceleb_a%2F2.0.1?fields=size%2Cgeneration%2Cupdated) has been stuck at 0 of 0 bytes for 61 seconds and will be aborted. CURL timing information: lookup time: 0.014351 (No error), connect time: 0 (No error), pre-transfer time: 0 (No error), start-transfer time: 0 (No error)
2021-10-22 04:49:19.928509: E tensorflow/core/platform/cloud/curl_http_request.cc:596] The transmission  of request 0x55c106c4f8d0 (URI: https://www.googleapis.com/storage/v1/b/tfds-data/o/dataset_info%2Fceleb_a%2F2.0.1?fields=size%2Cgeneration%2Cupdated) has been stuck at 0 of 0 bytes for 61 seconds and will be aborted. CURL timing information: lookup time: 0.015881 (No error), connect time: 0 (No error), pre-transfer time: 0 (No error), start-transfer time: 0 (No error)

Hi @justinge, thanks for getting in touch. For support issues, please use one of the resources below. That way your question will get better visibility.

TensorFlow forum
Keras mailing list

Thanks!


DeleteAsync not Intercepted by EF6 IDbCommandInterceptor

We've added an IDbCommandInterceptor from our DbConfiguration instance to capture all Reader, Scalar, NonQuery executions for logging.
AddInterceptor(new MyDbPerformance());
All of our own code passes through this interceptor and is logged, but any deletes we execute via EntityFramework.Extensions.BatchExtensions.DeleteAsync does not.
EntityFramework.Extended.<IP_ADDRESS>
Any ideas if\how we can get this to work?

You can use my update
https://github.com/loresoft/EntityFramework.Extended/pull/237

Hello @phil000 , @phucdat1d  ,
This library is no longer supported since 2015.
We recommend you to move to Entity Framework Plus which is also a free & open source library.
This issue have good chance to already be solved in EF+
Best Regards,
Jonathan


remove unnecessary objects

Remove all provider attributes

so it was not an issue with this PR, publishing compute algorithms works


Cannot resolve configuration property 'preliquibase.continueOnError'

What I did do wrong?

This is an IDE question. Not per se a question related to PreLiquibase.
Some IDEs have a feature whereby they know "know" (for Spring Boot) which configuration properties are valid. I'm guessing your IDE is IntelliJ IDEA?   The feature in that IDE relies on some metadata to figure this out. For third party libraries you'll often see this message. It doesn't mean you you've spelled anything wrong. It just means that the library author hasn't bothered to publish such metadata along with the library.
I've opened a feature request on this topic.
Closing this ticket as this is not a bug.

I purposed a PR that handle that : https://github.com/lbruun/Pre-Liquibase/pull/29


[RFC]: Make device agnostic for diverse hardware support

Motivation.
vLLM has already been adapted to many hardware devices, such as GPU, TPU, and XPU. However, adapting these backends requires implementing separate Worker/Executor/Model Runner frameworks for each, which leads to code redundancy and maintenance difficulties.
In fact, these hardware framework codes can be abstracted at the device layer, forming a unified framework. This way, only one set of code would need to be maintained, and different backends would only need to implement the device layer interfaces and any device-specific logic if necessary.
I also found some new features are only updated on GPU-related codes. In fact, these codes are also applicable to other hardware, but it is difficult for other hardware to perceive and follow these updates.
Proposed Change.
This RFC is intended to establish a unified framework.
Maybe there will be diffuculty for intergrating hardware framework to common framework, It makes sense to work towards this direction, the diagram below represents a proposed solution:

Taking Executor as example, for third-party hardware devices based on the pytorch ecosystem, the basic interfaces of torch have been well adapted, so after abstracting the device-related hard coding, such as torch.cuda, torch.xpu, GPU Executor could be used as the Common Executor  of all third-party devices.
Following https://github.com/vllm-project/vllm/pull/6080, different hardware backends can put their own device-specific code in NewBackendPlatform, so that the framework can be device-agnostic through current_platform. For example, torch.cuda.synchronize could use current_platform.synchronize.
Feedback Period.
To realize this idea will involve more files, so the following steps are currently sorted out to finally achieve the above purpose：

BackendPlatform

[x] Neuron
[x] Openvino

Backend Type Check

[x] is_cpu           ->  current_platform.is_cpu
[x] is_xpu           ->  current_platform.is_xpu
[x] is_openvino  ->  current_platform.is_openvino
[x] is_neuron      -> current_platform.is_neuron
[x] is_hip             -> current_platform.is_rocm

Backend Releated Func

[ ] seed_everything              -> current_platform.seed_everything
[ ] is_pin_memory_available -> current_platform.is_pin_memory_available
[ ] DeviceMemoryProfiler     -> current_platform.memory_profiler
[ ] wrap_device                     -> current_platform.wrap_device

Backend Releated Hard Coding

[ ] torch.xxx.get_device_name      -> current_platform.get_device
[ ] torch.xxx.Event                         -> current_platform.Event
[ ] torch.xxx.synchronize               -> current_platform.synchronize
[ ] torch.xxx.Stream                       -> current_platform.Stream
[ ] torch.xxx.stream                        -> current_platform.stream
[ ] torch.xxx.empty_cache              -> current_platform.empty_cache
[ ] torch.xxx.device_count              -> current_platform.device_count
[ ] torch.xxx.memory_allocated      -> current_platform.memroy_allocated
[ ] torch.xxx.set_device                   -> current_paltform.set_device
[ ] torch.xxx.current_device            -> current_platform.current_device
[ ] torch.xxx.get_device_capability  -> current_platform.get_device_capability

Try to unify hardware framework, cpu releated framework may have problem to intergrate.

[ ] gpu(neuron,openvino,tpu,xpu,..)_executor         -> common_backend_executor
[ ] gpu(neuron,openvino,tpu,xpu,..)_worker            -> common_backend_worker
[ ] gpu(neuron,openvino,tpu,xpu,..)_model_runner -> common_backend_model_runner

There must be omissions or difficulties in actual implementation here, keep updating.
CC List.
@youkaichao @WoosukKwon
Any Other Things.
No response
Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

we can do it step by step.

is_cpu -> current_platform.is_cpu
is_xpu -> current_platform.is_xpu
is_openvino -> current_platform.is_openvino
is_neuron -> current_platform.is_neuron

this can be the first step, and should be easy to do.
the rest might need some case-by-case discussion.

JFYI, refactoring of neuron backend checking is done by #9374

Although I think this device-agnostic framework is wonderful, it is actually quite challenging to find a balance between high-level abstraction and low-level performance.

can you elaborate on that?

Hey, I think the idea is very interesting and the problem surely must've been tackled many times across many projects.
Personally I think the unified interface that needs to be provided here is a bit too granular still, ie worker needs to call too much into accelerator-specific functions to carry out its logic.
Bringing one example to the table, ort https://onnxruntime.ai/docs/execution-providers/ has the concept of "ExecutionProvider", but the interface it's simple enough as to group common operations into higher level framework-specific abstractions so you don't have to implement dozens of functions. TFLite had delegates but I think the example isn't as good.
Some pain points off the top of my head: execution on cpu will likely implement a small subset of all the ops , executor/worker/interface logic has to have good defaults. Calling into accelerator closed source lib may not implement all the functions (ie not applicable here but still, CoreML), same point.

Each backend in ort will implement its own ExecutionProvider which based IExecutionProvider, i think it is similiar with xxxPlatform and Platform in vllm. For cpu, it is also easy to remove or reimplement some ops based Platform.

I think this part is fine:

for the rest, we should have more discussion before we take actions:

Of course, a full discussion is necessary.
Thanks for your diccusion and help, we have finished the first step of Backend Type Check and are ready to work for Backend Releated Func.
This second step wants to remove backend releated func which need if...else.. to process for different hardwares. The Platform will provide the same interface as func name, and each xxxPlatform can implement its own.
Can you give me some advice for the second step? Thanks.

Of course, a full discussion is necessary. Thanks for your diccusion and help, we have finished the first step of Backend Type Check and are ready to work for Backend Releated Func. This second step wants to remove backend releated func which need if...else.. to process for different hardwares. The Platform will provide interface with the same func name, and each xxxPlatform can implement its own. Can you give me some advice for the second step? Thanks.

@youkaichao could you give some advice on this?

you can try to find some code like this, very long if-else branching logic based on current_platform . they can be unified by current_platform.get_default_atten_backend or something like that.

we should start by sorting the number of if-else branches.
if there are more than 3 branches, it means at least 3 backends support this feature, and we can move it inside platforms .
if not, we can just keep them right now.

we should start by sorting the number of if-else branches.
if there are more than 3 branches, it means at least 3 backends support this feature, and we can move it inside platforms .
if not, we can just keep them right now.

Thanks! I got what you mean, I'll do this work step by step.

@youkaichao I list the remaining methods involving multiple backend branches, and will implement them one by one in the following PRs. If you have any suggestions, please let me know.

code path
func
func-refactor
num of related backends
other info

vllm/config.py
ModelConfig._verify_quantization
current_platform
rocm/tpu/neuron

vllm/config.py
DeviceConfig.__init__
current_platform.device_config_init
cuda_like/neuron/hpu/openvino/tpu/cpu/xpu

vllm/utils.py
is_pin_memory_available
current_platform.is_pin_memory_available
xpu/neuron/hpu/cpu/openvino
TODO: How to deal with in_wsl? Just leave it here?

vllm/model_executor/custom_op.py
CustomOp.dispatch_forward
current_platform.custom_forward
when enabled, rocm/cpu/hpu/tpu/xpu/cuda-for-default

please don't directly change DeviceConfig.__init__, but have current_platform.device_type to be a string, and call current_platform.device_type in DeviceConfig.__init__ .
let's do it step by step, others need further discussion.

I added https://github.com/vllm-project/vllm/pull/10402 as a first step to absorb some config checking and updating code into platforms/ . @wangshuai09 @MengqingCao if you are interested, welcome to do the same thing for xpu executor/open vino executor, etc.

I added #10402 as a first step to absorb some config checking and updating code into platforms/ . @wangshuai09 @MengqingCao if you are interested, welcome to do the same thing for xpu executor/open vino executor, etc.

Sure, I'll strat this work at xpu exectutor :-)

@MengqingCao CustomOp.dispatch_forward is more complicated, let's keep it unchanged right now.

@youkaichao What do you think about the refactoring of is_pin_memory_available? I noticed that wsl is only checked in is_pin_memory_available, thus I would prefer to just leave it here and add is_pin_memory_available for each platform.

@youkaichao What do you think about the refactoring of is_pin_memory_available? I noticed that wsl is only checked in is_pin_memory_available, thus I would prefer to just leave it here and add is_pin_memory_available for each platform.

This has been completed by #11163.


[WIP]fix: allow cross doamin resource sync

这个 PR 实现什么功能/修复什么问题:
允许跨域资源同步

[ ] 功能、bugfix描述
[ ] 冒烟测试
[ ] 单元测试编写

是否需要 backport 到之前的 release 分支:

release/3.4

/area region
/cc @swordqiu

Codecov Report

Merging #8075 into master will increase coverage by 11.00%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master    #8075       +/-   ##
===========================================
+ Coverage    6.22%   17.23%   +11.00%
===========================================
  Files         759       71      -688
  Lines      116620     7916   -108704
===========================================
- Hits         7260     1364     -5896
+ Misses     108706     6452   -102254
+ Partials      654      100      -554

Flag
Coverage Δ

#aFlag
17.23% <ø> (+11.00%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/compute/models/dbinstancenetworks.go

pkg/compute/models/networkinterfacenetwork.go

pkg/util/seclib2/aes.go

pkg/multicloud/ucloud/network.go

pkg/compute/models/specs.go

pkg/keystone/models/project_resources.go

pkg/baremetal/tasks/unmaintenance.go

pkg/compute/models/zoneresource.go

pkg/keystone/models/assignments.go

pkg/util/samlutils/encrypt.go

... and 676 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9a7eae6...f4d61d1. Read the comment docs.


Update mirror url for 4.14 candidate bits

4.14 is in dev phase so mirror url changed to ocp-dev-preview

/retest

/retest


Making repo private and public again makes dataset preview unavailable

I get a Server error "Unauthorized"
Slack discussion https://huggingface.slack.com/archives/C0311GZ7R6K/p1655366487928519

I'll try to create e2e tests for these use cases and fix them for good. I thought it had been fixed with the previous commits.

In the cache database, /splits collection, there are no entries for esc-bench/esc-diagnostic-dataset, esc-bench/esc-diagnostic-backup or polinaeterna/diagnostic.
Note: all these datasets are gated.
Maybe the datasets server never received the webhooks for some reason.
Also: even if we fix the webhook, we might want to automatically try to refresh the cache in this case.

Manually launched:
curl -X POST https://datasets-server.huggingface.co/webhook -H 'Content-Type: application/json' -d '{"event": "update", "repo": {"type": "dataset", "name": "esc-bench/esc-diagnostic-dataset"}}';

to refresh the cache, but it has created no job, and the cache has remained empty. It must be an issue in the webhook handler in the API service

Recovering:

Done. Note that something seems to be broken on the spgispeech config: https://huggingface.co/datasets/esc-bench/esc-diagnostic-dataset/viewer/spgispeech/other


[Chore] Update TRL to 0.1.0

Completed as part of the 0.14.0 release. The system's TRL version has since been updated to 0.1.1.


policy.errorlog.warningwindow not fully honored, leading to failed tests

Bug Report
General Troubleshooting steps

[X] Verified running the latest release of dbachecks?
[X] Verified errors are not related to permissions?

Version Information

Operating System (Name|Version): Windows 7
PowerShell Version: 5.1

Steps to Reproduce
<--
PLEASE include as much information as possible if this is a bug report.
The more you include the faster we can identify the problem and get it fixed
-->
To reproduce, all you need to do is to check the errorlog for an instance that has been up for a long while and has high-severity errors that occurred before the policy.errorlog.warningwindow starts.
Description of Bug
The behavior is that policy.errorlog.warningwindow is not fully checked, resulting in "old" high-severity errors causing failed tests.
In Instance.Assertions.ps1, Get-ErrorLogEntry does check the CreateDate of each log against policy.errorlog.warningwindow. This is effective as a performance tweak because searching logs that predate the policy.errorlog.warningwindow is pointless and time-consuming.
BUT, the LogDate of each error record must also be checked because not every site rotates their logs on a daily (or weekly or even monthly) basis and the current log can easily have "old" errors.
I will be submitting a patch for this shortly.
<--
PLEASE include as much information as possible if this is a bug
Explain what it does that you would like it not to do
or
What it doesnt do that you would like it to do :-)
-->

closed with #638


Add Certificate authentication to Set-AKSHCIRegistration, since this cmdlet requires an SPN secret despite many environments disallowing SPN secret creation

Title: Add SPN certificate authentication to Set-AKSHCIRegistration and entire deployment workflow for more secure automation purposes (-skiplogin doesn't work because this cmdlet seems to require an SPN text secret)
Description:
Request: I am requesting that certificate authentication of the type "Connect-AzAccount -certificatethumbprint <installed_cert_thumbprint> -Tenant  -ApplicationId <app_id> -serviceprincipal" could be used for this cmdlet since Connect-AzAccount is used under the hood anyway.  I am also requesting that the AKSHCI workflow can be done entirely with SPN certificate authentication and no secret text keys.
Environment:
Windows server 2022 physical servers
Public Azure tenant
Problem I am trying to solve: In environments where SPN text secrets are disallowed, this cmdlet does not seem to work.  Therefore it is not possible to deploy AKSHCI hybrid clusters even with an SPN having owner or contrib roles on the sub. In environments where I can use SPN text keys, it WILL work.   I think maybe the issue is that this specific cmdlet requires an SPN secret even when giving the cmdlet an access token for both graph and arm.  I tried to look as far as i could see in the code to see whether the cmdlet is looking for the secret key in the ARMcontext json, because I think that is what is happening rather than relying on context.
This feature could be considered as fixing a security weakness as well as possibly a bug?  It seems possible to use the certificate authentication context in order to call all the needed tokens for access instead of using a secret text key.
After testing with version 1.1.4 and 1.2.4 , it seems that neither of the following work  :

adding the ARM and Graph access tokens like so:
Connect-AzAccount -certificatethumbprint <installed_cert_thumbprint> -Tenant  -ApplicationId <app_id> -serviceprincipal;
$graphtoken =  Get-AzAccessToken -ResourceUrl 'https://graph.windows.net/'
$armtoken = Get-AzAccessToken -ResourceTypeName Arm
Set-AksHciRegistration -subscriptionId "$subid" -resourceGroupName $rg -TenantId $tenantid -GraphAccessToken $graphtoken.token -ArmAccessToken $armtoken.token -AccountId <spn_id>

Trying to bypass login to force this cmdlet to rely on the underlying account context:
Connect-AzAccount -certificatethumbprint <installed_cert_thumbprint> -Tenant  -ApplicationId <app_id> -serviceprincipal;
Set-AksHciRegistration -subscriptionId $subid -resourceGroupName $rg -TenantId $tenantid -SkipLogin

No matter what I do, this cmdlet claims "insufficient privileges error" despite the SPN having owner role over the entire subscription so long as the SPN does not have a secret text key.  In other environments where I can use SPN secret text keys, this cmdlet will work.
This may also be a possible fix for Set-AksHciRegistration is not saving SPN secret in AzureRmContext.json causes install-AksHci fail

The latest release of https://github.com/Azure/aksArc/releases/tag/AKS-arc-2411  has an update to decouple from Az Accounts module.
$credential = New-Object System.Management.Automation.PSCredential($AzureAppliacationId, ($servicePrincipalKey | ConvertTo-SecureString -AsPlainText -Force) )
Set-AksHciRegistration -SubscriptionId "${AzureSubscriptionID}" -TenantId $tenantId -Credential $credential -ResourceGroupName $azureResourceGroup
Install-AksHci -credential $credential


Consider replacing setup tools with hatch

hatch(ling) is recommended to use now, instead of setuptools

Is it really recommended or just an alternative option?

Currently, I'm looking into scikit-build-core, because it has good support for cmake.
As part of making gaussianfft work with Python 3.11 (and hopefully 3.12), I'm trying to make the compilation process simpler


Send image from destination

now the images are hard coded.
we want to send images as rest api call from the destination service.

#14
Done


weird signature in docs

// copy serverSettings from input to the store at /settings
signal('settingsOpened', [
  [
    getServerSettings, {
      success: [
        copy('input:serverSettings', 'state:settings')
      ]
      error: []
    }
  ]
]);

maybe better just leave only chains?:
// copy serverSettings from input to the store at /settings
export default [
  [
    getServerSettings, {
      success: [
        copy('input:serverSettings', 'state:settings')
      ]
      error: []
    }
  ]
]

agreed

fixed


fix(build): move BUILD rules next to go code

This should fix some errors I'm seeing on google3 import related to package boundaries.


Feature request: Please drop the direct use of int, short, etc in favour of int_least32_t, int_least16_t

I'm designing an ABI solution to programming across different platforms and this library looks like it would do nicely as the backend for game related APIs in it. The only thing that complicates that is the inconsistent ABI sokol currently uses by tying itself to the data model instead of using fixed size integers and the like.
I could probably get away with using only CPU register sized integers for the public API part but it would be nicer to use the same sized integers of a future version of sokol that drops the data model reliance in favour of fixed size integers.
For instance int sapp_gl_get_major_version(...) could instead become int_least16_t sapp_gl_get_major_version(...). Even if sokol were to somehow reach 30000 odd major versions and need to change to int_least32_t, by that point my library would've also changed ABI and version (because the ABI is tied to the version and accompanying launcher).

Are you planning to support platforms where int isn't 32-bit, if yes, which?

long can be 32 or 64 bit,
int can be 16, 32 or 64 bit
short can be 16 or 64 bit
void* can be 16, 32 or 64 bit
These details are all part of the data model. They're details I don't build into the libary's exposed API. They can be used internally and externally but never directly. For example I have something to the effect of this in my library:
// GNUC compilers mandated so this is not a problem
typedef union __attribute__((transparent_union))
{
    int_least64_t fsp; // File system position
   uint_least64_t abs; // Platforms where the registers are alias for the same register will always see the full pointer via this
            void *ptr;
      void const *cptr;
           uchar *top;
     uchar const *ctop;
} abs_t;

API_CB abs_t mrawcpy( abs_t dst, abs_t src, abs_t cpy );

#ifndef _LIB
inline size_t lrawcpy( void *dst, void const *src, size_t cpy )
    { return mrawcpy( dst, src, cpy ).abs; }
# ifdef COMPILE_APP_LOCALS
extern inline size_t lrawcpy( void *dst, void const *src, size_t cpy );
# endif
#else
inline size_t lmrawcpy( void *dst, void const *src, size_t cpy )
    { return mrawcpy( dst, src, cpy ).abs; }
# ifdef COMPILE_LIB_LOCALS
extern inline size_t lmrawcpy( void *dst, void const *src, size_t cpy );
# endif
#endif

Unless the inline in question does not use the data model in anyway it will be compiled as private to the software being compiled. So the app cannot access the library's internal inlines and vise versa. The library will always be compiled as native to the platform but the app will start as compiled for the architecture only. *.exe vs *.elf vs etc is combated via the launcher that will come with each version of the library. The launcher expects to find apps built for it's library in 3 places, each with their own purpose.
/.lib/lib/all_//.a
~/.lib/lib/all_//.a
$OTG_DIR/.lib/lib/all_//.a
Alternatively there's:
/.lib/all//.mak
~/.lib/all//.mak
$OTG_DIR/.lib/all//.mak
Or
/.lib/all//src//build.mak
~/.lib/all//src//build.mak
$OTG_DIR/.lib/all//src//build.mak
The library will hand MAIN_O and LIBMAIN_O variables to the makefiles along with the expected ABI path so the final app/lib can be made and passed into the PATH variable.
Basically I'm abusing a lot of common stuff to produce a new software ecosystem that does not need to know the operating system under which their binaries are being run. The option will be there to go native but it will not be needed for most software (such as eog which only needs to be able to open image files and can defer the file browsing prompt to whatever my library linked).

btw I don't really care which of the fixed width integer sizes you use so long as it's fixed width.

long can be 32 or 64 bit,
int can be 16, 32 or 64 bit
short can be 16 or 64 bit

Theoretically that's true, but the more important question is: are there any relevant platforms and compilers where int isn't a 32-bit signed integer? The only platforms I'm aware of are 16-bit platforms like Intel 80286, some early Amiga compilers and embedded 8-bit platforms like the Arduino, but the sokol headers don't support any of those platforms anyway.
It's the same as little- vs big-endian. In theory code should be endian agnostic, but in practice little-endian has won and big-endian is an exotic edge case that's not worth supporting anymore.

The backend of sokol would still be implementation defined in those cases but having a standard expectation of ABI built into the API would force those implementations to at least present the results in the expected sizes, even if it doesn't need those sizes for what it gives. This library and launcher of mine is aimed at any platform, that includes consoles from way back when like the nintendo gameboy. Not expected to be used fully in the final product for sure but it is expecting to at least make the initial development easier by allowing devs to develop on native platform then gradually prune the result until it fits on the target console's memory (since pruning would just mean reducing functions and data needed).

This would be a world of hurt. And almost no gain.

Not really, it's mainly just changing from variable sized integers (short, int, long) to fixed size integers. For anything native (such as the backend for instance) you would just slap something like #ifdef USE_NATIVE_SOKOL around it. Doesn't have to be all in one shot either, just something to help you start a programming session for instance (since the 1st 15min is the hardest to start). you can add a prefix too for where you want to draw the line between ABI and API. In my wrapper I'm using mpaw/'lmpaw for module/library functions, inlines & typedefs (which is where the ABI stuff is handled); paw for current binary stuff (everything is an inline to the mpaw stuff, data model is handled via a static object that the inlines pass onto the mpaw calls), and I'm using npaw for anything fully native (like npawfso_open()) which passes unverified parameters straight onto open(), CreateFileA() or CreateFileW. There's plenty to be gained with this approach because it directly removes unexpected behaviour caused by unnoticed assumptions about the ABI. If the ABI is well defined then you can force the middle layer (the defined ABI) to have optimised behaviour.
For example let's say you have something like void sokol_addivec4( int a[4], int b[4] ) {...} The optimisations applied on the assumption int is 4 bytes can lead to buggy behaviour on platforms where it's not (such as the LP32/ILP64/SILP64 models) whereas if it's say void msokol_addivec4( int32_t a[4], int32_t b[4] ) you can inline the former to msokol_addsvec4(...) in the LP32 model and msokol_addlvec4(...) in the ILP64/SILP64 models and know that even if sokol_addivec4(...) behaves wrong on some new data model it's not because of the functions it's inlined to but because the inline is mapping the wrong function on that model.
In short bug fixing becomes easier if you've already ruled out the size of the integers, structures, etc effecting the implementation.

As you point out int is not fixed size, but neither is int_least32_t so I can't see that changing to these can ever be useful unless you're working on systems where int is less than 32 bits.
Only retro machines had that, and they wouldn't be using Sokol for graphics.

The variable width of int_least*_t isn't a problem, it just helps me to create wrapper calls when I get round to the graphics side of things. My types are fixed in size via the _BitInt type so I only need to concern myself with what's the maximum the source API could be taking on any system. If the maximum is 32 bits then I can just declare pawe32d as the size the wrapper uses. On the other hand if you don't define those max width expectations then I have to use pawrd which is defined as the register width and cannot optimise array sizes as a result.

I'll close this ticket as 'not planned'. You can safely assume that int is 32 bits on all platforms supported by sokol_gfx.h.


Fails on Hoogle

Given a checkout of Hoogle from https://github.com/ndmitchell/hoogle I get:
C:\Neil\hoogle>dead-code-detection --root Main -isrc
dead-code-detection: Could not find module `Input.Type'
Use -v to see a list of the files searched for.

C:\Neil\hoogle>dead-code-detection --root Main -isrc -v
unrecognized option `-v'

The reason is that src/Hoogle.hs imports Input.Type. But src/Hoogle.hs isn't anywhere on the dependency graph from Root, so is unnecessary - it's a totally dead file which doesn't compile.
Saying use -v and then -v not working is a bit sad, but I suspect that's the GHC API doing it for you.

Here's a PR that gives a more sensible error message: #5. The output now clearly states that the error message is from ghc. I hope this will result in less confusion. (If you don't agree, I'd like to hear about better ideas.)
Regarding the behavior apart from the error messages: dead-code-detection intentionally doesn't use ghc to resolve the dependencies, but looks at the file system itself. This way it can find dead code in modules that aren't imported anywhere. (And as you point out, src/Hoogle.hs is a totally dead file, so the tool should find and report it.) I do consider this an important feature and I tend to think this is the right default behavior.
Would you want to have a way to tweak this default behavior? How exactly would you imagine that to work?
Apart from that: If you just delete src/Hoogle.hs dead-code-detection seems to be working fine.

Scanning the file system seems entirely reasonable. I'd be tempted to say that if a module can't be parsed you: 1) print out a message saying it can't be parsed, and 2) pretend it is a blank module with no uses and no exports. In this case seeing an error for this module, and then seeing useful results for the rest of the code, would be useful. In many of my projects I have travis.hs modules or similar that are only run on Travis, and where I might not have the necessary dependencies installed locally.

I should say the behaviour of continuing would be nice, but not essential - it sounds like its now clear how to avoid the issue, so not a big problem if you decide to do nothing.

In this case, the module could be parsed (or at least the import statements), but the imported modules couldn't be found.
In any case, I'll merge these changes.
Regarding the idea to ignore modules with errors while warning about them: I wonder if this would be bad behavior in the case where your project actually doesn't compile, because you have a type error or the like. You would see a warning that one module doesn't compile, but then all other modules that (transitively) import that module would also not compile. And also issue a warning, that they don't compile?
I always liked the idea that a prerequisite of dead-code-detection is, that your project (and every file in it) compiles.
But I can perfectly see, why you'd want to have files like travis.hs (or other scripts) that you want to exclude from being analyzed. What do you think about an additional command line flag to ignore certain files. For example:
dead-code-detection --root Main -isrc --ignore src/Hoogle.hs

An ignore flag works too, and is probably the simplest and most direct solution.

I've opened #8 and #7 instead of this ticket, therefore closing this. If you have further problems with hoogle, please, re-open.


Add support for Italian DGC settings

This is a prerequisite for #38.

Codecov Report

Merging #39 (cbd522b) into main (926a814) will decrease coverage by 10.88%.
The diff coverage is 39.86%.

@@             Coverage Diff             @@
##             main      #39       +/-   ##
===========================================
- Coverage   85.00%   74.11%   -10.89%
===========================================
  Files          10       12        +2
  Lines        1040     1294      +254
===========================================
+ Hits          884      959       +75
- Misses        156      335      +179

Impacted Files
Coverage Δ

dgc/src/cwt.rs
80.00% <ø> (ø)

dgc/src/dgc.rs
93.84% <ø> (ø)

dgc/src/dgc_container.rs
78.00% <ø> (ø)

dgc/src/lib.rs
100.00% <ø> (ø)

dgc/src/parse.rs
73.28% <ø> (ø)

dgc/src/recovery.rs
50.00% <ø> (ø)

dgc/src/test.rs
96.00% <ø> (ø)

dgc/src/trustlist.rs
58.33% <ø> (ø)

dgc/src/vaccination.rs
50.00% <ø> (ø)

dgc/src/valuesets.rs
100.00% <ø> (ø)

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 926a814...cbd522b. Read the comment docs.


Remove form content control from docx

Using python-docx, I would like to perform what MS Word calls, "Remove Content Controls".
In Word, one must select all, right click, and choose "Remove Content Controls", and save.
The result: populated text remains, and python-docx can now see all of the text.
Before removing content-control form fields, the content text is not easily accessible via python-docx.
Using python-docx==0.8.10

Please ask support questions on StackOverflow using the "python-docx" tag. This issues
list is for suspected bugs and new feature requests.
There is no API support for this in python-docx. You'd need to manipulate the underlying XML. Have a search on "python-docx workaround function" to get an idea how that could be easier starting with python-docx to get into the right part of the XML.

This was a feature request, not a bug.


Monomorphize WIP

I've branched off vcm/refactor-symb-eval about a week ago, so the result is a mess, but it gives a rough idea. Pirouette.Transformations.Monomorphization is the key module, and the rest of the changes in this PR are some very minor and non-semantic ones.
I'll rebase this on top of whatever once we decide on the merge sequence.

There's still work to do wrt substitutions that we discussed, but it can be done separately.


Change language: enables the user to change language on the modern front-end

Enables the user to change user interface language on the modern front-end just like on the old one

@rhh-ft any updates?

@rhh-ft should we take over this PR? Is there any plan to address comments from your end?

@tananaev Hi sorry for the late response I have been on holiday. It would be better if you guys would take over at this point

Hi @tananaev @rhh-ft
I've played this PR.
As an end user is it not better to keep language preference (including rtl/ltr) after the page reloads?

It is better. I don't think anyone is arguing otherwise.


Fixed innerRef prop type in HotKeys

Fixed the prop type for innerRef to receive function as value for cases where ref is not instantiated using React.createRef.
Current behavior:

Thanks, @shripadk. This is great.


有编译步骤吗？

有编译步骤吗？
直接Git clone之后，go build 报错

如果报错，请把错误贴上来

xspee@DESKTOP-7I2R7FR MINGW64 ~/Desktop
$ cd goflyway/

xspee@DESKTOP-7I2R7FR MINGW64 ~/Desktop/goflyway ((v2.0.0rc1-caddy))
$ ls
chinalist.txt  goflyway.conf  makefile  proxy/     script/
cmd/           LICENSE        pkg/      readme.md  test.acl

xspee@DESKTOP-7I2R7FR MINGW64 ~/Desktop/goflyway ((v2.0.0rc1-caddy))
$ cd cmd/

xspee@DESKTOP-7I2R7FR MINGW64 ~/Desktop/goflyway/cmd ((v2.0.0rc1-caddy))
$ cd goflyway/

xspee@DESKTOP-7I2R7FR MINGW64 ~/Desktop/goflyway/cmd/goflyway ((v2.0.0rc1-caddy))
$ go build
main.go:16:2: cannot find package "github.com/coyove/common/config" in any of:
        D:\Software\Go\src\github.com\coyove\common\config (from $GOROOT)
        C:\Users\xspee\go\src\github.com\coyove\common\config (from $GOPATH)
main.go:17:2: cannot find package "github.com/coyove/common/logg" in any of:
        D:\Software\Go\src\github.com\coyove\common\logg (from $GOROOT)
        C:\Users\xspee\go\src\github.com\coyove\common\logg (from $GOPATH)
main.go:18:2: cannot find package "github.com/coyove/common/lru" in any of:
        D:\Software\Go\src\github.com\coyove\common\lru (from $GOROOT)
        C:\Users\xspee\go\src\github.com\coyove\common\lru (from $GOPATH)
main.go:19:2: cannot find package "github.com/coyove/goflyway/cmd/goflyway/lib" in any of:
        D:\Software\Go\src\github.com\coyove\goflyway\cmd\goflyway\lib (from $GOROOT)
        C:\Users\xspee\go\src\github.com\coyove\goflyway\cmd\goflyway\lib (from $GOPATH)
main.go:20:2: cannot find package "github.com/coyove/goflyway/pkg/aclrouter" in any of:
        D:\Software\Go\src\github.com\coyove\goflyway\pkg\aclrouter (from $GOROOT)
        C:\Users\xspee\go\src\github.com\coyove\goflyway\pkg\aclrouter (from $GOPATH)
main.go:21:2: cannot find package "github.com/coyove/goflyway/proxy" in any of:
        D:\Software\Go\src\github.com\coyove\goflyway\proxy (from $GOROOT)
        C:\Users\xspee\go\src\github.com\coyove\goflyway\proxy (from $GOPATH)
main.go:22:2: cannot find package "golang.org/x/crypto/acme/autocert" in any of:
        D:\Software\Go\src\golang.org\x\crypto\acme\autocert (from $GOROOT)
        C:\Users\xspee\go\src\golang.org\x\crypto\acme\autocert (from $GOPATH)

xspee@DESKTOP-7I2R7FR MINGW64 ~/Desktop/goflyway/cmd/goflyway ((v2.0.0rc1-caddy))

go get ./... 就可以了


Add erase resource (button)

Route added in Knora-API: https://github.com/dhlab-basel/Knora/issues/1421

This needs a new method first in dasch-swiss/knora-ui core module. I will implement it...

The delete button will take place in the dasch-swiss/knora-ui repo in @knora/viewer module. I already started with a toolbar in resource viewer: wip/viewer-resource-toolbar

this issue is irrelevant due to the viewer refactoring and the integration of knora-api-js-lib -> close


Directions for users to configure extension

The G2G thread about the new extension has questions from users (of both Chrome and Firefox) about how to configure the extension.
We should have some concise, simple directions for configuration.
Where should this go? If they haven't figured out how to install the extension, it can't go in the extension. A G2G post would get stale very quickly. So the extension is separate from the core, but should there be some basic help about how to find, install, and open configuration page for the extension on the Help pages?

I've updated the space page with more information.  Feel free to add to it or edit it.  Yes, I think there should be something about the extension on Help pages and (definitely) on the Apps page.

I'll create a basic Help page (so people can find it when searching the Help pages), and link to the Space page from there.
Most details should be on the Space page (since only Leaders/Sysops can edit the Help pages) -- so any suggestions on what I should put on the Help page?

I would recommend something like, well, its a really rough draft
If you are using a Chrome browser or any of the following browsers Opera, Edge, Brave, and Vivaldi:

Install the extension blah blah
To configure the installed extension:

click the tool icon in the upper right corner of the browser window
under Extensions click on the three ... by WikiTree Browser Extension and then click Manage Extension
on the page for the extension scroll down and click on Extension Options.
on the WikiTree Extension options page click the button(s) to turn feature(s) on or off

If you are using the Firefox browser:

From the Tools menu in the Firefox top toolbar, select Add-ons and Themes
Search for WikiTree Browser to find the extension, then click on the WikiTree Browser Extension to install it
The Tools->Add-ons and Themes menu opens your Manage Your Extensions page. From this page, click on the ... next to WikiTree Browser Extension
Click Manage to see what version you have and to see the release notes
Click Preferences to open the Extension Options page

on the WikiTree Extension options page click the button(s) to turn feature(s) on or off

The way I configure the Chrome extension is a different way which seems easier to me:
After installing:

Click the puzzle piece icon to the right of the URL bar and press the pin next to WBE

Each time you want to edit configuration:

Click the WBE icon (tree) to the left of the puzzle piece and select "Options..."

I think extension deserves a link from the main WikiTree Menus. Maybe under Help / Extension.
It could link to Chrome store, but that is a problem for Firefox and IOS. Next best thing would be a Help:Extension page with links to all three stores and some First steps for each browser. Steve, Jamie and me can update the page as necessary. And there would also be links to Space pages with detailed descriptions.

Why not just link the help page to the space page and improve that?


fix(chart): chart datasource explore URL showing datasource name for druid

SUMMARY
Bug introduced on #9619, URL for druid datasources rendering name instead of explore_url
TEST PLAN

Create a druid cluster and datasource
Create a chart for the previous create druid datasource
Assert that the rendered URL for the datasource on the charts list links to the druid datasource

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #9839 into master will decrease coverage by 0.23%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #9839      +/-   ##
==========================================
- Coverage   71.02%   70.79%   -0.24%
==========================================
  Files         583      184     -399
  Lines       30615    17868   -12747
  Branches     3162        0    -3162
==========================================
- Hits        21745    12649    -9096
+ Misses       8759     5219    -3540
+ Partials      111        0     -111

Flag
Coverage Δ

#cypress
?

#javascript
?

#python
70.79% <0.00%> (-0.48%)
:arrow_down:

Impacted Files
Coverage Δ

superset/models/slice.py
83.78% <0.00%> (ø)

superset/views/database/mixins.py
58.92% <0.00%> (-21.43%)
:arrow_down:

superset/utils/cache.py
45.83% <0.00%> (-20.84%)
:arrow_down:

superset/db_engine_specs/mysql.py
78.26% <0.00%> (-13.05%)
:arrow_down:

superset/views/tags.py
35.13% <0.00%> (-4.11%)
:arrow_down:

superset/models/sql_types/presto_sql_types.py
72.41% <0.00%> (-4.06%)
:arrow_down:

superset/models/tags.py
58.58% <0.00%> (-3.80%)
:arrow_down:

superset/views/database/api.py
83.90% <0.00%> (-3.45%)
:arrow_down:

superset/views/sql_lab.py
59.06% <0.00%> (-3.29%)
:arrow_down:

superset/cli.py
36.59% <0.00%> (-3.14%)
:arrow_down:

... and 431 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5ca6ed7...2f3b0f4. Read the comment docs.


Is that any c++ implementation ?

Hi
I want to run this tracker on Nvidia Jetson and use C++ to improve performance.
how can I use this Tracker in C++?
is there a way to convert this model to tensorRT and use it in C++?

I'm afraid I'm not too familiar with deep learning inference frameworks, so I might not be the best person to answer your question. I haven't had the chance to work directly with inference frameworks like TensorRT either. However, if you have any ideas for optimization, I'd find that really intriguing and worth exploring!

I'm afraid I'm not too familiar with deep learning inference frameworks, so I might not be the best person to answer your question. I haven't had the chance to work directly with inference frameworks like TensorRT either. However, if you have any ideas for optimization, I'd find that really intriguing and worth exploring!

Hi thanks for your response ☺
Yes your method and paper is very strong for tracking (in the laboratory and science) but when you want to use this method in real and industrial environment your method will not succeed because of time consumption and requires hardware and environments...
That's why I trying to find a way that implementing this method with c++.
TensorRT is for embedded systems like Jetson.
I hope you are looking for a way to implement this method on C++
I suggest that use libtorch library based on torch.

I am also looking for a job recently, and I may not have enough time to do the aforementioned work. However, when I have free time, I will consider this matter. Thank you for your suggestion

Hi @SJavad !
I'm not sure if you are still looking for a way to use this model in C++ but you may want to look into onnx and onnx runtime. If you manage to export the model to the onnx format, you should be able to run it from anywhere such as a C++ environment.


SvelteKit types do not exist at route error

Describe the bug
I have a directory under routes, that doesn't have  any pages (just some components)
Ref:
https://github.com/xmlking/svelte-starter-kit/tree/main/src/routes/(app)/dashboard
I am getting the following error from Houdini plugin

Severity
annoyance
Steps to Reproduce the Bug

clone repo https://github.com/xmlking/svelte-starter-kit
change branch feature/houdini
npm run dev

Reproduction
https://github.com/xmlking/svelte-starter-kit/tree/main/src/routes/(app)/dashboard

I can also confirm this behaviour with the following packages on Windows :
"@sveltejs/adapter-auto": "next",
"@sveltejs/kit": "next",
"graphql": "^15.8.0",
"houdini": "^0.17.14",
"houdini-svelte": "^0.17.14",
"svelte": "^3.53.1",
"vite": "^3.2.4"

Sorry to hear you are running into this. If one of you wants to push a fix through, the logic that generates these types lives here.

Hey @xmlking,
I had a look at your repo, and it looks like the problem is because inside src/routes, Svelte only sees these files:
+page
+error
+layout
+server
You might consider moving these components inside blocks to src/lib/components/
In this link you can see the documentation about this point: https://kit.svelte.dev/docs/routing#other-files

Hey @AlecAivazis,
I debugged houdini-svelte/src/plugin/codegen/routes/index.ts and main/packages/houdini-svelte/src/plugin/kit.ts, and from what I understand, it shouldn't have this throw because Sveltekit only generates types for valid files (+page, +error, +layout, +server) for routes and ignores the rest. So, recursive walk_routes stops traversing the directory tree when it doesn't find a type, and for routes that Svelte ignores, it won't even have $types.d.ts.
Svelte $types documentation
Do you think this PR would be viable?
main/packages/houdini-svelte/src/plugin/kit.ts
+ // if type does not exists we return.
+ if (!fs.existsSync(svelteTypeFilePath)) {
+   return
+ }

- // if type does not exists we error.
- if (!fs.existsSync(svelteTypeFilePath)) {
-  throw Error(`SvelteKit types do not exist at route: ${svelteTypeFilePath}`)
- }

Most likely has to do with special characters (e.g. ( ) in a route. I'll take a look later today.

I made a pull request here and added further tests

So I actually prefer @524c's fix without the warning. There's nothing inherently wrong with a user not having kit route files in a directory. I'd like to treat the user like they know what they're doing. Since it was @524c's fix originally, i think we should let them open the PR with their contribution

Honestly, I also prefer not to have alerts, since not having types in certain routes is a normal behavior of Svelte.
I have a change from the previous proposal. Instead of the return we need a continue and let the walk_routes go on with its normal life.
I see my PR only as a patch, I couldn't see any possible new test 🤔
@sjcobb2022, can you think of any tests?
// if type does not exists we return.
if (!fs.existsSync(svelteTypeFilePath)) {
  continue
}

So I actually prefer @524c's fix without the warning. There's nothing inherently wrong with a user not having kit route files in a directory. I'd like to treat the user like they know what they're doing. Since it was @524c's fix originally, i think we should let them open the PR with their contribution

Alright I'll close mine

Honestly, I also prefer not to have alerts, since not having types in certain routes is a normal behavior of Svelte.
I have a change from the previous proposal. Instead of the return we need a continue and let the walk_routes go on with its normal life.
I see my PR only as a patch, I couldn't see any possible new test 🤔
@sjcobb2022, can you think of any tests?
// We will only visite valid routes for Svelte.
if (!fs.existsSync(svelteTypeFilePath)) {
  continue
}

I implemented some e2e checks so that we don't get the behaviour in the future but they aren't necessary in all honesty.

Honestly, I also prefer not to have alerts, since not having types in certain routes is a normal behavior of Svelte.
I have a change from the previous proposal. Instead of the return we need a continue and let the walk_routes go on with its normal life.
I see my PR only as a patch, I couldn't see any possible new test 🤔
@sjcobb2022, can you think of any tests?
// We will only visite valid routes for Svelte.
if (!fs.existsSync(svelteTypeFilePath)) {
  continue
}

I implemented some e2e checks so that we don't get the behaviour in the future but they aren't necessary in all honesty.

Ok, so I believe that if you find it necessary, you can do a PR with the tests.
My previous comment about a continue instead of a return is not right. The visite route is after the for.. 🤦


wasm_bindgen on async fn doesn't accept same types that it does on regular fn

Describe the Bug
Currently #[wasm_bindgen] on async fn seems to only accept types that are Into<JsValue>, but many wasm-bindgen types are only IntoWasmAbi, making it hard to convert synchronous code into asynchronous.
Steps to Reproduce
Try to compile code like following:
use wasm_bindgen::Clamped;

#[wasm_bindgen]
pub fn foo() -> Clamped<Vec<u8>> {
  Clamped(vec![])
}

#[wasm_bindgen]
pub async fn foo() -> Clamped<Vec<u8>> {
  Clamped(vec![])
}

Expected Behavior
Both functions compile successfully and generate their bindings.
Actual Behavior
Second function fails to compile due to:
   | #[wasm_bindgen]
   | ^^^^^^^^^^^^^^^ the trait `From<Clamped<Vec<u8>>>` is not implemented for `JsValue`

For fellow googlers. This is halfway fixed in https://github.com/rustwasm/wasm-bindgen/pull/3630


Code not reaching after the line val stream=EventHubsUtils.createUnionStream(ssc, ehParams)

Hi,
I have followed the steps provided in the link https://azure.microsoft.com/en-us/documentation/articles/hdinsight-apache-spark-eventhub-streaming/#comment-2693663776. I have written a sample eventhub receiver program as given below,
package com.onerm.spark
import org.apache.spark.streaming.{Seconds, StreamingContext}
import org.apache.spark.streaming.eventhubs.EventHubsUtils
import org.apache.spark.SparkContext
import org.apache.spark.SparkConf
import org.apache.spark.rdd.RDD
import org.apache.spark._
import org.apache.spark.sql.hive.HiveContext
import java.util.concurrent.{Executors, ExecutorService}
object HiveEvents {
def b2s(a: Array[Byte]): String = new String(a)
def main(args: Array[String]): Unit = {
val ehParams = MapString, String
val conf = new SparkConf().setAppName("Eventhubs Onerm")
val sc= new SparkContext(conf)
val hiveContext = new HiveContext(sc)
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
val pool:ExecutorService=Executors.newFixedThreadPool(5)
val ssc = new StreamingContext(sc, Seconds(120))
var dataString :RDD[String] =sc.emptyRDD
val stream=EventHubsUtils.createUnionStream(ssc, ehParams)
//lines below are not getting executed until I stop the execution
stream.print()
stream.foreachRDD {
rdd =>
if(rdd.isEmpty())
{
println("RDD IS EMPTY ")
}
else
{
dataString=rdd.map(line=>b2s(line))
println("COUNT" +dataString.count())
sqlContext.read.json(dataString).registerTempTable("jsoneventdata")
val filterData=sqlContext.sql("SELECT id,ClientProperties.PID,ClientProperties.Program,ClientProperties.Platform,ClientProperties.Version,ClientProperties.HWType,ClientProperties.OffVer,ContentID,Data,Locale,MappedSources,MarketingMessageContext.ActivityInstanceID,MarketingMessageContext.CampaignID,MarketingMessageContext.SegmentName,MarketingMessageContext.OneRMInstanceID,MarketingMessageContext.DateTimeSegmented,Source,Timestamp.Date,Timestamp.Epoch,TransactionID,UserAction,EventProcessedUtcTime,PartitionId,EventEnqueuedUtcTime from jsoneventdata")
filterData.show(10)
filterData.saveAsParquetFile("EventCheckpoint_0.1/ParquetEvent")
} }
ssc.start()
ssc.awaitTermination()
}}
The issue is that , the code after the line val stream=EventHubsUtils.createUnionStream(ssc, ehParams) is not reachable and is getting executed only when I am manually stopping the program execution by performing Ctrl+C.
After analyzing the logs of my spark-submit, I could observe that, when I am calling EventHubsUtils.createUnionStream(ssc, ehParams) its internally calling the EventHubReceiver class. The EventHubReceiver class has the EventHubsMessageHandler() class which internally has the below code which is getting looped continuously until the stopMessageHandler is set to false and I guess this value is getting set to true when I am stopping the program execution by doing ctrl+c.
Hence then only the statements after the line "EventHubsUtils.createUnionStream(ssc, ehParams)" are getting executed. Please correct me if I am wrong. Thank you!!!
while (!stopMessageHandler)
{
val message = receiverClient.receive()
if (message != null && message.getSequence > latestSequence) {
latestSequence = message.getSequence
processReceivedMessage(message)
}
val now = System.currentTimeMillis()
if(now > nextTime) {
if(offsetToSave != savedOffset) {
logInfo("writing offset to store: " + offsetToSave + ", partition: " + partitionId)
myOffsetStore.write(offsetToSave)
savedOffset = offsetToSave
nextTime = now + checkpointInterval
}
}
}

Hi @KothapallyAnitha , Do you have any workaround for this problem.
Am trying to do the same having the same issue which you faced.
Thank you for your help.

HI Ankush,
Actually later on when I  checked there is no issue with the API's being
used its was because of insufficient resources that it became unreachable.
Try changing or giving more executor memory and cores. Changing these
worked for me.
On Wed, Apr 5, 2017 at 9:52 AM, ankushreddy<EMAIL_ADDRESS>wrote:

Hi @KothapallyAnitha https://github.com/KothapallyAnitha , Do you have
any workaround for this problem.
Am trying to do the same having the same issue which you faced.
Thank you for your help.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hdinsight/spark-eventhubs/issues/3#issuecomment-291750287,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ASqZ8E_298oj9QKfhok7DctZt0QVVEeKks5rsxcCgaJpZM4ImXI6
.


Error with hardcoded image versions in ingress-nginx howto

Hi, the settings in the page give the ingress-nginx error, referred to here: https://github.com/kubernetes/ingress-nginx/issues/9928
E0712 15:44:57.996314       8 leaderelection.go:334] error initially creating leader election record: configmaps is forbidden: User "system:serviceaccount:ingress-nginx:ingress-nginx" cannot create resource "configmaps" in API group "" in the namespace "ingress-nginx"
Things seem to work ok if you take helm 4.7.1 and not override the default image versions

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: fd0e7d5a-37e9-07dc-f395-9cb5bb580e45
Version Independent ID: 7e4faf70-9724-7e7b-832b-1cd99a974920
Content: Create an ingress controller in Azure Kubernetes Service (AKS) - Azure Kubernetes Service
Content Source: articles/aks/ingress-basic.md
Service: azure-kubernetes-service
Sub-service: aks-networking
GitHub Login: @asudbring
Microsoft Alias: allensu

@adeturner
Thanks for your feedback! We will investigate and update as appropriate.

@adeturner
The error message you provided indicates a permission issue when creating a ConfigMap in the "ingress-nginx" namespace. The error suggests that the service account "ingress-nginx" does not have the necessary permissions to create ConfigMaps.
To resolve this issue, you can try granting the required permissions to the service account. Here's an example of how you can do this:
Get the name of the service account:
kubectl get serviceaccount -n ingress-nginx
Grant the necessary permissions to the service account:
kubectl create clusterrolebinding ingress-nginx --clusterrole=cluster-admin --serviceaccount=ingress-nginx:ingress-nginx -n ingress-nginx
This command creates a ClusterRoleBinding that assigns the "cluster-admin" cluster role to the "ingress-nginx" service account in the "ingress-nginx" namespace. This should provide the necessary permissions for creating the ConfigMaps.
Please note that granting the "cluster-admin" role to the service account may not be the most secure approach in a production environment. It's recommended to carefully review and restrict the permissions based on your specific requirements.
Additionally, you mentioned that using Helm version 4.7.1 and not overriding the default image versions resolved the issue for you. It's possible that there was a compatibility issue or a bug in a specific Helm version or image version you were using. Keeping the versions as recommended in the documentation is generally a good practice to ensure compatibility and stability.
If the issue persists or you need further assistance , I'd recommend working closer with our support team via an [Azure support request] (https://docs.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request). Or you can leverage our Q&A forum by posting your issue there so our community, and MVPs can further assist you in troubleshooting this issue or finding potential workarounds.
[Teams Q&A forum] (https://docs.microsoft.com/en-us/answers/topics/46488/office-teams-windows-itpro.html) for technical questions about the configuration and administration of Microsoft Teams on Windows.


derive Render

Offer a simple derive that used some default style and allows to do
#[derive(Serialize, RenderOutput)]
struct Foo { name: String, code: i32}

out.print(Foo { name: "hallo".into(), code: 42 })?;

The goal is to let people write even less code. This is an experiment, though, and only in addition to "manually" implemementing using the macros.

[x] Create proc macro crate output_derive
[x] Add a simple derive macro RenderOutput

[x] only works for structs for now
[x] generates the most trivial impl Render for $type

human output: just list all the fields separated by newlines
json output: same as render_json!();

[ ] Add attributes to fields to add some formatting/converting.

Works, and the fuzzy requirement of "formatting attributes" should be tracked in another issue.


toStringAsPrecision not working as expected

Hello dart team,
Dart SDK version: 2.9.2
I need to format double in significant figures.  I am expecting the toStringAsPrecision will perform round up.
e.g. 0.145 with precision 2 formats to 0.15.
but it formats to 0.14.
When I adding one more digi to value, e.g. 0.1451....to 0.1459, it can roundup correctly.  But not 0.1450.
Why rounding up the 2nd digi will depends on the 4th digi?

Thank you.

/cc @lrhn

My initial hunch is that it is most likely working as intended, because doubles just don't work like you expect them to!
Let's look at the numbers in question.
The real number 0.145 is indeed not precisely representable as a double. The closest double to 0.145 has the value

0.1449999999999999900079927783735911361873149871826171875

and that is the actual value your double literal 0.145 represents.
As you can see, that number is strictly below 0.145, so when rounded to two decimal digits, it chooses "0.14".
There is not a lot to do about that, it's just how doubles work. If you had written:
 print(0.14499999999999999.toStringAsPrecision(2));

you would probably have expected to print "0.14", but it's exactly the same number as 0.145, and we have to do the same thing in both cases.


Filtering issue with suggested demo application

Here is a demo application what you suggested to implement case insensitive filtering. Downloaded the code it builds fine returns values for simple queries. But filtering does not work at all.
Here I want to test case incendivity:
{
  persons(where: { name: "FRed" }) {
    id, name
  }
}

Response:
{
  "errors": [
    {
      "message": "The specified value type of field `name` does not match the field type.",
      "locations": [
        {
          "line": 2,
          "column": 25
        }
      ],
      "path": [
        "persons"
      ],
      "extensions": {
        "fieldName": "name",
        "fieldType": "StringOperationFilterInput",
        "locationType": "StringOperationFilterInput",
        "specifiedBy": "http://spec.graphql.org/June2018/#sec-Values-of-Correct-Type"
      }
    }
  ]
}

How can I write a query which satisfies all types? However I think Name property should be a string and query should be fine...
Thanks.

@PascalSenn Thanks for your reply. Yes meanwhile I have found the string query operators in you docs as well. And tried out the same query you suggested. But still getting error this time a different one. It seems System.String is threated like a complex object and filter operators as Properties. Or something like that...
Your query's response in Postman:
{
    "errors": [
        {
            "message": "The specified input object field `eq` does not exist.",
            "locations": [
                {
                    "line": 2,
                    "column": 27
                }
            ],
            "path": [
                "persons"
            ],
            "extensions": {
                "field": "eq",
                "specifiedBy": "http://spec.graphql.org/June2018/#sec-Input-Object-Field-Names"
            }
        }
    ]
}

Also when I try to use a very similar query in our application. Basically it is the same query on a different Entity type than your demo app, the eq operator matters. So even in your Query designer tool is suggesting the same problem in the designer. Despite of the shown error I still can send the query to the server. Which responds with the same error I got with Postman and your Demo app:

Any idea about this error? Thanks.

@majorimi  I see, in this particular demo application i also renamed the eq operations to equals

Sidenote. in graphql you do not need the ,

If you want to give it a try:
{
  persons(where: { name: { equals: "FRed" } }) {
    id
    name
  }
}

@PascalSenn Thanks for your response. Yes it worked with "equals". So operation name was overwritten. Get it.
However the problem is now every filtering stopped working except the "equals".
For example used to "contains" worked like this:
{
  persons(where: { { name_contains: "ed" } }) {
    id
    name
  }
}

Now we get error message: "message": "The specified input object field firstName_contains does not exist.",
Also trying to query NON string values does not work. Before adding the equal filter we could query ENUM values like this (type is an Enum):
{
  tags(where: { type: "ACTIVITY" }) {
    id
    name
    type
  }
}

Now Enum query returns error: "message": "The specified value type of field type does not match the field type.",
Tried to use it with the same "equals" syntax:
  tags(where: { type: {equals:  "ACTIVITY" }}) { ... }
Still getting error:"message": "The specified value type of field equals does not match the field type.",
It seems adding the QueryableStringInvariantEqualsHandler to the filter pipeline override all the existing filters. I was trying to add descriptor.AddDefaults(); and .AddDefaultFieldHandlers() etc. Also was trying to change the handler to override all kinds of operations by using: protected override int Operation => DefaultFilterOperations.All/.Any;. Not much luck...
So the question is the a way to use QueryableStringInvariantEqualsHandler and meanwhile have all the existing filter like "contains" and enum compare work as before?
Thansk.

This example uses the new filtering api:
https://chillicream.com/blog/2020/11/18/new-filtering-api


Note about environment availability is confusing/contradictory

Code of Conduct

[X] I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?
https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#about-environments
What part(s) of the article would you like to see updated?
https://github.com/github/docs/blob/ef82ef29daa7c29b121349444c4e1129cce11600/content/actions/deployment/targeting-different-environments/using-environments-for-deployment.md?plain=1#L28-L30

Note: You can only configure environments for public repositories. If you convert a repository from public to private, any configured protection rules or environment secrets will be ignored, and you will not be able to configure any environments. If you convert your repository back to public, you will have access to any previously configured protection rules and environment secrets.
Organizations with {% data variables.product.prodname_team %} and users with {% data variables.product.prodname_pro %} can configure environments for private repositories. For more information, see "{% data variables.product.prodname_dotcom %}'s products."

The first "paragraph" here says "You can only ... for public repositories". The second paragraph says "oh, actually, some organizations can do it for private repositories".
The text should be unified to list the kinds of repositories that can do it in a single sentence, or to not use only for a sentence that only covers a subset of the repositories that support the feature. Only after the kinds of repositories that can use the feature are listed should the effects of transitioning a repository to a kind that can't be listed.
Additional information
@JacobWeisenburger asked about environments in https://github.com/github/docs/discussions/22691

@jsoref Thanks so much for opening an issue! I'll triage this for the team to take a look :eyes:

@cmwilson21: I was on vacation last week ...

@cmwilson21 ?

Thanks for opening this @jsoref, I definitely see how the way this note is phrased is confusing. You or anyone else is free to open a PR that updates the note to use clearer language, such as:

Note: Users with {% data variables.product.prodname_free_user %} plans can only configure environments for public repositories. If you convert a repository from public to private, any configured protection rules or environment secrets will be ignored, and you will not be able to configure any environments. If you convert your repository back to public, you will have access to any previously configured protection rules and environment secrets.
Organizations with {% data variables.product.prodname_team %} and users with {% data variables.product.prodname_pro %} can configure environments for private repositories. For more information, see "{% data variables.product.prodname_dotcom %}'s products."


Update linked_list_cycle.cpp

Update solution  linked_list_cycle.cpp with time complexicity O(N).

please merge my pull request.


Make operator watch Node objects and act accordingly

That should work way more efficient from the current implementation, where we list all nodes on the cluster several times, iterate over all of them etc. It should also make tests much easier to struct.

This would also solve #140.


Minor improvement for improved readability

Q
A

content fix?
yes

New article?
no

Related issues?
no

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/spfx/known-issues-and-common-questions.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

thx Samuel, all fixes are welcome and a clear improvement in the readability. Thx.


Misleading or erroneous Access Policy documentation - clarity required.

I'm trying to follow the documentation https://www.medplum.com/docs/auth/open-patient-registration to add a default patient access policy to my project.
The documentation says 'Navigate to https://app.medplum.com/Project and select your project. In the "Edit" tab, set the "Default Patient Access Policy" field to your new policy and click "Save".'
However, there is no "Save" button.  Further when I add the example policy from https://www.medplum.com/docs/access/access-policies#patient-access and click the "Update" button, I see a "Success" message in the lower right.
Yet there is no visible change to the project.  The policy does not exist in any of the tabs at the top.
Perhaps there is some prerequisite that I missed?
Thanks in advance for any clarity.

@dooleydiligent thanks for bringing this to our attention. We've updated our docs to reflect that the button is named "Update" vs. "Save"
https://www.medplum.com/docs/auth/open-patient-registration

Yet there is no visible change to the project. The policy does not exist in any of the tabs at the top.

We don't expect a visible change to the project immediately after setting. Rather, a user logging in as a Patient should see a limited set of resources