guidance on handling the display of io::Error with file paths

Sorry for the length of this, but I think there are some aspects of this that I don't understand, so it's difficult for me to cut to the chase.
It is very well possible that I'm attacking this from the wrong angle, but the high level problem I'd like to solve is this: when I show errors whose underlying cause is an io::Error, it is very typically paired with a file path. For end users, both pieces of information are crucial: I need to show both the file path and the I/O error itself (which might be "not found" or "no permission" or whatever). Moreover, I'd like show it in the following format, which I feel is fairly standard:
path/to/foo: No such file or directory (os error 2)

As an application writer, the most natural-seeming path to take here was to use contexts. This is even highlighted as a prototypical example from the failure book:
let mut file = File::open(cargo_toml_path).context("Missing Cargo.toml")?;
file.read_to_end(&buffer).context("Could not read Cargo.toml")?;

But I can't actually use this approach, because it turns out that the the error messages emitted here are 1) fixed and 2) therefore might be wrong. That is, this might say that Cargo.toml is missing when the actual problem is that there was a permission error or something else entirely. An easy way to solve this problem is to surface the error message from the underlying I/O error. Being an inexperienced user of failure, I tried this approach:
let file = File::open(path).context(path.display().to_string())?;

If I had read the docs more carefully, I would have known that this indeed only prints the file path itself, and leaves out the underlying error. #182 seems to propose a fix to this, but I wonder whether the result of that would be this (the inverted format of my ideal error message):
No such file or directory (os error 2): path/to/foo

My next approach here was to just embed the io::Error's display string into my context:
let file = File::open(path).with_context(|e| format!("{}: {}", path.display(), e)?;

This gives me the output I want. As an application writer, I made this a bit easier by adding my own ResultExt trait with a blanket impl:
pub trait ResultExt<T, E>: failure::ResultExt<T, E> where E: fmt::Display {
    fn with_path<P: AsRef<Path>>(
        self,
        path: P,
    ) -> result::Result<T, failure::Context<String>>
    where Self: Sized
    {
        self.with_context(|e| format!("{}: {}", path.as_ref().display(), e))
    }
}

impl<T, E: fmt::Display>
    ResultExt<T, E> for result::Result<T, E>
where result::Result<T, E>: failure::ResultExt<T, E> {}

So that now I can just write
let file = File::open(path).with_path(path)?;

This was a somewhat annoying process, and being nudged to define my own trait here is slightly unfortunate. Regardless, as an application writer, I'm not too unhappy.

Let's switch gears. Instead of playing the role of an application writer, now let's talk about what I'm supposed to do as a library writer. To give away the ending here: I had to abandon derive(Fail)'s support for deriving Display and ignore some pieces of advice from the failure book. The end result is still better than the status quo with std::error::Error because I get causes/backtraces, but it took a quite a bit of will power/thinking to get here because I basically had to discard everything nice about failure and go out of my way to ignore your advice. :-)
OK, so, as a library writer, I really want to provide structured errors to my consumers. I do not want to use failure::Error because I want my errors to have types and be documented. I might be able to be convinced otherwise on this point, but I'm 99% sure this is non-negotiable. (For applications, I <3 failure::Error.)
Since I knew I wanted a structured error, I jumped right to the section on "An Error and ErrorKind pair." (I did look over the page on "A Custom Fail type," but I felt like the distinction between that page and "An Error and ErrorKind pair" wasn't especially clear to me. They almost look one in the same to me, at least, from the perspective of my particular problem.)
After reading "An Error and ErrorKind pair," I came up with this, which I figured would be wrong based on my experience above with contexts:
#[derive(Debug)]
pub struct Error {
    inner: Context<ErrorKind>,
}

#[derive(Debug, Fail)]
pub enum ErrorKind {
    #[fail(display = "{:?}", _0)]
    Path(PathBuf),
}

I also added the various Fail, Display, and From impls as suggested in your docs. Given my stumbling above with errors in my application, the problem here should be clear: the underlying io::Error won't get surfaced to the end user because the context swallows it up. (And if fixing #182 does permit it to be surfaced, then it seems likely it will be in the "wrong" format anyway.) At this point, it became clear to be that I needed to embed an io::Error into my ErrorKind enum. This was a difficult leap to make, because it seems strongly recommended against doing this in the section, "What should your ErrorKind contain?" In particular, it seems to specifically imply that the io::Error values should get swallowed into the context, and the ErrorKind should be responsible for specifying what went wrong from the perspective of the library.
Another problem with this approach is that I can't really use derive(Fail)'s Display auto-derive here because I can't use the Display impl for paths, which requires calling the display method. I believe an issue for that is filed already for this (#183, https://github.com/withoutboats/display_derive/issues/3).
So given that I 1) seemingly need to embed my own io::Error and 2) need to write my own Display impl, I finally landed with this:
#[derive(Debug)]
pub struct Error {
    inner: Context<ErrorKind>,
}

#[derive(Debug, Fail)]
pub enum ErrorKind {
    Io {
        #[cause]
        err: io::Error,
        path: PathBuf,
    },
}

Which is basically what I'd do today using std::error::Error, with of course the added benefit of implementing Fail, which gets me proper cause support, and of course, backtraces. Those are nice benefits!
My question is: is the above the ideal way to go about solving problem with failure? Is there a more idiomatic approach? It feels somewhat like I'm eschewing the happy path of failure here by embedding errors coming from dependencies (std in this case) into my own error types, instead of letting them become "causes" more naturally (at least, this is the philosophical impression I got from your book). In particular, one obvious downside from this approach is that this does seem to break my natural expectation of what causes would return here. Namely, if I iterate over all of the causes, then I get exactly one: my Error value. But since I've marked the io::Error as cause, I'd expect to get that too. In particular, that permits me to write something like this in my main function:
fn main() {
    if let Err(err) = try_main() {
        if let Some(ioerr) = err.root_cause().downcast_ref::<io::Error>() {
            if ioerr.kind() == io::ErrorKind::BrokenPipe {
                // broken pipe means our consume hung up, quit gracefully
                process:exit(0);
            }
        }
        eprintln!("{}", err);
        process::exit(1);
    }
}

It's clear to me from looking at the implementation of failure why this doesn't work. In particular, Error's cause method is implemented by calling Context's cause method, which in turn defers to its internal failure's cause method. Since I'm using Context::new, failure is a Either::This(Backtrace::new()), which has no cause. What isn't clear to me is why this is the implementation. My suspicion is that I've committed a category error, and that I probably shouldn't be using Context at all, and perhaps I should just create my own Backtrace.
Alternatively, I can eschew your recommended implementation of impl Fail for Error and write cause by skipping over the context completely:
impl Fail for Error {
    fn cause(&self) -> Option<&Fail> {
        self.inner.get_context().cause()
    }
}

This does indeed work as expected, but it is now abundantly clear that I should jettison Context altogether and just embed a backtrace directly into my error. The downside of this is that I (think) I now miss out on failure's conditional handling of backtraces with respect to no_std?
The final code I wind up with is this:
#[derive(Debug)]
pub struct Error {
    kind: ErrorKind,
    backtrace: Backtrace,
}

#[derive(Debug, Fail)]
pub enum ErrorKind {
    Io {
        #[cause]
        err: io::Error,
        path: PathBuf,
    },
}

impl Fail for Error {
    fn cause(&self) -> Option<&Fail> {
        self.kind.cause()
    }

    fn backtrace(&self) -> Option<&Backtrace> {
        Some(&self.backtrace)
    }
}

impl fmt::Display for Error {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        self.kind.fmt(f)
    }
}

And now I've finally wound up where I want to be, but I am completely and utterly uncertain about whether this is a good place to be. In particular, this particular type of error reporting is something I do very frequently, and I expect others will as well. That's why I've taken the time to write this issue, because I see this as an important problem to solve.

Let me pop up a level and add some meta discussion just to make sure my opinions/intent on this is clear:

I don't necessarily believe my final destination is bad. For the most part, I've been of the opinion that error handling using just std::error::Error is Pretty Much Great Except for Causes and Backtraces, so needing to write out what is in my mind write-once but somewhat boiler-platey code is basically a non-issue for me personally. (I recognize that this is absolutely not true in the wider ecosystem, and that many people will go to great lengths to eliminate every ounce of boiler plate as possible. :-))
My intent in filing this issue is to both give feedback on my own personal learning process, and to receive feedback on whether my proposed solution to my problem is the expected solution. If it is, I think the only real action item here would be documentation (from my perspective). If it isn't, then I'd very much appreciate being pointed in the right direction, because I definitely do not want to repeat this pattern over and over if it is not something that is intended to be supported well by failure.
It seems plausible that this experience has soured me on the use of Context inside a library, but perhaps it is unique to this specific problem. In particular, it seems like the issue I'm running into here is that the user facing error message requires the composition of two bits of information, but it seems like failure's design and documentation push toward a more hierarchical approach where errors don't really know about each other, and thus, can't be formatted holistically. And this is perhaps in turn a consequence of the fact that a bare io::Error is rarely a sufficient user displayable error on its own when dealing with file I/O, but does tend to be necessary in my experience.

After sleeping on this, I realized that a simpler solution might exist. Namely, I could go back to my types looking like this:
#[derive(Debug)]
pub struct Error {
    ctx: Context<ErrorKind>,
}

#[derive(Debug, Fail)]
pub enum ErrorKind {
    Path(PathBuf),
}

But in my application, I can display the error message by iterating over the causes and assembling the message. I got the idea from the #165 PR, which basically does exactly that.
I'm still a tad uneasy with this, and I would still appreciate advice on this entire affair. In particular: relying on causes to format the error message feels a little funny, almost too implicit, but I can't quite convince myself that it's much different than what I'd do more explicitly/manually. I think my brain just needs to adapt to it.

One other short-coming I can see is that this approach (I think) is that it makes it impossible or hard to pull out the original io::Error. (It does seem possible if you're determined, by taking advantage of causes/downcast_mut/mem::replace.) The use cases for that are hard to articulate though (I've certainly never needed it. I only ever need to inspect the io::Error.)

After sleeping on this, I realized that a simpler solution might exist. Namely, I could go back to my types looking like this:

I had just gotten to this point in my library, but for some reason it took me the longest time to do

But in my application, I can display the error message by iterating over the causes and assembling the message.

even after reading this whole issue a few times. For some reason I expected failure to do that automatically. Now that I've actually implemented it myself, though, that's probably an unreasonable expectation. There are many different ways it could be done (and probably specific to one's own Fail types), and I can imagine someone might not want such a thing to happen in their code.
I think the documentation ought to nudge users in the direction of putting causes in their Display impls, but then that makes deriving Display for Fail types much less useful.

For some reason I expected failure to do that automatically. Now that I've actually implemented it myself, though, that's probably an unreasonable expectation.

I suspect so as well, but on the flip side, I will literally need to write this code for every application that I write. I'll even want it for small one-offs. I suspect there is some middleground between "that's what failure does automatically" and "failure can be told to do it." :-)

I keep running into this. In my case I actually want to use the error kind pattern and use the kind to say what type of error I have and that wraps via ResultExt a inner io::Error. Sadly this now means I cannot attach more information into the outer error.
What I wanted is something like this:
#[derive(Debug)]
pub struct ConfigError {
    filename: PathBuf,
    inner: Context<ConfigErrorKind>,
}

impl Fail for ConfigError {
    fn cause(&self) -> Option<&Fail> {
        self.inner.cause()
    }

    fn backtrace(&self) -> Option<&Backtrace> {
        self.inner.backtrace()
    }
}

impl fmt::Display for ConfigError {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        write!(f, "{} (file {})", self.inner, self.filename().display())
    }
}

impl ConfigError {
    /// Creates a new config error
    pub fn new<P: AsRef<Path>>(p: P, kind: ConfigErrorKind) -> ConfigError {
        ConfigError {
            filename: p.as_ref().to_path_buf(),
            inner: Context::new(kind),
        }
    }

    /// Returns the filename that failed.
    pub fn filename(&self) -> &Path {
        &self.filename
    }

    /// Returns the error kind of the error.
    pub fn kind(&self) -> ConfigErrorKind {
        *self.inner.get_context()
    }
}

/// Indicates config related errors.
#[derive(Fail, Debug)]
pub enum ConfigErrorKind {
    /// Failed to open the file.
    #[fail(display = "could not open config file")]
    CouldNotOpenFile,
    /// Failed to save a file.
    #[fail(display = "could not write config file")]
    CouldNotWriteFile,
    /// Parsing a YAML error failed.
    #[fail(display = "could not parse yaml config file")]
    BadYaml,
}

The issue here now however is that I do not know how to actually construct the ConfigError with context()/with_context(). As far as I can tell there is no way other than to first construct the error with context and then to provide a second method that attaches the location info.

To clarify this is what I do at the moment:
macro_rules! ctry {
    ($expr:expr, $kind:expr, $path:expr) => {
        match $expr {
            Ok(val) => val,
            Err(err) => return Err(ConfigError::new($path, err.context($kind)))
        }
    }
}

And then accept the Context directly in the constructor of the config error.

A related problem I have is when I have
enum ErrorKind {
    Variant(PathBuf)
}

and I want to display the path, I really want to do #[fail(display = "text  {}", _0.display())], but for the moment I have to use the debug formatter.

I'm trying to figure this out, and the thing that worked the best for me wound up being a custom type implementing Fail that contains a generic error and a generic location, along with a convenience method on Result<T, E> that iff is_err annotates with a location.
Leaving out some of the details, it looks like this:
/// Location that an `Error` can occur
pub trait Location: Debug + Send + Sync + 'static {
  /// Format the location and error for display
  fn fmt_error(&self, f: &mut Formatter, error: &dyn Fail) -> fmt::Result;
}

// `Fail` is implemented for `Error` elsewhere
#[derive(Debug)]
pub struct Error<E: Fail, L: Location> {
  /// Error
  pub error: E,
  /// Location at which `error` occurred
  pub location: L,
}

impl<E: Fail, L: Location> Display for Error<E, L> {
  fn fmt(&self, f: &mut Formatter) -> fmt::Result {
    // defer to location so users can implement custom formatting
    self.location.fmt_error(f, &self.error)
  }
}

pub trait ErrAt<T, E: Fail> {
  /// Iff self.is_err() annotate with location
  fn err_at<L: Location, I: Into<L>>(self, location: I) -> Result<T, Error<E, L>>;
}

I wonder if this is a reasonable approach, or if I'm missing a more idiomatic way to do this.


Add explicit Java temporal types

can you rebase the PR ? i've squashed a couple of commits in master

Yes I will do it now

@vietj :+1: Done

please rebase without merge, there is a Foo commit that disappeared

Thanks


Playlist take long to play if there are inaccessible videos

What went wrong?
Playlist take long to play if there are inaccessible videos and the songs don't start until check the status of all the videos in the playlist
How do we reproduce this?
1_start the bot
2_+p playlist https://www.youtube.com/playlist?list=PLg0uzHBxKbxEJlb8YNvt0QBirjEg4-A_V
(example)
3_wait until check all the playlist and take too long, maybe add the playlist and check later when the song is blocked skip it.
Which version of the bot are you using?
release-260419
Which operating system are you using?
Windows 7 or higher
Just checking...

[X] I have confirmed that my issue has not already been reported.
[X] I am using Python 3.8 or higher.
[X] I have updated my Python dependencies.

+shuffle make to check again all the videos

Hi there 👋 thanks for opening an issue, would you mind sharing the branch you are using? I suspect you are using the master branch and I would encourage you to use the dev branch which has some overhauls to the way music is processed.

yes i follow the guide, im gonna check the dev branch 🫡

If you need any assistance switching feel free to ask here or on our support server

works really good 👌thanks for the info


getProduct parameter type

Hi,
I found  a discrepancy in the id type.
Is getProduct({ id }) string or number? README doc says it's a number but the source says it's a string.

Hi,
This has been fixed with the v2.0.0. Please see: https://github.com/lmsqueezy/lemonsqueezy.js/wiki/Products#getproduct


To update plugin or add again?

I'm the new maintainer of the Lost Grid–thus it has a new url. Should I submit a new plugin or is there a way to update existing plugins?
Thanks!

Hey! Sorry for the late response! You should update the existing one instead of adding another!
Steps:

fork the repo
edit these lines
Commit, push, and then open a pull request

Thanks!


Make threadpool size configurable in publisher/consumer builder

Currently it is very low level to override the default behaviour on how many threads are assigned to the publisher/consumer.
We are seeing too many GAX threads created for our usecase causing an overhead in memory consumption on k8s for low traffic situations.
There is some talk about it her
The documentation for override the pool size might be a bit to low leven for many situations.
It would be good if the pool would be configurable from outside especially since many users of pubsub are using a framework like spring-boot pubsub, or in our case apache camel google pubsub.

Thanks for raising this issue!

Currently it is very low level to override the default behaviour on how many threads are assigned to the publisher/consumer.

Can you explain this part a bit more? I'm a bit confused what you mean by very low level. I took a look at the pubsub docs and I think there is a setting inside pubsub to configure the executor provider:
     ExecutorProvider executorProvider =
          InstantiatingExecutorProvider.newBuilder().setExecutorThreadCount(4).build();

      // `setExecutorProvider` configures an executor for the publisher.
      publisher = Publisher.newBuilder(topicName).setExecutorProvider(executorProvider).build();

It would be good if the pool would be configurable from outside

I'm also a bit confused by this part. What do you mean by configurable from outside?

I think the internal threading model in pubsub might benefit from a better abstraction as eg. a configuration class, where users can override the default behaviour of pubsub (number of threads in pool asf.) , and then inject an instance of that when instantiating pubsub, or a method on the Publisher/Consumer to set no of threads.
In this way users do not have to deal with executors if they want to override the defaults.

For an abstraction for pubsub, I think this would probably better suited as a feature request for the pubsub folks. I'll transfer the issue there and CC the some of the devs.


Support for Panorama 'shared' objects

Feature request to add support for 'shared' objects in panorama.
I guess this is annoying because these objects are located at /config/shared. My initial thought is to add a special refresh method in panorama.Panorama much like the way the predefined module works, but I'm not sure on where to store objects as we need read/write access and the xpath for these is not like the normal objects.
That being said, I see this as an obvious must for #26

Given @btorresgil's ideas for #44 I no longer see the need for this. The Container Idea is much more robust and overall future proof all while implementing the same ends.


Error in bot startup

Hello , is experiencing an error when booting the bot, do not know what to do, please help me .
The following image:

Please ask this question in the #help section on slack. This is not an issue with the bot.


macOS: Memory usage skyrockets when window is occluded

Bevy version
0.8.1
[Optional] Relevant system information
MacBook Pro (16-inch, 2021)
`AdapterInfo { name: "Apple M1 Max", vendor: 0, device: 0, device_type: IntegratedGpu, backend: Metal }`

What you did

Cloned out the repository.
Checked out the v0.8.1 tag.
Tried the shader-instancing example locally on release mode: cargo run --release --example shader_instancing.

What went wrong
When occluding or minimizing the Bevy window, memory usages skyrockets at gigabytes per second until the window is shown again.
Additional information
The memory usage is caused by https://github.com/gfx-rs/wgpu/issues/1783. The following information stems from a long and somewhat redundant investigation on this issue, plus a bunch of other macOS-related issues (discord thread).
It was concluded that this is impractical to solve entirely at the wgpu level, and that there are two primary workarounds for this issue:

Detecting window occlusion using winit's WindowEvent::Occluded and avoiding any rendering at all when the window is occluded.
+ Probably easier to plumb into an existing codebase.

/ WindowEvent::Occluded was introduced on winit 0.27.0.
/ Care must be taken to throtte the winit loop (or put it on ControlLoop::Wait) when
  the window is occluded, (and/or drive the app world through a different means?),
  since without wgpu surface obtention throttling it at vsync, it would just collapse into
  a tight loop, causing massive CPU usage.

- Only mitigates the memory leak; on optimized builds it will still leak around 100 Mb
  when un-occluding the window.

Driving rendering through CVDisplayLink (macOS) / CADisplayLink (iOS) frame callbacks (both essentially do kind of the same).
Preliminary tests using the display-link 0.2 crate on raw wgpu examples were successful, but more interesting stuff is exposed on the raw APIs themselves (documentation). Apple itself explains how it works in this WWDC 2021 talk.
It is noted exploring such integration for Bevy is already being studied as part of https://github.com/bevyengine/bevy/issues/5713.
+ Fixes the memory leak.
+ If used as a way to drive winit redraw requests, can potentially ensure it keeps
  looping at target framerate even if the window is in the background.
+ Automatically handles system conditions (e.g. can throttle the application down
  automatically when thermals are high)

/ Likely the Apple recommended way to do it, given I have not seen any official Metal
  examples that *don't* use it.
/ Callback comes with nanosecond-precision timestamp(-s), could be used to derive
  delta timestamps?
/ Can be configured to target a specified framerate as long as it's divisible by vsync.

- Essentially forces Vsync.
- Likely very annoying to plumb on an existing codebase?

As part of filling information on the related issue on https://github.com/sotrh/learn-wgpu/issues/207#issuecomment-1235400560, I found out that approach 1 is not perfect when compiling with --release due to the heavy timing-dependent nature of the problem. On my testing with learn-wgpu, while it won't leak uncontrollably anymore, it will still leak around 100 Mb when un-occluding the window.


run-vcpkg failing with "dyld: Symbol not found: __ZNKSt3_V214error_category10_M_messageB5cxx11Ei"

I know I'm using old stuff. I inherited all of this and it was working perfectly until about 3 weeks ago and now out of the blue it stops working. I even tried it with old commits that had worked successfully in the past yet they are failing now too. To be clear, I'm using<EMAIL_ADDRESS>Is there any reason that this would stop working out of the blue? I would prefer to be able to fix it as it is but I would also be open to upgrading if that is what needed to be done, although I'm not sure what that would entail. I am a new C++ developer and I am still learning the basics of these packages. Please forgive my ignorance and thank you for any help in advance!

I forgot to mention that this is only happening on a macos-11 runner. My windows runner is working just fine regarding run-vcpkg.

@mrheffern thanks for reporting this! I think the issue is that run-vcpkg@v5 did not account for the version of the underlying OS in the computation of the cache key, and the old cached binaries built with previous macOS based image are being reused on a newer image and fail to run (they should be rebuilt instead, but run-vcpkg is not doing that in version 5).
I'd suggest to upgrade to run-vcpkg@v7 and then it should just be working with the same set of inputs. Or you could use a more recent run-vcpkg@v11, but this requires a recent vcpkg version  and also you need to switch to vcpkg's manifest mode (i.e., vcpkg.json).
Take a look at this C++ project template and you should get it working again.

Wow! That completely fixed it! I can't fully express how grateful I am to you. Thank you lukka!!! You totally saved me.
Any reason this would have stopped working in the last 3 weeks specifically, with no changes to my codebase? While I don't fully understand the implications of everything you pointed out, they didn't sound like issues that would have occured in the last three weeks.
Again, thank you lukka!

@mrheffern good question, the reason is likely that the binaries built long time ago on an older version of the macOS runner, now cannot run on a more recent macOS runner, and those needs to be rebuilt, which the newer action did automatically.
If it is working, that's good to know, thanks for the kind message! :)


Adjust height styles on form in SelectPanel

Context
This PR aims to fix a bug where the <dialog> with fit-content is not displayed properly in Safari. Paired with @keithamus for this fix!
Changelog

Changed

Adjusts styling of the form that lives within SelectPanel. This ensures that the height of the form will at minimum, be 100% of the dialog's height.

Adds height: initial and min-height: 100% to <form>
Adds story showcasing height: fit-content on <dialog>

Rollout strategy

[x] Patch release
[ ] Minor release
[ ] Major release; if selected, include a written rollout or migration plan
[ ] None; if selected, include a brief description as to why

Testing & Reviewing

Merge checklist

[ ] Added/updated tests
[ ] Added/updated documentation
[x] Added/updated previews (Storybook)
[ ] Changes are SSR compatible
[x] Tested in Chrome
[ ] Tested in Firefox
[x] Tested in Safari
[ ] Tested in Edge
[ ] (GitHub staff only) Integration tests pass at github/github (Learn more about how to run integration tests)

Hi!
I don't think this is the right fix either 😅
Setting min-height:100% on form instead of max-height, means the form can go well beyond the dialog, pushing the footer out


-watch flag doesn't work when using Vault backend

I'm using Vault as my backend.  I have everything setup and it seems to work fine with the onetime flag.  When I switch it to -watch, it starts but just sits there...   I've let it sit for an hour and I've made changes to the key in the Vault.  As I said, it works fine with the onetime flag so I'm sure the template and config are not a problem.  I'm running Confd on MacOS Sierra and the Vault server is a remote server.
Here is my command and the output:
confd -watch -backend vault -interval 5 -log-level debug -node http://xxx.xxx.xxx.xxx:8200 -auth-type token -auth-token xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
2017-01-09T17:30:33-07:00 dans-mbp.ad.mycompany.com confd[9289]: INFO Backend set to vault
2017-01-09T17:30:33-07:00 dans-mbp.ad.mycompany.com confd[9289]: INFO Starting confd
2017-01-09T17:30:33-07:00 dans-mbp.ad.mycompany.com confd[9289]: INFO Backend nodes set to http://xxx.xxx.xxx.xxx:8200
2017-01-09T17:30:33-07:00 dans-mbp.ad.mycompany.com confd[9289]: INFO Vault authentication backend set to token
2017-01-09T17:30:33-07:00 dans-mbp.ad.mycompany.com confd[9289]: DEBUG Loading template resources from confdir /etc/confd
2017-01-09T17:30:33-07:00 dans-mbp.ad.mycompany.com confd[9289]: DEBUG Found template: /etc/confd/conf.d/application.conf.toml
2017-01-09T17:30:33-07:00 dans-mbp.ad.mycompany.com confd[9289]: DEBUG Loading template resource from /etc/confd/conf.d/application.conf.toml

@bikerdan  vault it self dont support this kind of watch operation, you should store your secrets in vault, and your observable settings on consul or etcd.
from the hashicorp docs

Please note that Vault does not support blocking queries. As a result, Consul Template will not immediately reload in the event a secret is changed as it does with Consul's key-value store. Consul Template will fetch a new secret at half the lease duration of the original secret. For example, most items in Vault's generic secret backend have a default 30 day lease. This means Consul Template will renew the secret every 15 days. As such, it is recommended that a smaller lease duration be used when generating the initial secret to force Consul Template to renew more often.

Does that really matter?  All that is needed is that Confd does a periodic check.  Some interval based frequency.  It doesn't have to be a real time update...

If you want poll looping use the -interval flag.

Read the original post.  I did use the interval flag and it doesn't work.  Is the problem that I used the watch flag and the interval flag?

Use one or the other. Can you point to which doc so we can update it?

When I mean by "use one or the other", the first is for poll looping, and -watch is for watching. The -watch flag supersedes -interval. Watch is not supported for vault as @criloz mentioned, but on other backends it's an either/or.

I don't see it in the documentation here.  I may have seen it somewhere else.  I will comment if I find it.  I appreciate the help and will give it a try without the -watch flag.  Thanks for your help!


Allow graphics for rooms

Allow users to upload graphics to use for rooms.

Let them spice up text adventuring 80's style


cannot pull connect-sidecar

docker pull ghcr.io/skip-mev/connect-sidecar
Using default tag: latest
Error response from daemon: manifest unknown
I would like to suggest that we provide multiarch images, we can do that with the platform build tag in the docker build jobs (probably)

We do provide multi-arch builds, but they take a long time to complete so we haven't enabled them for each commit that gets pushed to main. We only do multi-arch builds for releases, and we haven't done the full changeover from slinky=>connect yet in a full release. We're aiming to do that very soon w/ a v2 release which will provide a connect-sidecar image and a full version bump.

Thanks for explaining @Eric-Warehime

https://github.com/orgs/skip-mev/packages?repo_name=connect


Single-tenant static Alertmanager support

To configure an alerting rule in a local test environment:
curl -v -XPOST --socks5-hostname localhost:8000 --globoff -H "X-Scope-OrgID: 1" 'http://configs.default.svc.cluster.local/api/configs/org/cortex' -d '{"rules_files": {"test.rules": "ALERT SomethingIsUp IF up == 1"}}'

If I read this correctly, this will send alerts to all users to this alert manager?

LGTM

If I read this correctly, this will send alerts [for] all users to this alert manager?

Yep.


ConfigMap name missing UI validation

Setup

Rancher version: v2.7.5
Rancher UI Extensions:
Browser type & version: Chrome Version 115.0.5790.98 (Official Build) (arm64)

Describe the bug
When attempting to create a configmap that has a capital letter in the name, the request is made with no error checks and you get an api error page instead of an error on the UI (or the string being lowercased before the request is made).
To Reproduce

Provision a v2.7.5 Rancher instance
Log into the Rancher instance
Click into the local cluster from the home page
On the left side of the page, click Storage -> ConfigMaps
Click the Create button in the upper right of the page
In the Name field, enter at least one uppercase character and whatever else you want
Click the Create button on the lower right of the page

Result
Receive an API error
Expected Result
Receive a UI error (or the text is lowercased for the request)
Screenshots

Reproduced in 2.7.5, but in master (2.7q3) we get taken back to the config maps list. The request still fails. There's no error presented to user.
Tries the same with secrets, but the error is shown to the user.
Given the limited impact, i don't think this is a blocker, moving out of 2.7q3


Wrong IMU topic for EKF with UM6/7 IMUs?

Line https://github.com/husky/husky/blob/862fd9c7df6bf3b57f9895f94e59003bcc460426/husky_control/config/localization.yaml#L18 refers to topic imu/data which doesn't exist at least on our Husky A2 with UM6 IMU. We have topic imu/data_raw, however.
I see the imu/data_raw topic in configs for both UM6 and UM7, whereas the microstrain nodes seem to publish on imu/data. Do we do something wrong, or is just everybody using Husky with UM* IMU running EKF without the IMU?
Also, setting the two_d_mode by default seems like something really unexpected to me...

The data gets filtered using imu_filter_madgwick which subscribes in imu/data_raw and publishes imu/data: https://github.com/husky/husky/blob/kinetic-devel/husky_bringup/launch/um6_config/um6.launch#L23

Thanks, Tony. Our Husky went now for a trip home which will take some time, so I'll get back to you later when we receive it again. Or @rouceto1 might chime in.
When testing, I'm pretty sure that in rqt_graph, I saw the IMU filter publish to some topic which was not imu/data, and we wondered why nobody is subscribing the filtered IMU... So let's wait and I'll check then if we don't have some local modifications which would break the publishing chain...

Ok, I think I got it.
It seems the um6 driver publishes mag as the old Vector3Stamped type. This was handled by the use_magnetic_field_msg: false config in madgwick filter config. However, the filter stopped supporting this option some time ago: https://github.com/ccny-ros-pkg/imu_tools/blob/melodic/imu_filter_madgwick/CHANGELOG.rst#115-2017-05-24 . This means the filter subscribes the right topic but with wrong type, which is confirmed by my console:
[ERROR] [1568128900.059345260]: Client [/imu_manager2] wants topic /imu/mag to have datatype/md5sum [sensor_msgs/MagneticField/2f3b0b43eed0c9501de0fa3ff89a45aa], but our version has [geometry_msgs/Vector3Stamped/7b324c7325e683bf02a9b14b01090ec7]. Dropping connection.

And the madgwick filter hangs on waiting for a correct mag message.
Now the question is, what is the correct fix for this? Is there a node that would translate the old mag type to the new one?

@peci1  Are you using a um6 or um7?  Which ROS distro are you targeting?

@tonybaltovski we are using um6 with melodic distro.

This should fix your issue: https://github.com/ros-drivers/um6/tree/mag_msg
I will work on getting the merged soon.  Let me know if it works for you.

It would and I will for Melodic since I released it already.  However, like imu_filter_madgwick it should eventually follow the REP.

@peci1 I have updated to um6 and um7.  They have also been bloomed.  I will be bloom the Husky asap.  This should solve the issue but I will wait for you to confirm before I close it.

Closing due to inactivity.  Please feel free to re-open if issue persists.


Setup OCIO

Use /prod/softprod/libs/ocio/aces_1.2/config.ocio
Setup for:

[ ] maya
[ ] nuke

It seems that in maya, the "Rendering Space", "Display" and "View" in the settings are not recognized. What should we set for these?

We saw that there was a problem on maya, @friquette  fixed it in another issue.
I think he wanted you to validate the colorspaces settings.

The error is due to the fact that the :
"Rendering Space", "Display" and "View" name are based on a OCIO v2 config from maya and we are using an OCIO v1 config
We will switch to this  OCIO v2 config : /prod/softprod/libs/ocio/OpenColorIO-Config-ACES-1.0.0/studio-config-v1.0.0_aces-v1.3_ocio-v2.0.ocio

Yes, with OpenPype, we have to look inside the .ocio files and put the exact Display and View name.
I will try to do this this afternoon

Yeah it's written "will be deprecated" but it's not clear. Only Nuke OCIO Config is working for now

so the maya part is deprecated but the global ocio doesn't work. so ocio doesn't work for maya

and neither this one :


Added Chrome specific text to onboarding page

Tested on Chrome and Firefox on Windows

Tested on Chrome and Safari on Mac

LGTM 👍 ! Awesome work :)


Add Kemal Akkoyun as Member

Signed-off-by: Kemal Akkoyun<EMAIL_ADDRESS>🚀  https://groups.google.com/g/prometheus-developers/c/8_20fF_r_VE

Welcome! :tada:


chore(sqllab): Relocate get bootstrap data logic

SUMMARY
Part 1 of SQLLab SPA migration: detail link
This commit relocates the existing _get_sqllab_tabs to the utils in sqllab in order to share the same logic in the sqllab api v1 endpoint.
This commit also refactors the defaultDBId by common config parameters which is less relevant to sqlLab root state.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A
TESTING INSTRUCTIONS
Go to SQLLab and no errors shown
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

cc: @EugeneTorap

Hi @michael-s-molina @john-bodley @villebro! Can you help with this PR?

To fix the Python Misc CI error, you need to fix the order of the imports in these files:

superset/views/core.py
superset/tests/integration_tests/core_tests.py

To fix the import order, just run the pre-commit hook.

Can we merge it?


Metadata API: enforce role name uniqueness in delegations

The spec does not say anything about role name uniqueness in a delegations object but I believe we cannot safely allow multiple roles with same role name in the roles array of a delegations object. If we did then the roles could have different keyids, and then we would end up in a situation where a metadata may be both a valid delegation and an invalid delegation at the same time, depending on how the role gets chosen and that does not seem like the intention of the design.
I don't think there are real uses cases for non-unique role names in delegations either -- and I assume that this situation is just an unintentional side-effect of making roles an ordered list -- so the spec should possibly clarify that uniqueness is required (although I guess theoretically some complex ordering issue could be solved by multiple roles per role name).
Regardless, metadata API should IMO enforce role name uniqueness on Delegations.roles unless a good case is made against that.

Potentially we could use roles: OrderedDict[str: DelegatedRole] to store the roles (and raise in from_dict() if input contains duplicate role names)?

I would suggest discussing and solving this issue in the spec repo first so that we make sure we are on the same page with the spec.
If there is a use case where role name duplicates are needed (which I doubt) then I hope they will be shared there.
Otherwise, a good catch, and I agree with the sentiment that from my perspective I see only problems in allowing duplicate role names in delegations.

I don't think there are real uses cases for non-unique role names in delegations either -- and I assume that this situation is just an unintentional side-effect of making roles an ordered list -- so the spec should possibly clarify that uniqueness is required (although I guess theoretically some complex ordering issue could be solved by multiple roles per role name).

The same is implied by the spec itself it just needs to be written explicitly I guess:

In order to accommodate prioritized delegations, the "roles" key in the DELEGATIONS object above points to an array of delegated roles, rather than to a hash table.

I want to work on this and will assign myself.
If somebody else is already working on it, please let me know.


Prevent default not working when used with jQuery

When used with jQuery and not jQlite the view doesen't scroll at all.  The problem is ithat jQuery normalize the event object, so the flag inserted in allowTouchMoveDefaults is cleared by jQuery.
See the documentation at:
https://api.jquery.com/category/events/event-object/

Closing in favor of #351


PowerShell Exception: System.IO.IOException

Oops, something went wrong.  Please report this bug with the details below.
Report on GitHub: https://github.com/lzybkr/PSReadLine/issues/new
Last 6 Keys:
Ctrl+v Enter
UpArrow Enter
Enter
LeftArrow
Exception:
System.IO.IOException: The parameter is incorrect.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.Console.SetCursorPosition(Int32 left, Int32 top)
at Microsoft.PowerShell.PSConsoleReadLine.SetCursorPosition(Int32 cursor)
at Microsoft.PowerShell.PSConsoleReadLine.BackwardChar(Nullable1 key, Object arg)    at Microsoft.PowerShell.PSConsoleReadLine.ProcessOneKey(ConsoleKeyInfo key, Dictionary2 dispatchTable, Boolean ignoreIfNoAction, Object arg)
at Microsoft.PowerShell.PSConsoleReadLine.InputLoop()
at Microsoft.PowerShell.PSConsoleReadLine.ReadLine(Runspace runspace, EngineIntrinsics engineIntrinsics)

Most likely a dupe of #292, reopen with additional information if you disagree.


Stay on page

Is there a way I can stay on the current page until the new one has loaded? So the user would see the page is loading as nprogress shows up but the page would be ready when he gets there.
Thanks :)

This is not really an issue for meteor-auto-nprogress.
You can look into something like waitOn hook in iron-router
http://iron-meteor.github.io/iron-router/


python3Packages.meshtastic: 1.2.57 -> 1.2.58

Motivation for this change
https://github.com/meshtastic/Meshtastic-python/releases/tag/1.2.58
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 156152 run on x86_64-linux 1

2 packages built:

python310Packages.meshtastic
python39Packages.meshtastic


Where is concurrency lib?

Where is _concurrency lib :-)
See discussion section!
Happy Holidays!

I wasn't used "new concurrency" on Android yet. For example, in this project - https://github.com/vgorloff/swift-everywhere-samples.
I conducted few examples on macOS 12 / Xcode 13.1, but not yet used "new concurrency" in production or commercial apps.
Will try to use few "new concurrency"  examples on Android.

Thanks!
Happy Holidays !

On Dec 29, 2021, at 10:02 AM, Vlad Gorlov @.***> wrote:
I wasn't used "new concurrency" on Android yet. For example, in this project - https://github.com/vgorloff/swift-everywhere-samples https://github.com/vgorloff/swift-everywhere-samples.
I conducted few examples on macOS 12 / Xcode 13.1, but not yet used "new concurrency" in production or commercial apps.
Will try to use few "new concurrency" examples on Android.
—
Reply to this email directly, view it on GitHub https://github.com/vgorloff/swift-everywhere-toolchain/issues/138#issuecomment-1002664585, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5F3PMCVFAWMUERQZ2XWOTUTMWK5ANCNFSM5K57HZOQ.
You are receiving this because you authored the thread.

Oh, and we use the new concurrency on iOS/macOS/Linux and its good now for normal operations.

On Dec 29, 2021, at 1:15 PM, John Burkey @.***> wrote:
Thanks!
Happy Holidays !

On Dec 29, 2021, at 10:02 AM, Vlad Gorlov @.*** @.***>> wrote:
I wasn't used "new concurrency" on Android yet. For example, in this project - https://github.com/vgorloff/swift-everywhere-samples https://github.com/vgorloff/swift-everywhere-samples.
I conducted few examples on macOS 12 / Xcode 13.1, but not yet used "new concurrency" in production or commercial apps.
Will try to use few "new concurrency" examples on Android.
—
Reply to this email directly, view it on GitHub https://github.com/vgorloff/swift-everywhere-toolchain/issues/138#issuecomment-1002664585, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5F3PMCVFAWMUERQZ2XWOTUTMWK5ANCNFSM5K57HZOQ.
You are receiving this because you authored the thread.

Building Concurrency example (https://github.com/vgorloff/swift-everywhere-toolchain/commit/645e75d91bcb423a6b349678820bf3cf756a6ebb) gives me the following error:
{0}vova@MacBook-Pro:sample-package$ swift-build-arm-linux-androideabi
<unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found
<unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found
<unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found
/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/Tests/sample-package/Sources/SAConcurrency/SAConcurrencyMain.swift:18:7: error: cannot find 'Task' in scope
      Task {
      ^~~~

Seems the Concurrencyfeature wasn't build when building Android toolchain.

Building Concurrency example (https://github.com/vgorloff/swift-everywhere-toolchain/commit/645e75d91bcb423a6b349678820bf3cf756a6ebb) gives me the following error:
{0}vova@MacBook-Pro:sample-package$ swift-build-arm-linux-androideabi
<unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found
<unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found
<unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found
/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/Tests/sample-package/Sources/SAConcurrency/SAConcurrencyMain.swift:18:7: error: cannot find 'Task' in scope
      Task {
      ^~~~

Seems the Concurrencyfeature wasn't build when building Android toolchain.

The Concurrency feature wasn't enabled during build.
# File CMakeCache.txt
//Enable build of the Swift concurrency module
SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY:BOOL=OFF

The Concurrency feature wasn't enabled during build.
# File CMakeCache.txt
//Enable build of the Swift concurrency module
SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY:BOOL=OFF

There is a chicken-egg problem "dispatch needs stdlib" and, at the same time, "stdlib needs dispatch".
# File: stdlib/public/Concurrency/CMakeLists.txt

if(SWIFT_CONCURRENCY_USES_DISPATCH)
  if(NOT CMAKE_SYSTEM_NAME STREQUAL Darwin)
    include_directories(AFTER
                          ${SWIFT_PATH_TO_LIBDISPATCH_SOURCE})

    # FIXME: we can't rely on libdispatch having been built for the
    # target at this point in the process.  Currently, we're relying
    # on soft-linking.
    list(APPEND swift_concurrency_link_libraries
      dispatch)
  endif()
endif()

There is a chicken-egg problem "dispatch needs stdlib" and, at the same time, "stdlib needs dispatch".
# File: stdlib/public/Concurrency/CMakeLists.txt

if(SWIFT_CONCURRENCY_USES_DISPATCH)
  if(NOT CMAKE_SYSTEM_NAME STREQUAL Darwin)
    include_directories(AFTER
                          ${SWIFT_PATH_TO_LIBDISPATCH_SOURCE})

    # FIXME: we can't rely on libdispatch having been built for the
    # target at this point in the process.  Currently, we're relying
    # on soft-linking.
    list(APPEND swift_concurrency_link_libraries
      dispatch)
  endif()
endif()

To build Concurrency feature the Dispatch subproject needs to be build during stdlib build. It is configured via cmake/modules/Libdispatch.cmake.

To build Concurrency feature the Dispatch subproject needs to be build during stdlib build. It is configured via cmake/modules/Libdispatch.cmake.

Dispatch subproject fails to build due compiler flag -Werror :)

Dispatch subproject fails to build due compiler flag -Werror :)

:-)
Sent from my iPhone

On Jan 1, 2022, at 8:40 AM, Vlad Gorlov @.***> wrote:

Dispatch subproject fails to build due compiler flag -Werror :)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

:-)
Sent from my iPhone

On Jan 1, 2022, at 8:40 AM, Vlad Gorlov @.***> wrote:

Dispatch subproject fails to build due compiler flag -Werror :)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

does it build?

I will try to figure out in coming days how to pass arguments to external project (dispatch in our case). After quick search I didn't managed to find "who" is passing that -Werror flag. Maybe it is a CMake default.

Maybe ping compnerd on slack channel for swift on android ?    He’s awesome and very aquatinted with stuff like this as he’s the windows build champ on top of all of his other magic.

On Jan 5, 2022, at 2:50 AM, Vlad Gorlov @.***> wrote:

I will try to figure out in coming days how to pass arguments to external project (dispatch in our case). After quick search I didn't managed to find "who" is passing that -Werror flag. Maybe it is a CMake default.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

https://join.slack.com/t/swift-android/shared_invite/zt-11grcjtps-UHeM3etApuYdayxAKv436A

Sounds like they want to do a strategy discussion with you too -   Nice folks .

Building of "libdispatch" is not a problem. There is already builder for it (https://github.com/vgorloff/swift-everywhere-toolchain/blob/master/lib/Builders/DispatchBuilder.js). Issue that CMake is a functional-programming thing which consumes time to figure out needed options to pass :)

Compnerd knows the swift builds super well just thinking he could save you time ;-)

On Jan 5, 2022, at 4:14 PM, Vlad Gorlov @.***> wrote:

Building of "libdispatch" is not a problem. There is already builder for it (https://github.com/vgorloff/swift-everywhere-toolchain/blob/master/lib/Builders/DispatchBuilder.js). Issue that CMake is a functional-programming thing which consumes time to figure out needed options to pass :)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

Seems "libDispatch" itself defined compiler flags here: https://github.com/apple/swift-corelibs-libdispatch/blob/swift-5.5.2-RELEASE/cmake/modules/DispatchCompilerWarnings.cmake
Strange why I don't have this issue when building libdispatch standalone. Maybe because i am passing C/CXX flags explicitly.

The issue with -Werror is fixed by adding code below:
    if(CMAKE_HOST_SYSTEM_NAME STREQUAL Darwin)
      add_compile_options($<$<OR:$<COMPILE_LANGUAGE:C>,$<COMPILE_LANGUAGE:CXX>>:-Wno-unused-command-line-argument>)
    endif()

The next error seems due minSDKVersion 21 (https://stackoverflow.com/a/53217341/1418981):
/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Sources/swift-corelibs-libdispatch/src/event/event_epoll.c:460:33: error: use of undeclared identifier 'EPOLL_CLOEXEC'
        _dispatch_epfd = epoll_create1(EPOLL_CLOEXEC);
                                       ^
1 error generated.

Cmake uses SDK 16 by default:
-- ANDROID_PLATFORM not set. Defaulting to minimum supported version
16.

Passing -DANDROID_PLATFORM=${ANDROID_PLATFORM} in cmake/modules/Libdispatch.cmake helped to build libswift_Concurrency.so for armv7.
Next error when building for aarch64:
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang
-- Check for working C compiler: /Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang - broken
CMake Error at /usr/local/Cellar/cmake/3.22.0/share/cmake/Modules/CMakeTestCCompiler.cmake:69 (message):
  The C compiler

    "/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang"

  is not able to compile a simple test program.

Configure is not finished successfully :0

When building stdlib, I am passing manually the C flag -L/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/21. And then Cmake picks it and passes to the libdispatch subproject. Maybe I just need not to pass it.

Next error:
FAILED: lib/swift/android/aarch64/libswift_Concurrency.so
: && /Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang --target=aarch64-none-linux-android21 --gcc-toolchain=/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64 --sysroot=/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -fPIC -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security  -L/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/21 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wmissing-field-initializers -Wno-comment -fdiagnostics-color -O2  -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libgcc_real.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments     -shared -Wl,-soname,libswift_Concurrency.so -target aarch64-unknown-linux-android21 --sysroot=/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -B /Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64/aarch64-linux-android/bin -lm -fuse-ld=gold -shared  -o lib/swift/android/aarch64/libswift_Concurrency.so stdlib/public/Concurrency/ANDROID/aarch64/_Concurrency.o lib/swift/android/aarch64/swiftrt.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/__/CompatibilityOverride/CompatibilityOverride.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/Actor.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/AsyncLet.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/Error.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/LinkCompatibilityShims.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/Task.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/TaskAlloc.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/TaskStatus.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/TaskGroup.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/TaskLocal.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/ThreadSanitizer.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/Mutex.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/AsyncStream.cpp.o stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/linker-support/magic-symbols-for-install-name.c.o -L/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-host/llvm-project/./lib   -L/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-aarch64/swift-stdlib/./lib/swift/android/aarch64   -L/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-host/swift/bin/../lib/swift/android/aarch64   -L/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-host/swift/bin/../lib/swift/android   -L/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/swift   -L/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/../lib/gcc/aarch64-linux-android/4.9.x   -L/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Install/darwin-aarch64/icu/lib   -L/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-aarch64/swift-stdlib/libdispatch-android-aarch64-prefix/lib -ldispatch  -ldl  -llog  /Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++abi.a  /Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so  /Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Install/darwin-aarch64/icu/lib/libicui18nswift.so  /Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Install/darwin-aarch64/icu/lib/libicuucswift.so  lib/swift/android/aarch64/libswiftGlibc.so  lib/swift/android/aarch64/libswiftCore.so  -latomic -lm && :
/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64/aarch64-linux-android/bin/ld.gold: warning: skipping incompatible /Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-aarch64/swift-stdlib/./lib/swift/android/aarch64/libdispatch.so while searching for dispatch
/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64/aarch64-linux-android/bin/ld.gold: warning: skipping incompatible /Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-aarch64/swift-stdlib/libdispatch-android-aarch64-prefix/lib/libdispatch.so while searching for dispatch
/Volumes/Shared/Data/Android/sdk/ndk/21.4.7075529/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64/aarch64-linux-android/bin/ld.gold: error: cannot find -ldispatch
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalImpl(swift::Job*): error: undefined reference to '_dispatch_queue_attr_concurrent'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalImpl(swift::Job*): error: undefined reference to '_dispatch_queue_attr_concurrent'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalImpl(swift::Job*): error: undefined reference to 'dispatch_queue_attr_make_with_qos_class'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalImpl(swift::Job*): error: undefined reference to 'dispatch_queue_create'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalImpl(swift::Job*): error: undefined reference to 'dispatch_queue_set_width'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalImpl(swift::Job*): error: undefined reference to 'dispatch_release'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalWithDelayImpl(unsigned long long, swift::Job*): error: undefined reference to 'dispatch_time'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalWithDelayImpl(unsigned long long, swift::Job*): error: undefined reference to 'dispatch_after_f'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalWithDelayImpl(unsigned long long, swift::Job*): error: undefined reference to '_dispatch_queue_attr_concurrent'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalWithDelayImpl(unsigned long long, swift::Job*): error: undefined reference to '_dispatch_queue_attr_concurrent'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalWithDelayImpl(unsigned long long, swift::Job*): error: undefined reference to 'dispatch_queue_attr_make_with_qos_class'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalWithDelayImpl(unsigned long long, swift::Job*): error: undefined reference to 'dispatch_queue_create'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalWithDelayImpl(unsigned long long, swift::Job*): error: undefined reference to 'dispatch_queue_set_width'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueGlobalWithDelayImpl(unsigned long long, swift::Job*): error: undefined reference to 'dispatch_release'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueMainExecutor: error: undefined reference to '_dispatch_main_q'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueMainExecutor: error: undefined reference to '_dispatch_main_q'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueMainExecutorImpl(swift::Job*): error: undefined reference to '_dispatch_main_q'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function swift_task_enqueueMainExecutorImpl(swift::Job*): error: undefined reference to '_dispatch_main_q'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/GlobalExecutor.cpp.o:GlobalExecutor.cpp:function dispatchEnqueueDispatchAsync(dispatch_queue_s*, void*, unsigned int): error: undefined reference to 'dispatch_async_f'
stdlib/public/Concurrency/CMakeFiles/swift_Concurrency-android-aarch64.dir/Task.cpp.o:Task.cpp:function swift_task_asyncMainDrainQueueImpl(): error: undefined reference to 'dispatch_main'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Obviously because libdispatch for some reason was built for armv7 instead for aarch64 :)
{0}vova@MacBook-Pro:swift-everywhere-toolchain$ file /Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-aarch64/swift-stdlib/lib/swift/android/aarch64/libswiftCore.so
/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-aarch64/swift-stdlib/lib/swift/android/aarch64/libswiftCore.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=994cfd8863a2c520eb9e21ff8538133e63c478ca, with debug_info, not stripped
{0}vova@MacBook-Pro:swift-everywhere-toolchain$ file /Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-aarch64/swift-stdlib/lib/swift/android/aarch64/libdispatch.so
/Volumes/Shared/Git/MyProjects/swift-everywhere-toolchain/ToolChain/Build/darwin-aarch64/swift-stdlib/lib/swift/android/aarch64/libdispatch.so: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=b8dfcdb8a04a347ce81569ad7dcec4457acf3e8f, with debug_info, not stripped
{0}vova@MacBook-Pro:swift-everywhere-toolchain$

For some reason the option CMAKE_SYSTEM_PROCESSOR set as armv7-a in file /.../CMakeFiles/3.22.0/CMakeSystem.cmake for libdispatch subproject. At the same time stdlib itself has value for the same option set as aarch64. Which is correct one.

The above issue fixed by passing Cmake flag -DANDROID_ABI=${ANDROID_ABI} to the libdispatch subproject.

Ok. I am able to build stdlib with libswift_Concurrency.so for all 4 Android archs: arm 32/64 bit and x86 32/64 bit.
Now it is time to make clean rebuild of Swift, stdlib, libDispatch, libFoundation and then try Async/Await example again.

Yay!

On Jan 6, 2022, at 4:13 AM, Vlad Gorlov @.***> wrote:

Ok. I am able to build stdlib with libswift_Concurrency.so for all 4 Android archs: arm 32/64 bit and x86 32/64 bit.
Now it is time to make clean rebuild of Swift, stdlib, libDispatch, libFoundation and then try Async/Await example again.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

But sample code which works on macOS crashes with Segmentation fault on Android :0

Getting closer!

On Jan 6, 2022, at 8:55 AM, Vlad Gorlov @.***> wrote:
But sample code which works on macOS crashes with Segmentation fault on Android :0
—
Reply to this email directly, view it on GitHub https://github.com/vgorloff/swift-everywhere-toolchain/issues/138#issuecomment-1006654303, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5F3PIP2MVFZVD6ODCD4J3UUWUONANCNFSM5K57HZOQ.
You are receiving this because you authored the thread.

I will try one idea, but if it won't work, then I will come back to troubleshooting after next major release of Swift - Swift 5.5.3 or Swift 5.6.0.
The Swift Concurrency is experimental thing.
Even Cmake option, which is used to activate this feature, defined as following -D SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY=TRUE/FALSE

I think that flag is the old flag when it was experimental - i haven’t had any issues with the feature since they fixed the bugs just post 5.5.
I’m sure compnerd has good advice on this - find him in the slack channel ?

On Jan 6, 2022, at 9:11 AM, Vlad Gorlov @.***> wrote:

I will try one idea, but if it won't work, then I will come back to troubleshooting after next major release of Swift - Swift 5.5.3 or Swift 5.6.0.
The Swift Concurrency is experimental thing.
Even Cmake option, which is used to activate this feature, defined as following -D SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY=TRUE/FALSE
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

The release which builds concurrency is ready https://github.com/vgorloff/swift-everywhere-toolchain/releases/tag/1.0.78
But on runtime new Swift concurrency seems will alway crash.

Got symbols ?

Not too much: <anonymous:f1d90000> instead of <unknown> :)
01-06 20:29:25.735  5762  5762 F DEBUG   : backtrace:
01-06 20:29:25.735  5762  5762 F DEBUG   :     #00 pc 000fb970  <anonymous:f1d90000>
01-06 20:29:25.735  5762  5762 F DEBUG   :     #01 pc 000834c7  /data/local/tmp/sample-Exe/libswift_Concurrency.so
01-06 20:29:25.735  5762  5762 F DEBUG   :     #02 pc 219021ac  <unknown>

The Concurrency actually working with latest toolchain, but for macOS and Android it works differently.
public class SAConcurrencyMain {

   private var task: Task<Void, Error>?

   public init() {
      print("[SAC] Concurrency: Init")
   }

   public func execute(completion: @escaping () -> Void) {
      print("[SAC] Concurrency: Start")
#if !false
      task = Task.detached(priority: .high) {
         let urls = await self.getURLs()
         print("[SAC] Concurrency: Got \(urls.count) urls.")
         print("[SAC] Concurrency: End")
         completion()
      }
#else // Below works on macOS but crashes on Android. See: https://github.com/vgorloff/swift-everywhere-toolchain/issues/138
      Task {
         let task = Task.detached(priority: .userInitiated) { () -> Int in
            let urls = await self.getURLs()
            return urls.count
         }
         let value = await task.value
         print("[SAC] Concurrency: Got \(value) urls.")
         print("[SAC] Concurrency: End")
         completion()
      }
#endif
   }

   private func getURLs() async -> [String] {
      Thread.sleep(forTimeInterval: 1)
      return ["https://docs.swift.org/", "https://google.com/", "https://ibm.com/"]
   }

}

The Task inside a Task not working on Android, but works on macOS:
Task {
         let task = Task.detached(priority: .userInitiated) { () -> Int in
             ...
         }
         let value = await task.value
        ...
}

So simple tasks work on droid but complicated ones (task inside task) crashes ?

On Jan 6, 2022, at 4:33 PM, Vlad Gorlov @.***> wrote:

The Concurrency actually working with latest toolchain, but for macOS and Android it works differently.
public class SAConcurrencyMain {
private var task: Task<Void, Error>?
public init() {
print("[SAC] Concurrency: Init")
}
public func execute(completion: @escaping () -> Void) {
print("[SAC] Concurrency: Start")
#if !false
task = Task.detached(priority: .high) {
let urls = await self.getURLs()
print("[SAC] Concurrency: Got (urls.count) urls.")
print("[SAC] Concurrency: End")
completion()
}
#else // Below works on macOS but crashes on Android. See: https://github.com/vgorloff/swift-everywhere-toolchain/issues/138
Task {
let task = Task.detached(priority: .userInitiated) { () -> Int in
let urls = await self.getURLs()
return urls.count
}
let value = await task.value
print("[SAC] Concurrency: Got (value) urls.")
print("[SAC] Concurrency: End")
completion()
}
#endif
}
private func getURLs() async -> [String] {
Thread.sleep(forTimeInterval: 1)
return ["https://docs.swift.org/", "https://google.com/", "https://ibm.com/"]
}
}
The Task inside a Task not working on Android, but works on macOS:
Task {
let task = Task.detached(priority: .userInitiated) { () -> Int in
...
}
let value = await task.value
...
}
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

True. Some pieces of Concurrency seems working on Android as on macOS. Others seems not.
Maybe some kind of execution environment on Android is not fully initialised/configured (e.g. MainActor or similar).

Let’s find the Linux concurrency people

On Jan 7, 2022, at 2:29 AM, Vlad Gorlov @.***> wrote:

True. Some pieces of Concurrency seems working on Android as on macOS. Others seems not.
Maybe some kind of execution environment on Android is not fully initialised/configured (e.g. MainActor or similar).
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

Hmm, looks like you're building the stdlib with the NDK clang, you have to make sure the Concurrency files are built with Apple's fork of clang.

Hey!   Thanks for this !    Can we jump on that slack channel on swift for android for higher bandwidth?   Gorloff you game?   I think Vlad is in Europe somewhere, so hoping to catch this "everyone is awake" window..  You don't really need me, i'm merely the pot stirrer.  :-)

Buttaface, any stability issues once you had it running?  Saleem mentioned to me that he had to work over the libdispatch on windows stuff to get things stable.

I'm not sure he's awake. 😉 I don't think there will be much to discuss, that is the only issue I've had with getting it working.
I can't really speak to stability, as I don't use the new Concurrency runtime on Android much. However, I run the full compiler validation testsuite on Android with the latest trunk tag once a month, and all the tests pass.

Wake up VLAD ! :-)
Well hoping to have a powwow as needed to get him to a good place.
Then later we can all talk about how we can get assistance getting things you guys need into main?
I can imagine thats “fun”.
We currently use “course grained” actors enough that we should shake any first things loose.

On Jan 7, 2022, at 11:25 AM, buttaface @.***> wrote:
I'm not sure he's awake. 😉 I don't think there will be much to discuss, that is the only issue I've had with getting it working.
I can't really speak to stability, as I don't use the new Concurrency runtime on Android much. However, I run the full compiler validation testsuite on Android with the latest trunk tag once a month, and all the tests pass.
—
Reply to this email directly, view it on GitHub https://github.com/vgorloff/swift-everywhere-toolchain/issues/138#issuecomment-1007590900, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5F3PJPZUMESCQF3KVPMY3UU4O2FANCNFSM5K57HZOQ.
You are receiving this because you authored the thread.

I think the issue is only because he doesn't use the official build-script, whereas I'm now down to a couple small patches since I use that. There's not much left to upstream for me, just submitting the last handful of patches.
If you run into any issues with Concurrency on Android, just post it in the forums or the bug tracker and we'll take a look.

Butta are you on the slack for swift for android ?
Higher bandwidth !

On Jan 7, 2022, at 11:47 AM, buttaface @.***> wrote:
I think the issue is only because he doesn't use the official build-script, whereas I'm now down to a couple small patches since I use that. There's not much left to upstream for me, just submitting the last handful of patches https://github.com/search?q=org%3Aapple+author%3Abuttaface&state=open&type=Issues.
If you run into any issues with Concurrency on Android, just post it in the forums or the bug tracker and we'll take a look.
—
Reply to this email directly, view it on GitHub https://github.com/vgorloff/swift-everywhere-toolchain/issues/138#issuecomment-1007604754, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5F3PLVWR7MJRMM5WNHV7LUU4RKHANCNFSM5K57HZOQ.
You are receiving this because you authored the thread.

Interesting. I didn't know that LLVM has a new flag, which seems emits something into binary which seems is needed to support Concurrency feature.

Can we all jump to slack for a couple minutes?
Butta you on that one?
swift-android.slack.com

On Jan 7, 2022, at 11:50 AM, Vlad Gorlov @.***> wrote:
Interesting. I didn't know that LLVM has a new flag, which seems emits something into binary which seems is needed to support Concurrency feature.
—
Reply to this email directly, view it on GitHub https://github.com/vgorloff/swift-everywhere-toolchain/issues/138#issuecomment-1007606840, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5F3PMHBKJPMIF4TWFKTC3UU4RWVANCNFSM5K57HZOQ.
You are receiving this because you authored the thread.

I didn't know about that one. Vlad and I occasionally chat on another Slack: he can find me on there now if he wants.

Which slack, can I jump there too?  I promise to be nice.

On Jan 7, 2022, at 11:58 AM, buttaface @.***> wrote:
I didn't know about that one. Vlad and I occasionally chat on another Slack: he can find me on there now if he wants.
—
Reply to this email directly, view it on GitHub https://github.com/vgorloff/swift-everywhere-toolchain/issues/138#issuecomment-1007612666, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5F3PN4NLCSTGNJFAPXEU3UU4SUNANCNFSM5K57HZOQ.
You are receiving this because you authored the thread.

I don't think we have invites or are even supposed to publicly talk about it yet. As you said, it's mostly me and him that might need to talk, so we'll get to it whenever. If you want us to look at something in particular, just post it here.

Cool, no worries.
We will be standing by to test, and we build with swiftpm on Mac using his kit.
Good luck gents.

John

On Jan 7, 2022, at 12:02 PM, buttaface @.***> wrote:
I don't think we have invites or are even supposed to publicly talk about it yet. As you said, it's mostly me and him that might need to talk, so we'll get to it whenever. If you want us to look at something in particular, just post it here.
—
Reply to this email directly, view it on GitHub https://github.com/vgorloff/swift-everywhere-toolchain/issues/138#issuecomment-1007615836, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC5F3PJO4PUBFP3W4SEVGWLUU4TDNANCNFSM5K57HZOQ.
You are receiving this because you authored the thread.

Seems like I have to research first which NDK contains clang with Swift concurrency support. It seems easier way than building stdlib with system provided clang and then linking with NDK clang/linker.

No, you can just use the forked Swift clang to both build and link the Swift stdlib, that works fine. That is what it does by default when you build the full Swift toolchain, so you must be overriding the stdlib build to use the NDK clang. Turn off that override and it should work fine.

Ok. Then I have to look closer which clang used where.
I past there were some build issues (in this not official toolchain) when building some components using clang built from fork (https://github.com/apple/llvm-project).
I even have an issue for this https://github.com/vgorloff/swift-everywhere-toolchain/issues/101

OK, I didn't know you had issues with the Swift-forked clang before. The problem is this line, where you pass in the NDK clang to build the stdlib instead. I used to do that too, but switched to the Swift-forked clang a couple months ago to fix this issue with building the Concurrency library.

Can we jump on slack for a few minutes ?  I would like to help with this effort.


settings page: Change cursor for date input to 'pointer'.

Currently, the cursor for the date input field in the settings page
is 'not-allowed' as it has the disabled attribute because we want
users to pick the date from the date picker. But this leads to
confusion whether the field is editable at all.
Change the cursor to 'pointer' to make it clear that the field has
a click action associated with it.

Testing Plan:
Manual testing in the browser.
GIFs or Screenshots:
Before:

After:

This is great, merged, thanks @agrawal-d!


Problem with uvloop and static files

Python/3.5.2 aiohttp/0.22.5 uvloop==0.5.2
Run this code:
import asyncio

import aiohttp.web
import uvloop

asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

app = aiohttp.web.Application()
app.router.add_static('/static', 'static')
aiohttp.web.run_app(app)

Run the following command:
curl http://<IP_ADDRESS>:8080/static/index.html
<html>
    <body>
        some static file
    </body>
</html>HTTP/1.1 200 OK
CONTENT-TYPE: text/html
LAST-MODIFIED: Tue, 16 Aug 2016 23:42:56 GMT
CONTENT-LENGTH: 62
DATE: Tue, 16 Aug 2016 23:43:57 GMT
SERVER: Python/3.5 aiohttp/0.22.5

Headers come after file.

Nice catch!
Funny bug.
I'll take a look.

Affects me as well (as mentioned in the issue referenced above).

Hmmm. Looks like duplicating socket FD removes all guarantees about write ordering.
@1st1 do you have any idea?
#1122 has failed tests for the issue

Hm. The general idea is to make sure that the headers are flushed, sent, and the original socket is closed. Then do sendfile on a duplicated socket. Not sure how to implement this though.
I also think it's time to add sendfile implementation to asyncio.  Will try to work on that.

Closing original socket effectively prevents HTTP pilpelining, which is especially effective for file sending.
As workaround I can disable sendfile for uvloop returning back to read-write loop.
Not the best solution though.

Does anyone use this in production?

Not yet, but I'm about to put an aiohttp-based application in production. I kept uvloop out while this issue is being fixed.

I mean you shouldn't ever serve static files with aiohttp in production -- you should use nginx (or another http server) for that.

Not in my case, that was going to be used just for a manual test which was going to be run once (or twice) and then disabled.
Your last point is not valid, since a number of applications do serve static files in production as the backend for CDNs. This means that they do provide the original files, although the "big hits" come to the CDN and not the end application. I've also worked with applications serving from Nginx or Apache to a CDN (in both cases it was Akamai), and today I work with an application that serves static files from CherryPy directly to a CDN (Amazon CloudFront). Both styles work well, what matters the most in all of these cases are the quality and features of the CDN.
Anyway, the issue happens in any environment, so it needs a fix either way :-)

@diogobaeder Sure :) Thank for explaining your use case though.

As workaround I can disable sendfile for uvloop returning back to read-write loop.
Not the best solution though.

@asvetlov Yeah, especially, considering that the old way of doing things (using loop.add_reader on sockets of transports) will be a RuntimeError in the next asyncio release. And, in fact, the same code might not work without uvloop either -- fundamentally this is a race condition.  So this has to be fixed.
My solution would be to dup the socket, write headers to the duped socket using socket.send/loop.add_writer APIs, and then use socket.sendfile.  The key thing is to send headers and file to the same duped socket, leaving the original transport's socket untouched.

@1st1 writing into socket by socket.send/loop.add_writer looks like antipattern. It ends up with reinventing transport but with sendfile support :)
Looks like we really need support sendfile in asyncio itself.

We'll have the same problem since libuv doesn't support sendfile for transports.
Maybe it is an anti pattern, but you're trying to use syscalls that asyncio does not support and it just happens to work properly.
While writing headers manually will involve some low level coding, it's the only solution that will work now and for months to come.

Just like to put in my $.02 as well. This affects me, and yes I hope to use this in production. I am running an app with gunicorn + aiohttp with uvloop worker right now, I am hoping to add an "admin panel" to the app. Why am I serving with aiohttp instead of nginx/some other server? simple, its an admin panel, not being used by customers, and it makes more sense to deploy it with the backend service.

I'll try to implement @1st1 suggestion for writing headers part without using transport.
Keep in touch.

Thanks a lot, Andrew!

Fixed by #1122


Templates in grunt targets resolved inconsistently

I run grunt-ts 4.2.0 and I have defined two targets:
    ts: {
      dev: {
        options: {
          sourceMap: true
        },
        src: '<%= typescript.src %>',
        outDir: '<%= typescript.outDir %>',
        baseDir: '<%= typescript.baseDir %>'
      },
      prod: {
        options: {
          sourceMap: false
        },
        src: '<%= typescript.src %>',
        outDir: '<%= typescript.outDir %>',
        baseDir: '<%= typescript.baseDir %>'
      }
    }

If I run either target, I get

Warning: Unable to create directory "path\to\output<%= typescript.baseDir %>" (Error code: ENOENT). Use --force to continue.

When I output target at https://github.com/TypeStrong/grunt-ts/blob/master/tasks/modules/compile.ts#L140 (where I think the error occurs), I get:
{ options: { sourceMap: true, mapRoot: undefined, sourceRoot: undefined },
  src: '<%= typescript.src %>',
  outDir: 'target/tmp/foo/src',
  baseDir: '<%= typescript.baseDir %>',
  out: undefined,
  reference: undefined }

Why is outDir correctly resolved but not src or baseDir?

Hi,
There is a bit of a problem with the order in which we resolve things that may be causing this.  I'm working on parameter resolution and templates now as part of the tsconfig.json support and I'll be sure to use your gruntfile in a test case.  Sorry about that.  It should be fixed soon.  Can you get away with hard-coding the values in your gruntfile for now (or just use variables instead of templates so they get evaluated ahead of time)?

This will be fixed in #275 when that is complete (still work-in-progress).

Thanks for the quick response. No, it is not a big deal. Just thought I'd mention it. Should I close this issue since it is tracked elsewhere now?

Nah I'll close it when #275 lands.

Hello, @bjorm - Would you please try<EMAIL_ADDRESS>and let me know if it resolves your issue?  Please note that this is a significant rewrite of the options resolution logic so it may have other issues.  Let me know if you see anything.

This should be corrected in v5, just released to npm.  Please reopen if not.


Upgrade to Citrix xDS Adapter v0.10.1

Current Behavior
The current Meshery Adapter for Citrix is running an older version of Citrix ADC with the xDS-API based service mesh.
Desired Behavior
Upgrade to Citrix xDS Adapter v0.10.1

Contributor Guides and Resources

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Layer5 Discussion Forum and Layer5 Community Slack

@dheerajng will you consider taking this enhancement?

@leecalcote is this an issue I can work on?

@haanhvu you're more than welcome to it, however, we have recently deprecated this adapter, given the lack of support for it in the community.

Perhaps, you wouldn't mind adding a deprecated notice in the readme.md.


2FA initialization failed on testnet, account got locked down

Problem
Hey, I tried to enable 2FA on my account danielwpz.testnet. I got 2FA successfully turned on, but when I tried to transfer tokens to others an error was thrown:
Querying call/danielwpz.testnet/list_request_ids,AQ4 failed: wasm execution failed with error: FunctionCallError(HostError(GuestPanic { panic_msg: "Multisig contract should be initialized before usage" })). { "block_hash": "FTDouqiRKnSUNJxtBbwwfYHekDsxKk7idKy7M8RagCP9", "block_height": 57483374, "error": "wasm execution failed with error: FunctionCallError(HostError(GuestPanic { panic_msg: \"Multisig contract should be initialized before usage\" }))", "logs": [] }

And seems I CANNOT disable 2FA on my account now, so it's ’locked down‘ now :(
Expected Behavior
I can successfully enable 2FA and transfer my token with 2FA confirmation.
If I want, I can always turn off 2FA any time.
Steps to reproduce

Enable 2FA on your account
Make a transfer

Looping in @marcinbodnar and @MaximusHaximus as we introduced some changes with the fungible token transfers, and it may have had an impact on the Testnet instance

@ihaid please test this loop, I tried myself but all worked normally

@stefanopepe I can't reproduce it

@stefanopepe Tested, I enabled 2FA, sent Nea, sent FT, after that deactivated 2FA - didn't receive any error.

😂, do you guys know if there is another way to disable 2FA?

btw i'm using email as 2FA method, are u guys using phone num?

btw i'm using email as 2FA method, are u guys using phone num?

We can try to change the email in the system and see if it works. Please open a ticket at https://nearhelp.zendesk.com/hc/en-us we'll take it from there. Specify that you want to change the 2FA, and it's a testnet account, so we don't begin the throughout verification process

cool, will do


String decoding in StackLoop doesn't work

Hello guys, thank You for your great repo!
I m playing around string encoding and found interesting thing which I want to share with you:
You have a code for stack loop encryption:
out[i] = in[i] ^ raw_key[i % sizeof(key)];
And decryption part is:
out[i] = in[i] ^ raw_key[i % sizeof(key)] ^ i;
I believe that decryption part should not contain ^i in the end
Thanks

I think you misread two lines:
https://github.com/open-obfuscator/o-mvll/blob/2a07dab881308add733d3a25ca9d16e7eb3abbb8/src/passes/strings-encoding/StringEncoding.cpp#L54-L68
https://github.com/open-obfuscator/o-mvll/blob/2a07dab881308add733d3a25ca9d16e7eb3abbb8/src/passes/strings-encoding/StringEncoding.cpp#L39-L52
So it works as expected

@andr-ggn, thanks for reporting this! I fixed this via bd1440f5ce8660781113d9ebb31d5852873b6e61, if you rebase to main it should work as expected. This is not really the best fix, but for now it should suffice. Feel free to let us know!


any workflow .json  example share ?

as the title

The README contains 16 example workflows - you can either download or directly drag the images of the workflows into your ComfyUI tab, and it load the json metdata that is within the PNGInfo of those images.
https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved#samples-download-or-drag-images-of-the-workflows-into-comfyui-to-instantly-load-the-corresponding-workflows


Adds Drop app to showcase

Adds the Drop app to the showcase page. Please see screenshot below:

Hey @drop-george, unfortunately we do not accept any PRs that modify the showcase


apvs-385 - get auto approval config from database

apvs-385 - get auto approval config from database
Checklist

[x] Unit tests
[ ] Code coverage checked
[ ] Coding standards
[ ] Error Handling
[ ] Security/performance considered?
[ ] Deployment changes considered?
[ ] README updated

tests failing