Sample rate issue during playback from Audio buffer

Describe the bug
First of all, thanks for your great work on this library! While writing a program for reading an mp3 using rodio and playing it back using wavy, I noticed that samples seem to be consumed from Audio buffers with .drain() at a rate that does not correspond to the rate specified in the Audio buffer. See the program below for an example. This results in the audio being quite distorted and slowed down / stuttering in my case. I tried manually feeding chunks to the SpeakersSink at the appropriate sample rate. This does seem to fix the distorted sound issue (minus some audible clicks inbetween feeding the chunks, which are expected), but then the speakers.play().await call freezes randomly after a few seconds in the song. Perhaps I am using the API in a wrong way?
To Reproduce
Here is the example code to measure the approximate samples per second consumed from the buffer:
use fon::{stereo::Stereo32, Sink, Audio};
use pasts::{exec, wait};
use wavy::{Speakers, SpeakersSink};
use rodio::decoder::Decoder;
use std::io::BufReader;
use std::time::Instant;

enum Event<'a> {
    Play(SpeakersSink<'a, Stereo32>)
}

struct State {
    buffer: Audio<Stereo32>,
    last_time: Instant,
    last_len: usize
}

impl State {
    fn event(&mut self, event: Event<'_>) {
        match event {
            Event::Play(mut speakers) => {
                speakers.stream(self.buffer.drain());
                let current_time = Instant::now();
                let current_len = self.buffer.len();
                let diff_time = current_time - self.last_time;
                let diff_len = self.last_len - current_len;
                //println!("Estimated samples per instant: {:?}/{:?}", diff_len, diff_time);
                let diff_time_ns = diff_time.subsec_nanos() as f64;
                let sps = (diff_len as f64) / diff_time_ns *<PHONE_NUMBER>.0;
                println!("Estimated samples per second: {:?}", sps);

                self.last_time = current_time;
                self.last_len = current_len;
            }
        }
    }
}

fn main() {
    let mut speakers = Speakers::default();
    let example_decoder = Decoder::new(BufReader::new(std::fs::File::open("/path/to/song.mp3").unwrap())).unwrap();
    let vector: Vec<f32> = example_decoder.map(|x| (x as f32) / 32768.0).collect();
    let buffer = Audio::<Stereo32>::with_f32_buffer(48_000, vector);
    let mut state = State { last_len: buffer.len(), buffer: buffer, last_time: Instant::now() };

    exec!(state.event(wait! {
        Event::Play(speakers.play().await),
    }))
}

Expected behavior
I would expect samples to be consumed at 48,000 samples per second as specified in the Audio buffer (and on my soundcard), but they are consumed at about ~20,000 samples per second on my system.
Screenshots
N/A
Desktop (please complete the following information):

OS: Linux 5.9.16

Additional context
The examples from the examples folder work fine (e.g. when recording using Microphone and immediately playing back like in monitor.rs, this issue does not occur).
Any idea what might be the cause?

@glu Thank you for the bug(?) report!  I'll look into this, and get back to you as soon as I can.  At a glance it looks like you're using the API correctly.  I'll have to read the rodio api docs to have more information on this.  speakers.play().await probably shouldn't freezing either, so I'll look into that as well.

@glu Thank you for the bug(?) report!  I'll look into this, and get back to you as soon as I can.  At a glance it looks like you're using the API correctly.  I'll have to read the rodio api docs to have more information on this.  speakers.play().await probably shouldn't freezing either, so I'll look into that as well.

Great, thank you! I will try to help out and see if I can find the issue.
On Mon, 11 Jan 2021, 00:51 Jeron Aldaron Lau<EMAIL_ADDRESS>wrote:

@glu https://github.com/glu Thank you for the bug(?) report! I'll look
into this, and get back to you as soon as I can. At a glance it looks like
you're using the API correctly. I'll have to read the rodio api docs to
have more information on this. speakers.play().await probably shouldn't
freezing either, so I'll look into that as well.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/libcala/wavy/issues/24#issuecomment-757566041, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ANKTIPZMWQWBR7AQ2OMCYVDSZI4RXANCNFSM4V4WWNOQ
.

Great, thank you! I will try to help out and see if I can find the issue.
On Mon, 11 Jan 2021, 00:51 Jeron Aldaron Lau<EMAIL_ADDRESS>wrote:

@glu https://github.com/glu Thank you for the bug(?) report! I'll look
into this, and get back to you as soon as I can. At a glance it looks like
you're using the API correctly. I'll have to read the rodio api docs to
have more information on this. speakers.play().await probably shouldn't
freezing either, so I'll look into that as well.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/libcala/wavy/issues/24#issuecomment-757566041, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ANKTIPZMWQWBR7AQ2OMCYVDSZI4RXANCNFSM4V4WWNOQ
.

@glu It looks like there are 3 different problems going on here:

speakers.play().await call freezes randomly

This is because of a bug I just discovered in the smelling_salts crate, where the speaker becomes ready before the waker is registered.

Distortion

Not 100% sure of the cause of this yet.

Samples consumed at the wrong rate.

I believe this may be an issue with your code making some assumptions about the sample rate returned from rodio.  Try with the following modifications:
use fon::stereo::{Stereo16, Stereo32};
use fon::{Sink, Audio};
use pasts::{exec, wait};
use wavy::{Speakers, SpeakersSink};
use rodio::decoder::Decoder;
use rodio::source::Source;
use std::io::BufReader;
use std::time::Instant;

enum Event<'a> {
    Play(SpeakersSink<'a, Stereo32>)
}

struct State {
    buffer: Audio<Stereo16>,
    last_time: Instant,
    last_len: usize
}

impl State {
    fn event(&mut self, event: Event<'_>) {
        match event {
            Event::Play(mut speakers) => {
                println!("Playing…");
                speakers.stream(self.buffer.drain());
                let current_time = Instant::now();
                let current_len = self.buffer.len();
                let diff_time = current_time - self.last_time;
                let diff_len = self.last_len - current_len;
                //println!("Estimated samples per instant: {:?}/{:?}", diff_len, diff_time);
                let diff_time_ns = diff_time.subsec_nanos() as f64;
                let sps = (diff_len as f64) / diff_time_ns *<PHONE_NUMBER>.0;
                println!("Estimated samples per second: {:?}", sps);

                self.last_time = current_time;
                self.last_len = current_len;
            }
        }
    }
}

fn main() {
    let mut speakers = Speakers::default();

    println!("Reading file…");

    let example_decoder = Decoder::new(BufReader::new(
        std::fs::File::open("/path/to/audio.mp3").unwrap()
    )).unwrap();

    // Read from rodio decoder into fon audio buffer.
    let buffer = Audio::<Stereo16>::with_i16_buffer(
        example_decoder.sample_rate(),
        example_decoder.collect::<Vec<i16>>(),
    );

    println!("Starting…");

    let mut state = State { last_len: buffer.len(), buffer: buffer, last_time: Instant::now() };

    exec!(state.event(wait! {
        Event::Play(speakers.play().await),
    }))
}

@glu It looks like there are 3 different problems going on here:

speakers.play().await call freezes randomly

This is because of a bug I just discovered in the smelling_salts crate, where the speaker becomes ready before the waker is registered.

Distortion

Not 100% sure of the cause of this yet.

Samples consumed at the wrong rate.

I believe this may be an issue with your code making some assumptions about the sample rate returned from rodio.  Try with the following modifications:
use fon::stereo::{Stereo16, Stereo32};
use fon::{Sink, Audio};
use pasts::{exec, wait};
use wavy::{Speakers, SpeakersSink};
use rodio::decoder::Decoder;
use rodio::source::Source;
use std::io::BufReader;
use std::time::Instant;

enum Event<'a> {
    Play(SpeakersSink<'a, Stereo32>)
}

struct State {
    buffer: Audio<Stereo16>,
    last_time: Instant,
    last_len: usize
}

impl State {
    fn event(&mut self, event: Event<'_>) {
        match event {
            Event::Play(mut speakers) => {
                println!("Playing…");
                speakers.stream(self.buffer.drain());
                let current_time = Instant::now();
                let current_len = self.buffer.len();
                let diff_time = current_time - self.last_time;
                let diff_len = self.last_len - current_len;
                //println!("Estimated samples per instant: {:?}/{:?}", diff_len, diff_time);
                let diff_time_ns = diff_time.subsec_nanos() as f64;
                let sps = (diff_len as f64) / diff_time_ns *<PHONE_NUMBER>.0;
                println!("Estimated samples per second: {:?}", sps);

                self.last_time = current_time;
                self.last_len = current_len;
            }
        }
    }
}

fn main() {
    let mut speakers = Speakers::default();

    println!("Reading file…");

    let example_decoder = Decoder::new(BufReader::new(
        std::fs::File::open("/path/to/audio.mp3").unwrap()
    )).unwrap();

    // Read from rodio decoder into fon audio buffer.
    let buffer = Audio::<Stereo16>::with_i16_buffer(
        example_decoder.sample_rate(),
        example_decoder.collect::<Vec<i16>>(),
    );

    println!("Starting…");

    let mut state = State { last_len: buffer.len(), buffer: buffer, last_time: Instant::now() };

    exec!(state.event(wait! {
        Event::Play(speakers.play().await),
    }))
}

Hi @AldaronLau, thanks for the quick reply. I'm glad that you were able to find the cause of the freezing issue; that was fast :).
As for the sample rate: my audio file was sampled at 48 kHz and this corresponds to the decoder.sample_rate() reported by rodio. Also, if the sample rate were incorrect, I think this would be audible as a lower / higher pitch rather than stuttering / a distorted sound.
That said, I have found something interesting that might bring us closer to the solution: I noticed that the sample rate started to increase the longer I listened to the song, while the distortion decreased. This made me suspect that it has something to do with the conversion to Vec here: https://docs.rs/fon/0.4.0/src/fon/audio.rs.html?search=#240. Perhaps this operation is taking too long, allowing only a handful of samples to be consumed from the buffer, especially in the beginning.
To test my hypothesis, I took a subset from the song, and ... it worked! If you add for example a .take(400000) before the collect() in my example above, (a few seconds from) the song will sound almost perfect. I think there is still a very faint distorted effect going on, but maybe it would be possible to remove that as well by splitting the buffer into even smaller chunks before feeding to .drain().

Hi @AldaronLau, thanks for the quick reply. I'm glad that you were able to find the cause of the freezing issue; that was fast :).
As for the sample rate: my audio file was sampled at 48 kHz and this corresponds to the decoder.sample_rate() reported by rodio. Also, if the sample rate were incorrect, I think this would be audible as a lower / higher pitch rather than stuttering / a distorted sound.
That said, I have found something interesting that might bring us closer to the solution: I noticed that the sample rate started to increase the longer I listened to the song, while the distortion decreased. This made me suspect that it has something to do with the conversion to Vec here: https://docs.rs/fon/0.4.0/src/fon/audio.rs.html?search=#240. Perhaps this operation is taking too long, allowing only a handful of samples to be consumed from the buffer, especially in the beginning.
To test my hypothesis, I took a subset from the song, and ... it worked! If you add for example a .take(400000) before the collect() in my example above, (a few seconds from) the song will sound almost perfect. I think there is still a very faint distorted effect going on, but maybe it would be possible to remove that as well by splitting the buffer into even smaller chunks before feeding to .drain().

@glu Thanks for the extra details.  Interestingly, though, your collect() happens before the main executor loop - so I think you're on to something, but I also think there might be something else going on.  With my testing, the audio was played back at the correct sample rate with my above code and didn't change over time (though severely distorted).  This might just be that my hardware is faster?  Or maybe double check your mp3 is stereo?  Otherwise I don't know how to reproduce the problem.  Optimizing drain() is definitely something that could be worked on.
I'm also not 100% sure, but guessing that rodio is returning interleaved audio frames that aren't reordered in any way, which would be correct - but, I couldn't find anything in the documentation about it.  Otherwise, samples might be taken from later in the song and played earlier, causing distortion, but that seems unlikely.

@glu Thanks for the extra details.  Interestingly, though, your collect() happens before the main executor loop - so I think you're on to something, but I also think there might be something else going on.  With my testing, the audio was played back at the correct sample rate with my above code and didn't change over time (though severely distorted).  This might just be that my hardware is faster?  Or maybe double check your mp3 is stereo?  Otherwise I don't know how to reproduce the problem.  Optimizing drain() is definitely something that could be worked on.
I'm also not 100% sure, but guessing that rodio is returning interleaved audio frames that aren't reordered in any way, which would be correct - but, I couldn't find anything in the documentation about it.  Otherwise, samples might be taken from later in the song and played earlier, causing distortion, but that seems unlikely.

Interesting, yes could be that it is something hardware related. Tomorrow
evening I could check on another device. And my mp3 is indeed stereo.
In any case, for my use case it is not really a problem to chunk the
decoded data before feeding it to the SpeakersSink, and with the fixed
freezing issue I think it will work well, so feel free to close this issue
if you want. Many thanks for your help and your time! Much appreciated.
On Mon, 11 Jan 2021, 23:19 Jeron Aldaron Lau<EMAIL_ADDRESS>wrote:

@glu https://github.com/glu Thanks for the extra details.
Interestingly, though, your collect() happens before the main executor
loop - so I think you're on to something, but I also think there might be
something else going on. With my testing, the audio was played back at the
correct sample rate with my above code and didn't change over time (though
severely distorted). This might just be that my hardware is faster? Or
maybe double check your mp3 is stereo? Otherwise I don't know how to
reproduce the problem. Optimizing drain() is definitely something that
could be worked on.
I'm also not 100% sure, but guessing that rodio is returning interleaved
audio frames that aren't reordered in any way, which would be correct -
but, I couldn't find anything in the documentation about it. Otherwise,
samples might be taken from later in the song and played earlier, causing
distortion, but that seems unlikely.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/libcala/wavy/issues/24#issuecomment-758260777, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ANKTIPYIDPF2I3WIWUVFD2TSZN2PHANCNFSM4V4WWNOQ
.

Interesting, yes could be that it is something hardware related. Tomorrow
evening I could check on another device. And my mp3 is indeed stereo.
In any case, for my use case it is not really a problem to chunk the
decoded data before feeding it to the SpeakersSink, and with the fixed
freezing issue I think it will work well, so feel free to close this issue
if you want. Many thanks for your help and your time! Much appreciated.
On Mon, 11 Jan 2021, 23:19 Jeron Aldaron Lau<EMAIL_ADDRESS>wrote:

@glu https://github.com/glu Thanks for the extra details.
Interestingly, though, your collect() happens before the main executor
loop - so I think you're on to something, but I also think there might be
something else going on. With my testing, the audio was played back at the
correct sample rate with my above code and didn't change over time (though
severely distorted). This might just be that my hardware is faster? Or
maybe double check your mp3 is stereo? Otherwise I don't know how to
reproduce the problem. Optimizing drain() is definitely something that
could be worked on.
I'm also not 100% sure, but guessing that rodio is returning interleaved
audio frames that aren't reordered in any way, which would be correct -
but, I couldn't find anything in the documentation about it. Otherwise,
samples might be taken from later in the song and played earlier, causing
distortion, but that seems unlikely.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/libcala/wavy/issues/24#issuecomment-758260777, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ANKTIPYIDPF2I3WIWUVFD2TSZN2PHANCNFSM4V4WWNOQ
.

@glu No problem!  I plan to at least leave the issue open until I fix the freezing issue (working on it right now), as well as the distortion issue.

@glu No problem!  I plan to at least leave the issue open until I fix the freezing issue (working on it right now), as well as the distortion issue.

@glu Sorry, you were right and I was wrong (I stupidly didn't know the tempo of the song I was playing).  I think I know how to fix this issue now, but it'll take a bit of code refactoring.  I recorded the audio output in audacity, which showed there was silence inserted randomly between samples.  Oddly enough, the played back sound from audacity sounded without distortion (I think the alsa driver just flakes out when it's not getting samples quickly enough).  I'll try to get a patched version of smelling_salts and fon released this weekend that should fix everything.  I currently plan to make Audio wrap a VecDeque instead of a Box<[]> so that it won't matter what the buffer size is, it will take the same amount of time to call drain().

@glu Sorry, you were right and I was wrong (I stupidly didn't know the tempo of the song I was playing).  I think I know how to fix this issue now, but it'll take a bit of code refactoring.  I recorded the audio output in audacity, which showed there was silence inserted randomly between samples.  Oddly enough, the played back sound from audacity sounded without distortion (I think the alsa driver just flakes out when it's not getting samples quickly enough).  I'll try to get a patched version of smelling_salts and fon released this weekend that should fix everything.  I currently plan to make Audio wrap a VecDeque instead of a Box<[]> so that it won't matter what the buffer size is, it will take the same amount of time to call drain().

No worries! Nice idea, thanks for your work on this. Let me know if I can help in any way. I'm quite new to Rust, so I'm learning a lot with this.

No worries! Nice idea, thanks for your work on this. Let me know if I can help in any way. I'm quite new to Rust, so I'm learning a lot with this.

@glu I would offer to mentor on this fix, but unfortunately it's quite complicated, involves some unsafe and is kind of a large internal change for the fon crate (I'm not quite sure what all the details are going to be yet).  I'm still in the process of figuring out how to make my crates more accessible for contributions; I know a lot of crates have "good first issue" labels on some issues, but it's difficult for me to either not fix it right away or even find simpler issues.  Thank you again for your interest, opening the issue, and discovering what's going on with fon and the drain() method.

@glu I would offer to mentor on this fix, but unfortunately it's quite complicated, involves some unsafe and is kind of a large internal change for the fon crate (I'm not quite sure what all the details are going to be yet).  I'm still in the process of figuring out how to make my crates more accessible for contributions; I know a lot of crates have "good first issue" labels on some issues, but it's difficult for me to either not fix it right away or even find simpler issues.  Thank you again for your interest, opening the issue, and discovering what's going on with fon and the drain() method.

No worries, thanks for the update!
On Thu, Jan 14, 2021 at 4:18 AM Jeron Aldaron Lau<EMAIL_ADDRESS>wrote:

@glu https://github.com/glu I would offer to mentor on this fix, but
unfortunately it's quite complicated, involves some unsafe and is kind of
a large internal change for the fon crate (I'm not quite sure what all the
details are going to be yet). I'm still in the process of figuring out how
to make my crates more accessible for contributions; I know a lot of crates
have "good first issue" labels on some issues, but it's difficult for me to
either not fix it right away or even find simpler issues. Thank you again
for your interest, opening the issue, and discovering what's going on with
fon and the drain() method.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/libcala/wavy/issues/24#issuecomment-759898697, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ANKTIP26IVUTSEDKWKHGOJDSZZO77ANCNFSM4V4WWNOQ
.

No worries, thanks for the update!
On Thu, Jan 14, 2021 at 4:18 AM Jeron Aldaron Lau<EMAIL_ADDRESS>wrote:

@glu https://github.com/glu I would offer to mentor on this fix, but
unfortunately it's quite complicated, involves some unsafe and is kind of
a large internal change for the fon crate (I'm not quite sure what all the
details are going to be yet). I'm still in the process of figuring out how
to make my crates more accessible for contributions; I know a lot of crates
have "good first issue" labels on some issues, but it's difficult for me to
either not fix it right away or even find simpler issues. Thank you again
for your interest, opening the issue, and discovering what's going on with
fon and the drain() method.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/libcala/wavy/issues/24#issuecomment-759898697, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ANKTIP26IVUTSEDKWKHGOJDSZZO77ANCNFSM4V4WWNOQ
.

Wavy 0.9 should be fixed!  fon's Audio now encapsulates a VecDeque and smelling_salts timing is now fixed, so it shouldn't freeze anymore.

Wavy 0.9 should be fixed!  fon's Audio now encapsulates a VecDeque and smelling_salts timing is now fixed, so it shouldn't freeze anymore.

Great! The sample rate issue seems solved, although there is still some audible distortion that I do not hear with other audio playback libraries (but that is a different issue or perhaps something on my end). One small remark: I got a compile error (error[E0658]) with stable Rust, because make_contiguous is considered an unstable feature. So users need nightly Rust to compile the current version. I don't know if this was intentional or not. Thanks for your work on this!

Great! The sample rate issue seems solved, although there is still some audible distortion that I do not hear with other audio playback libraries (but that is a different issue or perhaps something on my end). One small remark: I got a compile error (error[E0658]) with stable Rust, because make_contiguous is considered an unstable feature. So users need nightly Rust to compile the current version. I don't know if this was intentional or not. Thanks for your work on this!

@glu I never use nightly.  If you check the documentation, it says as long as you're running 1.48.0 you can use it.  I have the newest stable installed which is 1.49.0, no need for nightly.  At the current moment all of my crates target newest stable, maybe at some point I'll consider officially supporting a minimum version, but development is a lot easier for me without that restriction.
The audible distortion likely has something to do with this issue that I found while working on fixing this bug.  Otherwise, it could be that samples still aren't generated fast enough, or your conversion code is introducing distortion somehow.

@glu I never use nightly.  If you check the documentation, it says as long as you're running 1.48.0 you can use it.  I have the newest stable installed which is 1.49.0, no need for nightly.  At the current moment all of my crates target newest stable, maybe at some point I'll consider officially supporting a minimum version, but development is a lot easier for me without that restriction.
The audible distortion likely has something to do with this issue that I found while working on fixing this bug.  Otherwise, it could be that samples still aren't generated fast enough, or your conversion code is introducing distortion somehow.

Ah, it seems my toolchain was out of date :).

Ah, it seems my toolchain was out of date :).


Minor documentation vs generated code differences

The docs describes a View.empty function 🔗.
The generated code is for a View.none function 🔗.
Its a minor issue but I found it momentarily confusing.

Thanks @NduatiK !
(Just updated the docs site in the latest commit)


Adding Arduino Code and Webpage improvements

Description
Please include a summary of the change and which issue is fixed.
Please also include relevant motivation and context.
List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[X] This change requires a documentation update

Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] Any dependent changes have been merged and published in downstream modules

Reviewer: Vinit Shahdeo

@Severus-Matthew Can you please create a firebase-integration.html file and move your code there? Let's not change the home.html?

@vinitshahdeo is it fine now?


Add Storybook to develop and preview web (Lit) components

Type of change

[ ] feat: New feature
[ ] fix: Bug fix
[ ] perf: Performance improvement
[ ] docs: Documentation
[ ] test: Testing
[ ] refactor: Code refactor
[ ] style: Code style
[x] build: Build
[ ] ci: CI
[ ] revert: Revert
[ ] chore: Release

Breaking changes

[ ] BREAKING CHANGE: This pull request introduces breaking changes.

Description
Add Storybook to develop and preview web (Lit) components.
Related tickets and documents
QA instructions
Actions

[ ] Skip linter.
[ ] Skip build.
[ ] Skip test.
[ ] Skip code analysis.

Checklist
Pull request title

[x] The pull request title describes the change well.

Pull request titles are used to generate automatic release notes so well written titles help document each version.

Pull request labels

[x] The pull request label(s) are set and the BREAKING CHANGE one is present if needed.

Pull request labels are used to categorise the changes when generating automatic release notes; breaking changes are included in an individual category to help identify them.

Squash commit

[ ] The squash commit follows the Conventional commits specification.

Conventional commits are used to identify which should be the next version and so automate the release process; breaking changes must be included in the commit footer following the specification.

Duplicated by https://github.com/varodv/ui-elements-web/pull/15


Purpose quote

I used CSS transforming to vertically align the quotes with the text but that spaces the author line just the tiniest bit more then the figma, if its a problem let me know and I can try and look for another method.  (maybe custom fonts or replacing the quotes with images?)

I think this is fine for now. In the future we might want to just use an svg for this since there is so much custom styling.


Support for combining repetition_penalty, presence_penalty

Currently support for these parameters are mutually exclusive. This makes it hard to create an OpenAI compatible API.
cc @akshay-anyscale @Yard1

The support is included in the latest main branch, please see https://github.com/NVIDIA/TensorRT-LLM/pull/754.
Closing. Please feel free to comment if you have any question, thanks!


Update README.md

I fixed https://img.shields.io/github/release/qubyte/rubidium.svg

 You should sign our Contributor License Agreement in order to get your pull request merged.

Where can I sign? Thanks.

@Maryom I think you just click on the badge, https://cla-assistant.io/PerfectlySoft/Perfect?pullRequest=141


Adds localization support for the mdx files and fix for the broken pages

This Pr
i) Adds Localization support in mdx files (post/7-habit)
ii) Fix broken pages by using the correct data props in page components
iii) Removes unused graphql queries from non-page components
iv) Adds language json files for the blog page (7-habits)

Great work @romanr111. I have one small suggestion. Instead of directly passing the markdown content to the ReactMarkdown component, consider utilizing the children prop. This will make the component more flexible, as it will allow us to customize the content using component props in the future. While this change is not necessary at the moment, it's something I'm considering for future enhancements. Thank you for your understanding.


support for labels in recording rules

Some Mimir recording rules (ie. https://github.com/grafana/mimir/blob/main/operations/mimir-mixin-compiled/rules.yaml#L328) require labels to work properly, however the provider seems not to support those?
When running the following code against the provider (version 0.0.9)
  rule {
    record = "cluster_namespace_deployment_reason:required_replicas:count"
    expr   = "ceil(quantile_over_time(0.99, sum by (cluster, namespace) (cluster_namespace_job:cortex_distributor_received_samples:rate5m)[24h:]) / 240000)"
    labels = {
      deployment = "distributor"
      reason = "sample_rate"
    }
  }

the following error appears:
│ Error: Unsupported argument
│
│   on ../some_path.tf line 592, in resource "mimir_rule_group_recording" "mimir_rules":
│  592:     labels = {
│
│ An argument named "labels" is not expected here.

hello @shybbko you right, I totally forget labels in recording rules. Just implemented in version 0.1.0. Let me know if it's ok.

Thanks @fgouteroux , it works like a charm now.
I also greatly appreciate the blazingly fast support, 30 minutes wait for a fix? That's a premium grade support! <3

Thanks @fgouteroux , it works like a charm now. I also greatly appreciate the blazingly fast support, 30 minutes wait for a fix? That's a premium grade support! <3

I was working on it few hours ago before your issue so the code was fresh in my head ;-)


Add of authentication

This code adds authentication handling for downloads from  a server
which requires this. Source.
I tested the functionality with a Jenkins instance using the plugin "HTTP Header by reverse proxy" and an Apache 2.4 with Kerberos authentication (mod_auth_kerb 5.4). Another check was to "disable" any authentication that no credentials were required (Apache "Require all granted").
The service was created on Windows server 2008 with bind to a domain account.

It's a good point. I'm not a .NET programmer but I understood that this auth. is not supported on un-encrypted connections like HTTP and FTP, https://msdn.microsoft.com/de-de/library/system.net.credentialcache.defaultcredentials(v=vs.85).aspx.
I will check this, also a possible opt-in.

I enhancement is now supporting all kind of Microsoft SSPI authentication and a basic auth. All auth. types has been tested with Apache and the reverse proxy plugin for Jenkins.
Because it's my first C# programming I assume that the handling of the XML arguments is not the perfect one!?
I'm still extending the documentation.

@nightman68 Thanks for the change! From the functionality side it definitely makes sense to integrate it. From the code side I can take care of the refactoring/integration once you are fine with the documentation side

Hi Oleg,
I added some more details in the documentation and the sample configuration file. From my side this should be OK now. When you need additional changes please let me know.
Tomorrow afternoon I leave for vacation for 2 1/2 week without a laptop!
Regards
Torsten

Tomorrow afternoon I leave for vacation for 2 1/2 week without a laptop!

Have fun!
I will try to integrate the changes during this time.

I will; sounds good.
CU


Any chat example please to learn more how it works ?

Hi,
Is there any open source chat example using Autobahn with Thruway ?
Ratchet already have that, want to know if it' the case here too ?
If not, any details on how to procceed ?
Thank's

I don't think so, but why do you want to use WAMP? WAMP is a Web Application Messaging protocol and Thruway implemented that protocol.
Take a look on https://github.com/musonza/chat

@AdamSEY What are you on about lol ... He would probably like to use WAMP because that's what it's good for ...
Maybe that laravel lib is relevant, as in it can "maybe" help someone with handling chat data, but it won't help broadcast messages over websockets ..., and that's what this implementation of WAMP is for ...
@miranos No full featured demo that I know of, but I have Thruway running for an app pretty reliably with a RabbitMQ consumer and 0MQ producer.

@benfavre Why are you using RabbitMQ? it makes your application less scalable and “slower” because the central node adds latency and message envelopes are quite big.

Because it's easy to roll out and I was allready using it.
A client can get a sub 20 ms roundtrip to the backend to run anything I want right now and since it's just an abstraction it is plenty good enough.
Answer : because that's what I used, and I had issues with 0MQ.

@miranos - I don't believe we have any chat examples. We welcome PRs though if someone would like to make one.


imageglass: Update to version <IP_ADDRESS>

It seems that ImageGlass only provides 64bit builds since version 8, and the 32bit key is preventing auto-update.

/verify


Removes SQL Scripts that are no longer necessary

@FBRTMaka

I'll have to fix the deployment based on this change.


[Bugfix] Pass trust_remote_code_model=True for deepseek examples

SUMMARY:

Pass trust_remote_code_model=True for deepseek models
Needed due to the slight differences in how tokenizer and processor pull down their relevant configs

why do we need this now? what change caused this?

@horheynm Both AutoConfig and tokenizers share the same cache for AutoConfig, meaning that if you pass trust_remote_code to the processor, you do not have the pass it into oneshot (specifically, the instantiation of AutoConfig in oneshot).
However, the AutoConfig cache is not shared between AutoConfig and processors. Previously we were getting away with not passing trust_remote_code_model because it was sharing from the tokenizer cache, but now we do since processor and AutoConfig no longer shared from the same cache.


New version: Yuanfudao.Yuanfudao version 6.118.0

Created by 🥟 Dumplings in workflow run #9128.

Log
Updated: 6.117.0 → 6.118.0
Submitting WinGet manifests
Creating manifests
Uploading manifests and making commits
Creating a pull request

Microsoft Reviewers: Open in CodeFlow

@stephengillie Could you merge this PR?


[2.0] [POC] Deploy and Launch

Proof of concept for "Deploy and Launch".
As a user, connected via SSH in a terminal window, I can launch an application to the GUI thread. However, the same command does not work from SSHNet.
Feature Includes:

Fix: manually input default path of VSDbg
New: SshTool - BashSudo(string cmd)
New: Sample images to documentation for setup and configure
Auto-install cURL if it does not exist.
Code cleanup
Deprecated, "Publish" feature until further notice.

Breaking Change

NEW - Options: "Root folder of VSDBG" (RemoteVsDbgPath)

Not stale, still a WiP.


Test suite

Testing the interactive usage of the shell via expect (janet expect?) would be nice.

It is sparse, but there.


[Feature Request] How to run Whisper on the 8550 LE system

Hello，
I want to run Whisper on a GPU on an 8550 device.
Now I have downloaded the source code of qai hub and am able to successfully run python3- m pytest -- pyargs qai_hub_models.models.whiser_base_en.demo 、”python 3- m pytest -- pyargs qai_hub_models. models. wheeler_base_en.test” and other commands.
But these commands seem to not be able to see the specific information of converting voice to text. Is there any command that can convert my own audio into text and see the results of GPU operation.
thanks

Please record your voice into an audio file. The demo was just updated today (v0.9.0) with the ability to translate any file. You can pass --audio_file  as a flag to the demo to make that work.

Hi :) Commenting here rather than opening a new issue, tell me if you'd prefer otherwise.
I'm pretty sure the command you used does not execute Whisper on the 8550, and I too would like to know how running Whisper could be done on the 8550 GPU / NPU with Linux.
I already converted Whisper's encoder into a .dlc format but the doing so for the decoder is not possible: afaik QNN still does not support input tensors with variable shape which are needed for transformers.
The Qualcomm AI hub model website now lists the decoder as available on the 8550 with QNN (=AI Engine Direct) but downloading it provides a .so.

Which Linux OS should I use on the 8550 ?
Why is the downloaded model a .so and not a .dlc file? How should I use this shared library?

Have a great day :)


data: QR Code Generator

Add a new project to the list
I want to add a new project to the list
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Completed. Thanks for your contributions 🎉 ❤


kpt fn source should support output option consistent with other kpt fn commands

Both kpt fn and kpt live now uses a --output flag, but they do different things. We should try to align this to avoid confusion.


Consider using code-to-prompt to easily load code into context managers as a replacement for repo_compiler.sh

https://github.com/raphaelmansuy/code2prompt

https://github.com/addyosmani/git2txt

Okay, let's get that ticket created and added to your board. This will help track the decision-making process for standardizing your codebase analysis tool.
Here's a draft ticket you can adapt and use:
Ticket: Standardize Codebase Analysis Tool for LLM Input
Title: Standardize Codebase Analysis Tool for LLM Input
Description:
Currently, we are using a custom script to compile code from our repositories into a single file for use with Large Language Models (LLMs). However, we are aware of at least two other potentially superior tools that could streamline this process and improve the quality of the LLM input.
This ticket is for analyzing the available options, comparing their features, and making a team decision on which tool to adopt for future use. The goal is to standardize our workflow and ensure we are using the most efficient and effective method for preparing our codebase for LLM analysis.
Possible Tools to Consider:

Our current custom script (include the path to it in your actual ticket)
[Name of alternative tool 1] (include link or any relevant information)
[Name of alternative tool 2] (include link or any relevant information)
(Add any other tools the team is aware of)

Acceptance Criteria:

A document comparing the features, pros, and cons of each considered tool.
A team discussion and documented decision on the chosen tool.
A plan for migrating to the new tool (if applicable), including any necessary code changes or workflow adjustments.

Priority: High
Assignee: Alpha (or whoever is best suited to lead this analysis)
Due Date: [Set a reasonable due date - e.g., 3-5 days out]
Labels: Codebase, LLM, Tooling, Analysis, Standardization
Related Tickets: (None yet, but you might link this to future tickets related to implementing the chosen tool)
Notes:

The current custom script is functional but may be less efficient or feature-rich than other available tools.
We need to consider factors such as ease of use, maintainability, scalability, and the quality of the output generated for LLM consumption.

Adding this to your work log:
Here's how you can incorporate this into your existing work log, under the "Ticket Review and Prioritization" section:
2. Codebase and Ticket Analysis

Ticket Review and Prioritization:

Scenarios:

Ticket #123: (Description) - Priority: High - Assigned to: (RS1/Alpha)
Ticket #145: (Description) - Priority: Medium - Assigned to: (RS1/Alpha)
(Add other scenario-related tickets)

Infrastructure:

Ticket #201: (Description) - Priority: High - Assigned to: (RS1/Alpha)
(Add other infrastructure-related tickets)

Agent Coordination:

Ticket #310: (Description - related to fork management) - Priority: High - Assigned to: Alpha
(Add other coordination-related tickets)

Codebase Analysis Tool Standardization

Ticket # (Fill in with new ticket number): Standardize Codebase Analysis Tool for LLM Input - Priority: High - Assigned to Alpha - Due Date: [Insert Due Date] - (Description: Analyze and select the best tool for compiling code into a single file for LLM input. See ticket for details.)

Missing Tickets/Areas Needing Definition:

Need to create tickets for specific community engagement activities.
Need to further define the process for tokenizing agent-created tokens.

Next Steps:
Review the two alternatives and decide on a plan of action
This addition to the work log clearly outlines the new task, its priority, and the next steps involved. Remember to fill in the bracketed information with your specific details. Feel free to ask any further questions.


Ember['default']._LegacyEachView is undefined

I'm getting a TypeError (undefined) in app/initializers/ember-legacy-controllers.js at
initialize: function initialize() {
      /**
        Adds support for ArrayController in the legacy {{each}} helper
      */
      Ember['default']._LegacyEachView.reopen({

Am I supposed to use legacy views, too? I'm not even using the legacy controller types, just want to have SortableMixin back...

I can confirm that installing ember-legacy-views resolves this issue.
However, I don't want to have this dependency.

Ya, understood. We can likely guard the initializer, but we would also have to disable itemController functionality (I think)...

Sorry that you are running into issues, this repo / project is essentially deprecated and unmaintained.
I'm closing this (so that I can archive the repo and make it clearer to folks that this shouldn't be used going forward)...


Update logging.rst

For now, we don't push you to follow any predefined schema of issue, but ensure you've already read our contribution guide: https://open-metric-learning.readthedocs.io/en/latest/from_readme/contributing.html.

@SiddhantSadangi Hey! Why do you want to change this?)

looks like the code snippet missed a newline :)

@SiddhantSadangi Thank you! :)


Python Park schedules execution and reports an error

Search before asking

[X] I searched the issues and found no similar issues.

Linkis Component
linkis-commons
What happened + What you expected to happen
pythonspark调度执行，报错：initialize python executor failed ClassNotFoundException org.slf4j.impl.StaticLoggerBinder，具体如下：
Pythonspark schedules execution and reports an error: initialize Python executor failed classnotfoundexception org slf4j. impl. Staticloggerbinder, as follows:

Relevent platform

Reproduction script

Anything else
This question comes from the QA documentation of the Linkis community
QA Doc Link：https://docs.qq.com/doc/DSGZhdnpMV3lTUUxq
Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

The reason is that the spark server lacks slf4j-log4j12-1.7.25 Jar, copy the above jar report 00 to / opt / cloudera / parcels / cdh-6.3.2-1 cdh6. 3.2. p0. 1605554/lib/spark/jars


Install Steps Out of Date

Gitea v1.21.11
Going down the rabbit hole of figuring out where the folder needs to be in order for the CSS files to be picked up fine in a browser but regardless of where I put it, it's always 404 and never loads.
My CustomPath is /data/gitea, so the folder for the css files should be /data/gitea/public/css but that doesn't appear to work.
I have tried /data/gitea/public/assets/css as there was some mention of an assets folder in the Gitea documentation for customization, but that doesn't work either.
I have changed the ownership of the folders and files to the correct user of git, as well as added the necessary config information into app.ini to make dark-arc selectable but nothing seems to be working.
Any suggestions? Would be a huge help if you knew anything I could try.

Ok, I thought I exhausted all options for figuring this out but turns out I didn't and I was almost there for figuring it out before posting this but in any case I have found the issue.
https://blog.gitea.com/release-of-1.21.0/#-move-public-asset-files-to-the-proper-directory-25907
Rather than download the css file to $GITEA_CUSTOM/public/css/, they should be downloaded to $GITEA_CUSTOM/public/assets/ instead according to the breaking changes for 1.21 stated in the blog post above.
Theme loads just fine now, I suggest you may want to update the installation steps to reflect this.

Bit more playing around it does appear to be VERY 50/50 on whether it wants to load a theme, even with it being in the correct spot. But what can you expect..

Thanks for posting the solution, my gitea instance is still 1.20.3, so I will update soon and update the docs, thanks for posting your findings!

One thing I will suggest now that you have it working, is to clear your browser cache, that might possibly account for working 50/50

Yeah I think it was cache but not browser. I accidentally had "Cache Assets" on my nginx proxy manager for my gitea route.
But either way, things still changed. Not massively, just everything in the $GITEA_CUSTOM/public/assets folder, rather than just $GITEA_CUSTOM/public, if I'm not mistaken. So CSS files going in ../assets/css now :+1:

I updated and verify that all is working. Everything looks good so I updated the readme, thanks for the report!


Solarwatt: Parse error in case of a battery defect

Describe the bug
If the solarwatt battery is not working (in my case because of a battery damage), the URL endpoint used in evcc to get the SOC value returns a null value. This situation is not handled in the evcc code with following error log entries
[site  ] ERROR 2024/06/23 18:20:21 battery 1 soc: strconv.ParseFloat: parsing "<nil>": invalid syntax
Steps to reproduce

Damage your battery :-)  (no, don't do that)
Look at the evcc log

Configuration details
...

meters:
  - name: grid1
    type: template
    template: solarwatt
    usage: grid
    host: <Solarwatt energy manager URL>
  - name: pv2
    type: template
    template: solarwatt
    usage: pv
    host: <Solarwatt energy manager URL>
  - name: battery3
    type: template
    template: solarwatt
    usage: battery
    host: <Solarwatt energy manager URL>

...

Log details
[main  ] INFO 2024/06/23 18:19:48 evcc 0.126.2
[main  ] INFO 2024/06/23 18:19:48 using config file: /etc/evcc.yaml
[main  ] INFO 2024/06/23 18:19:48 starting ui and api at :7070
[db    ] INFO 2024/06/23 18:19:48 using sqlite database: /root/.evcc/evcc.db
[mqtt  ] INFO 2024/06/23 18:19:48 connecting evcc-239835967 at tcp://<IP_ADDRESS>:1883
[site  ] INFO 2024/06/23 18:19:50 site config:
[site  ] INFO 2024/06/23 18:19:50   meters:      grid ✓ pv ✓ battery ✓
[site  ] INFO 2024/06/23 18:19:50     grid:      power ✓ energy ✓ currents ✗
[site  ] INFO 2024/06/23 18:19:50     pv 1:      power ✓ energy ✓ currents ✗
[site  ] INFO 2024/06/23 18:19:50     battery 1: power ✓ energy ✓ currents ✗ soc ✓ capacity ✗
[site  ] INFO 2024/06/23 18:19:50   vehicles:
[site  ] INFO 2024/06/23 18:19:50     vehicle 1: range ✓ finish ✗ status ✓ climate ✗ wakeup ✓
[lp-1  ] INFO 2024/06/23 18:19:50 loadpoint 1:
[lp-1  ] INFO 2024/06/23 18:19:50   mode:        pv
[lp-1  ] INFO 2024/06/23 18:19:50   charger:     power ✓ energy ✓ currents ✓ phases ✓ wakeup ✗
[lp-1  ] INFO 2024/06/23 18:19:50   meters:      charge ✓
[lp-1  ] INFO 2024/06/23 18:19:50     charge:    power ✓ energy ✓ currents ✓
[lp-1  ] INFO 2024/06/23 18:19:50 vehicle updated: unknown -> BMW 320e
[main  ] INFO 2024/06/23 18:19:50 new version available: 0.127.2
[site  ] ERROR 2024/06/23 18:19:51 battery 1 soc: strconv.ParseFloat: parsing "<nil>": invalid syntax
[lp-1  ] INFO 2024/06/23 18:19:52 car connected
[lp-1  ] INFO 2024/06/23 18:19:52 start charging ->
[site  ] ERROR 2024/06/23 18:20:21 battery 1 soc: strconv.ParseFloat: parsing "<nil>": invalid syntax
[site  ] ERROR 2024/06/23 18:20:51 battery 1 soc: strconv.ParseFloat: parsing "<nil>": invalid syntax

What type of operating system are you running?
Docker container
Version
0.127.2

@thumm could you please share a trace log or really the actual HTTP response?


Too restrictive access permissions for sink file

Currently all sink files get read/write owner only permissions (600 in numeric). For some environments it's too restrictive. For example in our environment we have an application running under one user and a process thats checks the application's health status under another.

Env permissions are controlled by umask , so the fact that they are too restrictive is not directly related to Bootique. Having said that, I see no harm in being able to define permissions explicitly, so the PR is applied with some minor tweaks.

It's actually related. FileReportSink is implemented with the usage of  java.nio.file.Files#createTempFile(Path, String, String, FileAttribute[]) under the hood. createTempFile accepts file attribute array as the last parameter. When attribute array is omitted it uses a rather restrictive defaults (owner read and owner write to be precise). See Java Docs

Perhaps we should fix the "move" step then to ensure "umask" is honored?

Well, it would be better solution. But it breaks compatibility in some rare cases.

it breaks compatibility in some rare cases.
We can treat it as a bug fix :)

Any idea why "move from tmp file" step

To have a robust system for monitoring. Otherwise there's a race condition between a Bootique app writing the file, and monitoring agents reading it.

To have a robust system for monitoring. Otherwise there's a race condition between a Bootique app writing the file, and monitoring agents reading it. So it has to be atomic.

Right, ok. So, it means temp file creation should be performed without custom permissions so umask could be implied.


Download attachment

It doesn't look like there's a way to download an issue's attachments.
I can call jira.issue.getIssue() to get the attachment list, and then read the URL from response.fields.attachment[x].content. But I couldn't find a way to download the file from that URL (with the necessary authentication headers), or preferably, pipe it to a writable stream.

@jacobweber I'm using the request module (https://www.npmjs.com/package/request) to accomplish this.
This is a function wrote for an app that does exactly what you're trying to do (I'm using Express on the backend). I've dumbed it down a bit and made it a bit more generic for brevity's sake.
// ****************************************************************************
// Allows a user to get attachments from JIRA proxied via this server as
// as our API user. URLs will be provided as modified Base64-encoded strings.
// -----
// Custom string mappings to allow base64 strings in URI:
// ~ => /
// - => =
// _ => +
// -----
// ROUTE: GET /file/:url
// -----
// @param   Object      req     Express HTTP request object
// @param   Object      res     Express HTTP response object
// ****************************************************************************
function proxy_file(req, res) {
    // Get the jira download URL from the request object
    if (!req.params.url || req.params.url.length <= 0) throw new Error("Invalid URL provided!");
    const jira_url = Base64.decode(req.params.url.replace(/~/g,'/').replace(/\-/g,'=').replace(/_/g,'+'));

    // This is so we don't have to kill our proxy server everytime someone wants to load a file from Jira
    res.set('Cache-Control', 'public, max-age=31557600');

    // Make sure we have a URL before continuing...
    if (!jira_url|| typeof jira_url!== 'string' || jira_url=== 'undefined') {
        return res.download(this.env_config.server_app_path + 'public/img/generic_file.png');
    }

    // Make sure what we're fetching is an attachment, thumbnail, or avatar (we can add more if needed)
    if (!/attachment\/\d+\/.*$/.test(jira_url) && !/thumbnail\/\d+\/.*$/.test(jira_url) && !/\/secure\/viewavatar\?/.test(jira_url)) {
        throw new Error("We cannot fulfill your download request at this time.");
    }

    // Configure Request object to get file from Jira
    const req_options = {
        auth: {
            user: jira_config.username,
            pass: jira_config.password,
            sendImmediately: true
        }
    };

    // Make the request for the file...
    request.get(jira_url, req_options).on('response', response => {
        if (response.statusCode !== 200) {
            console.log("Could not get image.");
            return false;
        }
    }).on('error', (err) => {
        console.log("The error: ", err);
    }).pipe(res);
};

Hope that helps!

Thanks @kylefarris. I ended up doing something similar. Would be nice to have it as part of jira-connector though.

Its probably just a bit too outside the scope and fairly simple to implement.

I also got this issue
but solution @kylefarris doesn't work for me
I found that there is necessary header

Scripts that access Jira remotely may have trouble acquiring or returning a security token, or maintaining an HTTP session with the server. There is a way for scripts to opt out of token checking by providing the following HTTP header in the request:
X-Atlassian-Token: no-check

https://developer.atlassian.com/server/jira/platform/form-token-handling/

Could this be reconsidered?
The library has functionality to upload attachments so this doesn't look out of scope to me.
It's nice to be able to pass around a jira client object which I can use to make calls to the API but then when I need to download an attachment the complexity increases significantly because the library doesn't support it:

I need to import an http library and understand how Jira authentication works, and handle it myself with said http library.
I need to get the credentials from somewhere.


Frontend relative path tweaks.

Allow frontend to read configuration for where the API and events paths are hosted; this is more for different types of local development but useful for breaking the hardcoded dependency on frontend relative to backend.
Signed-off-by: Jason Sherman<EMAIL_ADDRESS>

Closing this PR, replaced with #374 and #368


Failed to parse Hetero-atom only file

Reading a PDB file with HETATM and CONECT fields only, returns empty parsed dictionary
from pidibble.pdbparse import PDBParser
p = PDBParser(pdb_code="50D_5ceo.pdb").parse()

50D_5ceo.zip
Edit: Reading the code, it seems that it does not support reading local files

Unfortunately I made the keyword confusing.  Use
from pidibble.pdbparse import PDBParser1
p = PDBParser(PDBcode="50D_5ceo").parse()

That is, the keyword argument for the pdb_code member is PDBcode, and the .pdb suffix is automatically added.


[Relay][Pass] Add pass to remove unused functions in relay module

https://discuss.tvm.ai/t/remove-unused-function-in-relay-module/4753
cc @icemelon9 @zhiics @jroesch @tqchen @MarisaKirisame

can you make the function accept a set of root function? they will not get removed under any circumstance.
by default it is the main function.

@MarisaKirisame is dead code elimination still broken? If not, we should try to merge these two passes, and add a few configuration knobs (e.g., interprocedural, intraprocedural).

@MarisaKirisame Changed as you suggested. Thank you!

Yes, we do need have a more powerful DCE pass. I think we may need to have some liveness analysis late on since @jroesch will probably need them for memory planning.

Dead Code Elimination is still broken until higher order pointer analysis is there. I will get back to it after the pldi dead line.

Thanks @wweic @jroesch @MarisaKirisame


kv-common vcpkg fix: Option C: release a no-op version of kv-common, then re-delete it from our SDK

This is the best option, in my opinion.
We will have to re-create the kv-common, release it together with core, then we can delete it from our SDK again, as we did initially. This will also give us the path to drop the constructor in the future, as we wanted (#3215).
Other options are #3237 and #3238.

If we are able to remove the port from the registry, that would be ideal. Let's aim for that first.
If we can't, for some reason, then I agree, this might be a good way to go as a workaround, if it has to come to that.


command to automatically diagnose common problems with your deploy

consider adding something about networking connectivity problems (timed out pinging after 600s, etc.)


aws_xray_sdk.core.exceptions.exceptions.SegmentNotFoundException: cannot find the current segment/subsegment, please make sure you have a segment open

With a basic integration, as described in https://github.com/aws/aws-xray-sdk-python/issues/147 , I can't migrate my database.
When I'm trying to run ./manage.py migrate, I get
cannot find the current segment/subsegment, please make sure you have a segment open
Traceback (most recent call last):
 File "./manage.py", line 10, in <module>
 execute_from_command_line(sys.argv)
 File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
 utility.execute()
 File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
 self.fetch_command(subcommand).run_from_argv(self.argv)
 File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
 self.execute(*args, **cmd_options)
 File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 327, in execute
 self.check()
 File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 359, in check
 include_deployment_checks=include_deployment_checks,
 File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 62, in _run_checks
 issues.extend(super(Command, self)._run_checks(**kwargs))
 File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 346, in _run_checks
 return checks.run_checks(**kwargs)
 File "/usr/local/lib/python2.7/site-packages/django/core/checks/registry.py", line 81, in run_checks
 new_errors = check(app_configs=app_configs)
 File "/usr/local/lib/python2.7/site-packages/django/core/checks/urls.py", line 16, in check_url_config
 return check_resolver(resolver)
 File "/usr/local/lib/python2.7/site-packages/django/core/checks/urls.py", line 26, in check_resolver
 return check_method()
 File "/usr/local/lib/python2.7/site-packages/django/urls/resolvers.py", line 256, in check
 for pattern in self.url_patterns:
 File "/usr/local/lib/python2.7/site-packages/django/utils/functional.py", line 35, in __get__
 res = instance.__dict__[self.name] = self.func(instance)
 File "/usr/local/lib/python2.7/site-packages/django/urls/resolvers.py", line 407, in url_patterns
 patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
 File "/usr/local/lib/python2.7/site-packages/django/utils/functional.py", line 35, in __get__
 res = instance.__dict__[self.name] = self.func(instance)
 File "/usr/local/lib/python2.7/site-packages/django/urls/resolvers.py", line 400, in urlconf_module
 return import_module(self.urlconf_name)
 File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
 __import__(name)
 File "/usr/src/app/icourt/urls.py", line 100, in <module>
 urls.include('cms.urls')
 File "/usr/local/lib/python2.7/site-packages/django/conf/urls/__init__.py", line 50, in include
 urlconf_module = import_module(urlconf_module)
 File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
 __import__(name)
 File "/usr/local/lib/python2.7/site-packages/cms/urls.py", line 19, in <module>
 urlpatterns = get_app_patterns()
 File "/usr/local/lib/python2.7/site-packages/cms/appresolver.py", line 189, in get_app_patterns
 site = get_current_site()
 File "/usr/local/lib/python2.7/site-packages/cms/utils/__init__.py", line 16, in get_current_site
 return Site.objects.get_current()
 File "/usr/local/lib/python2.7/site-packages/django/contrib/sites/models.py", line 63, in get_current
 return self._get_site_by_id(site_id)
 File "/usr/local/lib/python2.7/site-packages/django/contrib/sites/models.py", line 35, in _get_site_by_id
 site = self.get(pk=site_id)
 File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method
 return getattr(self.get_queryset(), name)(*args, **kwargs)
 File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 374, in get
 num = len(clone)
 File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 232, in __len__
 self._fetch_all()
 File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 1121, in _fetch_all
 self._result_cache = list(self._iterable_class(self))
 File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 53, in __iter__
 results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
 File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 887, in execute_sql
 cursor = self.connection.cursor()
 File "/usr/local/lib/python2.7/site-packages/aws_xray_sdk/ext/django/db.py", line 81, in cursor
 original_cursor = getattr(self, attr)(*args, **kwargs)
 File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 254, in cursor
 return self._cursor()
 File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 229, in _cursor
 self.ensure_connection()
 File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
 self.connect()
 File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 191, in connect
 self.init_connection_state()
 File "/usr/local/lib/python2.7/site-packages/django/db/backends/postgresql/base.py", line 208, in init_connection_state
 timezone_changed = self.ensure_timezone()
 File "/usr/local/lib/python2.7/site-packages/django/db/backends/postgresql/base.py", line 201, in ensure_timezone
 cursor.execute(self.ops.set_time_zone_sql(), [timezone_name])
 File "/usr/local/lib/python2.7/site-packages/aws_xray_sdk/core/models/subsegment.py", line 58, in __call__
 meta_processor=None,
 File "/usr/local/lib/python2.7/site-packages/aws_xray_sdk/core/recorder.py", line 417, in record_subsegment
 subsegment = self.begin_subsegment(name, namespace)
 File "/usr/local/lib/python2.7/site-packages/aws_xray_sdk/core/recorder.py", line 284, in begin_subsegment
 segment = self.current_segment()
 File "/usr/local/lib/python2.7/site-packages/aws_xray_sdk/core/recorder.py", line 267, in current_segment
 entity = self.get_trace_entity()
 File "/usr/local/lib/python2.7/site-packages/aws_xray_sdk/core/recorder.py", line 369, in get_trace_entity
 return self.context.get_trace_entity()
 File "/usr/local/lib/python2.7/site-packages/aws_xray_sdk/core/context.py", line 93, in get_trace_entity
 return self.handle_context_missing()
 File "/usr/local/lib/python2.7/site-packages/aws_xray_sdk/core/context.py", line 118, in handle_context_missing
 raise SegmentNotFoundException(MISSING_SEGMENT_MSG)
aws_xray_sdk.core.exceptions.exceptions.SegmentNotFoundException: cannot find the current segment/subsegment, please make sure you have a segment open

I believe that this is a known limitation when running Django locally. Please read more about it here. Specifically, it looks like the migrator is attempting to create a subsegment when no parent segment exists.
The documentation talks about how runserver may fail because runserver performs migration checks which will attempt to begin a subsegment. In your case, it looks like you're performing the migration directly and that's what's causing this issue. In the documentation, it talks about how you may either set the context missing strategy to LOG_ERROR so that it emits the error on server startup, or you may define a ready() method that is launched at startup and manually create a segment by calling:
from aws_xray_sdk.core import xray_recorder
xray_recorder.begin_segment("Migration Segment")

Would you be able to try that and see if it solves your problem?

I tried the LOG_ERROR setting, but now I'm getting another error:
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 327, in execute
    self.check()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 359, in check
    include_deployment_checks=include_deployment_checks,
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 62, in _run_checks
    issues.extend(super(Command, self)._run_checks(**kwargs))
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 346, in _run_checks
    return checks.run_checks(**kwargs)
  File "/usr/local/lib/python2.7/site-packages/django/core/checks/registry.py", line 81, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/usr/local/lib/python2.7/site-packages/django/core/checks/urls.py", line 16, in check_url_config
    return check_resolver(resolver)
  File "/usr/local/lib/python2.7/site-packages/django/core/checks/urls.py", line 26, in check_resolver
    return check_method()
  File "/usr/local/lib/python2.7/site-packages/django/urls/resolvers.py", line 256, in check
    for pattern in self.url_patterns:
  File "/usr/local/lib/python2.7/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/local/lib/python2.7/site-packages/django/urls/resolvers.py", line 407, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/usr/local/lib/python2.7/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/local/lib/python2.7/site-packages/django/urls/resolvers.py", line 400, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/src/app/icourt/urls.py", line 100, in <module>
    urls.include('cms.urls')
  File "/usr/local/lib/python2.7/site-packages/django/conf/urls/__init__.py", line 50, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/local/lib/python2.7/site-packages/cms/urls.py", line 19, in <module>
    urlpatterns = get_app_patterns()
  File "/usr/local/lib/python2.7/site-packages/cms/appresolver.py", line 189, in get_app_patterns
    site = get_current_site()
  File "/usr/local/lib/python2.7/site-packages/cms/utils/__init__.py", line 16, in get_current_site
    return Site.objects.get_current()
  File "/usr/local/lib/python2.7/site-packages/django/contrib/sites/models.py", line 63, in get_current
    return self._get_site_by_id(site_id)
  File "/usr/local/lib/python2.7/site-packages/django/contrib/sites/models.py", line 35, in _get_site_by_id
    site = self.get(pk=site_id)
  File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 374, in get
    num = len(clone)
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 232, in __len__
    self._fetch_all()
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 1121, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
  File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 887, in execute_sql
    cursor = self.connection.cursor()
  File "/usr/local/lib/python2.7/site-packages/aws_xray_sdk/ext/django/db.py", line 81, in cursor
    original_cursor = getattr(self, attr)(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 254, in cursor
    return self._cursor()
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 229, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
    self.connect()
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 192, in connect
    connection_created.send(sender=self.__class__, connection=self)
  File "/usr/local/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 193, in send
    for receiver in self._live_receivers(sender)
  File "/usr/local/lib/python2.7/site-packages/jsonfield/fields.py", line 166, in configure_database_connection
    loads=lambda x: x)
  File "/usr/local/lib/python2.7/site-packages/psycopg2/_json.py", line 178, in register_default_jsonb
    loads=loads, oid=JSONB_OID, array_oid=JSONBARRAY_OID, name='jsonb')
  File "/usr/local/lib/python2.7/site-packages/psycopg2/_json.py", line 147, in register_json
    register_type(JSON, not globally and conn_or_curs or None)
TypeError: argument 2 must be a connection, cursor or None

Tried beginning a segment in ready(), same error.
Did some debugging there, it looks like
<XRayTracedConn at 0x7f6f1885ca70 for psycopg2.extensions.connection at 0x7f6effb0f188>
is not compatible with psycopg2.extensions.connection at 0x7f6effb0f188, at least from psycopg2's point of view...

(this is with psycopg==2.7.7)

Yeah, if I do:
conn = not globally and conn_or_curs or None
register_type(JSON, conn.__wrapped__)

I don't get the error.

The problem is really this:
ipdb> from aws_xray_sdk.ext.dbapi2 import XRayTracedConn
ipdb> from psycopg2.extensions import connection
ipdb> issubclass(XRayTracedConn, connection)
False

Since the XRayTracedConn is not a subclass of psycopg's connection, this has the potential of breaking more things imho...

Nice job finding the root cause of the issue :)!  This is definitely something that will need some further investigation on my end. Does the solution you showed cause any other issues with your application? Is it a viable solution for now until we are able to find a more generalized solution with X-Ray?

This is not really a solution, I was only able to do this in a ipdb debugging session. One option could be to have a separate settings file for migration (with no xray), but since the real issue is the incompatibility between the wrapped db connection and psycopg2, there could be other things at risk.
It looks to me like that this should ideally be fixed in wrapt itself, there seem to be some open issues relating to this (i haven't read through them in detail, though), e.g.: https://github.com/GrahamDumpleton/wrapt/issues/130
I'll need to put the integration on hold until this is resolved. Happy to help if I can.

For now, I'll mark this as a bug until we can dig deeper to find a solution. It looks like the main issue is that wrapt is not properly subclassing the connection or cursor object, so it's not being marked as one as seen in the stacktrace you had sent.

I have added more details to this issue in #243 . I recommend merging this, #243 and #245 into a single issue.
I am currently testing a monkey-patched solution for this in Django, using .__wrapped__ if the instance is XRayTracedConn.

Hi @ezet Good suggestion. We can track this issue in #243 and reduce some clutter here.
@krisb78 Since the original error message was resolved, I'll close this one and tracking in #243
Thanks!

I have the same problem, and I am reading about lambda-powertools:
https://aws.amazon.com/es/blogs/developer/following-serverless-best-practices-with-aws-chalice-and-lambda-powertools/


[question] testset evaluation submission

hi authors.
i see that in https://huggingface.co/spaces/osunlp/TravelPlannerLeaderboard
Format of Submission:
{"idx":0,"query":"Natural Language Query","plan":[{"day": 1, "current_city": "from [City A] to [City B]", "transportation": "Flight Number: XXX, from A to B", "breakfast": "Name, City", "attraction": "Name, City;Name, City;...;Name, City;", "lunch": "Name, City", "dinner": "Name, City", "accommodation": "Name, City"}, {"day": 2, "current_city": "City B", "transportation": "-", "breakfast": "Name, City", "attraction": "Name, City;Name, City;", "lunch": "Name, City", "dinner": "Name, City", "accommodation": "Name, City"}, ...]} where in the plan, there are "day". however, I see that in https://github.com/OSU-NLP-Group/TravelPlanner/blob/main/evaluation/eval.py#L88, the key should be "days" ?
I also see in https://github.com/OSU-NLP-Group/TravelPlanner/blob/main/postprocess/openai_request.py , in the json format, they are "days" instead of "day".
May I know does this affect anything about evaluation ?
please advise. thanks.

besides, does the leaderboard support submission testing ? I mean just uploading several lines (rather than 1000) to have a format check. Also, does it support shuffled order ?

Hi Yanan,
It would not affect the evaluation since 'day' or 'days' is only used as an index.
Sorry, we still do not support the partial set test or shuffled order test. Maybe we will support these in the future, but now we have too much todo.

thanks a lot.


getEditorLayout on IEditorGroupsService ? Ideas?

Whilst we have applyLayout on EditorGroupsService I'd like to propose the read version maybe getEditorLayout() since trying to figure out ways to toggle and persist custom layouts and a way to switch easily between them I've realized the API is limited here. I can for example toggle a single pane as the only one (ctrl+w - o in vim mode) but restoring the layout which might be a complicated tree isn't supported without exposing the current layout - loads of issues with poor and repeated compromises since 2016 - restore the equal width (was it equal width in first place?), use shift+f4 to open in new window instead, if editor groups are horizontal the api can give you tabgroup numbers fron that could with a wish and whim match the layout you had previously and so on.. you can use this plugin or that plugin that uses the API restriction just mentioned..
Ultimately read dozens of issues which would benefit from this - and yet they all end up closed because apparently the velocity of the project means if doesn't get updated in X days its dead - or is there a persistent proposal system that could last more than X days - e.g. PEP in python and KEEPs in Kotlin etc.. .  This close if not-low-picking-fruit-to-fix-in-months leads to latest issues with strange amnesia comments like "what would be the use case for this?"  and seems many extension developers have just given up hitting this wall continuously.
As I explore the API I think the possibility of getEditorLayout() an EditorGroupLayout would be an big win for extension authors and I'd like to take a stab at this if there isn't any resistance -  e.g. "not needed, won't merge" or something already in progress?  also any pointers appreciated.

Related https://github.com/microsoft/vscode/issues/94817 and you follow the trail to dozens of layout / split / tabgroup plugin related requests

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for similar existing issues. See also our issue reporting guidelines.
Happy Coding!


Add a spec for server mode

Summary
This PR adds a spec for server mode that applies .rubocop.yml configuration changes even during server startup.
This is already achieved because the server loads .rubocop.yml configuration for each analysis request from the client.
Other Information
The feature mentioned in the following article has been implemented for configuration change of .rubocop.yml.

You should keep in mind that you'll need to restart the server manually
if you change RuboCop's configuration.

https://metaredux.com/posts/2022/07/26/rubocop-serves-much-faster.html
The test this PR adds ensure that behavior.

Before submitting the PR make sure the following are checked:

[x] The PR relates to only one subject with a clear title and description in grammatically correct, complete sentences.
[x] Wrote good commit messages.
[ ] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Ran bundle exec rake default. It executes all tests and runs RuboCop on its own code.
[ ] Added an entry (file) to the changelog folder named {change_type}_{change_description}.md if the new code introduces user-observable changes. See changelog entry format for details.

This is already achieved because the server loads .rubocop.yml configuration for each analysis request from the client.

Isn't this ineffective from a performance standpoint?

Yeah, I'm assuming this is as performant as a rubocop-daemon server process.
For the time being, it seems that there is a point where I may tune client process logic, so I’m investigating separately.
Then I’ll look further into tuning server process logic.


#6861: Enable Watcher/dprint tests on T3000 CI

CI Passing (w/ 5 passing runs for re-enabled dprint tests):
https://github.com/tenstorrent-metal/tt-metal/actions/runs/8460909086
https://github.com/tenstorrent-metal/tt-metal/actions/runs/8460910725

@tt-dma pls post a passing multi nebula unit tests run.

@tt-dma pls post a passing multi nebula unit tests run.

@tt-rkim the first link in the first comment is a passing multi nebula unit tests run, sorry if that wasn't clear!
https://github.com/tenstorrent-metal/tt-metal/actions/runs/8460909086

My bad - sounds good


[Question] Use load_dataset instead of custom implementation

The alpaca dataset implementation at src/llama_recipes/datasets/alpaca_dataset.py loads the data from a json file followed by using a custom implementation to load and tokenize it. Is there any particular reason to not use dataset from HuggingFace directly(tatsu-lab/alpaca).
I am also curious as to how is tokenization in InstructionDataset. __getitem__ different from SFTTrainer and why is SFTTrainer not being used altogether.
Thank you for your response.

@Tejaswgupta , in this repo, we mostly trying to use PyTorch native APIs, although we have a notebook with HF trainer (not SFTTrainer).
On the Alpaca dataset, I wonder if you are interested to make a PR, will be happy to help review and merge it.

Hi! It seems that a solution has been provided to the issue and there has not been a follow-up conversation for a long time. I will close this issue for now and feel free to reopen it if you have any questions!


[NEW ALGORITHM] Linear Discriminant Analysis (LDA) Algorithm

Issue will be closed if:

You mention more than one algorithm. You can create a separate issue for each algorithm once the current one is completed.
You propose an algorithm that is already present or has been mentioned in a previous issue.
You create a new issue without completing your previous issue.

Note: These actions will be taken seriously. Failure to follow the guidelines may result in the immediate closure of your issue.

Name:
[LDA ALGORITHM]
About:
A supervised dimensionality reduction technique that maximizes class separability.
Labels:
new algorithm, gssoc-ext, hacktoberfest, level1

Assignees:

[x] Contributor in GSSoC-ext
[x] Want to work on it

do not create unnecessary issues

@MastanSayyad I need a reasoning for why this unnecessary


Word count

The word count loses the comma after the thousands, so instead of [23,948] it shows as [23948] and it makes it harder to see the proper number.

Hmm.. For me in it shows as 23 948 with a tiny space between the thousands... Is that not happening for you?

I also prefer the comma over the space

Alrighty, I'll keep that in mind, and probably implement an option to toggle it. No promises on when however, working on other features atm.


Create macro for creating kwargs dict?

While calling methods with only positional arguments of different types is pretty easy (with tuples of IntoPy), for kwargs it gets complicated (see #1462).
Would it be useful to have a macro that can be used like so:
obj.call((pos_arg1, pos_arg2), kwargs! { key1=value1, key2=value2 })

Absolutely! I've thought about this before and I think we can do one better and support python collections a-la vec! and maplit::hashmap! etc. I think I'd make them look like this:
py_list!(py, [a, b, c]);
py_tuple!(py, (a, b, c));
py_dict!(py, {a => 1, b => 2, c => 3});
py_set!(py, {a, b, c});
py_frozenset!(py, {a, b, c});

I think these have more general use than a kwargs! macro.
Also when we add functionality to use vectorcall to call Python functions, then I think we might need some kind of py_call! macro which does the magic.

I wonder if we can take the Python instance from the scope implicitly:
list![a, b, c];
tuple!(a, b, c);
dict!{a => 1, b => 2, c => 3};
set!{a, b, c};
frozenset!{a, b, c};

so then
Python::with_gil(|py| {
    let my_list = list![1, 2, 3]
});

would expand to something like
Python::with_gil(|py| {
    let my_list = PyList::empty().unwrap();
    my_list.append(py, 1).unwrap();
    my_list.append(py, 2).unwrap();
    my_list.append(py, 3).unwrap();
});

I think it would look more familiar to Python users. I'm not sure though that being implicit here is a good idea. Also it relies heavily on the convention to name the Python instance py, which is brittle.

And it doesn't even work (with macro_rules macros) since you can't use outer identifiers in the expansion without passing them in. That shows that Rust doesn't want us to do this in the first place :)

Ah, right, thanks. Never mind then :)

Reminder for myself: I was thinking it might be nice if we can also use spread syntax in these macros.
e.g.
py_list!(py, [*left, *right]);


Add way to generate DataFrame from active_record with aggregated fields

Recently I stumbled on problem with speed when using daru, and found that a way to speed things up was to do more work in database - namely group and aggregate in ActiveRecord/database instead of on dataframe.
Here is what I wanted to use:
active_record = Provider.left_join(zip: :district).group(:id)

However I found out that I cannot give field like ANY_VALUE(district.id), because it gets converted to symbol infrom_activerecord, and subsequently pluck tries to convert it to table.column.
(At least thats how I understand it works).
So, we found out way to bypass this and I was thinking about adding this to daru, in something like this:
      # Load dataframe from AR::Relation
      #
      # @param relation [ActiveRecord::Relation] A relation to be used to load the contents of dataframe
      # @param with_sql_methods [Boolean] Enables giving fields with SQL methods
      #
      # @return A dataframe containing the data in the given relation
      def from_activerecord(relation, *fields, with_sql_methods: false)
        fields = relation.klass.column_names if fields.empty?

        fields = if with_sql_methods
                   fields.map(&:to_s)
                 else
                   fields.map(&:to_sym)

        result = relation.pluck(*fields).transpose
        Daru::DataFrame.new(result, order: fields).tap(&:update)
      end

Now I can create new DataFrame as
data_frame = Daru::DataFrame.from_activerecord(active_record,
                                              ["ANY_VALUE(district.id)"],
                                              with_sql_methods: true)

What do you think about that?

@janmpeterka - Thanks for this feature request suggestion! 🎉
You'd have to contribute this to the daru-io repository for this. We currently have the implementation of ActiveRecord importer here, wherein we support just normal field names and not sql methods. You can probably add has_sql_methods flag / keyword argument - the rest of the logic you might already find in the existing importer logic itself 😄
Would you like to contribute this feature, @janmpeterka?

Thanks, I will look into it. Not sure if I will be able to write the implementation myself, quite new to Ruby :)

Well, daru-io is quite inactive, so no contributing there.


Not working analytics when use docker image (0.12.0)

When I start project as Docker container using instructions from http://api-umbrella.readthedocs.io/en/latest/getting-started.html#running-with-docker, the "Analytics" have empty charts. API works OK, but all charts empty. I not found any issue in logs.
My api-umbrella.yml file:
services:
  - general_db
  - log_db
  - router
  - web
web:
  admin:
    initial_superusers:
      -<EMAIL_ADDRESS>    auth_strategies:
      enabled:
        - github
        - google
        - persona
      github:
        client_id: xxx
        client_secret: xxx
      google:
        client_id:
        client_secret:
elasticsearch:
  hosts:
    - http://<IP_ADDRESS>:14002
mongodb:
  url: mongodb://<IP_ADDRESS>:14001/api_umbrella
nginx:
  workers: 4
gatekeeper:
  workers: 4
hosts:
  - hostname: apinf.usubov.com
    default: true

I can test my backend:
$ curl --insecure "https://apinf.usubov.com/maps/api/geocode/json?api_key=hwBrTpoJhHwMszx904lDOjKZgDoD1oAX5aTJNVEr&address=1600+Amphitheatre+Parkway,+Mountain+View,+CA&key=AIzaSyDc97a13NBkefRoq6e5v2FpIyjEBS1rkl0"

Empty charts:

I can test that Elasticsearch works:
root@c5378d2bcfc9:/# curl http://<IP_ADDRESS>:14002
{
  "status" : 200,
  "name" : "Lady Mastermind",
  "cluster_name" : "api-umbrella",
  "version" : {
    "number" : "1.7.5",
    "build_hash" : "00f95f4ffca6de89d68b7ccaf80d148f1f70e4d4",
    "build_timestamp" : "2016-02-02T09:55:30Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
}

Any advice where I should find issue?

Ping @GUI. We need a bit of support here, to figure out what to do.
Ping @bajiat, this issue is related to our analytics not working recently.

@GUI I added debug code to Lua scripts, found that we successfully log data here https://github.com/NREL/api-umbrella/blob/master/src/api-umbrella/proxy/hooks/log_initial_proxy.lua#L276
But If I get ElasticSearch stats:
curl -XGET http://localhost:14002/_stats?pretty=true

    "api-umbrella-logs-v1-2016-08" : {
      "primaries" : {
        "docs" : {
          "count" : 0,
          "deleted" : 0
        },
        "store" : {
          "size_in_bytes" : 432,
          "throttle_time_in_millis" : 0
        },
        "indexing" : {
          "index_total" : 0,
          "index_time_in_millis" : 0,
          "index_current" : 0,
          "delete_total" : 0,
          "delete_time_in_millis" : 0,
          "delete_current" : 0,
          "noop_update_total" : 0,
          "is_throttled" : false,
          "throttle_time_in_millis" : 0
        },
        "search" : {
          "open_contexts" : 0,
          "query_total" : 138,
          "query_time_in_millis" : 270,
          "query_current" : 0,
          "fetch_total" : 0,
          "fetch_time_in_millis" : 0,
          "fetch_current" : 0
        },

@GUI Fixed issue by drop ElasticSearch data
curl -XDELETE 'http://localhost:14002/_all'

After that it stars works correctly and I see data on Analytics page.
Now prepare new installation from scratch and will provide for you more debug info.

I tried enable DEBUG level in ElasticSearch conf (logging.yml.mustache), enabled error_log for Nginx and added debug code in Lua scripts, but not found root case yet.
Delete ElasticSearch data after docker container start fix issue.
curl -XDELETE 'http://localhost:14002/_all'

hey guys, I have the same issue but @shaliko solution doesn't seem to solve it..
13.0 installed, getting this error after running the curl:

@ofirtwo Try restart docker containers.
docker-compose restart

@ofirtwo Actually later we found another issue. You need connect to you docker-container and install curl, the do restart.
docker exec -it CONTAINER_ID bash
apt-get update && apt-get install curl

Then from host machine:
docker restart CONTAINER_ID

works now, thanks

This should be fixed in the v0.14.0 release.

Version v0.14.4 same issue happens, the workaround doen't work


adding more string methods to prefer-lodash-method rule. fixes #136.

The following string methods are now reported on:

repeat
replace
split
toLowerCase (toLower)
toUpperCase (toUpper)
trim

Okay.  I addressed the comments you made and rebased.  Thanks again for this plugin!

Breaks for lodash v3 where toLower isn't available.

@omgaz - sorry about this.  i will look into how to fix using data from: https://github.com/wix/eslint-plugin-lodash/tree/master/src/util/methodDataByVersion

@omgaz - I believe I've addressed this in PR #140


Update TAO to the latest version

Feature Request
It would be really helpful to have the latest version of TAO. It lets you create a new "window" from an existing GTK window, which would make Dioxus the only library that can work with Wayland layershells—something I’m really excited to use.
I tried to update it on my side and it seems to have no braking changes.

Done in #2779


Fit the image

Hey, how to fit the image width  in the slider ? tried in the adapter it's not working.. maybe the image is too much small ?

Check glide docs for fit the image or etc. in slider here


Patch os.environ

Resolves #673

Coverage remained the same at 83.174% when pulling a288f0da8a7dbd23db2fa0000abf3381afbf183c on patch-environ-gdal-data into 3b557008d0372c7e1432f0e6b767ec42c9f9a5fa on maint-1.8.

Coverage remained the same at 83.174% when pulling a288f0da8a7dbd23db2fa0000abf3381afbf183c on patch-environ-gdal-data into 3b557008d0372c7e1432f0e6b767ec42c9f9a5fa on maint-1.8.

Coverage remained the same at 83.174% when pulling a288f0da8a7dbd23db2fa0000abf3381afbf183c on patch-environ-gdal-data into 3b557008d0372c7e1432f0e6b767ec42c9f9a5fa on maint-1.8.

Coverage remained the same at 83.174% when pulling a288f0da8a7dbd23db2fa0000abf3381afbf183c on patch-environ-gdal-data into 3b557008d0372c7e1432f0e6b767ec42c9f9a5fa on maint-1.8.

Coverage remained the same at 83.174% when pulling a288f0da8a7dbd23db2fa0000abf3381afbf183c on patch-environ-gdal-data into 3b557008d0372c7e1432f0e6b767ec42c9f9a5fa on maint-1.8.

Coverage remained the same at 83.174% when pulling a288f0da8a7dbd23db2fa0000abf3381afbf183c on patch-environ-gdal-data into 3b557008d0372c7e1432f0e6b767ec42c9f9a5fa on maint-1.8.


Determine the best way to lint source files

Currently npm run lint only lints Javascript files.  However, only the samples directory contains Javascript code.  The rest of the code is in TypeScript.
Determine the best approach for linting all of the code in this repo.

This is already fixed!  npm run lint is running eslint and gts check:
https://github.com/googleapis/nodejs-error-reporting/blob/master/package.json#L36


Overhaul of the readme

Issue #, if available:
Description of changes: Restructured the current README to make it easier to follow the various steps. Created separate prerequisites and installation sections. Added a section with basic usage examples for sharp, thumbor, and custom requests.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Hi - The README has been updated in the latest commit.


GPII-1733:  Dispatchers should pass next function to handler grades...

See GPII-1773 for details.

I also fixed various broken links in the documentation.


Slugify category names

This update will slugify categories such that multi-word category directory names will be URL friendly (e.g., "Community Support" --> community-support instead of community support).

Thank you!


Environment Changes

Question/Request.
Is it possible to get the records from the environment historie in LCS?
(Environment Details - History - Environment changes)
With this you would have a faster overview from failed/incorrect deployments and database restore actions. Does not have to be all records  but for example the last 10 (most recent) records from the LCS form.
Regards, Marcel

Everything is possible :), but in this case you do have a direct link in instance context menu to the page with all those details. Is it really worth implementing it in the app? Convince me.

:-)
Convince you, hahaha.. ok, the mentioned portal link is avaliable as a 'link' which means another prog has to open it and often requires a 'number' of account login's. It would be faster when this data is avaliable direct in 2LCS ?
No problem if you find this not really necessary but as indicated, just a question (and it is possible as I read  :-)   )

Oeps, forgotten to say. We notice that due to the ever increasing number of updates/environments (including customizations) and associated bugs and other issues, the course of these environments changes are quite important to know...
Regards, Marcel

I'd argue that everything which is not available in an API is worth implementing here, as LCS UX is not sufficient when working across numerous projects, and for me the features in 2LCS which I have contributed to thus far are those which allow the export of information across projects such that we can get a birds eye view.

Maybe a good intention for 2020? ;-)  But first of course the best wishes!

Ok. So if I will display the list of historic actions available in the grid this will fulfill your needs, right?

That sure will fulfill a lot!  Thnx  ;-)

See 0.27.0 and let me know your thoughts.

Hi Tomek,
Looks good and is the overview needed / I asked for  ;-)
Thnx for this addition!
I'll also close this request.


Ignore empty executables when processing doctests

When a doctest is annotated with no_run the persisted executable is an
empty file and tarpaulin would fail to parse it. Ignore empty files
to avoid the issue.

@tmiasko can you apply some fixes so this works in stable as well please? And also update the CHANGELOG once that's done :+1:

PTAL

Tested it and it works great :+1: merging now then releasing a new version


Change Show Icon to Channel Logo if Stream is "Just Chatting"

When twitch2tuner builds its epg, it assigns the shows the icon of whichever game is the "category" of the stream. That's fine for streams that use typical box art, but the most common stream type is "Just Chatting" which features an icon of a woman with a coffee cup in front of a microphone.
That's totally irrelevant for those streams.
Is there a way to detect if the category is "Just Chatting" and then use the channel's logo instead?

Hey @databoy2k, great suggestion. I've added conditional support for this feature based on an environment variable. Grab the latest Docker image, set USE_PROFILE_AS_JUST_CHATTING to true, and it should do what you want. Let me know what you think!

Absolutely awesome! It works like a charm. Quick refresh of guide data and I no longer have a coffee cup on my Just Chatting channels.
Thanks a ton!


TensorCache's forward_autoregressive is Private

I am in the process of implementing caching for one of my models and when building I encountered an error notifying me that the forward_autoregressive method for TensorCache is private. Is this intentional?

It is, though we might make it public, but we should add proper documentations before.