Support for variant streams, invocation from external code, progress hooks, verbosity

Everything is explained in the commits message;
after merge, can this version be tagged to 1.0.2 or something please?
I made a bot to download my university lessons, and this edits would be very useful to replace youtube-dl and to download them in parallel!
Thank you 🙏🏻

Thanks for the PR. The changes are too substantial for an unannounced PR, so I can't promise when I'll have time to do the review. At a glance I can get behind some changes but not all, and everything being in a single commit makes reviewing and accepting some changes extremely hard. It looks like you squashed many commits? Please separate them if possible (otherwise I might find the task dreadful enough and just do a clean room reimplementation...). Also, all code should be formatted with black. Some of the changes seem to do nothing but break the formatting (e.g., changes to argparse code), it would be nice to format with black and drop those distractions.
If the requested changes (to make things easier for me) are too time consuming for you then unfortunately it might end up taking a long time for me to get back to you... (I'm certainly not saying you're obligated to change anything. In case this sounds passive aggressive, that's not my intention, I'm just setting reasonable expectations.) Fortunately it's not hard to use your own fork in any setting (reference just in case you don't know how), so whatever you're trying to do shouldn't rely on my finding the time to review and merge the PR.

Some of the changes seem to do nothing but break the formatting

Oh, a little bit more on that: please don't make unnecessary changes like switching order of short and long options; this is my project so I'll switch back to my preference anyway, it'll just waste time for both of us.
The one unnecessary change I can get behind: I usually prefer qualified identifiers than bare imported identifiers (more readable without understanding context), meaning pathlib.Path not Path, but it seems I'm really using it a lot, and it's been a long time since pathlib was introduced in py34, so sure, let's just import Path alone.

I can't promise when I'll have time to do the review

Don't worry

It looks like you squashed many commits? Please separate them if possible

I didn't - I know, I should have... If I try to separate the commit now, I'll probably break something 😅

Fortunately it's not hard to use your own fork in any setting

Oh, nice to know!

so whatever you're trying to do shouldn't rely on my finding the time to review and merge the PR.

Yeah, you're absolutely right; I asked a pull request mainly because I thought it was the only way to distribute the modified code with pip to final users; I don't have this urge anymore due to your suggestion, so merge only if you wish 👍🏻

Btw I reverted some changes in particular as you asked, also the whole commit can be summarized in:

Split main() into make_arguments() __launch() __handle_arguments()
Add ffmpeg_loglevel progress_hooks and pass them around also with verbosity
Add invoke() to launch with arguments from external code, example:from caterpillar import caterpillar

def hook(data):
    print(data)

url = 'https://whatever'
out = 'video.mp4'
wd = 'video_temp'
hls_opts = {
    'm3u8_url': url,
    'output':   Path(out),
    'workdir':  Path(wd),
    'jobs':     4,
    'quiet':    24,
    'no_progress': True,
    'ffmpeg_loglevel': 'fatal',
    'progress_hooks': [lambda d: hook(d)] # lambda used to denote number of arguments
}
caterpillar.invoke(**hls_opts)

Add download_m3u8_playlist_or_variant: use the first variant stream found, if a variant streams playlist has been found
Call progress hooks when downloading the segments inside download_m3u8_segments

Thanks for making an effort.

I didn't

Ah that's what I dreaded. I can't merge all the changes in one commit even if I want to accept of them verbatim... That would be terrible for blaming and bisecting in the future.
However, your effort isn't in vain, at least it's been made easier for me to understand the changes you wan't to see, so I think I'll treat them as feature requests instead and make requested changes separately.
Some thoughts:

Split main() into make_arguments() __launch() __handle_arguments()

Sure, could refactor out the parser. But I definitely won't use dunder unless it's absolutely necessary somehow (it never is).

Add ffmpeg_loglevel

It's nice to be able to suppress logging, but it's not that simple. Critical functionality actually relies on log level being at least info: https://github.com/zmwangx/caterpillar/blob/f7c47e6f3b86b72b7ad461e07863566da5a762af/src/caterpillar/merge.py#L47-L62 so introducing an option like this that only affects one invocation (the final merge pass) but not all others is deceptive and confusing. I'd much rather select the log level automatically based on the existing verbosity level (controlled by repeatable -q and -v, as is standard), in fact I'm already suppressing logs based on that (see https://github.com/zmwangx/caterpillar/blob/f7c47e6f3b86b72b7ad461e07863566da5a762af/src/caterpillar/merge.py#L75-L79). So I'll do that if I can't think of any reason not to.

progress_hooks

I can add that, but your reporting is too elaborate for my taste. I'll implement a simpler reporting scheme instead, other logic should be in the caller as they see fit.

Add invoke() to launch with arguments from external code

Sure, I can add an API like that.

Add download_m3u8_playlist_or_variant: use the first variant stream found, if a variant streams playlist has been found

That's a useful addition. The nondescriptive remote0.m3u8 and remote1.m3u8 are a bit weird though, so I'll likely have to massage the implementation a bit.

To summarize, I'll probably implement these changes separately, but it's hard to give you committer credit when I'm not just fixing up minor things in your commits and signing them off. This makes me feel bad since I want to give you credit. So in the future, please consider scoping your commits as well as PRs (a series of PRs for unrelated things would make it a lot easier to accept or reject some).
I'll probably be able to find some time for this either today or within the next week, then I'll cut a release.

I asked a pull request mainly because I thought it was the only way to distribute the modified code with pip to final users;

Btw, just to be clear, I wasn't discouraging PRs, trying to upstream changes shows respect for an OSS project and I'm grateful for that (of course as I said scoped PRs would be a lot better execution-wise). Just saying you don't have to wait unnecessarily since I can't really promise anything (the other day I closed an issue I promised to fix precisely a year ago in another project so you get the idea...)

That would be terrible for blaming and bisecting in the future.

I'll learn how to use git properly 😅

But I definitely won't use dunder unless it's absolutely necessary somehow

I thought using double underscores to hide those function from external code - those functions should not be called directly.

Critical functionality actually relies on log level being at least info [...] in fact I'm already suppressing logs based on that

Oh..! I didn't notice that code; during some tests I passed 'quiet': 24 but I still got ffmpeg output, I probably missed something.
New idea then: add an argument to explicitly disable ffmpeg output.

I can add that, but your reporting is too elaborate for my taste.

Uh, ok; the final result I wanted to reach is this, but without rewriting the hook code for the progress bar.

The nondescriptive remote0.m3u8 and remote1.m3u8 are a bit weird though

I couldn't find better names, following this logic:

download playlist from given url (remote0.m3u8)
is this a variant stream playlist?

select first variant stream and download it (remote1.m3u8)
put remote1.m3u8 as final playlist to consider

download stream from final playlist

I'll probably implement these changes separately, but it's hard to give you committer credit

No problem 👍🏻 - feel free to do what you think best suits your needs.

just to be clear, I wasn't discouraging PRs

Yeah, I had no doubts on that; I mean, I probably would have said - to the final users of my project - to download from my fork and I would have asked for a pull request anyway.

I have implemented the feature requests I deem appropriate.

ffmpeg_loglevel: appropriate. Revamped in e3ebefc, should be completely consistent now.
You were seeing extraneous output despite filtering effort since previously suppression was based on a single (repurposed, or half-assed in this context) pattern, meaning a lot of stuff slip through the crack; and the final merging pass wasn't filtered at all, and you were only fixing that. The implementation here fixes everything.

progress_hooks: appropriate. Instead implemented more general, strictly typed event hooks in bbe9bb4. I wrote a simple adapter to the reporting scheme you implemented:

#!/usr/bin/env python3

import math
import pathlib

import caterpillar
from caterpillar.events import (
    Event,
    SegmentDownloadSucceededEvent,
    SegmentDownloadFailedEvent,
    SegmentsDownloadInitiatedEvent,
    SegmentsDownloadFinishedEvent,
)

class ProgressReporter:
    def __init__(self):
        self.segment_count = 0
        self.handled_count = 0
        self.downloaded_bytes = 0

    def event_hook(self, event: Event) -> None:
        # An alternative to isinstance: check event.event_type against
        # the EventType enum.
        if isinstance(event, SegmentsDownloadInitiatedEvent):
            self.segment_count = event.segment_count
            return
        elif isinstance(event, SegmentsDownloadFinishedEvent):
            data = dict(
                status="finished",
                fragment_index=self.handled_count,
                fragment_count=self.segment_count,
                downloaded_bytes=self.downloaded_bytes,
                # Warning: this is wrong, since errored segments haven't
                # been counted.
                total_bytes=self.downloaded_bytes,
            )
        elif isinstance(event, SegmentDownloadSucceededEvent):
            self.handled_count += 1
            self.downloaded_bytes += event.path.stat().st_size
            data = dict(
                status="downloading",
                # Warning: caterpillar uses imap_unordered which returns
                # results out of order, so calling it fragment_index is
                # not accurate.
                fragment_index=self.handled_count,
                fragment_count=self.segment_count,
                downloaded_bytes=self.downloaded_bytes,
                total_bytes_estimate=math.ceil(
                    self.downloaded_bytes / self.handled_count * self.segment_count
                ),
            )
        elif isinstance(event, SegmentDownloadFailedEvent):
            self.handled_count += 1
            return
        else:
            return
        ...

def main():
    reporter = ProgressReporter()
    caterpillar.process_entry(
        "...",
        pathlib.Path("..."),
        force=True,
        keep=True,
        event_hooks=[reporter.event_hook],
    )

if __name__ == "__main__":
    main()

You should probably consider implementing the event hook API natively though.

download_m3u8_playlist_or_variant: appropriate. Implemented variant streams support in e1dd54c. Instead of blindly selecting the first variant, I try to find the best variant. See variants.py.

invoke: not appropriate. Reason: there's already caterpillar.caterpillar.process_entry and caterpillar.caterpillar.process_batch that basically do this, the two are different enough that an application programmer probably won't use both indiscriminately and need their own dispatch anyway (if they use both, which they probably don't), so adding another trivial dispatcher or two seems pointless.

I think those are all the substantial requests. Let me know if you have questions or need anything else. I'll cut a release if there are no additional requests.
--
Unrelated Python clear-up:

I thought using double underscores to hide those function from external code - those functions should not be called directly.

No, you were thinking about name mangling, which only applies to methods, not module functions and variables, and even then only makes access difficult, not impossible. There's a very narrow use case that you'll know when you really, really need it (I was exaggerating a bit when I said "it never is", but the point is if you just want to mark something private, you shouldn't do this). https://docs.python.org/3/tutorial/classes.html#private-variables
If you just want to "hide" module functions from help and import *, then a single underscore is enough.
A simple example you can try:
mod.py:
__hidden_var = "nope, not hidden"

def __hidden_func():
    print("nope, not hidden")

class Foo:
    def __hidden_method(self):
        print("nope, still here")

main.py:
import mod

def main():
    print("mod.__hidden_var:", end="\t")
    print(mod.__hidden_var)
    print("mod.__hidden_func:", end="\t")
    mod.__hidden_func()
    foo = mod.Foo()
    print("mod.Foo.__hidden_method:", end="\t")
    try:
        foo.__hidden_method()
    except AttributeError as e:
        print(f"AttributeError: {e}")
    print("mod.Foo._Foo__hidden_method:", end="\t")
    foo._Foo__hidden_method()

if __name__ == "__main__":
    main()

$ python3 main.py
mod.__hidden_var:	nope, not hidden
mod.__hidden_func:	nope, not hidden
mod.Foo.__hidden_method:	AttributeError: 'Foo' object has no attribute '__hidden_method'
mod.Foo._Foo__hidden_method:	nope, still here

Let me know if you have questions or need anything else. I'll cut a release if there are no additional requests.

I would have some questions but can't ask right now; I'm in a hurry

No, you were thinking about name mangling, which only applies to methods, not module functions and variables, and even then only makes access difficult, not impossible. [...]

Ah, I see; thank you for the clarification.

I’m not in a hurry to cut a release, so no rush.

I released v1.1 while I still remember this... Closing, but feel free to ask questions.

ffmpeg_loglevel - determine loglevel by verbosity level;
download_m3u8_playlist_or_variant- try to find the best variant
I find this designs way better than mine 👍🏻

progress_hooks

appropriate. Instead implemented more general

Uhh, ok I guess; it would be nice adding that ProgressReporter snippet (or something like that) in the code [without using it directly], to avoid unnecessary copy&paste and re-implementations in dependents code
class ProgressReporter:
    STATUS_DOWNLOAD = "downloading"
    STATUS_FINISH = "finished"

    def __init__(self, callback: Callable):
        self.segment_count = 0
        self.handled_count = 0
        self.downloaded_bytes = 0
        self.callback = callback

    def event_hook(self, event: Event) -> None:
        status = ProgressReporter.STATUS_DOWNLOADING # default

        # An alternative to isinstance: check event.event_type against
        # the EventType enum.
        if isinstance(event, SegmentsDownloadInitiatedEvent):
            self.segment_count = event.segment_count
        elif isinstance(event, SegmentDownloadSucceededEvent):
            self.handled_count += 1
            self.downloaded_bytes += event.path.stat().st_size
        elif isinstance(event, SegmentDownloadFailedEvent):
            self.handled_count += 1
        elif isinstance(event, SegmentsDownloadFinishedEvent):
            status = ProgressReporter.STATUS_FINISHED

        if status is not None:
            estimate = (
                self.downloaded_bytes / self.handled_count * self.segment_count
                if self.handled_count > 0
                else 0
            )
            data = dict(
                status=ProgressReporter.STATUS_DOWNLOAD,
                handled_count=self.handled_count,
                segment_count=self.segment_count,
                downloaded_bytes=self.downloaded_bytes,
                total_bytes_estimate=math.ceil(estimate),
            )
            self.callback(data)

invoke

not appropriate. Reason: there's already caterpillar.caterpillar.process_entry

Ah I see; I wrote it to avoid rewriting the default values of certain arguments (and let argparse handle possibly future arguments of exclusive groups)


youtube-dl regularly produces video files that can't be deleted, moved or renamed

[ ] I'm reporting a broken site support issue
[x] I've verified that I'm running youtube-dl version 2019.08.02
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x] I've read bugs section in FAQ

Verbose log
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-f', '247+bestaudio', 'http://www.youtube.com
/watch?v=X0nk4HqPzrY']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2019.08.02
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-90425-gdb2a7c947e-sherpya, ffprobe N-90425-gdb2a7
c947e-sherpya
[debug] Proxy map: {}
[youtube] X0nk4HqPzrY: Downloading webpage
[youtube] X0nk4HqPzrY: Downloading video info webpage
[debug] Invoking downloader on 'https://r6---sn-o097znlk.googlevideo.com/videopl
ayback?expire=1565396341&ei=FblNXajZHZKZkwbngqLABw&ip=<IP_ADDRESS>&id=o-AEwNjeE
oZgoTXb4ZErn-bwUA7DBocuQNYClviX258u2V&itag=247&aitags=133%2C134%2C135%2C136%2C13
7%2C160%2C242%2C243%2C244%2C247%2C248%2C278&source=youtube&requiressl=yes&mm=31%
2C29&mn=sn-o097znlk%2Csn-n4v7sney&ms=au%2Crdu&mv=m&mvi=5&pl=21&initcwndbps=21175
00&mime=video%2Fwebm&gir=yes&clen=23123335&dur=151.317&lmt=1517563823035107&mt=1
565374610&fvip=1&keepalive=yes&c=WEB&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Cso
urce%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=ALgxI2wwRgIhANSM2mP6JsR2KX
6mXzIix_UclAl5AyeMk8a2xEFviBEpAiEA5WxiW9Moa4gXgXuB3llogrF8kySt4jCKPqp5v1nDr58%3D
&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylml4wRgIhAIdhL72xpX
hzYzt9LVYAHK2j0KosNL4oIaEOjytqmUv3AiEAkpYK0QNaLiyWW9WRi7mNvldf7gp9z48liPeVgCa_Pu
Q%3D&ratebypass=yes'
[download] Destination: Venera Gimadieva - The Golden Cockerel at Santa Fe Opera
-X0nk4HqPzrY.f247.webm
[download] 100% of 22.05MiB in 00:05
[debug] Invoking downloader on 'https://r6---sn-o097znlk.googlevideo.com/videopl
ayback?expire=1565396341&ei=FblNXajZHZKZkwbngqLABw&ip=<IP_ADDRESS>&id=o-AEwNjeE
oZgoTXb4ZErn-bwUA7DBocuQNYClviX258u2V&itag=251&source=youtube&requiressl=yes&mm=
31%2C29&mn=sn-o097znlk%2Csn-n4v7sney&ms=au%2Crdu&mv=m&mvi=5&pl=21&initcwndbps=21
17500&mime=audio%2Fwebm&gir=yes&clen=2635916&dur=151.361&lmt=1517562894906068&mt
=1565374610&fvip=1&keepalive=yes&c=WEB&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Cso
urce%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=ALgxI2wwRgIhAKCWcA5jgD02mL
sgVlM0nrVQ9ohGz4ar3GOdeEIGjNLWAiEAn85RObUJBo4jIPOgvludaIrfbEjTy3x7gYtjY1c315s%3D
&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylml4wRgIhAIdhL72xpX
hzYzt9LVYAHK2j0KosNL4oIaEOjytqmUv3AiEAkpYK0QNaLiyWW9WRi7mNvldf7gp9z48liPeVgCa_Pu
Q%3D&ratebypass=yes'
[download] Destination: Venera Gimadieva - The Golden Cockerel at Santa Fe Opera
-X0nk4HqPzrY.f251.webm
[download] 100% of 2.51MiB in 00:00
[ffmpeg] Merging formats into "Venera Gimadieva - The Golden Cockerel at Santa F
e Opera-X0nk4HqPzrY.webm"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:Venera G
imadieva - The Golden Cockerel at Santa Fe Opera-X0nk4HqPzrY.f247.webm" -i "file
:Venera Gimadieva - The Golden Cockerel at Santa Fe Opera-X0nk4HqPzrY.f251.webm"
 -c copy -map "0:v:0" -map "1:a:0" "file:Venera Gimadieva - The Golden Cockerel
at Santa Fe Opera-X0nk4HqPzrY.temp.webm"
Deleting original file Venera Gimadieva - The Golden Cockerel at Santa Fe Opera-
X0nk4HqPzrY.f247.webm (pass -k to keep)
Deleting original file Venera Gimadieva - The Golden Cockerel at Santa Fe Opera-
X0nk4HqPzrY.f251.webm (pass -k to keep)

Q:\Program Files\Portable\YouTube-dl0\command line\a>

Description
youtube-dl regularly produces video files that cannot be deleted, moved or renamed.
The files can be copied but the copy may have the same traits (undeletable, unmovable, unrenamable).
Restarting the computer can help but usually doesn't.
The files play fine in media players.
Attempting to delete or move one of these files results in a prompt that says "preparing to delete" of "moving" but the prompt never goes away.
Sometimes the cancel button will get rid of the prompt. If not, restarting the computer or killing the explorer.exe process appears to be the only way to get rid of it.
Attempting to rename one of these files results in the explorer window hanging, at which point the explorer.exe process needs to be killed or the computer restarted.
Sometimes, when renaming one of the files, the explorer window will just hang for a long time (rather than permanently) and the file will actually eventually be renamed.
In that case (when the file is renamed), it becomes possible to delete it.
youtube-dl is the only software that has ever produced such files on my drive.
At least two previous versions of youtube-dl also produced these files.
I still have video files (created by youtube-dl) from last year on my drive, that I still can't delete, move or rename. I even tried several pieces of software created for deleting 'undeletable' files and they were not able to delete the files.
The bug has been reproducable every time for me thus far using the following command:
YouTube-dl0\command line\a>youtube-dl -v -f 247+bestaudio http://www.youtube.com/watch?v=X0nk4HqPzrY

This has nothing to do with youtube-dl. Either your disk is damaged/has bads or something locks files. Use LockHunter or similar tool for the latter.


[BUG FIX]Kinesis lag keep increasing when there is no more new data for kinesis stream

Fixes https://github.com/apache/druid/issues/11005.

Description
check the value of AFTER_SEQUENCE_NUMBER. If empty it means there is no more new data for current stream. So just return 0L here.

Key changed/added classes in this PR

KinesisRecordSupplier.java

This PR has:

[x] been self-reviewed.

[ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[ ] added documentation for new or modified features or behaviors.
[x] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[x] been tested in a test Druid cluster.

Hi @clintropolis could you please take a look at this issue at your convince? I find you introduce this useful metrics in https://github.com/apache/druid/pull/9509. Maybe you will be more familiar with it :)

Hi @clintropolis Thanks for your review and approval! Job 75 (Compile=openjdk8, Run=openjdk11) leadership and high availability integration tests is failed. But I think job 75 is not this pr related. Retry may be passed :)

@zhangyue19921010 have you been able to run the kinesis integration tests locally to see if those tests continue to pass after this change? There are some instructions on how to run them here - https://github.com/apache/druid/blob/master/integration-tests/README.md#running-a-test-that-uses-cloud

Hi @suneet-s Thanks for reminding. I just run the kinesis integration tests locally and succeeded.

Thanks for the contribution @zhangyue19921010 !

Thanks a lot :) @suneet-s and @clintropolis


About Example bluetooth/bluedroid/ble/gatt_server_service_table (IDFGH-9569)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
v5
Operating System used.
Linux
How did you build your project?
Command line with idf.py
If you are using Windows, please specify command line type.
None
What is the expected behavior?
According to table creation(esp_gatts_attr_db_t gatt_db[HRS_IDX_NB] ) All values connected to char_value(uint8_t char_value[4]).
Therefore, if I write a value to any character, I must read the same value in the other character. Because i am updating the value array. Since the other character also connect on it, I need to read the same value as I wrote.
What is the actual behavior?
Since the value array will be updated when I write in any of the writable characters,
I can't read the same value when I read any character. I always read the default value
Steps to reproduce.
just use default example
Build or installation Logs.
No response
More Information.
https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/bluedroid/ble/gatt_server_service_table/main/gatts_table_creat_demo.c#L172

/**

@brief           This function is called to set the attribute value by the application

@param[in]       attr_handle: the attribute handle which to be set
@param[in]       length: the value length
@param[in]       value: the pointer to the attribute value

@return

             - ESP_OK : success

             - other  : failed

*/
esp_err_t esp_ble_gatts_set_attr_value(uint16_t attr_handle, uint16_t length, const uint8_t *value);
/**

@brief       Retrieve attribute value

@param[in]   attr_handle: Attribute handle.
@param[out]  length: pointer to the attribute value length
@param[out]  value:  Pointer to attribute value payload, the value cannot be modified by user

@return

             - ESP_GATT_OK : success

             - other  : failed

*/
esp_gatt_status_t esp_ble_gatts_get_attr_value(uint16_t attr_handle, uint16_t *length, const uint8_t **value);

If you want to get and set the value of the server database, please use the following API.：
esp_err_t esp_ble_gatts_set_attr_value(uint16_t attr_handle, uint16_t length, const uint8_t *value);
esp_gatt_status_t esp_ble_gatts_get_attr_value(uint16_t attr_handle, uint16_t *length, const uint8_t **value);


Set fuse resource in K8s YAML template

This change sets resource limits and requests for fuse-client in K8s.

Build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/6307/
Test FAILed.

Closed due to conflict


Allow for "per task" grouping of open tabs.

The idea is that if you want to work on let's say 3 tasks which may involve 10 open tabs per task, to allow to group those tabs into task groups, and instead of having all 30 tabs open at all times, be able to only see the 10 tabs that are related to that task.
As an additional benefit, it will allow you to quickly build some sort of a reference for useful files based on a task, which can be used for future devs. Pretty much finding code based on topic.

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."

DO NOT DOWNLOAD ANYTHING I HAVE POSTED AS A COMMENT MY ACCOUNT WAS COMPROMISED AND BEING USED TO COMPROMISE OTHERS
I AM WORKING ON DELETING AND POSTING THIS ON ALL THREADS IN WHICH THEY USED MY ACCOUNT TO POST THIS

@nxllxvxxd good luck and protect yourself from the losers of the internet :)


Get all images in multiple image tweet

Change the file name to datetime.
It might differ from different timezone for the original timestamp is for UTC time. This change makes me find a specific image based on pages easily.

The first image name will be just the datetime and the followed will add the "_n", which n represents the number.

Get thumbnails from videos?
After checking the pages, I found the API will return some tweets without the media.
After adding the "tweet_mode" = "extended" and it can get those missing images.
Furthermore, it also gets the thumbnails of videos.

更改檔案名稱從unix timestamp為時間，單純方便人工查找跟閱讀
當一則Tweet有多張圖片時，取得所有圖片，命名方式為：第一張為時間，第二張之後會加 "_n"，而n代表第幾張。
新增影片的縮圖。
為什麼會修改這一點是因為發現API回傳時會有一些Tweet明明有圖片，但是並沒有夾帶回傳，而且查找document無果，碰巧看到有人使用者個參數解決其他API的問題，就嘗試加進去，結果不但圖片有回傳，影片縮圖也有回傳，所以算是瞎貓碰上死耗子。

This program is very useful and makes me don't have to save any image one by one.
It seems that I don't know much about how the "fork" function works. I originally just want to make a version on my branch so I might change some features just for myself, like the filename.
I've tested and compared the result to the pages which are shown on the browser. Hope some feature is useful.

@jimy811111
Great (太好了)! I confirmed that your code works well. Thanks for your contribution.

It's my pleasure:D.


Initialize or fix the transition matrix in CRF layer

Hi, I'm using keras lstm + crf to solve a NER task. I found that sometimes the crf layer outputs impossible label sequences such as B-Person -> I-Organization.
To eliminate impossible label transitions, I wanted to manually initialize or fix the transition matrix in crf layer, but couldn't figure out how. Is there any API to do such things?
Here is my code. Thanks for your help.
model = Sequential()
model.add(Embedding(VOCAB_SIZE, output_dim=EMBEDDING_OUT_DIM, input_length=MAXLEN, mask_zero=True))
model.add(Bidirectional(LSTM(HIDDEN_UNITS, return_sequences=True)))
crf_layer = CRF(NUM_CLASS)
model.add(crf_layer)
model.compile('adam', loss=crf_layer.loss_function, metrics=[crf_layer.accuracy])

me too, have you got any idea? @ersinyar I'm considering writing another loss function to regularize those impossible transitions


small banners

removing an anime on continue watching makes the 4th anime next to it small
Steps to reproduce the behavior:

Go to home>continue watching
remove an anime
See error

https://drive.google.com/file/d/1uUlzQoKVIp5xEfmqUT4xnQbH_Q8EBtDZ/view?usp=sharing

Android 11
beta 1.9.0
RMX3081

Additional context
( dont scroll left or right in the continue watching to successfully replicate the error )
YES ITs A TINY BUG

Not exactly the 4th banner depends with your phone width/screen size dpi


Feature Request

Summary of the new feature/enhancement

Proposed technical implementation details (optional)

Closing for now. If there are details you wanted to add, feel free to add the missing information and then request that this be reopened. 🙂


Update dcom-machine-launch-restrictions-in-security-descriptor-defini…

…tion-language-sddl-syntax.md
The option described in this article, deny permission for al DCOM object

Why

Closes #[Issue Number]

Changes

Closing as duplicate of #11657


Error while updating property 'fill' of a view managed by: RNSVGPath

I'm having this issue, saw that the exact same issue was solved time ago, but the solution is not working for me.
Bug
After installing react-native-svg using "expo install react-native-svg", I try to add the following Svg:
<Svg width={18} height={20} viewBox="0 0 18 20" >
      <G
        transform="translate(1 1)"
        stroke="#000"
        strokeWidth={2}
        fill="none"
        fillRule="evenodd"
        strokeLinecap="round"
        strokeLinejoin="round"
      >
        <Path d="M16 18v-2a4 4 0 00-4-4H4a4 4 0 00-4 4v2" />
        <Circle cx={8} cy={4} r={4} />
      </G>
</Svg>

and this error shows up:
Error while updating property 'fill' of a view managed by: RNSVGPath

null

java.lang.Double cannot be cast to java.lang.String

Tried uninstalling and installing several versions. Same thing with all of them.
Current one: 9.13.3

same issue here.

Having the same issue as well.
"react": "16.13.1",
"react-native": "0.63.1",
"react-native-svg": "12.0.3",

I use fortawesome svg pacakges, could be conflicting with react-native-svg on some degree. I need to investigate further to confirm.
"@fortawesome/fontawesome-svg-core": "^1.2.30",
"@fortawesome/free-brands-svg-icons": "^5.14.0",
"@fortawesome/free-regular-svg-icons": "^5.14.0",
"@fortawesome/free-solid-svg-icons": "^5.14.0",
"@fortawesome/react-native-fontawesome": "^0.2.5",
"@react-native-community/masked-view": "^0.1.10",

Having the same issue as well.
"react": "16.13.1",
"react-native": "0.63.1",
"react-native-svg": "12.0.3",

I use fortawesome svg pacakges, could be conflicting with react-native-svg on some degree. I need to investigate further to confirm.
"@fortawesome/fontawesome-svg-core": "^1.2.30",
"@fortawesome/free-brands-svg-icons": "^5.14.0",
"@fortawesome/free-regular-svg-icons": "^5.14.0",
"@fortawesome/free-solid-svg-icons": "^5.14.0",
"@fortawesome/react-native-fontawesome": "^0.2.5",
"@react-native-community/masked-view": "^0.1.10",

Giving an update, I was able to solve the issue by using the Expo's version of the package. That worked for me because I only use react-native-svg as a prerequisite for fontawesome packages.
"react-native-svg": "git+https://github.com/expo/react-native-svg"

Has anyone found a fix for this yet?

This is still a persisting issue and have not found a solution. Any advice would be appreciated.

experiencing the same issue. while installing (https://github.com/skrafft/react-native-jitsi-meet#android-install)[https://github.com/skrafft/react-native-jitsi-meet#android-install]

i get the same issue and using
"react": "16.13.1"
"react-native": "0.63.4",
"react-native-svg": "12.0.3",
and i tried using "react-native-svg": "git+https://github.com/expo/react-native-svg.git"
or
"react-native-svg": "9.13.1",
still i get the same error

l have same error.
"react": "17.0.1",
"react-native": "0.64.0",
"react-native-svg": "12.0.3"
and i tried using "react-native-svg": "git+https://github.com/expo/react-native-svg.git", "react-native-svg": "9.13.1",
still i get the same error.
did you solve it?
There was no error until yesterday, but now it doesn't.

Still having the same error, any updates on this issue?

Any update on this issue? I am having still same problem.

Same issue on android only.

👀 any update on this?

any updates on this?

I have the same problem here !
Here is my dependencies:
{   ""dependencies": {     "@react-native-async-storage/async-storage": "^1.17.10",     "@react-navigation/drawer": "^6.4.4",     "@react-navigation/native": "^6.0.12",     "@react-navigation/native-stack": "^6.8.0",     "expo": "~46.0.8",     "expo-dev-client": "~1.2.1",     "expo-status-bar": "~1.4.0",     "react": "18.0.0",     "react-native": "0.69.5",     "react-native-reanimated": "2.9.1",     "react-native-safe-area-context": "4.3.1",     "react-native-screens": "3.15.0",     "react-native-svg": "^13.1.0",     "validator": "^13.7.0"   },   "devDependencies": {     "@babel/core": "^7.12.9",     "react-native-svg-transformer": "^1.0.0"   }, }

if you are using expo, sometimes when you run yarn start you will see a message like "react-native-svg V-XX.X.X" is incompactible with expo, install version XX.X.X
So i only change the version of react-native-svg and delete and install dependences again

Hey guys! I was having the same issue here. Try installing react-native-svg packages again, it worked for me
1 - stop expo app
expo install react-native-svg
2 - rerun the app

After rerunning the app it showed a warning that my react-native-svg version is incompatible with the installed expo version and after installing the expected version it was fixed.

I think it should be resolved in expo as well as for bare apps in the newest version of the library. I will close this issue then. Feel free to comment here with repro if something is wrong and we can always reopen it then.

Still having the same issue. If it is resolved than any kind of reference would be appreciated.

Installing<EMAIL_ADDRESS>this particular version worked for me.

Thanks @tier3guy this saved me from this issue. A bit frustrating that we have to be 4 major versions behind just to get it working on Android, but this did solve the Error while updating property 'fill' crash for me.

Make sure no outdated version installed or different duplicate versions.
clear node_modules and package.lock / yarn.lock

V12.3.0 is work for me


Packaging for production ignores node-modules

Node modules don't seem to get packaged when running desktop-build. As a very simply example, if we install lodash and simply require it in index.web.js, creating a production build, though successful, resulting in an error when the executable is run (module lodash not found).
Not very familiar with webpack so simply reporting the issue for now -- if I do manage to fix the problem I'll do a pull request.

Sorry, I didn't get the chance to check the issue. How was it resolved?

The issue wasn't with the provided base; I have multiple entry points from main.electron.js which necessitates a slightly different configuration. My apologies I closed the issue without comment -- should have noted that your provided solution works perfectly as is.
Also, thanks for this project, greatly helped me in getting started!

Glad to hear it! Good luck with your project.


Warning on external build tool configuration

Hello,
I have a project which uses an external build system target. The main target references this external build target in the Target Dependencies section of its Build Phases.
At compile time I get the following warning:
warning: [R.swift] project.pbxproj is internally inconsistent.
 - XCConfigurationList ([some id string here]) is not used
Perhaps a merge conflict?

The warning references the id of the XCConfigurationList associated with the PBXLegacyTarget section of the external build system target in the project.pbxproj file. Is this an R.swift issue, or is there any way to circumvent this warning? I don't think there is anything broken in the project.pbxproj file.
I'm using R.swift 4.0.0 with Xcode 9.4 (9F1027a).
Thanks!

Could be that we detect this as an issue while it's not a problem in this case. It would mean that the XCConfigurationList is declared in the Xcodeproj file, but is never referenced from anywhere. I'm not sure if that is the default thing that happens with external build system targets.
It would help a lot if you can recreate the setup in a sample project so we can investigate it easier. Would that be something you could do?

I'll see if I can put something together and will update here when I do...

Here ya go!
R.swiftTest.zip

Thank you for this example project!
It seems XcodeEdit, our library for parsing xcodeproj files doesn't support LegacyTargets. I'll add that to the library and we'll update the R.swift dependency in the next R.swift release.

Awesome! Thank you!

We've merged the fix, this will be in the next release!

Do you have an ETA for the next release? :)

Just released R.swift 5.0.0.alpha.1 that contains this fix.


Strange energy sensors results using integration/utility_meter

He everyone, did you try to create energy sensors from the thermostat power entity ?  First, you need to create an energy sensor with the integration platform wich will transform raw power in W to kWh, it integrates the power over a time lapse to give kWh. Here is how I did it :
Here is the raw Hilo sensor data in graph :

https://www.home-assistant.io/integrations/integration/
1st step is to use sensor - platform: integration (takes a sensor in W) to create kWh
sensor:

platform: integration
source: sensor.chambre_d_emilie
name: energy_chambre_d_emilie
unit_time: h
round: 2
unit_prefix: k
method: trapezoidal

It creates the entity sensor.energy_chambre_d_emilie with the following attibutes :
state_class: total
source: sensor.chambre_d_emilie
unit_of_measurement: kWh
friendly_name: energy_chambre_d_emilie
icon: mdi:chart-histogram
device_class: energy
Here is the graph of the energy sensor created and it keep increasing over time as his job is just that : always increase over time and never reset.  It does go to 0 when you reboot HA just fyi.  Just look at the strange increase, it should not be that high and just start to rise from the same level as before...

https://www.home-assistant.io/integrations/utility_meter/
2nd step is to use this integration sensor to feed the utility meter
utility_meter:
energy_chambre_d_emilie_daily:
source:  sensor.energy_chambre_d_emilie
cycle: daily
It creates the entity sensor.energy_chambre_d_emilie_daily with the following attibutes :
state_class: total_increasing
last_reset: 2021-10-21T04:00:00.034194+00:00
source: sensor.energy_chambre_d_emilie
status: collecting
last_period: 5.11
meter_period: daily
unit_of_measurement: kWh
friendly_name: energy_chambre_d_emilie_daily
icon: mdi:counter
device_class: energy
Be aware that you need to switch the thermostat in heating mode and it need to apply power to change status to collecting.
Great seems to work, also in energy dashboard.
Unfortunately, I see the utility meter does the job to reset the sensor at midnight but for an unknow reason the power consumption is not beginning at 0, but at another number like if the last value was used by the integration sensor and is cummulating overt time ...

I did use the same method before on tuya smart plug with power measurement capability and it is working fine.
I use the same method for my main entry, with the sensor.smartenergy that creates sensor.enregy_smartenergy but since it always increase and never stop consumming, it is almost invisible!

Do you have the same behavior ?
Any idea is welcome!

It is confirmed.  You need to use the method: left

Trapezoidal is going to give you those strange results way too high cause it continue to increase even at 0.


https://clickhouse.yandex/ is broken

http://i.imgur.com/MXyBkT3.jpg

It works fine

On "traffic by minute" chart, we see no issues.
Probably, temporary connectivity problems.
Tell me if this issue still present.

hmm. its interesting.
May be my ISP not support very fresh domain name like .yandex

@alexey-milovidov all works fine via 3g.
But.
sudo apt-get install clickhouse-server-common clickhouse-client
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 clickhouse-client : Depends: clickhouse-server-base (= 1.1.54198) but it is not going to be installed
 clickhouse-server-common : Depends: clickhouse-server-base (= 1.1.54198) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

We have no pre-built packages for 17.04.
You could build packages from sources according to the instruction:
https://github.com/yandex/ClickHouse/blob/master/doc/build.md

thx

http://andreybolonin.com/clickhouse/clickhouse-client_1.1.54197_amd64.deb
http://andreybolonin.com/clickhouse/clickhouse-compressor_1.1.54197_amd64.deb
http://andreybolonin.com/clickhouse/clickhouse-server-base_1.1.54197_amd64.deb
http://andreybolonin.com/clickhouse/clickhouse-server-common_1.1.54197_amd64.deb
Working fine for ubuntu 17.04

I'm using EC2 ubuntu 16.04 and got the error
ubuntu@ip-172-30-10-137:~$ sudo apt-get install clickhouse-server-base clickhouse-server-common clickhouse-client -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
clickhouse-server-base : Depends: libc6 (< 2.20) but 2.23-0ubuntu9 is to be installed
E: Unable to correct problems, you have held broken packages.

@seufagner,  16.04 ubuntu:
sed -i 's/trusty/xenial/' /etc/apt/sources.list.d/clickhouse.list
apt-get update
apt-get install clickhouse-server-base clickhouse-server-common clickhouse-client -y


Builtin testing🧪

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

175 of 175 (100.0%)  changed or added relevant lines in 7 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-1.01%) to 73.498%

Totals

Change from base Build<PHONE_NUMBER>:
-1.01%

Covered Lines:
1859

Relevant Lines:
2135

💛  - Coveralls


Improve test coverage for nc4dim.c

Test coverage for nc4dim.c is already 85% which is pretty reasonable. But I see some things that are not being tested, like bad parameter values for all functions.

These new tests have been merged. Thanks Ward!


Correct user id type from string to ArrayBuffer (compatible with BufferSource)

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (8f6c8b0) 92.85% compared to head (398565f) 92.85%.

:exclamation: Current head 398565f differs from pull request most recent head 751e761. Consider uploading reports for the commit 751e761 to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##           master     #148   +/-   ##
=======================================
  Coverage   92.85%   92.85%
=======================================
  Files          16       16
  Lines        6019     6019
=======================================
  Hits         5589     5589
  Misses        430      430

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


fix: replaces references to OARS with Aspects in the ADRs

Description
Updates the ADRs to replace references to "Open Analytics Reference System (OARS)" with "Aspects Analytics system (Aspects)".
Supporting information
Part of https://github.com/openedx/openedx-aspects/issues/61

Thanks for the pull request, @pomegranited! Please note that it may take us up to several weeks or months to complete a review and merge your PR.
Feel free to add as much of the following information to the ticket as you can:

supporting documentation
Open edX discussion forum threads
timeline information ("this must be merged by XX date", and why that is)
partner information ("this is a course on edx.org")
any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.
Please let us know once your PR is ready for our review and all tests are green.

@pomegranited 🎉 Your pull request was merged!  Please take a moment  to answer a two question survey so we can improve your experience in the future.


"show search bar" , "show search filter", "show 6 media thumbnail (under search bar)" option , Preferred width of Twitter feed - 900 pixels, 1000 pixels ,...

Hi! I need "show search bar" , "show search filter", "show 6 media thumbnail (under search bar)" option ,
Preferred width of Twitter feed - 900 pixels, 1000 pixels ,...
plz plz .. thank you so much

I support this as well, either search bar or button, as it has disappeared.
Thanks, the plugin looks great

I support this as well, either search bar or button, as it has disappeared.
Thanks, the plugin looks great

me too. there is nothing to do when you want to search but to deactivate the extension.

More width options and search is better in 4.0: https://github.com/thomaswang/minimal-twitter/releases/tag/v4.0.8

Thanks for the update ✨ I look forward to when it is available on the Firefox add-on library.


Check for regular file in Package.makeJar

Fixes #3010.

The build failure in safeUnitTests doesn't seem to be caused by me.


Upgrade instances to Amazon Linux 2014.09 v1.2.0

Upgrade Digital Marketplace instances running in Elastic Beanstalk to Amazon Linux 2014.09 v1.2.0.

How have these been tested?

They haven't been yet, @robyoung. That was the reason these haven't yet been merged.
I don't know how you do your testing, other than I suppose bringing up new Beanstalk instances.

I think that would be it. We have some smoke tests but they haven't been wired up yet. I would think that i we can deploy an app to the new instance type and the smoke tests pass then it's probably going to be ok.

As soon as we have URLs for the deployed apps we can easily point the smoke tests at them to run - configurable in the Jenkins job.

I've tested this in preview and it seems to be working with admin-frontend and api.
We had problems with deployment when moving from 1.0.9 to 1.1.0 and it appears to be ok with 1.2.0, so I'm :+1: to merge.

\o/


Python: Anthropic function calling fixes

Motivation and Context

Description

Contribution Checklist

[ ] The code builds clean without any errors or warnings
[ ] The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
[ ] All unit tests pass, and I have added new tests where possible
[ ] I didn't break anyone :smile:

Python Test Coverage Report • FileStmtsMissCoverMissingsemantic_kernel/connectors/ai/anthropic/services   anthropic_chat_completion.py158696%141, 147, 159, 165, 169, 375   utils.py32391%56, 90–93TOTAL16759185789%
Python Unit Test Overview

Tests
Skipped
Failures
Errors
Time

2966
4 :zzz:
0 :x:
0 :fire:
1m 23s :stopwatch:


Template Versioning - Refactoring template model

Renamed scope methods:

archived instead of valid,
latest instead of current
published instead of live. NOTE, published scope method could accept family_id to merge live and published methods into just one.

Adds customize class method to accommodate the customization action for templates controller.
Adds transfer_customisation class method to accommodate the transfer_customization action for templates controller
Adds keep_or_generate_version(family_id) class method to be reused across multiple controllers (e.g. template, phase, section, question, annotation)
deep_copy add options as an splat argument to tweak the cloning (e.g. modifiable flag, published, is_default, etc)
refactor set_creation_defaults method, i.e. remove the random generation of dmptemplate_id

when and where would the family_id/dmptemplate_id be created if we remove that functionality from the set_creation_defaults method?

the family_id is nil for a new template and for a versioning is the column id autogenerated by the db. Does it make sense?

also update the Plan model and remove the 'dmptemplate' it seems unnecessary (also update any references to it!) and move the 'base_template' method from there to the Template model if its even necessary.

closing issues that can't be tested in the UI


Not Using CUDA ?

The camera version runs better in windows than linux.(although the FPS is better in linux than windows, i believe it is because you are using parallel programming)
Linux
Ubuntu 16.04 Bionic
Nvidia GTX1060 16GB,
8GB RAM
CUDA and Cudnn
Windows 10
8GB RAM
No CUDA
Is it not using CUDA or what ?
Need help making it faster.

Unfortunately, we would be glad if you contribute the project.

Happy to do so !

Hey ! I ll be needing the data to be retrain the model.

You are right, but there is private information in the dataset. Shall we create a public dataset project? Sounds great!


Fixing and Updating the UI of Share your experience box

Fixing the share your experience for better user interface

@AsifQamar submit button is missing , add that too

@AsifQamar submit button is missing , add that too

Added, check my PR please and merge it

I'm also a contributor like you bro 😁 I can't merge 😁 I was just about to report the issue, then found your issue .

@AsifQamar submit button is missing , add that too

Added, check my PR please and merge it

I'm also a contributor like you bro 😁 I can't merge 😁 I was just about to report the issue, then found your issue .

Oh ok


Support > 4D permute conversion

Ticket
#459
Problem description
ttnn.permute can now support > 4D permute with https://github.com/tenstorrent/tt-metal/commit/b80a975f1d480c1deb5cf2e520c9797c058c0903. Remove the guard and add temporary workaround for the issue https://github.com/tenstorrent/tt-metal/issues/16188
The wrong shape on 3D permutation is also fixed now
What's changed

Remove > 4D guard and 3D shape workaround
Enable tests

Ho @swimdi , in the last commit https://github.com/tenstorrent/pytorch2.0_ttnn/pull/683/commits/297606ded699fe68cde7109b29fdb737021337a7 I'm trying to fix some wrong shapes being propogated on meta["val"]. Could you help me verify if the change makes sense to you?

Ho @swimdi , in the last commit https://github.com/tenstorrent/pytorch2.0_ttnn/commit/297606ded699fe68cde7109b29fdb737021337a7 I'm trying to fix some wrong shapes being propogated on meta["val"]. Could you help me verify if the change makes sense to you?

LGTM


Make scRNAseq queries live (EPIC)

Pushing scRNAseq data queries live: Can we do it this week?

Done (but double check status)

Data already live in DB
Working queries in place

Add query hooks to neurons (driven by scRNAseq flag) - already done?

Hack in via Cypher.  Don't wait for another DB build.
MATCH (primary:Class:Anatomy)<-[r:SUBCLASSOF*0..]-(a:Class:Anatomy)<-[r1:composed_primarily_of]-(c:Cluster:Individual)-[r3:has_source]->(ds:scRNAseq_DataSet:Individual) SET primary:hasScRNAseq
Query text for query for clusters: "Single cell transcriptomics data for  {%anatomy}"
Query details for XMI - run subClassOf query in OWLERY to generate input

TODO

Hack in DataSet label to node with scRNAseqDataSet => fix queries for data and license on cluster TermInfo.  @Robbie1977  to add
Implement suppression of display of relationship edges for all edges in Relationship TermInfo with hide_in_term_info: ['True'].  @dosumis  to look at rolling code.
Query hook for clusters - can just use "Cluster" for now - but in future it would be better to have an scRNAseq  label as we may well re use cluster for other purposes.  QUery tet: Genes expressed in ${cluster}
Changes to XMI - direct neo4j query - just takes ID of cluster (Individual)
Code needed to translate queries to results tables

@Robbie1977 has written - will link PR to this ticket.
columns as in https://github.com/VirtualFlyBrain/VFB_json_schema/issues/112#issuecomment-1283675439 - bit first table has publication and second table has cell_type.

Can we make sure code edits are not held up by clashing updates for other purposes?

MATCH (primary:Class:Anatomy)<-[r:SUBCLASSOF*0..]-(a:Class:Anatomy)<-[r1:composed_primarily_of]-(c:Cluster:Individual)-[r3:has_source]->(ds:scRNAseq_DataSet:Individual) RETURN * LIMIT 1

@hkir-dev  to work on moving anatomy -> cluster query to SOLR.

Items for follow-on sprint(s):

Update to latest build

Fix queries to cope with schema change
Fix xref-roller so that it generates multiple xrefs when accession is list with n > 1.


[🐛 Bug]:  Ensure ALL supported types are exposed to DevBindingsOptions

next-on-pages environment related information
No response
Description
DevBindingsOptions seems to be missing types around queues. I'm not sure if there are others (AI?).
Is there a reason all of Miniflare's supported binding types aren't listed?
Reproduction
No response
Pages Deployment Method
None
Pages Deployment ID
No response
Additional Information
No response
Would you like to help?

[ ] Would you like to help fixing this bug?

@admah it's not just the types but the missing bindings are actually not currently supported.
This is something was aware of, I did not add them for various reasons, mainly because their addition here (although not extremely hard) would not be trivial, so I thought that it would make more sense for now to include the current common and simple to support ones and to then add support for all of bindings when we use getBindingsProxy instead (since I am working on both these utilities I think I should just focus my efforts on getBindingsProxy instead of trying to implement the same logic twice, only to then deprecate the next-dev version).
For some context, I did implement the AI binding in #597, it worked fine, but again, it sounded safer/cleaner not to go with such implementation but wait for getBindingsProxy instead.
Regarding queues, they are not supported in wrangler pages dev that's why they haven't been added to next-dev (see #526). They being supported by next-dev but not wrangler pages dev doesn't really seem right to me (since as mentioned in the suggested workflow I think that ideally developers should be using both next-dev and wrangler pages dev).
So my suggestion here would be to wait for #623 to be resolved, once it is we can then revisit this issue (which should hopefully be already solved).
What do you think? does that sound ok to you? or would you prefer a different approach?
(note: I think that releasing getBidingsProxy should not take more then a couple of weeks maximum)

Thanks for the additional context here @dario-piotrowicz! I think continuing to focus on getBindingsProxy is the way to go.
Also agreed that queues support should be in both so we can revisit that later 👍


Implement difference_with operation

With this change, we'll now have the full suite of in-place set operations.

lgtm


Minor change to healthcheck url causes error

I am making the following change in the deployment.yam file:
livenessProbe:
           httpGet:
-            path: /ignite?cmd=version
+            path: /ignite?cmd=top&attr=false&mtr=true
             port: {{ .Values.service.restSecondPort }} # REST port number.

And am getting the following error:
helm upgrade -i ignite-dev . --namespace ignite-dev
Error: UPGRADE FAILED: failed to create patch: unable to find api field in struct Container for the json field "httpGet"

When I revert, the error disappears.
Output of helm version:
Client: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}

Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:13:03Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.1", GitCommit:"b1b29978270dc22fecc592ac55d903350454310a", GitTreeState:"clean", BuildDate:"2018-07-17T18:43:26Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.):
K8 on baremetal.

At first glance, this looks like an indentation issue on httpGet. It should be under the livenessProbe key indented two spaces from the left. Let me know if that solves your problem.
The helm lint <chart> command can ensure that a chart is well formatted. You may also want to make use of helm upgrade -i --dry-run --debug. Those flags (dry-run and debug) will give you an idea of what to expect when running a command without actually running those actions and also output the kubernetes manifests that would be installed.

I'm going to close this ticket due to inactivity, but please re-open if this still needs to be addressed. Thanks!


indicate default settings in the settings.json.

Type: Feature Request
Problem Statement

its been a long time since i started using VSCode and started editing the settings.json to change the settings and personalize my instance.
over time over time i realize that option a(default) in setting is better than option b(my custom change). and switched back to option a.
i have a long long settings.json and lots of settings which are actually pointing to the default setting config.

Expected: An option/indicator to identify the default settings in the settings.json in order to clean up the file.
VS Code version: Code - Insiders 1.74.0-insider (e764c5b816a99eb8673b2fc4fc028827f202872f, 2022-11-04T05:18:32.818Z)
OS version: Windows_NT x64 10.0.19044
Modes:
Sandboxed: Yes

Hi, is the request to add something like a yellow underline for settings where the configured value is the same as the current default value?
Something else to consider is that the default value of a setting sometimes changes between different versions of VS Code, especially if it is an experimental setting.

Yes, having a yellow underline would be great.

Is it possible to dim those settings instead? Also, it would be great if vscode could provide a code action to remove all of them.


Authentication Failed

Getting "Authentication Failed" when trying to connect
Password being used is from a password generator: TPAklk1UlV56%5PSi7^

I can see request is actually getting through to the server as it is spinning up an rcon thread

Server Properties:

Side note: Tried logging in with https://play.google.com/store/apps/details?id=com.jakubtomana.hypercon.minecraft&hl=en&gl=US and it works fine

Hello,
I did some testing yesterday and so far I couldn't reproduce the problem on my setup.

What version of server you are using?
Which operating system does the server run on?

Hello,
I did some testing yesterday and so far I couldn't reproduce the problem on my setup.

What version of server you are using?
Which operating system does the server run on?

Running forge-1.16.1-32.0.108 on AWS EC2, OS: Amazon Linux 2 AMI (ARM)

Hi,
This is just a guess but I am suspecting that Windows command line is doing something weird with the special characters in the password.
Try to put the password inside quotes and see if it works that way: mcrcon -H host -p "TPAklk1UlV56%5PSi7^" "say test"

mcrcon -H host -p "TPAklk1UlV56%5PSi7^" "say test"

Wrapping in " makes it work

I used this method, but it still shows "authentication failed!". How should I fix it?


on ios files are opened, but immediately closed themselves

On Android all work fine.
But when i open file on ios - it opens for a split second and then closes itself. There are no errors in the logs

Same here. RN 0.61

Can you send a snippet of your code?

I solved the problem. As far as I understand under ios, you are calling a modal window, which conflicts with my code. I made a delay(300ms) to start the opening of the  file so that my modal windows managed to close correctly

it's not a good idea, adding delay before opening file, do you have better solution?

I can reproduce this issue and use this onDismiss method can avoid the file-viewer window automatically close  immediately
`
import React, {PureComponent, Component} from 'react';
import {Platform, Image, Text, View, TouchableWithoutFeedback, Dimensions, StyleSheet} from 'react-native';
import Modal from 'react-native-modal';
import FileViewer from 'react-native-file-viewer';
import RNFetchBlob from 'rn-fetch-blob';
export default class TestScreen extends PureComponent {
constructor(props) {
super(props);
this.tabIndex = [];
this.state = {
isFileFetching: false
}
}
render() {
return <>

<TouchableWithoutFeedback onPress={()=>{
this.setState({isFileFetching:true});
    const filePath = downloadsPath.concat('/', fileName);
    RNFetchBlob.config({path: filePath}).fetch('GET', 'https://xxxx.com/yyy.pdf')
    .then(() => {
      FileViewer.open(filePath, {
        showOpenWithDialog: true,
        onDismiss:()=>this.setState({isFileFetching: false})
      }).catch((error) => {
        this.setState({ isFileFetching: false });
      });
    })
  }}><Text>download and open </Text></TouchableWithoutFeedback>
</>

}
}
`
I assume you also use the react-native-modal for the downloading spinner, I don't know if there is some conflicts between react-native-modal and react-native-file-viewer, if you FileViewer.open() right after setting isVisible=false, the FileViewer will be close automatically.
you can use the delay(300ms) or close the spinner in onDismiss callback function. I think the last one is better


FIX: Dark mode UI Contrast

Fixed misaligned navigation links when content is empty.
Fixed multiple contrast problem in dark mode:

Before ❌
After ✅
Contrast Result (dark mode)

Fixes #57
Fixes #164

How do I resolve this error in tests?

How do I resolve this error in tests? I think this is a permission error related to the automated tests and not from the changes I made.

I think we can merge this @aggre?

Will this region be light mode, even in the dark theme?

 Will this region be light mode, even in the dark theme?

Yes. Do you want me to invert this image color (in dark theme) instead?

Hi Simperfy, thanks for your contributions. Sorry, this was delayed because we navigate to new docs (docusaurus), will close this issue now. Feel free to contribute if u found something.

Hi Simperfy, thanks for your contributions. Sorry, this was delayed because we navigate to new docs (docusaurus), will close this issue now. Feel free to contribute if u found something and chat the pr or issues link on docs channel,

It's okay, no problem.


[Version Bump] v1.3 with upstream update

Docusaurus is bumped to v3.5.1 from upstream.
easyops-cn/docusaurus-search-local is bumped to v0.44.5 to accommodate docusaurus update.
Fix typo in AGR, search is now available on AGR as well.

[Version Bump] v1.3 with upstream update
Docusaurus is bumped to v3.5.1 from upstream.
easyops-cn/docusaurus-search-local is bumped to v0.44.5 to
accommodate docusaurus update.
Fix typo in AGR, search is now available on AGR as well.


Cannot activate theme

I cannot find the DevStyle tab in the version 2023-03. Is there another way to activate theme?

Same issue

Fixed by installing the prerequisite Darkest Dark.

Fixed by installing the prerequisite Darkest Dark.

Thank you sir, this should be mentioned on the Dracula install instructions BTW.


⚠ Bofrost.de has degraded performance

In 30b4049, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 2034 ms

Resolved: Bofrost.de performance has improved in 883b210.


Requests to Use CSV Uploader

This Epic is used to contain all stories related to requests from senders to participate in the CSV uploader pilot
Workflow for requests:

A request comes into the RS Inbox
Adrian (who solely monitors the inbox) creates a ticket under this Epic

Assigns it to both the O&O and Experiences Team
@ calls out the following folks in a comment:  Rick Hood, Victoria, Rachel, Lori
@ calls out the following folks in a slack: Rick Hood, Victoria, Rachel, Lori

Adrian moves the email to the "CSV Uploader" folder in the RS inbox
Rick Hood responds to the request with an email & link to the TOS agreement form
Adrian monitors the RS inbox for TOS agreement emails & let's Rick Hood know
Rick Hood configures the sender in RS
Rick Hood configures the sender in OKTA & triggers the activation email
Rick Hood responds to the TOS agreement email with a welcome email and links to the account registration and CSV uploader guides in RS
Adrian monitors the RS inbox for onboarding questions and notifies Rick Hood, Victoria, Rachel, Lori
Rick Hood responds to onboarding questions via email
Victoria captures metrics as necessary from onboarding pilot participants
Victoria captures feedback as necessary from pilot participants

@Adrian-Brewster @victorialb-usds I captured the basic WF as we discussed yesterday; we can look at this together on Wednesday with the others

@rhood23699 @Adrian-Brewster @victorialb-usds @rachelhanster
New request to be onboarded; this was in response to an email that CDPHE sent them inviting them to sign up


Styling enhancements for header and footer

[ ] use more space for header and footer
[ ] make icons larger (main logo, toolbar icons)
[ ] clean up header text styling

yeerrrrrrrrrr


[feat]

Describe the problem
I want to run an shell Code ./index-linux on start tauri-app
Describe the solution you'd like
I want to run an shell Code ./index-linux on start tauri-app
Alternatives considered
No response
Additional context
No response

I want a method such that when I launch the application I can run a shell command that will start the express server.

I want a method such that when I launch the application I can run a shell command that will start the express server.

Maybe this might point you in the right direction
https://tauri.app/v1/api/js/shell/


Tool for backfilling datetime column in pinot segments

Adding a tool in pinot admin, for the backfill of pinot segments with dateTime column.
The tool expects table name, timeSpec and dateTimeSpec as inputs, and will go over all pinot segments to backfill them with dateTime column, with a value corresponding to the time column value.
Updated segment metadata, to include information about date time columns.
If the segment consists of star tree, only raw docs are read, and star tree is generated by treating date time columns as dimensions and skipping them from default split order
Testing done:
Segments used:

Download a single segment, and backup and backfill it with date time
Backup and backfill of all segments of a table
Error generated if segment name provided does not exist

Segment configs:

Backfilled an existing segment - ensured that segment is queryable with new datetime
Backfilled an exsiting segment, with star tree - ensured that star tree was generated, and segment is queryable with new datetime
Backfilled an existing segment, with same dateTime column already present - ensured that segment is queryable with new datetime
Backfilled an existing segment, with same dateTime column already present, but differnet datetime column configs - ensured that segment is queryable with new datetime

Segment generation:

Generated segments from AVRO, JSON, PINOT
Generated segments from AVRO, with star tree config
Generated segments from AVRO, with dateTime in schema
Generated segments from AVRO, with dateTime in schema, and star tree enabled
Verified that these segments upload correctly, and can provide new datetime whenever available. The older segments and backfilled segments are all queryable together.

Query:
Queried for sum of metrics on backfilled segments, vs non backfilled segments, and verified the totals match
End to end:

Downloaded segment, backfilled, uploaded. Queried on segments to check for new column.

After this change is pushed, segments can be generated with the new datetime column, by specifying a dateTimeFieldSpec in the segment generation schema
All older segments can be backfilled using the tool.

I will review it, but I have a question. What happens if the segment is refreshed between the time it is downloaded and re-uploaded (after conversion)?
Perhaps you should introduce a refresh-if-crc-equals API in the controller?

This tool is doing the following

Download the segment,

upload the segment again.
Conceptually this is very similar to what a Pinot Minion does.

It will be great if we can model this as an ad-hoc or standalone Minion tool. I haven't looked into the interface of a Task in minion but if we can use the same interface to implement this task of adding date time column then we can leverage this tool to run any minion task.
@Jackie-Jiang @npawar what do you think?

@kishoreg As discussed offline, there are currently multiple constraints to model this as a minion task and run it via a tool that runs adhoc minion tasks. The task executor uses minionContext which is created using cluster configs on startup. We will need to duplicate minion setup steps to make this work. There's also some configs needed by pinotTaskConfigs which we will have to manually construct.
As a result, keeping it independent of the minion as a separate tool seems like the best option right now.

Codecov Report

Merging #1890 into master will decrease coverage by 0.1%.
The diff coverage is 40.96%.

@@            Coverage Diff             @@
##           master    #1890      +/-   ##
==========================================
- Coverage   69.58%   69.47%   -0.11%
==========================================
  Files         833      834       +1
  Lines       40453    40611     +158
  Branches     5421     5440      +19
==========================================
+ Hits        28148    28214      +66
- Misses      10427    10513      +86
- Partials     1878     1884       +6

Impacted Files
Coverage Δ

.../linkedin/pinot/common/data/DateTimeFieldSpec.java
78.04% <ø> (+4.87%)
:arrow_up:

...n/pinot/core/segment/creator/impl/V1Constants.java
32.14% <ø> (ø)
:arrow_up:

...ot/core/data/readers/PinotSegmentRecordReader.java
69.04% <0%> (-1.69%)
:arrow_down:

...indexsegment/generator/SegmentGeneratorConfig.java
58.73% <100%> (+0.16%)
:arrow_up:

...ment/creator/impl/SegmentColumnarIndexCreator.java
85.65% <16.66%> (-1.69%)
:arrow_down:

...kedin/pinot/core/segment/index/ColumnMetadata.java
79.25% <40%> (-4.62%)
:arrow_down:

...edin/pinot/core/minion/BackfillDateTimeColumn.java
40.44% <40.44%> (ø)

...inkedin/pinot/common/data/TimeGranularitySpec.java
75% <57.14%> (-1.93%)
:arrow_down:

...in/pinot/core/startree/OffHeapStarTreeBuilder.java
82.88% <6.25%> (-2.87%)
:arrow_down:

.../pinot/core/segment/index/SegmentMetadataImpl.java
77.85% <66.66%> (-0.23%)
:arrow_down:

... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c7ba3b...204028a. Read the comment docs.


[native filter] filters are displayed as out of scope if there are tabs on dashboard

filters are displayed as out of scope if there are tabs on dashboard
How to reproduce the bug
1,Open any dashboard with tabs (e.g. use `Sales Dashboard) or use your own
2,Create a native filter (e.g. time range) and apply it to all panels
3,Save the filter
4,Go to the "Dashboards" page
5,Open the dashboard from step 1
6,Pay attention to the native filter bar
Expected results
the scope of the filters has not changed for dashboard with tabs
Actual results
filter is displayed as out of scope on if there are tabs on dashboard (after re-visiting the dashboard), see the video below:
Screenshots
https://user-images.githubusercontent.com/81597121/149433547-fcd17a72-866a-4110-8904-c442965efb8e.mp4
Environment
(please complete the following information):

browser type and version:
superset version: master
python version: python --version
node.js version: node -v
any feature flags active:

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[x] I have reproduced the issue with at least the latest released version of superset.
[x] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context
Add any other context about the problem here.

Verified fix on superset master


Changing Populate to take an IRegistry instead of Registry

And removing overload taking ConfigurationExpression.
This is a breaking change, see #26

I'm wondering if the switch shouldn't be made also here and use IRegistry all the way down.

I'm wondering if the switch shouldn't be made also here and useIRegistry all the way down.

Yeah, also, the IServiceProviderFactory should also be IRegistry.

Looks like this will not work in the end... The IRegistry abstraction in only half backed in StructureMap.
The Container constructor takes a Registry and not an IRegistry, so do all the method below that it uses... :-(

The Container constructor takes a Registry and not an IRegistry, so do all the method below that it uses...

Yeah, I think I went down that road at some point and came to the same conclusion 😢

Ok then, let's close this.


denoising on uncompressed sound demo

Needs a new function resample_denoise_resample_chunk() to pipe it through the denoiser and resample back to 44.1k to be played back through the push_audio_chunk() function.
Unfortunate that we need this function just for this demo, and that it has to be recomputed on play, but it's all I got to keep it simple.

This has been done


chore: store examples in YAML

In this PR:

Terraform and CloudFormation examples moved from cf.go and tf.go to yaml,
all checks now refer to the yaml examples
added command for formatting examples

Although all the examples for a single check are in a single file, we need to reference it from multiple check fields. I suggest adding a new examples field that will point to a single file, while maintaining support for the current method of referencing examples for backwards compatibility.
Old example files will be deleted in another PR as they are still referenced by Go checks.

Will be ready for review after the release of Trivy v0.57

One thing to keep in mind with the examples is the avd generator. I believe it uses the go files as well. Could you confirm that @nikpivkin?

@simar7 I'll check, but as far as I remember avd-generator uses generated md files.

avd-generator uses only generated .md files https://github.com/aquasecurity/avd-generator/blob/main/docGen/defsec.go#L151-L161

Maybe we should change the file schema to the following?
id: avdid
examples:
   terraform:
   cloudformation:

This will allow us to associate examples and checks without using the trivy and trivy-checks packages to run the tests. To do this now, we need to load all the checks and then load all the examples based on the check metadata.

Maybe we should change the file schema to the following?
id: avdid
examples:
   terraform:
   cloudformation:

This will allow us to associate examples and checks without using the trivy and trivy-checks packages to run the tests. To do this now, we need to load all the checks and then load all the examples based on the check metadata.

It makes sense to me but we should make an announcement regarding this change as we will be changing the metadata schema. I suggest let's do it in a separate PR.


mapic: multistream: question mark support

LGTM
@gioelecerati Did you check in the box if it actually works (both cases: with and without ?)?

Yeah it's working in both cases in the box


Switch to GitHub Actions

Based on this example: https://github.com/actions/setup-elixir#matrix-example

Hooray! Thank you @nathany :tada:


Add url field to Message

Add an optional url field to Message, this is useful if you want to use django-stored-messages for notifications where you usually have an url pointing to the event.
While at it the create_message api is expanded to support passing the date.

I've updated the pull request to sync broadcast_message with add_message_for changes


WIP of the Kafka consumer wrapper for Subbu

This is what the consumer state machine will look when fully implemented.

please add this to the design doc in the open source, as well as to the doc
we shared and marked as FINAL Realtime design. thanks
-subbu
On Tue, Aug 2, 2016 at 4:44 PM, Jean-François Im<EMAIL_ADDRESS>wrote:

This is what the consumer state machine will look when fully implemented.
[image: Kafka consumer state machine]
https://cloud.githubusercontent.com/assets/246807/17349136/4931381e-58d0-11e6-8d4c-f56f185c8f60.png
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/linkedin/pinot/pull/338#issuecomment-237084698, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACLGfRAUNdkDk9_xVSnuvPHaXZGP4mF5ks5qb9ZdgaJpZM4JbKOw
.

Current coverage is 45.88% (diff: 0.00%)

No coverage report found for master at b6e7ea8.

Powered by Codecov. Last update b6e7ea8...a8e2aa6

Overall, I think the approach is workable. Here is what I would prefer:
A state actually does what it is named after (i.e. if it says it is trying to find a bootstrap server, it should be doing that without the need to call 'think'). Alternatively, the state can be named after its pre-condition, and an input can decide what that state does, and which state it goes to next. That would make the state machine more intuitive to understand, IMO. If you can make common methods out of error handlers, that will be great.
Please add unit tests in your final version, mocking kafka as needed (or, overriding it with a fake kafka). Best if unit tests can cover error conditions as well.
In order to support the segment completion protocols, the consumer should have methods like 'start consuming', 'stop consuming and give me the offset', 'catchup to consume upto this offset (exclusive)'. I think you are in the right direction to be able to add these methods, but just wanted to remind you. We can then build segment completion on top of the kafka consumer (and test the segment completion part with a fake kafka consumer wrapper!). The consumer code and the helix state machine code that has been validated is in https://github.com/linkedin/pinot/blob/LLConsumerPrototype/pinot-integration-tests/src/test/java/com/linkedin/pinot/validation/PartitionConsumer.java

I'm not sure how the state machine would work without having a method that's invoked periodically; there's no need to have multiple threads for this and I'd like to avoid synchronization between the Pinot code and some thread that's only there to make sure that on disconnections/connection establishment/leader re-election, we connect to the right node. If you have a good idea of how to make it work without that method and with no external thread, then I'm all ears.
With regards to the naming of the states, they do what the class is named for. Since there's no concurrent or asynchronous execution, the name of the state represents what is going to be done next (eg. if it's in connecting state, the next invocation of think() will do the actual connection).
There is no need to have the methods you have mentioned. All that this class does is process requests to consume between offset ranges and ensure that the client is connected to the appropriate Kafka broker for a given partition. The constructs that you have mentioned are something that should be handled in the Pinot realtime consumer, not the Kafka one. This class is only to separate the Kafka-specific details from the Pinot ones.
With regards to unit test, I'll see what I can do without having to create a Kafka consumer factory class and introduce additional complexity.

pls add unit tests at some point.

lgtm otherwise


Generator use unexpected dynamic

Describe the bug
I used concrete type as provider's build method parameter but generator regard it as dynamic.

@freezed
sealed class Video with _$Video {
  const factory Video.url(Uri uri) = IvsUri;
  const factory Video.vod(String id, {num? episodeId}) = VodId;

  const Video._();
}

@riverpod
int some(SomeRef ref, VodId vodOrEpisode) {
  return 3;
}

@riverpod
class Another extends _$Another {
  @override
  int build(VodId vodOrEpisode) { // 'Another.build' ('int Function(VodId)') isn't a valid override of '_$Another.build' ('int Function(dynamic)').
    return 3;
  }
}

To Reproduce
git clone -b riverpod-dynamic https://github.com/jeiea/flutter_example.git
cd flutter_example
dart run build_runner watch

Expected behavior
Code should be compiled.

Relying on generated types isn't supported. This is a known limitation of code generators. It's hard to bypass.
I'm working on something though. We'll see. But in any case, it's a duplicate, so I'll close it


Hello_There.c

Please merge
Linked Issue
If you have a pull request related to a current issue please link to that issue number.
That issue can be linked to the pull request by using the side panel in the Github UI or using the # symbol followed by the number of the associated issue.
To link a pull request to an issue to show that a fix is in progress and to automatically close the issue when someone merges the pull request, type the keyword "Closes" followed by a reference to the issue. For example, Closes #1977 or Closes ClJarvis/hello_There/issues/1977.
Description
What are you adding to the repo? What language are you writing "Hello There!"
Code of Conduct
By submitting this pull request, you agree to follow our Code of Conduct

Please Merge

This issue was assigned to someone else please don't work on issues until they are assigned to you.
Please read the contributing.md and code of conduct.
From contribute.md

Create a new branch titles Hello_There.extenesion, with the extension being the language you are adding. (EX Hello_There.py)


Fixed #24994 -- Documented, checked that settings.SECRET_KEY is valid unicode string

Documented that the settings.SECRET_KEY should be valid unicode, and checked
that it does indeed pass our expectations. In case of failure, it emits a
readable warning.

Closing due to inactivity. Please send a new PR if you can update it per the comments.


Make it clear that overridden methods may be overriden again

To make it clear why the 'override' word used in "virtual, abstract, or override" phrases in the text.
Summary
Describe your changes here.
Fixes dotnet/docs#Issue_Number (if available)

closing due to the lack of a response.
@volkan-yazilim If you want to continue working on this, you can reopen this PR, and tag me for a new review.


Widget ID (documentation hiccup)

The widget ID described in README.md (a_timeline) does not match the ID being sent in jobs/timeline.rb (ags_timeline).

Thanks, fixed.


Option to resize 198p to 360p during concat

The most common resolutions are 360p and 198p. Add an --aggressive (or just --resize) option to concat to resize 198p videos to 360p, in order to reduce the number of output videos.
The Problem:
Occasionally, during times of high network congestion or for whatever other reason, showroom streams get reduced from 360p to 198p. Such streams are blocky and almost unwatchable, and usually don't last very long. However, the audio is still intact. Unfortunately, they can't be merged with 360p streams without being re-scaled (and losing data).
The Old Solution:
When I originally wrote concat.py, I opted to keep 360p and 198p separate rather than rescale in order to preserve as much of the original video data as possible. However...  occasionally this results in a single stream getting split into 10+ videos of alternating 360p and 198p, and it's not like rescaling is going to make the video look noticeably worse.
The New Solution:
Add a --resize option to concat.py to complement the --generate and --merge options, and an --aggressive option that runs all three (--both will still only run --generate and --merge). --resize will work as follows:

scan the directory for 198p files
rename each 198p to resized/_198p.mp4
rescale each 198p to 360p (giving the new file the old name) w/ -crf 18

Occasionally streams also come in 396p and 720p, but these are rare (720p has only been seen once) and usually differ in other ways (e.g. 396p have lower frame rates), so this option will not affect them.

Instead of using -crf 18, added --bitrate option to specify the target bitrate for resized videos (defaults to 300k). Otherwise the new feature works as described above.


Migration guide example code - eager -> mode

In vast/website/docs/upgrade_guide.md, line 26, shouldn't
+ eager(Modes.ALL);

be
+ mode(Modes.ALL);

Yep, you are right. Mind submitting a quick PR? I'm away from my computer, traveling


fix: oneOf should not be allowed in message components

Description
This PR fixes that it currently possible to define messages with oneOf in components when the spec clearly doesn't allow it.
Currently allowed, but should not:
components:
  messages:
    UserSignedUp:
      oneOf:
        - payload:
            type: object
            properties:
              displayName:
                type: string
                description: Name of the user
              email:
                type: string
                format: email
                description: Email of the user

Related issue(s)
Fixes https://github.com/asyncapi/spec-json-schemas/issues/335

Thanks @dalelane 👍

/rtm


feature: extend serverless:exec to work for a simple (non cloud sql) Cloud Run app

In order to extend the serverless:exec gem to work on a basic sinatra Cloud Run app:

Added in a new product field to hold whether we're deploying an "app_engine" or "cloud_run" app.
Created a new start_cloud_run function to run the "cloud_build" strategy on the cloud run app
Added in a new default_product function to autodetect whether we're running on "app_engine" or "cloud_run".
Wrote a new version_info_cloud_run function to retrieve the cloud run metadata hash
Edited the start_build_strategy to retrieve the cloud run image

I think the version_info_cloud_run function could be refactored or combined with the old version_info function, but was unsure because we don't need the version field for Cloud Run.

@dazuma just pushed up the requested edits!

LGTM! Feel free to merge.


About LocMemCache database setting

Describe the problem
I want to setup the LocMemCache from django as the cache for the database implementation of this project. This avoids the need to setup a cache mechanism outside of the app.
I know it has it's problems, but my approach is to setup a timeout to the local memory cache so we know a key will be updated after X amount of time.
My problem is that the package, instead of discouraging the use, it prohibits using the django local cache implementation, which I think shouldn't be.
Steps to reproduce
Tell us how to reproduce it.
System configuration

Django version: 3.1.1
Python version: 3.8.5
Django-Constance version: 2.7.0

It's not the most common way to use it. But nevertheless, if you are willing to implement it, please suggest a pull request.


Fixes: #5

Pull Request Test Coverage Report for Build 1329ebd184a23790b36217efc00a77c14d2bedd1-PR-68

2 of 2 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.5%) to 91.495%

Totals

Change from base Build 0b8a38359bf5d1bf7cbc6eaa58b49d0c2b5cce10:
0.5%

Covered Lines:
710

Relevant Lines:
776

💛  - Coveralls


Dell Precision 3660 Driver are not available

Just letting you know the Dell Precision 3660 Driver are not available.
https://www.dell.com/support/kbdoc/en-us/000199139/precision-3660-tower-windows-10-driver-pack

I can confirm this.

So now I see it too... However "wmic computersystem get model" is showing "Precision 3660" on my device... Not sure which identifier I have, as in I am not sure how to check?

I can confirm that the DriverAutomationTool is creating the directory in MDT as ""Precision 3660 Tower", and that is NOT accurate.  It should be "Precision 3660".

I have the same issue wiht Latitude 5430. How do you procede, if there isn't a model available?
How do you import the package to sccm? Will it work, if the variable is added in Task Sequence?
Thanks
BR

I can confirm the drivers are downloading properly, but the model is not named correctly.  This issue can be closed.  I will created a separate issue in regards to the model naming.

As of at least 8/5/2022, the Precision 3660 is missing again.


MapStory TOS

MapStory has three main buckets of content:

Third party data from places like Open Street Map under various other open licenses, eg. ODDBL - aka Remote base map services
User contributed Story Layers, or data streams - any spatiotemporal data at any grain size - governed by default site license for data which is ODDBL
User media/stories built on top of these layers of data - defaulted currently at BY-SA 2.0 (but if they keep same license, we'd move them to 4.0)

What to look for (in addition to normal TOS review):

ODDBL and BY-SA compatibility - yes? if not, needed?
Could MapStory change its default terms for user-contributed data to CC0, considering its base layer pulling from sources that are ODDBL? End goal is just making it easier for data to work together cross-platforms, so viewing in this light, is this even desirable?
BY-SA 4.0 for user contributed media -- is that the optimal license re their intent? or is there a better license/tool we should recommend (this goes beyond TOS, but just to keep in mind)

Here's the TOS: http://wiki.mapstory.org/wiki/Terms_of_Service
Here's the licensing policy: http://wiki.mapstory.org/wiki/License_and_Copyright
We want to get some initial feedback to them by end of this month if possible.
Here's engagement doc with more background info (open to staff only).

Wrote down some initial feedback. Will likely require follow up once we talk to them, but I'll close this issue for now and we can reopen as needed.


Flax NNX and Orbax Checkpointing require hacking to work together

I'm building a system using flax.nnx and orbax.checkpointing. However, it is overly complicated on how to save and restore models due to the new jax.random.key() being used in flax.nnx rather than jax.random.PRNGkey().
I have had to create a workaround where all layers with rng and key in their path are changed from dtype=key<fry> to a format appropriate for saving. Then, upon restoration, they need to be shanged back.
I am attaching a link to a notebook explaining what I've done but I would be keen to hear if there are simpler workarounds? Or, preferably, if there is a way to simple save and restore models?
https://colab.research.google.com/drive/1ozln9ejG7eRtxvbkqHYU3K6OyPvveH9w?usp=sharing
Note: I am also adding an issue to orbax to see if there is a fix their side (#1337).

Thank you! I'll contact the Orbax team to see if they can fix this on their end.

Hey! Here's a quick and dirty solution.
Generally the idea is to use nnx.split with the NNX filter functionality to split the nnx.RngState types out of the state and then not save those.
graphdef, rng_state, other_state = nnx.split(model, nnx.RngState, ...)
and then just saving the other_state instead of the full thingy. I've edited your colab notebook to demonstrate this.
This means that RNG state will not be restored, which might be sub-optimal for certain scenarios but should work for most stuff. Hope it helps!

Another workaround for Dropout layers, and maybe custom layers too if they follow the same pattern, is to initialize them without the rngs arg, and only pass the RNG at __call__ time, like so:
import jax.numpy as jnp
import orbax.checkpoint as ocp
from flax import nnx

# Init dropout without rng arg.
model = nnx.Dropout(0.5)

# Pass RNG at call time.
output = model(jnp.ones(()), rngs=nnx.Rngs(0))

# This now works.
ckpt_dir = ocp.test_utils.erase_and_create_empty("/tmp/my-checkpoints/")
checkpointer = ocp.StandardCheckpointer()
checkpointer.save(ckpt_dir / "state", nnx.split(model)[1])

Versus if the RNG is supplied at initialization, the last line throws the following:
TypeError: Cannot interpret 'key<fry>' as a data type

But, this is only a workaround, as the RNG state will still not be serialized, and it makes for a more verbose call signature.


Font is not applied

hi,
I am using vscode on mac m3 pro. I have this theme which i love but it is not loading the font when i add the font-family in the settings. I have fontLigatures set to true as well. Need help

I was changing in the user settings but i didn't update the workspace one. shouldn't it be other way round?


Feature Request: Config option for tunnel IP version

Services in my local network use static or assigned IPv4 adresses.
Cloudflare by default sets IPv4 and IPv6 DNS records for tunnels.
As result, accessing a service from local network I dont go the direct way using the local IPv4 adress. Instead local requests resolve to the public IPv6 adress of the tunnel and require the authentication I have configured in zerotrust.
There is a argument for cloudflared to set the desired IP address family.
I'd very much love an config option to set a value for this flag to enforce the use of IPv4 only.
Is this already possible by some workaround? Might it be in a future version of this operator?
cloudflared tunnel --edge-ip-version <VERSION> run <UUID or NAME>

Possible values for VERSION: auto, 4, 6
Cloudflare Docs: edge-ip-version

For a one off use case, feel free to submit a PR like https://github.com/adyanth/cloudflare-operator/pull/100


Remove duplicate dead branch elim pass declaration

The function had two declarations in the same header with somewhat
different comments.

Rebased and pushed into master as 4b1577a0cbcfdbd


Updated SkiaSharp to stable version 1.56.2

Fixes #923

Codecov Report

Merging #928 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #928   +/-   ##
=======================================
  Coverage   60.46%   60.46%
=======================================
  Files         539      539
  Lines       20375    20375
  Branches     2909     2909
=======================================
  Hits        12320    12320
  Misses       7205     7205
  Partials      850      850

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aa17131...90e00a8. Read the comment docs.

There is one point to consider. Right now Skia# does not ship a native linux binary, so we are using a separate package instead. So before updating Skia# we need to manually check for linux compability.

Not compatible with latest changes to master. Closing.


Discrete chars

Is possible to create an AreaChart on discrete mode?
Like in this example (first chart)
https://jsfiddle.net/Xfx3h/426/

Hey @carlesnunez ,
Yes this should be relatively straight-forward: Check https://react-google-charts.com/area-chart for some examples.
The difference between a discrete and continuous chart is just in the column type. If you provide numbers or dates then a continuous chart is assumed, otherwise it's a discrete type.