implement str.splitlines

Refs #36

@cflee Regarding this:
I think another approach could be to iterate over the string (as a char array?) and check each byte to see whether it's a line ending. There's a special case if it's a \r, then check the next byte to see if it's \n. Then you can compute the start/end indexes (as seems to be done with i and j now) and substring the original string to add to the result list.

I attempted your suggestion locally and a major setback I faced is that the python representation of the line boundaries \x0b and the java representation of the same line boundary \u000B. If keeplines is true, I need to print the \x0b character, but if I use the java representation, it will print a new line, which is incorrect. I will research more on this as I think I do not fully understand how to deal with this yet.
Alternatively, the regex approach seems to pass and I will also try to test more thoroughly.
Thanks again for the feedback!! :)

@cflee @freakboy3742
Hi Chiang Fong Lee and Fede Isas, I followed the cpython implementation as suggested and also provided more test cases. Do let me know if it is okay, and I hope to be able to have my first ever contribution to open source soon! - Melvyn

@sngguojie The build is failing due to some small Java code style issues – please look into that so we can check the full test output.

I attempted your suggestion locally and a major setback I faced is that the python representation of the line boundaries \x0b and the java representation of the same line boundary \u000B. If keeplines is true, I need to print the \x0b character, but if I use the java representation, it will print a new line, which is incorrect.

To solve the above isse, I used a HashMap that takes the java line break character as the key and the string representation in python as the value. When the java line break character is detected, then I will append the python string representation given in the value.

I'm not quite sure I follow this – how is the Python representation involved here? It seems like all we are doing here is manipulating the strings in Java.
I'm quite curious about the choice of using a HashMap to get the String as well – could that be entirely avoided by adjusting the index used to substring from the char array? That way, there's no need to explicitly save/retrieve and append the linebreak char(s), and we wouldn't need those two representations (?) being put side-by-side.
I'm also a little split between having switching the HashMap/HashSet to a static object (since it doesn't change between invocations) or just having a large switch block.
I'll have another look at this after we can confirm that the newly added tests are passing.

@cflee Thanks for the quick feedback. I have fixed the style errors, sorry about that.

I'm not quite sure I follow this – how is the Python representation involved here? It seems like all we are doing here is manipulating the strings in Java.

Oh, I ran the tests to see what the outcome should be and manually added the strings in Java.

I'm quite curious about the choice of using a HashMap to get the String as well – could that be entirely avoided by adjusting the index used to substring from the char array? That way, there's no need to explicitly save/retrieve and append the linebreak char(s), and we wouldn't need those two representations (?) being put side-by-side.

For example, when I have the python test
s1 = 'a\nb'
print(s1.splitlines(True))

The expected result should be
['a\n', 'b']

However, when I attempted the implementation you mentioned (adjusting the index of the substring), the result is
['a
', b]

Which fails the test (Hence my alternative implementation). I hope my explanation on my choice to not use the indexing is clear. I'm not sure if my method is the best way, and I'm definitely open to learning how to improve!

I'm also a little split between having switching the HashMap/HashSet to a static object (since it doesn't change between invocations) or just having a large switch block.

Yup, I agree a static object or a large switch block would be more optimized than creating a new HashMap/HashSet every time the function is invoked. But this would depend on whether we establish that the indexing method you mentioned is possible.
Looking forward! :)

@sngguojie I don't have time to dig into this properly right at this moment, but some thoughts to try and unblock this...
My gut feel is that the issues around ['a\x0b', 'b] and ['a\n', 'b'] in the output are due to a faulty implementation of Str.__repr__(), which print() on the list invokes. I suspect the adjusting index approach is producing the correct string, but the output is wrong due to Str.__repr__(). Maybe you can verify whether this is (or is not!) the case.
You could check the length of the output strings as well, like 'a\n' must be two bytes long, not three. Is this the case for current implementation?
I think once #429 lands with ascii() builtin that could be used in the test as well, since it should escape non-ASCII characters.

@cflee Okay! Thanks for the direction! I will look into it!

Str.splitlines() has just been implemented in #566.
However, there's two things in this PR that would still be great contributions:

the improved Str.__repr__() – though it's still missing the quotes support per #568, and we probably should use a StringBuilder instead of a String, it's still an improvement over the existing implementation)
the understanding of handling all the non-\r\n newlines – the current implementation doesn't have this logic yet.

The tests in this PR are also failing on the newly-merged implementation, so some parts could be used a starting point for additional tests for the above two points too.
Since these pieces would be better handled in separate/new PRs, I'm closing this PR first. Thanks!

Ohh, I'm so sorry, I missed this PR while going through the list yesterday !
Thanks for bringing it up @cflee, I agree, +1 !


Update Suggestion: Alphabetization Consistency

I'm going to review the lists on each page since there are some inconsistencies with the alphabetization. IMO, items that start with "The" should be listed based on the first letter of the following word.
Example: "The Chaincode Podcast" is in the list as starting with "C". "The Bitcoin Basics Podcast" is under "T"
Which method should be used on the site, so I can make things consistent?

What if we just remove "The" so it fits better?

It would fit better, but I prefer to keep the name that the podcaster/author created for their work. We shouldn't change the name of "The Bitcoin Standard" to "Bitcoin Standard". It wouldn't be the correct name".

Funny, that's exactly what we did for the Japanese version of the book, but I see your point :)
But in that case, I guess the "The" should be considered in the alphabetization since you consider it part of the title?

That works, either is correct really. It should just be consistent.
I'll keep "The"s and put them as "T"


[BUG that I have a fix for] : Unchecking the "only save on run complete" setting does not work

As title says, unchecking this does not allow for new best checkpoints to be saved and replace the previous best if the run has not been finished.
I looked at the code and managed to fix this by simply commenting lines 300 to 304 included in the file Main.as :
if (raceTime <= 0 || deltaTime <= 0) {
DebugText("Checkpoint time negative..");
// not sure if this is necessary anymore? Made saving during half race bug, removed
//#if TMNEXT
//      waitForCarReset = true;
//#elif TURBO || MP4
//      ResetRace();
//#endif
return;
}
The lines commented start at #if TMNEXT
This made a call to ResetRace() before calling the function "UpdateSaveBestData" which emptied the current checkpoint times and of course prevented them from overriding the previous best.
I commented this as your comment said "not sure if this is necessary anymore?" so I hope it doesn't break anything elsewhere but it works on my end

Awesome thanks, I'll pop it in tonight and give it a shot. if all goes well I'll probably do an update on the weekend

No problem, also I tested on maniaplanet as it's what I play and for some reason that I don't remember the plugin didn't compile at first (before any modification on my end). I think it was related to the font (Resources::font did not exist or something ?).
So you should make sure that it also still properly runs on maniaplanet :)

seems like the Plugin hasnt been updated in the last 2 years, will this fix be released some day?

Probably no updates, between other projects and work I haven't looked into this for a long time.
I haven't touched Trackmania 2020 in probably the same two years, I prefer to play Trackmania Nations Forever, I think I was pinged a month or so ago in the openplanet discord about the plugin using an outdated api, so it might stop working at some point
I would like to rewrite the plugin but yeah it's unlikely
Happy to help out if someone wants to pick it up and fix a few of it's issues


Generic result of decode() method

Good day,
How can I use decode() method for generic result? Here's my code:
            SignalProducer<T, NoError> { sink, disposable in
                let request = self.request(method, URLString, parameters: parameters, encoding: encoding, headers: headers).responseJSON { response in
                    if let JSON = response.result.value{
                        if let decoded : T = decode(JSON){ //ERROR HERE: No 'decode' candidates produce the expected contextual result type 'Decoded?'
                            sink.sendNext(decoded)
                            sink.sendCompleted()
                        }
                    }
                }
                disposable.addDisposable({request.cancel()})
            }

I obtain error No 'decode' candidates produce the expected contextual result type 'Decoded?' on line with if let decoded : T = decode(JSON)

I'm not sure if this is possible, but try constraining your T to Decodable:
func decodedResponse<T: Decodable>() {
  return SignalProducer<T, NoError> ...
}

@jasin755 If you want to maintain the generic parameter through other functions you'll have to copy the exact generic constraints we use in our functions into your functions. @eimantas is close but you'll need the full constraint found in our decode.swift functions
<T: Decodable where T == T.DecodedType>

@jasin755 did you resolve your issue?

I'm closing this due to inactivity. @jasin755 if you still have an issue with this please feel free to reopen.


How do I inject IHttpContextAccessor into custom TelemetryInitializer , from a 4.8 framework web application?

List of NuGet packages and version that you are using: Microsoft.ApplicationInsights (2.20.0)

Runtime version (e.g. net461, net48, netcoreapp2.1, netcoreapp3.1, etc. You can find this information from the *.csproj file): v4.8

Hosting environment (e.g. Azure Web App, App Service on Linux, Windows, Ubuntu, etc.): Windows

What are you trying to achieve?
I'm trying to create a custom telemetry initializer (HttpContextRequestTelemetryInitializer) like the one mentioned in the official Microsoft docs and then use it in a .NET framework 4.8 app. The code for the telemetry initializer looks something like this -
public class HttpContextRequestTelemetryInitializer : ITelemetryInitializer
{
    private readonly IHttpContextAccessor httpContextAccessor;

    public HttpContextRequestTelemetryInitializer(IHttpContextAccessor httpContextAccessor)
    {
        this.httpContextAccessor =
            httpContextAccessor ??
            throw new ArgumentNullException(nameof(httpContextAccessor));
    }

    public void Initialize(ITelemetry telemetry)
    {
        var requestTelemetry = telemetry as RequestTelemetry;
        if (requestTelemetry == null) return;

        var claims = this.httpContextAccessor.HttpContext.User.Claims;
        Claim oidClaim = claims.FirstOrDefault(claim => claim.Type == "oid");
        requestTelemetry.Properties.Add("UserOid", oidClaim?.Value);
    }
}

What have you tried so far?
I'm able to use HttpContextRequestTelemetryInitializer as a singleton in .NET core application. But I'm not sure how to make it work in a .NET Framework 4.8 app, given that there are complexities of adding dependency injection and lack of httpContextAccessor in .NET Framework 4.8.

I am not sure where you got this example from - I don't see that in any official docs. If it is in any official doc - please share here (so as to remove it from the doc)
It won't work for ASP.NET.
It may work for ASP.NET Core, but definitely not safe.
You may look at the example initializers in this repo and borrow that approach for ASP.NET and ASP.NET Core.

@cijothomas see https://docs.microsoft.com/azure/azure-monitor/app/api-filtering-sampling#add-information-from-httpcontext

Thanks @pharring for sharing. I'll open a doc issue to remove it or fix it.
Old discussions: https://github.com/microsoft/ApplicationInsights-dotnet/issues/1313

@sushanti Please use the approach from the existing initializers in this repo for aspnet and aspnetcore.
I have opened a doc issue to address the correctness issue.

Thanks!


New terms for annotating scRNAseq datasets

This PR adds a set of new terms that will be needed to annotate scRNAseq datasets for their inclusion into FlyBase.
It also assigns an ID range to me, since it's my first contribution to FBcv.
related to #138

Don't worry about failing checks, looks there was a problem downloading CHEBI (happens sometimes) - it will probably succeed next time (or you can just restart the travis build if it happens again).


Get-PlasterTemplate : Add a possibility to retrieve all template module regardless of the version

Today Get-PlasterTemplate retrieve the templates only for the current module version (1.0.1).
It not possible to known which version of the Plaster module I must use for each Template.
And to filter the templates, Get-PlasterTemplate use the template module manifest not an information given by the user.

I use the latest build, and the Get-Plastertemplate do not return the type of the template (Project or Item);
>Get-PlasterTemplate

Title        : AddPSScriptAnalyzerSettings
Author       : Plaster project
Version      : 1.0.0
Description  : Add a PowerShell Script Analyzer settings file to the root of your workspace.
Tags         : {PSScriptAnalyzer, settings}
TemplatePath : C:\Program Files\WindowsPowerShell\Modules\plaster\1.0.2\Templates\AddPSScriptAnalyzerSettings

Title        : New PowerShell Manifest Module
Author       : Plaster
Version      : 1.0.0
Description  : Creates files for a simple, non-shared PowerShell script module.
Tags         : {Module, ModuleManifest}
TemplatePath : C:\Program Files\WindowsPowerShell\Modules\plaster\1.0.2\Templates\NewPowerShellScriptModule

I haven't finished updating the Get-PlasterTemplate cmdlet yet, Keith and I have been too busy with other projects lately to get the 1.1.0 release out with these changes.

Ok thanks.
All things come to him who waits :-)

Any update on this? Would be a great feature.

Haven't gotten around to this because of other project - feel free to contribute! Plaster is all PowerShell :)
Here's the code for Get-PlasterTemplate
https://github.com/PowerShell/Plaster/blob/master/src/GetPlasterTemplate.ps1


Fit Page to Diagram Option

I would like to request implementation of a "fit page to diagram" feature in the page setup dialogue.  I believe it should likely include a specification for a modifiable default margin, and the intent would be to automatically determine a custom page size from all deployed diagram elements at the time the feature is executed such that the page size is automatically resized to fit the diagram with the configured margin between the "outer" edges of the diagram elements and the edge of the page.
Thank you,
Nathan

Hi,
would you like to have "fit page to diagram" option in order to export the diagram only?
Or you would like to have this possibility so you can view only diagram in the editor (without surrounding canvas)?
Regards,

Hello,
I would like to have the "fit page to diagram" for the following two cases.

To export the diagram.  This seems the feature most impacted or eased by
this change.
To view the diagram with minimal canvas/background in the editor.  This
enhances the convenience of creating diagrams that fit the
canvas/background in the editor such that I would not have to resize my
diagram to fit on a particular page dimension.

To clarify, when I refer to the "page", I am referring to the default
background with gridlines that appears as if it is a "paper" page.
Currently, when I want this appearance, I go to the page setup dialogue,
and sort of fumble through different custom page sizes until I figure out
what it needs to be to just barely extend beyond the edges of my diagrams.
Thank you,
Nathan
On Mon, Nov 1, 2021 at 7:19 AM mararad @.***> wrote:

Hi,
would you like to have "fit page to diagram" option in order to export the
diagram only?
Or you would like to have this possibility so you can view only diagram in
the editor (without surrounding canvas)?
Regards,
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jgraph/drawio/issues/2366#issuecomment-956148123, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABNZBNNSH7HIBPUPW6TTBD3UJZZVFANCNFSM5HABRSJQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Thanks Nathan.

When exporting, there is the possibility to choose to export diagram only
"To view the diagram with minimal canvas/background in the editor." - this is something we don't support at the moment

Regards,

Hi,

When exporting, there is the possibility to choose to export
diagram only - I do not see this option.  When I export to PDF for
instance, if the diagram extends beyond the edges of the current page size,
I end up with multiple pages of partial diagram.  If I set the page size to
be larger than the diagram, then I have to do additional work to make it
fit "neatly" on the page.  Can you advise how I might use the feature you
are indicating from the web client (i.e. @ diagrams.net
http://diagrams.net)?
"To view the diagram with minimal canvas/background in the editor."

this is something we don't support at the moment - I thought I was
making a request to implement a new feature.

Thank you,
Nathan


Problem with building firmware for Teensy 4.0 and micro-ros

Issue
I have issue with building micro-ros-arduino for teensy 4.0.
Errors
    Linking .pio/build/teensy40/firmware.elf
    .pio/build/teensy40/src/main.cpp.o: In function `timer_callback(rcl_timer_t*, long long)':
    main.cpp:(.text._Z14timer_callbackP11rcl_timer_tx+0xc): undefined reference to `rcl_publish'
    .pio/build/teensy40/src/main.cpp.o: In function `setup':
    main.cpp:(.text.setup+0x16): undefined reference to `rmw_uros_set_custom_transport'
    main.cpp:(.text.setup+0x36): undefined reference to `rcutils_get_default_allocator'
    main.cpp:(.text.setup+0x4c): undefined reference to `rclc_support_init'
    main.cpp:(.text.setup+0x5a): undefined reference to `rclc_node_init_default'
    main.cpp:(.text.setup+0x60): undefined reference to `rosidl_typesupport_c__get_message_type_support_handle__std_msgs__msg__Int32'
    main.cpp:(.text.setup+0x6c): undefined reference to `rclc_publisher_init_default'
    main.cpp:(.text.setup+0x80): undefined reference to `rclc_timer_init_default'
    main.cpp:(.text.setup+0x90): undefined reference to `rclc_executor_init'
    main.cpp:(.text.setup+0x9a): undefined reference to `rclc_executor_add_timer'
    .pio/build/teensy40/src/main.cpp.o: In function `loop':
    main.cpp:(.text.loop+0x14): undefined reference to `rclc_executor_spin_some'

Steps to reproduce the issue
I have cloned micro-ros-arduino repo and I have used platformio for building it. I have put uros_lib inside "lib" folder of platformio.
I have included <Arduino.h> and then I have copied contents of this in main.cpp.
I know that teensy_transports.c.in is for mapping some functions of uros for arduino and also I have tried to cut its contents inside main.cpp, but nothing has changed.
How can I build correctly?
The mentioned functions (rclc_executor_spin_some, rclc_executor_init, ...) are also defined some where inside uros_lib but why they are not detected for compiler?

Platformio is not currently supported. Please use Arduino IDE or Arduino CLI.
And remember to patch the teensyduino addon

Any updates? I would like to also use platformIO instead of Arduino IDE or CLI and it throws the same errors mentioned here. I saw some recent issues mentioning platformIO as the used IDE so i bet it is compatible. Any hints on how to configure the project are welcome?
Thanks!

Hi @DaniGarciaLopez , you can use my github page link in order to get help. Of course it had been mention in #414 as well.

Hello @DaniGarciaLopez and @masoudir currently we do not have time to implement a solution for Platform.IO but it would be great if you contribute a possible solution so we can integrate it into the building ecosystem of micro-ROS.
@masoudir could you brief a tutorial on how to use the micro-ROS Arduino static libraries for Platform.IO so we can include it in this repo README.md or in micro.ros.org? It can be a section in the current README.md or even a new one .md file that we can reference from it. I would like it to be generic and not only for Teensy, is it possible?
Thanks a lot for the efforts!

Thanks for the answers. Adding a build_flag to platform.ini file correctly detects microros. I tried with the precompiled galactic version and with a custom compilation and both worked good.
[env]
platform = teensy
framework = arduino
upload_port = /dev/ttyACM0
build_flags = -L $PROJECT_DIR/lib/micro_ros_arduino/src/imxrt1062/fpv5-d16-hard -l libmicroros

[env:teensy40]
board = teensy40

Thanks @pablogs9 and @DaniGarciaLopez . Yes of course. It would be an honor for me to contribute more into your project. There are some small things needed to be considered during using platformio for microros. I will create a readme for that in future days.


Assist chips

The ChipsContainer test seems to be flaky:
Chrome Headless 109.0.5414.46 (Linux x86_64) absoluteChipsContainer.test.3095854908.js ChipsContainer when scroll buttons are clicked scrolls the container in the right and in the left direction FAILED
	Expected 32 to be less than 32.
	    at <Jasmine>
	    at UserContext.<anonymous> (/tmp/_karma_webpack_612838/commons.js:322384:33)
Chrome Headless 109.0.5414.46 (Linux x86_64): Executed 209 of 2623 (1 FAILED) (0 secs / 20.718 secs)
Chrome Headless 109.0.5414.46 (Linux x86_64) absoluteChipsContainer.test.3095854908.js ChipsContainer when scroll buttons are clicked scrolls the container in the right and in the left direction FAILED
	Expected 32 to be less than 32.
	    at <Jasmine>
	    at UserContext.<anonymous> (/tmp/_karma_webpack_612838/commons.js:322384:33)

I created #1447  to solve the flaky test

One thing caught my eye: When a measurement or drawing geometry was updated the elevation profile won't be updated.

One thing caught my eye: When a measurement or drawing geometry was changed the elevation profile won't be updated.

fixed with d5d6178564f76569b92ef84ec79ea288288bf059


Fixes typo in README.md

Fixes typo in README.md

Coverage remained the same at 100.0% when pulling 948f4dab4d0566904f5c633337a5b8dffe29b090 on kamronbatman:patch-1 into fd92642e768599adb1203ac5b566eae463fb8bdc on AndyBarron:master.

@AndyBarron, please merge, <3.


[Language Support] Add support for SAM apps written in Java

Currently the workshop instructions are based on SAM apps written in NodeJS. We have gotten feedback that customers would like to see examples of Java-based Serverless apps.

Hi Fernando,
I can help you with this.
Mark


dynamic redirect

empty result behave like old one
string result override the default in @Redirect
object result using template-url to generate url

https://github.com/pleerock/routing-controllers/issues/159

@Diluka I think it would also make sense to support @Redirect() with no arguments and allowing the controller method to return a string URL.

@marshall007
Diving into the driver, it decide whether there have redirections by checking the redirect string if not empty.

You forgot to make url argument optional in @Redirect decorator. Also please add docs regarding to this new functionality.

@pleerock
url argument is designed to be not optional and adding docs

Thank you! Released in 0.7.0-alpha.15


Slow CDN mirroring

It looks like it is taking over 1hr for some packages to mirror. For example:

Do we know what might be causing this?

Here's another example:

We see some errors in the channel-clone VMs, we are checking if this is related

Thanks Stefan! 🙏
Interested to learn what you discover 🙂

Adding a more recent example in case it is helpful

This one is approaching ~2.5hrs
Idk if there is something more going on with this case (hence mentioning it explicitly)

Another example after John referred me to this issue: 2+ hours after marking some packages as broken in https://github.com/conda-forge/admin-requests/commit/164a4271b51ee47fdcfdd52905e0137fbdf21003, packages are still not removed from the CDN (it's difficult to do quickfixes to wide-ranging breaks like this one if the turnaround is that long).
>mamba repoquery search -c conda-forge gcc_linux-aarch64=12.3.0=*_2 -p linux-64

Executing the query gcc_linux-aarch64=12.3.0=*_2

conda-forge/noarch                                  13.9MB @   3.8MB/s  3.6s
conda-forge/linux-64                                33.0MB @   4.2MB/s  7.8s

 Name              Version Build        Channel     Subdir
-------------------------------------------------------------
 gcc_linux-aarch64 12.3.0  hd570763_2   conda-forge linux-64            # should be empty

>python
Python 3.11.8 | packaged by conda-forge | (main, Feb 16 2024, 20:40:50) [MSC v.1937 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> pd.Timestamp.now("utc")
Timestamp('2024-03-12 05:43:31.911276+0000', tz='UTC')                  # marked broken ~2:20h before

We see some files generated by anaconda.org's dynamic repodata like linux-aarch64/awscli-1.29.39-py38he37f277_0.conda  that can't be downloaded, and others like noarch/strawberry-graphql-with-asgi-0.162.0-pyhd8ed1ab_0.conda that cannot be re-indexed as part of the CDN process.

Interesting, thanks Daniel! 🙏
Is there something in regards to those packages specifically that looks relevant?
Or did something unrelated to those packages occur (like a network outage or running out of disk space)?

Were we able to determine the cause here?

We were able to find packages that have repodata but no downloadable archive missing-packages.txt
We were also able to fix a bug that is more likely to be the cause, where we would have trouble re-downloading a package in the CDN process if the first clone failed. We were not able to find the precise cause.

Gotcha thanks for the update Daniel! 🙏
Looked at the first one on the list, linux-aarch64/awscli-1.29.39-py38he37f277_0.conda. It looks like other OS and arch combinations for 1.29.39 had validation issues ( https://github.com/conda-forge/awscli-feedstock/issues/765#issuecomment-1702350487 ), but linux-aarch64 was not one of these. These eventually this got sorted out. Can see a successful build and upload for this package. Here is the attached log. Though agree downloading this package doesn't work. So not sure what happened here. Maybe it should be marked broken (unclear whether that helps this issue)
The second package, noarch/boto3-stubs-lite-1.26.89-pyhd8ed1ab_0.conda, had a validation error with that package specifically ( https://github.com/conda-forge/boto3-stubs-lite-feedstock/issues/120#issuecomment-1465186684 ). Same with the third package, noarch/ca-policy-lcg-1.119-hd8ed1ab_0.conda, ( https://github.com/conda-forge/ca-policy-lcg-feedstock/issues/20#issue-1627340874 ). At least for these two cases, the conda-forge validation service notes the packages were not copied. However there do appear to be packages with 0 downloads for both boto3-stubs-lite and ca-policy-lcg. Visiting or downloading either results in a 404. Perhaps these should be marked broken as well
Looking at the latter two cases, do not see them in http://conda.anaconda.org/conda-forge/noarch/repodata.json.bz2. Nor do they show up in conda-forge-repodata-patches. So am wondering how these are added to the repodata when no package was copied
Idk if we can have aborted copy with the conda-forge validation service that might generate these issues, but that seems like one question that comes out of this
cc @beckermr (in case I'm missing anything here)

We download the dynamic anaconda.org repodata.json beefore creating the CDN version.

Can you please remind me which URL that lives under?

https://conda.anaconda.org/conda-forge/noarch/repodata-clone.json e.g.

Just saw a particularly severe case today (still nothing after 3.5h for linux-64, 2h for win-64):

The logs show that the CDN clone process downloaded that archive from https://conda-web.anaconda.org/conda-forge/linux-64/clangdev-18.1.2-default_h127d8a8_0.conda, had a bad archive at 2024-03-21T02:29:30 and was able to get a good archive at 2024-03-21T06:12

Not that I pretend to understand the cloning mechanism (or the reasons why it might fail), but would it make sense to have a shorter retry loop for failed clones? Like try again immediately after, or after X minutes delay?

It does retry frequently, there may be an intermediate cache issue.

The CDN appears to be down again.

Approaching the 500min mark 😬

We've addressed a disk-full issue.

Just ran into a a network issue:
conda.CondaMultiError: ('Connection broken: IncompleteRead(199522674 bytes read, 79146888 more expected)', IncompleteRead(199522674 bytes read, 79146888 more expected))
('Connection broken: IncompleteRead(199522674 bytes read, 79146888 more expected)', IncompleteRead(199522674 bytes read, 79146888 more expected))

Wondering if this is related

Approaching the 500min mark 😬 Should that metric be part of anaconda.statuspage.io?

Yes, I think we should start tracking this publicly somehow

CDN is at 37 minutes.

Should be resolved.

Last sync was done almost 10h ago now.

Looking at this case, mirroring has not completed after ~40mins

The linux-aarch64 (highlighted) and win-64 package above it are still mirroring. Approaching the ~1.5hr mark

Regarding cuda-tools, the linux-aarch64 package is now available on CDN

However the win-64 package is not. It has been ~3.25hrs since it was uploaded
% conda search 'conda-forge:cuda-tools[subdir=win-64]=12.4.1'
Loading channels: done
No match found for: cuda-tools=12.4.1[subdir=win-64]. Search: *cuda-tools*=12.4.1[subdir=win-64]

PackagesNotFoundError: The following packages are not available from current channels:

  - cuda-tools=12.4.1[subdir=win-64]

Current channels:

  - https://conda.anaconda.org/conda-forge/win-64
  - https://conda.anaconda.org/conda-forge/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

CONDA_DEBUG=1 conda search 'conda-forge::cuda-tools[subdir=win-64]=12.4.1'

Thanks Daniel! 🙏
More details below:

CONDA_DEBUG=1 conda search 'conda-forge::cuda-tools[subdir=win-64]=12.4.1'
% CONDA_DEBUG=1 conda search 'conda-forge::cuda-tools[subdir=win-64]=12.4.1'
DEBUG conda.gateways.logging:set_log_level(233): log_level set to 10
Loading channels: ...working... DEBUG conda.core.package_cache_data:_check_writable(321): package cache directory '/Users/jkirkham/miniforge/pkgs' writable: True
DEBUG conda.gateways.repodata:fetch_latest(827): Using cached repodata for https://conda.anaconda.org/conda-forge/win-64/repodata.json at /Users/jkirkham/miniforge/pkgs/cache/5afe41e9.json. Timeout in 1899 sec
DEBUG conda.gateways.repodata:read_cache(907): Loading raw json for https://conda.anaconda.org/conda-forge/win-64/repodata.json at /Users/jkirkham/miniforge/pkgs/cache/5afe41e9.json
done
No match found for: conda-forge/win-64::cuda-tools=12.4.1. Search: conda-forge/win-64::*cuda-tools*=12.4.1
Traceback (most recent call last):
  File "/Users/jkirkham/miniforge/lib/python3.10/site-packages/conda/exception_handler.py", line 17, in __call__
    return func(*args, **kwargs)
  File "/Users/jkirkham/miniforge/lib/python3.10/site-packages/conda/cli/main.py", line 83, in main_subshell
    exit_code = do_call(args, parser)
  File "/Users/jkirkham/miniforge/lib/python3.10/site-packages/conda/cli/conda_argparse.py", line 199, in do_call
    result = getattr(module, func_name)(args, parser)
  File "/Users/jkirkham/miniforge/lib/python3.10/site-packages/conda/cli/main_search.py", line 265, in execute
    raise PackagesNotFoundError((str(spec),), channels_urls)
conda.exceptions.PackagesNotFoundError: The following packages are not available from current channels:

  - conda-forge/win-64::cuda-tools=12.4.1

Current channels:

  - https://conda.anaconda.org/conda-forge/win-64
  - https://conda.anaconda.org/conda-forge/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Let's try reducing the cache TTL.
CONDA_LOCAL_REPODATA_TTL=0 CONDA_DEBUG=1 conda search 'conda-forge::cuda-tools[subdir=win-64]=12.4.1' --json

That looks promising

CONDA_LOCAL_REPODATA_TTL=0 CONDA_DEBUG=1 conda search 'conda-forge::cuda-tools[subdir=win-64]=12.4.1' --json
DEBUG conda.gateways.logging:set_log_level(233): log_level set to 10
DEBUG conda.core.package_cache_data:_check_writable(321): package cache directory '/Users/jkirkham/miniforge/pkgs' writable: True
DEBUG conda.gateways.repodata:fetch_latest(836): Local cache timed out for https://conda.anaconda.org/conda-forge/win-64/repodata.json at /Users/jkirkham/miniforge/pkgs/cache/5afe41e9.json
DEBUG conda.gateways.repodata.jlap.interface:__init__(41): Using ZstdRepoInterface
DEBUG conda.gateways.connection.session:add_binstar_token(247): Adding anaconda token for url <https://conda.anaconda.org/conda-forge/win-64/repodata.json.zst>
DEBUG urllib3.connectionpool:_new_conn(1055): Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG urllib3.connectionpool:_make_request(549): https://conda.anaconda.org:443 "GET /t/ja-0b7702b5-09c2-4eba-86f1-afde7a4f059b/conda-forge/win-64/repodata.json.zst HTTP/1.1"<PHONE_NUMBER>1
DEBUG conda.gateways.repodata.jlap.fetch:download_and_hash(243): https://conda.anaconda.org/conda-forge/win-64/repodata.json.zst {'Date': 'Wed, 10 Apr 2024 00:06:12 GMT', 'Content-Type': 'binary/octet-stream', 'Content-Length': '21595511', 'Connection': 'keep-alive', 'CF-Ray': '871e7d54f85f6b05-DFW', 'CF-Cache-Status': 'HIT', 'Accept-Ranges': 'bytes', 'Age': '624', 'Cache-Control': 'public, max-age=1200', 'ETag': '"265bb4f741e3a511d1b7fd13e0edee29"', 'Expires': 'Wed, 10 Apr 2024 00:26:12 GMT', 'Last-Modified': 'Tue, 09 Apr 2024 23:51:43 GMT', 'Vary': 'Accept-Encoding', 'x-amz-id-2': 'KsKq6Hrja0c+NuJOtbl7K2aZzL0wG6RjgNYYmQdqWpd3z3J4QGMToZG3p6uCst6RCFsGipoI7hk=', 'x-amz-request-id': '0TEK1J904ZF6JPAZ', 'x-amz-version-id': 'null', 'Set-Cookie': '__cf_bm=wiFBhoz93urWirCFc2ZG6eza4dPUj0Boq5Om5XXAAqg-1712707572-<IP_ADDRESS>-fImsderM0uAGUXTU1v5f2D6oevTo0EiN9RzvfapVIAJg8tz5A7xMZcag9R2Dk6LX_2rNKVn8hm_yraEI3GsI6RuELD2gizrmaUR0prPSIoM; path=/; expires=Wed, 10-Apr-24 00:36:12 GMT; domain=.anaconda.org; HttpOnly; Secure; SameSite=None', 'Server': 'cloudflare'}
INFO conda.gateways.repodata.jlap.fetch:download_and_hash(264): Download 21595511 bytes {'User-Agent': 'conda/24.3.0 requests/2.31.0 CPython/3.10.13 Darwin/23.4.0 OSX/14.4.1 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Accept': '*/*', 'Connection': 'keep-alive', 'if-none-match': '"6ca41bfb39370baa2abf1c3307d43f6e"'}
DEBUG conda.gateways.repodata.jlap.fetch:timeme(196): Download complete https://conda.anaconda.org/conda-forge/win-64/repodata.json Took 1.14s
INFO conda.gateways.repodata.jlap.fetch:request_url_jlap_state(431): Apply 0 patches 3796a3c1f32f8782… → 3796a3c1f32f8782…
DEBUG conda.gateways.repodata.jlap.fetch:timeme(196): Apply Patches Took 0.00s

{
  "cuda-tools": [
    {
      "arch": null,
      "build": "h57928b3_0",
      "build_number": 0,
      "channel": "https://conda.anaconda.org/conda-forge/win-64",
      "constrains": [],
      "depends": [
        "cuda-command-line-tools 12.4.1.*",
        "cuda-visual-tools 12.4.1.*"
      ],
      "fn": "cuda-tools-12.4.1-h57928b3_0.conda",
      "license": "LicenseRef-NVIDIA-End-User-License-Agreement",
      "md5": "12168c96051bbbc85dbb96ec9a8b0fac",
      "name": "cuda-tools",
      "platform": null,
      "sha256": "720c78b89fe36e19881347d69329f0ff8c1c4cd1630d39f3147d4fe08ace8583",
      "size": 20367,
      "subdir": "win-64",
      "timestamp":<PHONE_NUMBER>857,
      "url": "https://conda.anaconda.org/conda-forge/win-64/cuda-tools-12.4.1-h57928b3_0.conda",
      "version": "12.4.1"
    }
  ]
}

This case is approaching the 2hr mark, but does not appear to be picked up by CDN
Can find 1 of the 2 packages expected with CONDA_LOCAL_REPODATA_TTL=0 CONDA_DEBUG=1 conda search 'conda-forge::cuda-toolkit=12.4.1' --json

It looks like both packages are now available:
$ CONDA_LOCAL_REPODATA_TTL=0 CONDA_DEBUG=1 conda search 'conda-forge::cuda-toolkit=12.4.1' --json
DEBUG conda.gateways.logging:set_log_level(233): log_level set to 10
DEBUG conda.core.package_cache_data:_check_writable(321): package cache directory '/home/chl/.miniconda3-x86_64/pkgs' writable: True
DEBUG conda.gateways.repodata:fetch_latest(836): Local cache timed out for https://conda.anaconda.org/conda-forge/linux-64/repodata.json at /home/chl/.miniconda3-x86_64/pkgs/cache/497deca9.json
DEBUG conda.gateways.repodata.jlap.interface:__init__(41): Using ZstdRepoInterface
DEBUG conda.gateways.connection.session:add_binstar_token(247): Adding anaconda token for url <https://conda.anaconda.org/conda-forge/linux-64/repodata.json.zst>
DEBUG urllib3.connectionpool:_new_conn(1052): Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG urllib3.connectionpool:_make_request(546): https://conda.anaconda.org:443 "GET /t/ch-aad2628c-7f2b-496e-98ce-4fcff0ee47b9/conda-forge/linux-64/repodata.json.zst HTTP/1.1" 304 0
DEBUG conda.gateways.repodata.jlap.fetch:download_and_hash(243): https://conda.anaconda.org/conda-forge/linux-64/repodata.json.zst {'Date': 'Wed, 10 Apr 2024 17:33:49 GMT', 'Connection': 'keep-alive', 'CF-Ray': '87247bf1db4f6c79-DFW', 'CF-Cache-Status': 'HIT', 'Age': '1269', 'Cache-Control': 'public, max-age=1200', 'ETag': '"6134ed6707255ece9635559af12210f1"', 'Expires': 'Wed, 10 Apr 2024 17:53:49 GMT', 'Last-Modified': 'Wed, 10 Apr 2024 17:12:01 GMT', 'Vary': 'Accept-Encoding', 'x-amz-id-2': 'xXMe6GDWbJycN4+8p/BlbLkrY6hf+S8S8Ej7yaJjhjhoMx0RUMMY1OaU07Gb/yxsghhefqFxJqY=', 'x-amz-request-id': '4FRF0VNSKWS1SETG', 'x-amz-version-id': 'null', 'Set-Cookie': '__cf_bm=Zd4sY9TOVRGf_xkAFGWxZt22Jf9zV1CW1ZFappB_tHA-1712770429-<IP_ADDRESS>-Whna4UmYB02PdyGWm2hAjfjtN423iI9pahjBjPvhd19Y7Xir_WzKBEVZcHl7rVbWY94oHRFp5TOQeXEby1S1WrcfKFyABRrPsIO9Fo390pE; path=/; expires=Wed, 10-Apr-24 18:03:49 GMT; domain=.anaconda.org; HttpOnly; Secure; SameSite=None', 'Server': 'cloudflare'}
INFO conda.gateways.repodata.jlap.fetch:download_and_hash(264): Download 0 bytes {'User-Agent': 'conda/24.3.0 requests/2.31.0 CPython/3.11.7 Linux/6.5.0-26-generic linuxmint/21.3 glibc/2.35 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8 aau/0.4.4', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Accept': '*/*', 'Connection': 'keep-alive', 'if-none-match': '"6134ed6707255ece9635559af12210f1"'}
DEBUG conda.gateways.repodata.jlap.fetch:timeme(196): Download complete https://conda.anaconda.org/conda-forge/linux-64/repodata.json Took 0.09s
INFO conda.gateways.repodata.jlap.fetch:request_url_jlap_state(431): Apply 0 patches 4e79e31048896620… → 4e79e31048896620…
DEBUG conda.gateways.repodata.jlap.fetch:timeme(196): Apply Patches Took 0.00s
DEBUG conda.gateways.repodata:fetch_latest(836): Local cache timed out for https://conda.anaconda.org/conda-forge/noarch/repodata.json at /home/chl/.miniconda3-x86_64/pkgs/cache/09cdf8bf.json
DEBUG conda.gateways.repodata.jlap.interface:__init__(41): Using ZstdRepoInterface
DEBUG conda.gateways.connection.session:add_binstar_token(247): Adding anaconda token for url <https://conda.anaconda.org/conda-forge/noarch/repodata.json.zst>
DEBUG urllib3.connectionpool:_new_conn(1052): Starting new HTTPS connection (2): conda.anaconda.org:443
DEBUG urllib3.connectionpool:_make_request(546): https://conda.anaconda.org:443 "GET /t/ch-aad2628c-7f2b-496e-98ce-4fcff0ee47b9/conda-forge/noarch/repodata.json.zst HTTP/1.1" 304 0
DEBUG conda.gateways.repodata.jlap.fetch:download_and_hash(243): https://conda.anaconda.org/conda-forge/noarch/repodata.json.zst {'Date': 'Wed, 10 Apr 2024 17:33:52 GMT', 'Connection': 'keep-alive', 'CF-Ray': '87247c051e116b37-DFW', 'CF-Cache-Status': 'HIT', 'Age': '1272', 'Cache-Control': 'public, max-age=1200', 'ETag': '"3605ffb16e98f38bc071ce70257c459c"', 'Expires': 'Wed, 10 Apr 2024 17:53:52 GMT', 'Last-Modified': 'Wed, 10 Apr 2024 17:12:16 GMT', 'Vary': 'Accept-Encoding', 'x-amz-id-2': 'o7PiCRiTKh0i28OBv9US4lJb8Bne90hS+dQWjje44eZ8eGKGXf3KijXyyGIqkITCOeFXcv6WIwU=', 'x-amz-request-id': '4FR2PM0C9P3JSPP4', 'x-amz-version-id': 'null', 'Server': 'cloudflare'}
INFO conda.gateways.repodata.jlap.fetch:download_and_hash(264): Download 0 bytes {'User-Agent': 'conda/24.3.0 requests/2.31.0 CPython/3.11.7 Linux/6.5.0-26-generic linuxmint/21.3 glibc/2.35 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8 aau/0.4.4', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Accept': '*/*', 'Connection': 'keep-alive', 'if-none-match': '"3605ffb16e98f38bc071ce70257c459c"', 'Cookie': '__cf_bm=Zd4sY9TOVRGf_xkAFGWxZt22Jf9zV1CW1ZFappB_tHA-1712770429-<IP_ADDRESS>-Whna4UmYB02PdyGWm2hAjfjtN423iI9pahjBjPvhd19Y7Xir_WzKBEVZcHl7rVbWY94oHRFp5TOQeXEby1S1WrcfKFyABRrPsIO9Fo390pE'}
DEBUG conda.gateways.repodata.jlap.fetch:timeme(196): Download complete https://conda.anaconda.org/conda-forge/noarch/repodata.json Took 0.10s
INFO conda.gateways.repodata.jlap.fetch:request_url_jlap_state(431): Apply 0 patches 3a8d1a3f16e02a99… → 3a8d1a3f16e02a99…
DEBUG conda.gateways.repodata.jlap.fetch:timeme(196): Apply Patches Took 0.00s
{
  "cuda-toolkit": [
    {
      "arch": null,
      "build": "h7428d3b_0",
      "build_number": 0,
      "channel": "https://conda.anaconda.org/conda-forge/noarch",
      "constrains": [],
      "depends": [
        "__win",
        "cuda-compiler 12.4.1.*",
        "cuda-libraries 12.4.1.*",
        "cuda-libraries-dev 12.4.1.*",
        "cuda-nvml-dev 12.4.127.*",
        "cuda-tools 12.4.1.*"
      ],
      "fn": "cuda-toolkit-12.4.1-h7428d3b_0.conda",
      "license": "LicenseRef-NVIDIA-End-User-License-Agreement",
      "md5": "da683653aaadbeeb5a7a2561b2464728",
      "name": "cuda-toolkit",
      "noarch": "generic",
      "package_type": "noarch_generic",
      "platform": null,
      "sha256": "3f750755a089f61fb58e6e255706bd2d010fd217fcb0a503c2de9fe3a337b247",
      "size": 20519,
      "subdir": "noarch",
      "timestamp":<PHONE_NUMBER>371,
      "url": "https://conda.anaconda.org/conda-forge/noarch/cuda-toolkit-12.4.1-h7428d3b_0.conda",
      "version": "12.4.1"
    },
    {
      "arch": null,
      "build": "ha804496_0",
      "build_number": 0,
      "channel": "https://conda.anaconda.org/conda-forge/noarch",
      "constrains": [],
      "depends": [
        "__linux",
        "cuda-compiler 12.4.1.*",
        "cuda-libraries 12.4.1.*",
        "cuda-libraries-dev 12.4.1.*",
        "cuda-nvml-dev 12.4.127.*",
        "cuda-tools 12.4.1.*"
      ],
      "fn": "cuda-toolkit-12.4.1-ha804496_0.conda",
      "license": "LicenseRef-NVIDIA-End-User-License-Agreement",
      "md5": "e1e8cfdbb172f4b6558ce2db688e851f",
      "name": "cuda-toolkit",
      "noarch": "generic",
      "package_type": "noarch_generic",
      "platform": null,
      "sha256": "c3faecbb52cbdb82d1723e82ed572ae23bc187863b7f499f8afe7247cf1178c1",
      "size": 20097,
      "subdir": "noarch",
      "timestamp":<PHONE_NUMBER>443,
      "url": "https://conda.anaconda.org/conda-forge/noarch/cuda-toolkit-12.4.1-ha804496_0.conda",
      "version": "12.4.1"
    }
  ]
}

This latest delay is most likely due to the issues with the anaconda.org backend (xref: conda/infrastructure#899). The channel cloning process relies on various calls to .org's API; the .org database was sporadically triggering the OOM (out-of-memory) killer on the backend host. Anaconda's infrastructure team has expanded memory and scaling allocation for the database backend, and that should help stabilize things again.

Could we write a process that periodically checks repodata-clone.json versus repodata.json to keep track of any delay between packages appearing in the former versus the latter

Thanks Cheng and Daniel! 🙏
Daniel, think that is a good idea. If there is some way to share log details or maybe graphs on resource usage, that might help as well
We were also wondering if it would make sense to have a GH template for CDN issues. Are there specific pieces of info we should be capturing that would help narrow things down?

Seeing issues today with https://anaconda.org/conda-forge/hyperion-fortran/files, not available after 2h

Am seeing this with recent llvmlite RC packages. These were upload ~2hrs ago. However it looks like none are available yet

To know whether recent packages can be pulled, one tend to watch when the last update to the repodata.json was made with:
watch -n 1 "curl -sI https://conda.anaconda.org/conda-forge/linux-64/repodata.json | grep 'last-modified'"


Feature Request: Filter for transactions in wallet

Would be cool to filter by incoming/outgoing using similar UI to the Orders page

This would require adjustments to the server API so you could query transactions by incoming / outgoing.

If we decide to do this, it will need a matching server issue for the new data.

We would have to plan this one out. I assume it would require design changes. It would also be a non trivial refactor to things since you'd want to make sure that state is shared between the different lists (e.g. you bumb the fee on a transaction in the 'incoming' filtered list and the same transaction in the unfiltered list reflects that).


lnwallet: use tail not tip in getUnsignedAckedUpdates

Commits:

9d0d5bdfaf130af695cbb420a774da1bf028aa8f modifies a comment
97007fc4faf984932b0e67a29b7fb77df3d7e9c8 fixes logUpdate scope in range loop
7da2080a3dfbe0ca74ff9408f29e6006c0cf05d0 is the patch (+1, -1)
0547364091ee8bf3f6c8ceeb925a1f625666d473 is the previously failing unit test

Patch summary:

getUnsignedAckedUpdates is a function used to restore on channel restart the signed updates our peer has sent us, but that we haven't signed for yet.  It only includes settles and fails.  RevokeCurrentCommitment calls it.  The state transition looks like this, before we store them:

us     <----fail----  peer
       <----sig-----
       -----rev---->

getUnsignedAckedUpdates previously would only select remote updates with LogIndex satisfying:

< lc.localCommitChain.tail().theirMessageIndex

(update is locked-in on our local commitment)

>= lc.remoteCommitChain.tip().theirMessageIndex

(update is not on any of the remote's commitments)

This pull changes the second requirement from tip() to tail().  This means that we restore the remote update until we receive a revocation covering it, at which point our logs are compacted and we can forward the settle/fail.  Otherwise, the following subsequent state transition leads to buggy behavior:

us     ----sig---->  peer
       <---add-----
       <---sig-----
       ----rev---->

RevokeCurrentCommitment will call getUnsignedAckedUpdates again, but this time the fail will not be stored because it is on the remote's tip commitment.  If we restart, we will not restore the fail, and will have a lingering add leading to channel force close.
With this fix, when the peer finally sends a revocation and locks in the fail, the update logs will be compacted and the fail can be forwarded. Additionally, the existing filtering logic in AdvanceCommitChainTail (called by ReceiveRevocation) will ensure that we no longer attempt to restore the update on restart.

but this time the fail will not be stored because it is on the remote's tip commitment. If we restart, we will not restore the fail, and will have a lingering add leading to channel force close.

Ahh, ok this makes the scenario much clearer for me, thanks for this description!


Added location showing property to tweets in WallMap.Fixes #287

@fuksi As Aneesh is sick today, please review this PR.

I checked the code it should surely work. It helps next time if you can post it here a link/case that I can see the effect easily. Merge

@fuksi  Please delete the branch after merging.


AjaxDownload is implemented using hidden iframe

https://issues.apache.org/jira/browse/WICKET-6286

@solomax Please close this PR! We will use the solution in https://issues.apache.org/jira/browse/WICKET-6286. Thanks!


Introduce EventIterator API

DeleteTracker() is confusing to users. It is essentially just providing an iterator for updates and deletes, so let's call it that: EventIterator().
var tbl Table[*Foo]
wtxn := db.WriteTxn(tbl)
iter, err := tbl.EventIterator(wtxn, true /* also deletes */)
wtxn.Commit()
if err != nil { return }

for range iter.Watch(db.ReadTxn()) { // Whenever there is something new...
  for ev, rev, ok := iter.Next(); ok; ev, rev, ok = iter.Next() {
    // iterate over the changes
  }
}

Functional difference to before is that this does not allow "reiterating" as one could do with DeleteTracker's IterateWithError that stops on error and allows picking up iteration again to retry. But that's not needed with EventIterator as the caller can take care of retrying the failed event before iterating again.

I'm keeping the same pattern of providing the WriteTxn to EventIterator(). This way it's very clear how the modification is pushed into the database. We could also have EventIterator grab a WriteTxn, but that is wrought with deadlock dangers and it's less clear to the caller from what snapshot onwards will the events appear from. So I think it's still the way to go, even though it is confusing to user to take a WriteTxn when you're reading.
Hmm though maybe it could take a ReadTxn instead and then figure out does it need to internally do a WriteTxn against the target table or not... I'll try that!

Thanks, I think this naming is much better for users that do not look that much into statedb details and just want to use it.
Regarding ReadTxn vs WriteTxn, I agree that it's fine to just require a WriteTxn if it's mentioned in the docs why it is needed.
Another option would have two iterators:
UpdateEventIterator(txn ReadTxn)

DeleteEventIterator(txn WriteTxn)

but not sure if this would not bring more confusion..

I think I'm missing the benefit of taking a ReadTxn instead of a WriteTxn - is it that "logically" an event iterator should be a read-only operation?
To me, the trade-off to clarity of what's needed in the iface is not worth it, and I'd advocate (again 😁) for just taking a WriteTxn.

The benefit is that it's way less confusing to users. You just want to observe changes to the table, so why would you need to take a WriteTxn?

Hmm, would ChangeIterator(...) (Iterator[Change], error) be better than Event?

Thanks, I think this naming is much better for users that do not look that much into statedb details and just want to use it.
Regarding ReadTxn vs WriteTxn, I agree that it's fine to just require a WriteTxn if it's mentioned in the docs why it is needed.
Another option would have two iterators:
UpdateEventIterator(txn ReadTxn)

DeleteEventIterator(txn WriteTxn)

but not sure if this would not bring more confusion..

DeleteEventIterator to me implies it's only about the deletions.

Hey, I ended up changing this up a bit:

EventIterator() is now called Changes() and returns a ChangeIterator
The "includeDeleted" parameter is gone. If someone really just wants updates then one can just do LowerBound search on the revision index directly.


Take into consideration all files types and yaml

Also .yml files need to be taken into consideration

I'd argue this file should be removed all together. Since we just recently integrated both PHP CS Fixer and StyleCI. I no longer see the use of this file. It doesn't affect this package. Only for some users who have this integrated within there editors.

This file is probably fine to stay.

StyleCI itself has this file, even though StyleCI is enabled on StyleCI. :P

this are different things @vinkla , PHP-CS-Fixer primary goals is not to fix json, yaml and so on

Failing tests ? 5.1 stable ??

It exists for another purpose then the CS Fixer and StyleCI but it doesn't do anything for the library. It is custom setup for the developers building this package. That is my opinion.

Sure @vinkla
I presume @jasonlewis will be glad to get only 4 spaces in composer.json files and not tabs in commits, or at least to avoid it


Update Rocky Linux EPEL install

dnf config-manager --set-enabled powertools
dnf install epel-release

Fixed by PR #43


OpenSSL failed in speed evp test for all of the aes-xxx-siv algorithms

I found the issue in daily performance test on latest version of OpenSSL.  Run OpenSSL speed test, and it report errors on all of the aes-xxx-siv test cases.  Suppose the new add code cause this issue.

This issue happened on both x86 and aarch64, maybe it also happened on other archs. I am debugging and  fixing up this issue.


add streaming tokens and model_name

(#4583)
@hwchase17, @baskaryan
My test code is as follows.
import asyncio
from langchain.callbacks import get_openai_callback
from langchain.chat_models import ChatOpenAI
from langchain.schema import (
    HumanMessage,
    SystemMessage
)

example_messages = [
    SystemMessage(content="You are a helpful, pattern-following assistant that translates corporate jargon into plain English."),
    #SystemMessage(name="example_user", content="New synergies will help drive top-line growth."),
    #SystemMessage(name="example_assistant", content="Things working well together will increase revenue."),
    #SystemMessage(name="example_user", content="Let's circle back when we have more bandwidth to touch base on opportunities for increased leverage."),
    #SystemMessage(name="example_assistant", content="Let's talk later when we're less busy about how to do better."),
    HumanMessage(content="This late pivot means we don't have time to boil the ocean for the client deliverable."),
]

async def main():

    for model in [
        "gpt-3.5-turbo-0301",
        #"gpt-3.5-turbo-0613",
        #"gpt-3.5-turbo",
        #"gpt-4-0314",
        #"gpt-4-0613",
        #"gpt-4",
        ]:
        print(model)

        with get_openai_callback() as cb:
            chat = ChatOpenAI(model_name=model, streaming=False, temperature=0, max_tokens=100)

            resp = chat.generate([example_messages])
            print()
            print(resp)

            print("prompt_tokens : " + str(cb.prompt_tokens))
            print("completion_tokens : " + str(cb.completion_tokens))
            print("total_tokens : " + str(cb.total_tokens))
            print("total_cost : " + str(cb.total_cost))

        with get_openai_callback() as cb:
            chat = ChatOpenAI(model_name=model, streaming=False, temperature=0, max_tokens=100)

            resp = await chat.agenerate([example_messages])
            print()
            print(resp)

            print("prompt_tokens : " + str(cb.prompt_tokens))
            print("completion_tokens : " + str(cb.completion_tokens))
            print("total_tokens : " + str(cb.total_tokens))
            print("total_cost : " + str(cb.total_cost))

        with get_openai_callback() as cb:
            chat = ChatOpenAI(model_name=model, streaming=True, temperature=0, max_tokens=100)

            resp = chat.generate([example_messages])
            print()
            print(resp)

            print("prompt_tokens : " + str(cb.prompt_tokens))
            print("completion_tokens : " + str(cb.completion_tokens))
            print("total_tokens : " + str(cb.total_tokens))
            print("total_cost : " + str(cb.total_cost))

        with get_openai_callback() as cb:
            chat = ChatOpenAI(model_name=model, streaming=True, temperature=0, max_tokens=100)

            resp = await chat.agenerate([example_messages])
            print()
            print(resp)

            print("prompt_tokens : " + str(cb.prompt_tokens))
            print("completion_tokens : " + str(cb.completion_tokens))
            print("total_tokens : " + str(cb.total_tokens))
            print("total_cost : " + str(cb.total_cost))

        print()

if __name__ == '__main__':
    asyncio.run(main())

If these changes look risky, I have prepared a CustomCallback sample as another solution.
import asyncio
from langchain.callbacks.base import BaseCallbackHandler
from langchain.callbacks.openai_info import *
from langchain.chat_models import ChatOpenAI
from langchain.schema.output import ChatGenerationChunk, GenerationChunk
from langchain.schema import (
    BaseMessage,
    HumanMessage,
    SystemMessage
)
from typing import Any, Optional, Union
from uuid import UUID

class CustomCallbackHandler(BaseCallbackHandler):
    completion_tokens: int = 0

    def on_llm_new_token(
        self,
        token: str,
        *,
        chunk: Optional[Union[GenerationChunk, ChatGenerationChunk]] = None,
        run_id: UUID,
        parent_run_id: Optional[UUID] = None,
        **kwargs: Any,
    ) -> Any:
        #print(f"'{token}'")
        print(f"\033[36m'{token}'\033[0m")
        if token is not None and token != "":
            self.completion_tokens += 1

        return super().on_llm_new_token(
            token, chunk=chunk, run_id=run_id, parent_run_id=parent_run_id, **kwargs
        )

def get_openai_token_cost(chat: ChatOpenAI, model_name: str, messages: List[BaseMessage]):
    prompt_tokens = chat.get_num_tokens_from_messages(messages=example_messages)
    completion_tokens = chat.callbacks[0].completion_tokens
    total_tokens = prompt_tokens + completion_tokens

    total_cost: float = 0.0
    if model_name in MODEL_COST_PER_1K_TOKENS:
        completion_cost = get_openai_token_cost_for_model(
            model_name, completion_tokens, is_completion=True
        )
        prompt_cost = get_openai_token_cost_for_model(model_name, prompt_tokens)
        total_cost = prompt_cost + completion_cost

    return (
        f"Tokens Used: {total_tokens}\n"
        f"\tPrompt Tokens: {prompt_tokens}\n"
        f"\tCompletion Tokens: {completion_tokens}\n"
        f"Total Cost (USD): ${total_cost}\n"
    )

example_messages = [
    SystemMessage(content="You are a helpful, pattern-following assistant that translates corporate jargon into plain English."),
    #SystemMessage(name="example_user", content="New synergies will help drive top-line growth."),
    #SystemMessage(name="example_assistant", content="Things working well together will increase revenue."),
    #SystemMessage(name="example_user", content="Let's circle back when we have more bandwidth to touch base on opportunities for increased leverage."),
    #SystemMessage(name="example_assistant", content="Let's talk later when we're less busy about how to do better."),
    HumanMessage(content="This late pivot means we don't have time to boil the ocean for the client deliverable."),
]

async def main():

    for model_name in [
        "gpt-3.5-turbo-0301",
        #"gpt-3.5-turbo-0613",
        #"gpt-3.5-turbo",
        #"gpt-4-0314",
        #"gpt-4-0613",
        #"gpt-4",
        ]:
        print(model_name)

        chat = ChatOpenAI(model_name=model_name, streaming=True, callbacks=[CustomCallbackHandler()], temperature=0, max_tokens=100)

        resp = chat.generate([example_messages])
        print(resp)
        print(get_openai_token_cost(chat, model_name, example_messages))

        chat = ChatOpenAI(model_name=model_name, streaming=True, callbacks=[CustomCallbackHandler()], temperature=0, max_tokens=100)

        resp = await chat.agenerate([example_messages])
        print(resp)
        print(get_openai_token_cost(chat, model_name, example_messages))

if __name__ == '__main__':
    asyncio.run(main())

Id be wary of calling the tokeniser for every token, would slow down everyone else's apps

Id be wary of calling the tokeniser for every token, would slow down everyone else's apps

Thanks for your comment.
The function get_num_tokens_from_messages is called only once, outside of the chunk loop. It's not being called for each token, so I believe the latency should be minimal. What do you think?

I think the latency depends on the OpenAI API when streaming=true.

Before

gpt-3.5-turbo-0301
i : 0
generate timediff : 0.9258830547332764
agenerate timediff : 0.9537229537963867
i : 1
generate timediff : 0.915074348449707
agenerate timediff : 0.9698758125305176
i : 2
generate timediff : 0.9312689304351807
agenerate timediff : 1.018787145614624
i : 3
generate timediff : 0.9073448181152344
agenerate timediff : 0.9569604396820068
i : 4
generate timediff : 1.0726542472839355
agenerate timediff : 0.8910963535308838

avg generate timediff : 0.9504450798034668
avg agenerate timediff : 0.9580885410308838

after

gpt-3.5-turbo-0301
i : 0
generate timediff : 1.104553461074829
agenerate timediff : 0.8502066135406494
i : 1
generate timediff : 0.7971365451812744
agenerate timediff : 0.753244161605835
i : 2
generate timediff : 0.759894609451294
agenerate timediff : 0.6960177421569824
i : 3
generate timediff : 0.8439314365386963
agenerate timediff : 0.6662440299987793
i : 4
generate timediff : 0.6385622024536133
agenerate timediff : 0.8396048545837402

avg generate timediff : 0.8288156509399414
avg agenerate timediff : 0.7610634803771973

test code

import asyncio
import statistics
import time
from langchain.callbacks import get_openai_callback
from langchain.chat_models import ChatOpenAI
from langchain.schema import (
    HumanMessage,
    SystemMessage
)

example_messages = [
    SystemMessage(content="You are a helpful, pattern-following assistant that translates corporate jargon into plain English."),
    SystemMessage(name="example_user", content="New synergies will help drive top-line growth."),
    SystemMessage(name="example_assistant", content="Things working well together will increase revenue."),
    SystemMessage(name="example_user", content="Let's circle back when we have more bandwidth to touch base on opportunities for increased leverage."),
    SystemMessage(name="example_assistant", content="Let's talk later when we're less busy about how to do better."),
    HumanMessage(content="This late pivot means we don't have time to boil the ocean for the client deliverable."),
]

async def main():

    for model in [
        "gpt-3.5-turbo-0301",
        #"gpt-3.5-turbo-0613",
        #"gpt-3.5-turbo",
        #"gpt-4-0314",
        #"gpt-4-0613",
        #"gpt-4",
        ]:
        print(model)

        t1 = []
        t2 = []
        for i in range(5):
            print(f"i : {i}")
            with get_openai_callback() as cb:
                chat = ChatOpenAI(model_name=model, streaming=True, temperature=0, max_tokens=100)

                start = time.time()
                resp = chat.generate([example_messages])
                end = time.time()
                timediff = end - start
                t1.append(timediff)
                print("generate timediff : " + str(timediff))

            with get_openai_callback() as cb:
                chat = ChatOpenAI(model_name=model, streaming=True, temperature=0, max_tokens=100)

                start = time.time()
                resp = await chat.agenerate([example_messages])
                end = time.time()
                timediff = end - start
                t2.append(timediff)
                print("agenerate timediff : " + str(timediff))

        print()
        print("avg generate timediff : " + str(statistics.mean(t1)))
        print("avg agenerate timediff : " + str(statistics.mean(t2)))

if __name__ == '__main__':
    asyncio.run(main())

What's the progress on this?

Apologies for the slow review! Pr has some merge conflicts, happy to re-review if you'd like to resolve

@baskaryan
Sorry for the late reply. Since OpenAI now supports stream_options, I believe it would be best to review the source code.

https://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156
https://platform.openai.com/docs/api-reference/chat/create#chat-create-stream_options

However, because Azure OpenAI Service does not yet support stream_options, I have not been able to consider it.

https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#completions

I would like to close this merge request.

I see llm_output.usage as blank even if I use Langchain with OpenAI . Any plan on adding the same to Langchain ?

Sorry for the delayed reply. I mentioned below.

https://github.com/langchain-ai/langchain/issues/4583#issuecomment-2485189046


Release for Scala 3.0.0-M1

ScalaCheck 1.15.0 has just been deployed to Sonatype, which should allow munit to be released.

I will take a look at making a PR if no one objects.

A backported release of 0.7.15 would be great, too ☺

Thaaat might be harder as I have no idea how to release it locally :sweat_smile:

No need to back port, we can continue publishing new 0.7.x versions.

True, there's no need, but it's a lot more convenient for downstream dependencies. It avoids churn going forward with 3.0.0-M*.

0.7.16 with support for 0.27.0-RC1 and 3.0.0-M1 is now released.


Add backticks around 'apt-get install'.

Yes, this is fiddly.

Happy to take even fiddly fixes :).  Two quick things:

Can you sign the Dropbox CLA?
Can you tweak the commit message to match http://zulip.readthedocs.org/en/latest/code-style.html#commit-messages?

done and done :D

Merged.  Thanks for your contribution!

Hope to do some more substantial ones soon!


"missing TrueType font file"

How can I use a custom font type Courier?
func main() {

	doc, _, err := reader.ParsePDFFile("/Users/tommynanny/Desktop/PDFWorker/template3.pdf", reader.Options{})
	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}

	// if L := len(doc.Catalog.AcroForm.Flatten()); L != 65 {
	// 	fmt.Println(fmt.Errorf("expected 65 fields, got %d", L))
	// 	os.Exit(1)
	// }

	err = formfill.FillForm(&doc, formfill.FDFDict{
		Fields: []formfill.FDFField{
			{
				T: "z1",
				Values: formfill.Values{
					V: formfill.Text("Nanlin Sun"),
				},
			},
		},
	}, true)

	if err != nil {
		fmt.Println(err)    // << --- "missing TrueType font file"
		os.Exit(1)
	}

	out, err := os.Create("/Users/tommynanny/Desktop/PDFWorker/output.pdf")
	if err != nil {
		fmt.Println(err)
		os.Exit(1)
	}

	if err = doc.Write(out, nil); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}

Also, I realize my pdf full of Japanese characters doesn't seem to work with reader.ParsePDFFile which gives me the error of "unexpected type for Outline item: <nil>". I was wondering if there is a work around?

Hi !
The missing TrueType font file is triggered because the embedded font file is not found on the PDF file. I will relax the code to gracefully accept this scenario, but I suspect there is another issue..
The unexpected type for Outline item: <nil> error is either a bug or comes from a non spec compliant PDF, but Japanese characters should be accepted.. Could you send me your PDF sample so that I give another look ?

e9db32e should fix the missing TrueType font file error

Hi !
The missing TrueType font file is triggered because the embedded font file is not found on the PDF file. I will relax the code to gracefully accept this scenario, but I suspect there is another issue..
The unexpected type for Outline item: <nil> error is either a bug or comes from a non spec compliant PDF, but Japanese characters should be accepted.. Could you send me your PDF sample so that I give another look ?

Thanks for the info and for taking a look at this. Here is the pdf that I am working on at the moment:
template.pdf

The unexpected type for Outline item:  error is either a bug or comes from a non spec compliant PDF, but Japanese characters should be accepted.. Could you send me your PDF sample so that I give another look ?

This is #5, and now fixed. Thanks for the report.


Refactor from dispatcher

First commit refactors dispatchers.from() by pulling out two functions.
Second commit removes the boolean arg from imageContexts.new() and splits it into two functions (one for each branch).

oh needs a rebase too :angel:

Rebased, and build is green

Small changes to builder FROM dispatcher.
Build is green


Investigate using cosmos-ansible for non-gaiad binaries/chains

Top of the list:

[ ] REGEN

Inventory files for starting a local testnet and joining the regen-redwood-1 testnet and regen-1 mainnet have been tested successfully.
Where should we list them in the READMEs?

[ ] Interchain Security
[ ] Interchain Accounts

Duplicate of #106


20.02 model load err

Description
A clear and concise description of what the bug is.
model load err
TRTIS Information
20.02
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
i use onnx-tensorrt convert onnx to trt model；（set Max batch size:  32）
i meet error with version 20.02。
but version 19.10（tensorrt 6）can work
failed to load 'mfnb_trt' version 1: Invalid argument: unexpected configuration maximum batch size 32 for 'mfnb_trt_0_0_gpu0', model maximum is 1 as model does not contain an implicit batch dimension nor the explicit batch-dimension of 'data_input' is a wildcard.
`name: "mfnb_trt"
platform: "tensorrt_plan"
max_batch_size : 32
dynamic_batching {
preferred_batch_size: [ 4, 8 , 16, 24 ,32]
max_queue_delay_microseconds: 100
}
input [
{
name: "data_input"
data_type: TYPE_FP32
format: FORMAT_NCHW
dims: [3, 112, 112 ]
}
]
output [
{
name: "fc1_Y"
data_type: TYPE_FP32
dims: [1, 512 ]
}
]
instance_group [
{
count: 1
kind: KIND_GPU
gpus: [ 0 ]
}
]
`

Take a look at https://github.com/NVIDIA/tensorrt-inference-server/issues/1105. Especially this comment https://github.com/NVIDIA/tensorrt-inference-server/issues/1105#issuecomment-586474508 and the following one.

Take a look at #1105. Especially this comment #1105 (comment) and the following one.

./onnx2trt fr_mfnb.onnx -b -1 -o fr_mfnb.plan
Building TensorRT engine, FP16 available:1
Max batch size:<PHONE_NUMBER>3709551615
Max workspace size: 1024 MiB
[2020-03-05 01:47:08   ERROR] Parameter check failed at: ../builder/builder.cpp::setMaxBatchSize::113, condition: batchSize > 0 && batchSize <= MAX_BATCH_SIZE

onnx2trt is a utility included with TensorRT optimizer/runtime. You should ask you question there and they should be able to help: https://github.com/NVIDIA/TensorRT


New command: planner bucket remove

Usage
planner bucket remove
Description
Removes the Microsoft Planner bucket from a plan
Options

Option
Description

--bucketId [bucketId]
ID of the bucket to remove. Specify either bucketId or bucketName but not both.

--bucketName [bucketName]
Name of the bucket to remove. Specify either bucketId or bucketName but not both.

--planId [planId]
ID of the plan to which the bucket to remove belongs. Specify either planId or planName when using bucketName.

--planName [planName]
Name of the plan to which the bucket to remove belongs. Specify either planId or planName when using bucketName.

--ownerGroupId [ownerGroupId]
ID of the group to which the plan belongs. Specify ownerGroupId or ownerGroupName when using planName.

--ownerGroupName [ownerGroupName]
Name of the group to which the plan belongs. Specify ownerGroupId or ownerGroupName when using planName.

--confirm
Confirm removal of bucket.

 I've worked on some of the other Planner commands, so feel free to reach out to me if you face any issues with this.

Hey @ktskumar, are you still working on this?

Yep. I'm working on this. Will push the code by this weekend

It seems that this command has been lost sight of. @waldekmastykarz I can continue working on this if desired?

Please do @milanholemans 👏

@waldekmastykarz is it ok if I rename
--bucketId to -i, --id
and
--bucketName to --name
to keep it consistent with other commands?

Since we have multiple id's and names, I'd suggest that we don't implement shorts, which would be ambiguous. If need be, we can always add them later.


fix(di): Remove log when start DI

:tada: This PR is included in version 6.6.1 :tada:
The release is available on:

v6.6.1
GitHub release

Your semantic-release bot :package::rocket:


Updates to support Ubuntu and also adds more configuration options

I made updates which:

allow for installation on Ubuntu as well as Debian and RHEL systems.
downloads RHEL6 CIS rules
allows selection of CIS rules by distribution
Adds configuration options, namely

disable email_notification
add command  configuration
add <syslog_output> on the server

Thanks!!!


Program doesn't work if search.txt file format saved as Unicode

Is there an existing issue for this?

[X] I have searched the existing issues

I'm submitting a ...

[X] bug report
[ ] feature request
[ ] support request --> Contact me over mail for support https://github.com/MShawon

Description
As title said.
Some YouTube videos contains #,"|| some specific characters. And when I save the search file in ANSI format with unicode characters  the program just doesn't start. But when characters removed from exact title it cant find the video even it shows on search engine.
Environment
- OS : windows 10
- Python : exe
- Script version : 106.0.5249.62

config.json
{
    "http_api": {
        "enabled": true,
        "host": "<IP_ADDRESS>",
        "port": 5000
    },
    "database": true,
    "views": 1000,
    "minimum": 85.0,
    "maximum": 95.0,
    "proxy": {
        "category": "p",
        "proxy_type": "http",
        "filename": "",
        "authentication": true,
        "proxy_api": true,
        "refresh": 0.0
    },
    "background": false,
    "bandwidth": true,
    "playback_speed": 1,
    "max_threads": 5,
    "min_threads": 2
}

send me your search.txt


Turkish translations.

Hello fellas,
I've made a little contribution that covers the Turkish translation in the app. So, I hope you find it useful and it helps you to get Turkey based usage.

@LeaveNhA
Thank you for your contribution.
I merged them right away and I am beyond grateful.


Making the D3D base header public so that external programs that deri…

…ve from it can include it using an install directory rather than requiring a build directory.

No .cpp files needed.

It just links against the standard library and uses it as any other RenderManager client.


update changelog for v2

please :)
as far as i can tell, you just updated superagent, but it's nice to know none of supertest's APIs were changed

correct, the only change was to bump the version of superagent to 2.0.0.
As that version lists many breaking (and exciting new) changes, I decided to bump supertest to 2.0.0 as well since they are closely related.


Activate features

My copy-and-pasta was disabling features and that is probably the cause of my failures with gdal and Python 3.4.

Hi! This is the friendly conda-forge-admin automated user.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


Boilerplate files for the process CG

@tabatkins, I'm unsure what's wrong, as these failures don't seem related to the boilerplate I'm proposing to add.

@tabatkins, I'm unsure what's wrong, as these failures don't seem related to the boilerplate I'm proposing to add.

It's not you, there was a pygment bugfix that decided to change the rendering 😆 . Fix for tests in https://github.com/tabatkins/bikeshed/pull/1885

It's not you, there was a pygment bugfix that decided to change the rendering 😆 . Fix for tests in https://github.com/tabatkins/bikeshed/pull/1885

rebased, getting a different CI failure, which still doesn't seem related to what I'm proposing here.

rebased, getting a different CI failure, which still doesn't seem related to what I'm proposing here.

CI is fixed again

CI is fixed again


Create a new notebook

Create a new branch
Create a new .ipynb file
Do some stuff
Submit a Pull Request

Starter Algos
https://chat.openai.com/share/5dd94804-a6b4-4692-95df-e215845ece64


added menu swapping for the slayer cave for the dagganoth kings lair

This would fix #4102

This is a niche feature that is best suited for the plugin hub, and is currently covered by the Menu swapper extented plugin there. See runelite/plugin-hub#42


Redirect to the first requested URL after SAML SSO performed

Motivation:
Currently the web browser is redirected to the root(/). It'd be better if we can redirect it to the first requested URL.
Modifications:

Add ref parameter when redirecting an unauthenticated request to /link/auth/login.

The parameter will be sent to an identity provider and get back via relayState parameter.

Thanks, @hyangtack !


DELETE EOF

i am using f-api-v4 branch, when delete a file, the go-gitlab return error : EOF.
my golang version is 1.8.3, my os is osx.

I would need a bit more details in prder to be able to help. Like a short example code showing which calls you make, and the error output you see. Thx.

this is my code , in order to fix this "bug", i add the EOF check,it works. In fact, when return EOF, go-gitlab also delete file success.

And what response do you get when you try this with a curl cms, like shown here: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-5-stable/doc/api/repository_files.md#delete-existing-file-in-repository
Does that give the same response?
And/or, if you look at the second return value (which is the response), you could check the content of resp.Body and post that back as well... Thx!


Create EF6 package

Fixes #124

That's great @MisinformedDNA. Thank you!
So, actually, we can use SDK-style projects? Why we've been bothering all this time then? 😅 Now that I'm thinking, most of my libraries that support 4.x are SDK style too. Brain freeze.
@ngruson You were helping us with EF6 and you created the project. Do you have any reason why we shouldn't use SDK style projects? Do you have any specific use-case?
Btw, sorry that you have been waiting for so long. We simply got stuck with the legacy structure and never published it.

No specific reason not to use SDK style projects. At the time, I just created a new .NET Framework 4.8 class library in VS2019 which creates a non-SDK style project by default.

Thanks Nils. The package is generated and I see no issues.
But, the publish task is failing since we need .NET Framework runtime. I see I have an issue on that one #112, and probably we'll need a Windows environment. We'll check how to do that with Github Actions.


displaying model edges only

Is your feature request related to a problem? Please describe.
currently option to displaying mesh files (STL, VTK, etc) are for edge mesh i.e based on each of element mesh.
when hidden edge mesh is de/active by 'e' shortcut key  it's apply to a whole mesh.

Describe the solution you'd like
displaying model edges only could be useful to given insight or impression about the model.

Can you share the file?

below is attachment links of model files (stl & vtk)
model_files.zip

thanks for the hints, regarding to difficulty in implementation. i'm only share a screenshot and experiences using another similar software as the same with VTK libraries.
i frequently discussed with him also, even i'm not really know how is complex inside. only an ordinary user which given an opinion/ideas about the modeling & post-processing of engineering simulation.

I had a look at the dataset, all the mesh is connected, so we cannot rely on just boundaries.
I agree with @mwestphal we need to use vtkFeatureEdges to process the dataset.

Could we add some kind of "edge dihedral angle" config option to achieve this feature by filtering out edges that are "flatter" that a user-defined threshold?

@snoyer yes that's what vtkFeatureEdges is doing

A vtkFeatureEdges prototype is needed before we move forward with this.


[BugFix] incorrect pitch/roll moments on tapered elements crossing water line

** This pull request is ready to be merged**

Feature or improvement description
There were two negative signs missing in the calculation of the centroid for tapered members crossing the still water line, and one incorrect variable name.  The negative signs were missing in the original implementation plan.
Related issue, if one exists
This had been reported by a collaborator on an ARPA-E ATLANTIS project.
Impacted areas of the software
HydroDyn module.  Only tapered members crossing the still water surface are affected
Additional supporting information
Test results, if applicable
Added a very simple test case of a tapered cylinder (0.1m taper over 40 m) with a prescribed pitch motion.
Original results showing pitching moment that was way too large:

Corrected results in this PR:

@mattEhall, take a look at the code change for updating the plan.  This now matches your python script (I can upload the pages of algebra if you want the proof).
Should we add some of that derivation into the docs at some point?

These corrections look good, Andy. They match what I have in a Python verification script and match the corrected derivation. The trends in the plot of corrected moments look right. Thanks for tracking the errors down!


Use correct ThumbnailOption model in migrations

Coverage decreased (-0.2%) to 96.324% when pulling ab7929648effd815b2d56eaef51d54fe1107ff15 on hotfix/thumbnail_model into 2b4037872919288d642c9b67646288ec70da7caa on release/0.8.x.

Current coverage is 92.48% (diff: 100%)

Merging #321 into release/0.8.x will decrease coverage by 0.32%

@@           release/0.8.x       #321   diff @@
===============================================
  Files                 22         22
  Lines               1224       1224
  Methods                0          0
  Messages               0          0
  Branches             147        147
===============================================
- Hits                1136       1132     -4
- Misses                42         45     +3
- Partials              46         47     +1

Powered by Codecov. Last update 2b40378...ab79296

Coverage decreased (-0.3%) to 96.242% when pulling 5fb767feb22afbfc352813de6cc14d46d848aa8c on hotfix/thumbnail_model into 2b4037872919288d642c9b67646288ec70da7caa on release/0.8.x.

Coverage decreased (-0.3%) to 96.242% when pulling 5fb767feb22afbfc352813de6cc14d46d848aa8c on hotfix/thumbnail_model into 2b4037872919288d642c9b67646288ec70da7caa on release/0.8.x.

Coverage decreased (-0.3%) to 96.242% when pulling 5fb767feb22afbfc352813de6cc14d46d848aa8c on hotfix/thumbnail_model into 2b4037872919288d642c9b67646288ec70da7caa on release/0.8.x.

Coverage remained the same at 96.569% when pulling a74348df2b670cd7f767facd574c4bc35f8c7ebf on hotfix/thumbnail_model into 2b4037872919288d642c9b67646288ec70da7caa on release/0.8.x.


isl_x_x: fix tarball urls

Resolves #141819

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

I do not really know how best to test this change.
It just downloads from the package from hydra; the nix hash seems to still be the same.
nixpkgs-review also did basically nothing.

After parameterizing the version, all of the isl packages want to be rebuilt.
All isl packages succeeded being built locally. (I will therefore remove the draft status)
Also, nixpkgs-review wants to rebuilt an insane amount of packages.
The amount of package rebuilds is just way to much for me to run.


implemented the exit dialog button in all cases of closing

Issue Number
fixes #103
Describe the changes you've made
I made the exit dialog box to appear each time while closing in any way.
for this i used protocao function with WM_DELETE_WINDOW.
Describe if there is any unusual behaviour of your code(Write NA if there isn't)
NA
Additional context (OPTIONAL)
Add any other context or screenshots about the feature request here.
Test plan (OPTIONAL)
A good test plan should give instructions that someone else can easily follow.
How someone can test your code?
Checklist

[ ] I have added my name in the contributors list at the end of README.md file.
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.
[x] The title of my pull request is a short description of the requested changes.
[ ] Open Source Program names(OPTIONAL: If you participated in any open-source program then please mention the program name here)

Below screenshots showing the corrected bug output,
here we tried to close the window from the very starting page, and we got the exit dialog box,

@QAZIMAAZARSHAD , i have created a pr for making the exit dialog box to appear every time when closing in any way, please check it


Remember window size and position

I love many features this app already has, but there is something that bothers me every time. I can't belive that no one brought this up by now, but for me it is highly annoying that the window size and position is not saved and restored after reopening. I am aware that this can be handled by using external software that manages all windows positions and sizies, but I think that the swiss army chainsaw of terminal emulators should be able to implement not only very sophisticated features but also those simple and tiny things that make our every day life so much easier :)
Nevertheless great work is done here, keep it up!

You're right. :smile_cat:
I've personally kind of gotten used to it mainly because when I open a terminal I leave it open for days and weeks at a time. The good news is that this project is changing gears a bit to filling out the feature list with a lot of the boring but expected features which every terminal app needs. I've been able try out some of the original features so far like the frames and now the easy up/downloads, but I'm missing some of the basics too.
You are welcome to help fill out the issue tracker here with desired 'standard' functionality.


Setup a mesop-rc pip package

This will enable us to Mesop pre-release versions

Example PR on how to do this: https://github.com/google/mesop/pull/334
Will add docs to internal/publishing.md


Do we want to keep unused code in backscope?

backscope has unused code. For example, see https://github.com/numberscope/backscope/blob/aeb7595e03e3fac502be666b259d0d5685131f90/flaskr/nscope/models.py#L12.
Should we keep it?
I don't think we should keep it. If we ever need it, can get it using Git.

Also, see:

I believe the plan is to remove unused code/files at the next opportunity.


fix(FileDetailsLabel): Add dark mode styling

I missed adding dark mode styling on the first pass - this should match the design.

Preview: https://virtual-assistant-pr-virtual-assistant-97.surge.sh


Cleanup DEPRECATED TLS config

issue: https://github.com/containerd/containerd/issues/8577

/ok-to-test


can not download private youtube videos

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've searched known issues and the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood
I have a link which is private youtube video. I can not down load it by yt-dlp even I use command --cookies-from-browser firefox.
The error message is
"no matches found
Provide verbose output that clearly demonstrates the problem

[ ] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[ ] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
No response

that is not a verbose log
quote the url


v1 post release ideas

push notifications?

there is a demand for equal functionality to the gig o website

Explore changing screen files to stateless widgets and moving all stateful widgets within to their own classes.

home page change left side gray line to a container light gray outline for each gig.

long pull down on home page rebuilds page with updated info

compact vs expanded view on the home page

auto hyperlink for any gig info text


BAU: Update no-ipv but client oauth session error description

Proposed changes
What changed

Change ClientOAuthSessionError description to be more relevant to the cause of the error.

Why did it change

It was "Missing context", which is vague. We should want the error description to implicate the cause of the error (there is a ClientOAuthSession but not an IpvSession).

Issue tracking

PYIC-6245

I have put a message on #di-orchestration-team to check if the error description is relied upon


@broskoski => Uses updated LotStanding endpoint instead of BidderStatus

This PR hopefully fixes some of the weirdness in the active bids module, following: https://github.com/artsy/metaphysics/pull/401 and https://github.com/artsy/gravity/pull/10349
Fixes a couple of things:

Your bidder_status used to be incorrect if you'd bid more than 10 times (because it only looked at the first page of me/bidder_positions
Date was not showing up next to the artwork title :)

In your collector profile, it now shows the lots that you've bid on for all live auctions under "My Active Bids" and lots that you've bid on in now-closed auctions under "Bid History". This also fixes the bug where it was showing multiple widgets for each lot (one per position instead of one per lot).

Very nice 💯


remove method of Plotter

plt = Plotter(N=2)
s = Sphere()
plt.add(s, at=0)
plt.add(s, at=1)
plt.remove(s, at=1)

When I remove s from renderer at index 1, the remove method will clear the plt.objects. But another s is in the actors list of the renderer at index 0.
Then I show this test and the camera is not reset. But if I remove another Sphere at index 1, the parameter resetcam=False is useless.
...
plt.remove(s, at=0)
plt.show(resetcam=False, at=1).interactive()

Eh, I didn't find the reason, the relation between "resetcam" and "sharecam" of Plotter is puzzling.

Reset_camera method or resetcam parameter of Plotter is not for all actors in it's instance, but the actors of the first renderer.
plt = vedo.Plotter(N=2, sharecam=True)
s1 = vedo.Sphere()
s2 = vedo.Sphere(pos=(30, 30, 30))
plt.add(s1, at=0)
plt.add(s1, at=1)
plt.show().interactive()

When sharecam is True, the Sphere at index 1 is out of the window.

It's a bit confusing indeed because it's complicated to address many use cases (jupyter etc) so show() is doing a number of extra things... Note that sharecam means that all renderers share the same camera - so it's normal that in
import vedo
plt = vedo.Plotter(N=2, sharecam=True)
s1 = vedo.Cube()
s2 = vedo.Sphere(pos=(30, 30, 30)).lw(1)
plt.at(0).add(s1, vedo.Axes(s1)).reset_camera()
plt.at(1).add(s2, vedo.Axes(s2))
plt.show().interactive().close()

one of the 2 goes out of the window! (click and press r to manually reset the cam). Also at=... does not shift the focus of the current renderer, maybe using .at(). is more convenient.
if you want to skip the show() this is maybe an option:

import vedo
plt = vedo.Plotter(N=2, sharecam=False)
s1 = vedo.Cube()
s2 = vedo.Sphere(pos=(30, 30, 30)).lw(1)
plt.at(0).add(s1, vedo.Axes(s1)).reset_camera().add_renderer_frame(lw=5)
plt.at(1).add(s2, vedo.Axes(s2)).reset_camera().add_renderer_frame(lw=5)
plt.render().interactive().close()

Nice! Take notes.
Method at changes the current renderer.
Thank you!

plt = Plotter(N=2)
s = Sphere()
plt.add(s, at=0)
plt.add(s, at=1)
plt.remove(s, at=1)

When I remove s from renderer at index 1, the remove method will clear the plt.objects. But another s is in the actors list of the renderer at index 0.

This question.
The value of property plt.objects is changed I remove objs from a renderer, even if these objs had been added in other renderers.
So it can't even used as amount of objs in current renderer.

Thanks for pointing out this. In fact the .object attribute is quiet useless and should be revised or even eliminated..
Consider:

from vedo import *
plt = Plotter(N=2)
s = Sphere().lw(1).alpha(0.8)
s.name = "MySphere"
c = Cube().alpha(0.8)
e = Ellipsoid().alpha(0.8)
plt.add(s, at=0)
plt.add(s, c, e, at=1)
plt.remove("MySphere", at=1) # you can also remove stuff by name
# print(plt.objects) # useless/wrong
print(plt.get_meshes(at=1))
plt.show().interactive()

Oh, let me try this method.


ASSIST - Prolog documentation

If I run the Prolog documentation server, as per the instructions here
https://github.com/ge-high-assurance/RACK/tree/master/assist/bin#prolog-api-usage
I get a series of errors as follows:
ERROR:  .../RACK/assist/bin/documentation/serve.pl:20:
ERROR:    import/1: No permission to import requirements:mitigates/2 into user (already imported from hazard)
ERROR: .../RACK/assist/bin/documentation/serve.pl:20:
ERROR:    import/1: No permission to import requirements:assert_author/2 into user (already imported from hazard)
ERROR: .../RACK/assist/bin/documentation/serve.pl:20:
ERROR:    import/1: No permission to import requirements:assert_mitigates/2 into user (already imported from hazard)
ERROR: .../RACK/assist/bin/documentation/serve.pl:20:
ERROR:    import/1: No permission to import requirements:author/2 into user (already imported from hazard)
ERROR: .../RACK/assist/bin/queries/hazard_structure.pl:18:
ERROR:    import/1: No permission to import requirements:mitigates/2 into hazard_structure (already imported from hazard)
ERROR: .../RACK/assist/bin/queries/hazard_structure.pl:18:
ERROR:    import/1: No permission to import requirements:assert_author/2 into hazard_structure (already imported from hazard)
ERROR: .../RACK/assist/bin/queries/hazard_structure.pl:18:
ERROR:    import/1: No permission to import requirements:assert_mitigates/2 into hazard_structure (already imported from hazard)
ERROR: .../Work/RACK/assist/bin/queries/hazard_structure.pl:18:
ERROR:    import/1: No permission to import requirements:author/2 into hazard_structure (already imported from hazard)

Incidentally, since the Prolog ontology files are no longer included in the RACK distribution, as per
https://github.com/ge-high-assurance/RACK/commit/b90f96c4df07c8c72ab7b5b3c47e4cbd2e96f526
the instructions for running the documentation server should mention the need to auto-generate the ontology files.

Indeed that's embarassing, hah Prolog!
I will see if I can conditionally list the ontology files based on whether they have been generated or not.
Will update.

Thanks for looking into this - I ran into another problem today that might be related:
?- [assist/bin/ontology/prov-s].
ERROR: source_sink `assist/bin/ontology/prov-s' does not exist
ERROR: In:
ERROR:   [20] throw(error(existence_error(source_sink,...),_11088))
ERROR:   [16] '$resolve_source_path'(... / prov-s,_11120,[expand(true)]) at /opt/homebrew/Cellar/swi-prolog/8.2.4/libexec/lib/swipl/boot/init.pl:2318
ERROR:   [15] '$load_file'(... / prov-s,user,[expand(true)]) at /opt/homebrew/Cellar/swi-prolog/8.2.4/libexec/lib/swipl/boot/init.pl:2292
ERROR:    [9] <user>
ERROR:
ERROR: Note: some frames are missing due to last-call optimization.
ERROR: Re-run your program in debug mode (:- debug.) to get more detail.
?- [assist/bin/ontology/provs].
ERROR: import/1: No permission to import 'prov-s':entity/1 into user (already imported from rack)
true.

It looks as though SWI Prolog doesn't like modules names that include a minus sign - maybe they need to be quoted in some way.
As you can see, renaming the file solved the problem, and left me with the familiar No permission error.

In SWI Prolog the consult(filename) command (with alternate syntax [filename]) expects the filename portion to be an atom.  SWI Prolog will automatically interpret bare text as an atom, but if that atom contains special characters, it should be single-quoted to make the atom explicit.
For the case reported above, the syntax should be ['assist/bin/ontology/prov-s'].


'heroku' and 'html' icons stacking issue

Hi,
I am experiencing an issue with the icon-html and icon-heroku when using the stack feature;
<span .fa-stack .fa-2x>
  <i .fa .fa-circle .fa-stack-2x>
  <a href="http://heroku.com" target="_blank">
    <i .icon-heroku .fa-stack-1x>

The icons appear to be too small, and not centred.
This has only arisen in version 2, so I have reverted back to 1 for the time being, at the expense of the mongodb icon.
thanks

Screenshot would make this much easier to visually understand.

v1.2

v2.1

v2.2

Interesting.  The main change between 1.2 and 2.x was the version of fontcustom -- which compiles the font and creates the .css.  That's likely the culprit.  You could diff the .css between the versions to see what changed as far as display styling.  I'd be interested what you find out.

Doesn't appear to be a css issue, which makes sense as not all icons are affected.
My own worked around is just to add padding to the couple of icons I need;
.icon-heroku
  padding:3px 0 0 15px

.icon-html
  padding:3px 0 0 6px