modules: Introduce hal-nxp-mcux-sdk

Introduces the new GitHub-based distribution of the MCUXpresso SDK as a
module in the west manifest. This new distribution provides the same
bare metal drivers as those downloaded from mcuxpresso.nxp.com and
imported into the hal_nxp module, however in this distribution they are
organized in a different directory structure that doesn't duplicate each
driver for every device. It can be integrated as-is into Zephyr without
the reorganization done in the hal_nxp module, and will therefore
simplify future updates of the SDK into Zephyr.
Currently the west manifest points to an NXP-hosted GitHub repo, but
will be updated once we have a Zephyr-hosted fork available.
Signed-off-by: Maureen Helm<EMAIL_ADDRESS>
[ ] Depends on #32278
[ ] Depends on NXPmicro/mcux-sdk#6

cc: @nxp-susan

I assume its reasonable to say that for a given board/SoC we'd never mix drivers from one HAL or the other? If this is the case, seems like we can simplify the Kconfig a bit.

I hope so, but I'm not sure yet.

Feels like you can use the same Kconfig symbols in both the current zephyr HAL repo and the new mcux-sdk based repo, and just have a high level switch on MCUX v MCUX_SDK.

I agree having both MCUX and MCUX_SDK prefixed symbols feels like a lot of duplication, but I also don't think there should be any interdependencies between the two modules. If we did have a unified set of Kconfig symbols, where would they be defined? Also, the MCUXpresso SDK is considering adding support for Kconfig in NXPmicro/mcux-sdk#6, in which case we would only use the symbols in Zephyr, not define them in Zephyr.

Don't the symbols need to be defined in Zephyr in order not to break Kconfig for people with custom west manifests where the MCUX SDK is not included?

Don't the symbols need to be defined in Zephyr in order not to break Kconfig for people with custom west manifests where the MCUX SDK is not included?

Now that you mention it, I think I remember hearing this during one of the module discussions.
@tejlmand can you give some guidance? Is there a way we can use Kconfig symbols defined in a module? These symbols are not specific to Zephyr and could be used by other projects.
This is a simplified version of what I want to do:
In the module:
config HAS_MCUX_SDK_DRIVER_FLEXCAN
	bool

config MCUX_SDK_DEVICE_MK64F12
	bool "MK64F12"
	select HAS_MCUX_SDK_DRIVER_FLEXCAN

config MCUX_SDK_DRIVER_FLEXCAN
	bool "FLEXCAN"
	depends on HAS_MCUX_SDK_DRIVER_FLEXCAN
	help
	  Build the FlexCAN bare metal driver

In Zephyr:
config CAN_MCUX_FLEXCAN
	bool "MCUX FlexCAN driver"
	depends on HAS_MCUX_SDK_DRIVER_FLEXCAN
	imply MCUX_SDK_DRIVER_FLEXCAN
	help
	  Build the FlexCAN shim driver

Don't the symbols need to be defined in Zephyr in order not to break Kconfig for people with custom west manifests where the MCUX SDK is not included?

Now that you mention it, I think I remember hearing this during one of the module discussions.
@tejlmand can you give some guidance? Is there a way we can use Kconfig symbols defined in a module? These symbols are not specific to Zephyr and could be used by other projects.

If you refer to a Kconfig symbol defined in a module from a Kconfig file in the Zephyr repo, then that symbol must also exists inside Zephyr itself, in order for everything to work when the module is not present.
This is why all known Zephyr modules have their Kconfig files inside the Zephyr repo: https://github.com/nrfconnect/sdk-zephyr/tree/master/modules
If you only want the settings visible / configurable to the user when the module is actively present, then you can just have a dummy duplication inside Zephyr.
Like this:
if 0
# These are dummy definitions to avoid unknown symbols when the module is missing.
config HAS_MCUX_SDK_DRIVER_FLEXCAN
	bool

config MCUX_SDK_DRIVER_FLEXCAN
	bool
endif

config CAN_MCUX_FLEXCAN
	bool "MCUX FlexCAN driver"
	depends on HAS_MCUX_SDK_DRIVER_FLEXCAN
	imply MCUX_SDK_DRIVER_FLEXCAN
	help
	  Build the FlexCAN shim driver

That will ensure Zephyr builds, and only if the module is present, then the settings are configurable as those settings are now available in the module with prompts.
This is in principle what we do here:
https://github.com/zephyrproject-rtos/zephyr/blob/0106d8f2a391fffcefabd053e17c683ea2e53d8a/modules/Kconfig#L69-L74
and then as soon as the module is present, the Kconfig file is sourced properly with all symbols available to the user :smile:


Support Etag in AppConfig APIs

There are several APIs that would benefit from having support for ETag.

For my own interest, this issue looks similar to https://github.com/Azure/azure-sdk-for-java/issues/9279

For my own interest, this issue looks similar to #9279

Duplicate issue so close it.


How to retrain the network

Thank you for releasing this wonderful work. Is there a plan to release the code for training? Particularly, the losses as described in the paper.

Hi @diamondspark, thanks for your question. We plan to release training code in the future for some new models that we're working on (upgraded versions of this model using newer modules) , but likely not for this one. All the best, R.


Filter instance and array references in ClassGenerator

These should be ignored and not added as locals.

Codecov Report

Merging #217 into master will not change coverage.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master     #217   +/-   ##
=========================================
  Coverage     79.57%   79.57%
- Complexity      377      379    +2
=========================================
  Files            46       46
  Lines           935      935
  Branches        175      175
=========================================
  Hits            744      744
  Misses          114      114
  Partials         77       77

Impacted Files
Coverage Δ
Complexity Δ

...ine/testgenerator/jimpleevosuite/ClassGenerator.kt
87.27% <100%> (ø)
35 <0> (+2)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 23086a5...99e0881. Read the comment docs.

@JSAbrahams They shouldn't be added, because they are not locals (they are expressions using locals). Field references to the library should be persisted (they are not 'unbound variables'), and array references should not be added as complete references (e.g. a[2]) but only as base objects (a), which are in the use-def boxes, anyways

@FWDekker I don't see how these test cases would represent valid usecases. The ClassGenerator creates its own class and adds the test method(s) to it - at no point are fields added to the test class, and thus there is also no difference between library field uses and internal field uses. Both are external to the test class, and thus dealt with in the same way


if I use JWT Auth as api auth and use Illuminate\Support\Facades\Auth as web auth,what to do ?

if I use jwt Auth as api auth and use Illuminate\Support\Facades\Auth as web auth,in my user model, what is Authenticatable ?

Hi, @lvtu0316!
This package doesn't depend on Authenticatable class in any way. Just make any model reacterable following to the Setup Reacterable documentation section. In example code block I provided Authenticatable because it's default User model from the first Laravel installation.
If you still have questions - please describe them in more detail.

The User Model is:
use Illuminate\Foundation\Auth\User as Authenticatable;
class User extends Authenticatable implements JWTSubject, LikerContract
use Cog\Contracts\Love\Liker\Models\Liker as LikerContract;
use Cog\Laravel\Love\Liker\Models\Traits\Liker;
{
    use Liker;
}

The Api/TopicsController is:
// 点赞
    public function like(Topic $topic, User $user)
    {

        if ($user->hasLiked($topic))
        {
            $user->unlike($topic);
        }else{
            $user->like($topic);
        }

        $likers = $topic->collectLikers();

        return $this->response->array($likers);

    }

but
"message": "Type error: Return value of Cog\\Laravel\\Love\\Likeable\\Services\\LikeableService::getLikerUserId() must be of the type integer, null returned",`

I sloved by:
public function like(Topic $topic)
    {
        $user = \Auth::user();
        if ($user->hasLiked($topic))
        {
            $user->unlike($topic);
        }else{
            $user->like($topic);
        }

        $likers = $topic->collectLikers();

        return $this->response->array($likers);

    }

Ah... you are using old version of the package. Old version was tightly coupled with default Laravel application. If it's not on production yet, I recommend you to start integration from the scratch and take v8.1.2 as the basis. If you can't switch to the new version for some reason - tell me what exact version are you using.

I used v5.1, ok, I will upgrade to the latest version! Thanks .

I'm closing this issue. Feel free to open new issues if there will be another questions.


Update nerves systems

Coverage increased (+0.03%) to 31.178% when pulling 29818ad0bda79f6d7a8a0190ef8ecd5d48420421 on system-updates into 189181b1386658cfe4a7b20e074a71268bf61967 on next.


Adopt aspects from Eventuous (The Right Way)

I really like your fully end-to-end approach with its thorough design and documentation. Great work.
But I would love to see in your solution the adoption of some aspects of Alexey Zimarevs The Right Way and its C# library Eventuous, e.g.

Separation of (immutable) state management usable without aggregates

Required for getting rid of aggregates (behaviour) from the core of a generic ES implementation

Support for functional domain models and functional services by using isolated lambdas instead of state mutation within aggregates

Functional programming helps in managing side effects, e.g. due to state mutations within multi-threaded applications

Event Store support for real-time subscriptions usable for projecting read models or publishing integration events

For the (in Java land) missing C# (and go-lang) construct of Channels the Java-based CSP (Communicating Sequential Processes) implementation JCSP can be used

This feature request goes against the initial goals of this project.
The goal of my project is to show that event sourcing and CQRS can be added to almost any existing application with PostgreSQL database with little efforts.
You can install and manage Event Store or buy their cloud solution and learn new approaches suggested by Eventuous but this is optional and you can stay with PostgreSQL and still have event sourcing.
I've never worked for Event Store company (like the author of the article you are referencing to), so I don't see any reason to promote their approach and vision. Also, I don't like clickbait headlines like "The Right Way". There are many ways with with their pros and cons.
I want my project be as simple as possible. I'm not building a Java-port of a C# framework.

Sorry, I didn't want to offend you.

The goal of my project is to show that event sourcing and CQRS can be added to almost any existing application with PostgreSQL database with little efforts.

As I said, I like your design and documentation. Unluckily your implementation is not usable for me - it's not generic, but closely coupled to Spring, which doesn't fit into my current tech stack.

If the problem is with the tight integration with Spring Framework, it's another question. I think I can make some "core" Gradle module that will not depend on Spring, and then wire everything up with the Spring config. It will be possible to use another DI framework.

Spring Boot is still so much more popular

No need to sell Spring to me - I've been using Spring in commercial environments since 2003 and helped to build the first tooling support ;-)
But luckily today there're other alternatives available than back in the old J2EE days. Newcomers (like Quarkus and Micronaut) may not be mainstream yet - but in the J2EE days it took Spring some time to get traction as well.

To have less confusion, I emphasis that the project is built with Spring Boot:

A reference implementation of an event-sourced system that uses PostgreSQL as an event store built with Spring Boot.


Add readme.md files for all the folders inside level 2.

Description of  the change
Add readme.md files for all the folders inside level 2.
Domain

[x] Readme

Are you contributing under any open-source program ?
SSOC'22

Great issue start working on it !


Cannot delete content of multiple cells

Problem description
If a user wants to delete content from A1:A4 (for example), it is not possible at the moment. Something similar happens if you want to remove content from an entire row. You can only make changes to cells one by one (double click on each cell).

Adding more context to this issue.
Steps to reproduce it

The user uploads a csv/xls file.
He/she/they want to select a range to delete data.

Right now it is not possible to select more than 1 cell to edit data.
Suggested solution
Allow the user to select more than one cell and remove data with the Backspace keyboard key

I tested the change in Mac and I have two comments:

To delete cells content in Mac the user needs to select fn + backspace. When performing this action, content from selected cells is remove.
However, when a user selects a range and then backspace, nothing happens.
Could we also implement the "delete" when using just the backspace key? Can we make sure this also works for other OS?

Sure. Will add a PR also for the backspace key

@romicolman @roll Please check again that it is working with the backspace key press as well. Before it was only working with the delete.

I tested this on the PR. It is not working on Mac


Add tls support

When setting up the zipkin endpoint, it'd be helpful to enable TLS
OkHttpSender.create("https://<IP_ADDRESS>:9411/api/v2/spans")
Ideally, we could pass in a jks/pkcs12 cert, password, and a CA bundle

Referencing https://github.com/openzipkin/brave/issues/691 as that should be a dependency

wrong project zipkin-reporter-java is what you want


WIP: introduce NodeInclusionPolicy in PodTopologySpread

Signed-off-by: kerthcet<EMAIL_ADDRESS>
part of https://github.com/kubernetes/enhancements/issues/3094#issuecomment-1082526739

/assing
/hold for exception, this KEP has been removed from the milestone
/sig-scheduling

close as Exception is rejected. I'll create a new one to release v1.25 then.

/milestone clear


Move the "version" from "diddiparser.parser" to "diddiparser.init"

Maybe this is not an issue, but I'll show it.
In many 3rth-party packages, you can access to the version variable with this:
>>>from package_name import __version__
>>>__version__
9.2.7

And it can be really useful with the issue tracking, version checks, etc. I really like that feature.
But I tried a similar code with DiddiParser, and returned AttributeError: 'diddiparser' has no attribute '__version__'. Then I used diddiparser.parser instead of just diddiparser, and it worked fine! I propose to move that variable to diddiparser/__init__.py to avoid these problems.

By now, let's just move the __version__. I will also merge PR #5 now.

We will talk about __author__ later... but now I will close the issue.


Conversions to and from Godot types should be very effortless now for the user.


Roll engine 8176f4d3c467..0fa159a2646d (2 commits)

https://github.com/flutter/engine/compare/8176f4d3c467...0fa159a2646d
git log 8176f4d3c467..0fa159a2646d --no-merges --oneline
0fa159a26 Add debug info to figure out why the bots are unhappy (flutter/engine#6369)
e031ccef7 Add missing binding for pushOffset (flutter/engine#6367)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

@rmistry @liyuqian : New roll has been created by https://autoroll.skia.org/r/flutter-engine-flutter-autoroll

Trybots failed. These were the failed builds: https://flutter-dashboard.appspot.com/build.html

Commit queue failed; closing this roll.


Post: About Top Teams in CA

Team history, location, size and age groups, so it's easier to decide what team to join for new people.
--
E.g. Interesting things like the following about Alto Velo (http://www.cyclingnews.com/news/alto-velo-settles-naming-rights-lawsuit-scraps-domestic-elite-team/)


fix: some tensors are in bf16

numpy does not support bf16, so we conservatively upcast to f32.
I encountered some loras with (some?) tensors in bf16.

Yes, I forgot to do it for lora, I did it for model weights before.

ah right. thanks for correcting :)

ah, you seem to down cast it to f16, have you seen any issues with that? I went with the safe route of upcasting to f32, since bf16 is a chopped-off f32.

Nothing special, just consistency with the model. I also forget why I switched to an f16 in the first place, it seems like an f32 would be a better choice.


Custom TERRAKUBE ADMIN group

Unable to change and validate TERRAKUBE_ADMIN group name
I've edited every entry related with TERRAKUBE_ADMIN and also configured ldif files but i'm unable to create an organization
Unable to find logs related with the organization creation.
Is there any other documentation related with that? I've been checking dex connector but seems that there's something missing

Hello @Eric-Llistar if you are using the helm chart you just need to overwrite the group name like this:
## Terrakube Security
security:
  adminGroup: "<<CHANGE_THIS>>"

That will overwrite the value here
https://github.com/AzBuilder/terrakube-helm-chart/blob/efd3585024cd7236cabfd389ed090d3d2add721f/charts/terrakube/templates/secrets-api.yaml#L19
The container is using this environment variable to get the admin group name
TERRAKUBE_ADMIN_GROUP
I hope this can help

Sorry! I've deployed via docker compose

Sorry! I've deployed via docker compose

It will be the same just update that environment variable
https://github.com/AzBuilder/terrakube/blob/9ef544546474031188c955d1bc68b6d98a93f97f/api/src/main/resources/application.properties#L58

Hi Alfredo!
First of all thanks for your quick answers.
Unfortunately still not working.. I've just edited this variable twice:
org.terrakube.owner=${TERRAKUBE_ADMIN_GROUP:MY_GROUP_NAME}
org.terrakube.owner=$"MY_GROUP_NAME"
But didn't work
I've also modified the config-ldap.ldif on ##group definition##  section
dn: cn=MY_GROUP_NAME,ou=XXX,ou=XXX,dc=XXX,dc=XXX
objectClass: groupOfNames
cn: MY_GROUP_NAME
member: cn=username,ou=MY_GROUP_NAME,ou=XXX,ou=XXX,dc=XXX,dc=XXX
Is that configuration correct?
Thanks!

you need to make sure to update the org name
    dn: {{ .Values.openldap.baseRoot }}
    dc: example
    objectClass: dcObject
    objectClass: organization
    o: Example, Inc

The user here
    dn: cn={{ .Values.openldap.adminUser }},ou=users,{{ .Values.openldap.baseRoot }}
    objectClass: inetOrgPerson
    sn: Admin
    cn: Admin
    mail<EMAIL_ADDRESS>    userpassword: {{ .Values.openldap.adminPass }}

This part also
    dn: cn=TERRAKUBE_ADMIN,ou=Groups,{{ .Values.openldap.baseRoot }}
    objectClass: groupOfNames
    cn: TERRAKUBE_ADMIN
    member: cn={{ .Values.openldap.adminUser }},ou=users,{{ .Values.openldap.baseRoot }}

And you also need to change this environment variables for the openldap container
https://github.com/AzBuilder/terrakube/blob/90a5eb260293ea1c734b3534a70e882003df9712/docker-compose/docker-compose.yaml#L38
I hope that can help you

Sure. Still the same. Let me share with you which files I've modified in order to try to find what is the problem:

terrakube/api/src/main/resources/application.properties:

################
#OWNER INSTANCE#
################
org.terrakube.owner=${TERRAKUBE_ADMIN_GROUP:**MY_ADMIN GROUP_NAME**}

terrakube/docker-compose/api.env
TERRAKUBE_ADMIN_GROUP=**"MY_ADMIN GROUP_NAME"**

terrakube/docker-compose/config-ldap.ldif

dn: dc=**my_company,dc=com**
dc: **my_company**
objectClass: dcObject
objectClass: organization
ou: **my_company**

dn: ou=**users_group,dc=my_company,dc=com**
ou: **users**
objectClass: organizationalunit

dn: cn=**admin_user,ou=users_group,dc=my_company,dc=com**
objectClass: inetOrgPerson
sn:
cn: **admin_user**
mail: **email**
userpassword: **admin_password**

# Group definitions.

dn: ou=**users_group,ou=users,dc=my_company,dc=com**
objectClass: organizationalUnit
ou:**users_group**

# TERRAKUBE_ADMIN
dn: cn=**MY_ADMIN GROUP_NAME,ou=users_group,ou=users,dc=my_company,dc=com**
objectClass: groupOfNames
cn: **MY_ADMIN GROUP_NAME**
member: cn=**admin_user,ou=users_group,dc=my_company,dc=com**

terrakube/docker-compose/config-ldap.yaml

connectors:
- type: ldap
  name: OpenLDAP
  id: ldap
  config:
    # The following configurations seem to work with OpenLDAP:
    #
    # 1) Plain LDAP, without TLS:
    host: **MY_HOSTNAME**:389

    # This would normally be a read-only user.
    bindDN: cn=**admin_user,ou=users_group,dc=my_company,dc=com**
    bindPW: **admin_password**

    usernamePrompt: Email Address

    userSearch:
      baseDN: **ou=users_group,ou=users,dc=my_company,dc=com**
      filter: "(objectClass=person)"
      username: userPrincipalName
      # "DN" (case sensitive) is a special attribute name. It indicates that
      # this value should be taken from the entity's DN not an attribute on
      # the entity.
      idAttr: DN
      emailAttr: userPrincipalName
      nameAttr: cn

terrakube/docker-compose/docker-compose.yaml

ldap-service:
    image: bitnami/openldap:2.6.4-debian-11-r4
    environment:
      LDAP_TLS_VERIFY_CLIENT: try
      LDAP_ADMIN_USERNAME: "**admin_user**"
      LDAP_ADMIN_PASSWORD: "**admin_password**"
      LDAP_ROOT: "**dc=my_company,dc=com**"
      LDAP_CUSTOM_LDIF_DIR: "/ldifs"

ldap.yaml seems to be ok since I'm able to log in with domain credentials, but at the time to create an organization, I receive a "without permissions" message to create it
Is there any other file or section that should I fill?
Thanks!!
PD: Asterisks used to be to mark text as Bold but not working very well


Playing the same raw url on different domains

Hi,
I have an issue regarding the youtube-dl -g [url] command. As the extracted information is only valid for the ip adresses that are in the same domain, I would like to know if there is anyway to use the same url in different ip domains? ,
I tried the cookies option and forwarded the cookie file to different ip domains but no luck.. Currently it seems that only ip is the barrier to share the url on different ips/machines.

See https://github.com/rg3/youtube-dl/issues/9001#issuecomment-201991818.

Hi Yan12125, the problem is with the ip section at the URL.
Fx I have the following raw URL extracted by youtube-dl where the IP address is bold and underlined :
https://redirector.googlevideo.com/videoplayback?ip=<IP_ADDRESS>
and I want to play this video at the <IP_ADDRESS>
I dont think this is the issue for the redirector, at least as I try the redirector at the another IP it does not work..

The redirector approach is just a workaround and some people found it works. Essentially there's no guarantee that the URL resolved from one IP is valid for another IP. If you want a valid URL for <IP_ADDRESS>, run youtube-dl on <IP_ADDRESS>, too.
Other options can be proxies, while they're far out of the scope of youtube-dl.

Using the redirector and same ip is it still necessary to also have the cookie.  Or would it using the redirector and same IP be enough?


Removed cell content aligment options from table editor context menu.

All other options were left intact.

Merged in 6c9d202057babfb1fbe59a1bd88a9f4b5273a846 - thanks!


Fetching leaderboards using javascript

Hello, I would like to display the results of three different competitions on my own website and sum the scores of the challenges to provide the total score, something like this:

where the separate scores are from 3 different leaderboards of Codabench challenges (I can't link them since they are not published yet).
To fetch the scores, I am using javascript as follows:
async function getData() {
 \\ Here in the end we will have 3 URLs to 3 different challenges, once they are published
  const url = "https://www.codabench.org/api/competitions/2626/results.json";
  try {
     const response = await fetch(url);
     if (!response.ok) {
       throw new Error(`Response status: ${response.status}`);
     }

     const json = await response.json();
     console.log(json);

    \\ Results in the table are achieved by hardcoding the json to show an example of the desired output
    \\ json = {"User":"a3d3","Score A3D3":"0.8","Score imageomics":"0.6","Score iHARP":"0.7"};
    json["Total score"] = calculateTotalScore(json);
    displayTable(json);

     // Check if the response contains an error message
     if (json.detail) {
       displayError(json.detail);
     } else {
     // Assuming json contains valid data, display it as a table
       displayTable(json);
     }
   } catch (error) {
     console.error(error.message);
     displayError('Failed to load data.');
   }
}

If I put the URL https://www.codabench.org/api/competitions/2626/results.json directly in the browser, I can view the results of my unpublished competition. However, using fetch(url) in js throws an error:

Is there a way to fetch the leaderboard results from Codabench via javascript fetch(url)? Or maybe there is an alternative way that I could use to fetch the results of challenges to be displayed on my website? many thanks!

This is expected problem when cross-origin requests are not allowed. You can use different tools to get the required result e.g. using Postman app.
I am closing this because it is not an issue. This article may help you
https://medium.com/@dtkatz/3-ways-to-fix-the-cors-error-and-how-access-control-allow-origin-works-d97d55946d9


130-Fixing the latest release url

Signed-off-by: sibyabin<EMAIL_ADDRESS>resolves #130

@dennyglee  could you review this please ( ref : #130 )

Looks great - thanks @sibyabin for the contribution!    Could you ping me your details via Slack?  Thanks!


Enhanced Readability on Small Screens

Adjust text size for better readability on smaller screens
@media (max-width: 480px) {
.footer-text {
font-size: 12px;
}
}

@Sanatan25 Can you describe the problem in detail?


Enhance the CustomProcessUpload sample with import BPMN use case

This is an enhancement to cover the import of bpmn diagrams via the API.

CLA check status is not updated even if the CLA was signed:


ROBUSTNESS: Add assertion that the rserver process started

Issue
Two users reported the following problem when we upgraded from RStudio Server  2022.02.4+500 to 2023.03.0+386:
$ rsc start --verbose
TTY detected. Printing informational message about logging configuration. Logging configuration loaded from '/etc/rstudio/logging.conf'. Logging to '/wynton/protected/home/legrand/nick/.local/share/rstudio/log/rserver.log'.
2023-03-30T17:46:24.296317Z [rserver] ERROR Unexpected exception: File name too long [system:36]; LOGGED FROM: int main(int, char* const*) src/cpp/server/ServerMain.cpp:941

alice, your personal RStudio Server 2023.03.0+386 running R 4.1.3 is available on:

  <http://<IP_ADDRESS>:59507>

Importantly, if you are running from a remote machine without direct access to pdevel.example.org, you need to set up SSH port forwarding first, which you can do by running:

  ssh -L 59507:devel.example.org:59507<EMAIL_ADDRESS>
in a second terminal from your local computer.

Any R session started times out after being idle for 120 minutes.
WARNING: You now have 10 minutes, until 2023-03-30 10:56:33-07:00, to connect and log in to the RStudio Server before everything times out.
Your password for this RStudio Server instance is: aq67I5mTxNEGYKtObS7j
Shutting down RStudio Server ...
Shutting down RStudio Server ... done
$

Suggestion
After launching RStudio Server, wait for the process to start, and possible a few more moments to give it time to fail, before reporting that it's available to the user.

Updated rsc start to check for startup failures.  The above error should now give:
$ rsc start --verbose
TTY detected. Printing informational message about logging configuration. Logging configuration loaded from '/etc/rstudio/logging.conf'. Logging to '/wynton/protected/home/legrand/nick/.local/share/rstudio/log/rserver.log'.
2023-03-30T17:46:24.296317Z [rserver] ERROR Unexpected exception: File name too long [system:36]; LOGGED FROM: int main(int, char* const*) src/cpp/server/ServerMain.cpp:941
ERROR: It looks like RStudio Server failed during launch ['rserver' process (PID 1423936 on hb-x1-2023) no longer exists]
Traceback:
1: assert_rserver_running() on line #804 in bin/rsc
2: main() on line #1633 in bin/rsc
Exiting (exit 1)

$


fixed an issue with ggplot

Description of changes:
Fixed an issue with ggplot. cleaned up the notebook.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@yuanzhua @aaronmarkham


chez-racket: init at 2021-12-11

Motivation for this change
This is basically Chez Scheme but better, notably it supports aarch64-linux (and likely aarch64-darwin too!).
This can be used to make Idris 2 work on more platforms with its Chez Scheme backend, albeit it is not done in this PR.
Things done
I've added chez-racket as a separate directory with its own files, since it's a permanent fork.
The only common part is the setup hook.
It would be great if someone on aarch64-darwin could test this.

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 131833 at 1ad518f5 run on x86_64-linux 1

1 package built successfully:

chez-racket

4 suggestions:

warning: stale-substitute
Stale substituteInPlace detected.
substituteStream(): WARNING: pattern '/bin/ln' doesn't match anything in file './workarea'
Near pkgs/development/compilers/chez-racket/default.nix:52:0:
   |
52 |     description  = "Fork of Chez Scheme for Racket";
   | ^

warning: stale-substitute
Stale substituteInPlace detected.
substituteStream(): WARNING: pattern '/bin/cp' doesn't match anything in file './workarea'
Near pkgs/development/compilers/chez-racket/default.nix:52:0:
   |
52 |     description  = "Fork of Chez Scheme for Racket";
   | ^

warning: patch-phase
patchPhase should not be overridden, use postPatch instead.
Near pkgs/development/compilers/chez-racket/default.nix:25:3:
   |
25 |   patchPhase = ''
   |   ^

warning: missing-phase-hooks
configurePhase should probably contain runHook preConfigure and runHook postConfigure.
Near pkgs/development/compilers/chez-racket/default.nix:43:3:
   |
43 |   configurePhase = ''
   |   ^

Bump

I've fixed all the problems and in the process also added support for cross-compilation!

Result of nixpkgs-review pr 131833 run on x86_64-darwin 1

1 package built:

chez-racket

Package builds for aarch64-darwin but does not run:
[nix-shell:~/.cache/nixpkgs-review/pr-131833]$ ./results/chez-racket/bin/scheme
Killed: 9

While I'm not able to test this myself, it seems to me that the chance that this is a bug upstream is somewhat high.


Add the findUTXO feature of the UTXOSet.

Currently, the required parameters hash, index are required in the findUTXO in UTXOSet.
In addition, it is necessary to find UTXO setHash directly without index in UTXO pool.
Index is If size_t.max, find the hash parameter by UTXO hash
#205

Could you add a simple unittest? You could modify one of the existing tests. This is to ensure we don't accidentally break it in the future.

added unit test code

In general we don not want to make methods public just because they're used in a far away unittest. However I'll merge it for now, because UtxoSet.d does not have any unittests of its own. I'll file an issue for this.


TTL is not preserved automatically during edit

Change the signature of GuaranteedUpdate so that TTL can
be more easily preserved. Allow a simpler (no ttl) and
more complex (response and node directly available, set ttl)
path for GuaranteedUpdate.  Add some tests to ensure this
doesn't blow up again.
@lavalamp - tried to keep the changes to GuaranteedUpdate
minimal.  @justinsb and I found a transactional bug with
one of the controllers (ipallocator) that requires having
access to the etcd index in the update - I wanted to also
ensure we have res available for that case in the future
although there are other changes required there as well.
Prereq to #8641

Reduced abstraction scope with tools.ResponseMeta{TTL int64} and added comments.

Any remaining comments?

Making another pass.

Just one comment. LGTM otherwise.

Verified 0 in the etcd client means no TTL, therefore I'm going to default to 1 for negative values.  Will open a follow up to investigate whether we need to lie about the TTL from inside tools/etcd_helper (set to 1) automatically to guarantee that any client that sees a TTL will use it properly.

Who do I have to bribe to get a merge?

Who do I have to bribe to get a merge?

@thockin

Trade you a rebase for a merge

actually, I'll be done soon, it will have to sit until tomorrow.  Trying to stabilize jenkins, so slow merges

@rjnagal can I get into your queue on this?

It's in the queue :) I am waiting for the e2e from last batch to come green.


FEAT: ChatGLM3  tool calls

Support tool calls for ChatGLM3, but the response function arguments are different:
Post URL: http://localhost:42183/v1/chat/completions
payload json:
{
    "model": "test_tool",
    "messages": [
        {
            "role": "user",
            "content": "帮我查询股票10111的价格"
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "track",
                "description": "追踪指定股票的实时价格",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "symbol": {
                            "description": "需要追踪的股票代码"
                        }
                    },
                    "required": [
                        "symbol"
                    ]
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "text-to-speech",
                "description": "将文本转换为语音",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "text": {
                            "description": "需要转换成语音的文本"
                        },
                        "voice": {
                            "description": "要使用的语音类型（男声、女声等）"
                        },
                        "speed": {
                            "description": "语音的速度（快、中等、慢等）"
                        }
                    },
                    "required": [
                        "text"
                    ]
                }
            }
        }
    ],
    "stop": [
        "\n"
    ]
}

ChatGLM3 response (The arguments is a json string):
{
    "id": "chatcmpl-095b7d07-74ed-44aa-9405-a3a999f73ebf",
    "model": "test_tool",
    "object": "chat.completion",
    "created":<PHONE_NUMBER>,
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": null,
                "tool_calls": [
                    {
                        "id": "call_095b7d07-74ed-44aa-9405-a3a999f73ebf",
                        "type": "function",
                        "function": {
                            "name": "track",
                            "arguments": "{\"symbol\": \"10111\"}"
                        }
                    }
                ]
            },
            "finish_reason": "tool_calls"
        }
    ],
    "usage": {
        "prompt_tokens": -1,
        "completion_tokens": -1,
        "total_tokens": -1
    }
}

ChatGLM: https://github.com/THUDM/ChatGLM3/blob/main/tool_using/README.md
ChatGLM cpp: https://github.com/li-plus/chatglm.cpp/blob/main/examples/chatglm3_demo.py
Closes: https://github.com/xorbitsai/inference/issues/676

LGTM

0.0:43309, pid=26826] string indices must be integers
Traceback (most recent call last):
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/api/restful_api.py", line 787, in stream_results
    iterator = await model.chat(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/context.py", line 227, in send
    return self._process_result_message(result)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/pool.py", line 657, in send
    result = await self._run_coro(message.message_id, coro)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/pool.py", line 368, in _run_coro
    return await coro
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/api.py", line 306, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
    result = await result
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/utils.py", line 33, in wrapped
    ret = await func(*args, **kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 67, in wrapped_func
    ret = await fn(self, *args, **kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 252, in chat
    return await self._call_wrapper(_wrapper)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 200, in _call_wrapper
    return await asyncio.to_thread(_wrapper)
  File "/root/miniconda3/envs/infer/lib/python3.10/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/root/miniconda3/envs/infer/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 240, in _wrapper
    getattr(self._model, "chat")(prompt, *args, **kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/model/llm/pytorch/chatglm.py", line 158, in chat
    return self._tool_calls_completion(msg[0], self.model_uid)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/model/llm/pytorch/chatglm.py", line 120, in _tool_calls_completion
    "name": msg["name"],
TypeError: [address=<IP_ADDRESS>:43309, pid=26826] string indices must be integers
Traceback (most recent call last):
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/queueing.py", line 407, in call_prediction
    output = await route_utils.call_process_api(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/route_utils.py", line 226, in call_process_api
    output = await app.get_blocks().process_api(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/blocks.py", line 1550, in process_api
    result = await self.call_function(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/blocks.py", line 1199, in call_function
    prediction = await utils.async_iteration(iterator)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 519, in async_iteration
    return await iterator.__anext__()
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 623, in asyncgen_wrapper
    async for response in f(*args, **kwargs):
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/chat_interface.py", line 437, in _stream_fn
    first_response = await async_iteration(generator)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 519, in async_iteration
    return await iterator.__anext__()
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 512, in __anext__
    return await anyio.to_thread.run_sync(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 495, in run_sync_iterator_async
    return next(iterator)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/chat_interface.py", line 111, in generate_wrapper
    for chunk in model.chat(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/client/common.py", line 49, in streaming_response_iterator
    raise Exception(str(error))
Exception: [address=<IP_ADDRESS>:43309, pid=26826] string indices must be integers

0.0:43309, pid=26826] string indices must be integers
Traceback (most recent call last):
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/api/restful_api.py", line 787, in stream_results
    iterator = await model.chat(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/context.py", line 227, in send
    return self._process_result_message(result)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/pool.py", line 657, in send
    result = await self._run_coro(message.message_id, coro)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/pool.py", line 368, in _run_coro
    return await coro
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/api.py", line 306, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
    result = await result
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/utils.py", line 33, in wrapped
    ret = await func(*args, **kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 67, in wrapped_func
    ret = await fn(self, *args, **kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 252, in chat
    return await self._call_wrapper(_wrapper)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 200, in _call_wrapper
    return await asyncio.to_thread(_wrapper)
  File "/root/miniconda3/envs/infer/lib/python3.10/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/root/miniconda3/envs/infer/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 240, in _wrapper
    getattr(self._model, "chat")(prompt, *args, **kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/model/llm/pytorch/chatglm.py", line 158, in chat
    return self._tool_calls_completion(msg[0], self.model_uid)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/model/llm/pytorch/chatglm.py", line 120, in _tool_calls_completion
    "name": msg["name"],
TypeError: [address=<IP_ADDRESS>:43309, pid=26826] string indices must be integers
Traceback (most recent call last):
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/queueing.py", line 407, in call_prediction
    output = await route_utils.call_process_api(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/route_utils.py", line 226, in call_process_api
    output = await app.get_blocks().process_api(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/blocks.py", line 1550, in process_api
    result = await self.call_function(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/blocks.py", line 1199, in call_function
    prediction = await utils.async_iteration(iterator)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 519, in async_iteration
    return await iterator.__anext__()
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 623, in asyncgen_wrapper
    async for response in f(*args, **kwargs):
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/chat_interface.py", line 437, in _stream_fn
    first_response = await async_iteration(generator)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 519, in async_iteration
    return await iterator.__anext__()
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 512, in __anext__
    return await anyio.to_thread.run_sync(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 495, in run_sync_iterator_async
    return next(iterator)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/chat_interface.py", line 111, in generate_wrapper
    for chunk in model.chat(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/client/common.py", line 49, in streaming_response_iterator
    raise Exception(str(error))
Exception: [address=<IP_ADDRESS>:43309, pid=26826] string indices must be integers

What is the error message? Did you test function calling on this branch?

0.0:43309, pid=26826] string indices must be integers
Traceback (most recent call last):
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/api/restful_api.py", line 787, in stream_results
    iterator = await model.chat(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/context.py", line 227, in send
    return self._process_result_message(result)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/pool.py", line 657, in send
    result = await self._run_coro(message.message_id, coro)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/backends/pool.py", line 368, in _run_coro
    return await coro
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xoscar/api.py", line 306, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
    result = await result
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/utils.py", line 33, in wrapped
    ret = await func(*args, **kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 67, in wrapped_func
    ret = await fn(self, *args, **kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 252, in chat
    return await self._call_wrapper(_wrapper)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 200, in _call_wrapper
    return await asyncio.to_thread(_wrapper)
  File "/root/miniconda3/envs/infer/lib/python3.10/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/root/miniconda3/envs/infer/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/model.py", line 240, in _wrapper
    getattr(self._model, "chat")(prompt, *args, **kwargs)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/model/llm/pytorch/chatglm.py", line 158, in chat
    return self._tool_calls_completion(msg[0], self.model_uid)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/model/llm/pytorch/chatglm.py", line 120, in _tool_calls_completion
    "name": msg["name"],
TypeError: [address=<IP_ADDRESS>:43309, pid=26826] string indices must be integers
Traceback (most recent call last):
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/queueing.py", line 407, in call_prediction
    output = await route_utils.call_process_api(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/route_utils.py", line 226, in call_process_api
    output = await app.get_blocks().process_api(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/blocks.py", line 1550, in process_api
    result = await self.call_function(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/blocks.py", line 1199, in call_function
    prediction = await utils.async_iteration(iterator)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 519, in async_iteration
    return await iterator.__anext__()
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 623, in asyncgen_wrapper
    async for response in f(*args, **kwargs):
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/chat_interface.py", line 437, in _stream_fn
    first_response = await async_iteration(generator)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 519, in async_iteration
    return await iterator.__anext__()
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 512, in __anext__
    return await anyio.to_thread.run_sync(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/gradio/utils.py", line 495, in run_sync_iterator_async
    return next(iterator)
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/core/chat_interface.py", line 111, in generate_wrapper
    for chunk in model.chat(
  File "/root/miniconda3/envs/infer/lib/python3.10/site-packages/xinference/client/common.py", line 49, in streaming_response_iterator
    raise Exception(str(error))
Exception: [address=<IP_ADDRESS>:43309, pid=26826] string indices must be integers

What is the error message? Did you test function calling on this branch?

yes, i merge your branch and test it

tool is {'role': 'system', 'content': 'Answer the following questions as best as you can. You have access to the following tools:', 'tools': []}

it seem tool is a empty slice, not null

@waltcow how did you call? Using python client or jusr via RESTful api?

@waltcow how did you call? Using python client or jusr via RESTful api?

curl -X 'POST' \
  'http://<IP_ADDRESS>:9997/v1/chat/completions' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "af1b28d0-9020-11ee-9058-fdd89f67f6c7",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "What is the largest animal?"
        }
    ]
  }'

@waltcow how did you call? Using python client or jusr via RESTful api?

curl -X 'POST' \
  'http://<IP_ADDRESS>:9997/v1/chat/completions' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "af1b28d0-9020-11ee-9058-fdd89f67f6c7",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "What is the largest animal?"
        }
    ]
  }'

I didn't reproduce it, could you check your openai version? it needs to later than v1.0 pip show openai

(infer) ➜  inference git:(tool) pip show openai
Name: openai
Version: 1.3.6
Summary: The official Python library for the openai API
Home-page:
Author:
Author-email: OpenAI<EMAIL_ADDRESS>License:
Location: /root/miniconda3/envs/infer/lib/python3.10/site-packages
Requires: anyio, distro, httpx, pydantic, sniffio, tqdm, typing-extensions
Required-by: xinference

@waltcow thanks for your feedback, I've reproduced it, this issue happens with pytorch format, and we will fix it in this PR.


Feature app runner

Now working for #14.

https://github.com/spring-petclinic/spring-petclinic-kotlin


Got Uncaught (in promise) DOMException

First I want to thank you for your great work. I meet an error when I try to set the same callback function like the demo then I found there's the same error in the live demo. The error message is as follow:
Live Demo:

My project:

OS: the client is windows 10 and the server is ubuntu(WSL)
Browser: Chrome 76
Version(In my project, it's installed from NPM): "vue-audio-recorder": "^3.0.0",

Hello, I have encountered the same problem. Have you found the solution?

Also, the same error appears in the live demo provided by readme.

Hello @ljhsmileking , I didn't find any solution, and I replaced "recorder" with "vue-dictaphone"


应该装在devDependencies

装在dependencies会报错...
[vite] Dep optimization failed with error:
Dependency vite-transform-globby-import is attempting to import Node built-in module fs.
This will not work in a browser environment.
Imported by: node_modules\vite-transform-globby-import\dist\index.js

Tip:
Make sure your "dependencies" only include packages that you
intend to use in the browser. If it's
a Node.js package, it
should be in "devDependencies".

If you do intend to use this dependency in the browser and the
dependency does not actually use these Node built-ins in the
browser, you can add the dependency (not the built-in) to the
"optimizeDeps.allowNodeBuiltins" option in vite.config.js.

If that results in a runtime error, then unfortunately the
package is not distributed in a web-friendly format. You should
open an issue in its repo, or look for a modern alternative.

给下一个踩坑的
yarn add -D vite-transform-globby-import or npm i --dev vite-transform-globby-import

装在dependencies会报错...
[vite] Dep optimization failed with error:
Dependency vite-transform-globby-import is attempting to import Node built-in module fs.
This will not work in a browser environment.
Imported by: node_modules\vite-transform-globby-import\dist\index.js

Tip:
Make sure your "dependencies" only include packages that you
intend to use in the browser. If it's
a Node.js package, it
should be in "devDependencies".

If you do intend to use this dependency in the browser and the
dependency does not actually use these Node built-ins in the
browser, you can add the dependency (not the built-in) to the
"optimizeDeps.allowNodeBuiltins" option in vite.config.js.

If that results in a runtime error, then unfortunately the
package is not distributed in a web-friendly format. You should
open an issue in its repo, or look for a modern alternative.

给下一个踩坑的

yarn add -D vite-transform-globby-import or npm i --dev vite-transform-globby-import

装在dependencies会报错...
[vite] Dep optimization failed with error:
Dependency vite-transform-globby-import is attempting to import Node built-in module fs.
This will not work in a browser environment.
Imported by: node_modules\vite-transform-globby-import\dist\index.js

Tip:
Make sure your "dependencies" only include packages that you
intend to use in the browser. If it's
a Node.js package, it
should be in "devDependencies".

If you do intend to use this dependency in the browser and the
dependency does not actually use these Node built-ins in the
browser, you can add the dependency (not the built-in) to the
"optimizeDeps.allowNodeBuiltins" option in vite.config.js.

If that results in a runtime error, then unfortunately the
package is not distributed in a web-friendly format. You should
open an issue in its repo, or look for a modern alternative.

yarn add -D vite-transform-globby-import or npm i --dev vite-transform-globby-import
给下一个踩坑的

It is optimizing behavior of vite about the 'dependencies'. And this is a plugin of development, you should better install it in ’ devDependencies‘


Settings sync merge conflict UI renders button offscreen when diff is in inline view

This is exactly the same issue as #105767 back to 2020 which was marked as duplicated.
However, the issue seems to still be there now. @sandy081 Could you point me out to the main issue? Thanks.
cc @RMacfarlane

Version: 1.65.2 (user setup)
Commit: c722ca6c7eed3d7987c0d5c3df5c45f6b15e77d1
Date: 2022-03-10T14:33:55.248Z
Electron: 13.5.2
Chromium: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Windows_NT x64 10.0.19044

/duplicate
https://github.com/microsoft/vscode/issues/104672


AttributeError when calling train in tf.estimator.DNNClassifier

System information

I wrote a custom script, using the tf.estimator.DNNclassifier. Source code below.
OS Platform and Distribution: MacOS 10.12.5
TensorFlow installed from: source
TensorFlow version:  v1.2.0-1741-g88633a8eb 1.2.1
Python version: 3.6
Bazel version (if compiling from source): 0.5.2
CUDA/cuDNN version: N/A
GPU model and memory: N/A
Exact command to reproduce:
Additional libraries: numpy (1.13.0), protobuf (3.3.0), tensorflow-tensorboard (0.1.2)

Describe the problem
When using tf.estimator.DNNClassifier train function, I get the an AttributeError listed below in the logs, relate to the definition of values=tuple(six.itervalues(features)) in _dnn_model_fn.
Note: when using tensorflow.contrib.learn.DNNClassifier instead (with fit, instead of train) no error occurs.
Source code
import tensorflow.contrib.learn as skflow
feature_columns = skflow.infer_real_valued_columns_from_input(totA.astype(np.float32))
clf = tf.estimator.DNNClassifier(feature_columns=feature_columns,hidden_units=dnntfDef.hidden_layers,
                               optimizer=dnntfDef.optimizer, n_classes=numTotClasses,
                               activation_fn=dnntfDef.activationFn, model_dir=model_directory)

clf.train(input_fn=lambda: input_fn(A, Cl2), steps=2000

logs
    clf.train(input_fn=lambda: input_fn(A, Cl2), steps=dnntfDef.trainingSteps)   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 241, in train     loss = self._train_model(input_fn=input_fn, hooks=hooks)   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 616, in _train_model     model_fn_lib.ModeKeys.TRAIN)   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 601, in _call_model_fn     features=features, labels=labels, **kwargs)   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/estimator/canned/dnn.py", line 265, in _model_fn     config=config)   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/estimator/canned/dnn.py", line 84, in _dnn_model_fn     values=tuple(six.itervalues(features)),   File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/six.py", line 578, in itervalues     return iter(d.values(**kw)) AttributeError: 'Tensor' object has no attribute 'values'

Pardon me for not understanding all of the context, but should totA be defined, or come from an import, in your example?

Sorry for the missing information. totA come from an import. In details:
`totA = np.vstack((A, A_test))`

where A is my training tensor, and A_test is my validation tensor.
As I said, the code is exactly the same for when I run DNNClassifier through tf.estimator or through
tensorflow.contrib.learn
Please let me know if you have any more questions.

OK, I built a stand-in for totA using a random normal numpy array.
Now the example breaks with "NameError: name 'dnntfDef' is not defined".
Is there something else I should stub out to replicate?

Sorry about that, those were previously defined variables. Given a train set A, with related classes Cl, and a test set A_test, with related labels Cl_test, the code proceeds as follows:
totA = np.vstack((A, A_test))
totCl = np.append(Cl, Cl_test)
numTotClasses = np.unique(totCl).size

import tensorflow.contrib.learn as skflow
feature_columns = skflow.infer_real_valued_columns_from_input(totA.astype(np.float32))
clf = tf.estimator.DNNClassifier(feature_columns=feature_columns,hidden_units=[20],
                               optimizer="Adagrad", n_classes=numTotClasses,
                               activation_fn="tanh", model_dir=model_directory)

clf.train(input_fn=lambda: input_fn(A, Cl2), steps=200)

The code using tensorflow.contrib.learn is the same other than the last two lines:
clf = tensorflow.contrib.learn.DNNClassifier(feature_columns=feature_columns,hidden_units=[20],
                                        optimizer="Adagrad", n_classes=numTotClasses,
                                        activation_fn="tanh", model_dir=model_directory)
clf.fit(input_fn=lambda: input_fn(A, Cl2), steps=200)

I hope this helps. Thanks for your help.

@jhseu any ideas?

Since you switched to core canned Estimators, can you try switching to core FeatureColumns as well?
https://www.tensorflow.org/versions/master/api_docs/python/tf/feature_column

Interesting suggestion, will try. What is the closest API in tf.feature_column to the one I used from learn: https://www.tensorflow.org/api_docs/python/tf/contrib/learn/infer_real_valued_columns_from_input
Thanks

OK, instead of using:
feature_columns = skflow.infer_real_valued_columns_from_input(totA.astype(np.float32))
I used:
feature_columns = tf.feature_column.numeric_column(key='totA', shape=totA.shape)
This still gives the exact same error.

Can you paste a full runnable example? Specifically I need to see what your input_fn is returning.
From what I can tell, that error shouldn't happen unless you have some mixed Tensor/dict type.

Here it is. Will provide train and test files to run shortly
`#!/usr/bin/env python3
import numpy as np
import sys, os.path
#**********************************************
''' Main '''
#**********************************************
def main():
print(" Using training file: ", sys.argv[1],"\n")
En, Cl, A = readLearnFile(sys.argv[1])
En_test, Cl_test, A_test = readLearnFile(sys.argv[2])
learnFileRoot = os.path.splitext(sys.argv[1])[0]
clf_dnntf, le_dnntf  = trainDNNTF(A, Cl, A_test, Cl_test, learnFileRoot)

#**********************************************
''' DNNClassifier '''
#**********************************************
def trainDNNTF(A, Cl, A_test, Cl_test, Root):
import tensorflow as tf
import tensorflow.contrib.learn as skflow
from sklearn import preprocessing
model_directory = Root + "/DNN-TF_"
print("\n  Training model saved in: ", model_directory, "\n")

#**********************************************
''' Initialize Estimator and training data '''
#**********************************************
print(' Initializing TensorFlow...')
tf.reset_default_graph()

totA = np.vstack((A, A_test))
totCl = np.append(Cl, Cl_test)
numTotClasses = np.unique(totCl).size

le = preprocessing.LabelEncoder()
totCl2 = le.fit_transform(totCl)
Cl2 = le.transform(Cl)
Cl2_test = le.transform(Cl_test)

feature_columns = skflow.infer_real_valued_columns_from_input(totA.astype(np.float32))

''' tf.estimator version '''
#clf = tf.estimator.DNNClassifier(feature_columns=[totA], hidden_units=[20],
#                           optimizer="Adagrad", n_classes=numTotClasses,
#                           activation_fn="tanh", model_dir=model_directory)

''' tf.contrib.learn version '''
clf = skflow.DNNClassifier(feature_columns=feature_columns, hidden_units=[20],
                                optimizer="Adagrad", n_classes=numTotClasses,
                                activation_fn="tanh", model_dir=model_directory)

#**********************************************
''' Train '''
#**********************************************

''' tf.estimator version '''
#clf.train(input_fn=lambda: input_fn(A, Cl2), steps=2000)

''' tf.contrib.learn version '''
clf.fit(input_fn=lambda: input_fn(A, Cl2), steps=100)

accuracy_score = clf.evaluate(input_fn=lambda: input_fn(A_test, Cl2_test), steps=1)
print('\n  ================================')
print('  \033[1mDNN-TF\033[0m - Accuracy')
print('  ================================')
print("\n  Accuracy: {:.2f}%".format(100*accuracy_score["accuracy"]))
print("  Loss: {:.2f}".format(accuracy_score["loss"]))
print("  Global step: {:.2f}\n".format(accuracy_score["global_step"]))
print('  ================================\n')

return clf, le

#**********************************************
''' Format input data for Estimator '''
#**********************************************
def input_fn(A, Cl2):
import tensorflow as tf
x = tf.constant(A.astype(np.float32))
y = tf.constant(Cl2)
return x,y
#**********************************************
''' Read learn File '''
#**********************************************
def readLearnFile(learnFile):
try:
with open(learnFile, 'r') as f:
M = np.loadtxt(f, unpack =False)
except:
print('\033[1m' + ' Learning file not found \n' + '\033[0m')
return
En = np.delete(np.array(M[0,:]),np.s_[0:1],0)
M = np.delete(M,np.s_[0:1],0)
Cl = ['{:.2f}'.format(x) for x in M[:,0]]
A = np.delete(M,np.s_[0:1],1)
return En, Cl, A

#************************************
''' Main initialization routine '''
#************************************
if name == "main":
sys.exit(main())`

The zipped train+test files can be found here:
https://goo.gl/2no9AA
to run the code above (assuming it's called DNNClassifier.py):
./DNNClassifier.py train.txt test.txt
You can toggle between the tf.estimator and tf.contrib.learn version within the code.

There are many issues with the script, but it works for me with many modifications:

feature_columns must be an array
input_fn must return a dictionary for features where the key matches the feature column key.
activation can't be a string in the new version
You'll have to get shapes right. It would probably be easier just to change your input_fn to use numpy_input_fn: https://www.tensorflow.org/versions/master/api_docs/python/tf/estimator/inputs/numpy_input_fn

Closing because there's no bug here.

(Note that we'll have some new tutorials for core estimators up soon. They're not up for core canned estimators because they're not in an official release yet.)

Actually, will reopen and reassign to improve the error message.


org-roam migration to v2 not working

What did you expect to happen?
It migrated correctly my files from v1 to v2.
What actually happened?
After I run org-roam-migrate-wizard I get No node at point and if I close Emacs it says Save file /home/portstone/org/roam/20210509215114-msrp.org
Describe your attempts to resolve the issue
I deleted that file (/home/portstone/org/roam/20210509215114-msrp.org) since I didn't know what could be wrong with it. It runs and completes but still nodes are missing after I type spc n r f. I restart Emacs and type again spc n r f and it freezes at Processing modified files....50%
Content of /home/portstone/org/roam/20210509215114-msrp.org
#+title: MSRP
#+roam_alias: "RRP" "SRP"

MSRP stands for manufacturer's suggested retail price.
RRP stands for recommended retail price.
SRP stands for suggested retail price.

At first I did it like it said in the documentation but I thought reinstalling Doom Emacs completely will may fix it since may version of Doom Emacs was around half a year old. Therefore I deleted my .emacs folder and ran those two commands:
git clone --depth 1 https://github.com/hlissner/doom-emacs ~/.emacs.d
~/.emacs.d/bin/doom install

Steps to reproduce

Delete .emacs folder
Make sure in init.el it is set to org roam2
Install Emacs (with those two commands)
Start Emacs and run org-roam-migrate-wizard
After it finished restart Emacs and use keybinding: spc n r f
Freezing at Processing modified files...50%

System Information
https://pastebin.com/AAKQ6jKz

Does unpinning org-roam fix the issue? I.e.
1. Add `(unpin! org-roam)` to `~/.doom.d/packages.el`

2. Run `$ doom sync -u`

3. Restart Emacs

No it still hangs. I also checked if it maybe because of my folder structure since I have tons of pictures and other files that I reference to. Therefore I deleted everything except the .org files but it keeps hanging at Processing modified files...48% (I don't know why it is now hanging at 48% and not 50%).
It needs to convert 197 org files. I also don't have a feedback if it is finished with migrating. It just says roam/20210509215114-example.org 33L, 200C written and after some minutes it doesn't changed therefore I assume it finished since it is not saying something like finished migrating.
Also my process of migrating is that it outputs roam/20210509215114-example.org 33L, 200C written and after some seconds it does  Processing modified files...done and again does output roam/20210509215114-another-example.org 33L, 200C written for the rest of files it converted. Shouldn't it write first everything and after that process the files?

Since it isn't an issue with Doom Emacs itself I am going to close it.


Possibly allow connection to remote device through web using secure password / Authenticator

Sorry if this was mentioned before!

I was wondering if it would be possible to instead of using github or microsoft to login, you could change a setting when running your server and setup a password that would allow the user to access the files when on the URL. Adding onto that, for more security, support for apps like Google Authenticator to have a random 6 digit code that changes every so often would have more security and create another option to sign in.
Idk if doing one of these would hurt features like settings sync as they rely on github/microsoft to work.
I would love this feature to exist in VS Code Server as people that sometimes have restricted devices cannot signin with Github or Microsoft.
So if i could signin via password, I could easily access my files without having to worry about logging in with github,

The underlying service is an Azure service which requires a user account for allocation. Running or allowing users to run separate services to authenticate to the tunnel is very unlikely to happen, and is outside the scope of VS Code.
However, in the future we will allow people to log in with their Microsoft account, which may be sufficient to get around the blockers.


Update requirements.txt - New chromadb version does not work

Recently the 0.4 version of chromadb has been released (https://pypi.org/project/chromadb/#history). When running the code with the last version it raises an error. However, installing the 0.3.29 version solves the error.

Hi, thanks for your PR. Merged!


Create recipe template for generating a BadPixelMap Calibration

Is your feature request related to a problem? Please describe.
No recipe template exists for creating BadPixelMap from L1 data.
Describe the solution you'd like
A recipe template needs to be written to generate a BadPixelMap Calibration file from L1 data.
Additional context
In order to test this you will need to get your hands on the appropriate input data from II&T or TVAC and you will quantitatively compare the calibration file to the outputs of the II&T data pipeline. In addition please describe in your own words (~several sentences to a paragraph) what the recipe does and what the inputs and outputs are. Finally, please make sure the appropriate table entry is complete within the Confluence DRP Implementation Document table in Section 2.
Here's a checklist:

[ ] Recipe Written
[ ] Test Written and passing
[ ] Table Entry Complete

I've written the recipe and test file for this, but have a few outstanding questions:

How should walker.py handle the master dark and master flat? I've added additional arguments to the walk_corgidrp function to accept these inputs for now, but I'd like to confirm if this approach is what you had in mind.
The TVAC noise maps I have access to have multiple fields swapped between what the code expects to find in the primary and extension header, which causes several issues. Additionally, KGAIN is not defined in either of the headers for the TVAC images. I've made modifications to darks.py and the Image class to address the swapped header values, but I'm wondering if this is specific to the data files I have or if you’ve encountered similar issues with the datasets you’re working with.

The master dark and master flat should be calibration files specified as part of the recipe json file. They should be specified as part of the bad pixel map creation step (see the other step functions as examples). You shouldn't be modifying the walk_corgidrp function.
I've not encountered such issues. The code I pointed out in the l1_to_l2b_e2e.py should work just fine to handle all of the calibration data you need. Have you gotten access to the box folder yet?

Got it, thanks. I reverted walk_corgidrp to its original arguments. For that, can I make the assumption that only one Dark and one FlatField calibration file will be in the caldb at a time?
Ok, good to know. It may just be an issue with the data I have then, since I did have to modify a lot to handle the noise maps. I don't have access to Box yet - since the invite was sent to my jpl.nasa.gov email address I have to get NASA approval for an account.

If you set the calibration files to be chosen automatically in the recipe (with the "AUTOMATIC" value), then the caldb will automatically pick the most suitable calibration, even if there are multiple files in the caldb.
If we can share the Box folder with a different email, just DM me.


Can't download butler

The links in the documentation seems to be wrong. Or is the server down?
https://broth.itch.ovh/butler
https://broth.itch.ovh/butler/linux-amd64/LATEST/archive/default
I'm getting itch.io API error (403): /games/239683/uploads: invalid key

Good catch! Something did break indeed, it's all back up now.


User Interface: Dashboard

Description

Can consider using the following components from mantine:

Table: https://mantine.dev/core/table/
Ring Progress: https://mantine.dev/core/ring-progress/
Semi-circle Progress: https://mantine.dev/core/semi-circle-progress/

Design

Resolved by #51


Complete crash at profile selection

I'm submitting a ...

[x] bug report
[ ] feature request
[ ] support request

General information
Prerequisites

[x] Are you running the latest version? => yup, master branch 10 Nov. 18

Description
I'm able to connect to my netflix account, I can see my profiles, but when I try to select any, as soon as I press the down key, it completly crashes OSMC. The Netflix full size background appears for less than a second, then I have the :disappointed: error from OSMC.
Context (Environment)
Installation

[x] I installed the plugin via zip from the Releases page
[ ] I´m using the Netflix Repo
[ ] I´m using a different source (Please tell which)

Operating System

[x] Linux (x86/x64)
[ ] OSX (x86/x64)
[ ] Windows (x86/x64)
[ ] Linux (ARM)
[ ] Android

Additional informatin on the environment
Kodi 18 beta 5, OSMC 2018-10, Rpi 3
Debug log
Journalctl :
nov. 11 16:49:00 bear sudo[1228]: pam_unix(sudo:session): session closed for user osmc
nov. 11 16:49:00 bear mediacenter[318]: /usr/bin/mediacenter: line 142:  1228 Segmentation fault      sudo -u osmc $KODI --standalone -fs --lircdev /var/run/lirc/lircd
nov. 11 16:49:00 bear mediacenter[318]: Kodi exited with return code 139 after 0 hours, 0 minutes and 51 seconds
nov. 11 16:49:00 bear kernel: Console: switching to colour frame buffer device 240x67

kodi.log:
16:49:00.654 T:1917657088   DEBUG: Previous line repeats 7 times.
16:49:00.654 T:1917657088   DEBUG: [Warning] CGUITextureManager::GetTexturePath: could not find texture 'noop'
16:49:00.655 T:1798718192   DEBUG: ClampLimits: 1920x1080 1280x720 -> 1280x720 clamped=0
16:49:00.656 T:1762652912    INFO: easy_acquire - Created session to https://…
16:49:00.671 T:1798718192   DEBUG: COMXCoreComponent::Initialize OMX.broadcom.image_decode input port 320 output port 321 m_handle 0x71b093d8
16:49:00.672 T:1798718192   DEBUG: COMXCoreComponent::AllocInputBuffers component(OMX.broadcom.image_decode) - port(320), nBufferCountMin(2), nBufferCountActual(2), nBufferSize(81920), nBufferAlignment(16)
16:49:00.675 T:1798718192   DEBUG: COMXCoreComponent::Initialize OMX.broadcom.egl_render input port 220 output port 221 m_handle 0x71b08d88
16:49:00.676 T:1798718192   DEBUG: COMXCoreComponent::UseEGLImage component(OMX.broadcom.egl_render) - port(221), nBufferCountMin(1), nBufferCountActual(1), nBufferSize(15360) nBufferAlignment(16)
16:49:00.699 T:1798718192   DEBUG: COMXCoreComponent::Deinitialize : OMX.broadcom.image_decode handle 0x71b093d8
16:49:00.700 T:1798718192   DEBUG: COMXCoreComponent::Deinitialize : OMX.broadcom.egl_render handle 0x71b08d88
16:49:00.701 T:1798718192   DEBUG: DecodeJpegToTexture: decoded special://masterprofile/Thumbnails/9/9bc830dc.jpg 1280x720
16:49:00.703 T:1798718192   DEBUG: COMXImage::CreateThumb /home/osmc/.kodi/addons/plugin.video.netflix/resources/fanart.jpg 1600x900 progressive images not supported by decoder

Seeing something very similar. No backtrace seems to be generated, even after installing gdb.
Attaching to kodi before the crash reveals this:
 #0  0x015045b4 in X509_NAME_cmp ()
 #1  0x014aab80 in OBJ_bsearch_ex_ ()
 #2  0x733b3cbe in ?? () from /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.2

Seems to be a crash from a python script invoking some ssl code.

I had the same error with Plex and Youtube plugins, which worked perfectly on a Kodi 17 OSMC installation, so it has to be related to Kodi.
I'm closing this issue because it's not related to the Netflix plugin. If you investigate further, can you please link the new issue ? Thanks.

This addon requires Kodi 18+, so that's nmot an option. Apparently I was missing a few dependencies (found here: https://raw.githubusercontent.com/zjoasan/netflix-install-script/master/netflix_prep_install.sh).

Great, thanks for your feedback, I will retry this week.


Update munit to 1.0.0-RC1

About this PR
📦 Updates org.scalameta:munit from 1.0.0-M11 to 1.0.0-RC1
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalameta", artifactId = "munit" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.scalameta", artifactId = "munit" }
}]

labels: library-update, early-semver-pre-release, semver-spec-pre-release, commit-count:1

Superseded by #315.


Update EKS config map to allow account-admin and read-only SSO roles

What does this PR do?
It allows our roles to access clusters that are created in the CI. By default only the role that created the cluster can access it. This is a bit blocking when you want to inspect clusters created by the CI. With that change account-admin role will always get full access to the EKS cluster and read-only will be able to view resources inside the cluster.
It also "repair" the console so that we can see the resources directly in the Resources panel on the EKS page
Which scenarios this will impact?
Motivation
Additional Notes

FYI, we should move to the IAM way, ConfigMap way is legacy and is probably going to be deprecated at some point in the future.

Good to know, was not sure if we chose the Configmap authentication on purpose.
I will create a card for us to migrate to authentication with EKS API


Feat/postgres 16

Problem
Hopefully none
Summary of changes
This adds postgres-v16 as a vendored postgresql version, and adapts the code to support this version
Checklist before requesting a review

[x] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.
[ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
[ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

[ ] Do not forget to reformat commit message to not include the above checklist

One thing came up in a conversation with @knizhnik today: We have made a couple of small change to some WAL record format. Heap update is one example that I remember, to add the command id, not sure if there are others.
Now would be a good chance to revisit that. We still need the command id for WAL generated by neon, but does our pageserver code handle both vanilla Postgres WAL records and the ones created with neon-patched postgres? If the pageserver can handle both, that makes it possible to stream WAL from unmodified Postgres to Neon storage. If we need to make changes the format, add a flag or something, we should do it now for v16.

One thing came up in a conversation with @knizhnik today: We have made a couple of small change to some WAL record format. Heap update is one example that I remember, to add the command id, not sure if there are others.
Now would be a good chance to revisit that. We still need the command id for WAL generated by neon, but does our pageserver code handle both vanilla Postgres WAL records and the ones created with neon-patched postgres? If the pageserver can handle both, that makes it possible to stream WAL from unmodified Postgres to Neon storage. If we need to make changes the format, add a flag or something, we should do it now for v16.

Here is the list of changes in WAL format we have made:
@@ -108,6 +108,7 @@ typedef struct xl_heap_delete
 {
        TransactionId xmax;                     /* xmax of the deleted tuple */
        OffsetNumber offnum;            /* deleted tuple's offset */
+       uint32      t_cid;
        uint8           infobits_set;   /* infomask bits */
        uint8           flags;
 } xl_heap_delete;
@@ -145,6 +146,7 @@ typedef struct xl_heap_header
 {
        uint16          t_infomask2;
        uint16          t_infomask;
+       uint32      t_cid;
        uint8           t_hoff;
 } xl_heap_header;

@@ -186,6 +188,7 @@ typedef struct xl_multi_insert_tuple
        uint16          datalen;                /* size of tuple data that follows */
        uint16          t_infomask2;
        uint16          t_infomask;
+       uint32      t_cid;
        uint8           t_hoff;
        /* TUPLE DATA FOLLOWS AT END OF STRUCT */
 } xl_multi_insert_tuple;
@@ -215,9 +218,9 @@ typedef struct xl_heap_update
        OffsetNumber old_offnum;        /* old tuple's offset */
        uint8           old_infobits_set;       /* infomask bits to set on old tuple */
        uint8           flags;
+       uint32       t_cid;
        TransactionId new_xmax;         /* xmax of the new tuple */
        OffsetNumber new_offnum;        /* new tuple's offset */
-
        /*
         * If XLH_UPDATE_CONTAINS_OLD_TUPLE or XLH_UPDATE_CONTAINS_OLD_KEY flags
         * are set, xl_heap_header and tuple data for the old tuple follow.
@@ -279,6 +282,7 @@ typedef struct xl_heap_lock
 {
        TransactionId locking_xid;      /* might be a MultiXactId not xid */
        OffsetNumber offnum;            /* locked tuple's offset on page */
+       uint32       t_cid;
        int8            infobits_set;   /* infomask and infomask2 bits to set */
        uint8           flags;                  /* XLH_LOCK_* flag bits */
 } xl_heap_lock;

So there are several affected WAL records and several functions where this structures are serialized/deserialized.
Unfortunately we do not have any Neon specific flag/format/version number in XLOG which can be used to determine format. We can add such flag now but then backward compatibility will be lost.
Alternatively we can somehow pass this flag to xlogreader, specifying with which format we are working now.
Just want to remember why it may be needed: there are many complaints from customers concerning slow import of data in Neon. Right now it mostly can be done only using pg_dump/pg_restore, which actually reconstruct heap an indexes from scratch. Using pg_basebackup is certainly more efficient because it allows just copy files.
But there are several problems with basebackup:

User should use exactly the same version of Postgres which is supported by Neon.
We should provide new way of launching compute. Right now basebackup is sent to compute from PS. And we need to be able to use client's basebacklup file
pg_basebackup include WAL, so we need our compute to be able to process vanilla WAL (without added t_cid)

Right now we are speaking about 3), although 2) can be also very non-trivial task and 1) can be show stopper.
It seems to be that least invasive solution will be to add some global variable or may be even GUC, (i.e. neon_wal_compatibility) and use it in redo handlers. I can make such change if there are no better proposals.

It seems to be that least invasive solution will be to add some global variable or may be even GUC, (i.e. neon_wal_compatibility) and use it in redo handlers. I can make such change if there are no better proposals.

Let's make it so that you can distinguish the record types from the WAL record itself. An extra flag on the records or something. That way, we don't need any extra bookkeeping of what WAL was generated with unmodified PostgreSQL and which with neon.

It seems to be that least invasive solution will be to add some global variable or may be even GUC, (i.e. neon_wal_compatibility) and use it in redo handlers. I can make such change if there are no better proposals.

Let's make it so that you can distinguish the record types from the WAL record itself. An extra flag on the records or something. That way, we don't need any extra bookkeeping of what WAL was generated with unmodified PostgreSQL and which with neon.

Looks likes there are two free bits in xl_info So in principle we can use one of this bits to indicate Neon format.
But then there will be incompatibility with current Neon format because definitely this bit is not set here.
So in addition to this bit we need also to add GUC indicating with this bit should be taken in account or not.

Ok, apart from XLog changes requested upthread which is presumably best solved with custom RMGR records, for a new custom RMGR Neon (remember to register the ID at the wiki), the following tests failed locally:
-- stuck on index creation --
test_runner/regress/test_pg_regress.py::test_pg_regress[debug-pg16]
test_runner/regress/test_pg_regress.py::test_pg_regress[release-pg16]
test_runner/regress/test_pg_regress.py::test_isolation[debug-pg16]
test_runner/regress/test_pg_regress.py::test_isolation[release-pg16]

-- RuntimeError: timed out waiting 30s for sk_id=1 to trim WAL to 24.00MB --
test_runner/regress/test_wal_acceptor.py::test_s3_wal_replay[debug-pg16-local_fs]
test_runner/regress/test_wal_acceptor.py::test_s3_wal_replay[debug-pg16-mock_s3]
test_runner/regress/test_wal_acceptor.py::test_s3_wal_replay[release-pg16-local_fs]
test_runner/regress/test_wal_acceptor.py::test_s3_wal_replay[release-pg16-mock_s3]

-- timeout --
test_runner/regress/test_wal_acceptor_async.py::test_wal_lagging[debug-pg16]

I run regression tests for pg16 branch at MacOS.
For debug build test_isolation and test_sql_tests are passed and 3 tests are failed for test_pg_regress:

portals.sql returns different output for queries ... without order by. Is it really correct test?
Another failed tests is tablespace.py - it messed "tablespace "regress_tblspace" is not empty" error.
Also wonder how it worked for pg14/15 because we are actually do not write something to table spaces (because compute is stateless).
Cluster,.sql returns one extra row

The problem with tablespace.sql was caused by not porting alternative expected file (without tablespace not-empty errors).
And problems with portal.sql and cluster.sql seems to have the same source: sync seqscan.
For some reasons current position is not reset at the beginning of the table and as a result next scan is started not with first page which cause unmatched result.
I still didn't find the reason why this position is not correctly reset. I suspect that something is lost during merge. I will continue investigation but @MMeent  may be you have an idea why sync scan is not working as expected?

Well, it seems to be a problem in vanilla as well!
I do not know whether it can be considered as bug or not but behaviour in any case seems to be strange.
Sp if shared_buffers are reduced to 1Mb and so syncscan criteria in heapam.c is not satisfied for tenk1 table, then result of the test doesn't match.
For some reasons start position is not reset and so next seqscan is started not with 0.

Oh that's funny, I was just looking at why my buildfarm member 'chipmunk' has been failing for a long time: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=chipmunk&dt=2023-08-06 08%3A20%3A21. Same thing! chipmunk also runs with a tiny shared_buffers, because it's a very old raspberry pi that really doesn't have a lot of RAM.

The problem with tablespace.sql was caused by not porting alternative expected file (without tablespace not-empty errors).

I fail to see why it is correct to drop a tablespace which may still contain files? AFAIK the alternative expected file was not correct - the materialized view testschema.amv should still have data in that tablespace...

Well, notion of tablespace has no sense in Neon.  Should we spent a lot of effort trying to emulate work with tablespaces? I do not think so.
I do not understand what is not correct in the alternative expected file (by the way - it is exactly the same as for PG 15).
testschema.amv is mentioned in two places:
-- let's try moving a materialized view from one place to another
CREATE MATERIALIZED VIEW testschema.amv AS SELECT * FROM testschema.atable;
ALTER MATERIALIZED VIEW testschema.amv SET TABLESPACE regress_tblspace;
REFRESH MATERIALIZED VIEW testschema.amv;
SELECT COUNT(*) FROM testschema.amv;
 count
-------
     3
(1 row)

and
drop cascades to materialized view testschema.amv

What is not correct?

For some reasons current position is not reset at the beginning of the table and as a result next scan is started not with first page which cause unmatched result.

That's something that should be considered correct and expected behaviour, right?

Tom doesn't think so: https://www.postgresql.org/message-id/flat/367973.1691369736%40sss.pgh.pa.us#8e3682f38fc9ccf180c0c75289867ccf

Intersestingly, some of the v14 and v15 tests are failing too. I wonder what's up with those? I believe CI on 'main' has been pretty green, so it seems like too many failures to be just flaky tests.
Looks like PostGIS needs to be updated to make it compile..

This includes a bunch of refactoring to the python tests. They all makes sense to me, but let's extract them to a separate PR.

For some reasons current position is not reset at the beginning of the table and as a result next scan is started not with first page which cause unmatched result.

That's something that should be considered correct and expected behaviour, right?

Tom doesn't think so: https://www.postgresql.org/message-id/flat/367973.1691369736%40sss.pgh.pa.us#8e3682f38fc9ccf180c0c75289867ccf

This has now been fixed in upstream, so once we refresh our postgres code again, can revert the 'portals' and 'cluster' test changes again.

Comparing the diff on v15 and v16:
git diff --stat  REL_15_4..origin/REL_15_STABLE_neon
# vs v16:
git diff --stat d5feb6011e629cc1096bbaa10af5ffcc6c17f908..2f1145c369a9916ec2fd972a3e6769845987e28c

I think we lost the changes to 'user.c` in the rebase. Commits ac6ca4df7a55cc0e3c6d552bcf3fe02179011ff4 and 4834eb7beda9de9d86d4bca330588772bef1765e on the REL_15_STABLE_neon branch. I presume that's because they were added after the v16 branch was originally created, so please check any other later commits on the REL_15_STABLE_neon branch too, if they're missing.

I think we lost the changes to 'user.c` in the rebase.

Never mind. That was intentional. The behaviour of GRANT was changed in v16, so that we don't need those neon patches anymore.

I merged 'origin/main' to this branch, to fix the conflicts and make CI run again

Summary of the current status:

All the extension compilation errors have been fixed. v16 compute image builds successfully now. I had to disable a bunch of extensions to make that work, so we'll need to revisit them later, but we can release v16 with some extensions missing.
I added libicu67 to the 'neon' docker image, so initdb should work now. Need to ensure that our storage nodes in staging and production have that library installed too
One regression failure on v14. I believe it's a flaky test and will probably go away if we rerun the tests
A few regression failures on v16. These passed at one point, so I believe they are caused by the recent changes to WAL-logging UPDATE records. @MMeent, you are working on those, right?

Fixed an incorrect reference to XlNeonHeapUpdate and cleaned up some leftover structs that are no longer used. That was causing regression tests to hang. There are still some failures:
2023-09-12T06:53:15.536960Z ERROR wal_connection_manager{tenant_id=d54b4bfbf23ccf2176def12c31205783 timeline_id=744d48c600f902ad9f0fb583453c2a2c}:connection{node_id=1}:panic{thread=walreceiv
er worker location=/home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bytes-1.4.0/src/buf/buf_impl.rs:254:9}: assertion failed: self.remaining() >= dst.len()

Stack backtrace:
   0: utils::logging::tracing_panic_hook
             at /home/heikki/git-sandbox/neon/libs/utils/src/logging.rs:166:21
   1: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/alloc/src/boxed.rs:2007:9
   2: std::panicking::rust_panic_with_hook
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:709:13
   3: std::panicking::begin_panic_handler::{{closure}}
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:595:13
   4: std::sys_common::backtrace::__rust_end_short_backtrace
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/sys_common/backtrace.rs:151:18
   5: rust_begin_unwind
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:593:5
   6: core::panicking::panic_fmt
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/panicking.rs:67:14
   7: core::panicking::panic
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/panicking.rs:117:5
   8: bytes::buf::buf_impl::Buf::copy_to_slice
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bytes-1.4.0/src/buf/buf_impl.rs:254:9
   9: bytes::buf::buf_impl::Buf::get_u32_le
             at /home/nonroot/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bytes-1.4.0/src/buf/buf_impl.rs:481:9
  10: pageserver::walrecord::v16::rm_neon::XlNeonHeapUpdate::decode
             at /home/heikki/git-sandbox/neon/pageserver/src/walrecord.rs:364:31
  11: pageserver::walingest::WalIngest::ingest_neonrmgr_record::{{closure}}
             at /home/heikki/git-sandbox/neon/pageserver/src/walingest.rs:743:37
  12: pageserver::walingest::WalIngest::ingest_record::{{closure}}
             at /home/heikki/git-sandbox/neon/pageserver/src/walingest.rs:111:18
...

but I didn't investigate that yet

This PR includes a bunch of test changes that are unrelated to v16. I think they're good refactorings, but let's do them as separate PRs

I merged main to this branch, to get latest changes. And I reverted the unrelated test changes.


FR: Support For EPS Files

It would be rad if Imager could generate thumbnails and transforms on EPS files - I know Imagick can interpret them so Imager could create cached PNG and/or JPG thumbnails for the Craft backend and front-end template display.
Thanks!

Wont add. Would make more sense as a separate plugin.