Question on Path Aliasing

When you create following rules:
{
  "rules": [
    { "pathname": "/foo/**", "dest": "foo.another.com" }
  ]
}

Why do I still have to use foo.example.com/foo/bar instead of just using foo.example.com/bar? The problem is that the app running on foo.another.com needs to be aware of the path used in rules.json which makes it kinda coupled.

@jamo any thoughts ? I think it would be cool if you can rewrite paths.

yeah, it's coming eventually

Any news @jamo on path rewriting?

@sedubois it's on the roadmap. We'll post updates here as soon as we have them 👍


Bugs in demo

when you scroll to any position != 0 and rotate the device, the function finishIfNeeded will close the activity
if I disable this behaviour and go to the permissions page and rotate the device, then I can swipe to the next page (without animation though)

Those bugs are not happening with the play store version, but with the on from the repo's master...
In my clone I added a function to disable the finishIfNeeded check before adding slides and enable them again afterwards, which solves 1, but the second thing still happens, I don't know yet why though...

Alright the first thing is solved.

The second thing has something to do with the SwipeBlockableViewPager. I'm investigating this.

Solved the second issue.

Facing the same issue. When can we get the fixed version?

Today

Tried it, can confirm it's working basically...
I still see 2 problems:

I start reading data from the internal storage in onPageSelected (one page after the permissions slide) and it happens, that the ViewPager does call this function after rotation shortly and trying to swipe => can't reproduce that now and it's simply solveable for me with checking the permissions again in the callback
rotate device on the page with the permission, press back, there's a visual glitch, the ViewPager does scroll one page forward and then to the previous one

Saw another thing after trying around 50 times or so, it happened, that I could swipe over the permissions slide without granting the permission... Only once though in all this "tests"... Point one seems to be this issue...
I will solve that by forcefully scrolling back if the permission is not granted in onPageSelected for now to make sure, that this does not have any influence on my app...

Does the glitch happen even in the latest commit?

I'll check, you just changed the layouts...

PS: It does not fit in here, but I would rethink the last change... You force people now to include ConstraintLayout and DataBinding to their projects for very simple slide layouts...
And the glitch happens in the last commits demo app as well... Just swipe to the permission slide, rotate the phone and press the hardware back button and you'll see it

Why are you forcing us to include ConstraintLayout and DataBinding ...


StandupsListTest crashes

Description
Running StandupsListTest crashes consistently at line 521 of the extension to OrderedSet. It looks to me as though some code is missing from the guard statement because when _elements has zero items it attempts to remove item[0].
Checklist

[ ] I have determined whether this bug is also reproducible in a vanilla SwiftUI project.
[X] If possible, I've reproduced the issue using the main branch of this package.
[X] This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior
I'd expect the test to complete normally
Actual behavior
It crashes as described above
Steps to reproduce
Just run the test
The Composable Architecture version information
1.14.0
Destination operating system
IOS 17
Xcode version information
Xcode 15.5
Swift Compiler version information
SWift 5

Hi @easiwriter, I am not able to reproduce this. It passes just fine for me:

And fwiw, these tests run on CI with every PR and they have been passing for a long time. Can you provide some proof of the crash?
I'm not sure this is an issue with the library, and so for now I am going to convert it to a discussion.  Feel free to continue the conversation over there.


Build failure: stig

Steps To Reproduce
Steps to reproduce the behavior:

build stig

Build log
error: asynctest-0.13.0 not supported for interpreter python3.11

Additional context
I got this when switching from 23.05 to 23.11
Notify maintainers
@doronbehar
Metadata

system: "x86_64-linux"
host os: Linux 6.1.67, NixOS, 23.11 (Tapir), 23.11.2130.d65bceaee0fb
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.18.1
channels(root): "nixos-23.11, nixos-hardware, nixos-unstable"
channels(ao): ""
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Thanks for the report, this is supposed to be fixed in https://github.com/NixOS/nixpkgs/pull/276014 . I will wait for Darwin CI to be green there and merge it. Ping me if I forget. Follow that PR at:
https://nixpk.gs/pr-tracker.html?pr=276014

Great, thanks!


Transform local matrix relative to world space or other matrix

Done
See Matrix4x4.RELATIVE
and
ComponentTransform. rebase, transform, lookAt, showTo


Retain heap based allocation

Is there an existing issue for this?

[X] I have searched the existing issues

Description
Add a feature flag sgx2 so that features can be merged quickly. E.g. otherwise it is difficult to merge mmap() without providing brk() in the same run.
Acceptance Criteria
No response
Suggestions for a technical implementation
No response

@jarkkojs can this be closed?

Yes.


Open File Location dialog

Description
When users ingest/upload data to the ODE, the app makes a copy of the file/s-folder and adds it to the ODE folder in the laptop.
To make this process transparent to the user, we need to let they know this and provide an easy way to see the exact location of the data.
Steps...
When the user uploads data to the ODE for the first time and after the file/folder is selected for the ingestion process, we need to display this dialog:

Figma file here.
Please, instead of Open FILE location, let's change the text of that button to Open location (to keep it generic in those cases when users ingest other type of data such as folders).
NOTE: I'm creating another ticket to implement the Open location feature.

As we discussed in the stand-up meeting, @guergana, please, do not implement the Open location button, just the Okay one

Hey,
The flow which leads to the position of the Open Location dialog works like this:
User clicks on Upload your data -> File import dialog pops up -> User selects the folder/file action -> Windows/Mac/Linux file browser pops up -> The selected folder/file is imported, and the Open Location dialog overlays
Please be sure to visit the Prototype, the Figma page here and the checkbox for the "Do not show again" here
https://github.com/user-attachments/assets/a935299f-51a4-46ec-9c32-55331c4f966c


Release process update

Added verification step into release process.

@pahor167 Since Truffle is being sunset, can we add steps for verifying via Hardhat?


Add build_operators step to deploy edpm

This PR has:

[x] Appropriate testing (molecule, python unit tests)
[x] Appropriate documentation (README in the role, main README is up-to-date)

/assign @cjeanner - PTAL, it should work only with CI for now, needs few more work to adapt to run in other repos.

/lgtm
that's already a nice addition! Thanks @viroel !


Use strip_path=false and preserve_host=true on TLS negotiation routes

What this PR does / why we need it:
cert-manager creates temporary ingress resource for Let's Encrypt TLS negotiation but it fails with default strip_path and preserve_host values. The doc suggests creating KongIngress resource for this operation but who wants to create another resource manually for getting an automated SSL certificate.
So, I've added few lines for changing default values and preventing KongIngress override for routes starts with "/.well-known/acme-challenge/".

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.ismail baskin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

ping @hbagdi

@ismailbaskin ping for any updates on this. Thank you!

@ismailbaskin another friendly ping for an update here.

Opened up #303 which includes this change along with a unit test.


[DESIGN-PENDING] feat: add a way to enter recovery codes

This PR resolves MOPLAT-489
Description

Final design pending. Logic ready.
https://user-images.githubusercontent.com/100233/202164088-2cb11f72-2f29-437e-909e-3f56d9bb2dc7.mov
PR Checklist

[ ] I tested my changes on iOS / Android.
[ ] I added screenshots or videos to illustrate my changes.
[ ] I added Tests and Stories for my changes.
[ ] I added an app state migration.
[ ] I hid my changes behind a feature flag.
[ ] I have prefixed changes that need to be tested during a release QA with [NEEDS EXTERNAL QA] on the changelog.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
Changelog updates

Cross-platform user-facing changes

iOS user-facing changes

Android user-facing changes

Dev changes

Need help with something? Have a look at our docs, or get in touch with us.

1 small copy change and then I think we can merge this as-is. Thanks!


Copilot blocks 'Enter' key in Python editor

I'm having a strange issue: when the copilot window is open, I am unable to edit python code. I will open the copilot window, go back to an open .py file in the VSCode editor, press the Enter key, and nothing will happen. When I close the copilot window the commands I issued while it was open are performed.
Strangely, it only seems to happen with the 'Enter' key - I can enter other text. Also, some shortcuts seem to work while other do not, e.g. "Ctrl+Enter" will insert a new line while the copilot is open, but "Shit+Enter" (my keybinding for executing a line in the interactive python terminal) does not.
Logs here:
https://s3.console.aws.amazon.com/s3/object/kite-client-logs/prod/317531/kite_local/client.log.2020-05-21_01-24-04-PM.bak.gz

Do you happen to have the "focus on docs" feature enabled in the Copilot? It's the 3rd button from the right at the bottom of the Copilot.
If so, can you try disabling this feature and seeing if you still have this problem?

The issue persists whether "focus on docs" is enabled or not.

Any update on this?

A few additions:

this doesn't happen in Atom, I can edit python code normally.
The copilot also blocks the execution of Jupyter notebooks in VSCode.
In case it wasn't clear, the copilot doesn't interfere with editing other languages (e.g. MATLAB, markdown), just python.


How to sign and verify using HS512

Does this library support signing with HS512? Also, what is the actual difference to HS256?

Hi @nostalgiatune ,
Yes, it does.
Options opt = new Options(); opt.setAlgorithm(Algorithm.HS512); final String token = signer.sign(claims, opt);
The difference is basically the hash's size. 512 bits for HS256 and 1024 for HS512 and HS384.


What are the parameters / arguments for reproducing the result in the paper?

Hi, I'm trying to train the model and reproduce the result. But noticed that some details are missing here.
What are the parameters / arguments for reproducing the result in the paper? (e.g.,lr_decrease_start, num_epochs, etc...)
Could you please provide the full training instruction?
Thanks.

Thanks for your attention.
Sorry, I forget the specific training parameters. But it should be okay to use the default training parameters.
Although this would affect several points in terms of precision, this should not affect the major conclusions of this paper.


ROSCon 2024 talk slide (draft)

Do not merge this until final slides are published.

https://github.com/user-attachments/assets/a5d4dda6-cd6b-41d1-8038-40e4906082b0

https://github.com/user-attachments/assets/6b6d8038-6d5a-4aaf-aace-bb9af4995145


added my name to guestbook.md

Hi Sean. I am submitting my pull request to add my name to your guestbook.md file. Cheers!

Thank you Yamila!


Android lint misc issues

Refactoring various lint issues. Best reviewed commit by commit.

Files touched by commit https://github.com/dimagi/commcare-odk/commit/99698b8b234900ddc687f45cd235f9da11e18b81 don't seem to be used. Can you verify and remove them?

Removed unused resources in commit https://github.com/dimagi/commcare-odk/commit/99698b8b234900ddc687f45cd235f9da11e18b81 (and a few others too).

@dcluna Looks great! I pushed a commit (https://github.com/dimagi/commcare-odk/commit/77bab19f086a0e99f03261c8ef44dcfe325726fe) that does some extra null checks and uses try/finally for closing cursors. It is a little verbose, but good to be cautious.
Let me know when you've check that commit and I'll merge the PR.


LS Life & Native UI Bug-Method not found?

The log reads as:
[10:34:07] [DEBUG] Instantiating script LSlife.LSL ...
[10:34:07] [ERROR] Failed to instantiate script LSlife.LSL because constructor threw an exception: System.MissingMethodException: Method not found: 'NativeUI.Sprite NativeUI.UIMenu.get_BannerSprite()'.
at LSlife.LSL.Setup()
at LSlife.LSL..ctor() in C:\Users\Mcal\source\repos\LSLife\LSlife\Main.cs:line 403
I haven't gotten it to work with Enable All Interiors.

Old version of https://github.com/Guad/NativeUI most likely, but anyway, not a SHVDN problem.


Do-1203 Enabling the sonarcloud integration

This addition is required to integrate the repo to sonarcloud.

duplicate


zIndex of the DrawerNavigator

Is there a way to have an element, specifically a TouchableOpacity, to still be clickable when the drawer navigator is open? I'm trying to have a button that opens and closes my drawer navigator, but even though the drawer is not rendered on top of the button, it's not clickable when the drawer is open.
It seems like you cannot manipulate the zIndex of the drawer navigator. Any tips?

I haven't had much luck with setting zIndex or elevation. A simpler approach may be to just add a close to the Drawer itself.
Something like the following will work:
DrawerNavigator(routes, {
  contentComponent: (props) => (
    <View>
      <TouchableOpacity onPress={() => props.navigation.navigate('DrawerClose')}>
        <Icon
          name="close"
          size={40}
          color="lightgrey"
        />
      </TouchableOpacity>
      <DrawerView.Items {...props} />
    </View>
  )
});

Hi @jurassix I am also facing similar issue. When there is a button on the screen, it just floats on top of the side drawer when side drawer is opened. Is there a way of setting the zIndex of the DrawerLayout so that the drawer is the top most component when it slides out.
Thanks for your help.

I believe you will need to create a custom version of the DrawerNavigator in order to inject styles into it to affect the zindex.
You might have a cleaner solution to adjust the zindex and elevation on the button to be 0 or some low value.
A screen shot and a simple example and I could help more.

Hi @jurassix
Screenshot as below:-

index.android.js
import React, { Component } from 'react';
import {
    AppRegistry,
    StyleSheet,
    Text,
    View,
    Button,
    StatusBar,
    TouchableOpacity,
    Icon
} from 'react-native';
import { DrawerNavigator } from "react-navigation";

class AboutMeScreen extends Component {
    static navigationOptions = {
        title : "About Me Screen",
        drawer : () => {
            label : "About Me"
        }
    }
    render(){
        let { navigator } = this.props.navigation;
        return (
            <View>
                <StatusBar />
                <Text>About Me Screen</Text>
            </View>
        );
    }
}

class HomeScreen extends Component {
    static navigationOptions = {
        title : "Home Screen",
        drawer : () => {
            label : "Home"
        }
    }
    render(){
        let { navigate } = this.props.navigation;
        return (
            <View>
                <Text>Home Screen</Text>
                <Button title="Go to About Me" onPress={ ()=> navigate("AboutMe") } style="{ { zIndex : 0 } }" />
            </View>
        );
    }
}

const routeOptions = {
    Home : { screen : HomeScreen },
    AboutMe : { screen : AboutMeScreen }
};

const drawerOptions = {
    drawerWidth : 280,
    drawerPosition : "left",
};

const sidenav = DrawerNavigator(routeOptions,drawerOptions);

AppRegistry.registerComponent('sidenav', () => sidenav);

Screenshot shows the button overlapping the side drawer when the side drawer is opened.
Thank you.

@tonyooi I've built your component in iOS and the behavior is working fine, even without any styles added to the button.

I do see an error with your code above. When you added styles to your button the syntax is incorrect.
Change:

<Button title="Go to About Me" onPress={ ()=> navigate("AboutMe") } style="{ { zIndex : 0 } }" />

To: (notice style is now an object)
<Button title="Go to About Me" onPress={ ()=> navigate("AboutMe") } style={{ zIndex : 0 }} />

I'll check this out in Android and see if I can reproduce ur issue.

@tonyooi I just verified that your code works on Android as well.

Here are my package.json dependencies. This could just be a version issue?
  "dependencies": {
    "react": "~15.4.0",
    "react-native": "0.42.3",
    "react-navigation": "^1.0.0-beta.7"
  }

@jurassix Thank you so much.
My package.json file
{
  "name": "sidenav",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "start": "node node_modules/react-native/local-cli/cli.js start",
    "test": "jest"
  },
  "dependencies": {
    "react": "15.4.2",
    "react-native": "0.42.3",
    "react-navigation": "^1.0.0-beta.7"
  },
  "devDependencies": {
    "babel-jest": "19.0.0",
    "babel-preset-react-native": "1.9.1",
    "jest": "19.0.2",
    "react-test-renderer": "15.4.2"
  },
  "jest": {
    "preset": "react-native"
  }
}

Seeing that you got it working and looks like there isn't much difference in the packages, I tried again by reinstalling the node modules and now it works fine.

Hi there! This issue is being closed because it has been inactive for a while. We're closing issues after a period of inactivity. Please do not take this personally :)
If you think this issue should remain open, please let us know. The following information is helpful when it comes to determining if the issue should be re-opened:

Does the issue still reproduce on the latest release? Post a comment with the version you tested.
If so, is there any information missing from the bug report? Post a comment with all the information required by the issue template.
Is there a pull request that addresses this issue? Post a comment with the PR number so we can follow up.
If you would like to work on a patch to fix the issue, contributions are very welcome! Read through the contributors guide, and feel free to hop into #react-navigation if you need help planning your contribution.


Add all python requirements in requirements.txt

requirements.txt exist and when running make install it will install the requirements automatic


Wrong type for [hosts]

Checklist for submitting an issue to KickThemOut:

[x] I have carefully read the README file and haven't managed to resolve my issue.
[x] I have searched the issues of this repo and believe that this is not a duplicate.
[x] I am running the latest version of KickThemOut.

OS name & version:  Mac OS High Sierra 10.13.6
Python version:  3.7.0
Scapy version: scapy-python3 0.25
Nmap version:  Nmap version 7.70
Link of Gist: <"https://gist.github.com/denningsrogue/f26750c37dbad96a231840753493d5a4.js"

Description:

When I run the script here is the result:
ERROR: Gateway IP could not be obtained. Please enter IP manually.
kickthemout> Enter Gateway IP (e.g. <IP_ADDRESS>): <IP_ADDRESS>
ERROR: Default Interface MAC Address could not be obtained. Please enter MAC manually.
kickthemout> Enter MAC Address (MM:MM:MM:SS:SS:SS): xx:xx:xx:xx:xx:xx
Scanning your network, hang on...
ERROR: Network scanning failed. Please check your requirements configuration.
If you still cannot resolve this error, please submit an issue here:
https://github.com/k4m4/kickthemout/issues
Details: Wrong type for [hosts], should be a string [was <class 'NoneType'>]

Hi there. Apologies for the delayed response. Are you still experiencing this issue? Thanks!

Hi, I am receiving this issue as well.

Hey there @denningsrogue & @iWrek. Please try to reinstall KickThemOut and all its dependencies; I have upgraded Scapy to the latest Kamene build. Let me know if the new release solves your issue. Thanks!

Good to hear that!


Special Interest Group Proposal: Credential Format Comparison

Introduction/background material
I would like to propose a SIG dedicated to maintaining information about available credential formats for the benefit of OWF projects and the wider community. The topic is more complex than one might assume on first sight, since there are more than 14 formats for representing digital credentials and most of those formats can be combined with different signature algorithms, ways to represent cryptographic keys (with alone more than a hundred DID methods), status management methods, trust management methods and so on.
There is pre-existing work started at Internet Identity Workshop (IIW 34, Spring 2022) and extended and augmented during Rebooting the Web of Trust (RWOT-XI, The Hague, Sept 2022).
It consists of a sheet, the “credential format comparison matrix”, containing information about the technical options in the different dimensions (formats, signature algorithms, …) as well as known credential profiles, i.e. concrete combinations used in implementations and an article explaining the “matrix”.
Sheet: https://docs.google.com/spreadsheets/d/1X93ptJcmfX1NZEo5E7ElnqJ-knDS4Dj6JOYSJ_2PsUw
Article: https://github.com/WebOfTrustInfo/rwot11-the-hague/blob/master/final-documents/credential-profile-comparison.pdf
Source Code: https://github.com/vcstuff/credential-profile-comparison
The sheet is maintained by an informal group of maintainers, who would donate it to the new SIG in order to give it a long term home. Those maintainers would also be willed to continue their work to maintain the information and are seeking for more contributors.
Objectives
The aim of the SIG is to provide information and tools to facilitate transparency of the technical options regarding credential formats and adjacent technologies in order to allow for comparison and effective decision making on credential formats.
List of deliverables or work products (optional)
The SIG shall maintain, evolve, and publish the credential format comparison matrix based on contributions from the wider identity community.
Leader
Mirko Molik
Initial Participant List
Paul Bastian
Mirko Molik
Maaike van Leuken
Andre Kudra
Torsten Lodderstedt

Approved by TAC on May 31, 2023

[ ] Create Discord channel - @tkuhrt
[ ] Work with Torsten to get GitHub repo transferred to OpenWallet Foundation - @tkuhrt @tlodderstedt
[ ] Determine leader of the SIG @tlodderstedt
[ ] Add information about Special Interest Groups to TAC website - @tkuhrt
[ ] Add link to the above to the OpenWallet Foundation GitHub Profile - @tkuhrt

@tkuhrt can you create a github repo so I can fill it with content?
@tlodderstedt you haven't created already one, have you?

Am 24. Juni 2023, 13:33 +0200 schrieb Mirko Mollik @.***>:
@tkuhrt can you create a github repo so I can fill it with content?
@tlodderstedt you haven't created already one, have you?
I haven’t.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

Created https://github.com/openwallet-foundation/credential-format-comparison-sig


Optimize Genesis for CL Module

Closes: https://github.com/osmosis-labs/osmosis/issues/5414
What is the purpose of the change
Optimizes the genesis size of CL module.
Previously 400,000 positions would lead up to genesis size of 550MB, this PR reduced this to 150MB.
This change mainly includes three major changes:

Remove uninitialized uptime tracker from state export. This was happening due to how proto unmarshalling works.
Before:

{
                        "pool_id": "2",
                        "tick_index": "-150900000",
                        "info": {
                            "liquidity_gross": "379834025645510.897800136583901047",
                            "liquidity_net": "379834025645510.897800136583901047",
                            "spread_reward_growth_opposite_direction_of_last_traversal": [],
                            "uptime_trackers": {
                                "list": [
                                    {
                                        "uptime_growth_outside": []
                                    },
                                    {
                                        "uptime_growth_outside": []
                                    },
                                    {
                                        "uptime_growth_outside": []
                                    },
                                    {
                                        "uptime_growth_outside": []
                                    },
                                    {
                                        "uptime_growth_outside": []
                                    },
                                    {
                                        "uptime_growth_outside": []
                                    }
                                ]
                            }
                        }
                    },

After:
                {
                        "pool_id": "2",
                        "tick_index": "-150900000",
                        "info": {
                            "liquidity_gross": "379834025645510.897800136583901047",
                            "liquidity_net": "379834025645510.897800136583901047",
                            "spread_reward_growth_opposite_direction_of_last_traversal": [],
                            "uptime_trackers": {
                                "list": []
                            }
                        }
                    },

All tick info has pool id. This is unnecessary since the tick structs are already with the pool struct, which contains the pool id.
Put position data into pool data, this way we can remove unnecessary pool id for each and every position

Testing and Verifying
Changed according tests
Documentation and Release Note

[ ] Does this pull request introduce a new feature or user-facing behavior changes?
[ ] Changelog entry added to Unreleased section of CHANGELOG.md?

Where is the change documented?

[ ] Specification (x/{module}/README.md)
[ ] Osmosis documentation site
[ ] Code comments?
[ ] N/A

lool, great job

Technically due to JSON marshalling, we may be making size worse in indented form by having positions as part of the pool object. I think this is perfectly fine though.

Technically due to JSON marshalling, we may be making size worse in indented form by having positions as part of the pool object. I think this is perfectly fine though.
Maybe longer term we should be imagining genesis' as a mix of only using json for modules we want to edit, otherwise using protobuf

Yeah! I am even for supporting two different types of state exports, one being protobuf and the other being json

@p0mvn Is there anything else to do for this PR? Merge with main for now

@p0mvn Is there anything else to do for this PR? Merge with main for now

Please make sure that the go.mod for the scripts is updated, and then we can merge

@p0mvn done in https://github.com/osmosis-labs/osmosis/pull/5453/commits/a97b9c3e7b21a04d3dd5aa42b655999650c79048


refactor: esm compatibility.

superseded by https://github.com/orbitdb/ipfs-pubsub-peer-monitor/pull/4


[FEATURE] Add an explanation on how to reference the API documentation

The framework has been converted to FastAPI, which allows for interactive API documentation referencing.
I would like to add an explanation on this point.


crypto.subtle error when working with the W3C credential format in React Native

There is an issue with digitalbazaar/rdf-canonize that causes trouble when working with W3C credentials in React Native (resulting in a 'crypto.subtle not found.' error). @nbAmit (who reported it to me) was able to work around it by following the instructions of this comment.
I haven't looked into it yet. I just wanted to drop this here in case others are also bumping into it.

I saw that there's already a 5.x.x release of digital credentials fork of vc-js while we're still on 1.x.x. Might be worth updating dependencies and see if some issues are resolved. the changelog mentions some issues that have been resolved for React Native in the releases


fix: fix CJS import of log at session.ts

Description
This is update for following TODO.
Updated package.json in "@wdio/electron-utils/log" to fix this TODO.

TODO: Fix CJS import of log from '@wdio/electron-utils/log'

Memo
In my understand, when CommonJS is set module  at tsconfig.json, then moduleResolution will be set node10 (same as node) as default.(docs)
Then exports field of package.json(imported module side) will be ignored. (exports field is added Node v12.7.0.)
So this PR adds a typesVersions field and tells typescript to the appropriate types file.

In order to avoid such complications, one idea is to introduce a bundler such as rollup and build both esm and cjs source code from esm.

@goosewobbler
thank you for merging this PR!


Flutter and Dart ci added

Problem / Issue No.
Fixes #12
Description
Flutter and Dart ci added

Path
.github\workflows\flutter-ci.yml

Additional Information
GSSOC 2024

@andoriyaprashant Please review my pull request


Newsletter #37: s/removes/deprecates/ C-Lightning newaddr "address" field

Mentioned by @rustyrussell here: https://twitter.com/rusty_twit/status/1105610129934700544
(Sorry!  And, wow, thanks for the amazing compliment!)

utACK


[DRAFT] Add view resource modal to search results rows

NOTE: This is a draft PR which I am submitting without tests in order to get feedback on this approach before investing in test changes.
Fixes #2172

Why was this change made?
To allow catalogers to view resources among search results before editing or copying them.
How was this change tested?
In the browser, locally.
Which documentation and/or configurations were updated?
None

@mjgiarlo, I agree with your message on Slack about the props.readOnly  being to spread through-out the components because they weren't initially designed for it. You've likely already considered this, but instead of passing down the readOnly boolean through props, could a boolean be set at the editor level in state (similar to currentResource) that the components could use in place of props?

@jermnelson :speech_balloon:

@mjgiarlo, I agree with your message on Slack about the props.readOnly being spread through-out the components as they weren't initially designed for support readOnly. You've likely already considered this, but instead of passing down the readOnly boolean through props, could a boolean be set at the editor level in state (similar to currentResource) that the components could use in place of props?

I had a hack at that. Take another look and let me know how close I got?

I had a hack at that. Take another look and let me know how close I got?

@mjgiarlo this is exactly what I was thinking, looks good!

This is exciting! Thank you! Overall looks good. I like the buttons on the bottom. A few suggestions (if it's possible):

Reduce the width of the modal so there isn't so much empty space on the right.
Can we white out the background so the modal pops more?
Could you change the buttons to this blue: 00548F (not sure if you're looking for that level of feedback here?)
Make the "x" in the upper right black and bold so it's more noticeable and easy to click.
Is it possible to remove the horizontal scroll bar?

I meant to comment rather than approve, as this is only a draft.

@astridu :speech_balloon:

This is exciting! Thank you! Overall looks good. I like the buttons on the bottom. A few suggestions (if it's possible):

Reduce the width of the modal so there isn't so much empty space on the right.
Can we white out the background so the modal pops more?
Could you change the buttons to this blue: 00548F (not sure if you're looking for that level of feedback here?)
Make the "x" in the upper right black and bold so it's more noticeable and easy to click.
Is it possible to remove the horizontal scroll bar?

I believe I've gotten all of these. Can you take a look at the new screencast above to verify and see if there are any other changes I should make? Thank you muchly!

Looks excellent!! Thank you!

@astridu Note that I had to back out the change that blurs the background because the method I used to achieve this is not compatible with React applications, and we do not have a good solution for this at this time (after consulting with both @justinlittman and @jermnelson who confirmed this).
Note to reviewers: the tests pass locally, but I get the dreaded warning about act(() => { ... }). And wrapping lines of the feature test in act() are not sufficient to make the warning go away. I'm not sure how much more time to sink into figuring this out. (I've read Kent Dodds' blog post about this phenomenon, and it seems like getting past this is a timesink.)


Undertale [PCSE01116]

Game summary

Game name: Undertale
Game serial: PCSE01116

Vita3K summary

Version: v0.1
Build number: 523
Commit hash: https://github.com/vita3k/vita3k/commit/32e9a31

Test environment summary

Tested by: IllusionMan1212
OS: Windows 10 Pro
CPU: AMD Ryzen 5 1600
GPU: NVIDIA GTX 1050Ti
RAM: 8 GB

Issues
Displays an error message
Screenshots

Log
vita3k.log

Issue:

Ugly sound, bad performance, and have graphic bug issue in left border, get blue font in place black font, use other font in option fix that

Screenshots:

Log:
vita3k.log

Issue :

graphic issue is now fixed.
have strange lag/freeze render but game continue move, and press button re get it move.

Screenschots :

I think this game is fully playable now.
App summary

App name: Undertale
App serial: PCSE01116
App version: 1.00

Vita3K summary

Version: v0.1.5
Build number: 2663
Commit hash: https://github.com/vita3k/vita3k/commit/e8151664
CPU backend: Dynarmic

Test environment summary

Tested by: gymzatan
OS: Windows 11
CPU: AMD Ryzen 7 5800H
GPU: AMD Radeon(TM) Graphics
RAM: 16 GB

Issues
Game runs at full speed without any issue.
Screenshots

Log
vita3k.log
Recommended labels

Playable

E

vita3k android v3
game: undertale [PCSE01116]
i got game crash when i was date papayrus and step one triangel


Add Validated Open Source Software list as a data source

There is a Validated Open Source Software list at https://www.ibm.com/community/z/open-source-software/ that the team @cwsolee runs maintains with links to instructions they maintain for popular open source software. It would be great to get this as a data source so folks can use the SDT for finding things on the table, instead of having to use both.

We now have a JSON endpoint that we can query and build our lists from: https://www.ibm.com/community/z/open-source-software/output/json/ :tada:

package_build needs a good amount of refactor. Hopefully this can be added as a todo during mentorship period

@pleia2 I have extracted information from the validated open source list and converted it into json files for our requirements you can find those files here do tell me if we need some improvements for it.
Thanks !

Nice job! I loaded everything up in my development instance and it looks good.
A couple corrections with the file names:

Ubuntu is based on Debian, but it's not Debian, so we'll want to be clear that it's instead Ubuntu 18.04 and Ubuntu 20.04 (not Debian Buster and Bullseye). And it's not openSUSE 15, it's SUSE Linux Enterprise Server 15 (short name is SLES 15).
And I think the file names should instead be like: IBM_Validated_OSS_List_Ubuntu_1804.json and IBM_Validated_OSS_List_SLES_15.json (leading with what it is makes them easy to identify in the alphabetized lists)

@arshPratap I think the last component of this would be taking https://github.com/arshPratap/SDT-Validated-Open-Source-List/blob/main/Validated_OpenSource.ipynb and crafting it into a script that we can include in the software-discovery-tool project. I will create a separate issue for that in that repository though.


Apply Modifiers should maybe skip the armature modifier

This is more an enhancement. I've been caught several times by this issue.
When you check the Apply Modifiers, the armature modifier will be applied too.
If you have an animation with a selected frame, the mesh will be exported with the pose applied. This will result in wrong skinning as the mesh does not match the bind pose.
It's all logical and expected, but it kind of ruin the option when you have a rigged model with animations.
Most often you will have a mirror modifier that it's nice to apply, else you'll have only half the mesh exported, but you don't want the armature to be applied to have proper skinning.
So you have to apply the mirror manually and export without the "Apply modifers", which is too bad because you loose the ability to go back on this part of the model.
Another work around is to disable the armature modifier before exporting. It works.... but you have to remembrer it every time you export and that's frustrating when you work with iterations...

Looks like the FBX exporter just skips armature modifiers, even when "Apply Modifiers" is selected. Maybe we should do the same...

Fixed in new import/exporter, https://github.com/KhronosGroup/glTF-Blender-IO.


~!POOL/LIVE!! Mosconi Cup 2024  LIVE 𝐒𝐭𝐫𝐞𝐚𝐦𝐬 FrEe BrOaDcAsT oN TV Guide & 𝐒𝐭𝐫𝐞𝐚𝐦𝐢𝐧𝐠 (Saturday 30th November)

Mosconi Cup 2024: FULL schedule, date, stream, TV channel and format as Team Europe eye fifth straight win over USA
Up for the cup
POOL fans are in for a treat as the Mosconi Cup is back and the action unfolds TODAY!
CLICK HERE TO WATCH LIVE STREAM NOW
CLICK HERE TO WATCH LIVE STREAM NOW
CLICK HERE TO WATCH LIVE STREAM NOW
Team Europe humiliated USA in Alexandra Palace to lift the title last year with a staggering 11-3 scoreline.
And as a result, it means that Europe has now won FOUR consecutive 9-ball tournaments.
But the location of this year's event could play into the hands of the Americans as they search for their first trophy since 2019.
However, it will be another huge test for the USA with the groundbreaking news that Jayson Shaw is set to captain Team Europe.
When is the Mosconi Cup 2024?
The Mosconi Cup 2024 will get underway on Saturday, November 30 before concluding on Tuesday, December 3.
Caribe Royale Resort in Orland, Florida is the chosen venue for this huge event and it can host approximately 5,000 fans.
Team Europe beat USA 11-3 at Alexandra Palace last year.
Read more on pool
GOLDEN HOUR What is a golden break in nine-ball pool and how do you complete it?
RACK 'EM UP The difference between American and British pool explained
What TV channel is it on and can it be live streamed?
The Mosconi Cup 2024 will be broadcast LIVE on Sky Sports in the UK.
Sky Sports customers can live stream all the action via the NOW app.
In the US, DAZN will hold the broadcast rights.
You can purchase a DAZN monthly subscription for $24.99 a month or $224.99 a year.
DAZN also provide an option for $19.99 per month, but you will be locked into a 12-month contract.
What are the teams?
Team Europe
Jayson Shaw (Captain of Team Europe)
Eklent Kaçi (Two-time Mosconi Cup winner)
David Alcaide (Four-time Mosconi Cup winner)
Mickey Krause (European Open Champion)
Francisco Sanchez Ruiz (Former World Champion)
Team USA
Skyler Woodward (Captain of Team USA)
Fedor Gorst (World Champion)
Shane Van Boening (Five-time US Open Champion)
Tyler Styer (Two-time Mosconi Cup winner)
Billy Thorpe (Two-time Mosconi Cup winner)
What is the new Mosconi Cup 2024 format?
The respective captains of Team Europe and Team USA will select five players to compete in the tournament.
Should the final day be required, five matches of singles will be played to decide the winner.
All matches are played as best of nine frames.
But in a slight twist to the usual rules, captains will not only lead their teams, but also compete on the table themselves - this has not happened since 2004.
What has been said?
Emily Frazer expressed her excitement ahead of the next edition of the Mosconi Cup.
The CEO said: "This is one of the most exciting opportunities in the history of the Mosconi Cup.
“After celebrating 30 years of history, today we are thrilled to reveal the latest edition of the World Nineball Tour’s centrepiece.
"This competition had flourished year on year and built a significant audience in the United States over the last three decades.
"There is no atmosphere in the world like the Mosconi Cup. Florida paves a path for Nineball fans, new and old, to witness new history and be part of this extraordinary event.”

bt

dsf

(https://sportsnowonline.com/live/soccer/)


Can't view large np array in Spyder

Hello,
Dep is:
numpy : 1.16.2
spyder : Beta
Problem:
Small array can be interpreted , but when I try to see one larger array I get error.

How to solve this issue?
Thanks

Please report the error using the form that we provide for you (i.e. the third window from to right in your screenshot above).
Just fill in the required details and send it again by providing your Github credentials (they'll be asked after you press the "Submit to Github" button on the bottom).

done,
thanks
https://github.com/spyder-ide/spyder/issues/8885#issue-417212092


[ci:component:github.com/gardener/logging:v0.48.0->v0.49.0]

Release Notes:
`docker_id` is replaced by `container_id`.

Gardener-specific multi-tenancy can be switched off by `EnableMultiTenancy`.

/kind enhancement
/retest
/cc @vlvasilev

The testmachinary must be adjusted and one line into the configuration so I am closing the PR. I will make a new one.

/close


There is a simpler way to do the DNS resolution between two VNETs

[Enter feedback here]
It seems that the Section "Name resolution between two connected virtual networks" is based on the name resolution for on-prem called "Name resolution between a virtual network and a connected on-premises network". To do a name resolution between two VNETs you do not need to set up a custom domain server. For most customers who do not have a custom DNS server on Azure it would make no sense to set one up just for HDInsights due to the mangament/operational costs. Therefore the alternative should be mentioned: Use Azure Private DNS Zones instead of a custom DNS server.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: ee1f7cfb-ea01-a540-eccc-1661000a3daa
Version Independent ID: c232a47a-ec3b-0268-5122-4e5feda20e1a
Content: Create virtual networks for Azure HDInsight clusters
Content Source: articles/hdinsight/hdinsight-create-virtual-network.md
Service: hdinsight
GitHub Login: @sreekzz
Microsoft Alias: v-sreiyer

@kasimrehman
Thanks for your feedback! We will investigate and update as appropriate.

@kasimrehman
Thank you for bringing this to my attention
Azure Private DNS Zones allow you to create and manage custom DNS domains within Azure. By associating a virtual network with a private DNS zone, you can resolve names within that virtual network using the zone's DNS servers. This approach eliminates the need to set up and manage a custom DNS server specifically for HDInsight or any other Azure service.
Using Azure Private DNS Zones is a valid alternative for name resolution between virtual networks in Azure, including for HDInsight deployments.
Please add your ideas in below link, so our production team can review it and update the same.
Ideas · Community (azure.com)

This is not a product idea. It is a documentation issue. The use of Azure Private Zones need to be documented.


promtool tsdb create-blocks-from rules should create a minimal number of blocks

Proposal
We are using https://github.com/slok/sloth to generate SLO recording rules, and want rules backfill to be part of our workflow. create-blocks-from rules writes a block for every block duration, for every rule. There's a lot of rules that we're generating, and the number of resulting blocks cause too large an impact to cortex startup time.
I'd like to add support for --max-block-duration similar to https://github.com/prometheus/prometheus/pull/8919, and have all input rules for the command output to a single series of blocks.
I'm working on this right now, but putting this out there to get some early feedback.

We have this working on a private fork, and waiting for approval to contribute upstream. Because the block ranges depend on a rule group's eval interval, we're writing all rules from a group into the same block.

@william-tran thanks for the issue :). We have planned on making the block size configurable for the rule backfilling just haven't got around to it yet. We want to make the block size config hidden because if you run the  backfiller for a large a time frame it will use up a lot of memory on the device where the backfill is running because all the samples will be stored in memory until the block is created and flushed to disk.  If you want to submit a change to implement this, it would be very welcomed. Thank you!


⚠ LPDM Handbook (hosted on Divio) has degraded performance

In 3d04972, LPDM Handbook (hosted on Divio) (https://repohubdev.tk) experienced degraded performance:

HTTP code: 500
Response time: 5344 ms

Resolved: LPDM Handbook (hosted on Divio) performance has improved in 2105bec.


feat: inversion of control location

This will allow to check any url for GitHub URLs
e.g
const url = new URL(document.referref)
console.log(isRepoDiscussionList(url)) //? true

Blocks https://github.com/sindresorhus/refined-github/pull/2974

@fregante are you ok with this? Should I add inversion of control to all functions?

The ideal way to support this would be to support plain strings as parameter, but that complicates things and forces each function to parse it every time. So your current solution of accepting a URL is interesting.
I think it’s fair, however the default should be url: URL | Location = location since location is already URL-like and doesn’t need to be reparsed every time.

however the default should be url: URL | Location = location since location is already URL-like and doesn’t need to be reparsed every time.

Ok. I agree with this. As a note, they differ a little bit
https://www.typescriptlang.org/play/index.html?ssl=1&ssc=1&pln=2&pc=61#code/MYewdgzgLgBAqgJwDYHUCWUAWICuUAyIwAhlGuAFwwCiAHsEjgCYCmAPANYsCeIAZvABK+ADQwuvAYRJlwAPgBQoSLGmlyYdFlxRESKnQbN2E-jDWywY0wLjC5QA
After first iteration, I thought maybe functions should receive a object, so more options can be passed. Exact types can be infered from used helpers parameters
e.g.
type Defaults = {
	url: Location | URL
	title: string
}

export const getDefaults = (): Defaults => ({
	url: location,
	title: document.title,
});

export const is404 = (o = getDefaults()): boolean => o.title === 'Page not found · GitHub';

type IsBlameOptions =
  & Parameters<typeof getRepoPath>
  & {} // ...
export const isBlame = (o: IsBlameOptions = getDefaults()): boolean => String(getRepoPath(o)).startsWith('blame/');

No, let’s leave DOM-based tests out of this. In a separate PR, maybe, those functions can accept a document object, which also has document.location

I'm thinking of cases where you need both url and title.

I updated to use url: URL | Location = location on isRepoDiscussionList and all nested calls

Looks good. Can you update all the URL-based functions? These are only 10 out of ~50

Ok.

I think this is also an opportunity to drop the domTestsIdentifiers regex in test.ts since DOM-based functions will now have a distinctive signature:
- 	if (validURLs === 'combinedTestOnly' || domTestsIdentifiers.test(detect.toString())) {
+ 	if (validURLs === 'combinedTestOnly' || detect.toString().startsWith('() =>') {

And in the future this would be changed to
+ 	if (validURLs === 'combinedTestOnly' || detect.toString().startsWith('(document_ = document) =>') {

@fregante done. It seems they are more than 10 :slightly_smiling_face:

Don't worry about rebasing the PR, it will be squashed anyway

@fregante domTestsIdentifiers change, breakes tests.

It's failing because those 3 functions are not URL-based but DOM-based. 2 use isUserProfile and one uses exists. You need to revert the changes to those 3 functions.

Published as 1.1.0. Thank you!
If needed, feel free to add a document_ = document argument to DOM-based detections as well


Parallel fetch with refactor to use "MultiBackend."

A Backend has only a FetchSingleSeries method, while a MultiBackend has only a FetchMultipleSeries method.
There are two MultiBackends defined (both in api/backend/multibackend.go). There are SequentialMultiBackend and ParallelMultiBackend. Both take a Backend as a field which they use for performing fetches.
The ParallelMultiBackend should be constructed using the NewParallelMultiBackend( ) function which takes a Backend and a limit (for maximum number of simultaneous fetches).
Everywhere that previously used a Backend now uses MultiBackend, using SequentialMultiBackend wherever needed.
There are no tests unit tests for these multibackends (only integrations with existing backends) and no tests for the ParallelMultiBackend at all.

LGTM


Full e2e train using library and new CI runners

Built on top of #1556
adds non-legacy training to the A10 runners. These ones should be able to run natively without disabling flash-attn.
adds flash-attn, packaging, wheel pip install's to the e2e yaml.
adds some generated data to train off of, and the model in use is granite-7b-lab.
This PR  also introduces some new handling for installing all necessary packages and reinstalling some existing ones
when doing dnf upgrade we get a nccl version mismatch because a bunch of the cuda deps are upgraded
also, the path was set to the wrong values. The outdated values were not pointing to the right nvcc or nccl causing issues when running nvidia-smi and any cuda processes.

@russellb I added some comments, made a catch-all follow up. lmk if that satisfies all of the requirements here. if not, I am happy to add some more stuff!

whoops @RobotSail it seems I pushed at the same time you reviewed, sorry If you missed the two new things I just changed

tiran's request for review was from early on the PR and was placed by Nathan. there's not outstanding feedback from him that needs another look or anything, so I'm going to let this merge.


Not working for Windows

On Windows the only thing I can do is adding folders, No Code Files are detected and also no Editor is opening.

@ivanjeremic does anything show up in the sidebar after you add a folder?
I don't have a windows machine so it's hard to diagnose without having to spin up a vm but maybe you're interested to help fix or debug it? 😅

I just ran it on a buddy's pc. Everything seemed to work fine...
@ivanjeremic can you provide the steps to reproduce the problem?

I just ran it on a buddy's pc. Everything seemed to work fine...
@ivanjeremic can you provide the steps to reproduce the problem?

Sorry but there are no steps I click on the + Button then I select an empty folder the folder is then in the left sidebar, but what now? If I add .js files in this folder nothing is shown inside the app, the only thing this app is doing for me is showing folders in the left sidebar without that whats in them.

@ivanjeremic do you have any TODO comments in those files? The app only shows a list of the todos in your project. Not all files in the folder.

Looks like a misunderstanding of how to use the app. Will update README to make it more clear. Closing this out.


Recommended way to send FormData

What is the recommended way to send a file using rails-ranger. I tried copying the form-data example from the axios repo, but it doesn't seem to work. The form has one file in it, but Chrome just shows it sending an empty json object to the server.

Here is what I tried with rails-ranger
import RailsRanger from 'rails-ranger'

const config = {
  axios: {
    baseURL: '/merchandising',
    headers: {
      'Content-Type': 'multipart/form-data',
      'Accept': 'multipart/form-data'
    }
  }
};

export default new RailsRanger(config);

// FDapi is for form-data api
FDapi.post('shelf_tag_templates/upload_image', fd,
{headers: {
  'Content-Type': 'multipart/form-data',
  'Accept': 'multipart/form-data'
}})

Rails server log:
Parameters: {"{}"=>nil}

Here is what works with Axios
import axios from 'axios';
axios.post('/merchandising/shelf_tag_templates/upload_image', fd,
{headers: {
  'Content-Type': 'multipart/form-data',
  'Accept': 'multipart/form-data'
}})

Rails server log:
Parameters: {"custom_file"=>#<ActionDispatch::Http::UploadedFile:0x00007fc8000e7c78 @tempfile=#<Tempfile:/tmp/RackMultipart20201125-27867-1ml5lx0.gif>, @original_filename="amazed.g
if", @content_type="image/gif", @headers="Content-Disposition: form-data; name=\"custom_file\"; filename=\"amazed.gif\"\r\nContent-Type: image/gif\r\n">}

Hmmmm, I never considered the file transfer scenario for the library 🤔
In fact I would suggest a very different approach instead:
Usually uploading files into the rails server is not a good idea because it will lock the request thread/process for a long time, increasing the overall response time of the application.
If it suits your scenario you could try asynchronous upload (in which the user uploads the file to S3/Azure/etc via js and then it only sends a link to rails server to keep track of).
More info in the Active Storage docs: https://guides.rubyonrails.org/v5.2.0/active_storage_overview.html

However, if the solution above doesn't fit your needs I will have to test around a bit to see if there is something that can be done or if it should be implemented as a new feature 🤔

I don't think ActiveStorage work for this situation because files may need to be uploaded to a different directory on disk based on the class name. With ActiveStorage, files have a single root directory. Shrine allows uploading files to different directories, but it is more complicated than using Ruby's File.open method. I agree that asynchronous uploads are better for performance, but I'll wait to let the other team member decide if he wants to use Shrine or upload the file directly.


Update aggregated status checks for ZenExtension

Add ZenExtension to and remove Routes from ServiceStatus when front door is enabled

looks fine to me.


Add a Release highlight page

The just released version 1.11 includes 33 changes according to the Revision history (and even more, 57, if one looks at the github release page). But a majority are technical/language fixes or improvements of little interest to users of the conventions document. In the worst case important enhancements or changes may go unnoticed.
From my very own tentative assessment about half of the entries in the Revision history are of little interest to the reader/user (other than perhaps a vague feeling that "things are now clearer more easy to understand" :-) ) :

Issue #481: Introduce units_metadata attribute and clarify some other aspects of units
Issue #147: Clarify the use of compressed dimensions in related variables
Issue #486: Fix PDF formatting problems and invalid references
Issue #490: Simple correction to Example 6.1.2
Issue #457: Creation date of the draft Conventions document
Issue #445: Updates concerning the Polar Stereographic Grid Mapping
Issue #468: Update section 2.3 to clarify recommended character set
Issue #147: Clarify the use of compressed dimensions in related variables
Issue #483: Add a missing author
Issue #463: Convert URLs with HTTP protocol to HTTPS if available, fixed a few dead links
Issue #383: Link to the CF website and deleted the Preface sectionReference UGRID conventions in CF
Issue #472: Fix incorrect formating for some <= symbols
Issue #458: Fix broken link to Unidata documentation.
Issue #423: Always use "strictly monotonic" when describing coordinate variables
Issue #420: Add List of Figures
Issue #210: Correct errors in examples H9-H11
Issue #374: Clarify rules for packing and unpacking in Section 8.1
Issue #449: Typo in end-date in Example 7.12
Issue #266: Updates to some links in the bibliography
Issue #286: Some labels of examples contain "Example" so that their label in the list of examples contains "Example" (affects four examples); corrected captions of three tables and five examples
Issue #418: Add missing examples to TOE (table of examples); cof themorrected captions of three tables and three examples
Issue #367: Delete obsolete references in section 3.3 for mappings of CF standard names to GRIB and PCMDI tables
Issue #405: Update ch. 4.4 text on reference time vs. UDUNIT
Issue #406: Remove duplicate section 8.2 in the conformance document
Issue #391: Correct link to Snyder and typo in the bibliography
Issue #437: Correct link to NUG in the bibliography
Issue #428: Recording deployment positions
Issue #430: Clarify the function of the cf_role attribute
Pull request #408: Deleted a sentence on "rotated Mercator" under Oblique Mercator grid mapping in Appendix F
Issue #265: Clarification of the requirements on bounds variable attributes
Issue #260: Clarify use of dimensionless units
Issue #410: Delete "on a spherical Earth" from the definition of the latitude_longitude grid mapping in Appendix F
Issue #153: Reference UGRID conventions in CF

The remaining ones could perhaps be divided into something like
"Major enhancements and new features"

Introduce units_metadata attribute and clarify some other aspects of units
Reference UGRID conventions in CF

"Clarifications" (some of the listed items may not qualify?)

Clarify the use of compressed dimensions in related variables
Updates concerning the Polar Stereographic Grid Mapping
Update section 2.3 to clarify recommended character set
Clarify the use of compressed dimensions in related variables
Always use "strictly monotonic" when describing coordinate variables
Clarify rules for packing and unpacking in Section 8.1
Recording deployment positions
Clarify the function of the cf_role attribute
Deleted a sentence on "rotated Mercator" under Oblique Mercator grid mapping in Appendix F
Clarification of the requirements on bounds variable attributes
Clarify use of dimensionless units
Delete "on a spherical Earth" from the definition of the latitude_longitude grid mapping in Appendix F

The "Major enhancements and new features" items should then be described with a few sentences that also refer to the relevant sections in the document. Similarly, the "Clarifications" items should be explained with a sentence, if possible lumping together related items, like for example the ones related to CRS/map projections (Polar Stereographic, rotated Mercator, and spherical Earth).

Dear @JonathanGregory
I thought a about this for a while, but looking at the list of issues that have been closed now with 1.11 I think these two categories are not fine-grained enough. While I do admit tht ther eisa degree of value judgement and sa few boundary cases on the list above, I think that we can agree that most of the ones stricken over are not of interest to the readers/users, .e.g.

https://github.com/cf-convention/cf-conventions/issues/483: Add a missing author
https://github.com/cf-convention/cf-conventions/issues/463: Convert URLs with HTTP protocol to HTTPS if available, fixed a few dead links

just to pick two random ones.
These do not need to be mentioned at all (I am not sure that all such are labelled defect, but this can of course be improved for the future).
Then we have a middle category, which are the ones that I tentatively called "Clarifications". These  are relevant to briefly highlight because they do provide a clarification in relation to what has been discussed,or asked, or otherwise influence users' understanding and interpretation of the conventions. E.g. these ones:

https://github.com/cf-convention/cf-conventions/issues/468: Update section 2.3 to clarify recommended character set
https://github.com/cf-convention/cf-conventions/issues/430: Clarify the function of the cf_role attribute
https://github.com/cf-convention/cf-conventions/issues/265: Clarification of the requirements on bounds variable attributes
https://github.com/cf-convention/cf-conventions/issues/260: Clarify use of dimensionless units
https://github.com/cf-convention/cf-conventions/issues/410: Delete "on a spherical Earth" from the definition of the latitude_longitude grid mapping in Appendix F

And then there are the (usually few) enhancements that add important new features or otherwise have more far-reaching consequences that should be highlighted.
So, I do think that with three categories we can substantially improve this. Moreover, I understand that github have more advanced mechanisms for supporting and automating categorising issues/PRs into different types thus helping out with this. But I do not have any deeper insights in how it works.

Dear Jonathan,
I realise that did not express the idea I had in mind very well (rather the opposite I am afraid). The intention was not to focus on issue categories, but rather to highlight the essence of the new release in the main webpage of cfconventions.org (hence the issue title and having the issue in this repo).
What I intended is something like these two examples
Mockup 1:

Conventions: Latest release (1.11) HTML PDF Highlights •  Working draft HTML PDF
Vocabularies: Standard names  •  Area types  •  Standardized regions

where the "Highlights"  is a link to a summary of the major new elements in the release.
Mockup 2:

Conventions: Latest release (1.11) HTML PDF  •  Working draft HTML PDF
                          Release Highlights
Vocabularies: Standard names  •  Area types  •  Standardized regions

And the same for the "Release Highlight"

And the link points to a separate web page or, perhaps preferably, to a discuss/#issue that announces the new release. Something like this ("zero-order-draft"):
Release of CF-1.11
A brief text as introduction, perhaps something like:
The Climate and Forecasting Conventions has been update to version 1.11 on 2023-12-XX. The full document is available as html and as PDF. Major new and improved features are highlighted below. Additionally, a number of minor technical and textual improvements have been made, see full details here.
Highlights

Units: Several improvements and clarifications have been made in how CF deals units. A major new addition is the introduction of the strongly recommended attribute units_context that should be attached to a data variable involving temperature data. This attribute, which is only relevant for temperature data, can take one of these three values: on_scale, difference, or unknown (default). With this attribute a data producer can unambiguously express how the units of a data variable should be converted between degree_Celsiusand Kelvin. Without this attribute, or if it is specified as unknown, such conversion is not always possible. See section 3.1 for details. In addition, the use of dimensionless units as well as the relationship between CF and the UDUNITS software package has been clarified throughout the text.
....
....
Coordinate reference systems (CRS) and map projections: Appendix F  sees three clarifications and changes:

It is now clarified that the Latitude-longitude grid mapping is not limited only to spherical representation of the Earth.
The jargon term "rotated Mercator" is now removed from a comment under the Oblique Mercator. THe reason is that it is ambiguous in relation to well-established terminology. Instead use either Oblique Mercator or Transverse Mercator.
For the Polar stereographic grid mapping the parameter straight_vertical_longitude_from_pole is deprecated in favor of longitude_of_projection_origin, which is consistent with what is used by CF for other grid mappings. In addition, it is clarified that the closest relevant proj library link is stere.

....
....

Dear @larsbarring
Sorry I missed your point. I probably read the title of the issue, but forgot about it once I started reading your categorisation of the revisions included in 1.11! I agree that a web page of release notes would be useful, highlighting the changes which users should be made aware of. Not to be deterred from my theme, though (!), I would suggest that it's the enhancement issues that ought to be described in the highlights web page, while the defect issues do not need to be described because they do not change what users do. Moreover, the text to be contributed to the highlight page could be written in the revision history of the conventions document when the PR is prepared i.e. longer text than the one line we currently have for each issue. It would be better to write these as we go along, rather than waiting for the release, by which time their essence may have been forgotten. It would be an easy extra task on building the new version to make the highlights web page by copying the text from the revision history.
Best wishses
Jonathan


Allow pass custom decoder for decoding LogGroup

Currently, we can only convert a record of LogGroup to a JSONObject
and emit to internal queue as String, with all fields like source, tags,
topic etc. However, that's may cost too much memory and time for unnecessary
fields like topic. We can skip this fields when decoding LogGroup.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


iOS: Terminated due to memory issue

After uploading one or multiple files the app crashes due to a memory issue. This happens at around 1:30 min. after the upload is started/finished (not sure).
The only log I get is: Message from debugger: Terminated due to memory issue
On my iPad it is reproducible 100% of the time.
Commit: dbdc47d33905cc05ae939ab2d5bec2e678923703
My three plist keys are:
FUMaximumConnectionsPerHost: 1
FUMaximumUploadOperation: 1
FUTimeoutInSeconds: 3600
My call:
taskId = await FlutterUploader().enqueue(
        url: url,
        files: [FileItem(path: file.path)],
        method: UploadMethod.POST,
        data: data,
        headers: headers,
      );

Device Info:

OS Version: iPhone OS 13.5 (Build 17G68)
Hardware model: iPad7,5

How big is the file, roughly?

It's just a tiny .pages file with 100KB

I have the same issue and it's five 20Mb picture

this is a really bad issue theres some sort of memory leak going on

@joshoconnor89 @Curvel please retry this with the latest beta version 2.0.0-beta.5. If you can still reproduce it, please provide a sample app.

I still see what appears to be a memory leak on 3.0.0-beta.3. I have plenty of successful uploads for a minute or two, then start ramping up memory usage
Most RawUploads are between 100B and 4KB
URLSessionDidSendBodyData: chillisource.flutter_uploader.upload.background.0FF9B967-1FEA-4158-AC81-69A3D0342895, byteSent: 132, totalBytesSent: 132, totalBytesExpectedToSend: 132, progress:100.0
URLSessionDidReceiveData:
URLSessionDidCompleteWithError: chillisource.flutter_uploader.upload.background.0FF9B967-1FEA-4158-AC81-69A3D0342895 with response: <NSHTTPURLResponse: 0x28307f180> { URL: http://localhostnamegoeshere.private:8000/v1/stream/foo/bar } { Status Code: 200, Headers {
    "Content-Length" =     (
        1
    );
    "Content-Type" =     (
        "application/json"
    );
    Date =     (
        "Tue, 07 Dec 2021 23:02:27 GMT"
    );
} } and status: 200
URLSessionDidCompleteWithError: upload completed
URLSessionDidCompleteWithError: response: 6, task: completed
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_RESOURCE RESOURCE_TYPE_MEMORY (limit=1850 MB, unused=0x0)
    frame #0: 0x00000001bda8cd40 libobjc.A.dylib`AutoreleasePoolPage::AutoreleasePoolPage(AutoreleasePoolPage*) + 88
libobjc.A.dylib`AutoreleasePoolPage::AutoreleasePoolPage:
->  0x1bda8cd40 <+88>:  str    w12, [x19, #0xc]
    0x1bda8cd44 <+92>:  adrp   x12, 17
    0x1bda8cd48 <+96>:  add    x12, x12, #0xe85          ; =0xe85
    0x1bda8cd4c <+100>: str    w22, [x19]
Target 0: (Runner) stopped.
Lost connection to device.
Exited (sigterm)

From a profile build (and debug looks about the same), I can see memory holding steady without the enqueue calls. With the enqueue calls, the RSS memory builds up and periodically returns to baseline. Eventually, it builds up to some iOS cutoff and the app is terminated

I could be wrong but it seems like there is no mechanism to revoke a completed task from the cached stream handler (used by the progress and result event streams). Since these methods may receive multiple calls for each upload task would they always grow in size? I would expect to generate hundreds of uploads per minute with my code that exhibits the problem.
Looking at https://github.com/fluttercommunity/flutter_uploader/blob/main/ios/Classes/CachingStreamHandler.swift#L10
The cache is a [String: [String: Any]], but I am not familiar with the code enough to know how that should grow. The SwiftFlutterUploaderPlugin seems like it would be updating this map whenever there is a state change for result or progress
There is an add and a clear method but no remove or dequeue method for this cache. The clearUploads hook isn't a realistic option to prevent memory pressure issues; however, if I call clearUploads prior to enqueue, then the are memory usage is fine.

Canceling every upload on the first result callback seems to work well so far. It doesn't require the clearUploads call either. I will post if I see some other behavior.
I am using FlutterUploader instance like this now:
// One-time listener installation to cancel any tasks and delete the uploaded file
uploader!.result.listen((result) {
  final path = pendingUploadTasks[result.taskId];
  if (path != null) {
    File(path).delete();
    pendingUploadTasks.remove(result.taskId);
    uploader!.cancel(taskId: result.taskId);
  }
});
....
// Enqueue a file and save the taskId and path key-value pair
final file = File('adsf');
await file.writeAsBytes(bufferedBytes, mode: FileMode.write, flush: true);
final taskId = await uploader!.enqueue(
  RawUpload(
    url: 'https://foo.bar/v1/stream/foobar',
    path: file.path,
  ),
pendingUploadTasks[taskId] = file.path;

You're right that CachingStreamHandler may be the issue here. I'll investigate. Thx @trueb2


remove dvc lib requirement

it should be done w/o DVC api. I don't like that we keep depending on it. It's really heavy, it's a very strong assumption that it's installed this way ...

Originally posted by @shcheklein in https://github.com/iterative/dvclive/issues/381#issuecomment-1344611831
Dvclive now depends on the dvc api for a few things like:

saving experiments -- can we call the cli?
checking for dvc repro execution -- is there another way to detect this?
anything else?

Mixed feelings.

it's a very strong assumption that it's installed this way

Not saying this is not True, but I think we should use the existing analytics for loyal users and check the numbers.

It's really heavy

I agree it's heavy. Not a good excuse but in the Python data science context (DVCLive scope), dvc is probably not a heavy dependency.

saving experiments -- can we call the cli?
checking for dvc repro execution -- is there another way to detect this?

My main concern here is that the quick alternatives like using CLI and/or reimplementing existing logic (like we do with make_checkpoint) have more than a few downsides, IMO.

anything else?

If we consider this really critical, we could consider what are the critical parts that can be decoupled from the rest of the DVC repo and create a separate, lightweight package.
For some things is trickier, like exp save, although as we discussed in the P.R. I do believe most (all?) of the operations DVCLive really cares about could be done with scmrepo only.

On my machine and a fresh .venv with only pip install dvc, these are the sizes of the dependencies sorted by size:

 14M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/pygit2
 12M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/networkx
 10M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/pip
9,1M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/cryptography
7,5M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/pygments
3,4M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/setuptools
3,4M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/dulwich
3,2M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/dvc
3,0M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/future
3,0M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/celery
3,0M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/asyncssh
2,6M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/pytz
2,1M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/aiohttp
1,9M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/rich
1,6M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/psutil
1,5M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/kombu
1,1M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/pycparser
1,0M	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/fsspec
828K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/urllib3
724K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/pyparsing
684K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/cffi
672K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/omegaconf
672K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/click
668K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/billiard
492K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/idna
476K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/wcwidth
472K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/tqdm
428K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/gitdb
404K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/requests
380K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/commonmark
372K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/amqp
352K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/scmrepo
348K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/diskcache
328K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/tomlkit
300K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/certifi
276K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/packaging
268K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/funcy
248K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/grandalf
236K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/yarl
232K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/dvclive
200K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/voluptuous
196K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/pathspec
176K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/tabulate
160K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/colorama
156K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/multidict
144K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/smmap
116K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/frozenlist
116K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/dictdiffer
112K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/distro
 92K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/dpath
 88K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/shtab
 76K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/filelock
 56K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/vine
 56K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/shortuuid
 52K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/attrs
 16K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/nanotime
 12K	/Users/daviddelaiglesiacastro/Desktop/test-dvc-deps/.venv/lib/python3.9/site-packages/aiosignal

If we are willing to invest efforts, it might be more beneficial to optimize how dependencies are handled in DVC (make optional imports here and there + configure install requirements) than to invest time in decoupling / moving common code away.

If we are willing to invest efforts, it might be more beneficial to optimize how dependencies are handled in DVC (make optional imports here and there + configure install requirements)

Any specific ideas about this?
Asking because I can't think about a good solution from the top of my head. The only thing that comes to my mind is to make dvc[dvclive] that would not even install DVC as a CLI (so that you could keep using global CLI), but would install only very basic requirements that are needed for DVCLive. What that would be?

Any specific ideas about this?
Asking because I can't think about a good solution from the top of my head. The only thing that comes to my mind is to make dvc[dvclive] that would not even install DVC as a CLI (so that you could keep using global CLI), but would install only very basic requirements that are needed for DVCLive. What that would be?

Something like that but not so dvclive specific.
I was thinking something like dvc[api] (IMO make_experiment makes sense as part of the public Python API) and only include the minimum required for using the public Python API .
All the UI/CLI and queue stuff would be not needed and those appear to be most of the heaviest dependencies

dvc[api] can become too wide open (Repo API, etc).
may be dvc[experiments]?

Is a thin version of dvc something that's worth spending time on at the moment? Many of the heaviest deps are scm-related, so they are needed anyway, and I don't really want to limit what parts of dvc can be used by dvclive if it's not necessary.
It's useful to be able to install dvclive without dvc, but as an optional dependency dvclive[dvc], I think it makes sense if you want to use dvc features like saving experiments or making checkpoints. Let's say dvclive writes out images or model files that should be tracked by dvc. scmrepo won't be enough to track these files. Including it as an optional dependency (and removing the "fake" dvclive dependency from dvc) would also help maintain version compatibility between them since the new dvclive features are only compatible with the latest dvc versions.
To reiterate some points that I think we all already know: It's common for other helper libraries like this to depend on their "parent" library (lightning, keras, etc.), and dvclive already depends on these much heavier ML frameworks. This pattern is so common that I think trying to go against it will only confuse most Python/DS users. Also, looking at our event data, only about 5% come from binary installs. I'm not sure the global cli approach is that common among DS, because many DS are used to the conda or other python packaging system and terrified to change versions of a package that is working for their current model setup.

because many DS are used to the conda or other python packaging system and terrified to change versions of a package that is working for their current model setup.

Exactly, thats why isolating (e.g. install CLI globally) would be an easier way for users to deal with this. DVC opens the whole "can of worms" with all its dependencies- there will a lot of case when some wheel is missing, some conflict between tqdm that DVC needs and being used by my code, etc, etc. It's a very large surface of additional conflicts and pain.

It's common for other helper libraries like this to depend on their "parent" library (lightning, keras, etc.)

Hmm. I'm not sure this is true tbh. Loggers come as part of the main frameworks mostly? And don't require any additional dependencies? Especially of the DVC size, especially the one that installs and overrides potentially CLI. Unless I'm missing something.

@shcheklein I don't want to dispute your legitimate concerns, which we can keep discussing, but I want to acknowledge the reality of the pydata community, which encompasses most of our users.
For tf, they started to bundle keras with tf2, but keras still has tf as a dependency and uses it as a lib. For torch, it doesn't come with any high-level framework, and the high-level frameworks for it (lightning, fast.ai) depend on it and use it as lib (they even have cli). Those deps lists include things like pandas, numpy, scipy, sklearn, etc. that are all way heavier than dvc (same with mlflow).
My take: DVC is still changing frequently, so I would be wary to have a global install that could break my projects with updates or leave me with an old version forever. I think most DS would rather deal with dependency hell than risk a version change breaking the reproducibility of their project, so they install the latest version of dvc inside their venv at the start of each project and never touch it again (which the lack of binary installs seems to support). I'm not saying it's a good engineering practice, but this is what users do.

I thought that most of them were including loggers out of the box, like this https://docs.wandb.ai/guides/integrations/lightning :
from pytorch_lightning.loggers import WandbLogger
from pytorch_lightning import Trainer

but from what I see wandb is quite lightweight anyways: https://github.com/wandb/wandb/blob/main/requirements.txt (no heavy DS/ML deps)
For MLFlow there is https://pypi.org/project/mlflow-skinny/ . But I agree, that if people have to install it as a regular full mlfow with all extras - that's problematic as well. I doubt ppl are happy about this.
(btw, I don't think that keras, tf, pytorch, etc - are good examples in this case. They are main frameworks, it's expected that they are heavy, might depend on each other, etc - it's natural. I would try to analyze callbacks, loggers in this case. If it becomes painful to install an extra logger that becomes painful and people might not have enough motivation)

@shcheklein, any specific concerns regarding dvc's dependency? I know dvc[all] is pretty heavy, but plain pip install dvc is relatively lightweight. Users of dvclive usually will have even heavier dependencies.

@skshetry a few concerns:

pip install dvc will be overriding the global CLI install while not providing the necessary remotes support. Thus it will be breaking users setup / workflow. Means we have to depend on dvc[all] for this to not break existing setups.
If we setup it before hand (like we do with devcontainers) we have to setup it with all to make it more or less general, otherwise it breaks the point, we would have to have a separate "transition" state that will be manual, involved, etc.
If we install dvc into users space with pip before everything else (again, devcontainers experience) it's a path into conflicts with the existing requirements that people have. It's one thing when people prepared everything gradually, and maintain their file (even it's painful), another is when we want to smoothly transition them into the ecosystem.
I'm not sure DVC is that small and lightweight even w/o clouds support. What is the complete recursive list of deps?

@skshetry a few concerns:

pip install dvc will be overriding the global CLI install while not providing the necessary remotes support. Thus it will be breaking users setup / workflow. Means we have to depend on dvc[all] for this to not break existing setups.
If we setup it before hand (like we do with devcontainers) we have to setup it with all to make it more or less general, otherwise it breaks the point, we would have to have a separate "transition" state that will be manual, involved, etc.
If we install dvc into users space with pip before everything else (again, devcontainers experience) it's a path into conflicts with the existing requirements that people have. It's one thing when people prepared everything gradually, and maintain their file (even it's painful), another is when we want to smoothly transition them into the ecosystem.
I'm not sure DVC is that small and lightweight even w/o clouds support. What is the complete recursive list of deps?

Recursive list of deps (click to expand)
$ pipdeptree -p dvc -e dvc-http
Warning!! Cyclic dependencies found:
* dvc-http => dvc => dvc-http
* dvc => dvc-http => dvc
------------------------------------------------------------------------
dvc==2.33.3.dev137+g198daec42
  - appdirs [required: >=1.4.3, installed: 1.4.4]
  - colorama [required: >=0.3.9, installed: 0.4.6]
  - configobj [required: >=5.0.6, installed: 5.0.6]
    - six [required: Any, installed: 1.16.0]
  - distro [required: >=1.3.0, installed: 1.8.0]
  - dpath [required: >=2.0.2,<3, installed: 2.0.8]
  - dvc-data [required: ==0.28.4, installed: 0.28.4]
    - attrs [required: >=21.3.0, installed: 21.4.0]
    - dictdiffer [required: >=0.8.1, installed: 0.9.0]
    - diskcache [required: >=5.2.1, installed: 5.4.0]
    - dvc-objects [required: ==0.14.0, installed: 0.14.0]
      - fsspec [required: >=2022.10.0, installed: 2022.11.0]
      - funcy [required: >=1.14, installed: 1.17]
      - shortuuid [required: >=0.5.0, installed: 1.0.11]
      - tqdm [required: >=4.63.1,<5, installed: 4.64.1]
      - typing-extensions [required: >=3.7.4, installed: 4.4.0]
    - funcy [required: >=1.14, installed: 1.17]
    - nanotime [required: >=0.5.2, installed: 0.5.2]
    - pygtrie [required: >=2.3.2, installed: 2.5.0]
    - shortuuid [required: >=0.5.0, installed: 1.0.11]
  - dvc-render [required: ==0.0.15, installed: 0.0.15]
  - dvc-task [required: ==0.1.8, installed: 0.1.8]
    - celery [required: >=5.2.0,<6, installed: 5.2.7]
      - billiard [required: >=<IP_ADDRESS>,<4.0, installed: <IP_ADDRESS>]
      - click [required: >=8.0.3,<9.0, installed: 8.1.3]
      - click-didyoumean [required: >=0.0.3, installed: 0.3.0]
        - click [required: >=7, installed: 8.1.3]
      - click-plugins [required: >=1.1.1, installed: 1.1.1]
        - click [required: >=4.0, installed: 8.1.3]
      - click-repl [required: >=0.2.0, installed: 0.2.0]
        - click [required: Any, installed: 8.1.3]
        - prompt-toolkit [required: Any, installed: 3.0.33]
          - wcwidth [required: Any, installed: 0.2.5]
        - six [required: Any, installed: 1.16.0]
      - kombu [required: >=5.2.3,<6.0, installed: 5.2.4]
        - amqp [required: >=5.0.9,<6.0.0, installed: 5.1.1]
          - vine [required: >=5.0.0, installed: 5.0.0]
        - vine [required: Any, installed: 5.0.0]
      - pytz [required: >=2021.3, installed: 2022.6]
      - vine [required: >=5.0.0,<6.0, installed: 5.0.0]
    - funcy [required: >=1.17, installed: 1.17]
    - kombu [required: >=5.2.0,<6, installed: 5.2.4]
      - amqp [required: >=5.0.9,<6.0.0, installed: 5.1.1]
        - vine [required: >=5.0.0, installed: 5.0.0]
      - vine [required: Any, installed: 5.0.0]
    - shortuuid [required: >=1.0.8, installed: 1.0.11]
  - dvclive [required: >=1.2.2, installed: 1.2.2]
    - dvc-render [required: >=0.0.12, installed: 0.0.15]
    - ruamel.yaml [required: >=0.17.11, installed: 0.17.21]
  - flatten-dict [required: >=0.4.1,<1, installed: 0.4.2]
    - six [required: >=1.12,<2.0, installed: 1.16.0]
  - flufl.lock [required: >=5, installed: 7.1.1]
    - atpublic [required: >=2.3, installed: 3.1.1]
    - psutil [required: >=5.9.0, installed: 5.9.4]
  - funcy [required: >=1.14, installed: 1.17]
  - grandalf [required: ==0.6, installed: 0.6]
    - future [required: Any, installed: 0.18.2]
    - pyparsing [required: Any, installed: 3.0.9]
  - hydra-core [required: >=1.1.0, installed: 1.3.0]
    - antlr4-python3-runtime [required: ==4.9.*, installed: 4.9.3]
    - omegaconf [required: ~=2.2, installed: 2.2.3]
      - antlr4-python3-runtime [required: ==4.9.*, installed: 4.9.3]
      - PyYAML [required: >=5.1.0, installed: 5.4.1]
    - packaging [required: Any, installed: 21.3]
      - pyparsing [required: >=2.0.2,!=3.0.5, installed: 3.0.9]
  - iterative-telemetry [required: ==0.0.6, installed: 0.0.6]
    - appdirs [required: Any, installed: 1.4.4]
    - distro [required: Any, installed: 1.8.0]
    - filelock [required: Any, installed: 3.8.2]
    - requests [required: Any, installed: 2.28.1]
      - certifi [required: >=2017.4.17, installed: 2022.9.24]
      - charset-normalizer [required: >=2,<3, installed: 2.1.1]
      - idna [required: >=2.5,<4, installed: 3.4]
      - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.13]
  - networkx [required: >=2.5, installed: 2.8.8]
  - packaging [required: >=19.0, installed: 21.3]
    - pyparsing [required: >=2.0.2,!=3.0.5, installed: 3.0.9]
  - pathspec [required: >=0.9.0,<0.10.0, installed: 0.9.0]
  - psutil [required: >=5.8.0, installed: 5.9.4]
  - pydot [required: >=1.2.4, installed: 1.4.2]
    - pyparsing [required: >=2.1.4, installed: 3.0.9]
  - pygtrie [required: >=2.3.2, installed: 2.5.0]
  - pyparsing [required: >=2.4.7, installed: 3.0.9]
  - requests [required: >=2.22.0, installed: 2.28.1]
    - certifi [required: >=2017.4.17, installed: 2022.9.24]
    - charset-normalizer [required: >=2,<3, installed: 2.1.1]
    - idna [required: >=2.5,<4, installed: 3.4]
    - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.13]
  - rich [required: >=10.13.0, installed: 12.6.0]
    - commonmark [required: >=0.9.0,<0.10.0, installed: 0.9.1]
    - pygments [required: >=2.6.0,<3.0.0, installed: 2.13.0]
  - ruamel.yaml [required: >=0.17.11, installed: 0.17.21]
  - scmrepo [required: ==0.1.4, installed: 0.1.4]
    - asyncssh [required: >=2.7.1,<3, installed: 2.12.0]
      - cryptography [required: >=3.1, installed: 38.0.3]
        - cffi [required: >=1.12, installed: 1.15.1]
          - pycparser [required: Any, installed: 2.21]
      - typing-extensions [required: >=3.6, installed: 4.4.0]
    - dulwich [required: >=0.20.46, installed: 0.20.50]
      - urllib3 [required: >=1.25, installed: 1.26.13]
    - fsspec [required: >=2021.7.0, installed: 2022.11.0]
    - funcy [required: >=1.14, installed: 1.17]
    - gitpython [required: >3, installed: 3.1.29]
      - gitdb [required: >=4.0.1,<5, installed: 4.0.10]
        - smmap [required: >=3.0.1,<6, installed: 5.0.0]
    - pathspec [required: >=0.9.0,<0.10.0, installed: 0.9.0]
    - pygit2 [required: >=1.10.0, installed: 1.11.1]
      - cffi [required: >=1.9.1, installed: 1.15.1]
        - pycparser [required: Any, installed: 2.21]
    - pygtrie [required: >=2.3.2, installed: 2.5.0]
  - shortuuid [required: >=0.5.0, installed: 1.0.11]
  - shtab [required: >=1.3.4,<2, installed: 1.5.8]
  - tabulate [required: >=0.8.7, installed: 0.9.0]
  - tomlkit [required: >=0.11.1, installed: 0.11.6]
  - tqdm [required: >=4.63.1,<5, installed: 4.64.1]
  - typing-extensions [required: >=3.7.4, installed: 4.4.0]
  - voluptuous [required: >=0.11.7, installed: 0.13.1]
  - zc.lockfile [required: >=1.2.1, installed: 2.0]
    - setuptools [required: Any, installed: 65.5.0]

$ pipdeptree -p dvc-http -e dvc
Warning!! Cyclic dependencies found:
* dvc-http => dvc => dvc-http
* dvc => dvc-http => dvc
------------------------------------------------------------------------
dvc-http==2.27.2
  - aiohttp-retry [required: >=2.5.0, installed: 2.8.3]
    - aiohttp [required: Any, installed: 3.8.3]
      - aiosignal [required: >=1.1.2, installed: 1.3.1]
        - frozenlist [required: >=1.1.0, installed: 1.3.3]
      - async-timeout [required: >=4.0.0a3,<5.0, installed: 4.0.2]
      - attrs [required: >=17.3.0, installed: 21.4.0]
      - charset-normalizer [required: >=2.0,<3.0, installed: 2.1.1]
      - frozenlist [required: >=1.1.1, installed: 1.3.3]
      - multidict [required: >=4.5,<7.0, installed: 6.0.2]
      - yarl [required: >=1.0,<2.0, installed: 1.8.1]
        - idna [required: >=2.0, installed: 3.4]
        - multidict [required: >=4.0, installed: 6.0.2]
  - fsspec [required: Any, installed: 2022.11.0]

Flattened list of dependencies (83 in total)
aiohttp==3.8.3
aiohttp-retry==2.8.3
aiosignal==1.3.1
amqp==5.1.1
antlr4-python3-runtime==4.9.3
appdirs==1.4.4
asyncssh==2.12.0
async-timeout==4.0.2
atpublic==3.1.1
attrs==21.4.0
billiard==<IP_ADDRESS>
celery==5.2.7
certifi==2022.9.24
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.3
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
colorama==0.4.6
commonmark==0.9.1
configobj==5.0.6
cryptography==38.0.3
dictdiffer==0.9.0
diskcache==5.4.0
distro==1.8.0
dpath==2.0.8
dulwich==0.20.50
dvc-data==0.28.4
dvc-http==2.27.2
dvclive==1.2.2
dvc-objects==0.14.0
dvc-render==0.0.15
dvc-task==0.1.8
filelock==3.8.2
flatten-dict==0.4.2
flufl.lock==7.1.1
frozenlist==1.3.3
fsspec==2022.11.0
funcy==1.17
future==0.18.2
gitdb==4.0.10
GitPython==3.1.29
grandalf==0.6
hydra-core==1.3.0
idna==3.4
iterative-telemetry==0.0.6
kombu==5.2.4
multidict==6.0.2
nanotime==0.5.2
networkx==2.8.8
omegaconf==2.2.3
packaging==21.3
pathspec==0.9.0
prompt-toolkit==3.0.33
psutil==5.9.4
pycparser==2.21
pydot==1.4.2
pygit2==1.11.1
Pygments==2.13.0
pygtrie==2.5.0
pyparsing==3.0.9
pytz==2022.6
PyYAML==5.4.1
requests==2.28.1
rich==12.6.0
ruamel.yaml==0.17.21
scmrepo==0.1.4
setuptools==65.5.0
shortuuid==1.0.11
shtab==1.5.8
six==1.16.0
smmap==5.0.0
tabulate==0.9.0
tomlkit==0.11.6
tqdm==4.64.1
typing_extensions==4.4.0
urllib3==1.26.13
vine==5.0.0
voluptuous==0.13.1
wcwidth==0.2.5
yarl==1.8.1
zc.lockfile==2.0

@daavoo Was this the issue you meant to close? I don't follow how it's fixed by #413.

@daavoo Was this the issue you meant to close? I don't follow how it's fixed by #413.

Didn't notice that GitHub considered the issue linked because of the pending to resolve #413 phrase

I think this is a good goal that we should see if we can work towards.
In the short term, looking at issue like https://github.com/iterative/studio-support/issues/73, I would suggest that we are better off making DVC a dependency of DVCLive (and removing DVCLive from DVC) until we are able to stop calling DVC as a lib. At least things will be more explicit, and if DVCLive is installed, DVC will also be installed as a lib and reduce the chances of things breaking.

In the short term, looking at issue like iterative/studio-support#73, I would suggest that we are better off making DVC a dependency of DVCLive (and removing DVCLive from DVC) until we are able to stop calling DVC as a lib. At least things will be more explicit, and if DVCLive is installed, DVC will also be installed as a lib and reduce the chances of things breaking.

Opened https://github.com/iterative/dvclive/pull/481 , doing the opposite to the issue title 😅 but I think it is the best to do for now.

We need to double check if we need to change VS Code or Studio now to install DVCLive explicitly. Also demo projects.

I thought that most of them were including loggers out of the box, like this https://docs.wandb.ai/guides/integrations/lightning :
from pytorch_lightning.loggers import WandbLogger
from pytorch_lightning import Trainer

Since we are discussing it, we should put DVCLive callbacks into the ML frameworks, and it shouldn't be dependent on this issue (none of the logger libs are installed as deps).

Also demo projects.

Both example-get-started and example-get-started include dvclive explicitly already in their requirements.

Closing in favor of https://github.com/iterative/dvc/issues/9709


Perf and usability improvements

Parallelizes file uploads, handles rate limiting, and avoids creating the assistant every time if it already exists.
Pushed some related improvements to assistants to do ANN against thousands of files in parallel:
https://github.com/datastax/astra-assistants-api/commit/547d63e34259683b988c9d44320b67a5070b21dc
Something I'm not clear on is when / how do we want to trigger reindexing given that rows are based on different commit hashes? My original plan was to just checkout to a /HASH/ directory but that changed in some of the restructuring. I guess the smart thing to do would be to keep track of file hashes and only re-index files that changed.
Thoughts?

Just pasting my remark from chat:
On the reindex question, I’ve thought about this but never tried it: In an embedding store you have tags on the docs usually. So tags can include repo and git hash. So maybe at index time for each file you mark it with every hash that has an exercise and for which the file hasn’t changed from that version. Then you should be able to query at every hash and get the right version without indexing redundancy.
If multiple processes absolutely needed to concurrently navigate the same repo at different hashes, that could still avoid multiple clones using git worktrees or git show.


kube-down.sh should remove entries from ./kube/...

When a cluster is torn down the corresponding cluster information under ./kube/... should be removed.

@jlowdermilk if you like you can re-assign this to me and I can do it for GCE.

If you have cycles to work on it now, have at it. Otherwise I can get to it later this afternoon.

Oh, not today, I am totally focused on the doc fixit.


Hide 'seitan' from UI display

please tag @keybase/react-hackers if you're editing front end code @joshblum

will do. any further changes for this one?

No all good