Blog post about GoCD and our experiences working with it.

Thanks @nbozic, your post will be up on our website soon!


Improve directory name extraction from branch name.

Fixes #493
Fixes #494
This improves directory extraction to handle two previously unhandled cases:

a multi-segment directory with a non-standard separator is now properly reconstructed by joining on /
the - included in the depname-version portion is properly accounted for when the delimiter is -.

I did not address the fact that the existing non-standard separator test uses a separator (|) that does not appear to be valid according to the dependabot documention.

👋 thanks for this! I'm on paternity leave right now so hesitant to merge this until I return, as I don't want to risk any problems that others would have to cleanup... but when I return in June I plan to take a look at this and hopefully ship it.

There are some merge conflicts that you will need to resolve by rebasing/merging and re-running the build script before I can merge this

Hi @dreid ,
Can you please resolve the merge conflicts so that we may proceed to complete the code merge.
Conflicting files
dist/index.js
src/dependabot/update_metadata.ts


Change JSON operations in LDP derived tables to work with both json and jsonb types

LDP 2.0 is planned to use the jsonb data type instead of json, which will be a breaking change.  This PR attempts to convert all JSON operations in LDP derived tables to support both data types, in order to minimize the impact of upgrading to LDP 2.0.
Note that these changes use the -> and ->> JSON operators (and other techniques) to allow us to continue supporting LDP 1.x for some period of time while also supporting LDP 2.0.  However, for Metadb derived tables, I would still recommend standardizing on functions such as jsonb_extract_path_text() as they are significantly more readable.
No need to approve this PR, but comments are welcome.

That sounds exciting @nassibnassar . I noticed that when a type conversion took place after the extraction function (json_extract...), the brackets of the function were not deleted. I'm not sure what the implications of this are, or if it's even necessary.

That sounds exciting @nassibnassar . I noticed that when a type conversion took place after the extraction function (json_extract...), the brackets of the function were not deleted. I'm not sure what the implications of this are, or if it's even necessary.

Thanks, yes, the parentheses are required when doing a type conversion, because otherwise the type conversion is evaluated first.

Thank you for tagging me in this so I can stay in the loop. I wasn't aware of LDP 2.0. Since Metadb is supposed to take over eventually is 2.0 going to be the last iteration of the OG LDP? Can we keep calling it "LDP1" when trying to distinguish between it and Metadb?

Thank you for tagging me in this so I can stay in the loop. I wasn't aware of LDP 2.0. Since Metadb is supposed to take over eventually is 2.0 going to be the last iteration of the OG LDP? Can we keep calling it "LDP1" when trying to distinguish between it and Metadb?

My guess is that 2.x will be the last versions of LDP, but it is hard to know for sure.
Good question about the naming.  My efforts to fix the naming problems have not succeeded.  But in any case, it might be useful to differentiate between LDP 1 and 2.


setting and retrieving content

I have been wanting to use the webcomponent version of tinymce and have been able to successful add it to a webpage. However, I did not find a way (attribute) to set content or retrieve content to the component.
In my use-case I am adding it as a component within my own custom webcomponent that has other editing components as well (such as file input etc)
Thanks

Hi @rrrepos
I think you should be able to get/set the value content of the editor by using the element value as with any other input element. E.G.:
document.getElementById('my-editor').value = '
new content
';
console.log(document.getElementById('my-editor').value)

Ah yes ... as simple as that :+1:  However, the following error seems to show on the console
Uncaught TypeError: Cannot read property 'toString' of undefined
    at Object.isEditorUIElement (tinymce.min.js:2)
    at tinymce.min.js:2
    at E (tinymce.min.js:2)
    at Object.getParent (tinymce.min.js:2)
    at am (tinymce.min.js:2)
    at tinymce.min.js:2
    at Object.each (tinymce.min.js:2)
    at Object.hd (tinymce.min.js:2)
    at Ei.executeHandlers (tinymce.min.js:2)
    at HTMLDocument.s (tinymce.min.js:2)

The html is:
<tinymce-editor id="editor" width="90%" height="300" api-key="..."></tinymce-editor>

Sorry @rrrepos I have tried replicating that behaviour without luck. Can you elaborate a bit more on the circumstances when that happens? Does it happens every time?

Let me check it again @jscasca and share more details of it. Thanks

I'm closing the issue due to a lack of activity but feel free to reopen if you are still experiencing issues with the component


allow any number of kubeconfig files

Restricting .kubeconfig files to only four locations made it see exactly where files were coming from, but it made extending the mechanism just about impossible.  This allows a separate precedence order slice that allows easy extensions.  ExplicitPath (previously CommandLinePath) is special and still called out because if the ExplicitPath is set, the referenced file must be present.  Missing files in the rest of the precedence order are simply ignored.

@smarterclayton This is what @fabianofranz wanted for https://github.com/openshift/origin/pull/992.  It makes extension of loading order to allow more files possible.

2 comments otherwise LGTM.

Seems reasonable to me.


Fix: fixed Y & Y2 ticks misalignment (fixes #209)

What is the purpose of this pull request? (Check any that's applicable)

[ ] Feature
[x] Bug fix
[ ] Dependency updates
[ ] Documentation updates
[ ] Build related changes
[ ] Changes an existing functionality
[ ] Other, please explain:

Does this introduce a breaking change?

[ ] Yes
[x] No

What changes did you make? (Give an overview)
Updated the logic behind displaying ticks. Previously, it was done automatically through the D3.js nice function, resulting in a different number of ticks, and hence misalignment. This was updated to be based on a fixed number for both Y & Y2 axis, and the values are calculated manually. The default number of ticks depends on the range of the Y & Y2 axis, but this can be overridden by passing a ticksCount property in the graph configuration object.
Add any screenshots
Before:

After:

I guess we need to add ready for ux review label for this pr. Please correct me if i am wrong.

I guess we need to add ready for ux review label for this pr. Please correct me if i am wrong.

Yes, I believe you're right. But I don't seem to have permissions to update labels.

@sdadn LGTM analysis has given one alert for one change. Can you take a look at that alert?
https://lgtm.com/projects/g/cerner/carbon-graphs/rev/pr-91b8a7059028d447b4a25349f910ce7622b8dede

@sdadn LGTM analysis has given one alert for one change. Can you take a look at that alert?
https://lgtm.com/projects/g/cerner/carbon-graphs/rev/pr-91b8a7059028d447b4a25349f910ce7622b8dede

@Veera81 LGTM check is now passing.

Can you fix the LGTM alert?

:tada: This PR is included in version 2.5.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


rerolling missions

Describe the Bug
when rerolling a mission, it will pick 1 of 2 types of missions consistantly
Reproduction Steps

go to missions
reroll selected
it becomes 1 of 2 types of missions
(it appears to determine the mission on boot up of game, further testing required)
when rerolling again, it switches to the other mission of the 2 picked
(if 'place blocks' and 'ride animal' missions are chosen, then one will become the other and vice versa)

https://github.com/Kryeit/Missions/assets/58529249/0809c4ac-6462-4900-b6ba-fb723f47cafc
Expected Result
a random mission of the X amount total would be picked with a highly unlikelihood chance of a repeat
(i dont remember how many missions there are total)
Additional Context
No response

version?

1.20.1
missions:0.3.2

can you test with 0.3.4?

I can, ittle be a few hours til I can tho

it now appears to crash the game entirely
latest.log

i really didnt mean for the log to look suspicious
i thought it would show the text inside

same happens with 0.3.3
latest.log

Please delete config/missions/ folder as said in the changelog

Will do

once there you have the new rerolling logic, so maybe this issue is fixed :)

it functions normally now :)
thank you for your help :)


Type error thrown when using StateNotifierProvider.family

Describe the bug
In riverpod 2.0.2 I started to get the error below:
[log] type '((StateNotifierProviderRef<Y0, Y1>) => Y0, {String? name, Family<dynamic>? from, Object? argument, List<ProviderOrFamily>? dependencies, (() => String)? debugGetCreateSourceHash}) => StateNotifierProvider<AnimatorNotifier, AnimatorState>' is not a subtype of type '((StateNotifierProviderRef<AnimatorNotifier, AnimatorState>) => AnimatorNotifier, {Object? argument, List<ProviderOrFamily>? dependencies, Family<dynamic> from, String? name}) => StateNotifierProvider<AnimatorNotifier, AnimatorState>' of 'function result'

To Reproduce
It's hard to reproduce, the error appears at random times and very rarely but the code that causes is:
    ref.listen<Type of someVal>(
      someProvider{ERROR THROWN HERE}(someArg).select((state) => state.someVal),
      (_, __) {},
    );

The whole code sequence works without an error, but after X time this error is thrown for the same code that worked a couple of minutes ago.
Edit: This problem can be fixed only by HotRestart. When I leave the screen, invalidate this provider and go back or try to use this provider again, then this problem is thrown.

Logs

[log] type '((StateNotifierProviderRef<Y0, Y1>) => Y0, {String? name, Family<dynamic>? from, Object? argument, List<ProviderOrFamily>? dependencies, (() => String)? debugGetCreateSourceHash}) => StateNotifierProvider<AnimatorNotifier, AnimatorState>' is not a subtype of type '((StateNotifierProviderRef<AnimatorNotifier, AnimatorState>) => AnimatorNotifier, {Object? argument, List<ProviderOrFamily>? dependencies, Family<dynamic> from, String? name}) => StateNotifierProvider<AnimatorNotifier, AnimatorState>' of 'function result'
[log] #0      FamilyBase.call (package:riverpod/src/framework/family.dart)
package:riverpod/…/framework/family.dart:1
#1      WidgetRefAnimatorX.listenIndexAnimation
package:package_name/…/extension/widget_ref_animator_extension.dart:107
#2      _MyWidgetState.build
package:package_name/…/components/my_widget.dart:115
#3      StatefulElement.build
package:flutter/…/widgets/framework.dart:4992
#4      ConsumerStatefulElement.build
package:flutter_riverpod/src/consumer.dart:488
#5      HookElement.build
package:flutter_hooks/src/framework.dart:416
#6      ComponentElement.performRebuild
package:flutter/…/widgets/framework.dart:4878
#7      StatefulElement.performRebuild
package:flutter/…/widgets/framework.dart:5050
#8      Element.rebuild
package:flutter/…/widgets/framework.dart:4604
#9      BuildOwner.buildScope
package:flutter/…/widgets/framework.dart:2667
#10     WidgetsBinding.drawFrame
package:flutter/…/widgets/binding.dart:882
#11     RendererBinding._handlePersistentFrameCallback
package:flutter/…/rendering/binding.dart:378
#12     SchedulerBinding._invokeFrameCallback
package:flutter/…/scheduler/binding.dart:1175
#13     SchedulerBinding.handleDrawFrame
package:flutter/…/scheduler/binding.dart:1104
#14     SchedulerBinding._handleDrawFrame
package:flutter/…/scheduler/binding.dart:1015
#15     _rootRun (dart:async/zone.dart:1391:13)
#16     _CustomZone.run (dart:async/zone.dart:1293:19)
#17     _CustomZone.runGuarded (dart:async/zone.dart:1201:7)
#18     _invoke (dart:ui/hooks.dart:150:10)
#19     PlatformDispatcher._drawFrame (dart:ui/platform_dispatcher.dart:318:5)
#20     _drawFrame (dart:ui/hooks.dart:115:31)

I'd love to help, but without something that I can execute, I don't know what's going on

Do you have some stacktrace maybe?

Once I hit this error again I'll provide the stack trace, sorry for that. In the meantime, I'm trying to come up with a minimal reproducible example. Can you hint me please on this error, what's it about, so I can research this problem further? From the description, I can think of maybe some type inference related issue, the provider is not recognizing the type...

That's the thing. I've never seen this before
It could be a bug. But if so, you're the first to report it. So I can't really help you without more info

Ok, I return with more info soon

Attached logs and update a little bit of code that causing the problem, when I come up with MRE, I post it as well.

Are you maybe overriding the provider? What's your provider definition?

Provider definition:
final animatorNotipod = StateNotifierProvider.family<AnimatorNotifier, AnimatorState, String>(
  (ref, key) {
    return AnimatorNotifier();
  },
  name: 'animatorNotipod',
);

No, I'm not overriding this provider, just use family. But I'm extending WidgetRef.listen method to write additional logic and in this logic, I'm updating AnimatorNotifier but it should be fine since I'm not updating the value I'm listening to.

If I remove the .family from this provider without changing the other code, the error is not reproducible. Something in the Family modifier causing this.

I was going to open an issue about this problem. In my case, the error only appears after I hot reload a few times the file where my providers are declared. I'm just watching on a StreamProvider.family, nothing custom.

I managed to create a minimal reproducible example.
Run this code and do a hot reload by adding a blank line for example.
Hope this helps.
import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';

final counter = StreamProvider.family<int, void>((ref, _) async* {
  yield 0;
});

void main() {
  runApp(const ProviderScope(child: MyApp()));
}

class MyApp extends ConsumerStatefulWidget {
  const MyApp({super.key});

  @override
  ConsumerState<MyApp> createState() => _MyAppState();
}

class _MyAppState extends ConsumerState<MyApp> {
  @override
  Widget build(BuildContext context) {
    Future.microtask(() {
      const int iterations = 1000; // A smaller number will trigger the error slower.
      for (int i = 0; i < iterations; i++) {
        ref.read(counter(i));
      }
    });

    Future.microtask(() {
      setState(() {});
    });

    return Container();
  }
}

Providing my implementation of custom listen and state I operate with:
State:
@freezed
class AnimatorState with _$AnimatorState {
  const factory AnimatorState({
    @Default(<AnimationModel>[]) List<AnimationModel> animations,
    @Default(<RunningAnimation>[]) List<RunningAnimation> running,
  }) = _AnimatorState;
}

listen extension:
// IndexAnimation implements AnimationModel
void listenIndexAnimation<T extends IndexAnimation>(
    String key,
    String uid,
    int index,
    TickerFuture Function(T animation) animator, {
    void Function()? whenComplete,
  }) {
    listen<List<AnimationModel>>(
      // the bug is thrown here
      animatorNotipod(key).select((s) => s.animations),
      (prev, curr) async {
        if ((prev?.length ?? 0) >= curr.length) return;

        final animations = curr.whereTypeAndIndex<T>(index);

        if (animations.isEmpty) return;

        if (animations.length > 1) {
          throw Error();
        }

        final animation = animations.first;

        final state = read(animatorNotipod(key));
        final notifier = read(animatorNotipod(key).notifier);
        final running = RunningAnimation(uid, animation);
        final isRunning = state.running.contains(running);

        if (isRunning) return;

        notifier.addRunning(running);

        animator(animation).whenCompleteOrCancel(() {
          if (!animation.isCompleted) {
            animation.complete();
            notifier.removeAnimation(animation);
          }
          notifier.removeRunning(running);

          whenComplete?.call();
        });
      },
    );
  }

I hope this helps.

Sadly not really haha

I tried your example @roscale but didn't reproduce the problem (besides it destroying my CPU haha)

It's weird because the stacktrace is broken
Honestly it sounds like a Dart bug related to hot-reload.

Yeah, it's actually really weird. From my understanding, FamilyBase receives the wrong providerFactory after the hot-reload, but I don't think it should be possible because you always give it the right constructor.

Ok I found the problem. It's both a Dart bug and a Riverpod bug.
In the internals of Family, it's passing the Provider's constructor as tear-off.
But the constructor has for prototype Whatever Function(..., {Family? from}) yet the Family internals expect Whatever Function(..., {Family from})
Notice the Family? vs Family
In the function prototype, it's missing either a ? or a required.
So the types mismatch.
But for some reason, the Dart compiler doesn't catch it .... unless you do a hot-reload somehow?

Actually, Riverpod's code is valid thinking about it. The types should match.
It's probably the opposite instead. The Dart compiler works, but hot-reload doesn't

Ok removing the constructor tear-off did fix it.
Changing:
class StreamProviderFamily<R, Arg> extends FamilyBase<StreamProviderRef<R>,
    AsyncValue<R>, Arg, Stream<R>, StreamProvider<R>> {
  /// The [Family] of [StreamProvider].
  StreamProviderFamily(
    super.create, {
    super.name,
    super.dependencies,
  }) : super(providerFactory: StreamProvider<R>.new);
}

to:
/// The [Family] of [StreamProvider].
class StreamProviderFamily<R, Arg> extends FamilyBase<StreamProviderRef<R>,
    AsyncValue<R>, Arg, Stream<R>, StreamProvider<R>> {
  /// The [Family] of [StreamProvider].
  StreamProviderFamily(
    super.create, {
    super.name,
    super.dependencies,
  }) : super(providerFactory: (create, {name, from, argument, dependencies}) {
          return StreamProvider<R>(
            create,
            name: name,
            from: from,
            argument: argument,
            dependencies: dependencies,
          );
        });
}

remove the error.

Done, here's the Dart issue: https://github.com/dart-lang/sdk/issues/50148
I don't feel like changing all the providers to remove the constructor tear-off. It's a bit tedious and error-prone.
To begin with, I refactored families to use constructor tear-offs on purpose ...
It'd be great if the Dart team could fix this and making cherry-pick the bug.

Describe the bug In riverpod 2.0.2 I started to get the error below:
[log] type '((StateNotifierProviderRef<Y0, Y1>) => Y0, {String? name, Family<dynamic>? from, Object? argument, List<ProviderOrFamily>? dependencies, (() => String)? debugGetCreateSourceHash}) => StateNotifierProvider<AnimatorNotifier, AnimatorState>' is not a subtype of type '((StateNotifierProviderRef<AnimatorNotifier, AnimatorState>) => AnimatorNotifier, {Object? argument, List<ProviderOrFamily>? dependencies, Family<dynamic> from, String? name}) => StateNotifierProvider<AnimatorNotifier, AnimatorState>' of 'function result'

To Reproduce
It's hard to reproduce, the error appears at random times and very rarely but the code that causes is:
    ref.listen<Type of someVal>(
      someProvider{ERROR THROWN HERE}(someArg).select((state) => state.someVal),
      (_, __) {},
    );

The whole code sequence works without an error, but after X time this error is thrown for the same code that worked a couple of minutes ago.
Edit: This problem can be fixed only by HotRestart. When I leave the screen, invalidate this provider and go back or try to use this provider again, then this problem is thrown.
Logs

I believe I'm having the same error https://github.com/rrousselGit/riverpod/discussions/1984
In my case, my StateProvider is a bool that is changed when a scroll position on a listview as at a certain point
NotificationListener<ScrollNotification>(
                    onNotification: (scrollNotification) {
                      if ((scrollNotification.metrics.extentBefore > 70) && (ref.read(showTitleProvider(widget.myhash).notifier).state == false))
                      {
                                   ....
                                 ref.read(showTitleProvider(widget.myhash).notifier).state = true;
                      }
...
}

Its hard to make the bug come out, but I can make it come out if I very very very quickly scroll back and fourth at that point (70pixels in my case)forcing the flag to repeatedly switch between true and false.  If I do it enough times and fast enough then the bug comes out.  Otherwise I have no rhyme or reason as to how many times forces it to trigger.

The error was fixed in the Dart SDK recently. So I'll close this

flutter 3.7 still has this error.

type '((StateProviderRef) => Y0, {String? name, Family? from, Object? argument, List? dependencies, (() => String)? debugGetCreateSourceHash}) => StateProvider' is not a subtype of type '((StateProviderRef) => int?, {Object? argument, List? dependencies, Family from, String? name}) => StateProvider' of 'function result'

The error was fixed in the Dart SDK recently. So I'll close this

in which Dark SDK version was it fixed? this error started popping after upgraded to flutter 3.7 / dart 2.19.
The following _TypeError was thrown building DismissibleHint(dirty, dependencies: [UncontrolledProviderScope], state: _ConsumerState#065b9):
type '((AutoDisposeProviderRef) => Y0, {String? name, Family? from, Object? argument, List? dependencies, (() => String)? debugGetCreateSourceHash}) => AutoDisposeProvider' is not a subtype of type '((AutoDisposeProviderRef) => bool, {Object? argument, List? dependencies, Family from, String? name}) => AutoDisposeProvider' of 'function result'
• Flutter version 3.7.0 on channel stable
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision b06b8b2710 (11 days ago), 2023-01-23 16:55:55 -0800
• Engine revision b24591ed32
• Dart version 2.19.0
• DevTools version 2.20.1

Same here:
type '((AutoDisposeStateNotifierProviderRef<Y0, Y1>) => Y0, {String? name, Family<dynamic>? from, Object? argument, List<ProviderOrFamily>? dependencies, (() => String)? debugGetCreateSourceHash}) => AutoDisposeStateNotifierProvider<TimerNotifier, TimerState>' is not a subtype of type '((AutoDisposeStateNotifierProviderRef<TimerNotifier, TimerState>) => TimerNotifier, {Object? argument, List<ProviderOrFamily>? dependencies, Family<dynamic> from, String? name}) => AutoDisposeStateNotifierProvider<TimerNotifier, TimerState>' of 'function result'
I got the same error with:

Flutter 3.7
Dart 2.19

@rrousselGit
Unfortunately, still seeing this problem even with the latest version
Flutter 3.7.3 • channel stable • https://github.com/flutter/flutter.git
Framework • revision<PHONE_NUMBER> (3 days ago) • 2023-02-08 15:46:04 -0800
Engine • revision 248290d6d5
Tools • Dart 2.19.2 • DevTools 2.20.1

If I see this correctly, the commit fixing the mentioned Dart Issue is only in Dart 3.0.0 ?
https://github.com/dart-lang/sdk/commit/88400bdc4249b592714fd76c61df251e212cca20

If I see this correctly, the commit fixing the mentioned Dart Issue is only in Dart 3.0.0 ?
dart-lang/sdk@88400bd

Ah, ok thanks.  Will wait it out

════════ Exception caught by widgets library ═══════════════════════════════════
type '((AutoDisposeStreamProviderRef) => Stream, {required String? name, required Iterable? dependencies, required Iterable? allTransitiveDependencies, required (() => String)? debugGetCreateSourceHash, Family<Object?>? from, Object? argument}) => AutoDisposeStreamProvider<List>' is not a subtype of type '((AutoDisposeStreamProviderRef<List>) => Stream<List>, {required Set? allTransitiveDependencies, Object? argument, required (() => String)? debugGetCreateSourceHash, required Iterable? dependencies, Family<Object?> from, required String? name}) => AutoDisposeStreamProvider<List>' of 'function result'
I occasionally  get this error and I can't figure out the issue

start stopwatch
type '((ChangeNotifierProviderRef) => Y0, {required String? name, required Iterable? dependencies, required Iterable? allTransitiveDependencies, required (() => String)? debugGetCreateSourceHash, Family<Object?>? from, Object? argument}) => ChangeNotifierProvider' is not a subtype of type '((ChangeNotifierProviderRef) => TimerChangeNotifier, {required Set? allTransitiveDependencies, Object? argument, required (() => String)? debugGetCreateSourceHash, required Iterable? dependencies, Family<Object?> from, required String? name}) => ChangeNotifierProvider' of 'function result'

I've written some patches for version 2.3.6 that work around the issue by replacing constructor tear-offs with anonymous functions.
patches.zip
cd ~/.pub-cache/hosted/pub.dev/riverpod-2.3.6
git apply fix_riverpod.patch
cd ~/.pub-cache/hosted/pub.dev/flutter_riverpod-2.3.6
git apply fix_flutter_riverpod.patch

@roscale Thank you, this works for me. :-)
Hope Dart 3 will soon be stable anyway.


Plugin system to support new ES syntax?

When using flow, lack of new syntax support is a pain.
https://github.com/facebook/flow/labels/ES2015%2B
Will hegel support plugin system or reuse babel transforms? So it can support some new syntax from community plugin, instead of letting users waiting for core support for a long time?

Hi. Hegel uses Babel as a parser, but doesn't compile your code, but, Hegel needs to inference the result of an invocation of new operators, so we will try to provide analysis for any new syntax.
Thank you for contribution ^_^

Glad to hear that! Flow lacks optional chaining for a really long time... It just ignores that useful syntax.
I will close this issue since this is just a question, not a real big issue.


Is it possible to disable the PWA/worker on certain routes?

I want to disable the PWA for certain routes where I do not need PWA functionality. Is that possible with this module?

https://github.com/vite-pwa/vite-plugin-pwa/issues/218#issuecomment-1024580718

Thank you very much @userquin. I've tried that as the following unde pwa in my nuxt.config.ts but it did not work.
workbox: {
      navigateFallback: "/",
      navigateFallbackDenylist: [/^\/chat\/embed/],
      globPatterns: ["**/*.{js,css,html,png,svg,ico}"],
    },

So I am trying to exclude /chat/embed/** routes from the PWA. But I still see the 'install app' button on those pages (also devtools show an sw is registered on the page).
Does navigateFallbackDenylist fully exclude a page from the scope of the PWA or am I doing something wrong?

You can try moving the app under a subpath and then use the sw scope using that subpath, ofc you will need to add a layout to add the webmanifest and register the sw. Anyway, if you're excluding those paths, what's the problem?
I have never tested it.

You should also exclude the html pages and their assets using globIgnores, but this would be a problem


About the number of hard negative candidates

I want to know whether your models are trained with hard negatives.
In the paper's data sections (section 3.2 and 6.3), it seems that the M-BEIR query may have hard negative candidates.
But in the code's training config (like src/models/uniir_clip/clip_scorefusion/configs_scripts/large/train/inbatch/inbatch.yaml), the number hard negatives is set to 0.
So, I want to know whether your models in Table 2 are trained with hard negatives.

By default, the UniIR models(the experiment results we show In the paper) are not trained with hard-negatives.


miniserve: 0.25.0 -> 0.26.0

Description of changes
Following #279383.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@ofborg build miniserve miniserve.passthru.tests

Result of nixpkgs-review pr 284429 run on aarch64-darwin 1

1 package built:

miniserve


Query Many to Many

In this milestone, I did the following Activities

Understand the different types of relationships between tables.
Prepare complex queries that answer analytical questions.
Use primary key & foreign key mechanisms for joining tables.

Create a table named vets with the following columns:

id: integer (set it as autoincremented PRIMARY KEY)
name: string
age: integer
date_of_graduation: date

thank you very much 👍


Just gave 1.0 a try and noticed a few issues

Sometimes drag and drop results in checker-boarding black/white squares.
Unable to maintain aspect ratio when resizing, I'm not even sure what window auto dimensions is supposed to do.
There's no easy way to reset values to defaults
Zooming in too far with some scalers like FSR results in checker-boarding (falling back to bilinear instead would be better than showing nothing).
Profiles are confusing, as numbers they make me assume it will do scaling within that range but it appears it's not the case.
No way to enable onscreen navigation for images.
No way to separate scaling for Downscaling/Luma/Chroma ect.
No indication of the image's name or filepath (I guess it could be added to window title to save space?)

There were quite a few others I noticed but those were the main issues.
If you need me to supply extra info let me know.

Sometimes drag and drop results in checker-boarding black/white squares.

Don't think I understand, I would like more info on this.

Unable to maintain aspect ratio when resizing, I'm not even sure what window auto dimensions is supposed to do.

If you are talking about maintaining aspect ratio when resizing the window, that's is not implemented yet.
Window auto dimensions is auto sizing window when you open new image to dimensions of that image. If it would fit on the screen, and if not it still auto sizes but to max 90% of the screen resolution, while keeping the window aspect ratio same as the image aspect ratio.

There's no easy way to reset values to defaults

For now you can delete config file for this.

Zooming in too far with some scalers like FSR results in checker-boarding (falling back to bilinear instead would be better than showing nothing).

I would like more info on this.

Profiles are confusing, as numbers they make me assume it will do scaling within that range but it appears it's not the case.

With profiles you can set different scaling settings for different scale factor ranges. 0.0, 0.0 profile will be applied if no other profile covers current scale factor. Scaling factor of 1.0 will mean no scaling, less than 1.0 will be downscaling and larger than 1.0 will be upscaling.

No way to enable onscreen navigation for images.

Don't think I understand.

No way to separate scaling for Downscaling/Luma/Chroma ect.

Would have to separate luma and chroma since images will be decoded into RGB. Will look into this idea.
Downscaling you can handle with separate options using scale profiles, for an example 0.0, 1.0 will cover all downscaling.

No indication of the image's name or file path (I guess it could be added to window title to save space?)

You can change Window name setting to show you file name or full path.

There were quite a few others I noticed but those were the main issues.

I would like to hear about other issues as well.

Don't think I understand, I would like more info on this.
It will look like this, like it failed to render. Though this specific image is when trying to zoom in with FSR scaling (no other option an auto dimensions set) but that's what it looks like.

For now you can delete config file for this.

I was talking about resetting a specific setting to default, deleting whole config isn't really ideal.

I would like more info on this.

Same as the first image, as soon as I try to zoom in it will break. Other scaling algorithms (blackman in this example) seem to artifact like:

(the white dots)

With profiles you can set different scaling settings for different scale factor ranges. 0.0, 0.0 profile will be applied if no other profile covers current scale factor. Scaling factor of 1.0 will mean no scaling, less than 1.0 will be downscaling and larger than 1.0 will be upscaling.

Ah I see now but it seemed to use the scaling methods even when zooming in higher than the specified profile amount.

Don't think I understand.

Look at Window's photos app, next and previous buttons show up when hovering close to the left/right side of the image.

You can change Window name setting to show you file name or full path.

Ah, it doesn't change on setting change but does on restart.

I would like to hear about other issues as well.

They aren't really issues so much as they are things I personally found annoying, mainly with defaults like opening such a tiny window (it should have some sort of scaling with Window res, ugly grey background, tiny settings window, when there's no image open a button to open image would be nice (or maybe use it to display recent image thumbnails) ect.

About issues with Modified FSR and possibly other filters, I can reproduce. This is expected behavior due to wrong parameters. For an example: Modified FSR parameters have to be  b != 0, b != 2, c != 0 (b - parameter1, c - parameter2). Probably should explain all filters in user manual and warn users when they try to set incorrect parameters.

..when there's no profile to support that zoom level it should fall back to default or bilinear.

It will fall back to the default profile 0.0, 0.0 and whatever settings are set for that profile.

Look at Window's photos app, next and previous buttons show up when hovering close to the left/right side of the image.

For now you can use Next and Previous from context menu or arrow keys. Also I can imagine this interfering with user wanting to drag image.

Ah, it doesn't change on setting change but does on restart.

It will be applied on the next opened image.

ugly grey background

Its neutral. That's just default, you can change it in general options to any color.

tiny settings window

Looks good on my end. Maybe is tiny due to high pixel density or something else. I would like to investigate this. Could you provide screenshot or take picture of you screen with camera?

when there's no image open a button to open image would be nice

For now you can open image from context menu or using shortcut CTRL+ O.

With profiles you are unable to change the min scaling above 0 until you change the max, it should just check when you try saving instead.

The problem with this is that it checks when you enter value is the range correct. Example upper bound (right box) is 0.0 and you wanna set lower bound (left box) first, on confirming value into lower bound it will do the check. But since you have to add profile to the list anyway I may rework this.

Drag opening an image doesn't bring the app to the foreground.

I'll consider this as an issue. Will look into fixing this.

I'd also really like to be able to see the current zoom amount somehow.

Will look into implementing this.

Opening an image moves the window to the center of the screen.

This is expected behavior, this will only be the case if you have Auto window dimensions enabled.

About issues with Modified FSR and possibly other filters, I can reproduce. This is expected behavior due to wrong parameters. For an example: Modified FSR parameters have to be b != 0, b != 2, c != 0 (b - parameter1, c - parameter2). ~Probably should explain all filters in user manual~ (user manual is updated) and warn users when they try to set incorrect parameters.

I just used default params, it probably should default to the correct params as not everyone knows/wants to look at a guide just to get scaling working.

It will fall back to the default profile 0.0, 0.0 and whatever settings are set for that profile.

It doesn't seem to be doing that or it is breaking somehow.

For now you can use Next and Previous from context menu or arrow keys. Also I can imagine this interfering with user wanting to drag image.

When it's just on the edges it doesn't really interfere with anything.

Looks good on my end. Maybe is tiny due to high pixel density or something else. I would like to investigate this. Could you provide screenshot or take picture of you screen with camera?

I'm just on 1440p 27" which is very standard these days but I'd imagine it'd be worse at 4k.

By default the write changes just gets cut off for some tabs.

This is expected behavior, this will only be the case if you have Auto window dimensions enabled.

It's honestly rather annoying, is there a way to have it resize without the centering?

I just used default params, it probably should default to the correct params as not everyone knows/wants to look at a guide just to get scaling working.

Can agree with this, will look further into this.

When it's just on the edges it doesn't really interfere with anything.

Maybe it gets implemented after all.

It doesn't seem to be doing that or it is breaking somehow.

So far I couldn't find issue with this, it worked as expected.
Looking at those images its same as on my end. Are you referring to font size? Maybe should add option to use larger text and window.

By default the write changes just gets cut off for some tabs.

The settings window can be resized and it can be scrolled.

It's honestly rather annoying, is there a way to have it resize without the centering?

May implement option to enable or disable centering.

Looking at those images its same as on my end. Are you referring to font size? Maybe should add option to use larger text and window.

The settings window can be resized and it can be scrolled.

Sure but it's window sizes will reset on next launch....

This should be addressed now among some other things in the new release v1.1.0.

Just tried 1.2:
Looks like the black checker-boarding was entirely scaling errors. After setting params properly it seems fine (though the user shouldn't have to do that).
Params don't reset when changing scaler so it's easy to accidentally use wrong params.
I don't understand the correct params for FSR. I ended up setting B=0.1, C=1 which seems fine but I don't get what B should be (anything over 0.5 looks terrible it seems).
It would be nice to be able to edit the profile scaling conditions, currently I have to add a new profile and delete the old one which is a little tedious.
The settings window remembering position doesn't seem to work too well if the app is maximized, settings window moved to the right-most then app is minimized. It's not completely hidden though but it doesn't retain the positions if it's maximized again (even if it didn't get moved).
The overlay is nice for debugging but wasn't really what I was after, something like Window's photo viewer which combines the titlebar with such information and quick settings would be nice but probably difficult, otherwise a small dedicated bar at the bottom (possibly also include mouse navigation) would probably be best but very subjective. A bar that disappears when there's no interaction for a some time could also be pretty cool (obviously if you are hovered over it already it shouldn't disappear).
Some wishlist requests would be ability to upscale (without zooming) then downscale (with a diff scaler) for the final output would be interesting. Being able to save the filtered (and possibly scaled) image would be awesome but out of scope for a viewer.

..(though the user shouldn't have to do that for basic scaling).
Params don't reset when changing scaler so it's easy to accidentally use wrong params.

Don't see some perfect solution for this. Think we should simply set defaults instead of 0.0, 0.0 to 0.16, 1.0 cause this will be valid params for all functions.

I don't understand the correct params for FSR.

Can have look into my research, these params worked well in the research. For Modified FSR in general can tell you that C acts as a kernel blur.

It would be nice to be able to edit the profile scaling conditions

I agree with this, should look into implementing this.

..but that meant it was using it with no zoom

Its already optimized to skip any scaling if scale would be 1.0.
https://github.com/garamond13/w-image-viewer/blob/d284294863caf122a06420b9072cca60fe9c3adf/w-image-viewer/renderer.cpp#L71C4-L71C33

I don't know if it's possible to see scaling higher than 6 digits

Scale is float so anything beyond 6 decimal places can be considered to be trash.

The settings window remembering position doesn't seem to work too well if the app is maximized, settings window moved to the right-most then app is minimized. It's not completely hidden though but it doesn't retain the positions if it's maximized again (even if it didn't get moved).

Don't know what do you mean by this. Did quick test of resizing settings window, moving it around all while maximizing application, minimizing, closing, reopening. All worked as expected.

The overlay is nice for debugging but wasn't really what I was after, something like Window's photo viewer which combines the titlebar with such information and quick settings would be nice but probably difficult, otherwise a small dedicated bar at the bottom (possibly also include mouse navigation) would probably be best but very subjective. A bar that disappears when there's no interaction for a some time could also be pretty cool (obviously if you are hovered over it already it shouldn't disappear).

So something like menu bar on top (under title bar) and like status bar bottom with which you can interact? W Image Viewer is imagined to be clean and simple, but maybe implement all that so that it can be turned on or off.

Also noticed that zooming doesn't seem to take into account the position you are trying to zoom in

It looks like its always zooming in center and not where your mouse cursor is. May look into changing this.

Some wishlist requests would be ability to upscale (without zooming) then downscale (with a diff scaler) for the final output would be interesting.

I don't see what good would this do.

Being able to save the filtered (and possibly scaled) image would be awesome but out of scope for a viewer.

I think this shouldn't be too difficult to handle.

Don't see some perfect solution for this. Think we should simply set defaults instead of 0.0, 0.0 to 0.16, 1.0 cause this will be valid params for all functions.

Its already optimized to skip any scaling if scale would be 1.0.

Ah, well I'd still like to see scaling done at 1.0 but upscaling then downscaled back to 1x.

Don't know what do you mean by this. Did quick test of resizing settings window, moving it around all while maximizing application, minimizing, closing, reopening. All worked as expected.

https://github.com/garamond13/w-image-viewer/assets/1760158/a4f309ca-3307-4d98-9b15-1c4cb5872b22

So something like menu bar on top (under title bar) and like status bar bottom with which you can interact? W Image Viewer is imagined to be clean and simple, but maybe implement all that so that it can be turned on or off.

I was referring to using the title bar for such things like Windows Photo viewer to save viewing space but idk how to do that, might be hard.

Having an intractable bar at the bottom wouldn't be too bad if it's automatically hidden after 400ms or so after not hovering near the bottom (it's how I have MPV set up so I'm used to it).

It looks like its always zooming in center and not where your mouse cursor is. May look into changing this.

I don't think it's always in the center as zooming out can cause the image to be completely off-screen.

I don't see what good would this do.

In subjective tests I've seen better results upscaling past native resolution and downscaling back to native with BC-Spline (b=0.3, c=0.35). So for a 1080p image, upscaling it something like 2x with FSR then downscaling to my native 1440p with BC-Spline (b=0.3, c=0.35) results in a better image than just simply upscaling from 1080p to 1440p. Although I did all my subjective tests with video in MPV so I'm not sure if that would translate to just images.

Default params are already set to 0.16, 1.0 in the commit 305b26a.

Ah, well I'd still like to see scaling done at 1.0 but upscaling then downscaled back to 1x.

Any resampling will be destructive which means image quality will be objectively degraded compared to the original.

I was referring to using the title bar for such things like Windows Photo viewer to save viewing space but idk how to do that, might be hard.

Would have to switch to some custom title bar.
The issue with settings window shown in video is kind of expected behavior, even its not that nice.

Default params are already set to 0.16, 1.0 in the commit 305b26a.

But is that really a good solution for all scalers? Wouldn't it be better to use better values as default for each?

Any resampling will be destructive which means image quality will be objectively degraded compared to the original.

I think maybe it depends on the original images quality, if it's poor quality it seems to benefit subjectively but it will deviate far from the original. I'm obviously not asking for it to be default but the option to do it would be nice.

Would have to switch to some custom title bar.

Hmm, I guess it would be rather tricky, for now would it be possible to have an option to remove the title bar with maybe a quick way to restore it somehow? For example: MPC-HC you can remove it all and re-add it in context menu:

The issue with settings window shown in video is kind of expected behavior, even its not that nice.

Hmm... It's a tad annoying but not the end of the world. I guess you could save dimensions for 2 states separate for when window is minimized/maximized.
Also I found a small bug when maximizing the window then going fullscreen and exiting fullscreen, trying to minimize will do nothing.

But is that really a good solution for all scalers? Wouldn't it be better to use more optimal values as default for each?

No. But we can't have optimal params for all use cases anyway.

..would it be possible to have an option to remove the title bar with maybe a quick way to restore it somehow?

Not too optimistic about this, we need title bar to move the window. But maybe it gets implemented in some way.

Also I found a small bug when maximizing the window then going fullscreen and exiting fullscreen, trying to minimize will do nothing.

I tried to reproduce this. It all worked fine on my end.

Another feature request: Being able to easily swap between profiles from context menu, though not sure how to go about that? Stick with that profile until auto profiles are restored or change profile when the conditions for the profile other than the one you changed from are met?

So multiple profiles covering same range, and user chooses which one will be used. Maybe introduce custom names for profiles.

Not too optimistic about this, we need title bar to move the window. But maybe it gets implemented in some way.

I guess you could dedicate the top of the window for dragging the window around but not sure how difficult that is.

I tried to reproduce this. It all worked fine on my end.

Maybe a Windows10 issue? However no other apps seem to behave that way.
https://github.com/garamond13/w-image-viewer/assets/1760158/0c88539b-fe48-4f7d-ab00-341f4d510caa

So multiple profiles covering same range, and user chooses which one will be used. Maybe introduce custom names for profiles.

Actually I meant changing to a profile, completely ignoring ranges.

Maybe a Windows10 issue?

Don't thinks so. Think I see what you mean, will look into fixing this.

Actually I meant changing to a profile, completely ignoring ranges.

That can work too. Although wouldn't wanna overcomplicate things, its imagined to be simple to use and powerful image viewer.

Just found a newish bug, with center dimensions disabled it doesn't even try to fit the full image on screen or remember window positions:
https://github.com/garamond13/w-image-viewer/assets/1760158/353edc35-f0f7-4ba6-b9e1-7742f864ab1a
Also, I think being able to set minimum width/height is much needed, otherwise opening stuff like icons results in this:

Just found a newish bug, with center dimensions disabled it doesn't even try to fit the full image on screen or remember window positions:

Should be fixed now.

Also, I think being able to set minimum width/height is much needed, otherwise opening stuff like icons results in this:

Will look into fitting this inside settings.


digest auth broken on release WsgiDav==3.0.0a4

Not sure if I am the only one experiencing this issue but it appears that digest auth is broken in newest release WsgiDav==3.0.0a4. I have made the fix in a feature branch based off of master but am unable to push the feature branch and open a PR (403 from github).
I would love to contribute to the project and open a PR if you give me access (assuming this is a real issue).
2018-11-23 21:24:55.758 - <140603089622784> wsgidav.error_printer       ERROR   :  Caught HTTPRequestException(HTTP_INTERNAL_ERROR)
Traceback (most recent call last):
  File "/vagrant/venv/lib/python3.5/site-packages/wsgidav/error_printer.py", line 46, in __call__
    app_iter = self.next_app(environ, sub_app_start_response)
  File "/vagrant/venv/lib/python3.5/site-packages/wsgidav/http_authenticator.py", line 211, in __call__
    return self.auth_digest_auth_request(environ, start_response)
  File "/vagrant/venv/lib/python3.5/site-packages/wsgidav/http_authenticator.py", line 492, in auth_digest_auth_request
    req_username, realm_name
UnboundLocalError: local variable 'req_username' referenced before assignment

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/vagrant/venv/lib/python3.5/site-packages/wsgidav/error_printer.py", line 83, in __call__
    raise as_DAVError(e)
wsgidav.dav_error.DAVError: DAVError(500 Internal Server Error: An internal server error occurred
    Source exception: 'local variable 'req_username' referenced before assignment')

2018-11-23 21:24:55.758 - <140603089622784> wsgidav.error_printer       ERROR   :  e.src_exception:
local variable 'req_username' referenced before assignment
2018-11-23 21:24:55.758 - <140603089622784> wsgidav.wsgidav_app         INFO    :  <IP_ADDRESS> - (anonymous) - [2018-11-23 21:24:55] "OPTIONS " elap=0.005sec -> 500 Internal Server Error

Just realized you guys probably follow the Fork PR Workflow? I can do that.

Thanks, forking first should work.


Add the NVD API Key

Fixes #146

@rfennell  and @Saturate  this is already supported if you use the addition argument.
All you have to do is to check up on the CLI documentation for finegraiend control
https://jeremylong.github.io/DependencyCheck/dependency-check-cli/arguments.html
ex. --nvdApiKey YOUR_API_KEY
I personally do not think exposing all fields thru the extension is a good idea, however the documentation could be  more clear on how you can utilize the underlying CLI and pass params to it.
Personally i dont see this PR as proving much benefit but a PR with improved documentation would be awesome i think.

I agree in general not to add all CLI arguments as AZDO Task parameters.
However, given that --nvdApiKey is becoming in effect essential when using the 9.x.x version of the CLI tool

NVD API Key Highly Recommended
With 9.0.0 dependency-check has moved from using the NVD data-feed to the NVD API. Users of dependency-check are highly  encouraged to obtain an NVD API Key; see https://nvd.nist.gov/developers/request-an-api-key Without an NVD API Key dependency-check's updates will be extremely slow. Please see the documentation for the cli, maven, gradle, or ant integrations on how to set the NVD API key.

It should be exposed at the task parameter level.

I agree in general not to add all CLI arguments as AZDO Task parameters.
However, given that --nvdApiKey is becoming in effect essential when using the 9.x.x version of the CLI tool

Hi @rfennell,
it doesn't mean that the --nvdApiKey parameter is always used. In my case I have a dedicated pipeline for updating the NVD-CVE database and I use the --noupdate parameter in this task.
I agree with @tlogik for a PR for improving the documentation on this.

I agree in general not to add all CLI arguments as AZDO Task parameters.
However, given that --nvdApiKey is becoming in effect essential when using the 9.x.x version of the CLI tool

Hi @rfennell,
it doesn't mean that the --nvdApiKey parameter is always used. In my case I have a dedicated pipeline for updating the NVD-CVE database and I use the --noupdate parameter in this task.
I agree with @tlogik for a PR for improving the documentation on this.

With the changes to the NVD, using the API KEY will be the main use case for this extension right? So I think adding this parameter would be pretty useful for the majority of users - as it mostly won't work without one.
I'm all for not adding everything, but the most used ones should be there.

The way to obfuscate the API key in Azure DevOps is to save the key as a secret variable and then reference it by name in the pipeline in the from $(nvdapikey). This could be passed in via the existing additionalArguments or the proposed nvdapikey parameters.
- task: dependency-check-build-task@6
  displayName: "Vunerability Scan Exploited Vulnerabilities update check"
  inputs:
    projectName: 'Maintainance'
    scanPath: '.'
    format: 'HTML'
    additionalArguments: '--nvdApiKey $(nvdapikey)'

If you pass in the key this way, Azure DevOps will automatically obfuscate the secret in it's logs as shown below
Invoking Dependency Check...
Path: D:\a\_tasks\dependency-check-build-task_47ea1f4a-57ba-414a-b12e-c44f42765e72\6.1.3\dependency-check\bin\dependency-check.bat
Arguments: --project "Maintainance" --out "D:\a\1\TestResults\dependency-check" --scan "D:\a\1\s" --format HTML --nvdApiKey ***
C:\Windows\system32\cmd.exe /D /S /C "D:\a\_tasks\dependency-check-build-task_47ea1f4a-57ba-414a-b12e-c44f42765e72\6.1.3\dependency-check\bin\dependency-check.bat --version"
Dependency-Check Core version 9.0.3

Searching for left over lock files...
found no left over lock files, continuing...
C:\Windows\system32\cmd.exe /D /S /C "D:\a\_tasks\dependency-check-build-task_47ea1f4a-57ba-414a-b12e-c44f42765e72\6.1.3\dependency-check\bin\dependency-check.bat --project Maintainance --out D:\a\1\TestResults\dependency-check --scan D:\a\1\s --format HTML --nvdApiKey ***"

@rfennell Thanks for your quick response, but thats exactly what I'm doing and the api key is exposed in cleartext in the pipeline run.
The only difference is, that the secret is stored in an Azure keyvault and referenced as a secret in Azure DevOps -> Pipeline Library.

Ah, seen that before, it looks to be difference in behavior between running on Windows (a .bat file) or Linux (a .sh file). Somewhere in the chains of execution the obfuscation gets lost when string are combined.
The problem line is dependency-check-build-task.ts#L158
We need to edit console.log(Arguments: ${args}); to do some string replacement of the --nvdapikey 1234 with --nvdapikey ****. I have to do similar on one of my other Azure DevOps tasks as it also shells out to command line tool and secret tracking gets lost.
I will add something to this PR

Closing this as it will be solved with https://github.com/dependency-check/azuredevops/pull/155
Thanks for the PR!


Modifier suggestion: Assign inverse

Potentially useful when you want to have a mathematical function and allow calling its inverse with inv F, or allow more dual operations using under F like read-write, open-close, etc. Mathematical duals also appear in graphics domain I think.
J calls this assign obverse, or just "obverse".

I could add this, although it would be limited in that it would only let you assign inverses to operations where no information is lost in the result. This would make it usable with invert and under, but the number of functions this is useful for is limited.
The problem is that a lot of underable operations inherently destroy information.
An example is first. You can't invert first because once you have the first row, the rest of the array is gone.
under first works because it actually generates a "before" function and an "after" function, neither of which is exactly the same as first itself. These generated functions use special instructions that save data to a hidden stack so that it can be used when undoing an operation. These instructions are not available to the user.

This has been added! (experimentally)
See setinv and setund


Overlay support in language module?

Thanks for this great project! :heart:
It's what I dreamt numtide/devshell could be & more :heart_eyes:
I'm looking for a devenv-native, streamlined way to change nodejs version and also have the package managers use that version, for which the best solution I found (see discussion in discourse) was to use an overlay:
{
  overlays = [
    (final: prev: {
      nodejs = cfg.package;
    })
  ];
}

And then all added packages use that, too:
{
  packages = with pkgs; [
    yarn
    nodePackages.pnpm
  ];
}

→ pnpm node --version && yarn node --version is the selected package
For supporting this in e.g. languages/javascript.nix all we'd need is a way to add an overlay in the language module - is there a way?
A possible option (but less great & flexible):
{
  languages = {
    javascript = {
      package-manager = "pnpm";
    };
  };
}

And then in the languages/javascript.nix if packageManager is yarn add (yarn.override { nodejs = pkgs.nodejs-16_x; })
Coming from #475
Vaguely related to #16

I really like the idea of having an package-manager option in the javascript module and on non npm we could use nodejs slim to remove npm.

@tennox were you able to get these overlays working in devenv? I'm having a similar problem were even some package json dependencies fail because yarn is using the wrong node version. I'm currently still to novice at nix sadly to get it working myself.

Thanks will give that a go. Does this also work without flakes?
Just like you I came here since this is an amazing project (first tried devbox which is also an amazing project but sadly too basic for our needs). But I'm also worry about the extra complexity this introduces everywhere, as long as it works not really that big of an easy, but things like this make it tougher to sell.

Thanks will give that a go. Does this also work without flakes?

I don't see a way without integrating this into devenv (which I meant to initiate with this issue, maybe I'll craft a PR 🧐)

I'm also [worried] about the extra complexity this introduces everywhere, as long as it works not really that big of a deal, but things like this make it tougher to sell.

Exactly.
If it helps you, I can say that I have used devenv for many projects and almost never had to fiddle with/update the flake.nix in any 'difficult way' (only e.g. change nixpkgs channel/branch when a new stable nixos release came out)
Exceptions are nodejs version (see this issue) - and of course when building software via nix. (but that's just out of scope for devenv)

It seems there are two requests here:
a) supporting other package managers than npm, which we support now as per examples in https://github.com/cachix/devenv/tree/main/examples
b) supporting local overlays which we already have an issue for https://github.com/cachix/devenv/issues/478

@domenkozar  Problem is not really other package managers, problem is that package managers are using different nodejs version. Overlays are a way to fix this, but ideally it doesn't happen in the first place.
    package = pkgs.nodejs_21;
    enable = true;
    # corepack.enable = true;
    yarn = {
      enable = true;
      install.enable = true;
    };
  };

node --version
v21.7.2
yarn node --version
v20.11.1

This results in problems with packages such as better-sqlite3 when building dependencies:
yarn node -e 'console.log(process.config)'
-> node_module_version: 115,
node -e 'console.log(process.config)'
-> node_module_version: 120,

Error:
node_modules/better-sqlite3/build/Release/better_sqlite3.node'
was compiled against a different Node.js version using
NODE_MODULE_VERSION 115. This version of Node.js requires
NODE_MODULE_VERSION 120. Please try re-compiling or re-installing
the module (for instance, using `npm rebuild` or `npm install`). {

Then we should always override that they use the canonical package of nodejs

Yes indeed. Seeing this comment, would it be possible to do something similar for this snippet?
{
  overlays = [
    (final: prev: {
      nodejs = pkgs.nodejs_21;
    })
  ];
}

recently started to work for a project that already had a devcontainer with specific versions of node, pnpm and deno. Here's what I did to get the exact same versions:
devenv.yaml
# yaml-language-server: $schema=https://devenv.sh/devenv.schema.json
inputs:
  nixpkgs:
    url: github:cachix/devenv-nixpkgs/rolling
  # https://www.nixhub.io/packages/nodejs
  nodejs-18_18_0:
    url: github:NixOS/nixpkgs/517501bcf14ae6ec47efd6a17dda0ca8e6d866f9
  # https://www.nixhub.io/packages/nodePackages.pnpm
  pnpm-8_6_6:
    url: github:NixOS/nixpkgs/12d2f0da28a47f907825939630b4a0e8122fabe3
  # https://www.nixhub.io/packages/deno
  deno-1_39_0:
    url: github:NixOS/nixpkgs/2b9c57d33e3d5be6262e124fc66e3a8bc650b93d

devenv.nix
{ pkgs, lib, config, inputs, ... }:

let
  node-pkgs = import inputs.nodejs-18_18_0 { system = pkgs.stdenv.system; };
  pnpm-pkgs = import inputs.pnpm-8_6_6
    { system = pkgs.stdenv.system;
      overlays = [(final: prev: {nodePackages = prev.nodePackages.override { nodejs = node-pkgs.nodejs; }; })];
    };
   deno-pkgs = import inputs.deno-1_39_0 { system = pkgs.stdenv.system; };
in {
  languages.javascript.enable = true;
  languages.javascript.package = node-pkgs.nodejs;
  languages.javascript.pnpm.enable = true;
  languages.javascript.pnpm.package = pnpm-pkgs.nodePackages.pnpm;
  languages.javascript.pnpm.install.enable = true;

  languages.deno.enable = true;
  languages.deno.package = deno-pkgs.deno;
}

the difficult bit was to make sure that pnpm was using the same node version (pnpm node -v vs `node -v), was able to get that with an overlay and override.

vaguely related to #1382


Add fuse-7z-ng package

Fuse-7z-ng allows you to mount various archive file types with fuse.

@StijnDW Any updates on this?

Thanks for the feedback. I amend'ed my commit and rebased on nixos/master, is that the correct way to fix this? The commit now uses fetchFromGitHub.

Almost identical expression was merged in the meantime from #10462.


update distances.md

优化了指示函数的描述

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


🌱 manifestlint: move from kubeval to kubeconform

kubeval is not maintained anymore, and suggest moving to kubeconform. Change manifestlint.sh to use kubeconform.
Pin the used kubeconform image.

There is chicken/egg issue with project-infra. https://github.com/metal3-io/project-infra/pull/462

/test-ubuntu-integration-main

/hold
Holding to sync merging, due chicken/egg issue.

/unhold
We have all labels in place, and infra is merged, let's get this retested and merged.
/test manifestlint

/cherry-pick release-1.2
/cherry-pick release-1.3

/cherry-pick release-1.3

/cherry-pick release-1.3

/cherry-pick release-1.1


Custom vNets prevent Windows containers from reaching public  internet

Is this a request for help?:
Yes

Is this an ISSUE or FEATURE REQUEST? (choose one):
Issue

What version of acs-engine?:
v.18.1, v.18.5

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm)
Kubernetes v1.10.4
What happened:
Deploy a private Windows/Linux hybrid cluster within a custom vNet

Can access public internet from the Windows host

Cannot access public internet from the Windows container (iis/windowsservercore-1803)

Can access public internet from the Linux host

Cannot access public internet from the Linux container.

Deploy a private Windows/Linux hybrid cluster without a custom vNet

Can access public internet from the Windows host

Can access public internet from the Windows container (iis/windowsservercore-1803)

Can access public internet from the Linux host

Cannot access public internet from the Linux container.

What you expected to happen:
Windows containers should be able to reach the public internet.
How to reproduce it (as minimally and precisely as possible):
Deploy a hybrid cluster with a custom vNet.
Anything else we need to know:
Working with a customer this week on getting their ASP .NET 4.6 application containerized and orchestrated on Kubernetes. Toward the end of the week we ran into this blocker with custom vNets. Was hoping to document the issue here in more detail, but not sure where to look on the windows agent nodes for relevant logs/information.

By private you mean with PrivateCluster set to Enabled: true correct?

@CecileRobertMichon yep!

I'm experiencing the same, but with only Custom vnets and Azure CNI. There's a new azure-cni-networkmonitor daemonset that fails on windows nodes:
$> kg pod --all-namespaces -w -o wide
NAMESPACE     NAME                                            READY     STATUS              RESTARTS   AGE       IP               NODE
kube-system   azure-cni-networkmonitor-99v7p                  1/1       Running             0          8m        <IP_ADDRESS>      k8s-linuxpool1-43313860-vmss000000
kube-system   azure-cni-networkmonitor-hftd8                  0/1       ContainerCreating   0          8m        <IP_ADDRESS>      43313acs000001
kube-system   azure-cni-networkmonitor-xz4js                  0/1       ContainerCreating   0          8m        <IP_ADDRESS>       43313acs000000
kube-system   heapster-6f7b9bdf-ngdg5                         2/2       Running             0          8m        <IP_ADDRESS>      k8s-linuxpool1-43313860-vmss000000
kube-system   kube-addon-manager-k8s-master-43313860-0        1/1       Running             0          7m        <IP_ADDRESS>   k8s-master-43313860-0
kube-system   kube-apiserver-k8s-master-43313860-0            1/1       Running             0          7m        <IP_ADDRESS>   k8s-master-43313860-0
kube-system   kube-controller-manager-k8s-master-43313860-0   1/1       Running             0          7m        <nil>            k8s-master-43313860-0
kube-system   kube-dns-v20-cdb774968-r7gmw                    3/3       Running             0          8m        <IP_ADDRESS>      k8s-linuxpool1-43313860-vmss000000
kube-system   kube-dns-v20-cdb774968-wpzp6                    3/3       Running             0          8m        <IP_ADDRESS>      k8s-linuxpool1-43313860-vmss000000
kube-system   kube-proxy-fcv5c                                1/1       Running             0          8m        <IP_ADDRESS>   k8s-master-43313860-0
kube-system   kube-proxy-j8njl                                1/1       Running             0          8m        <IP_ADDRESS>      k8s-linuxpool1-43313860-vmss000000
kube-system   kube-scheduler-k8s-master-43313860-0            1/1       Running             0          8m        <IP_ADDRESS>   k8s-master-43313860-0
kube-system   kubernetes-dashboard-65c8bbc84b-cf4w5           1/1       Running             0          8m        <IP_ADDRESS>      k8s-linuxpool1-43313860-vmss000000
kube-system   metrics-server-5c74cf6d4f-f99tz                 1/1       Running             0          8m        <IP_ADDRESS>      k8s-linuxpool1-43313860-vmss000000
kube-system   tiller-deploy-d85ccb55c-zq4lq                   1/1       Running             0          8m        <IP_ADDRESS>      k8s-linuxpool1-43313860-vmss000000

Here's my deployment (1.10, 1803, customvnet, two pools, managed disks):
{
  "apiVersion": "vlabs",
  "properties": {
    "orchestratorProfile": {
      "orchestratorType": "Kubernetes",
      "orchestratorRelease": "1.10"
    },
    "masterProfile": {
      "count": 1,
      "dnsPrefix": "k8svnet",
      "vmSize": "Standard_D2_v2",
      "vnetSubnetId": "/subscriptions/12c7e9d6-967e-40c8-8b3e-4659a4ada3ef/resourceGroups/k8svnet/providers/Microsoft.Network/virtualNetworks/KubernetesCustomVNET/subnets/KubernetesSubnet",
      "firstConsecutiveStaticIP": "<IP_ADDRESS>",
      "vnetCidr": "<IP_ADDRESS>/16"
    },
    "agentPoolProfiles": [
      {
        "name": "linuxpool1",
        "count": 1,
        "vmSize": "Standard_D2_v2",
        "storageProfile" : "ManagedDisks",
        "vnetSubnetId": "/subscriptions/12c7e9d6-967e-40c8-8b3e-4659a4ada3ef/resourceGroups/k8svnet/providers/Microsoft.Network/virtualNetworks/KubernetesCustomVNET/subnets/KubernetesSubnet",
        "availabilityProfile": "VirtualMachineScaleSets"
      },
      {
        "name": "windowspool1",
        "count": 2,
        "vmSize": "Standard_D2_v2",
        "storageProfile" : "ManagedDisks",
        "availabilityProfile": "VirtualMachineScaleSets",
        "vnetSubnetId": "/subscriptions/12c7e9d6-967e-40c8-8b3e-4659a4ada3ef/resourceGroups/k8svnet/providers/Microsoft.Network/virtualNetworks/KubernetesCustomVNET/subnets/KubernetesSubnet",
        "osType": "Windows"
      }
    ],
    "windowsProfile": {
      "adminUsername": "azureuser",
      "adminPassword": "",
      "windowsPublisher": "MicrosoftWindowsServer",
      "windowsOffer": "WindowsServerSemiAnnual",
      "windowsSku": "Datacenter-Core-1803-with-Containers-smalldisk"
    },
    "linuxProfile": {
      "adminUsername": "azureuser",
      "ssh": {
        "publicKeys": [
          {
            "keyData": ""
          }
        ]
      }
    },
    "servicePrincipalProfile": {
      "clientId": "",
      "keyvaultSecretRef": {
        "vaultID": "",
        "secretName": "spp",
        "version": ""
      }
      }
    }

}

Looks loosely related to https://github.com/Azure/acs-engine/issues/2971 ; not even kube-dns pods can resolve hostnames:
$> kubectl --namespace=kube-system exec -it kube-dns-v20-7fcdddfcb8-pbpzw  -c kubedns -- ping google.com
PING google.com (<IP_ADDRESS>): 56 data bytes
^C
--- google.com ping statistics ---
8 packets transmitted, 0 packets received, 100% packet loss
command terminated with exit code 1

I spin up two identical clusters, one with custom vnets and one without, the latter has connectivity for windows pods. I diff everything I could think of, they are virtually the same (the only difference, the vnetResourceGroup in azure.json was populated in the broken cluster; I set it to "" on masters and nodes, rebooted everything but no help).
I'm at loss :( I can keep the clusters around for a live debug session, they're easy to recreate (btw, I'm building acs-engine from master, on commit c4b1324c).

@ams0 were you able to repro with previous acs-engine versions? (maybe 0.16 or 0.17) ie. is this a known regression or something that never worked?

Going to try with 1.7, i'll let you know! thanks @CecileRobertMichon

Can you try accessing an exteranl site both by DNS name and IP? We're still working through a different set of DNS related problems too.
Compare:
invoke-webrequest -UseBasicParsing http://ogfg.link
invoke-webrequest -UseBasicParsing http://<IP_ADDRESS>

There are known issues with Windows + custom vnet, support was never completely added and the option to enable is available as experimental: see https://github.com/Azure/acs-engine/issues/1767


ContextMap empty with spring boot 3 in spring-cloud-gateway filter

We are currently facing an issue while migrating our codebase to Spring Boot 3. We use spring cloud gateway and spring webflux in our application.
Problem description:
We have a custom filter that implements SCG GlobalFilter. However, during our migration, we've observed that the context is not consistently being populated as expected. It is empty intermittently. We've confirmed that this issue was not there with spring boot 2.x with the exact same configuration/code.
Steps Taken:
In our attempts to address this issue, we have taken the following actions:

Initially, we were using chain.filter(exchange).subscriberContext(putMDCIntoReactorContext());. However, we recognized that subscriberContext is deprecated as mentioned here
We tried different alternatives, including:

return Mono.deferContextual(contextView -> chain.filter(exchange).contextWrite(putMDCIntoReactorContext()));
return chain.filter(exchange).transformDeferredContextual((mono, contextView) -> mono.contextWrite(putMDCIntoReactorContext()));

We also experimented by removing publishOn(Schedulers.boundedElastic()) to ensure that the issue is not related to multiple threads.

Observed Behavior:
Despite our efforts, the issue persists. Sometimes, the contextMap is populated correctly, but other times, it remains empty. This inconsistency is causing unexpected logs.
Example Responses:
Here are two examples of logs from the service:

Response for URL : /api/v1/url/, responseCode: 200, Service: *** contextMap: {traceId=, spanId=}
Response for URL : /api/v1/url/, responseCode: 200, Service: *** contextMap: {}

As you can see in the below image that thread is also same across the request

Expected Outcome:
We expect the context to be consistently populated as intended, ensuring that the necessary information is available throughout our application's processing flow.
We would greatly appreciate your insights, suggestions, or potential solutions to this issue. Your assistance will help us ensure the seamless migration of our service to Spring Boot 3.

May @jonatan-ivanov or @chemicL can comment?

Hey, @amittal4 👋
Observability is a major theme across Spring Boot 3 and other portfolio projects. It might be worth considering replacing your custom logic with what's now part of the offering. The below resources should help you navigate both your custom case but should also highlight hints about these things being available out-of-the-box.
Unfortunately, as I don't have access to any code and don't know what some terms mean to you, I'm going to make a guess, so please correct me where I'm wrong and where this is not being helpful.
It seems contextMap is referring to your logger's MDC. It feels that you capture the contents of the MDC in your filter, store that into the Reactor's Context and then expect that MDC to be restored upon any Thread hop in your reactive chains. There seems to be something missing in your report, which is responsible for making that restoration. In the past it used to be some form of a Hook. E.g. Hooks.onEachOperator? Or perhaps you were using Spring Cloud Sleuth?
Regardless of that, these mechanisms have been now replaced with more reliable ones, using the context-propagation library and its integration into other projects. For custom use cases, there is now a ThreadLocalAccessor that you can use with it for interacting with the MDC. It has been recently introduced in version 1.1.1.
Project Reactor integrates with context-propagation library. You can read more about it in our reference documentation.
To make every operator have MDC populated, you need to enable automatic propagation mode, which can be achieved using a Spring Boot property spring.reactor.context-propagation=auto.
For more background behind some of these changes in Reactor, please consult this three-part blog series.
There's also a recording of a presentation I gave on this: https://www.youtube.com/watch?v=xF7aZJlfTSw

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.


A POC get_config that illustrates one of the main purposes of config2py.

One of the main purposes of config2py is to provide a simple interface for acquiring configuration values while encapsulating the potentially complex details of how the configuration value for the given key is actually acquired. The get_config(key: KT) -> VT function provides a straightforward way to retrieve a configuration value by key, but there are various complexities that may need to be taken into consideration when implementing this function, such as:

Sources: The configuration value may be obtained from various sources, such as environmental variables, configuration files, user input, or remote services. The implementation may need to consider which sources to prioritize and how to handle conflicts or missing values.
Formats: If the configuration value is obtained from a file, the file format may vary (e.g., INI, JSON, YAML, TOML, etc.). The implementation may need to be able to parse and validate the value based on the file format.
Validation: The configuration value may need to be validated before being returned to ensure that it meets certain criteria (e.g., it is of the correct type, within a certain range, or conforms to a particular format).
Caching: To improve performance, the configuration value may be cached in memory or persistent storage to be retrieved from there next time it is needed. The implementation may need to consider how to handle cache invalidation and expiration to ensure that the cached value remains up-to-date.
Fallback values: If a configuration value is not found, the implementation may need to provide a fallback value to prevent the system from failing or behaving unexpectedly.
Encryption/decryption: If the configuration value is sensitive or needs to be kept secure, the implementation may need to support encryption and decryption of the value to prevent unauthorized access.
Merging: If configuration values are obtained from multiple sources, the implementation may need to merge the values together in a consistent manner to prevent conflicts and ensure that all relevant values are considered.
Hierarchical keys: Configuration values may be organized into a hierarchical structure, with keys that contain sub-keys. The implementation may need to support hierarchical keys and provide a way to retrieve values from nested keys.
Concurrency: If multiple threads or processes are accessing the configuration values concurrently, the implementation may need to handle concurrency issues to prevent race conditions, deadlocks, or other issues.

Design
At the time of writing this, we'll chose the functional way of dealing with parametrization. That is, we'll write a function with arguments that can be set using functools.partial to produce a function that needs only one argument, the key.
The function would have an interface along the lines of:
from typing import KT, VT, Callable, Iterable, Mapping

def get_config(
    key: KT,
    sources: Iterable[Union[Callable, Mapping]],
    *,
    default: VT = no_default_sentinel,
    egress: Callable = None,
    val_is_valid: Callable[[VT], bool] = lambda x: True,
    config_not_found_exceptions: Iterable[Exception] = (Exception,)
):
    """Get a config value from a list of sources"""

This approach is not without cons. For example, the parameters handling will have to be done at every call (and we can't use lru_cache as is to get rid of this re-computation when the key chances since lru_cache caches according to the full input).
An object oriented approach would be more appropriate here (with python's limited functional support), where we give the params in the init (and they are handled/compiled at init time, only once).

Further, note that the design we're proposing here is appropriate when fetching a single key, or, if we want to fetch multiple keys with, where the params (sources, default, egress, val_is_valid etc.) are fixed for all these keys. We can envision a different situation, that will probably need a different approach (or additional tool): One that facilitates the definition of how to retrieve a configuration value for each individual key. What we have in mind for that, at this point, is using a mapping from key to it's own parametrized get_config function to retrieve it.
from typing import KT, VT, Callable, Mapping
from functools import partial

def get_config_with_getter_for_key(
    key, getter_for_key: Mapping[KT, Callable[[], VT]]
) -> VT:
    return getter_for_key[key]()

def mk_key_getter_mapping(getter_params_for_key, default=None):
    return {
        key: partial(get_config, key=key, **params)
        for key, params in getter_params_for_key.items()
    }

source_1 = {'a': 1, 'b': 2}
source_2 = {'a': 10, 'c': 3}

getter_params_for_key = dict(
    a=dict(sources=[source_1, source_2], egress=print),
    b=dict(sources=[source_2, source_1], egress=lambda k, v: k * v),
)

my_get_config = partial(
    get_config_with_getter_for_key,
    getter_for_key=mk_key_getter_mapping(getter_params_for_key)
)

assert my_get_config('a') == None
# and this is printed:
# a 1

assert my_get_config('b') == 'bb'


feat(plugin_runner): wasmer@3

Description:
This PR reattempts https://github.com/swc-project/swc/pull/5456.
Most of changes are for the breaking changes of wasmer@3, as well as enabling rkyv's strict mode (https://github.com/swc-project/swc/pull/6922). This could not be seperated since wasmer@3 enables strict mode by default without way to turn it off.
There are couple of changes worth to note:

Disabling in-memory module lookup: https://github.com/swc-project/swc/pull/7197/files#diff-3bda5def6ce2b7553c3b3a5ad241c0bdb7021e67b7de1e594df4cd5a54d403b3R154-R159
Disabling plugin_runner in bindings_wasm: https://github.com/swc-project/swc/pull/7197/files#diff-dc3ded556a1fd709a129acd588e5eda651b842c6acc3f5340d40088a1f927facR310-R312
Skipping plugin compat test: https://github.com/swc-project/swc/pull/7197/files#diff-531197dfcefba05faca53f0cf442ecc2dc6b59d5ead01979f5ffb912aa36249aR64-R66

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: kwonoj:x: djkoloskiYou have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: kwonoj:x: djkoloskiYou have signed the CLA already but the status is still pending? Let us recheck it.

Unsure what to do with

https://github.com/swc-project/swc/actions/runs/4649648894/jobs/8228139357
CLA: https://github.com/swc-project/swc/pull/7197#issuecomment-1500950948

About cargo deny, you can add a crate to allowlist
About CLA, we can ignore it.

It seems we can't make an exception for the unlicensed pkgs, (https://github.com/swc-project/swc/actions/runs/4649770971/jobs/8228348524?pr=7197#step:5:15) only global config would work. https://github.com/EmbarkStudios/cargo-deny/issues/451


Leftover Container Fix

This fixes #142 by skipping containers that are empty.
In the future, it would be nice to output container indices that were not used, or something similar.

This looks good - it'd be great if you could amend the original unit test or add a new one that would not pass before the additions and pass afterwards ?
The current test has an even distribution of items to containers, so it doesn't run into the issue.

This looks good - it'd be great if you could amend the original unit test or add a new one that would not pass before the additions and pass afterwards ?
The current test has an even distribution of items to containers, so it doesn't run into the issue.

Thanks for the push on that one @radumg !
The unit tests are updated. (I apologize if there is a better way than below to share the results, feel free to let me know if I did it wrong.)
Result before Code Fix:

Result After:

Thanks for the push on that one @radumg !
The unit tests are updated. (I apologize if there is a better way than below to share the results, feel free to let me know if I did it wrong.)
Result before Code Fix:

Result After:


Add IE 10+ compatibility to supportsCustomEvent

The problem on line 4 has to do with the IE 10 implementation of new CustomEvent( ).
The value needs to be one of the predefined values or it will throw a "Object doesn't support this action" error
document.createEvent("CustomEvent"); - this will work
and then you use
CustomEvent::initCustomEvent() - to define the type as something other than "CustomEvent"
Or for IE 10 just fallback to the IE9 workaround as shown in the PR.

I have completed the CLA.


[6.0.0-rc.4] Error on submitting form

This all previously worked, but I'm updating react/redux and redux-form and now I have js errors.
This is a component loaded into a product page.
import React, { Component, PropTypes } from 'react';
import { reduxForm } from 'redux-form';
import { addProduct } from './actions';

class AddProductForm extends Component {

  onSubmit(props) {
    this.props.addProduct(props);
  }

  render() {
    const { fields: { productID }, handleSubmit } = this.props;

    return (
      <form onSubmit={handleSubmit(props => this.onSubmit(props).bind(this))}>
        <input type="hidden" {...productID} />
        <button type="submit">Add to Cart</button>
      </form>
    );
  }
}

const formData = {
  form: 'AddProductForm',
  fields: ['productID']
}

export default reduxForm(formData, null, {addProduct})(AddProductForm);

the error that's returned is "You must either pass handleSubmit() an onSubmit function or pass onSubmit as a prop".
If I update that one line of code to be <form onSubmit={handleSubmit}>  I no longer get the error.  However, on submit the same error comes up.
[update]  I've fudged the code around to get rid of that error, but it looks like I might have taken one step forward and took another step back. addProduct isn't a part of props any more.  So how would I now call that function from my actions?

Found the solution in #1491.
// export default reduxForm(formData, null, {addProduct})(AddProductForm);
export default connect(null, {addProduct})(reduxForm(formData)(AddProductForm));

I think this would be a good candidate to include in the migration docs.

Thanks God (well, not really, thanks you).
It was driving me mad.
Couldn't see anything in the docs.
I'm tired. Thanks. Really tired.


AcceptButton is always set on the parent form

In WizardControl.cs you are setting the parent form's AcceptButton property to the nextButton. Under most circumstances this is good, but my problem is I have a multi-line textbox in one of my wizard pages. The user can't press the enter key in the textbox to go to the next line because you've set the AcceptButton to handle the enter key event and advance them to the next page.
As a workaround, I set the AcceptButton property to null in my form's load event. However, I have to hope that the DisplyColorOrCompositionChanged event handler you've defined doesn't fire, otherwise it will set the AcceptButton back to nextButton.
It would be nice if there was a way to disable his behavior.

I actually don't see how this would work as you have then effectively disabled the Next/Finish button from advancing pages and I don't expose that button for reattachment. Can't you simply set the AcceptsReturn property on the TextBox?

You're right. I hadn't thought about AcceptsReturn. That did what I needed. My workaround was effectively disabling the ability to press enter to advance pages.


use any-cloud-storage to abstract artifact storage

Pull Request
Description
Use https://github.com/nalbion/any-cloud-storage to abstract storage of artifacts
Checklist
Please review and check the following items to ensure your pull request can be processed smoothly:

[x] My code adheres to the project's coding standards and style guidelines.

[ ] I have tested my changes thoroughly, including unit tests where applicable.

[x] I have updated the documentation to reflect any changes made in this pull request.

[x] My branch is up-to-date with the latest changes from the main branch.

[ ] I have received code review feedback and addressed any comments or concerns.

References, Fixes and Closes

Better implementation of #100

I believe these changes would not be a standard agent protocol SDK that's 1:1 with the spec. For changes to the spec, there needs to be an RFC written out that details the changes.
However, I can see this being useful for the js sdk, and I'm thinking that what we could do here is take some of these changes and incorporate it into some example agents. If you can take a look at the config options RFC in the V2 branch and see if that "changes the game" per se for your implementations.

@jzanecook I don't think that the implementation of createArtifact(), registerCreateArtifact() and registerGetTaskArtifact() have anything to do with the spec.
I'd be happy to move AnyCloudArtifactStorage and ArtifactStorageFactory into a README or examples and remove the dependency on any-cloud-storage. Maybe the example could show how artifact_provider could be used to configure the storage for each request?


Two classes

Add support for two node classes, with different bandwidth and validatorsPerNode values.
Also:

rename numberValidators to numberNodes
set chi globally, and set validatorsPerNode per class

rebased to solve merge conflicts


[Question] connection to specific database

Hi guys,
Thanks for this wonderful tool.
I was wondering if there is a way to specify the connection to a specific database. My question is because, in the example, the connection is made into the SQL warehouse and that connection will give me access to all databases, but what if I want to make a connection just with a database in a way if someone makes a query to a table that is not there will get some kind of error connection.
This is my code:
with sql.connect(
        server_hostname=settings.DATABRICKS_SERVER_HOSTNAME,
        http_path=settings.DATABRICKS_HTTP_PATH,
        access_token=settings.DATABRICKS_ACCESS_TOKEN,
    ) as conn:
        with conn.cursor() as cursor:
            cursor.execute(
                f"SELECT * FROM {settings.DATABRICKS_DATABASE_URL} LIMIT 5;"
            )

            for row in cursor.fetchall():
                print(row)

Thi is ok if the query string does not change, but if I want a complex query I will have to change the string in the cursor.execute and I will have access to any database

Thanks for your question. You can specify a catalog and schema when you build your connection like this:
with sql.connect(
        server_hostname=settings.DATABRICKS_SERVER_HOSTNAME,
        http_path=settings.DATABRICKS_HTTP_PATH,
        access_token=settings.DATABRICKS_ACCESS_TOKEN,
        catalog=settings.SOME_CATALOG,
        schema=settings.SOME_SCHEMA
    ) as conn:
        with conn.cursor() as cursor:
            cursor.execute(
                f"SELECT * FROM {settings.DATABRICKS_DATABASE_URL} LIMIT 5;"
            )

            for row in cursor.fetchall():
                print(row)

This is not a form of security, but just convenience. Anyone with access to this connection could run a USE <catalog> command or fully-qualified table name to see that data. For true access control you can use the data-access permissions features of Databricks itself.

Hi @susodapop thanks for your quick response. I will read about data-access permissions from Databricks.


write to S3 is very slow

Environment

OS: [Ubuntu 20.04]
Hardware (GPU, or instance type): [H800]

To reproduce
I got a 2G jsonl.gz text file, I tokenized it with data stored as numpy array.  Writer is defined as below:
   out = MDSWriter(
        columns={"input_ids": f"ndarray:int32:{args.seq_len}",
                 "token_type_ids": f"ndarray:int8:{args.seq_len}",
                 "attention_mask": f"ndarray:int8:{args.seq_len}",
                 "special_tokens_mask": f"ndarray:int8:{args.seq_len}"},
        out=out_path,
        compression=None
    )

tokenized data is pre-processed and loaded already, so time won't be wasted at tokenization.
def parse_data_2_mds_format(tokenized_dataset):
    input_ids = np.array(tokenized_dataset['input_ids']).astype(np.int32)
    token_type_ids = np.array(tokenized_dataset['token_type_ids']).astype(np.int8)
    attention_mask = np.array(tokenized_dataset['attention_mask']).astype(np.int8)
    special_tokens_mask = np.array(tokenized_dataset['special_tokens_mask']).astype(np.int8)
    return {'input_ids': input_ids, 'token_type_ids': token_type_ids,
            'attention_mask': attention_mask, 'special_tokens_mask': special_tokens_mask}

 with Pool(processes=args.mds_num_workers) as inner_pool:
                with tqdm(total=len(tokenized_datasets), desc="Writing Out MDS File") as pbar:
                    for result in inner_pool.imap(parse_data_2_mds_format, tokenized_datasets):
                        out.write(result)
                        pbar.update()

With code above, the writing time to S3 takes 30 minutes with mds_num_workers set as 200. If I set to 1, it
takes 1 hour to finish. It's just so slow, I have huge data to process. How to accelerate? Is that possible to write
a block of data once rather than one by one? Please give some suggestion to accelerate.
Expected behavior

Additional context

Hey @charliedream1, have you tried the parallel dataset conversion approach as detailed in our docs below? https://docs.mosaicml.com/projects/streaming/en/stable/preparing_datasets/parallel_dataset_conversion.html
Please let us know if that works for you.

I'm just using the parallel format You just mentioned. Please check my code. If i am using it wrongly or the speed is just as this kind of slow.
By the way, I try to copy files to the s3 With a command 30g Only takes less than five minutes. But currently I need to use half an hour with our lib To transmit samples one by one with parallel. I have a huge data, if speed is this kind of slow I cannot use this.
Please give some help.
---Original---
From: "Saaketh @.>
Date: Fri, Oct 25, 2024 21:27 PM
To: @.>;
Cc: "Optimus<EMAIL_ADDRESS>Subject: Re: [mosaicml/streaming] write to S3 is very slow (Issue #812)
Hey @charliedream1, have you tried the parallel dataset conversion approach as detailed in our docs below? https://docs.mosaicml.com/projects/streaming/en/stable/preparing_datasets/parallel_dataset_conversion.html
Please let us know if that works for you.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

I also try to output the data to the local disk but the speed is still the same. However, if i am using the dataset lib to save to the disk. It is only a few seconds.
---Original---
From: "Saaketh @.>
Date: Fri, Oct 25, 2024 21:27 PM
To: @.>;
Cc: "Optimus<EMAIL_ADDRESS>Subject: Re: [mosaicml/streaming] write to S3 is very slow (Issue #812)
Hey @charliedream1, have you tried the parallel dataset conversion approach as detailed in our docs below? https://docs.mosaicml.com/projects/streaming/en/stable/preparing_datasets/parallel_dataset_conversion.html
Please let us know if that works for you.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>


Cleanup YAML project code.

Summary
Few things in this PR:

Added update_if_exists flag to namespace and tag creation.
Made the above flag default to False everywhere. It used to be True on the node, which I think is not great default. cc @anhqle
Added system project namespace cleanup when run inside validation.

Test Plan
Unit tests and my local examples.

[ ] PR has an associated issue: #
[ ] make check passes
[ ] make test shows 100% unit test coverage

Deployment Plan
Asap.

Changes largely LGTM!
Only one comment:

Made the above flag default to False everywhere. It used to be True on the node, which I think is not great default.

@agorajek From what I've seen, a lot of people seem to rely on this flag being true... of course we can announce that this is a breaking change, but I'm wondering if that's just the default behavior that people expect or prefer?

@shangyian I am worried that the current setup (created_node --> update_if_exists=True) will lead to silent overrides. I think I could do the following:

keep this change, create_node( ..., update_if_exists: default False) and
add a new method to the client create_or_update_node( ..., update_if_exists: default True)
This should make it easy for users to pick the expected behavior easily. cc @anhqle

Thoughts?

I'm good with changing the default of update_if_exists to False.
In that case, it'd be nice to tell user about the update_if_exists functionality when they run into node-already-exists-error.

Yes, we have that already, example:
datajunction.exceptions.DJClientException: Node `default.account_type_table` already exists. Use `update_if_exists=True` to update the node.