Idea: match platform scrolling behavior with Flutter using HTML scrollable area

The own scrolling functionality on Flutter Web has already caused problems a few times. The biggest disadvantage of the current implementation is the fact that it just doesn't feel natural. The speed, the acceleration and the bouncing when you reach the end, for example, are for the most part very different from "real" web pages.
So yesterday in the shower, I got an idea on how to fix all these problems in one go, and I thought it might be good to have a Flutter developer take a closer look at this idea:
In the background of the Flutter web application there should be an empty and invisible real HTML element and when the user scrolls in the browser e.g. with the mouse wheel, one should scroll directly inside this element. Then you just read with Javascript using a listener on which position this HTML scrolling element is currently located and then pass this position to the Flutter Canvas Kit where this value is then passed e.g. to the position.pixels value of a ScrollController, which then actually scrolls the app.
The big advantage of this would be that the scrolling would feel exactly the same as its implementation in every browser. In this way one would not "reinvent the wheel", but simply use what already exists and what has already been successfully tested and used.
Thanks for reading. Maybe that would be something that might be realized.

Interesting Idea. In my opinion scrolling is defenetly the most annoying thing in flutter web at the moment.
My problem is just, that the scrolling is not smooth. On click rotation at the mouse whele moves the page instantly. Like in 2000. I think, how hard can it be to add a little animation with animate too...

@FelixMittermeier
Thanks for coming up this with idea / issue. There's an active open umbrella issue to discuss and fix scrollbar issues related to web and desktop and I think you should comment this idea there to get traction and for easy tracking at one place.
I am going ahead and closing this as duplicate in favor of umbrella issue. If you disagree, write in comments and I'll reopen to address it.

Well I do not think that this is related to the scrollbar itself. It is about the scrolling functionality in general so the linked issue does not fit in my opinion.

You are right. I probably mis-read the umbrella issue.

Related issues:
https://github.com/flutter/flutter/issues/56257
https://github.com/flutter/flutter/issues/32338
https://github.com/flutter/flutter/issues/77714
https://github.com/flutter/flutter/issues/75850
https://github.com/flutter/flutter/issues/59404

Thanks for the idea. It's an interesting idea. Currently the architecture of Flutter engine and framework prefers a different approach. However, this may have potential. If anything, this may be implementable using platform views as a package.

A similar idea https://github.com/flutter/flutter/issues/64501

A similar idea #64501
Maybe this is better as it adds scrolling fidelity as well as better performance, as stated in this article about scrolling being off the main thread on most browsers

https://blogs.windows.com/msedgedev/2017/03/08/scrolling-on-the-web/

Hey are their any updates on this?

We've implemented syncing of a HTML scroll area and a Flutter scroll area in an app. When scrolling a <div>, the ScrollController is updated to keep it in sync.
The code we have is for a very niche case, but I can confirm this is possible and does work pretty well. The scrolling feels a lot better and more "native".
I'll try to separate it out into a generic POC soon.

Okay, managed to turn this into a very early preview package.
Live example: https://native-scroll.web.app
Package: https://pub.dev/packages/native_scroll
Code: https://github.com/tomgilder/native_scroll
Only tested on macOS with Safari and Chrome. Will probably break under lots of conditions. Almost certainly not mobile-friendly yet.
But it at least works on HTML and CanvasKit renderers.

@tomgilder Wow, thank you so much for the great work.
I tried it out and it looks very promising. Seems like the performance is a bit worse but apart from that...
Maybe it would be more performant if one could implement it at a lower level of flutter.
I'm already looking forward to how this idea is developing 😄

Could this be implemented on other platforms? Desktop (or at least Linux when I tested it) has the same scroll behaviour as web, and Android/iOS scrolling still isn't perfect. It's much better than Web, but some people think that it's still slightly off.

Okay, managed to turn this into a very early preview package.
Live example: https://native-scroll.web.app Package: https://pub.dev/packages/native_scroll Code: https://github.com/tomgilder/native_scroll
Only tested on macOS with Safari and Chrome. Will probably break under lots of conditions. Almost certainly not mobile-friendly yet.
But it at least works on HTML and CanvasKit renderers.

Nice work, I'm on windows, so I needed to hide the ugly system scrollbar and add the flutter one:
Wrap your ListView or whatever ScrollView you are using with Scrollbar()
Add this to the header in web/index.html
<style>
  ::-webkit-scrollbar {
    display: none;
  }
</style>

But my drag to scroll stopped to work when I using your package, any thoughts on how to make it work again?

Flutter Native Scroll package does work for simple use cases, and it's code is very simple!
https://pub.dev/packages/native_scroll

native_scroll_web.dart
@JS()
library native_scroll;

import 'package:flutter/widgets.dart';
import 'package:js/js.dart';
import 'dart:html';
import 'dart:ui' as ui;

class NativeScrollBuilder extends StatefulWidget {
  final Widget Function(BuildContext context, ScrollController controller)
      builder;

  const NativeScrollBuilder({
    Key? key,
    required this.builder,
  }) : super(key: key);

  @override
  _NativeScrollBuilderState createState() => _NativeScrollBuilderState();
}

class _NativeScrollBuilderState extends State<NativeScrollBuilder> {
  static int _globalId = 0;

  late String _viewId;
  late ScrollController _scrollController;

  final _heightDiv = DivElement();

  @override
  void initState() {
    super.initState();
    _scrollController = ScrollController();
    _globalId++;
    _viewId = 'native-scroll-view-$_globalId';

    ui.platformViewRegistry.registerViewFactory(
      _viewId,
      (_) {
        // Create a scroll container - this is a <div> with scrolling overflow.
        // When it scroll, the Flutter ScrollController gets updated.
        return DivElement()
          ..id = _viewId
          ..style.overflow = 'scroll'
          ..style.width = '100%'
          ..style.height = '100%'
          // We need to cancel scroll events to stop them getting to Flutter
          ..onWheel.listen((event) => event.stopPropagation())
          ..onMouseWheel.listen((event) => event.stopPropagation())
          ..onScroll.listen((event) {
            final target = event.target as DivElement;
            _onNativeScroll(target.scrollTop);
          })
          ..append(_heightDiv);
      },
    );

    _scrollController.addListener(_onFlutterScroll);

    WidgetsBinding.instance!.addPostFrameCallback((timeStamp) async {
      _setScrollHeight();

      _scrollController.position.addListener(() {
        _setScrollHeight();
      });
    });
  }

  void _onFlutterScroll() {
    // There was a scroll in Flutter (e.g. scrollController.jumpTo called)
    // We need to update our shadow HTML scrolling elements
    _setScrollHeight();

    _heightDiv.scrollTop = _scrollController.position.pixels.toInt();
  }

  void _onNativeScroll(int scrollTop) {
    // There was a scroll in HTML land, update Flutter
    _scrollController.jumpTo(scrollTop.toDouble());
  }

  double? _lastScrollHeight;

  void _setScrollHeight() {
    // This updates a child <div> to have the same height
    // as the scroll contents from Flutter

    final scrollContentsHeight = _scrollController.position.viewportDimension +
        _scrollController.position.maxScrollExtent;

    if (scrollContentsHeight != _lastScrollHeight) {
      _lastScrollHeight = scrollContentsHeight;
      _heightDiv.style.height = '${scrollContentsHeight}px';
    }
  }

  @override
  void dispose() {
    _scrollController.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Stack(
      children: [
        ScrollConfiguration(
          behavior: ScrollConfiguration.of(context).copyWith(scrollbars: false),
          child: widget.builder(context, _scrollController),
        ),
        HtmlElementView(viewType: _viewId),
      ],
    );
  }
}

native_scroll_vm.dart
import 'package:flutter/widgets.dart';

class NativeScrollBuilder extends StatefulWidget {
  final Widget Function(BuildContext context, ScrollController controller)
      builder;

  const NativeScrollBuilder({
    Key? key,
    required this.builder,
  }) : super(key: key);

  @override
  _NativeScrollBuilderState createState() => _NativeScrollBuilderState();
}

class _NativeScrollBuilderState extends State<NativeScrollBuilder> {
  late ScrollController _scrollController;

  @override
  void initState() {
    super.initState();
    _scrollController = ScrollController();
  }

  @override
  void dispose() {
    _scrollController.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return widget.builder(context, _scrollController);
  }
}

NativeScrollBuilder(
    builder: (BuildContext context, ScrollController controller) {
    return ListView.builder(
        controller: controller,
        itemCount: 100,
        itemBuilder: (context, _) {
            return Text('I scroll nicely on the web');
        },
    ),
);

So what is solution?
If flutter has problems with Instagram browser, Telegram browser and etc.
Maybe problem not in browsers? Problem is on flutter.
And HTML works fine. So scroll problem it's just in core flutter scrolling mechanism?

Hey @NelepovDmitry,
did you manage to fix it?
Also using a Flutter webapp inside of Telegram webapp, but the scrolling just keeps closing my webapp... -.-
Would appreciate some tips :)

my workaround with same problem:
css:
body {
  height: 10000vh !important;
  position: static !important;
  overscroll-behavior: none;
}
flt-glass-pane {
  position: fixed !important;
  max-width: 100vw !important;
  max-height: 100vh !important;
}

js:
window.scroll(0, window.document.body.scrollHeight);

Greetings @ibsbk
I found the compiled code in your repositories and it works fine
That's incredible!)
But unfortunately your hack doesn't work properly on my hello world code((

Good: telegram window does not minimize when scrolling down.
Bad: any interaction with app elements doesn't work

Could you please advise what I'm doing wrong or give me the minimal code
Thanks

main.dart
void main() {
runApp(MyApp());
}

class MyApp extends StatelessWidget {
const MyApp({super.key});

@override
Widget build(BuildContext context) {
  return MaterialApp(
    title: 'FlutterWebApp',
    themeMode: ThemeMode.dark,
    darkTheme: ThemeData.dark(),
    home: Scaffold(
      body: ListView(
        children: [
          ...List.generate(
            30,
            (index) => ListTile(
              title: Text('ListTile #$index'),
            ),
          )
        ],
      ),
    ),
  );
}
}

index.html
<!-- ... -->
<script src="https://telegram.org/js/telegram-web-app.js" defer=""></script>
<script src="js.js" defer></script>

<!--hahahahah-->
<style>
  body {
    height: 10000vh !important;
    position: static !important;
    overscroll-behavior: none;
  }

  flt-glass-pane {
    position: fixed !important;
    max-width: 100vw !important;
    max-height: 100vh !important;

  }
</style>
<!-- ... -->

js.js
function resize_frame(height){
  console.log('height is ', height)
  document.body.style.height = `${height}px`
  window.Telegram.WebView.postEvent('resize_frame', false, {height: height});
}

window.scroll(0, window.document.body.scrollHeight);

Hi @WBgLyErCMc
I had same problem after update on flutter 3.16 (or 3.13). In my case it was enough replace element name in css
flutter-view instead flt-glass-pane

So what is solution?
If flutter has problems with Instagram browser, Telegram browser and etc.
Maybe problem not in browsers? Problem is on flutter.
And HTML works fine. So scroll problem it's just in core flutter scrolling mechanism?

my workaround with same problem:
css:
body {
  height: 10000vh !important;
  position: static !important;
  overscroll-behavior: none;
}
flt-glass-pane {
  position: fixed !important;
  max-width: 100vw !important;
  max-height: 100vh !important;
}

js:
window.scroll(0, window.document.body.scrollHeight);

This method solved my issue, but on android telegram it looks nice, but on ios telegram it looks ugly, all texts just moved down

So what is solution?
If flutter has problems with Instagram browser, Telegram browser and etc.
Maybe problem not in browsers? Problem is on flutter.
And HTML works fine. So scroll problem it's just in core flutter scrolling mechanism?

my workaround with same problem:
css:
body {
  height: 10000vh !important;
  position: static !important;
  overscroll-behavior: none;
}
flt-glass-pane {
  position: fixed !important;
  max-width: 100vw !important;
  max-height: 100vh !important;
}

js:
window.scroll(0, window.document.body.scrollHeight);

This method solved my issue, but on android telegram it looks nice, but on ios telegram it looks ugly, all texts just moved down

I just changed height: 10000vh to 1vh

so just use this code to be happy

-- The solutions is--
Add this code to index.html "head" tag:
<style>
    body {
      height: 10000vh !important;
      position: static !important;
      overscroll-behavior: none;
    }

    flutter-view {
      position: fixed !important;
      max-width: 100vw !important;
      max-height: 100vh !important;
    }

    /* Safari-specific CSS */
    @supports (-webkit-touch-callout: inherit) {
      body {
        height: 1vh !important;
      }
    }
  </style>

Add this code to "body" tag:
<script>
    window.scroll(0, window.document.body.scrollHeight);
  </script>

@elyum05 this is not working in iOS

my workaround with same problem:
css:
body {
  height: 10000vh !important;
  position: static !important;
  overscroll-behavior: none;
}
flt-glass-pane {
  position: fixed !important;
  max-width: 100vw !important;
  max-height: 100vh !important;
}

js:
window.scroll(0, window.document.body.scrollHeight);

Greetings @ibsbk
I found the compiled code in your repositories and it works fine That's incredible!)
But unfortunately your hack doesn't work properly on my hello world code((

Good: telegram window does not minimize when scrolling down.
Bad: any interaction with app elements doesn't work

Could you please advise what I'm doing wrong or give me the minimal code Thanks
main.dart
index.html
js.js

When I do this, the web becomes unclickable

@bugrevealingbme you need to replace flt-glass-pane with flutter-view

@bugrevealingbme you need to replace flt-glass-pane with flutter-view

This time the code doesn't work

This worked for me #77883 (comment). But you need to change to flutter-view

@khamidjon-khamidov Hi there, this has also worked for me but only on IOS.
on Android, the app still closed when swiping down. did you have the same result?
I used your code posted in Stackoverflow

@farshid-shokrolahi Did you have any scrollable view (ListView, SingeChildScrollView, etc.) on main page?

@farshid-shokrolahi Did you have any scrollable view (ListView, SingeChildScrollView, etc.) on main page? The solution I posted works both for android and ios (at least on my android device Samsung S24+)

Not in the main page but in the pages that has scroll view or bottom sheet, when swiping down, the app closes. is there any solution to prevent the whole app to not close? not in the page that has scrollable widget.

I developed a Flutter web application. When the link is opened in LINE, it uses LINE's built-in browser. However, there is a behavior where swiping down closes the LINE browser, which prevents my web application from scrolling properly with gestures. This issue is specific to the LINE default browser on iOS with Flutter web.

 document.addEventListener("touchmove", function(event) {
    event.preventDefault();
}, {
    passive: false
});

This code can be added to the index.html file of a Flutter web project to resolve scrolling conflicts within Telegram's in-app browser. It works by listening to the touchmove event and calling event.preventDefault() to block the default scrolling behavior when a user swipes on the screen. The passive: false option ensures that the browser does not treat the event listener as passive, allowing the prevention of unwanted scrolling interactions. This approach helps smoothen the user experience inside Telegram's browser.

I developed a Flutter web application. When the link is opened in LINE, it uses LINE's built-in browser. However, there is a behavior where swiping down closes the LINE browser, which prevents my web application from scrolling properly with gestures. This issue is specific to the LINE default browser on iOS with Flutter web.


Switch to FileUri.fsPath

What it does
As we've updated Theia, we can now rely on FileUri.fsPath and URI to compute file paths.
This should also avoid OS-specific issues across Linux, Windows and Mac (e.g. #49).
Fixes https://github.com/eclipse-cdt-cloud/cdt-cloud-blueprint/issues/49
Contributed on behalf of STMicroelectronics.
How to test

Open CDT Cloud Blueprint and open an empty workspace folder.
Generate an example from the welcome page on Linux and Windows.
Check that the example has been generated in to the workspace folder.

@vinodappu Could you please test this PR and verify that the generation of examples also works for you with this change? Thank you!

@vinodappu confirmed it works on Windows too! See https://github.com/eclipse-cdt-cloud/cdt-cloud/discussions/29#discussioncomment-4923298


-Estudos do dia 10/12/2021

[x] Understanding Case Sensitivity in Variables
[x] Explore Differences Between the var and let Keywords
[x] Declare a Read-Only Variable with the const Keyword
[x] Add Two Numbers with JavaScript
[x] Subtract One Number From Another with JavaScript
[x] Multiply Two Numbers with JavaScript
[x] Divide One Number by Another with JavaScript
[x] Increment a Number with JavaScript
[x] Decrement a Number with JavaScript
[x] Create Decimal Numbers with JavaScript

Todos os exercícios do dia foram concluídos. Revisar as anotações em prosseguir para o próximo!


[BUG] Huge difference in performance using picamera2 vs libcamera-vid on RPi 5

Hello,
I am trying to achieve 1080p@60 FPS (*) recording on my Raspberry Pi 5 using picamera2 but I am having issues.
(* even if the camera is supposed to only support 1080p@50FPS, it seems to still be able to do 1080@60FPS)
When using libcamera-vid I do:
libcamera-vid --level 4.2 --framerate 60 --width 1920 --height 1080 -o test-libcamera.mp4 -t 60s --denoise cdn_off -n
And then I count the number of frames using ffprobe:
ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 test-libcamera.mp4
I get 3527 frames, which is quite close to the 3600 frames I am supposed to be having at 60FPS (60 frames per seconds * 60 seconds), I have only a 2% loss of frames which is quite decent.
However, when using picamera2 I do:
#!/usr/bin/python3
import time

from libcamera import controls

from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
from picamera2.outputs import FileOutput

picam2 = Picamera2()
video_config = picam2.create_video_configuration(main={"size": (1920, 1080)}, controls={'FrameRate': 60})
picam2.configure(video_config)
picam2.set_controls({"NoiseReductionMode": controls.draft.NoiseReductionModeEnum.Off})

encoder = H264Encoder()
output = FileOutput('test-picamera2.h264')

picam2.start_recording(encoder, output)
time.sleep(60)
picam2.stop_recording()

And when I count the number of frames using ffprobe:
ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 test-picamera2.h264
I only get 2345 which is way further from the 3600 frames I am supposed to get, this is a loss of 35% of frames !
I am very surprised by these results. Is this behavior expected ?
Many thanks in advance for your help !

You don 't mention which camera module is being used, but the most likely reason is the modes that libcamera-vid and picamera2 have chosen to run the camera in.
libcamera-vid has most likely chosen a fast but cropped mode to try and achieve the framerate you've requested, while picamera2 has chosen a higher resolution but slower mode to attempt to match the requested 1920x1080 format.
A check of the INFO output message for the 'Selected sensor format' and 'Selected CFE format' each method has chosen may give a clue, or a check on the width of field of view of both recorded videos may also indicate the differing choices.
'The Sensor Configuration' guide in Section <IP_ADDRESS> of the Picamera2 manual gives some guidance on methods to 'force' picamera2 to select a specific mode. https://datasheets.raspberrypi.com/camera/picamera2-manual.pdf
And https://www.raspberrypi.com/documentation/computers/camera_software.html#list-cameras indicates how to list the available modes.

Hi @sandyol55 and thank you for your answer.
I am using the camera module v3, and indeed, it would seem that the format selected is different:
libcamera-vid:
[24:35:02.967150978] [77300]  INFO Camera camera.cpp:1197 configuring streams: (0) 1920x1080-YUV420 (1) 1536x864-BGGR_PISP_COMP1
    [24:35:02.967248700] [77305]  INFO RPI pisp.cpp:1450 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B

picamera2:
[24:36:29.561050846] [77422]  INFO Camera camera.cpp:1197 configuring streams: (0) 1920x1080-XBGR8888 (1) 2304x1296-BGGR_PISP_COMP1
[24:36:29.561170217] [77425]  INFO RPI pisp.cpp:1450 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a - Selected sensor format: 2304x1296-SBGGR10_1X10 - Selected CFE format: 2304x1296-PC1B

Please excuse my ignorance, but I do not understand what is "CFE", is it the sensor "resolution" ?
If so, I am not sure I understand how I can achieve a 1920x1080 footage with a CFE format that is "less" that the requested dimensions ?
Would you be enable to enlighten me ? I thank you in advance for your help !

The V3 camera module has 3 operating modes.
Modes: 'SRGGB10_CSI2P' : 1536x864 [120.13 fps - (768, 432)/3072x1728 crop]
                         2304x1296 [56.03 fps - (0, 0)/4608x2592 crop]
                         4608x2592 [14.35 fps - (0, 0)/4608x2592 crop]

By changing your libcamera-vid command you have now allowed both methods to choose mode 1, which has a maximum possible framerate of 56fps.  libcamera is getting closer than picamera2 to the maximum 56fps but neither can achieve 60 fps in that mode.
To get picamera2 to choose mode 0, (with its faster maximum rate),  you can add a sensor definition in the configuration, something like this:-
#!/usr/bin/python3
import time

from libcamera import controls

from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
from picamera2.outputs import FileOutput

picam2 = Picamera2()

# Fastest framerate mode (0  for V3 Camera)
mode=picam2.sensor_modes[0]
# Pick out key parameters from mode
sensor={'output_size': mode['size'],'bit_depth':mode['bit_depth']}
# And apply them to the configuration
video_config = picam2.create_video_configuration(main={"size": (1920, 1080),"format":"YUV420"},
                                                 sensor=sensor,
                                                 controls={'FrameRate':60,
                                                           "NoiseReductionMode" : controls.draft.NoiseReductionModeEnum.Off})

picam2.configure(video_config)

encoder = H264Encoder()
output = FileOutput('test-picamera2.h264')

picam2.start_recording(encoder, output)
time.sleep(60)
picam2.stop_recording()
picam2.close()

(This code snippet also changes to the YUV420 format as that is likely to more computationally efficient than the default format)
With these changes picamera2 should now get closer to 60 fps.
Although mode 0 only has a resolution of 1536x864 , both libcamera and picamera2 will be upscaling that to the requested 1920x1080, before passing it to be encoded.

@sandyol55 thank you so much, the whole sensor stuff it is still not 100% clear for me but at least I get things working just fine by specifying the sensor mode.
One last thing, if I'm ok reducing the FPS a little bit, do you agree that selecting mode 1 will provide a better image quality since there won't be any upscaling done ?
[{'bit_depth': 10,
  'crop_limits': (768, 432, 3072, 1728),
  'exposure_limits': (9, 77208384, None),
  'format': SRGGB10_CSI2P,
  'fps': 120.13,
  'size': (1536, 864),
  'unpacked': 'SRGGB10'},
 {'bit_depth': 10,
  'crop_limits': (0, 0, 4608, 2592),
  'exposure_limits': (13, 112015443, None),
  'format': SRGGB10_CSI2P,
  'fps': 56.03,
  'size': (2304, 1296),
  'unpacked': 'SRGGB10'},
 {'bit_depth': 10,
  'crop_limits': (0, 0, 4608, 2592),
  'exposure_limits': (26, 220417486, None),
  'format': SRGGB10_CSI2P,
  'fps': 14.35,
  'size': (4608, 2592),
  'unpacked': 'SRGGB10'}]

Yes, assuming the ~56 fps suits your requirement, then I'd expect that  the downscaling from mode 1, instead of the upscaling from mode 0, would be perceptibly higher in quality - maybe not by much though?? .

Closing as this is resolved thanks to @sandyol55


WindowsError: [Error 216] This version of %1 is not compatible

Hi,
I was trying to build libchromiumcontent on my windows 7 32-bit Enterprise Edition. I have python 2.7 and Visual Studio 2013 Express installed.
When I tried update script, i got below error, please kindly help, thank you!

Extracting...
Traceback (most recent call last):
  File "script/update", line 213, in <module>
    sys.exit(main())
  File "script/update", line 32, in main
    download_source_tarball(version)
  File "script/update", line 86, in download_source_tarball
    tar_xf(xz_name)
  File "script/update", line 96, in tar_xf
    subprocess.call([xz(), '-dqk', filename])
  File "C:\Python27\lib\subprocess.py", line 522, in call
    return Popen(*popenargs, **kwargs).wait()
  File "C:\Python27\lib\subprocess.py", line 710, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 958, in _execute_child
    startupinfo)
WindowsError: [Error 216] This version of %1 is not compatible with the version
of Windows you're running. Check your computer's system information to see wheth
er you need a x86 (32-bit) or x64 (64-bit) version of the program, and then cont
act the software publisher

libchromiumcontent can only be built on 64bit Windows, even if you are building for 32bit target.


RTMDET How to get "Grided concatenated outputs" without NMS

Hi, I am testing and trying to export RtmDet with an output containing the detections, rescaled bounding boxes and label scores as yolo models would which would provide a final tensor of dimension [batch_size, detection_dimension, detection_size]. Where detection_dimension depends on the input size and detection_size would thus correspond to the concatenation of {bounding_boxes, score, label scores} for instance {4, 1, 80} in the case of a coco trained model.
The model has to be TensorRT friendly, thus operation such as NonZero can not be used.
I have tried exports as per defined in the doc, using mmdeploy but BatchedNMS and operation afterwards are not compliant with my requirements.
I also have tried to directly modify base_dense_head.py which is being used by the model unfortunately, it is batch dependent due to this loop, shall I still create my own head or is there anyone who knows how to do it seamlessly ?

Adding code below to deploy.py, I could get the onnx model that has the networks prior to BatchedNMS. Also, by using trtexec, I could get TensorRT model from this onnx model.
After using this technique, you should be able to implement the post-processing yourself and get the model you want.
I don't know if this method will help you.
partition_config = dict(
    type='tensorrt',
    apply_marks=True,
    partition_cfg=[
        dict(
            save_file='model.onnx',
            start=['detector_forward:input'],  # [mark_name:input, ...]
            end=['multiclass_nms:input'],  # [mark_name:output, ...]
            output_names=['boxes', 'scores']) # output names
    ])


[Feature Request] DLL Chainloading

Would it be possible to implement DLL chainloading into DXVK, to make it easier to load D3D wrappers like ENB, Reshade etc.? I'm not sure if this would be of much use on Linux, but on Windows it would allow these to be used without using DXVK system-wide, which is currently (as far as I am aware) the only way to do this. I am aware wrappers like these have no official support, but it would be nice to at least have the ability to load them.

No, because these D3D wrappers don't get loaded by D3D11 on Windows, they load D3D11 themselves, so they'd have to do the same thing with DXVK somehow. It's just not our responsibility to load random third-party libraries.
Both ENB and Reshade work on Linux, so there's that.


Raise IncorrectAlgorithm if token has no alg header

Closes https://github.com/jwt/ruby-jwt/issues/410

Thanks for the fix.
As long as the alg header is required by the validation i think this is a great change. In general I think the alg header should be of almost no relevance and maybe we should make it less important when validating the signature.


Cleanup Github Actions and fix fprettify

This PR fixes a number of issues:

fprettify will now ignore the entire file src/cuda/exec_dist.f90 to avoid it messing with the CUDA chevrons
Github Actions no longer reports fprettify failing as a true failure; instead it posts a warning, however there's now no obvious indication that a run has failed.
Updated various github actions to latest versions and pinned their Python version.
Removed check for Conventional Commits.

Note: the fprettify chevron bugfix has been submitted so the above hack to just remove the file entirely will be no longer needed once the corresponding PR has been merged and fprettify updated.

Looks good to me. Though I'd also remove mentions of the "conventional commit" thing in the documentation. I think it is mainly here: https://github.com/xcompact3d/x3d2/blob/main/docs/source/developer/contributing_guidelines.rst


Expose sql_type for columns in resultset #167

@jstastny-cz This looks clean, but I'm wary about adding something to the interface without knowing what someone would do with it.  I'm not sure how one would make use of this information.  What was the use case for this?  Thanks.

@mkleehammer Hello, we used pyodbc for ODBC interface/driver tests. What we needed to do was to check the actual SQL data type returned by the driver.

If I understand the PR correctly, then we have the same use case. I am using pyodbc to generate SQL Server create tables DDL based on the result set. If the actual SQL type is returned, we can create exact DDL. Right now we are creating approximate DDL.

Yes please! We have a use case that requires us to know the actual SQL type behind the column. We have been unable to come up with a clean workaround.


bug: ionic cli add does not find bower packages

Type: bug
Platform: desktop browser
Components that a 'bower search' can find sometimes result in the following error message after executing 'ionic add'
Failed to find the bower component [...]
Are you sure it exists? (CLI v1.6.1)

Try, for example, well-known and useful packages angularfire and firebase: they appear in bower searches, but result in the error above.

This issue was moved to driftyco/ionic-cli#530


Fixed an error thrown when selecting thread_pools['name'].

String was used instead of BytesRef.
Since integration tests don't catch this error, due to transformation happening to faciliate easy assertions, added an assertion on the abstract class that handles those expressions to catch such errors in the future and without adding and perf overhead for normal runtime.

also why is only thread_pools affected here, whats about all the other objects returning a string like sys.nodes.fs ?

related to @chaudum comment, maybe adding a testing matcher helper asserting for a bytesref is a better solution?

@matriv found another occurence in https://github.com/crate/crate/blob/master/sql/src/main/java/io/crate/operation/reference/sys/node/NodeThreadPoolsExpression.java#L57

@seut in the FS expressions it is correctly returned https://github.com/crate/crate/blob/master/sql/src/main/java/io/crate/operation/reference/sys/node/fs/NodeStatsFsDisksExpression.java#L49

@chaudum @seut Plz check again. I have added a check in test layer to avoid Strings.
By disabling JDBC I've run all tests and caught another error (wait_for_active_shards).
I wouldn't remove the assertion since it only applies on tests and the fix in test layer only catches errors when run with JDBC.
Another commit is changing the type of ThreadPools to BytesRef (won't be backported to not break streaming compatibility)
As @mfussenegger we should add (maybe in blackbox tests?) an exhaustive test that gets all columns from information_schema.columns and selects them with HTTP protocol to avoid such bugs in the future.

Codecov Report

Merging #6676 into master will decrease coverage by 0.11%.
The diff coverage is 57.14%.

@@             Coverage Diff              @@
##             master    #6676      +/-   ##
============================================
- Coverage     83.02%   82.91%   -0.12%
+ Complexity    13375    13369       -6
============================================
  Files          1527     1527
  Lines         49697    49696       -1
  Branches       5956     5957       +1
============================================
- Hits          41259    41203      -56
- Misses         5603     5656      +53
- Partials       2835     2837       +2

Impacted Files
Coverage Δ
Complexity Δ

.../java/io/crate/metadata/sys/SysNodesTableInfo.java
93.01% <100%> (ø)
29 <1> (ø)
:arrow_down:

...ference/sys/node/NodeStatsArrayTypeExpression.java
33.33% <40%> (-16.67%)
2 <1> (ø)

...ration/aggregation/statistics/moment/Variance.java
66.66% <0%> (-33.34%)
5% <0%> (-2%)

...ion/aggregation/impl/GeometricMeanAggregation.java
59.09% <0%> (-21.22%)
11% <0%> (ø)

...main/java/io/crate/breaker/ArraySizeEstimator.java
80% <0%> (-20%)
4% <0%> (-1%)

...a/io/crate/exceptions/ContextMissingException.java
81.81% <0%> (-18.19%)
1% <0%> (-1%)

...o/crate/analyze/SafeExpressionToStringVisitor.java
83.33% <0%> (-16.67%)
7% <0%> (-1%)

...peration/aggregation/impl/VarianceAggregation.java
64.7% <0%> (-15.69%)
11% <0%> (ø)

...aggregation/impl/StandardDeviationAggregation.java
64.7% <0%> (-15.69%)
11% <0%> (ø)

... and 37 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update df1d391...88ef8af. Read the comment docs.

lgtm

@matriv think you missed testSysNodesCgroup, see travis failure

@seut yep thx! that's because this part that fails is only for linux.


chore: fix typos

fix a typo: propogate->propagate

To discourage airdrop farmers from spamming PRs, we do not accept simple typo fixes like this.


hides the toolbar for pages out of django-cms

This fixes an issue in which the toolbar appears even if outside the django CMS paths

Coverage decreased (-0.01%) to 90.77% when pulling 860dbe5995e646a947f7be5e63506f3f46816aa5 on pa0lin082:fix_toolbar_hide into e251624c0bf7e68074d065e1fe2d96ea38f68658 on divio:develop.

Coverage decreased (-0.01%) to 90.77% when pulling 860dbe5995e646a947f7be5e63506f3f46816aa5 on pa0lin082:fix_toolbar_hide into e251624c0bf7e68074d065e1fe2d96ea38f68658 on divio:develop.

Coverage decreased (-0.01%) to 90.77% when pulling 860dbe5995e646a947f7be5e63506f3f46816aa5 on pa0lin082:fix_toolbar_hide into e251624c0bf7e68074d065e1fe2d96ea38f68658 on divio:develop.

Coverage decreased (-0.01%) to 90.77% when pulling 860dbe5995e646a947f7be5e63506f3f46816aa5 on pa0lin082:fix_toolbar_hide into e251624c0bf7e68074d065e1fe2d96ea38f68658 on divio:develop.

Coverage decreased (-0.01%) to 90.77% when pulling 860dbe5995e646a947f7be5e63506f3f46816aa5 on pa0lin082:fix_toolbar_hide into e251624c0bf7e68074d065e1fe2d96ea38f68658 on divio:develop.

Coverage decreased (-0.01%) to 90.77% when pulling 860dbe5995e646a947f7be5e63506f3f46816aa5 on pa0lin082:fix_toolbar_hide into e251624c0bf7e68074d065e1fe2d96ea38f68658 on divio:develop.

Coverage decreased (-0.01%) to 90.77% when pulling 860dbe5995e646a947f7be5e63506f3f46816aa5 on pa0lin082:fix_toolbar_hide into e251624c0bf7e68074d065e1fe2d96ea38f68658 on divio:develop.

@mkoistinen @evildmp what's your opinion on this solution? The bug is legitimate

Thanks @pa0lin082 !


local dev plugin localization failed

lazy.nvim supports dev configuration, which allows specifying a directory to put local plugins inside; for the specific plugin, lazy.nvim will look up in the specified directory for such a plugin, if the plugin is configured with dev = true.
When I set a local plugin, telescope-lazy.nvim successfully displays the plugin name in telescope's picker list, but it seems to fail to find the correct location of the plugin. For example, when I press <CR> it emits an error message Could not perform action. Readme file doesn't exist.; <C-F> and others don't work either.

Thanks for the issue.
I've added support for dev mode plugins in 1b98d08.


Add description of params/N/enum key

@dgelessus @GreyCat @jchv  I would appreciate a review of the params/enum key description I added, is it clear enough?

@dgelessus @GreyCat Thanks for the review, merging in.


Fix #39509: Invalid computations on ARMv7 running under NodeJS

Please make against master. We don't accept PRs against release branches (r$x.$y) after the final release ended, except if we are doing a patch release. We do patch releases mostly for security issues and only taking in stuff that is present on master branch.
Once this is in master, we can evaluate if it is worth creating a cherry-pick against r2.2. TF2.2 is not a long term support release so if we don't need to issue a patch release for it we won't merge anything else on r2.2.
Is the bug you're seeing also affecting 1.15 and 2.1 (the LTS versions)? How about 2.0?

Well r2.2 is the first one where our model can trigger, so I can't ensure the issue is present on 2.1 or 2.0. Looks like the fix was accepted on ruy side, so what would be the best way to fix on master? update tf master with ruy master or just add the fix as a patch applied in tensorflow?

Looks like it will be taken care of by guy's mainteneur : https://github.com/tensorflow/tensorflow/issues/39509#issuecomment-635434843, so I'm closing this PR.

I'm not sure what would be the best fix as I'm not an owner of tflie/ruy code.

FYI: the TF->Ruy repository reference update bringing this fix in, had been through 3 successive rollbacks, but now it looks like it's finally sticking:
https://github.com/tensorflow/tensorflow/commit/c2548337173ca6d4c1949225bc3dee2dc3f1a7dc#diff-2c39bc19b761ac36dc046245d1d47fe6


Clear stack navigation breaks in 1.8.34

It seems like 1.8.34 created a regression.  The following function has been used to clear the stack on navigation:
fun DestinationsNavigator.navigateOntoEmptyBackstack(direction: Direction, graph: NavGraph) {
    navigate(direction) {
        launchSingleTop = true
        popUpTo(graph.route ?: graph.startDestinationRoute ?: graph.findStartDestination().navigatorName) {
            inclusive = true
        }
    }
}

This seemed to work fine up until 1.8.34 when the following error started occurring:
java.util.NoSuchElementException: ArrayDeque is empty.
    at kotlin.collections.ArrayDeque.removeLast(ArrayDeque.kt:163)
    at androidx.navigation.NavController.launchSingleTopInternal(NavController.kt:1849)
    at androidx.navigation.NavController.navigate(NavController.kt:1810)
    at androidx.navigation.NavController.navigate(NavController.kt:1773)
    at androidx.navigation.NavController.navigate(NavController.kt:2173)
    at androidx.navigation.NavController.navigate$default(NavController.kt:2168)
    at androidx.navigation.NavController.navigate(NavController.kt:2153)
    at com.example.ScreenKt.navigateOntoEmptyBackstack(Screen.kt:150)

Just to be sure, try 1.8.35-beta but I'm guessing it doesn't have anything to do with it. If there is a regression here, it's very likely from the version updates that were done to the official compose navigation dependency between these two versions.
But why exactly do you want to do this? just to clear the backstack? Do this instead:

navigator.navigate(YourDirection) {
     popUpTo(NavGraphs.root) {
        inclusive = true
    }
}

https://issuetracker.google.com/issues/195342722
Reportedly fixed in Navigation 2.4.0-alpha06

Alright, a further update.
The use of NavGraphs.root unfortunately does not clear the stack when switching graphs from my testing.  Maybe I am doing something wrong but the construct I was using continues to work.
However, removing the launchSingleTop when clearing the stack like you also suggested does avoid the crash even with the latest library.  I will go with this.  Thanks for your help.
Closing this issue as resolved.

I had updated to 1.8.35-1.8.36. still had this issue.  Back to 1.8.33-beta, the app is safe for now

Again guys, this most certainly is not something this library is doing. It is very likely some change in the official compose navigation library that broke something for you, I’m guessing a misuse of the APIs that (since it wasn’t an expected usage of it) wasn’t considered when some change happened.
please make sure you’re using appropriate APIs and using them well. I’m pretty sure that if you are, then it will be working correctly.
@RMF09 were you also using launchSingleTop?

Again guys, this most certainly is not something this library is doing. It is very likely some change in the official compose navigation library that broke something for you, I’m guessing a misuse of the APIs that (since it wasn’t an expected usage of it) wasn’t considered when some change happened.
please make sure you’re using appropriate APIs and using them well. I’m pretty sure that if you are, then it will be working correctly.
@RMF09 were you also using launchSingleTop?

Yea.. I am using that

Are you sure you need it?

Or that it makes sense to use it?


Limit local cache size (cache eviction)

Discussed in https://github.com/vercel/turbo/discussions/1264

Originally posted by maxburs May 18, 2022
I'm caching the /node_modules/.turbo folder in CI. This is causing problems because of its size. I've seen it get as large at 4.5gb. Is there a good way to limit the size of the cache?
TURBO-2071

We just spoke on this amongst the team and we've settled on that this is something that we should do. We have it on our board and we will get it prioritized at some point in the future. We're not sure when at the moment but having controls around cache eviction is something we're interested in being a feature.

@anthonyshew An idea would be to track when you actually hit a cache. So you can present how many unused cache files you have at the moment.
turbo build

...

 Tasks:    A successful, B total
Cached:    C cached, D total, E unused for F amount of time to (turbo cache clean --unused)
  Time:    Gs >>> FULL TURBO

That's a good idea and we want a mapping of [code ref] -> [turbo hash] for other reasons also.


More granular interface for our use of jQuery

This catches bugs when we pass unexpected values to jQuery functions.
Cc @mikesall

lgtm


Error mapping Map<String, Integer> with JdbcTypeCode(JSON)

Error mapping a Map<String, Integer> stored as JSON using EntityViews

So my Entity has a Map defined like this:

    @JdbcTypeCode(SqlTypes.JSON)
    private Map<String, Integer> inventory;

I tried to return this field in a EV. I tried no Annotation, i tried "MappingSingular" and i tried "Basic"-Annotation.
With "MappingSingular" i get:
java: > oder ',' erwartet

(produced code looks like this):
    public AttributePath<T, Map<String java.lang.Integer>, Map<String java.lang.Integer>> inventory() {
        MethodSingularAttribute<CompanyAccessTokenPublicCompanyView, Map<String java.lang.Integer>> attribute = CompanyAccessTokenPublicViewCompanyAccessTokenPublicCompanyView_.inventory;
        return attribute == null ? getWrapped().<Map<String java.lang.Integer>>get("inventory") : getWrapped().get(attribute);
    }

Using no annotation or Basic-Annnotation results in a runtime error:
java.lang.IllegalArgumentException: There are error(s) in entity views!
The resolved possible types [java.util.Map] are not assignable to the given expression type 'java.util.Collection<java.lang.Integer>' of the mapping expression declared by the attribute inventory[de.alteravitarp.govnet.database.government.company.CompanyAccessTokenPublicView$CompanyAccessTokenPublicCompanyView.getInventory]!
	at com.blazebit.persistence.view.impl.EntityViewManagerImpl.<init>(EntityViewManagerImpl.java:285) ~[blaze-persistence-entity-view-impl-jakarta-1.6.10.jar:1.6.10]
	at com.blazebit.persistence.view.impl.EntityViewConfigurationImpl.createEntityViewManager(EntityViewConfigurationImpl.java:257) ~[blaze-persistence-entity-view-impl-jakarta-1.6.10.jar:1.6.10]
	at de.alteravitarp.govnet.config.blaze.BlazeConfiguration.createEntityViewManager(BlazeConfiguration.java:36) ~[classes/:na]
	at de.alteravitarp.govnet.config.blaze.BlazeConfiguration$$SpringCGLIB$$0.CGLIB$createEntityViewManager$1(<generated>) ~[classes/:na]
	at de.alteravitarp.govnet.config.blaze.BlazeConfiguration$$SpringCGLIB$$FastClass$$1.invoke(<generated>) ~[classes/:na]
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:258) ~[spring-core-6.0.13.jar:6.0.13]
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) ~[spring-context-6.0.13.jar:6.0.13]
	at de.alteravitarp.govnet.config.blaze.BlazeConfiguration$$SpringCGLIB$$0.createEntityViewManager(<generated>) ~[classes/:na]
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[na:na]
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:139) ~[spring-beans-6.0.13.jar:6.0.13]
	... 51 common frames omitted

Expected behavior
I expect the application to start up and run.
Steps to reproduce

Use Spring Boot 3.1.5 + Spring Boot JPA + Spring Boot WebMVC
Create a Entity with a Map and use "@JdbcTypeCode(SqlTypes.JSON)"
Create a EV returning the Map
Error

Environment

Version:            1.6.10
JPA-Provider:       Hibernate 6.2.13-FINAL
DBMS:               MariaDB
Application Server: Spring Boot 3.1.5

Hi and thanks for the report. The compilation error you are observing happens for the generated metamodel. As a workaround, you should be able to disable metamodel generation by e.g. excluding the blaze-persistence-entity-view-processor dependency. The runtime error is due to the fact that you did not annotate the entity view mapping with @MappingSingular, which is necesary: https://persistence.blazebit.com/documentation/1.6/entity-view/manual/en_US/index.html#singular-collection-type-mappings

Hi and thanks for the report. The compilation error you are observing happens for the generated metamodel. As a workaround, you should be able to disable metamodel generation by e.g. excluding the blaze-persistence-entity-view-processor dependency. The runtime error is due to the fact that you did not annotate the entity view mapping with @MappingSingular, which is necesary: https://persistence.blazebit.com/documentation/1.6/entity-view/manual/en_US/index.html#singular-collection-type-mappings

Hey, i stated above in my report, that i tried using @MappingSingular. We rescheduled the project involved with this issue so i dont have time to check excluding the dependency right now. :)

I encountered the same issue with:
@MappingSingular
    Map<String, Object> getInfo();

It will generate the wrong type: MethodSingularAttribute<PersonView, Map<StringObject>>, obviously missing a comma between String and Object in the generics of Map.

The version I am using is 1.6.11. Currently, the issue can be resolved by removing the blaze-persistence-entity-view-processor dependency, but then we lose the benefits of performance improvement in static typing brought by blaze-persistence-entity-view-processor.

It will generate the wrong type: MethodSingularAttribute<PersonView, Map>, obviously missing a comma between String and Object in the generics of Map.

Thanks for the details. That should be easy to fix then.

@beikov Thanks for your work, looking forward to it being fixed.


ohai plugin bad parsing with --param foo=bar in cc-opt

If configure flags as below are used
node.override["nginx"]["configure_flags"]  = [
  "--with-http_gunzip_module",
  "--with-threads",
  "--with-stream",
  "--with-stream_ssl_module",
  "--with-file-aio",
  "--with-openssl=/usr/src/ssl",
  "--with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2'",
  "--with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,--as-needed'"
]

Ohai plugin can't handle parsing properly, --with-cc-opt is split into two elements because of --params inside, there is no short call for --params, thus it breaks chef_nginx::source, because configure arguments are never equal to configure_flags and nginx is recompiled with EACH chef run.
Any idea what we can do with this? Whitelisting --params?

Could we potentially split on --with?

I think we can just change the regex to \s--(?!param) to avoid this particular issue. It won't prevent all the potential issues though.


[build-script] Allow combining '-o' with other options.

What's in this pull request?
Applied a22d8569 fix to -o and  --test-optimized.
Also, moved to run_tests_group group.
And, added a note about why we split these declarations.
CC: @gribozavr
Any reason you didn't applied this?

Before merging this pull request to apple/swift repository:

[ ] Test pull request on Swift continuous integration.

CC: @gribozavr
Any reason you didn't applied this?

I was doing a minimal change.  Your patch LGTM.

@swift-ci Please test and merge


Change Goban addStone capture obj

I should change the Goban addStone capture to return consistent capture object with loadStonesFromMovetree.

realistically, not going to do this.


OverflowError: Range exceeds valid bounds

My Convolutional neural net is giving me an error on the first DenseLayer, saying "OverflowError: Range exceeds valid bounds". My code looks correct given the other examples that I have consulted, but I'm not really sure.
IMAGE_HEIGHT = 6
IMAGE_WIDTH = 200
NUM_PEOPLE = 18

def gen_model():
    """
    Generates the model to be used
    :return: the model, untrained
    """
    model = Sequential()

    model.add(Convolution2D(5, 5, 5, input_shape=(1, IMAGE_HEIGHT, IMAGE_WIDTH)))
    model.add(AveragePooling2D(pool_size=(2, 2)))
    model.add(Convolution2D(5, 5, 5))
    model.add(AveragePooling2D(pool_size=(2, 2)))
    model.add(Flatten())
    model.add(Dense(output_dim=20))
    model.add(Activation('relu'))
    model.add(Dense(output_dim=18))
    model.add(Activation('softmax'))
    model.compile(optimizer="adagrad", loss="categorical_crossentropy", metrics=['accuracy'])
    return model

And here is the full traceback error:
Traceback (most recent call last):
File "/home/chris/Desktop/KerasCNN/model.py", line 63, in <module>
main()
File "/home/chris/Desktop/KerasCNN/model.py", line 52, in main
model = gen_model()
File "/home/chris/Desktop/KerasCNN/model.py", line 31, in gen_model
model.add(Dense(output_dim=20))
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 142, in add
output_tensor = layer(self.outputs[0])
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 458, in __call__
self.build(input_shapes[0])
File "/usr/local/lib/python2.7/dist-packages/keras/layers/core.py", line 596, in build
name='{}_W'.format(self.name))
File "/usr/local/lib/python2.7/dist-packages/keras/initializations.py", line 59, in glorot_uniform
return uniform(shape, s, name=name)
File "/usr/local/lib/python2.7/dist-packages/keras/initializations.py", line 30, in uniform
return K.variable(np.random.uniform(low=-scale, high=scale, size=shape),
File "mtrand.pyx", line 1565, in mtrand.RandomState.uniform
(numpy/random/mtrand/mtrand.c:16656)
OverflowError: Range exceeds valid bounds
UPDATE
Resolved issue by adding border_mode='same' to Convolutional layers. Now receiving error:
Exception: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 arrays but instead got the following list of 540 arrays
UPDATE 2
Resolved previous issue by doing train_concat = np.arry(train). Now receiving error:
ValueError: ('Bad input argument to theano function with name "/usr/local/lib/python3.5/dist-packages/keras/backend/theano_backend.py:514"  at index 0(0-based)', 'setting an array element with a sequence.')

Same problem here! I transfered my scripts to a different machine and the error popped up. On my old machine it is working flawlessly.
My code causing the problem:
model = Sequential()

model.add(Convolution2D(self.nb_conv_filters, self.conv_kernel_size, self.conv_kernel_size, border_mode='valid', input_shape=(1, self.img_rows, self.img_cols)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(self.pool_kernel_size, self.pool_kernel_size)))

model.add(Flatten())

model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dropout(0.5))

model.add(Dense(self.nb_classes))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
				  optimizer='adadelta',
				  metrics=['accuracy'])

It could be different software version. Can you compare them?.
On Thu, Sep 15, 2016 at 10:37 AM, jagiella<EMAIL_ADDRESS>wrote:

Same problem here! I transfered my scripts to a different machine and the
error popped up. On my old machine it is working flawlessly.
My code causing the problem:
model = Sequential()
model.add(Convolution2D(self.nb_conv_filters, self.conv_kernel_size, self.conv_kernel_size, border_mode='valid', input_shape=(1, self.img_rows, self.img_cols)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(self.pool_kernel_size, self.pool_kernel_size)))
model.add(Flatten())
model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(self.nb_classes))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='adadelta',
metrics=['accuracy'])
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/2681#issuecomment-247347006,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AALC-_nSdzBgWywIVZR85J0lj5HSz3Cnks5qqVgmgaJpZM4Iawx-
.

I updated my docker with the latest build just now and i get this too. I wasn't getting this before.
<ipython-input-6-95e4eaaa23f7> in train(BATCH_SIZE)
      3     X_train = (X_train.astype(np.float32) - 127.5)/127.5
      4     X_train = X_train.reshape((X_train.shape[0], 1) + X_train.shape[1:])
----> 5     discriminator = discriminator_model()
      6     generator = generator_model()
      7     discriminator_on_generator =         generator_containing_discriminator(generator, discriminator)

<ipython-input-3-57fe563bb1fd> in discriminator_model()
     11     model.add(MaxPooling2D(pool_size=(2, 2)))
     12     model.add(Flatten())
---> 13     model.add(Dense(1024))
     14     model.add(Activation('tanh'))
     15     model.add(Dense(1))

/opt/conda/lib/python2.7/site-packages/keras/models.pyc in add(self, layer)
    306                  output_shapes=[self.outputs[0]._keras_shape])
    307         else:
--> 308             output_tensor = layer(self.outputs[0])
    309             if type(output_tensor) is list:
    310                 raise Exception('All layers in a Sequential model '

/opt/conda/lib/python2.7/site-packages/keras/engine/topology.pyc in __call__(self, x, mask)
    485                                     '`layer.build(batch_input_shape)`')
    486             if len(input_shapes) == 1:
--> 487                 self.build(input_shapes[0])
    488             else:
    489                 self.build(input_shapes)

/opt/conda/lib/python2.7/site-packages/keras/layers/core.pyc in build(self, input_shape)
    693
    694         self.W = self.init((input_dim, self.output_dim),
--> 695                            name='{}_W'.format(self.name))
    696         if self.bias:
    697             self.b = K.zeros((self.output_dim,),

/opt/conda/lib/python2.7/site-packages/keras/initializations.pyc in glorot_uniform(shape, name, dim_ordering)
     57     fan_in, fan_out = get_fans(shape, dim_ordering=dim_ordering)
     58     s = np.sqrt(6. / (fan_in + fan_out))
---> 59     return uniform(shape, s, name=name)
     60
     61

/opt/conda/lib/python2.7/site-packages/keras/initializations.pyc in uniform(shape, scale, name)
     30
     31 def uniform(shape, scale=0.05, name=None):
---> 32     return K.random_uniform_variable(shape, -scale, scale, name=name)
     33
     34

/opt/conda/lib/python2.7/site-packages/keras/backend/theano_backend.pyc in random_uniform_variable(shape, low, high, dtype, name)
    138
    139 def random_uniform_variable(shape, low, high, dtype=_FLOATX, name=None):
--> 140     return variable(np.random.uniform(low=low, high=high, size=shape),
    141                     dtype=dtype, name=name)
    142

mtrand.pyx in mtrand.RandomState.uniform (numpy/random/mtrand/mtrand.c:13528)()

OverflowError: Range exceeds valid bounds

I have the same problem, but when I typed model.summary() it looked like I have negatief shapes

I could solve this by changing
input_shape=(1, IMAGE_HEIGHT, IMAGE_WIDTH)
To
input_shape=(IMAGE_HEIGHT, IMAGE_WIDTH,1)

I could solve the problem by adding the following two lines at the top of my program
from keras import backend as K
K.set_image_dim_ordering('th')

check your ~/.keras/keras.json
if "image_dim_ordering": is "th" and  "backend": "theano", your input_shape must be (channels, height, width)
if "image_dim_ordering": is "tf" and  "backend": "tensorflow", your input_shape must be (height, width, channels)

anujshah1003 suggestion was good. K needs to be imported explicitly.

@lizhiyuanUSTC what if my "image_dim_ordering" is "th" and "backend" is "tensorflow"? Thanks!

@qilicun just change the image_dim_ordering to tf or backend to theano


autogpt doesn't write on file

Why AUTOGPT doesn't write on file? And also, it is connected to internet?

Is it possibile to have a code already compiled by you that gives the permissions to chatgpt?


Reading images concurrently in a threaded setup may result in a segfault

Not quite sure why this is the case, but reading multiple jpeg files concurrently in a threaded setup may result in a segfault. Compare the stacktrace below.
*** Error in `/root/oli/rfl/env/bin/python3': double free or corruption (fasttop): 0x00007fc1046223e0 ***
======= Backtrace: =========
/usr/lib/libc.so.6(+0x70c4b)[0x7fc1f72fec4b]
/usr/lib/libc.so.6(+0x76fe6)[0x7fc1f7304fe6]
/usr/lib/libc.so.6(+0x777de)[0x7fc1f73057de]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(H5MM_xfree+0xe)[0x7fc174c86ece]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(+0x9470917)[0x7fc174b09917]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(H5E_clear_stack+0x35)[0x7fc174b09a85]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(H5Fis_hdf5+0x2b)[0x7fc174b0c6bb]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(_ZN2H56H5File6isHdf5EPKc+0xd)[0x7fc174aca0fd]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(_ZN3itk11HDF5ImageIO11CanReadFileEPKc+0x110)[0x7fc1747a4ba0]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(_ZN3itk14ImageIOFactory13CreateImageIOEPKcNS0_12FileModeTypeE+0x25c)[0x7fc174f4765c]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(+0x3865515)[0x7fc16eefe515]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(+0x370c569)[0x7fc16eda5569]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(+0x370cc28)[0x7fc16eda5c28]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(+0x335c038)[0x7fc16e9f5038]
/root/oli/rfl/env/lib/python3.5/site-packages/SimpleITK/_SimpleITK.so(+0x335cf35)[0x7fc16e9f5f35]
/usr/lib/libpython3.5m.so.1.0(PyCFunction_Call+0xc9)[0x7fc1f78f8bc9]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x799f)[0x7fc1f796fa3f]
/usr/lib/libpython3.5m.so.1.0(+0x1285e3)[0x7fc1f79715e3]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x5d8e)[0x7fc1f796de2e]
/usr/lib/libpython3.5m.so.1.0(+0x1285e3)[0x7fc1f79715e3]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalCodeEx+0x23)[0x7fc1f79716c3]
/usr/lib/libpython3.5m.so.1.0(+0x92fd8)[0x7fc1f78dbfd8]
/usr/lib/libpython3.5m.so.1.0(PyObject_Call+0x47)[0x7fc1f78b12b7]
/usr/lib/libpython3.5m.so.1.0(+0x7ed14)[0x7fc1f78c7d14]
/usr/lib/libpython3.5m.so.1.0(PyObject_Call+0x47)[0x7fc1f78b12b7]
/usr/lib/libpython3.5m.so.1.0(+0x185ee2)[0x7fc1f79ceee2]
/usr/lib/libpython3.5m.so.1.0(PyObject_Call+0x47)[0x7fc1f78b12b7]
/usr/lib/libpython3.5m.so.1.0(+0x119a1e)[0x7fc1f7962a1e]
/usr/lib/libpython3.5m.so.1.0(+0x95946)[0x7fc1f78de946]
/usr/lib/libpython3.5m.so.1.0(+0x95c00)[0x7fc1f78dec00]
/usr/lib/libpython3.5m.so.1.0(+0xc3475)[0x7fc1f790c475]
/usr/lib/libpython3.5m.so.1.0(PyObject_Call+0x47)[0x7fc1f78b12b7]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x3d6b)[0x7fc1f796be0b]
/usr/lib/libpython3.5m.so.1.0(+0x1285e3)[0x7fc1f79715e3]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalCodeEx+0x23)[0x7fc1f79716c3]
/usr/lib/libpython3.5m.so.1.0(+0x930c5)[0x7fc1f78dc0c5]
/usr/lib/libpython3.5m.so.1.0(PyObject_Call+0x47)[0x7fc1f78b12b7]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0xee6)[0x7fc1f7968f86]
/usr/lib/libpython3.5m.so.1.0(+0x1285e3)[0x7fc1f79715e3]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalCodeEx+0x23)[0x7fc1f79716c3]
/usr/lib/libpython3.5m.so.1.0(+0x930c5)[0x7fc1f78dc0c5]
/usr/lib/libpython3.5m.so.1.0(PyObject_Call+0x47)[0x7fc1f78b12b7]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0xee6)[0x7fc1f7968f86]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x88ca)[0x7fc1f797096a]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x88ca)[0x7fc1f797096a]
/usr/lib/libpython3.5m.so.1.0(+0x1285e3)[0x7fc1f79715e3]
/usr/lib/libpython3.5m.so.1.0(PyEval_EvalCodeEx+0x23)[0x7fc1f79716c3]
/usr/lib/libpython3.5m.so.1.0(+0x92fd8)[0x7fc1f78dbfd8]
/usr/lib/libpython3.5m.so.1.0(PyObject_Call+0x47)[0x7fc1f78b12b7]
/usr/lib/libpython3.5m.so.1.0(+0x7ed14)[0x7fc1f78c7d14]
/usr/lib/libpython3.5m.so.1.0(PyObject_Call+0x47)[0x7fc1f78b12b7]
/usr/lib/libpython3.5m.so.1.0(PyEval_CallObjectWithKeywords+0x47)[0x7fc1f7967da7]
/usr/lib/libpython3.5m.so.1.0(+0x1629a2)[0x7fc1f79ab9a2]
/usr/lib/libpthread.so.0(+0x7454)[0x7fc1f7633454]
/usr/lib/libc.so.6(clone+0x5f)[0x7fc1f73767df]

Do you have sample code to reproduce the problem? Is it occasional or does it always happen?

This call stack stays this is an HD5 issue. The HDF5 library has a configuration option to ensure it's thread safe:
https://support.hdfgroup.org/hdf5-quest.html#tsafe
I have had success concurrently reading tiff files. There may be other issues with other ImageIO types, caution should be used with concurrent IO. We should ensure that the mechanism to determine the ImageIO type is concurrent thread safe.

Here is some sample code that reproduces the problem on my machine always, assuming you have a few jpeg images.

ArchLinux
Python 3.5.2
SimpleITK 0.10.0

import glob
import SimpleITK as sitk
from multiprocessing.pool import ThreadPool

files = glob.glob('/home/oli/Projects/rfl/tests/data/2d/images/*.jpg')
files = files * 1000

def load_file(path):
    print('Loading file', path)
    image = sitk.ReadImage(path)
    data = sitk.GetArrayFromImage(image)

with ThreadPool() as p:
    list(p.imap(load_file, files))

I also don't understand why HDF5 is involved in reading jpeg images. Additionally, I have the same problem with other image formats (2D as well as 3D).

The HDF5 ImageIO is used when determining the file format. Here is a first pass to try to address the issue in ITK:
http://review.source.kitware.com/#/c/21811/

I think you references the wrong ticket @blowekamp, since the linked one just replicates the HDF5 cmake options in ITK. These changes shouldn't be related to this issue.

Yeah, that looks like a reasonable thing to do. But it probably takes some time until this lands in SimpleITK in form of a new release, I would assume.
Does this patch might fix this issue as well?

We are writing a test for multi-threading I/O with a couple different formant to determine how stable this operation is and for what formats.
I expect that ITK patch will be in the next branch of SimpleITK today. It's not too hard to build the SimpleITK superbuild from the source. It just takes a little time. When this ITK patch is merged and in SimpleITK's next I'd recommend compiling it and trying the potential fix out for your files.
The next SimpleITK release will probably be at the end of next month. Also there are some nightly binaries on the dashboard for the master branch, which may be of help.

I have written a test and experimented with the fix. It seems to resolve the problem. The usability of concurrent reading and writing of image in SimpleITK is subject to the thread-safety of the individual ITK ImageIO, and the underlying third-party library used for IO. So this feature can be used with caution, and testing.
When the underlying C++ ITK code is called from SimpleITK the Python GIL is unlocked. This enables light weight Python threads to be used effectively, unlike with normal pure Python code. While many cases the IO will be bound by storage access, the approach appears to be quite effective.
I have a directory of the Visible Human as 1878 png files.
Here is the performance improvement I see:
In [9]: %timeit -n1 -r3 p.map(sitk.ReadImage, files)
1 loop, best of 3: 2.86 s per loop
In [10]: %timeit -n1 -r1 map(sitk.ReadImage, files)
1 loop, best of 1: 1min 28s per loop

This issue is still haunting me in SimpleITK 2.2.0. It doesn't matter if I use sitk.ReadImage, sitk.ImageFileReader or if I manually select an ImageIO. I've also tried newer versions of HDF5 since that was identified in this old issue (though unsure if SimpleITK python package is using it). I am quite sure this is a problem in ITK too.
My workaround is to use nibabel 😢

We could try rebuilding ITK/SimpleITK with this option enabled: https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/ThirdParty/HDF5/src/CMakeLists.txt#L27
Do you have a small block of code to reproduce the issue?

This is a small piece of an image batcher used for training neural networks. There's more following it that post-processes the loaded images, but that's unimportant here.
Sorry for all the extra .copy()s... I had numpy memory corruption issues too (diagnosed using MD5 sums)! Yes, you can still shoot yourself in the foot C-style in Python apparently. It just seems harder to know that you're shooting yourself in the foot because Python hides all the details.
    def _load_patients(self, patientIds):
        volumeMaskPairs = []

        def load_one_patient(patientId):
            imageFiles = [ f"normalized{i+1}_aligned.nii.gz" if i > 0 else "normalized_aligned.nii.gz" for i in range(self.numChannels) ]

            if "normalized8_aligned.nii.gz" == imageFiles[-1]:
                imageFiles[-1] = "image8_aligned.nii.gz"

            imageFiles = [ os.path.join(self.dataRoot, "Images", patientId, imageFile) for imageFile in imageFiles ]

            maskFile = os.path.join(self.dataRoot, "Masks", patientId, "mask_aligned.nii.gz")

            sitkVolumes = [ LoadImage(imageFile, use_nib=True) for imageFile in imageFiles ]

            assert all(sitkVolumes), str(patientId)

            npVolumes = [ sitk.GetArrayViewFromImage(sitkVolume) for sitkVolume in sitkVolumes ]

            assert all([npVolumes[0].shape == npVolume.shape for npVolume in npVolumes])

            sitkMask = LoadMask(maskFile, numClasses=self.numClasses, dilateUnknown=self.dilateUnknown, use_nib=True)

            assert sitkMask is not None

            npMask = sitk.GetArrayViewFromImage(sitkMask).astype(np.int32) # XXX: Prevent issue with -1
            npMask[npMask == 255] = -1
            npMask[np.logical_and(npVolumes[0] < 253, npMask == 1)] = -1

            for npVolume in npVolumes:
                npMask[npVolume < -188] = -1

            npVolumes = [ npVolume[None, None, ...].copy() for npVolume in npVolumes ]
            npMask = npMask[None, ...].copy()

            return np.concatenate(tuple(npVolumes), axis=1), npMask

        with ThreadPoolExecutor(max_workers=min(8, self.batchSize)) as exe:
            volumeMaskPairs = list(exe.map(load_one_patient, patientIds))

Here's LoadImage (now modified to workaround this race using nibabel).
def LoadImage(path, use_nib=False):
    try:
        if use_nib:
            import nibabel as nib
            img = nib.load(path)
            img = np.asarray(img.dataobj)
            img = img.transpose(2,1,0)
            img = sitk.GetImageFromArray(img)
            return img
        else:
            return sitk.ReadImage(path)
    except:
        return None

LoadMask calls this in addition to doing some complicated re-labeling of masks based on ITK-SNAP label text files.

Can you create a minimal reproducible example?

Sure. But I want you to understand that this problem is not strictly reproducible.
For example, I have hundreds of experiments using 4 image channels (1 nifti file per channel) that sparsely crash due to this issue. I thought failing hardware was the cause (in fact, some machines had hardware problems). Cranking this up to 8 image channels causes a crash rate of 50%, all with "Segmentation fault". All traceable back to SimpleITK (via gdb backtrace command).
Now, you may have to wait 10 seconds or 5 days for a crash... and it may not even crash.

Try the attached out.
SimpleITKCrash.zip
A screenshot is included to show the random nature of the reproducibility. I even added a "No crash!" line later to show successful terminations.

If I change sitk.ReadImage to nib.load, I can't seem to make it crash anymore. Indeed, I have jobs that have been running for a day now without crashing using the nibabel workaround. Although, I also some have jobs using SimpleITK that have been running for a few days now... but they can still crash. I had a 1000 epoch SimpleITK-based experiment crash at about 950 epochs... just says "Segmentation fault". And I have about half of them that completed successfully over several days using SimpleITK (and more than half of them successful when using fewer nifti images to form pytorch tensors).

@nslay For that particular issue demonstrated in crash.py I was able to use the stack trace to locale the global initialization not being concurrent thread safe. This issue appears different than the reported issue of random failure doing IO of NIFT.
The code above uses GetArrayViewFromImage which creates a reference to SimpleITK managed memory. It can be tricky in python and NumPy to keep track of the owner which can lead to segmentation fault. If you are frequently using GetArrayViewFromImage over the safer GetArrayFromImage then there may be incorrect uses or alias that could cause the problem.
If you have another minimal reproducible example of threading issue please create a new issue with the bug report.

I don't think that's the problem because memory is only shared within load_one_patient(). Everything returned by load_one_patient is a deep copy. The concatenate is obvious, but the LoadMask is also a deep copy because no one stores 32 bit integer masks. The astype(np.int32) performs a deep copy.

Have you tried the multi-process based Pool instead of the thread based one? With SimpleITK Images supporting pickling, the usage should be rather transparent.
It may be that the C++ NIFTI library is no concurrent thread safe.

If I had to take a guess... I'd guess this is partly related to factories and their initialization. And ITK loves using lots and lots of factories everywhere!

Your report indicates that it is irregular after long execution periods. Once the factories are initialized and your code is running ( for hours ) the factories are not going to change.

The one issue is being addressed with the patch in ITK linked above.
We need a stack trace of the failure to narrow down the cause. To do that we need to reproduce the problem, with the SimpleITK symbols not stripped from the binary. Otherwise we are just guessing.

Or maybe valgrind or address sanitizer. A crash may occur somewhere that doesn't make any sense... caused by something else. I'll see what I can do for you.


Referencing source code of plugin raises exception "No platform plugin found. Did you install the nuget package in your app project as well?"

Please check all of the platforms you are having the issue on (if platform is not listed, it is not supported)

[ ] iOS
[X] Android
[ ] UWP
[ ] .NET Standard

Feature Library

[ ] Core
[X] BluetoothLE
[ ] Locations
[ ] HTTP Transfers
[ ] Notifications
[ ] Sensors

Version of Library
commit 1d098d8ef08af1f8fb8bde61cbb1427cf931bf4b on branch 6.3.0
Version of OS(s) listed above with issue
Android 9.0
Steps to Reproduce

Clone the git repository and move on branch 6.3.0
Remove not used target information from csproj (monoandroid81;monoandroid90;xamarinios10;xamarinmac20;xamarin.TVOS10)
Reference the bluetooth LE project from a Xamarin.Forms project
Call CrossBleAdapter.Current

Expected Behavior
Both AdapterScanner and Current are not null
Actual Behavior
AdapterScanner and Current are null and the following exception is raised: "No platform plugin found. Did you install the nuget package in your app project as well?"
When I use the nuget package no exceptions are raised the adapter is correctly managed. I can perform scan operations without any issue or exception.
Now I need to check some features available on branch 6.3.0, but I'm not able to use it.
Do I have to do some extra steps not written here link ?
I cannot understand how the plugin is initialized, can initialization be the cause of the problem?
NOTE: Since my Xamarin.Forms app is .netstandard2.0, I removed the following lines from csproj file in order to avoid build errors on platforms I don't use:
<TargetFrameworks>netstandard2.0;monoandroid81;monoandroid90;xamarinios10;xamarinmac20;xamarin.TVOS10</TargetFrameworks>
<TargetFrameworks Condition=" '$(OS)' == 'Windows_NT' ">uap10.0.16299;$(TargetFrameworks)</TargetFrameworks>

and replaced them with:
<TargetFrameworks>netstandard2.0</TargetFrameworks>
In MainActivity.cs I'm requesting bluetooth access in this way:
System.String permission = Manifest.Permission.AccessFineLocation;

if (ContextCompat.CheckSelfPermission(this, permission) != Permission.Granted)
{
	ActivityCompat.RequestPermissions(this, new System.String[] { Manifest.Permission.AccessCoarseLocation, Manifest.Permission.AccessFineLocation }, 0);
}
else
{
	System.Diagnostics.Debug.WriteLine("Permission Granted!!!");
}

I'm sorry - this isn't the type of request I'm willing to walk you through.  This is a multitargeted library.  Nuget manages the implementation that is used based on the compile target.  You can find some documents out there on this.


Link to individual errors in 'Voter Resources'

Link to individual ElectionAdministration, Department, and VoterService errors.
Pivotal Card
This is based on https://github.com/votinginfoproject/Metis/pull/329
This is blocked by https://github.com/votinginfoproject/data-processor/pull/209, which creates a database function element_type, that makes it all work.

LGTM!


[silgen] Fix createUnsafeCopyUnownedValue to be correct from an ownership perspective.

[silgen] Fix createUnsafeCopyUnownedValue to be correct from an ownership perspective.
When I originally implemented this, I did not understand why an unowned(unsafe)
value was unsafe. I thought it was unsafe since the value was not meant to be
paired with (which is why I used the unmanaged_retain_value). Instead it just
means that we perform an unconditional copy without checking if the unowned
value is actually live. In contrast, an unowned(safe) value, we use
copy_unowned_value which performs an assert that the unowned value is still
alive before copying.
From a non-semantic sil perspective, since I still was retaining the value
properly, there was not a bug by using the unmanaged_retain_value instruction.
rdar://31880847

@swift-ci smoke test and merge

Not sure why autolink failed. My change should not have affected that. Lets try again.

@swift-ci smoke test linux platform


Add static extraction

I based this on the keyframes branch since it's where you've been committing.
This goes through the PostCSS AST and turns it into two sets of rules, one set of rules that's static and can be put into a css file and another set of rules that's dynamic and must be in js(this is really inefficient right now but can be improved later). It then writes all the static rules in a file to filename.css(just like glam) and puts the dynamic rules in the js just like before.
Currently this adds an import declaration for the css, I think this is a good default(there isn't an option to disable it yet but it would be trivial to add) so that users don't have to deal with the underlying css files and remembering to add imports(and remove imports since if there is static css in a file, then there isn't static css the css file will still be there) as long as they have the appropriate loader/some method for importing css
Currently this only extracts static css in styled and css.
I made this opt-out with an extract option that if it's set to false will disable it(it is also disabled if the filename is unknown so it works fine in the example site), I think this is needed for tests because without it jest-utils(And hopefully jest-glamor-react soon!) won't take into account static css.
Tests

For the non babel tests I set extract to false so .toMatchSnapshotWithEmotion would work with static css
For the babel tests, most of them have no filename set so they don't extract css but I added some that do and they use a mocked fs
I copied some other tests I wrote for glam to test writing to the filesystem(with a mock)

Also random idea, maybe instead of the css filename being filename.css it should be filenamewithoutextension.emotion.css e.g. main.emotion.css vs main.js.css.
unrelated to this PR but it seems that lib/ isn't published in emotion 3

Codecov Report

Merging #23 into keyframes will increase coverage by 1.79%.
The diff coverage is 100%.

@@              Coverage Diff              @@
##           keyframes      #23      +/-   ##
=============================================
+ Coverage      79.49%   81.29%   +1.79%
=============================================
  Files             11       11
  Lines            395      433      +38
  Branches          67       78      +11
=============================================
+ Hits             314      352      +38
  Misses            67       67
  Partials          14       14

Impacted Files
Coverage Δ

src/parser.js
100% <100%> (ø)
:arrow_up:

src/inline.js
100% <100%> (ø)
:arrow_up:

src/babel.js
97.14% <100%> (+0.77%)
:arrow_up:

src/index.js
69.23% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ecf50ff...88fd4d2. Read the comment docs.

Sorry about me commiting to that random branch. Master is up to date now.

unrelated to this PR but it seems that lib/ isn't published in emotion 3

Fixed in v3.0.2

Also random idea, maybe instead of the css filename being filename.css it should be filenamewithoutextension.emotion.css e.g. main.emotion.css vs main.js.css.

This is a great idea.

What I've done:

the name of css files are now filenamewithoutextension.emotion.css e.g. main.emotion.css for a file named main.js
css is only extracted if there are no @applys in it and css vars are used for dynamic bits
the extract option is removed and is replaced with an inline option which when true causes the babel plugin to not extract anything


when toolchains are empty, message is confusing

When .elan folder is empty (any toolchain installed) and we try to do an elan update the next message appears (which is confusing, because it doesn't give a clue about what to do next).

PR #82


doc: Update core tem members

Not all team leaders are listed and need to be update the page
Please include the following team members:

Abhipsa Mishra - Leader
Silke Nodwell - Leader
Ekaterine Oshkhereli - Leader

The photos and complete details could be asked for tem on our slack.

I can take this one

I've create a pull request for Silke and Abhipsa #132, will create a new one for Eka once I get her details

Please include Ekaterine Oshkhereli, before Airat Yusuff.
And we also have a new Leader 🙈
Sonali Goel
Position: Senior/Lead Engineer at Yoox-Net-a-porter


[HIPIFY] Changing compiler and cuda environment related section to ROCm?

I am hipifying RapidCFD, an on GPU version of OpenFOAM.
How to modify, the following, compiler and cuda environment related section in the settings file to use hipcc and the Thrust implementation provided by ROCm:
# Compiler settings
# ~~~~~~~~~~~~~~~~~
unset gcc_version gmp_version mpfr_version mpc_version
unset MPFR_ARCH_PATH GMP_ARCH_PATH

# Location of compiler installation
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
if [ -z "$foamCompiler" ]
then
    foamCompiler=system
    echo "Warning in $WM_PROJECT_DIR/etc/config/settings.sh:" 1>&2
    echo "    foamCompiler not set, using '$foamCompiler'" 1>&2
fi

case "${foamCompiler}" in
OpenFOAM | ThirdParty)
    case "$WM_COMPILER" in
    Gcc | Gcc++0x | Gcc48 | Gcc48++0x)
        gcc_version=gcc-4.8.2
        gmp_version=gmp-5.1.2
        mpfr_version=mpfr-3.1.2
        mpc_version=mpc-1.0.1
        ;;
    Gcc49 | Gcc49++0x)
        gcc_version=gcc-4.9.0
        gmp_version=gmp-5.1.2
        mpfr_version=mpfr-3.1.2
        mpc_version=mpc-1.0.1
        ;;
    Clang)
        # using clang - not gcc
        export WM_CC='clang'
        export WM_CXX='clang++'
        clang_version=llvm-3.4.2
        ;;
    *)
        echo 1>&2
        echo "Warning in $WM_PROJECT_DIR/etc/config/settings.sh:" 1>&2
        echo "    Unknown OpenFOAM compiler type '$WM_COMPILER'" 1>&2
        echo "    Please check your settings" 1>&2
        echo 1>&2
        ;;
    esac

    # optional configuration tweaks:
    _foamSource `$WM_PROJECT_DIR/bin/foamEtcFile config/compiler.sh`

    if [ -n "$gcc_version" ]
    then
        gccDir=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER_ARCH/$gcc_version
        gmpDir=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER_ARCH/$gmp_version
        mpfrDir=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER_ARCH/$mpfr_version
        mpcDir=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER_ARCH/$mpc_version

        # Check that the compiler directory can be found
        [ -d "$gccDir" ] || {
            echo 1>&2
            echo "Warning in $WM_PROJECT_DIR/etc/config/settings.sh:" 1>&2
            echo "    Cannot find $gccDir installation." 1>&2
            echo "    Please install this compiler version or if you wish to use the system compiler," 1>&2
            echo "    change the 'foamCompiler' setting to 'system'" 1>&2
            echo
        }

        _foamAddMan     $gccDir/man
        _foamAddPath    $gccDir/bin

        # add compiler libraries to run-time environment
        _foamAddLib     $gccDir/lib$WM_COMPILER_LIB_ARCH

        # add gmp/mpfr libraries to run-time environment
        _foamAddLib     $gmpDir/lib
        _foamAddLib     $mpfrDir/lib

        # add mpc libraries (not need for older gcc) to run-time environment
        if [ -n "$mpc_version" ]
        then
            _foamAddLib     $mpcDir/lib
        fi

        # used by boost/CGAL:
        export MPFR_ARCH_PATH=$mpfrDir
        export GMP_ARCH_PATH=$gmpDir
    fi
    unset gcc_version gccDir
    unset gmp_version gmpDir  mpfr_version mpfrDir  mpc_version mpcDir

    if [ -n "$clang_version" ]
    then
        clangDir=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER_ARCH/$clang_version

        # Check that the compiler directory can be found
        [ -d "$clangDir" ] || {
            echo 1>&2
            echo "Warning in $WM_PROJECT_DIR/etc/config/settings.sh:" 1>&2
            echo "    Cannot find $clangDir installation." 1>&2
            echo "    Please install this compiler version or if you wish to use the system compiler," 1>&2
            echo "    change the 'foamCompiler' setting to 'system'" 1>&2
            echo 1>&2
        }

        _foamAddMan     $clangDir/share/man
        _foamAddPath    $clangDir/bin
    fi
    unset clang_version clangDir
    ;;
system)
    case "$WM_COMPILER" in
    Clang)
        export WM_CC='clang'
        export WM_CXX='clang++'
        clang_version=llvm-9.0.0

        cudaHome="$(dirname $(dirname $(which nvcc)))"
        [ -d "$cudaHome" ] || {
            echo 1>&2
            echo "Error in $WM_PROJECT_DIR/etc/config/settings.sh:" 1>&2
            echo "    Cannot find CUDA installation." 1>&2
            echo "    Found only $cudaHome." 1>&2
            echo 1>&2
            exit 1
        }
        export CUDA_HOME=$cudaHome
        #an option to override thrust
        if [ -d "$THRUST_HOME" ]
        then
            export THRUST_INCLUDE=-I$THRUST_HOME
        else
            export THRUST_INCLUDE=
        fi
        unset cudaHome
        ;;
    esac
    # okay, use system compiler
    ;;
*)
    echo "Warn: foamCompiler='$foamCompiler' is unsupported" 1>&2
    echo "   treating as 'system' instead" 1>&2
    ;;
esac

#
# add c++0x flags for external programs
#
if [ -n "$WM_CXXFLAGS" ]
then
    case "$WM_COMPILER" in
    Gcc*++0x)
        WM_CXXFLAGS="$WM_CXXFLAGS -std=c++0x"
        ;;
    esac
fi

Hello @klausbu,
I don't quite understand how your question relates to HIPIFY tools.

Well, I hippified the RapidCFD source code but all the settings files that call CUDA to compile the code instead of HIP or whatever ROCm software to be replaced remained unchanged. I need to know what to change to call the c++ compiler and hipcc (HIP) and thrust (RapidCFD uses primarily the trust library).   There were also libraries like -lcudart which were not removed from compilation related files - which is hopefully all to be done with them.

First, I'm glad to hear that you successfully hipified your sources. Second, as for building your OpenFOAM stuff, unfortunately, it is out of HIPIFY's scope, sorry.


pyglet.gl.ContextException: Could not create GL context

Hi
When I ran  python src/visualize/render_smplx.py，this problem occurred.
May I ask what is the reason for this?
the version of Opengl to be 3.3?
@anindita127
Looking forward to your reply!

I have another question, what is the pyglet version corresponding to opengl version 3.1.0?


Block hash mismatch error while syncing Madara.

Is there an existing issue?

[X] I have searched the existing issues

Description of bug
After running madara as full node, there is an error while syncing:
[2024-10-11 11:49:48 INFO] :sparkles: Imported #218483 (0x146c0b...531539) and updated state root (0x38a6a6...143729)
[2024-10-11 11:49:48 INFO] :hourglass_flowing_sand: Gracefully closing the database...
Error: Block hash mismatch: expected 0x495aaa2ae4e2784fcdf4c50a5d21415e388c593a3d0b6db0958758dadd829d9, got 0x7646fc987f5ecc3941804c3fabc6bf684691756d5b2fd21f94fc0b114459188

Steps to reproduce
Run madara with these flags
- "--name"
        - "madara"
        - "--base-path"
        - "/madara/data"
        - "--network"
        - "test"
        - "--l1-endpoint"
        - "http://anvil-service:8545"
        - "--full"
        - "--gateway-key"
        - "$(GATEWAY_KEY)"
        ```

Did you simply let it run without touching it or its base path? It seems that you shut downed the process before it mismatch right? If you restart it which logs will you get?

I could not replicate the error by cheating and using the --unsafe-starting-block parameter. Since it's a block hash error and not a state root commitment error, cheating like this should return the error. I believe we also have nodes that have synced past this block without issue.
I am a bit stumped by this, but there is a chance that #333 solves the issue. I am thinking that it might be because the rust std HashMap uses the os random generator to hash the keys (to avoid a DoS vector) and so the hashmap order may not be the same as what we have. There are holes in that theory though, but it might be worth a try.
Would you be so kind as to try that pull request? @Uacias

@cchudant  Sure thing, I'll come back with feedback asap
@antiyro I simply let it run, I actually did not shut down the process. It is all running on kubernetes, these logs may be relevant for you:

kubectl logs -f madara-1
Defaulted container "madara-1" out of: madara-1, wait-for-anvil (init)
[2024-10-15 12:53:19 INFO] 🥷  Madara Node
[2024-10-15 12:53:19 INFO] ✌  Version 0.7.0-75d58ac4a59
[2024-10-15 12:53:19 INFO] 💁 Support URL: https://github.com/madara-alliance/madara/issues
[2024-10-15 12:53:19 INFO] 🏷  Node Name: madara
[2024-10-15 12:53:19 INFO] 👤 Role: Full Node
[2024-10-15 12:53:19 INFO] 🌐 Network: Starknet Sepolia (chain id `SN_SEPOLIA`)
[2024-10-15 12:53:19 INFO] 💻 Operating system: Linux 12 Debian GNU/Linux
[2024-10-15 12:53:19 INFO] 💻 CPU architecture: x86_64
[2024-10-15 12:53:19 INFO] 💻 CPU: AMD Ryzen Threadripper 3970X 32-Core Processor
[2024-10-15 12:53:19 INFO] 💻 CPU cores: 64
[2024-10-15 12:53:19 INFO] 💻 Memory: 128674MB
[2024-10-15 12:53:19 INFO] 💻 Kernel: 6.10.12-200.fc40.x86_64
[2024-10-15 12:53:19 INFO] 💾 Opening database at: /madara/data
[2024-10-15 12:53:19 INFO] 🚀 Subscribed to L1 state verification
[2024-10-15 12:53:19 INFO] ⛓  Starting L2 sync from block 218484
[2024-10-15 12:53:19 INFO] 📱 Running JSON-RPC server at <IP_ADDRESS>:9944 (allowed origins=["http://localhost:*", "http://<IP_ADDRESS>:*", "https://localhost:*", "https://<IP_ADDRESS>:*"])
[2024-10-15 12:53:19 INFO] 📈 Prometheus endpoint started at <IP_ADDRESS>:9615
[2024-10-15 12:53:19 WARN mc_telemetry] ⚠  Could not connect to telemetry endpoint 'wss://starknodes.com/submit': Reqwest(reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("starknodes.com")), port: None, path: "/submit", query: None, fragment: None }, source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })
[2024-10-15 12:53:28 INFO] ⏳ Gracefully closing the database...
Error: Block hash mismatch: expected 0x495aaa2ae4e2784fcdf4c50a5d21415e388c593a3d0b6db0958758dadd829d9, got 0x7646fc987f5ecc3941804c3fabc6bf684691756d5b2fd21f94fc0b114459188

This is madara pod setup:
  containers:
    - name: madara-1
      image: uacias/madara:latest
      ports:
        - containerPort: 9944
      env:
        - name: RUST_LOG
          value: "info"
      args:
        - "--name"
        - "madara"
        - "--base-path"
        - "/madara/data"
        - "--network"
        - "test"
        - "--l1-endpoint"
        - "http://anvil-service:8545"
        - "--full"
      volumeMounts:
        - name: madara-data-1
          mountPath: /madara/data
  volumes:
    - name: madara-data-1
      persistentVolumeClaim:
        claimName: madara-pvc
        ```
This is madara.dockerfile

Stage 1: Build the application
FROM rust:1.81 AS builder
Install build dependencies
RUN apt-get -y update &&
apt-get install -y clang git &&
apt-get autoremove -y;
apt-get clean;
rm -rf /var/lib/apt/lists/*
Set the working directory
WORKDIR /usr/src/
Clone the Madara repository
RUN git clone https://github.com/madara-alliance/madara.git
WORKDIR /usr/src/madara/
Installing scarb
ENV SCARB_VERSION="v2.8.2"
ENV SCARB_REPO="https://github.com/software-mansion/scarb/releases/download"
ENV PLATFORM="x86_64-unknown-linux-gnu"
ENV SCARB_TARGET="/usr/src/scarb.tar.gz"
RUN curl -fLS -o $SCARB_TARGET
$SCARB_REPO/$SCARB_VERSION/scarb-$SCARB_VERSION-$PLATFORM.tar.gz &&
tar -xz -C /usr/src/ --strip-components=1 -f $SCARB_TARGET &&
mv /usr/src/bin/scarb /bin
Build the application in release mode
RUN cargo build --release
Stage 2: Create the final runtime image
FROM debian:bookworm-slim
Install runtime dependencies
RUN apt-get -y update &&
apt-get install -y openssl ca-certificates &&
apt-get autoremove -y;
apt-get clean;
rm -rf /var/lib/apt/lists/*
Set the working directory
WORKDIR /usr/local/bin
Copy the compiled binary from the builder stage
COPY --from=builder /usr/src/madara/target/release/madara .
chain presets to be monted at startup
VOLUME /usr/local/bin/crates/primitives/chain_config/presets
VOLUME /usr/local/bin/crates/primitives/chain_config/resources
Set the entrypoint
ENTRYPOINT ["./madara"]

Just to mention, if you would like to replicate this one you could checkout out our repo:

https://github.com/neotheprogramist/starknet-rpc-tests/blob/main/scripts/cluster-run.sh

Also possible by simply creating kind cluster and the applying
- anvil (its service is provided as l1 for madara)
- madara

Yes now that there is a block mismatch your db is corrupted.
Something you may want to use in the future are the backups parameters for checkpoints:
--backup-dir <PATH>
          Directory for backups. Use it with `--restore-from-latest-backup` or `--backup-every-n-blocks <NUMBER OF BLOCKS>`

          [env: MADARA_BACKUP_DIR=]

--restore-from-latest-backup
          Restore the database at startup from the latest backup version. Use it with `--backup-dir <PATH>`

          [env: MADARA_RESTORE_FROM_LATEST_BACKUP=]

Note that they will use way more storage.
I see that you have a gatway key now you can just restart it from scratch after we merge https://github.com/madara-alliance/madara/pull/333

My apologies @Uacias your db is not corrupted, we compute hashes before storing, you could just pull and restart.

I've just redeployed the pod, I'll get back with a feedback asap

Unfortunately:
[2024-10-16 12:04:24 INFO] ✨ Imported #218482 (0x2c4bfc...128478) and updated state root (0x32934a...500c5d)
[2024-10-16 12:04:24 INFO] ✨ Imported #218483 (0x146c0b...531539) and updated state root (0x38a6a6...143729)
[2024-10-16 12:04:24 INFO] ⏳ Gracefully closing the database...
Error: Block hash mismatch: expected 0x495aaa2ae4e2784fcdf4c50a5d21415e388c593a3d0b6db0958758dadd829d9, got 0x7646fc987f5ecc3941804c3fabc6bf684691756d5b2fd21f94fc0b114459188

I can confirm this bug is reproductible, the same has been happening to our node on the benchmark server

Actually, interestingly enough Pathfinder seems to have the same bug.
pathfinder_runner  | 2024-10-21T10:14:09  INFO Error during bulk syncing blocks, falling back to normal sync block=218488 error=Verifying block contents
pathfinder_runner  | 2024-10-21T10:14:09  INFO Error during bulk syncing blocks, falling back to normal sync block=218484 error=Verifying block contents
pathfinder_runner  | 2024-10-21T10:14:09  WARN L2 sync process terminated with: Block hash mismatch


refactor: Prepare code for installing older Northstar versions

This PR aims to refactor Northstar installation code to make it easier to later add functionality to install a specific Northstar versions

@catornot you wanna review this PR? At least all the changes in here are backend/Rust stuff and AFAIK you have the most Rust knowledge + FlightCore knowledge combo out of the people I know ^^


[ BUG ] Engineers goggles

Version:  8a4b812
Branch: Master
You are unable to:

Quick Equip Googles
Manually Equip Googles

After Equipping via commands:

GUI for stress and speed does not show up

Related to recent merge of #5

Related to the disabling of the stress mechanic, issue is irrelevant.


btnEvent is not triggering

Hello , im using the new node jabra sdk ( 4.0.1 ) with a Blueparrot Headset.
Id like to capture BlueParrot Button Events but its not working
here is my code:
const j = require("@gnaudio/jabra-node-sdk"); j.createJabraApplication('900').then((jabra) => {      jabra.on('attach', (device) => {         console.log('Press any key on Jabra device ' + device.deviceName);                 device.on('btnPress', (btnType, btnValue) => {           console.log('New input from device is received: ', j.enumDeviceBtnType[btnType], btnValue);         });	     }); });
the device.on action is not triggering by any button pressed
my blueparrot headset model is : C300-XT MS
please some one can help me?

This repository is not suited to for such questions. You should try ask on StackOverflow.

"Sir, this is a Wendy's."
This isn't the place for such questions; you might want to try Stack Overflow.

It looks like this is a question rather than a bug report. This issue tracker is for tracking bugs and active work on TypeScript itself, rather than a general forum for programmers using TypeScript to get help or ask questions.
You can ask questions on sites like Stack Overflow. We are not able to provide one-on-one support on the issue tracker. Please read the issue template carefully - it has important information on what kinds of reports can be acted on here, as well as links to useful TypeScript resources. Thanks!