SSL verification error when using requests sessions

The problem
When using requests sessions to connect to a https website with a self signed certificate, requests throws a SSLError. It is working pefectly fine when using requests without sessions.
What I am using:

python2.7
requests (2.18.4)
exported REQUESTS_CA_BUNDLE environment variable
Imported certificate to ca-certificates on underlying Debian os

Note: The certificate validates more than one hostname (e.g. kubernetes, kubernetes.default, kubernets.default.svc, kubernetes.default.svc.cluster.local, ...)
Steps to reproduce
# Normal request
import requests
requests.get('https://kubernetes.default.svc')
<Response [401]>

# Session request
s = requests.Session()
request = requests.Request('GET', 'https://kubernetes.default.svc')
prepared_request = s.prepare_request(request)
response = s.send(prepared_request)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 612, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 504, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kubernetes.default.svc', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)'),))

Sorry, this was supposed to be posted at Bravado, not requests!

You should prepare the request using the Session:
request = requests.Request('GET', 'https://kubernetes.default.svc')
prepared_request = s.prepare_request(request)
response = s.send(prepared_request)

Thanks for the quick respnse Lukasa, I did make a mistake copy/pasting on the issue. Sorry.
I will update the issue too. Could you please reopen?
Here is the request output using the session to prepare:
import requests
# Normal request
requests.get('https://kubernetes.default.svc')
<Response [401]>

# Session request
s = requests.Session()
request = requests.Request('GET', 'https://kubernetes.default.svc')
prepared_request = s.prepare_request(request)
response = s.send(prepared_request)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 612, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 504, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='kubernetes.default.svc', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)'),))

Yup, you're missing some environmental settings here. This is something that is not well handled by the prepared request flow. You can try doing this:
s = requests.Session()
request = requests.Request('GET', 'https://kubernetes.default.svc')
prepared_request = s.prepare_request(request)

settings = s.merge_environment_settings(prepared_request.url, None, None, None, None)
response = s.send(prepared_request, **settings)

Thanks Lukasa, this works like a charm! I will try to figure out what exactly is wrong with my environment.

Nothing is wrong with it: just the prepared request flow doesn't normally consider the environment. =)

That is good to know. Maybe it can be added to the documentation page:
http://docs.python-requests.org/en/master/user/advanced/#session-objects
This page documents the usage of Sessions, but it doesn't mention the environment. Also the SSL section does not mention this.

It should probably go into the prepared request section, not the session section. =) I'd welcome a patch though.

Do you mean a patch on the documentation, or to fix the prepared request to use the environment?
I am not really familiar with the code, but I can take a crack at it.

A patch on the documentation. :smile:

I know this issue is closed but I had the same issue and with hours of research and attempting potential solutions nothing seemed to work. Ended up uninstalling and reinstalling Python and have not ran into this issue since, I highly suggest you try this if you are still experiencing this issue and have tried multiple potential resolutions.


Provide mechanism to disable package comparison

In some cases, due to odd packaging practices, certain packages are
expected to have different versions across different arches. This change
allows admins to define a list of packages that should not be compared.
Signed-off-by: Luiz Carvalho<EMAIL_ADDRESS>

Updated release notes draft:

New mechanism added to bypass component comparison based on server configuration


ci: enable CI via GitHub workflows

This patch is loosely based on code from OP-TEE/optee_os repo. It uses the same idea with custom container to build and run OP-TEE OS + Zephyr OP-TEE client + Zephyr OP-TEE test suite on QEMU.

LGTM Reviewed-by: Oleksii Moisieiev<EMAIL_ADDRESS>I'm not an expert in the github CI. but as I saw from the script - you fetch only optee-client. I suppose you have some prefetched version of zephyr-optee-test. If yes - there should be dependency or at least compatibility check. because tests are dependent from optee version.


Update sklearn example storage uri

Update sklearn example storage uri to be consistent with the
kserve PR, https://github.com/kserve/kserve/pull/1997
Signed-off-by: Chin Huang<EMAIL_ADDRESS>
"Fixes #issue-number" or "Add description of the problem this PR solves"
Proposed Changes

Thanks @chinhuang007 !
/approve


Fix tutorials to validate all scenarios in MacOS

Fix tutorials to validate all scenarios in MacOS, not just a few

Resolving by turning off Macos here per #869


fix(knative): Make knative broker name configurable

https://github.com/apache/camel-k/issues/2864

Add e2e knative tests
Change the kamelet-binding-broker yaks test to use a custom broker name

Release Note
Make knative broker name configurable

WIP: This PR depends on a camel-knative fix. This is for an early review.

@christophd @squakez  for review.

@claudio4j many thanks! some e2e tests failed. is this because the fixed Camel component is not part of Camel K yet?

some e2e tests failed. is this because the fixed Camel component is not part of Camel K yet?
Yes, we should wait the release train of camel artifacts.

@claudio4j will users of the component be able to listen for all events on a broker with this fix, too? At the moment users need to filter for a given event type and can not listen for all events on that broker.

@claudio4j will users of the component be able to listen for all events on a broker with this fix, too? At the moment users need to filter for a given event type and can not listen for all events on that broker.

Yes, there is a e2e knative-broker test that checks all messages are received.

@claudio4j awesome! this is a much appreciated enhancement

@claudio4j can we rebase this please?

@claudio4j can we rebase this please?

rebased.

I see all of the failing CI tests are timeouts.

For what I could see, the check errors are the same we are experincing on main (likely releated to the new runtime?). I'm merging also this and we can focus on fixing directly any error on main


Cleanup unused APIs on Export type

What was wrong?
There were multiple property methods on the Export type which are not used (as measured by running code coverage)
How was it fixed?
Removed them since they weren't needed.
Cute Animal Picture

If there's anything you'd like cleaned up I'll do it post merge.


Fix scope usage of SysTime/TimeZone

Currently, inout gives a free STC.return_ lifetime to functions without an actual lifetime restriction on the returned value. See: dlang/dmd#12689
The SysTime struct has a Rebindable!(immutable TimeZone) _timezoneStorage; field with a getter that's inferred scope because Rebindable.get() is qualified inout. This PR makes SysTime and TimeZone compatible with scope without relying on the inout bug.

std/datetime/interval.d(8355):        cannot pass argument `&stFunc` of type
`SysTime delegate(scope const(SysTime) st) pure nothrow @safe` to parameter
`SysTime delegate(scope const(SysTime)) func`

Huh

It looks like previous work on scope in std.datetime (https://github.com/dlang/phobos/pull/6604 https://github.com/dlang/phobos/pull/6618) relied on Rebindable swallowing lifetimes through inout, for example through cast(SysTime) sysTime:
SysTime stFunc(scope const SysTime st) { return cast(SysTime) st; }

st is clearly not scope here. It doesn't matter for the other cases with Date, TimeOfDay and DateTime since those don't have a pointer member (class TimeZone) like SysTime, but stFunc here correctly gets return inferred now. I have no clue why return scope doesn't show up in the error message, I couldn't reduce it to a small test case so far.


Using cloud-init volumes is not idempotent: Error: volume.0.user_data attribute is immutable

Description
Adding a server with specified user-data will make it impossible run terraform apply more than once. To do so, following lifecycle paradigm must be added:
  lifecycle {
    ignore_changes = [ volume.0.user_data ]
  }

Expected behavior
At least terraform should not break, if user data is used, so at least, the fact that user data is not returned from the cloud api, should lead to ignoring this field, when state is refreshed.
Environment
Terraform version:
Terraform v0.15.4

Provider version:
+ provider registry.terraform.io/hashicorp/local v2.2.2
+ provider registry.terraform.io/hashicorp/tls v3.3.0
+ provider registry.terraform.io/ionos-cloud/ionoscloud v6.2.3

OS:
on linux_amd64

Configuration Files

How to Reproduce

Error and Debug Output

Additional Notes

References

Nevermind, was using a complete outdated terraform version. /close

Im having the same problem on:
Terraform v1.2.9
on linux_amd64
+ provider registry.terraform.io/ionos-cloud/ionoscloud v6.3.2

When trying to plan or apply onto a created 'ionosccloud_server' resource I cant can't run plan or apply:
 Error: volume.0.user_data attribute is immutable, therefore not allowed in update requests
Anyone got a fix to this?

User_data can only be set on server creation. You need to first destroy the server, change user_data and then run apply. The other option would be for us to change user_data so that it forces server re-creation on update.

I believe re-creation would be the Terraform native approach.
The problem with the current setup is that it blocks most Terraform actions(plan, apply, plan -replace, plan -destroy, ...).


Failed to build LoadBalancer configuration due to unable to fetch subnets.

kubebuilder/controller "msg"="Reconciler error" "error"="failed to build LoadBalancer configuration due to unable to fetch subnets. Error: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors"  "controller"="alb-ingress-controller"
Am unable to get over this issue, any pointers please?

I have the subnets tagged by eksctl
article reference awsdocs

NoCredentialProviders: no valid providers in chain you didn't setup aws permissions properly.
https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html

Feel free to reopen this if you still encounter issues with the permissions.


Project / Parent Relationship bug?

Seems like the "project" parent task feature doesn't work?
I have tried the following:
project task:
{
start: new Date(2022, 0, 3),
end: new Date(2022, 0, 30),
name: Phase 1",
id: "big",
type: "project",
progress: 100,
dependencies: [],
styles: { progressColor: "blue", progressSelectedColor: "#ff9e0d", barCornerRadius: 5 },
hideChildren: true,
},
child task:
{
start: new Date(2022, 0, 3),
end: new Date(2022, 0, 17),
name: "Task 1",
id: 1,
type: "task",
progress: 100,
dependencies: [],
styles: { progressColor: "blue", progressSelectedColor: "#ff9e0d", barCornerRadius: 5 },
project: "big",
},
but the Gantt isn't recognizing that Project Name: phase 1, Id: big has child Name: Task 1, if i have set this up wrong please let me know but as far as i can tell it's just not working as it should based on what i have seen in the documentation / example

eventually sorted it, however the instructions could do with enhancing there is quite a bit im having to dig way past the current scope of the instructions to figure out that the demo shows off

@Freem11 How did you solve this ? I have the same issue.


Virus in contailner image

Hi
PHP 7.4 fpm image has a kdevtmpfsi virus miner and It utilized 100% of all CPUs and RAM...
I think it is not a coincidence problem!

Thanks for reporting! I'm investigating the issue.

Hi @rezakho, I'm trying to reproduce the problem, but I need a little more context, please.
I've attempted to create (without launching) containers from chialab/php:7.4, chialab/php:7.4-apache, and chialab/php:7.4-fpm images, then inspecting the container's filesystem looking for a binary named kdevtmpfsi, but found nothing:
$ docker create --name test chialab/php:7.4
0f4ff4f8987061f3fb2098608b230260f095c382dfc5e410f3f56e9af11dfc51
$ docker export test | tar t | grep kdevtmpfsi
$ docker rm -fv test
test

$ docker create --name test chialab/php:7.4-apache
28ffc81e0ded750c0ce1622508b5c4e3ab8111dc87e1802b085e2af56791ed59
$ docker export test | tar t | grep kdevtmpfsi
$ docker rm -fv test
test

$ docker create --name test chialab/php:7.4-fpm
91b78221d265100cee016b2ff968f03d997f289780616c1c2a69bb3e06c9c83c
$ docker export test | tar t | grep kdevtmpfsi
$ docker rm -fv test
test

I understand you're using chialab/php:7.4 image, right? Which flavour are you using? I assume it is chialab/php:7.4-apache, am I correct?
I've also started a container with the latest chialab/php:7.4-apache image, then inspected the running processes with docker top, and I'm unable to see anything weird:

Can you please help me with some more steps so that I can reproduce the problem? If possible, please share your Dockerfile and build scripts so that we can investigate the issue furtherly — either by including it in the issue, or you can email it to me at the email address in my GitHub profile if there is anything you don't want to share publicly.

Seems it is a nginx + php-fpm vulnerability!!!
[https://www.trendmicro.com/vinfo/in/security/news/vulnerabilities-and-exploits/php-fpm-vulnerability-cve-2019-11043-can-lead-to-remote-code-execution-in-nginx-web-servers]

The vulnerability you're pointing out was patched as of October 2019:

Administrators and IT teams using NGINX with PHP-FPM are recommended to update their PHP to their latest or stable versions (7.2.24 or 7.3.11), which have addressed the vulnerability

Right now, our images run PHP 7.2.34 and 7.3.24. PHP 7.4 was released after that vulnerability was fixed, and we must assume that every stable release of 7.4.x was unaffected, unless the bug was somehow reintroduced…

No! Unfortunately this vulnerability exists in php 7+ but not related directly to your repo or docker:
https://github.com/neex/phuip-fpizdam

6.This exploit works only for PHP 7+, but the bug itself is present in earlier versions (see below).

Hello @rezakho,
our images do not include nginx out of the box, and the vulnerability is patched in PHP 7.4, 7.3.11, 7.2.24 and 7.1.33 (see the changelog) so the only versions that could be susceptible are 5.6 and 7.0.
Could you share more information on you setup? What flavor are you using, php, php-apache or php-fpm? What makes you think that the problem is the vulnerability you linked?

I'm closing this issue, as we're unable to reproduce the scenario described starting from the base image.
Feel free to open a new issue, but please include detailed steps to reproduce the problem with the chialab/php image — keep in mind that we can't fix PHP's own vulnerabilities in this repository apart from installing and possibly pre-configuring it with sane, reasonable defaults.


fix(codegen): remove redundant semicolon in PropertyDefinition

Current dependencies on/for this PR:

#2511  👈
main

This stack of pull requests is managed by Graphite.


[ISSUE-635] Support WBT disabling

Purpose
Resolves #635
Changes described in proposal:

Add watcher for Wbt configmap to update it every 60 sec
On NodeStage: try to read and edit wbt_lat_usec
On NodeUnstage: restore default value for wbt_lat_usec

PR checklist

[x] Add link to the issue
[x] Choose Project
[x] Choose PR label
[x] New unit tests added
[x] Modified code has meaningful comments
[ ] All TODOs are linked with the issues
[ ] All comments are resolved

Testing
Provide test details

@safronovD this fix must also be merged to release-1.0 branch.


remove_address_record function is called two times per Address during Processor shutdown

Observed behavior
remove_address_record is called two times for the same Address
Steps to reproduce
use core::time::Duration;
use ockam_core::async_trait;
use ockam_core::{Processor, Result};
use ockam_node::Context;

#[ockam_macros::test]
async fn starting_processor(ctx: &mut Context) -> Result<()> {
    ctx.start_processor("dummy_processor", DummyProcessor)
        .await?;
    ctx.sleep(Duration::from_secs(1)).await;
    ctx.stop_processor("dummy_processor").await?;

    ctx.sleep(Duration::from_secs(1)).await;
    ctx.stop().await
}

struct DummyProcessor;

#[async_trait]
impl Processor for DummyProcessor {
    type Context = Context;

    async fn process(&mut self, _ctx: &mut Context) -> Result<bool> {
        Ok(true)
    }
}

Desired behavior
Each address should in the system should have clear lifecycle. It should be removed only once
Ockam Version
0.116.0

Hello can I work for it?
But I need to know where is relevant code located
@SanjoDeundiak

Sure @danik292 , go ahead! Most of the source code you'll need to touch must be in ockam_node/src/router directory. The tricky part of that issue is that access to the address map happens from different places/different source files/at different point of an address lifecycle. So could take a while to untangle

Thank you☺
@SanjoDeundiak


Close #13 - better lockstep API

Assuming:
class Foo:
    @lockstep
    def process(self):
        # ...

foo = Foo()

foo.process() - run the entire process
with foo.process.lockstep() as process: - step into the process
yield from foo.process.lockstep() - for lockstep nesting

Not 100% sure about the yield from API. Maybe now that we are explicitly running the lockstep, we should also give the yield from it's own API? Something like yield from foo.process.delegate()?

BTW: please let me merge this. I want to merge it and immediately update testlight - otherwise it'll break the installation.

Shouldn't it be
with foo.process.lockstep() as process:
    yield from process

?

Currently we allow yielding from the context manager itself.
Yielding from the process object is easy to implement, but I generally prefer APIs that allow their user to avoid binding new names.
I would like to have a separate method for yielding the steps, because having it under .lockstep() forces me to use _LockstepContextManagerWrapper instead of a regular contextmanager, but I struggle to find a good name for it:

yield from foo.process.unwrap_lockstep()?
yield from foo.process.nested_lockstep()?
yield from foo.process.lockstep_steps()?

?


Flutter web not working on iOS web having OS version less than 14.1

When testing the Flutter web app on iOS safari with OS 13.4 and 13.7, it is not working.
We tested it on OS 15.2 and 15.4, it is working fine.
Sample: https://github.com/deepak786/ios_web_issue
web build: https://deepak786.github.io/ios_web_issue_build
It also works fine on the Desktop browsers and Android web browsers.
I inspected the console using the Safari web inspector, it showed the following error:
[Error] SyntaxError: Unexpected token '='. Expected an opening '(' before a method's parameter list.
	(anonymous function) (flutter.js:23)
[Error] ReferenceError: Can't find variable: _flutter
	(anonymous function) (ios_web_issue_build:80)

The same issue happens with Flutter Gallery website.
When I searched about the above error, it seems that flutter.js is declaring the variables in a way that is not supported by iOS 13.
This feature is added in iOS 14.1.
Ref: https://stackoverflow.com/questions/60026651/safari-unexpected-token-expected-an-opening-before-a-methods-paramet
Video showing the error:
https://drive.google.com/file/d/1nFupkQ2ZqHZnXDaBRuP9CO8GMXdkVCPb/view?usp=sharing

flutter doctor -v
[✓] Flutter (Channel stable, 3.0.0, on macOS 12.3.1 21E258 darwin-x64, locale
    en-US)
    • Flutter version 3.0.0 at /Library/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision ee4e09cce0 (2 weeks ago), 2022-05-09 16:45:18 -0700
    • Engine revision d1b9a6938a
    • Dart version 2.17.0
    • DevTools version 2.12.2

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/appleapple/Library/Android/sdk
    • Platform android-31, build-tools 30.0.2
    • ANDROID_HOME = /Users/appleapple/Library/Android/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7590822)

[✓] IntelliJ IDEA Community Edition (version 2019.3.3)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.66.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.40.0

[✓] Connected device (4 available)
    • Deepak’s iPhone (mobile) • 00008101-00117D690A80801E            • ios
      • iOS 15.4.1 19E258
    • iPhone 11 Pro (mobile)   • 56817DBD-D6E9-4308-84F2-65A261C2095C • ios
      • com.apple.CoreSimulator.SimRuntime.iOS-13-7 (simulator)
    • macOS (desktop)          • macos                                •
      darwin-x64     • macOS 12.3.1 21E258 darwin-x64
    • Chrome (web)             • chrome                               •
      web-javascript • Google Chrome 101.0.4951.64

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

Hi @deepak786, Thanks for filing the issue. I am able to reproduce the issue on stable and the master channel.  Flutter web app does not load on ios 13.
Flutter gallery

flutter counter app (master)

flutter doctor -v (mac)
[✓] Flutter (Channel stable, 3.0.1, on macOS 12.3 21E230 darwin-arm, locale en-IN)
    • Flutter version 3.0.1 at /Users/mahesh/Documents/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision fb57da5f94 (3 days ago), 2022-05-19 15:50:29 -0700
    • Engine revision caaafc5604
    • Dart version 2.17.1
    • DevTools version 2.12.2

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-32, build-tools 31.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
    • All Android licenses accepted.

[!] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! CocoaPods 1.10.2 out of date (1.11.0 is recommended).
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin
        usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To upgrade see https://guides.cocoapods.org/using/getting-started.html#installation for instructions.

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 61.2.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.66.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.40.0

[✓] Connected device (4 available)
    • Redmi K20 Pro (mobile) • <IP_ADDRESS>:5555                     • android-arm64  • Android 11 (API 30)
    • iPhone 12 Pro (mobile) • 19FD0231-BFF0-441D-B584-AD94C4084525 • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-15-2 (simulator)
    • macOS (desktop)        • macos                                • darwin-arm64   • macOS 12.3 21E230 darwin-arm
    • Chrome (web)           • chrome                               • web-javascript • Google Chrome 101.0.4951.64

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[✓] Flutter (Channel master, 3.1.0-0.0.pre.854, on macOS 12.3 21E230 darwin-arm, locale en-IN)
    • Flutter version 3.1.0-0.0.pre.854 at /Users/mahesh/Documents/flutter_master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision ec20ea80ad (35 hours ago), 2022-05-21 16:28:06 -0400
    • Engine revision 1bfe551d6b
    • Dart version 2.18.0 (build 2.18.0-142.0.dev)
    • DevTools version 2.13.1

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-32, build-tools 31.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)
    • All Android licenses accepted.

[!] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! CocoaPods 1.10.2 out of date (1.11.0 is recommended).
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin
        usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To upgrade see https://guides.cocoapods.org/using/getting-started.html#installation for instructions.

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.11+0-b60-7772763)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 61.2.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.66.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.40.0

[✓] Connected device (4 available)
    • Redmi K20 Pro (mobile) • <IP_ADDRESS>:5555                     • android-arm64  • Android 11 (API 30)
    • iPhone 12 Pro (mobile) • 19FD0231-BFF0-441D-B584-AD94C4084525 • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-15-2 (simulator)
    • macOS (desktop)        • macos                                • darwin-arm64   • macOS 12.3 21E230 darwin-arm
    • Chrome (web)           • chrome                               • web-javascript • Google Chrome 101.0.4951.64

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

I also tested that it works on 14.1 but not before that.

@yjbanov

Users affected by this can sidestep the issue by using the old index.html syntax (by injecting the main.dart.js file directly, instead of using flutter.js).
I'm working on a fix right now. We need to use the constructor function instead of using public field declarations, as pointed out by @deepak786!
Thanks for the report!

I'm currently testing a fix for this in the iOS simulator, as suggested by @maheshmnj.

So the fix for this has landed. There are some workarounds that you can do, @deepak786:

Switch flutter to master, and do a flutter clean and flutter build web again. The flutter.js file will use the new syntax (look for constructor() inside of FlutterLoader)
If you want to stay in your current channel, you can. Just create a new, separate, flutter web app, and do a flutter build web there.

Inside build/web you'll find the new flutter.js that you can use in your current app. (This is more cumbersome, because every time you do a flutter clean you'll have to restore the flutter.js file coming from master to your old app)

Apologies for the breakage, and thanks for reporting!

@ditman Thank you for the update.
Switching to master would be a solution for now but this could break other functionalities in the app.
Any estimate about when this will land in a stable channel.

@deepak786 Fixes on the master take approximately 4 months to stabilize and land on stable you can read more about the release channels here https://github.com/flutter/flutter/wiki/Flutter-build-release-channels


Add logging_client_events

Notes
This request is not associated with any specifific function
This is a periodical request of instagram app at any version which contains a bunch of logs of users interaction with the app and a validation code.
I've looked on github many instagram apis and so far nobody was able (or interested) in reverse engineer it. I do have a little suspect that it can be related to other problems i've in my project.
Does anyoe have the skills to reverse it?
i'm pasting here a sample request of what im talking about
/logging_client_events HTTP/1.1
X-IG-Connection-Type: WIFI
X-IG-Capabilities: 3brTvw==
X-IG-App-ID:<PHONE_NUMBER>52427
User-Agent: Instagram <IP_ADDRESS>.129 Android (26/8.0.0; 480dpi; 1080x1920; samsung; SM-A520F; a5y17lte; samsungexynos7880; pt_BR; 161478664)
Accept-Language: pt-BR, en-US
Content-Type: multipart/form-data; boundary=WclRaGE_-iaawkAGZnprHXDKaqrJ8r
Accept-Encoding: gzip, deflate
Host: graph.instagram.com
X-FB-HTTP-Engine: Liger
Connection: keep-alive
Content-Length: 1576

--WclRaGE_-iaawkAGZnprHXDKaqrJ8r
Content-Disposition: form-data; name="access_token"

567067343352427|f249176f09e26ce54212b472dbab8fa8
--WclRaGE_-iaawkAGZnprHXDKaqrJ8r
Content-Disposition: form-data; name="format"

json
--WclRaGE_-iaawkAGZnprHXDKaqrJ8r
Content-Disposition: form-data; name="cmsg"; filename="ab67f151-d0ae-4c63-a45e-3fa8eedf56ab_1_zero.batch.gz"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

?T?n?6?=GqQ?Mb??kg`???.
??(?%?Z??A????i&?>?E?z??????%h???<d]??P?#?	?h??A ???9D???^?LT}?A
#?)??=S{??!a?`(a?CE)#?0?<
S?<N??r!?,?9??#I)?9?d???	NIH2???dy??lU?j[
12?,???*?RWRFD?8?\?,'T&.??9?Jw???q???
#;U????7	b?c`?\o=#????????TZ????7?!;\??T?T?WWm'??,????F?4?X?O0$?rF ??8?iK???#?k^x/?+OW+??=??q
?6?????z	JQ.????8b?Q?]???,??%?????]???????b??b?(?`F?SiW8-|3?-???r???r?:N?Y??ls?'H?`??7r^5????x?xV(?[?7ll?CR??hwI??????8?y1~?&????/???o?Fl.d?
;????c????n?????*??j???6G1?q`?2-?m?N<-_rT??~????G???_^???F???+g.u+????\?V?'v?j?QF?V?w????E?i+?c??9?\??u?????'???k??"? ?	F???K???+?`
?F:N8?????!??5??\??n????t??u
??|B???"??n???0}S1g?y???K7?q?????n??h?j???]er?O???t*??????Kr??4c?w?T?????????g????!!?QFd?????;?,?X?B/???b?P@ "???????????3?}6????O??^?v??5?-??dy?Lff??n?D?Fk?X-??\?????~4m?n\l??G????p??ek??q?C[??,0v?q???J(/???W
--WclRaGE_-iaawkAGZnprHXDKaqrJ8r
Content-Disposition: form-data; name="sent_time"

1566875183.401
--WclRaGE_-iaawkAGZnprHXDKaqrJ8r
Content-Disposition: form-data; name="cmethod"

deflate
--WclRaGE_-iaawkAGZnprHXDKaqrJ8r--

Content-Disposition: form-data; name="cmsg"; filename="ab67f151-d0ae-4c63-a45e-3fa8eedf56ab_1_zero.batch.gz"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
Content-Disposition: form-data; name="cmethod"
deflate

The contetnt is deflated (the input starts with 0x78). You have to infalte it.
I can't reproduce the multipart/form-data, for me it's only urlencoded. If you could state what you did in order to get this request it would be helpful. Maybe it's the version (you're using v100, I'm using v136). You can also send the data base64 compressed so it's decompressable without data-loss. (In Burp you can use [right-click] > Send to decoder and en-/decode it).

I can't reproduce the multipart/form-data, for me it's only urlencoded.
As the endpoint suggests, this is some sort of user log... it is send periodicaly from the instagram app without the need of any specific human interaction.

if the log is small the app simple urlencode a normal json as you said... if the log is larger then instagram compresses it and send this form...
In order to get this multpart form simple roam around with the app for 10 or 15 minutes... like some photos, search something... do any normal interaction with the app and then you will get it....
i do believe that the data thats is deflated is the very same urlencoded... THAT CONTAINS SOME SORT OF DATA VALIDATION CRYPTOGRAPHY which im not able to replicate, and i would like very much to know if anyone is

how do you calculate checksum?

I know the data is deflated, this is not the concern of the question, that was just an example
you got very much same response as me for small json.
"config_checksum":"qs|c=51a78d39a32ed5fed29c6b3e55fc1beb&ts=1566858438&qv=v1&s=1595da72d9f415d8a7b8eff881b0b0cf",
   "data":[

   ]

the question
HOW TO CALCULATE CHECKSUM?

The config checksum is basically from the last response. I don't know why you're freaking out about this.

1- im not freaking out... i have a question and i came to a public space ask for help... i thought the github was for that
2- the goal of any "instagram private api" out there is to behave like real instagram app... NONE of those apis do implement the request to this endpoint and it exists since instagram v35 so to instagram is pretty easy to identify who is a bot or not... "device_id" too long without sending log...
3- did you try do what you said? because if i try to post the request you proposed or any other without a real checksum i get the response:
{
   "error":{
      "message":"Invalid OAuth access token signature.",
      "type":"OAuthException",
      "code":190,
      "fbtrace_id":"AfI6ADiLg3WjKuQAMwWQ6yI"
   }
}

that is clear instagram reject it.
im sorry if i didnt started this issue with a clear question and i missused the therm cryptography for checksum... im doing lots of stuff in paralel. but if you are able to help with this subject, please dont it

I will explain my situation and maybe you will understand why i want this:
i have an android app that uses a java version of mgp api (which i personally translated from php to java during 2 years)...
The users of my app often are caught with the famous "action_blocked" message, although the app is not intended for spam, it does follow, comment and like stuff...
the point is at the exact same moment the users receive the "action_blocked" message in the app, if they go to the original instagram app they are able to follow, like and comment.
this is running under real android devices so this is not a ip/proxy related problem...
instagram is flagging users device_id as spam... so any other device_id is able to comment under same ip/account.
well this is happening over a long time and i'm out of new ideas about how to handle it. one last shot is try to see if replacating this request it will make instagram more permisive.

did you try do what you said?

Kind of. I didn't have a token, so it's null and I received 200 OK with this response:
{
  "checksum": "",
  "config": "",
  "config_owner_id": "",
  "app_data": "{}"
}

But looking at the requests, the last checksum that was sent is used in the next requests.
This is also visible in the code:
// v.133
// on save:
public final void 0Uj.BGe(InputStream p0,boolean p1)	//method@16aa
{
   // ...
   if (p0) {
      0oW v0 = this.A00.createParser(p0);
      v0.null_nextToken();
      if ((v6 = 0XI.parseFromJson(v0)) && (v0_1 = v6.checksum) && !v0_1.isEmpty()) {
         String checksum = v6.checksum;
         Map mainConfigHahmap = v6.mainConfigHashmap;
         Set blacklist = v6.blacklist;
         String configOwnerId = v6.configOwnerId;
         0XX v4 = 0XW.A01(0XW.A00());
         13c.A02(v4.A02(configOwnerId).A07());
         v4.A02(configOwnerId).A07().A01 = 1;
         String targetKey = "__config_checksum__";
         13c.A02(v4.A02(configOwnerId).A07());
         if (!checksum) {
            13d.notNullOrThrow(targetKey);
            v4.A02(configOwnerId).A07().A00.put(targetKey, 13d.emptyObj);
         }else {
            13d.notNullOrThrow(targetKey);
            v4.A02(configOwnerId).A07().A00.put(targetKey, checksum);
         }
         // ....
}
// on load:
public final String 0XX.A03(String p0)	//method@1862
{
   13d config = this.getSomePigionSamplingPolicy(p0);
   String ret = null;
   13d.A01(config);
   Object v1 = config.A02;
   _monitor_enter(v1);
   if ((String checkSum = config.A04.get("__config_checksum__"))) {
      ret = checkSum;
   }
   _monitor_exit(v1);
   return ret;
}

i think checksum comes in the response of the first request with a new device_id (~100k)

In the latest uptade(<IP_ADDRESS>.110), no more deflated json in logging_client_events api, What do you think why instagram did this?

@mmadymarov The Http requests are already compressed. Then there's no need to add another layer of compression.

I am working replicating this right now, if anyone's interested in synchronising work, we can connect via telegram or facebook messenger (same username as here)

I am working replicating this right now, if anyone's interested in synchronising work, we can connect via telegram or facebook messenger (same username as here)

do you have a repo? are you pushing here? i'm interested but i have zero progress on this

@bolds07 no, no repo yet. Currently have few examples collected, probably it'll help to collect more

@bolds07 no, no repo yet. Currently have few examples collected, probably it'll help to collect more
Edit: this repo looks interesting, though: https://github.com/DamianoMagrini/instaverse/blob/857273e475a868585c91c07f626cbba5553ccb73/reversed-api/src/instagram/9568347_events.ts

This is for the web api.

1- im not freaking out... i have a question and i came to a public space ask for help... i thought the github was for that
2- the goal of any "instagram private api" out there is to behave like real instagram app... NONE of those apis do implement the request to this endpoint and it exists since instagram v35 so to instagram is pretty easy to identify who is a bot or not... "device_id" too long without sending log...
3- did you try do what you said? because if i try to post the request you proposed or any other without a real checksum i get the response:
{
   "error":{
      "message":"Invalid OAuth access token signature.",
      "type":"OAuthException",
      "code":190,
      "fbtrace_id":"AfI6ADiLg3WjKuQAMwWQ6yI"
   }
}

that is clear instagram reject it.
im sorry if i didnt started this issue with a clear question and i missused the therm cryptography for checksum... im doing lots of stuff in paralel. but if you are able to help with this subject, please dont it

Make sure you send your data to URL without strain slash! That was my issue!
Should be https://graph.instagram.com/logging_client_events and not https://graph.instagram.com/logging_client_events/

@350d @Nerixyz @caffeinum
do you guys have anything on it... i'm starting to map the events and would be good to join efforts

Content-Disposition: form-data; name="cmsg"; filename="*.gz"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
Content-Disposition: form-data; name="cmethod"
deflate

The contetnt is deflated (the input starts with 0x78). You have to infalte it.
I can't reproduce the multipart/form-data, for me it's only urlencoded. If you could state what you did in order to get this request it would be helpful. Maybe it's the version (you're using v100, I'm using v136). You can also send the data base64 compressed so it's decompressable without data-loss. (In Burp you can use [right-click] > Send to decoder and en-/decode it).

How do you inflate the content? I've tried to decode it using burpsuit but nothing is happening. I've successfully decoded some other responses using it though

burp doesn't inflate, it only deflates... (at least the free version)
you must use burp and encode it as base64, then create  a program which reads base64, decode then deflate... there are a few sites there that do it but is easier simple make your own
you cant simple control c + control + v because the inflated code has to much non ascii characteres so the copy paste process lose bunch of information...

I know the data is deflated, this is not the concern of the question
"config_checksum":"qs|c=51a78d39a32ed5fed29c6b3e55fc1beb&ts=1566858438&qv=v1&s=1595da72d9f415d8a7b8eff881b0b0cf",
   "data":[

   ]

the question
How to calculate checksum??
this qs field name reminds me of another checksum field that is found during the REGISTRATION REQUEST, which i basically dont know anybody that was able to reverse engineer it.

I know the data is deflated, this is not the concern of the question
"config_checksum":"qs|c=51a78d39a32ed5fed29c6b3e55fc1beb&ts=1566858438&qv=v1&s=1595da72d9f415d8a7b8eff881b0b0cf",
   "data":[

   ]

the question
How to calculate checksum??
this qs field name reminds me of another checksum field that is found during the REGISTRATION REQUEST, which i basically dont know anybody that was able to reverse engineer it.

If you still need it pm me

@brutalsuperman, this is github... there is no such a thing as "PM me"
either you have a solution for that and you can share here for the actual and future watchers of this topic
if you somehow are looking for sell something, no thanks

Qs field is for their native library called libquicksand. It's not soo hard
to figure out
On Fri, 11 Jun 2021 at 15:08, Bolds @.***> wrote:

@brutalsuperman https://github.com/brutalsuperman, this is github...
there is no such a thing as "PM me"
either you have a solution for that and you can share here for the actual
and future watchers of this topic
if you somehow are looking for sell something, no thanks
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dilame/instagram-private-api/issues/1140#issuecomment-859608404,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMNLMBKEWCNHZCLHYZFQAULTSIKG5ANCNFSM4MIYMSYQ
.

Anyone knows, if instagram currently uses this for bot-detection?

https://github.com/DrunkSkipper/qs_stamp/blob/master/python/quicksand.py
found this implementation in python, anyone knows how to use this?

They use safetynet now. Looking into quicksand is useless
On Thursday, 5 August 2021, crylonblue @.***> wrote:

https://github.com/DrunkSkipper/qs_stamp/blob/master/python/quicksand.py
found this implementation in python, anyone knows how to use this?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dilame/instagram-private-api/issues/1140#issuecomment-893655388,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMNLMBJQJPZT5WDQE7PP2VLT3LEBJANCNFSM4MIYMSYQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

I dont think they do so on ios. From what i saw on the ios app so far (<IP_ADDRESS>.119). They have to channels for logging_client_events "regular" and "micro-batch". The regular one does not have the config checksum, the micro-batch has.

To maybe solve this issue, the channel "regular" has no qs token. You can just send your events there.
A simple implementation in python since, i am not that good in typescript.
`
import base64
import zlib
import requests
def logging_client_events(message_dict):
data = {
"format": "json",
"access_token": constants.APP_ID + "|" + constants.ACCESS_TOKEN,
"sent_time": self.api.get_client_time(),
"system_uptime": self.get_system_uptime(),
"compressed": "1",
"messsage": base64.b64encode(zlib.compress(bytes(json.dumps(message_dict).encode()), 1)).decode()
}
log_request = self.session.post("https://graph.instagram.com/logging_client_events", data=data)

return log_request

`

@crylonblue what data we should send logging_client_event ? i know that json data but how to do this and stay undetected ?

I was able to test for video views, #1624

Which data = [] did you use in the message to achieve this. I tried to replicate it, and the response is as always the same but the view count stays the same.
I used the following events:
instagram_organic_impression
instagram_organic_sub_impression
instagram_video_viewability_changed
video_should_start
video_buffering_started
video_fetched
video_format_changed
video_started_playing
video_buffering_finished
video_played_time
video_exited
instagram_organic_viewed_impression
instagram_organic_time_spent
instagram_organic_vpvd_imp

I think some of the events get sent via MQTT instead of HTTP now. (But not sure)

ACCESS_TOKEN

To maybe solve this issue, the channel "regular" has no qs token. You can just send your events there. A simple implementation in python since, i am not that good in typescript.
import base64
import zlib
import requests

def logging_client_events(message_dict):
    data = {
        "format": "json",
        "access_token": constants.APP_ID + "|" + constants.ACCESS_TOKEN,
        "sent_time": self.api.get_client_time(),
        "system_uptime": self.get_system_uptime(),
        "compressed": "1",
        "messsage": base64.b64encode(zlib.compress(bytes(json.dumps(message_dict).encode()), 1)).decode()
    }

    log_request = self.session.post("https://graph.instagram.com/logging_client_events", data=data)

    return log_request

So i think its pretty much solved, now all you have todo is to add logging_client_events in each api_call, that has a event involved for example "ig_main_feed_request_succeeded" etc.

hi, where do you get the constants.ACCESS_TOKEN? i don't find it even in burpsuite, is it account or hardware related? can i just randomize it? thanks.

ACCESS_TOKEN

To maybe solve this issue, the channel "regular" has no qs token. You can just send your events there. A simple implementation in python since, i am not that good in typescript.
import base64
import zlib
import requests

def logging_client_events(message_dict):
    data = {
        "format": "json",
        "access_token": constants.APP_ID + "|" + constants.ACCESS_TOKEN,
        "sent_time": self.api.get_client_time(),
        "system_uptime": self.get_system_uptime(),
        "compressed": "1",
        "messsage": base64.b64encode(zlib.compress(bytes(json.dumps(message_dict).encode()), 1)).decode()
    }

    log_request = self.session.post("https://graph.instagram.com/logging_client_events", data=data)

    return log_request

So i think its pretty much solved, now all you have todo is to add logging_client_events in each api_call, that has a event involved for example "ig_main_feed_request_succeeded" etc.

hi, where do you get the constants.ACCESS_TOKEN? i don't find it even in burpsuite, is it account or hardware related? can i just randomize it? thanks.

ACCESS_TOKEN is a fixed key for Instagram, you can hardcode both app_id and access_token in your application
567067343352427|f249176f09e26ce54212b472dbab8fa8

constants.APP_ID =<PHONE_NUMBER>52427
constants.ACCESS_TOKEN = f249176f09e26ce54212b472dbab8fa8

I was able to test for video views, #1624

Which data = [] did you use in the message to achieve this. I tried to replicate it, and the response is as always the same but the view count also stays the same.
I used the following events: instagram_organic_impression instagram_organic_sub_impression instagram_video_viewability_changed video_should_start video_buffering_started video_fetched video_format_changed video_started_playing video_buffering_finished video_played_time video_exited instagram_organic_viewed_impression instagram_organic_time_spent instagram_organic_vpvd_imp

Have you had any success today?


Connection issues with 1.4 beta

After 1.3 worked great with my lamps, I forgot to disable beta branches and ended up with 1.4b4 -- which isn't working anymore. Downgrading to 1.3 solved my issue, but maybe these error messages can help to avoid bringing new issues into 1.4 stable.
After HA startup, the integration was not showing the lamps. Looking to the logs I found the following messages repeated every 30 seconds:
WARNING (AwoxMeshCommands-U7GFU0NF) [custom_components.awox.awox_mesh] [U7GFU0NF] Command failed, retry 1
WARNING (AwoxMeshCommands-U7GFU0NF) [custom_components.awox.awox_mesh] [U7GFU0NF] Command failed, retry 2
WARNING (AwoxMeshCommands-U7GFU0NF) [custom_components.awox.awox_mesh] [U7GFU0NF] Command failed, retry 3

As downgrading back to 1.3 solved this issue I did not search for the cause of it, but please let me know if you need more information.

Could you maybe test latest beta? Reverted the use of bleak for now.

Hi,
I would weigh in this discussion (if I can help in any way please let me know, I updated today to the latest version 0.1.4b5 , no difference so far.)
Basically since the major updates of HA earlier this year (when the bluetooth lib changed), I have near permanent connections issues with the AWOX Mesh.
My logs show no errors upon rebooting/reload of HA or reload of AWOX (on RPI), all lights are found at those times and if the mesh is present it works fine.
Apart from the "Awox mesh becomes unavailable" (sensor.awox_mesh), I can only notice it when I switch a light on/off and it then can take up to 1min until the light reacts or not at all.
There is no Issue when using the official Awox App on my phone (except loading time, about ~10-15s).
Or is the App the culprit ?
Although I only use it when HA does not react and I close the App after to avoid probable access fight.
(The app shows no firmware updates for any light, so I gather being up to date as best possible)
There are 2 disabled entities (no enabling possible to my knowledge)
sensor.awox_mesh_last_connection
sensor.awox_mesh_last_rssi_check
I am considering external issues which might impact the general low-power bluetooth connection, but except a 2 older devices which still use 2.4Ghz Wifi there is nothing apparent & unrelated to laundry machine running or alike.
All lights I use are max. 3-4m apart.
(then again, it worked rather smoothly before the BT lib changes/updates)
I am not excluding user error on my part, but I am puzzled to what else I might do ?
Please let me know what / if I can help out more towards a more stable mesh.

@wickedgalen

There are 2 disabled entities (no enabling possible to my knowledge)
sensor.awox_mesh_last_connection
sensor.awox_mesh_last_rssi_check

These are disabled by default and can be enabled on the settings tab of each sensor in the advanced settings area
Regarding the connectivity issues. Do you have the HA Bluetooth integration enabled? That gives my some issues. Should be fixed in latest HA patch release, didn't test yet.
I disabled the HA Bluetooth integration and that improved the connection a lot.
Could you post a debug log here maybe there is some info I can get out of it.

Could you maybe test latest beta? Reverted the use of bleak for now.

Sadly the issue remains with 1.4b5. I tried restarting HA and my Raspberry without success. A rollback to 1.3 made the lights available again directly after the restart of HA.
For me 1.3 is the most stable version I've ever seen with this extension, I haven't had a single issue in the last two weeks.

Thank you for your quick reply.
I started with disabling this + reboot : bad idea, BT completly down.

Then I notices more updates : Core, Hacs (must have missed them earlier) + reboot.
iBeacon appeared & found a bunch of jpunzadi devices.
Device Tracker (although for all of them) :

&
Estimated Distance

FYI: The lights are installed on walls/ceilings and I sat on my desk during the time shown ... If it be off by 1 m I 'd be ok with it ... but it seems the time/space continuum is off here !!?!? The walls did not move ;).
Anyhow need to read up on it, too late for today, I'll get back in a few days when it sunk in and could do some testing.

I had the same that all Bluetooth devices got a RSSI of 0 when HA was running for a while. It was caused by the HA Bluetooth integration that is constantly running a scan and restarting the Bluetooth controller on failure.
I disabled the HA Bluetooth integration and now the Bluetooth controller is usable again.

Hi back,
probably unrelated but, due to adding another integration and by that, multiple reboots with browser cache clears seem to have fixed all issues / let all updates take propper effect.
Running now:
Awox Smart Connect v0.1.4b5
with

(No settings whatsoever changed after install)

The lights now work ON CLICK =)
No changes on my dasboard needed, previous buttons work.
I'll be perma-testing it anyhow, but the whole give a very good stability impression.
The distancing has very nice accuracy and reactiveness.

I'm just unsure about what to read out of this now?
sensor.awox_mesh
shows either unavailable or the power relay, long before (the power plug use) it seemed to me as the awox network status or is this a misinterpretation of mine ?

Thanks a lot so far for your great work =)
& the HA BT guys too.


No.11 - Test case: TC443 (owncloud instance bad gateway)

vers - v1.4.1-rc2
Deployed, but at the same time 502 error  when clicking on the link

is that still the case?

Ive deleted  Contracts List and now its work

yeeep , now it work


SSL Certificate Pinning using flutter

I want to add a trusted certificate to my HttpClient, but this is not very efficient. Certificates expire so I want to pin the public key. Is there a way to do this in dart/flutter?

We publish a plugin : https://pub.dartlang.org/packages/ssl_pinning_plugin to check pinning with fingerprint (SHA-1).

Is this available yet?  Documentation?

That would be great.  So you have a new release for the 0.5.7 beta?   It doesn't seem to be compatible with the latest flutter beta. getting error:
The current Flutter SDK version is 0.5.7.
Because ssl_pinning_plugin requires Flutter SDK version ^0.4.4, version solving failed.

I guess this should better be asked in dart-lang/sdk because it is about dart:io

https://github.com/dart-lang/sdk/issues/33115 mentions that this is now fixed in Flutter
Can anyone confirm?

yes it works

Great! Thanks for the update @long1eu 👍

Well, as I can see, only sha1 is supported. This is a shame, sha1 is very weak. Any thoughts to add sha256? Sha1 is not even worth

@giaur500 please create a feature request in https://github.com/dart-lang/sdk/issues

@zoechi @mit-mit  thanks for certficate pinning support. Can the SecurityContext validate a public key instead of a certificate? This is more future proof as certificate pinning comes with issues of certificate expiry. or can X509Certificate include public key sha256 signature. Since most of native apps are build on public key pinning. This has become one major blockage for our client. Can we reopen the current issue and  https://github.com/dart-lang/sdk/issues/33115 . since there has been any activity on dart sdk issue from months.

Is this issue resolved for SHA2 certificates ?
I used webview in my application and it stopeed after i installed ssl on my server.

The ssl_pinning_plugin makes its own HTTPS request and checks the certificate.  It does not check certificates used by the app's real requests.  I see no way to accomplish that with the plugin.  This means that apps using the plugin are still vulnerable to MITM.


[BUG] Cluster status YELLOW after configuring Security Plugin in single-node clusters

What is the bug?
Upon activating the Security plugin in a single node cluster, the cluster status will always be YELLOW because of unassigned shards in 3 different indices:

.plugins-ml-config
.opensearch-sap-pre-packaged-rules-config
.opensearch-sap-log-types-config

How can one reproduce the bug?
Steps to reproduce the behavior:

Have single node cluster
Configure Security plugin per instructions in docs.
Observe Yellow cluster status after restarting
Query /_cat/shards API to see unassigned shards

What is the expected behavior?
Default sharding for above indices should be 1 primary / 0 replicas to account for single-node clusters. Or perhaps some degree of intelligent sharding based on cluster size.
What is your host/environment?

OS: Debian 11.7
Version: Opensearch 2.9
Plugins: Security

Do you have any screenshots?
$ curl https://example.org:9200/_cat/shards?v -u admin
Enter host password for user 'admin':
index                                     shard prirep state
.plugins-ml-config                        0     p      STARTED
.plugins-ml-config                        0     r      UNASSIGNED
.opensearch-observability                 0     p      STARTED
.opensearch-sap-pre-packaged-rules-config 0     p      STARTED
.opensearch-sap-pre-packaged-rules-config 0     r      UNASSIGNED
.opensearch-sap-log-types-config          0     p      STARTED
.opensearch-sap-log-types-config          0     r      UNASSIGNED
.opendistro_security                      0     p      STARTED

Do you have any additional context?
Related to the sentiment behind https://github.com/opensearch-project/anomaly-detection/issues/847, that plugins enabled on a single node Opensearch cluster should Just Work and maintain GREEN cluster status.

[Triage] Thank you for filing this issue @williamtrelawny. Looking at this issue, it seems like you have configured index configurations in such a way you have an unassigned shard. The Security plugin does not have any impact on sharding strategies.
Closing this issue.

I'm also having this issue with the default setup for running a single node in Docker. The instructions at https://hub.docker.com/r/opensearchproject/opensearch indicate that the following starts up a single node cluster:
$ docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" --name opensearch-node -d opensearchproject/opensearch:latest

However, the cluster remains in "yellow" status:
$ curl -sX GET "https://localhost:9200/_cluster/health" -ku admin:admin | jq -r .status
yellow

Many of the internal indices are being replicated but those replicas are not assigned to nodes, being a single node cluster.
$ curl https://localhost:9200/_cat/shards -ku admin:admin
.opensearch-observability                 0 p STARTED     0   208b <IP_ADDRESS> f9447c29352c
.plugins-ml-config                        0 p STARTED     1  3.8kb <IP_ADDRESS> f9447c29352c
.plugins-ml-config                        0 r UNASSIGNED
.opensearch-sap-pre-packaged-rules-config 0 p STARTED              <IP_ADDRESS> f9447c29352c
.opensearch-sap-pre-packaged-rules-config 0 r UNASSIGNED
.opensearch-sap-log-types-config          0 p STARTED              <IP_ADDRESS> f9447c29352c
.opensearch-sap-log-types-config          0 r UNASSIGNED
security-auditlog-2023.08.15              0 p STARTED     5 63.7kb <IP_ADDRESS> f9447c29352c
security-auditlog-2023.08.15              0 r UNASSIGNED
.opendistro_security                      0 p STARTED    10 74.8kb <IP_ADDRESS> f9447c29352c

My understanding of OpenSearch is insufficient for me to be able to configure these indices so that they are not replicated. Whilst I understand it may be possible to alter these indexes after the fact to use no replicas:
$ curl -XPUT https://localhost:9200/security-auditlog-2023.08.15/_settings -H 'Content-Type: application/json' -d '{"index":{"number_of_replicas":0}}' -ku admin:admin
{"acknowledged":true}

However, I have no idea what this will do to the cluster. This strategy also falls down when we try to modify the .plugins-ml-config index, in which I receive the following permissions related error:
{"error":{"root_cause":[{"type":"security_exception","reason":"no permissions for [] and User [name=admin, backend_roles=[admin], requestedTenant=null]"}],"type":"security_exception","reason":"no permissions for [] and User [name=admin, backend_roles=[admin], requestedTenant=null]"},"status":403}

Ideally, we are looking for a very minimal OpenSearch image we can run integration tests against in automated tests. It may be that I'm unaware of many of the configuration settings that can help, but I'm also struggling to find comprehensive documentation of these things.
Right now we are using an Elasticsearch image instead as a workaround.

[Triage] Thank you for filing this issue @williamtrelawny. Looking at this issue, it seems like you have configured index configurations in such a way you have an unassigned shard. The Security plugin does not have any impact on sharding strategies.
Closing this issue.

Please do not close this issue as it is not resolved. I have not made any changes to index parameters, sharding, replication, etc. at all. I have simply installed Openersearch and configured the Security plugin.
For whatever reason, 2 shards are created by default on all deployments of the Security Plugin, regardless of the number of nodes in the cluster.
If the Security Plugin does not affect sharding strategies, then why is it that other default indices not part of the Security Plugin do not have this issue?
Whether the root cause is within the Security Plugin code or not, the issue arises only after initializing the plugin, so from a procedural standpoint the issue does lie here.
And it definitely IS an issue if Opensearch w/ Security Plug-in does not work "out of the box" on a single node.

@scrawfor99: I think there was a misunderstanding when this issue was closed. The named indices are created with a replica configuration by default which makes it impossible to get them to green for a single-node setup. For anomaly detection, this was already acknowledged as requiring a change.

@dennisoelkers Thanks for calling this out - OpenSearch-Project wide we should have a consistent philosophy.  We should re-triage this issue with this context in mind

Per the cluster health documentation [link] it would suggest that the security plugin should allow for a configuration with no replicas.

OpenSearch expresses cluster health in three colors: green, yellow, and red. A green status means all primary shards and their replicas are allocated to nodes. A yellow status means all primary shards are allocated to nodes, but some replicas aren’t. A red status means at least one primary shard is not allocated to any

Personally, I have a bias towards multi-node clusters as I have experience with single machine sources of failure causing significant impact.  As much as I think all clusters should be multi-node, that is a preference and pushing that preference via the cluster health check is not transparent to operators of OpenSearch.

[Triage] Given the feedback on this issue, we will make an action item here to allow for 1 node clusters to be green. We will need to change it so that 1 node clusters can be set to 1 so that it is green.

I will start looking into this. Can you assign it to me @scrawfor99 ?

.opensearch-sap-log-types-config also breaks upgrading older versions of OpenSearch (<2.10) to latest OpenSearch on multi clusters When one of the data nodes are updated (rolling upgrade), it will apply the new index to the node. However the index can not be replicated to the "old nodes" which have not been upgraded. This breaks the upgrade.
We had to do this:
PUT /.opensearch-sap-log-types-config/_settings
{
  "index" : {
    "auto_expand_replicas" : "false",
    "number_of_replicas" : 0
  }
}

Also, setting auto_expand_replicas: true seems unnecessary as a default setting??
To let the status go to GREEN and continue the upgrades of all the data nodes

Is it fixed? Looks good on v2.12.0

We using 2.11.1 and I dont see the problem. I think, that problem was fixed in v 2.11.0.

This problem appears to still exist in 2.12.0 and is not isolated to single node clusters, I am pretty sure we just saw this on a 20 node cluster when we expanded it to 40 nodes.  We saw the following error message:
"explanation": "there are too many copies of the shard allocated to nodes with attribute [host.rack], there are [40] total configured shard copies for this shard id and [7] total attribute values, expected the allocated shard count per attribute [7] to be less than or equal to the upper bound of the required number of shards per attribute [6]"

So the problem is related to rack affinity for shard allocation.
I fixed it by following the advice above, but rather than disabling plugins and setting replicas to 0 I set it to the original number of replicas.
curl -XPUT -H 'Content-Type: application/json' http://es-foo.bar.lan/.opensearch-sap-log-types-config/_settings -d '{ "index" : {  "auto_expand_replicas" : "false", "number_of_replicas" : 20 } }'


QUESTION - Help with Event::fire and sending email

In my plugin on boot()
I have added:
    public function boot()
    {
        \RainLab\Notify\Classes\Notifier::bindEvents([
            'rainlab.user.register' => \RainLab\User\NotifyRules\UserRegisteredEvent::class
        ]);
        \RainLab\Notify\Classes\Notifier::instance()->registerCallback(function ($manager) {
            $manager->registerGlobalParams([
                'user' => Auth::getUser()
            ]);
        });
    }

in my component, I have added
Event::fire('rainlab.user.register', [$user, $data]);

But nothings happens,
How  I can debug where could be the problem?

Is what I am doing, the right path at least?

@roulendz have you tried attaching other listeners to that event to see if it is firing as expected?

If you still need support with this, feel free to send us an email.


Start of ManipulationHandler rework

Overview

Adds PrimitiveManipulator component to experimental, which is ultimately intended to replace ManipulationHandler.

Changes some of the enums to use flags. This allowed me to simplify things within ManipulationHandler, but also exposes some more functionality to the user.
Removed the ManipulationHandler internal state machine, as it was too simple to be necessary and simply obfuscated pointer events.
Removed RotateAboutObjectCenter option, as it only behaved differently for one handed near interaction, and this different behaviour felt unnatural.
Some minor cleanup of ManipulationHandler and the move/rotate/scale logic classes.
PrimitiveManipulator does not differ hugely from ManipulationHandler from the perspective of the user yet, but separating these classes now allows us to make bigger changes that will affect the user later.

Improved the ManipulationHandler inspector and added the PrimitiveManipulator inspector.

Different sections are now contained in foldout sections.
Some sections are only displayed when certain conditions are met. These conditions are made clear in help boxes.
Added a warning for ManipulationHandler inspector, informing the user that ManipulationHandler will soon be removed. Also provided a button that will automatically upgrade the current object to use PrimitiveInspector.

Added a new test for ManipulationHandler and new tests for PrimitiveInspector.

Changes

Fixes: #5464, #4396, #5776.

New Inspector

Upgrade Button

public class TwoHandRotateLogic

I think similar to the comment that you ad in ManipulationMoveLogic, we should probably be consistent here with making sure that this (and TwoHandScaleLogic) are internal classes to manipulation handler (unless we want to make the statement that these actually should be pulled out to be more general classes i.e. moving them to some lower layer)

Refers to: Assets/MixedRealityToolkit/Utilities/Physics/TwoHandRotateLogic.cs:20 in 9b01488. [](commit_id = 9b01488c3968d810be1fdad03d7929d6518a74ec, deletion_comment = False)

For this change, can you elaborate on the deficiencies for the existing ManipulationHandler and describe in what way this new handler improves the situation and ultimate goals.
Lists of new features are good, but it's better to understand why this is needed

For this change, can you elaborate on the deficiencies for the existing ManipulationHandler and describe in what way this new handler improves the situation and ultimate goals.
Lists of new features are good, but it's better to understand why this is needed

@SimonDarksideJ So the main thing addressed in this PR is that the ManipulationHandler inspector properties. At the moment they imply behaviour is possible, without explaining what you need to change about the object in order to actually enable that behaviour.
Release behaviour, for example, implies that the object can maintain it's velocity on release, without explaining to the user that you need a Rigidbody component on the object.
The reason why I feel like we need to separate these changes into their own class now, is that there are other planned changes to ManipulationHandler, some of which I already have a good portion of the work done for, that will be severe breaking changes. Separating now and adding to experimental means that we can continue refactoring this class without worrying as much about breaking changes.
For example, I have a lot of changes locally refactoring contraints, in the hopes of making them more consistent between ManipulationHandler and BoundingBox. I decided to pause work on those changes so I could get this PR out, rather than having one Monolithic PR.
On top of all this there was some cleanup I felt was necessary to make ManipulationHandler more readable and approachable for developers, these will aid the future changes I mentioned.


How to make bucket public?

How to do it with Pulumi? https://cloud.google.com/storage/docs/access-control/making-data-public
Click on the name of the bucket that contains the object you want to make public, and navigate to the object if it's in a subdirectory.

Click the more actions menu () associated with the object that you want to make public.

Select Edit access from the drop-down menu.

In the overlay that appears, click the + Add entry button.

Add a permission for allUsers.

Select Public for the Entity.
Select allUsers for the Name.
Select Reader for the Access.
Click Save.

Hi @nsisodiya
To make an object public, it would need to be under pulumi management and then you would be able to change it's attributes
https://www.pulumi.com/registry/packages/gcp/api-docs/storage/bucketobject/
Paul


Add pprof endpoint to pilot-agent

To help debug performance issues, memory leaks, deadlocks, etc.
This can safely be added over localhost port 15020 - we already have far more dangerous quitquitquit there so no problem adding another admin endpoint.
Would match similar code in istiod:
pilot/pkg/xds/debug.go
150:            s.addDebugHandler(mux, "/debug/pprof/", "Displays pprof index", pprof.Index)
151:            s.addDebugHandler(mux, "/debug/pprof/cmdline", "The command line invocation of the current program", pprof.Cmdline)
152:            s.addDebugHandler(mux, "/debug/pprof/profile", "CPU profile", pprof.Profile)
153:            s.addDebugHandler(mux, "/debug/pprof/symbol", "Symbol looks up the program counters listed in the request", pprof.Symbol)
154:            s.addDebugHandler(mux, "/debug/pprof/trace", "A trace of execution of the current program.", pprof.Trace)

@howardjohn hi, i would like to do this, and i find that there are lots of ports to listen, is there a plan about it?
Or only need to implement GRPC server?

We should add here: https://github.com/istio/istio/blob/4e465fc12484a74fe26383eca726f48c34d35363/pilot/cmd/pilot-agent/status/server.go#L202 on port 15020. I think @carolynhu may be working on it already possibly

@daixiang0 I'm recently working on dashboard related stuff, haven't got a chance to look into much more about this. Sure, if you'd like to work on this ticket, I'm going to assign this to you. thanks.

@carolynhu i will do this :)


Modular javascript and the option to trigger/reinitialize!

Right now I struggle with forms. I have a simple input field, and everything works out great. Until I create some input element with code and append it to DOM. In your code, there is this piece:
// Add active if input element has been pre-populated on document ready
    $(document).ready(function() {
      $(input_selector).each(function(index, element) {
        if($(element).val().length > 0) {
          $(this).siblings('label, i').addClass('active');
        }
      });
    });

It's nice, but it runs only once, because the whole forms plugin (as well as all other plugins) are initialized one time only! So now I have to copy-paste that code and re-run it every time I attach something to the DOM, which is killing me.
In my opinion, you should think of modular way to provide plugins so that they can be initialized/refreshed once one needs to.
p.s.
Forgot to mention that I already did this with collapsible and call it each time I add an item (facepalm)

duplicate of https://github.com/Dogfalo/materialize/issues/250


Significantly simplify SwiftSyntaxBuilder’s implementation

This significantly simplifies the implementation of SwiftSyntaxBuilder. The main cleanups are:

Removing the separate syntax node hierarchy in SwiftSyntaxBuilder and instead re-using the SwiftSyntax hierarchy

This is possible since BasicFormat now formats an entire SwiftSyntax tree instead of requiring the careful intraction with SwiftSyntaxBuilder’s Buildable types
All convenience initializers from SwiftSyntaxBuilder are now extensions on the SwiftSyntax nodes. We can even consider sinking them down into SwiftSyntax again

Removing the ExpressibleAs* types

With the integration of the parser into SwiftSyntaxBuilder, you create way fewer syntax nodes explicitly. Looking at the test cases, I think the regressions on the client side aren’t too significant and this hugely simplifies the SwiftSyntaxBuilder’s implementation
Having written a fair amount of SwiftSyntaxBuilder code myself now, I never found the ExpressibleAs* protocols super useful because it was non-trivial to discover which types are expressible as other types.
IMHO things are also clearer without the ExpressibleAs conformances because the conversion from ExprBuildable to InitializerClause implicitly inserted a =, which is a non-trivial fix up, I think.

I think the best way to review this is to skim over the new generated files (don’t bother looking at the diff because it’s way too noisy) as well as the updated test cases. https://github.com/ahoppen/swift-syntax/commit/f8af287eec02a934028212111b54b984f68775a6 contains the changes that need to be made to CodeGeneration to adopt to this simplified SwiftSyntaxBuilder.
@fwcd and @kimdv I would really value your thoughts on this change and whether you think getting rid of ExpressibleAs is a change that we can make without making the library harder to use. You wrote the most SwiftSyntaxBuilder code and can probably judge this best.

@swift-ci Please test

Some initial toughts:

I like the removal of ExpressibleAs* 🎉
Some of the convince initialiser removals I'm not sure all are could. Specially those where we have to add Expr("someThing"). It makes a bit harder to get started in my opinion.

@swift-ci Please test

@swift-ci Please test

@swift-ci Please test

I’m merging this now as a starting point to reconsider which convenience initializer we still need now that we have the parser integration. We can always go and add those we want back in.

Wow, you're moving quickly, sorry I didn't have time to look at this in detail. :D
Skimming over the proposal I'd say that I especially like the unification of the node hierarchies! These changes make things conceptually a lot simpler, which could be especially useful e.g. for tools that both parse and generate Swift code (e.g. source code transformation tools). IIUC, they could leverage the expressive power of SwiftSyntaxBuilder without the boilerplate of converting between different representations.
If we can address the added verboseness of explicit Expr wraps through protocols like ExprBuildable I see no downside there 👍

Yeah, sorry for merging this so quickly, I just have other PRs that depend on this and I wanted to make progress.

If we can address the added verboseness of explicit Expr wraps through protocols like ExprBuildable I see no downside there 👍

I’ve got an alternative proposal: We should provide more hand-written convenience initializers that use the parser interop to eliminate the need for most of the syntax tree construction altogether. With https://github.com/apple/swift-syntax/pull/1012 (and the corresponding CodeGeneration changes in https://github.com/ahoppen/swift-syntax/commit/fe30a08fdb00b1a70ed8b33aec8cf8d0e2ad6e94) we only have two places in CodeGeneration left where a call to Expr is now needed because we no longer have the implicit conversion from String.


Axiomatize molecular function regulators

At the LEGO meeting, it would be useful to have equivalence axioms for MF regulator, activator, inhibitor - to see the inference directly in Noctua
cc @ukemi @vanaukenk

We now have logical definitions for

molecular function regulator
enzyme regulator activity
enzyme activator activity
enzyme inhibitor activity
channel regulator activity
channel activator activity
*channel inhibitor activity
*receptor regulator activity
*receptor activator activity
*receptor inhibitor activity

I think that cover its !


Add Blankskip to Zipformer+CTC

Accelerating RNN-T Training and Inference Using CTC guidance
https://arxiv.org/abs/2210.16481

Hi, yifan, why not reduce frames at middle layers ?

Hi, yifan, why not reduce frames at middle layers ?

And  another question, is it possible the reduced encoder frames shorter than label sequence(For the case deletion error in CTC) ?

Please add .DS_Store to .gitignore.

OK


CFN Condition usage within conditions and functions

eg:
    HasManagedPolicies:
      !Or [ { Condition: UseManagedPolicyARN }, { Condition: UseECR } ]

or even:
    CreateMetricsStack:
      Condition: UseAutoscaling

It feels like I should be able to do this:

    const useManagedPolicyARN = new cdk.Condition(...);
    const useEcr = new cdk.Condition(...);
    const hasManagedPolicies = new cdk.Condition(this, "HasManagedPolicies", {
      expression: new cdk.FnOr(
        useManagedPolicyARN,
        useEcr
      )
    });

But the cdk.Condition class does not match the type definition of the predicate functions (which require FnCondition.
After some mucking around, I found the following workaround, but it relies on Fn simply re-emitting exactly what it's passed:
    const hasManagedPolicies = new cdk.Condition(this, "HasManagedPolicies", {
      expression: new cdk.FnOr(
        new cdk.Fn("Condition", "UseManagedPolicyARN"),
        new cdk.Fn("Condition", "UseECR")
      )
    });

This applies to referring to Conditions within FnOr and so forth within resources as well.
What is the correct way to express this in CDK?

We've generally avoided Conditions so far, since we prefer expressing conditions in the CDK app.
I'm assuming you're using CDK to generate reusable templates that will be deployed by someone else?

Yep, this is me evaluating CDK as "a tool to synthesize templates" rather than "a tool to manage AWS".
We're not big users of conditions anyway, but nice to have coverage.
That example unfortunately doesn't work - ref is a string, and FnOr expects an FnCondition.


⚠ Bofrost.at has degraded performance

In 8280aea, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1855 ms

Resolved: Bofrost.at performance has improved in b58c2f6 after 7 minutes.


Batch size auto is wrong?

Batch size auto is not working correctly (with generate_until tasks?).
lm_eval --model hf --model_args pretrained=HuggingFaceH4/zephyr-7b-alpha,dtype=bfloat16 --tasks polemo2_in --device cuda:0 --batch_size auto
Traceback (most recent call last):
  File "/net/tscratch/people/plgkwrobel/llm-benchmark/venv/bin/lm_eval", line 8, in <module>
    sys.exit(cli_evaluate())
  File "/net/tscratch/people/plgkwrobel/llm-benchmark/lm-evaluation-harness/lm_eval/__main__.py", line 231, in cli_evaluate
    results = evaluator.simple_evaluate(
  File "/net/tscratch/people/plgkwrobel/llm-benchmark/lm-evaluation-harness/lm_eval/utils.py", line 415, in _wrapper
    return fn(*args, **kwargs)
  File "/net/tscratch/people/plgkwrobel/llm-benchmark/lm-evaluation-harness/lm_eval/evaluator.py", line 150, in simple_evaluate
    results = evaluate(
  File "/net/tscratch/people/plgkwrobel/llm-benchmark/lm-evaluation-harness/lm_eval/utils.py", line 415, in _wrapper
    return fn(*args, **kwargs)
  File "/net/tscratch/people/plgkwrobel/llm-benchmark/lm-evaluation-harness/lm_eval/evaluator.py", line 325, in evaluate
    resps = getattr(lm, reqtype)(cloned_reqs)
  File "/net/tscratch/people/plgkwrobel/llm-benchmark/lm-evaluation-harness/lm_eval/models/huggingface.py", line 1051, in generate_until
    batch_size = self._detect_batch_size()
  File "/net/tscratch/people/plgkwrobel/llm-benchmark/lm-evaluation-harness/lm_eval/models/huggingface.py", line 610, in _detect_batch_size
    batch_size = forward_batch()
  File "/net/tscratch/people/plgkwrobel/llm-benchmark/venv/lib/python3.10/site-packages/accelerate/utils/memory.py", line 134, in decorator
    raise RuntimeError("No executable batch size found, reached zero.")
RuntimeError: No executable batch size found, reached zero.

The benchmark works with at least batch_size 2 (on 40GB VRAM card).

Thanks for reporting this, will check it out!
I suspect that this is because Mistral has a max length in its config of 32768 , and due to caution we calculate our max batch size based on the model's reported max length to ensure no OOMs will occur at that batch size.
You should be able to get around this with --batch_size auto by also passing --model_args max_length=4096 or some other value that is greater than the documents in this task.

I believe the first instance of batch size calculation is anomalous in this respect (which is perhaps worth changing.)

@haileyschoelkopf could it make sense to just set the auto batch size to one rather than yielding No executable batch size found, reached zero. ?

Other option is to just run with defined batch size and if OOM occurs then decrease and repeat.

A potential hacky workaround:
    batch_size = "auto"
    try:
        results = run_evaluation(eval_request=eval_request, task_names=[task.benchmark], num_fewshot=task.num_fewshot,
                                 batch_size=batch_size, device=DEVICE, use_cache=None, limit=LIMIT)
    except RuntimeError as e:
        if "No executable batch size found" in str(e):
            batch_size = 1
            results = run_evaluation(eval_request=eval_request, task_names=[task.benchmark], num_fewshot=task.num_fewshot,
                                     batch_size=batch_size, device=DEVICE, use_cache=None, limit=LIMIT)
        else:
            raise

This seems like an improvement over the current code to me. Whether or not we write a more robust fix later, opening a PR with this would be an improvement.

@StellaAthena I think we can do that here: https://github.com/EleutherAI/lm-evaluation-harness/blob/756eeb6f0aee59fc624c81dcb0e334c1263d80e3/lm_eval/models/huggingface.py#L620
Let me do a pull request real quick


Export utils and commands to share (with sentinel client, or others)

This supports redis-sentinel-client. It should not change or break any node_redis functionality, it simply separates and/or exports pieces that can be used by other modules. (Which should be good architecture regardless of the sentinel client or other modules using it.)
Thanks!

Ping @benbuckman


[Task]: Add spec conformance tests for regexp langlib

Description
$subject
Describe your task(s)
No response
Related area
-> Compilation
Related issue(s) (optional)
No response
Suggested label(s) (optional)
No response
Suggested assignee(s) (optional)
No response

PR - https://github.com/ballerina-platform/ballerina-spec/pull/1222

We are closing this issue because it is not being actively worked on. Let's reopen when we resume conformance test efforts.


[Coder] Sometimes Coder shows another selector for a method

In version 0.8.186, there is an error in the refresh of the clases when there is scroll in the list and then one class is opened for review.

This is the sequence of actions that lead me to get this error:

Nice catch :).

Yes, I was trying to understand the difference between  and , and I got lucky to find this kind of difficult bug.

Tested in version 0.8.210, and the error is still there, while analyzing class BlFlowLayoutCompositionExamples. The method exampleText has now the pragma gtExample, instead of example.

This is due to the infinite element cache.

Works now!


Add syntax highlighting to code blocks on appendices

https://adoring-einstein-5ea514.netlify.app/appendices/

I assume this should apply to everything, not just appendices?

probably, I just happened to notice it in the appendices

This should be fixed by:
https://github.com/wbamberg/matrix-hugo/commit/5b8b712699abcbaf4d9fbb55cebb2b813edd9825
https://github.com/wbamberg/matrix-hugo/commit/50bc61be43d0ae65069abbfe26288f933ba50184
https://github.com/wbamberg/matrix-hugo/commit/e7d23fd934cd86dbc518cf8310027af915a2de94
https://github.com/wbamberg/matrix-hugo/commit/bd399b8dac94b25e6dd033ea3c1d34beaf0a5bfd