Cant retrieve metrics from Kubernetes kind's

I have a Kubernetes cluster installed on AWS EKS version 1.25. I installed krr on my local computer with brew. I am publishing the Prometheus service with port forwarding, and while krr reports that it is connected, I am receiving the following errors.
krr simple -n monitoring --history_duration 10

|  __ \     | |             | |        | |/ /  __ |  __
| |) |_ | |__  _   _ | | __ _  | ' /| |) | |) |
|  _  // _ | '_ | | | / | / ` | |  < |  _  /|  _  /
| | \ \ () | |) | || _ \ || (| | | . | | \ | | \
||  __/|_./ _,|/__,| ||__|  __|  _\
Running Robusta's KRR (Kubernetes Resource Recommender) v1.6.0
Using strategy: Simple
Using formatter: table
[INFO] Using clusters: ['cluster-name']
on 0: [INFO] Listing scannable objects in cluster-name
on 0: [INFO] Connecting to Prometheus for cluster-name cluster
on 0: [INFO] Using Prometheus at https://xxxxxxxxxxxxxx.gr7.xx-xxxx-x.eks.amazonaws.com/api/v1/namespaces/monitoring/services/prometheus-server:80/proxy for
cluster cluster-name
on 0: [INFO] Prometheus found
on 0: [INFO] Prometheus connected successfully for cluster-name cluster
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment monitoring/prometheus-alertmanager/prometheus-alertmanager
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for DaemonSet monitoring/prometheus-node-exporter/prometheus-node-exporter
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment monitoring/prometheus-alertmanager/prometheus-alertmanager
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment monitoring/prometheus-alertmanager/prometheus-alertmanager-configmap-reload
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment monitoring/prometheus-msteams/prometheus-msteams
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for DaemonSet monitoring/prometheus-node-exporter/prometheus-node-exporter
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment monitoring/prometheus-server/prometheus-server
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment monitoring/prometheus-server/prometheus-server-configmap-reload
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment monitoring/prometheus-alertmanager/prometheus-alertmanager
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment monitoring/prometheus-alertmanager/prometheus-alertmanager-configmap-reload
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment monitoring/prometheus-msteams/prometheus-msteams
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for DaemonSet monitoring/prometheus-node-exporter/prometheus-node-exporter
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment monitoring/prometheus-server/prometheus-server
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment monitoring/prometheus-server/prometheus-server-configmap-reload
on 0: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment monitoring/prometheus-alertmanager/prometheus-alertmanager
on 1: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment monitoring/prometheus-alertmanager/prometheus-alertmanager-configmap-reload
on 1: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment monitoring/prometheus-msteams/prometheus-msteams
on 1: [WARNING] Prometheus returned no MemoryAmountLoader metrics for DaemonSet monitoring/prometheus-node-exporter/prometheus-node-exporter
on 2: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment monitoring/prometheus-server/prometheus-server
on 2: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment monitoring/prometheus-server/prometheus-server-configmap-reload
on 2: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment monitoring/prometheus-alertmanager/prometheus-alertmanager-configmap-reload
on 3: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment monitoring/prometheus-msteams/prometheus-msteams
on 4: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment monitoring/prometheus-server/prometheus-server
on 4: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment monitoring/prometheus-server/prometheus-server-configmap-reload
Calculating Recommendation |████████████████████████████████████████| 6 in 3.7s (1.02/s)
The values in the table also appear as 'unset'.

Where am I making a mistake?

Additionally, when I scan a namespace with defined request-limit, it shows the existing resource values but does not provide any recommendations. I am receiving the same warning logs.


Give the cards content

Using the FDND api

Card 0: Intro
Card 1: Me
Card 2: My school
Card 3: My work
Card 4: My hobbies
Card 5: My social life
Card 6: Outro

Added content to all the cards


⚠ Bofrost.de has degraded performance

In 98c6a89, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1229 ms

Resolved: Bofrost.de performance has improved in a0498c6.


Release a new intellij plugin

Can you release a new version with recent fixes? Thanks a lot!

coming soon


fix #206 : add ipv6 validation method

@fabian-hiller this is potential fix for #206, i added additional ipv6 validation method.

@fabian-hiller what do you think about PR, will you merge it, you have any questions or should i add the docs to the same PR?

I plan to review it on the weekend

I want to let you know that I have rescheduled the review for next week.

I will close this PR. However, I will keep it in mind when improving our IPv6 validation in the future.


Entsoe pricing forecast till mignight

Describe the bug
Since last week I'm using the entsoe integration (region Netherlands). However, the planner only seems to fetch prices up untill midnight. At midnight it shows prices for the rest of the day. For example now (21:00) it only shows prices untill 00:00 as shown below. Whereas entsoe, has prices available up until 00:00 the next day.

Steps to reproduce
1.Configure entsoe integration
2.Configure region BZN|NL
3.Switch to PV mode
4. Select departure time
5. Select departure tab
...
Configuration details
# tariffs are the fixed or variable tariffs
tariffs:
  currency: EUR # three letter ISO-4217 currency code (default EUR)
  grid:
    # or variable tariffs
    type: entsoe
    domain: BZN|NL
    securitytoken: 83xxxx
    #tax: 0.21
    charges: 0.1724

Log details
[entsoe] ERROR 2023/10/17 21:27:26 EOF
[main  ] ERROR 2023/10/17 21:27:26 failed configuring grid tariff: cannot create tariff 'entsoe': EOF

What type of operating system are you running?
HomeAssistant Add-on
Version
0.121.0

Logfile does not match error description?

Afaikt we are requesting +24 hours:
	"PeriodEnd":    {now.Add(duration).Format(numericDateFormat)},

If that does not work Entsoe probably doesn't return the data.
Closing due to missing logfile that could demonstrate the problem.

the same for me

Closing due to missing logfile that could demonstrate the problem.

"the same for me" does not change that in any way :(

Were can I get the log file?

I have the same issue.
EVCC is only showing pricing untill midnight (no errors in log file).
If I run the api call separately I do get the full results including the next day (see attached file).
API call results in a dataset containing 2 TimeSeries (mRID 1 & 2).
I believe only the first set of TimeSeries is captured by EVCC.
api result.pdf

Fixed in https://github.com/evcc-io/evcc/commit/8f82e1a55b8f0ecb20fdea66cceee3b6cb0222af


FEATURE: Cache for News view(s)

Background:  The News view of today shows a webpage with information about the application.   Very soon it will also be showing News from the local healthcare authorities to the people of their city and region.
Desire: Many locations have limited internet connectivity.  In these locations the user might go to an internet cafe, quickly visit the news, and then leave.  Later that day they would want to be able to reference that news from a site with no internet connection.  So the last downloaded news should be cached.

I can look into this issue. Please assign it to me.

Note: there might be more than one HCA active for a user at any time.  i.e. they may have added their own manual authority and they might be using the recommended one for their county/area/state

Closing this issue out as we have taken it into consideration and documented the feedback on this page in Jira.


[docs] Define backport criteria

Motivation for this change
Make it more definitive on what can be backported.
cc @alyssais
"retired", but active RMs: @domenkozar @vcunat
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Perhaps one of these could link to the other?  (Probably CONTRIBUTING.md -> the manual)  That way we wouldn't have to maintain the same documentation in two places.

I think the manual can go deeper, with more examples; while the CONTRIBUTING.md is more like cliff notes. Just for now they are essentially the same thing.

thanks everyone

I know I'm very late, but the formulation

Version updates which include new functionality (but no breaking changes)

seems a bit too permissive to me.  I read it as basically allowing arbitrarily large and risky updates, assuming they don't contain (intentional/known) incompatibilities.

The intention there was to review the upstream's changelog, if there's no stated breaking changes we should take upstream at their word.
Obviously the world isn't perfect, but desired affect was to allow for things like zfs 2.1 to be backported


Test multi-process execution on GPUs and find out performance issue

Recently we have seen that when using multiple ranks per GPU we don't see any performance improvement.
We need to find out the reason for this by using small test example.

The issue is related to CUDA MPS service not being active on the target nodes. One can use nvidia-cuda-mps-control -d  to start MPS service. Otherwise, wrapper script can be used instead of executable:
$ cat wrapper_special

#!/bin/bash

#enable mps
nvidia-cuda-mps-control -d > /dev/null 2>&1

# call actual binary
shift
./x86_64/special "$@"

[BBP-internal note: HELP-14142 may be of interest]


conflicting limits

gs <- gsplot() %>%
  lines(1:10, 1:10, xlim=c(0,100)) %>%
  points(2:7, 7:2, xlim=c(-10,-5))

should throw a warning saying that args are conflicting for a given view

Maybe work this in with #124


Java line length 100 instead of 120

Using a custom line length with google-java-format brings some challenges. It's a feature available only in the mvn plugin and the creators of google-java-format have shared that they will not accept it.
That means that the other tools like the official Intellij plugin for google-java-format will never support it.
My advice is to not use this feature in the mvn plugin so that users like me can format the code in Intellij directly. This will require a new PR where we change the line length to 100 and reformat all the Java files with the burden for the open PR to apply the formatter before rebasing.
WDYT?
@sureshms @harshach @mithmatt

@amiorin, we have spent a considerable amount of energy on this already.
Currently, all you need to do is just code without worrying about formatting in your IDE. Run mvn clean install to test your code, the code automatically gets formatted and your IDE picks up the new formatted code. This gives a new level of freedom of not having to worry about formatting the code while coding. Formatter formats the code when you build and run tests.
FYI, last time we messed with this behavior, many of us had to do a large number of code merges, for the following reason:

I create a PR. Before PR is merged, the pre-commit hook formats the code in PR.
Meanwhile, the branch I created PR from, I continue to work and create another PR.
The code I have and the code in the main branch have considerably changed and now I need to merge a large number of changes.

I understand that you want to format the code in Intellij directly. But mvn clean install does that for you anyway. Can we please just stick to the current process?

We decided to stick to 120.


Consider uploading to fir.im?

Not very friendly to non-ios developers. Consider uploading to fir.im?

I've tried uploading to fir.im and it seems to only provide a download service and doesn't provide the codesign service, so even uploading to fir.im users can not install correctly.


Group 3 - Text2SQL Implementation

This pull request introduces several changes, including the addition of SQL table creation and sample data insertion scripts, environment variable definitions, and significant updates to the RAGHelper.py file to support database connectivity and text-to-SQL translation. The most important changes are summarized below:
Environment Variable Definitions:

Added database connection environment variables (db_host, db_port, db_username, db_password, db_name) to the .env.template file (server/.env.template).

Database Connectivity and Text-to-SQL Translation:

Introduced PostgreSQL connection and text-to-SQL translation functionalities in RAGHelper.py, including methods to connect to the database, retrieve schema, translate natural language queries to SQL, and execute SQL queries (server/RAGHelper.py). [1] [2] [3]

Code Formatting and Cleanup:

Reformatted import statements and various methods for better readability and consistency (server/RAGHelper.py). [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]

Report on Pull Request #36: Text-to-SQL Implementation for RAGMeUp Framework

Overview
The pull request titled "Group 3 - Text2SQL Implementation" integrates a Text-to-SQL (text2sql) feature into the RAGMeUp framework, enabling users to query structured PostgreSQL databases using natural language. This enhancement bridges the gap between intuitive natural language input and SQL-based database querying, expanding the framework's functionality significantly. The pull request includes updates to the backend, user interface, and configuration files to support the new feature.

Enhancements Introduced

Text-to-SQL Integration:

Integrated a pre-trained text2sql model that translates natural language queries into SQL queries.
Designed the workflow specifically for PostgreSQL databases, leveraging its schema structure and SQL dialect.

Dynamic Schema Retrieval:

Implemented functionality to dynamically fetch database schema details, including table names and their columns, from PostgreSQL.
This schema information is formatted and used as context for the text2sql model to ensure accurate SQL generation.

SQL Query Execution:

Added logic to execute SQL queries generated by the text2sql model on a connected PostgreSQL database.
Robust error handling ensures meaningful feedback is provided to users for issues like syntax errors or connection failures.

User Interface Enhancements:

Introduced a toggle button in the upper-right corner of the UI to allow users to switch between the traditional RAG workflow and the new Text-to-SQL workflow.
Enhanced the input handling to process natural language queries for both workflows seamlessly.

Environment Configuration Updates:

Updated the .env.template file to include the following PostgreSQL configuration variables:

db_host: Hostname or IP address of the database server.
db_port: Port number of the database server.
db_username: Username for authentication.
db_password: Password for the database user.
db_name: Name of the database.

Users must populate these variables for the feature to function correctly.

Specific Code Changes

Backend Changes:

server.py:

Modified the /chat endpoint to accept an isText2SQL flag in the request payload, allowing the server to distinguish between workflows.
Added logic to call the relevant functions in RAGHelper for either RAG or Text-to-SQL workflows.

RAGHelper.py:

Added connect_to_postgres to establish connections to PostgreSQL databases.
Implemented get_schema to dynamically retrieve and format schema details.
Added translate_to_sql for natural language to SQL translation using the text2sql model.
Introduced execute_sql_query to run generated SQL queries on PostgreSQL and return results.
Enhanced handle_user_interaction to support both workflows based on the isText2SQL flag.

RAGHelperLocal.py:

Extended RAGHelperLocal to inherit and utilize Text-to-SQL capabilities from RAGHelper.

Frontend Changes:

index.scala.html:

Added a toggle button in the UI to switch between workflows, ensuring an intuitive user experience.
Adjusted the input field logic to process queries based on the toggle state.

add.scala.html:

Updated the layout to align with the new workflow options.

Frontend JavaScript:

Updated event handlers to include the isText2SQL flag in requests based on the toggle state.
Enhanced query submission logic to process Text-to-SQL queries dynamically.

Environment Configuration:

.env.template:

Added five new variables required for PostgreSQL connectivity:

db_host, db_port, db_username, db_password, db_name.

Challenges Addressed

PostgreSQL-Only Focus:

This implementation focuses exclusively on PostgreSQL, leveraging its schema structure and SQL dialect. Adapting the feature for other databases was beyond the current scope.

Dynamic Schema Retrieval:

Fetching and formatting large schemas with complex relationships posed a challenge. The implementation streamlined this by focusing on essential table and column information.

Model Limitations:

The pre-trained text2sql model faced difficulties with ambiguous or incomplete natural language queries, which sometimes led to invalid SQL.

Error Handling:

Comprehensive mechanisms were added to manage common errors during SQL generation and execution, ensuring meaningful feedback for users.

UI Integration:

Adding the toggle feature required careful design to maintain compatibility with existing workflows while providing an intuitive user experience.

Usage Instructions

Configure Environment Variables:

Populate the .env.template file with the following PostgreSQL details:

db_host: Hostname or IP (e.g., localhost).
db_port: Port number (default: 5432).
db_username: Username for database access.
db_password: Password for the user.
db_name: Name of the database.

Example:
db_host=localhost
db_port=5432
db_username=rag_user
db_password=securepassword
db_name=rag_db

Toggle Feature:

Use the toggle button in the upper-right corner of the UI to switch between the RAG and Text-to-SQL workflows.

Start the Application:

Ensure the PostgreSQL database is running and accessible using the provided credentials.
Launch the RAGMeUp framework and submit natural language queries via the UI.

Future Improvements

Support for Additional Databases:

Expand functionality to support relational databases such as MySQL, SQL Server, and Oracle to enhance versatility.

Connect Databases via UI:

Allow users to input database connection details directly through the UI, reducing reliance on manual environment configuration.

Schema Visualization:

Add a feature to visualize the database schema in the UI, helping users craft precise queries.

Enhanced Error Feedback:

Provide users with detailed feedback, such as the generated SQL query and suggestions for refining inputs when queries fail.

Model Customization:

Fine-tune the text2sql model on domain-specific datasets to improve query accuracy and adaptability.

Caching for Performance:

Implement schema and query result caching to reduce response times for frequently accessed data.

Conclusion
This pull request introduces a PostgreSQL-specific Text-to-SQL feature to the RAGMeUp framework, significantly enhancing its ability to handle structured data. The addition of the toggle button in the UI ensures seamless workflow transitions, and the updated configuration provides a streamlined setup process.
Future work should focus on expanding database compatibility, enabling dynamic database connections via the UI, and refining the user experience to further extend the framework’s usability and versatility. This implementation is a major step forward in making RAGMeUp more robust and user-friendly.


Workflow for Automating PR Creation for Release Updates

Closes https://github.com/prometheus-operator/prometheus-operator/issues/6877. With this workflow, a PR will be automatically created when the upstream release in prometheus-operator repository does not match the release version in https://github.com/prometheus-operator/website/blob/main/data/prometheusOperator.json.

Discussed this today, some plans for testing:

Create a new branch and change the operator version to any that we've released in the past
Update this PR to update that branch instead of main, also add on: dispatch... so we can manually trigger this.
Merge this PR
Run workflow, fix it if it fails
New PR updating workflow target to main
Profit!!!


Make Clair connection configurable

https://github.com/vmware/harbor/blob/82dc75806447a2f3cc25d98502b91d45ef0cef3a/src/common/const.go#L72
It would be nice to have this value configurable instead of hardcoded.

Please see https://github.com/vmware/harbor/issues/3245#issuecomment-330740543

This is duplicated by https://github.com/vmware/harbor/issues/4269

Assign to @yixingjia since he is working on something related.

Clair  endpoint is now configurable.


feat: Added upgrade-python-requirements workflow

JIRA: https://openedx.atlassian.net/browse/BOM-2602

@Jawayria I retract my comment, @stvstnfrd is correct here.

@Jawayria I retract my comment, @stvstnfrd is correct here.

Okay

@stvstnfrd please review


support custom/external event callback

An alternative way to achieve https://github.com/luvit/luv/pull/345

ready to merge


[REQUEST] Automatically resize images

Describe your problem
It's more a feature request, not a bug issue. Sorry if i put it in the wrong place.
Is there a way to automatically resize uploaded images?
Steps to reproduce the problem
Yes, it's easy to convert images on your computer before uploading it to the web site. But if you use a phone to write content it would be super handy to set the desired width and quality beforehand, so Bludit would automatically resize images.
Bludit version
3-10-0

Good call. It would be fantastic to see something like this added:

That would be great!
But setting at least one standard size and quality apart from thumbnails would already be a nice step.

I'd like to write a plugin for that matter. Can you at least add the afterImagesUpload hook? Or any other way to get and modify the uploaded images?

Any news on this? Would be great!

Is not implemented yet, but I added a new library for manage Images for Bludit v4 and will we possible to do it.

Is not implemented yet, but I added a new library for manage Images for Bludit v4 and will we possible to do it.

Cool, thanks, would be perfect for photo blogging!

Hello, with the new version of Bludit v4.0 rc1, I would like to close the old Github issues. If you feel that your issue is not resolved in the latest version, create a new ticket.

Help and Support use the Forum https://forum.bludit.org
Bugs and new requests here in Github https://github.com/bludit/bludit/issues


Decode GIFs as static images on Android

Problem
https://github.com/Expensify/App/issues/14751
Platform: Android
GIFs are not being decoded as static images, and therefore are resulting in a decoding error when uploading avatars.
Screencast from 2023-02-13 04:28:30 AM.webm
Solution
Use the ForceStaticImage option from the Fresco decoder to decode the first frame only in a GIF image.
Result
Screencast from 2023-02-13 04:34:10 AM.webm

@Beamanator I don't think we have a template for forked PRs right?
I've approved the PR and take care of the checklist in the App PR.

@mananjadhav That is correct. Looks like the package spec in the app is based on the merge commit hash, so we will have to wait until this is merged to open the other other PR.

@redstar504 I meant the package.json for react-native-image-manipulator

@mananjadhav Yes it makes sense, but the current package.json is only referencing the commit of the last fix that was merged into this fork, so I am not sure we need to worry about versioning:
"@oguzhnatly/react-native-image-manipulator": "github:Expensify/react-native-image-manipulator#c5f654fc9d0ad7cc5b89d50b34ecf8b0e3f4d050",

Yeah I saw, I would then wait for @Beamanator to comment here. We do that for some of the other dependencies, I am not sure about this one. Can you please add @Beamanator as a reviewer please?

@Beamanator we're ready for review.

It's a bit weird I wasn't automatically added as a reviewer, I'll have to look into why 👍
@mananjadhav true we don't have PR templates for forks I believe, I wonder if this is something we should look into! Maybe ask in #contributor-plus if you feel like it, otherwise I'll try to remember to bring it up later this week when I'm back full time :D
I currently see what @redstar504 is saying and I believe we should continue just using the latest commit hash in App's package.json - though it's a bit funky that some forks we publish, some we don't. @mananjadhav do you mind starting a convo in #expensify-open-source about this? If you're busy no worries I can start the conversation but it will probably wait till Wednesday

For now @redstar504 can you please proceed with an App PR referencing the latest commit in package.json? 🙏


Disable user verification (UV)

Overview
We've had some reports of operating systems requiring a PIN to be entered before a Yubikey can be used for authentication. Any subsequent use of the Yubikey then appears to force a PIN before use.
Reading some documentation, this appears to be defined by a feature called user verification (UV) - something only applicable to FIDO2 and WebAuthn.
Yubico recommend explicitly enabling or disabling  user verification to avoid unintended, or unexpected user interaction: "For second factor flows, we recommended to set UV to discouraged to prevent a PIN prompt when using a YubiKey for authentication."
Expected outcome
I'm suggesting we (by default) disable UV to avoid unintended circumstances of requiring PINs to be set on users' Yubikeys.
Notes
See spec: https://www.w3.org/TR/webauthn/#userVerificationRequirement
Yubico summary: https://developers.yubico.com/WebAuthn/WebAuthn_Developer_Guide/User_Presence_vs_User_Verification.html which outlines the broad scenarios that this could kick off: "User verification can take various forms, such as password, PIN, fingerprint, public key credential, etc."

Note, Yubico has provided this recommendation in a document sent directly to the module maintainers.

Good find. Maybe it can be disabled by default but configurable.


Markdown parsing error for www.grav.dev literal

I have this Markdown:
At the base of my development stack is my local server run by MAMP PRO. For each project website I have a separate local folder that MAMP PRO see as a site and serves from it. So, for example, I have www.br.dev (for Grav), www.brp.dev (for Statamic) and www.materialize.dev (for Grav), plus a bunch more.

and it generates this output from Grav with Parsedown Extra:

but this from the Parsedown tester:

Do I have something set up wrong, or is there a problem within Grav?

Is the site running on www.br.dev?

Yes and no.
The URL that generated the example is http://localhost:5757/blog/my-web-workflow-and-tools?ckcachecontrol=1448887144.
But that is being served via CodeKit's internal server that uses MAMP PRO for PHP and MAMP PRO is serving www.br.dev.
However, if I use the URL, http://www.br.dev/blog/my-web-workflow-and-tools, I get this output:

which looks fine.
Very strange. Something with CodeKit's server in the chain?

Really strange. I don't get why it changes that string, which has nothing to do with the server it's running on, even the domain name is not involved.
What about running the site on MAMP without going through CodeKit? I am running MAMP, tried the same string, nothing happens.

It looks fine for me, too, straight from MAMP PRO. That's the screenshot that looks fine, above.

So definitely something in the CodeKit changing the output for some way


The query is not properly shown

Start the server, and hit it with an HTTP request that has a query part. Check that currently that query is not properly shown in all places. It should.

Fixed by #11


Protocol Oriented Programming

As you know, Apple has presented new way of programming - which will make Crusty happy.
I know it's all about personal taste... But I really like the idea and I think it can release the great power of Swift.
This whole XcodeServerEntity-subclass model in existing SDK is a great candidate to redesign it with Protocol Oriented Programming. This will allow to drop this big inheritance tree and implement only necessary stuff. What's more it can allow to replace classes with structs which will also have a great impact on performance.
@czechboy0 what do you think?

The reason why everything inherits from XcodeServerEntity is because its init(json: NSDictionary) constructor parses out common fields like id, rev and tinyID. Protocols wouldn't allow for that unfortunately, but overall I agree with you that we should be aiming for a protocol-based approach wherever possible.

Old protocols wouldn't but the new ones? I mean, now we can create extensions with default implementation - this should be a game changer.

Sure, but the way the objects are now parsed is that the constructor takes a JSON object, pulls out the fields relevant to it and then calls super with the same objects, which does the same thing... until it reaches the XcodeServerEntity, which pulls out the above mentioned fields.
Protocol extensions enable you to only provide a default implementation for all objects conforming to a certain protocol, but AFAIK they don't support inheritance, which is exactly this case.

Sure, with the current architecture of project it's impossible but I was rather thinking about deeper refactoring and dropping this verbose XcodeServerEntity inheritance thing. I'd probably do things similar to you but something (maybe a little Crusty in my head) is telling me that it can be resolved with different approach.
I will rethink this and If I come with any kind of solution or idea I will bring it to you! 😉

Yeah do, thanks!

I've came to this little article today and I wonder maybe this will be a good approach?

Interesting, Chris mentions the library called JSONCodable, which I took a look at. Looks very useful for parsing and generating JSON into structs.
There is a couple of reasons why I don't think it would be of much benefit here, mostly because we give different names to properties and to the keys in the JSON. Plus the parsing pattern is very similar to what we're using already, with the exception that we use inheritance to parse the objects, because that's how they are treated by Xcode Server (everything has an _id and a _rev).
Let's open this if we come up with more actionable ideas.


MNT: Re-rendered with conda-smithy 2.0.1 (didn't matrix Windows properly)

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


Fix typo

Hi, I noticed a small typo on line 203
https://github.com/takahirox/glTF-Transform-lod-script/blob/main/generate_lod.mjs#L203

      lodOcclusionTexture !== material.getOcculusionTexture()) {

Occulusion should be Occlusion

Thanks for the report. Would you like to make a PR?

Fixed in #3


Mitigation Credits

Problem description
Non-offset GHG reductions, such as those resulting from corporate energy conservation or efficiency initiatives, are currently not supported in Guardian. These are generally measured as the delta between the baseline and current emission levels, there are no concepts/tools which would allow Guardian users to account for these, and link/reflect them on the ESG scorecards ( #1017).
Requirements
Introduce the ability to mint Mitigation Asset Type tokens as the result of the calculation of the diff between planned (and reported on the ESG) and actual results of the calculations based on the MRV data for a reporting period. This would likely require:

New type of blocks in the policy definition language specifying 'target' numbers.
Policy Engine ability to mint different types of tokens depending on the conditions
Needs to be linked with #1017 logic

Definition of done

Policy engine supports Mitigation Tokens
Sample policy is created demonstrating workflow
Documentation for the API and the sample policy

Acceptance criteria
A company is able to automatically account for mitigation efforts through MRV data processing by the policy engine resulting in the production of mitigation tokens.

Should be assigned to Max to how we report to Mitigation. Would have to have CET's first to how we know how to deal at a business level.

@anvabr @envisionblockchainpm is there an update on this?

@sergmetelin as per the comment above this ticket is waiting for @Mpinnola's input

I assume "mitigation" in this context refers to non-offset GHG reductions, such as those resulting from corporate energy conservation or efficiency initiatives? In general, this would be measured as the delta between the baseline and current emission levels. These reductions aren't typically traded/tradable. For example, if a Company A reduces (aka "mitigates") their emissions by 100 tCO2e compared to their baseline, but they only needed to reduce by 50 tCO2e to meet their goals, they can't just sell the other 50 to Company B, or at least that's not really how it works now.
Just putting a random thought out there;  if we did want to enable such a transaction, perhaps instead of Company A selling Company B 50 "mitigation credits" they could instead accept 50 CETs from Company B at a certain price (i.e. Company B could essentially pay Company A to take 50 CETs off their hands). So maybe if an offsets trades at ~10$, maybe CETs can be traded at ~-10$. This would allow the monetization of mitigation without the need for a new asset type.
I could use a bit if clarification of what a "mitigation asset" refers to, but if I am understanding clearly, I and not sure of the feasibility/utility/necessity of this new asset type.

@Mpinnola In the blockchain space there are limited tools to represent real world concepts, chiefly a single tool - tokens.  Everything needs to be modelled as a token (of some type), to enable accounting/tracking of mitigation results they need to be recorded as tokens. They don't need to be tradable, although I think it would be prudent to leave the capability for this for the future.

@anvabr
One existing system that may be able to guide us here is the cap-and-trade system, which already provides a mechanism to incentivize mitigation. While this is a regulatory mechanism, perhaps it can guide us in the concept of a voluntary “mitigation credit.”
Under Canada’s cap-and-trade system, subjected entities are given emissions limits. If measured emissions are below the limit, “surplus credits” are issued. If emissions are above the limit, the entity must make a payment, or purchase and retire surplus credits or offsets. In similar systems, entities are issued carbon allowances that represent their limit. After a compliance period, any surplus of allowances can be sold, or they can purchase additional allowances if they emit above their limit.
https://www.canada.ca/en/environment-climate-change/services/climate-change/pricing-pollution-how-it-will-work/output-based-pricing-system/overview.html
I think what we need to determine is; do these systems make the concept of a “mitigation credit” obsolete, especially if an entity is already subject to a cap-and-trade program? Effectively, a surplus credit or an allowance credit may achieve the same function as a hypothetical “mitigation credit” in the sense that mitigation can be measured as the delta between a limit (or baseline or target) and actual emissions, and rewarded via the issuance of surplus credits, or the actual surplus of allowance credits.
My next question is, why not focus on tokenizing surplus credits, or allowances, and then perhaps facilitate voluntary cap-and-trade systems? In a voluntary system, the limit could be a voluntary target or a baseline. If emissions are lower that the limit, mitigation has occurred, and credits can be issued. If emissions are over the limit, credits must be purchased. We could call them “mitigation credits” if we want, but effectively this is a voluntary cap-and-trade system.
Let’s model an example based on the concept of “surplus credits”

Company A has a baseline or target of 1,000 mtCO2e
Emissions are measured via CETs
At the end of 2022 they have a balance of 800 mtCO2e
200 “mitigation credits” are issued
Company B has a baseline or target of 1,000 mtCO2e
At the end of 2022 they have a balance of 1,200 mtCO2e
They are required to offset 200 mtCO2e or they can perhaps purchase 200 “mitigation credits” from company A

Or we can model the “allowance credit” system

Company A has a baseline or target of 1,000 mtCO2e
1,000 allowances are issued at the beginning of the monitoring period
At the end of the at the end of 2022 they have a balance of 800 CETs
800 tokenized allowances are burned, and they have 200 left, that can be sold

Both are modeling cap-and-trade systems, both can be tracked with tokenized credits and emissions, both measure and incentivize mitigation. In addition, there are existing systems and standards that can guide us. If we don’t want to go this path, the next step should be to identify what functions the hypothetical “mitigation credits’ would serve that are not captured by tokenizing a cap-and-trade system.


How to read the complete response body after executing the HTTP request?

I have tried the below query and its also posted in open test documentation but its not working for me.I am getting JSON values as null.

description: Verify that the branches array exists and is not empty
script: |
       var branches =$output.body;

       if (!branches || (branches.length == 0)) {
           $fail("Response body invalid or contains no branch elements.");
       } else {
           $log($format(
             "The  API returned {0} branches",
             branches.length));
       }

 - description: display
   script:

       var output1= JSON.stringify($output.body);

       var newArr = [];
       newArr= branches.split( );

       for (var i = 0; i < newArr.length; i++)
       {
       $log("Json values" +newArr[i]);
       }

You need to read the $output.body value immediately after the HttpRequest action has executed. You can store it in a variable if you are planning to use it later in the test, or use it right away for whatever it is that you need to do, but you have to read it in the very next action after HttpRequest. The $output object is reset and repopulated at the end of every test action, since it needs to store the output values from the last action that finished execution.
In the example you gave above, the $output object stores the output values for the script action that finished executing in the top part of your example. Script actions don't have a body output value, so, the expression $output.body will return null.

Its working . Thanks:)


[Request for advice] Sending keystrokes from a remote device (Bluetooth COM to USB relay)

I apologize in advance if this question is going to be offtopic. I'm asking about a feature that may not be a good fit for the project. I'm using a Bluetooth to USB relay to connect a BT keyboard/mouse to android-based projector (bluetooth is in the specs, but it doesn't work on my device).
Now, as a bonus, I would like to add support for sending media keys to the projector from my notebook/home PC.
As I understand it, windows does not support bluetooth server mode, so the best approach would be to send commands via bluetooth SPP profile to the RPi working as Bluetooth COM to USB relay. As a plan B, I can use rest api, but I would prefer to use same protocol everywhere.
I have no problem writing to BT COM port on the windows side, but I can't find a suitable library for the bluetooth com port server on the Raspberry Pi side.
I've tried some projects of https://github.com/ukBaz/python-bluezero/wiki, but most of them are in alpha and the rest look abandoned.
Maybe I'm just using the wrong keywords to search, so I thought I'd ask an expert :)
Could you please suggest which of the existing libraries makes sense to use to create a com-server on the RPi side?
And one more question - would such a feature be useful for Bluetooth to USB relay? I'd be happy to add PR after I get the prototype working.

As chance would have it, I am contributing to a project called RelayKeys that might be a perfect fit for your use case :)
You need an inexpensive BLE dongle (e.g., nRF52840 USB Key with TinyUF2 Bootloader) to allow your Windows machine to act as a Bluetooth device. Then you may use RK to send keyboard and mouse commands. Currently there are only a handful of multimedia keys, but the team around @willwade would highly appreciate PRs adding more!
If the team decided to switch to adafruit_hid, there would be more multimedia keys out of the box. But this would require quite a bit of rework.
After all any contribution to RK is for a good cause, with Ace Centre being a charity devoted to developing solutions for people like myself, who are reliant on assistive technology.
Since I can't use my arms anymore, I use an eye-controlled keyboard running on a Windows tablet mounted to my wheelchair. Actually this is intended for speech synthesis. But RelayKeys allows me to turn this tablet into a Bluetooth keyboard. The issued Bluetooth commands are in turn sent to a Pi 0W that is running bluetooth_2_usb and is connected to a KVM switch.
To me my setup sounds quite similar to what you have in mind?

Hi @quaxalber!
Thank you so much for both the project and the warm attitude! I'm really glad I asked - it's exactly what I was looking for:)
I've already ordered the nRF52840 dongle, will definitely report back when it arrives and I can try RelayKeys.

Glad to hear :) looking forward to your feedback. In case you need help with setting up let me know!

Nice! Thanks @quaxalber
@ig-sinicyn what media keys are you needing. From memory it would be pretty easy adding missing ones in. Volume we do have. See current keys here https://docs.acecentre.org.uk/products/v/relaykeys/using-relaykeys/relaykeys-cli#command-keypress-key-modifier
let us know how you get on :)

@ig-sinicyn How have you been getting on with this? Need any help?

Hi, @quaxalber! Thanks for the great advice,  I've managed to use RelayKeys without any issue, works fine!
While waiting for the dongle I've checked some third-perty libs and long story short - I have a BLE GATT server that allows to send keystrokes remotely. There's a prototype version but it is not completed yet.

Great stuff! I bet @willwade is happy about that too!
Curious to learn more about your PR, interesting stuff :)


ROX-18609: Introduce ClusterHealthResponse

Description
This PR adds a response to the RawClusterHealthInfo message from Sensor, establishing constant writes from Central->Sensor in the interval of RawClusterHealthInfo messages being sent (currently 30s).
The reason for this is when Central is exposed behind an Ingress controller that does TLS termination, our current way of keepalives via gRPC isn't working as expected. This is due to the ingress controller not forwarding the HTTP/2 frame (as a reference, here's an issue for NGINX explaining this).
This leads to infrequent writes from Central->Sensor (the latest will be the reprocessing interval duration, i.e. 4 hours). This then triggers write timeouts defined within the ingress controller.
Currently, the only way to deal with this is to increase the timeout to an increased value (like the aforementioned 4 hours). This is also suggested by the NGINX ingress controller documentation for long-running bi-directional streams.
As suggested in the issue opened for NGINX, an alternative is to introduce health checks / constant flow of messages in the application, which this PR does by responding to the frequently sent cluster health messages.
As a note for reviewers, this PR superseeds #7161. The introduction of central capabilities will be split into a different PR.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed

see CI (especially GKE compatibility tests)

Manual tests (for the test environment Kong was used as ingress controller of choice, you probably can use any other ingress controller):

Create an EKS cluster on infra.

Have a secondary K8S cluster (this can be either local, OS, or GKE).

Below steps are all executed on the EKS cluster

Install Kong following the documentation here for EKS. Alternatively, you can use any other ingress controller, but have to adjust the below manifest and steps accordingly.

For the Kong service, ensure that the load balancer type ELB is used.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "80"
    service.beta.kubernetes.io/aws-load-balancer-type: elb

Associate a public domain with the load balancer public IP and create certificates for it e.g. via certbot:

sudo certbot certonly --manual -d <your-domain-name> --preferred-challenges dns

Create a secret within the stackrox namespace with the generated cert/key pair from certbot:

kubectl create ns stackrox
kubectl create secret tls stackrox-rsa-custom -n stackrox --cert=fullchain.pem --key=privkey.pem

Generate a Central bundle via roxctl:

# PSP = false
# Expose = none (we will create a custom Ingress for this)
roxctl central generate interactive

Adjust the config map manifest containing the endpoint config (01-central-09-endpoints-config.yaml):

...
data:
  endpoints.yaml: |
    endpoints:
      - listen: "8080"
        tls:
          disable: true

Adjust the central service manifest to include the 8080 port and the kong annotation(01-central-14-service.yaml):

...
  annotations:
    konghq.com/protocol: "grpc"
...
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 8080

Deploy the central by running:

./central-bundle/central/scripts/setup.sh
kubectl create -R -f central-bundle/central

Afterwards, create an ingress that exposes the 8080 port with TLS terminated using the previously created stackrox-rsa-custom secret:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    konghq.com/protocols: grpc,grpcs,https,http
    kubernetes.io/ingress.class: kong
  name: central-ingress
  namespace: stackrox
spec:
  rules:
  - host: stackrox-old.dhaus.io
    http:
      paths:
      - backend:
          service:
            name: central
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - <custom domain name>
    secretName: stackrox-rsa-custom

Verify that the UI opens successfully and you can connect to your Central using https://<custom domain name>.

Below commands are all executed on the secondary K8S cluster (OpenShift, local, GKE)
13. Deploy the sensor within the cluster:
export API_ENDPOINT="https://<custom domain name>:443"
export ROX_ADMIN_PASSWORD=$(cat central-bundle/password)
./deploy/sensor.sh
kubectl set env -n stackrox deploy sensor CENTRAL_ENDPOINT=https://<custom domain name>:443

Verify that sensor is successfully connecting to Central and shown as healthy.

Observe that the connection is not interrupted between Sensor and Central. You can determine this by the pod restarts of Sensor as well as observing the logs of Central. On local clusters with little to no workload, the pod previously restarted after latest 10 minutes. On OpenShift or GKE clusters with more workloads, this usually takes not longer than 30 minutes.

Current dependencies on/for this PR:

master

PR #7310   👈

This comment was auto-generated by Graphite.

/test

/test gke-version-compatibility-tests


Strange error: The type Y cannot inherit from both X and IY

I have the following code:
public interface IY
{
    int Value { get; set; }
}

public class X
{
    public int Value { get; set; }
}

public class Y : X, IY
{

}

This code produces the following compilation error:

The type Y cannot inherit from both X and IY because both those types have a member with the script name set_value. You have to rename the member on one of the base types, or refactor your code.

The IY is interface. Why it cannot be compiled to javascript?

This functionality is correctly implemented in Bridge.
http://deck.net/11f892ef12746979edcedd64e147a402
public class Program
{
    public static void Main()
    {
        var y = new Y();

        y.Value = 33;

        // Should be true
        Console.WriteLine(y is IY);

        // Should be 33
        Console.WriteLine(y.Value);

        // Cast to IY type
        var iy = (IY)y;

        iy.Value = 44;

        // Should be 44
        Console.WriteLine(iy.Value);

        // Should be true
        Console.WriteLine(y is X);

        // Cast to X type
		var x = (X)y;

        x.Value = 55;

        // Should be 55
        Console.WriteLine(x.Value);
    }
}

public interface IY
{
    int Value { get; set; }
}

public class X
{
    public int Value { get; set; }
}

public class Y : X, IY
{

}

Hope this helps.


powerdevil: backport fix for debug log spam

Motivation for this change
The version of powerdevil we're using prints a few debug messages every time an
inhibition is triggered, including some spurious messages about not being able
to contact consolekit.  This can cause a lot of spam in the system journal,
especially when a web browser or other application that plays video is running,
because such applications often trigger such inhibitions very frequently.
A recent upstream commit to powerdevil changes the logging behavior of the
program so that it doesn't print debug messages anymore.  This commit backports
that patch to our version of powerdevil (5.17.5).
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Come to think of it, since this is a backport, should I target this PR to the 20.03 branch instead of master?


Process in CodeBuild hangs

Expected Behavior
it runs
Actual Behavior
it hangs for more than 10 mins, no error messages
Steps to Reproduce the Problem
Run the following commands in CodeBuild:
[Container] 2022/03/25 18:39:00 Running command echo "kics_img_name $kics_img_name"
kics_img_name checkmarx/kics:v1.5.4

[Container] 2022/03/25 18:39:17 Running command docker run -v $(pwd):/path $kics_img_name scan --no-progress --ignore-on-exit all -p /path -o /path --report-formats junit --output-name kics-report --log-level=DEBUG --type CloudFormation
Unable to find image 'checkmarx/kics:v1.5.4' locally
v1.5.4: Pulling from checkmarx/kics
Digest: sha256:628ed3b084e6ace14838b2772bf261187a606066d7bb31ed77087beb66ed0847
Status: Downloaded newer image for checkmarx/kics:v1.5.4
6:39PM DBG Could not find string flag ci
6:39PM DBG console.scan()
6:39PM DBG console.scan()

                   .0MO.
                   OMMMx
                   ;NMX;
                    ...           ...              ....
WMMMd     cWMMM0.  KMMMO      ;xKWMMMMNOc.     ,xXMMMMMWXkc.
WMMMd   .0MMMN:    KMMMO    :XMMMMMMMMMMMWl   xMMMMMWMMMMMMl
WMMMd  lWMMMO.     KMMMO   xMMMMKc...'lXMk   ,MMMMx   .;dXx
WMMMd.0MMMX;       KMMMO  cMMMMd        '    'MMMMNl'
WMMMNWMMMMl        KMMMO  0MMMN               oMMMMMMMXkl.
WMMMMMMMMMMo       KMMMO  0MMMX                .ckKWMMMMMM0.
WMMMMWokMMMMk      KMMMO  oMMMMc              .     .:OMMMM0
WMMMK.  dMMMM0.    KMMMO   KMMMMx'    ,kNc   :WOc.    .NMMMX
WMMMd    cWMMMX.   KMMMO    kMMMMMWXNMMMMMd .WMMMMWKO0NMMMMl
WMMMd     ,NMMMN,  KMMMO     'xNMMMMMMMNx,   .l0WMMMMMMMWk,
xkkk:      ,kkkkx  okkkl        ;xKXKx;          ;dOKKkc

Scanning with Keeping Infrastructure as Code Secure v1.5.4

6:39PM INF Scanning with Keeping Infrastructure as Code Secure v1.5.4
6:39PM DBG storage.NewMemoryStorage()
6:39PM DBG Looking for queries in executable path and in current work directory
6:39PM DBG helpers.GetDefaultQueryPath()
6:39PM DBG helpers.GetExecutableDirectory()
6:39PM DBG Queries found in /app/bin/assets/queries
6:39PM DBG source.NewFilesystemSource()
6:39PM DBG engine.NewInspector()
6:39PM DBG Custom library not provided. Loading embedded library instead
6:39PM DBG Custom library not provided. Loading embedded library instead
6:39PM DBG Could not open embedded library data for cloudFormation platform
6:39PM INF Inspector initialized, number of queries=496
6:39PM INF Query execution timeout=1m0s
6:39PM DBG provider.NewFileSystemSourceProvider()
6:39PM DBG parser.NewBuilder()
6:39PM DBG resolver.Add()
6:39PM DBG resolver.Build()

here it hangs, you have to stop CodeBuild manually. Could you help, please?

ping :( If I can get more verbose debug please let me know.

Hello, @ALutchko!
Thank you so much for reaching us 😊
Can you give us details about your project, please?

Hi, @ALutchko! Can you provide us with the project that you are running against KICS, please?

I am really sorry but no, it is not OpenSource. If you provide me with more details what kind of information is needed then most likely I'll be able to provide it.
Buildspec for CodeBuild looks like:
      - ecr_domain="$CURR_ACC_ID.dkr.ecr.$REGION.amazonaws.com"
      - echo "ecr_domain $ecr_domain"
      - kics_img_name=$(grep checkmarx/kics ../ecr_images.txt)
      - aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $CURR_ACC_ID.dkr.ecr.$REGION.amazonaws.com/checkmarx
      - echo "kics_img_name $kics_img_name"
      - docker pull $ecr_domain/$kics_img_name
      - cd ../..  # repo root
      - pwd  # check we're in the correct place
      - docker run -v $(pwd):/path $kics_img_name scan --no-progress --ignore-on-exit all -p /path -o /path --report-formats junit --output-name kics-report --log-level=DEBUG --type CloudFormation

The repo root has a few directories which contain CloudFormation yamls.

I understand 😊 Let's try to figure out what is happening.

Can you provide us with the size of the project?
Can you use the flag --profiling MEM and let us know the value of Total MEM usage for prepare_sources, please?
Is it stop at resolver.Build()? If not, please send us the entire log.

Thank you. I will try to simulate a scan in CodeBuild with an approximate size project
Yes, for example: docker run -v $(pwd):/path $kics_img_name scan --no-progress --ignore-on-exit all -p /path -o /path --report-formats junit --output-name kics-report --log-level=DEBUG --type CloudFormation --profiling MEM
For how long did you wait until you stopped?

The result (I've added mem just before):

[Container] 2022/04/12 09:57:32 Running command free -h
              total        used        free      shared  buff/cache   available
Mem:          3.6Gi       709Mi       403Mi       0.0Ki       2.5Gi       2.7Gi
Swap:            0B          0B          0B
[Container] 2022/04/12 09:57:32 Running command docker run -v $(pwd):/path $kics_img_name scan --no-progress --ignore-on-exit all -p /path -o /path --report-formats junit --output-name kics-report --log-level=DEBUG --type CloudFormation --profiling MEM
Unable to find image 'checkmarx/kics:v1.5.4' locally
v1.5.4: Pulling from checkmarx/kics
Digest: sha256:628ed3b084e6ace14838b2772bf261187a606066d7bb31ed77087beb66ed0847
Status: Downloaded newer image for checkmarx/kics:v1.5.4
9:57AM DBG Could not find string flag ci
9:57AM DBG console.scan()
9:57AM DBG console.scan()

                   .0MO.
                   OMMMx
                   ;NMX;
                    ...           ...              ....
WMMMd     cWMMM0.  KMMMO      ;xKWMMMMNOc.     ,xXMMMMMWXkc.
WMMMd   .0MMMN:    KMMMO    :XMMMMMMMMMMMWl   xMMMMMWMMMMMMl
WMMMd  lWMMMO.     KMMMO   xMMMMKc...'lXMk   ,MMMMx   .;dXx
WMMMd.0MMMX;       KMMMO  cMMMMd        '    'MMMMNl'
WMMMNWMMMMl        KMMMO  0MMMN               oMMMMMMMXkl.
WMMMMMMMMMMo       KMMMO  0MMMX                .ckKWMMMMMM0.
WMMMMWokMMMMk      KMMMO  oMMMMc              .     .:OMMMM0
WMMMK.  dMMMM0.    KMMMO   KMMMMx'    ,kNc   :WOc.    .NMMMX
WMMMd    cWMMMX.   KMMMO    kMMMMMWXNMMMMMd .WMMMMWKO0NMMMMl
WMMMd     ,NMMMN,  KMMMO     'xNMMMMMMMNx,   .l0WMMMMMMMWk,
xkkk:      ,kkkkx  okkkl        ;xKXKx;          ;dOKKkc

Scanning with Keeping Infrastructure as Code Secure v1.5.4

9:57AM INF Scanning with Keeping Infrastructure as Code Secure v1.5.4
9:57AM DBG storage.NewMemoryStorage()
9:57AM DBG Looking for queries in executable path and in current work directory
9:57AM DBG helpers.GetDefaultQueryPath()
9:57AM DBG helpers.GetExecutableDirectory()
9:57AM DBG Queries found in /app/bin/assets/queries
9:57AM DBG source.NewFilesystemSource()
9:57AM DBG engine.NewInspector()
9:57AM DBG Started MEM profiling for get_queries
9:57AM DBG Custom library not provided. Loading embedded library instead
9:57AM DBG Custom library not provided. Loading embedded library instead
9:57AM DBG Could not open embedded library data for cloudFormation platform
9:58AM DBG Stopped MEM profiling for get_queries
9:58AM INF Total MEM usage for get_queries: 226.13MB
9:58AM INF Inspector initialized, number of queries=496
9:58AM INF Query execution timeout=1m0s
9:58AM DBG provider.NewFileSystemSourceProvider()
9:58AM DBG parser.NewBuilder()
9:58AM DBG resolver.Add()
9:58AM DBG resolver.Build()
9:58AM DBG Started MEM profiling for prepare_sources

it hanged in this state for 15 mins then I stopped the build.

Hello again, @ALutchko!
I think I finally reproduced the issue! There is a strange behaviour with the flag --type.
With 3 GB memory, 2 vCPUs:

Scan time without --type flag: 3/4min
Scan time with --type flag: It fails and "it does not end"

With 7 GB memory, 4 vCPUs:

Scan time without --type flag: 3/4 min
Scan time with --type flag: 9/10/11 min

P.S. I used this "project": https://github.com/rafaela-soares/codebuild-kics-scan
We will try to understand this behaviour and fix it. We apologize for it.

Hello again, @ALutchko!
I think I finally reproduced the issue! There is a strange behaviour with the flag --type.
With 3 GB memory, 2 vCPUs:

Scan time without --type flag: 3/4min
Scan time with --type flag: It fails and "it does not end"

With 7 GB memory, 4 vCPUs:

Scan time without --type flag: 3/4 min
Scan time with --type flag: 9/10/11 min

P.S. I used this "project": https://github.com/rafaela-soares/codebuild-kics-scan
We will try to understand this behaviour and fix it. We apologize for it.

Hello again, @ALutchko!
I think I finally reproduced the issue!
With 3 GB memory, 2 vCPUs:

Scan time without --type flag: It fails

Scan time with --type flag: It fails

With 7 GB memory, 4 vCPUs:

Scan time without --type flag: ~3/4 min
Scan time with --type flag: ~9/10/11 min

P.S. I used this "project": https://github.com/rafaela-soares/codebuild-kics-scan
We will try to understand why there is a difference in the scan time and fix it. There is a strange behaviour when using the flag --type. We apologize for it.

@ALutchko, can you try with 7 GB memory, 4 vCPUs and let us know if the build succeeded, please?

Have just tried, waited 10 mins, the same result:
Scanning with Keeping Infrastructure as Code Secure v1.5.4
7:39PM INF Scanning with Keeping Infrastructure as Code Secure v1.5.4
7:39PM DBG storage.NewMemoryStorage()
7:39PM DBG Looking for queries in executable path and in current work directory
7:39PM DBG helpers.GetDefaultQueryPath()
7:39PM DBG helpers.GetExecutableDirectory()
7:39PM DBG Queries found in /app/bin/assets/queries
7:39PM DBG source.NewFilesystemSource()
7:39PM DBG engine.NewInspector()
7:39PM DBG Started MEM profiling for get_queries
7:39PM DBG Custom library not provided. Loading embedded library instead
7:39PM DBG Custom library not provided. Loading embedded library instead
7:39PM DBG Could not open embedded library data for cloudFormation platform
7:40PM DBG Stopped MEM profiling for get_queries
7:40PM INF Total MEM usage for get_queries: 252.36MB
7:40PM INF Inspector initialized, number of queries=496
7:40PM INF Query execution timeout=1m0s
7:40PM DBG provider.NewFileSystemSourceProvider()
7:40PM DBG parser.NewBuilder()
7:40PM DBG resolver.Add()
7:40PM DBG resolver.Build()
7:40PM DBG Started MEM profiling for prepare_sources

What will be the application behavior if I just remove --type CloudFormation? It will scan for all possible types and try to guess?

Yes. Can you try without the flag --type, please?

The same. Switched off after 27 min from start. Also, I've updated to 1.5.5
9:25AM INF Scanning with Keeping Infrastructure as Code Secure v1.5.5
9:25AM DBG storage.NewMemoryStorage()
9:25AM DBG Looking for queries in executable path and in current work directory
9:25AM DBG helpers.GetDefaultQueryPath()
9:25AM DBG helpers.GetExecutableDirectory()
9:25AM DBG Queries found in /app/bin/assets/queries
9:25AM DBG source.NewFilesystemSource()
9:25AM DBG engine.NewInspector()
9:25AM DBG Started MEM profiling for get_queries
9:25AM DBG Custom library not provided. Loading embedded library instead
9:25AM DBG Custom library not provided. Loading embedded library instead
9:25AM DBG Could not open embedded library data for cloudFormation platform
9:25AM DBG Stopped MEM profiling for get_queries
9:25AM INF Total MEM usage for get_queries: 260.02MB
9:25AM INF Inspector initialized, number of queries=496
9:25AM INF Query execution timeout=1m0s
9:25AM DBG provider.NewFileSystemSourceProvider()
9:25AM DBG parser.NewBuilder()
9:25AM DBG resolver.Add()
9:25AM DBG resolver.Build()
9:25AM DBG Started MEM profiling for prepare_sources
9:35AM DBG Stopped MEM profiling for prepare_sources
9:35AM INF Total MEM usage for prepare_sources: 1.48GB
9:35AM DBG Started MEM profiling for start_scan
9:35AM DBG service.StartScan()
9:35AM DBG service.StartScan()
9:35AM DBG engine.Inspect()

So it is not related to the behaviour of the flag --type.
Can you try to use 15 GB memory, 8 vCPUs, please?

@ALutchko, can you send us only the IaC code to<EMAIL_ADDRESS>Only for debugging purposes to understand what is happening.

Large: just for test purpose, because using such size in real workflow won't make sense due to price. Tested, switched off after 20 min from start
10:21AM INF Scanning with Keeping Infrastructure as Code Secure v1.5.5
10:21AM DBG storage.NewMemoryStorage()
10:21AM DBG Looking for queries in executable path and in current work directory
10:21AM DBG helpers.GetDefaultQueryPath()
10:21AM DBG helpers.GetExecutableDirectory()
10:21AM DBG Queries found in /app/bin/assets/queries
10:21AM DBG source.NewFilesystemSource()
10:21AM DBG engine.NewInspector()
10:21AM DBG Started MEM profiling for get_queries
10:21AM DBG Custom library not provided. Loading embedded library instead
10:21AM DBG Custom library not provided. Loading embedded library instead
10:21AM DBG Could not open embedded library data for cloudFormation platform
10:21AM DBG Stopped MEM profiling for get_queries
10:21AM INF Total MEM usage for get_queries: 225.68MB
10:21AM INF Inspector initialized, number of queries=496
10:21AM INF Query execution timeout=1m0s
10:21AM DBG provider.NewFileSystemSourceProvider()
10:21AM DBG parser.NewBuilder()
10:21AM DBG resolver.Add()
10:21AM DBG resolver.Build()
10:21AM DBG Started MEM profiling for prepare_sources
10:30AM DBG Stopped MEM profiling for prepare_sources
10:30AM INF Total MEM usage for prepare_sources: 1.48GB
10:30AM DBG Started MEM profiling for start_scan
10:30AM DBG service.StartScan()
10:30AM DBG service.StartScan()
10:30AM DBG engine.Inspect()

I'll try to send something.

Sent.

Hello again, @ALutchko! Thank you so much for sending it.
We ran KICS against your folder and the scan was quick.

Did you try to run KICS locally against your (entire) project?
Are your CF IaC samples in the same folder inside your project?

Locally it works, even on VM having 1GB ram so memory isn't the issue here. Looks strange but timestamps are in GMT, not my local time.
r@l $ docker run -v /home/ans/clsm-ci-cd:/path checkmarx/kics scan  --ignore-on-exit all -p /path -o /path --report-formats junit --output-name kics-report --log-level=DEBUG --profiling MEM
2:19PM DBG Could not find string flag ci
2:19PM DBG console.scan()
2:19PM DBG console.scan()

   [ logo ]

Scanning with Keeping Infrastructure as Code Secure v1.5.5

2:19PM INF Scanning with Keeping Infrastructure as Code Secure v1.5.5
2:19PM DBG storage.NewMemoryStorage()
2:19PM DBG Looking for queries in executable path and in current work directory
2:19PM DBG helpers.GetDefaultQueryPath()
2:19PM DBG helpers.GetExecutableDirectory()
2:19PM DBG Queries found in /app/bin/assets/queries
2:19PM DBG source.NewFilesystemSource()
2:19PM DBG engine.NewInspector()
2:19PM DBG Started MEM profiling for get_queries
2:19PM DBG Custom library not provided. Loading embedded library instead
2:19PM DBG Custom library not provided. Loading embedded library instead
2:19PM DBG Could not open embedded library data for cloudFormation platform
2:20PM DBG Stopped MEM profiling for get_queries
2:20PM INF Total MEM usage for get_queries: 273.14MB
2:20PM INF Inspector initialized, number of queries=496
2:20PM INF Query execution timeout=1m0s
2:20PM DBG provider.NewFileSystemSourceProvider()
2:20PM DBG parser.NewBuilder()
2:20PM DBG resolver.Add()
2:20PM DBG resolver.Build()
2:20PM DBG Started MEM profiling for prepare_sources
2:20PM DBG Stopped MEM profiling for prepare_sources
2:20PM INF Total MEM usage for prepare_sources: 273.14MB
2:20PM DBG Started MEM profiling for start_scan
2:20PM DBG service.StartScan()
2:20PM DBG service.StartScan()
2:20PM DBG engine.Inspect()
2:20PM DBG match: true :: 6.256660416485939
2:20PM DBG match: true :: 5.037401197654112
2:20PM DBG engine.Inspect()

2:22PM DBG Stopped MEM profiling for start_scan
2:22PM INF Total MEM usage for start_scan: 403.90MB
2:22PM DBG model.CreateSummary()
2:22PM DBG HTTP POST to descriptions endpoint
2:22PM DBG HTTP Status: 200 OK 253.227496ms
2:22PM DBG console.resolveOutputs()
2:22PM DBG helpers.PrintResult()
Files scanned: 38
Parsed files: 38
Queries loaded: 496
Queries failed to execute: 0

------------------------------------

EC2 Not EBS Optimized, Severity: INFO, Results: 1
Description: It's considered a best practice for an EC2 instance to use an EBS optimized instance. This provides the best
....
Results Summary:
HIGH: 67
MEDIUM: 38
LOW: 76
INFO: 1
TOTAL: 182

2:22PM INF Files scanned: 38
2:22PM INF Lines scanned: 8594
2:22PM INF Parsed files: 38
2:22PM INF Lines parsed: 8594
2:22PM INF Queries loaded: 496
2:22PM INF Queries failed to execute: 0
2:22PM INF Inspector stopped
2:22PM DBG console.printOutput()
2:22PM DBG Output formats provided [junit]
2:22PM DBG helpers.GenerateReport()
2:22PM DBG Started MEM profiling for generate_report
2:22PM INF Results saved to file /path/junit-kics-report.xml fileName=junit-kics-report.xml
Results saved to file /path/junit-kics-report.xml
2:22PM DBG Stopped MEM profiling for generate_report
2:22PM INF Total MEM usage for generate_report: 403.90MB
Scan duration: 3m46.054786187s
2:22PM INF Scan duration: 3m46.054786187s

The repo root contains files like below:
aws-ci-cd # (the dir in the archive)
...
kics.config
README.md

Just in case:
$ cat kics.config

log-file: true
log-level: DEBUG
#log-path: .
silent: false
type:
  - CloudFormation
# Ansible, AzureResourceManager, Buildah, CloudFormation, DockerCompose, Dockerfile,
# GRPC, GoogleDeploymentManager, Kubernetes, OpenAPI, Terraform
verbose: true

But the same config exists locally.
Looks like the issue is in relations with CodeBuild.

I've double-checked, "PrivilegedMode" set to "true" for this CodeBuild

Yes, it seems that is some CodeBuild issue...
You can specify with directories/files you want to scan. Maybe it will help in the CodeBuild.
For example: docker run -v $(pwd):/path $kics_img_name scan --no-progress --ignore-on-exit all -p /path/<IaCFolderOrFileName>,/path/<IaCFolderOrFileName2> -o /path --report-formats junit --output-name kics-report --log-level=DEBUG --type CloudFormation
Let me know if it helps 🤞

This scan completed in 1.5 mins:
[Container] 2022/04/13 14:59:58 Running command path_to_check="/path/aws-ci-cd/infra-auto/infra-auto.v2.yml"
[Container] 2022/04/13 14:59:58 Running command docker run -v $(pwd):/path $kics_img_name scan --ignore-on-exit all -p "$path_to_check" -o /path --report-formats junit --output-name kics-report --log-level=DEBUG --profiling MEM

So, imho it makes sense to add debug to kics before and after each file and/or dir to check which one breaks test. What do you think?

You can try. But as you showed, the scan is quick locally. I do not know why CodeBuild is crashing 😢

I cannot try, it should be done from inside of KICS :) You have that loop inside which iterated over dirs/files :)

Unfortunately, we can not do much if we do not have access to the entire project.
Regarding the part of the project that you send us, we did not find anything that breaks or crashes. We test it locally, and it was quick. And we tested it in the CodeBuild (in a private repo, of course), and it was fast too (even with 3 GB memory, 2 vCPUs).

Furthermore, as you showed, your entire project takes ~3/4 min locally. So, it seems that it is not a problem from the KICS side.

I don't blame "it's on your side" but this is the only way I can think about to debug. It's impossible to try files one by one in CodeBuild env :(

Maybe the issue can be escalated to the main team?

The main team is already aware of this issue.

Did you try the suggestion of pointing only the IaC files in the scan?
docker run -v $(pwd):/path $kics_img_name scan --no-progress --ignore-on-exit all -p /path/aws-ci-cd/infra-auto,/path/aws-ci-cd/infra-manual,/path/aws-ci-cd/pipelines -o /path --report-formats junit --output-name kics-report --log-level=DEBUG --type CloudFormation

Even worse: stopped after 20 mins. Maybe there's a typo or need spaces between comas?  And I see this hang-in-silence regardless of "--log-level=DEBUG" 👎👎👎

This completes in 2 mins

There is no typo and no need for spaces between comas:
[Container] 2022/04/14 07:50:38 Running command docker run -v $PWD:/path checkmarx/kics scan --no-progress --ignore-on-exit all -p /path/aws-ci-cd/infra-auto,/path/aws-ci-cd/infra-manual,/path/aws-ci-cd/pipelines -o /path --report-formats junit --output-name kics-report -v --log-level DEBUG --profiling MEM --type CloudFormation
31 | Unable to find image 'checkmarx/kics:latest' locally
32 | latest: Pulling from checkmarx/kics
33 | 40e059520d19: Pulling fs layer
34 | c45c8fc16a24: Pulling fs layer
35 | d839455a985f: Pulling fs layer
36 | eb9b894d736d: Pulling fs layer
37 | 52f6c41608e2: Pulling fs layer
38 | 4f4fb700ef54: Pulling fs layer
39 | eb9b894d736d: Waiting
40 | 52f6c41608e2: Waiting
41 | 4f4fb700ef54: Waiting
42 | 40e059520d19: Verifying Checksum
43 | 40e059520d19: Download complete
44 | 40e059520d19: Pull complete
45 | d839455a985f: Verifying Checksum
46 | d839455a985f: Download complete
47 | eb9b894d736d: Verifying Checksum
48 | eb9b894d736d: Download complete
49 | 4f4fb700ef54: Verifying Checksum
50 | 4f4fb700ef54: Download complete
51 | 52f6c41608e2: Verifying Checksum
52 | 52f6c41608e2: Download complete
53 | c45c8fc16a24: Verifying Checksum
54 | c45c8fc16a24: Download complete
55 | c45c8fc16a24: Pull complete
56 | d839455a985f: Pull complete
57 | eb9b894d736d: Pull complete
58 | 52f6c41608e2: Pull complete
59 | 4f4fb700ef54: Pull complete
60 | Digest: sha256:024bc81f6ba68fdac9a194a52e88d2bc341b3a65467cd451faa160a53257ee74
61 | Status: Downloaded newer image for checkmarx/kics:latest
62 | 7:50AM DBG Could not find string flag ci
63 | 7:50AM DBG console.scan()
64 | 7:50AM WRN Any kics.config file will be ignored, please use --config if kics.config is wanted
65 | 7:50AM DBG console.scan()
66 | 7:50AM WRN Any kics.config file will be ignored, please use --config if kics.config is wanted
67 |
68 | .0MO.
69 | OMMMx
70 | ;NMX;
71 | ...           ...              ....
72 | WMMMd     cWMMM0.  KMMMO      ;xKWMMMMNOc.     ,xXMMMMMWXkc.
73 | WMMMd   .0MMMN:    KMMMO    :XMMMMMMMMMMMWl   xMMMMMWMMMMMMl
74 | WMMMd  lWMMMO.     KMMMO   xMMMMKc...'lXMk   ,MMMMx   .;dXx
75 | WMMMd.0MMMX;       KMMMO  cMMMMd        '    'MMMMNl'
76 | WMMMNWMMMMl        KMMMO  0MMMN               oMMMMMMMXkl.
77 | WMMMMMMMMMMo       KMMMO  0MMMX                .ckKWMMMMMM0.
78 | WMMMMWokMMMMk      KMMMO  oMMMMc              .     .:OMMMM0
79 | WMMMK.  dMMMM0.    KMMMO   KMMMMx'    ,kNc   :WOc.    .NMMMX
80 | WMMMd    cWMMMX.   KMMMO    kMMMMMWXNMMMMMd .WMMMMWKO0NMMMMl
81 | WMMMd     ,NMMMN,  KMMMO     'xNMMMMMMMNx,   .l0WMMMMMMMWk,
82 | xkkk:      ,kkkkx  okkkl        ;xKXKx;          ;dOKKkc
83 |
84 |
85 | Scanning with Keeping Infrastructure as Code Secure v1.5.5
86 |
87 |
88 | 7:50AM INF Scanning with Keeping Infrastructure as Code Secure v1.5.5
89 | 7:50AM DBG storage.NewMemoryStorage()
90 | 7:50AM DBG Looking for queries in executable path and in current work directory
91 | 7:50AM DBG helpers.GetDefaultQueryPath()
92 | 7:50AM DBG helpers.GetExecutableDirectory()
93 | 7:50AM DBG Queries found in /app/bin/assets/queries
94 | 7:50AM DBG source.NewFilesystemSource()
95 | 7:50AM DBG engine.NewInspector()
96 | 7:50AM DBG Started MEM profiling for get_queries
97 | 7:50AM DBG Custom library not provided. Loading embedded library instead
98 | 7:50AM DBG Custom library not provided. Loading embedded library instead
99 | 7:50AM DBG Could not open embedded library data for cloudFormation platform
100 | 7:51AM DBG Stopped MEM profiling for get_queries
101 | 7:51AM INF Total MEM usage for get_queries: 257.19MB
102 | 7:51AM INF Inspector initialized, number of queries=496
103 | 7:51AM INF Query execution timeout=1m0s
104 | 7:51AM DBG provider.NewFileSystemSourceProvider()
105 | 7:51AM DBG parser.NewBuilder()
106 | 7:51AM DBG resolver.Add()
107 | 7:51AM DBG resolver.Build()
108 | 7:51AM DBG Started MEM profiling for prepare_sources
109 | 7:51AM DBG Stopped MEM profiling for prepare_sources
110 | 7:51AM INF Total MEM usage for prepare_sources: 257.19MB
111 | 7:51AM DBG Started MEM profiling for start_scan
112 | 7:51AM DBG service.StartScan()
113 | 7:51AM DBG engine.Inspect()
114 | 7:51AM DBG service.StartScan()
115 | 7:51AM DBG engine.Inspect()
116 |
117 | 7:52AM DBG Stopped MEM profiling for start_scan
118 | 7:52AM INF Total MEM usage for start_scan: 336.61MB
119 | 7:52AM DBG model.CreateSummary()
120 | 7:52AM DBG HTTP POST to descriptions endpoint
121 | 7:52AM DBG HTTP Status: 200 OK 204.0446ms
122 | 7:52AM DBG console.resolveOutputs()
123 | 7:52AM DBG helpers.PrintResult()
124 | Files scanned: 7
125 | Parsed files: 7
126 | Queries loaded: 496
127 | Queries failed to execute: 0
128 |
129 | ------------------------------------
130 |
131 | EC2 Not EBS Optimized, Severity: INFO, Results: 1
132 | Description: It's considered a best practice for an EC2 instance to use an EBS optimized instance. This provides the best performance for your EBS volumes by minimizing contention between Amazon EBS I/O and other traffic from your instance
133 | Platform: CloudFormation
134 |
135 | [1]: ../../path/aws-ci-cd/infra-auto/infra-auto.v2.yml:543

...

Hi @ALutchko, great. Can you check the other paths also one by one so you can pinpoint the problematic path and who knows the problematic file.
We will also consider you suggestion of adding more debug when processing files.
Let us know what other debugging info you are missing so we can consider it as well in our next soon-to-be release!
Thanks

@nunoocx

"Let us know what other debugging info you are missing"
In this case most likely we have a problematic file but it's impossible to see which file it is. At the moment the only way is to check all files narrowing down using -p, which is really time- and resource-consuming. It would be very helpful to see in the debug mode which file was passed and which just opened for checking then something happened. So, a message like "opening somedir/fileXYZ" will resolve this problem very easy.

Trying to narrow down meanwhile.

Hello again, @ALutchko 🙂
We launched the new KICS version with more log messages. I hope it helps!
I would like to ask you for more information if you do not mind:

What is the operating system that you are using in your CodeBuild environment?
What image are you using in your CodeBuild environment?
The folders of the project that you provide us (/path/aws-ci-cd/infra-auto,/path/aws-ci-cd/infra-manual,/path/aws-ci-cd/pipelines) have all the files that you have in the same folders?
What is the number of files in the entire project?

Hello @ALutchko,
Can you give us an update, please?

@rafaela-soares Sorry, I had no time to do that, I don't use the tool since that. Maybe later.
Thanks!

Thanks, @ALutchko!


[Bug] [MongoDB-CDC] Incremental data kind error in snapshot phase

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
Incremental data kind error in snapshot phase
SeaTunnel Version
dev
SeaTunnel Config
none

Running Command
none

Error Exception
none

Zeta or Flink or Spark Version
No response
Java or Scala Version
No response
Screenshots
cdc-base logic

MongoDB cdc logic

trigger error location

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

MongoDB's sourceRecord can't get the operation under Envelope.operationFor, right?

Please assign it to me @hailin0


Mention that Search add-on currently doesn't work with elasticsearch version 8.x

There must be a caution on the first page of Search add-on documentation.
The issue to fix it in the framework: https://github.com/jmix-framework/jmix/issues/624

Already fixed in JD-CR-73


update + - expand collapse icons

@topowright please review and merge

@jmccausland please let me know once we have new builds so I can place them on our Beta site

all up to date


fix xio erasing of zero in complex printing

xio used to print
1.232+.3123j in some cases.
Now it prints 1.232+0.3123j.

Where does the ' ' character come from.

@SylvainCorlay I just found another minor problem.
The ' ' comes from the float printing. It either prints a - or ' '. But since for the imaginary part the absolute value is handed to the printer, we can expect it to be ' ' all the time.
There was a small issue with the float printing as it didn't work correctly if the biggest decimal was 0.
I added test cases for both. Should look even better now :)


add examples

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Pull request type

Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[x] Documentation content changes
[ ] Other (please describe):

Codecov Report
Base: 60.49% // Head: 47.21% // Decreases project coverage by -13.28% :warning:

Coverage data is based on head (44b5d0f) compared to base (07cff18).
Patch has no changes to coverable lines.

Additional details and impacted files
@@             Coverage Diff             @@
##           master     #407       +/-   ##
===========================================
- Coverage   60.49%   47.21%   -13.29%
===========================================
  Files         234      231        -3
  Lines       25303    24860      -443
===========================================
- Hits        15308    11738     -3570
- Misses       9035    12433     +3398
+ Partials      960      689      -271

Flag
Coverage Δ

1.16.x
47.05% <ø> (-13.09%)
:arrow_down:

1.18.x
38.32% <ø> (-22.06%)
:arrow_down:

1.19.x
38.30% <ø> (-22.08%)
:arrow_down:

connection
?

database_sql
?

discovery
?

e2e
27.23% <ø> (-21.76%)
:arrow_down:

integration
?

macOS-latest
38.29% <ø> (+0.03%)
:arrow_up:

ratelimiter
?

scripting
?

table
27.23% <ø> (-0.71%)
:arrow_down:

topic
?

ubuntu-latest
47.20% <ø> (-13.24%)
:arrow_down:

unit
38.31% <ø> (-0.09%)
:arrow_down:

windows-latest
?

Flags with carried forward coverage won't be shown. Click here to find out more.

 Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Username and feedback fields cannot be empty if they are empty then the api request will not hit and it will show a warning toast

Fixes Issue

Changes proposed

Check List (Check all the applicable boxes)

[ ] My code follows the code style of this project.
[ ] My change requires changes to the documentation.
[ ] I have updated the documentation accordingly.
[ ] All new and existing tests passed.

I think you have raised a new PR for this..so closing this!


Bump StartUpDG.jl compat to 1.1+

Currently, StartUpDG.jl <1.1.0 imports NamedArrayPartition from RecursiveArrayTools.jl. Due to compat bounds put in place because of #1789, this prevents us from using newer versions of StartUpDG.jl.
This PR moves NamedArrayPartition back into StartUpDG.jl and lowers the compat requirement on RecursiveArrayTools.jl, hopefully making it compatible with Trixi.jl again.

Thanks!
Looks like the CI failure is real:
https://github.com/trixi-framework/Trixi.jl/actions/runs/9780885936/job/27003604009?pr=2001#step:7:1317
  LoadError: "Memory access for Kronecker.KroneckerProduct{Float64, Matrix{Float64}, Matrix{Float64}} not implemented yet."

Thanks! I should be able to get to this after the holiday (July 4 in the US)

It turns out there was a bug in Hex elements of SBP approximation type. Running julia> convergence_test(joinpath(EXAMPLES_DIR, "elixir_euler_curved.jl"), 3) on main gives suboptimal convergence rates
Dict{Symbol, Any} with 3 entries:
  :variables => ("rho", "rho_v1", "rho_v2", "rho_v3", "rho_e")
  :l2        => [0.438461, 0.607849, 0.607727, 0.414621, 0.482715]
  :linf      => [0.240406, 0.676716, 0.68089, 0.144242, 0.296871]

Fixing it using (what will be) v1.1.4 of StartUpDG gives
Dict{Symbol, Any} with 3 entries:
  :variables => ("rho", "rho_v1", "rho_v2", "rho_v3", "rho_e")
  :l2        => [3.23863, 3.60149, 3.60946, 3.26518, 3.70695]
  :linf      => [3.30159, 3.5868, 3.5791, 3.39908, 3.75305]

@sloede @ranocha the format check is failing for test/test_dgmulti_3d.jl, but I formatted it using the most recent version of JuliaFormatter (1.0.60). Any suggestions?


Feat/bring up to date

Let me know if something needs to be split out
FIX

default v2
handle case where there is no curretn or nex event
rename things so they match the language in OnTime
update HELP.md
rare case where timer/public/Lower Third message is undefined in newly created actions where the user is yet to type anything

FEAT

add more ontime variables

*MessageVisible
Blink/Blackout
Now and Next Cue

Toggle/On/Off on all relevant actions
feat: Load and sart Cue action
Better Add / remove action
feedback Time added or removed
this timer message feedback

lets drop v1 here

would this be an appropriate way to notify v1 users


Handle updating email address field on ACME Issuers

Is your feature request related to a problem? Please describe.
Currently if a user updates the email field on their Issuer resource, cert-manager does not update the registered ACME account with the new email address.
This is at best misleading, and at worst dangerous as users may stop receiving expiry warning notifications that they expect to receive.
Describe the solution you'd like
We should validate that the email address is up to date when a change is observed. This will probably require including additional cached account information in the issuer's status field in order to prevent excessive calls to the ACME API.
/kind feature
/area acme
/priority important-soon
/milestone v0.9

/assign
/lifecycle active


The lines commented out were a fix for NuGet and dotnet versions whic…

…h are probably no longer even supported. You can't simply rename a package source as other sections of the file rely on those names, e.g.  The packageSourceMapping section is about to become more commonly used as central package management complains about it be missing if there is more than one package source.
Task name: This is used by NuGetCommand@2
Description: As mentioned about, I removed lines that renamed a package source
Documentation changes required: (Y/N) N
Added unit tests: (Y/N) N
Attached related issue: (Y/N) I reported issue here, https://github.com/microsoft/azure-pipelines-tasks/issues/17056
Checklist:

[ ] Task version was bumped - please check instruction how to do it
[ ] Checked that applied changes work as expected

What would be the fucking point?
You find a bug.
You report the bug.
You provide a fix for the bug.
And what?

Created the same PR https://github.com/microsoft/azure-pipelines-tasks-common-packages/pull/157 in the new repo.  I will double check if this work around is still needed or not.  If not, the PR will get merged.


Version 0.4.0 fails to build

The changes in https://github.com/dignifiedquire/async-tar/pull/10 probably need to be reverted.  Attempting to update the library here: https://github.com/swift-nav/esthri/pull/188 results in a compilation error:
[2021-07-28T21:21:01.527Z] error[E0433]: failed to resolve: could not find `io` in `async_std`
[2021-07-28T21:21:01.527Z]  --> /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/async-tar-0.4.0/src/archive.rs:9:5
[2021-07-28T21:21:01.527Z]   |
[2021-07-28T21:21:01.527Z] 9 |     io::prelude::*,
[2021-07-28T21:21:01.527Z]   |     ^^ could not find `io` in `async_std`
[2021-07-28T21:21:01.527Z]
[2021-07-28T21:21:01.527Z] error[E0432]: unresolved imports `async_std::io`, `async_std::path`, `async_std::prelude`, `async_std::stream`, `async_std::task`
[2021-07-28T21:21:01.527Z]   --> /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/async-tar-0.4.0/src/archive.rs:8:5
[2021-07-28T21:21:01.527Z]    |
[2021-07-28T21:21:01.527Z] 8  |     io,
[2021-07-28T21:21:01.527Z]    |     ^^ no `io` in the root
[2021-07-28T21:21:01.527Z] 9  |     io::prelude::*,
[2021-07-28T21:21:01.527Z] 10 |     path::Path,
[2021-07-28T21:21:01.527Z]    |     ^^^^ could not find `path` in `async_std`
[2021-07-28T21:21:01.527Z] 11 |     prelude::*,
[2021-07-28T21:21:01.527Z]    |     ^^^^^^^ could not find `prelude` in `async_std`
[2021-07-28T21:21:01.527Z] 12 |     stream::Stream,
[2021-07-28T21:21:01.527Z]    |     ^^^^^^ could not find `stream` in `async_std`
[2021-07-28T21:21:01.527Z] 13 |     task::{Context, Poll},
[2021-07-28T21:21:01.527Z]    |     ^^^^ could not find `task` in `async_std`

I attempted to add a build scenario that checks for this error here: https://github.com/silverjam/async-tar/pull/3 -- which shows similar results.  FWIW I tried enabling more features in async_std to get it to compile, but it seems too many things async-tar needs are behind the "default" feature, which also enables the runtime (which is the bit the original PR was trying to avoid).  For example see: https://github.com/async-rs/async-std/blob/master/src/lib.rs#L316 which enables the fs and path module.
cc @danieleades

Hmm I'll take a look. Off the top of my head I can't think why missing crate features would allow async-tar to build, but fail compilation when it's included as a dependency

this is confusing. the standard library re-exports that your build is complaining can't be found are behind the std and default feature in async-std. The question then is why this library and its tests builds fine and passes all its tests...
if i remove the async_std dev-dependency (which has default features) then i can reproduce these errors. If i put the dev-dependency back, the build is successful. I find this behaviour very surprising. How should one test that their dependencies are complete, if dev-dependencies are always also included?
looking at the async-std source code, the runtime and the fs modules are behind the same cfg flags. it's all or nothing. I think the change does need to be reverted.
Any ideas on how we can get the CI to fail with the current setup?

see https://github.com/async-rs/async-std/issues/438
though it looks like the fs module relies on the executor. no idea why, haven't dug that far into the source code yet.

See also https://github.com/rust-lang/cargo/issues/7916
Which introduces a nightly feature that addresses this behaviour

I switched to resolver = "2" for cargo in #22 which fixes the issue of this not being discovered in CI. For now I am going back to include full async-std to make sure we have a working release.

I switched to resolver = "2" for cargo in #22 which fixes the issue of this not being discovered in CI.

good shout. i'm glad to see a concrete use-case for the new resolver!
as for the original problem, it doesn't look like async-std provides much granularity in selecting features. There may be good technical reasons for this. Or there may be bad technical reasons for this.
compare this to tokio-

Feature flags
Tokio uses a set of [feature flags] to reduce the amount of compiled code. It
is possible to just enable certain features over others. By default, Tokio
does not enable any features but allows one to enable a subset for their use
case. Below is a list of the available feature flags. You may also notice
above each function, struct and trait there is listed one or more feature flags
that are required for that item to be used. If you are new to Tokio it is
recommended that you use the full feature flag which will enable all public APIs.
Beware though that this will pull in many extra dependencies that you may not
need.

full: Enables all Tokio public API features listed below except test-util.
rt: Enables tokio::spawn, the basic (current thread) scheduler,
and non-scheduler utilities.
rt-multi-thread: Enables the heavier, multi-threaded, work-stealing scheduler.
io-util: Enables the IO based Ext traits.
io-std: Enable Stdout, Stdin and Stderr types.
net: Enables tokio::net types such as TcpStream, UnixStream and UdpSocket,
as well as (on Unix-like systems) AsyncFd
time: Enables tokio::time types and allows the schedulers to enable
the built in timer.
process: Enables tokio::process types.
macros: Enables #[tokio::main] and #[tokio::test] macros.
sync: Enables all tokio::sync types.
signal: Enables all tokio::signal types.
fs: Enables tokio::fs types.
test-util: Enables testing based infrastructure for the Tokio runtime.

Thanks for the quick turnaround!


gpio.pe2.into_open_drain_output()

I'm new to Rust and embedded programming and am trying to work through some examples. I have a  dht11 chip and so came across your repository while looking for examples of how to use it. My nucleo-64 board has a stm32f411re which does not have a pe2 pin, but I think I can use GPIOA and the pa8 pin. I am trying to understand
    let pin = gpio.pe2.into_open_drain_output();

Is it really correct that this is configured as output?  I think '.into_open_drain' might be doing an alternate function setting. Could you explain what this really does?

The function is correct, .into_open_drain_output() will configure a pin as a GPO in open-drain mode, without any alternate function attached. Without going into details, open-drain means that the pin can only force one electrical state (in this case, a logical 0), and has to rely on a pull-up resistor to put a 1 on the wire. It is commonly used for wires which can be driven by multiple sources.
In the case of the DHT11, this is kind of a trick/hack: on most microcontrollers, the electrical state of an open-drain output can also be read as an input, so basically the pin acts both as an input and an output that can only pull down the bus to a logical 0. This works well in this case due to the protocol implemented by the DTH11, which is a variation of a 1-Wire bus. If you want additional details, you could have a look at the DHT11 datasheet (section 5).
The alternative way to do this would be to switch the pin between output and input during communication, which right now is hard to do in a hardware-independent way, and would require additional traits and probably a far bit of hack-ish code. I hope I've cleared things up for you, otherwise feel free to ask.

Thank you very much for the detailed explanation!


Prevent user from editing tag groups

Solved #40

The .gitignore file contains some additional exceptions for the windows environment.


Fix CI release for main and v0.11.x branches

We have two actively maintained branches: main and v0.11.x and ci release should do patch release v0.10. for main and v0.11. for v0.11.x branches.

Make sure to bump rollkit to using .github version v0.3.0

The main challenge is the the tag release action we use doesn't natively support bumping old versions, it only bumps latest.
We should simplify and not use the CI release for patch fixing old releases.


Update data-catalog-common-scenarios.md

Thank you for your contribution. Closing this pull request unmerged, because this change was addressed in another pull request in the private repo.


[chip-test, otp_ctrl] otp_ctrl_calibration

Test point name
otp_ctrl_calibration
Host side component
Rust?
Opentitantool infrastructure implemented
Yes
Silicon Validation (SiVal)
Yes
Emulation targets

[ ] None
[ ] CW310
[ ] Hyperdebug + CW310

Contact person
Checklist
Please fill out this checklist as items are completed. Link to PRs and issues as appropriate.

[ ] Check if existing test covers most or all of this testpoint (if so, either extend said test to cover all points, or skip the next 3 checkboxes)
[ ] Device-side (C) component developed
[ ] Bazel build rules developed
[ ] Host-side component developed
[ ] Test added to dvsim nightly regression (and passing at time of checking)
[ ] For SiVal test cases, test is running relevant FPGA or silicon regression

This is tested as part of the manufacturing flow.


AppVeyor Code Coverage Failure

master AppVeyor build has been failing since #757 was merged with the following:

You are using Legacy parameter set that adapts Pester 5 syntax to Pester 4 syntax. This parameter set is deprecated, and does not work 100%. The -Strict and -PesterOption parameters are ignored, and providing advanced configuration to -Path (-Script), and -CodeCoverage via a hash table does not work. Please refer to https://github.com/pester/Pester/releases/tag/5.0.1#legacy-parameter-set for more information.
The running command stopped because the preference variable "ErrorActionPreference" or common parameter is set to Stop: Please provide pester results with code coverage using the -CodeCoverage parameter

@JanDeDobbeleer @rkeithhill you guys know the Pester world better than I do. Ideas?

Does the AppVeyor build not run for a PR?  I guess not because PRs have been passing their builds after that PR was merged.
BTW this is not using the Legacy parameter set:
Invoke-Pester -Configuration $configuration -ErrorAction SilentlyContinue

That is the Advanced parameter set (see below).  Could it be the -ErrorAction SilentlyContinue that is making Pester think that the Legacy parameter set is being used?  That seems to be a stretch.
     {CodeCovera… Named    False     False    False  All             String
CodeCoverageOutputFil… {CodeCovera… Named    False     False    False  All             String
EnableExit             {En*}        2        False     False    False  All             SwitchParameter
ExcludeTagFilter       {ExcludeT*}  Named    False     False    False  All             String[]
FullNameFilter         {Name, N*}   1        False     False    False  All             String[]
OutputFile             {OutputFi*}  Named    False     False    False  All             String
OutputFormat           {OutputFo*}  Named    False     False    False  All             String
PassThru               {Pas*}       Named    False     False    False  All             SwitchParameter
Path                   {Script, Sc… 0        False     False    False  All             String[]
PesterOption           {Pe*}        Named    False     False    False  All             Object
Quiet                  {Q*}         Named    False     False    False  All             SwitchParameter
Show                   {Sh*}        Named    False     False    False  All             OutputTypes
Strict                 {St*}        Named    False     False    False  All             SwitchParameter
TagFilter              {Tags, Tag}  4        False     False    False  All             String[]

    Command: pester/Invoke-Pester
    Set:    Advanced

Name                   Aliases      Position Mandatory Pipeline ByName Provider        Type
----                   -------      -------- --------- -------- ------ --------        ----
Configuration          {Con*}       Named    False     False    False  All             PesterConfiguration

I suppose a quick fix would be to use the same "invocation" approach we use in Travis?

Does the AppVeyor build not run for a PR? I guess not because PRs have been passing their builds after that PR was merged.

Code Coverage secret isn't available in PRs, so that part of the build doesn't run except on AppVeyor on master.

#809 didn't work:

Please provide pester results with code coverage using the -CodeCoverage parameter

CodeCoverage doesn't seem to be part of results?

#809 didn't work:

Please provide pester results with code coverage using the -CodeCoverage parameter

CodeCoverage doesn't seem to be part of results?