Make timeline and deadline dates fully data driven

Changed the Settings to add a second exclusion so the main config page can ignore the timeline dates (they begin with 'config.timelineDates.') and to add a tab for Timeline Dates filtering on the same. The config values are meant all to be dates. As code and value are both 100 characters and the database doesn't care I use the code as the label to display. Finally added an entry to the configuration to read the config.timelineDates values and add an array of label and date entries.
Created a CutoffDateWarning component that takes a date key (i.e. the label) and if the date value of that is in the past it shows a warning not to register without first contacting registration by email to the email in ContactEmail configuration. If that date is in the future it shows nothing.
Created a TimelineList component that takes the array of lable/date values, and an optional ignore before date. Again, it's a label of any of the dates. It takes the array, filters out any dates before the date with that label, sorts it in ascending order on the dates and displays them. If the date is in the past it uses the class deadlineExpired, otherwise deadline.
Changed the class deadlineExpired from red (error.main) to a 700 gray with strike through.
In use it cleans up the Welcome page.
ACUS' is now:
<Page title='Welcome' titleElement={titleElement}>
  <MdxWithExternalLinks>
    <WelcomeContent />
  </MdxWithExternalLinks>
  <DynamicMemberContent />
  <CutoffDateWarning cutoffDateConfig='Membership Payment In Full Due' />
  <TimelineList ignoreBeforeDateConfig='Registration Open' />
</Page>

More imortantly, the cutoff date warning doesn't require reading and interpreting, but shows as needed.
And the timeline dates are easily maintained in settings and ones can be added/removed, or, for this year, left out, by leaving their value set to last year's.
Resolves: #166, #167

Builds fail for lack of the timelineDates in configuration.
But ACUS fails on code that didn't change because configuration[node] is unhappy. I think the zod schema definition of timeline dates and not use the z.record type as it flows through to places where configuration[x] of configuration.x now get undesirable possible values.

Canceling.
Need to consider how to strictly validate the config, and have this, while "Create a new year" needs to be set up.


Error handler callstack bump to fix GH-191

This fixes #191.  Note that I couldn't figure out how to run the tests, so I can't see whether the test actually passes or not (it did when I tried the test code in minisphere).

The usual tests I do before a merge is make apitest and make qecmatest which has decent coverage, especially if you run with assertions enabled (I just edit the Makefile locally to uncomment -DDUK_OPT_ASSERTIONS).
make codepolicycheck is also useful because it catches various nits; in particular it reports FIXME tags in code which I use as a mini task list in branches: as a first step I often just add FIXMEs for things to figure out. make codepolicycheck will then remind me what's still to be done. Finally, I rebase to make the commits clean for merge.
For 1.4 I'm going to add an automatic test run for pull requests, probably Travis at least initially, so that it'd be easier to see pull request status.

I use Travis for minisphere, it's very nice.  Since I develop on Windows, it's useful as a way to ensure me I didn't break the build on Linux (if it builds on Linux it will usually build on OSX as well).

I've also used Travis in other projects and it mostly works well. However, there are Duktape test runs that take hours and hours so it won't be enough by itself. Also it'd be nice to have test coverage for all three major platforms and one or two exotic ones (including a "mixed endian" ARM). But Travis is an easy first step.

Ok, I can run the tests too as soon as I get off work.

Re: That ARM platform: There really are mixed endian platforms?  I thought those were a myth! ;-). Mixed endian never made sense to me, how do you make the hardware efficient with the bytes shuffled around like that?

I'm working with several of them. CPUs fetch data into cache lines anyway and shuffling is more or less cost free. That's why there are also configurable endianness processors.

Yeah, you'll have to run the tests, I can't get them going on Cygwin, there's a long string of these before it finally fails:
{ [Error: ENOENT, no such file or directory 'D:\tmp\runtests-7220-1']
  errno: -4058,
  code: 'ENOENT',
  path: 'D:\\tmp\\runtests-7220-1',
  syscall: 'open' }

Followed by:
Testcase execution done, running diffs
D:\src\duktape\runtests\runtests.js:571
            if (res.stdout !== res.testcase.expect) {
                   ^
TypeError: Cannot read property 'stdout' of undefined
    at <node stacktrace>
Makefile:523: recipe for target 'apitest' failed
make: *** [apitest] Error 1

Some API tests currently write to /tmp which is not nice. make qecmatest doesn't do that though.
Anyway, I'll run the tests.

Btw, add yourself to AUTHORS.rst. It's both a nice list of contributing people but also relevant to licensing.

Oh, no problem.  I should probably rebase this to master while I'm at it.

I'm running the tests now; it'll take a while for the assertions enabled test.

I think there are some obscure corner cases that will break with finalizer torture here: when the longjmp state is set up, there may be a finalizer (at least from the point of view of torture testing which sometimes generates cases that are not actually triggerable) which would cause another longjmp while preparing for this one.
But if that's the case I'll fix the issue as I deal with the other torture test issues in #314.

It might be safer to use callstack_max = DUK_DEFAULT_CALLSTACK_MAX + 10 rather than += so that if assertions are disabled and there'd be some control flow bug, recursive error calls wouldn't keep bumping the limit upwards (potentially without limit). Using a straight assignment would eventually break out of such a process.

Good point.
On a sidenote, I actually found I had to add a little more than 10, as Duktape apparently sometimes adds calls in-between.  For example the recursion limit is 10000 but the recursive f() function throws at ~9990.

You have Windows newlines in the test case at least - could you check that both source and testcase are clean of them? CR LF pairs cause the testcases to break now.
(I need to get that git attributes thing in place.)

Oh, hm.  I know Windows Git converts them to LF before commit, Cygwin Git apparently does not.  I use Notepad2 for editing, which preserves Unix newlines for existing files but creates new files as CR LF.
I'll fix that too. :)

I think that's related to the callstack growth - it throws when the callstack size would go over the maximum, e.g. if it were to resize from 9990 to 10020, that would trigger the callstack limit. It's intentionally tied to the resize so that it's only checked from time to time.

Codepolicycheck detects CR FL newlines, and covers API tests but not yet Ecmascript tests because the FIXMEs haven't been cleaned up yet. Maybe I could finally clean them up for 1.3 release so that newline errors would be easy to detect in pull requests.

This is the full whine BTW, there's also a non-leading tab on line 25:
vim +1     tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - /*
vim +2     tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - *  Error handling callstack headroom (GH-191)
vim +3     tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - *
vim +4     tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - *  When a RangeError is thrown due to the Ecmascript callstack limit being
vim +5     tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - *  reached, there should be enough headroom for at least 10 further recursions.
vim +6     tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - */
vim +7     tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace -
vim +20    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - ===*/
vim +21    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace -
vim +22    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - try {
vim +23    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - function recurse(n) {
vim +24    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - print(n);
vim +25    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - if (--n > 0) recurse(n);
vim +26    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - }
vim +26    tests/ecmascript/test-err-callstack-headroom.js : non-leading tab (idx 25) - }
vim +27    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace -
vim +28    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - Duktape.errCreate = function(e) {
vim +29    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - recurse(10);
vim +30    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - return e;
vim +31    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - };
vim +32    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - function f() { f(); }
vim +33    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - f();
vim +34    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - }
vim +35    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - catch(e) {
vim +36    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - print(e);
vim +37    tests/ecmascript/test-err-callstack-headroom.js : trailing whitespace - }
*** Total: 26 problems

I actually fixed all that (I saw the code policy update when I went to rebase), and also added an additional assert to verify the bump.  I didn't add an assert before the longjmp because it wasn't really clear what to do there.
Rebased and force pushed the branch.

Oops, just realized I forgot to make the test custom. :(

You can fix that or not -- the idea is that if comparison tests are executed for other engines, custom test errors won't cause warnings (because those engines won't be able to run the test). But the testcases are not "pure" in this respect now. So feel free to fix you need to touch something or leave as is if not.

There's now a basic Travis setup in master. For now it's just codepolicycheck, but should still be useful to catch trivia in pull requests. (If you rebase there should be a Travis run for the pull.)

Ok, apitest and qecmatest pass with assertions enabled (two tests time out with current timeout limits but that's normal).

Including the new test for this fix?

Yeah, I checked out the branch from your repo.

Hmm, you're still missing from AUTHORS?

Yeah, sorry, haven't been near a computer for the past few hours.  I'll fix it in a bit, along with adding a custom tag to the testcase.

Didn't mean to pester you - I'm going to sleep myself so no hurry from my side :) I added apitest and qecmatest to master Travis tests too, they take about 5 minutes in total. I'd like to run the tests with assertions enabled but that'd probably take too long to be the default for every push.

That must be why my debug builds got so slow recently - I enabled assertions.  They really do slow everything way down!

Yes, assertions are slow because they apply to many low level operations like refcounts. Also some pre- and post-checks are quite expensive but important for asserting state.

Okay, just rebased again and added myself to AUTHORS.rst.  Looks like all tests check out, except for one:
test-to-number.c: fail; 15 diff lines; known issue: number parsing bug for strings containing NUL characters (e.g. '\u0000')

Since it says "known issue" though, I assume this one isn't my fault. :-)

Yes, known issues are known issues and don't cause a test failure. I don't want to hide them from the test output though, because it'd be easy to forget they exist :)

That should do it.  The comment after the assert bugs me because it's so far past the 80-column mark but it didn't feel right adding a whole new line just for the short comment.

Looks merge ready except for one minuscule nit: "reset reclimit" should probably be "reset callstack limit", reclimit refers to the heap native call recursion limit.

Okay, that was easy enough. :)

Sorry if I kept you from sleep. ;-)

I kept myself from sleep improving the code policy checks and Travis stuff. Also webhooks in preparation for a repository-based Wiki setup :)

It occurs to me that the bug testcase for this is not strictly correct--the bump only guarantees 10 additional entries while the test actually requires 12--1 for the errCreate, 10 for the recurse() call and another for the print().


fix: update ansi-regex resolution

Fixes

Fixes #123
See snyk notes for why I think this is a fix.

cc: @StudioMaX @apepper

Read about referenced issues here. Replace words with this Pull Request's context.

Good to see progress in this area!
AFAIK resolutions is a yarn only feature (see e.g. https://stackoverflow.com/q/52416312 ). There is an overrides npm flag in the making but it will probably take weeks to months until this is implemented.

@apepper pnpm supports resolutions too.
View the pnpm-lock.yaml in this PR for reference. 👍

Npm's "overrides" is now available. I created a pull-request to fix the warning for npm - https://github.com/yowainwright/es-check/pull/155.


ingest: Automated workflow failing during fetch from NCBI

Blocked by upstream issue: https://github.com/ncbi/datasets/issues/419
Failed during automated ingest on 2024-10-31.
[batch] [2024-10-31T17:11:13+00:00] Error: Download error: stream error: stream ID 3; INTERNAL_ERROR; received from peer

I manually re-ran the workflow thinking it was a transient error, but it hit the same error.
[batch] [2024-10-31T18:19:14+00:00] Error: Download error: stream error: stream ID 3; INTERNAL_ERROR; received from peer

Then realized this was already reported as an error in https://github.com/ncbi/datasets/issues/419.

Unblocked probably, at least in my tests

The automated ingest hit the error again yesterday, but seems to be running fine today

Ingest failed yesterday and today with the same error again.

Yeah I already reported (reopened my old issue) over at their Github issue tracker

Ingest completed successfully today


🛠 Tooling: add package-lock to gitignore

Bug Report Checklist

[X] I have tried restarting my IDE and the issue persists.
[X] I have pulled the latest main branch of the repository.
[X] I have searched for related issues and found none that matched my issue.

Overview
Since we use pnpm in our tooling, but it's totally reasonable to expect that other folks might use npm out of habit when getting started with development on the site, we could put the package-lock in gitignore to avoid any accidental commits of a lockfile.
Additional Info
No response

Hi, do you need help. May I pick up this task?


Migrate Identity to azure-json

In addition to the migration there's a couple other changes in here (that were prompted by these things needing migration, and then me realizing we just don't need them anymore.)

Removal of legacy storage mechanism for credentials for the IntelliJCredential. (all the kdbx stuff.)
Removal of a wrapper type we had for missing MSAL functionality which has since been implemented.

Also includes a fix to azure-json to allow parsing jsonc, which we need for VisualStudioCodeCredential.
Fixes #39799

/check-enforcer evaluate

/check-enforcer evaluate

API change check
API changes are not detected in this pull request.

/azp run azure - identity

/azp run java - identity

/azp run java - identity

/azp run java - identity

/azp run java - identity

/azp run java - identity


:hour-range usage

Hello,
I have the mix and max range saved by the user from another form input. How do I use this in the hour-range?
I see :hour-range only takes numeric values and not variable names that hold values.
For example, if starttime and endtime hold the hour-range, how can I use it with vue-timepicker?
Thanks

I see :hour-range only takes numeric values and not variable names that hold values.

You can create a data variable, or a computed variable, and set it as the value for :hour-range
data () {
  return {
    // Accepts value from other inputs
    startHour: 8,
    endHour: 17,
    ...
  }
},

computed: {
    myHourRange () {
      return [[this.startHour, this.endHour]]
    },
    ...
}

Then in the HTML template --
<vue-timepicker :hour-range="myHourRange"></vue-timepicker>


Typo in bridge icon reference

As shown in the screenshot there is a typo when referencing the icon for the tests-finished event.

Environment

Bridge version:<PHONE_NUMBER>44

Affected Component

[ ] Docs
[ ] CLI
[ ] Kubernetes Integration
[ ] Openshift Integration
[ ] Helm
[ ] Istio
[x] Bridge
[ ] Gatekeeper
[ ] Datastore
[ ] REST API
[ ] eventbroker / distributor
[ ] jmeter

The icon is called perfromance-health in barista components. So I guess the typo is in barista components, we're just victims :D

No action required.


Adds .src and .dest methods, parallel compilation option

Built on #45, since these changes were originally open against it, but @rbuckton hasn't had time to work on it.

Adds a src() method to IncrementalCompiler that returns a vinyl-fs stream for the source files detected by the compiler. As a result, incrementalCompiler.src() can be used in place of gulp.src() to collect expected build inputs.
Adds a dest() method to IncrementalCompiler that returns a vinyl-fs stream to write compilation outputs to the output directory specified in tsconfig.json (either outDir, or the parent directory of outFile or out).
Adds a compile() method to IncrementalCompiler that calls itself.
Adds support to the create() function to also accept a path pointing to a directory containing tsconfig.json.
Reworks the IncrementalCompiler into a class-like to leverage the class prototype, while maintaining the fact that IncrementalCompiler is callable.
Switched from typings to @types for dependent declaration files.
Added a "typings" entry to package.json.
Enables noImplicitAny, strictNullChecks, noUnusedLocals, and noImplicitThis on the project's TS compilation.
Includes polyfills for Object.assign and Promise such that older versions of node (0.12) can still run gulp-tsb.
Adds a "parallel" compilation option, which causes a worker thread to be spun up to run the compilation on, rather than stalling the main thread with the build.

I still need to see if accepting the new baselines was all I had to do to fix the cross-plat line ending bug - if travis is green, then that's fixed, I think.

Great - can you resolve merge conflicts?


chore(deps): update module github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper to v1.8.6

This PR contains the following updates:

Package
Type
Update
Change

github.com/terraform-ibm-modules/ibmcloud-terratest-wrapper
require
patch
v1.8.4 -> v1.8.6

Release Notes

terraform-ibm-modules/ibmcloud-terratest-wrapper
v1.8.6
Compare Source
Bug Fixes

deps: update github.com/ibm-cloud/bluemix-go digest to b58456b (#496) (d4cd5fc)

v1.8.5
Compare Source
Bug Fixes

deps: update module github.com/ibm/platform-services-go-sdk to v0.35.0 (#494) (020338a)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 2.5.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[SDL 0158] Cloud App Transport Adapter

Proposal: Cloud App Transport Adapter

This proposal will detail a possible solution for allowing SDL-enabled cloud applications to be used in a vehicle. This proposal will cover the process of obtaining cloud application server endpoints, enabling which cloud apps should appear in the HMI app list, opening connections between Core and a cloud based app server, and how a cloud application can authenticate connected head units.

Review Issue: smartdevicelink/sdl_evolution#493
Revisions to the review issue: https://github.com/smartdevicelink/sdl_evolution/issues/666
Steering Committee Decision:

The Steering Committee voted to accept this proposal with revisions. The revisions will include changing the app_icon_url policy table attribute to icon_url as mentioned in this comment, include the affecting platforms based on the protocol changes mentioned in this comment, and rename cloudAppAuthToken to authToken noted in this comment.
Additionally, the author stated that OEMs have the ability to specify secure websockets or unsecured websockets whether you have WS or WSS specified in cloud_transport_type in the policy table.

Revisions have been made

Closed via #147


[AutoRelease] t2-dnsresolver-2022-12-27-51748(can only be merged by SDK owner)

https://github.com/Azure/sdk-release-request/issues/3592
Live test success
https://dev.azure.com/azure-sdk/internal/_build?definitionId=4423

issue link:https://github.com/Azure/sdk-release-request/issues/3592


Provide concrete example of building custom events

To help us debug your issue fill in the basic information below using the options provided
Serverless Java Container version: eg. 1.5
v1 (Spring Boot 2) and v2 (Spring Boot 3.x)
Implementations: Jersey / Spring / Spring Boot / Spring Boot 2 / Spark
Spring Boot
Framework version: eg SpringBoot 2.2.6.RELEASE
Spring Boot 2.x and Spring Boot 3.x
Frontend service: REST API / HTTP API / ALB
VPC Lattice
Deployment method: eg SAM, Serverless Framework, Console
N/A
Scenario
Describe what you are trying to accomplish
This is a more general ask related to this issue. The documentation provided here indicates that just four classes need to be implemented to build the custom class. However, pursuant of trying to implement the VPC Lattice event, I discovered that in addition to creating the POJOs for the event model itself, I needed to also implement ExceptionHandler, SecurityContext, SecurityContextWriter. It would helpful to include a practical example of how to create the custom event, outline the minimum and optional attributes for the servlet request, and how to set fields for the response payload.
Expected behavior
Describe how you would expect the application to behave
Ability to simply or at least clear extend the framework with custom event payloads
Actual behavior
Describe what you are seeing instead
Difficulty following the documentation

Thanks for raising this. I'll discuss this internally with the team and get back to you


SimpleITK, ITK and Examples cannot be built

Hi.
I am trying to build SimpleITK to use it with C++.  I have tried to build on Windows 10 with Visual Studio 2019, following the guide on https://simpleitk.readthedocs.io/en/master/buildingWindows.html . I have done everything that is written in the document and nothing else. To provide some more information, here is the result after configuring and generating build files with Cmake 3.19.4

After this, I build the ALL_BUILD.vcxproj with release mode and I get following errors:

I am unable to see something out of ordinary so any help would be appreciated.

It's hard to see what the problem is from just your screen capture.  Could you try building SimpleITK from the command line and then sharing the entire output here?
To build from the command line open up Windows PowerShell and enter this:
cd C:\Users\YBurakD\Desktop\SimpleITK\SimpleITK-build
cmake --build . --config Release

Also, as described in the instructions, it's better to not place the build directory so deep in the file system.  CMake can have problem with long path names.  It's better to have the build directory be something like C:\SimpleITK-build, right at the top of the file system.


ISPN-11241 Deprecate Indexing mode

https://issues.redhat.com/browse/ISPN-11241

Oops, another rebase is needed

Rebased

I've added several comments. It's generally ok. I'd like to have at least the ones regarding naming of configs and methods resolved before merging. I've done that myself, locally, because I believe Gustavo might be PTO today. I'll merge once I see my local test run is fine.

It seems odd that an new xml attribute was added but the Serializer is not updated in this PR. Definitely an issue!

It seems odd that an new xml attribute was added but the Serializer is not updated in this PR. Definitely an issue!

Let me check that

The reason the serializer works is because the changes are backwards compatible, it does not remove any config and can read and interpret the old config, and also it produces the XML with the "old" config.
Originally I develop this PR for 10.1.x branch, but now I am not sure it will be there given the late product cycle...

Addressed comments

All good now. merged! Thanks @gustavonalle !


RELATED: RAIL-3048 Upgrade SDK to 8.5.0

JIRA: RAIL-3048

The code coverage is the same after your change 🤟
This is a good thing if you are not sure ;)
Report (compared to build #262)
Covered lines: 23.39%, change: 0%
Covered branches: 38.46%, change: 0%
Coverage report
https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/265/artifact/ci/results/coverage/lcov-report/index.html

Build succeeded (check pipeline).

https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/265/ : SUCCESS in 1m 05s
https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-oss-check/215/ : SUCCESS in 7s

The code coverage is the same after your change 🤟
This is a good thing if you are not sure ;)
Report (compared to build #262)
Covered lines: 23.39%, change: 0%
Covered branches: 38.46%, change: 0%
Coverage report
https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/266/artifact/ci/results/coverage/lcov-report/index.html

Build succeeded (gate pipeline).

https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-merge-compliance-check/100/ : SUCCESS in 16s
https://checklist.intgdc.com/job/client-libs/job/gooddata-create-gooddata-react-app-unit-tests-zuul-docker-el7/266/ : SUCCESS in 1m 03s
https://checklist.intgdc.com/job/compliance/job/gooddata-create-gooddata-react-app-fossa-scanning-zuul-el7/65/ : FAILURE in 4m 18s (non-voting)


How to use Web API (json) in angular2-infinite-scroll

You can show me. simple way to use Web API (json) in angular2-infinite-scroll

please look at the example plunkr.
for using http requests utilizing json - please consult other resources.


Use updated seleniarm docker image

The Issue
The Docker Composer file is pinned against a really old version of the seleniarm image.
To do: update documentation regarding Docker image version.
How This PR Solves The Issue
Updates the tag for the image to latest, and users can pin to a specific version if desired.
Manual Testing Instructions
Update docker-compose.selenium-chrome.yaml in an existing project to use the latest tag and run ddev poweroff && ddev start
Automated Testing Overview

Related Issue Link(s)
Release/Deployment Notes

Code looks good. Unclear to me what is the cause of the test failure - https://github.com/ddev/ddev-selenium-standalone-chrome/actions/runs/6483056158/job/17603806583?pr=31#step:12:465

Maybe it's pickier about YAML now? Trying quotes around the image tag.

117.0 seems to be a movable tag as well. Should it be something like 117.0-chromedriver-117.0-grid-4.10.0-20230926?

Usually tags like 117.0 may be updated but will stay with the same basic version of the local stuff. For example, solr:8 stays with solr 8. Or solr:8.8 might stays with solr 8.8*

In local testing, I found that PHPUnit would hang before trying to start a Functional JavaScript test, which I believe is what is happening now on CI. Using noVNC, it appears the browser is not starting at all (with the --headless option removed from our driver args).

I'll just close this until we (or someone else) can find an updated seleniarm Docker image that works. It would be nice to test with a newer version of Chromium (especially >=112 for the updated --headless=new mode), but this does work as-is. I wonder if a simpler image with just Chromium and ChromiumDriver would suffice. It's my understanding that Behat needs to use Chrome directly through the remote debugging port, while PHPUnit needs to access Chrome through a web driver. I believe that is due to differences in how each implements user login steps.


Feature Request: Ability to Customize "Actively Exploited CVEs:" Text

In the <IP_ADDRESS>614 pre-release, when utilizing SOFA functionality the deadline is set to either 14 or 28 days, depending on if CVE fixes are associated with it.
If an update has CVE fixes included is now indicated in the GUI of Nudge. Currently the display text isn't customizable. It shows "Actively Exploited CVEs: True" or "Actively Exploited CVEs: False". Having the ability to customize or hide that text could be beneficial for various organizations.

I think I will add an option to hide this UI, but not to change the text as the left side has traditionally been hardwired in the code and as soon as I open this one up, I have to open the others up..

showActivelyExploitedCVEs will be in the next release


Required if with uploaded file does not work

Hi!
When I try
'document_snils.*' => 'required|uploaded_file:0,10M,pdf,jpeg,jpg', , required flag works.
When I try
'document_snils.*' => 'required_if:type,entrepreneur|uploaded_file:0,10M,pdf,jpeg,jpg', required_if flag does not work.

Yeah, doesn't work for me too.

This is also true for required_with, so I assume (without testing) that all required_* rules fail for files.


Add CLI Settings Source

Resolves #209.
This PR implements a CliSettingsSource with the following main features outlined below. Refer to docs/index.md for more details.
Current Examples
Fields

Same expectations as setting environment variables
Proposed _cli_parse_args can accept arg array or bool set to True for sys.argv

from pydantic import BaseModel

from pydantic_settings import BaseSettings

class DeepSubModel(BaseModel):
    v4: str

class SubModel(BaseModel):
    v1: str
    v2: bytes
    v3: int
    deep: DeepSubModel

class Settings(BaseSettings):
    v0: str
    sub_model: SubModel

print(Settings(_cli_parse_args=[
    '--v0=0',
    '--sub_model={"v1": "json-1", "v2": "json-2"}',
    '--sub_model.v2', 'nested-2',
    '--sub_model.v3', '3',
    '--sub_model.deep.v4', 'v4'
]).model_dump())
'''
{'v0': '0', 'sub_model': {'v1': 'json-1', 'v2': b'nested-2', 'v3': 3, 'deep': {'v4': 'v4'}}}
'''

Lists

Proposed intermixing of three list styles:

JSON style --field='[1,2,3]'
Argparse style --field 1 --field 2 --field 3
Lazy style --field=1,2,3

from pydantic import BaseModel
from typing import List
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    my_list: List[int]

print(Settings(_cli_parse_args=[
    '--my_list', '[1,2]',
    '--my_list', '3,4',
    '--my_list', '5', '--my_list', '6'
]).model_dump())
'''
{'my_list': [1, 2, 3, 4, 5, 6]}
'''

Dictionaries

Proposed intermixing of two dictionary styles:

JSON style --field='{"k1": 1, "k2": 2}'
Env Var style --field k1=1 --field k2=2

These styles can be used in conjunction with list forms as well:

--field k1=1,k2=2 --field k3=3 --field '{"k4: 4}' etc.

from pydantic import BaseModel
from typing import Dict
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    my_dict: Dict[str, str]

print(Settings(_cli_parse_args=[
    '--my_dict', '{"k1": "a", "k2": "b"}',
    '--my_dict', '{"k3": "c"}, {"k4": "d"}',
    '--my_dict', '{"k5": "e"}', '--my_dict', '{"k6": "f"}',
    '--my_dict', '[k7=g,k8=h]',
    '--my_dict', 'k9=i,k10=j',
    '--my_dict', 'k11=k', '--my_dict', 'k12=l',
]).model_dump())
'''
{'my_dict': {
    'k1': 'a',
    'k2': 'b',
    'k3': 'c',
    'k4': 'd',
    'k5': 'e',
    'k6': 'f',
    'k7': 'g',
    'k8': 'h',
    'k9': 'i',
    'k10': 'j',
    'k11': 'k',
    'k12': 'l',
}}
'''

Subcommands and Positionals

Proposed help text generation based off of field descriptions and class documentation
Proposed _cli_prog_name has same meaning as in argparse
Proposed CliSubCommand annotation for marking subcommands
Proposed CliPositionalArg annotation for marking positional arguments

from pydantic import BaseModel
from typing import Tuple, Type
from pydantic_settings import BaseSettings, CliSubCommand, CliPositionalArg, PydanticBaseSettingsSource

class FooPlugin(BaseModel, use_attribute_docstrings=True):
    '''git-plugins-foo - Extra deep foo plugin command'''

    my_feature: bool = False
    '''Enable my feature on foo plugin'''

class BarPlugin(BaseModel, use_attribute_docstrings=True):
    '''git-plugins-bar - Extra deep bar plugin command'''

    my_feature: bool = False
    '''Enable my feature on bar plugin'''

class Plugins(BaseModel, use_attribute_docstrings=True):
    '''git-plugins - Fake plugins for GIT'''

    foo: CliSubCommand[FooPlugin]
    '''Foo is fake plugin'''

    bar: CliSubCommand[BarPlugin]
    '''Bar is also a fake plugin'''

class Clone(BaseModel, use_attribute_docstrings=True):
    '''git-clone - Clone a repository into a new directory'''

    repository: CliPositionalArg[str]
    '''The repository to clone'''

    directory: CliPositionalArg[str]
    '''The directory to clone into'''

    local: bool = False
    '''When the resposity to clone from is on a local machine, bypass ...'''

    shared: bool = False
    '''Force the clone process from a reposity on a local filesystem ...'''

class Init(BaseModel, use_attribute_docstrings=True):
    '''git-init - Create an empty Git repository or reinitialize an existing one'''

    directory: CliPositionalArg[str]
    '''The directory to clone into'''

    quiet: bool = False
    '''Only print error and warning messages; all other input will be suppressed'''

    bare: bool = False
    '''Create a bare repository. If GIT_DIR environment is not set, it is set ...'''

class Git(BaseSettings, use_attribute_docstrings=True):
    '''git - The stupid content tracker'''

    clone: CliSubCommand[Clone]
    '''Clone a repository into a new directory'''

    init: CliSubCommand[Init]
    '''Create an empty Git repository or reinitialize an existing one'''

    plugins: CliSubCommand[Plugins]
    '''Fake GIT plugin commands'''

print(Git(_cli_prog_name='git', _cli_parse_args=['--help']))
'''
usage: git [-h] {clone,init,plugins} ...

git - The stupid content tracker

options:
  -h, --help            show this help message and exit

subcommands:
  {clone,init,plugins}
    clone               Clone a repository into a new directory
    init                Create an empty Git repository or reinitialize an existing one
    plugins             Fake GIT plugin commands
'''

print(Git(_cli_prog_name='git', _cli_parse_args=['clone', '--help']))
'''
usage: git clone [-h] [--local bool] [--shared bool] REPOSITORY DIRECTORY

git-clone - Clone a repository into a new directory

positional arguments:
  REPOSITORY     The repository to clone
  DIRECTORY      The directory to clone into

options:
  -h, --help     show this help message and exit
  --local bool   When the resposity to clone from is on a local machine, bypass ...
  --shared bool  Force the clone process from a reposity on a local filesystem ...
'''

print(Git(_cli_prog_name='git', _cli_parse_args=['plugins', 'bar', '--help']))
'''
usage: git plugins bar [-h] [--my_feature bool]

git-plugins-bar - Extra deep bar plugin command

options:
  -h, --help         show this help message and exit
  --my_feature bool  Enable my feature on bar plugin
'''

@hramezani and @samuelcolvin
I am most of the way through but hit this limitation in the underlying environment variable parsing:

JSON is only parsed in top-level fields, if you need to parse JSON in sub-models, you will need to implement validators on those models.

Obviously, this would be very nice to have from a CLI perspective (e.g., nested lists etc.). I didn't know this was the case until I started writing nested tests. Do you have suggestions on what the best course of action is here? Ideally, I would love for it to be handled in environment parsing. If not, from CLI perspective the next easiest path would be nested dictionaries of strings. Is that something I can throw to Pydantic and have a reasonable expectation it will sort it out?

Sorry @hramezani, I should have been more specific with my question. I am at a fork and am looking for guidance on which direction to take.
Is there any background on why nested JSON isn't supported in EnvSettingsSource? Was there something that scared us away from adding it? Is it a massive pile of work? etc.
My preference would be to solve it there so that the CLI and ENV are consistent. Just trying to ascertain if that is a reasonable direction or not.

Got it. So it's either a bug or something invalid in the test case. Below is an example of the issue I am seeing:
import os
from pydantic import BaseModel
from typing import Optional, List
from pydantic_settings import BaseSettings

class Obj(BaseModel):
    val: int

class Child(BaseModel):
    num_list: Optional[List[int]] = None
    obj_list: Optional[List[Obj]] = None
    str_list: Optional[List[str]] = None
    union_list: Optional[List[Obj | int]] = None

class Cfg(BaseSettings, env_nested_delimiter='__'):
    num_list: Optional[List[int]] = None
    obj_list: Optional[List[Obj]] = None
    union_list: Optional[List[Obj | int]] = None
    str_list: Optional[List[str]] = None
    child: Optional[Child] = None

os.environ['CHILD__NUM_LIST'] = '[1,2,3]'
print(Cfg().model_dump())
'''
Traceback (most recent call last):
  File "/home/kschwab/pydantic-settings/counter_example.py", line 23, in <module>
    print(Cfg().model_dump())
  File "/home/kschwab/pydantic-settings/pydantic_settings/main.py", line 93, in __init__
    super().__init__(
  File "/home/kschwab/.local/lib/python3.10/site-packages/pydantic/main.py", line 171, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Cfg
child.num_list
  Input should be a valid list [type=list_type, input_value='[1,2,3]', input_type=str]
    For further information visit https://errors.pydantic.dev/2.6/v/list_type
'''

It's a bug. Here is the related issue

The bulk of the functionality is now complete. Will continue to work on cleanup, comments, docs etc. over next few days. The only key remaining feedback items needed for any adjustments are:

Subcommand and Positional Argument Implementation (refer to GIT CLI example for reference)

[ ] Does the CliSubCommand and CliPositionalArg annotations for denoting their respective meanings look OK?

There are a few ways to accomplish this. I took the stance of a more explicit annotation at the user level for denoting subcommands and positional arguments. Overall, it is quite clean and makes intent clear when reading the model. The alternative approach is implicit coercion, with union fields into subcommands and required fields into positionals. This takes flexibility away from user, who may want a similar experience to that as environment variable overrides. Therefore, the default experience is in line with how Pydantic is already structured, and deviation from the expectation is expressed through these annotations.

[ ] Is single subparser per model OK?

I took an opinionated stance of only supporting a single subcommand group (single subparser) per model. If we want to support multiple subparsers, we would likely need to go back to an earlier version I had here. I changed from that direction however because:

Better control of generated help text. With a union implementation, the help text for the subcommand gets pulled from the class doc, which works so long as it is not long. However, in our application, we work with larger models that do have verbose class documentation so it can flood the help text. Simple workaround would be parsing the first sentence of the class doc etc. but not worth it IMO. With current implementation, the subcommand help text is the field doc string making it very straightforward. Then, the class doc itself is displayed when using help on the subcommand itself. e.g. git --help shows condensed help for the clone subcommand, whereas git clone --help shows the expanded class doc help text there.
Explicit naming of subcommands. In union implementation, the subcommand name must be coerced from the class name or some other mechanism. With current implementation, it is whatever the subcommand field name is.
Avoids the need for Discriminated Unions. Absolutely an option but requires more boilerplate, which takes away from the clean aesthetic. This was actually the forcing issue that caused me to change direction. The normal union selection was not strong enough so users would have needed to add discriminated unions for any subcommand.

[ ] Positional arguments always placed before subcommands?

Another opinionated stance, although of smaller concern. As is today, there is no requirement with regards to field ordering within the definition of a model. However, the resultant CLI will reorder positional args before subcommands. This is mostly to prevent users from shooting themselves in the foot. Perhaps this is best to just change to a flag, i.e., turn the safety off?

Outside of the above, I think everything else is pretty straightforward and aligns with precedence already set by environment variables.

Hi @hramezani, need some help with resolving the below:

Many of the doc examples use CLI --help to view generated help text etc. However, this results in an exit, which pytest does not like. How to update pytest such that exit is allowed in these expected cases?
Many of the examples use the recently merged use_attribute_docstrings flag, which does not appear available in tests. How to update the tests such that the pydantic version includes this commit?
Is there a way to generate the docs locally for viewing?

Many of the doc examples use CLI --help to view generated help text etc. However, this results in an exit, which pytest does not like. How to update pytest such that exit is allowed in these expected cases?

You can use
```py test="skip"

to skip running on those blocks

Many of the doc examples use the https://github.com/pydantic/pydantic/pull/6563 use_attribute_docstrings flag, which does not appear available in tests. How to update the tests such that the pydantic version includes this commit?

This will be available on Pydantic 2.6. It is in beta and will be released soon. We are planing to release the next pydantic-settings after Pydantic 2.6. So, in the end, we can use use_attribute_docstrings flag.

Is there a way to generate the docs locally for viewing?

Not right now. pydantic-settings docs will be rendered under pydantic docs. But I will work on it to make it possible to render docs in pydantic-settings

Hi @hramezani, almost ready to move this into PR. There are three final items:

I do not plan to add coverage for this line, unless there is a safe/easy way in pytest to modify sys.argv. Is this acceptable? If so, how to exclude?
I would like to exclude this function from coverage or leverage the original test case if you can help locate. I pulled this from pydantic._internal._repr.py. Please provide guidance on desired direction.
Also related to above number two, I am hitting a mypy linting issue related to typing_extensions.TypeAliasType here. This is directly copied from the _repr version, how would you like me to proceed?

I do not plan to add coverage for this line, unless there is a safe/easy way in pytest to modify sys.argv. Is this acceptable? If so, how to exclude?

Can't we use monkeypatch fixture like monkeypatch.setattr("sys.argv", ["test1", "-f", "test2"])?

I would like to exclude this function from coverage or leverage the original test case if you can help locate. I pulled this from pydantic._internal._repr.py. Please provide guidance on desired direction.

There are some tests in pydantic repo

https://github.com/pydantic/pydantic/blob/c1dff153c8ba2894e2b060cf6e53de6851ba4cd9/tests/test_utils.py#L83
https://github.com/pydantic/pydantic/blob/c1dff153c8ba2894e2b060cf6e53de6851ba4cd9/tests/test_utils.py#L106

Also related to above number two, I am hitting a mypy linting issue related to typing_extensions.TypeAliasType here. This is directly copied from the _repr version, how would you like me to proceed?

Please ignore it

Thanks @hramezani for the feedback. Ready for review 👍🏾

please update

@hramezani @samuelcolvin Thank you for the feedback, really appreciate it and will make updates accordingly. We found a couple of changes we'll also bring in after playing with it internally:

The enum support should be moved deeper into the environment variable parsing. Will open a separate issue and address it there.
We plan to integrate this with other existing parsers (one of which is pytest), so will be providing some support to "export" the internally generated parser and likely a pytest specific example. Still working on how to best accomplish this but will push once ready.

The enum support should be moved deeper into the environment variable parsing. Will open a separate issue and address it there.

Opened #252 to address this.

@hramezani @samuelcolvin @dmontagu updates are complete and ready for review.

@hramezani @samuelcolvin @dmontagu updates are complete and ready for review.

Thanks @kschwab for the update.
We will review it later and probably this will be included in the next release

@kschwab Sorry for the late response. Finally, I found time to work on this and test it.
The implementation is good and the doc is very well 👍
There are something that need to be fixed:

If you enable SettingsConfigDict(cli_parse_args=True), then the CLI source will be the first priority and there is no way to change the priority. This is not like other sources that we've added before like toml and yaml. you include them and change the priority as well. So, this needs to be changed and we should have the same behavior as other new sources.

It doesn't work with aliases. with the following model:

class Settings(BaseSettings):
    apple: str = Field(alias='alias')

    model_config = SettingsConfigDict(cli_parse_args=True)

we can't use --alias foo.
As alias support is included in other sources, we have to have it in CLI as well.

It doesn't work with case_sensitive config and always expects the field name. With the following model:

class Settings(BaseSettings):
    foo: str

    model_config = SettingsConfigDict(cli_parse_args=True, case_sensitive=False)  # default is `False`

Then --FOO bar doesn't work. in env source, we convert all the env names to lowercase in when case_sensitive=False

It doesn't work well with dataclasses in sub model. for example with:

@pydantic_dataclasses.dataclass
class MyDataclass:
    foo: int

class Settings(BaseSettings):
    n: MyDataclass

You have to pass --n '{"foo": 123} but --n.foo 123 doesn't work. but it works for normal basemodel submodels
Thanks for your great work here ❤

Thanks @hramezani for the review, we'll address the feedback items soon.
With respect to the CliSettingsSouce prioritization, originally, we had prioritization that could be changed but removed it after our discussion here. The conclusion was to use fixed prioritization so as to not break backwards compatibility, and then revisit dynamic prioritization once this issue is fixed in V3.
We can change it back to the original if you would prefer, let me know your thoughts.

Thanks @hramezani for the review, we'll address the feedback items soon.
With respect to the CliSettingsSouce prioritization, originally, we had prioritization that could be changed but removed it after our discussion here. The conclusion was to use fixed prioritization so as to not break backwards compatibility, and then revisit dynamic prioritization once this issue is fixed in V3.
We can change it back to the original if you would prefer, let me know your thoughts.

Yeah. that was my first thought. but after talking with the team we decided to have the CLI source like other sources.
Can't we implement the new CLI source like other new sources(toml, yaml) and include it like them? it would be good to
Sorry about that.

No worries!

Can't we implement the new CLI source like other new sources(toml, yaml) and include it like them?

I haven't looked at the latest toml, yaml source changes and how it was integrated, but will try and align with it if possible. I don't think there should be any issues. The only thing that comes to mind is the external parser support, but I think it should be fine. We'll make the updates and see where we land 🙂

@hramezani I've addressed the issues raised. For case-insensitive matching, this is not supported on external parsers, it is only supported on the internal parser.

Thanks @kschwab for addressing the issues. I will check it later.

Thanks @samuelcolvin for the review.
I added the differentiation with the last commit. Required fields are marked as (required) and optional fields are marked as (default: val). This also aligns with argparse ArgumentDefaultsHelpFormatter in how it delineates optional/default arguments.
usage: demo.py [-h] [--foo str] [--bar int]

Settings for the application.

options:
  -h, --help  show this help message and exit
  --foo str   Set the foo value. (required)
  --bar int   (default: 123)

@kschwab Thanks for your update.
I am going to merge the PR if you are done with your updates.
It will be released soon 🚀

Thanks @kschwab for your great effort here ❤

Thanks @hramezani! Glad to see it go in 👍🏾


[release]: prepare release v0.6.8 and framework v6

Is the main genesis needed to be reset?

maybe next time, after: https://github.com/rooch-network/rooch/issues/2402  ?


presubmit test broken - 10.29

See https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubeflow_gcp-blueprints/137/kubeflow-gcp-blueprints-presubmit/1321719992365879296/

Here's the error
INFO|2020-10-29T09:40:52|/workspace/testing-repo/py/kubeflow/testing/util.py|72| Error from server (InternalError): error when applying patch:
INFO|2020-10-29T09:40:52|/workspace/testing-repo/py/kubeflow/testing/util.py|72| {"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"serviceusage.cnrm.cloud.google.com/v1beta1\",\"kind\":\"Service\",\"metadata\":{\"annotations\":{\"cnrm.cloud.google.com/deletion-policy\":\"abandon\",\"cnrm.cloud.google.com/disable-dependent-services\":\"false\"},\"labels\":{\"app.kubernetes.io/managed-by\":\"configmanagement.gke.io\",\"auto-deploy-base-name\":\"kf-ci\",\"auto-deploy-group\":\"gcp-blueprint-master\",\"kf-name\":\"kf-vbp-1029-99d\",\"tekton.dev/pipeline\":\"deploy-gcp-blueprint\",\"tekton.dev/pipelineRun\":\"deploy-kf-master-txrwf\",\"tekton.dev/pipelineTask\":\"deploy-gcp\",\"tekton.dev/task\":\"deploy-gcp-blueprint\",\"tekton.dev/taskRun\":\"deploy-kf-master-txrwf-deploy-gcp-ng9wf\"},\"name\":\"anthos.googleapis.com\",\"namespace\":\"kubeflow-ci-deployment\"}}\n"},"labels":{"auto-deploy-base-name":"kf-ci","blueprint-repo-commit":null,"kf-name":"kf-vbp-1029-99d","tekton.dev/pipelineRun":"deploy-kf-master-txrwf","tekton.dev/taskRun":"deploy-kf-master-txrwf-deploy-gcp-ng9wf"}}}
INFO|2020-10-29T09:40:52|/workspace/testing-repo/py/kubeflow/testing/util.py|72| to:
INFO|2020-10-29T09:40:52|/workspace/testing-repo/py/kubeflow/testing/util.py|72| Resource: "serviceusage.cnrm.cloud.google.com/v1beta1, Resource=services", GroupVersionKind: "serviceusage.cnrm.cloud.google.com/v1beta1, Kind=Service"
INFO|2020-10-29T09:40:52|/workspace/testing-repo/py/kubeflow/testing/util.py|72| Name: "anthos.googleapis.com", Namespace: "kubeflow-ci-deployment"

Based on test logs the management cluster is
gke_kubeflow-ci_us-central1_kf-ci-management
cnrm-webhook-manager
Is reported as unhealthy

kubectl --context=kubeflow-ci-management -n cnrm-system describe pods cnrm-webhook-manager-544bfccb5d-ts5fk
Name:           cnrm-webhook-manager-544bfccb5d-ts5fk
Namespace:      cnrm-system
Priority:       0
Node:           gke-kf-ci-management-kf-ci-management-d6b2e6ad-jgkd/<IP_ADDRESS>
Start Time:     Mon, 19 Oct 2020 01:32:50 -0700
Labels:         cnrm.cloud.google.com/component=cnrm-webhook-manager
                cnrm.cloud.google.com/system=true
                pod-template-hash=544bfccb5d
Annotations:    cnrm.cloud.google.com/version: 1.9.1
Status:         Running
IP:             <IP_ADDRESS>
Controlled By:  ReplicaSet/cnrm-webhook-manager-544bfccb5d
Containers:
  webhook:
    Container ID:  docker://87d89a1ddfc88138500f220c3103c0f145b149f334cdafee11a1bef8e98feec8
    Image:         gcr.io/cnrm-eap/webhook:97b6128
    Image ID:      docker-pullable://gcr.io/cnrm-eap/webhook@sha256:4d727354e9cde8efeafbaba16ef79766b1b1a92df3787df59f933b59b0eb1d61
    Port:          <none>
    Host Port:     <none>
    Command:
      /configconnector/webhook
    Args:
      --stderrthreshold=INFO
    State:          Running
      Started:      Mon, 19 Oct 2020 01:33:07 -0700
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     40m
      memory:  64Mi
    Requests:
      cpu:      20m
      memory:   32Mi
    Readiness:  exec [cat /tmp/ready] delay=3s timeout=1s period=3s #success=1 #failure=3
    Environment:
      NAMESPACE:  cnrm-system (v1:metadata.namespace)
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from cnrm-webhook-manager-token-b6f7k (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  cnrm-webhook-manager-token-b6f7k:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  cnrm-webhook-manager-token-b6f7k
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                    From                                                          Message
  ----     ------     ----                   ----                                                          -------
  Warning  Unhealthy  103s (x41111 over 9d)  kubelet, gke-kf-ci-management-kf-ci-management-d6b2e6ad-jgkd  Readiness probe failed: OCI runtime exec failed: exec failed: cannot exec a container that has stopped: unknown

I deleted the pod and a new one restarted.
kubectl --context=kubeflow-ci-management -n cnrm-system get pods
NAME                                            READY   STATUS        RESTARTS   AGE
cnrm-controller-manager-0                       1/1     Running       0          10d
cnrm-deletiondefender-0                         1/1     Running       0          10d
cnrm-resource-stats-recorder-64c9b6d496-7xsfc   1/1     Running       0          10d
cnrm-webhook-manager-544bfccb5d-ts5fk           0/1     Terminating   0          10d
cnrm-webhook-manager-544bfccb5d-w4qrq           1/1     Running       0          83s

The latest run of the pipelinerun now appears to be stuck getting started; there doesn't appear to be any taskruns
kubectl --context=kf-ci-v1 -n auto-deploy describe pipelineruns deploy-kf-master-5djt4
Name:         deploy-kf-master-5djt4
Namespace:    auto-deploy
Labels:       auto-deploy-base-name=kf-ci
              auto-deploy-group=gcp-blueprint-master
Annotations:  <none>
API Version:  tekton.dev/v1alpha1
Kind:         PipelineRun
Metadata:
  Creation Timestamp:  2020-10-29T14:40:34Z
  Generate Name:       deploy-kf-master-
  Generation:          1
  Resource Version:    226255709
  Self Link:           /apis/tekton.dev/v1alpha1/namespaces/auto-deploy/pipelineruns/deploy-kf-master-5djt4
  UID:                 b3a2e05b-19f4-11eb-9df6-42010a8e00b4
Spec:
  Params:
    Name:   artifacts-gcs
    Value:  gs://kubernetes-jenkins/pr-logs/pull/kubeflow_gcp-blueprints/137/kubeflow-gcp-blueprints-presubmit/1321824194224197632
    Name:   junit-path
    Value:  artifacts/junit_deploy-kf-master-
    Name:   test-target-name
    Value:  deploy
  Pipeline Ref:
    Name:  deploy-gcp-blueprint
  Resources:
    Name:  blueprint-repo
    Resource Spec:
      Params:
        Name:   url
        Value:  https://github.com/kubeflow/gcp-blueprints.git
        Name:   revision
        Value:  refs/pull/137/head
      Type:     git
    Name:       testing-repo
    Resource Spec:
      Params:
        Name:            revision
        Value:           master
        Name:            url
        Value:           https://github.com/kubeflow/testing.git
      Type:              git
  Service Account Name:  default-editor
  Timeout:               1h0m0s
Events:                  <none>

I think we also need to upgrade our current version of config connector(1.9.1) to the latest one as mentioned https://github.com/GoogleCloudPlatform/k8s-config-connector/issues/252#issuecomment-696133051

It looks like there are a huge number of TektonPipelineRuns accumulating in the kf-ci-v1 namespace
Looks like we have 49749 pipelineruns. I'm wondering if that is affecting the Tekton pipelineruns controller and explain why the latest runs aren't starting.
pipelineruns.txt
Lets try deleting all the pipelineruns
kubectl --context=kf-ci-v1 -n kf-ci delete pipelineruns --all=tru

I'm running the program in kubeflow/testing#767 to try to GC all the finished taskruns and see if that fixes things.

Tekton pipelines appear to be running again.
/test all

The auto deployments are unhealthy. It looks to me like  we are running short on quota for

service accounts
PD
static ip addresses

in project kubeflow-ci-deployment
gcloud --project=kubeflow-ci-deployment container clusters list --format="table(name, zone, createTime)" --sort-by=createTime
NAME            LOCATION       CREATE_TIME
deployapp       us-east1-d     2019-04-26T22:35:53+00:00
apps            us-central1-a  2019-09-18T23:04:32+00:00
myapp2          us-central1-a  2019-10-03T04:18:04+00:00
kfctl-3263      us-central1-a  2020-10-22T11:03:24+00:00
kfctl-121b      us-central1-a  2020-10-22T14:08:04+00:00
kfctl-c19d      us-central1-a  2020-10-22T18:58:12+00:00
kfctl-4dda      us-central1-a  2020-10-27T22:26:54+00:00
kfctl-b3cf      us-central1-a  2020-10-27T23:28:22+00:00
kfctl-796e      us-central1-a  2020-10-27T23:33:47+00:00
kfctl-9ad9      us-central1-a  2020-10-28T03:33:27+00:00
kfctl-b485      us-central1-a  2020-10-28T07:32:36+00:00
kfctl-fc27      us-central1-a  2020-10-28T07:34:23+00:00
kfctl-297d      us-central1-a  2020-10-28T07:35:04+00:00
kfctl-9df7      us-central1-a  2020-10-28T11:41:12+00:00
kfctl-cbcc      us-central1-a  2020-10-28T19:16:14+00:00
kfctl-5f95      us-central1-a  2020-10-28T19:35:51+00:00
kfctl-18dc      us-central1-a  2020-10-28T19:36:08+00:00
kfctl-d3f9      us-central1-a  2020-10-28T23:43:09+00:00
kfctl-07dc      us-central1-a  2020-10-29T03:37:24+00:00
kfctl-987d      us-central1-a  2020-10-29T03:44:42+00:00
kfctl-a11d      us-central1-a  2020-10-29T07:40:16+00:00
kfctl-1c1c      us-central1-a  2020-10-29T11:11:20+00:00
kfctl-fd8d      us-central1-a  2020-10-29T14:32:29+00:00
kf-v1-1029-49d  us-east1-c     2020-10-29T20:04:58+00:00

I'm not sure where all of the kfctl clusters are coming from.

My conjecture is these are coming from the kfctl repo.
https://github.com/kubeflow/kfctl/blob/v1.0-branch/prow_config.yaml
In particular I think they are coming from the previous release branches which are still running tests.

I selected all of the clusters except the "kf-v1" cluster for deletion.
I'm not sure what all the apps clusters are for; I suspect at least some of them were for click to deploy which we are no longer using.
I also deleted a bunch of service accounts and IP addresses. This should hopefully free up some quota.
Judging by the cluster timestamps it looks like resources are still being GC'd as we don't have any older than 10/22. But we will need to find and shutdown the processes that keep creating them.
Hopefully the PRs referenced above will do this.

So it looks like the blueprint clusters were able to get created. The endpoints aren't accessible though.
gcloud --project=kubeflow-ci-deployment container clusters list --format="table(name, zone, createTime)" --sort-by=createTime
Reauthentication required.
Please insert and touch your security key
Timed out while waiting for security key touch.
Please insert and touch your security key
NAME              LOCATION       CREATE_TIME
kf-v1-1-1030-d8b  us-central1-c  2020-10-30T02:55:18+00:00
kf-vbp-1030-b1f   us-central1-c  2020-10-30T02:55:33+00:00
kf-v1-1030-72e    us-east1-c     2020-10-30T08:08:28+00:00

Looking at the logs it looks like there was a timeout waiting
error: timed out waiting for the condition on containerclusters/kf-vbp-1030-b1f

It looks like the last blueprint cluster might have been deployed around the same time as I was cleaning up kubeflow-ci-deployment was getting deployed.

Another auto-deployment just started and it looks like the cluster was correctly created and it progressed to deploying the applications.

The latest autodeployment is up and healthy.
https://kf-vbp-1030-3b5.endpoints.kubeflow-ci-deployment.cloud.goog/

Thank you @jlewi!


Template repos’ README

Lets make a template to share between all gateway services repos.
This template will be the outline for all service reposn readmes

Gateway services  README Template

📗 Table of Contents

📖 About the Project
🔍 Arch Map

🛠 Built With

Tech Stack
Key Features

🚀 Live OpenAPI Docs

💻 Getting Started

Prerequisites
Setup
Install
Usage
Run tests
Deployment

👥 Authors
🤝 Contributing
⭐ Show your support
🙏 Acknowledgements
❓ FAQ (OPTIONAL)
📝 License

📖 [project_name]

Describe your project in 2 paragraphs.

[project__name] is a...

🔭  Mermaid Arch maps

Describe the arch map for the project.

(back to top)
🛠 Built With
Tech Stack

Describe the tech stack and include only the relevant sections that apply to your project.

Server

Name link

Data store

Name link

Key Features

Describe between 1-n* key features of the application.

[key_feature_1]
[key_feature_2]
[key_feature_3]

(back to top)

🚀 Live Docs

Add a link to your deployed project.

Live Docs Link

(back to top)

💻 Getting Started

Describe how a new developer could make use of your project.

To get a local copy up and running, follow these steps.
Prerequisites
In order to run this project you need:
Example command:
npm i -g x

Setup
Clone this repository to your desired folder:
Example commands:
  git clone git@github.com:myaccount/my-project.git
  cd my-folder

Install
Install this project with:
Example command:
  cd my-project

Usage
To run the project, execute the following command:
Example command:

Run tests
To run tests, run the following command:
Example command:

Deployment
You can deploy this project using:
Example:

(back to top)

👥 Authors

Mention all of the collaborators of this project.

👤

GitHub: @githubhandle
GitHub: @githubhandle

(back to top)

🤝 Contributing
Contributions, issues, and feature requests are welcome!
Feel free to check the issues page.
(back to top)

⭐ Show your support

Write a message to encourage readers to support our project

If you like this project...join the zoom calls or loom at the label for good-first-issue
(back to top)

🙏 Acknowledgments

Give credit to everyone who inspired our codebase.

I would like to thank...
(back to top)

❓ FAQ (OPTIONAL)

Add at least 2 questions new developers would ask when they decide to use your project.

[Question_1]

[Answer_1]

[Question_2]

[Answer_2]

(back to top)

📝 License
This project is MIT licensed.
NOTE: we recommend using the MIT license - you can set it up quickly by using templates available on GitHub. You can also use any other license if you wish.
(back to top)

we could always add badges too I guess..
https://github.com/Ileriayo/markdown-badges

Looks great... let's make this into a PR that we keep open. I'll start with this as a template for one of the services, and we can incorporate any tweaks from that exercise back into this template.

Skip contributors
Don't use the Frequency logo
Our stuff is Apache 2.0 licensed

send @wilwade the MD, he will edit and open a draft PR


Switching to LCEL

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [X] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Dear Pamela,
Thank you for the excellent work across all your repositories. It's truly inspiring and rich in content. I was wondering if there is a plan to switch to LCEL to reduce code complexity?
Best regards,
Maged

I've added a langchain example to demonstrate the very basic Langchain usage (which has a single LCEL expression). I intentionally use the OpenAI SDK in these examples to complement my slides at https://pamelafox.github.io/my-py-talks/llms-python/ but I do like to be able to show other frameworks as well.


If there was gpu allocation issue I would rather expect error message about gpu/cuda

I'm not sure what's wrong with it. I'll try to find out.
Sorry for the inconvinience.

OK , my bad , I had already output/lora_vision_test create with data from phi3 training.
So in this case maybe add some information regarding that:
weights already in output/lora_vision_test, or output/lora_vision_test is not empty ...


the fancy zone doesn't work well

Microsoft PowerToys version
0.72.0
Installation method
PowerToys auto-update
Running as admin
Yes
Area(s) with issue?
FancyZones, FancyZones Editor
Steps to reproduce
I use 3 monitors split programs and the one monitor work well with merge areas. For example I drag a program the middle of the divide line. then It grow with merged area. but other monitor it doenst work
✔ Expected Behavior
No response
❌ Actual Behavior
No response
Other Software
No response

Same issue as this? https://github.com/microsoft/PowerToys/issues/28078

Are you still experiencing this issue? /needinfo


Need to update to v2 Android Plugin APIs

Hi guys,
After I upgrade flutter to version 1.12.x, I got this issue.
The plugin 'flutter_full_pdf_viewer' is built using an older version of the Android plugin API which assumes that it's running in a full-Flutter environment. It may have undefined behaviors when Flutter is integrated into an existing app as a module. The plugin can be updated to the v2 Android Plugin APIs by following https://flutter.dev/go/android-plugin-migration.
Please help !!!

Same here. It used to work on my app, but after I upgraded to embedding v2, it just gives me:
E/flutter (14088): [ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: PlatformException(error, Attempt to invoke virtual method 'android.support.v4.media.session.MediaControllerCompat$TransportControls android.support.v4.media.session.MediaControllerCompat.getTransportControls()' on a null object reference, null)


Inconsistent results were detected using the same SQL

Test Data
start_time,end_time,id
2022-06-24 10:18:03,2022-06-24 10:18:03,4
2022-07-01 00:00:00,2022-07-01 01:00:00,6
2022-07-01 05:00:00,2022-08-01 00:00:00,10

gorm result: 1
2022/08/16 20:41:10 test.go:57
[0.219ms] [rows:1] SELECT * FROM `test` WHERE start_time >= "2022-07-01 00:00:00"

sqlite teminal result: 2
sqlite> SELECT * FROM `test` WHERE start_time >= "2022-07-01 00:00:00";
2022-07-01 00:00:00|2022-07-01 01:00:00|6
2022-07-01 05:00:00|2022-08-01 00:00:00|10

gorm result: 3
[0.195ms] [rows:3] SELECT * FROM `test` WHERE e_time < "2022-08-01 00:00:00"

sqlite terminal result: 2
sqlite> SELECT * FROM `test` WHERE e_time < "2022-08-01 00:00:00";
2022-06-24 10:18:03|2022-06-24 10:18:03|4
2022-07-01 00:00:00|2022-07-01 01:00:00|6

Inconsistent results were detected using the same SQL
sqlite> .version
SQLite 3.39.2 2022-07-21 15:24:47 698edb77537b67c41adc68f9b892db56bcf9a55e00371a61420f3ddd668e6603
zlib version 1.2.12
gcc-5.4.0 20160609
go version
go version go1.19 linux/amd64
gorm.io/driver/sqlite v1.3.6
gorm.io/gorm v1.23.8
github.com/mattn/go-sqlite3 v1.14.14

使用golang原生的sql查询 结果与预期相符；所以确定问题是由gorm导致的

sqlite 查询time.Time 被转为：”2022-07-01 00:00:00+00:00“ 串，查出结果与”2022-07-01 00:00:00“ 结果不一致

It's a time zone issue, it has nothing to do with gorm, even if you use other drivers, it only depends on whether you use the interpolate params or not.


No media shown after login

Installation and login worked fine. However, I'm unable to see my files stored on Google Drive, though I can browse the folder structure in the left panel.
System: Mac OS X 10.14.3

This occurs for me too, and seems to be a recent issue - I'm guessing Google Drive has changed something in the last few weeks.
System: Mac OS X 10.13.6

Update: I've fixed this within my own app, and would guess that it'll work here too @alexkim205 @MiMoHo . To solve it, create a preload.js script file containing the following:
if (!window.chrome) {
  window.chrome = {}
}

Then add the following option wherever the BrowserWindow or BrowserView is created, to run the preload script before the page loads:
webPreferences: {
  preload: path.join(__dirname, '/preload.js'),
}

The issue was that drive.google.com was referencing window.chrome, which didn't exist. (There wasn't a console error though, so I diagnosed it by spotting a Network request to https://drive.google.com/drive/jserror?script=https%3A%2F%2Fdrive.google.com%2Fdrive%2Fmy-drive&error=chrome%20is%20not%20defined&line=Not%20available.)

Thank you for the update. I think this would help a lot of people having the same issue. Would you mind making a pull request?


Document the consent API

Pending release of feature

Based on the existence of https://developers.cloudflare.com/zaraz/consent-management/api/, think this can be closed out as outdated.... but feel free to re-open if I'm mistaken here.


Development environment (spawned with make local-setup) does not support targetRef in the wasmplugin object.

Motivated by the deprecation of workload selectors on the issue #302, we found an Istio issue on using  the targetRef of the wasmpluging when the targeted gateway is istio-ingressgateway (or any manually deployed Gateway API gateway). Thus, currently, our development environment (spawned with make local-setup) does not support targetRef in the wasmplugin object.
Istio engineers provided a workaround, but clearly stated that it was intended not to apply to Istio ingress gateway.
So, there are three ways we can take in order to deprecate workload selectors with targetRef:

Replace istio-ingressgateway with automatically deployed Gateway

Note: Sometimes it is very handy to switch Envoy's log level to debug to see what decisions are being maked at the gateway level regarding auth and rate limiting filters. For the istio-ingressgateway, it is just about k edit deployments istio-ingressgateway -n istio-system and then set "--proxyLogLevel=debug". However, for automatically deployed Gateways, the deployment is fully reconciled and modifications of the log level are not allowed.

Investigate the workaround provided in the Istio issue and keep using Istio ingress gateway.
Postpone the replacement of the workload selectors with the targetRef and wait for Istio to fix it (if they ever decide to fix it).

PS: My take is to go for the first one, i.e. use an automatically deployed gateway. Some of the Istio design decisions regarding Gateway API on-boarding are explicitly avoiding Istio ingress gateway, and more issue may happen in the future like this one. Postponing does not seem a good option as well IMO. The targetRef is clearly a simplification that can benefit the users of kuadrant and does not seem fair to block it because there is an issue with out development environment.

@adam-cattermole that's what I was talking about

#298 did:

Replace istio-ingressgateway with automatically deployed Gateway


Issue #743 make setup steps skippable

… due to incorrect Administrator role check
Fixes #643 (skips were missing in setup)

CI failed, but this does not seem related to these changes, pushing.


Does it support the rule index's name like regex result $1 $2

First I had many databases to sync, and ES 6 has no type,  So I add a prefix to every index name by table_name
I'll set Each tables like this
[[rule]]
schema = "db1"
table = "table1"
index = "db1-table1"
type = "_doc"

Does this support a wildcard result?
[[rule]]
schema = "db1"
table = "(.*)"
index ="db1-$1"
type = "_doc"

em, seem we don't support wildcard like this now. but maybe we can do it, refactoring rule, for every table, get associated index and type dynamically.

@fly-studio Did you solve the rule wildcard problem later?
I also need to prefix all index like this:
mysql table: table_1 table_2
db1 to es index:db1_table_1 db1_table2
db2 to es index:db2_table_1 db2_table2
thx!

@10000go I wrote a new sync tool by myself,
So the new tool could renames  like the variant ${SCHAME} ${TABLE}
or splits the index per a date maths like index_name-{created_at|YYYY.mm.dd}

@fly-studio is mysql-es?Has this tool been applied to online production?thx

@10000go Yes, I run this tool for 4 months, 0 data lost, of the last release.
base on alibaba/canal, zero-lost.
But the last release is not the newst version. I will releasing lately

Ok, thx,I've added watch @fly-studio


GPU memory not fixed when training

Thank you for your great work！
Why does the GPU memory change all the time during training instead of being fixed all the time?
Except for changing scale to 2, I use the default configs.

Hi @ProBURN-E, thanks. VRAM usage varies due to multiple factors, it's normal. There are internal bottlenecks that causes it, as well as different network designs that influences it. For example, networks like as compact and span have a stable gpu-util and vram usage, as opposed to omnisr hich has more fluctuations.
Another reason could be slow dataloading times. If you have an SSD, it is recommended that you put your dataset in it, as it will avoid high reading times from HDD.


increase CPU request to 500m for app server container

Description
(as title says)
Type of change

[ ] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[x] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library

Related Tickets & Documents

Related Issue #
Closes #

Checklist before requesting a review

[x] I have performed a self-review of my code.
[x] PR has passed all pre-merge test jobs.
[x] If it is a core feature, I have added thorough tests.

Testing

Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

/lgtm
/approve


Shift text highlighting. Also Ctrl / Alt detection

Finishing the highlight job I started last year :feelsgood:

Wohoo! Will boost my motivation to do Krom's cutCopyPaste / https://github.com/Kode/Kha/issues/847. :hurtrealbad:


SIGSEGV in pmtiles serve

I encountered a segfault when running pmtiles serve (version 1.7.0):
$  pmtiles serve --cors="*" .

2024/02/28 16:30:10 main.go:115: served /uprn/16/32744/21777.mvt in 12.216941ms
2024/02/28 16:30:10 main.go:115: served /uprn/16/32746/21777.mvt in 5.569058ms
2024/02/28 16:30:10 main.go:115: served /uprn/16/32745/21776.mvt in 18.636425ms
2024/02/28 16:30:10 main.go:115: served /uprn/16/32746/21776.mvt in 27.242473ms

2024/02/28 16:30:36 server.go:139: fetching uprn 41561-8008
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x280 pc=0x5b762e]

goroutine 1839 [running]:
compress/gzip.(*Reader).Read(0x0?, {0xc0003a3000?, 0x7fc5d9ad8a18?, 0x0?})
        /opt/local/lib/go/src/compress/gzip/gunzip.go:247 +0x2e
bufio.(*Reader).fill(0xc0000287e0)
        /opt/local/lib/go/src/bufio/bufio.go:106 +0xff
bufio.(*Reader).ReadByte(0xc0000287e0)
        /opt/local/lib/go/src/bufio/bufio.go:265 +0x2c
encoding/binary.ReadUvarint({0x1971320, 0xc0000287e0})
        /opt/local/lib/go/src/encoding/binary/varint.go:133 +0x82
github.com/protomaps/go-pmtiles/pmtiles.deserialize_entries(0x1973a20?)
        /Users/bdon/workspace/protomaps/go-pmtiles/pmtiles/directory.go:141 +0x18c
github.com/protomaps/go-pmtiles/pmtiles.(*Server).Start.func1.1()
        /Users/bdon/workspace/protomaps/go-pmtiles/pmtiles/server.go:175 +0x9ab
created by github.com/protomaps/go-pmtiles/pmtiles.(*Server).Start.func1
        /Users/bdon/workspace/protomaps/go-pmtiles/pmtiles/server.go:127 +0x8ac

Unfortunately the uprn.pmtiles file is large and contains information that I can't re-distribute (or, at least, that I can't re-distribute in its entirety).
I'm running pmtiles inside docker inside WSL: I'm not sure whether this is relevant.
The crash occured before the URL of the requested tile was logged, and it's not obvious to me how to reconstruct the x/y tile index from the offset and size (41561-8008), so I can't easily test if the crash is reproducible in v1.18.0.

Looking at the console more carefully, I think that the uprn.pmtiles file was re-written by tippecanoe without the pmtiles serve process being restarted.
PMtiles could check to see if a file has been updated and handle this gracefully, but just telling users not to edit files would also be reasonable.

As of v1.18 the http, s3 and file backends handle Etags gracefully. Versions prior to v1.16, and azure/gap buckets can read garbage offsets if the file changes from underneath.

Great - in that case, I don't think the crash I encountered is a bug in the current version of pmtiles.


Could it be an issue with the in-memory database when there is a lot of data?

Hi,
I'm looking for a static site generator (SSG) for my project in which I need to extract a large amount of data from my database and provide it to the SSG to generate the static site. Statik seems to be exactly what I want. However, there is one thing concerns me:
From the wiki:

Statik allows you to define your own data models in YAML format, and instances of those data models either in YAML or Markdown. This is all loaded into an in-memory SQLAlchemy SQLite database when rendering your views.

I wonder if there is a lot of data (e.g. >20G), would the in-memory database consume all my ram and freeze my computer?

Interesting question! See http://stackoverflow.com/questions/784173/what-are-the-performance-characteristics-of-sqlite-with-very-large-database-file
It could possibly work, depending on how you define your models. Otherwise this would be a good case to integrate with a larger relational database (e.g. PostgreSQL or MySQL).
Unfortunately Statik doesn't support those databases just yet. But it'd definitely be possible (Statik uses SQLAlchemy under the hood to connect to th database, and works with most relational databases).
The trouble is, Statik uses an ephemeral database for a specific reason: I wanted extreme flexibility in the database structure.
I'd have to give it some thought as to how best to integrate with other relational databases for large datasets. Could be tricky.
Is it possible to describe your use case in a bit more detail?

Is it possible to describe your use case in a bit more detail?

I maintain an ecommerce website for my friend's business. The website is heavily and slow with a lot of products (> 5000 products) and thousand documents/articles. Most customers don't use the buy/checkout function. They go to our website to see the product pictures and information, then contact our sale via online chat or phone. So I'm planning to convert our website to a static site with 3rd party services for online chat/discussion and even for ecommerce functions (with services like https://www.marketcloud.it or https://www.moltin.com).

Your use case sounds, to me at least, like more of a case to use caching servers like Varnish, to be honest. Doesn't sound like the sort of thing where a static site generator would be useful (in fact, it'll just complicate things significantly).

If i may, how many customers do you have? If it isn't in the thousands concurrently, then perhaps the following can be of help. You say you have around 5000 products. In my experience when there is a performance problem with such "few" records, then there might be other forms of optimization:

check your SQL queries, are there many joins? If yes, are they cartesian products, i.e. multiplying entire tables with each other? Can you simplify your joins? Perhaps less normalization, thus less joins?
do you have all the important indices set? This is mostly the first thing people do, but perhaps this is a case for you?
What DB are you using? Depending on how you use your schema, some DBs are better than others. I've seen cases where using PosgreSQL lead to problems (using Java and Hibernate), and just switching to MariaDB solved everything in one go. This might not be possible for you, but is certainly worth looking into.
Use your DBs query optimizer and find queries which are performing poorly.
And then a layer higher up, perhaps your code can be optimized? This is of course the most difficult of all optimizations.

Varnish is a really good thing, but personally i think one should first check these things before looking into caching systems, which can bring with them their own issues.

Thank you @thanethomson and @eitch


DSND-2940: Finish GET ACSP full profile error handling

Describe the changes

Fixed a bug where an error thrown during full profile get would return 403 forbidden.
Added more unsuccessful iTests.

Related Jira tickets
DSND-2940
Developer check list
General

[ ] Is the PR as small as it can be?
[ ] Has the Jira ticket been updated with any relevant comments?
[ ] Is the README up to date?
[ ] Has the POM been updated for the latest versions of dependencies?
[ ] Has the code been double-checked against main?
[ ] Does the code adhere standards in this repository, including SonarQube checks?

Testing

[ ] Do the code changes have unit and integration tests with code coverage > 80%?
[ ] Has the code been tested locally and/or passed the API Karate tests on Tilt?
[ ] Have mandatory manual test cases been run?
[ ] Are extra Karate tests required?
[ ] Are all the test data resources up to date?

Release preparation
Where possible, add links to the respective Jira tickets when an item is checked

[ ] Have these changes been included in a release ticket?
[ ] Are changes required to the docker-chs-development deployment or test data?
[ ] Are any changes required to the environment added to deployment repositories?

Notes to the tester
Add any testing hints or instructions here.

CI: No security warnings found


Use /proc/self/stat on Linux (#1)

Tested under WSL 1.

CI runs on MSRV of 1.28. This is the 2015 edition, so some sort of import for std is necessary. NonZero*::parse also hadn't existed yet.

Diff looks good. I'll test it locally later.


eslint-config-react-app doesn't show unused vars when destructuring at least in stateless components

Current configuration:

Use case, check onEdit prop:

Changing no-unused-vars args prop to all:

Result:

[eslint] 'onEdit' is defined but never used. (no-unused-vars)

I can create a PR for this if needed, just wanted to know if this was intentional and why, thanks!

This one made it to the latest alpha and maybe it was intentional setting before.
Now code like this will be marked with warnings:
const Foo = props  => (
component that doesnt use props
)

handleEvent(event, otherArg) {
    doStuff(otherArg)
}

Whats your opinion on that @goncy?

In my opinion props is still a unused prop in the first scenario, just that we are used to have it there, i don't think that is bad having it as a warning, the second one is true, are you getting a no-unused-vars warning for the first argument?, in that case we can change the all in args to after-used
all - all named arguments must be used.```

After used:

All:

Sounds like the default setting strikes a nice balance. However the more I think about it the previous setting could be useful sometimes, where you use a callback function from a library which sends arguments it can be nice to define those regardless if you use them in your specific case which would make it easier for fellow developers on your team getting a gut feeling about the callback api when they approach your code.
Lets get some other opinions on this one before deciding.

Maybe /*eslint no-unused-vars: ["error", { "args": "after-used" }]*/ is a good in between?


chore: bump webassemblyjs

What kind of change does this PR introduce?

Did you add tests for your changes?

Does this PR introduce a breaking change?

What needs to be documented once your changes are merged?

Closes https://github.com/webpack/webpack/issues/7495
We have now a check into our CI: https://github.com/xtuc/webassemblyjs/pull/389

Thanks


chore: delete handwritten samples

The only samples remaining are the ones created using SnippetsGen.
This PR was created by running the generation/delete_non_generated_samples.sh script added in a separate commit in this PR.

@meltsufin Snippet-bot is not happy about removing region tags in use. Merging this PR will cause the devsite build to break.
I added "do not merge" label.

@tmatsuo These modules were copied from split repos. Those split repos are retaining all of these samples and the docs should not break.

Yeah I manually checked the source code of the docs for the first violation reported by snippet-bot (dataproc_create_autoscaling_cluster).
The source code contains:
 'language_snippets': [{'display': 'Java',
                        'filename': 'java-dataproc/samples/snippets/src/main/java/CreateClusterWithAutoscaling.java',
                        'keyword': 'Java',
                        'org': 'googleapis',
                        'repo': 'google-cloud-java',
                        'slug': 'java'}],

I think the devsite build will break because of this.

@tmatsuo It looks like it was autogenerated by Sample Browser. @jmdobry Do you know what happens if there are duplicates for region tags?

Ok, DRIFT is debugging the data. Also I manually checked those URLs and they all have docs/samples so quite likely they're all sample browser URLs (not guaranteed).
I removed the do-not-merge. Thanks for your patience.

@tmatsuo Are we not worried about the Sample Browser being broken?

@meltsufin AFAIK, the sample browser will detect the removal and remove the page as well (it will be self heal).

@meltsufin JFYI, I told that it would be deleted, but in this case, the page:
https://cloud.google.com/dataproc/docs/samples/dataproc-create-autoscaling-cluster
has been rewritten with the sample code from the split repository and now the link goes to:
https://github.com/googleapis/java-dataproc/blob/HEAD/samples/snippets/src/main/java/CreateClusterWithAutoscaling.java

Thanks for the update. This is perfect!


Part 2 of ASLLVD corrections - THANK YOU !!!

Please let me know if you have any questions.   THANKS so much !!!
asllvd-corrections-part2.xlsx

Done

Thanks very much- but one further correction.  When the gloss is changed from PRISON to SCREEN/FILTER, the example needs to be moved into the proper group.   THANKS!

Same issue when PERSPECTIVE_2 is changed to PERSPECTIVE - the example needs to be moved

Also, when a new entry/variant BUILD_2 was created, which was intended to be listed under main entry BUILD, it still needs to be a distinct entry/variant.
BUILD
     BUILD_2

Fixed

looks good


bug show create table show index comment error

Proposed changes
Fix two bugs that cause show create table index comment error

comment has extra single quotes.
quota does not escape in show create table result.

COMMENT ''test index" comment'' to COMMENT 'test index\" comment'

Refs PR:
https://github.com/apache/doris/pull/6560
https://github.com/apache/doris/pull/28346
before

now

Issue Number: close #xxx

run buildall

run buildall

run buildall

run performance

run p0

run p0

run buildall

replace screenshot with text for git msg and search engine firendly

@morrySnow ok, done

@morrySnow ok, done

still have screenshot in description

@morrySnow  removed.

run buildall

@morrySnow May need to remove dev/2.0 tags, doesn't seem to affect 2.0？


Failure status: strange behaviour

Hi,
i am one of the organizer of the following competition
https://www.codabench.org/competitions/1534/
A competition participant reported a strange behaviour regarding the status. His submission went through the whole pipeline (ingestion program, scoring program), he even obtained a score but the status indicates "Failed".
I don't understand why, it does not make sense to me, and there is no visible issue within the logs.
Could you do something about it?
Thanks.

Hi, do you know the ID of the submission and the username of the submitter? Did it happen to other submissions?

Hi.
Yes, it happenned for several submissions. Here is the complete list of submissions that failed like that:
username: 404Error
Ids that failed: 47879, 47703, 47702, 47594, 47592, 47591, 45824,
username: binhnk
Ids: 45161,44932

Update: the re-run did not fail. I have currently no explanation on the "Failed" status on the first run.
We will try to investigate this problem. In the meantime, you can indicate your participants to submit again if an error occur.

This is quite strange because when i have a look at the submission it did fail as showed below.

According to the submitter, it fails afterward which seems to be consistent with your test. Any idea?

Oh sorry, you are right. I made a confusion between "48322" and "48332"

I close this issue because it is very specific. Feel free to re-open it if needed.


Added new preference for viewing temperature in Fahrenheits.

To change the settings, just go to the settings fragment from the bottom navigation view and select the temperature unit option. There are two options over there, one for Celsius and one for Fahrenheits. Changing this changes the units of all the temperature values across the application!
Below are the screenshots:

Hey can you assist me in resolving the build errors I am getting, it would help me a lot!

Hey can you help me in resolving the build errors. I have implemented the feature and tested it properly, you can refer the above screenshots!

Hey can you help me in resolving the build errors. I have implemented the feature and tested it properly, you can refer the above screenshots! #11

Hi @abhishekjagushte first of, thank you so much for taking out time to build out this feature! It is super great and useful. Secondly, so sorry about the build failure, I added some API keys as secrets to the repo here and GitHub Actions doesn't make them available to forked copies of this project hence, the build will fail on your end as expected. I've gone through the build logs to ascertain the failure was because the keys weren't available so we are all good 👍 . I would review the code and definitely get it merged 💯 . Thank you! :)

Thanks! Learnt a lot from your work!!

Thanks! Learnt a lot from your work!!

Awesome! Glad you did 💯


WalletConnect hanging

Hi,
Just testing out the create react app sample
Working well with Chrome extension connect, but with the wallet connect with mobile station app I get "ready to connect" screen hanging indefinitely .
Is this a known issue?

@williammortondev It's because SSL is not set. (SSL setting is required during development because walletconnect communicates with https / wss) template is as simple as possible, so SSL setting was not enforced.
Please refer to the links below.

You need to set local Root CA configuration: https://github.com/FiloSottile/mkcert
Also you have to set Root CA to your mobile device: https://github.com/FiloSottile/mkcert#mobile-devices
And set the cert files to react-scripts : https://github.com/terra-money/wallet-provider/blob/main/packages/package.json#L7
https://create-react-app.dev/docs/using-https-in-development/

There may be the same problem after the above settings. It happens when the Walletconnect development server is in bad condition and not responding properly.
Then replace the bridge server as shown below.
<WalletProvider
      connectorOpts={{ bridge: 'https://tequila-walletconnect.terra.dev/'}}
      ...
    >


Fixes bond index parsing for w/c/t/ctu labels in CXSMILES/CXSMARTS

In functions parse_coordinate_bonds(), parse_doublebond_stereo() and parse_wedged_bonds() we attempt to apply the corresponding features on bonds which have the specified indexes in the RDKit mol, instead of those that in the SMILES string. These do match for simple SMILES, but in cases where rings are involved, these might not match.
The issue was discovered and described for wiggly bonds, since these are specified using both an atom and a bond, and an error was raised when bond wasn't connected to the atom.
This is an example of the issue:
mb = """
     RDKit          2D

  0  0  0  0  0  0  0  0  0  0999 V3000
M  V30 BEGIN CTAB
M  V30 COUNTS 9 10 0 0 0
M  V30 BEGIN ATOM
M  V30 1 C 2.366025 0.000000 0.000000 0
M  V30 2 C 0.866025 0.000000 0.000000 0
M  V30 3 C -0.433013 0.750000 0.000000 0
M  V30 4 N -0.433013 -0.750000 0.000000 0
M  V30 5 C -1.183013 -2.049038 0.000000 0
M  V30 6 C -2.683013 -2.049038 0.000000 0
M  V30 7 C -3.433013 -3.348076 0.000000 0
M  V30 8 C -3.433013 -4.848076 0.000000 0
M  V30 9 C -4.732051 -4.098076 0.000000 0
M  V30 END ATOM
M  V30 BEGIN BOND
M  V30 1 1 1 2
M  V30 2 1 2 3
M  V30 3 1 3 4
M  V30 4 1 4 5
M  V30 5 2 5 6 CFG=2
M  V30 6 1 6 7
M  V30 7 1 7 8
M  V30 8 1 8 9
M  V30 9 1 4 2
M  V30 10 1 9 7
M  V30 END BOND
M  V30 END CTAB
M  END
"""

m = Chem.MolFromMolBlock(mb)

# The crossed double bond will be encoded as "w:" CX extension
# which requires an atom and a bond index.
b = m.GetBondWithIdx(4)
assert b.GetBondType() == Chem.BondType.DOUBLE
assert b.HasProp('_MolFileBondCfg')
assert b.GetIntProp('_MolFileBondCfg') == 2

m.RemoveAllConformers()  # Don't write coords to CXSMILES
cxsmi = Chem.MolToCXSmiles(m)
print(cxsmi)
# CC1CN1C=CC1CC1 |w:4.5|

m2 = Chem.MolFromSmiles(cxsmi)
# [11:32:52] atom 4 is not associated with bond 5 in w block

The issue is easily fixed if we can find the indexes of the bonds in the original SMILES string.
Note that the test for "t:" and "c:" labels make use of the new stereo algorithm. This is because with the legacy one these stereo labels are reset during the "clean up" stage of assignStereochemistry(): since there isn't a conformation available, these bonds never receive bond directions in https://github.com/rdkit/rdkit/blob/812568eb29a28ba6d0fbf40f8c4fca5d909d04ed/Code/GraphMol/SmilesParse/SmilesParse.cpp#L440-L446

Rather than trying to fix both problems at the same time, I would suggest just changing the title of this and adjusting the description. Then we can fix the SMILES/SMARTS difference (which requires a modification to the SMARTS parser) separately

Done!


Updated German translation to 1.2.0

Here we go :nail_care:

Everything looks perfect to me. Did you include this patch though: https://github.com/HugoGiraudel/sass-guidelines/pull/259?

:scream: doesn't look like. Damn. Will update soon!


Missing "libpng-dev" requirement breaks thumbnails generation

Description
Compiling Pillow without libpng-dev installed breaks thumbnail generation for png images on django-filer.
Solution
It would be useful to add libpng-dev on Pillow dependencies.
# Pillow dependencies && apk add jpeg-dev zlib-dev freetype-dev lcms2-dev openjpeg-dev tiff-dev tk-dev tcl-dev **libpng-dev** \

Debian slim-buster supports libpng-dev. Shouldn't be a problem anymore. My current image configuration (since I don't see Pillow dependencies...):
  # Image dependencies (some for performance)
  && apt-get install -y libtbb2 libtbb-dev libdc1394-22 libjpeg62-turbo libtiff-dev libpng-dev libavformat-dev \

Some of these dependencies are also for opencv, e.g. libtbb2 and libjpeg62-turbo. When dealing with Pillow dependencies, check their installation docs.

Since we switched to Debian Slim, Pillow is installed from precompiled wheel. We no longer install from source and compile at install time, so these dependencies shouldn't be needed.
More details on Alpine downsides here.
I think we can close this issue now.


MatrixSDK has a minimum deployment target of iOS 10.0 - integration error

Hello,
I've downloaded the sample Riot app from xcode11 branch and run the app with my custom server successfully. But when I'm integrating Riot to my app, I'm getting below MatrixSDK error.
Compiling for iOS 9.0, but module 'MatrixSDK' has a minimum deployment target of iOS 10.0: /Users/raju/Library/Developer/Xcode/DerivedData/MetavoxxBusiness-dsfuxkgcqrielscbdgbeouojilvg/Build/Products/Debug-iphonesimulator/MatrixSDK/MatrixSDK.framework/Modules/MatrixSDK.swiftmodule/x86_64-apple-ios-simulator.swiftmodule
my xcode version : 11.2

Cannot reproduce. The xcode11 is updated a lot. You may have tried to build at bad time. Sorry for that.


Update license

@edjafarov would you like to change the LICENSE to be with your name?
I'm a Google employee and so Karma is owned by Google. I'm changing all the licenses to "Google", which means dealing with CLA and other bullshit. I didn't contribute to this plugin that much - you did it, so I think it would make more sense if we change the license to your name. Let me know what you think.

Closing this for now as it is pretty outdated


Change src

Hi there,
How can I change the animation? I wanted to use my own image, can I just point to my images folder?
I tried to use the [template] but did not work

I got it working it’s just that the image does not appear centered as the original animation. What can I do to make my image to appear in the center of the page like the animation does?

In the template string you can define a css class such as 'custom-spinner-template' and make your css centering changes there.
in your component ts file:
template: string =<img class="custom-spinner-template" src="<path>/yourImage.gif" />;
in your component css file:
.custom-spinner-template {     top: 43% !important;     left: 45% !important;     opacity: 1 !important; }