datatypes in file structure can't have commas (like Decimal)

SELECT *
FROM file('a', 'CSV', 'a Decimal(10,1)')

Received exception from server (version 19.11.2):
Code: 49. DB::Exception: Received from localhost:9000. DB::Exception: Odd number of elements in section structure: must be a list of name type pairs.

0 rows in set. Elapsed: 0.062 sec.

Workaround & proper behaviour
SELECT *
FROM file('a', 'CSV', 'a Decimal64(1)')

Received exception from server (version 19.11.2):
Code: 107. DB::Exception: Received from localhost:9000. DB::Exception: File /var/lib/clickhouse/user_files/a is not exist.

0 rows in set. Elapsed: 0.069 sec.

It was implemented in deficient way from the beginning. Need to apply real parser to parse table declaration, similar to CREATE query.


Regenerated build.ninja is not reloaded properly.

Example:
src/ninja% ninja
ninja: no work to do.
------
src/ninja% touch configure.py
------
src/ninja% ninja
[1/1] python2 configure.py
wrote build.ninja.
duplicate rule 'cxx'%

I can't reproduce this. What version? Where is the duplicate rule coming from?

This seems to mostly be caused by the addition of the "Globals" struct. The changeset that's attached to is about refactoring the tools. Is there a reason the globals struct needed to be added? The code was more straightforward before its addition.
In terms of things that were actually changed, the clean tool used to pass the BuildConfig as a constant reference. Now it's effectively being passed as a mutable object.


Using the version before the swift 3 one

Attempting to load the view of a view controller while it is deallocating
"Attempting to load the view of a view controller while it is deallocating is not allowed and may result in undefined behavior (<UISearchController: 0x100817850>)
fatal error: unexpectedly found nil while unwrapping an Optional value"
and then it crashes here:
EPcontact.swift -             phoneNumbers.append((phone.stringValue,phoneNumber.label!))
EPcontactpicker.swift-                 contact =  EPContact(contact: contactsForSection[indexPath.row])

I am getting the same type fo error. It crashes before opening the picker. any fix for this yet?


Duplicate Home Page URLs: / and /index Both Accessible

When using Nextra to create a home page, an /index.mdx file is required to define the content. This setup results in two accessible URLs leading to the same content:

https://nextra.site/
https://nextra.site/index

This duplication can be confusing and may lead to potential SEO issues or unwanted behavior.
Steps to Reproduce

Create a new Nextra site or use an existing one.
Add an index.mdx file to define the home page content.
Access the home page via both https://nextra.site/ and https://nextra.site/index.

Expected Behavior
Only one URL should be accessible for the home page content, preferably https://nextra.site/.
Actual Behavior
Both https://nextra.site/ and https://nextra.site/index show the same home page content, leading to duplicate content accessibility.
Suggested Solution
Consider implementing a redirect or a mechanism to ensure that only one URL (e.g., https://nextra.site/) is accessible for the home page, and the other URL (https://nextra.site/index) is redirected or results in a 404 error.

I think it’s an upstream issue from Next.js? Should be fixed with app router migration in Nextra 4


Add managed resources to the operator status

This adds the various daemon sets to the operator status: the engine
daemonset, the route agent daemonset, and the globalnet daemonset.
Currently the status of these daemon sets isn’t updated outside the
reconciliation loop, which means the status is inaccurate.
Signed-off-by: Stephen Kitt<EMAIL_ADDRESS>

In addition to Daniel’s comment, this needs changes to register the daemonsets, otherwise their status is never updated.


Instead of actually selecting the joint, leave the mesh faces selected for that joint

Avoids confusion when pressing F to focus
Helps user actually better see what's selected

fixed in 448ff6c


Use Optimist

Trollop is a deprecated gem, and has been renamed/replaced by Optimist.
This will probably also need to happen in conjunction with manageiq-providers-vmware, since that also has a script that uses this.
That said, this is not intended to be backported, and should just be used to remove the deprecation warnings the "proper" way.
Links

Org search of current usage of old gem (hides some irrelevant repos):  search

I think manageiq-api has trollop references as well

I didn't realize you had already opened a PR for the api repo: https://github.com/ManageIQ/manageiq-api/pull/515 and now it's merged :)


upgrade pyYAML supported versions in setup.py

Support pyYAML 6.0
Fixes #1472 .

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 96.748%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2886

Relevant Lines:
2983

💛  - Coveralls


Bug: Password with special characters not accepted by GraphQL

What are you trying to achieve?
I was trying to get the token by using this mutation:
mutation token {
 tokenCreate(
   email<EMAIL_ADDRESS>   password: "[secret|password.#"
 ) {
   token
   user {
     id
     email
   }
   refreshToken
   csrfToken
 errors {
   code
   message
 }
 }
}

And got INVALID_CREDENTIALS error even though the same password worked when trying to log into dashboard.
Steps to reproduce the problem

Create a strong password that uses special characters (e.g. using pass generate)
Use the mutation given above
INVALID_CREDENTIALS error shows
Log into panel, go to settings.
Change password to something simple (e.g. using pass generate -n)
Use the same mutation with new pass
Token is returned

What did you expect to happen?
I should have gotten a proper token.
Logs
No response
Environment
Saleor version: whatever is deployed on eu.saleor.cloud

The pass that caused the failure: _9#fC\n]x0p|XOQOOo(0hhQ.y


neverlink is not propagated to transitive runtime_deps

when we set "neverlink=true" for 3rd party dependencies, eg
maven.artifact("io.swagger.core.v3", "swagger-jaxrs2", versions["swaggerJaxrs2"], neverlink = True), the "neverlink" property is only applied to the artifact itself and not propagated to transitive deps. This will introduce more runtime deps when we assemble a package with all runtime classpaths.

I would like to prepare PR to fix this issue. Anya advices regaring compabilty? Should I change behaviour of neverlink param or introduce new param?


lean cloud live doesn't validate brokerage credentials

If you enter incorrect brokerage credentials, the CLI doesn't show an error. It displays the live URL and members need to navigate to the site to see if their deployment was successful.
For example, if you enter an invalid account tier for FTX, the command line makes it seem like the deployment was successful. However, you only see that there was an issue after you open the website.

Algorithm.Initialize() Error: tier (Parameter 'ftxus Tier passed cannot be recognized. Please use one of the following values: Tier1,Tier2,Tier3,Tier4,Tier5,Tier6,Tier7,Tier8,Tier9,VIP1,VIP2,VIP3,MM1,MM2,MM3') Stack Trace: tier (Parameter 'ftxus Tier passed cannot be recognized. Please use one of the following values: Tier1,Tier2,Tier3,Tier4,Tier5,Tier6,Tier7,Tier8,Tier9,VIP1,VIP2,VIP3,MM1,MM2,MM3'): StackTrace: tier (Parameter 'ftxus Tier passed cannot be recognized. Please use one of the following values: Tier1,Tier2,Tier3,Tier4,Tier5,Tier6,Tier7,Tier8,Tier9,VIP1,VIP2,VIP3,MM1,MM2,MM3')

Validating brokerage credentials isn't really available in the cloud either since we need to run Lean, connect to the brokerage and all. But this seems like a bug in the suggested tier options for Ftxus

solved by https://github.com/QuantConnect/lean-cli/pull/91
Previously, it was using click.prompt( type=str) . Now it's using click.prompt( type=click.Choice()) therefore user will only be able to select from provided options


Setup Python version matrix for GHA

@gliptak thank you a lot :) did not know that it can be done this more simple & clear way. Thank you!


Fix #4530: no-string-throw fixer creates syntax errors

PR checklist

[x] Addresses an existing issue: #4530
[x] New feature, bugfix, or enhancement

[x] Includes tests

[ ] Documentation update

Overview of change:
Insert whitespace after throw keyword (if not present) when fixing no-string-throw.
Input
throw('component requires CSS height property');
Fix (Before)
thrownew Error(('component requires CSS height property'));
Fix (After)
throw new Error(('component requires CSS height property'));
Is there anything you'd like reviewers to focus on?

The double parentheses behavior is unrelated to this change, and existed before this bugfix.

I am happy to create an issue for this if so desired.

CHANGELOG.md entry:
[bugfix] no-string-throw fix inserts whitespace if not present after throw keyword

@JoshuaKGoldberg finished those changes you requested.
Oddly enough, I'm seeing that my implementation is failing on CircleCI, but all tests are passing locally on my end. I suspect it has to do with the fact that a Replacement fix is applied to the same position (expression.getStart). Any thoughts as to why there might be this disparity?

Very spooky 👻... Have you tried installing a version of TypeScript that's failing in CI, such as 2.7 or 3.0? Perhaps .pos instead of .getStart() might help?
Alternately, you could try reordering the lines? The TSLint application of fixes is known to be not great at overlapping fixes, but that doesn't seem like it should be affecting these?

I have tried all versions of TypeScript that are failing on CI: 2.1, 2.4, 2.7, 2.8, 2.9, 3.0, rc, and next. All tests are all passing locally with each of these TypeScript versions installed, yet the only version passing on CI is next. Using .pos instead of .getStart did not remedy the situation either. I will continue to investigate this, but I am a bit stumped...

To clarify, my workflow is this.

Checkout to fresh version of this branch
yarn
yarn compile
yarn test (all tests pass)
yarn add<EMAIL_ADDRESS>yarn test (again, all tests pass)
Repeat steps 5 & 6 for all failing TypeScript versions

Am I missing something?

@JoshuaKGoldberg ready for your pass.


Mentioning that drift_db_viewer v2.0.0 is breaking due to changes in Flutter v3.22+ Material implementation

As I understand drift_db_viewer is mentioned here as a tool to show drift data tables on device (and even web).
From v3.22 of Flutter (now Flutter beta channel and Flutter main/master channels) this library is breaking during build.
Please take a look here: https://github.com/vanlooverenkoen/db_viewer/issues/57

Thanks for the report, I've submitted a PR to that repository.

I hope @vanlooverenkoen sees your PR soon...

@RoarGronmo thanks for the ping! I'm on it. You can expect a new release in the coming hours!

Fixed in 1.1.0 for db_viewer and 2.1.0 for drift_db_viewer


Efficiently enumerate symlinked files from manifest

Get file statuses in bulk when symlinked
files have single common parent directlory.
Supersedes https://github.com/prestosql/presto/pull/5349

failures seem unrelated

Add a test to make sure that Parquet works, since that's the primary usage of symlink input format. (even though it shouldn't have worked and only did by accident...)


delete key only work when press shift + delete in some devices

The system related version shows below

node -v --> v11.10.0

npm -v --> 6.7.0
uname -a --> Linux webssh-server 3.16.0-4-amd64 #1 SMP Debian 3.16.51-3 (2017-12-13) x86_64 GNU/Linux
cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 8 (jessie)"
NAME="Debian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

WebSSH2 release version

grep version app/package.json --> "version": "0.2.8"

OS and Version of SSH server connecting to

sshd -v --> OpenSSH_6.7p1 Debian-5+deb8u7, OpenSSL 1.0.2l  25 May 2017

Hi author,
I am glad you offer such a great project.
In most cases, webssh2 works very well.  However, when I use webssh2 to log into some switches, delete key only works when I press shift + delete. I do not know if this is a device-related problem or webssh2 problem

I do not know if this is a device-related problem or webssh2 problem

If you use Putty on those devices, do you have to press shift+delete ?
What TERM variable does that device expect?


Remove instances of testing against byte strings

The astropy 4.1 release includes the fixes to astropy.io.votable by @tomdonaldson that previously caused byte columns to be returned in Astropy tables. The changes in this pull request alter the handful of cases where we showed selection using byte strings, to remove these tests. These code changes should make the notebooks work with both astropy 4.1 and with previous versions.
Some additional cleanup of whitespace and wording in KNOWN_ISSUES.md has also been done, as well as suppressing a new warning from importing aplpy in the CS_Image_Access notebook.

In KNOWN_ISSUES,
services.to_table()[0]['short_name', ivoid', 'access_url']
doesn't look right to me.

I checked it. The line was missing a single quote. After that is fixed, it works.

I've reinstalled the environment as specified in environment.yml. All the notebooks run successfully.


Filename not formatted in form data

When uploading a file, the formadata isn't being turned into JSON.  The request payload includes the following as part of the string: "[object FormData]&...".  It should be converted to JSON and look something like "photo[image]=pic1.jpg&..."
I set up the application using the rails tutorial at
https://github.com/blueimp/jQuery-File-Upload/wiki/Rails-setup-for-V5
If the widget isn't initialized, and I use normal page-submit instead of an ajax submit, the photo saves just fine.

The Rails example is a user contribution and I'm glad if someone keeps it up to date.
The plugin itself does not turn the formData into JSON, as the formData is supposed to be a representation of a normal form submit. However, you can easily turn a formData variable into a JSON format, if your setup requires this.

Hi, thanks for responding!  The plugin is quite nice.
The issue is the file itself isn't appearing in the request formData, or anywhere that I can see in the request Payload or Request Headers.  When I step through in the chrome debugger, the file can be found in various data variables, but it doesn't make it into the request.
I don't think this is a rails issue.
Here is the request payload:
[object FormData]&authenticity_token=6gtB1C8vr7VWWRs2GNrF8nHhAewfmL4sfmDOOot7OT4%3D
To me the: [object FormData] looks like something that didn't get evaluated and turned into a proper URI string.  I've been digging through the code in a debugger for about a day now but can't quite figure out the issue.
Thanks for the help!

Also, it appears that lines 252 - 254 in jquery.fileupload.js are extracting form data
$.each(this._getFormData(options), function (index, field) {
formData.append(field.name, field.value);
});
Is that not true?

I only test the plugin with the provided PHP example and the python-based App Engine demo.
I doubt that this issue is on the client-side, as the plugin works as expected in those scenarios.
Regarding the Rails example, you might want to contact one of the contributing users:
https://github.com/blueimp/jQuery-File-Upload/wiki/Rails-setup-for-V5/_history

When you test with the python and PHP versions, how does the file appear the request payload / Request in general?  Rails only comes into play when the request is being processed, which, to me, appears to be occur the problem.

It's a standard multipart/form-data payload, the same type of data that is submitted via a HTML form submit.
You'll get something like this when watching the request with Chrome's Network tab on the demo:
------WebKitFormBoundarycdwjdwkx1vA2VC1n
Content-Disposition: form-data; name="protocol"

http
------WebKitFormBoundarycdwjdwkx1vA2VC1n
Content-Disposition: form-data; name="files[]"; filename="Walka.jpg"
Content-Type: image/jpeg

------WebKitFormBoundarycdwjdwkx1vA2VC1n--

Chrome will not display the actual file content, but it's submitted to the server.

Figured it out.  This was caused by some JQuery code I had a the base Javascript file.
In a JS file included in a base template, I had an $().ajaxSend() method that appended an authentication token to the request data.  This wasn't playing very well with the ajax request made by file uploader.
That's a day I'm not getting back. :(


Labels - add support to save ethernet interface name

Labels - add support to save ethernet interface name.
Currently, fablib determines the interface name using the mac address for an interface. Since, AM already executes a task to determine the PCI address, same task has been extended to grab the ethernet interface name as well.
This will help avoid extra SSH calls via fablib as this information would be available in Fim model.
Currently there is no place to store the ethernet interface name in Labels. Hence the request.

I can easily accommodate this today as part of https://github.com/fabric-testbed/InformationModel/pull/175 if you prefer. It will be a new 1.4.x release

the local_name field already provides the required capability, so no need to do anything.


Feature request: select best for a given container

The current behaviour for bestaudio+bestvideo is format agnostic and therefore many youtube videos are merged as mkv (mp4+vorbis). It would be good to be able to download the best for a given container e.g. webm - download vp9+vorbis or similarly for mp4 - mp4+aac
Thank you!

It's already possible, read FAQ.


Primary key shouldn't be nullable by default

For now, even if you define a field is primary key, it still enable nullable by default, I think this is not a good practice in most case.

So then do you want us to produce an error when a primary key columns can contain null? Or a warning?

Actually I want to the generated field should be annotationed with @NonNull instead of @Nullable when a field is primary key.

Ah, for the model you mean? Yes that seems reasonable.
On Thu, Nov 17, 2016, 9:07 PM Jiawen Geng<EMAIL_ADDRESS>wrote:

Actually I want to the generated field should be annotationed with
@NonNull instead of @Nullable when a field is primary key.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/square/sqldelight/issues/544#issuecomment-261429094,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEEEZBqtyp1GBfH0HS6anc_c38vJMcpks5q_QhVgaJpZM4KoDPG
.


Crash - Assertion failed: m_lock_info && m_lock_info->m_file.get_path() == m_filename

How frequently does the bug occur?
Sometimes
Description
We are seeing this assertion failure:
Assertion failed: m_lock_info && m_lock_info->m_file.get_path() == m_filename
which asks us to:
please_report_this_issue_in_github_realm_realm_core_v_13_26_0.
The crash is happening on multiple code paths that all lead to opening a Realm and adding records via:
let realm = try Realm(configuration: configuration)

try realm.write {
    realm.add(records, update: .modified)
}

Based on the assertion that is being triggered we are assuming it is related to opening the realm file on disk. The crash seems to only be happening when the app is in the background. We have not had any success in reproducing the crash. All Realm files are in a subdirectory of the app's documents directory that uses URLFileProtection.completeUntilFirstUserAuthentication. This directory can contain multiple Realm files that the app may switch between during a session.
We would appreciate any information on what scenarios might trigger this assertion. Hopefully more information can help us narrow down the cause.
Stacktrace & log output
OS Version: iOS 17.3.1 (21D61)
Report Version: 104

Exception Type: EXC_CRASH (SIGABRT)
Crashed Thread: 6

Application Specific Information:
Assertion failed: m_lock_info && m_lock_info->m_file.get_path() == m_filename

Thread 6 Crashed:
0   libsystem_kernel.dylib          0x3a937dfbc         __pthread_kill
1   libsystem_pthread.dylib         0x3ee67d67c         pthread_kill
2   libsystem_c.dylib               0x32a593b8c         abort
3   ChildCareAttendance             0x201060fa4         please_report_this_issue_in_github_realm_realm_core_v_13_26_0
4   ChildCareAttendance             0x2010611e4         realm::util::terminate_internal
5   ChildCareAttendance             0x2010610c4         realm::util::terminate_with_info
6   ChildCareAttendance             0x201060fb8         realm::util::terminate
7   ChildCareAttendance             0x200e127ec         realm::util::InterprocessMutex::set_shared_part
8   ChildCareAttendance             0x200e10bac         realm::DB::open
9   ChildCareAttendance             0x200e17928         realm::DB::create
10  ChildCareAttendance             0x200ea8e08         realm::_impl::RealmCoordinator::open_db
11  ChildCareAttendance             0x200ea9a68         realm::_impl::RealmCoordinator::do_get_realm
12  ChildCareAttendance             0x200ea9984         realm::_impl::RealmCoordinator::get_realm
13  ChildCareAttendance             0x200eea67c         realm::Realm::get_shared_realm
14  ChildCareAttendance             0x200dae3f4         +[RLMRealm realmWithConfiguration:confinedTo:error:]
15  ChildCareAttendance             0x200dadf90         +[RLMRealm realmWithConfiguration:queue:error:]
16  ChildCareAttendance             0x2010e9904         RLMRealm.__allocating_init
17  ChildCareAttendance             0x2010ea618         Realm.init
18  ChildCareAttendance             0x200673ef0         ChildCareCentralRepository.updateRecords<T>
19  ChildCareAttendance             0x200711b10         AttendancePushProcessor.process
20  ChildCareAttendance             0x200711dbc         AttendancePushProcessor
21  libswift_Concurrency.dylib      0x330a29ac0         swift::runJobInEstablishedExecutorContext

Can you reproduce the bug?
No
Reproduction Steps
No response
Version
10.46.0
What Atlas Services are you using?
Local Database only
Are you using encryption?
No
Platform OS and version(s)
iOS 17.3.1 and 17.1.2
Build environment
Xcode version: 15.1
Dependency manager and version: SPM

Are you ever deleting the Realm files? This could happen if you delete the file while it's open and then a new file is created at a different path and it happens to get the same inode number.
There also might be a race condition in InterprocessMutex when a file is reopened while in the process of being closed on another thread. The code appears to have correct locking but it's somewhat complicated and this could definitely happen if the locking is slightly wrong, so I'll review it to see if I can find anything.

We do delete Realm files on occasion, but it's quite rare (a few times per year), so I don't think it's the cause of this issue.
The second scenario you mentioned, opening a file that is in the process of being closed is very possible. We have a lot of code paths that open Realm files on different threads (i.e., user interaction, background tasks, and APNs). Most Realm interactions are wrapped in an autoreleasepool block which I imagine causes a lot of Realm file closes.
Looking at our code that handles Realm file management, it's certainly possible that a race condition exists on our end. Moving some of that code into an Actor to force synchronous access might be an option.

After looking more into the code producing the assertion failure my conclusion is that we can just delete it entirely as it was a solution to a problem that we have since solved in a much better way.

👍
Glad this issue led to the discovery of some unnecessary code, but I wonder if the fact it was being triggered at all means we should take a hard look at our Realm file access code.


ScrollRestoration access to session store error in iframe

What version of Remix are you using?
1.6.5
Steps to Reproduce
Create a fresh remix application and simply embed in an iframe on another site.
DOMException: Failed to read the 'sessionStorage' property from 'Window': Access denied for this document.
Expected Behavior
Work or fail gracefully.
Actual Behavior
Threw an error DOMException: Failed to read the 'sessionStorage' property from 'Window': Access denied for this document.

Duplicate of #3659


More information saved for Gyms and Pokestops

Is it possible to store more information in the database for Gyms and Pokestops? I'd like to have the name and the picture with it.

I believe this is something that needs to be implemented on the backend map repo: https://github.com/PokemonGoMap/PokemonGo-Map/issues

Ah you're probably right at that one. Didn't notice the separate back-end. Apologies.

It's tripped me up a lot, too. Further complicating it is mchristopher forked the map repo and has implemented some things himself.


[otbn] Harden req/ack on OTP interface

OTBN specific version of https://github.com/lowRISC/opentitan/issues/17437

estimate 4

estimate range: 2 - 4

Closing in favour of https://github.com/lowRISC/opentitan/issues/17772


Confusing behavior when using two part publish syntax

Found while investigating https://github.com/acorn-io/acorn/issues/703
Using the following acornfile:
containers: {
	app: {
		image: "nginx"
		ports: "80/http"
		scale: 1
		files: {
			// Simple index.html file
			"/usr/share/nginx/html/index.html": "
AAACCCOOOORRRNNN
 "
		}
	}
}

Run with this command(s):
acorn run -p test.org:80 single
or
acorn run -p test:80 single

If you have a two-part publish syntax, the first part needs to be the name of a container in the acorn. If no container name is matched, what should happen? Seems like it should error out.

Hi @cjellick , is this issue is still open?

Hello @cjellick,
I investigated this reported issue and here are my findings and possible solutions:
Environment Details:
Acorn Version: v0.8.0+89036514
Cluster type: kind

Observation:
Case 1: test:80 (L143)
When using the command acorn run -p test:80 test, the app is created but not published. This occurs when the first part of the command does not match the container name. No ingress is created for this case.

Case 2: test.org:80 (L136).
In contrast, when using the command acorn run -p test.org:80 test, the app is successfully created and published. It is live and accessible via the Acorn domain.

A note about Case 2 is that, even though this specific case is not mentioned in the documentation under Publish individual ports, there is implementation for this case.
Reference:

Proposed Solution:

To address this issue, I suggest adding validation logic alongside the existing parsePortBindings functionality.
Should we also consider updating the public documentation with currently supported flag values as per code?

I would like to work on this issue and would greatly appreciate any feedback, guidance, or additional insights you can provide.
Thank You!

Thanks for the analysis! I agree with your approach.
One thing to consider: if you can do something on the cli, it should also be achievable directly in the acornfile. So you'll want to make sure it errors out in the same way through that flow.
Definitely want the docs updated as well.

@cjellick, I've raised PRs to address this issue, Could you kindly review them?
PRS:

#2271
#2289


Fixed comment in Glossary which indicated wrongfully that put is a bl…

…ocking effect (fixes #626)

thank :)


allowing import of multiple accounts with identical names

Since i have multiple accounts with the same name but f3 requiering unique names, i included a name-conflict detection where the account is added with its name and the account number, if the name exists more than onces.

Nice feature!! Will be game changing for other users who had multiple accounts with the same name.


[UPDATE] Backend

Unified base class BaseProcessor
Threading as default backend
Evaluator and EnsembleTransformer use mlens.config as default for backend

Coverage decreased (-0.4%) to 90.879% when pulling 63fa62235eed60eab94ae47d61bdd4088b40372c on backend into cd2a973cdfdc21073c690ec6209d32a72dc663e6 on master.


Low throughput (0.61 reqs/sec) when served with RESTful API

What I'm trying to do is to measure the requests mii can handle per second.
So I have started the server with the following code,
client = mii.serve(
            model_path,
            deployment_name=deployment_name,
            enable_restful_api=True,
            restful_api_port=8080,
)

and sent 10 requests to the localhost endpoint asynchronously.
(A request is composed of one prompt, to reflect real-world use case.)
What I'm seeing as a result is that only 0.61 requests are processed per second, which is pretty low to what I have expected after seeing what's written in blog.
Maybe rest api is processing the requests sequentially, not asynchronously?

@noiji please try with the latest main branch. I have made improvements that allow us to match performance the RESTful API to our Python API (see #328).

Thanks for the update!
When run with default(32) restful processes, I see that throughput increases to 3.59 req/s, which is six times bigger than original throughput (0.62/s).

Glad to hear! Closing the issue, but please open a new issue if you see any other problems.


Externalizing CSS for newLink, login, user/index, and header PHP Pages 🛠🎨

Related Issue
Closes: #361
Description of Changes
This PR addresses the need to optimize the codebase by externalizing CSS for the newLink, login, user/index, and header PHP pages. Currently, these pages employ internal and inline CSS, which can lead to code redundancy and maintenance complexities. To enhance code structure, improve maintainability, and align with coding best practices, I have transitioned all internal and inline CSS to dedicated external files for these pages.
Key Changes:
Moved embedded CSS to distinct external CSS files for the newLink, login, user/index, and header PHP pages. 🔗📂
Updated the corresponding PHP pages to reference the newly externalized CSS files. 🔄
By organizing CSS externally, we are promoting better modularity, fostering a streamlined development process, and facilitating easier maintenance. This enhancement contributes to an overall optimized codebase and improved performance.
Your review and feedback on this PR are highly appreciated as we work towards refining our codebase and enhancing our development practices. 🌟📝🚀
Checklist:

[ ] My code adheres to the established style guidelines of this project.
[ ] I have conducted a self-review of my code.
[ ] I have included comments in areas that may be difficult to understand.
[ ] I have made corresponding updates to the project documentation.
[ ] My changes have not introduced any new warnings.

@apoorvaron Please assign me this issue


How do I get a handle on returned items?

This is more of a support request:
How do I get a handle on the returned items before I show them? I would like to animate or fade them in etc. The api is not very clear to me :(
Here is my call:
/*
        * Filter Case studies
        */

        var options = {
    	    valueNames: ['category'],
    	};

        var featureList = new List('case-studies-list', options);

        $('#filter-bfsi').click(function() {
            featureList.filter(function(values) {
            	if (values.category == "BFSI") {
                	return true;
                } else {
                    return false;
                }
            });
            return false;
        });

Hmm, at the moment you can't get a handle on the items before you show them. But it sounds like something that could be interesting to add in the future.
You could probably "hack" this by doing something like this:

$('#filter-bfsi').click(function() {
    var items = featureList.get('category', 'BFSI');

    // Add you handle to items

    featureList.filter(function(values) {
        if (values.category == "BFSI") {
            return true;
        } else {
            return false;
        }
    });
    return false;
});

Well I get a handle on the items but that don't quite help with what I am trying to do, which is fading them in. Here's my code with your 'hack' again:
var options = {
    	    valueNames: ['category']
    	};

        var featureList = new List('case-studies-list', options);

        $('#filter-bfsi').click(function() {

        	 var items = featureList.get('category', 'BFSI');
        	 var $items = $(items);

            featureList.filter(function(values) {
            	if (values.category == "BFSI") {
            		$items.hide();
            		return $items.fadeIn('slow') && true;
                } else {
                    return false;
                }
            });
            return false;
        });

What should I be doing to get this going?

Here is how I think you could do
$('#filter-bfsi').click(function() {
    var items = featureList.get('category', 'BFSI');
    for (var i = 0; i < items.length; i++) {
           $(items[i].elm).hide();
    }
    // Add you handle to items

    featureList.filter(function(values) {
        if (values.category == "BFSI") {
            return true;
        } else {
            return false;
        }
    });

    for (var i = 0; i < items.length; i++) {
           $(items[i].elm).fadeIn('slow');
    }

    return false;
});

Hmm..This should have worked but it isn't. Here is a demo page with the current code implemented.
http://bit.ly/xso4Cs
There must be an easier way to do this, right?

Ouch, found the problem!
Then only one item matches the .get() criteria a single item is return instead of a array. I realize that this is confusing and I'll change that in upcoming releases.
Until then, use this code.
var items = featureList.get('category', 'BFSI');
if (!$.isArray(items)) {
     items = [items];
}


allow ignoring all other args in for_call except those specified

What:
Allow asserting only some call arguments as requested
Why:
Feature request
How:
Add new argument ignore_other_args and ignore_other_kwargs for mock_callable and mock_async_callable
Change get base_runner.can_accept_args to return True if provided args match, if any of the switches provided
Risks:
Checklist:

[X] Added tests, if you've added code that should be tested
[X] Updated the documentation, if you've changed APIs
[X] Ensured the test suite passes
[X] Made sure your code lints
[X] Completed the Contributor License Agreement ("CLA")

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

16 of 16 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.02%) to 94.202%

Totals

Change from base Build<PHONE_NUMBER>:
0.02%

Covered Lines:
2616

Relevant Lines:
2777

💛  - Coveralls

Have you considered using for_partial_call_args to keep the idiom of for_call?


This doesn't seem to be enough?

I believe I've completed the steps above, and in my powershell function app's function I make a call to "Get-AzKeyVault" with the parameters "-ResourceGroupName $ResourceGroupName"
then it crashes and gives me the feedback: "ERROR: Get-AzKeyVault : 'this.Client.SubscriptionId' cannot be null."
This is correct, as when I run "Get-AzContext -ListAvailable" I get an output with no subscription name or id, only tenant, environment and so on.
If I missed something here I am sorry; otherwise it would be nice to have a link to a page explaining the use of cmdlets requiring a default subscription id, in the context of using a managed identity.
I can get it to work if I give the application permissions for the resource group the keyvault and app is in(the same rg(and same subscription for that matter)), then it works without a problem, but that isn't specified as needed as far as I can see, and where I work this needs someone higher in the chain of command to do, so doing it as stated would be preferrable when creating automatic deployments and such.
I use "Connect-AzAccount -Identity" for connecting

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7b8bb1a2-3999-6542-8e85-a4613c516449
Version Independent ID: 57414851-4f75-f358-4721-3797b74c2799
Content: Use an system-assigned managed identity to access Azure Key Vault
Content Source: articles/key-vault/managed-identity.md
Service: key-vault
Sub-service: general
GitHub Login: @msmbaldwin
Microsoft Alias: mbaldwin

@petteremil Thanks for your feedback! We will investigate and update as appropriate.

hmm, after having verbalized this, and after sleeping on it I can sort of see that listing all the keyvaults probably would be access on the resource group level? and not on the keyvault scope..
If I'm not mistaken of course, I think what may have confused me is that the app trying to list the keyvaults is in the same resource group as the vaults, so in my rationalization this meant it didn't need rg access, but I guess that's not the case?
anyways, I guess this would be pertinent to another article in that case, not one about accessing a specific keyvault?
although it's still a bit fuzzy to me that I can access a specific keyvault in a resource group without rg access, but not all of them, sort of.. if you get where I'm coming from :)

@petteremil What is role permissions you have given to application at Management Plane (RBAC) at Resource Group and Subscription level ?

@We will now proceed to close this thread as we haven't heard back. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


About pop3 support

Hi,
Is there any special reason about not installing pop3 (and pop3-ssl)? Seems that with the courier modules and a couple of lines in the config (TLS certs and port), it should work.
I'm asking because if you want to use the mail server with gmail pop3 check, it may not work since this only supports imap.
Great work anyway, these docker images is really helpful.
UPDATE: just saw this issue https://github.com/tomav/docker-mailserver/issues/32, sorry about the duplication.

Hey @crash7, thank you for your message.
If you want to add pop3 support in this public image:

pop3 should be disabled by default
enabling pop3 could be done using an ENV variable
don't forget to add integration tests

If you PR I'll merge. I can help you if you have any question.

@crash7 Let me know if you work on it or not.
Thank you.

Hi!
Yes, I will try to submit this during the weekend.

Thanks!

I have ENV variable set for POP. But I cannot telnet into the pop ports. Why is this?
In docker ps it shows that the pop ports 110 and 993 arenot showing <IP_ADDRESS>    but instead
no IP address.


add rttr::type::get_direct_base_classes()

add rttr::type::get_direct_base_classes()
Outline of changes

added vector storage for direct base classes to class info
reused class info get_base_types by adding parameter to prevent/allow recursion on base classes
added collection of direct base classes to type_register.cpp

Coverage decreased (-0.04%) to 92.242% when pulling 27cacf2fd1ef44704531e97dead9a2c9ae0458fe on dand-oss:get_direct_base_classes into b3a131cf6a42c8bbcd7dd75c2acc9c61c0722775 on rttrorg:master.

Coverage decreased (-0.04%) to 92.242% when pulling 27cacf2fd1ef44704531e97dead9a2c9ae0458fe on dand-oss:get_direct_base_classes into b3a131cf6a42c8bbcd7dd75c2acc9c61c0722775 on rttrorg:master.


Fix typo: "srlen" to "strlen" that may have caused some big issues

This maybe will fix a big issue en rapport avec les chaussettes.


python API [14466]

hey, i want to know how to match python APIs and C++ APIs, cause most of examples are C++ version. It's a little difficult to convert C++ version to python version

Hi @Xuezha-Guo,
Which examples are you looking at? Fast DDS currently includes 2 APIs, the old FastRTPS API which is no longer supported and the DDS PIM standard compliant API whose examples you can find in the DDS folder. The DDS API is practically the same as the Python API. Also, you might find helpful the Python API Reference included in Fast DDS documentation.

ok, i seem to understand. thanks a lot!

Hi @duchengyao ,
As stated in this other comment user transports cannot be set with the current Fast DDS-Python API. It's not possible to create the required SharedMemTransportDescriptor.
However, Fast DDS uses both UDPv4 and SHM transports by default, preferring the latter for data transfer if possible. No configuration should be required to communicate over Shared Memory. Should you need additional transports, these can be set via XML profiles.


Consider deprecating/removing the provider-specific date/time types

Npgsql provides NpgsqlDate, NpgsqlDateTime and NpgsqlTimeSpan for reading and writing date/time values that cannot be represented with the standard BCL DateTime/TimeSpan (which are more limited in range). However, now that we support NodaTime, it may be a good idea to obsolete these provider-specific types.
These types are probably pretty inefficient and may contain bugs (they're seldom-used and a bit untested), and it just doesn't seem right for a database driver to come with its own date/time types.

This is an Npgsql issue, not an EF Core issue, moved to https://github.com/npgsql/npgsql/issues/2009.


Network remove is_clone flag.

Remove the is_clone flag as it is no longer needed with the Contract having mv_elemL/R flags
Remove rowrank semi-colon in Network file.

I just want to make a commend that, this modification potentially can raise errors to the current version. You may want to test the case, for example, that two tensors in the network are the same.

Why removing is_clone flag but not set the default to False?
For cases we want to do A.contract(A) with Network, this might fail?

This is done in #95 .


Home page too wide for mobile chrome (HTC One)

Only on the home page, the width is too wide. See menu button. The page can be slid a few pixels left and right. (Chrome on HTC One. Not in stock browser. that looks fine)

Home page revamped!


Keshav frontent

Changes proposed

links teams and event page to their corresponding sections in hero page
added all events link to event page to visit the doc similar to teams page

Check List (Check all the boxes which are applicable)

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

Screenshots and video
events page button

hyperlink on hero page

teams page

events page

I forgot to update the screenshot of the events page for all events button in readme.md and am currently not on my laptop to do the same. Please @pradeeptosarkar or @mnnkhndlwl do the same.


Make minDvrSize open

Goal
Change minDvrSize visibility to open
How to test
1 . Run all automated tests

This PR will be close since it is not related to DAI


Internal Error: Could not install

Hello!
I installed dart, then activated fvm.
fvm releases works well.
But fvm install stable raises error mentioned above.
Ubuntu 20.04 LTS

Heh, works via install flutter and then flutter pub global activate fvm ))

Hello, I have the same issue.
ubuntu 18.04 LTS

@ValeriusGC could you pass the --verbose flag?

I made this with workaround - installed entire flutter and then fvm .. from that.

@ValeriusGC I am still curious to see what the issue was. The --verbose should be added on the 'fvm install stable' so it can show a stack trace.
I think this is a duplicate of this issue https://github.com/leoafarias/fvm/issues/213

Encountered this with fvm 1.3.8 on Windows 10 x64 today. 1.3.7 Works fine. Maybe related?
Could not install <2.0.2>
#0      installWorkflow (package:fvm/src/workflows/install_version.workflow.dart:44:5)
<asynchronous suspension>
#1      FlutterCommand.run (package:fvm/src/commands/flutter_command.dart:30:7)
<asynchronous suspension>
#2      CommandRunner.runCommand (package:args/command_runner.dart:197:13)
<asynchronous suspension>
#3      FvmCommandRunner.run (package:fvm/src/runner.dart:60:14)
<asynchronous suspension>
#4      main (file:///C:/Users/Przemek/AppData/Local/Pub/Cache/hosted/pub.dartlang.org/fvm-1.3.8/bin/main.dart:4:3
<asynchronous suspension>

@Hexagonale can you run it with --verbose flag?

Ran as developer
fvm --verbose flutter build apk --release
[     0.044s] FVM: Running version 2.0.2
[     0.057s] Exception: On Windows FVM requires to run in developer mode or as an administrator
Could not install <2.0.2>
#0      installWorkflow (package:fvm/src/workflows/install_version.workflow.dart:44:5)
<asynchronous suspension>
#1      FlutterCommand.run (package:fvm/src/commands/flutter_command.dart:30:7)
<asynchronous suspension>
#2      CommandRunner.runCommand (package:args/command_runner.dart:197:13)
<asynchronous suspension>
#3      FvmCommandRunner.run (package:fvm/src/runner.dart:60:14)
<asynchronous suspension>
#4      main (file:///C:/Users/Przemek/AppData/Local/Pub/Cache/hosted/pub.dartlang.org/fvm-1.3.8/bin/main.dart:4:3)
<asynchronous suspension>

@Hexagonale seems like it was not able to create the symlink which could still be permission.
Please download the lates fvm
pub global activate fvm 2.0.0-dev.4
And run the same command with --verbose... it will output the exact error just to confirm that is permission.

All commands as administrator:
pub global activate fvm 2.0.0-dev.4
Package fvm is currently active at version 1.3.8.
...
Activated fvm 2.0.0-dev.4.
Confirm version:
fvm --version
2.0.0-dev.4
And it's working:
fvm --verbose flutter build apk --release
[     0.330s] Version: 2.0.2 - already installed.
fvm: running version "2.0.2"
 Building with sound null safety
Running Gradle task 'assembleRelease'...                           15,7s
√ Built build\app\outputs\flutter-apk\app-release.apk (19.6MB).

Great! I will go ahead and close this issue since the initial problem was for Ubuntu.

I'm getting the same error on Windows

@oliverbytes please open up a new issue with the --verbose flag on the command and "fvm doctor"

Commenting here for visibility: I had the same issue and fixed it by running my terminal as administrator. This is recommended by the --verbose tag already, but somehow the verbose tag did not produce this tip at the beginning.

please be sure to remove or rename the folder in your flutter project .vfm
then run vfm use ...
or install again if not working


TraceWebServletAutoConfiguration should back of if spring-webmvc is not on the classpath

Problem description
We're using spring-cloud-sleuth for tracing instrumentation of Jersey-based REST endpoints, Spring's RestTemplate and others.
As we don't want to unnecessarily pollute the classpath, we exclude certain transitive dependencies which we don't need. E.g.:
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-sleuth</artifactId>
  <exclusions>
    <exclusion>
      <groupId>io.zipkin.brave</groupId>
      <artifactId>brave-instrumentation-jms</artifactId>
    </exclusion>
    <exclusion>
      <groupId>io.zipkin.brave</groupId>
      <artifactId>brave-instrumentation-messaging</artifactId>
    </exclusion>
    <!-- ... -->
  </exclusions>
</dependency>

Generally this is working fine as e.g. in case of TraceMessagingAutoConfiguration there is a @ConditionalOnClass(MessagingTracing.class) present that ensures the auto configuration backs off.
However in case of spring-webmvc this is not the case. The TraceWebServletAutoConfiguration always tries to initialise itself and imports SpanCustomizingAsyncHandlerInterceptor which can't resolve HandlerInterceptorAdapter as it's not on the classpath.
Envionment
spring-boot-2.3.4.RELEASE with spring-cloud-sleuth-core-2.2.5.RELEASE
I didn't check if the refactoring in 3.0.0 fixes the issue but on the first look it doesn't seem to.
Potential solution
Current workaround is @EnableAutoConfiguration(exclude = { TraceWebServletAutoConfiguration.class }) but that won't work for long anymore because TraceWebServletAutoConfiguration has already been marked as @Deprecated.
Simple solution would probably be adding @ConditionalOnClass(HandlerInterceptorAdapter.class).
If you confirm this approach I'm happy to prepare a pull request.

Yeah absolutely be my guest and go with the @ConditionalOnClass(HandlerInterceptorAdapter.class)  Please prepare a PR against the 2.2.x branch. A test would be more than welcome :)


Port to another editors and applications?

Do you have any interest in porting this theme to another editors like zed, helix or other applications like warp and kitty terminal?
I tried to to this port and already did it for the following ones:

zed editor
helix editor
warp terminal

so i mean, is that ok for you? does it make sense to open a PR with this additions? maybe create another repos? wdyt?

and really thank you for this theme! it turned into my daily driver and isn't aggressive to my eyes even in the morning!

Wow never thought of porting darkvoid to other editors or terminals but nice idea in this way we can make a whole ecosystem with this color scheme.
For now I don't think adding these in this repo will be a great idea right, but you can make seperate repos with this colorscheme for porting to other applications, and then we can make a common repo which will have link to all those repos for different terminals and editors, what do you think?
Just like some other themes like catpuccin ?

and really thank you for this theme! it turned into my daily driver and isn't aggressive to my eyes even in the morning!

Hey @zoedsoupe you know what? Thanks this made my day 😁❤

Rather than chase after each and every thing individually, I'd suggest it would be better to create a base16 and/or base24 colorscheme since it could then be used by tinty

yeah it can make sense, but that way you kinda lock theme usage to tiny (for better experience) or depends on base16/24 support, which some apps do not handle
also, porting a theme to base16/24 at least with my experience makes the porting, specially for editor (aka treessiter) kinda difficult while losing some colors, but i'm being biased here so you can just ignore this last paragraph

hey @Aliqyan-21 does you mind to create a GitHub org for this? i think it's better than a single "centralized" repo
with an org everyone could collaborate too (:
i can create it too, but i think as you are the main author, it should be your responsibility
wdyt?

zed theme probably need some enhancement but helix and warp themes are working like a charm, as i'm using it as my daily driver

@Aliqyan-21 here we go:
helix version

source: https://github.com/zoedsoupe/dark-void-helix
zed version

source: https://github.com/zoedsoupe/darkvoid-zed

Do you have any interest in porting this theme to another editors like zed, helix or other applications like warp and kitty terminal?
I tried to to this port and already did it for the following ones:

zed editor
helix editor
warp terminal

so i mean, is that ok for you? does it make sense to open a PR with this additions? maybe create another repos? wdyt?

Wow looks cool @zoedsoupe!
So you really want a github org? Should we do it? And if yes what should we name it?😁

in my opinion is an easier way to other people contribute and manage different port "authors". it's also a way to centralize the contribution docs and the semantics of the theme
i think the name can be darkvoid-theme or even only darkvoid like the catppuccin one. wdyt?

Yes okay @zoedsoupe Darkvoid was what I was thinking too

But Darkvoid is taken

How about Darkvoid-Theme?

that's ok, no problem

i'll move the repositories for it

i'll move the repositories for it

yes sure @zoedsoupe

think we can close it rn


SSL certificate

I'm new to SSL certificates, so my question may be odd.
I've downloaded  MockServer root CA certificate from the website and installed the .pem file (with the help of "update-ca-certificates" command on the Linux machine). Installed MockServer and it runs perfectly.
I have an REST client application which is able to make HTTPS calls to the MockServer (on the same machine) via "localhost". That is, I'm able to run/test my application https://localhost/api/blablabla with MockServer .
Whenver I install MockServer on another machine (and install MockServer root CA certificate), my application fails to access MockServer because of SSL certification. That is, I'm unable to run/test my application https://<IP_ADDRESS>/api/blablabla with MockServer . MY app's running on <IP_ADDRESS>...
Were you able to have MockServer & REST client on different machines?

Hi all,
I've accomplished the setup with the help of mockserver.properties file.
"mockserver.sslSubjectAlternativeNameIps" saved my life :)
jamesdbloom , you may close this issue. Thanks.

Ok good, thanks


New extended pylontech file   correction

The original file pylontech_Force-H.yaml  extended with voltage and temperature sensors of the module cells. The set includes 3 modules.
Replacement of wrong pull request #275

I had to resolve conflicts outside of github: feat: Add extended Pylontech Force profile


Saver uses injected page instead of global capybara reference fo current path

#180

Thank you @ryanmcgarvey, this is great.  Sorry it's taken me so long to surface and merge :(


No CONTAINS keyword.

Current version of cypher-dsl don't support CONTAINS keyword

Also add STARTS WITH, ENDS WITH

Thanks @dMitin. We are rebooting this project. The next version supports this already:
var n = anyNode("n");
var foobarProp = property("n", "foobar");
var statement = Cypher.match(n)
	.where(foobarProp.contains(literalOf("baz")))
	.or(foobarProp.startsWith(literalOf("a")))
	.or(foobarProp.endsWith(literalOf("b")))
	.returning(n)
	.build();

assertThat(cypherRenderer.render(statement))
	.isEqualTo("MATCH (n) WHERE (n.foobar CONTAINS 'baz' OR n.foobar STARTS WITH 'a' OR n.foobar ENDS WITH 'b') RETURN n");


Best way to initialize?

Hi there! This is probably a very newbie question but I searched like whole internet and can't figure it out. So, I have stream.rb in config/initializers with:
require 'stream'
stream = Stream::Client.new(Rails.application.secrets.stream_key, Rails.application.secrets.stream_secret, Rails.application.secrets.stream_id, location: 'eu-central')

The problems is reaching this stream variable from controllers/models. Is there like a common practice on how to do this? I know this is probably simpler with stream-rails but it doesn't seem like its working with rails5 so I decided to go with ruby version.

@benjam1n I'd use something like ::Stream = Stream::Client... but a better pattern would be putting it in your main ApplicationController and accessing from there, like @stream = Stream::Client... and having @stream scoped to any other controller. If you're also going to need it in the models, then the stream-rails gem will probably work better for you. (I'll be working on Rails 5 support in the very near future)


Update Managed Files

Coming back for the tsan issues on this and the other failing repos. Its an external dep bug


HTTP2: Add graceful goaway

Fixes #24237
Implements graceful goaway at the HTTP2 layer.
Before this PR, the behavior of the HTTP2 server on shutdown was -

Send a GOAWAY frame with last stream ID filled with the last stream ID that was accepted.
Keep the connection alive till the number of streams goes down to 0.
Additionally, it would keep accepting new streams even after sending this GOAWAY frame. This is broken behavior as per the HTTP2 spec.

After this PR, the HTTP2 server would perform a graceful shutdown as recommended by the spec -

Send a GOAWAY with last stream ID as 2^31-1 indicating to the client that no new streams should be started.
Waits for an RTT period by sending a ping (and waiting for an ACK). A hard limit of 20 seconds is considered as the limit beyond which we don't wait for the ACK anymore. While this is in progress, we continue to accept new streams to handle streams that were started before the client saw the GOAWAY.
After the RTT, we send a GOAWAY with last stream ID as the last stream that was accepted till this point. No more new streams are accepted after this.
The connection then remains alive till the existing streams are finished.

Thanks for reviewing!
(Ignoring the C# infra flake - ```unzip:  cannot find or open ../../../../input_artifacts/csharp_nugets_windows_dotnetcli.zip, ../../../../input_artifacts/csharp_nugets_windows_dotnetcli.zip.zip or ../../../../input_artifacts/csharp_nugets_windows_dotnetcli.zip.ZIP.


download cover option

Describe your suggested feature
make the covers downloadable in 2.0.0
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
[X] I have written a short but informative title.
[X] If this is an issue with the app itself, I should be opening an issue in the app repository.
[X] I have updated the app to version 1.1.19.
[X] I will fill out all of the requested information in this form.

That's a feature request for https://github.com/LNReader/lnreader


Putting

Basically each WHATWG standard has three script elements that would be nicer if they could be placed at the end of <head> so fetching can start earlier (they're all async/defer).
If the WHATWG-specific template thing gets fixed first though I reckon we no longer need this.
cc @domenic

What's the "WHATWG-specific template thing"?
If all the WHATWG specs use this the same way, it should just be in the boilerplate. Are y'all all putting the same <script> elements manually into each of your specs?

I thought you and @domenic had a plan to create WHATWG-specific boilerplate.
And yeah, that's what's happening. I guess I always forget that we can uplift things to Bikeshed.

I basically keep delaying because when we do so we need to update every spec. So I'd rather do it once. Which means I'd like to roll in https://github.com/whatwg/meta/issues/23 as well. Which is somewhat blocked on DOM/MIME Sniffing breaking the pattern, plus on https://github.com/tabatkins/bikeshed/issues/1058.

What if we keep the current setup around as well as legacy so we can gradually roll this out?

So some specs use whatwg and some use whatwg-new or something?

Basically, to ease the transition and spread the risk.

Yeah, that'll definitely work; it just means a few extra meta commits per spec which I was trying to avoid, but it's probably the right thing to do. I can work on it, although getting #1058 fixed first would be ideal.

Should I do anything with this, then, or can I close it and y'all will adjust/add boilerplate?

I think we can handle this entirely in the boilerplate, yeah, but maybe we can wait for @annevk to confirm he didn't have a different idea.

Sounds fine.


update contributor list

Description
update contributor list

Job PR-288/2 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-288/2/index.html


'unknown' data name when installing pygrib with pip

I faced the following issue while using pygrib 2.1.4 installed using pip.
File "src/pygrib/_pygrib.pyx", line 538, in pygrib._pygrib.open.select
ValueError: no matches found

The grib file i used is from ECMWF and contain '2m Temperature' and 'Total precipitation' data.
Ubuntu 22.04.2 LTS, Python==3.8.13 in a conda environement.
Using pip
pip install pygrib
Test case :
import pygrib

grb = pygrib.open('resources/forecast.grib')

for entry in grb.read(10):
	print(entry.name)

Output :
unknown
2 metre temperature
unknown
2 metre temperature
unknown
2 metre temperature
unknown
2 metre temperature
unknown
2 metre temperature

Using conda
conda install pygrib -c conda-forge
Test case :
import pygrib

grb = pygrib.open('resources/forecast.grib')

for entry in grb.read(10):
	print(entry.name)

Output :
Total precipitation
2 metre temperature
Total precipitation
2 metre temperature
Total precipitation
2 metre temperature
Total precipitation
2 metre temperature
Total precipitation
2 metre temperature

Same issue in #125 and #182 ?

Some grib records don't have a name attribute, apparently.   This is not the fault of pygrib.

probably conda is using a newer version of eccodes


Repo CleanUp - Step 1

[x] Start ~an airtable~ a google spreadsheet audit with all cagov repos
[x] Make sure each repo has a description so when people go to https://github.com/cagov they have an idea of what each repo is for.

Possible future steps

Identify which repos are active/inactive
Identify a lead dev contact for each repo
Archive inactive repos
Implement README template

@ChadABratton Anything to add here?

Closing because it's been a year with no activity.
This work typically happens ad hoc and in the Engineering Coda


Implement Forest Clean

Issue summary

Implement a Forest command clean. The usage could be along of lines of forest clean. The idea is to provide a high-level command to clean-up Forest's data directory, e.g. past snapshots or database that needs to be cleaned because it got too large without playing with dangerous rm -rf and guessing where the heck Forest is storing all this data.
It's a bit of a destructive thing to do, so we should prompt the user and explain the severity (e.g. lost database).
Task summary

[ ] implement prompting the user, providing description of the clean-up action to come,
[ ] implement --force to skip prompting the user, e.g. in automatic workflows,
[ ] implement --snapshots-only flag to delete only the snapshots,
[ ] implement --database-only flag to delete only the database,
[ ] implement reporting how much space was freed thanks to the action. Perhaps it should be included in the first prompt as well.

Acceptance Criteria

[ ] the command should respect the --chain flag, e.g. forest --chain calibnet clean should not touch anything related to mainnet.
[ ] user is able to remove old data without looking for Forest data directory and invoking deletes manually, trying to understand Forests directory structure.

Other information and links

Feel free to propose your own API providing similar functionality, this is just a suggestion how I roughly see it.

Hey @LesnyRumcajs, after looking into this, I have some questions.
Does it make sense to implement different clean func as subcommand of subcommand?  e.g.  forest fetch clean.  Also forest fetch list might be helpful for someone who cannot remember the default snapshot dir like myself, and then another forest fetch remove [snapshot] might be intuitive

@hanabi1224 Sounds interesting, though keep in mind it's not only about downloaded snapshots but also the RocksDB.
.local/share/forest
❯ ls
calibnet/  filecoin-proof-parameters/  keystore  keystore.json  libp2p/  mainnet/  snapshots/

forest fetch list sounds a bit bizarre. Might be worth moving some commands around so it's more self-explaining. That is to say, something like forest snapshot list. Then maybe the download part should be in forest snapshot fetch. Right now it seems to be a mess.
Might be an opportunity for another task that would need to be done before this one? :)

I like forest snapshot  series commands!

Snapshot cleanup will be covered by snapshot prune introduced by this PR https://github.com/ChainSafe/forest/pull/2039


vmTag & Dynamic Templating & Hot Code Pushes

How can I add vmTag to dynamic template? I'm having trouble with page contents loading dynamically into same component templates instead of rendering a new component template for each page's items as defined in each page object in a collection of pages (see below).
(I realize that rendering an app based on a collection is not the best thing but a while ago I ran with the idea of having a whole app be produced from one json file + a UX library so that I can have one file that distinguishes one of my clients from another, with the rest of meteor project being essentially the same, and so that once I create an admin back-end I can use it to add pages to the app/website).
See result here -> http://cch-website-61199.onmodulus.net.
Going to about page from home page breaks the layout, however refreshing about page after that renders it correctly of course. I don't want to have to create separate templates for each page because the layout for each page is defined in the page collection and components are produced by:

<template name="page">
  {{#if page}}
    {{> Template.dynamic template="recursion" data=page }}
  {{/if}}
</template>

<template name="recursion">
  {{#if layout}}
    {{> Template.dynamic template=this.layout data=this }}
  {{ else }}
    {{#if items}}
      {{#each items}}
        {{> Template.dynamic template=this.type data=this }}
      {{/each}}
    {{ else }}
      {{> Template.dynamic template=this.type data=this }}
    {{/if}}
  {{/if}}
</template>

A page data object in the collection of pages looks like this:
{
  'i': 2,
  'id': 'about',
  'name': 'About',
  'landing': {
    'type': 'container',
    'items': [
      {
        'type': 'header',
        'text': 'Welcome'
      },
      {
        'type': 'paragraph',
        'text': 'Welcome to this landing page'
      },
      {
        'type': 'anchor',
        'href': 'http://google.com/',
        'text': 'Go to Google'
      }
    ]
  }
}

As you can see in the recursion template, the type of each object in the page object items array defines a component to be rendered.

Maybe has to do with #210 ? I'm using v4.0.12

Found a solution that seems to work for now (for me at least...I have a vm for every single html element which is kinda why I need this for hot code push purposes so that I don't have to attach an _id to every single element that is rendered from a site json). Was wondering if you could let me know if there are any problems with the following, @ManuelDeLeon:
ViewModel.mixin({
element: {
  id: '',
  vmTag() {
    return this.id() ? this.id() : this.parent().children(this.type()).findIndex((vm) => {
      return vm.templateInstance === this.templateInstance;
    });
  }
}

Aside from the performance hit, there is a chance for the json to change between HCP. You have to decide how likely is that to happen in your app. It may be that the answer is "extremely unlikely".
I have no clue how to make your life easier on this. I keep thinking "it should work without it" but I don't want you to waste time making a repro just to satisfy my curiosity.

It seems to be working well but I like the hot code push feature where the client gets refreshed automatically, and once I start creating admin back-ends to edit pages (site json) so that I can work through a portal and be mobile instead of at a computer, I'd like for the page to still refresh automatically for the client; not sure yet...there's such a small window of time for and such a small chance that someone can't just refresh the page for new data, but I like to always have the server running while I'm locally developing for now so that I don't have to hit F5 1000 times a day.

Still trying to figure out how I wanna do some things and developing my boilerplate


State Storage Roadmap

Great works guys on releasing to public preview... wondering when reliable collections & volume drivers will be released... maybe add to FAQ section?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 33213fe3-c523-df14-797e-856e1597f31d
Version Independent ID: b54b22bd-f275-b4eb-de66-31380ac95fab
Content: State storage options on Azure Service Fabric Mesh
Content Source: articles/service-fabric-mesh/service-fabric-mesh-storing-state.md
Service: service-fabric-mesh
GitHub Login: @rwike77
Microsoft Alias: ryanwi

@Thanks for the feedback! I have assigned the issue to the content author to evaluate and update as appropriate.

@MedAnd  We're working on adding support for reliable collections and volume drivers, but don't have an estimate right now for when it will be ready.  Thanks for the question.  #please-close

We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


Unable to read configuration.

In my config.exs, I have added the following per the docs:
config :ex_state, repo: MyApp.Repo

When I try to do anything with ExState, it fails with:
** (ArgumentError) could not fetch application environment ExState for application :ex_state because configuration at ExState was not set
    (elixir 1.10.2) lib/application.ex:653: Application.fetch_env!/2
    (ex_state 0.2.0) lib/ex_state.ex:106: ExState.repo/0
    (ex_state 0.2.0) lib/ex_state.ex:113: ExState.create/1

Sorry, looks like that configuration change got out of sync with the documentation. The config you have there should work with version 0.3. Alternatively, 0.2 should work as:
config :ex_state, ExState, repo: MyApp.Repo


Updating requirements inside of web DM

Updated the fields cookie, http_referrer, and site within the Web Datamodel to be allowed

:tada: This PR is included in version 1.3.13-develop.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.6.0-develop.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.6.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Experimental oop api

Here's a lot of the progress with the /reportbacks and /reportback-items endpoints.
Example Reportback Retrieve
// REQUEST: http://dev.dosomething.org:8888/api/v1/reportbacks/1209.json

// RESPONSE:
{
  data: {
  id: "1209",
  created_at: "1427226250",
  updated_at: "1428442705",
  flagged: false,
  quantity: 12,
  why_participated: "Because I like doing some things!",
  items: {
    total_retrieved: 2,
    data: [
      {
        id: "2633",
        caption: "Some awesome caption here.",
        uri: "http://dev.dosomething.org:8888/api/v1/reportback-items/2633",
        media: {
        uri: "http://dev.dosomething.org:8888/sites/default/files/styles/480x480/public/uid_170415.jpeg",
        type: "image"
        },
        created_at: "1427226250",
        status: "approved"
      },
      {
        id: "2634",
        caption: "Some other awesome caption here.",
        uri: "http://dev.dosomething.org:8888/api/v1/reportback-items/2634",
        media: {
        uri: "http://dev.dosomething.org:8888/sites/default/files/styles/480x480/public/uid_170412.jpeg",
        type: "image"
        },
        created_at: "1427226585",
        status: "approved"
      }
    ]
  },
    campaign: {
      id: "1321",
      title: "Hunt Day 5: Band-Aid Brigade"
    },
    user: {
      id: "100005"
    }
  }
}

Example Reportback Items Collection
// REQUEST: http://dev.dosomething.org:8888/api/v1/reportback-items.json?campaigns=345,1321&count=2&random=true

// RESPONSE:
{
  total_retrieved: 2,
  data: [
    {
      id: "233",
      caption: "Monuments",
      uri: "http://dev.dosomething.org:8888/api/v1/reportback-items/2313",
      media: {
        uri: "http://dev.dosomething.org:8888/sites/default/files/styles/480x480/public/reportbacks/1321/uid_1713.jpg",
        type: "image"
      },
      created_at: "1423695826",
      status: "approved",
      reportback: {
        id: "1838",
        created_at: "1422458628",
        updated_at: "1424298714",
        flagged: false,
        quantity: 15,
        why_participated: "Good times! Yo!"
      },
      campaign: {
        id: "1321",
        title: "Hunt Day 5: Band-Aid Brigade"
      },
      user: {
        id: "100001"
      }
    },
    {
      id: "984",
      caption: "DoSomething? Just did!",
      uri: "http://dev.dosomething.org:8888/api/v1/reportback-items/984",
      media: {
        uri: "http://dev.dosomething.org:8888/sites/default/files/styles/480x480/public/reportbacks/1321/uid_129-nid_1321-0.jpg?itok=S20O7GvV",
        type: "image"
      },
      created_at: "1408659318",
      status: "approved",
      reportback: {
        id: "1747",
        created_at: "1408659318",
        updated_at: "1408659318",
        flagged: false,
        quantity: 1,
        why_participated: "test"
      },
      campaign: {
        id: "1321",
        title: "Hunt Day 5: Band-Aid Brigade"
      },
      user: {
        id: "100002"
      }
    }
  ]
}

Along with the new resources created in Services, there are also 3 new classes created in the dosomething_reportback module: ReportbackTransformer, Reportback and ReportbackItem.
Reportback and ReportbackItem are subclasses of ReportbackTransformer which allows all the data transformations to be dictated in one class via an assortment of methods. So regardless of which endpoint is hit, and whether retrieving a single item or a collection of items all the data is always returned in a standard format as expected.
The data in the JSON response is also properly typed so no 0 or 1 when it should be a boolean false or true, and where expected numbers are integers instead of strings (except for ids).
There's still a bit more work to be done on the /reportback.json index method, and also allowing for and including pagination data in the responses. More to come on that, but this PR was already getting ginormous ;)
@jonuy @drewish @angaither
CC: @DeeZone @DFurnes
*Note: There currently is no Feature for this endpoint, so you will need to manually toggle it on within the Services Admin.

The nice thing about passing off a lot of the logic to dedicated classes is that if we decide to switch from Services Module to RESTful Module, there isn't much overhaul. In whatever fashion RESTful module addresses resources and catches query parameters, we just pass those along by creating a new instance of the respective class (either Reportback or ReportbackItem) and let it do it's thing ;)

First time taking a look at what you're up to. Thrilled to see reportback-items.json as I can think of a great application in the Message Broker generated transactional campaign signup and digest messages.
Nice work!

thanks @DeeZone! :)

looking good, just a couple of style questions
NOIIICE @weerd

References #4049


[Explore]: Figure out ways to improve payload validation

Background
A forerunner for #2123
We are using the go playground validator library to validate all our api payloads. There are two problems with the library:

In the resulting validation error message all fields are displayed in camel case i.e. the way they are defined in the code and not the way they are defined in json.
The error message does not indicate where exactly in the hierarchical structure of the payload did the error occur

Use this explore to look for ways we could address the problem and present to the team about our options. I have captured a few of the ideas we have discussed below:

Tweak the way we use the current library to address the problem.
Find and explore other validation libraries

@matt-royal has done initial research and shared some alternatives and his thoughts about them. 👍🏽

ozzo-validation - seems closer to Rails/Sequel validation and more flexible. Over 3k stars on github
valix - looks similar to our current solution and not many stars on github.
validator - looks similar to our current solution. Over 1k stars on github. Unclear if we can customize error messages.
validating - seems similar to ozzo-validation on the surface. ~200 stars on github.

Acceptance Criteria
Capture your findings and present it to the team in a meeting.
Timebox
No timebox
Dev Notes
No response

I had done some research on this in https://github.com/cloudfoundry/korifi/issues/1947. I had looked at:

santhosh-tekuri/jsonschema which didn't seem great;
eirini-forks/validus which is a spike of mine, see https://github.com/cloudfoundry/korifi/issues/1947#issuecomment-1329096979 for an example;
gookit/validate.

Here's an example using ozzo-validation. Running this yields the following output:

Error:  version cannot be blank, applications.0.memory must use a supported unit (B, K, KB, M, MB, G, GB, T, or TB), applications.0.instances must be 0 or more, applications.0.routes.0.route is not a valid route

package main

import (
	apierrors "code.cloudfoundry.org/korifi/api/errors"
	"errors"
	"fmt"
	validation "github.com/go-ozzo/ozzo-validation"
	"gopkg.in/yaml.v3"
	"io"
	"regexp"
	"strings"
)

type Manifest struct {
	// Error field names will be pulled from the `json` tag. We can globally change this to `yaml` by setting
	// validation.ErrorTag = "yaml", but this will break json-encoded payloads. Alternatively, we can ensure that
	// yaml-encoded payloads also have `json` tags. We can also manually build up the errors ourselves, as shown
	// here: https://go-ozzo.github.io/ozzo-validation/#validation-errors
	Version      int                   `yaml:"version" json:"version"`
	Applications []ManifestApplication `yaml:"applications" json:"applications"` //validate:"dive"`
}

func (m Manifest) Validate() error {
	return validation.ValidateStruct(&m,
		validation.Field(&m.Version, validation.Required),
		validation.Field(&m.Applications),
	)
}

type ManifestApplication struct {
	Name                         string          `yaml:"name" json:"name"`                                                       //validate:"required"`
	Instances                    *int            `yaml:"instances" json:"instances"`                                             //validate:"omitempty,gte=0"`
	Memory                       *string         `yaml:"memory" json:"memory"`                                                   //validate:"amountWithUnit,positiveAmountWithUnit"`
	HealthCheckInvocationTimeout *int64          `yaml:"health-check-invocation-timeout" json:"health-check-invocation-timeout"` //validate:"omitempty,gte=1"`
	HealthCheckType              *string         `yaml:"health-check-type" json:"health-check-type"`                             //validate:"omitempty,oneof=none process port http"`
	Timeout                      *int64          `yaml:"timeout" json:"timeout"`                                                 //validate:"omitempty,gte=1"`
	Routes                       []ManifestRoute `yaml:"routes" json:"routes"`                                                   //validate:"dive"`
}

func (m ManifestApplication) Validate() error {
	return validation.ValidateStruct(&m,
		validation.Field(&m.Name, validation.Required),
		validation.Field(&m.Instances, validation.Min(0).Error("must be 0 or more")),
		validation.Field(&m.HealthCheckInvocationTimeout, validation.Min(1)),
		validation.Field(&m.HealthCheckType, validation.In("none", "process", "port", "http")),
		validation.Field(&m.Timeout, validation.Min(1)),
		validation.Field(&m.Routes),
		validation.Field(&m.Memory, validation.By(validateAmountWithUnit)),
	)
}

type ManifestRoute struct {
	Route *string `yaml:"route" json:"route"` // validate:"route"`
}

func (m ManifestRoute) Validate() error {
	routeRegex := regexp.MustCompile(
		`^(?:https?://|tcp://)?(?:(?:[\w-]+\.)|(?:[*]\.))+\w+(?:\:\d+)?(?:/.*)*(?:\.\w+)?$`,
	)
	return validation.ValidateStruct(&m,
		validation.Field(&m.Route, validation.Match(routeRegex).Error("is not a valid route")))
}

func DecodeAndValidateYAMLPayload(r io.Reader, object validation.Validatable) error {
	decoder := yaml.NewDecoder(r)
	decoder.KnownFields(false)
	err := decoder.Decode(object)
	if err != nil {
		return apierrors.NewMessageParseError(err)
	}

	return validatePayload(object)
}

func validatePayload(object validation.Validatable) error {
	err := object.Validate()
	if err != nil {
		return apierrors.NewUnprocessableEntityError(err, strings.Join(errorMessages(err), ", "))
	}
	return nil
}

func errorMessages(err error) []string {
	return errorMessagesWithLeafInfo("", err)
}

func errorMessagesWithLeafInfo(field string, err error) []string {
	errors, ok := err.(validation.Errors)
	if !ok {
		return []string{field + " " + err.Error()}
	}

	prefix := ""
	if field != "" {
		prefix = field + "."
	}
	var messages []string
	for f, err := range errors {
		ems := errorMessagesWithLeafInfo(f, err)
		for _, e := range ems {
			messages = append(messages, prefix+e)
		}
	}
	return messages
}

func validateAmountWithUnit(value any) error {
	var unitAmount = regexp.MustCompile(`^\d+(?:B|K|KB|M|MB|G|GB|T|TB)$`)
	v, isNil := validation.Indirect(value)
	if isNil {
		return nil
	}

	if !unitAmount.MatchString(v.(string)) {
		return errors.New("must use a supported unit (B, K, KB, M, MB, G, GB, T, or TB)")
	}

	return nil
}

func main() {
	manifest := new(Manifest)
	err := DecodeAndValidateYAMLPayload(strings.NewReader(`---
      #version: 1
      applications:
      - name: app1
        default-route: true
        memory: 128
        disk_quota: 256M
        instances: -1
        routes:
        - route: "0"`),
		manifest)

	if err != nil {
		fmt.Println("Error: ", err.(apierrors.UnprocessableEntityError).Detail())
		return
	}

	fmt.Println("No errors")
}

This example demonstrates...

Custom validations
Customizing the message for built-in validations
Using the json field name in errors
Showing the full path to errors on nested fields (e.g. applications.0.memory)
We were able to replace the custom route validator with the built-in regexp validation with a custom error message

Based on my @gcapizzi and my notes at the top, we reviewed the following packages:

https://github.com/RussellLuo/validating
https://github.com/eirini-forks/validus
https://github.com/gookit/validate
https://go-ozzo.github.io/ozzo-validation

From these, ozzo-validation seemed like the most flexible and intuitive option. We were able to verify in a spike that it meets all of our needs (see comment above). The one major concern with this package is that it doesn't seem to be actively maintained. However, it is by far the most popular library we explored, it has only 1 production dependency and there is a recent fork that claims to be actively maintained. We believe that we can mitigate the risk of using this package by using the active fork, and that we will be able to make changes ourselves should we run into issues.
Regarding the other packages:

RussellLuo/validating looks less intuitive than ozzo-validation, and it wasn't clear from their documentation if it would allow us to properly explain nested errors to users
validus is young and will require us to build out a fair amount of logic ourselves. We believe we would be better served by modifying a more seasoned package when we run into trouble.
gookit/validate seems mostly focused on processing web forms, doesn't encourage encapsulation of validation logic, and also uses tags to configure validation. We believe we'll have many of the same problems with this package as we do with our current one.

We also explored getting better error messages out of our current validation package. We may be able to correct the bad field names for the manifest by specifying json tags for each field, but we don't believe there's a way to properly display nested errors.
Recommendation
We recommend that we use the https://github.com/jellydator/validation fork of ozzo-validation for #2123

I like the look of this. Compared to go-playground/validator, I prefer the explicitness of defining the validation in code in a Validate method. It's quick enough to add out-of-the-box validations, and equally clear how to add custom ones when required.
When we make the actual change, I think it will be good to adopt the (already half-implemented) change to inject the payload decoder/validator into the handler and test it in the payloads package. We can then mock a single validation error in the handlers tests, and go into the detail in the payloads tests.
It probably makes little performance difference, but in the example code it might be better to declare and assign the regexes outside of the methods so they are only compiled once.


get_docids_for_value_range is broken w/certain fast fields that use a GCD inverse

Describe the bug
When inserting fast fields in a certain order, segments will occasionally end up with a column that matches extra documents when calling get_docids_for_value_range using nonexistant values.
The testcase at the bottom of this report illustrates a failing case, though you need to run it repeatedly to cause it to fail.
Expected behaviour: passing fastfield values/ranges to get_docids_for_value_range that cannot possibly exist in a given segment reader should return no matching documents.
Which version of tantivy are you using?
0.19.0
To Reproduce
This code will reproduce the bug, but it won't happen every time. Running it repeatedly will eventually give you a SegmentReader with the min/max values [6341073085727221<PHONE_NUMBER>471920086], and that SegmentReader will always match document zero when querying in the invalid value<PHONE_NUMBER>999999.

    #[test]
    fn test_gcd_bug() {
        let dir = RamDirectory::create();
        let mut schema_builder = Schema::builder();
        let url_norm_hash_field = schema_builder.add_i64_field("url_norm_hash", FAST | INDEXED);
        let schema = schema_builder.build();
        let index = Index::create(dir, schema, IndexSettings::default()).unwrap();
        let mut writer = index.writer(50_000_000).unwrap();
        for i in [6341073085727221_i64,<PHONE_NUMBER>727221_i64,<PHONE_NUMBER>727221_i64,<PHONE_NUMBER>727221_i64,<PHONE_NUMBER>727221_i64,<PHONE_NUMBER>727221_i64,<PHONE_NUMBER>471920086_i64,<PHONE_NUMBER>471920086_i64,<PHONE_NUMBER>471920086_i64,<PHONE_NUMBER>471920086_i64] {
            writer.add_document(doc! {
                url_norm_hash_field => i,
            }).unwrap();
        }
        writer.commit();
        let reader = index.reader().unwrap();
        let searcher = reader.searcher();
        for segment in searcher.segment_readers() {
            let field = segment.fast_fields().i64(url_norm_hash_field).unwrap();
            let (min, max) = (field.min_value(), field.max_value());
            println!("{} {}", min, max);
            if<PHONE_NUMBER>999999_i64 >= min &&<PHONE_NUMBER>999999_i64 <= max {
                let mut vec = vec![];
                field.get_docids_for_value_range(7999999999999999_i64..=7999999999999999_i64, 0..u32::MAX, &mut vec);
                println!("{:?}", vec);
            }
        }
    }

https://gist.github.com/mmastrac/a06dddbdc2734bde74764edae7418333

confirmed as a bug

Thanks for the report!
For better performance, we try to convert the users value to the value space of the fast field when running get_docids_for_value_range . With GCD, the value space gets smaller. e.g. [0, 100, 200] is mapped to [0,1,2]. We cannot map a incoming user value of 50 to that user space exactly.
Currently inverse mapping is executed self.monotonic_mapping.inverse(50), which returns 0. Instead we need to detect that we are outside the value space and map to the closest next value. +1 for the start of the range, and -1 for the end of the range
Minimal Example
    #[test]
    fn test_gcd_bug_regression_1757() {
        let mut schema_builder = Schema::builder();
        let num_field = schema_builder.add_u64_field("url_norm_hash", FAST | INDEXED);
        let schema = schema_builder.build();
        let index = Index::create_in_ram(schema);
        {
            let mut writer = index.writer_for_tests().unwrap();
            writer
                .add_document(doc! {
                    num_field => 100i64,
                })
                .unwrap();
            writer
                .add_document(doc! {
                    num_field => 200i64,
                })
                .unwrap();
            writer.commit().unwrap();
        }

        let reader = index.reader().unwrap();
        let searcher = reader.searcher();
        let segment = &searcher.segment_readers()[0];
        let field = segment.fast_fields().u64(num_field).unwrap();
        let mut vec = vec![];
        field.get_docids_for_value_range(150..=150, 0..u32::MAX, &mut vec);
        assert_eq!(vec.len(), 0);
    }

Buggy Code
This part of the code needs to be able to handle user data
    fn get_docids_for_value_range(
        &self,
        range: RangeInclusive<Output>,
        doc_id_range: Range<u32>,
        positions: &mut Vec<u32>,
    ) {
        self.from_column.get_docids_for_value_range(
            self.monotonic_mapping.inverse(range.start().clone())
                ..=self.monotonic_mapping.inverse(range.end().clone()),
            doc_id_range,
            positions,
        )
    }


chore: update fill_and_empty tests

Decreasing db_size to make tests faster without losing the efficacy of the tests. Still, around 3k leaves are created in total for each fill and empty test. The one using 64 workers is averaging 200 range extension requests

#445  👈
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @gabriele-0201 and the rest of your teammates on  Graphite


Parser hangs with certain CREATE TABLE statement

The following statement fails to return from parsing:
create table t1 (a int32 primary key(a),) pARTITION by hASH(a)  PARTITIONS 2, RANGE(a) (PARTITION 99 < VALUES <= (100)) REPLICAS 1;

Also appears to be an issue for certain ALTER TABLE statements, e.g.: alter table test add column col int32;.


IOr value

Hello,
I have been using Vavr extensively in my projects and had a need for an IOr value type. I understand that there is no such value type in Vavr and was wondering if Vavr needs it. In my opinion, it would be a nice addition for Vavr.
I have already implemented it and would love to contribute it to the project. Let me know if you are interested in it and I'll create a PR.

Hi Daniel,
forgot to mention, this is inclusive-or which compliments either (exclusive-or) and it was inspired by Cats IOr from Typelevel. The use case would be similar to Either but in cases when you have non-fatal errors on the left side that you want to collect.
It is ok if you think that IOr wouldn't fit into Vavr because it is not the part of Scala standard library. However, Vavr's Validation is not part of the Scala standard library too but it fits nice in Vavr.

Hi, thanks for the clarification!
Yes, you are right, Validation is fitting well into Vavr on the one hand but it is not part of the standard library on the other hand. My last efforts focused on simplifying Vavr. It would be good to move Validation out of Vavr, into its own library that builds on top of Vavr, similar like Cats built on top of Scala. IOr could then accompany Validation.

Sounds good!
Let me know if there will be a project that builds on top of Vavr. I'd be happy to contribute.