Missing ability to include another YAML file in a short way (ie combine load + func call)

In ytt, there is the possibility to load values from files, but it does not seem that it is possible to load another YAML file from the main ytt file.
Could this be considered as an addition?
Something like this:
main.yaml
---
test: #@ include("hello.yaml")

hello.yaml
---
#@ def world()
"world"
#@ end

hello: #@ world()

the result would be:
---
test:
  hello: "world"

Having a way to pass variables to the included file would be nice. What this is is basically a #@ load() but with the function definition implicit in the included file.

hi @mildred ,
ytt in fact allows for loading files with function definitions in other files.
Here is an example from the ytt playground with a very similar usecase to what you explained above: https://get-ytt.io/#example:example-load

Yes, I understand, but I wanted a way to avoid having to explicitly declare the functions. I would like to have instead of funcs.lib.yml
func.lib.yml:
name: max
cities:
- SF
- LA

without the #@ def func1():. The whole file would be by itself a single function.
Perhaps that can be made into the load() builtin. So when there is no #@ def in the loaded file, an implicit _main function could be defined.
The main template could then be:
main.yml
#@ load("func.lib.yml", func1="_main")

func1_key: #@ func1()

which would result in:
func1_key:
  name: max
  cities:
  - SF
  - LA

It's just that I'm used to have include statements, and having to define functions seems a bit odd when you mean to have the whole file as a single function.

i was actually considering adding this functionality at some point, but went with a more conservative approach for now.
this was due to awkwardness with parametarization. ie how does one specify which args whole file takes. i guess it might be worth adding this feature without allowing args, but then "upgrade" from no-args to args is a bit weird. wdyt?

You're right, what I'm asking is not really necessary, and is more convenience. in any case it's not very important.
Upgrade can happen simply by adding a #@ def statement enclosing the whole file that map to the default function name.

feel free to close this issue or file it for later if you think it's relevant.

What can be useful though, is having a way to perform both the load() and the function call.
A way to shorten the following file:
#@ load("parts/a.yml", parta="main")
#@ load("parts/a.yml", partb="main")
#@ load("parts/a.yml", partc="main")
---
#@ parta()
---
#@ partb()
---
#@ partc()

@cppforlife, how do you suggest this should be implemented ?
I noticed that load() with only the file name specified was currently invalid (load statement must import at least 1 symbol). I suggest to change load() from a statement to an expression (it would return a value) the following way:

load() with no imports defined is returning a function corresponding to the complete loaded document
load() with a more imports, in addition to importing the functions in the current scope, would return a dict containing the function objects indexed by their function name

I suggest load() with no arguments should be equivalent to enclosing the loaded file with:
#@def the_anonymous_main_function(*args, **kwargs):
...
#@end

(and returning the function)

Oh, I didn't see load() was part of Starlark, and as such it might not be possible to convert it to an expression... too bad.

yeah load() is part of starlark so it might have to be something else if we continue on this route.
i think this issue actually relates to a different problem ive been thinking about recently. given that ytt supports custom libraries (examples: https://get-ytt.io/#example:example-load-custom-library, https://github.com/k14s/k8s-lib), there should be facilities to reuse non-library files (ie templates) from a library.
in the example linked above, k8s-lib, offers library with make() method that returns another struct. this is fine as that repo was designed to be used as a library. however, in other cases, repo may not be designed to be used as a library and just have bunch of templates (no function to call to get them). currently if those templates are "imported" via _ytt_lib there would be no way to access them or pass in specific data/values to them.
i was imagining adding something like below, which may solve this issue as well... this example tries to use https://github.com/k14s/k8s-simple-app-example/tree/master/config-step-2-template as a library:
#@ load("@ytt:lib", "lib")
#@ load("@ytt:template", "template")

#! no custom data/values
#@ sample_app1 = lib.load("github.com/k14s/k8s-simple-app-example/config-step-2-template")

#! with custom data/values
#@ sample_app2 = lib.load("github.com/k14s/k8s-simple-app-example/config-step-2-template", {"hello_msg": "another-msg"})

--- #@ template.replace(sample_app1.templates())
--- #@ template.replace(sample_app2.templates())

and for this issue:
#@ load("@ytt:lib", "lib")
stuff: #@ lib.self().template("hello.yaml")

one challenge with stuff: #@ lib.self().template("hello.yaml") though, is that templating hello.yaml returns document set, which cannot be value of a map item. one way to solve that would be to add unwrap() (or something similar) to yamlfrag so it will end up like this:
stuff: #@ lib.self().template("hello.yaml").unwrap()
#! or
stuff: #@ lib.self().unwrapped_template("hello.yaml")
#! or
stuff: #@ lib.self().inline("hello.yaml")

That sounds like a good idea if you believe it's possible to load files outside of Starlark load statement.
The unwrap() might not be necessary if yaml fragments are indexable. Just .template("hello.yml")[0] might suffice.

A way to pass values to loaded templates, would be to use the existing @ytt:data library. inside the loaded template, data.values.FOO can resolve to:

FOO defined when loading the template via lib.load("lib/", {"FOO": "bar"}).template("hello.yml")
FOO defined in a #@data/values annotated file in the lib/ library directory

reusing existing mechanisms and providing composability.

reusing existing mechanisms and providing composability

yup, that's what i was thinking as well. though need to also figure out how end user can customize data/values given to that library. top level template's data/values can be customized via --data-value flags. i've noted down some thinking in here about using overlays as well: https://github.com/k14s/ytt/issues/11.
one challenge remains is how do you reference particular data/values since library can be templated multiple times (in example above, we have sample_app1 and sample_app2). nothing good came to mind so far...

Do you mind if I attempt an implementation of that ? (implement @ytt:lib.load, extract code from pkg/cmd/template/cmd.go in a separate package to reuse it for load(0 implementation)

@mildred sure, go ahead!

See WIP: https://github.com/k14s/ytt/compare/master...mildred:ytt-lib

@mildred do you want to start a PR. i'd like to leave a few comments as i look deeper into the code.

oops. didnt notice you've already done that...

overriding values:

after spending morning with @nimakaviani we have came to the following proposal:
module == single file (can export variables, functions, or be templated => some type of result e.g. yaml structure, or string, or None)
package == single directory, contains modules
library == collection of packages

load path format:
[@[library]:][package/]module

with libraries being:
- "" (empty) for current library
- "std" (or "ytt) for standard library provided
- ".+" for any library path under _ytt_lib directory

any number of packages in path
single module at the end (std library modules dont have an extension)

examples covering all variations:
# find module within current directory (ie relative path, does not support '..')
funcs.yml

# find module within package 'dir' that's inside current package
dir/funcs.yml

# same as @ytt:overlay
@std:overlay

# use "current library" and load particular module
@:common/stuff.yml

# use "github.com/k14s/k8s-lib/app" library and load
# particular module from it; this library must be
# located under _ytt_lib (eg _ytt_lib/github.com/k14s/k8s-lib)
@github.com/k14s/k8s-lib/app:module.yml

note that @:common/stuff.yml case covers the ability to include modules from packages relative to current library root (instead of supporting symlinks or ../.. in paths). for example load("@:common/funcs.lib.yml", "common1") would work from app1/config.yml, assuming that directory structure is:
app1/config.yml
common/funcs.lib.yml

this loading behaviour can also be used with library module from std library (included via load("@std:library", "library")).
# loading library object into a variable
a = library.get("")
a = library.get("std") # not supported
a = library.get("github.com/k14s/k8s-lib/app")

# library object has following methods
.data_values({...}) # returns new library object with added data values overrides (takes yaml structure as well for input)
.yaml(["common/stuff.yml"]) # returns yaml structures from "common/stuff.yml"
.text(["common/stuff.yml"]) # returns templates string from "common/stuff.yml"
.funcs("common/stuff.yml") # returns module that has exported methods (see module package)
.list() # ["file1.yml", "file2.yml"]
.read("file1.yml") # <raw contents of file1.yml>

what do you think @mildred ?

This looks like a good proposal, I have a few questions:
If I'm located in a module in /_ytt_lib/lib1/pkg1/mod1.yaml (@lib1:pkg1/mod1.yaml)

Would @:pkg2/mod2.yaml reference /_ytt_lib/lib1/pkg2/mod2.yaml or /pkg2/mod2.yaml?
Would @lib2:pkg1/mod1.yaml reference /_ytt_lib/lib2/pkg1/mod1.yaml or /_ytt_lib/lib1/_ytt_lib/lib2/pkg1/mod1.yaml or /_ytt_lib/lib1/pkg1/_ytt_lib/lib2/pkg1/mod1.yaml, or perhaps any of these in a specific search order?

Also, in your API proposal:
.yaml(["common/stuff.yml"]) # returns yaml structures from "common/stuff.yml"
.text(["common/stuff.yml"]) # returns templates string from "common/stuff.yml"
.funcs("common/stuff.yml") # returns module that has exported methods (see module package)

Why does yaml takes an array and func does not? Why not take varargs in the function (less syntax cluttering, but not important)? Also, a feature I wanted in this module was the ability to get all the text or all the yaml from all files from the library if no argument was provided. it frees the library user from knowing the library internals.

In particular, if already within /_ytt_lib/lib1/pkg1/mod1.yaml (@lib1:pkg1/mod1.yaml), if @lib2:pkg1/mod1.yaml can reference /_ytt_lib/lib2/pkg1/mod1.yaml (either directly or through fallback) it allows more powerful things. In particular it allows to avoid duplication of libraries used in different places

I believe

@:pkg2/mod2.yaml reference /_ytt_lib/lib1/pkg2/mod2.yaml as the context for @: is the current library, which in this case is lib1.
same with @lib2:pkg1/mod1.yaml referencing  /_ytt_lib/lib1/pkg1/_ytt_lib/lib2/pkg1/mod1.yaml

yes we are going to have duplication. However, from the slack discussion we had here, I believe we kinda settle on the fact that in the absence of mounting, maybe we can continue with duplication? so having duplicated - but potentially different version of a given library - in multiple _ytt_lib directories is probably ok.

Why does yaml takes an array and func does not?

it is probably an oversight. or did you have particular thoughts @cppforlife ?

Why not take varargs in the function (less syntax cluttering, but not important)

+1 on using variadics here. makes it cleaner.

Also, a feature I wanted in this module was the ability to get all the text or all the yaml from all files from the library if no argument was provided. it frees the library user from knowing the library internals.

I prefer things being explicit. makes readability a lot easier. Recently, I made some concourse adjustments using ytt and the team that used the templates were super happy about having explicit references as to where to look for what function or yaml structure. I can see using something like .yaml(["common/*.yml"]) being a convenience when writing templates but reading those is going to make life harder for some people. Again this is from first hand feedback I received from some other engineers on a project I am contributing to.

If I'm located in a module in /_ytt_lib/lib1/pkg1/mod1.yaml (@lib1:pkg1/mod1.yaml)

Would @:pkg2/mod2.yaml reference /_ytt_lib/lib1/pkg2/mod2.yaml or /pkg2/mod2.yaml?
Would @lib2:pkg1/mod1.yaml reference /_ytt_lib/lib2/pkg1/mod1.yaml or /_ytt_lib/lib1/_ytt_lib/lib2/pkg1/mod1.yaml or /_ytt_lib/lib1/pkg1/_ytt_lib/lib2/pkg1/mod1.yaml, or perhaps any of these in a specific search order?

@: means root of current library. and definition of current library depends on how it was loaded from whoever loaded it. with following files:
_ytt_lib/github.com/lib1/pkg1/mod1.yaml
_ytt_lib/github.com/lib1/pkg2/mod2.yaml

@:lib1/pkg2/mod2.yaml will reference mod2.yaml if mod1.yml was loaded via@github.com/lib1:pkg1/mod1.yml.
if @github.com:lib1/mod1.yaml was used then current library (@:) would be rooted at _ytt_lib/github.com (instead of github.com/lib1) which would cause @:pkg2/mod2.yml not to resolve (from inside mod1.yaml).
the reason why we need to depend on the "initiating" load is because we do not have any filesystem markers that signify where one library begins vs ends (github.com/some-name is up to interpretation to mean github.com library with some-name as a package vs github.com/some-name is the library.)

... or perhaps any of these in a specific search order

all load locations always point to one thing, ie not searching thru multiple directories etc.
Also, in your API proposal:
.yaml(["common/stuff.yml"]) # returns yaml structures from "common/stuff.yml"
.text(["common/stuff.yml"]) # returns templates string from "common/stuff.yml"
.funcs("common/stuff.yml") # returns module that has exported methods (see module package)

Why does yaml takes an array and func does not?

we can append yaml documents together without losing anything. however, with functions we cannot "append" without losing functions that are named the same. i thought it was a safest thing for now, and we can iterate what does it mean to load a package vs a module.
.text(["common/stuff.yml"]) is also a bit weird as appending plain text isnt that useful.

Why not take varargs in the function (less syntax cluttering, but not important)?

being a bit more explicit and leaving possibility of adding future options as the last arg.

Also, a feature I wanted in this module was the ability to get all the text or all the yaml from all files from the library if no argument was provided. it frees the library user from knowing the library internals.

i'd like that as well. just havent formed an opinion on what that should look like. goes back to above point about merging funcs with same names together. possibly extend those functions to support accepting just the package name (eg .funcs(["common"])) or no arguments at all to indicate everything.
probably should also rename funcs to defs to align it better with def identifier.

Well, the main question I had was about _ytt_lib and if nesting it would make sense... If a single _ytt_lib location should be the target for all loads (like GOPATH) or if instead we should nest them (like node_modules)
@nimakaviani, you seem to have answered in favor of the node_modules approach, that is a library loaded as @github.com/lib1 that itself is loading a library @github.com/lib2 would search for it in /_ytt_lib/github.com/lib1/_ytt_lib/github.com/lib2. Advantages are that you always encapsulate the dependency, but the drawback is that you end up with so many of them, with no unity.
The other approach would be to always load libraries in top level _ytt_lib folder only. The example above would look for the library in /_ytt_lib/github.com/lib2 instead. @cppforlife, do you confirm that is not what you want ?
And this makes me think that perhaps separate from ytt itself, we could have a package manager which would populate _ytt_lib automatically, but that's something else entirely.

Also, a feature I wanted in this module was the ability to get all the text or all the yaml from all files from the library if no argument was provided. it frees the library user from knowing the library internals.

I'd like that as well. just havent formed an opinion on what that should look like. goes back to above point about merging funcs with same names together. possibly extend those functions to support accepting just the package name (eg .funcs(["common"])) or no arguments at all to indicate everything.

Actually, getting the complete YAML output from a library is the main use case I was thinking of. Just as by default, when you execute ytt CLI on a directory it outputs everything and you are not forced to select which files to render
The idea of selecting a package name to get the output from it seems a nice idea, including the root package (indicated as "/" ?). Could it be the default if no option is provided or do you prefer it to be explicit (.yaml() would be a shortcut for .yaml(["/"]))

@nimakaviani, you seem to have answered in favor of the node_modules approach, that is a library loaded as @github.com/lib1 that itself is loading a library @github.com/lib2 would search for it in /_ytt_lib/github.com/lib1/_ytt_lib/github.com/lib2. Advantages are that you always encapsulate the dependency, but the drawback is that you end up with so many of them, with no unity.

yes I think i have been advocating for a self-contained modularized model from the get-go. The idea of the library being self-contained is very appealing to me.
yes there is definitely the problem with each library potentially inheriting a copy of a dependency but again if they are all pulled from a unified source (like git for example) the resolution should be relatively easy. Also broken links and dealing with versioning of loaded modules becomes much less of a problem.

Well, the main question I had was about _ytt_lib and if nesting it would make sense... If a single _ytt_lib location should be the target for all loads (like GOPATH) or if instead we should nest them (like node_modules)

my previous comment didn't explicitly mention it but yes, nesting of _ytt_lib is absolutely supported and encouraged to lock down correct dependencies. there is no need to searching for paths in this case either.

Actually, getting the complete YAML output from a library is the main use case I was thinking of. Just as by default, when you execute ytt CLI on a directory it outputs everything and you are not forced to select which files to render

yup yup. just wanted to limit scope at the beginning.

Could it be the default if no option is provided or do you prefer it to be explicit (.yaml() would be a shortcut for .yaml(["/"]))

i dont mind making it a default if no args are provided.

Not sure if it related to that particular PR, but it makes my situation solved. What I want - is to have some "templates" or "main" set component, would place that into root folder and then reuse in each particular implementation of that component, something like:
app1/deployment.yaml
app2/deployment.yaml

And in each of those deployments, I would refer to common component file, like:
load("../components.lib.yaml", "deployment")
However, it seems not possible yet (tested with ytt 0.18.0)

And in each of those deployments, I would refer to common component file, like: load("../components.lib.yaml", "deployment")

correct it's not possible to use ../ in load paths.
thanks as well for letting us know that this will be helpful for your use case.
to unblock you for now, you can do the following (as prev proposed by @mildred in a different issue):
$ ls -l
app1
app2
shared.lib.yml

$ ls -l app1/
config.yml
shared.lib.yml -> ../shared.lib.yml

$ ls -l app2/
config.yml
shared.lib.yml -> ../shared.lib.yml

given above fs structure, you can use --allow-symlink-destination to allow "inside project" symlinks:
$ ytt -f . --allow-symlink-destination .

+1 nice feature to add. Very helpful indeed

https://github.com/k14s/ytt/releases/tag/v0.23.0 includes @ytt:library feature that yall may find appropriate for your use case /cc @carlosflorencio @tfpereira @jdiasamaro @ioben. since this issue covered a lot of use cases, let me know in slack or here if it fits your needs.

https://github.com/k14s/ytt/releases/tag/v0.24.0 includes ability to use / in load statement to load files relative to library root.

Noting:

it's been more than a year since any progress of this issue (and related PRs are long closed/abandoned)
several useful bits have been harvested, as noted above.

This "discussion" has become inactive (and is most decidedly not in-progress). Closing.


Metaspeedup

This dumps header info to the JSON's as they are made for book keeping purposes as well as updates some missing driver attributes


Expand gateway request logging in AWS, GCP, and airnode-client

Closes #1351.
User story for the motivation and context of this PR:
As a [developer], I want to [access request processing logs on any of GCP, AWS, or the airnode-client without needing knowledge of Airnode request handling], so jthat [I can debug failing requests].

I would use the log options with the meta field everywhere where you want to add some data to the log message. Similar to how it's done in Airseeker. Or do you feel like it's better this way?

Ah sorry it wasn't ready for re-review just yet 😅 I pushed that commit in order for CI to build the images that I could then test. But I think the meta option will be the better route 👍

I'm a bit skeptical about how much will this help with the debugging process.

Agreed, but it's a start and provides a lot more than what the user / Airnode operator gets access to currently. I actually found it useful in debugging encoded parameters for a signed gateway request 😅

I would use the log options with the meta field everywhere where you want to add some data to the log message

For the local client, a requestId gets generated and added as logging metadata (with meta: as you suggested) to distinguish requests, but I found this is not needed for AWS & GCP as each generates their own form of unique ID for an incoming request that groups logs nicely (RequestId and executionId, respectively).


Feature: Add custom filepath to monitor for FsNotify

Modify or add  a function to take custom filepaths to monitor and pass them to fsnotify watcher.

@biplavxyz issue resolved in new PR #4

Completed


Model: made LoginCommand work limited to Facebook

Pull Request Test Coverage Report for Build 254

22 of 36 (61.11%) changed or added relevant lines in 8 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.4%) to 90.82%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/main/java/seedu/address/logic/LogicManager.java
0
2
0.0%

src/main/java/seedu/address/model/AddressBook.java
4
6
66.67%

src/main/java/seedu/address/model/account/Account.java
7
9
77.78%

src/main/java/seedu/address/model/ModelManager.java
2
4
50.0%

src/main/java/seedu/address/ui/BrowserPanel.java
1
7
14.29%

Totals

Change from base Build 249:
-0.4%

Covered Lines:
1939

Relevant Lines:
2135

💛  - Coveralls


Model links, and planes links

Nice work.
Model links, and road planes links are not accessible. Need to use VPN?
Thanks.

Yes, these files are saved on Google Drive.

Ok～，thanks.


Add support for "include"

As the code base grows and the number of pages increases, do you think this would be possible to add an "include" syntax that could help factorize some common elements in the pages?
Typically, I'd like to include shell.sql on top of each page to keep a common structure and the customizations I need accross all pages.
I suspect this could quickly complexify the structure so it might not be the right way to meet this need.

Aaand... right after posting I find
https://github.com/lovasoa/SQLpage/blob/3e9987971ec993b6f7d1c0cd9489f6eb98f3011d/examples/corporate-conundrum/game.sql#L1-L3
which simply solves the problem by storing the configuration in the database
https://github.com/lovasoa/SQLpage/blob/3e9987971ec993b6f7d1c0cd9489f6eb98f3011d/examples/corporate-conundrum/sqlpage/migrations/00_base_structure.sql#L52-L62
Simple but efficient. I guess this should be enough for all common cases.

Indeed, I am not a fan on adding non-standard or non-SQL syntax to SQLPage. The whole mantra of the project is that It's just good old SQL.
But I'll rework the examples to make more consistent use of the kind of "page configuration database tables" like the one you highlighted that allows you to make a simple SELECT * FROM shell. Another advantage of the technique is that once you have that, you can make interface elements configurable by the site's users if you want, using a simple form.

Hi Friends, is it possible to use a similar approach for protected pages preface? I want to add the following code to all protected pages but with a DRY approach:
SET username = (SELECT username FROM login_session WHERE id = sqlpage.cookie('session'));

SELECT 'redirect' AS component,
        'signin.sql?error' AS link
WHERE $username IS NULL;

Hi @mnesarco !
This is on my roadmap, but is currently not available. What you can do at the moment to avoid repetition, if you are using a database that supports stored procedures, is
select * from my_auth_function(sqlpage.cookie('session'))

Hi @lovasoa !
Thank you. I have created a framework for Authentication and Authorization using stored procedures in postgresql for this. I will share it on github soon.

I'd like to re-open this, since there is still no good general feature to allow code re-use in SQLPage.
What I am currently thinking of is a sqlpage.run_sql(sql_file, parameters) function that returns json that can in turn be used by the dynamic component. That would be powerful, because it would be composable with existing features.
One could easily do things such as
set personal_data = json_object('person_id', $id);
select 'dynamic' as component, sqlpage.run_sql('personal_info.sql', $personal_data);

Returning a boolean is a little bit limiting. What I was thinking is having it return json, in a way that is compatible with the dynamic component. So, in your example, you would do
select 'card' from component, 'You are logged in' as title
where sqlpage.run_sql('filters/logged-in.sql')->>0->>'authorized' -- access the 'authorized' column of the first row returned by 'filters/logged-in.sql'

With filters/logged-in.sql being something like:
select whatever as authorized;

Yes, I see the JSON approach is quite flexible. And in SQLite, we'd do the following:
select 'card' from component, 'You are an editor' as title
where json_extract(sqlpage.run_sql('filters/editor.sql'), '$[0].access_level') = 4;

where sqlpage.run_sql('filters/editor.sql') returns a JSON array (not an object since we're returning rows):
[
    {
        "username" : "myusername",
        "access_level" : 4
    }
]

and filters/editor.sql is something like:
select username, access_level from users
where username = (
    select username from sessions where token = sqlpage.cookie('session_token')
);

Truly flexible! And would avoid a lot of repeated code 🙂

I'm working on this in https://github.com/lovasoa/SQLpage/pull/253
Currently, the behavior is the following:

Variable values are passed from the including file to the included file:

set x=1
select 'dynamic' as component, sqlpage.run_sql('included.sql')

($x will be 1 inside included.sql)
2. Variable values are not passed back from included to including (forcing the including file to extract the result value of sqlpage.run_sql
What do you think ?

Awesome to hear @lovasoa!
I like the simplicity of automatic passing and blocking included variable values 👏
I do see a benefit to forcing the values explicitly -- preventing unintended side effects if other variables are not intended to be shared with included.sql -- but I think side effects could be mitigated with good naming conventions.

I prefer the explicit args, that prevents shooting yourself in the foot. Implicit passing (aka gobal vars) can generate name collisions hard to debug.

I'm not really sure about that... The included file simply has access to the current request, that is, the URL parameters (like ?x=1) through $x, as well as other request-specific parameters such as the current path (through sqlpage.path()).
The variables are not global, their scope is limited to the current request being executed. Do you see an example where this could be problematic ?
I don't want to think about SQLPage as a traditional programming framework, made to accommodate large and complex applications. SQLPage tries to give priority to simplicity, accessibility, and maintainability. I think this "variables can go only one way" rule works towards these goals.
I think most of the use of this feature will be something like
select 'dynamic' as component, sqlpage.run_sql('sqlpage/header.sql') as properties;

and then header.sql can implement some global rules for the application, handling authentication, generating the shell, etc... And it can use URL parameters for that.
What do you think ?

Hi,
This is awesome ! Thanks for your amazing work.
In my usage, I don't see any scenario where parameters should be useful. For sure, it can add a level of dynamism. Maybe you should release this first and add parameters later if it is needed.

Implemented in v0.20! Please test it and report any issue! 😁


Sorting test data with new GRSISort

I had a go at sorting a run (#7847 in /tig/daq08_data/Tests/) that we took with a single HPGe clover in the lab with a freshly downloaded GRSISort. At the moment I see nothing in the FragmentTree. I'm almost certain that I'm missing something (some flags etc.). Image attached.
This run sorts successfully on grifuser (i.e. where we skip the piled-up hits). We used a Bi-207 source. I think this run used firmware version grif16_30Sep16.sof, writing data in the GRF4 format. The data we will take in the array will use a different version, but the data format should be the same. Has anyone else tried sorting these lab runs?

Just noticed that runinfo has 0 channels, perhaps this is an issue. With the test daq, we often omit some of the information that is present in the MIDAS files taken with the array daq.

0 channels in RunInfo is just a bug that is in the new code. There are still 84 Channels in the TChannel from the look of it.
It’s doing something weird because there is no sub run number. I don’t think this is the issue, but it is one issue.
I personally won’t get a chance to look at this data until I’m in Vancouver on Monday.

Ok. Yup I think with these test runs I always needed to provide a .cal file so that TChannel behaves properly.
Rather naively I tried changing the name of the midas file from run07847.mid to run07847_000.mid but this didn't help with the sort.

Are you sure the data words are coming thorough properly? Have you tried running without errors suppressed and seeing where it’s failing?

It might be that it is failing all the time. I ran the command above with no suppress-error flag. Is it suppressed automatically?

I don’t believe so. This makes me think there is something up with the MIDAS event such that it isn’t being recognized as GRIFFIN.

Yeah, this is because the merging of the new version discarded the check for the bank type GRF4. I've fixed this and am checking it right now. Once I've verified it's sorting now, I'll push the fix.

Fixed by PR #782.

BTW, we might want to fix the code so that it gives an error if the bank name is unknown. I started this in PR #783, but right now it's just an error message with the midas event serial number, but no bank name, maybe we can add that later.

Great, I can check this on the S1601 account.

There is a brand new run taken in the array with two revision 1 GRIF-16s, grsmid00:/data3/griffin/Tests/run07768_000.mid. The source is 152Eu. Fragment sorts fine (from what I can tell) but I am having some issues sorting into analysis files, which I would like to use to look at the CFD time difference (probably not optimized at the moment anyway). If I do:
grsisort -a fragment07768_000.root
It appears to sort OK (showing ~1.9 million events) but hangs up here:

Unless I am just being impatient.

Can you try something like adding the command --sort-depth 100000000 . Add less zeros if it crashes
On Oct 9, 2016, at 5:42 PM, Michael Bowry <notifications<EMAIL_ADDRESS>wrote:
There is a brand new run taken in the array with two revision 1 GRIF-16s, grsmid00:/data3/griffin/Tests/run07768_000.mid. The source is 152Eu. Fragment sorts fine (from what I can tell) but I am having some issues sorting into analysis files, which I would like to use to look at the CFD time difference (probably not optimized at the moment anyway). If I do:
grsisort -a fragment07768_000.root
It appears to sort OK (showing ~1.9 million events) but hangs up here:
[capture3]https://cloud.githubusercontent.com/assets/14794662/19225042/8b6e47ce-8e47-11e6-84a7-5acedb605402.PNG
Unless I am just being impatient.

You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/GRIFFINCollaboration/GRSISort/issues/781#issuecomment-252524254, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFLTwAup1fXqjT_nBCOcremlzn9no3A8ks5qyYnzgaJpZM4KRiCJ.

We tried sorting here this morning and it worked. It might just be getting stuck on a lock somewhere. The computers at TRIUMF had problems with the old (independent) version as well. Not sure what is going on exactly.

We think we've found the problem. There is something strange going on when you sort with -a with no -q option. It gets stuck in the command prompt somehow. We will look into this some more but for now the solution it to sort with the -aq.

the -aq flag works for me, even without the --sort-depth flag.

Have all the data features from the new firmware been examined for consistency with the specification document, GRF4 format is Table 5.9 of https://rawgit.com/wiki/GriffinCollaboration/GRSISort/technical-docs/GRIFFIN_Event_Format.pdf? And check that the results look correct? If any bugs need to be fixed in the firmware we need to identify these as soon as possible. Thanks.

We will get right on it.

Just a quick comment,  when simply giving the -a command alone, the program assumes you want to look at the tree, so it stays open.  It is also why the "normal" status isn't displayed that comes with the minus q command.  Particularly if you are using the new histogramming feature one can examine the histogram live from the terminal while the file sorts in the background.
Pressing enter would make the prompt reappear saying it is ready for commands.

I don’t believe it was making the prompt appear. It was just creating more whitespace.

well that is new.  I had a prompt locally, but I am still using root 5...
On Wed, Oct 19, 2016 at 11:32 AM, Ryan Dunlop<EMAIL_ADDRESS>wrote:

I don’t believe it was making the prompt appear. It was just creating more
whitespace.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/GRIFFINCollaboration/GRSISort/issues/781#issuecomment-254849833,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAacI6YL2a1_V9kyc7_lTiJrRnnqQ3Bpks5q1jgqgaJpZM4KRiCJ
.

I think this is done.


'All' button missing in Sandcastle

Broken in #3232
This promise doesn't ever seem to resolve.
@mramato

Hello World also shows up at the end of the list now, rather than alphabetical.  Also marking as a showstpper.


Named colors

Add new color expression allowing named colors.
Add an example
Add naive tests

I don't really like the syntax namedColor('orange'). I know we have the same issue for the column properties as in the hex colors (hex('#ABC')) but here there are so many letters.
Maybe @makella could help us with this, but we need to thought also about implicit cast for these text colors.

@Jesus89 Yes, namedColor is ugly, but the String API could use names without this.
In fact, @IagoLast , it would be nice if you add that feature, it's another else if block in https://github.com/CartoDB/renderer-prototype/blob/master/src/core/style/parser.js#L133
Regarding the JS API, if you come up with a good name perfect, but color is too generic, and, in any case, I don't think this feature is very useful outside the String API scope.

@Jesus89 @davidmanzanares can you update the named reference in the integration tests?

Yes :point_up:

What is the final String API?

hey guys, sorry missed this earlier.
so what is the final notation for HEX and named colors?


[MTSRE-481] Added secrets and pullSecretName validation

py-mtcli
Description
Added the following validation checks:

secret names should be unique
pullSecretName should be one of secrets's names

/ok-to-test


Eliminate use of abort()

Summary:
Current implementation call abort() in cases when it cannot handle the error. This patch eliminate the use of abort() by either return proper error code, or log error to error log and move on.
For critical errors, what rocksdb do is to turn the DB into read only mode, and wait for user to handle the error (instead of crashing). I'll send a patch to do the same later.
Signed-off-by: Yi Wu<EMAIL_ADDRESS>

Codecov Report

Merging #26 into master will increase coverage by 0.55%.
The diff coverage is 60.95%.

@@            Coverage Diff             @@
##           master      #26      +/-   ##
==========================================
+ Coverage   82.45%   83.01%   +0.55%
==========================================
  Files          42       42
  Lines        2377     2390      +13
==========================================
+ Hits         1960     1984      +24
+ Misses        417      406      -11


Fix error in growth ODE test

Growth ode test in ../test/growth.jl L877 errors at this test:
    @test any(diff(C_cover; dims=1) .< 0) ||
        "ODE never decreases, du being restricted to >=0."
    @test any(diff(C_cover; dims=1) .>= 0) ||
        "ODE never increases, du being restricted to <=0."

This is due to p.X_mb and p.rec being updated outside of the time step loop (i.e. given non-zero values) and being treated as a local variable inside the timestep loop. Due to this, inside the timestep loop p.X_mb and p.rec are matrices of zeros and hence the ODE shows no changes and the test errors. Could perhaps be fixed by moving the time step loop into a function.

Test errors like these should be fixed now as it points to an issue in the model or means that the test needs to be updated due to a change. I don't think it should be done by someone new to the project with no context of how the changes relate to one another.

From what I can see it's not an issue with the ODE but due to ODE variables being out of scope when they are used in the for loop, but I'll find a way to fix it.

Closed in #573


Uploading objects from the UI is always blocked for OpenShift Users

Environment info

NooBaa Version: VERSION
Platform: OpenShift 4.1

Actual behavior

When users logges-in via OpenShift we don't allow them to sign out and they are not considered as the system owners. Currently, we do not allow users that are not the system owners to upload object via the UI, meaning that they will never be able to upload.

Expected behavior

Remove the Uploading UI block, it is no longer relevant.

Steps to reproduce

Login via OpenShift -> try to upload an object via the UI

More information - Screenshots / Logs / Other output

Le'ts remove it all together

Removed the option to upload from the UI - not relevant for our demos / POCs anymore


Fixed and simplified init db script

Hey @rungokarol, the SQL file has been removed as you advised. I believe that the initialisation scripts are going to be used only to set up the local development database. I agree that for deployment purposes we'll need to figure out how to pass creds to the service but currently the app is far away from being deployed anywhere :smile:


How to uninstall?

I have installed it on my pfSense box and it was running great.
But I sold my Ubiquiti systems and now I want to uninstall the software.
How can I do that?

Can someone please help me out?
Many thanks!

The package maintainer doesn't provide an uninstall script, best I can tell you to do is read through the install script and manually remove each package from shell in the opposite order it was installed..
That is until this gets turned into a REAL pfSense package..... (sooner the better!)

Will do a fresh install of pfSense than and restore my settings.
Probably a bit quicker...
Thanks anyways.


Upgrade to Support Node 18 Please

Feature Suggestion

Upgrade to Support Node 18 Please. My backstage app is using node 18, and in order to continue to develop and contribute to roadie plugins that my app uses, I need the plugins to depend on node 18. This will require testing all plugins. I am happy to help test the plugins we contribute to.

not stale


doc: release-notes: add pinctrl release notes

Add list of platforms that have a new pinctrl driver to the release
notes.

@carlescufi ping


My first branch

This pull request adds Sheffin-shaji-george's file to the _posts folder.

add quote

add Sheffin's file

_git add


Handle stopping a publication early

I've encountered a situation where if you subscribe to a huge publication, or any-sized publication really, and you want to stop it early, the publication won't stop until all the cursors have finished running (i.e. onStop doesn't get called until the cursors have returned). This PR allows the onStop to be called immediately in order to stop the cursors. This PR potentially fixes other issues people are having that they may have perceived as a performance issue.
The code on publication.js isn't so great but it works. I am open for someone to clean that up so it isn't so terrible.

This caused unintended behavior changes, see #124
It seems maybe this should be reverted ?
For now we had to lock to 1.7.0


[stable/prometheus-operator] add config to enable or disable rules in kube-apiserver-error.yaml

Is this a new chart

NOTE: We're experiencing a high volume of PRs to this repo and reviews will be delayed. Please host your own chart repository and submit your repository to the Helm Hub instead of this repo to make them discoverable to the community. Here is how to submit new chart repositories to the Helm Hub.

No
What this PR does / why we need it:
This PR adds a config value to enable or disable rules in kube-apiserver-error.yaml (default to enable). This makes it consistent with all the other rules, which all have such an enable value.
Which issue this PR fixes
N/A
Special notes for your reviewer:
Have locally tested with helm lint and install
Checklist

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/mychartname])

@bismarck @gianrubio  what else is needed for this to be merged?

/ok-to-test

force-push to bump the chart version, as it has been bumped by other commits since the last submit.

Could someone help with the test failure? Not sure if it's caused by the PR change or the testing environment.

1043
I0206 19:12:57.861]   Normal   NotTriggerScaleUp  5s (x2 over 15s)  cluster-autoscaler  pod didn't trigger scale-up (it wouldn't fit if a new node is added): 1 node(s) didn't have free ports for the requested pod ports, 1 node(s) didn't match node selector
1044
I0206 19:12:57.862]   Warning  FailedScheduling   2s (x4 over 17s)  default-scheduler   0/3 nodes are available: 2 node(s) didn't match node selector, 3 node(s) didn't have free ports for the requested pod ports.

@qingkunl try to /retest looks like it is CI issue

/retest

/retest

I see same build issue for all other opened [stable/prometheus-operator] PRs. So probably - just need to wait when it will be fixed

/retest

/retest

This CI issue has existed for a few days. Is this being worked on, and is there an issue open to track it?

/retest

/retest

/retest

/assign @gianrubio

/resign

/lgtm


ResourceListingParser#read always returns null

https://github.com/swagger-api/swagger-parser/blob/master/modules/swagger-compat-spec-parser/src/main/java/io/swagger/models/reader/ResourceListingParser.java#L20-L92
returns null with many lines of code commented out.
Why now throw an UnsupportedOperationException instead?
Caveat - I've never looked at this code before today. So I don't know if returning null is consistent.

Currently returning null tells the processor chain to go to next. Not pretty but expected.


[Diagnostics] Stop using unicode line characters

ASCII characters are more reliable.
┆    → :
│    → |
├─   → |-
╰─   → `-
╭───   +---
│    → |
╰───   +---

@swift-ci Please test

This makes me a little sad, but given the issues we've had across various platforms... IMO it seems like the most reasonable route to take for now.

https://github.com/apple/swift/pull/72754
@swift-ci Please test

This makes me a little sad, but given the issues we've had across various platforms... IMO it seems like the most reasonable route to take for now.

Why not use different annotation characters based on the context?

Why not use different annotation characters based on the context?

Unfortunately, we don't really have enough context. We could try to detect when we're in a terminal, but some terminals handle Unicode properly and some do not. And when we're emitting diagnostics to a file, we have no notion of whether the tool reading that file will handle Unicode. So at best, we'd be able to introduce the Unicode in a few places where we know it would work... but then folks would be surprised at the output differing in different environments.


Rebase and merge with TurtleArt and other changes related to Port

[x] Rebase and merge with TurtleArt

[x] Port from GStreamer Gst.Message.structure to get_structure()

[x] Port to Python3

[x] Port to GTK3

[x] Port from GConf to GioSettings

Fixes #4, Fixes #1.
@quozl @pro-panda Please review.

Getting this error while testing

@srevinsaju @chimosky can you review please.

@Saumya-Mishra9129 as the error suggests, there is no *.gschema file as there is on turtleart-activity.

Yeah Thanks, I probably forgot to add that file. I will fix this in morning
😁
On Wed 10 Jun, 2020, 2:27 AM Srevin Saju<EMAIL_ADDRESS>wrote:

@Saumya-Mishra9129 https://github.com/Saumya-Mishra9129 as the error
suggests, there is no *.gschema .xml file as there is on turtleart-activity.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sugarlabs/activity-turtle-flags/pull/8#issuecomment-641565385,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKZH63PNOOFZYB6DTLJ3N4DRV2O2JANCNFSM4NNMCMKQ
.

Ok; I will review the diff tomorrow morning too. (got held up with some work)

Reviewed, thanks.
As this is an update of upstream changes, all your commit messages should reflect that.

@quozl @chimosky @JuiP @srevinsaju , Please review and test.

Ok as I can't do a huge diff review, I will do an activity testing :D

@Saumya-Mishra9129 what is the test procedure for this activity? I was not able to click on any of the flags, does it use any key combination to activate them?

@Saumya-Mishra9129 what is the test procedure for this activity? I was not able to click on any of the flags, does it use any key combination to activate them?

No, Actually there were some minor bugs in the code. Fixed them. @chimosky @quozl @srevinsaju Please review and merge.

@chimosky I have tested activity works fine. Please merge.

Reviewed, thanks.
Collaboration seems to be broken for the activity, although this PR has nothing to do with it but if you have the time you can fix it in a new PR. Thanks.


[Feature]: "replicaValues" field needed in helm chart's values.yaml

Is there an existing issue for this?

[X] I have searched the existing issues

Summary
Helm chart's values.yaml now does not have any field for statefulset replicas number. I think I need a field that can describe it.
Why should this be worked on?
With this field I can set different replicas value when I install an app(now it defaults by 1), or when Appsmith is not used for now, I can set the replicas to 0 to free the resource for my environment by like: helm upgrade ...  --set replicas=0.

Well If you like to accept this feature, I can work on this if you don't mind

Hey @LronDC, thank you for raising this. We are actually already working on this to support clustering of the Appsmith stateful set by just changing the replicas.
I'm happy to see you're interested in working a solution for this, and I hate to reject. But, it's unfortunately a little more involved, and is tied to another fix we are working on. Predominantly around being able scale Appsmith, MongoDB and Redis replicas separately and reliably. So, we'll get back with a solution to this soon.
Thanks again! 🍻

Very good to know that

Hey @LronDC, thank you for raising this. We are actually already working on this to support clustering of the Appsmith stateful set by just changing the replicas.
I'm happy to see you're interested in working a solution for this, and I hate to reject. But, it's unfortunately a little more involved, and is tied to another fix we are working on. Predominantly around being able scale Appsmith, MongoDB and Redis replicas separately and reliably. So, we'll get back with a solution to this soon.
Thanks again! 🍻

Very good to know that. Thank all your fabulous works.

Hey @LronDC any update when would this feature go live ? Thanks in advance

Hey @pruthvi-itribe, the solution is currently in review, and we are working on a migration path and the new Chart with this support may need some manual effort to upgrade. We're testing this out a little more before pushing it out. Unfortunately, I can't give a timeline right now. Thank you for your patience.


Do you have any number for the expected speed-up between one GPU and multiple GPUs?

Hi, Do you have any number for the expected speed-up between one GPU and multiple GPUs? Results of MNIST and Cifar10 dataset are shown as followings respectively. Are they correct?
Thank you,
Chien-Lin
Input: 1024(=1x32x32) dimensions; 60,000 images
Output: 10 dimensions, a sparse vector
Network: 2 Convolution NNs + 1 linear NN
GPU1 total time: 138.024 sec, Batch size: per node = 1, total = 1
GPU2 total time: 108.990 sec, Batch size: per node = 1, total = 2
GPU4 total time:  89.639 sec, Batch size: per node = 1, total = 4
GPU6 total time:  82.023 sec, Batch size: per node = 1, total = 6

Input: 3x32x32 dimensions; 50,000 images
Output: 10 dimensions, a sparse vector
Network: 4 Convolution NNs + 1 linear NN
GPU1 total time: 128.232 sec, Batch size: per node = 32, total = 32
GPU2 total time: 117.435 sec, Batch size: per node = 16, total = 32
GPU4 total time:  98.507 sec, Batch size: per node =  8, total = 32

Seems like reasonable speed ups. Remember, MNIST and CIFAR are tiny networks (by today's standards) and do not benefit much from distributed learning. The only reason I included them was because they are in the public domain and well known.

Thank you Zak. You did really amazing works. I had results on comparing the raw computational efficiency of two toolkits “nn.DataParallelTable” and “torch-distlearn” using simulated data.
Results of nn.DataParallelTable (the bigger number is better):

Results of torch-distlearn (the bigger number is better):


Failed submission

Thanks for the good job and the long-term maintenance of the server.
I encounter some submission error these two days and comments show:
[ERROR] Evaluation failed. No available key left!  and [ERROR] Evaluation failed. 'choices'
It seems like the test server is down?

Fixed. The keys are expired and we replaced them. Please try to re-submit again.

@ChonghaoSima Hello, I am experiencing the same problem~ What can I do to solve it?

Thanks, it works now.


Feature: Catch Keypad Input

Hey,
thank you for your great work! I was just wondering if it's possible to get the entered PINs to HA or at least the name of the corresponding PIN?
The idea is to use the entered PIN as an event to open multiple doors. Like PIN1 for garage and PIN2 for entry door.

Hi,
No this is not possible, neither the numbers nor the PIN entered is exposed by the API.

Thank you for your fast answer!


Backmerge: #3176 – Drawing gets broken after dragging a Functional Group onto an atom

How the feature works? / How did you fix the issue?
(Screenshots, videos, or GIFs, if applicable)
closes #3176
Check list

[ ] unit-tests written
[ ] e2e-tests written
[ ] documentation updated
[x] PR name follows the pattern #1234 – issue name
[x] branch name doesn't contain '#'
[x] PR is linked with the issue
[x] base branch (master or release/xx) is correct
[x] task status changed to "Code review"
[ ] reviewers are notified about the pull request

The failing test is working fine locally:


.option("filter", filter) will overwrite the pushdown filters

Currently, if we run
spark.read.format("bigquery").option("fitler", "_PARTITIONDATE > '2019-01-01'")....createOrReplaceTempView("t")
spark.sql("select * from t where a =1").show()

Then only the partition column filter is pushed down.
Have you considered combining both predicates with "AND"?

Fixed in 0.9.2-beta


Unable to index WARC-Target-URI with angle brackets (as produced by newer versions of wget)

Does the following error message mean a page was not imported:
[2020-01-12 21:20:05] [info] WARNING: FAILED canonicalize(<https://abs.twimg.com/a/1576611360/css/t1/images/ui-bg_flat_0_aaaaaa_40x100.png>)

I can't get anything to actually show up in searches even though they're in the WARCs that are being imported.

It seems that contents of WARCs I create with wget on Debian 9 can be displayed through OpenWayback on either Debian 9 or Debian 10. WARCs I create on Debian 10 cannot be displayed in either.
Is there a way to resolve this issue in my OpenWayback instance?

It appears that wget creates invalid WARC files with the value of the WARC-Target-URI field incorrectly enclosed in angle brackets:
WARC-Target-URI: <http://example.com/>

Looks like pywb has support for wget-style WARCs: https://github.com/webrecorder/pywb/issues/294

Strange, thanks for identifying the problem. Internet Archive has a proxy that I can use with wget to generate compliant WARCs and for now I'll use that since it seems to be getting updated regularly and is a pretty neat drop-in in front of any old web browser or web downloader like wget.


not accepting 0228 as integer (showing : invalid octal digit)

#include
using namespace std;
class Employee
{// private:     //it would anyways be private by default
int SecretPassword;
public:
string Name;
string Profession;
int Age;
Employee(string Name, string Profession, int Age, int SecretPassword){
    this->Name=Name;
    this->Profession=Profession;
    this->Age=Age;
    this->SecretPassword=SecretPassword;
}

void get_SecretPassword(){
    cout<<"The Secret Password of the Employee is "<<this->SecretPassword;
}

};
int main(){
Employee Dev("Devansh Singh","Coder", 25, 0228 );
}

/question


Sorting by price does work on Basilisk?

https://beta.kodadot.xyz/bsx/explore?page=1&tab=GALLERY&sort=updatedAt_DESC&sort=blockNumber_DESC&sort=price_ASC&listed=true
Seems like different order by prices than from low to high?

If several sort options are selected, the first sort condition will take effect.

https://beta.kodadot.xyz/bsx/explore?page=1&tab=GALLERY&sort=updatedAt_DESC&sort=blockNumber_DESC&sort=price_ASC&listed=true

In this case, the order of nfts is based on updated time. If the nfts have the same updated time, then they will be sorted based on price.
So, If we would like to see the nfts sorted by priced first, the url would be https://beta.kodadot.xyz/bsx/explore/?page=1&tab=GALLERY&sort=price_ASC&sort=updatedAt_DESC&sort=blockNumber_DESC&listed=true
IMO, the sort options dropdown menu should be changed to a sort options list, so that user could see which sort condition is the first.


I found an issue with GiveMinionsToo

When the time Ends minions become ghosts .

it's fixed

LOL I WISHED!


Storm should detect correct nimbus.host on mutli-master mesos cluster

My setup:
3 masters (mesos, marathon, chronos, storm) - ubuntu trusty
ips: 192.168.56.{10,11,12} - MASTER(1,2,3)
Using latest build (storm-mesos 0.9.3) + mesos 0.21.1
Running storm UI on all masters
Nimbus get started from Master1
After startup mesos elects MASTER2 to be leader.
storm.yaml

mesos.master.url: "zk://<IP_ADDRESS>:2181,<IP_ADDRESS>:2181,<IP_ADDRESS>:2181/mesos"

storm.zookeeper.servers:
  - <IP_ADDRESS>
  - <IP_ADDRESS>
  - <IP_ADDRESS>

nimbus.host: <IP_ADDRESS>
mesos.executor.uri: "//opt/storm-mesos-0.9.3.tgz"

storm.messaging.transport: "backtype.storm.messaging.netty.Context"

storm.local.dir: "/opt/storm_local"
storm.log.dir: "/var/log/storm"

ui.port: 9998

mesos.framework.role: "*"
mesos.framework.checkpoint: false
mesos.framework.name: "Storm 0.9.3"

When i correcly specify MASTER2(current leader) as nimbus.host - UI works, but when i use MASTER1 or MASTER3 ip i get connection refused. My problem is that setup is automated (Ansible) and dont know which master get elected.

Problem solved - wrong entries in /etc/hosts - MesosNimbus get stuck waiting for acknowledgent (which was sent to wrong host)


Misleading info in FAQ section

[Enter feedback here]
The following section is misleading

When should I use these commands?
We recommend using these commands to migrate a URL ping test to a standard test and take advantage of the available capabilities. Remember, this migration is optional.

Migration is not optional. I'm viewing this page because you've notified me that ping tests will be retired.
It even says so earlier on the same page:

On 30 September 2026, the URL ping tests will be retired, and ping tests. Before that date, you'll need to transition to standard tests.

Also the second part of the first sentence in the above infobox makes no sense: "..., and ping tests." that's not a valid sentence.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 86e42c56-f857-3db0-7b7e-37422e8eda8f
Version Independent ID: 27f33f23-7bd1-142c-be35-988a90e5cbb9
Content: Migrate from Azure Monitor Application Insights classic URL ping tests to standard tests - Azure Monitor
Content Source: articles/azure-monitor/app/availability-test-migration.md
Service: azure-monitor
Sub-service: application-insights
GitHub Login: @AaronMaxwell
Microsoft Alias: aaronmax

@krillerdk
Thanks for your feedback! We will investigate and update as appropriate.


No longer supports set -e

I have a script with set -e set to allow it to bail if there's an error, but with exceptions within. In this script, the NVM sourcing is under set +e and the nvm install is under set -e again (see test script below).
When running nvm install, a cryptic error of "No such file or directory" is output. It appears that nvm_version_path is returning empty, which is causing the error.
Found '/path/to/project/.nvmrc' with version <v6.2.2>
Downloading https://nodejs.org/dist/v6.2.2/node-v6.2.2-darwin-x64.tar.xz...
######################################################################## 100.0%
mkdir: : No such file or directory
Binary download failed, trying source.
######################################################################## 100.0%
Checksums empty
mkdir: : No such file or directory
Binary download failed, trying source.

This was not an issue with v0.29.0 (nor it seems v0.30.0, v0.30.1 or v0.31.0), but appears to be an issue since v0.31.1. A small test file to recreate:
#!/bin/sh
set -e

set +e
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
set -e

nvm install 6.2.2

With version v0.29.0
> $ nvm --version
0.29.0
> $ ./test.sh
Downloading https://nodejs.org/dist/v6.2.2/node-v6.2.2-darwin-x64.tar.xz...
######################################################################## 100.0%
WARNING: checksums are currently disabled for node.js v4.0 and later
Now using node v6.2.2 (npm v3.9.5)

With version v0.31.1
> $ nvm --version
0.31.1
> $ ./test.sh
N/A: version "v6.2.0" is not yet installed
Downloading https://nodejs.org/dist/v6.2.2/node-v6.2.2-darwin-x64.tar.xz...
######################################################################## 100.0%
mkdir: : No such file or directory
Binary download failed, trying source.
######################################################################## 100.0%
Checksums empty
mkdir: : No such file or directory
Binary download failed, trying source.
Detected that you have 8 CPU thread(s)
Set the number of jobs to 8 - 1 = 7 jobs to speed up the build
Installing node v1.0 and greater from source is not currently supported

@alistairjcbrown can you try with v0.31.2? If there's still an issue, this should definitely be fixed.

@ljharb It's still an issue in v0.31.2
> $ nvm --version
0.31.2
> $ ./test.sh
Downloading https://nodejs.org/dist/v6.2.2/node-v6.2.2-darwin-x64.tar.xz...
######################################################################## 100.0%
mkdir: : No such file or directory
Binary download failed, trying source.
######################################################################## 100.0%
Checksums empty
mkdir: : No such file or directory
Binary download failed, trying source.
Detected that you have 8 CPU thread(s)
Set the number of jobs to 8 - 1 = 7 jobs to speed up the build
Installing node v1.0 and greater from source is not currently supported

In this case, the mkdir is failing on the binary install, forcing it to fall back to the source install, which isn't supported - so it should be failing with or without the set -e.
The mkdir warning is the part I find troubling - especially since that didn't fail in v0.29.

This appears to be working for me locally - does your $NVM_DIR exist? What does nvm debug say?

Details below with updated script. Interestingly changing from #!/bin/sh to #!/bin/bash causes the script to work, so this may be a compatibility issue with bourne shell.
Script used:
#!/bin/sh
set -e

echo "NVM_DIR: '$NVM_DIR'"
test -e $NVM_DIR && echo 'NVM_DIR exists'
echo ""

set +e
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
set -e

set +e
nvm debug
echo ""
set -e

nvm install 6.2.2

Output:
> $ ./test.sh
NVM_DIR: '/Users/alistairbrown/.nvm'
NVM_DIR exists

nvm --version: v0.31.2
$SHELL: /bin/zsh
$HOME: /Users/alistairbrown
$NVM_DIR: '$HOME/.nvm'
$PREFIX: ''
$NPM_CONFIG_PREFIX: ''
nvm current: v5.2.0
which node: $NVM_DIR/versions/node/v5.2.0/bin/node
which iojs:
which npm: $NVM_DIR/versions/node/v5.2.0/bin/npm
npm config get prefix: $NVM_DIR/versions/node/v5.2.0
npm root -g: $NVM_DIR/versions/node/v5.2.0/lib/node_modules

Downloading https://nodejs.org/dist/v6.2.2/node-v6.2.2-darwin-x64.tar.xz...
######################################################################## 100.0%
mkdir: : No such file or directory
Binary download failed, trying source.
######################################################################## 100.0%
Checksums empty
mkdir: : No such file or directory
Binary download failed, trying source.
Detected that you have 8 CPU thread(s)
Set the number of jobs to 8 - 1 = 7 jobs to speed up the build
Installing node v1.0 and greater from source is not currently supported


I would like to request pull by putting the mock server function in this library. Is it possible?

Hello, I am a developer who is developing a gateway using the fastapi_gateway library. But when the service to connect to the gateway was not developed, I wanted to use the gateway as a mock server. Therefore, by adding the bool type mock parameter of the route decoder, I will distinguish whether the corresponding API of the gateway is connected to the service, and if the mock is True, I will respond to the sample data added to the responses parameter. Is this effective in your opinion? If you think it's efficient, Is it okay if I develop the function and make a pull request?

@uk1996, hello!
It seems to me that this is clearly not the task of the gateway, but you need to mock the answer on the final server.

I understand, thank you for taking the time to reply!


Add samtools stats for mapping stats

It would be nice to have the mapping stats from BWA in the multiqc report.

Agreed! BWA doesn't output any summary statistics, so it's not possible to add it directly to MultiQC. However, we could add a process that runs samtools stats, which will give summary statistics to show in the report.

Done in #51


Delete email from script

Subject

[ ] I would like to contribute to the project (code, documentation, advocacy, integration, ...)
[ ] I would like some help for integrating this container with another service (rainloop, PostFWD, ...)
[* ] I would like to configure a not documented mail server use case
[ ] I would like some feedback concerning a use case
[ ] I have questions about TLS/SSL/STARTTLS/OpenSSL
[ ] Other

Description
Hi,
I created a script for mass creating mailboxes. It also needs the same to remove them.
What is the command, the equivalent of this:
docker run --rm -e MAIL_USER = $ {user} -e MAIL_PASS = $ {email} -ti tvial / docker-mailserver: latest / bin / sh -c 'echo "$ MAIL_USER | $ (doveadm pw -s SHA512- CRYPT -u $ MAIL_USER -p $ MAIL_PASS) "'>> config / postfix-accounts.cf

You can use ./setup.sh email del <email>
setup.sh takes also care about aliases if configured.
Or invoke delmailuser directly.

Thank you. It help.


Is there support for conditional multinomial logistic regression?

I wonder if this type of cMNL (the type you use in Choice Based Conjoint/ DCE is supported?

Hi rolandkofler,
Could you be more specific about the type of cMNL (any links?) that you are wondering about, or the type of features you are wondering about?
If you're wondering if this package can be used to help design discrete choice experiments, then no this package doesn't have those capabilities. However, PyLogit can estimate conditional logit models after the data from such an experiment has been collected.
If you're wondering about support for features such as individually varying sensitivities (i.e. mixed logit models), they're not yet implemented.


Reformatted exhaustion list into tabular data & added full stops

To keep with the style that the original reference used, I've added full stops to the end of each line in the bullets sections of Conditions.
I also reformatted the data under "exhaustion" to be tabular, since it's more semantically correct, and also IMO looks a bit nicer. (Though would be nicer again with some CSS to make the first column a little wider and centre the numbering in the first line.)

looks good!

I would prefer not to add CSS that is specific for some item to quickref.css, to keep the data fully decoupled from the design. You could add an inline style to your table, or just style all table headers with left alignment (i.e, just add th { text-align: left; } to quickref.css). Otherwise, looks good!
By the way, I have changed how bullet point lists look like. They look ugly on smartphones, so I have replaced them with paragraphs separated by horizontal rules. I like this new design better, but let me know if it breaks anything.

Hmm yeah. Found it looks just fine by adding a couple of empty table headers/datas, and having the header be left aligned.
And yeah, I noticed the new bullets. Looks much nicer this way.


[xxxx] Early sign off

Context
Early sign off
Changes proposed in this pull request
Off the cuff conversation
Guidance to review
https://register-pr-4878.test.teacherservices.cloud/reports/performance-profiles

Important business

[ ] Does this PR introduce any PII fields that need to be overwritten or deleted in db/scripts/sanitise.sql?
[ ] Does this PR change the database schema? If so, have you updated the config/analytics.yml file and considered whether you need to send 'import_entity' events?
[ ] Do we need to send any updates to DQT as part of the work in this PR?
[ ] Does this PR need an ADR?

NB: Please notify the #twd_data_insights team and ask for a review if new fields are being added to analytics.yml

 Early sign off

Closing as it will live in backlog given the timeline we at the moment


[Bug?] All exceptions are catched in the connection block

Hi,
It seems like all exceptions are being catched by the connection block :
Cloudflare.connect(token: cloudflare_token) do |connection|
        raise "Exception"
end

I would like my service to crash if anything fails or if the cloudflare authentication fail.
Could this be fixed ? :)

Yes, this does seem like bad behaviour.


updating vesting branch

Codecov Report

:exclamation: No coverage uploaded for pull request base (hannan@6f863a8). Click here to learn what that means.
The diff coverage is 99.21%.

@@            Coverage Diff            @@
##             hannan      #60   +/-   ##
=========================================
  Coverage          ?   91.22%
=========================================
  Files             ?       30
  Lines             ?     2427
  Branches          ?      322
=========================================
  Hits              ?     2214
  Misses            ?      178
  Partials          ?       35

Impacted Files
Coverage Δ

...tringUtilsLib/truffle/contracts/StringUtilsLib.sol
100% <ø> (ø)

TokenLib/truffle/contracts/BasicMathLib.sol
100% <ø> (ø)

LinkedListLib/truffle/contracts/LinkedListLib.sol
100% <ø> (ø)

TokenLib/truffle/contracts/TokenLib.sol
91.35% <ø> (ø)

...okenLib/truffle/contracts/TokenLibTestContract.sol
100% <ø> (ø)

...stLib/truffle/contracts/LinkedListTestContract.sol
90.9% <ø> (ø)

.../truffle/contracts/ArrayUtilsTestContractThree.sol
100% <100%> (ø)

ArrayUtilsLib/truffle/contracts/Array32Lib.sol
100% <100%> (ø)

ArrayUtilsLib/truffle/contracts/Array8Lib.sol
100% <100%> (ø)

ArrayUtilsLib/truffle/contracts/Array256Lib.sol
100% <100%> (ø)

... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6f863a8...2caeead. Read the comment docs.


Corresponds to authorization code grant flow.

Corresponded part of code grant flow.
Currently __authz endpoints can return "Code" only from cookie information. (ID - Password is not supported yet)
To obtain cookies, you need to do the following on the __token endpoint:
POST
__token
grant_type=password&username={id}&password={password}&p_cookie=true

If cookie is set, it is possible to return code from the __authz endpoint.
GET
__authz?response_type=code&redirect_uri={redirectUri}&client_id={appUrl}&state={state}

Released in v1.6.9.


impl(bigquery): V2 GetJob api - Implementation

Implementation of Custom job class, http response, rest response, and rest stub with unit tests.
fyi: Not all fields are included  in the custom classes for the V2 job proto. I have included selected fields based on use cases. Some classes QueryParameterType and QueryParameterStructType and self referential and recursive mirroring the job proto messages.

This change is

I am going to ask @dbolduc to review these changes, because I lost track of the desired state here.

build changes LGTM.


Kubernetes-reporter: Get all ConfigMaps

What this PR does / why we need it:
This PR changes tests suite kubernetes-custom-reporter ConfigMaps
report scope from Kubevirt namespaces to all namespaces.
This enable us to debug issues that  caused by ConfigMaps that
are not related to Kubevirt deployment.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
This is needed to debug #3850
Special notes for your reviewer:
Release note:

NONE

/lgtm

/approve


Feature/first steps for offline use

This pull request marks progress towards enabling our progressive web app (PWA) to operate with reduced functionality while offline. The initial step involves determining when the app can function without current data.
In particular, for the "total prices" view, once the app has retrieved and shown the current hour's electricity prices, it shouldn't refresh this information within the same hour since changes are improbable. Therefore, if a user exits and then returns to the app within the same hour, the app does not need to update this data. However, if there's a change in the hour between the user's departure and return, the app must refresh to display the latest information.

Relates to https://github.com/tvup/poweruse/issues/190


certain expressions might be wrongly parsed by the text2chem

Here if we try to parser the material optimally doped Tl2212 single crystal we obtain as formula Tl2212Tl2212, from the text2chem.

@kensei-te I'm sure I've asked you already.
The expression Tl2212 has a specific transformation in a normal formula?

Tl2212 is, Tl2Ba2CaCu2O8+delta.  Here, delta stands for some value between 0 to 1. (depending on material of solubility of oxygen, delta range can be narrower in practice)
Tc varies drastically (0-90 K) by this amount of delta, while determining delta is possible but not easy from experiment.  One needs much effort to do that. Other way is to speculate delta value by using some empirical value, but this does not work when any special case exists.
From machine learning point of view, either "Tl2212" or "Tl2Ba2CaCu2O8+delta" is not suitable as a train data UNLESS delta is clearly specified.  This can be true for any other abbreviated guys.

Thank you @kensei-te!
I try to recognise this particular way to writing by regex, to exclude them as material names.
I'm thinking the following sequence:
a) one or two characters and
b) optional space / dash
c) 3 or 4 numbers
in regex would be something like [A-Za-z]{1,2}[ -]?[0-9]{3,4}.
Such regex would match the following:
Ti-2212
Ti2212
Ti 2212
Y-123
Y123
Y 123
My question, do you think there are additional forms that could escape?


Let the user know if their !theme {themeName} failed.

The user should get a whisper if their theme command failed to switch the theme. This will help people who submit a request but spell something wrong and their theme doesn't go through.

Nice idea @John-Kryspin!  Let's discuss it on stream.
I'm going to leave this link here so I can reference it then.
https://dev.twitch.tv/docs/irc/guide/#command--message-limits
Specifically want to talk about our whisper rate limits.  We're sending whispers for !theme list but even that may hit the limit.  The whisper limits are not exactly clearly stated.
In the meantime, let's get all-contributors involved to show love for the ideas.
@all-contributors please add @John-Kryspin for ideas

Let's do this but rather than a whisper let's write the message in chat.  Maybe something like:
{twitchUsername}, {theme} is not a valid theme name or isn't installed.  You can use !theme list to get a list of available themes.

Awesome, Let me change the PR to support this idea.


add a webhook check for the agent-tls-mode setting

make sure that it can't be enabled unless the clusters have all reported they can accept the strict setting.

QA Testing
Root cause
The webhook doesn't validate changes to the new agent-tls-mode setting's value and default fields.
What was fixed, or what changes have occurred
If agent-tls-mode has default or value updated from system-store to strict, then all non-local clusters must
have a status condition AgentTlsStrictCheck set to True, unless the new setting has an overriding
annotation cattle.io/force=true.
Areas or cases that should be tested
What areas could experience regressions?
Steps

Get a Rancher instance meant to be used with this version of the webhook. Configure it to use system-store as the
value of the agent-tls-mode setting.
Provision two downstream clusters.
Note the v3.Clusters' conditions in status. AgentTlsStrictCheck should be True.
Set the condition to False or anything else for one of them.
Try changing the value of the agent-tls-mode setting to strict.
Observe an error from the webhook.
Try this again, but also set the overriding annotation cattle.io/force=true.
Ensure the webhook issues no errors, and the setting's been updated.

🧪 Test Cases

#
Priority
Description & Link
PASS/FAIL

1
P1
set agent-tls-mode setting during helm install to system-store
⏸ NOT TESTED YET

2
P1
set agent-tls-mode setting during helm install to strict
⏸ NOT TESTED YET

3
P0
Changing agent-tls-mode setting with kubectl
⏸ NOT TESTED YET

4
P0
Changing agent-tls-mode setting with the UI
⏸ NOT TESTED YET

5
P1
Adding force annotation using kubectl, change setting with kubectl
⏸ NOT TESTED YET

6
P1
Adding force annotation using kubectl, change setting with the UI
⏸ NOT TESTED YET

7
P1
Upgrade
⏸ NOT TESTED YET

🚨 8 test cases... CLICK TO EXPAND! (For table links to work) ⬅

1 / set agent-tls-mode setting during helm install to system-store Status: ⏸ NOT TESTED YET

:small_red_triangle: back to top

Test 1 details... Click to expand
Test Steps for Validation

Install Rancher via helm

Include this set:

--set agentTLSMode=system-store

Once Rancher is setup check that status of the setting via kubectl
kubectl get setting agent-tls-mode -o yaml

You should see the setting as system-store

✅ Expected Outcome
For system-store to be set
✅ Actual Outcome

2 / set agent-tls-mode setting during helm install to strict Status: ⏸ NOT TESTED YET

:small_red_triangle: back to top

Test 2 details... Click to expand
Test Steps for Validation

Install Rancher via helm

Include this set:

--set agentTLSMode=strict

Once Rancher is setup check that status of the setting via kubectl
kubectl get setting agent-tls-mode -o yaml

You should see the setting as strict

✅ Expected Outcome
For strict to be set
✅ Actual Outcome

3 / Changing agent-tls-mode setting with kubectl Status: ⏸ NOT TESTED YET

:small_red_triangle: back to top

Test 3 details... Click to expand
Test Steps for Validation

Create a rancher with --set agentTLSMode=system-store

Create a downstream cluster

Ensure this downstream cluster has AgentTlsStrictCheck set to false

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck

If it's not already set as False, you can change it to False and save the yaml
status: 'False'
type: AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using kubectl
kubectl edit setting agent-tls-mode

Change the value from system-store to strict

This should be blocked by webhook since at least 1 downstream cluster has AgentTlsStrictCheck set to 'False'

✅ Expected Outcome
For webhook to block this change to the agent-tls-mode setting
✅ Actual Outcome

4 / Changing agent-tls-mode setting with the UI Status: ⏸ NOT TESTED YET

:small_red_triangle: back to top

Test 4 details... Click to expand
Test Steps for Validation

Create a rancher with --set agentTLSMode=system-store
Create a downstream cluster
Ensure this downstream cluster has AgentTlsStrictCheck set to false

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck

If it's not already set as False, you can change it to False and save the yaml
status: 'False'
type: AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using the Rancher UI
This can be accessed from starting at dashboard/home > click Global Settings (bottom left globe icon) > Settings > click the ellipsis (three dots) at the setting agent-tls-mode > Edit Setting > change the value to Strict > Save
This should be blocked by webhook as long as at least one downstream cluster has AgentTlsStrictCheck set to false

✅ Expected Outcome
The UI action should be blocked by webhook
✅ Actual Outcome

5 / Adding force annotation using kubectl, change setting with kubectl Status: ⏸ NOT TESTED YET

:small_red_triangle: back to top

Test 5 details... Click to expand
Test Steps for Validation

Create a rancher with --set agentTLSMode=system-store

Create a downstream cluster

Ensure this downstream cluster has AgentTlsStrictCheck set to false

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck

If it's not already set as False, you can change it to False and save the yaml
status: 'False'
type: AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using the Rancher UI

This can be accessed from starting at dashboard/home > click Global Settings (bottom left globe icon) > Settings > click the ellipsis (three dots) at the setting agent-tls-mode > Edit Setting > change the value to Strict > Save

This should be blocked by webhook as long as at least one downstream cluster has AgentTlsStrictCheck set to false

Now edit the agent-tls-mode setting using kubectl > kubectl edit setting agent-tls-mode

Add this annotation under metadata:
metadata:
  annotations:
    cattle.io/force: "true"

After this annotation is added try changing the agent-tls-setting to strict using kubectl

Now even if there is a downstream cluster that has AgentTlsStrictCheck set to false the webhook will allow it now

✅ Expected Outcome
The webhook should allow the setting to be changed now, even with a downstream cluster that has AgentTlsStrictCheck set to false
✅ Actual Outcome

6 / dding force annotation using kubectl, change setting with the UI Status: ⏸ NOT TESTED YET

:small_red_triangle: back to top

Test 6 details... Click to expand
Test Steps for Validation

Create a rancher with --set agentTLSMode=system-store

Create a downstream cluster

Ensure this downstream cluster has AgentTlsStrictCheck set to false

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck

If it's not already set as False, you can change it to False and save the yaml
status: 'False'
type: AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using the Rancher UI

This can be accessed from starting at dashboard/home > click Global Settings (bottom left globe icon) > Settings > click the ellipsis (three dots) at the setting agent-tls-mode > Edit Setting > change the value to Strict > Save

This should be blocked by webhook as long as at least one downstream cluster has AgentTlsStrictCheck set to false

Now edit the agent-tls-mode setting using kubectl > kubectl edit setting agent-tls-mode

Add this annotation under metadata:
metadata:
  annotations:
    cattle.io/force: "true"

After this annotation is added try changing the agent-tls-setting to strict using the Rancher UI

Now even if there is a downstream cluster that has AgentTlsStrictCheck set to false the webhook will allow it now

✅ Expected Outcome
The webhook should allow the setting to be changed now, even with a downstream cluster that has AgentTlsStrictCheck set to false
✅ Actual Outcome

7 / Upgrade Status: ⏸ NOT TESTED YET

:small_red_triangle: back to top

Test 7 details... Click to expand
Test Steps for Validation

Start with Rancher 2.8.5
Upgrade Rancher to 2.9-head
Add this env var to the rancher deployment > CATTLE_AGENT_TLS_MODE > value: system-store
Create a downstream cluster
Ensure this downstream cluster has AgentTlsStrictCheck set to false

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck

If it's not already set as False, you can change it to False and save the yaml
status: 'False'
type: AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using the Rancher UI
This can be accessed from starting at dashboard/home > click Global Settings (bottom left globe icon) > Settings > click the ellipsis (three dots) at the setting agent-tls-mode > Edit Setting > change the value to Strict > Save
This should be blocked by webhook as long as at least one downstream cluster has AgentTlsStrictCheck set to false

✅ Expected Outcome
The UI action should be blocked by webhook
✅ Actual Outcome

✅ PASSED
Validation Environment

Component
Version / Type

Rancher version starting
v2.9-3c4ccdc5bc9fde3510089153b5ad58fdbe604880-head

Rancher version upgraded
not applicable

Rancher commit link
https://github.com/rancher/rancher/commit/3c4ccdc5bc9fde3510089153b5ad58fdbe604880

Installation option
Helm (high availability)

RKE binary version used
v1.6.0-rc8

If Helm Chart k8s cluster
v1.30.2

Cert Details
external tls aws acm

Docker version
20.10.7, build f0df350

Helm version
v2.16.8-rancher2

Downstream cluster type
Linode k3s

Downstream K8s version
v1.30.2+k3s2

Authentication providers enabled
local

Logged in user role
admin

Browser type
Google Chrome

Browser version
126.0.6478.127 (Official Build) (x86_64)

Dashboard
master 102f1e6

Webhook version
rancher/rancher-webhook:v0.5.0-rc13

🏗 Rancher Installation Details... CLICK TO EXPAND! ⬅
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.15.0/cert-manager.crds.yaml

# Add the Jetstack Helm repository
helm repo add jetstack https://charts.jetstack.io

# Update your local Helm chart repository cache
helm repo update

# Install the cert-manager Helm chart
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.15.0

helm install rancher rancher-alpha/rancher --devel \
  --namespace cattle-system \
  --set hostname=$URL_VAR \
  --set ingress.tls.source=letsEncrypt \
  --set letsEncrypt.email=$EMAIL_VAR\
  --set letsEncrypt.ingress.class=nginx \
  --set bootstrapPassword=$PW_VAR \
  --set rancherImage=rancher/rancher \
  --set rancherImageTag=v2.9-head \
  --version 2.9.0-alpha7 \
  --set global.cattle.psp.enabled=false \
  --set agentTLSMode=system-store \
  --set privateCA=true

https://letsencrypt.org/certificates/ >>> ISRG Root X1 >>> pem
kubectl -n cattle-system create secret generic tls-ca \
  --from-file=cacerts.pem=./cacerts.pem

🧪 Test Cases

#
Priority
Description & Link
PASS/FAIL

1
P1
set agent-tls-mode setting during helm install to system-store
✅ PASS

2
P1
set agent-tls-mode setting during helm install to strict
✅ PASS

3
P0
Changing agent-tls-mode setting with kubectl
✅ PASS

4
P0
Changing agent-tls-mode setting with the UI
✅ PASS

5
P1
Adding force annotation using kubectl, change setting with kubectl
✅ PASS

6
P1
Adding force annotation using kubectl, change setting with the UI
✅ PASS

7
P1
Upgrade
✅ PASS

8
P1
Set tls-mode Strict with multiple downstream AgentTlsStrictCheck=True
✅ PASS

🚨 8 test cases... CLICK TO EXPAND! (For table links to work) ⬅

1 / set agent-tls-mode setting during helm install to system-store Status: ✅ PASS

:small_red_triangle: back to top

Test 1 details... Click to expand
Test Steps for Validation

Install Rancher via helm

Include this set:

--set agentTLSMode=system-store

Once Rancher is setup check that status of the setting via kubectl
kubectl get setting agent-tls-mode -o yaml

You should see the setting as system-store

✅ Expected Outcome
For system-store to be set
✅ Actual Outcome
Value system-store was set

2 / set agent-tls-mode setting during helm install to strict Status: ⏸ NOT TESTED YET

:small_red_triangle: back to top

Test 2 details... Click to expand
Test Steps for Validation

Install Rancher via helm

Include this set:

--set agentTLSMode=strict

Once Rancher is setup check that status of the setting via kubectl
kubectl get setting agent-tls-mode -o yaml

You should see the setting as strict

✅ Expected Outcome
For strict to be set
✅ Actual Outcome
Strict was correctly set

3 / Changing agent-tls-mode setting with kubectl Status: ✅ PASS

:small_red_triangle: back to top

Test 3 details... Click to expand
Test Steps for Validation

Create a rancher with --set agentTLSMode=system-store

Create a downstream cluster

Ensure this downstream cluster has AgentTlsStrictCheck set to false

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck
status: 'False'
type: AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using kubectl
kubectl edit setting agent-tls-mode

Change the value from system-store to strict

This should be blocked by webhook since at least 1 downstream cluster has AgentTlsStrictCheck set to 'False'

✅ Expected Outcome
For webhook to block this change to the agent-tls-mode setting
✅ Actual Outcome
Getting the expected error:
error: settings.management.cattle.io "agent-tls-mode" could not be patched: admission webhook "rancher.cattle.io.settings.management.cattle.io" denied the request: value.default: Forbidden: AgentTlsStrictCheck condition of cluster c-m-c7bshql9 isn't 'True'

4 / Changing agent-tls-mode setting with the UI Status: ✅ PASS

:small_red_triangle: back to top

Test 4 details... Click to expand
Test Steps for Validation

Create a rancher with --set agentTLSMode=system-store
Create a downstream cluster
Ensure this downstream cluster has AgentTlsStrictCheck set to false

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck

If it's not already set as False, you can change it to False and save the yaml
status: 'False'
type: AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using the Rancher UI
This can be accessed from starting at dashboard/home > click Global Settings (bottom left globe icon) > Settings > click the ellipsis (three dots) at the setting agent-tls-mode > Edit Setting > change the value to Strict > Save
This should be blocked by webhook as long as at least one downstream cluster has AgentTlsStrictCheck set to false

✅ Expected Outcome
The UI action should be blocked by webhook
✅ Actual Outcome
Getting the expected error of:
admission webhook "rancher.cattle.io.settings.management.cattle.io" denied the request: value.default: Forbidden: AgentTlsStrictCheck condition of cluster c-m-c7bshql9 isn't 'True'

5 / Adding force annotation using kubectl, change setting with kubectl Status: ✅ PASS

:small_red_triangle: back to top

Test 5 details... Click to expand
Test Steps for Validation

Create a rancher with --set agentTLSMode=system-store

Create a downstream cluster

Ensure this downstream cluster has AgentTlsStrictCheck set to false

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck

If it's not already set as False, you can change it to False and save the yaml
status: 'False'
type: AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using the Rancher UI

This can be accessed from starting at dashboard/home > click Global Settings (bottom left globe icon) > Settings > click the ellipsis (three dots) at the setting agent-tls-mode > Edit Setting > change the value to Strict > Save

This should be blocked by webhook as long as at least one downstream cluster has AgentTlsStrictCheck set to false

Now edit the agent-tls-mode setting using kubectl > kubectl edit setting agent-tls-mode

Add this annotation under metadata:
metadata:
  annotations:
    cattle.io/force: "true"

After this annotation is added try changing the agent-tls-setting to strict using kubectl

Now even if there is a downstream cluster that has AgentTlsStrictCheck set to false the webhook will allow it now

✅ Expected Outcome
The webhook should allow the setting to be changed now, even with a downstream cluster that has AgentTlsStrictCheck set to false
✅ Actual Outcome
Successful edit with annotation added

6 / dding force annotation using kubectl, change setting with the UI Status: ✅ PASS

:small_red_triangle: back to top

Test 6 details... Click to expand
Test Steps for Validation

Create a rancher with --set agentTLSMode=system-store

Create a downstream cluster

Ensure this downstream cluster has AgentTlsStrictCheck set to false

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck

If it's not already set as False, you can change it to False and save the yaml
status: 'False'
type: AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using the Rancher UI

This can be accessed from starting at dashboard/home > click Global Settings (bottom left globe icon) > Settings > click the ellipsis (three dots) at the setting agent-tls-mode > Edit Setting > change the value to Strict > Save

This should be blocked by webhook as long as at least one downstream cluster has AgentTlsStrictCheck set to false

Now edit the agent-tls-mode setting using kubectl > kubectl edit setting agent-tls-mode

Add this annotation under metadata:
metadata:
  annotations:
    cattle.io/force: "true"

After this annotation is added try changing the agent-tls-setting to strict using the Rancher UI

Now even if there is a downstream cluster that has AgentTlsStrictCheck set to false the webhook will allow it now

✅ Expected Outcome
The webhook should allow the setting to be changed now, even with a downstream cluster that has AgentTlsStrictCheck set to false
✅ Actual Outcome
Successful edit with annotation added

7 / Upgrade Status: ✅ PASS

:small_red_triangle: back to top

Test 7 details... Click to expand
Test Steps for Validation

Start with Rancher 2.8.5
Upgrade Rancher to 2.9-head
Add this env var to the rancher deployment > CATTLE_AGENT_TLS_MODE > value: system-store
Create a downstream cluster
Ensure this downstream cluster has AgentTlsStrictCheck set to false

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck
status: 'False'
type: AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using the Rancher UI
This can be accessed from starting at dashboard/home > click Global Settings (bottom left globe icon) > Settings > click the ellipsis (three dots) at the setting agent-tls-mode > Edit Setting > change the value to Strict > Save
This should be blocked by webhook as long as at least one downstream cluster has AgentTlsStrictCheck set to false

✅ Expected Outcome
The UI action should be blocked by webhook
✅ Actual Outcome
Action is blocked by webhook

8 / Set tls-mode Strict with multiple downstream AgentTlsStrictCheck=True Status: ✅ PASS

:small_red_triangle: back to top

Test 8 details... Click to expand
Test Steps for Validation

Create a rancher with --set agentTLSMode=system-store

Create two downstream clusters

Ensure both downstream clusters have AgentTlsStrictCheck set to 'True'

This can be checked by going to cluster management > edit the downstream cluster as yaml > check under status at the bottom for AgentTlsStrictCheck

Now try changing the agent-tls-mode setting to strict using kubectl
kubectl edit setting agent-tls-mode

Change the value from system-store to strict

This should be allowed by webhook since both downstream cluster have AgentTlsStrictCheck set to 'True'

✅ Expected Outcome
For the change to be successful since both downstream clusters have AgentTlsStrictCheck set to 'True'
✅ Actual Outcome
The change was successful since both downstream clusters have AgentTlsStrictCheck set to 'True'


Add Latvian translations

Added Latvian translations.
Deletions that show up in files are simply added commas after Macedonian translations. I re-did this twice, thinking that it may have been something on my end, only to realize that everything seems to work fine, just difference looks weird. If anyone has suggestions how to avoid this here or in future, let me know.
This is my first contribution to open source and I picked this project because it really helped me to improve my knowledge and application of CSS flexbox.

Hey @Akirtovskis, welcome to the open source world and thanks for your contribution!
Adding a comma will display like that so no need to worry or do anything different. The translation looks good, but there are a few spots in levels.js where you added a line return in the middle of the string:

line 75
line 196
line 843

Can you delete the returns so all the text is on a single line? You make make a new commit for it, then I'll merge in your work. Let me know if you have any questions.

Looks like you missed one at line 841. Otherwise looks ready to go!

Thanks, though right now it still shows english version when I choose Latvian on the site!

Yes, that was the case! Thanks!


If card year is 0 return false for isExpired

Summary
Not sure how I missed this, but stripe sends the exp_year as 0 when it's not set (rather than null or something) so it would always be a date in the past. If it's 0, just return false.
Ticket Link
https://mattermost.atlassian.net/browse/MM-30104
Related Pull Requests
N/A
Screenshots
N/A

@stevemudie we can test this once merged on the cloud test servers.

/cherry-pick cloud

/cherry-pick cloud

Cherry pick is scheduled.


Fix: Removed jstype=string notation from resources.proto

What this PR does / why we need it:
Removes jstype=string notation from resources.proto file since javascript supports Bigint from ES2020
One of the two PRs addressing this issue, the other PR is for the js-sdk repository replacing Long with BigInt.
Which issue(s) this PR fixes:

One of the two PRs that should Fix #1023
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
No

NONE

Additional documentation:

Checklist:

[x] Added relevant tests or not required
[x] Didn't break anything

Summary by CodeRabbit

Bug Fixes

Streamlined data representation for ChangeID, Checkpoint, and TimeTicket messages by reverting certain fields to numeric types, enhancing compatibility and performance.

These changes improve the overall efficiency of data handling within the application without altering existing functionalities.

@Aswinr24 Thanks for your contribution. Could you sign the CLA?
https://github.com/yorkie-team/yorkie/blob/main/CONTRIBUTING.md#contributor-license-agreement-cla

@Aswinr24 Thanks for your contribution. Could you sign the CLA? https://github.com/yorkie-team/yorkie/blob/main/CONTRIBUTING.md#contributor-license-agreement-cla

Done !