error in sobolev_alignment_clf.krr_model_selection()

Report
Hi Soufiane, the scVI training runs no problem, but the KRR gives the following error:

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[27], line 3
      1 #KRR approximation
      2 # Select best model for KRR
----> 3 sobolev_alignment_clf.krr_model_selection(
      4     X_source=source_an,
      5     X_target=target_an,
      6     M=500,
      7     same_model_alignment_thresh=0.95
      8 )

File /mambaforge/envs/sobolev/lib/python3.9/site-packages/sobolev_alignment/sobolev_alignment.py:1018, in SobolevAlignment.krr_model_selection(self, X_source, X_target, M, same_model_alignment_thresh)
   1016 X_input = {"source": X_source.X, "target": X_target.X}
   1017 if self._fit_params["log_input"]:
-> 1018     X_input = {k: np.log10(X_input[k] + 1) for k in X_input}
   1020 # Compute sigma after re-scaling data (if required)
   1021 if self._fit_params["frob_norm_source"]:

File /mambaforge/envs/sobolev/lib/python3.9/site-packages/sobolev_alignment/sobolev_alignment.py:1018, in <dictcomp>(.0)
   1016 X_input = {"source": X_source.X, "target": X_target.X}
   1017 if self._fit_params["log_input"]:
-> 1018     X_input = {k: np.log10(X_input[k] + 1) for k in X_input}
   1020 # Compute sigma after re-scaling data (if required)
   1021 if self._fit_params["frob_norm_source"]:

File /mambaforge/envs/sobolev/lib/python3.9/site-packages/scipy/sparse/_base.py:462, in _spbase.__add__(self, other)
    460         return self.copy()
    461     # Now we would add this scalar to every element.
--> 462     raise NotImplementedError('adding a nonzero scalar to a '
    463                               'sparse array is not supported')
    464 elif issparse(other):
    465     if other.shape != self.shape:

NotImplementedError: adding a nonzero scalar to a sparse array is not supported

There are raw counts in both .X and .layers['counts'] for both source and target
Version information

-----
anndata             0.9.2
numpy               1.24.4
pandas              2.0.3
scanpy              1.9.3
scvi                1.0.3
session_info        1.0.0
sobolev_alignment   0.4.1
torch               2.0.0
tqdm                4.66.1
-----
Cython              3.0.0
PIL                 9.4.0
absl                NA
aiohttp             3.8.5
aiosignal           1.3.1
annotated_types     0.5.0
anyio               NA
arrow               1.2.3
asttokens           NA
async_timeout       4.0.3
attr                23.1.0
babel               2.12.1
backcall            0.2.0
backoff             2.2.1
brotli              1.0.9
bs4                 4.12.2
certifi             2023.07.22
cffi                1.15.1
charset_normalizer  3.2.0
chex                0.1.7
click               8.1.7
cloudpickle         2.2.1
colorama            0.4.6
comm                0.1.4
contextlib2         NA
croniter            NA
cycler              0.10.0
cython              3.0.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.8
decorator           5.1.1
deepdiff            6.3.1
defusedxml          0.7.1
docrep              0.3.2
entrypoints         0.4
executing           1.2.0
falkon              0.8.3
fastapi             0.101.1
fastjsonschema      NA
flax                0.6.1
fqdn                NA
frozenlist          1.4.0
fsspec              2023.6.0
future              0.18.3
gmpy2               2.1.2
h5py                3.9.0
hyperopt            0.2.7
idna                3.4
igraph              0.10.6
importlib_metadata  NA
importlib_resources NA
ipykernel           6.25.1
ipython_genutils    0.2.0
ipywidgets          8.1.0
isoduration         NA
jax                 0.4.14
jaxlib              0.4.14
jedi                0.19.0
jinja2              3.1.2
joblib              1.3.2
json5               NA
jsonpointer         2.0
jsonschema          4.17.3
jupyter_events      0.6.3
jupyter_server      2.7.1
jupyterlab_server   2.24.0
keopscore           2.1.2
kiwisolver          1.4.4
leidenalg           0.10.1
lightning           2.0.7
lightning_cloud     NA
lightning_fabric    2.0.7
lightning_utilities 0.9.0
llvmlite            0.40.1
markupsafe          2.1.3
matplotlib          3.7.2
ml_collections      NA
ml_dtypes           0.2.0
mpl_toolkits        NA
mpmath              1.3.0
msgpack             1.0.5
mudata              0.2.3
multidict           6.0.4
multipart           0.0.6
multipledispatch    0.6.0
natsort             8.4.0
nbformat            5.9.2
networkx            3.1
numba               0.57.1
numpyro             0.13.0
nvfuser             NA
opt_einsum          v3.3.0
optax               0.1.7
ordered_set         4.1.0
orjson              3.9.5
overrides           NA
packaging           23.1
parso               0.8.3
past                0.18.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.10.0
prometheus_client   NA
prompt_toolkit      3.0.39
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pvectorc            NA
py4j                <IP_ADDRESS>
pydantic            2.1.1
pydantic_core       2.4.0
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.16.1
pykeops             2.1.2
pyparsing           3.0.9
pyro                1.8.6+4be5c2e
pyrsistent          NA
pythonjsonlogger    NA
pytorch_lightning   2.0.7
pytz                2023.3
requests            2.31.0
rfc3339_validator   0.1.4
rfc3986_validator   0.1.1
rich                NA
scipy               1.11.2
seaborn             0.12.2
send2trash          NA
setuptools          68.1.2
six                 1.16.0
sklearn             1.3.0
sniffio             1.3.0
socks               1.7.1
soupsieve           2.3.2.post1
sparse              0.14.0
stack_data          0.6.2
starlette           0.27.0
statsmodels         0.14.0
sympy               1.12
texttable           1.6.7
threadpoolctl       3.2.0
tomli               2.0.1
toolz               0.12.0
torchaudio          2.0.0
torchmetrics        1.0.3
torchvision         0.15.0
tornado             6.3.3
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
uri_template        NA
urllib3             2.0.4
uvicorn             0.23.2
wcwidth             0.2.6
webcolors           1.13
websocket           1.6.2
websockets          11.0.3
xarray              2023.8.0
yaml                6.0.1
yarl                1.9.2
zipp                NA
zmq                 24.0.1
zoneinfo            NA
-----
IPython             8.14.0
jupyter_client      7.4.9
jupyter_core        5.3.1
jupyterlab          3.6.5
notebook            6.5.5
-----
Python 3.9.17 | packaged by conda-forge | (main, Aug 10 2023, 07:02:31) [GCC 12.3.0]
Linux-3.10.0-1160.90.1.el7.x86_64-x86_64-with-glibc2.17

Hi Joe,
Thanks a lot for raising this issue.
It seems that there is an issue with the scipy sparse matrix structure. Could you please tell me what happens if you "densify" the matrix prior to running Sobolev Alignment, e.g.,:
data_an.X = np.array(data_an.X.todense())
If it resolves the problem, I can add it as a pre-processing step within the fit procedure.
Have a great day,
Souf

Thanks Soufiane, this did get around that error. Now I have a different issue with Falkon. I have Pytorch 2.0.0 and CUDA 11.7, and I can access GPU and run scVI no problem. I have compiled Falkon with the wheel for these specific versions, which should be tested and supported but is not working. I am following up with them on it.

Thanks a lot. I will add that in the pre-processing pipeline, and in the tutorial.
For Falkon, it tooks me quite some trial-and-error to install Falkon. I gave some advice here: https://github.com/NKI-CCB/sobolev_alignment#issues-with-the-compiler
I am not sure that cuda version plays a role here, but pytorch should be at 1.11 (2.0 is very different). The compiler must also be chosen differently. I'll follow the correspondence you have with them and add it to the README.

Falkon is working now, but I came across another error further down the line:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[61], line 2
      1 #Estimate number of shared principal vectors
----> 2 n_similar_pvs = sobolev_alignment_clf.permutation_test_number_similar_pvs(n_permutations=10)

AttributeError: 'SobolevAlignment' object has no attribute 'permutation_test_number_similar_pvs'

I installed through pip, and running the command gives:
Requirement already satisfied: sobolev-alignment in /mambaforge/envs/sobolev/lib/python3.9/site-packages (1.0.0)
yet:
tail -1 /mambaforge/envs/sobolev/lib/python3.9/site-packages/sobolev_alignment/__init__.py
__version__ = "0.4.1"

I tried to force reinstall:
pip install --upgrade --no-deps --force-reinstall sobolev-alignment
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting sobolev-alignment
  Downloading sobolev_alignment-1.0.0-py3-none-any.whl (34 kB)
Installing collected packages: sobolev-alignment
  Attempting uninstall: sobolev-alignment
    Found existing installation: sobolev_alignment 1.0.0
    Uninstalling sobolev_alignment-1.0.0:
      Successfully uninstalled sobolev_alignment-1.0.0
Successfully installed sobolev-alignment-1.0.0

but still:
tail -1 /mambaforge/envs/sobolev/lib/python3.9/site-packages/sobolev_alignment/__init__.py
__version__ = "0.4.1"


Build is erroring

https://travis-ci.org/tildeio/glimmer/builds/191162501

I restarted the build to see if that solves the issue. Haven't really looked at the travis logs yet though...

Error: Command failed: npm run prepublish
npm ERR! Linux 4.8.12-040812-generic
npm ERR! argv "/home/travis/.nvm/versions/node/v6.9.4/bin/node" "/home/travis/build/tildeio/glimmer/node_modules/.bin/npm" "run" "build"
npm ERR! node v6.9.4
npm ERR! npm  v3.10.8
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>build: `../../../build/build`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>build script '../../../build/build'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the @glimmer/runtime package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     ../../../build/build
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs @glimmer/runtime
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls @glimmer/runtime

Version 2.1.5
lib/syntax/functions.ts(81,13): error TS2536: Type '0' cannot be used to index type 'T'.
lib/syntax/functions.ts(87,15): error TS2536: Type '0' cannot be used to index type 'T'.
tsc -vlib/syntax/functions.ts(81,13): error TS2536: Type '0' cannot be used to index type 'T'.
lib/syntax/functions.ts(87,15): error TS2536: Type '0' cannot be used to index type 'T'.


Use an Existing View Controller in Storyboard layout rather than XIB file for controller.

any luck on achieving this?

In ViewController.swift that was provided, call the view controllers from the storyboard, add an identifier for the view controller in the mainStoryboard
let mainStoryboard: UIStoryboard = UIStoryboard(name: "Main",bundle: nil)
let controller1 = mainStoryboard.instantiateViewControllerWithIdentifier("TrendingViewController") as! TrendingViewController
controller1.parentNavigationController = self.navigationController
controller1.title = "TRENDING"
controllerArray.append(controller1)

As @kvnbautista said, you must get the view controller from the storyboard and then append it to the controller array of the page menu.


Tabs: Item: data attributes not passed to DOM

🐛 Bug Report
We are trying to pass data-instance-id prop to our Tabs Item components. The Item components are showing "data-key" but not our custom data prop.
For example, this is what we are trying:
🤔 Expected Behavior
The Test documentation says the data props should render at attributes for React-Spectrum components.
https://react-spectrum.adobe.com/react-spectrum/testing.html
😯 Current Behavior
The custom data attribute is not present in the DOM for rendered Item components.

💁 Possible Solution

🔦 Context
We are in process of upgrading our React Spectrum components from V2 to V3 and our E2E tests are breaking.
💻 Code Sample
import React from "react";
import { Provider, defaultTheme, Tabs, TabList, Item } from "@adobe/react-spectrum";
import "./styles.css";

export default function App() {
  return (
    <Provider theme={defaultTheme}>
      <Tabs isQuiet={true}>
        <TabList>
          <Item key={'test'} data-instance-id={'test'}>
              test
          </Item>
        </TabList>
      </Tabs>
    </Provider>
  );
}

🌍 Your Environment

Software
Version(s)

react-spectrum
3.13.0

Browser
Chrome

Operating System
Mac

🧢 Your Company/Team
Adobe/Asset Essentials
🕷 Tracking Issue (optional)

Great! I'll definitely check it out

Thanks for the help! Another question, though. Do I make the changes directly in the code or do I have to do it through Storybook?

Hi @reidbarber , I try to add the code based on #2429 (comment) and #2047 but still didn't understand how I should write the test. Do you mind helping me out?


Auto select port

#104

Pull Request Test Coverage Report for Build 377

13 of 14 (92.86%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 44.211%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

sofa-ark-parent/core/common/src/main/java/com/alipay/sofa/ark/common/util/PortSelectUtils.java
13
14
92.86%

Totals

Change from base Build 375:
0.3%

Covered Lines:
1031

Relevant Lines:
2332

💛  - Coveralls


[GLUTEN-1680][CH] Fix ExpandTransform exception

What changes were proposed in this pull request?
(Fixes: #1680)
How was this patch tested?
manual test

Could you please describe more details? Thanks

Could you please describe more details? Thanks

The method of ExpandTransform::prepare() have a statement input.setNeeded() which  is incorrectly placed inside the if (!input.hasData()) {}, this may cause exception Cannot pull block from port which is not needed.

cc @liuneng1994

Run Gluten Clickhouse CI


logging with time #432

#432

As mentioned in #432, we don't want to make this the default. However, what I can do is make these binaries importable (instead of pure scripts as they are now) so that you can wrap them yourself (e.g. my_t2t_decoder.py where main does some setup/modifications and then calls t2t_decoder.main(argv)). I'll try to have that for the next release.


VSCode fails to search files (and cmd+p) inside folders stored in Google File Stream

VSCode Version: 1.34.0
OS Version: highSierra and Mojave

Steps to Reproduce:

Store any project inside a Google File Stream volume
open that folder with vscode
try to search for a file inside that folder using cmd+P (for the first time, i.e. without opening/close the file previously)

Does this issue occur when all extensions are disabled?: Yes
I also tried a few things but it didn't work:

to make the folder available offline in Google File Stream
open code --verbose to check for error messages but nothing different compared to a regular local folder shows up

Side notes (to rule out problems in Google File Steam):
I tried to open the directory with IntelliJ Idea and search works fine.
Surprisingly when I try to do the same test with Atom, the result is the same as that in vscode,
the search returns no result, despite the fact the there is a match.
Could that be a problem of a library shared with Atom or Electron?

Atom and vscode both use ripgrep so it's possible that ripgrep won't search that drive for some reason. Do you know what Google File Stream does to actually mount that drive and what might be unusual about it?
It seems like Google File Stream is only available through work/school enterprise accounts? So I'm not sure if it's possible for me to test this?

Google file stream uses fuse.
here is the mount point description:
drivefs on /Volumes/GoogleDrive (dfsfuse_DFS, local, nodev, nosuid, synchronous, mounted by dan)
Internally I believe it caches a lot of stuff as needed, such that for the point of view of other regular applications they don't 'feel' like accessing a network drive.
I wonder if a flag on fuse leaks it out and it's somehow is detected by ripgrep or other culprit.
And yes, only Academic or business account can use google file stream unfortunately.
What tests can I do for you?

Other people have reported that fuse works but is extremely slow. Does text search return anything?

I've also experience some issues with Google File Stream so I can contribute to the discussion.
Text search does return some results, but only from files that you've previously opened, i.e., if you just opened a project without opening any files, your search will return nothing.

/duplicate of https://github.com/microsoft/vscode/issues/69333


withMessage does not render i18next messages

Hi,
When I start the express app I first initialize i18next and then I set the validators in the api routes.
The problem is that when the .withMessage() is run in the validation chain the i18next hasn't manage to load the translation resources before the withMessage is being called.
So when I attach something like .withMessage(i18next.t('validation.Error')) to a validation chain, the i18next.t() can't find the key validation.Error.
Is there a way to make i18next keys been rendered inside withMessage()?
Before I was using the legacy api and it was working as expected.
I use express v4.x.x, i18next v10.x.x and express-validator v4.3.0

I think your use case is quite the same as #466; it seems like deferring withMessage() is something that needs to be done!

Hello @tioschi this is now possible express-validator v5.0.0.
Check this section of the docs for more information on how to do it.


Issue with create request

Hi I have been using your library to access and pull data from Microsoft Dynamics successfully, recently I have been trying to create records in the CRM with the library and have been getting an error as a response to the create the error body is
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"> 	<s:Body> 		<s:Fault> 			<faultcode xmlns:a="http://schemas.microsoft.com/net/2005/12/windowscommunicationfoundation/dispatcher">a:DeserializationFailed</faultcode> 			<faultstring xml:lang="en-US"> 			The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://schemas.microsoft.com/xrm/2011/Contracts/Services:entity. The InnerException message was 'Error in line 1 position 882. Expecting state 'Element'.. Encountered 'Text'  with name '', namespace ''. '.  Please see InnerException for more details. 			</faultstring> 		</s:Fault> 	</s:Body> </s:Envelope>
Here is the code used to generate the error.
orgService = new OrganizationServiceProxy(Constant.ENDPOINT, CRMLogin.getRequestInterceptor());

Entity workOrder = new Entity("x_workorder");

final String activityid = service_appointment.getValue(SAptEntry.ACTIVITY_ID);

Log.d("pushUpdatesToCRM>>>","Creating WorkOrder for Activity: "+activityid);

orgService.Create(workOrder, new Callback<UUID>() {

		@Override
		public void success(UUID uuid, Response response) {
			Log.d("pushUpdateToCRM>>>", "WorkOrder created for Activity:" + activityid);
		}

		@Override
		public void failure(RetrofitError error) {
			Log.d("pushUpdateToCRM>>>", "WorkOrder not created for Activity: " + activityid);
			Log.d("pushUpdateToCRM>>>", "WorkOrder create error: " + error.getMessage());
			Log.d("UGG", error.getBody().toString());
		}
	});

Do you have any suggestions on how to proceed? Thanks.

Okay I solve this this was a problem with the library I can submit the code if you would like.

Yes, please.

The problem was that the payload to the create request was a mixture of escaped and unescaped xml.
I added this function to The Utils Library:
public static String decodeXML(String data) {
return data
.replace("&", "&")
.replace("<", "<")
.replace(">", ">")
.replace(""", """)
.replace("'", "'");
}
and altered the create statement in Organization Service Proxy to read:
content.append(Utils.decode(Utils.objectToXml(entity, "d:entity", false)));
I am sure that you are probably more familiar with the library and could fix the objectToXML for this case better. But that got it posting

With the huge new merge of changes this was fixed.


1st parameter 0x12 missing

While debugging the blue screen 0x13A I came across the first parameter 0x12 which is not described on this page. I am asking for a description of this parameter

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 25edc27f-390e-725f-7ee7-638a3cbeb5bf
Version Independent ID: baaa985d-e78a-ff2d-5fb8-ea9e63358eff
Content: Bug Check 0x13A KERNEL_MODE_HEAP_CORRUPTION - Windows drivers
Content Source: windows-driver-docs-pr/debugger/bug-check-0x13a--kernel-mode-heap-corruption.md
Product: windows-hardware
Technology: windows-devices
GitHub Login: @DOMARS
Microsoft Alias: domars

Ist parameter 0x12. All I can tell is that dxgmm2.sys faulted. No idea why.

@MrPepka - The parameter list has been updated to include the more recent parameter ones such as 0x12. Thanks for this issue, and your interest in improving the stop code documentation.


Class CrystalSlice is defined twice

CrystalSlice is defined here and here.
I can see an argument for both elements and rscrystal. Maybe the tools for updating the crystal, such as the updates after slices, implementing Franz-Nodvik, etc should go in rscrystal, whereas the final propagation algorithm should go in elements?

Paul and I were thinking keeping all the crystal slice stuff in one place is better.

When this issue is closed, the elements/ directory should be deleted from the repository.

The relationship between Crystal() and CrystalSlice() is unclear.
Please create a separate issue for this and assign to you, me and Boaz.
Add the constructor item 4 from above to the new issue.


Add kubeseal and inletsctl to get command

Signed-off-by: Tuan Anh Tran<EMAIL_ADDRESS>Add kubeseal and inletsctl to arkade get command
Description
Add kubeseal and inletsctl to arkade get command
Motivation and Context
Add kubeseal and inletsctl to arkade get command

[ ] I have raised an issue to propose this change (required)

How Has This Been Tested?
Tested on macOS

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I've read the CONTRIBUTION guide
[x] I have signed-off my commits with git commit -s
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

[ ] I have tested this on arm, or have added code to prevent deployment

@alexellis updated as per comment
inletsctl needs to use URLTemplate and BinaryTemplate as well.
eg: url https://github.com/inlets/inletsctl/releases/download/0.5.4/inletsctl-darwin.tgz
extracted file name inletsctl-darwin | inletsctl | inletsctl.exe

Sorry @tuananh I just merged another PR and this needs another rebase
Alex

@alexellis rebased.


Creating StaticRouter.sol

Hello @ylv-io 👋,
Thank you for your presentation at DevCon. I found it very helpful.
I had a quick question regarding the creation of StaticRouter.sol. I believe from reading the synthetic v3 hardhat-router util that this file is created at deployment. Is this correct?
I put the needed files from this repo into a contracts folder, modified hardhat configure it to include @synthetixio/hardhat-router and deploy. When I attempt to do this I get an error from the plugin looking for a Proxy.sol contract.
I created a sample repo where I tried to replicate the creation for your StaticRouter.sol contract and document my current approach and error.
https://github.com/ssolson/StaticRouterTest.git
Any pointers on how to create the StaticRouter.sol would be greatly appreciated. 🙏

I think I got it and have updated my example repo.


I want to get property layerX and layerY from PointerEvent in lib.dom.d.ts

Suggestion

🔍 Search Terms

List of keywords you searched for before creating this issue. Write them down here so that others can find this suggestion more easily and help provide feedback.
✅ Viability Checklist

My suggestion meets these guidelines:

[ ] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[ ] This wouldn't change the runtime behavior of existing JavaScript code
[ ] This could be implemented without emitting different JS based on the types of the expressions
[ ] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
[ ] This feature would agree with the rest of TypeScript's Design Goals.

⭐ Suggestion

📃 Motivating Example

💻 Use Cases
What do you want to use this for?
Resolve click event better
What shortcomings exist with current approaches?
Without layerX and layerY declaration in lib.dom.d.ts
What workarounds are you using in the meantime?
Declare e as an any variable or add  / / ts ingore above it

To expedite the triage process, we need everyone to follow the issue template and instructions.
When you clicked "Create New Issue", the issue form was pre-populated with a template and some instructions. We need you to read those instructions completely, follow them, and then fill in all the fields in that template.
We are not able to assist with issues that don't follow the template instructions as they represent a significantly larger amount of work compared to issues which are correctly specified. Thank you for understanding.


Reworded error message

This PR changes an error message which was flagged here: https://security.lfx.linuxfoundation.org/#/a092M00001If9uZQAR/non-inclusive-language

Codecov Report

Merging #1482 (3ac3da9) into main (b6fbca2) will increase coverage by 0.08%.
The diff coverage is 0.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1482      +/-   ##
==========================================
+ Coverage   78.89%   78.97%   +0.08%
==========================================
  Files         200      200
  Lines       21724    21779      +55
  Branches     4403     4406       +3
==========================================
+ Hits        17139    17200      +61
+ Misses       2355     2350       -5
+ Partials     2230     2229       -1

Flag
Coverage Δ

py-unittests
78.97% <0.00%> (+0.08%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...entimelineio-bindings/otio_serializableObjects.cpp
26.59% <0.00%> (ø)

tests/test_cmx_3600_adapter.py
99.59% <0.00%> (+0.02%)
:arrow_up:

src/opentimelineio/serialization.cpp
47.99% <0.00%> (+0.18%)
:arrow_up:

...opentimelineio/opentimelineio/adapters/cmx_3600.py
85.06% <0.00%> (+0.28%)
:arrow_up:

tests/test_plugin_detection.py
91.46% <0.00%> (+0.67%)
:arrow_up:

...-opentimelineio/opentimelineio/plugins/manifest.py
91.50% <0.00%> (+4.36%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6fbca2...3ac3da9. Read the comment docs.


Only works on one-to-one of which was to be many-to-many, webrtc

I am developing a conference style application (many-to-many) for video calls this style: http://spaceify.net/talk/. He even is available on GitHub but I can not for my nodejs to work, which is not much novelty, hence I decided to create my own using PHP that never leaves me.
I created the server using WebSockets, is simple he receives the message and forwards it to all connected .. Just that, nothing more nothing less.
But my problem is that it does not connect with more than one person, to connect with the third person other streams fail. You can only connect one-to-one.
I do not know consider whether the error is in JavaScript or some need to improve server .. What can I do to make it connect to all who come?
See my code:
var socket = new WebSocket('ws://<IP_ADDRESS>:1805/');
var stunServer = "stun.l.google.com:19302";
var localStream = null;
var peerConn = null;
var started = null;
var mediaConstraints = {
    'mandatory': {
        'OfferToReceiveAudio': true,
        'OfferToReceiveVideo': true
    }
};

function createPeerConnection() {
    var servers = [];
    servers.push({
        'url': 'stun:' + stunServer
    });
    var pc_config = {
        'iceServers': servers
    };
    peerConn = new webkitRTCPeerConnection(pc_config);
    peerConn.onicecandidate = function(event) {
        if (event.candidate) {
            sendMessage({
                type: 'candidate',
                label: event.candidate.sdpMLineIndex,
                id: event.candidate.sdpMid,
                candidate: event.candidate.candidate
            });
        }
    };
    peerConn.onaddstream = function(event) {
        console.log("Added remote stream");
        $("body").append('<video width="320" height="240" autoplay="true" src="' + window.URL.createObjectURL(event.stream) + '"></video>');
    };
    peerConn.onremovestream = function(event) {
        console.log("Remove remote stream");
    };
}

function sendMessage(message) {
    var mymsg = JSON.stringify(message);
    console.log("SEND: " + mymsg);
    socket.send(mymsg);
}

function connect() {
    if (localStream) {
        console.log("Creating PeerConnection.");
        createPeerConnection();
        console.log('Adding local stream...');
        peerConn.addStream(localStream);
        started = true;
        peerConn.createOffer(setLocalAndSendMessage, null, mediaConstraints);
    }
}

socket.addEventListener("message", function(evt) {
    console.log("RECEIVED: " + evt.data);
    processSignalingMessage(evt.data);
}, false);

function processSignalingMessage(message) {
    var msg = JSON.parse(message);
    if (msg.type === 'offer' && localStream && !started) {
        started = true;
        createPeerConnection();
        console.log('Adding local stream...');
        peerConn.addStream(localStream);
        peerConn.setRemoteDescription(new RTCSessionDescription(msg));
        console.log("Sending answer to peer.");
        peerConn.createAnswer(setLocalAndSendMessage, null, mediaConstraints);
    } else if (msg.type === 'answer') {
        peerConn.setRemoteDescription(new RTCSessionDescription(msg));
    } else if (msg.type === 'candidate') {
        var candidate = new RTCIceCandidate({
            sdpMLineIndex: msg.label,
            candidate: msg.candidate
        });
        peerConn.addIceCandidate(candidate);
    }
}

function setLocalAndSendMessage(sessionDescription) {
    peerConn.setLocalDescription(sessionDescription);
    sendMessage(sessionDescription);
}

function startVideo() {
    try {
        navigator.webkitGetUserMedia({
            audio: true,
            video: true
        }, successCallback, errorCallback);
    } catch (e) {
        navigator.webkitGetUserMedia("video,audio", successCallback, errorCallback);
    }

    function successCallback(stream) {
        $("body").append('<video width="320" height="240" autoplay="true" src="' + window.URL.createObjectURL(stream) + '"></video>');
        localStream = stream;
    }

    function errorCallback(error) {
        console.log('An error occurred: [CODE ' + error.code + ']');
    }
}

full code here: http://pastebin.com/3iWDLU9Y

have you considered using a library to do this such as this one?
Currently, you're creating a single peerconnection object. But you need one peerconnection per pair of people that will get connected.

@fippo The library is very large, I would like something smaller and lighter.
You refer to this function of the code that must be recreated for each new connection?
createPeerConnection

it's large because it solves a bunch of problems... like working in Firefox which your code will not.
Anyway: yes, you might want to check http://webrtc.github.io/samples/src/content/peerconnection/multiple too

@fippo It is only a prototype, calm ... very good example I based myself in it and got something significant: I can connect one-to-one but when I connect the third person, the third person connects only with the first.
The first connects with the second and third.
The second connecting with the first.
The third connecting with the first.
I'm lost in solving this impasse ..

Adding a little context to this conversation, this is the StackOverflow post where I suggested to this user that SimpleWebRTC be used. http://stackoverflow.com/questions/31479356/
I'm closing this issue. The poster is asking for help restructuring basic PeerConnection code to support multiple user connections, which is exactly what this library solves, but then also says they don't want to use a library for "personal reasons."
The answer is: a lot of work has gone into development of this library to solve the exact problem OP is asking us to solve (among others), and OP is welcome to take code or inspiration from this library (per the license).

I do not want to have to use such an extensive library of which I try to ride my own code. If not able to help, even though the solution I thank you for having closed this topic.

@xdumaine  I do not want to have to use such an extensive library of which I try to ride my own code. If not able to help, even though the solution to my problem, I thank you for having closed this topic.I seek help elsewhere, hugs.

@thebestclass: FWIW, you seem to be greatly underestimating the amount of time and effort it takes to write a webrtc library. This library is now more than two and a half years old and far from done.

@fippo  Nonsense, of course I'm not. But I think not necessary or useful to use something outsourced, and if you ever need maintenance? I will have to wait update you guys? I hope you understand.


entities macro breaks when subscriptions are moved to separate namespace

I tried creating a separate module for all my subscription definitions, and all of the subscriptions broke that were using the entities macro. The compiler spit out some undeclared variable warnings, but other than that, there was nothing useful for me to debug it. Replacing usages of entities with entity at least got rid of those warnings, but it took moving everything back into the same module for the usages entities to work again.
The subscriptions are nothing fancy either, so I have no idea what the issue was. Here's an example:
(defsub :notes
  [?eids <- (acc/by-fact-id :e) :from [:note/creation-date]]
  [(<- ?notes (entities ?eids))]
  =>
  {:notes ?notes})

Really sorry about that! The entities macro was hard to implement. It does some weird stuff, namely splitting the rule it occurs in into two rules. That said there were some var resolution issues having to do with CLJS namespace resolution issues in Clara that were fixed as of 0.18.0 (I think). #111 and #112 talk about very similar issues if not the same you have. Looks like I may have fixed some of it in the latest master but the yet unmerged PR #116 that upgrades Clara to 0.18.0 may fix it fully.
I hope this helps until I can take a closer look.

Okay, good to know. I'll just keep things in the same module for now.
Is there anything blocking #116, or does it just need QA?

Just ran the todomvc example with it. More QA would be good (especially with rules split across namespaces). Nothing's blocking it other than nervousness on my part.


Filtering on DateTime in DataGrid results in crash

Describe the bug
Filtering on DateTime in DataGrid results in crash: Microsoft.Data.SqlClient.SqlException (0x80131904): Conversion failed when converting date and/or time from character string. (full stack trace below).
FormatString works fine for formatting column values, but the filtered date is formatted incorrectly and an error occurs when building the EF query. In both these cases it seems FormatString and FilterDateFormat are ignored and the date format of the environment/system is used instead.
I think the issues are related to this: when displaying the filtered date and building the EF query, column.GetFilterValue() is used. It returns the DateTime value as an object, but is used like a string, so it's just casted and no explicit formatting occurs.
I guess this isn't an issue in US, but that the DateTime format of my region (Oslo/Norway) is unexpected by Radzen and/or EF.
Full stack trace:
Error: Microsoft.Data.SqlClient.SqlException (0x80131904): Conversion failed when converting date and/or time from character string.
   at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at Microsoft.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows)
   at Microsoft.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more)
   at Microsoft.Data.SqlClient.SqlDataReader.Read()
   at Microsoft.EntityFrameworkCore.Storage.RelationalDataReader.Read()
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.MoveNext()
   at System.Linq.Enumerable.Single[TSource](IEnumerable`1 source)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at System.Linq.Queryable.Count[TSource](IQueryable`1 source)
   at Radzen.Blazor.RadzenDataGrid`1.get_View()
   at Radzen.PagedDataBoundComponent`1.get_PagedView()
   at Radzen.Blazor.RadzenDataGrid`1.DrawGroupOrDataRows(RenderTreeBuilder builder, IList`1 visibleColumns)
   at Radzen.Blazor.RadzenDataGrid`1.<>c__DisplayClass6_0.<DrawRows>b__0(RenderTreeBuilder builder)
   at Microsoft.AspNetCore.Components.Rendering.RenderTreeBuilder.AddContent(Int32 sequence, RenderFragment fragment)
   at Radzen.Blazor.RadzenDataGrid`1.BuildRenderTree(RenderTreeBuilder __builder)
   at Microsoft.AspNetCore.Components.Rendering.ComponentState.RenderIntoBatch(RenderBatchBuilder batchBuilder, RenderFragment renderFragment)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.RenderInExistingBatch(RenderQueueEntry renderQueueEntry)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()
ClientConnectionId:64afdacd-03b8-4e8b-9607-0041a76485d1
Error Number:241,State:1,Class:16

To Reproduce
Steps to reproduce the behavior:

Be in Oslo/Norway.
Make a DataGrid with a DateTime column and filtering (only tested SimpleMode).
Try to format the DateTime with FormatString and FilterDateFormat.
Run the app and observe that the DateTime column values are formatted according to FormatString.
Try to filter on a date and observe that the filtered date is not formatted according to FormatString or FilterDateFormat.
Click Apply and experience a crash.

Expected behavior
The filtered date should be displayed according to FormatString or FilterDateFormat, and the DataGrid should update according to the DateTime filter without crashing, like it does for other columns.
Screenshots
None right now.
Desktop (please complete the following information):

OS: Windows 10
Browser: Chrome
Version: 95.0.4638.69

Additional context

.NET 5
Radzen 3.12.2 (I tried many different versions, same issue)

I tested this on another computer and was not able to reproduce the crash. The system still controls the DateTime format, but it causes no issues with building the EF query.
On the computer where I experience the crash, the DB is in the cloud. The issue might be caused by a mismatch between the DateTime format of the server and my computer.

I found the issue, it was on my side. I used "datetime" in SQL table, changing it to "datetime2" solved the issue.


Output number of satellite in view

Hi, is there a command or function to output the number of satellite seen by the GNSS?

Hi @Buttosaii,
This is the sort of thing you could Google yourself (e.g. search on "NMEA number of satellites seen"), but in short the answer is yes, you can use the numSV attribute in NMEA messages like GGA, GNS or GSV, e.g.,
from serial import Serial
from pynmeagps import NMEAReader
stream = Serial('/dev/ttyACM0, 38400, timeout=3)
nmr = NMEAReader(stream)
(raw_data, parsed_data) = nmr.read()
if parsed_data.identity in ("GGA", "GNS", "GSV"):
   print (f"Number of satelllites tracked:  {parsed_data.numSV}")

Hope this helps.
FYI the u-blox support forum is also a useful resource for technical queries though, as I say, this sort of information is in the public domain.


[feat] allow to render error pages from handleError

Fixes #2824 by optionally rendering error pages if renderError is returned from the handleError hook. This solution is better than #3171 because the right status code is returned and the user can reload the page in order to try again.
Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

[x] It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
[x] This message body should clearly illustrate what problems it solves.
[x] Ideally, include a test that fails without this PR but passes with it.

Tests

[x] Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

[x] If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpx changeset and following the prompts. All changesets should be patch until SvelteKit 1.0

Thank you. Adding new API is unnecessary though — all we need to do is this...
// pseudo-code
try {
  return handle();
} catch (e) {
  handle_error(e);
  return render_error_page();
}

...and wrap all the error page rendering logic in a try-catch (previously, some of it was outside the try-catch) so that that's guaranteed not to throw. I've opened #3239, so I'll close this and #3171


Wow, Seriosly?  I'm Sitting Here Creating my Function.json file.  Yep.  Never did one before, but now I'll just go ahead and do that.  LOL.  Wooo.  Hilarious.

This is a late night joke, right?  I have no clue how to create my very own json function file.  LOL.
The instructions state: "In version 2.x of the Azure Functions runtime, you have to explicitly register the binding extensions (binding types) that you use in your function app."   ...Hmmmm, help me out here.  What does this mean?  How do I do it?
"When you develop functions locally, you can install the extensions you need by using the Azure Functions Core Tools from the Terminal or from a command prompt.  Ah Ha, I see.  Yes, yes... But wait!! How does one do this???
Ooh, I see...here we go.  The info to help me do this:
"After you have updated your function.json file to include all the bindings that your function needs, run the following command in the project folder."  ... ! Wait! It's assumed that I was able to create the file I need by being told I need to create the file.  Hahahahahaha!!  Nope!  I don't know how.  Still don't know....  There is a missing piece to this puzzle, right?
The part that's missing is There is no instruction regarding how to do this.  Seriously???  Yup.  OK.  I'll go do that.  I'll create my function.  yup.  going to create my very own function.json now.  mmmmhhhhm.  yup.  LOL. WOW.  What was I thinking?
Ooops, looks like i guessed poorly....This is hilarious.  Seriously.  Come on man.  Tell me how to create a function.json file.  I'm new to this... need some hand-holding....   Hahahahahaha HAH!  Wooooo,
func : Unable to find project root. Expecting to find one of host.json, local.settings.json in project root.
At line:1 char:1
+ func extensions install
+ ~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Unable to find ...n project root.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b2646ad4-34b7-fc5e-dcc9-d53a061df394
Version Independent ID: bbff4fda-24e4-2b62-24b9-94cda5f53bed
Content: Work with Azure Functions Core Tools
Content Source: articles/azure-functions/functions-run-local.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@dhmorrison Thanks for your feedback. Could you clarify which documentation you're referring to? Please provide us with the URL of the documentation so that we ensure your feedback is addressed by the right team.

@dhmorrison Based on the error message that you are seeing, I believe you are running the command from the wrong folder.
After running func init, you should have the host.json and local.settings.json files generated and running func new should have a function.json file generated as well (except in the case of C#/Java Functions). The CLI Quickstart is a good doc to start with.
You can read more about Triggers and Bindings in the docs. And for specifics about modifying the function.json file when working with different triggers/bindings, refer to the reference docs for them which can be found under Reference > Bindings in the functions docs.

I hope that clears some things up.
That being said, I understand the confusion created because of the sequence of sections in this doc. I believe a better flow would be to have Register extensions after the Create a function section instead.

@dhmorrison sorry about the confusion. We are planning to add a better overview to the Core Tools article to clarify the general steps to create and publish function apps.

@PramodValavala-MSFT please assign this issue to me.

This is really DUMB! How come before the VS 2017 cloud wizard generated the localsettings.json file? Now I have to jump through hoops and figure out how to manually generate a localsettings.json file that has the right entries for the storage locations? I feel like we have gone backwards here folks!

Another note, the documentation SUCKS and is written with a ton of assumptions for a brand new construct that is pretty poor design.

It's all good.  I know it can be challenging to document as you create cool new components.  I'm no angel when it comes to that either.  I suspect you are targeting your documentation toward the developer community whose interest is more than academic.  IMHO, it makes sense to put a premium on solid code examples that go a bit deeper than writing a function that returns... Yo!  My name is Lil Deep.  For me anyhow in depth real world samples are worth their weight in gold.

@dhmorrison  your suggestion is very correct and we are in the process of reworking the Functions quickstarts to add Storage bindings to be more realistic.
@ScottyMac52 if you are creating C# Functions in.Visual Studio, we have an article for working with Visual Studio tools. I think it’s easier to use VS than Core Tools for C# Functions.

OK, good news on this thread. There is now a new extension bundles feature that makes it much easier to install the binding extensions needed by your project.
If there are still things in this article that you think we need to address, please re-open this issue.
#please-close


Clearer messaging to user when they sign out from their phone (uProxy related)

The WeChat login flow involves scanning a qr-code from your phone - the user can then revoke this login from the same phone app, which will silently disable signaling messages in uProxy. This isn't buggy behavior, but could be clearer (basically uProxy should realize that it has been signed out remotely).

Deprioritizing, the user should hopefully know that they signed out and understand why it breaks uProxy.


User Search ComboBox, long text causes popup to immediately close when clicked

Provide a general summary of the issue here
In Chrome/Chromium, any overflowing text typed into the input box causes the popout to fail to display
🤔 Expected Behavior?
the popout should be displayed
😯 Current Behavior
the popout briefly shows, but immediately disappears
💁 Possible Solution
This bug only occurs in Chrome/Chromium (we discovered it Electron) but does not occur in Firefox and Safari (tested on MacOS) ... so probably helps narrow-down the source of the problem?
🔦 Context
This is a critical bug that blocks a release as the popout list of choices is not displayed
🖥 Steps to Reproduce
https://react-spectrum.adobe.com/react-aria/examples/user-combobox.html

type any arbitrary long text that reaches the end of the input field, just enough to cause the entered text to "scroll" at least one character
click on the "arrows" icon button to attempt to reveal the popout list of choices

Version
latest in official website
What browsers are you seeing the problem on?
Chrome
If other, please specify.
No response
What operating system are you using?
MacOS
🧢 Your Company/Team
No response
🕷 Tracking Issue
No response

Might be related to https://github.com/adobe/react-spectrum/issues/5653#issuecomment-1922662612 and https://github.com/adobe/react-spectrum/issues/5653 in general.

Thank you!
I'm able to reproduce with the example and steps you shared.
Closing as a duplicate.


DIV-8008, DIV-8018: Creating installation scripts.

@himeshr-egov and @egov-joy I am adding the DIVOC installer into our repo itself. Please review the readme as well.


add python3.6 compatibility

Tested the happy path only for 3.6 compatibility.
It's a very cheap change, otherwise I wouldn't even bother.

Confirmed forwards compatible at least to 3.7.3


⚠ Bofrost.lu has degraded performance

In 741b510, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 992 ms

Resolved: Bofrost.lu performance has improved in 7235257 after 20 minutes.


Global context

In the case of using nestjs-pino with microservices, I had to filter logs in elasticsearch by microserviceName and I thought that would be great to have the option to set variable global once and then use it automatically with all logger actions.
I also did that set global context had a higher priority to data and context in the case of namespacing conflicts. E.g. once set global context parameters can not be rewritten by others.

Hi, @ly4she, thanks for PR!
Just asking, have you tried using pino options, maybe name, mixin, base  can do what you want? If not, just want to get more clear understanding of adding this feature

Hello, @iamolegga, thanks for reply.
I checked pino options again. Yes, I totally missed out  base property from native pino options.
Sorry for disturbing you. Thanks for nestjs-pino lib.

Glad to help! 😊


Feat: integration of ngx-auth-firebaseui (collaboration )

Recently, I developed ngx-auth-firebaseui, Angular Material UI component for firebase authentication.

here is the demo app https://ngx-auth-firebaseui.firebaseapp.com/
on medium
features - why-to-use-ngx-auth-firebaseui: readme
that would be awesome 👍

tell me what do you think about this library..

@AnthonyNahas I will take it into consideration!


Add more devices, but only if they really exist

This will bring /dev/kvm, /dev/ttyUSB*, and a bunch of other niceties in the container, but conditionally. (Because if we try to specify non-existent devices, Docker will boo at us.)

what were you trying to do i wonder hahaha but coooool!


Bootstrap docusaurus docs

This is the continuation of https://github.com/davidmigloz/langchain_dart/pull/517 by @bettdouglas

Available in https://beta.langchaindart.dev/


Add layout=narrow initial param to the ModalStackNavigator

Details

Breaking down https://github.com/Expensify/App/pull/31476 into several PRs as explained here
Fixed Issues

$ https://github.com/Expensify/App/issues/30528
Tests

N/A

[x] Verify that no errors appear in the JS console

Offline tests

N/A
QA Steps

N/A

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

N/A

Android: mWeb Chrome

N/A

iOS: Native

N/A

iOS: mWeb Safari

N/A

MacOS: Chrome / Safari

N/A

MacOS: Desktop

N/A

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If the PR modifies the form input styles:

[ ] I verified that all the inputs inside a form are aligned with each other.
[ ] I added Design label so the design team can review the changes.

[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

@rayane-djouah  Noticed a strange bug.

Go to search page
Remove ?isInRHP=true from the url and enter
then click on the back icon

https://github.com/Expensify/App/assets/75031127/b50a8a98-710f-4170-b1f4-e9d34546632f

I think we might need to rename this since Search ended up being moved to a new "left-hand-pane".
So I think instead of isInRHP, the URL params can just be layout=narrow or something.

Also, I've looked into the problem you've described a bit, but I don't think we should worry about it at this time.
The reason I say that is because my impression for the time being is that react-navigation is not in the business of re-writing browser history (or more accurately, coercing the browser to use its version of the history rather than the native browser history).
There's additional context in these comments. I think that to solve the problem, we would ultimately need react-navigation to enforce a different version of the browser history, which is something they don't want to do for now.
It's not a huge deal either – I think we can agree for now that a user manually removing URL params is an edge case we don't need to worry about and which is unlikely to affect any real users.

Just a Q: @roryabraham  layout=narrow will always be added regardless of the screen size, i wonder if this will create a confusion, probably we could add it conditionally?

Hmmm I don't think it's really a problem, so let's just move forward

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

This was causing a deplooy blocker so to unblock the deploy we went with a straight revert https://github.com/Expensify/App/pull/34552/files

🚀 Deployed to production by https://github.com/mountiny in version: 1.4.25-10 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

This also caused another deploy blocker which we were able to fix without reverting this. There was some code looking for empty route params which was broken by this. We fixed it by looking for the presence of a specific route param rather than that they were empty.


cmd/scollector: Send a hi metric with a value of 1

This can be used to create an alert for scollector being alive and then used as a dependency.

Code looks good. Not sure what your plan is, but you might get better results using scollector.collect.queued instead. It should have the same frequency as this metric (every 15s by default), but if you get a high count you know there is more data waiting to be sent. Basically scollector.collect.queued should function exactly the same as this in the average case (only set to 0 instead of 1) and in cases where there was an outage it will actually give a better signal indicating how many data points are waiting to be sent. Also alerts on unknown values are tricky since you might just be seeing old datapoints that are being burned down in the queue (i think we are FIFO not LIFO)
Now that I think about it something like a scollector.collect.queued_since that is the seconds since the latest timestamp of the last batch of sent metric might work best, since that could easily be used in the alert logic to determine if you should go crit or just warn that scollector is back online and burning down the queue. Not sure how long the burndown time is on average, but I know if things get backed up significantly it can take a bit to return to 0.

The timestamp of the value 1 should be when the collector created it. The thing I like about the way I did it is it is very straightforward in what it does. Using scollector.collect.queued as an additional (or additional logic) check makes sense to me though, as does adding the scollector.collect.queued_since metric (not sure entirely what you mean there though as to which timestamp).
My plan is this can be a dependency, and if the host can be pinged but this isn't there, it isn't host down but rather (scollector down, not current, etc).


[BUG] STS GetCallerIdentity called many times with duplicate information to detect account ID

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Whenever we scrape any discovery, static, or custom namespace job, the GetCallerIdentity API is called to detect the account ID of the scrape job's credentials.
For my 13 services across >20 roles, this results in hundreds of calls to the GetCallerIdentity API. Most of these calls are duplicates of each other with the same credentials, because GetCallerIdentity happens at the for every job, for every role, for every region:

2 nlb discovery jobs for the same role in different regions
elb and nlb discovery jobs for the same role in the same region
A discovery job and a static job for the same role in the same region

This STS API call happens in multiple goroutines, outside the semaphores used to guard concurrent API access, so hundreds of API calls are happening at almost the exact same time. In practice this is not too much of an issue because they are spread across multiple accounts, but every few minutes yace still causes a flood of API calls.
Expected Behavior
When a role ARN is specified, yace should probably not call GetCallerIdentity at all. It is only used to fetch the account ID, and there is no need to perform API calls to determine the account for a role. The role ARN includes the account ID, and arn.Parse(roleArn) can be used to extract this information.
When a role ARN is not specified, yace should call GetCallerIdentity once on startup and cache the account ID. Or at worst, yace should call GetCallerIdentity once per scrape (i.e. once every few minutes) if there is a concern that AWS credentials on disk may be changed to point at a different AWS account while yace is running.
Steps To Reproduce

Configure yace to scrape a many services/regions for many roles
Observe the number of GetCallerIdentity API calls

Anything else?
I'd be happy to contribute this change, let me know if you would prefer a single GetCallerIdentity on startup to detect the current account, one per scrape, or something else.

This same behaviour seems to also affect the IAM ListAccountAliases API call in the latest version.
It seems like the work in #1432 will address this once it's wired in, since the account ID and alias are looked up once at the start of the scrape.


Both docs and behavior for webpack.LoaderOptionsPlugin are wrong

bug
What is the current behavior?
webpack.LoaderOptionsPlugin doesn't pass options to sass-loader
While in need for customised sass loadpath and separate css fiile output I've spent some time configuring bleeding edge Webpack 2 with bleeding edge sass-loader (:
Good news is that old-school approach works:
If the current behavior is a bug, please provide the steps to reproduce.
// this passing nothing
new webpack.LoaderOptionsPlugin({
      sassLoader: {
        includePaths: [
          path.resolve('./node_modules')
        ]
      }
    })

// this passing undefined
new webpack.LoaderOptionsPlugin({
  options: {
    sassLoader: {
      includePaths: [
        path.resolve('./node_modules')
      ]
    }
  }
})

What is the expected behavior?
sass-loader to use  value of includePaths:
{
        test: /\.(sass|scss)$/,
        loader: ExtractTextPlugin.extract({
          fallbackLoader: 'style-loader',
          loader: [
            'css-loader',
            'resolve-url-loader',
            {
              loader: 'sass-loader',
              query: {
                sourceMap: true,
                includePaths: [path.resolve('./node_modules')]
              }
            }
          ]
        }),
      }

Webpack 2.2.0-rc.3

Have you tried just this?
new webpack.LoaderOptionsPlugin({
  options: {
      includePaths: [
        path.resolve('./node_modules')
      ]
  }
})

It should set includePaths option for all loaders and you make this behaviour specific only to sass loader by defining test in addition to options.
Not 100% sure all will work though, I've got a somewhat similar issue ATM.

Have you tried just this?
new webpack.LoaderOptionsPlugin({
  options: {
      includePaths: [
        path.resolve('./node_modules')
      ]
  }
})

It should set includePaths option for all loaders and you make this behaviour specific only to sass loader by defining test in addition to options.
Not 100% sure all will work though, I've got a somewhat similar issue ATM.

@kachkaev , thanks Alex, I have another workaround in place, opened an issue coz LoaderOptionsPlugin doesn't work as expected or as error message suggests

@kachkaev , thanks Alex, I have another workaround in place, opened an issue coz LoaderOptionsPlugin doesn't work as expected or as error message suggests

The proper workaround to get sass-loader working with Webpack2 is to put BOTH context and path in the options section
https://github.com/bholloway/resolve-url-loader/issues/33#issuecomment-249830601

Yes, after debugging breakpoint inside LoaderOptionsPlugin, finally find out the current behavior(webpack2 version) will wipe out the original context's options, then, ** it breaks down a bunch of loaders** acquiring some data from the original options property of loaderContext.

@sokra because context object already has a property named "options", if user defines a options property for LoaderOptionsPlugin, it will override the original options object inside the context object.
One simple way to solve this is to use Object.assign that simply combine object property of context.
But it may let the user get the chance to override other context properties.
Therefore, another way is to add a new property like loaderOptions or other reasonable name. It holds the options passed from LoaderOptionsPlugin, isolating the custom loader options from context object.

@kachkaev i am not understanding the docs on LoaderOptionsPlugin
is the plugin provided in webpack_config.plugins or loader.options.plugins for example
This might be trivial to you but i am not able to interpret in the correct way
I tried these two configs of ExtractTextPlugin in css-loader, sass-loader but the same error comes

Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema.
 - configuration has an unknown property 'postcss'. These properties are valid:
   object { amd?, bail?, cache?, context?, dependencies?, devServer?, devtool?, entry, externals?, loader?, module?, name?, node?, output?, performance?, plugins?, profile?, recordsInputPath?, recordsOutputPath?, recordsPath?, resolve?, resolveLoader?, stats?, target?, watch?, watchOptions? }
   For typos: please correct them.
   For loader options: webpack 2 no longer allows custom properties in configuration.
     Loaders should be updated to allow passing options via loader options in module.rules.
     Until loaders are updated one can use the LoaderOptionsPlugin to pass these options to the loader:
     plugins: [
       new webpack.LoaderOptionsPlugin({
         // test: /\.xxx$/, // may apply this only for some modules
         options: {
           postcss: ...
         }
       })
     ]
 - configuration.resolve.extensions[0] should not be empty.

npm ERR! Linux 3.19.0-68-generic
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "production"
npm ERR! node v5.0.0
npm ERR! npm  v3.3.12
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>production: `webpack -p--config webpack.production.config.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>production script 'webpack -p--config webpack.production.config.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the belong package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     webpack -p--config webpack.production.config.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls belong
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/avinash/code/flash/npm-debug.log

Config 1
const webpack = require('webpack');
const path = require('path');
const autoprefixer = require('autoprefixer');
const ExtractTextPlugin = require('extract-text-webpack-plugin');
const BundleTracker = require('webpack-bundle-tracker');

const ENV_VARS = {
  staging: {
    publicPath: 'https://flash-staging.s3.amazonaws.com',
    filename: './webpack-stats.staging.json',
  },
  production: {
    publicPath: 'https://d2li6e5dkf687u.cloudfront.net',
    filename: './webpack-stats.production.json',
  },
};

module.exports = {
  cache: true,
  entry: {
    build: ['./flash/ver_static/js/react/app.js'],
    vendor: ['react', 'lodash', 'moment', 'react-router', 'redux'],
  },
  output: {
    filename: '[name]-[hash].min.js', // this is the default name, so you can skip it
    chunkFilename: '[name]-[hash].js',
    path: path.resolve(__dirname, 'flash/ver_static'),
    publicPath: `${ENV_VARS['production'].publicPath}/static/`,
    sourceMapFilename: '[file].map',
  },
  devtool: 'source-map',
  module: {
    loaders: [{
      rules: [
        {
          test: /\.(js|jsx)?$/,
          exclude: [
            /node_modules/,
            /.\/flash\/ver_static\/js\/libs\//,
          ],
          use: [
            {
              loader: 'babel',
              options: {
                presets: [['es2015', { modules: false }], 'stage-0', 'react'],
                plugins: [
                  'transform-runtime',
                  'transform-react-remove-prop-types',
                  'add-react-displayname',
                ],
              },
            },
          ],
        }, {
          test: /\.scss$/,
          loader: ExtractTextPlugin.extract({
            fallback: 'style-loader',
            use: [
              {
                loader: 'css-loader',
                options: {
                  plugins: [new webpack.LoaderOptionsPlugin({ minimize: true })],
                },
              },
              {
                loader: 'postcss-loader',
                options: {
                  plugins: [autoprefixer({ browsers: ['> 1%', 'last 2 versions'] })],
                },
              },
              {
                loader: 'sass-loader',
                options: {
                  plugins: [
                    new webpack.LoaderOptionsPlugin({
                      sourceMap: true,
                      indentedSyntax: 'sass',
                      includePaths: [path.join(__dirname, 'flash/ver_static/css')],
                    }),
                  ],
                },
              },
            ],
          }),
        },
      ],
    }],
  },
  resolveLoader: {
    moduleExtensions: ['-loader'],
  },
  externals: {
    // don't bundle the 'jquery' npm package with our bundle.js
    jquery: 'jQuery',
  },
  resolve: {
    extensions: ['.js', '.jsx'],
  },
  plugins: [
    new webpack.DefinePlugin({ 'process.env.NODE_ENV': '"production"' }),
    new webpack.optimize.CommonsChunkPlugin({ name: 'vendor', filename: 'vendor-[hash].min.js' }),
    new webpack.optimize.UglifyJsPlugin({ minimize: true, compress: { warnings: false, drop_console: false } }),
    new ExtractTextPlugin({ filename: 'belong-[hash].min.css', allChunks: true }),
    new webpack.IgnorePlugin(/^\.\/locale$/, [/moment$/]),
    new BundleTracker({ filename: ENV_VARS['production'].filename }),
    function ExitFailedBuildProcess() {
      this.plugin('done', (stats) => {
        if (
          stats.compilation.errors &&
          stats.compilation.errors.length &&
          process.argv.indexOf('--watch') === -1
        ) {
          process.exit(1);
        }
      });
    },
  ],
};

Config 2
const webpack = require('webpack');
const path = require('path');
const autoprefixer = require('autoprefixer');
const ExtractTextPlugin = require('extract-text-webpack-plugin');
const BundleTracker = require('webpack-bundle-tracker');

const ENV_VARS = {
  staging: {
    publicPath: 'https://flash-staging.s3.amazonaws.com',
    filename: './webpack-stats.staging.json',
  },
  production: {
    publicPath: 'https://d2li6e5dkf687u.cloudfront.net',
    filename: './webpack-stats.production.json',
  },
};

module.exports = {
  cache: true,
  entry: {
    build: ['./flash/ver_static/js/react/app.js'],
    vendor: ['react', 'lodash', 'moment', 'react-router', 'redux'],
  },
  output: {
    filename: '[name]-[hash].min.js', // this is the default name, so you can skip it
    chunkFilename: '[name]-[hash].js',
    path: path.resolve(__dirname, 'flash/ver_static'),
    publicPath: `${ENV_VARS['production'].publicPath}/static/`,
    sourceMapFilename: '[file].map',
  },
  devtool: 'source-map',
  module: {
    loaders: [{
      rules: [
        {
          test: /\.(js|jsx)?$/,
          exclude: [
            /node_modules/,
            /.\/flash\/ver_static\/js\/libs\//,
          ],
          use: [
            {
              loader: 'babel',
              options: {
                presets: [['es2015', { modules: false }], 'stage-0', 'react'],
                plugins: [
                  'transform-runtime',
                  'transform-react-remove-prop-types',
                  'add-react-displayname',
                ],
              },
            },
          ],
        }, {
          test: /\.scss$/,
          loader: ExtractTextPlugin.extract({
            fallback: 'style-loader',
            use: [
              { loader: 'css-loader' },
              {
                loader: 'postcss-loader',
                options: {
                  plugins: [autoprefixer({ browsers: ['> 1%', 'last 2 versions'] })],
                },
              },
              { loader: 'sass-loader' },
            ],
          }),
        },
      ],
    }],
  },
  resolveLoader: {
    moduleExtensions: ['-loader'],
  },
  externals: {
    // don't bundle the 'jquery' npm package with our bundle.js
    jquery: 'jQuery',
  },
  resolve: {
    extensions: ['.js', '.jsx'],
  },
  plugins: [
    new webpack.LoaderOptionsPlugin({ minimize: true }),
    new webpack.LoaderOptionsPlugin({
      sourceMap: true,
      indentedSyntax: 'sass',
      includePaths: [path.join(__dirname, 'flash/ver_static/css')],
    }),
    new webpack.DefinePlugin({ 'process.env.NODE_ENV': '"production"' }),
    new webpack.optimize.CommonsChunkPlugin({ name: 'vendor', filename: 'vendor-[hash].min.js' }),
    new webpack.optimize.UglifyJsPlugin({ minimize: true, compress: { warnings: false, drop_console: false } }),
    new ExtractTextPlugin({ filename: 'belong-[hash].min.css', allChunks: true }),
    new webpack.IgnorePlugin(/^\.\/locale$/, [/moment$/]),
    new BundleTracker({ filename: ENV_VARS['production'].filename }),
    function ExitFailedBuildProcess() {
      this.plugin('done', (stats) => {
        if (
          stats.compilation.errors &&
          stats.compilation.errors.length &&
          process.argv.indexOf('--watch') === -1
        ) {
          process.exit(1);
        }
      });
    },
  ],
};

I'm going to close this manually as it appears this is outdated. Unless there is a custom loader involved you shouldn't require LoaderOptionsPlugin anymore.


fix: modify addon payload compliant with AS whenever talking to AS APIs

Signed-off-by: Yashvardhan Kukreja<EMAIL_ADDRESS>py-mtcli
Description
AS APIs expect the req. payload to contain addOnParameters as
{
    "parameters": [<parameter_obj_1>, <parameter_obj_2>, ....]
}

(directly as a slice of parameter-objects),
unlike the old CS-API which expected addOnParameters as:
{
    "parameters": {
        "items": [<parameter_obj_1>, <parameter_obj_2>, ....]
    }
}

This PR adds the support for tweaking the addOnParameters with respect to the API being called (AS or CS)

I remember ciaran mentioning this API change, but if I'm not mistaken, CS also accepts that format.


acme.json in an invalid JSON, Traefik can't read certs

Welcome!

[X] Yes, I've searched similar issues on GitHub and didn't find any.
[X] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you do?
I use Traefik as a Docker container on several servers, with both the Docker Provider and the File Provider working side by side (the File Provider will soon be deprecated).
Each Traefik container handles about 100 services (websites in their own Docker containers). I use the HTTP challenge.
My configuration works fine, and generates certificates accordingly. The certs are written to the acme.json file automatically by Traefik. This is where the problem lies.
What did you see instead?
For unknown reasons and at (apparently) random times, Traefik fails to correctly and coimpletely write the certs to the acme.json file, resulting in an incomplete ans so invalid JSON file.
If I cat the acme.json file, I can clearly see that the file is invalid, like so :
{
  "acmeResolver": {
    "Account": {
      "Email": "[REDACTED]",
      "Registration": {
        "body": {
          "status": "valid",
          "contact": [
            "mailto:[REDACTED]"
          ]
        },
        "uri": "https://acme-v02.api.letsencrypt.org/acme/acct/[REDACTED]"
      },
      "PrivateKey": "[REDACTED]",
      "KeyType": "4096"
    },
    "Certificates": [
      // For simplicity, only one shown here, but imagine ~100 elements in the array.
      {
        "domain": {
          "main": "[REDACTED]"
        },
        "certificate": "[REDACTED]",
        "key": "someKeyStringThatsRedacted....dISnJVMF
>>EOF

// OBVIOUSLY SOME STUFF IS MISSING HERE
// EOF is indicative, it doesn't appear when using `cat`

The cut certificate array element is the $N$th one in the array, where $1 < N < 100$
I have found no clue to why this occurs. Theories I've come up with up until now are :

Concurrency issues if Traefik attempts two cert writes simultaneously, or any other concurrency issue
Traefik crashes during the write process
Traefik container restarts during the write process

I haven't looked over Traefik's code yet, but if the write process is streamed or chunked, a single sync write to the acme.json might fix this. Of course this might cause other issues.
What version of Traefik are you using?
This issue has occurred over the years since I've updated Traefik from v1.7 to v2.5.
This issue has occurred on v2.5, v2.10 & v2.11.
What is your environment & configuration?
log:
  level: ERROR
global:
  checkNewVersion: false
  sendAnonymousUsage: false
api:
  insecure: false
  dashboard: true
  debug: false
ping:
  entryPoint: traefik
providers:
  file:
    directory: /etc/traefik/dynamic
    watch: true
  docker:
    endpoint: unix:///var/run/docker.sock
    exposedByDefault: false
certificatesResolvers:
  acmeResolver:
    acme:
      email: [REDACTED]
      storage: /etc/traefik/acme.json
      httpChallenge:
        entryPoint: web
      caServer: https://acme-v02.api.letsencrypt.org/directory
defaultEntryPoints:
  - websecure
  - web
  - traefik
entryPoints:
  web:
    address: :80
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
          permanent: true
  websecure:
    address: :443
    http:
      tls:
        certResolver: acmeResolver
  traefik:
    address: :8080

If applicable, please paste the log output in DEBUG level
No really obvious log can be found relating to this issue except the "Can't read ACME file" once the issue has occured.
My Traefik containers are configured to automatically restart on crashes (Docker's restart: always policy). Logs often get squashed by repeated crash logs.

Hi @Clovel,
Is the acme.json shared across multiple Traefik instances?
If yes, that's the reason of your issue:
https://doc.traefik.io/traefik/https/acme/#storage

For concurrency reasons, this file cannot be shared across multiple instances of Traefik.

Hi @emilevauge,
No, the acme.json file is not shared across multiple Traefik instances.
I do have another container reading that file on rare occasions to check if it has been proken or not to restore a backup of the file if necessary, but this bug existed before this feature in my app. This feature is a workaround to fall back on my feet if the acme.json file is corrupt.
Can reading the file cause these concurrency issues ? To read it I use a basic fs.promises.readFile call in NodeJS v14 :
const lACMEFileContents: string = await fs.promises.readFile(
  TRAEFIK_ACME_CONFIG_FILE_PATH,
  {
    encoding: 'utf8',
    flag: 'r',
  },
);

I then backup the file as such just in case (mind that this will not be the backup I restore if it is broken) :
/* Write the ACME file to the backup directory */
await fs.promises.writeFile(
  `${TRAEFIK_ACME_CONFIG_FILE_PATH}.backup`,
  lACMEFileContents,
  {
    encoding: 'utf8',
    flag: 'w',
  },
);

Maybe the fact that it is in the same directoy might cause issues ?
Then I try to parse the file, and if it fails I then rewrite it :
let lParsedACMEFileContents: TraefikACME | null = null;
try {
  lParsedACMEFileContents = JSON.parse(lACMEFileContents) as TraefikACME;
} catch(pException) {
  console.error(`[ERROR] <TraefikManager.backupACMEFile> Failed to parse ACME file contents :`, pException);
  throw pException;
}

I only rewrite to the file if it is broken, using a previous backup that is valid.

Hey @Clovel,
Thank you for your feedback.
To move forward on the analysis, we'd like to know if you have an idea of the best way to reproduce the issue and if you could provide a reproducible case (for instance, full Kubernetes manifest to reproduce the issue).
In the meantime, if any community member can help us find verified steps to reproduce and fix the issue if possible, we would love the help.
Thanks in advance.

To move forward on the analysis, we'd like to know if you have an idea of the best way to reproduce the issue and if you could provide a reproducible case

To be honest, I don't have a reproduction process for this bug. For the moment, it appears to be random (of course it most likely isn't). I haven't found a specific case that causes this issue.
If I find anything, I'll be sure to update you on this thread.

for instance, full Kubernetes manifest to reproduce the issue

I don't use K8s, but raw Docker on Linux (Debian 12). I deploy the various proxied apps via a (containerized) NodeJS app that uses the Dockerode library.
Here is an excerpts of the dynamic configuration (File Provider) :
http:
  routers:
    FXw8Xfh7brjjkNo4p:
      rule: Host(`[REDACTED]`,`[REDACTED]`)
      service: FXw8Xfh7brjjkNo4p
      tls:
        certResolver: acmeResolver
        domains:
          - main: [REDACTED]
            sans:
              - [REDACTED]
      middlewares:
        - waf
...
...
  services:
    FXw8Xfh7brjjkNo4p:
      loadBalancer:
        servers:
          - url: http://<IP_ADDRESS>:80
...
  middlewares:
    waf:
      plugin:
        traefik-modsecurity-plugin:
          MaxBodySize: 10485760
          # And more redacted variables related to ModSecurity plugin

I'd like to point out that all the middleware stuff about ModSecurity was added later. The bug this thread is all about predates these features.

I don't really know where to start to build a reproductible case here.
What would you recommend ?
I almost need to have a full server of about 100 Docker containers to deploy, 1 or 2 domain names for each container to use for cert challenges, and an app to deploy all of these to have something that fits my case. The test case is my app so to speak...
We'd have to mock almost all the components to have a solid test case, which would require a lot of time. I'm sure there is a better way to build a test case.
Or maybe by analyzing the code that manages the writes to the ACME file we can find something worthy of an upgrade/fix.

Hello @Clovel,

What would you recommend ?

TBH this issue is hard to reproduce. We haven't been able to set up a reproducible use case.
For this reason, if you face the issue again, we'd be interested in having more information on the environment to move forward.
Please keep us updated even if you do not face the problem again (in such a case we'll close the issue).
In the meantime, I left the issue open, waiting for more news from you.

Please keep us updated even if you do not face the problem again (in such a case we'll close the issue).

Well, I had it twice last week on two different servers, but no logs or specific events seem to point to a cause.
I will keep you updated.
P.S. : If I ever have the time, I'll try too look at the Traefik source code too, Maybe something is obvious (not necessarily a bug, but an explanation for this behaviour)

Still haven't found anything.
I was wondering : Does the Traefik source code ensure atomic writing into the acme.json file ? In other words, if Traefik starts to write to the file and shuts down (docker restart for example), does it gracefully wait for the end of the write operation or does it interrupt the write operation ?


SSKeychain class in SumUp SDK breaks iOS 10

Hi,
Apparently SumUp's SDK declares a class called SSKeychain which is in conflict with the system one. This is causing us issues with key chain functions (i.e. our app doesn't work when compiled for iOS 10).
Could you remove this class from your SDK ?
objc[67397]: Class SSKeychain is implemented in both /Users/xx/Downloads/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/System/Library/PrivateFrameworks/StoreServices.framework/StoreServices (0x11752d490) and /Users/xx/Library/Developer/CoreSimulator/Devices/169BEXXX-11CC-41B7-AAD7-86A3A2C9B58E/data/Containers/Bundle/Application/E8F77XXX-4EEC-4D4F-B9B9-A6E7EB75C85E/xx.app/xxApp (0x1060ecf00). One of the two will be used. Which one is undefined.

thanks for the report. We'll look into it.

Hi @dclerc .
We've just released 1.3.0b4 which hopefully fixes your issue. Would you mind giving it a try and let us know? https://github.com/sumup/sumup-ios-sdk/tree/develop
Thanks,
Lukas

It works, thanks!

Thanks for raising it. Please keep in mind to add NSBluetoothPeripheralUsageDescription to your info plist when building on iOS 10 as it will crash if not present and firing up BTLE.
See https://developer.apple.com/library/mac/documentation/General/Reference/InfoPlistKeyReference/Articles/CocoaKeys.html#//apple_ref/doc/uid/TP40009251-SW20


Fail if users configured USE_FIND_ALGORITHM=true and VALIDATE_ALL_CODEBASE=false at the same time

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Currently, super-linter allows setting USE_FIND_ALGORITHM=true and VALIDATE_ALL_CODEBASE=false.
In this case, super-linter silently ignores the USE_FIND_ALGORITHM setting. Users might be confused by this behavior.
Expected Behavior
Given that this is an unsupported configuration, we should explicitly fail, in this case.
Anything else?
No response

I think this should be a supported algorithm

When VALIDATE_ALL_CODEBASE is false, we need to diff against something, and we use Git for that. What shall we diff against without using Git?

A situation is happens when using flag IGNORE_GITIGNORED_FILES=true there is an error "fatal: not a git repository: /<path/to/the/git/worktree>/worktrees/fix_java"

Currently, this is expected because we use git check-ignore for that https://github.com/super-linter/super-linter/blob/7a21f934b424c138cdcbeb2c070932f71f8a184b/lib/functions/buildFileList.sh#L230
To support this use case, we'd have to duplicate that logic, or find a tool that takes one or multiple .gitignore files, and tells if a given file is ignored according to those .gitignore files.

And, in the current implementation, we should actually enable "git repository validation" if IGNORE_GITIGNORED_FILES=true, as we now do when USE_FIND_ALGORITHM=false (#4953)

The cause is that the git repo is actually a git worktree, if mounted in docker container, it becomes an invalid git repo.


feat(*): use typed secrets for projects, builds, and jobs

Assign a type to kubernetes secrets storing configuration for projects,
builds, and jobs.
This will simplify queries and provide distinction for debugging.
$ kubectl get secrets
NAME                                                             TYPE                                  DATA      AGE
brigade-830c16d4aaf6f5490937ad719afd8490a5bcbef064d397411043ac   brigade.sh/project                    8         1h
brigade-brigade-api-token-t4vw8                                  kubernetes.io/service-account-token   3         1h
brigade-brigade-cr-gw-token-m8sss                                kubernetes.io/service-account-token   3         1h
brigade-brigade-ctrl-token-2z5zz                                 kubernetes.io/service-account-token   3         1h
brigade-brigade-github-gw-token-kxg77                            kubernetes.io/service-account-token   3         1h
brigade-f1ccf2ad3495a1c68b82c78789061036240a9eb5ea2f0b0c4c20f9   brigade.sh/project                    6         1h
brigade-worker-01c5qe5qf8ypdbmz7zh5q3qrxk-589e1502               brigade.sh/build                      7         28m
brigade-worker-token-656m2                                       kubernetes.io/service-account-token   3         1h
default-token-tbczv                                              kubernetes.io/service-account-token   3         2d
node-runner-1517983823420-589e1502                               brigade.sh/job                        3         18m
slack-notify-1517983827460-589e1502                              brigade.sh/job                        6         18m

Good catch!

Love this idea. I think we should do it.