Refactor cstr to fix benchmark issues

Description
This PR aims to address and refactor the cstr model to fix benchmark issues as outlined in Issue #60.
Solution strategy causing issues:

[x] Big-M MINLP reformulation
[x] Hull MINLP reformulation
[ ] GLOA
[x] Enumeration
[x] LOA
[x] LBB
[x] RIC

Branch
This PR targets the benchmark_zd branch.
Steps to Reproduce

Run the benchmark tests for the cstr module before applying the changes.
Apply the changes from this PR.
Re-run the benchmark tests to verify performance improvements.

Notes

Ensure that all sub-solvers are tested thoroughly after refactoring.
Additional optimizations may be required based on benchmark results.

Feel free to adjust any details as necessary before submitting the PR.

GLOA log output
Starting GDPopt version 22.5.13 using GLOA algorithm
iterlim: None
time_limit: 3600
tee: true
logger: <Logger pyomo.contrib.gdpopt (INFO)>
integer_tolerance: 1e-05
constraint_tolerance: 1e-06
variable_tolerance: 1e-08
subproblem_initialization_method: <function restore_vars_to_original_values at 0x7fd517337ce0>
call_before_subproblem_solve: <class 'pyomo.contrib.gdpopt.util._DoNothing'>
call_after_subproblem_solve: <class 'pyomo.contrib.gdpopt.util._DoNothing'>
call_after_subproblem_feasible: <class 'pyomo.contrib.gdpopt.util._DoNothing'>
force_subproblem_nlp: false
subproblem_presolve: true
tighten_nlp_var_bounds: false
round_discrete_vars: true
max_fbbt_iterations: 3
init_strategy: None
init_algorithm: set_covering
custom_init_disjuncts: []
max_slack: 1000.0
OA_penalty_factor: 1000.0
set_cover_iterlim: 8
discrete_problem_transformation: gdp.bigm
call_before_discrete_problem_solve: <class 'pyomo.contrib.gdpopt.util._DoNothing'>
call_after_discrete_problem_solve: <class 'pyomo.contrib.gdpopt.util._DoNothing'>
call_before_master_solve: <class 'pyomo.contrib.gdpopt.util._DoNothing'>
call_after_master_solve: <class 'pyomo.contrib.gdpopt.util._DoNothing'>
mip_presolve: true
calc_disjunctive_bounds: false
obbt_disjunctive_bounds: false
mip_solver: gams
mip_solver_args:
  add_options: ['option threads=1']
  tee: true
nlp_solver: gams
nlp_solver_args:
  add_options: ['option threads=1;', '$onecho > baron.opt', 'FirstLoc 1', '$offecho', 'GAMS_MODEL.optfile=1', 'option optcr=1e-2;']
  solver: baron
  tee: true
minlp_solver: gams
minlp_solver_args:
  add_options: ['option threads=1;', 'option optcr=1e-6;']
  solver: baron
  tee: true
local_minlp_solver: gams
local_minlp_solver_args:
  add_options: ['option threads=1;', '$onecho > baron.opt', 'FirstLoc 1', '$offecho', 'GAMS_MODEL.optfile=1', 'option optcr=1e-2;']
  solver: baron
  tee: true
small_dual_tolerance: 1e-08
bound_tolerance: 0.01

If you use this software, you may cite the following:
            - Implementation:
            Chen, Q; Johnson, ES; Bernal, DE; Valentin, R; Kale, S;
            Bates, J; Siirola, JD; Grossmann, IE.
            Pyomo.GDP: an ecosystem for logic based modeling and optimization
            development.
            Optimization and Engineering, 2021.

- GLOA algorithm:
        Lee, S; Grossmann, IE.
        A Global Optimization Algorithm for Nonconvex Generalized
        Disjunctive Programming and Applications to Process Systems.
        Comp. and Chem. Eng. 2001, 25, 1675-1697.
        DOI: 10.1016/S0098-1354(01)00732-3.
Original model has 100 constraints (17 nonlinear) and 10 disjunctions, with 76 variables, of which 20 are binary, 0 are integer, and 56 are continuous.
---Starting GDPopt initialization---
Starting set covering initialization.
=============================================================================================
Iteration | Subproblem Type | Lower Bound | Upper Bound |   Gap    | Time(s)

--- Job model.gms Start 10/17/24 20:21:43 47.6.0 c2de9d6d LEX-LEG x86 64bit/Linux
--- Applying:
    /package/gams/47.6.0/gmsprmun.txt
--- GAMS Parameters defined
    Input /tmp/tmpadv92w2s/model.gms
    Output /tmp/tmpadv92w2s/output.lst
    ScrDir /tmp/tmpadv92w2s/225a/
    SysDir /package/gams/47.6.0/
    CurDir /tmp/tmpadv92w2s/
    LogOption 3
Licensee: David Bernal / SECQUOIA Research Group         G240524|0002AS-GEN
          Purdue University, Davidson School of Chemical EngineeringDCE3427
          /local/scratch/a/ctristan/.local/share/GAMS/gamslice.txt
          License Admin: David E Bernal<EMAIL_ADDRESS>
          Evaluation license: Not for commercial or production use
          The evaluation period of the license will expire on Apr 14, 2026
Processor information: 1 socket(s), 48 core(s), and 96 thread(s) available
GAMS 47.6.0   Copyright (C) 1987-2024 GAMS Development. All rights reserved
--- Starting compilation
--- model.gms(1286) 2 Mb
--- Starting execution: elapsed 0:00:00.016
--- model.gms(835) 3 Mb
--- Generating MIP model GAMS_MODEL
--- model.gms(839) 3 Mb
---   280 rows  114 columns  655 non-zeroes
---   65 discrete-columns
--- Range statistics (absolute non-zero finite values)
--- RHS       [min, max] : [ 1.000E+00, 2.000E+01] - Zero values observed as well
--- Bound     [min, max] : [ 1.000E+00, 1.000E+01] - Zero values observed as well
--- Matrix    [min, max] : [ 9.500E-01, 2.000E+01]
--- Executing CPLEX (Solvelink=5): elapsed 0:00:00.020

IBM ILOG CPLEX   47.6.0 c2de9d6d Sep 12, 2024          LEG x86 64bit/Linux

--- GAMS/CPLEX licensed for continuous and discrete problems.
--- GMO setup time: 0.00s
--- GMO memory 0.54 Mb (peak 0.55 Mb)
--- Dictionary memory 0.00 Mb
--- Cplex <IP_ADDRESS> link memory 0.01 Mb (peak 0.02 Mb)
--- Starting Cplex

Version identifier: <IP_ADDRESS> | 2022-11-28 | 9160aff4d
CPXPARAM_Advance                                 0
CPXPARAM_Threads                                 1
CPXPARAM_MIP_Display                             4
CPXPARAM_MIP_Pool_Capacity                       0
CPXPARAM_TimeLimit                               3599.7754099676386
CPXPARAM_MIP_Tolerances_AbsMIPGap                0
Generic callback                                 0x50
Tried aggregator 3 times.
MIP Presolve eliminated 155 rows and 42 columns.
MIP Presolve modified 59 coefficients.
Aggregator did 31 substitutions.
Reduced MIP has 93 rows, 41 columns, and 231 nonzeros.
Reduced MIP has 16 binaries, 0 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.00 sec. (0.62 ticks)
Found incumbent of value 55.000000 after 0.00 sec. (0.84 ticks)
Probing changed sense of 4 constraints.
Probing time = 0.00 sec. (0.07 ticks)
Cover probing fixed 0 vars, tightened 1 bounds.
Tried aggregator 1 time.
Detecting symmetries...
MIP Presolve eliminated 11 rows and 4 columns.
MIP Presolve modified 10 coefficients.
Reduced MIP has 82 rows, 37 columns, and 197 nonzeros.
Reduced MIP has 12 binaries, 0 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.00 sec. (0.18 ticks)
Probing time = 0.00 sec. (0.04 ticks)
Clique table members: 16.
MIP emphasis: balance optimality and feasibility.
MIP search method: dynamic search.
Parallel mode: none, using 1 thread.
Tried aggregator 1 time.
No LP presolve or aggregator reductions.
Presolve time = 0.00 sec. (0.03 ticks)
Initializing dual steep norms . . .

Iteration log . . .
Iteration:     1   Dual objective     =            85.000000
Root relaxation solution time = 0.00 sec. (0.07 ticks)

        Nodes                                         Cuts/
   Node  Left     Objective  IInf  Best Integer    Best Bound    ItCnt     Gap

*     0+    0                           55.0000       85.0000            54.55%
Found incumbent of value 55.000000 after 0.00 sec. (1.30 ticks)
*     0     0      integral     0       85.0000       85.0000        1    0.00%
Elapsed time = 0.00 sec. (1.31 ticks, tree = 0.00 MB)
Found incumbent of value 85.000000 after 0.00 sec. (1.31 ticks)

Root node processing (before b&c):
  Real time             =    0.00 sec. (1.31 ticks)
Sequential b&c:
  Real time             =    0.00 sec. (0.00 ticks)
                          ------------
Total (root+branch&cut) =    0.00 sec. (1.31 ticks)

--- MIP status (101): integer optimal solution.
--- Cplex Time: 0.00sec (det. 1.31 ticks)

--- Returning a primal only solution to GAMS (marginals all set to NA).
--- Fixing integer variables and solving final LP...

Version identifier: <IP_ADDRESS> | 2022-11-28 | 9160aff4d
CPXPARAM_Advance                                 2
CPXPARAM_Threads                                 1
CPXPARAM_MIP_Display                             4
CPXPARAM_MIP_Pool_Capacity                       0
CPXPARAM_TimeLimit                               3599.7754099676386
CPXPARAM_MIP_Tolerances_AbsMIPGap                0
Tried aggregator 1 time.
LP Presolve eliminated 276 rows and 111 columns.
Aggregator did 3 substitutions.
All rows and columns eliminated.
Presolve time = 0.00 sec. (0.10 ticks)

--- Fixed MIP status (1): optimal.
--- Cplex Time: 0.00sec (det. 0.18 ticks)

Proven optimal solution
MIP Solution:           85.000000    (1 iterations, 0 nodes)
Final Solve:            85.000000    (0 iterations)

Best possible:          85.000000
Absolute gap:            0.000000
Relative gap:            0.000000

--- Reading solution for model GAMS_MODEL
--- Executing after solve: elapsed 0:00:00.028
--- model.gms(1286) 5 Mb
--- Putfile results /tmp/tmpadv92w2s/results.dat
--- Putfile statresults /tmp/tmpadv92w2s/resultsstat.dat
*** Status: Normal completion
--- Job model.gms Stop 10/17/24 20:21:43 elapsed 0:00:00.029
WARNING: implicitly casting 'YR_is_recycle[1].indicator_var' value 1 to bool
WARNING: implicitly casting 'YP_is_cstr[1].indicator_var' value 1 to bool
WARNING: implicitly casting 'YR_is_recycle[2].indicator_var' value 0 to bool
WARNING: implicitly casting 'YP_is_cstr[2].indicator_var' value 1 to bool
WARNING: implicitly casting 'YR_is_recycle[3].indicator_var' value 0 to bool
WARNING: implicitly casting 'YP_is_cstr[3].indicator_var' value 1 to bool
WARNING: implicitly casting 'YR_is_recycle[4].indicator_var' value 0 to bool
WARNING: implicitly casting 'YP_is_cstr[4].indicator_var' value 1 to bool
WARNING: implicitly casting 'YR_is_recycle[5].indicator_var' value 0 to bool
WARNING: implicitly casting 'YP_is_cstr[5].indicator_var' value 1 to bool
--- Job model.gms Start 10/17/24 20:21:44 47.6.0 c2de9d6d LEX-LEG x86 64bit/Linux
--- Applying:
    /package/gams/47.6.0/gmsprmun.txt
--- GAMS Parameters defined
    Input /tmp/tmp71nz7lio/model.gms
    Output /tmp/tmp71nz7lio/output.lst
    ScrDir /tmp/tmp71nz7lio/225a/
    SysDir /package/gams/47.6.0/
    CurDir /tmp/tmp71nz7lio/
    LogOption 3
Licensee: David Bernal / SECQUOIA Research Group         G240524|0002AS-GEN
          Purdue University, Davidson School of Chemical EngineeringDCE3427
          /local/scratch/a/ctristan/.local/share/GAMS/gamslice.txt
          License Admin: David E Bernal<EMAIL_ADDRESS>
          Evaluation license: Not for commercial or production use
          The evaluation period of the license will expire on Apr 14, 2026
Processor information: 1 socket(s), 48 core(s), and 96 thread(s) available
GAMS 47.6.0   Copyright (C) 1987-2024 GAMS Development. All rights reserved
--- Starting compilation
--- model.gms(407) 2 Mb
--- Starting execution: elapsed 0:00:00.015
--- model.gms(257) 3 Mb
--- Generating MINLP model GAMS_MODEL
--- model.gms(261) 5 Mb
--- Reset Solvelink = 2
---   48 rows  45 columns  129 non-zeroes
---   83 nl-code  40 nl-non-zeroes
---   4 discrete-columns
--- Range statistics (absolute non-zero finite values)
--- RHS       [min, max] : [ 1.000E-02, 9.000E+00] - Zero values observed as well
--- Bound     [min, max] : [ 1.000E+00, 1.000E+01] - Zero values observed as well
--- Matrix    [min, max] : [ 9.500E-01, 5.000E+00] - Zero values observed as well
--- model.gms(261) 3 Mb
--- Executing BARON (Solvelink=2): elapsed 0:00:00.016

GAMS/BARON       47.6.0 c2de9d6d Sep 12, 2024          LEG x86 64bit/Linux

===========================================================================
 BARON version 24.5.8. Built: LNX-64 Wed May 8 10:06:40 EDT 2024
 Running on machine dantzig.ecn.purdue.edu

 BARON is a product of The Optimization Firm.
 For information on BARON, see https://minlp.com/about-baron

 If you publish work using this software, please cite publications from
 https://minlp.com/baron-publications, such as:

 Khajavirad, A. and N. V. Sahinidis,
 A hybrid LP/NLP paradigm for global optimization relaxations,
 Mathematical Programming Computation, 10, 383-421, 2018.
===========================================================================
 This BARON run may utilize the following subsolver(s)
 For LP/MIP/QP: CLP/CBC, ILOG CPLEX
 For NLP: MINOS, SNOPT, External NLP, IPOPT, FILTERSQP
===========================================================================
 Doing local search
 Preprocessing found feasible solution with value 3.13020
 Solving bounding LP
 Starting multi-start local search
 Done with local search
===========================================================================
  Iteration    Open nodes         Time (s)    Lower bound      Upper bound
          1             1             0.18      1.14300          3.13020
*         8+            4             0.33      2.18511          3.13019
*       426            15             0.81      3.03166          3.13017
*       485            13             0.90      3.03166          3.13016
*       654            11             0.99      3.03166          3.13014
       2010             0             1.78      3.13013          3.13013

                         *** Normal completion ***

 Wall clock time:                     1.79
 Total CPU time used:                 1.78

 Total no. of BaR iterations:    2010
 Best solution found at node:    1208
 Max. no. of nodes in memory:      20

 All done
===========================================================================

Solution      = 3.13012906092058  found at node 1208
Best possible = 3.13012906092058
Absolute gap  = 0  optca = 1E-9
Relative gap  = 0  optcr = 1E-6

--- Reading solution for model GAMS_MODEL
***
*** Solver did not provide marginals for model GAMS_MODEL
***
--- Executing after solve: elapsed 0:00:01.836
--- model.gms(264) 3 Mb
--- model.gms(407) 5 Mb
--- Putfile results /tmp/tmp71nz7lio/results.dat
--- Putfile statresults /tmp/tmp71nz7lio/resultsstat.dat
*** Status: Normal completion
--- Job model.gms Stop 10/17/24 20:21:45 elapsed 0:00:01.837
        0        subproblem          -inf       3.13013       inf%      2.37  *

Solved in 0 iterations and 2.36730 seconds
Optimal objective value 3.1301290609
Relative optimality gap inf%


PyRuntime Error during inference: ModuleNotFoundError

I am trying to run inference on an onnx model compiled using onnx-mlir. When I execute the python script, it throws the following error:
ModuleNotFoundError: No module named 'PyRuntime'
The path to my PyRuntime is:
root@onnx-mlir:~/llvm-project/build/onnx-mlir/build/src/Runtime/CMakeFiles# ls
ExecutionSession.dir  OMTensorUtils.dir  PyRuntime.dir  cruntime.dir

So, I created a create a symbolic link to it:
root@onnx-mlir:~/llvm-project/build/onnx-mlir/build/Debug/bin# ln -s /llvm-project/build/onnx-mlir/build/src/Runtime/CMakeFiles
Then on running my script:
root@onnx-mlir:~/llvm-project/build/onnx-mlir/build/Debug/bin# python3 inference_googlenet.py
Traceback (most recent call last):
  File "inference_googlenet.py", line 2, in <module>
    from PyRuntime import ExecutionSession

I would greatly appreciate help with this.

http://onnx.ai/onnx-mlir/UsingPyRuntime.html
How about creating a link to onnx-mlir/build/lib?

See my other issue.
PyRuntime actually requires building, and builds to build/Debug/lib

http://onnx.ai/onnx-mlir/UsingPyRuntime.html
How about creating a link to onnx-mlir/build/Debug/lib?

Sorry, did not work.

From which directory should I execute the make PyRuntime?
Also, the /build/Debug/lib contains: PyRuntime.cpython-38-x86_64-linux-gnu.so and set ONNX_MLIR_HOME=/root/llvm-project/build/onnx-mlir/build/onnx-mlir/build/Debug/lib.
Problem still persists:
root@onnx-mlir:~/llvm-project/build/onnx-mlir/build/Debug/bin# python3 test_onnxmlir.py  Traceback (most recent call last):   File "test_onnxmlir.py", line 2, in <module>     from PyRuntime import ExecutionSession ModuleNotFoundError: No module named 'PyRuntime'

hi @Diksha-G !
execute make PyRuntime under build to generate the .so file.
Set ONNX_MLIR_HOME=/root/llvm-project/build/onnx-mlir/build/onnx-mlir/build/Debug/ . note that lib is not part of the variable, and is concatenated on by the RunONNXModel.py

Let me know if it works! I struggled HARD through this process and I want you to have to struggle a bit less. So I'm here for you.

hi @Diksha-G !
execute make PyRuntime under build to generate the .so file.
Set ONNX_MLIR_HOME=/root/llvm-project/build/onnx-mlir/build/onnx-mlir/build/Debug/ . note that lib is not part of the variable, and is concatenated on by the RunONNXModel.py

hi @renxida
I tried the following like you suggested:
root@onnx-mlir:~/llvm-project/build/onnx-mlir/build# make PyRuntime make: *** No rule to make target 'PyRuntime'.  Stop. root@onnx-mlir:~/llvm-project/build/onnx-mlir/build#
Thanks for your help. Would appreciate any lead :)

To help us figuring out what is going on, do you mind trying the end to end example I recently added:
https://github.com/onnx/onnx-mlir/blob/main/docs/mnist_example/README.md
I ran it on my mac and it worked fine. Let me know also the machine on which you run, Linux, Mac, Windows?

hi @renxida
I tried the following like you suggested:
root@onnx-mlir:~/llvm-project/build/onnx-mlir/build# make PyRuntime make: *** No rule to make target 'PyRuntime'. Stop. root@onnx-mlir:~/llvm-project/build/onnx-mlir/build#

Hi Diksha! from the following quote

Also, the /build/Debug/lib contains: PyRuntime.cpython-38-x86_64-linux-gnu.so and set ONNX_MLIR_HOME=/root/llvm-project/build/onnx-mlir/build/onnx-mlir/build/Debug/lib.

You seem to have already have PyRuntime compiled.
So try removing the /lib at the end of ONNX_MLIR_HOME=/root/llvm-project/build/onnx-mlir/build/onnx-mlir/build/Debug/lib and it might work.

To help us figuring out what is going on, do you mind trying the end to end example I recently added:
https://github.com/onnx/onnx-mlir/blob/main/docs/mnist_example/README.md
I ran it on my mac and it worked fine. Let me know also the machine on which you run, Linux, Mac, Windows?

Hey Alexander,
I am running this on Ubuntu 20.04 on a VM. I am trying the new example right now that you mentioned in the earlier post.

I can see the PyRuntime.cpython-…so  file in my lib folder. I have created the symbolic link from Debug/bin to Debug/lib folder. But when I open python3 and type “from PyRuntime import ExecutionSession”, it throws the error that PyRuntime not found.

I can see the PyRuntime.cpython-…so  file in my lib folder. I have created the symbol link from Debug/bin to Debug/lib folder. But when I open python3 and type “from PyRuntime import ExecutionSession”, it throws the error that PyRuntime not found.

Update @AlexandreEichenberger
I have been able to solve the PyRuntime Not found Error. When I execute the python script for resnet50 model, I get the following:
  File "mlir-test.py", line 6, in <module>     session = ExecutionSession(model, "run_main_graph") RuntimeError: Cannot open library: 'resnet50.so'
What am I missing here?

Hi @Diksha-G !
If you don't mind, I would love to have a chat by zoom and see if I can help you. I just sent you an email!

@renxida

Let me know if it works! I struggled HARD through this process and I want you to have to struggle a bit less. So I'm here for you.

Sorry about this. Can you make a list of things that, in retrospect, would have helped you. I really want to make this a smoother experience for external users. We are in the process to reorganize the docs, esp the readme, so any feedback from an external source would be very useful. Please make sure to specify which OS you are using, thanks

@Diksha-G

I have been able to solve the PyRuntime Not found Error. When I execute the python script for resnet50 model, I get the following:

File "mlir-test.py", line 6, in  session = ExecutionSession(model, "run_main_graph") RuntimeError: Cannot open library: 'resnet50.so'

What am I missing here?

We are building resnet in our make check-onnx-backend. Is it failing there, or when you are trying on a separate model.

@AlexandreEichenberger
I was able to get onnx-mlir to compile the onnx model and run inference on my VM after setting the LD_LIBRARY_PATH variable.

@AlexandreEichenberger
I was able to get onnx-mlir to compile the onnx model and run inference on my VM after setting the LD_LIBRARY_PATH variable.


added warning for extreme weather conditions

I added a pop-up component to be rendered when an extreme weather condition is occurring in the city searched. The component should render a warning to alert the user whenever the city searched is experiencing a thunderstorm, squall, or tornado. It should update accordingly when a new city is searched.

@indywip could you also post a few sample screenshots of the same?

Update:
Cleared merge conflicts
A working sample:

The feature works as intended.


feat: note command

Overview

[x] Closes #1427

Changes

[x] Introduces new @rustbot note command, allows the user to make a comment in an issue/PR that creates a Summary Entry.

Command Specification
The following comment would create a new summary entry
@rustbot note abc

details details details

If this was the first @rustbot note command to be triggered in this specific issue, it would add a new section in the Issue Body Description (or "Original Post") of the issue called "Summary Notes":
### Summary Notes

- ["abc" by @chazkiker2](link-to-comment)

If this was not the first @rustbot note command to be triggered in this issue, rustbot would simply append to the existing summary notes:
### Summary Notes

- ["abc" by @chazkiker2](link-to-chaz-comment)
- ["xyz" by @nikomatsakis ](link-to-niko-comment)

Cool =)

Just fyi - please ping me here (and likely best on Zulip as well) when you're ready for a review or would like some help, otherwise it'll likely get lost in my notifications :)

@nikomatsakis — Here's a quick screen recording with the current functionality, please let me know if it's what you expected or what you'd like to change. More notes/questions below
https://user-images.githubusercontent.com/70249988/151492393-acbec388-552e-4686-89a1-1d547d03a471.mov

One thing I noticed is that if I put new notes under a pre-existing <!-- TRIAGEBOT_SUMMARY_START --> ... <!-- TRIAGEBOT_SUMMARY_END -->. Wondering if that's expected behavior from interactions.rs or if that's a bug—and if so, whether I introduced it. I'll look further into it, but if that's expected functionality, please let me know 😄
Is the formatting of the comment markdown to your liking? Where did you want to place it in the issue body? Is "Summary Notes" an appropriate title, or would you prefer something else?

One thing I noticed is that if I put new notes under a pre-existing <!-- TRIAGEBOT_START --> ... <!-- TRIAGEBOT_END -->, triagebot starts a new section for the EditIssueBody. Wondering if that's expected behavior from interactions.rs or if that's a bug—and if so, whether I introduced that or if it was pre-existing. I'll look further into it, but if that's expected functionality, please let me know

Can you clarify what you mean by this? I'm not sure I'm following what you're saying -- maybe a link to an issue where this happened or another screencap would be useful.
I think it would be nice but definitely not required for the user to be able to directly edit the list, and in general feels relatively painful to get that to work well (editing markdown vs. re-generating from scratch is annoying) -- we can consider it a future improvement, but not more than that, I think.

Would you like any restrictions on the command? Users that are capable of using it? Should the summary details meet any requirements—the simplest of which would be "to be there" in the first place?

I don't think we should impose any restrictions -- ideally, @rustbot note <foo>  is sufficient, and the details could be before or after that. We should also try to support multiple notes per comment -- but that might be a little annoying, not sure if the parser is well-suited to that just yet.

We may also want to add a @rustbot note remove <foo> command, which drops a note from the top-level description. I suspect a note called literally "remove" is unlikely, so we don't need to worry about that.

One thing I noticed is that if I put new notes under a pre-existing <!-- TRIAGEBOT_START --> ... <!-- TRIAGEBOT_END -->, triagebot starts a new section for the EditIssueBody. Wondering if that's expected behavior from interactions.rs or if that's a bug—and if so, whether I introduced that or if it was pre-existing. I'll look further into it, but if that's expected functionality, please let me know

Can you clarify what you mean by this? I'm not sure I'm following what you're saying -- maybe a link to an issue where this happened or another screencap would be useful.

@Mark-Simulacrum Here's a screen cast of this bug and a link to the demo issue https://github.com/chazkiker2/triagebot-playground/issues/4:
https://user-images.githubusercontent.com/70249988/152027722-83ecad6e-b0a3-43cf-ad1b-4f9cf4a43a7c.mov

Haven't caught up, but something that came up when I was discussing this with @joshtriplett -- it might be nice if the edits are made to a "first comment" that we add automatically to the beginning of the issue -- that would of course require adding this comment for every new issue.

Haven't caught up, but something that came up when I was discussing this with @joshtriplett -- it might be nice if the edits are made to a "first comment" that we add automatically to the beginning of the issue -- that would of course require adding this comment for every new issue.

@nikomatsakis when you say "first comment" you mean a new comment as opposed to the issue body (or "original post")?

Hm, that seems like it's probably a bug in interactions.rs (pre-existing, almost certainly). I don't think it needs to block this PR necessarily but we should likely try to fix it. At a quick glance the code there is pretty poorly commented, so not completely sure where the bug is. It should be possible to look at just the section between the triagebot start/end comments I think, not sure how hard that would be.
re: the first comment, I'm pretty skeptical -- I don't think it's a good idea to start leaving such a comment on every issue and every PR, it seems pretty noisy. I'd start by editing the issue for now. In the future it might make sense to use the first comment where we control it (e.g., on MCPs, we already post one; on PRs we will once highfive functionality is migrated in).

As far as @rustbot note remove XXX functionality goes

My latest commit supports the following functionality:

@rustbot note remove XXX will remove the first summary entry with the title XXX from the data and the markdown
If there are duplicate XXX entries, only the first is removed
If there exists no such XXX entry, main will panic—This surely needs to be fixed... see below

I will make updates to support:

TODO: If there exists no such XXX entry instead, we will actually handle the error and make a rustbot comment noting the error

Questions:

Should we make a comment if the XXX entry doesn't exist?
Should we support for more granular control over which comment is deleted? What if I wanted to delete the second XXX entry instead—we could number the list and allow for @rustbot note remove 3 which would remove the 3rd comment?

I don't think we should expect more than one note with the same title, removing the first one seems OK for now. I would not post comments if users don't get it right, we can just silently ignore it: they should be able to edit the comment and make the deletion happen, I think.

@Mark-Simulacrum another question for ya (this is a possible bug): If a user edits the title of an existing @rustbot note to-edit entry and replaces the title with say edited, triagebot treats it like a new entry.
Demo issue https://github.com/chazkiker2/triagebot-playground/issues/8 and screencast:
https://user-images.githubusercontent.com/70249988/152057049-7e686543-8dbb-4de6-bdb7-6fe124072db1.mov

@Mark-Simulacrum and @nikomatsakis , I think PR is ready for another review... let me know your thoughts 😄

To split into separate issues:

[ ] Allow user to directly edit the summary note list
[ ] Allow user to make multiple note entries in the same comment
[ ] (Unrelated bug) Editing below <!-- TRIAGEBOT_START -->...<!-- TRIAGEBOT_END --> makes interactions.rs think it has to make a whole new entry

To do otherwise

[ ] Actually add documentation into the wiki entry for this command

Potential edge cases:

[ ] What if a user deletes a summary comment (either by deleting the comment itself or simply removing the @rustbot note xxx portion of the comment—I assume we probably want to remove that entry from the summary section?

re: the first comment, I'm pretty skeptical -- I don't think it's a good idea to start leaving such a comment on every issue and every PR, it seems pretty noisy.

I was proposing this for the case where rustbot (or for now, rust-highfive) already posts the first comment.
Long-term, for RFCs I'm hoping that first comment includes the "rendered" link, and these notes, and links to any FCPs started, and other such things.

I'm inclined to merge this in as-is and then iterate in the future -- the changes look good to me, and I think even a partial implementation with some bugs is likely to already be quite useful. We can leave most of the nice-to-have features for future (potentially near future, but future) PRs.
@chazkiker2 Do you see any blockers to doing so?

@Mark-Simulacrum sounds good to me!

Just added an initial wiki entry for this command, @Mark-Simulacrum feel free to edit / remove / add anything you think necessary :)

re: the first comment, I'm pretty skeptical -- I don't think it's a good idea to start leaving such a comment on every issue and every PR, it seems pretty noisy.

I was proposing this for the case where rustbot (or for now, rust-highfive) already posts the first comment.
Long-term, for RFCs I'm hoping that first comment includes the "rendered" link, and these notes, and links to any FCPs started, and other such things.

Split to #1579

@chazkiker2 thanks for working on this!


pull request for e2e test at 11/27/2016 00:56:03

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Build all corefx packages

CoreFX includes a number of packages that are not built when using source-build. For example, Microsoft.NETCore.Platforms is not built. It is consumed via NuGet. This causes two issues:

The bits included in the final cli build are not all compiled from source. This goes against overall goal of source-build.

Any changes made to such packages in corefx do not flow through. As an example, this means additional runtime IDs added to Microsoft.NETCore.Platforms are not included in the final built cli.

Fix that by passing a flag to indicate all of CoreFX should be built.
Fixes #650

Yeah I do think we get ourselves into trouble trying to consume packages during the build of a repo that produces the same version'ed package.
This is a tricky one as we have conflicting requirements, as our official shipping branches only want to version and build what we make changes to, but source build wants to be able to always build. I'm not sure what our best strategy is for handling these situations in general.
For the Platforms package in particular I think the easiest fix is to bump the version of the package, even if we don't plan to ship it we can keep the higher version in source to allow for source-build to work. Does that work for your scenarios @omajid?

For the Platforms package in particular I think the easiest fix is to bump the version of the package, even if we don't plan to ship it we can keep the higher version in source to allow for source-build to work. Does that work for your scenarios @omajid?

My scenario is really bootstrapping new (or updated) versions of Linux distributions. Take an example: Fedora 29. This is not present in the current runtime.json file for the 2.1 branches. I would like to:

Add fedora.29 to the runtime.json

Build a version of .NET Core, say 2.1.2, that corresponds as closely to Microsoft's release of .NET Core 2.1.2 as possible with the exception that it knows about this different runtime id (and so can run on Fedora 29).

Does "shipping" a higher version of Microsoft.NETCore.Platforms into a Linux distribution cause any issues down the road? If this works, I am completely okay with bumping the version of the Platforms package.

Does "shipping" a higher version of Microsoft.NETCore.Platforms into a Linux distribution cause any issues down the road?

Depends on what shipping it means :) If by shipping you only have it show up in the shared framework deps.json file then I don't think anything bad will happen. If however you include your version of Microsoft.NETCore.Platforms on a nuget feed or cache that might conflict with the same version when we ship it later. I think to get what you want you don't really need to ship the nuget package you just need it to transfer the RID graph into the shared framework, so I think bumping the version will work fine for you.
For my own curiosity is there a reason you need to add a new RID for a new version of an existing platform? I would expect it to fallback to the older version if the new one isn't found. Only reason you should need a new RID is if it is a complete new family or if you have breaking changes that will not run on the older version and thus you need a new version to target.

Depends on what shipping it means

I think you have it right: I am going to stick this in the shared framework that's used when running with .NET Core supplied on Fedora. So

For my own curiosity is there a reason you need to add a new RID for a new version of an existing platform? I would expect it to fallback to the older version if the new one isn't found.

Hm. That's interesting. I thought runtime ids were opaque strings and so something like rhel.7.8 will not fallback to rhel.7.7 or rhel.7 without explicit entries in the runtime graph?

Hm. That's interesting. I thought runtime ids were opaque strings and so something like rhel.7.8 will not fallback to rhel.7.7 or rhel.7 without explicit entries in the runtime graph?

You're correct I misspoke because I confused myself for a minute. They do have a fallback but it drops all the way back to "linux". See https://github.com/dotnet/core-setup/pull/1689. Perhaps that will work for you still but maybe not depending on the scenario.

@dotnet-bot test RHEL7.2 Unshared Release please (build expired)

Online tarball builds are obsolete

Aaah. This explains a lot. The change worked locally for me (tarball builds), but was failing in CI. What's the difference between an "online tarball build" and a regular tarball build (./build-source-tarball.sh + build.sh inside the tarball)?

I don't think there's any (intentional) difference except the offline tarball leg disables the network, looking at https://github.com/dotnet/source-build/pull/702/files.
I'm not sure if I'm comfortable merging this without the online one successful, since that means you must disable networking or the tarball will fail to build. Makes sense why disabling the network would cause the problem not to show up, due to the conflict between what's on the feed vs. what was built locally, but that makes building the tarballs not very approachable. Since issues like this can show up, I'm not so sure anymore that we should even remove the online tarball legs.

Yeah, that seems like we should add the legs back then.  I will revert the CI change.

I'm not sure if I'm comfortable merging this without the online one successful, since that means you must disable networking or the tarball will fail to build.

I agree. I don't like this change if it means users must be offline for it to work.

Closing and reopening to get new CI.

This issue is still repro'ing locally for me, let's see if it shows up in CI again...
@dotnet-bot test CentOS7.1 Tarball Release please
@dotnet-bot test RHEL7.2 Tarball Release please

Now I'm seeing it in CI as well.
@dotnet-bot test RHEL7.2 Tarball Release please

@dotnet-bot test CentOS7.1 Unshared Release please (restore failure, but a different one from usual: Decrypt failed with OpenSSL error - SSL_ERROR_SSL.     error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac)

Thanks, @crummel !

No problem, thanks for getting this started!


Merge pull request #2 from cqfn/master

Merge to head

@rultor merge

@rultor merge

@kniazkov OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@kniazkov Done! FYI, the full log is here (took me 9min)

@rultor merge

@kniazkov The pull request is closed already, so I can't merge it

@rultor merge

@kniazkov I'm sorry, I don't understand you :( Check this page and try again please


What encryption algorithm is used?

I want to create own SDK in python, but what encryption algorithm does this project use? I cant find this info.

The best place to look to the SDK is on the repos, all the encryption is implemented there:
Rust lang: https://github.com/FuelLabs/fuels-rs
TypeScript: https://github.com/FuelLabs/fuels-ts

Hi @Jaammerr
I would look at the fuel-specs.  First would be getting the ABI encoding and decoding.
Public-Key Cryptography is here too

Hopefully, @QuinnLee answered your question, @Jaammerr. Closing the issue.


CLOUD-48916 change azure platform name to AZURE_RM

Jenkins build finished, all tests passed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/1877/

Jenkins build finished, all tests passed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/1878/


Replace manual Vision library with GAPIC and helpers

This PR replaces the existing Vision client library with an
autogenerated library supplemented with helpers, following the reference
of NodeJS and
Python.
As such, it substantially changes the public surface.
(See also #1828 for analogous change to Language.)
cc: @jbolinger @lukesneeringer

@frankyn Will you be handling a migration guide for this like with the Language migration? If so, would you be able to review as well?

Note: I plan to update the tests to use the standard acceptance/data directory.

@geigerj, summoning @remi.

@remi has been summoned!
I'm checking this out & I'll update the GAPIC Vision Ruby code samples to work using this and see how the code looks 😄

Will we wait on approving/merging until external documentation is ready?

@blowmage I would not delay on approving/merging, although I defer to @remi on release timing.

I would prefer to have a Migration Guide published at cloud.google.com/vision/docs before publishing a new google-cloud-vision gem
Whether this is merged before then is up to y'all!
trying out the library...

Is there an estimate for when external documentation would be updated?

@remi Agreed. It would also be best to get the Migration Guide up first before publishing so that we can link to it from the docs/post-install message when we do publish the gem, as we did with Language.

@geigerj I just saw "Allow Hash shorthand for features" 😄
Trying out the latest... I'll have my LGTM or any additional requests in by EOD

I'd like to synchronize the gem publish && new samples on website && migration guide
I updated the samples & am making the Migration Guide now

@remi 👍 I am going to merge this PR. Will wait for docs update before publishing.

Is there an ETA for when this can be released?


Find better name for singleton

What the types does is not what is commonly understood as the singleton pattern.
It is just an instance that has a constant storage key and allows you to operate on that key without repeating the name if the key.
I don't have a great idea at the moment. Just writing it down for later.

In Java, the term "singleton" also also used for collection constructors with one element: Collections.singletonList, Collections.singletonMap and Collections.singleton (set). This is more or less the same that is happening here.


PR #1 - Add search vendors around bar and update dependencies

What does this PR do?
This first pull request for the search bar do basically two main things:

Fix dependency issue with Jquery and include project's script.js.
Implement the bar/card to search the vendors around (just HTML skeleton) .

Why it is needed?

The Jquery .min.js file did not include AJAX and in order to import external HTML files we need that library. We want to import external HTML files because we want to separate the repeated elements such as Navbar, Footer, Search from all the HTML files.
The search vendors around allows the user to retrieve the nearest vendors that matches user's query.

How?

Importing a new Jquery file and including it on every html.
Adding bootstrap's forms and make them ready for querying.

Testing?

Using AJAX's exclusive functions without having any error.
Visual test only (just HTML changes).

Anything Else?
This PR shows how we can separate commons HTML files and import them to improve readability.
Looking forward to start the servlets for the search bar feature and its respective JS functions.

LGTM! Just one thing. Google's JavaScript style guide mentions that all file names must be in lowercase. Is it ok to change views/COMMONS to views/commons ?

Does that apply to all files? I thought it was only for *.js. :o

LGTM! Just one thing. Google's JavaScript style guide mentions that all file names must be in lowercase. Is it ok to change views/COMMONS to views/commons ?

Does that apply to all files? I thought it was only for *.js. :o

The Google HTML/CSS style guide doesn't mention anything about file names, but I think it would keep consistency through all the project if we use the JS file name convention, what do you think?

LGTM! Just one thing. Google's JavaScript style guide mentions that all file names must be in lowercase. Is it ok to change views/COMMONS to views/commons ?

Does that apply to all files? I thought it was only for *.js. :o

The Google HTML/CSS style guide doesn't mention anything about file names, but I think it would keep consistency through all the project if we use the JS file name convention, what do you think?

Booth options seems ok to me, I set the name in uppercase because I was thinking in the Global variables, but no problem changing the name


Use new Helix queue for Windows CI builds.

Skip CI please.

@dotnet-bot test CI please

cc @A-And @jkotas @morganbr

@crummel, do we need to use the Wasm queue for all Windows builds or just the Wasm flavor (see #5845)?

@crummel this is broken because Java is not on the path on the machines.

@morganbr the machines are updated, should try again if you have not already done so..

@morganbr Does that look better?

@dotnet-bot test CI please

I can't comment on that line on GitHub, but there's a check for Windows_NT around this comment // This opens the set of build steps that will be run. that we'll probably need to change from (os == 'Windows_NT') to "starts with Windows_NT".

We'll probably also need to disable the CoreCLR and CoreFX tests for now.

@dotnet-bot build and test Windows_NT Debug WebAssembly please

@dotnet-bot test this please

@dotnet-bot build and test Windows_NT Debug WebAssembly please

@morganbr There's no point trying to run these. Without addressing what I wrote in https://github.com/dotnet/corert/pull/5858#issuecomment-400824121 these won't work.

@dotnet-bot test CI please

@dotnet-bot build and test Windows_NT Debug WebAssembly please

@dotnet-bot build and test Windows_NT Debug WebAssembly please

@dotnet-bot test Windows_NT_Wasm Debug and CoreCLR tests WebAssembly

@crummel, does the queue actually have the new image hooked up to it? This is the same failure as the last image that would indicate emsdk activate latest --global wasn't run on it.

@dotnet/dnceng or @MattGal will hopefully be able to confirm.

@morganbr can confirm that was the older image, please try again.  (ran on wasmw10gt000011, new agenst start at 01N)
@dotnet-bot Linked test Windows_NT_Wasm Debug and CoreCLR tests WebAssembly

@morganbr looks like the new image doesn't work.  I'll work with the FRs to help understand this.

@dotnet-bot test CI please

@dotnet-bot test Windows_NT_Wasm Debug and CoreCLR tests WebAssembly

@dotnet-bot test Windows_NT_Wasm Debug and CoreCLR tests WebAssembly

Thanks, @crummel! I made a couple of minor tweaks to this in #5845 and merged.


Fix footer display

I did not notice before, but the recent pydata sphinx theme versions had unset our footer. I will send a PR later to fix the 0.5.0 docs in napari.github.io, but this fixes the immediate issues and changes the wording on the footer items as requested in #12 .

What do you think of Forum for image.sc and Ask a Q for Zulip?
Seems more natural?
Also, I recall there being some opposition to "Chat" due to connotations with AI chatbots.

We talked about it at the meeting and kinda decided on chat if I remember correctly? But yes, forum would be my first choice, I just followed the existing suggestions from the issue 😊

I'm ambivalent about the chat part, but rather strongly feel that image.sc should be labeled as forum.

I was the one with the concern about using chat, but I have actually revised that and am in favour as I think it's the option that makes most sense and is pretty unambiguous.
It's also totally different to how most chat bots are shown, with the little bubble in the right hand corner that inevitably makes some Clippy-like remark about helping you out.

I don't mind "ask a q" vs "forum" for image.sc, so I'm not gonna press merge. Since Peter feels strongly about Forum and it's shorter, let's go with that. I'm happy we got consensus on chat. 😊

Updated


How do we use get_all_message_ids_in_chat()?

Using the included docker file with Python 3.7.
I'm trying to get all messages from all chats like this:
def get_all_chats_and_messages(driver):
    print('------------------ OLD CHATS')

    for chat in driver.get_all_chats():
        print('----- CHAT -----', chat.id, '-----------------------------')
        for message_id in driver.get_all_message_ids_in_chat(chat):
            print(message_id)

But this is the result:
------------------ OLD CHATS
----- CHAT -----<EMAIL_ADDRESS>-----------------------------
<EMAIL_ADDRESS>----- CHAT -----<EMAIL_ADDRESS>-----------------------------
<EMAIL_ADDRESS>----- CHAT -----<EMAIL_ADDRESS>-----------------------------
<EMAIL_ADDRESS>----- CHAT -----<EMAIL_ADDRESS>-----------------------------
<EMAIL_ADDRESS>----- CHAT -----<EMAIL_ADDRESS>-----------------------------
<EMAIL_ADDRESS>
As you can see only one message from each chat is being printed.

there is you get all chats ids. If you want to return all messages from chat, use get_all_messages_in_chat by ids in foreach loop

And, I can say for a fact that, it is erratic: sometimes it is able to get some messages, never all of them, some other only the first one or last one, IDK...

I've also noticed erratic functioning from both chat.get_messages and from driver.get_all_messages_ids_in_chat(chat).
Used two phones to test against. In the chat with only 5 chats all went always smooth, including chat.get_all_earlier_messages(), etc, but in the phone with thousand contacts it did fail.


LGTM

I prefer the strict version actually, because with this one I'm likely to see the warnings but just commit things anyway. I was already in the habit of copying my commit message in case it got shot down...

Could we at least echo back the message on error so that we can copy/paste it back in and then edit?

Could we at least echo back the message on error so that we can copy/paste it back in and then edit?

:+1: I think failing hard and outputting the offending commit message is probably a reasonable middle ground.

I'm ok with this change if this is what others would prefer. I can always tweak my local commit hook to be stricter.

My logic for making everything a warning was I figured a big block of red text was pretty obvious, and it is easier to amend a commit message if you mess it up than it is to copy and paste it and then fix it.
I don't have a strong opinion either way, as copying and pasting the commit message isn't too much work :trollface:.

Code LGTM.


Static page build on Render.com fails with "ctx.route.replaceAll is not a function"

Hello!
I am having issues generating static website on Render.com after configuring this sitemap module. It works with no issues on my local machine. Could you please help me with that? Thank you 🙏🏻
nuxt.config.ts
modules: [
  ['@funken-studio/sitemap-nuxt-3', {
    generateOnBuild: true,
    hostname: 'https://domain.com',
    gzip: true,
  }],
],

Build Command: npm run generate (nuxt generate)
Publish Directory: .output/public
Build log:
Jan 12 05:43:19 PM  ℹ Client built in 6772ms
Jan 12 05:43:19 PM  ℹ Building server...
Jan 12 05:43:23 PM  ✔ Server built in 4058ms
Jan 12 05:43:23 PM  ✔ Generated public .output/public
Jan 12 05:43:23 PM  ℹ Initializing prerenderer
Jan 12 05:43:25 PM  ℹ Prerendering 6 initial routes with crawler
Jan 12 05:43:25 PM    ├─ / (135ms)
Jan 12 05:43:25 PM    ├─ /page-1 (22ms)
Jan 12 05:43:25 PM    ├─ /page-2 (19ms)
Jan 12 05:43:25 PM    ├─ /page-3 (19ms)
Jan 12 05:43:25 PM
Jan 12 05:43:25 PM   ERROR  ctx.route.replaceAll is not a function
Jan 12 05:43:25 PM
Jan 12 05:43:25 PM    at node_modules/@funken-studio/sitemap-nuxt-3/dist/module.mjs:419:92
Jan 12 05:43:25 PM    at node_modules/hookable/dist/index.mjs:39:82
Jan 12 05:43:25 PM    at async prerender (node_modules/nitropack/dist/shared/nitro.c8278d90.mjs:2961:7)
Jan 12 05:43:25 PM    at async node_modules/nuxt/dist/index.mjs:1675:7
Jan 12 05:43:25 PM    at async build (node_modules/nuxt/dist/index.mjs:2236:5)
Jan 12 05:43:25 PM    at async Object.invoke (node_modules/nuxi/dist/chunks/build.mjs:56:5)
Jan 12 05:43:25 PM    at async Object.invoke (node_modules/nuxi/dist/chunks/generate.mjs:32:5)
Jan 12 05:43:25 PM    at async _main (node_modules/nuxi/dist/cli.mjs:50:20)
Jan 12 05:43:25 PM
Jan 12 05:43:26 PM  npm ERR! code ELIFECYCLE
Jan 12 05:43:26 PM  npm ERR! errno 1
Jan 12 05:43:26 PM  npm ERR! @ generate: `nuxt generate`
Jan 12 05:43:26 PM  npm ERR! Exit status 1
Jan 12 05:43:26 PM  npm ERR!
Jan 12 05:43:26 PM  npm ERR! Failed at the @ generate script.
Jan 12 05:43:26 PM  npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
Jan 12 05:43:26 PM
Jan 12 05:43:26 PM  npm ERR! A complete log of this run can be found in:
Jan 12 05:43:26 PM  npm ERR!     /opt/render/.cache/_logs/2023-01-12T16_43_26_067Z-debug.log
Jan 12 05:43:26 PM  ==> Build failed 😞

Running into the same issue. Have you found a solution for this?

Okay, found the root cause and the solution. Render.com uses Node version 14.17.0 when you don't specify something yourself. The replaceAll function was introduced in version 15 if I'm not mistaken. So, in order to resolve this you need to create a file in the root of your directory called .node-version  and add a single line saying lts. You can find more details here
Hope this helps!

@StefanVDWeide Good point! Works for me as well with the .node-version file. Thank you for resolving this. I am closing this one as according the Nuxt 3 docs, we should be using 16.x version.


Add in claim by same order for Relics

Currently, Relics can only be won back by the Relics Realm.
We need to update this so that any Realm of the same order can actually claim back the relic for the original owner.
set_relic_holder -> update this function
We will need to recurse over the relics owned by the losing Realm and then return all of them back to their rightful owners.

To add to this task.
On the unsettling of a Realm, we must send back all the relics to their original owners. If we do not do this, then relics will be stuck in limbo.


Add support for parts  CR 10-19

Description of the change: (free-text)
We should be able to convey part metadata via ninjs.
In NewsML-G2, items can have many wrappers, each expressing properties for an identifiably separate part of the content; for example each of the shots, or segments, which make up a video.
Each part should have properties for:

an ID for the part
a sequence number
an icon that may help to visually identify the content of the part
the start time and duration of the segment
And all of the other ninjs properties, except for renditions and associations.

To compare with partMeta in NewsML-G2 / IPTC News Architecture:
Here's a multi-part video example:
<?xml version="1.0" encoding="ISO-8859-1"?>
<newsItem
    xmlns="http://iptc.org/std/nar/2006-10-01/"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://iptc.org/std/nar/2006-10-01/
        ./NewsML-G2_2.28-spec-All-Power.xsd"
    standardversion="2.28"
    guid="tag:example.com,2008:407624"
    version="12"
    standard="NewsML-G2"
    conformance="power"
    xml:lang="en">
    <catalogRef
        href="http://www.iptc.org/std/catalog/catalog.IPTC-G2-Standards_34.xml" />
    <catalogRef
        href="http://www.example.com/metadata/newsml-g2/catalog.NewsML-G2.xml" />
    <rightsInfo>
        <usageTerms>
            Access only for Eurovision Members and EVN / EVS Sub-Licensees.

            Coverage cannot be used by a national competitor of the contributing
            broadcaster.
        </usageTerms>
    </rightsInfo>
    <itemMeta>
        <itemClass qcode="ninat:video" />
        <provider qcode="providercode:EBU">
            <name>European Broadcasting Union - EVN</name>
            <organisationDetails>
                <contactInfo>
                   <web>http://www.eurovision.net</web>
                   <phone>+41 22 717 2869</phone>
<EMAIL_ADDRESS>                   <address role="AddressType:Office">
                       <line>Eurovision Sports News Exchanges</line>
                       <line>L Ancienne Route 17 A</line>
                       <line>CH-1218</line>
                       <locality>
                           <name>Grand-Saconnex</name>
                       </locality>
                       <country qcode="ISOCountryCode:ch">
                           <name xml:lang="en">Switzerland</name>
                       </country>
                   </address>
                </contactInfo>
            </organisationDetails>
        </provider>
        <versionCreated>2019-10-19T10:54:04Z</versionCreated>
        <firstCreated>2008-11-06T10:22:28Z</firstCreated>
        <pubStatus qcode="stat:usable" />
        <service qcode="servicecode:EUROVISION">
            <name>Eurovision services</name>
        </service>
        <edNote>Originally broadcast in Germany</edNote>
        <link rel="irel:associatedWith"
            href="http://www.example.com/video/407624/index.html"/>
    </itemMeta>
    <contentMeta>
        <contentCreated> 2008-11-05T19:04:00-08:00</contentCreated>
        <located type="cptype:city" qcode="city:345678">
            <name>Berlin</name>
            <broader type="cptype:statprov" qcode="state:2365">
                <name>Berlin</name>
            </broader>
            <broader type="cptype:country" qcode="iso3166-1a2:DE">
                <name>Germany</name>
            </broader>
        </located>
        <creator qcode="codesource:DEZDF">
            <name>Zweites Deutsches Fernsehen</name>
            <organisationDetails>
                <location>
                   <name>MAINZ</name>
                </location>
            </organisationDetails>
        </creator>
        <contributor qcode="codeorigin:DEZDF" role="rolecode:TechnicalOrigin">
            <name>Zweites Deutsches Fernsehen</name>
        </contributor>
        <creator qcode="codesource:GBRTV">
            <name>Reuters Television Ltd</name>
        </creator>
        <language tag="en" role="langusecode:VoiceOver" >
            <name>English</name>
        </language>
        <genre qcode="genre:biog">
            <name xml:lang="en-GB">Biography</name>
            <name xml:lang="fr">biographie</name>
        </genre>
        <subject type="cpnat:abstract" qcode="medtop:01000000">
            <name xml:lang="en-GB">Arts, Culture and Entertainment</name>
            <name xml:lang="fr">Arts, culture, et spectacles</name>
            <narrower type="cpnat:abstract" qcode="medtop:20000003">
                <name xml:lang="en-GB">Animation</name>
                <name xml:lang="fr">Dessin animé</name>
            </narrower>
        </subject>
        <headline>Loriot retrospective</headline>
        <description role="descrole:dopesheet">
            Yesterday evening (November 5) an exhibition opened in Berlin in
            honour of German humorist Vicco von Bülow, better known under the
            pseudonym "Loriot", to commemorate his 85th birthday. He was born
            November 12, 1923 in Brandenburg an der Havel and comes from an old
            German aristocratic family. He is most well-known for his cartoons,
            television sketches alongside late German actress Evelyn Hammann and
            a couple of movies. Under the name "Loriot" in 1971 he created a
            cartoon dog named "Wum", which he voice acted himself. In 1976 the
            first episode of the TV series "Loriot" was produced.

        </description>
        <description role="descrole:shotlist">
            Berlin, 05/11/2008

            - vs. Vicco von Bülow entering exhibition

            - vs. Loriot and media

            - sot Vicco von Bülow

            "Since 85 years I didn't succeed in pursuing a job that could be
            called a profession."

            - vs exhibition

            - sot Irm Herrmann, actress

            "Loriot is timeless. You always can watch him and I can always
            laugh."

            - actor Ulrich Matthes in exhibition

            sot Ulrich Matthes, actor

            " I would say: one of the great German classics. Goethe, Kleist,
            Schiller, Thomas Mann, Loriot. That's the way I would say it."
             Goethe, Kleist,
            Schiller, Thomas Mann, Loriot. That's the way I would say it."</description>
    </partMeta>
    <contentSet>
        <remoteContent href="http://www.example.com/video/407624.avi"
            rendition="vidrnd:avi25"
            format="fmt:avi"
        duration="111" durationunit="timeunit:seconds"
        videocodec="vcdc:c155"
        videoframerate="25"
        videoaspectratio="16:9" />
    </contentSet>
</newsItem>

As an example,
<partMeta partid="Part1_ID" seq="1">
    <icon href=" http://www.example.com/video/Keyframes/407624.jpeg"/>
    <timeDelim renditionref="vidrnd:avi25" start="0" end="446" timeunit="timeunit:editUnit"/>
    <language tag="en" role="langusecode:VoiceOver" />
    <description>Vicco von Bülow entering exhibition </description>
</partMeta>

could become
"partmeta": [
{
"partid": "Part1_ID - should be a URI?",
"sequencenumber": 1,
"icon": {
"href": "http://www.example.com/video/Keyframes/407624.jpeg" // could this be a rendition instead?
},
"timedelimiter": {
"renditionref": "http://example.com/vidrnd/avi25", // refers to a rendition block given elsewhere?
"start": "0",
"end": "446",
"timeunit": "http://cv.iptc.org/newscodes/timeunit/editUnit",
},
"language": "EN",
"languagerole": "http://cv.iptc.org/newscodes/languagerole/voiceOver",
"description": "Vicco von Bülow entering exhibition"
}
]

Questions:

Should we declare defaults for things like timeunit, to make things simpler?
The original request says we can include anything in ninjs "except for associations and renditions". But why would we want to ignore those? A part of a video might be associated with a URL, and a rendition could be used to represent a thumbnail. Which leads to the next question:
Should we just use renditions instead of icon?
If we want users to include any valid ninjs property in a part, then we have two choices:

To have some declared properties such as "partid", "sequencenumber" and "timedelimiter", and then a reference to a child object which can contain any valid ninjs property (what to call it?)
Or to put the new properties into the main ninjs schema as top-level properties, and just make "parts" a list of ninjs objects, the same way "associations" is defined.

Use cases for parts:

Spatial parts (G2 construct is regionDelim):

Image regions - defining people and products in an image
Recommending crop boundaries for different devices / sizes

Time-based parts (G2 construct is timeDelim):

describing segments of a radio broadcast
declaring rights and copyright ownership of parts of a video

Would we need to declare metadata for a channel in a video stream? E.g. different channels of an MPEG file containing

Notes:

We should allow overlapping parts - image regions will often overlap, and audio streams may overlap too
We could (should?) allow parts to be defined by both space and time - e.g. someone's face during an interview (as long as the person and the camera don't move!)

Further research:

W3C timed text formats: Here's an example.
https://github.com/w3c/IMSC-1.1_Text_TestContent/blob/master/IMSC1-1_TEXT_Test-Reel_FMS_v4-0-1_2019-11-20.xml

Could subtitles / closed captions / audio description be a use case for parts in ninjs files?
For comparison, W3C has defined a timed text format, IMSC, based on TTML.
Here's an example.
https://github.com/w3c/IMSC-1.1_Text_TestContent/blob/master/IMSC1-1_TEXT_Test-Reel_FMS_v4-0-1_2019-11-20.xml

This issue came up again today when we discussed a requirement from TV2 Danmark.
We discussed W3C Media Fragments which are simple URI extensions that cover both temporal and spatial fragments: https://www.w3.org/TR/media-frags/
Examples of temporal fragments:
http://ninsuna.elis.ugent.be/DownloadServlet/mfwg/fragf2f.ogv?t=21&track='ogg_1'
http://ninsuna.elis.ugent.be/DownloadServlet/mfwg/fragf2f.ogv?t=npt:0:00:21.0
http://ninsuna.elis.ugent.be/DownloadServlet/mfwg/fragf2f.ogv?t=smpte-25:00:00:21:00&track='ogg_2'

"partmetas": [
{
"partid": "Part1_ID - should be a URI?",
"icons": [{
"href": "http://www.example.com/video/Keyframes/407624.jpeg" // could this be a rendition instead?
}],
"regiondelimiters": [
{
"renditionref": "http://example.com/vidrnd/avi25", // refers to a rendition block given elsewhere?
"x":  25,
"y": 25,
"width": 100,
"height": 200
}
],
"language": "EN",
"languagerole": "http://cv.iptc.org/newscodes/languagerole/voiceOver",
"description": "Vicco von Bülow entering exhibition"
}
]

Questions raised today:

Which properties can be used inside a partMeta object (element of the partMeta array)? On one hand, we could allow anything in the ninjs schema, the same as we do in associations - but that's probably going too far. On the other, we could specify which properties can be re-used inside a partMeta and referring to their existing definition.
What would the link look like - the value of the renditionref property? @iyoung mentions JSONPath, it works well for them internally.


[Easy Dictionary] open ai custom system prompt setting support

Extension
https://www.raycast.com/isfeng/easydict
Description
open ai custom system prompt setting support
Who will benefit from this feature?
No response
Anything else?
No response

PR wanted 🥺


Ubuntu crashes on file read

Windows Version
Microsoft Windows [Version 10.0.22621.1555]
WSL Version
<IP_ADDRESS>
Are you using WSL 1 or WSL 2?

[X] WSL 2
[ ] WSL 1

Kernel Version
<IP_ADDRESS>
Distro Version
Ubuntu 22.04
Other Software
R version 4.1.2
Repro Steps
[In Ubuntu terminal:]
R
[Within R]
load("my_8_gb_Rdata_file.RData")
[Ubuntu crashes, terminal window disappears]
The error seems to happen when reading any large Rdata file from any location. These RData files were saved from Windows R, but that should not prevent their being read in by Linux.
Expected Behavior
I would expect that the file would be loaded into R, as happens when I run the same code on Windows; if that didn't work for some reason, I would expect R to throw an error ("file does not exist" if that were the problem, or "system memory exceeded") --the point is that a failure to read the file should be contained in R; the fact that it is causing Ubuntu to crash indicates that this is not a problem with the R code.
Actual Behavior
Ubuntu terminal suddenly disappears with no error or warning.
Diagnostic Logs
No response

/logs


When the viewer area is less than a quarter of the tile area, the SVG filter fails

I am working on a project based on qt5, using SVG filters. Everything's fine, except when viewer's area is less than a quarter of tile's area, SVG filters don't work .
I need to drag zoom in or zoom out the viewer to get SVG to work again.
Part of the code is as follows:
<filter id="rgb">
		<feColorMatrix type="matrix" values=" 1 0 0 0 0
                                                      0 1 0 0 0
                                                      0 0 1 0 0
                                                      0 0 0 1 0"
						   color-interpolation-filters="sRGB">
		</feColorMatrix>
</filter>

document.getElementsById("viewer").style.webkitFilter= "url(#rgb)";

Wow... I've never thought of trying that! So this is an otherwise generic OpenSeadragon viewer, and you're applying this SVG filter over it? What browsers are you using? How's the performance (just curious)? Is the problem you are encountering consistent across browsers?
I think in general people use https://github.com/usnistgov/OpenSeadragonFiltering to do this kind of filtering. Would be neat if the SVG filter could work as another option.
Could you share a link to an example with the issue, like a CodePen or something?


Add Initial Conformance Test Framework

We want to ensure that controllers using these APIs are conformant. Having a good conformance test suite will help establish consistency across community implementations and help developers identify any missing pieces in their implementation. This should work similarly to the existing Kubernetes conformance tests and be accessible with the same workflows.

The current conformance suite was proposed and accepted on the mailing list
https://groups.google.com/d/msg/kubernetes-sig-network/dAt78O-JUCo/vvOXt3-XCgAJ
kubernetes/org#1328
Conformance testing needs to test not just the v2 API, but also Ingress v1beta1 and v1. I am not sure that it should live in this repo, since it has a broader scope. I would prefer to avoid building 2 separate suites and ending up with different overlapping tests.
Probably @aledbf and @alexgervais should discuss?

When https://github.com/kubernetes-sigs/ingress-controller-conformance started, it was intended to support v1beta1 and v1 Ingress APIs.
It is desirable to have a single conformance test suite for v1beta1, v1 Ingress and v2 Service APIs.  We have not committed to using the sonobuoy in kubernetes-sigs/ingress-controller-conformance, but building on a common ground would make total sense. The worst possible thing would be to completely diverge on approaches here and have the 2 suites diverge so early in their inception.
@aledbf , thoughts?

xref kubernetes-sigs/ingress-controller-conformance#29

/remove-lifecycle stale

/remove-lifecycle stale
/lifecycle frozen

I think this issue was resolved by #969


Program to Add Two Matrices

Program to Add two Matrices in C.

Please go ahead with your changes


463 broke everything

So, 30 min ago I got the dev build 463 and the whole thing broke down.
I was mostly using the resizing feature and the autostop.
Now the resizing doesn't work at all. the player is still small even if YTC is set to Fit to content and it's largish in size, as I've set it up, almost whole screen. I've set the player to 980px but it's only 850.
Autoplay doesn't fire on every instance and it's not the HTML5 bug, I've set HTML5 video disabled through an extension. The player always load in Flash (forced flash).
Config button is missing from the pages, I can find it in settings page, but not on all pages, or the user bubble menu.

Update to 466. Dupe of #1386.

Running 466 and still have this same issue.

Try updating to 467.

If you're still experiencing the issue dump your console (CTRL + SHIFT + J and CTRL + SHIFT + K).


Fixed: Incorrect user in footer of profile

Footer had previously shown targeted user instead of author.

poggers


Slow autocomplete for ssh config host names

Autocompletion for server names specified in ~/.ssh/config works, but is slow. It takes a few seconds to autocomplete the server names. Other autocomplete functionality is working instantly, only the ssh config autocomplete functionaltity is very slow. On native linux this is working instantly.
My windows build: 10.0.15046 (however, I have had this on all builds so far)

For what it's worth, on my box, it seems bash_completion is set to complete for machines that are reporting their uptime but the package which actually manages that was not installed by default.
Ref from /usr/share/bash-completion/bash_completion:
    # Add hosts reported by ruptime.
    COMPREPLY+=( $( compgen -W \
        "$( ruptime 2>/dev/null | awk '!/^ruptime:/ { print $1 }' )" \
        -- "$cur" ) )

The package rwho package provides ruptime so you can just sudo apt install rwho to get that binary.
Or you can just comment out the part of the script that deals with polling ruptime.

Installing rwho worked great on Ubuntu Xenial.
Not sure that this issue is specific to Microsoft WSL....
I've been suffering from this problem for more than a year, and I just finally got frustrated enough to look for a solution by searching for "ssh autocomplete slow" and found this.
Thank you!

Not sure that this issue is specific to Microsoft WSL....

Possibly. Performance (if that is the word) seems alright on the bash-completion that ships with 20.04 on recent WSL1, without having installed rwho.

Someone inside the intersection of (a) able to reproduce the behavior and (b) highly motivated, could do an strace -tt -f -o bash.strace /bin/bash and see where bash is spending its time. But whatever it is doing, it is (at the end of the day) going to take whatever time it takes.


DocArray v2 third alpha release note

DocArray v2 alpha release
DocArray v2 has its third alpha release as planned in the roadmap
What is new ?
Tensorflow support (#1064) (#1098)
In this version we added TensorFlow support to v2.
Most importantly, this includes a TensorFlowTensor class and a corresponding TensorFlowCompBackend.
import tensorflow as tf

from docarray import BaseDocument, DocumentArray
from docarray.typing import TensorFlowTensor

class MyDoc(BaseDocument):
    title: str
    tensor: TensorFlowTensor

da = DocumentArray[MyDoc](
    MyDoc(title=f'hello {i}', tensor=tf.zeros((224, 224, 3))) for i in range(100)
)

Pretty printing with rich (#1043)
Add pretty print and .summary() for Document's as well as DocumentArray's:
For a Document:

For a DocumentArray:

Display of different multi modal data type (#1113) (#1136)
You can now display your multi modal data with our predefined documents and types from a notebook! This applies to audio, image, video, as well as 3D data.
You can simply call .display() on the Documents url or its tensor(s):
For PointCloud3D:
doc = PointCloud3D(url='tests/toydata/tetrahedron.obj')
doc.tensors = doc.url.load(samples=10000)
doc.tensors.display()
# or via url
doc.url.display()

For Mesh3D:
doc = Mesh3D(url='tests/toydata/tetrahedron.obj')
doc.tensors = doc.url.load()
doc.tensors.display()
# or via url
doc.url.display()

Pytorch Multi Modal dataset
You can now easily utilise DocumentArrays in PyTorch training scripts using MultiModalDataset.
All you need is a DocumentArray and a dictionary of preprocessing functions and you’re up and running.
from torch.utils.data import DataLoader
from docarray import DocumentArray, BaseDocument
from docarray.data import MultiModalDataset
from docarray.documents import Text

class Thesis(BaseDocument):
    title: Text

class Student(BaseDocument):
    thesis: Thesis

da: DocumentArray[Student] = get_students()
ds: MultiModalDataset[Student] = MultiModalDataset[Student](da, preprocessing={"thesis.title": embed_title, "thesis": normalize_embedding})
loader: DataLoader = DataLoader(ds, batch_size=4, collate_fn=MultiModalDataset[Student].collate_fn)

# Use your loader just like any other dataloader for awesome DL training

More serialization options

Is this still a WIP? I see a title at the bottom with nothing below it

yes still wip

OK, ping me when ready for me to review

As discuss before we won't need this release anymore


TCPGet: Separated the node.connected property for each instance

Coverage remained the same at 88.015% when pulling 57bbfcda35895d8dba654b84484502b86b3b02c2 on natcl:tcpgetfix into 74a1713e993ad2ee165ecafeb34bf309a0c3e542 on node-red:master.


Unable to open HomeActivity

After building project is success, the app crashes immediately right after launching.

java.lang.RuntimeException: Unable to start activity ComponentInfo{com.saulmm.cui/com.saulmm.cui.HomeActivity}: java.lang.IllegalStateException: You need to use a Theme.AppCompat theme (or descendant) with this activity.

If you build from this commit which is the head in master at a time, you shouldn't have that problem.
That used to happen when your activity is inheriting from AppCompatActivity but the theme set to it is not related with AppCompat
i.e:
This will crash firing your exception
<style name="AppTheme" parent="@android:style/Theme.Material">
HomeActivity extends AppCompatActivity
This will work
<style name="AppTheme" parent="Theme.AppCompat.Light.NoActionBar">
HomeActivity extends AppCompatActivity

I have the same problem. In AdnroidManifest.xml, I change
android:theme="@style/AppTheme"

to
<EMAIL_ADDRESS>
It work for me.

Yes,AppTheme is declared only for v23. That's why.

@Arkar-Aung is right, what a mistake, solved here. Thanks for the feedback guys


A possible typo in README.md

In the RecyclerView part,
mRecyclerView.setAdapter(new MaterialViewPagerHeaderDecorator());
does it mean
mRecyclerView.addItemDecoration(new MaterialViewPagerHeaderDecorator()); ?

mRecyclerView.setAdapter(new MaterialViewPagerHeaderDecorator());
means add HeaderDecorator with mRecyclerView.
also your mRecyclerView start after HeaderDecorator and scroll with top of HeaderDecorator.
If you not add than HeaderDecorator is behind the Recycleview.


Test parser should retain source line numbers

Originally submitted to Google Code by bryan.oakley on 7 May 2010

It would be nice if the internal robot data model could remember line
numbers from the original file, so that error reports could reference back
to the original file.
Even if exact numbers aren't possible it would still be useful. For
example, if a single keyword call is extended over several lines it might
only save the line number of the first line.
This would make it easy(ish) to write a grep utility that is keyword aware,
and can return file and line numbers to make it easy to jump to the keyword
via a text editor.

Up!

YES!
This lack makes debugging a tiresome slog

It would be great benefit if we have line numbers published in the error file. Scanning the whole 'log' file to locate the error becomes a heavy task when we have huge number of TCs.

I'd really like this too, but I don't think it makes sense with the current parsing architecture and as long as we support the HTML format. We are most likely going to deprecate other than the plain text format in RF 3.1, which would allow us to re-implement parsing in RF 3.2. I cannot make any promises when RF 3.2 would be created (most likely not during 2017) nor will this feature be included then. If this is an important feature for you, please contact me and we can see how you could help getting it done via code contributions or direct sponsorship.

I really would love to know how you can track down test code errors without line numbers. Not only is it massively time consuming not to be able to nail down where a problem is, but it's also decades behind every single other development paradigm in existence. Hell, even BASIC had "?SYNTAX ERROR IN 20".

@keithdtyler When test code (ie. library code) fails, it spits out Python's stack trace which has exactly what line and what module is failing and what the exception was.  This issue is more about having that information embedded in the data model to enable writing eg. your own debuggers

@Tattoo
From your comments, it seems it is possible to print the python's stack when a failure happens in python code. Can you tell me how to do that? thanks

New parser (#3076) ought to make this possible.

This is an awesome feature!
Allows to show this in IDEs and jump to the issue!!

Is it at all possible to get line number information out of a dry-run generated output.xml file?
It does show line numbers for things like unknown variables in the Settings area when importing like
Library    ${test.py}

where ${test.py} should be passed in from the cli and if a library does not exist. But for some reason the other errors like keywords not existing and duplicate keywords don't have any line number information in the outputted xml file.

Line numbers aren't in general saved to output.xml some error messages contain them. Is the problem here that there's an error without them?

Yeah, I'm basically trying to something that can identify where in the script file syntax errors occur - at the moment I'm just showing the html log file generated from a dry-run to the user, however, I'd like to do some inline styling to show exactly where the errors are in the robot ace editor I'm working on. I've tried some hacky ways to parse the dry-run output.xml to grab anything with a line number (which works fine) and then any other kw nodes with a status of FAIL, but these latter ones don't have line number information so I can't display where they are in the editor.
You can see an example output.xml here: https://gist.github.com/SullyJHF/012409a06ab8aa0095448013abd11f22
And the script that goes along with this here: https://gist.github.com/SullyJHF/eb4f46eb6b653055cd88effabe015063
As you can see in the xml file, some things in the <errors> node have line endings (but some don't, see the duplicate keyword message). And also typo'd or just plain missing keywords aren't even in the errors node, and also don't have any line numbers attached to them.
Any assistance on this would be greatly appreciated.

Sounds like an interesting use case. Perhaps you can tell us more about that on Slack or elsewhere.
It seems we missed errors related to creating user keywords when adding line numbers to error messages as part of this issue. I submitted #4201 about that and will fix it soon.
Non-existing keywords are only detected at execution time. Source information hasn't been added to them because you can also see where the error occurred in log file. I'm afraid adding that would make these errors pretty verbose but we can experiment with that. Please submit a separate issue if you think that would be a good idea.

Perfect thanks for this!


Bug Found | Needs Fixing

/cps loaded command not show default allowed protocol
1.type /cps loaded
Desktop (please complete the following information):

OS: Windows
Java Version: 1.8.0_191
Plugin Version: v1.2.4
BungeeCord Version: #1417

Fixed (i know i am crazy)


Visual Studio 2013 bug fixes

I started playing with dukluv project and I had to make few changes in order to make it working out of the box with visual studio 2013. Would you mind to review these changes and see if they make sense to you, and eventually merge them into the project?

This is awesome, thanks!


Update babylon.abstractMesh.ts

Add method "RotateAround", to rotate a mesh around a given point and axis.
We implemented it because it suited our needs, but it might be useful to others.
Not sure the way we compose translation and rotation matrices to then extract their components is the best way to go, but as far as I know their is no "apply transform matrix to mesh" method already implemented. Or is there ?

Thanks for the feedback DeltaKosh,
I get it about the GC optimization, I fixed it but some odd behaviours persist, depending uppon which Tmp.Matrix[index] I pick. I'll investigate it !

Thanks a lot (a good talk for your symposium by the way;))


Rename libgfortran to libfortran-ng?

As of the creation of https://github.com/conda-forge/ctng-compilers-feedstock (AFAICT), the GCC stack switched from libgfortran to libgfortran-ng, and the libgfortran feedstock has been archived for a long time.
I think it would be nice to use the same naming for both linux & osx, i.e. do the same rename here? Alternatively, I'd also be fine to rename the GCC back to libfortran (remove -ng suffix).
CC @conda-forge/ctng-compilers

We decided to keep the libgfortran name but with the SO version in the package name (e.g. libgfortan5).

Of course we could also get rid of all the unversioned libgfortran packages. Currently we're still releasing new builds of libgfortran-ng with every CI run of ctng-compilers.

Right yes, we never finished the transition. I have not thought through how to fully transition. We could simply stop building the -ng packages. We may also want to patch packages that depend on a version of libgfortran-ng where we have a corresponding libgfortranSONAME. Not all versions of libgfortran-ng have this and so we'd either have to increase the min pin for older builds or simply leave them behind in terms of using the latest libgfortran at run time.

Leaving them behind is likely the best solution, but ofc the forward compatibility should make moving the min pin fine since there is never a max pin to worry about within the same SONAME.


Build fails on Flutter 3.24.2

Describe the bug
Build of the flutter project with flutterflow fails on dartTonconect package:

implementation code

Environment (please complete the following information):
OS: ubuntu22..0.4
darttonconnect: ^1.0.3
you can refer this link
https://github.com/jonataslaw/get_cli/issues/263

I didn't resolve it

Good news everyone!
I forked this project https://github.com/aap17/dart_ton_plus/
Bugs fixed, improvements made, UI implemented


Expose the ModalBarrier and bodyScrimColor Scaffold used with the bottom sheet

Use case
Need to change the background color when a full-size Scaffold#bottomSheet contains transparent parts. Or you need to use a custom widget to render that background (like gradient).
Proposal
Make it possible to pass ScaffoldState#_bodyScrimColor and ModalBarrier as Scaffold parameters

Hello @RareScrap. Could you show an example of this? Like what is currently possible vs what is expected if this feature is implemented? And any current workarounds there may be?

@exaby73 Imagine you want to keep some free space at the top of the bottom sheet. In this case, you are expecting two things:

Scaffold body does not get darker when the bottom sheet reaches its maximum height
You are still able to interact with widgets on the scaffold body

Suppose you have implemented it like this:
import 'package:flutter/material.dart';

void main() {
  runApp(
    const MaterialApp(
      title: 'Flutter Demo',
      home: ExampleScreen(),
    ),
  );
}

class ExampleScreen extends StatelessWidget {
  const ExampleScreen({Key? key}) : super(key: key);

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      home: _buildHome(),
    );
  }

  Widget _buildHome() {
    return Scaffold(
      backgroundColor: Colors.green,
      body: SafeArea(
        child: Align(
          alignment: Alignment.topCenter,
          child: ElevatedButton(
            onPressed: () {},
            child: const Text("I'm blocked button :("),
          ),
        ),
      ),
      bottomSheet: _buildBottomSheet(),
    );
  }

  Widget _buildBottomSheet() {
    return DraggableScrollableSheet(
      maxChildSize: 0.9,
      snap: true,
        expand: false,
      builder: (BuildContext context, ScrollController scrollController) {
        return Container(
          color: Colors.red,
          child: ListView.builder(
            controller: scrollController,
            itemCount: 100,
            itemBuilder: (context, index) {
              return Text("Item $index");
            },
          ),
        );
      },
    );
  }
}

This code causes inability to press the button when the bottom sheet reaches its maximum size:

You can't press the button because of hardcoded ModalBarrier of the scaffold. Also hardcoded grey tint is applied to the body of scaffold.
There is a workaround to solve this using a custom scaffold state implementation:
class WorkaroundScaffold extends Scaffold {
  const WorkaroundScaffold({
    Key? key,
    PreferredSizeWidget? appBar,
    Widget? body,
    Widget? floatingActionButton,
    FloatingActionButtonLocation? floatingActionButtonLocation,
    FloatingActionButtonAnimator? floatingActionButtonAnimator,
    List<Widget>? persistentFooterButtons,
    Widget? drawer,
    DrawerCallback? onDrawerChanged,
    Widget? endDrawer,
    DrawerCallback? onEndDrawerChanged,
    Widget? bottomNavigationBar,
    Widget? bottomSheet,
    Color? backgroundColor,
    bool? resizeToAvoidBottomInset,
    bool primary = true,
    DragStartBehavior drawerDragStartBehavior = DragStartBehavior.start,
    bool extendBody = false,
    bool extendBodyBehindAppBar = false,
    Color? drawerScrimColor,
    double? drawerEdgeDragWidth,
    bool drawerEnableOpenDragGesture = true,
    bool endDrawerEnableOpenDragGesture = true,
    String? restorationId,
  }) : super(
          key: key,
          appBar: appBar,
          body: body,
          floatingActionButton: floatingActionButton,
          floatingActionButtonLocation: floatingActionButtonLocation,
          floatingActionButtonAnimator: floatingActionButtonAnimator,
          persistentFooterButtons: persistentFooterButtons,
          drawer: drawer,
          onDrawerChanged: onDrawerChanged,
          endDrawer: endDrawer,
          onEndDrawerChanged: onEndDrawerChanged,
          bottomNavigationBar: bottomNavigationBar,
          bottomSheet: bottomSheet,
          backgroundColor: backgroundColor,
          resizeToAvoidBottomInset: resizeToAvoidBottomInset,
          primary: primary,
          drawerDragStartBehavior: drawerDragStartBehavior,
          extendBody: extendBody,
          extendBodyBehindAppBar: extendBodyBehindAppBar,
          drawerScrimColor: drawerScrimColor,
          drawerEdgeDragWidth: drawerEdgeDragWidth,
          drawerEnableOpenDragGesture: drawerEnableOpenDragGesture,
          endDrawerEnableOpenDragGesture: endDrawerEnableOpenDragGesture,
          restorationId: restorationId,
        );

  @override
  ScaffoldState createState() => SDScaffoldState();
}

class SDScaffoldState extends ScaffoldState {
  @override
  void showBodyScrim(bool value, double opacity) {
    // Disable hardcoded bottomsheet tint and ModelBarrier
  }
}

But this solution cannot be called reliable and easy to maintain. It would be much easier if scaffold exposed the background color of the bottom sheet and the barrier widget as a property.
But this solution cannot be described reliable and easy to maintain. It would be much easier if scaffold exposed the background color of the bottom sheet and the barrier widget as a property.

Thank you for the information @RareScrap. Labelling for further insights from the team

Same here!

yeap, same. I need to customize it.
Current workaround: using bottomNavigationBar, but is not consistent.

Any updates here?


Foundation: extract cfSource from RunLoop._Source

Making cfSource a member within the structure prevents allowing
overriding of the open class as the type it returns cannot be
deserialised.  Because the member is not a stored property, extract it
to an extension which removes it from the vtable.  This allows the use
of the private type in an external context.

Please also file separately to figure out why Windows could not handle this computed variable in this context?

@swift-ci please test and merge


open_mfdataset memory leak, very simple case. v0.12

MCVE Code Sample

import glob
import xarray as xr
import numpy as np
from memory_profiler import profile

def CreateTestFiles():
        # create a bunch of files
        xlen = int(1e2)
        ylen = int(1e2)
        xdim = np.arange(xlen)
        ydim = np.arange(ylen)

        nfiles = 100
        for i in range(nfiles):
                data = np.random.rand(xlen, ylen, 1)
                datafile = xr.DataArray(data, coords=[xdim, ydim, i], dims=['x', 'y', 'time'])
                datafile.to_netcdf('testfiles/datafile_{}.nc'.format(i))

@profile
def ReadFiles():
        xr.open_mfdataset(glob.glob('testfiles/*'), concat_dim='time')

if __name__ == '__main__':
        # write out files for testing
        CreateTestFiles()

        # loop thru file read step
        for i in range(100):
                ReadFiles()
~
~

usage:
mprof run simplest_case.py
mprof plot
(mprof is a python memory profiling library)
Problem Description
dask version 1.1.4
xarray version 0.12
python 3.7.3
There appears to be a persistent memory leak in open_mfdataset. I'm creating a model calibration script that runs for ~1000 iterations, opening and closing the same set of files (dimensions are the same, but the data is different) with each iteration. I eventually run out of memory because of the leak. This simple case captures the same behavior. Closing the files with .close() does not fix the problem.
Is there a work around for this? I've perused some of the issues but cannot tell if this has been resolved.

Output of xr.show_versions()

INSTALLED VERSIONS
commit: None
python: 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0]
python-bits: 64
OS: Linux
OS-release: 3.10.0-693.17.1.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
libhdf5: 1.10.4
libnetcdf: 4.6.2
xarray: 0.12.0
pandas: 0.24.2
numpy: 1.16.2
scipy: 1.2.1
netCDF4: 1.4.2
pydap: None
h5netcdf: None
h5py: None
Nio: 1.5.5
zarr: None
cftime: <IP_ADDRESS>
nc_time_axis: None
PseudonetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.2.1
dask: 1.1.4
distributed: 1.26.0
matplotlib: 3.0.2
cartopy: 0.17.0
seaborn: None
setuptools: 41.0.1
pip: 19.1.1
conda: None
pytest: None
IPython: 7.3.0
sphinx: None

Hi,
xarray doesn't have any global objects that I know of that can cause the leak - I'm willing to bet on the underlying libraries.

given your installed packages, open_mfdataset should be defaulting NetCDF4. Please try your measure again after setting it explicitly open_mfdataset(..., engine='netcdf4')
See if the problem disappears if you pass engine='h5netcdf'
Once you have confirmed the actual underlying library, try using it directly without xarray in your ReadFiles test: for every file returned by glob, open it with the netCDF4 package and load into memory all coords (not the data).
Once NetCDF4 is confirmed to be the culprit, if you can it would be great if you could rewrite the test (only the read part) in C using the NetCDF C library to figure out if the leak is in it or in the Python wrapper.

Oh but first and foremost - CPython memory management is designed so that, when PyMem_Free() is invoked, CPython will hold on to it and not invoke the underlying free() syscall, hoping to reuse it on the next PyMem_Alloc().
An increase in RAM usage from 160 to 200MB could very well be explained by this.
Try increasing the number of loops in your test 50-fold and see if you get a 50-fold increase in memory usage too (from 160MB to 2.2GB). If yes, it's a real leak; if it remains much more contained, it's normal CPython behaviour.

Thanks for the profiling script. I ran a few permutations of this:

xarray.open_mfdataset with engine='netcdf4' (default)
xarray.open_mfdataset with engine='h5netcdf'
xarray.open_dataset with engine='netcdf4' (default)
xarray.open_dataset with engine='h5netcdf'

Here are some plots:
xarray.open_mfdataset with engine='netcdf4': pretty noticeable memory leak, about 0.5 MB / open_mfdataste call:

xarray.open_mfdataset with engine='h5netcdf': looks like a small memory leak, about 0.1 MB / open_mfdataste call:

xarray.open_dataset with engine='netcdf4' (default): definitely has a memory leak:

xarray.open_dataset with engine='h5netcdf': does not appear to have a memory leak:

So in conclusion, it looks like there are memory leaks:

when using netCDF4-Python (I was also able to confirm these without using xarray at all)
when using xarray.open_mfdataset

(1) looks like by far the bigger issue, which you can work around by switching to scipy or h5netcdf to read your files.
(2) is an issue for xarray. We do do some caching, specifically with our backend file manager, but given that issues only seem to appear when using open_mfdataset, I suspect it may have more to do with the interaction with Dask, though to be honest I'm not exactly sure how.
Note: I modified your script to xarray's file cache size to 1, which helps smooth out the memory usage:
def CreateTestFiles():
        # create a bunch of files
        xlen = int(1e2)
        ylen = int(1e2)
        xdim = np.arange(xlen)
        ydim = np.arange(ylen)

        nfiles = 100
        for i in range(nfiles):
                data = np.random.rand(xlen, ylen, 1)
                datafile = xr.DataArray(data, coords=[xdim, ydim, [i]], dims=['x', 'y', 'time'])
                datafile.to_netcdf('testfile_{}.nc'.format(i))

@profile
def ReadFiles():
        # for i in range(100):
        #         ds = xr.open_dataset('testfile_{}.nc'.format(i), engine='netcdf4')
        #         ds.close()
        ds = xr.open_mfdataset(glob.glob('testfile_*'), engine='h5netcdf', concat_dim='time')
        ds.close()

if __name__ == '__main__':
        # write out files for testing
        CreateTestFiles()

        xr.set_options(file_cache_maxsize=1)

        # loop thru file read step
        for i in range(100):
                ReadFiles()

Also, if you're having memory issues I also would definitely recommend upgrading to a newer version of xarray. There was a recent fix that helps ensure that files get automatically closed when they are garbage collected, even if you don't call close() or use a context manager explicitly.

Awesome, thanks @shoyer and @crusaderky for looking into this. I've tested it with the h5netcdf engine and it the leak is mostly mitigated... for the simple case at least. Unfortunately the actual model files that I'm working with do not appear to be compatible with h5py (I believe related to this issue https://github.com/h5py/h5py/issues/719). But that's another problem entirely!
@crusaderky, I will hopefully get to trying your suggestions 3) and 4). As for your last point, I haven't tested explicitly, but yes I believe that it does continue to grow linearly more iterations.

I was iterating today over a large dataset loaded with open_mfdataset, and had been observing memory usage growing from 2GB to 8GB+.
I can confirm that xr.set_options(file_cache_maxsize=1) kept memory use at a steady 2GB, properly releasing memory.
libnetcdf                 4.8.1           nompi_h261ec11_106    conda-forge
netcdf4                   1.6.0           nompi_py310h0a86a1f_103    conda-forge
xarray                    2023.1.0           pyhd8ed1ab_0    conda-forge
dask                      2023.1.0           pyhd8ed1ab_0    conda-forge
dask-core                 2023.1.0           pyhd8ed1ab_0    conda-forge


Rename master to main

As per ASWF guidance, master branch should be renamed to main. Local branches will need to:
git branch -m master main
git fetch origin
git branch -u origin/main main
git remote set-head origin -a

Watch out for code in python/aswfdocker/utils.py which hard codes master branch:
def get_docker_org(repo_uri: str, source_branch: str) -> str:
    if not source_branch and not repo_uri:
        return constants.TESTING_DOCKER_ORG
    if (
        source_branch == "refs/heads/master"
        and constants.MAIN_GITHUB_ASWF_DOCKER_URL.endswith(repo_uri)
    ):

LF Ticket:
https://jira.linuxfoundation.org/plugins/servlet/desk/portal/2/IT-27216

Addressed in PR #218


Clean up See also sections

In docs/visual-basic directory
Blank line updates related to #6923

#11692 is much larger...


quickstart scenario in e2e smoke test

🌮 Taqueria PR
Close #1724
🪁 Description
added the full docs quickstart scenario to the smoke test
🪂 Pre-Merge Checklist (Definition of Done)
🚦 Required to merge:

[ ] ⛱ I have completed this PR template in full and updated the title appropriately
[ ] ⛵ My code builds cleanly, and I have manually tested the changes
[ ] 🏄♂ Another team member has built this branch and done manual testing on the change
[ ] 🏖 New and existing unit tests pass locally and in CI
[ ] 🔱 The test plan has been implemented and verified by an SDET
[ ] 🦀 Automated tests have been written and added to this PR
[ ] 🐬 I have commented my code, particularly in hard-to-understand areas
[ ] 🤿 Corresponding changes have been made to all documentation
[ ] 🐚 Required changes to the VScE have been made
[ ] 🪸 Required updates to scaffolds have been made
[ ] 🚢 The release checklist has been completed

🛩 Summary of Changes
added the full docs quickstart scenario to the smoke test
🎢 Test Plan
will pass in cicd and locally
🛸 Type of Change

[x ] 🧽 Chore ➾
[ ] 🛠 Fix ➾
[ ] ✨ Feature ➾
[ ] 👷 Refactor ➾
[ ] 🧪 Pre-Release ➾
[ ] 🚀 Release ➾

test is working except on mac minis it refuses to get past like 62,