Add checks for lat_lon_land

Issue resolution

Resolves #622 (specifically the issue at https://github.com/E3SM-Project/zppy/issues/622#issuecomment-2417599936)

Select one: This pull request is...

[x] a bug fix: increment the patch version
[ ] a small improvement: increment the minor version
[ ] an incompatible (non-backwards compatible) API change: increment the major version

1. Does this do what we want it to do?
Objectives:

Add dependency checking for the lat_lon_land set.
Add new helper function check_parameter_defined .
Add/modify tests accordingly.

Required:

[ ] Product Management: I have confirmed with the stakeholders that the objectives above are correct and complete.
[ ] Testing: I have added or modified at least one "min-case" configuration file to test this change. Every objective above is represented in at least one cfg.
[ ] Testing: I have considered likely and/or severe edge cases and have included them in testing.

If applicable:

[ ] Testing: this pull request introduces an important feature or bug fix that we must test often. I have updated the weekly-test configuration files, not just a "min-case" one.
[ ] Testing: this pull request adds at least one new possible parameter to the cfg. I have tested using this parameter with and without any other parameter that may interact with it.

2. Are the implementation details accurate & efficient?
Required:

[ ] Logic: I have visually inspected the entire pull request myself.
[ ] Logic: I have left GitHub comments highlighting important pieces of code logic. I have had these code blocks reviewed by at least one other team member.

If applicable:

[ ] Dependencies: This pull request introduces a new dependency. I have discussed this requirement with at least one other team member. The dependency is noted in zppy/conda, not just an import statement.

3. Is this well documented?
Required:

[ ] Documentation: by looking at the docs, a new user could easily understand the functionality introduced by this pull request.

4. Is this code clean?
Required:

[ ] Readability: The code is as simple as possible and well-commented, such that a new team member could understand what's happening.
[ ] Pre-commit checks: All the pre-commits checks have passed.

If applicable:

[ ] Software architecture: I have discussed relevant trade-offs in design decisions with at least one other team member. It is unlikely that this pull request will increase tech debt.

Checked for each set in "lat_lon","zonal_mean_xy","zonal_mean_2d","polar","cosp_histogram","meridional_mean_2d","annual_cycle_zonal_mean","zonal_mean_2d_stratosphere","enso_diags","qbo","diurnal_cycle","streamflow","tropical_subseasonal","tc_analysis","lat_lon_land that dependencies are added.

As far as I can tell, e3sm_diags.py never had a dependency addition for when lat_lon_land was in sets. That is, https://github.com/E3SM-Project/zppy/issues/622#issuecomment-2417599936 has been a bug ever since #548 was merged and hasn't been caught until now. Perhaps the climo task was just always happening to finish first.
In any case, an explicit dependency check has been added with this PR.

The min-case test min_case_e3sm_diags_lat_lon_land also doesn't work. It appears that one wasn't tested run as part of merging #608. (Note the caveat "I haven't actually run these minimal cases because there are quite a few of them. My plan is to modify them as needed when actually using them to debug/test future pull requests." on https://github.com/E3SM-Project/zppy/pull/608#issuecomment-2237816140)

 Error description
$ cd /lcrc/group/e3sm/ac.forsyth2/zppy_min_case_e3sm_diags_lat_lon_land_mvm_1_output/test-633/v3.LR.historical_0051/post/scripts
$ grep -v "OK" *status
ERROR (3)
$ grep -in error climo_land_monthly_climo_1985-1988.o607365
64:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
68:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
72:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
76:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
80:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
84:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
88:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
89:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
91:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
93:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
95:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
97:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
98:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
102:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
106:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
107:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
112:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
113:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
115:srun: error: chr-0499: task 0: Exited with exit code 1
116:srun: error: chr-0495: task 0: Exited with exit code 1
117:srun: error: chr-0495: task 0: Exited with exit code 1
118:srun: error: chr-0495: task 0: Exited with exit code 1
119:srun: error: chr-0499: task 0: Exited with exit code 1
120:srun: error: chr-0494: task 0: Exited with exit code 1
121:srun: error: chr-0494: task 0: Exited with exit code 1
122:srun: error: chr-0494: task 0: Exited with exit code 1
123:srun: error: chr-0499: task 0: Exited with exit code 1
127:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
131:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
135:ncks: ERROR nco_rgr_wgt() Hail Mary algorithm reports final failure since product of identifed horizontal dimension sizes in the 2D input data file does not equal the map-file 1D dimension size = 21600.
136:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
138:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
140:ncremap: ERROR Failed to horizontally regrid. cmd_rgr[0] failed. Debug this:
142:srun: error: chr-0496: task 0: Exited with exit code 1
143:srun: error: chr-0496: task 0: Exited with exit code 1
144:srun: error: chr-0496: task 0: Exited with exit code 1
145:ncclimo: ERROR monthly regrid cmd_rgr[1] failed. Debug this:

$ cd /lcrc/group/e3sm/ac.forsyth2/zppy_min_case_e3sm_diags_lat_lon_land_mvm_2_output/test-633/v3.LR.historical_0051/post/scripts
$ grep -v "OK" *status
climo_land_monthly_climo_1995-1998.status:ERROR (3)
e3sm_diags_lnd_monthly_mvm_lnd_model_vs_model_1995-1998_vs_1985-1988.status:WAITING 607367
# similar regridding errors

So far so good on my latest version of the cfg from https://github.com/E3SM-Project/zppy/issues/622#issuecomment-2417449209 though: the other two diags tasks are running while the tc_analysis diags task waits for tc_analysis to finish.

Test the cfg from #622
Re: the issues with the cfg at https://github.com/E3SM-Project/zppy/issues/622#issuecomment-2369252993.

 The cfg (5th iteration from the original cfg)
# Test `main` after merging #623
[default]
# NEW in v3: Set `dry_run = False`
dry_run = False
input = /lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.historical_0051
#output = /lcrc/group/e3sm2/ac.zhang40/E3SMv3/v3.LR.historical_0920
output = /lcrc/group/e3sm/ac.forsyth2/issue_622v5/v3.LR.historical_0051
case = v3.LR.historical_0051
#www = /lcrc/group/e3sm/public_html/diagnostic_output/ac.zhang40/E3SMv3_0920
www = /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/issue_622v5
partition = compute
environment_commands = "source /lcrc/soft/climate/e3sm-unified/load_latest_e3sm_unified_chrysalis.sh"
campaign = "water_cycle"

[climo]
active = True
years = "1985:2014:30",
walltime = "1:00:00"

  [[ atm_monthly_180x360_aave ]]
  input_subdir = "archive/atm/hist"
  mapping_file = map_ne30pg2_to_cmip6_180x360_aave.20200201.nc
  frequency = "monthly"

  [[ atm_monthly_diurnal_8xdaily_180x360_aave ]]
  input_subdir = "archive/atm/hist"
  input_files = "eam.h3"
  mapping_file = map_ne30pg2_to_cmip6_180x360_aave.20200201.nc
  vars = "PRECT"
  frequency = "diurnal_8xdaily"

  [[ land_monthly_climo ]]
  frequency = "monthly"
  input_files = "elm.h0"
  input_subdir = archive/lnd/hist
  vars = ""

[ts]
active = True
years = "1985:2014:30"
walltime = "00:50:00"

  [[ atm_monthly_180x360_aave ]]
  input_subdir = "archive/atm/hist"
  input_files = "eam.h0"
  frequency = "monthly"
  mapping_file = /home/ac.zender/data/maps/map_ne30pg2_to_cmip6_180x360_aave.20200201.nc
  vars = "FSNTOA,FLUT,FSNT,FLNT,FSNS,FLNS,SHFLX,QFLX,TAUX,TAUY,PRECC,PRECL,PRECSC,PRECSL,TS,TREFHT,CLDTOT,CLDHGH,CLDMED,CLDLOW,U,ICEFRAC,LANDFRAC,OCNFRAC,PS,CLDICE,CLDLIQ,T,AODDUST"
  ts_fmt = "cmip"

  [[ atm_daily_180x360_aave ]]
  input_subdir = "archive/atm/hist"
  input_files = "eam.h1"
  frequency = "daily"
  mapping_file = /home/ac.zender/data/maps/map_ne30pg2_to_cmip6_180x360_aave.20200201.nc
  # Needed for Wheeler Kiladis
  vars = "FLUT,PRECT,U850"

  [[ land_monthly ]]
  input_subdir = "archive/lnd/hist"
  input_files = "elm.h0"
  frequency = "monthly"
  mapping_file = map_r05_to_cmip6_180x360_aave.20231110.nc
  vars = "FSH,RH2M,LAISHA,LAISUN,QINTR,QOVER,QRUNOFF,QSOIL,QVEGE,QVEGT,SOILICE,SOILLIQ,SOILWATER_10CM,TSA,TSOI,H2OSNO,TOTLITC,CWDC,SOIL1C,SOIL2C,SOIL3C,SOIL4C,WOOD_HARVESTC,TOTVEGC,NBP,GPP,AR,HR"
  extra_vars = "landfrac"
  ts_fmt = "cmip"

  [[ rof_monthly ]]
  input_subdir = "archive/rof/hist"
  input_files = "mosart.h0"
  mapping_file = ""
  frequency = "monthly"
  vars = "RIVER_DISCHARGE_OVER_LAND_LIQ"
  extra_vars = 'areatotal2'

[tc_analysis]
active = True
scratch = "/lcrc/globalscratch/$USER"
# Make walltime very short to reproduce this error
# NEW in v4: try with longer run time
walltime = "01:00:00"
years = "1985:2014:30",

[e3sm_diags]
active = True
walltime = "4:00:00"
years = "1985:2014:30",
ts_num_years = 30
# NEW in v3: Added `ref_end_yr` (zppy.utils.ParameterNotProvidedError: ref_end_yr)
ref_end_yr = 2014
ref_start_yr = 1985
ref_final_yr = 2014
multiprocessing = True
num_workers = 8

  [[ atm_monthly_180x360_aave ]]
  short_name = 'v3.LR.historical_0051'
  grid = '180x360_aave'
  reference_data_path = '/lcrc/soft/climate/e3sm_diags_data/obs_for_e3sm_diags/climatology'
  obs_ts = '/lcrc/soft/climate/e3sm_diags_data/obs_for_e3sm_diags/time-series'
  dc_obs_climo = '/lcrc/group/e3sm/public_html/e3sm_diags_test_data/unit_test_complete_run/obs/climatology'
  climo_diurnal_subsection = "atm_monthly_diurnal_8xdaily_180x360_aave"
  climo_diurnal_frequency = "diurnal_8xdaily"
  ts_daily_subsection = "atm_daily_180x360_aave"
  sets="lat_lon","zonal_mean_xy","zonal_mean_2d","polar","cosp_histogram","meridional_mean_2d","annual_cycle_zonal_mean","qbo","diurnal_cycle","zonal_mean_2d_stratosphere","aerosol_aeronet","tropical_subseasonal","tc_analysis",
  output_format_subplot = "pdf",

  [[ lnd_monthly_mvm_lnd ]]
  # Test model-vs-model using the same files as the reference
  grid = 'native'
  # NEW in v5: renamed this parameter
  climo_land_subsection = "land_monthly_climo"
  diff_title = "Difference"
  partition = "compute"
  qos = "regular"
  short_name = v3.LR.piControl
  ref_name = "20231209.v3.LR.piControl-spinup.chrysalis"
  ref_start_yr = 0051
  ref_final_yr = 0100
  ref_years = "0051-0100",
  reference_data_path = "/lcrc/group/e3sm/ac.zhang40/tests/20231209.v3.LR.piControl-spinup.chrysalis_land_diags/post/lnd/native/clim"
  run_type = "model_vs_model"
  sets = "lat_lon_land",
  short_ref_name = "20231209.v3.LR.piControl-spinup"
  swap_test_ref = False
  tag = "model_vs_model"
  ts_num_years_ref = 50

  [[atm_monthly_180x360_aave_mvm]]
  ref_years = "0001-0050",
  ref_start_yr = 1
  ref_final_yr = 50
  ts_num_years = 30
  ts_num_years_ref = 10
  ts_subsection = "atm_monthly_180x360_aave"
  short_name = 'v3alpha04-COARE.piControl'
  grid = '180x360_aave'
  ref_name = '20230924.v3alpha04_trigrid.piControl.chrysalis'
  short_ref_name = 'v3alpha04-CTL.piControl'
  tag = 'v3alpha04i-COARE_vs_CTL'
  run_type = "model_vs_model"
  reference_data_path = '/lcrc/group/e3sm2/ac.xzheng/E3SMv3_dev/20230924.v3alpha04_trigrid.piControl.chrysalis/post/atm/180x360_aave/clim'
  climo_diurnal_subsection = "atm_monthly_diurnal_8xdaily_180x360_aave"
  climo_diurnal_frequency = "diurnal_8xdaily"
  climo_subsection = "atm_monthly_180x360_aave"
  sets="lat_lon","zonal_mean_xy","zonal_mean_2d","polar","cosp_histogram","meridional_mean_2d","annual_cycle_zonal_mean","qbo","diurnal_cycle","zonal_mean_2d_stratosphere","aerosol_budget"
  diff_title = 'Difference'

$ cd /lcrc/group/e3sm/ac.forsyth2/issue_622v5/v3.LR.historical_0051/post/scripts
$ grep -v "OK" *status
# No errors

These links show viewers:

https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/issue_622v5/v3.LR.historical_0051/e3sm_diags/atm_monthly_180x360_aave/model_vs_obs_1985-2014/viewer/ (all 13 sets)
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/issue_622v5/v3.LR.historical_0051/e3sm_diags/atm_monthly_180x360_aave_mvm/v3alpha04i-COARE_vs_CTL_1985-2014/viewer/ (10 of 11 sets)

=> Missing "aerosol_budget"

https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/issue_622v5/v3.LR.historical_0051/e3sm_diags/lnd_monthly_mvm_lnd/model_vs_model_1985-2014/viewer (1 set)

Some sets still need better handling
Looking at the sets used in this cfg, it looks like "aerosol_aeronet" and "aerosol_budget" don't appear much in the zppy code:
$ git grep -n aerosol
zppy/templates/cryosphere.cfg:2:sets= "lat_lon","zonal_mean_xy","zonal_mean_2d","polar","cosp_histogram","meridional_mean_2d","enso_diags","qbo","diurnal_cycle","annual_cycle_zonal_mean","streamflow","zonal_mean_2d_stratosphere","aerosol_aeronet","aerosol_budget"
zppy/templates/default.ini:205:# All available sets (17) = "aerosol_aeronet","aerosol_budget","annual_cycle_zonal_mean","area_mean_time_series","cosp_histogram","diurnal_cycle","enso_diags","lat_lon","meridional_mean_2d","polar","qbo","streamflow","tc_analysis", "tropical_subseasonal", "zonal_mean_2d","zonal_mean_2d_stratosphere","zonal_mean_xy"
zppy/templates/default.ini:209:sets = string_list(default=list("lat_lon","zonal_mean_xy","zonal_mean_2d","polar","cosp_histogram","meridional_mean_2d","annual_cycle_zonal_mean","zonal_mean_2d_stratosphere","aerosol_aeronet","aerosol_budget"))
zppy/templates/water_cycle.cfg:2:sets = "lat_lon","zonal_mean_xy","zonal_mean_2d","polar","cosp_histogram","meridional_mean_2d","enso_diags","qbo","diurnal_cycle","annual_cycle_zonal_mean","streamflow","zonal_mean_2d_stratosphere","aerosol_aeronet","aerosol_budget"

That means those 2 sets don't occur in either the .py or .bash file for E3SM Diags. These were added in #289 and #352. However, these 2 PRs only updated the default sets for e3sm_diags. It appears that parameter checking and dependency checking in e3sm_diags.py and e3sm_diags.bash have not included them. I suppose the dependency handling hasn't come up because they've typically been included with other sets that did force a dependency requirement.
Example error from /lcrc/group/e3sm/ac.forsyth2/issue_622v5/v3.LR.historical_0051/post/scripts/e3sm_diags_atm_monthly_180x360_aave_mvm_v3alpha04i-COARE_vs_CTL_1985-2014_vs_0001-0050.o607385:
2024-10-16 15:19:15,254 [ERROR]: core_parameter.py(_run_diag:269) >> Error in e3sm_diags.driver.aerosol_budget_driver
Traceback (most recent call last):
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.10.0_chrysalis/lib/python3.10/site-packages/e3sm_diags/parameter/core_parameter.py", line 266, in _run_diag
    single_result = module.run_diag(self)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.10.0_chrysalis/lib/python3.10/site-packages/e3sm_diags/driver/aerosol_budget_driver.py", line 148, in run_diag
    metrics_dict_test[aerosol] = generate_metrics_dic(
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.10.0_chrysalis/lib/python3.10/site-packages/e3sm_diags/driver/aerosol_budget_driver.py", line 82, in generate_metrics_d\
ic
    elvemis = data.get_climo_variable(f"{aerosol}_CLXF", season)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.10.0_chrysalis/lib/python3.10/site-packages/e3sm_diags/driver/utils/dataset.py", line 161, in get_climo_variable
    variables = self._get_climo_var(filename, *args, **kwargs)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.10.0_chrysalis/lib/python3.10/site-packages/e3sm_diags/driver/utils/dataset.py", line 386, in _get_climo_var
    derived_var = func(*variables)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.10.0_chrysalis/lib/python3.10/site-packages/e3sm_diags/derivations/acme.py", line 1654, in <lambda>
    (("bc_a?_CLXF",), lambda *x: molec_convert_units(sum(x), 12.0)),
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.10.0_chrysalis/lib/python3.10/site-packages/e3sm_diags/derivations/acme.py", line 175, in molec_convert_units
    if var.units == "molec/cm2/s":
AttributeError: 'int' object has no attribute 'units'

@chengzhuzhang I'll look into this more, but let me know if anything stands out as immediately wrong (either in how zppy is calling Diags or in Diags itself).
Action items to do before merging this PR

[ ] Debug the missing set described above
[ ] Update e3sm_diags.py and e3sm_diags.bash to include parameter/dependency checks for the aerosol sets.
[ ] default.ini shows 17 possible sets for e3sm_diags. Update the weekly comprehensive tests to test all 17 -- that is, make sure none were inadvertently missed in #628.
[ ] Get the min-case mvm-land test working

@chengzhuzhang I need more context for the "aerosol_aeronet","aerosol_budget" sets. #289 and #352 only added these to default sets.

That implies everything about them is covered through CoreParameter, correct?
I'm assuming the only dependency is on the climo task? Is that true?

AttributeError: 'int' object has no attribute 'units' implies somehow Diags is getting an int instead of a variable, but it's unclear if that's because of something in the Diags code or a missing processing step in zppy. If there really doesn't need to be extra processing of these two sets beyond adding them to the default sets, that would seem to imply the error is in E3SM Diags then.

@chengzhuzhang I need more context for the "aerosol_aeronet","aerosol_budget" sets. #289 and #352 only added these to default sets.
* That implies everything about them is covered through `CoreParameter`, correct?

Yes, this is correct

* I'm assuming the only dependency is on the `climo` task? Is that true?

Yes, this is true

AttributeError: 'int' object has no attribute 'units' implies somehow Diags is getting an int instead of a variable, but it's unclear if that's because of something in the Diags code or a missing processing step in zppy. If there really doesn't need to be extra processing of these two sets beyond adding them to the default sets, that would seem to imply the error is in E3SM Diags then.

This seems to be a diags error.

@chengzhuzhang I've renamed this PR to better cover that the changes affect a number of sets. We need two changes to the e3sm_diags code base before I can proceed though:

Move the non-empty dat file check to the tc_analysis set in E3SM Diags. Then I can remove that error in the tc_analysis task in zppy. (Addresses https://github.com/E3SM-Project/zppy/pull/628#discussion_r1805574596)
Fix the aerosol type (int vs variable) bug. Then, I can properly test the aerosol sets in zppy (Addresses https://github.com/E3SM-Project/zppy/pull/633#issuecomment-2420037419)

Thanks!

Thanks. yes, it makes sense to update the upstream tool first.

@chengzhuzhang I've renamed this PR to better cover that the changes affect a number of sets. We need two changes to the e3sm_diags code base before I can proceed though:
* Move the non-empty `dat` file check to the `tc_analysis` set in E3SM Diags. Then I can remove that error in the `tc_analysis` task in `zppy`. (Addresses [Improve input validation and testing #628 (comment)](https://github.com/E3SM-Project/zppy/pull/628#discussion_r1805574596))

* Fix the aerosol type (int vs variable) bug. Then, I can properly test the `aerosol` sets in `zppy` (Addresses [Update to diags handling #633 (comment)](https://github.com/E3SM-Project/zppy/pull/633#issuecomment-2420037419))

Thanks!

@forsyth2 item one has been taken care of by https://github.com/E3SM-Project/e3sm_diags/pull/851/commits/ffbdf287a91eca57fd383a1de4c0cf22e9fce829.
Though I'm not sure if item two is a blocker for zppy, we can improve error handling in e3sm diags, but it doesn't seem to be critical for running zppy...

item one has been taken care of

Great, thanks! I'll remove the empty-dat check in this PR then.

I'm not sure if item two is a blocker for zppy

The only thing is no plots will generate for the aerosol sets if we run into that bug in the weekly testing too (now that the aerosol sets have been added to weekly testing). But correct, zppy can still function.

The only thing is no plots will generate for the aerosol sets if we run into that bug in the weekly testing too (now that the aerosol sets have been added to weekly testing). But correct, zppy can still function.

The aerosl_budget table won't be created because the variables are not included in the standard v3 output that zppy weekly tests run with.
@forsyth2 sorry I accidentally messed up your comment by editing...

@forsyth2 the PR looks good, I just had a clarification regarding the change of   climo_subsection for land.

Thanks @chengzhuzhang! Let me know if my answer to your question is sufficient, and then I can merge this.


BatchMatMul for TFLite

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template
System information

TensorFlow version (you are using):tf2.0.0-rc1
Are you willing to contribute it (Yes/No):No

Describe the feature and the current behavior/state.
Currently tf.matmul hides batch matmuls in code and this means seemingly benign operations (tf.matmul is listed as supported) will cause a model to not be convertable to TFLite without using TF_SELECT_OPS and compiling a custom interpreter.
Will this change the current api? How? No
Who will benefit with this feature? All users who wish to convert models to TFLite models and use matmul for batch multiplication.
Any Other info.

Is this problem solved?

This is fixed. Thanks.

@TimCapes,
Sorry for the delayed response. As per this comment,

BatchMatMul can be supported via Select TF ops and we offer a selective build mechanism for Select TF ops for slim binary size deployment.

Can you please take a look? Thanks!


Can't find the settings

Using PF2 system. Screenshot below.

Should work with 0.8.1


Update to zbus 3

This PR starts the process of updating to the upcoming version of zbus 3.0. The most notable change made in the new major version is much tighter integration with tokio with the relevant feature, which significantly reduces the number of dependencies required to use zbus in a Tokio-centric build tree.
Dependencies built invoking cargo test --no-default-features --features tokio:
3.0:  143
2.0: 154
The build times between the two branches are about as expected given the size of tokio vs async-std. I would not consider these changes a win for compile times despite the lower dependency count.
All tests pass on Ubuntu 22.04.
Closes #42

The build times between the two branches are about as expected given the size of tokio vs async-std

There may be a confusion here still. zbus does not depend on async-std. It makes use of the smol-rs crates though, which are designed to be as small and self-contained as possible.

I would not consider these changes a win for compile times despite the lower dependency count.

This doesn't surprise me really. If you recall, I did try and make a point about this in #42. Practically speaking, this won't bring a lot of significant benefits to the table, even if I removed the remaining smol-rs dep: async-broadcast.

I would not consider these changes a win for compile times despite the lower dependency count.

This doesn't surprise me really. If you recall, I did try and make a point about this in #42. Practically speaking, this won't bring any significant benefits to the table, even if I removed the remaining smol-rs dep: async-broadcast.

This is not to say 3.0 doesn't bring real benefits for tokio users. 3.0 does bring tighter integration with the runtime itself so all the tokio tools will work out of the box with zbus now. E.g all internal tasks are not tokio tasks launched in whatever application runtime zbus API is used with.
These benefits are unrelated to lowered dependencies though.

There may be a confusion here still. zbus does not depend on async-std. It makes use of the smol-rs crates though, which are designed to be as small and self-contained as possible.

Sorry, that was poorly worded on my part. In my mind the async-* crate ecosystem makes up "async-std" and that was what I was referring to via the async-io feature of zbus. I understand that zbus does not literally depend on that crate.

This doesn't surprise me really. If you recall, I did try and make a point about this in https://github.com/hwchen/secret-service-rs/issues/42. Practically speaking, this won't bring any significant benefits to the table, even if I removed the remaining smol-rs dep: async-broadcast.

This is not to say 3.0 doesn't bring real benefits for tokio users. 3.0 does bring tighter integration with the runtime itself so all the tokio tools will work out of the box with zbus now. E.g all internal tasks are not tokio tasks launched in whatever application runtime zbus API is used with.

These benefits are unrelated to lowered dependencies though.

I'm with you on those points 💯. The lowered dependencies is nice for auditing processes and the better integration is great because it avoids more cases where you'll get panics if you try using Tokio-specific utilities.

E.g all internal tasks are not tokio tasks launched in whatever application runtime zbus API is used with.

Just to be clear, the "not" here was a typo. It's "now". :)


Make prepped structure caching more "cache" like

Currently the way structure caching works is more like a dump than a cache, requiring all structures that need to be prepped be present in the cache.
Instead the cache should act as a an actual "cache ", ie searching for if the target has been prepped before by a name based match, and IF NOT prepping it and adding to the cache.
One question here is how we avoid collisions, eg with structures that have unclear names, eg protein.pdb. We should probably also add some heuristics to avoid overwrrites and collisions.

adding on to this, it seems like the protein_prep_workflow quietly deletes and replaces whatever output_dir you give it, which might be better to have as an option

Also the current workflow waits until all the prepped complexes are made to save them - should be done in parallel otherwise if something bad happens you lose everything: https://github.com/choderalab/asapdiscovery/blob/0432f69fc99c85a923a498bf6ba6f117d3e703f3/asapdiscovery-modeling/asapdiscovery/modeling/workflows/protein_prep.py#L260-L271

I guess my idea was that saving to disk should be seperate of the computation, ie you can cache prepped complexes any time, not just when they are being prepped.
Would a partially complete "cache" not be misleading? You would then have to remember that it was only partially finished (and most probably remove it). This is also partially due to the limitations in current caching which treats a "cache" (inverted commas) as a fully complete dump of the prepped structures.

Fixed


Sql Timeout

Hello,
I'm coding along with your book and I have a major problem whenever I launch the app. Fetching the data is either extremely slow or I get timed out. Here is the exception:
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]       An unhandled exception has occurred while executing the request. Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.  ---> System.ComponentModel.Win32Exception (258): Dépassement du délai d'attente.    at Microsoft.Data.SqlClient.SqlCommand.<>c.<ExecuteDbDataReaderAsync>b__164_0(Task1 result)
at System.Threading.Tasks.ContinuationResultTaskFromResultTask2.InnerInvoke()    at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location where exception was thrown ---    at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread) --- End of stack trace from previous location where exception was thrown ---    at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)    at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)    at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)    at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable1.AsyncEnumerator.InitializeReaderAsync(DbContext _, Boolean result, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func4 operation, Func4 verifySucceeded, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable1.AsyncEnumerator.MoveNextAsync()    at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable1 source, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable1 source, CancellationToken cancellationToken)    at WorldCities.Data.ApiResult1.CreateAsync(IQueryable1 source, Int32 pageIndex, Int32 pageSize, String sortColumn, String sortOrder, String filterColumn, String filterQuery) in E:\Software development\Portfolio\Projects\WorldCities\Data\ApiResult.cs:line 77    at WorldCities.Controllers.CitiesController.GetCities(Int32 pageIndex, Int32 pageSize, String sortColumn, String sortOrder, String filterColumn, String filterQuery) in E:\Software development\Portfolio\Projects\WorldCities\Controllers\CitiesController.cs:line 34    at lambda_method(Closure , Object )    at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker)    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)    at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)    at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context) ClientConnectionId:42b3bf42-7977-4fc2-a127-afa95f02fc0f Error Number:-2,State:0,Class:11
If I wait long enough (like a really long time), the fetching is getting better (after loads of refreshes). I've tried with the chapter_07 repo and have the same issue.

Hi Remi,
Chapter 7 has paging enabled, therefore you should call the GetCities API using a pageSize of 10: can you confirm that? (place a debug breakpoint on CitiesController.cs on the first line of the GetCities() method to check it).
Assuming such scenario, here's the underlying SQL query:
SELECT [c].[Id], [c].[Name], [c].[Lat], [c].[Lon], [c0].[Id] AS [CountryId], [c0].[Name] AS [CountryName]
FROM [Cities] AS [c]
INNER JOIN [Countries] AS [c0] ON [c].[CountryId] = [c0].[Id]
ORDER BY [c].[Name]
OFFSET 0 ROWS FETCH NEXT 10 ROWS ONLY

The above query can be obtained using the EF debugging techniques that you can find in page 430 (Chapter 8). If you run the Chapter 9's WorldCities app, you will be able to fetch it from the sql string variable in the ApiResult.cs file (line 96).
Check that your query is identical to mine, then try to execute it using on your DB using SSMS and see how much time it takes: it's a very light query, it should take a bunch of milliseconds to complete. If it doesn't, there's a high chance that the issue relies on your SQL Server instance.

I've tried those requests
https://localhost:5001/api/cities => very fast
https://localhost:5001/api/cities/?pageIndex=0&pageSize=10 => very fast
https://localhost:5001/api/cities/?pageIndex=0&pageSize=10&sortColumn=name => very very slow and sometimes get the above error
To answer your question, I confirm that pageSize is set to 10.

That line "fires" the above query: you need to try that using SSMS (not API) and benchmark it. There must be something odd in the name column and/or in the DMBS itself.

I have implemented the ToSql method extensions, this is the query I get when reaching GetCities()
SELECT [c].[Id], [c].[Name], [c].[Lat], [c].[Lon], [c0].[Id] AS [CountryId], [c0].[Name] AS [CountryName] FROM [Cities] AS [c] INNER JOIN [Countries] AS [c0] ON [c].[CountryId] = [c0].[Id] ORDER BY [c].[Name] OFFSET @__p_0 ROWS FETCH NEXT @__p_1 ROWS ONLY
and here is the message I get from SSMS
Msg 137, Level 15, State 2, Line 5 Must declare the scalar variable "@__p_0". Msg 137, Level 15, State 2, Line 5 Must declare the scalar variable "@__p_1".

Replace p0 and p1 with the actual values (you can see them if you put a breakpoint, execute the command and look at the output window with the "debug" log active): they should respectively be 0 and 10, just like I wrote early on.
Please check the query with SSMS to see if the slowness is there.

everything works fine in SSMS when I replace @__p_0 and @__p_1 with respectively 0 and 10.

I have noticed that if I wait a few minutes without touching anything and then click on cities or countries, everything works fine and is fast

Definitely a mistery: I cannot reproduce it by any means.


data/manifests/musan_cuts.jsonl.gz can not found when running alimeeting train script ,plz check script issue

FileNotFoundError: [Errno 2] No such file or directory: 'data/manifests/musan_cuts.jsonl.gz'
plz check the prepare script put the musan_cuts.jsonl.gz to data/fbank , not data/manifests.
i solved it by copy the musan_cuts.jsonl.gz file from data/fbank to data/manifests
plz check if the prepare script and trainning scipt mismatch

thank you so much!
this pr should fix this issue: https://github.com/k2-fsa/icefall/pull/1750


Transfer connection url when opening session

Why are the changes needed?

Now we support multiple frontend services.
We need transfer the connection url into session to identify the connection type.
It can also fix #3575
How was this patch tested?
Pass existing UT.

how do you think about this? @yaooqinn

Codecov Report

Merging #3577 (768646b) into master (8419b7b) will increase coverage by 0.02%.
The diff coverage is 95.23%.

@@             Coverage Diff              @@
##             master    #3577      +/-   ##
============================================
+ Coverage     51.74%   51.77%   +0.02%
  Complexity       13       13
============================================
  Files           482      482
  Lines         26975    27001      +26
  Branches       3762     3762
============================================
+ Hits          13959    13980      +21
- Misses        11661    11665       +4
- Partials       1355     1356       +1

Impacted Files
Coverage Δ

...ache/kyuubi/server/KyuubiRestFrontendService.scala
79.74% <ø> (-1.21%)
:arrow_down:

...g/apache/kyuubi/client/api/v1/dto/SessionData.java
41.86% <50.00%> (+0.83%)
:arrow_up:

.../engine/spark/session/SparkSQLSessionManager.scala
80.00% <100.00%> (+0.25%)
:arrow_up:

...kyuubi/engine/spark/session/SparkSessionImpl.scala
75.60% <100.00%> (+5.02%)
:arrow_up:

...apache/kyuubi/service/AbstractBackendService.scala
98.36% <100.00%> (ø)

...a/org/apache/kyuubi/service/TFrontendService.scala
91.20% <100.00%> (-0.27%)
:arrow_down:

...la/org/apache/kyuubi/session/AbstractSession.scala
96.33% <100.00%> (+0.03%)
:arrow_up:

...ala/org/apache/kyuubi/session/SessionManager.scala
86.01% <100.00%> (ø)

.../org/apache/kyuubi/events/KyuubiSessionEvent.scala
96.55% <100.00%> (+3.00%)
:arrow_up:

...rg/apache/kyuubi/server/BackendServiceMetric.scala
93.44% <100.00%> (ø)

... and 18 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Instead of continuously changing these func definitions, I would expect all newly add information to propagate via configurations

good idea

thanks, addressed.

cc @pan3793  @ulysses-you

thanks @ulysses-you @yaooqinn , merged to master

hi @lightning-L  could you rebase your pr on this?

Seems CI hangs starting from this PR. @turboFei @lightning-L would you please take a look?


Implement Email Verification Feature for User/Canteen Accounts

Description
This pull request introduces comprehensive email verification functionality for user accounts. Prior to this implementation, users could register without email verification. With this update, users are required to verify their email addresses upon registration before being able to log in. Additionally, options for resending verification emails and disabling email verification have been added, enhancing user control over their account security. Three new email templates have been included: one for requesting email verification and another for confirming successful email verification. Furthermore, a third email template notifies users if their verification token has expired. This feature greatly improves user communication and security throughout the verification process.
Changes Made

Implemented email verification feature, making it mandatory for user registration.
Users are unable to log in until their email address is verified.
Added resend verification email functionality.
Added option to disable email verification via changes in the .env file.
Included three email templates: one for requesting email verification, one for confirming successful email verification, and one for notifying users if their verification token has expired.
Enhanced user control over account security.
Utilized Gmail's SMTP server and nodemailer for sending emails.

Testing

Thoroughly tested email verification process, including registration, verification, resend functionality, and disabling.
Ensured proper rendering of email templates.
Verified integration with Gmail's SMTP server for sending emails.

Additional Notes

This feature significantly enhances user account security and ensures valid user contact information.
Improves user experience with clear email communication throughout the verification process, including notification of token expiration.

Screenshots

Email Verification Email

Successfully Verified

Token Expired

Resend Verification Email Page

Related Issues

Fixes #9

@Slacky300 Can you share your discord id !! I will accept this PR but i want to make some changes

@Slacky300 Can you share your discord id !! I will accept this PR but i want to make some changes

@VanshKing30 here's my discord id:<PHONE_NUMBER>70716702

Not this one
eg : Vansh5238

Not this one eg : Vansh5238

You mean username? then it's: rehman_khan_300


[Bugfix] Sync cpu_cores of BEs when replaying master FE info to other FEs (#9580)

What type of PR is this：

[x] bug
[ ] feature
[ ] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #9581.
Problem Summary(Required) ：

This is cherry-picked from #9580。

run starrocks_fe_unittest

run starrocks_fe_unittest


Missing icon mapping

The following RBB icons are not mapped to any Amchart icon:

RBB key
Icon

300002

300003

300004

300005

300006

If one of these icons appears in the RBB data, there is no mapping and no icon is displayed in the module.

RBB provides SVG graphics for their icons, e.g. key 300006:
https://www.rbb24.de/basis/grafik/icons/wetter/svg/300006.svg

A possible solution is to implement the ability to switch between Amchart and RBB icons with a config flag. It is also possible to use RBB icons as a fallback if no Amchart icon is mapped (maybe this should be the first enhancement).


[BACKLOG-21083] Database Connection: Override driver class for a dialect

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  core/src/main/java/org/pentaho/di/core/database/BaseDatabaseMeta.java
  core/src/main/java/org/pentaho/di/core/database/GenericDatabaseMeta.java
  dbdialog/src/main/java/org/pentaho/ui/database/event/DataHandler.java
  dbdialog/src/main/resources/org/pentaho/ui/database/databasedialog.properties
  dbdialog/src/main/resources/org/pentaho/ui/database/databasedialog.xul
  dbdialog/src/main/resources/org/pentaho/ui/database/generic_native.xul

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.core.database.BaseDatabaseMeta

Complexity Change: -0.5% :small_red_triangle_down:
Instruction Change: -0.4% :small_red_triangle_down:
Line Change: -0.6% :small_red_triangle_down:
Method Change: -1.1% :small_red_triangle_down:

org.pentaho.di.core.database.GenericDatabaseMeta

Branch Change: -66.9% :small_red_triangle_down:
Complexity Change: -69.4% :small_red_triangle_down:
Instruction Change: -64.9% :small_red_triangle_down:
Line Change: -72.8% :small_red_triangle_down:
Method Change: -76.8% :small_red_triangle_down:

org.pentaho.di.job.entries.sql.JobEntrySQL

Instruction Change: -3.3% :small_red_triangle_down:
Line Change: -3.9% :small_red_triangle_down:

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change: -1.3% :small_red_triangle_down:
Complexity Change: -1.9% :small_red_triangle_down:
Instruction Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Branch Change: -2% :small_red_triangle_down:
Complexity Change: -1.7% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.synchronizeaftermerge.SynchronizeAfterMergeMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta

Branch Change: -0.9% :small_red_triangle_down:
Complexity Change: -0.8% :small_red_triangle_down:

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta

Branch Change: -0.5% :small_red_triangle_down:

org.pentaho.di.trans.steps.salesforceinsert.SalesforceInsertMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.salesforceupsert.SalesforceUpsertMeta

Instruction Change: -0.1% :small_red_triangle_down:

License header violations

:heavy_exclamation_mark: Copyright year is not the current year on core/src/main/java/org/pentaho/di/core/database/BaseDatabaseMeta.java. Found 2017. Was expecting 2018

:heavy_exclamation_mark: Copyright year is not the current year on core/src/main/java/org/pentaho/di/core/database/GenericDatabaseMeta.java. Found 2017. Was expecting 2018

Checkstyle violations
The code submitted is not complying with one or more of our guidelines. To perform these checks locally, please use this tool.
/home/buildguy/project/core/src/main/java/org/pentaho/di/core/database/BaseDatabaseMeta.java

:heavy_exclamation_mark: ')' is not preceded with whitespace. at Line: 2,298 Column: 70

/home/buildguy/project/core/src/main/java/org/pentaho/di/core/database/GenericDatabaseMeta.java

:heavy_exclamation_mark: '(' is not followed by whitespace. at Line: 137 Column: 52

:heavy_exclamation_mark: ')' is not preceded with whitespace. at Line: 286 Column: 105

:heavy_exclamation_mark: '(' is not followed by whitespace. at Line: 881 Column: 38

:heavy_exclamation_mark: ')' is not preceded with whitespace. at Line: 881 Column: 69

:heavy_exclamation_mark: '(' is not followed by whitespace. at Line: 883 Column: 26

:heavy_exclamation_mark: ')' is not preceded with whitespace. at Line: 883 Column: 57

/home/buildguy/project/dbdialog/src/main/java/org/pentaho/ui/database/event/DataHandler.java

:heavy_exclamation_mark: 'for' is not followed by whitespace. at Line: 249 Column: 8

:heavy_exclamation_mark: 'if' is not followed by whitespace. at Line: 1,234 Column: 9

:heavy_exclamation_mark: 'if' child has incorrect indentation level 8, expected level should be 6. at Line: 1,330

:heavy_exclamation_mark: 'if' has incorrect indentation level 8, expected level should be 6. at Line: 1,331

:heavy_exclamation_mark: 'if' is not followed by whitespace. at Line: 1,331 Column: 11

:heavy_exclamation_mark: 'if' child has incorrect indentation level 10, expected level should be 8. at Line: 1,332

:heavy_exclamation_mark: 'if rcurly' has incorrect indentation level 8, expected level should be 6. at Line: 1,333

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  core/src/main/java/org/pentaho/di/core/database/BaseDatabaseMeta.java
  core/src/main/java/org/pentaho/di/core/database/GenericDatabaseMeta.java
  dbdialog/src/main/java/org/pentaho/ui/database/event/DataHandler.java
  dbdialog/src/main/resources/org/pentaho/ui/database/databasedialog.properties
  dbdialog/src/main/resources/org/pentaho/ui/database/databasedialog.xul
  dbdialog/src/main/resources/org/pentaho/ui/database/generic_native.xul

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.core.database.BaseDatabaseMeta

Complexity Change: -0.5% :small_red_triangle_down:
Instruction Change: -0.4% :small_red_triangle_down:
Line Change: -0.6% :small_red_triangle_down:
Method Change: -1.1% :small_red_triangle_down:

org.pentaho.di.core.database.GenericDatabaseMeta

Branch Change: -66.9% :small_red_triangle_down:
Complexity Change: -69.4% :small_red_triangle_down:
Instruction Change: -64.9% :small_red_triangle_down:
Line Change: -72.8% :small_red_triangle_down:
Method Change: -76.8% :small_red_triangle_down:

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.csvinput.CsvInputMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Branch Change: -2% :small_red_triangle_down:
Complexity Change: -1.7% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta

Branch Change: -1.8% :small_red_triangle_down:
Complexity Change: -1.1% :small_red_triangle_down:
Instruction Change: -0.4% :small_red_triangle_down:
Line Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.ivwloader.IngresVectorwiseLoader

Branch Change: -0.7% :small_red_triangle_down:
Complexity Change: -1.1% :small_red_triangle_down:
Instruction Change: -0.6% :small_red_triangle_down:
Line Change: -0.9% :small_red_triangle_down:

org.pentaho.di.trans.steps.ivwloader.IngresVectorwiseLoader.FifoOpener

Branch Change: -16.7% :small_red_triangle_down:
Complexity Change: -12.5% :small_red_triangle_down:
Instruction Change: -7.1% :small_red_triangle_down:
Line Change: -10.5% :small_red_triangle_down:
Method Change: -20% :small_red_triangle_down:

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.setvalueconstant.SetValueConstantMeta

Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.sort.SortRowsMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta

Instruction Change: -0.1% :small_red_triangle_down:

Checkstyle violations
The code submitted is not complying with one or more of our guidelines. To perform these checks locally, please use this tool.
/home/buildguy/project/dbdialog/src/main/java/org/pentaho/ui/database/event/DataHandler.java

:heavy_exclamation_mark: 'if' is not followed by whitespace. at Line: 1,331 Column: 9

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  core/src/main/java/org/pentaho/di/core/database/BaseDatabaseMeta.java
  core/src/main/java/org/pentaho/di/core/database/GenericDatabaseMeta.java
  dbdialog/src/main/java/org/pentaho/ui/database/event/DataHandler.java
  dbdialog/src/main/resources/org/pentaho/ui/database/databasedialog.properties
  dbdialog/src/main/resources/org/pentaho/ui/database/databasedialog.xul
  dbdialog/src/main/resources/org/pentaho/ui/database/generic_native.xul

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.core.database.BaseDatabaseMeta

Complexity Change: -0.5% :small_red_triangle_down:
Instruction Change: -0.4% :small_red_triangle_down:
Line Change: -0.6% :small_red_triangle_down:
Method Change: -1.1% :small_red_triangle_down:

org.pentaho.di.core.database.GenericDatabaseMeta

Branch Change: -66.9% :small_red_triangle_down:
Complexity Change: -69.4% :small_red_triangle_down:
Instruction Change: -64.9% :small_red_triangle_down:
Line Change: -72.8% :small_red_triangle_down:
Method Change: -76.8% :small_red_triangle_down:

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta

Branch Change: -1.8% :small_red_triangle_down:
Complexity Change: -1.1% :small_red_triangle_down:
Instruction Change: -0.4% :small_red_triangle_down:
Line Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change: -1% :small_red_triangle_down:
Complexity Change: -1% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:
Line Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.ifnull.IfNullMeta

Branch Change: -2% :small_red_triangle_down:
Complexity Change: -1.9% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.insertupdate.InsertUpdateMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.pgbulkloader.PGBulkLoaderMeta

Branch Change: -0.9% :small_red_triangle_down:
Complexity Change: -0.9% :small_red_triangle_down:

org.pentaho.di.trans.steps.systemdata.SystemDataMeta

Branch Change: -2% :small_red_triangle_down:
Complexity Change: -2% :small_red_triangle_down:
Instruction Change: -0.3% :small_red_triangle_down:
Line Change: -0.8% :small_red_triangle_down:

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta

Branch Change: -0.9% :small_red_triangle_down:
Complexity Change: -0.8% :small_red_triangle_down:

org.pentaho.di.trans.steps.webservices.WebServiceMeta

Branch Change: -1.7% :small_red_triangle_down:
Complexity Change: -1.2% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.ui.repository.pur.repositoryexplorer.model.UISecurityRole

Branch Change: -1.8% :small_red_triangle_down:
Complexity Change: -1.5% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:
Line Change: -0.5% :small_red_triangle_down:

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'dbdialog,core' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  core/src/main/java/org/pentaho/di/core/database/BaseDatabaseMeta.java
  core/src/main/java/org/pentaho/di/core/database/GenericDatabaseMeta.java
  dbdialog/src/main/java/org/pentaho/ui/database/event/DataHandler.java
  dbdialog/src/main/resources/org/pentaho/ui/database/databasedialog.properties
  dbdialog/src/main/resources/org/pentaho/ui/database/databasedialog.xul
  dbdialog/src/main/resources/org/pentaho/ui/database/generic_native.xul

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.core.database.BaseDatabaseMeta

Complexity Change: -0.5% :small_red_triangle_down:
Instruction Change: -0.4% :small_red_triangle_down:
Line Change: -0.6% :small_red_triangle_down:
Method Change: -1.1% :small_red_triangle_down:

org.pentaho.di.core.database.GenericDatabaseMeta

Branch Change: -66.9% :small_red_triangle_down:
Complexity Change: -69.4% :small_red_triangle_down:
Instruction Change: -64.9% :small_red_triangle_down:
Line Change: -72.8% :small_red_triangle_down:
Method Change: -76.8% :small_red_triangle_down:

org.pentaho.di.job.entries.evalfilesmetrics.JobEntryEvalFilesMetrics

Branch Change: -0.4% :small_red_triangle_down:
Complexity Change: -0.5% :small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Branch Change: -0.7% :small_red_triangle_down:
Complexity Change: -0.6% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:
Line Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.checksum.CheckSumMeta

Branch Change: -1.2% :small_red_triangle_down:
Complexity Change: -1.4% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.csvinput.CsvInputMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change: -4.1% :small_red_triangle_down:
Complexity Change: -4.2% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:
Line Change: -0.7% :small_red_triangle_down:

org.pentaho.di.trans.steps.ifnull.IfNullMeta

Branch Change: -2% :small_red_triangle_down:
Complexity Change: -1.9% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change: -1.1% :small_red_triangle_down:
Complexity Change: -1.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.setvalueconstant.SetValueConstantMeta

Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.webservices.WebServiceMeta

Branch Change: -1.7% :small_red_triangle_down:
Complexity Change: -1.2% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.gpload.GPLoadMeta

Branch Change: -0.9% :small_red_triangle_down:
Complexity Change: -0.8% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

Note: The list of offending coverage changes extends beyond the presented results and was truncated to prevent reaching the comment size limit.


Flexbox issue : justify-* and items-* are reversed

Describe the bug
justify-* makes vertical alignments but should be aligned horizontally
items-* makes horizontal alignments but should be aligned vertically
To Reproduce
import {Text, View} from "react-native";

const StyledView = styled(View);

export default function App() {
    return <StyledView className={`flex-1 items-start justify-end`}>
        <Text>123</Text>
    </StyledView>;
}

Expected behavior
The text should be on the top right corner

I just understood that we should specify the flex direction first...


rules_oci does not work on linux arm64 hosts due to jq dependency

While resolving toolchains for target //bin/services/external_proxy:image: No matching toolchains found for types @aspect_bazel_lib//lib:jq_toolchain_type.
To debug, rerun with --toolchain_resolution_debug='@aspect_bazel_lib//lib:jq_toolchain_type'
If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro.

//bin/services/external_proxy:image being a oci_image target
if this is a known issue perhaps stating compatible architectures might be handy

I was able to work around this issue by building my own jq arm64 binary and registering a toolchain within my workspace locally:
on a mac arm64 host:
docker run --rm -it ubuntu:18.04
apt update
apt upgrade
apt install git build-essential make libtool autoconf
git clone https://github.com/jqlang/jq
git submodule update --init --recursive
autoreconf -fi
./configure --with-oniguruma=builtin
make -j8

load("@aspect_bazel_lib//lib/private:jq_toolchain.bzl", "jq_toolchain")

jq_toolchain(
    name = "jq_linux_arm64_tools",
    bin = ":jq", # commited prebuilt jq binary locally
)

toolchain(
    name = "jq_linux_arm64_toolchain",
    exec_compatible_with = [
        "@platforms//os:linux",
        "@platforms//cpu:arm64",
    ],
    toolchain = ":jq_linux_arm64_tools",
    toolchain_type = "@aspect_bazel_lib//lib:jq_toolchain_type",
)

FYI: This can soon be resolved using yq since stamping support is coming.

if anyone is looking for a quick patch-around, this works for my own usecases but I make no guarantees that it works with all: https://github.com/archen/rules_oci/pull/1

Tested out your above patch and although it looks like it might break something else based on the tests it did let me keep developing on my linux/arm64 machine. I don't think I will ever purchase another arm machine 😞

@michaeljs1990 yeah I haven't had time to replace all of the filtration in the image.sh.tpl with yq-compat. It's funny... ARM hasn't been as painful as I thought it would be, but I sure am glad I'm using bazel to compile tools from code as needed for my dev corps instead of relying on prebuilds.

See https://github.com/bazel-contrib/rules_oci/pull/337#issuecomment-1718621054

Will be fixed by #386

When will we upgrade to bazel-lib 2.0? Is there a definite schedule? This hinders some tests (all my personal machines are arm64, linux/mac).

fixed by https://github.com/bazel-contrib/rules_oci/pull/386

When will we upgrade to bazel-lib 2.0

That will happen with the 2.x version of rules_oci, which we are already landing PRs to.


Existing check on login table / migrations

Just a check to make sure the table doesn't already exist before creating

closed, one moment


Permission denied in mutate job

Hi, i'm trying to use the mutate_job service but it's giving me errors. The next code is the example i use to try to pause one campaign.
campaign_path = client.path.campaign(customer_id, campaign_id)
operation = client.operation.update_resource.campaign(campaign_path) do |c|
  c.status = :PAUSED
end
mutate_operation = client.operation.mutate
mutate_operation.campaign_operation = operation

batch_job_srv = client.service.mutate_job
resource_name = batch_job_srv.create_mutate_job(customer_id.to_s).resource_name
batch_job_srv.add_mutate_job_operations(resource_name, '', [mutate_operation])
batch_job_srv.run_mutate_job(resource_name)

All works fine but the response is User doesn't have permission to access customer, while the synchronous way have permission and pause the campaign correctly.
Thanks

Sorry I missed this issue. That feature hasn't yet been released widely. The library should be working fine but you're being rejected from the API side. Once the API side lifts the restriction it should work fine.


Fix error message for the GDScript utility function

Currently, if fails, it prints the message like:

After the fix, it will be correct:

And added some minor naming fixes

I wrote #70948 before seeing this. My problem with the approach here is that the names are really only needed in debug builds, so adding them to the bytecode is wasteful. Having different bytecode for debug and release are also not ideal since this can introduce subtle bugs in one of the targets when changing something.
Though I missed the names in the error messages in #70948 so this PR could still be used for this after mine is merged.

Needs a rebase (or redo from scratch?) based on #70948.

Ah, this need to be re-created from scratch. Currently, I lost interest.


Add tests and automated publishing the package to NPM

This PR:

introduces automated publishing of the package using the changeset action. The setup is identical to the one in single-spa-react
introduces Jest tests on the API

For automated publishing to work, it is necessary that:

The GITHUB_TOKEN is granted permissions to create and approve pull requests. This can be done in Settings -> Actions -> General -> Workflow permissions -> Allow GitHub Actions to create and approve pull requests
The admin of the repo creates an NPM_TOKEN secret with an NPM token with write permissions to the organization (or at least the corresponding package, for publishing).

Thanks for your work on this, these tests will help a lot. I've left a few comments and questions
Do I need to add an NPM_TOKEN environment variable to this repo before merging it?

For automated publishing to work, you will need to do the following, before merging, @joeldenning:

The GITHUB_TOKEN is granted permissions to create and approve pull requests. This can be done in Settings -> Actions -> General -> Workflow permissions -> Allow GitHub Actions to create and approve pull requests
The admin of the repo creates an NPM_TOKEN secret with an NPM token with write permissions to the organization (or at least the corresponding package, for publishing).

I read through the rollup documentation linked to in the warning you shared (https://rollupjs.org/guide/en/#warning-treating-module-as-external-dependency) and the solution seems to be to add @rollup/plugin-node-resolve rather than to change it to a CJS import. Is there an error even when rollup node resolve is present?
Have you tested that import-map-overrides works server-side after this change?

To unblock this PR I have reverted all changes related to the warning I get when building import-map-overrides-server.js.
I tracked the problem in this new issue and I can tackle it in a separate PR.


Add 'bullet' package

Official:

http://bulletphysics.org/wordpress/

Official GitHub repo:

https://github.com/bulletphysics/bullet3

Note: CMake based

Available in https://github.com/ruslo/hunter/releases/tag/v0.19.190


Add Ability to Subdivide Tabs

Rather than limiting tab divisions to Exclusively Horizontal or Vertical, allow users to subdivide in both directions.

This does not seem related to anything in the toolchain.


Need documentation on returning task status from gradle tasks

https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/build/gradle?view=azure-devops
There's no documentation on how you can return the status of a gradle task.
https://docs.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml says that you can make a step do something if the Agent.JobStatus is SucceededWithIssues. How do I make a gradle build task return SucceededWithIssues? I want to create a task that either returns Succeeded in most cases but if there are some issues we want to warn about but not break the build return SucceededWithIssues.

@vijayma How can this be accomplished?
We have a Azure DevOps pipeline that has a stage failed but pipeline is "SuccededWithIssues". Any idea about this too?


Update README.md

Updated user guide to cover new features
Minor changes asked by client in the previous demo session

Codecov Report

Merging #22 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #22   +/-   ##
=========================================
  Coverage     83.55%   83.55%
  Complexity      118      118
=========================================
  Files            13       13
  Lines           383      383
  Branches         23       23
=========================================
  Hits            320      320
  Misses           55       55
  Partials          8        8

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a04c4d...9505cbb. Read the comment docs.


Google drive verification code

Hi,
I am running the script on a remote cluster which has no gshell cmodule. Instead I am using gdrive. I am wondering could you send me the verification code for downloading the ckpts and data from the google drive address?
email<EMAIL_ADDRESS>

Hi, gshell does not seem to be supported anymore, but it should be straightforward to go to the Google Drive links following the readme and download the data from there. Another possibility is to use gdown.


Fix broken link in README.md to "Introducing Ramda by Buzz de Cafe"

Seems like Buzz de Cafe now doesn't support the forward slash (/) at the end.
Replace :
http://buzzdecafe.github.io/code/2014/05/16/introducing-ramda/
with:
http://buzzdecafe.github.io/code/2014/05/16/introducing-ramda

gah, updated jekyll this weekend. did not realize this was a side-effect


Cyrillic clipboard from remote to local buggy

Hello, dear support team and developer.
I found new issue with clipboard when copying Cyrillic text from remote to local.
It doesn't represent correctly.
Tested with HTML 5 websocket with 2.5.5 latest version.
Please see attached screenshots.

Hi, thanks for reporting this issue. I normally use UTF8 encoding into Myrtille to support any character set, but there must be  some parts of the code which don't. It may also be related to FreeRDP because Myrtille read the clipboard content from it.
I was thinking to use a base64 encoding to send the clipboard content, then decode it into the browser. I may eventually do this.

#158 duplicate


Document generator for Exported XML schema

Do we have any document generator for the exported xml from Office PnP Schema?
Please provide me some pointers.

You can find the schema details and additional documentation from https://github.com/OfficeDev/PnP-provisioning-schema repository where the remote provisioning schema is version. Would also suggest to use the PnP Yammer group in future for these kind of questions to get faster response on your questions. PnP Yammer group has more than 4000 members and you can find that from http://aka.ms/OfficeDevPnPYammer.


feature: Webpack Chunking

This PR modifies our Webpack bundling strategy to follow several common patterns used elsewhere in the industry.
Creates a minimal chunk for each App
Creates a minimal commons chunk for each App
Create a framework chunk for @artsy framework code to ensure that it will not be cache-invalidated
Create a framework chunk for core framework code to ensure that it will not be cache-invalidated
Creates library chunks for any large node_module dependencies
Creates shard chunks for any code used by multiple pages
Additional Optimizations

Reduces the number of low KiB bundles by using naming common preloaded chunks
Restricts bundles to 240 KiB unzipped
Cleans up legacy code that was accidentally imported by the orders page

Notable Impacts:
Reduces the Artist page JS bundle payload by ~275 KiB compressed

Nice! 👍


Remove object attribute from Panel class

Closes #546
Inheriting classes define this attribute individually.

Thanks! I think the _get_objects method should probably also be moved to the ListPanel class.


Rename the default branch to main

The default branch for this repository will soon be renamed from master - a potentially offensive term - to main. Many other GitHub communities are also renaming their default branches. To learn more about this welcome news, check out Git’s statement about why branch renaming matters.
You may need to make changes if you have a local copy or fork of this repository. You can view our guidance on this Github issue.

We’ve just finished renaming the default branch in this repo.
If you’re working on a contribution and keep a local copy or fork, you might need to make some changes.


Represent TIMESTAMP WITH TIME ZONE as ZonedDateTime in JDBC

I found this PR valuable.
Although the implementation of getObject is not backward compatible and it's not 100% consistent with respect to offset-literal-timestamps with JVM timezone,  I found it much valuables due to the ability to perform timezone conversions with AT TIME ZONE.
Here is an example to illustrate this:
JVM timezone is set to "America/Bahia_Banderas" (UTC-5)

SQL text
Before
After

SELECT current_timezone()
America/Bahia_Banderas
America/Bahia_Banderas

SELECT timestamp '2019-05-27 10:00:00'
2019-05-27 10:00:00
2019-05-27 10:00:00

SELECT timestamp '2019-05-27 10:00:00 +05:17'
2019-05-26 23:43:00
2019-05-27 10:00:00

SELECT timestamp '2019-05-27 10:00:00 UTC'
2019-05-27 05:00:00
2019-05-27 10:00:00

SELECT timestamp '2019-05-27 10:00:00' AT TIME ZONE 'UTC'
2019-05-27 10:00:00
2019-05-27 15:00:00

SELECT timestamp '2019-05-27 10:00:00 UTC' AT TIME ZONE 'UTC'
2019-05-27 05:00:00
2019-05-27 10:00:00

SELECT ts_col FROM tbl (tbl.ts_col is a timestamp column)
2019-05-27 10:00:00
2019-05-27 10:00:00

SELECT ts_col AT TIME ZONE 'UTC' FROM tbl
2019-05-27 10:00:00
2019-05-27 15:00:00

@findepi @electrum @dain

Based on https://github.com/prestosql/presto/pull/6152, other than that hopefully ready for review.

Depends on https://github.com/prestosql/presto/pull/6208, but otherwise ready for review.


Could you show your config file? You may mixed up the key pretrained and load_from.

@UrwLee
@CheckpointLoader.register_scheme(prefixes='')
def load_from_local(filename, map_location):
"""load checkpoint by local file path.
Args:
    filename (str): local checkpoint file path
    map_location (str, optional): Same as :func:`torch.load`.

Returns:
    dict or OrderedDict: The loaded checkpoint.
"""

if not osp.isfile(filename):
    raise IOError(f'{filename} is not a checkpoint file')
checkpoint = torch.load(filename, map_location=map_location)
checkpoint = _process_mmcls_checkpoint(checkpoint)
return checkpoint

pretrained

yes, you are right, i need to download the pretrain weights instead


Add uchicago to edu domain

School URL: https://www.uchicago.edu/en
Full school name: The University of Chicago

@yixinliu99 I am sorry, the domain uchicago.edu is removed from our trusted domains list because it was abused in the past. If you are the student of this school, please enroll to GitHub Student Developer Program. Then apply for JetBrains educational subscription with your GitHub account (GitHub tab at https://www.jetbrains.com/shop/eform/students).


Problem with prepopulated data via submission

Hi,
I'd like to pre-populate rendered form with some values. So I've added EventEmitter and binded it to [refresh] output. Then I've emitted some values as submission. It's working well. But when I want to edit any pre-populated data in form, old values are kept in (change) and (submit) event (change event is detected correctly but it doesn't have any affect on event.data). What's really strange is that when I edit data inside datagrid update is working fine. But adding new row to datagrid or updating textfields which are inside containers is not working at all.
Is there any solution to this problem?
I'm using angular-formio 3.11.6 and formiojs 3.20.0 with Angular 7.
Thanks for your help!

It would be helpful to have a jsfiddle of this issue that you are experiencing so that we can take a look.

Sorry for bothering you. There was problem with additional dependencies and I accidentialy rendered form before emitting submission values which cause that strange behaviour.

Closing this thread as it is outdated. Please re-open if it is still relevant. Thank you for your contribution!


react-scripts imports all NodeJS types into browser-facing code

Describe the bug
By default, the TypeScript template creates react-app-end.d.ts which includes:
/// <reference types="react-scripts" />

The react-scripts module in turn includes react-app.d.ts which has the following reference:
/// <reference types="node" />

This results in all NodeJS types being imported into the project, even if they have been excluded by setting "types": [] in the tsconfig.json. So, if I try to use setTimeout(), TypeScript sees both the DOM version of this function and the  version from @types/node, which are not compatible (the latter returns an instance of the NodeJS.Timeout class instead of a numeric reference).
Even worse is that the @nodejs/types types import all the DOM types that are supported in the latest version of NodeJS. And since NodeJS supports all the same things as a modern version of Chromium, it results in very new things like BigUint64Array being treated as valid types, even if the project is explicitly targeting and older ES lib in tsconfig.json.
Did you try recovering your dependencies?

N/A
Which terms did you search for in User Guide?

typescript node types incorrectly included
Environment

  current version of create-react-app: 3.4.1
  running from /Users/jonathanrimmer/.npm/_npx/56315/lib/node_modules/create-react-app

  Binaries:
    Node: 13.8.0 - ~/.nvm/versions/node/v13.8.0/bin/node
    Yarn: 1.22.10 - /usr/local/bin/yarn
    npm: 6.13.6 - ~/.nvm/versions/node/v13.8.0/bin/npm
  Browsers:
    Chrome: 86.0.4240.80
    Firefox: Not Found
    Safari: 14.0
  npmPackages:
    react: ^16.13.1 => 16.13.1
    react-dom: ^16.13.1 => 16.13.1
    react-scripts: 3.4.3 => 3.4.3
  npmGlobalPackages:
    create-react-app: Not Found

Steps to reproduce

npx create-react-app cra-types-bug --template typescript
Open App.tsx
Add const handle: number = setTimeout(() => {}, 1000);

Expected behavior
It should work without errors.
react-scripts should reference only those specific NodeJS types, such as process that it makes available at build-time. It should not reference the entirety of NodeJS's type declarations.
Actual behavior

TypeScript thinks it is calling the Node version of setTimout and gives an error:

Reproducible demo

https://github.com/jonrimmer/cra-types-bug

git clone https://github.com/jonrimmer/cra-types-bug.git
cd cra-types-bug
yarn build

Just to clarify, in case anyone else found this issue by google, you can bypass the error by explicitly access the window object, e.g. window.setTimeout for the example above.

I believe the issue still exists, it just hasn't received any attention.


Update README-Japanese.md

Adding language bar to make it consistent with README.md

Changes made in patch


bot.rust-lang.org misconfiguration results in 404s

19:02 < elly> bot.rust-lang.org 404?
19:02 < brson> i see it
19:03 < elly> bot.rust-lang.org is serving me github's "Page does not exist!" page
19:03 < brson> ha!
19:04 < brson> I get that all the time at the office
19:04 < brson> and have to use the IP
19:04 < brson> bot.rust-lang.org is not even hosted on github
19:04 < elly> then why is it an alias for graydon.github.com?
19:05 < brson> that doesn't sound right
19:05 < elly> ~$ host bot.rust-lang.org
19:05 < elly> bot.rust-lang.org is an alias for graydon.github.com.
19:05 < elly> that's asking <IP_ADDRESS>
19:05 < elly> <IP_ADDRESS> says the same
19:05 < elly> something's screwy
19:06 < elly> also the 'normal' record has ttl 300, but the bogus record has ttl 42993
19:08 < elly> I wonder where honestdns got that from

Fixed (hopefully).


Your very awesome plugin....

Does not seem to work with 41.2 anymore...

@kaipradel does work for me, are you using the latest version? Please use manual updates, instead of Sketch toolbox


Replace kotlin-compile-testing with our own code

Since tschuchortdev/kotlin-compile-testing#336 seems to be going unaddressed, we investigated forking the library to discover that it is actually rather simple to code up the bits we require for ourselves.  This PR, then, removes our dependency on external compile-testing libraries, replacing them with direct invocations of the Kotlin compiler.

Codecov Report

Merging #533 (406c2a8) into master (29aab79) will not change coverage.
The diff coverage is n/a.

 Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Whisper finetuning support for other languages

Is there a plan to support whisper finetuning for English?

already support, just add these lines to your yaml


GH Actions

Description
Adds actions consistent with other Pusher SDKs. Uses labels to automatically bump versions and publish releases.
CHANGELOG

[ADDED] GH actions for automatic releases.

Codecov Report

Merging #22 (508181d) into master (f6580db) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master       #22   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            6         6
  Lines          291       296    +5
=========================================
+ Hits           291       296    +5

Impacted Files
Coverage Δ

...ec/pusher/push_notifications/configuration_spec.rb
100.00% <100.00%> (ø)

spec/pusher/push_notifications/delete_user_spec.rb
100.00% <100.00%> (ø)

...c/pusher/push_notifications/generate_token_spec.rb
100.00% <100.00%> (ø)

...er/push_notifications/publish_to_interests_spec.rb
100.00% <100.00%> (ø)

...pusher/push_notifications/publish_to_users_spec.rb
100.00% <100.00%> (ø)

spec/pusher/push_notifications/client_spec.rb
100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04344aa...508181d. Read the comment docs.


Colors specified in palette.gpl instead of toolbar.ini

Hello,
in the recent update of Xournal, specifiing color in the toolbar.ini file does not work anymore. The colors are converted to index numbers of palette.gpl file. The change is discussed here and documented here
I made such palette.gpl file with the dracula colors. Modified the toolbar.ini to reflect them and added a line into installation.md
As far as I know there is not yet a better way to switch palettes than manually.
Hope this helps someone:)
Before and After images

Hi @BasileosFelices, sorry for being so slow, I have merged this in e39c469 with a couple small tweaks! (Couldn't figure out how to get the GitHub UI to show it as merged properly, sorry)


Enable parameterization of buttons and labels

The current labels for "Remove all", "Upload all" and the progress ("x% of y...") are hardcoded into the components. Please enable them to be parameterized, so we can translate/change them in our apps.

Now it supports labels for "Remove all", "Upload all"


multiple TXT records should not always be concatenated.

This library concatenates all strings in a TXT record. Many (probably most) applications treat the records individually.
See for example:
http://www.zeroconf.org/rendezvous/txtrecords.html
The thing with concatenating TXT records seem to come from some weird email RFC (SPF?) but I can't find anything about it in the more generic mdns RFCs. Anyway I think it is better to let applications interpret the records themselves.

Have you seen this issue on the real data? I think the mistake of the current implementation is that it assumes that multiple records can be present in a single answer (i.e. single RRData). I hardly remember that I've investigated that an reached a conclusion that it should not be possible. So while current implementation is inefficient and doesn't handle non-utf8 data it should handle two records easily.
But, I have to study the RFC again to be sure :)
I.e. I think we need to change txt::Record to struct Record<'a>(pub &'a [u8]) (because TXT records are actually binary).

I have seen it. Currently I'm parsing data coming from video surveillance cameras (of the brand Axis) and they sometimes have multiple fields in one record. Also rendezvous is Apples implementation of mdns/zeroconf/bonjour/whatever and since they make it very easy to create several fields in a record I think chances are there many apple devices and applications that might have it as well. I don't have any apple devices though so I can't test it.
You can look up mDNSResponderPosix if you want to try it out. It's fairly well used in devices such as routers and other IOT devices to provide Zeroconf services and is simple to configure.
I agree [u8] is better!

Okay, you're right, sorry for confusion.


Add denoising algorithm to points

Research and find a way to denoise the output data using temporal cohesion

Look if it’s possible to train a custom AI for mediapipe that can do this

This issue is way to vauge. In the future a kalman filter might be useful but currently this is not an issue


Add an option to set anchor for UserLocationMarker

Right now you can set the anchor for regular markers, via the addMarker function, but it is not possible to change the anchor of a user's marker.

I'm already working on that for 0.61.0 #432
we only need to add impl for web


Max line length

Small update to have max-line-length as a argument.
Sorry, but didn't have time to make tests pass. Some magic happens below :stuck_out_tongue:
Closes https://github.com/DanielNoord/pydocstringformatter/issues/90

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

2 of 2 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
471

Relevant Lines:
471

💛  - Coveralls

Thanks once again @mdczaplicki, also for prompting me to final fix our CI 🎉


C++/162_Find_Peak_Element.cpp

seems that this problem is also included in #11
I'll just close it and merge the other one.


[SPARK-43836][BUILD] Make Scala 2.13 as default in Spark 3.5

What changes were proposed in this pull request?
This PR aims to make Scala 2.13 the default Scala version in Apache Spark 3.5.
Why are the changes needed?
The current releases of Scala community are Scala 3.2.2 and Scala 2.13.10.

https://scala-lang.org/download/all.html

Although the Apache Spark community has been using Scala 2.12 by default since Apache Spark 3.0 and Scala community will release Scala 2.12.18 for Java 21 support, we had better focus on Scala 2.13+ more from Apache Spark 3.5 timeline to adopt Scala community's activity.
Since SPARK-25075 added Scala 2.13 at Apache Spark 3.2.0, the Apache Spark community has been using it as a second Scala version. This PR aims to switch only the default Scala version from 2.12 to 2.13. Apache Spark will support both Scala 2.12 and 2.13 still.
Does this PR introduce any user-facing change?
Yes, but we still have Scala 2.12.
How was this patch tested?
Pass the CIs.

All tests passed.

Thank you, @MaxGekk . Merged to master.

Thank you @dongjoon-hyun
This is quite a big change for Spark. Perhaps we should inform everyone with an email to @dev. After all, it's holidays now, so I don't think everyone knows about this PR.

Yes, I agree with you, @bjornjorgensen . I'm also planning to send out this as a part of the followings Apache Spark 3.5.0 Expectations soon.

SPARK-40497 Upgrade Scala to 2.13.11
SPARK-43347 Remove Python 3.7 Support
SPARK-43447 Support R 4.3.0
SPARK-43831 Build and Run Spark on Java 21
SPARK-43832 Upgrade Scala to 2.12.18
SPARK-43836 Make Scala 2.13 as default in Spark 3.5

I'm very happy to see this change @dongjoon-hyun :)

Thank you always in many ways, @LuciferYang ! :)

For the record, I sent an email to dev@spark here.

https://lists.apache.org/thread/3x6dh17bmy20n3frtt3crgxjydnxh2o0 Apache Spark 3.5.0 Expectations (?)


Upgrading ruby buildpack to v1.4.1

Upgrading to latest stable version of ruby buildpack
--Buildpacks Team

Please note that this is the "skinny" buildpack.


Required labels rely on fragments folder which may not be there

Currently, patchomator assumes that /usr/local/Installomator/fragments  exists and is populated with data from the Installomator github repo.
The problem is that if --write hasn't been run interactively on the workstation, that folder never gets generated.
When using --required, this problem results in Errors being reported despite no actual Installomator errors occurring.
Please add a function to check for this folder, and if it doesn't exist then populate it.
Alternately: Perhaps the fragments folder never needs to live locally on the device at all? All of the information from that folder is contained within the Installomator.sh script.
Here is an example of how to scrape the name= label variable from the label googlechromepkg. Perhaps the dependency on the fragments folder can be discarded if this method is used to grab the necessary data.
sed -n '/# label descriptions/,$p' /usr/local/Installomator/Installomator.sh | grep -i -A 50 "googlechromepkg" | grep -m 1 "name=" | sed 's/.*=//' | sed 's/"//g'
This same trick can be used to get any label value from any ${label}

It's an order of operations problem I didn't anticipate. Initially, I wasn't really planning on patchomator running unattended.
The way the labels are used is each one is run as a sub-script to check the current version, etc. Parsing through Installomator.sh each time would probably slow things down (though I'd have to try it to be sure).
I'll move the self-update/install steps higher in the script to make sure those fragments are there and up to date before patchomator tries to read them.


OCR don't detect my sentence

Hello,
I take a screenshot, always the same thing, to detect the phrase "Votre demande de paris a bien été prise en compte".
But, sometimes, the sentence is not in .TXT. Why ??  Tesseract V5 alpha.
Originals Images here :
https://mega.nz/file/1whm2JLR#lVUc46As7Glv-vm--F5KcPbU-9ROJinSQE1ZL2zpRos

Please respect guidelines for posting issue: use tesseract user forum for asking questions/support.
Do not forget to READ DOCUMENTATION before posting topic.


[AmortizedLDA] Unintentionally lower bounded cell_topic_posterior_sigma

https://github.com/YosefLab/scvi-tools/pull/1356#discussion_r809391190
As Pierre mentioned in this comment, the already non-negative variance output by the Encoder is softplussed which lower bounds around 0.6.

let's get this into 0.15.2

Closed by #1445


Lambda Layer - August Release blog post

This PR is publish Release blog for Lambda layer release

LGTM other than Resolve conflicts


[UI] - Rebuild button is gone

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
The issue has been spotted by Cypress:
1) Applications testing
       Push basic application and check we can restart and rebuild it:
     AssertionError: Timed out retrying after [100](https://github.com/epinio/epinio-end-to-end-tests/runs/6686673320?check_suite_focus=true#step:3:101)00ms: Expected to find content: 'Rebuild' within the selector: 'li' but never did.

Expected Behavior
We should see the rebuild option in the menu.
Steps To Reproduce

Deploy latest epinio in the Rancher UI with epinio-dev branch
Create an app and try to rebuild it via its menu

Environment
- OS: Linux
- Architecture: x86_64
- Epinio Version: main
- Cluster:
  - Provider: k3s
  - Options:
  - Kubernetes Version:

I verified that deploymend.stage_id is undefined here: https://github.com/rancher/dashboard/blob/8dce4f9c6f726dff95161ffc2a67fe976874695b/pkg/epinio/models/applications.js#L131
I also verified that the epinio server is sending the stage_id over the wire:
$ curl -k  -u admin:password https://epinio.<IP_ADDRESS>.omg.howdoi.website/api/v1/namespaces/workspace/applications/sample4 | jq

{
  "meta": {
    "name": "sample4",
    "namespace": "workspace",
    "createdAt": "2022-06-07T11:51:12Z"
  },

...

  "status": "running",
  "statusmessage": "",
  "stage_id": "ac9e56e05f3ae853",
  "image_url": "registry.epinio.svc.cluster.local:5000/apps/workspace-sample4:ac9e56e05f3ae853"
}

So the problem must be somewhere between receiving the reponse and unmarshaling it to an object. I'm not sure where this happens in the front end code.

ok found it, the stage_id is not under deployment, it's a top level key. Simply changing the deploymend.stage_id to just stage_id makes the button appear (and work).

I just realized we have this field both as a top level (on the app) and on the workload.
Looking at the code, we set a label on the deployment when one is set in the deployment request: here
That's is the stage id we return when we "show" the app: here
We need to check if the "deploy" request has the stage id set correctly.

I verified that the "deploy" request had the Stage ID set:
req = {App:{Meta:{Name:sample13 Namespace:workspace CreatedAt:0001-01-01 00:00:00 +0000 UTC}} Stage:{ID:e0e73d481354852d} ImageURL:registry.epinio.svc.cluster.local:5000/apps/workspace-sample13:e0e73d481354852d Origin:{Kind:1 Container: Git:<nil> Path:sample-app.zip}}

I think I know now. The app deployment has the label epinio.suse.org/stage-id: e0e73d481354852d but we've recently changed all epinio.suse.org to epinio.io.  The problem is that the helm-chart still refers to the last application chart release which doesn't have this change.
I think we just need to cut a new release of the application chart and bump it.

I cut a new release of the application helm chart: https://github.com/epinio/helm-charts/releases/tag/epinio-application-0.1.18
I'll bump it in the epinio chart now and see.

I works! I created a couple of PRs to bring the fix to epinio/epinio


[AutoPR hdinsight/data-plane] [HDInsight] - Add FINISHING to application state

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/4906

(message created by the CI based on PR content)

This PR has been merged into https://github.com/Azure/azure-sdk-for-python/pull/4056


feature: Created class for Chart Info (chart.yaml)

@zbrookle hey zach when you get a chance take a look at this PR