[EDGCONX-45/EDGCONX-46] - Enhance HTTP Endpoint Security with TLS and FIPS-140-2 Compliant Cryptography

https://folio-org.atlassian.net/browse/EDGCONX-45
WebServer creation occurs in edge-common, so additional changes in particluar module for this story is not needed.
https://folio-org.atlassian.net/browse/EDGCONX-46
OrdersOkapiClinentFactory is responsible for creating WebClients with SSL or without SSL support based on system properties that is implemented in scope of this story

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: azizbekxm:x: BKadirkhodjaevYou have signed the CLA already but the status is still pending? Let us recheck it.


[move-prover] Changes to bytecode instruction set to allow writeback to have edge information.

Motivation
Changes to bytecode instruction writeback to allow for edge information.
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
Current coverage is fine (as edge information not actually used yet).

Please give the PR a descriptive title, and add [move-prover] before the title.

Oops. Accidentally the PR title as branch name!

/land


Highlight command line components in color

Command line components should have a color difference on web pages. This should also be done for pieces of the program text for various languages (Python, Solidity, Rust, bash, for example).
npm install <options>

Command line items are now highlighted for the line marked "bash". Marking the command line as "sh" does not work.

I would also like to change the background of CLI to a dark one, as it is now done in solana docs


#62

Airplane.active now an integer, 0 1 or 2. 0 is off screen not moving, 1 is off screen moving, 2 is on screen and moving and can crash into other airplanes.
Function for starting location will be called again if starting location is less than 100 pixels away from another airplane's location.

PR #70 overrides these changes, closing for #70


The style of the body that was in the demo.html has been sent to the pell.css file

I'm a student, I'm learning to collaborate on open source projects. It was a small collaboration, I hope I have helped.

Maybe it's not a big contribution but most of all it's an idea or option that the text editor might have.

What @hgm1284 said. 😄


Enable cross-version support for Scala sources.

This allows attaching unmanaged sources on per-ScalaVersion basis.
For example, with scalaBinaryVersion set to 2.11, the files in
src/{main,test}/scala-2.11 as well src/{main,test}/scala would be
available in the unmanagedSources list.
This makes unmanagedSources consistent with managedSources (which is allowed to be scala version specific).
This is one of the older techniques which I initially felt would get irrelevant with time. But I am finding myself still using this repeatedly :)
Consider this more of a bouncing off idea with @jsuereth and @eed3si9n . We first need to see if this makes sense OOTB, and if it does, should we just append scalaBinaryVersion to scalaSource with no user control (like in the case of managedSources) or should we allow users some control via, say crossSource key (yet another Key!).

Can one of the admins verify this patch?

I like the idea, and I agree it becomes something which should be default.
Ideally we avoid having a flag to enable this, and instead just silently add src/main/scala-<binaryVersion> and src/main/scala-<fullVersion> to the list of sources.
However, for compatibility, it probably makes sense to have a flag (or autoplugin) which enables this.

+1. I think it makes sense to have this one out of the box.
I think we should be symmetric with the crossTarget and only add src/main/scala-<binaryVersion>. The reuse of crossPaths is fine with me.

scripted source-dependencies/relative-source-error is failing on Travis. I restarted it and failing on the same test.

Having flag is something I am wanting to avoid now -- instead users can override unmanagedSourceDirectories completely.
Indeed, using scalaBinaryVersion instead of of full scalaVersion is to be consistent with crossTarget. This is also to stay consistent with sbt's treatment of scalaBinaryVersion pre-2.10 and 2.10 onwards (see the updated commit message for more). The unintended side effect of this is that src/{main,test}/scala-2.10 would be included but src/{main,test}/scala-2.9 would not be included (src/{main,test}/scala-<scalaVersion> would be included instead). We have to choose between consistency with crossTarget's behavior and element of surprise (why src/{main,test}/scala-2.9 is not selected would be un-obvious).
scripted source-dependencies/relative-source-error could have been due to docker tmp mount point issue on Travis (have had this earlier). This didn't occur on my local env or even on the Travis build of my fork.
Regardless, have restructured a bit and added a small test to test the functionality.

LGTM


Terraform tutorials and nav

New section for Terraform with:
Export logs
Export metrics
Manage database
Provision a cluster (already existed)
https://cockroachlabs.atlassian.net/browse/DOC-7037
https://cockroachlabs.atlassian.net/browse/DOC-7038

@marksoper I still have some placeholder text to fill in, but what do you think of this page structure?

Reopening here https://github.com/cockroachdb/docs/pull/17509 due to rebase issues


Is Osprey implemented in current version or in the future plan?

Osprey: Weak Supervision of Imbalanced Extraction Problems without Code
https://ajratner.github.io/assets/papers/Osprey_DEEM.pdf
This paper looks very promising and convenient. Is there an implementation or there will be one?

Hi @holdonyb great question!  I can't speak for the version running at Intel, but the main concept here is that we can build interfaces (e.g. excel spreadsheet-based ones like in Osprey) that sit on top of, and compile down to, Snorkel.  This is something we are indeed working on- stay tuned!
There are also a number of other pipeline innovations, somewhat specific to the Osprey setting as described, that are separate from this interface- these we will add to the list; however right now pushing on a big new overall release and refactor (v0.9) so we'll revisit after that's out in early August!
Are there any specific components / aspects of Osprey that you are particularly interested in seeing in Snorkel?  Any detail you can give is great!

Closing for now, feel free to re-open!

@ajratner any update on this? The balancing, interface and generative/discriminative model blending are all of interest to me as I have faced a partnership extraction task myself and this paper directly addresses the huge cost in time, software and Turks I spent gathering this business graph dataset: https://blog.datasyndrome.com/open-sourcing-relatos-business-graph-database-fca220daadd8

Oh, and is the code for Osprey published somewhere? I can’t find it.

Hi @rjurney unfortunately any code release up to my co-authors (at Intel)!  As for some of that gen / disc model blending, not on the immediate road map, but should be straight forward to try out if you want!

Understood. Yeah, their ensemble game method is a one line definition in the paper so it seems easy enough to add to the book, just reference, paraphrase and implement an extension to the text extraction model I’m building already. Thanks!


Bug/turbo/42

Fixed the bug for multiple handlers for different verb under same path.
Updated the signatures to respond with route and error.
Updated the func names for  typed fetch for Path and Query params.

Closes #42


YouTube script refactor, removing shorts from feed

Describe your changes

Refactored script to use dataclasses (acting as interfaces like those in interfaces.ts for the website)
Excluded YouTube shorts from feed

Used ayncronous approach to remain performant.

Added per-channel limit for YouTube videos displayed (improving scalability to many channels)

Related issue number/link
Fixes #200
and other changes not mentioned in issues

Will merge tomorrow/day after once the YT API quota resets.


Multiple issues with getting-started.md

Mention the black pins that should hold the ribbon cable in.
Add warning that "Arduino ESP32 Boards" is not the same as 'esp32'. I made this mistake, presumably others will too.
Latest v2 version of esp32 is v2.0.18, I assume this is OK.
I'd like to explain more about 'Choose an example' but I'm not sure what to say.
I've addressed some of the minor grammatical errors.

Other issues, not addressed here:
Image shows that the vibrating engine needs to be attached, which is no longer the case.
Instructions say to insert ribbon cable before inserting battery, but figure shows battery being connected before the ribbon cable.


Signal not emitted for ScrollContainer

**Godot version:3.1

OS/device including version:

Issue description:

When the scrolling is triggered by dragging on the scroll bar or by the InputEventPanGesture, no scroll_started or scroll_started is emitted.
Steps to reproduce:
Minimal reproduction project:

Can reproduce in 3.1 thru 3.2.3.rc3.
Can get the scrollbars and use their scrolling signal, but nothing from the container itself.
MRP: 31819.zip

Closing as duplicate of #22936


obtain orbital average kinetic energy?

This might be more of a physics question than a strictly pyscf related issue, in which case I'm sorry! Just getting started in this field.
I am trying to compute ionization cross sections of heavy atoms (e.g. Xe, I) using a BEB method. To do so, I need occupancy number, binding energy and average kinetic energy of each occupied orbital. The first two I can easily obtain with pyscf (I used an HF calculation), however it is not entirely clear to me how I could go about obtaining the latter. Is it possible to obtain with the SCF methods included in pyscf? If so, how would I go about doing that?
Thanks in advance!

You can compute the orbital kinetic energy by contracting the orbital coefficients C = mf.mo_coeff with the kinetic energy matrix T = mol.intor_symmetric('int1e_kin'). For example
import pyscf

mol = pyscf.M(
    atom = 'H 0 0 0; F 0 0 1.1',  # in Angstrom
    basis = 'ccpvdz',
    symmetry = True,
)

mf = mol.HF()
mf.kernel()

occs = mf.mo_occ
C = mf.mo_coeff
T = mol.intor_symmetric('int1e_kin')
for icol, occ in enumerate(occs):
    if occ>0.0:
	print(f'Orbital {icol+1} kinetic energy {C[:,icol].T @ T @ C[:,icol]}')

prints out
$ python 00-simple_hf.py
converged SCF energy = -99.9873974403488
Orbital 1 kinetic energy 37.254262161040366
Orbital 2 kinetic energy 3.8278166626243366
Orbital 3 kinetic energy 2.462086461589103
Orbital 4 kinetic energy 3.1640185085938737
Orbital 5 kinetic energy 3.1640185085938737

This is exactly what I was looking for, thank you!


Wrong React import in code examples

Documentation
Relevant Radix Component(s)
The React import is wrong in the code examples on the getting started page. It should be 'react', not 'React'.
I could make a PR to fix it on this page but it would probably be best to check the whole project for this mistake.
Also, is the import even necessary? Back in the day we needed to import React for JSX to work, but in modern projects it should't be an issue I think? If I paste the code in a .jsx or .tsx file, the editor complains about an unused variable.

If you have time to run a quick find and replace for erroneous imports we'd certainly appreciate it.

Also, is the import even necessary?

It's environment specific really and I don't see the harm in keeping them, they can always be omitted later if not needed.

If you have time to run a quick find and replace for erroneous imports we'd certainly appreciate it.
Sure, I'll fork the repo and submit a PR
It's environment specific really and I don't see the harm in keeping them, they can always be omitted by the consumer if not needed.
Yeah, makes sense


fix ci/install.ps1

From the AppVeyor CI results in a previous PR, I noticed that in environments with Python 3.9 and later, Test-Path $env:PYTHON in ci/install.ps1 returns False, and it falls back to Python 3.4.
As a result, even in cases where UIA_SUPPORT=YES, comtypes is not actually installed, and the functionality of backend='uia' is not being tested.
I'm experimenting with various solutions to fix this.
I tried removing the fallback conditional, and while it worked on Python 3.8 and earlier, but it failed on Python 3.9 and later.

(Is C:\\Python3x\Scripts\pip.exe truly missing from the env?)

I welcome other maintainers to contribute to resolving this issue.
Thank you.

I've made several commits for trial and error, and I've been investigating various aspects.
What I've found is that the default Build worker image (when image: is not specified) doesn't support the newer versions of Python.
https://www.appveyor.com/docs/windows-images-software/#python
On the other hand, when specifying a newer Build worker image, we encounter issues like failing to install packages specified in dev-requirements.txt, or even when the installation succeeds, some tests fail.
It seems like we'll need to use appropriate workarounds for each Python version.
Any opinions would be appreciated.

By specifying APPVEYOR_BUILD_WORKER_IMAGE separately in appveyor.yml, the tests have been successful in environments with Python 3.8 and below.
https://www.appveyor.com/docs/build-environment/#using-multiple-images-for-the-same-build
However, issues arise in environments that support newer Python versions like Visual Studio 2019 and Visual Studio 2022. Installation of Pillow fails in Python 3.9, and PyQt5 installation fails in Python 3.10 and Python 3.11.
Does anyone know of a workaround for this issue?

Python 3.10 and Python 3.11 were successfully installed with the latest version of PyQt5 and by using nose2 instead of nose. (nose has been in maintenance mode for a while, and it became unusable with Python 3.10 due to the removal of collection.Callable.)
For Python 3.9, it will probably install successfully if you can specify the appropriate version of Pillow.
However, I'm not very familiar with workarounds for installing Pillow or with the command-line optional arguments for nose and nose2. I haven't been able to handle this properly.
If anyone knows more about this, please advise.

At 70ee60b, pywinauto.unittests.test_application.DesktopUiaWindowSpecificationTests.test_folder_list was failing.
However, at 99b8c9f, adding a command to show file extensions in File Explorer in runTestsuite.ps1, test_folder_list is no longer failing.
It appears that the different default behaviors of the GUI, depending on the image specified in AppVeyor, might be the reason for the test failures.

Are we using nose instead of Python's built-in unittest command in our CI because we want to exclude testall.py? As previously mentioned, nose doesn't work with all Python versions supported by pywinauto, and we need to switch to nose2.
If we move testall.py to a different directory, we should be able to use Python's built-in unittest command (or a combination of coverage and `unittest).
What is the reason for continuing to use nose?


include web Mercator projection (EPSG:3857) in ERDDAP WMS for using easy leaflet maps with nice basemaps

CC: @ehazen, @7yl4r,
Hi @BobSimons,
Thank you, thank you for a great and useful product of ERDDAP focused on the many ways to slice, dice and represent data for a plethora of analysis and visualization.
One of the more useful and easy representations of data is as a Web Map Service (WMS) to easily drop into a "slippy" online map using JavaScript libraries like leaflet and all the pretty basemap's like Esri.OceanBasemap and others from Leaflet Provider Demo:

However, the ERDDAP WMS currently only serves geographic projection EPSG:4326 and not the web Mercator EPSG:3857 needed to use these great basemaps. Heres' some futzing I did trying various combinations to make this work:
WMS Testing for Fixing Explorer Environmental Map

Could this be a matter of simply adding or updating instances in these Java files where 4326 shows up?

EDDTableFromMultidimNcFiles.java

EDDTableFromNcCFFiles.java

Thanks very much, Ben

Yes. "Add support for Web Mercator projection" is already on the To Do list.
Because of the nature of the graphics library that ERDDAP uses to generate maps (which was selected long ago, before most of the current libraries existed), it is not possible to simply add a new projection. So it is a more difficult project. Similarly, it would/will not be easy to switch to another graphing library. The To Do list is long. I just haven't done this item yet. I'm sorry.

Ok, glad to hear Web Mercator projection for slippy maps is on your TODO. Too bad it's not an easy fix. I'm not a Java programmer, but please let me know if I can help.
Seems like a common approach for serving WMS image and other OGC services (WCS raster, WFS vector) is to setup a GeoServer alongside ERDDAP for the same data sources. For instance, I see this with these projects:

PacIOOS
CeNCOOS
IOOS Animal Acoustic Telemetry (AAT)

This might not be the best place to ask, but do you have any pointers on how to best to link ERDDAP and GeoServer?
Ideally, you could discover one through the other (eg metadata mention of alternate providers through catalog search), but not sure where such information would live other than highest level of a website.
CC'ing Axiom folks who work setup docker-erddap and docker-geoserver in case they might have any tips: @kwilcox, @shane-axiom, @rsignell-usgs

@bbest I'm going to guess the dataset you are trying to access through ERDDAP's WMS is netCDF based, and the short answer is that netCDF and GeoServer don't play nicely together. There was some movement towards a GeoServer netCDF backend at some point.. maybe 8 years ago... but I believe progress stalled and was never completed. This could use a little bit of research (let me know if you find anything recent).
If you are looking for WMS from a netCDF data source in a projection other than 4326, you'll have to look towards:

ncWMS
sci-wms

Can you link me to the ERDDAP dataset you are trying to plot? I'll poke at it to see if it moves.

IIRC when I looked at the GeoServer netCDF reader a few years ago we decided against using it because it didn't handle a lot of real world complexities (e.g. anti-meridian wrapping) and didn't include support for community conventions like CF because it didn't make use of Unidata's netcdf-java library. That being said, if your use case isn't complex you might take a look and see if it works for you.

Thank you very much @kwilcox and @srstclair for weighing in on this given all your experience. I'm generally interested in quickly visualizing remotely sensed variables like SST and chlorophyll globally (ie WMS), as well as summarizing these data over time by arbitrary polygons (ie ERDDAP slicing/fetching). For example from the
example I previously shared WMS Testing for Fixing Explorer Environmental Map, I used the Multi-scale Ultra-high Resolution (MUR) SST Analysis using R's leaflet:
addWMSTiles(
    baseUrl = 'https://coastwatch.pfeg.noaa.gov/erddap/wms/jplMURSST41mday/request?',
    layers = "jplMURSST41mday:sst",
    options = WMSTileOptions(
      version = "1.3.0", format = "image/png", transparent = T, opacity = 0.7,
      time = "2018-07-16T00:00:00Z"))

The ncWMS and sci-wms look great, particularly for slicing by time and variable. The  NetCDF — GeoServer 2.15.x User Manual suggests good support for COARDS compliance (custom, Time, Elevation, Lat, Lon).
I'm also working with MBON and interested in being able to programmatically search the catalog of datasets in the portal at https://mbon.ioos.us developed by Axiom, but not sure how to do that, even with the excellent documentation. Do you know the base ERDDAP URL? I'm able to search through the GeoServer OGC GetCapabilities endpoints, but that's just a short name and not the other metadata, hence interest in how datasets could be cross-tagged across ERDDAP and GeoServer.

WCS (raster): https://data.axds.co/gs/ows?service=WCS&version=2.0.1&request=GetCapabilities
WFS (vector): https://data.axds.co/gs/ows?service=wfs&version=2.0.0&request=GetCapabilities
WMS (image): https://data.axds.co/gs/ows?service=wms&version=1.3.0&request=GetCapabilities

@kwilcox @srstclair @bbest  I dug up some old links on GeoServer and netCDF support I had been forwarded previously.  NOAA/NWS had funded recently (< 8 yrs ago) some work to allow GeoServer to support a particular model dataset (Rapid Refresh: RAP) in GRIB2 .  Info here:
https://github.com/bencaradocdavies/geoserver/wiki/RAP-Native-Grid and
https://github.com/bencaradocdavies/geoserver/wiki/RAP-Native-Grid-ImageMosaic
Including use of netCDF-Java to read the source data to feed to GeoTools and in turn GeoServer (see the first link).
However, I'm not sure whether they were merged into the primary GeoServer product or if it's only a custom build as outlined in the wiki link.  It looks like part of the project involved adding support for the specific RAP grid to netCDF-Java, so clearly netCDF-Java is part of the equation for at least this fork.
There's a lot of configuration involved in order to make GeoServer read netCDF (or GRIB2 in this case) via GeoTools ImageMosaic, so that's still a limitation/inconvenience, but maybe even so there's a path forward to make ERDDAP and GeoServer play well together.   I would be interested to see that.  It could go a ways towards adding some of the functionalities you're looking for to ERDDAP (or maybe alongside ERDDAP).
Maybe the magic of GitHub mentions will help, if @bencaradocdavies can share if this has gone anywhere since he worked on it.

@mwengren @kwilcox @srstclair @bbest the current GeoServer NetCDF ("netcdf") plugin is built on the Unidata NetCDF-Java library and is well supported. I added a NetCDF example in the GeoServer Quickstart in the OSGeoLive image.
The NetCDF Output ("netcdf-out") plugin is used for WCS delivery of NetCDF 3 and 4. I do not think this what you want.
The RAP Native work I did in 2016-2017 added projection support and netcdf-out improvements. It was all merged into GeoServer 2.9. The use-case in the links above is the most complicated, and involves a mosaic that concatenates multiple GRIB2 files for delivery as NetCDF via WCS. GeoServer configuration to deliver WMS from NetCDF is much, much simpler.

Connecting ERDDAP and GeoServer has been on my list of things to To Do (or at least explore) for years. If any of you want to pursue it, please do. If, as it sounds, there is a netcdf-java to GeoServer connection, then it seems likely that there would be a way to take all the information that an ERDDAP has about its datasets and automatically make corresponding datasets in GeoServer (at least for the gridded datasets). I suspect that the current netcdf-java to GeoServer connection assumes that the data is gridded geographic data. It would also be interesting to see if the tabular datasets in ERDDAP which have a CF DSG/cdm_data_type/featureType could also be made into appropriate datasets in GeoServer and hence available via GeoServer's WFS. If anyone wants to work on this and wants to bounce ideas off me, please email me.

@BobSimons I suspect that you are right: as far as I know, GeoServer supports gridded NetCDF data sources ("coverages") but not NetCDF vector data sources (equivalent to ERDDAP tabular?). The GeoServer devel list would be a great place to start. I am no longer active in the GeoServer project but there are several active developers supporting NetCDF data sources. The NetCDF libraries are there; support would require a GeoTools DataStore implementation and a GeoServer plugin to package it and possibly add a web UI for configuration. A prototype DataStore could even be a GSoC project.

@bencaradocdavies thanks for the update!
That would be great if an ERDDAP GeoTools DataStore implementation could be built, at least for gridded datasets.  I like the idea of a GSoC project as well.
@BobSimons if you wanted to suggest that, you could go through the ESIP GSoC repository to suggest it, see: https://github.com/ESIPFed/gsoc/issues.  I don't know if they are still accepting new project ideas or not, but they may still be!
Someone would have to provide guidance from the GeoTools standpoint too, since it would obviously be developed in GeoTools.  OSGeo also participates in GSoC.  Maybe someone in the GeoTools project could be looped in later.  What do you think?
On the projection question, I am not sure if gridded geographic data is a requirement or not, it may be that as long as the projection can be specified within GeoTools/GeoServer, any projection may work (maybe someone knows better about this than I do).  GeoServer can reproject dynamically, of course at a performance cost.

@mwengren the geotools-devel mailing list should be the first stop for anyone contemplating a new DataStore implementation because you can find other interested developers and mentors. The GeoTools Developer Guide has more. It is straightforward to get permission to create a new unsupported module and the commit access to maintain it. Contributors need to submit signed code contribution agreements; this can be the slowest step for employees of bureaucracies!

Thank you all for weighing in on this issue around connecting GeoServer and ERDDAP. Perhaps I don't understand the suggestions for more direct integration between GeoServer and ERDDAP, but having a separate data catalog that describes the various representations and web services for a given dataset seems the way to go to me.
The logical choice for data catalog software is CKAN, which is in keeping and replicable with data.ioos.us (see github.com/ioos/catalog) and data.gov (see github.com/GSA/data.gov).
This way the underlying dataset storage could be much more flexible than any one suite of software and be transformed for optimization of services. For instance, netcdf could be served through ERDDAP, but also transformed into a stack of TIFFs, which may be more efficient for serving through GeoServer (as suggested by this tutorial Using the ImageMosaic plugin for raster time-series data — GeoServer 2.16.x User Manual).
Decoupling the data catalog from the underlying data representations and web services enables many more possibilities, while ensuring queryable access through the CKAN API with Python and R clients.
I'm curious what you all think about this approach which seems to already be in place?
Overview
netcdf → ERDDAP → various formats (csv, png, ...)   ↘
  ↓                                                   → CKAN
tiff → GeoServer → OGC services (WMS, WFS, WCS,...) ↗

datasets | data.ioos.us
Note how ERDDAP and other web services using GeoServer (ie WMS, WCS) are both visible and queryable for dataset search and browse:

dataset detail | data.ioos.us
Note how all the links are available through the catalog to a variety of representations served by different endpoints, ie ERDDAP, GeoServer or other...

...

The system you propose may make sense for various reasons, but not for the
immediate need here and not for the other uses I had hoped for, which is to
allow ERDDAP to use the WMS/mapping/reprojection features of GeoServer to
make Web Mercator maps (and do lots of other things). For these needs and
for ease of setup, I think it would be great if an administrator could
install ERDDAP and GeoServer, and then have all of the datasets in ERDDAP
automatically also show up as datasets in GeoServer.  ERDDAP has aggregated
datasets (not just a bunch of .nc files), with metadata about each dataset
(enhanced over what is in the .nc files) and it would be nice if that
information could be used to automate the connection to GeoServer. Then,
ERDDAP could know that there are corresponding datasets in GeoServer and
thus use/rely on those services. With your architecture, those things are
not true and it would be more work for the administrator (setting up
datasets separately in ERDDAP, GeoServer, and CKAN), and working to make
the metadata the same in ERDDAP, GeoServer and CKAN..
But that's just my view for my/ERDDAP's purposes. Anyone can set up any
architecture they want.
On Mon, Mar 18, 2019 at 11:55 AM Ben Best<EMAIL_ADDRESS>wrote:

Thank you all for weighing in on this issue around connecting GeoServer
and ERDDAP. Perhaps I don't understand the suggestions for more direct
integration between GeoServer and ERDDAP, but having a separate data
catalog that describes the various representations and web services for a
given dataset seems the way to go to me.
The logical choice for data catalog software is CKAN https://ckan.org/,
which is in keeping and replicable with data.ioos.us (see
github.com/ioos/catalog) and data.gov (see github.com/GSA/data.gov).
This way the underlying dataset storage could be much more flexible than
any one suite of software and be transformed for optimization of services.
For instance, netcdf could be served through ERDDAP, but also transformed
into a stack of TIFFs, which may be more efficient for serving through
GeoServer (as suggested by this tutorial Using the ImageMosaic plugin for
raster time-series data — GeoServer 2.16.x User Manual
https://docs.geoserver.org/latest/en/user/tutorials/imagemosaic_timeseries/imagemosaic_timeseries.html
).
Decoupling the data catalog from the underlying data representations and
web services enables many more possibilities, while ensuring queryable
access through the CKAN API https://docs.ckan.org/en/2.8/api/ with
Python https://github.com/ckan/ckanapi and R
https://github.com/ropensci/ckanr clients.
I'm curious what you all think about this approach which seems to already
be in place?
Overview
netcdf → ERDDAP → various formats (csv, png, ...)   ↘
↓                                                   → CKAN
tiff → GeoServer → OGC services (WMS, WFS, WCS,...) ↗
datasets | data.ioos.us
Note how ERDDAP and other web services using GeoServer (ie WMS, WCS) are
both visible and queryable for dataset search and browse:
[image: image]
https://user-images.githubusercontent.com/2837257/54555122-4fc6b500-4973-11e9-93c7-039c82d6de4d.png
dataset detail | data.ioos.us
Note how all the links are available through the catalog to a variety of
representations served by different endpoints, ie ERDDAP, GeoServer or
other...
[image: image]
https://user-images.githubusercontent.com/2837257/54555165-6705a280-4973-11e9-9abd-ac3721964e3c.png
...
[image: image]
https://user-images.githubusercontent.com/2837257/54555195-72f16480-4973-11e9-913b-cb1fcbb4fbf9.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/BobSimons/erddap/issues/24#issuecomment-474054997,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABarOIMmfDz-jDWN3ZsgPohMFDlz5-ksks5vX-EvgaJpZM4apFpZ
.

--
Sincerely,
Bob Simons
IT Specialist
Environmental Research Division
NOAA Southwest Fisheries Science Center
99 Pacific St., Suite 255A
Monterey, CA 93940
Phone:<PHONE_NUMBER>
Fax:<PHONE_NUMBER>
Email<EMAIL_ADDRESS>The opinions in this message are mine personally and do
not necessarily reflect any position of the U.S. Government
or the National Oceanic and Atmospheric Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><

Ah thank you @BobSimons for this extra insight. I will be developing a data management portal for IEA ecosystem status indicators, so will become more familiarized with ERDDAP configuration. In the process I can chip away at sync'ing datasets between ERDDAP and GeoServer initially with some simple scripts and keep these considerations in mind for trying to generalize more broadly for a tighter integration. Everybody's input here has been much appreciated 🙏

Apologies for hijacking this thread with a tangent, but: are ERDDAP, CKAN, THREDDS, (others?) all providing a similar "middleman" functionality?
I have minimal exposure to CKAN and THREDDS and want to better understand the niche of each of these relative to ERDDAP.

@BobSimons @bbest I think both pursuits would be worthwhile.  I think @BobSimons is considering general purpose ERDDAP use, and I totally agree about the value of a GeoServer 'extension' point for ERDDAP (if that's an appropriate phrase).  I would love to see that happen.  @bbest it sounds like your goal is to develop a purpose-built application on both of those tools, as well as CKAN for the data catalog - also a good idea, in my book!  Maybe they should be separated into distinct issues though to avoid confusion.
@bbest on the architecture you showed, I would add that there is some - possibly manual - metadata configuration that happens in the middle in order for the CKAN representation to look the way it does.  PacIOOS does a phenomenal job maintaining their metadata WAFs, but I don't know how much of it is a fully automated process.  Either way, there would be some additional software to stitch together the TIFF-based services emanating from GeoServer and those from ERDDAP.  Nonetheless, no reason it wouldn't work!

I view them as significantly different. Each has its own niche that makes
each better for certain purposes.
THREDDS is a DAP+WMS+WCS data server (just for gridded data) that just
provides 1 catalog view: a hierarchy of terms, eventually terminating in
either data files or aggregated datasets at the leaves. THREDDS has its own
metadata system which is separate from the dataset's metadata. So if you
specify, e.g.,  a new summary, it appears in the THREDDS catalog for the
dataset, not in the dataset's .das metadata (although there may be a way to
do that separately). The THREDDS catalog has no system for full-text
searches, which is why e.g., NOAA groups and others uses/used CKAN,
Lucene/Solr, or ESRI GeoPortal for catalog services. For brokering, THREDDS
can only point to the URLS of datasets in other THREDDS (and simplistically
to other URLs), but since there are no catalog services that is no help for
searching.
CKAN is a more general purpose catalog system, not a data server (in any
significant way). As such, it is useful in that it can point to all kinds
of data services in all kinds of remote data servers. But since the data
sources are diverse and because CKAN makes no effort to broker those data
services, there is no consistent way to access data from the different
datasets.
ERDDAP is a dataserver (for gridded datasets (slightly more limited than
THREDDS) and tabular datasets) with catalog services (full text search,
faceted search, view all, and advanced search) for its datasets.  ERDDAP
emphasizes brokering: ERDDAP seeks to be able to connect to data from lots
of remote data services (e.g., THREDDS, GRADS, DAPPER, relational
databases, Cassandras, AWS S3 buckets, WFS servers, other ERDDAPs, ...) and
present those datasets in exactly the same way that it presents local
datasets (e.g., .nc files, ,csv files, and lots of specialized file types).
I hope that is reasonably fair. Some of that is debatable. I hope that
helps.
 Then,
another goal of ERDDAP is to encourage better metadata for each of those
datasets. (I don't think THREDDS or CKAN view it the same way.) Part of the
problem is that the community's idea of "good metadata" changes over time
(e.g,. new versions of CF, ACDD, and ISO 19115) so ERDDAP helps by making
it easy to modify each dataset's metadata via the ERDDAP datasets.xml setup
for each dataset, without editing/changing each data file (which e.g., NCEI
doesn't want to do because they want to preserve the original data
submission) or trying to get a remote data source to change the metadata.
Then, if each dataset is aggregated and has extensive, good metadata, then:

Search in ERDDAP works better because there is more and better metadata
available to search.
You can automatically create ISO 19115 metadata from the dataset's
metadata (as ERDDAP does).
That's an important point, because then you don't have to separately
maintain the dataset's local metadata and the dataset's ISO 19115 metadata.
You can automatically populate other catalog systems directly or via the
ISO 19115 documents.
You should be able to make a system to automatically connect to other
data servers, e.g., GeoServer.

On Mon, Mar 18, 2019 at 1:03 PM Tylar<EMAIL_ADDRESS>wrote:

Apologies for hijacking this thread with a tangent, but: are ERDDAP, CKAN,
THREDDS, (others?) all providing a similar "middleman" functionality?
I have minimal exposure to CKAN and THREDDS and want to better understand
the niche of each of these relative to ERDDAP.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/BobSimons/erddap/issues/24#issuecomment-474078755,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABarOE5YlV8CKMnrbchPb_-bkixLCFloks5vX_EGgaJpZM4apFpZ
.

--
Sincerely,
Bob Simons
IT Specialist
Environmental Research Division
NOAA Southwest Fisheries Science Center
99 Pacific St., Suite 255A
Monterey, CA 93940
Phone:<PHONE_NUMBER>
Fax:<PHONE_NUMBER>
Email<EMAIL_ADDRESS>The opinions in this message are mine personally and do
not necessarily reflect any position of the U.S. Government
or the National Oceanic and Atmospheric Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><

Many thanks for the detailed response. Following this discussion has been a great help for me to better understand the relationships between these softwares.


text position: use size instead of bounds

Objective

#13846 introduced a bug where text not bound was not displayed

Solution

bounds are infinite
use computed size instead, that already should be using the available bounds

I think this will revert to the original behaviour in the test code I posted in the pr that caused the problem ..?

I guess we can just use the size if !bounds.x.is_finite(). I don’t really understand what alignment should mean with infinite bounds otherwise.

change it to use the bound when its finite, otherwise the size


InConditionBuilder proper null handling

Q
A

Is bugfix?
✔

New feature?
❌

Breaks BC?
❌

Tests pass?
✔

Fixed issues
#17661

Seems to fail tests: https://travis-ci.org/yiisoft/yii2/jobs/618641372

I've fixed tests and added some test cases to condition provider

Although i see that build fails for php 7.4

That's OK. Same in master.

Merged. Thank you!


Refactor + features + bug fix

Refactore source code
Fix infinite delegate call for currentTime notification
New optional for delegate methods (not necessary implements all delegate methods)
Notification of finish
PS: Broken changes: setUrl(..) changed to url = ...

I like some of the thinks you added, but I won't merge because of the refactor of braces { }
Im gonna add you features by my self

It is an old mistake about braces. Look for big frameworks to see that the old braces usage makes code less readable. It is like  "composite foreign key" for the old database structure. Avoid wrong patterns.


24.1.r1 - Gateway groups gone

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[X] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[X] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug
Somewhere on the path to 24.1.r1 the gateway groups were lost. When accessing the group section no entries are presented.
First reported on the forum https://forum.opnsense.org/index.php?topic=38231.0
To Reproduce
Expected behavior
Groups should be preserved during the upgrade
Describe alternatives you considered
Screenshots
Relevant log files
Additional context
Environment
Software version used and hardware type if relevant, e.g.:
OPNsense 24.1.r1 (amd64).

Please check your configuration history for changes after migration.

Hi Ad,
If you're referring to gateway or gateway group changes there were none since before snapshots/openssl branch was published. The last changes on this FW in a while were last weekend when I migrated DHCP to Kea and opened the two issues,
Looking in config.xml the old GW group names appear only in the GW section of the  rules however the <gateway_group> tag only appears for a new group I just created under a different name to contrast and compare.
Unsure what to think really, I was happy things were working properly when I set these groups up last summer and they've been left as is. There's no reasonable explanation for the groups to be missing, plus there's the forum thread that wasn't started by me - so that would be two occurrences.(?)
Was there a migration that would have touched the config.xml file ? Doubt that's the case but I want to make sure I understood what you meant :)

Another user appears to have lost the GW group...
https://forum.opnsense.org/index.php?topic=38231.msg187256#msg187256

Gateway groups haven’t changed by my knowledge. If the gui doesn’t show entries which are available in the config.xml there’s something someone can reproduce, at this point there’s not much to go on… when entries have vanished, someone need to check their history to find out when that happened (gateway migration perhaps?)

The most likely culprit is https://github.com/opnsense/core/blob/98878a9eb90c1150b232bfbc7e9a012a3e3462a0/src/opnsense/mvc/app/models/OPNsense/Routing/Migrations/M1_0_0.php#L115
… but if that’s the case, there should be a migration in the history removing these entries.

Getting closer, Jan 12th is when it happened. On the 13th I was already on 24.1.r_3 opening the Kea issues.
I've looked around in /var/log, GW logs in the GUI and a few other places - unsure though where that history file that could have any migration info might be located.

-rw-r--r--  1 root  wheel   344K Jan  2 04:17 config-1704187023.3998.xml
-rw-r--r--  1 root  wheel   344K Jan 10 12:05 config-1704906311.9269.xml
-rw-r-----  1 root  wheel   348K Jan 12 11:10 config-1705075828.5096.xml
-rw-r-----  1 root  wheel   348K Jan 17 04:12 config-1705482752.6161.xml
-rw-r-----  1 root  wheel   348K Jan 17 04:12 config-1705482756.3832.xml
-rw-r-----  1 root  wheel   348K Jan 17 04:12 config-1705482764.2576.xml
-rw-r-----  1 root  wheel   348K Jan 17 04:12 config-1705482765.6325.xml
-rw-r-----  1 root  wheel   348K Jan 20 10:28 config-1705764524.2759.xml
-rw-r-----  1 root  wheel   348K Jan 20 10:29 config-1705764593.7023.xml
root@OPNsense:~ # cd /conf/backup/
root@OPNsense:/conf/backup # grep way_g config-1704187023.3998.xml
<gateway_group>
</gateway_group>
<gateway_group>
</gateway_group>
<gateway_group>
</gateway_group>
root@OPNsense:/conf/backup # grep way_g config-1704906311.9269.xml
<gateway_group>
</gateway_group>
<gateway_group>
</gateway_group>
<gateway_group>
</gateway_group>
root@OPNsense:/conf/backup # grep way_g config-1705075828.5096.xml
root@OPNsense:/conf/backup #

You can check the history via the gui, just search for history in the top right corner.

I completely missed the moment when History was added to the GUI, sorry about that. It's great to be able to diff backups in the GUI. Side by side might improve readability but WinMerge or Meld are not far away.
Coming back to the two logs, it looks like when adding the UUID to the GWs caused this issue. The diff file has all the relevant changes
diff.txt

I completely missed the moment when History was added to the GUI

It has been there for at least a decade ;)
Cheers,
Franco

This one goes into RC2.


Please "git tag" your RAKU modules versions

can you please "GIT TAG" your RAKU modules with it's version, similar and at the same time you maintain it's "version": "x.y.z" in your META6.json file?
this is highly appreciated in regards to "reproducible builds" and especially for the "Rakudo Star" modules
THANK YOU!

I think this is now done.  Please re-open if you disagree.


Remove unnecessary Subjectable boxing

For example in TestMatchesPattern.

Ok, TestMatchesPattern receives Subjectable because it's used both in pattern()->test() and pattern()->match()->test(), which receives either string or Base as subject


mousai: 0.3.2 -> 0.4.1

Motivation for this change
https://github.com/SeaDve/Mousai/releases/tag/v0.4.0
https://github.com/SeaDve/Mousai/releases/tag/v0.4.1
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Added a release notes entry if the change is major or breaking
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 124112 run on x86_64-linux 1

1 package built:

mousai

Works great :+1:


User "system:serviceaccount:test-namespace:default" cannot list resource

Our team developed an xxxx Operator with Operator-sdk framework v1.0, basically it worked. Somehow we got issue,we could not find the root cause.

Development environments:
go 1.13

require (
github.com/go-logr/logr v0.1.0
github.com/onsi/ginkgo v1.14.2
github.com/onsi/gomega v1.10.3
k8s.io/api v0.18.6
k8s.io/apimachinery v0.18.6
k8s.io/client-go v0.18.6
sigs.k8s.io/controller-runtime v0.6.2
)

Issue background

we deploy 3  xxxx Operators in the differnent namespace in the same cluster. Initially it worked, somehow we get the following issue, then controller seems not to reconcile any Custom resource.
The logs was grabbed from namespace "test-namespace"
E1111 14:27:43.129782       1 reflector.go:383] pkg/mod/k8s.io/client-go@v0.18.6/tools/cache/reflector.go:125: Failed to watch *v1.Deployment: unknown (get deployments.apps)
E1111 14:27:44.435006       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.6/tools/cache/reflector.go:125: Failed to list *v1.Deployment: deployments.apps is forbidden: User "system:serviceaccount:test-namespace:default" cannot list resource "deployments" in API group "apps" in the namespace "test-namespace"
E1111 14:27:46.631244       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.6/tools/cache/reflector.go:125: Failed to list *v1.Deployment: deployments.apps is forbidden: User "system:serviceaccount:test-namespace:default" cannot list resource "deployments" in API group "apps" in the namespace "test-namespace"
E1111 14:27:50.593543       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.6/tools/cache/reflector.go:125: Failed to list *v1.Deployment: deployments.apps is forbidden: User "system:serviceaccount:test-namespace:default" cannot list resource "deployments" in API group "apps" in the namespace "test-namespace"
E1111 14:34:03.600671       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.6/tools/cache/reflector.go:125: Failed to list *v1alpha1.Mycustomresource: Mycustomresources.bird.com is forbidden: User "system:serviceaccount:test-namespace:default" cannot list resource "Mycustomresources" in API group "bird.com" in the namespace "test-namespace"
E1111 14:34:30.066145       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.6/tools/cache/reflector.go:125: Failed to list *v1.Deployment: deployments.apps is forbidden: User "system:serviceaccount:test-namespace:default" cannot list resource "deployments" in API group "apps" in the namespace "test-namespace"
E1111 14:34:35.840234       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.6/tools/cache/reflector.go:125: Failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:test-namespace:default" cannot list resource "secrets" in API group "" in the namespace "test-namespace"
E1111 14:34:39.842596       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.6/tools/cache/reflector.go:125: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:test-namespace:default" cannot list resource "services" in API group "" in the namespace "test-namespace"
E1111 14:34:58.563342       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.6/tools/cache/reflector.go:125: Failed to list *v1alpha1.Mycustomresource: Mycustomresources.bird.com is forbidden: User "system:serviceaccount:test-namespace:default" cannot list resource "Mycustomresources" in API group "bird.com" in the namespace "test-namespace"

The question is:
(1) what is User "system:serviceaccount:test-namespace:default"? What is its meaning? It seems that I have not defined it.
(2) Why User "system:serviceaccount:test-namespace:default" cannot list resource ?
(3) How to fix it?

A few possibilities

It looks like your RBAC doesn't have the "list" verb specified for certain types. What does your config/rbac/role.yaml look like?
The namespace used (test-namespace) does not look like the one created by default. How are you deploying your operator? Are you using make deploy?

/triage support

Hi @estroz ,
This is a piece of role.yaml, since it is too long, I could not show all here. I saw here for deployments, there are a few of verbs(create/delete/get/list)
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: null
  name: manager-role
rules:
- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch

/assign

@barryzhounb when you run make deploy, kustomize applies the namespace in config/default/kustomization.yaml to all resources. If this name does not equal the one you're trying to deploy in, then your role binding will not bind the role to the correct service account. If you update that line to test-namespace then run make deploy, those errors should go away.

I have been met same problem like this
make-deploy can't update all resource that my controller want to use in rbac
my pod log  has create these error log
1 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.2/tools/cache/reflector.go:156: Failed to watch *v1.Deployment: failed to list *v1.Deployment: deployments.apps is forbidden: User "system:serviceaccount:opdemo-system:default" cannot list resource "deployments" in API group "apps" at the cluster scope
E0724 08:00:55.020734       1 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.2/tools/cache/reflector.go:156: Failed to watch *v1.Deployment: failed to list *v1.Deployment: deployments.apps is forbidden: User "system:serviceaccount:opdemo-system:default" cannot list resource "deployments" in API group "apps" at the cluster scope
E0724 08:01:27.222765       1 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.2/tools/cache/reflector.go:156: Failed to watch *v1.Deployment: failed to list *v1.Deployment: deployments.apps is forbidden: User "system:serviceaccount:opdemo-system:default" cannot list resource "deployments" in API group "apps" at the cluster scope


Create endpoint for fetching music

Handles a scenario where a user searches for a song in the UI.

This is now done. Thanks to you @sankaire


feat: add WYSIWYG editor

🙋 Feature Request
It would be great if this library would have some sort of WYSIWYG editor like CKEditor.
🔦 Context
Sometimes you'll need to format text in different styles or add basic images. It would be great if there is no need for another library which doesn't fit into fluent-ui's design scheme.
💻 Examples
https://ckeditor.com/ckeditor-5/demo/feature-rich/

Hi,
We will not be including any 3rd party libraries in our package in the near future. If we choose CK, people will say 'we want TinyMCE'. If we choose tiny others will cry for some other solution. No way we can make everyone happy with this. Lots of issues wrt legal aspects involved as well.
If we would supply something for this, it would most probably not be part of the core package but be an extension and based on Microsoft technology.

@vnbaaij I don't want CKEditor included. I just put it up as a reference.

@MarvinKlein1508 We've already had this discussion here #738 and here #359.
We believe that excellent third-party HTML editor solutions exist, and that there's no need to add a new one to this library at present. In my opinion, the main problem will be : (1) what format to propose (HTML, Markdown, ...) and (2) what functional limits to bring (bold/italic/bullets/...)
It's a huge job, and unfortunately we don't have the time to get into it right now (creation, follow-up, maintenance, Accessibility issues, etc.).

Hi @dvoituron
surely this ain't a easy task. But this library would even benefit from a very basic editor. Sure there are plenty of editors out there but as you've mentioned they don't follow the design principle of fluent-ui.
I think even a very basic editor would be a huge benefit for this component library. With basic I mean:

bold
italic
underline
headings
links
lists

I think with HTML and Markdown you would have covered like most of all use-cases.

@MarvinKlein1508 I did not assume you meant CK specifically. Just went with your example and to make clear that if we choose that one, only some would be happy.
IT is not about theuse cases you mention. That part is clear. As I sai, it is about including 3rd party libraries. For both HTML and Markdown we would need additional tools (like MarkDig) and we can't do that from a legal perspective


Highlight active heading in table of contents

Do you have any plans to add colour highlighting of the active heading in table of contents?
It would be great to have this feature and dynamically update when scrolling.
I attached an image how it may look (but it is just an example).
PS: great theme, love the minimalistic design

Thank you for the suggestion. It would be a nice touch, but I don't think it would be possible to add without Javascript. The theme currently doesn't use any JS and I don't think it's worth changing that for a non-essential feature.


Bugfix on Biometrics

Description:
This is a small change - I missed the "s" in includes on 3 methods and had an extra "m" in the anchor link on the docs.

Not sure why it failed, is this normal? https://jenkins-oss.wixpress.com/job/detox-ios-59-8-pr/1395//console

Thanks

This is such a useful new feature, great work! @LeoNatan Would you mind publishing a release with this fix?

Out


add a requirements file for yaml, needed on piz daint for python3

launch daint jenkins --test=root<EMAIL_ADDRESS>

launch daint jenkins --test=root<EMAIL_ADDRESS>

jenkins test is green. Somehow it did not commit back to this PR.
I will merge


Fix crash Android in virtual Method

@jerson It's supposed to crash. That's why it's in the try .. catch block and then ignores the exception.

no, for some reason, the try catch does not work, probably because it is a virtual method

Crash in android 4.4 or lower in 5 or 6 its ok

Tested also on a real Device with Android 4.4

@jerson Wait.. the try..catch doesn't work? I don't a lot about java so maybe @GantMan can help me here.
Perhaps you're mistaking the stack trace printout to be a crash (in the catch)?

try catch, catch the error, but the application is closed anyway

That does seem weird. I'll give it a poke later on some devices. We just added this package to an app we have in production and haven't made an upgrade yet so I'll definitely take a look at this soon.
Maybe its better to rewrite it so it doesn't have the try .. catch at all? With what you added we shouldn't need the try .. catch.

if it sounds weird, but that prevents the application to crash, the try catch think it is only for that possible NullPointerException so maybe we could remove it and just leave if !=null

This is the mistake that captures the try catch, but the application error is closed
E/BluetoothAdapter( 6165): Bluetooth binder is null
E/LALA    ( 6165): java.lang.NullPointerException
E/LALA    ( 6165): 	at com.learnium.RNDeviceInfo.RNDeviceModule.getConstants(RNDeviceModule.java:78)
E/LALA    ( 6165): 	at com.facebook.react.bridge.BaseJavaModule.writeConstantsField(BaseJavaModule.java:452)
E/LALA    ( 6165): 	at com.facebook.react.bridge.NativeModuleRegistry.writeModuleDescriptions(NativeModuleRegistry.java:82)
E/LALA    ( 6165): 	at com.facebook.react.bridge.CatalystInstanceImpl.buildModulesConfigJSONProperty(CatalystInstanceImpl.java:368)
E/LALA    ( 6165): 	at com.facebook.react.bridge.CatalystInstanceImpl.initializeBridge(CatalystInstanceImpl.java:126)
E/LALA    ( 6165): 	at com.facebook.react.bridge.CatalystInstanceImpl.access$200(CatalystInstanceImpl.java:39)
E/LALA    ( 6165): 	at com.facebook.react.bridge.CatalystInstanceImpl$1.call(CatalystInstanceImpl.java:97)
E/LALA    ( 6165): 	at com.facebook.react.bridge.CatalystInstanceImpl$1.call(CatalystInstanceImpl.java:92)
E/LALA    ( 6165): 	at com.facebook.react.bridge.queue.MessageQueueThreadImpl$1.run(MessageQueueThreadImpl.java:73)
E/LALA    ( 6165): 	at android.os.Handler.handleCallback(Handler.java:733)
E/LALA    ( 6165): 	at android.os.Handler.dispatchMessage(Handler.java:95)
E/LALA    ( 6165): 	at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:31)
E/LALA    ( 6165): 	at android.os.Looper.loop(Looper.java:136)
E/LALA    ( 6165): 	at com.facebook.react.bridge.queue.MessageQueueThreadImpl$3.run(MessageQueueThreadImpl.java:193)
E/LALA    ( 6165): 	at java.lang.Thread.run(Thread.java:841)
D/dalvikvm( 6165): GC_FOR_ALLOC freed 855K, 12% free 8141K/9176K, paused 65ms, total 65ms
I/ReactNativeJS( 6165): Running application "Setbeat" with appParams: {"initialProps":{},"rootTag":1}. __DEV__ === true, development-level warning are ON, performance optimizations are OFF

yes, it is very strange, but that prevents the application to crash, the try catch think it is only for that possible NullPointerException so maybe we could remove it and just leave if !=null

It'd probably read a bit better this way as well.

@jerson You solution worked for me :) :+1: :heart:

:/ can you just merge it ?

@jerson I'm not a collaborator but didn't we agree on removing the try .. catch?
//cc @GantMan

I think the try catch if it is useful, but maybe for other errors

I think the try catch trying of catch the exception however this combierte in error and can not be controlled by the catch try and the application closes immediately on the device.  I also think this error should be treated with an if, and support the try catch. //cc @jerson :)

Same annoying error, Android 6.0 Nexus 5x emulator. @jerson fix works.

remove try catch and update the code like this
BluetoothAdapter mBluetoothAdapter = BluetoothAdapter.getDefaultAdapter();
if (mBluetoothAdapter == null) {
    // Device does not support Bluetooth
} else {
    if (!mBluetoothAdapter.isEnabled()) {
        // Bluetooth is not enable :)
    }else{
//permission enabled get whatever you want
}
}

this works fine


Update bunsen. Attempt to get captures working

This project uses semver, please check the scope of this pr:

[X] #patch# - backwards-compatible bug fix
[ ] #minor# - adding functionality in a backwards-compatible manner
[ ] #major# - incompatible API change

CHANGELOG
Please add a description of your change here, it will be automatically prepended to the CHANGELOG.md file.

Visual Acceptance tests:
No changes


pokemon match cards Game

🛠 Fixes Issue (Number)
Closes #446
👨💻 Changes proposed

✅ Check List (Check all the applicable boxes)

[x] My code doesn't break any part of the project (Zero Octave-Javascript-Projects).
[x] This PR does not contain plagiarized content.
[x] My Addition/Changes works properly and matches the overall repo pattern.
[x] The title of my pull request is a short description of the requested changes.

📄 Note to reviewers
It will be a memory pokemon match card game with different pokemon hidden behind the card you have to memorize and have to find the right match
📷 Screenshots

We've changes the files for the website, so i suggest you to update your forked repo and add your project in cards.json and readme instead of index and cardsdata file

@NOiR-07 all changes are made #594


Add information about OAuth client Registration

Summary

The current version of the documentation contains a statement that we need an OAuth client registration, but it does't contain information, how we can do this.
Why do we need this ?

To simplify OAuth client registration.
What is already there? What do you see now?

https://www.thethingsindustries.com/docs/reference/api/authentication/#oauth-access-tokens
What is missing? What do you want to see?

Add CLI command for creation of the OAuth client
How do you propose to document this?

Add CLI command for creation of the OAuth client
Can you do this yourself and submit a Pull Request?

Yes

Thanks @romach for filing this issue.
We're blocking this on Console support: https://github.com/TheThingsNetwork/lorawan-stack/issues/4300
Once we have that, we can still provide an CLI example, but then at least it is less error prone.


Playground currently broken in Chrome and Edge

What happened?

PRQL input
# Default example

SQL output
# Nothing

Expected SQL output
# Default example

MVCE confirmation

[X] Minimal example
[X] New issue

Anything else?
No response

Hmmm... Both the public Playground and the latest Dev Container e798ddc7 work fine for me on macOS 12.6.7. I saw both a PRQL query on the left and a bunch of SQL on the right. (I did not verify the SQL code.)
I tried:

Chrome Version 120.0.6099.35 (Official Build) beta (x86_64)
Edge Version 119.0.2151.72 (Official build) (x86_64)
Firefox 120.0 (64-bit)
Safari Version 17.1 (176<IP_ADDRESS>.9, 17616)

Thanks for trying to reproduce this @richb-hanover .
I'm on Windows 10 Pro (Version 10.0.19045 Build 19045).
My browser versions are:

Chrome Version 119.0.6045.160 (Official Build) (64-bit)
Edge

Initially: Version 119.0.2151.58 (Official build) (64-bit)
After upgrade: Version 119.0.2151.93 (Official build) (64-bit)

I can't get any SQL output on any of those.

And which version/commit of the Playground are you using? Thanks.

That's quite weird. It works fine for me on both Firefox & Chrome, on MacOS.
@snth do you see anything in the dev tools console?

Yeah, so I've been looking into it with the dev tools console. There are two requests that fail/are blocked by my ad blocker (pi-hole):

https://www.googletagmanager.com/gtag/js?id=G-LQJDD599T4
https://cdn.jsdelivr.net/npm/<EMAIL_ADDRESS>
I added whitelist rules for:

googletagmanager.com
cdn.jsdelivr.net

Interestingly, the first whitelist rule was ineffective and that particular request still fails. I thought it couldn't be the second one because while the "Query Results" panel may fail, that shouldn't block the "Compiled SQL" panel. However once I added the cdn whitelist rule the playground now works.

I then removed the whitelist rules again just to prove my thesis but everything keeps working. I figured maybe the files are in my browser cache so I cleared the browser cache but it's still working. I guess maybe my ISP is caching some of that CDN content. I'll wait for it to fail again and then try again.

Bottom line: given that many people use ad blockers, I'm relieved that having the googletagmanager/Google analytics blocked doesn't affect things. To me it's a bit odd that having the DuckDB wasm file blocked breaks the whole playground.
My pi-hole version is probably out of date since it probably shouldn't be blocking a JS and NPM CDN. I guess we can close this issue.


Visual Studio vsproj generation broken after #66242

Godot version
4.0.beta (
System information
Windows 10 & 11, Visual Studio
Issue description
As expected in #66242 since I developed it on Linux and didn't get to test on Windows/MSVC, the vsproj=yes option broke.
Here's the error that @EricEzaM reported:
[Initial build] scons: *** [godot.vcxproj] InternalError : Sizes of 'buildtarget' and 'variant' lists must be the same.
Traceback (most recent call last):
  File "C:\Users\Eric\AppData\Local\Programs\Python\Python38\lib\site-packages\SCons\Action.py", line 1318, in execute
    result = self.execfunction(target=target, source=rsources, env=env)
  File "C:\Users\Eric\AppData\Local\Programs\Python\Python38\lib\site-packages\SCons\Tool\msvs.py", line 1845, in GenerateProject
    GenerateDSP(dspfile, source, env)
  File "C:\Users\Eric\AppData\Local\Programs\Python\Python38\lib\site-packages\SCons\Tool\msvs.py", line 1795, in GenerateDSP
    g = _GenerateV10DSP(dspfile, source, env)
  File "C:\Users\Eric\AppData\Local\Programs\Python\Python38\lib\site-packages\SCons\Tool\msvs.py", line 1196, in __init__
    _DSPGenerator.__init__(self, dspfile, source, env)
  File "C:\Users\Eric\AppData\Local\Programs\Python\Python38\lib\site-packages\SCons\Tool\msvs.py", line 448, in __init__
    raise SCons.Errors.InternalError("Sizes of 'buildtarget' and 'variant' lists must be the same.")
SCons.Errors.InternalError: Sizes of 'buildtarget' and 'variant' lists must be the same.
scons: building terminated because of errors.

I'm not familiar with the way this feature implemented nor with how it's used as I almost never use Visual Studio, so it would be great if a VS user could have a look at fixing this, and making sure that the set of build configs defined in #66242 makes sense for contributors and users making custom builds.
Steps to reproduce

Install Windows 10 or 11 with VS 2019 or 2022
scons vsproj=yes

Minimal reproduction project
No response

Seems it has to do with there being .dev equivalents of all of the build targets:
I put a breakpoint at the line in Scons where the first error message occurs (the one about Sizes of 'buildtarget' and 'variant' lists must be the same.), and checked what it had a problem with.  Seems that the env['buildtarget'] list and the variants list aren't the same size and it has a problem with that.
pprint(env['buildtarget'])
['bin\\godot.windows.editor.x86_32.exe',
 'bin\\godot.windows.editor.dev.x86_32.exe',
 'bin\\godot.windows.editor.x86_64.exe',
 'bin\\godot.windows.editor.dev.x86_64.exe',
 'bin\\godot.windows.template_release.x86_32.exe',
 'bin\\godot.windows.template_release.dev.x86_32.exe',
 'bin\\godot.windows.template_release.x86_64.exe',
 'bin\\godot.windows.template_release.dev.x86_64.exe',
 'bin\\godot.windows.template_debug.x86_32.exe',
 'bin\\godot.windows.template_debug.dev.x86_32.exe',
 'bin\\godot.windows.template_debug.x86_64.exe',
 'bin\\godot.windows.template_debug.dev.x86_64.exe']

(Pdb) pprint(variants)
['editor|Win32',
 'editor|x64',
 'template_release|Win32',
 'template_release|x64',
 'template_debug|Win32',
 'template_debug|x64']

I locally added the .dev versions to the variants in methods.py
self.arg_dict["variant"] += [
f'{config}{dev}{f"_[{name}]" if name else ""}|{platform}'
for config in ModuleConfigs.CONFIGURATIONS
for platform in ModuleConfigs.PLATFORMS
for dev in ModuleConfigs.DEV_SUFFIX
]
@staticmethod
def for_every_variant(value):
return [value for _ in range(len(ModuleConfigs.CONFIGURATIONS) * len(ModuleConfigs.PLATFORMS) * len(ModuleConfigs.DEV_SUFFIX))]
It's building again, but being completely unfamiliar with the build system, that might be the completely wrong way to fix it. I also still have to double check that it's actually building the right thing.

That change also worked for me except one part - don't include {dev} in the 'variant' name, otherwise scons will not work as e.g. editor.dev is not a valid target (while editor is). Here is my diff
@@ -745,7 +745,7 @@ def generate_vs_project(env, num_jobs):

             @staticmethod
             def for_every_variant(value):
-                return [value for _ in range(len(ModuleConfigs.CONFIGURATIONS) * len(ModuleConfigs.PLATFORMS))]
+                return [value for _ in range(len(ModuleConfigs.CONFIGURATIONS) * len(ModuleConfigs.PLATFORMS) * len(ModuleConfigs.DEV_SUFFIX))]

             def __init__(self):
                 shared_targets_array = []
@@ -774,6 +774,7 @@ def generate_vs_project(env, num_jobs):
                     f'{config}{f"_[{name}]" if name else ""}|{platform}'
                     for config in ModuleConfigs.CONFIGURATIONS
                     for platform in ModuleConfigs.PLATFORMS
+                    for dev in ModuleConfigs.DEV_SUFFIX
                 ]

Seems I was overly worried about the resulting VS configurations. I thought using the same name would result in a bunch of duplicate configurations in VS, but while distinct names do produce 6 configs, your version only creates the 3 that actually matter.

Looks good to me, feel free to PR that change.

Not sure if you referred to EricEzaM or me, but just in case I created a PR.

The PR may still need a slight tweak. When I used it , the vcxproj still has .dev in the exe names:
<NMakeOutput Condition="'$(Configuration)|$(Platform)'=='editor|Win32'">bin\godot.windows.editor.dev.x86_32.exe</NMakeOutput>
... which doesn't match the exe that gets built by scons, so VS fails to find/launch the newly built exe.
Not related to the changes in this PR I think but additionally - even after I removed .dev from the setting above so that VS could launch the exe, I couldn't debug due to "module was built without debug symbols". I had to also add "debug_symbols=1" into the NMakeBuildCommandLine so that scons would create a pdb. After that I'm able to build & debug 4.0 beta 2 in VS 2022.

Running scons from a cmd window generates the same file for me - no discrepancies between vsproj and scons via cmd. Make sure you have dev_build=yes when running from scons, as the vsproj has that.
scons p=windows dev_build=yes -j15 -> bin\godot.windows.editor.dev.x86_64.exe
<NMakeOutput Condition="'$(Configuration)|$(Platform)'=='editor|x64'">bin\godot.windows.editor.dev.x86_64.exe</NMakeOutput>

This might turn into a conceptual question. For ease of use I'd prefer to create all 6 configurations, but if I understand the code correctly, all configs use the same command line options and whether to use dev_build depends on just the environment variable.
Somebody with a better overview of how things tie together correct me here, but I see two options:
-Instead of adding duplicate variants to match the build target count, I should have reduced the build targets (make the names depend on the env variable in the same way as the NMake command line, rather than creating all 6 options). Downside is that changing between dev/non-dev requires recreating the project file.
-The user friendly version: create all combinations as VS configurations. That might require some bigger changes to how the command line and config names are constructed.

Thanks @EricEzaM , I was indeed missing dev_build in my env. I guess I missed an instruction somewhere in the scons/visual studio setup guide. After adding that, the vcxproj resulting from this PR change worked fine for debugging.

@afestini that code diff does appear to make more sense, if the end result is the same.
I don't think all combinations are needed in the vsproj. The assumption I think is that if you are using vsproj, you are in development and thus want the dev build. If you want a nondev build it is easy enough do do that from the terminal.

@EricEzaM that's a good point I lost sight off. Since the project still goes through Scons, there is not much point in creating one if you're not going to debug. I usually build projects through VS so I can easily check all the options and settings (and potentially tweak them). That doesn't really apply here.
Looking at the docs, the description for VS is in fact focusing on debugging and the current approach would change the command line to achieve that (meaning, require updating the documentation). Would we create any inconsistencies by making dev_build the default and require an explicit dev_build=no otherwise?

I am pretty sure that previously the vsproj build command included the commands to essentially do dev_build=yes

Going through the original change and the description, the second version would be in line with how I understand it, specifically:

Note: Unlike previously, dev_build defaults to off so that users who
compile Godot from source get an optimized and small build by default.
Engine contributors should now set dev_build=yes in their build scripts or
IDE configuration manually.

While it's arguably a less useful default for the VS project creation, it might still be the least surprising and consistent with the original PR.
I played through the scenarios of both versions and the first (current PR) always places .dev in the binary name, while the actual build option is only set when using dev_build=yes during project creation. The second one seems to behave the same, except with the correct binary name. So I will update my PR with the second version.

In case anyone actively using VS is still interested, I have one more and (maybe) final version that creates all six configurations ("editor", "editor (dev build)", etc.). Instead of making VS mess with the config name to extract the target, it uses the targets directly to create different cmdargs for each variant. It somewhat triggers my OCD by moving target= and dev_build= to the end of the command line, but otherwise seems to work. Side effect: the values of "variant" can be modified based on what you want to show in VS without worries about breaking the scons target names.
@@ -742,11 +742,12 @@ def generate_vs_project(env, num_jobs):
             PLATFORMS = ["Win32", "x64"]
             PLATFORM_IDS = ["x86_32", "x86_64"]
             CONFIGURATIONS = ["editor", "template_release", "template_debug"]
-            DEV_SUFFIX = ".dev" if env["dev_build"] else ""
+            DEV_SUFFIX = ["", ".dev"]

             @staticmethod
             def for_every_variant(value):
-                return [value for _ in range(len(ModuleConfigs.CONFIGURATIONS) * len(ModuleConfigs.PLATFORMS))]
+                count = len(ModuleConfigs.CONFIGURATIONS) * len(ModuleConfigs.PLATFORMS) * len(ModuleConfigs.DEV_SUFFIX)
+                return [value for _ in range(count)]

             def __init__(self):
                 shared_targets_array = []
@@ -772,27 +773,26 @@ def generate_vs_project(env, num_jobs):
                     defines = []
                 self.names.append(name)
                 self.arg_dict["variant"] += [
-                    f'{config}{f"_[{name}]" if name else ""}|{platform}'
+                    f'{config}{f" (dev build)" if dev else ""}{f"_[{name}]" if name else ""}|{platform}'
                     for config in ModuleConfigs.CONFIGURATIONS
                     for platform in ModuleConfigs.PLATFORMS
+                    for dev in ModuleConfigs.DEV_SUFFIX
                 ]
                 self.arg_dict["runfile"] += [
-                    f'bin\\godot.windows.{config}{ModuleConfigs.DEV_SUFFIX}.{plat_id}{f".{name}" if name else ""}.exe'
+                    f'bin\\godot.windows.{config}{dev}.{plat_id}{f".{name}" if name else ""}.exe'
                     for config in ModuleConfigs.CONFIGURATIONS
                     for plat_id in ModuleConfigs.PLATFORM_IDS
+                    for dev in ModuleConfigs.DEV_SUFFIX
                 ]
                 self.arg_dict["cpppaths"] += ModuleConfigs.for_every_variant(env["CPPPATH"] + [includes])
                 self.arg_dict["cppdefines"] += ModuleConfigs.for_every_variant(env["CPPDEFINES"] + defines)
-                self.arg_dict["cmdargs"] += ModuleConfigs.for_every_variant(cli_args)
+                self.arg_dict["cmdargs"] += [
+                    f'target={config}{" dev_build=yes" if dev else ""}{" {cli_args}" if cli_args else ""}'
+                    for config in ModuleConfigs.CONFIGURATIONS
+                    for _ in ModuleConfigs.PLATFORMS
+                    for dev in ModuleConfigs.DEV_SUFFIX]

             def build_commandline(self, commands):
-                configuration_getter = (
-                    "$(Configuration"
-                    + "".join([f'.Replace("{name}", "")' for name in self.names[1:]])
-                    + '.Replace("_[]", "")'
-                    + ")"
-                )
-
                 common_build_prefix = [
                     'cmd /V /C set "plat=$(PlatformTarget)"',
                     '(if "$(PlatformTarget)"=="x64" (set "plat=x86_amd64"))',
@@ -806,7 +806,6 @@ def generate_vs_project(env, num_jobs):
                 common_build_postfix = [
                     "--directory=\"$(ProjectDir.TrimEnd('\\'))\"",
                     "platform=windows",
-                    f"target={configuration_getter}",
                     "progress=no",
                     "-j%s" % num_jobs,
                 ]

In case anyone actively using VS is still interested, I have one more and (maybe) final version that creates all six configurations ("editor", "editor (dev build)", etc.). Instead of making VS mess with the config name to extract the target, it uses the targets directly to create different cmdargs for each variant. It somewhat triggers my OCD by moving target= and dev_build= to the end of the command line, but otherwise seems to work. Side effect: the values of "variant" can be modified based on what you want to show in VS without worries about breaking the scons target names.

I am interested in this, having to regenerate visualstudio solution when debugging custom modules (the core of my game, actually...) is a pain, and running my game in zero-optimization that dev build enables is also a pain.


MSUnmerged: Deploy with GRID tools

Impact of the new feature
MSUnmerged
Is your feature request related to a problem? Please describe.
We decided to clear all the unneeded files from the CMS unmerged namespace through remote deletions This means we will have to issue the deletion requests from the micro service itself, regardless if it would run on a VM or a K8 pod, and the later would most probably be the solution we are implementing right now. For that to happen wee will need to have the minimal set of GRID middleware software stack, mostly for authentication and deletion tools to be used once the proper deletion protocol for a given site have been established. The easiest way of having those is to have the cvmfs + ceph volume mounted on a K8 pod.
Describe the solution you'd like
Having the ability to start k8 pods with cvmfs + ceph volume mounts.
Describe alternatives you've considered
N/A
Additional context
N/A

During the rucio transition meeting yesterday we  mentioned there is an alternative to the external filesystem mount. There exists some  gfal bindings packages [1] both C and python which we may include directly into our micro services stack.
[1]
https://gitlab.cern.ch/dmc/gfal2-bindings
https://github.com/cern-fts/gfal2-python
https://pypi.org/project/gfal2-python/

Here is the full list of direct dependencies of gfal2-python3 :
yum install -y \
    gfal2-python3 \
    gfal2-plugin-file \
    gfal2-plugin-gridftp \
    gfal2-plugin-http \
    gfal2-plugin-srm \
    gfal2-plugin-xrootd \
    xrootd-client

Then it comes with few more yum install cmake glib2 boost gcc-c++, that are needed in order for it's shared library to be compiled (it is mandatory to be with the python version it is about to be used). But those they will be requested as dependencies of the above.
And we will need this to be deployed in the docker container which will run the service. So I'd like to ask @vkuznet and @muhammadimranfarooqi how would be the best to proceed here. Should I continue relying on the old rpm based structure expressed in the cmsdit repository  - and in this case is it a must to create a separate *.spec file for every one of those depending packages. Or should we just do something similar to how @ericvaandering  builds the Rucio containers:
https://github.com/rucio/containers/blob/aaae2516ac3d98c5fb4b22d16f71e9a4834092bf/daemons/Dockerfile#L39-L51
p.s. we'd like to follow the short(est)cut here :).

@todor-ivanov the best approach is to create a base image (or multiple base images) for your services which will encapsulate all your required dependencies. For instance, we have cmsweb-base image which is used to build cmsweb services. You may have dmwm-base image which will contain all dependencies dmwm services need. This approach has many advantages. For instance, today we use RPMs, tomorrow we may switch to another package management, e.g. install manually or through pypi or through anaconda. Think of your base image as your OS where you can run your service. Then you'll only need to adjust your base image to provide you full set of dependencies, while your service image will only contain your service codebase (WMCore). This approach allows to optimize base image, e.g. reduce its size, re-use it with other services, use different package management, etc.

Thanks @vkuznet  this is a great suggestion. And I agree. In this case though it is a little bit different. This dependency is introduced because of the need of MSUnmerged to interact with GRID services directly. And the package we are talking here about is not particularly well packaged in pip. This is something which we already realized while debugging its installation. It builds a shared library on the fly and exposes a python3 interface to the outer world etc. .... it did not break my expectations at all. So the bottom line is - I'd like to keep it as down the chain as possible close to the docker image of the service actually needs it. And that's  why I just created this commit [1], putting it as a dependency only for the reqmgr2ms container and avoid having it in a base image. And Thanks for the prompt reaction and the good advice again!
[1]
https://github.com/dmwm/CMSKubernetes/pull/612/commits/764f09c51f79cde8350030bb0c66bb72f8a10a0e

@todor-ivanov once again, use layers, dmwm-base -> dmwm-base-gfal -> reqmgr2ms-unmerged. Doing this way you can again have separate images for individual parts, which can be reused.

And just for my personal log:
If one builds a virtual environment and the possibility of the command:pip intall gfal2-pythono3 inside the virtual environment to fail is quite big. So the work around is to follow the regular deployment process from the OS packaging (being it rpm, deb etc.) and then the so installed shared library needs to be linked through the $PYTHONPATH variable for the virtual environment.
Here is the dump of the information one needs to do the above:
[... WMCoreDev.d]$ rpm -ql gfal2-python3
/usr/lib64/python3.6/site-packages/gfal2.so

[... WMCoreDev.d]$ echo $PYTHONPATH
/usr/lib64/python3.6/site-packages/:/afs/cern.ch/work...

And once the virtual environment is started the result should be:
(WMCore.venv3) [... WMCoreDev.d]$ ipython
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.16.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import gfal2
In [2]:

Definitely use layers. The one thing that I can see where there is room for improvement is this:
"Then it comes with few more yum install cmake glib2 boost gcc-c++, that are needed in order for it's shared library to be compiled (it is mandatory to be with the python version it is about to be used). But those they will be requested as dependencies of the above."
Those are temporary RPMs needed just to build something else and they are quite heavy. The most efficient way to handle this is, as Valentin would point out, to use a multi-source docker build so you build the libraries and bindings in one image and just copy them into the image you ultimately run.
But a "poor mans" substitute could be to, in a single Dockerfile step run an install script which deletes these development RPMs at the end. That way your resulting image is smaller. You can see a simpler version of that here:
https://github.com/dmwm/CMSKubernetes/blob/master/docker/rucio-server/install_mail_templates.sh
where I need to go through a whole bunch of hoops just to get access to a few small files.

Thanks @ericvaandering  @vkuznet  let me process this:
dmwm-base -> dmwm-base-gfal -> reqmgr2ms-unmerged

I was thinking that  what I am currently doing is kind of similar and looks something like this:
dmwm-base -> reqmgr2ms + added dependency from gfal2

Which is almost the same only that this way gfal would be installed on every micro service not only MSUnmerged, but this was a side affect I convinced myself to live with, because of the lack of time to rebuild the whole construction of all the microservices image builds right now.

I have not thought about it enough to know which order or layers you should have, just that in general, they are something you want.

@todor-ivanov , with my recent observation on a number of vulnerabilities which I posted in this ticket https://github.com/dmwm/WMCore/issues/10623 and to hypernews I strongly advise that you evaluate a risk of having monolithic image which may contains lots of vulnerabilities from different dependency chains. Instead, making smaller and independent images ensures better security model.
Please also note, that cmsweb base image was our initial attempt to do the migration from VM based procedure to k8s one. But it is not the best approach! It contains our RPM based procedure which we used in VM which is not required in k8s. You may end-up with much smaller image if you will not start from cmsweb, but rather from small CC/Debian image and only build layers which you need to run your service. Of course cmsweb provides a way to build all cmsweb images but it has its downside, as I posted above we discovery plenty of vulnerabilties which comes from different software stack within an image and it will take us a while to fix them. This is why I strongly advice to start migrating to pypi based installation which you can control number of dependencies you put into image. At the end to run a dmwm python based app you only need python and few dependencies, but you don't need per-se RPMs. Of course, I understand that it may not be feasible right now to get away from RPM based procedure, but I want to make sure that you'll not freeze your mind that you'll always use it and will always depend on cmsweb image.
Said that, the better approach for you would be to start thinking about which layers you need. I think you have the following chain: python -> dependencies -> dmwm/wmcore -> microservices -> additional dependencies -> specialized microservice(s). Therefore, the images you need would be base image, e.g. CC/Debian, dmwm-deps, dmwm, microservice, etc. Using layered approach you may build your dependencies as you like and then only copy necessary pieces to your final image. This will increase your image security, reduce its size, make layers independent and re-usable, etc.
Meanwhile, you need to evaluate what is in cmsweb image before you should use it. May be you only need cmsweb-base image to start with. Open up Dockerfile for each to see what they provide, and start from appropriate one. As I said you may first create your own dmwm-base image with only software you need and nothing else.

Hi @vkuznet  thanks!  this is all extremely helpful and a good suggestion. I am afraid I do not have the time to experiment  with anything beyond the already tested base image of cmsweb being it huge and rpm dependent. I doubt you may find anybody who would like to step away from rpms more than I do. But unfortunately now is not the time. we have more urgent issues to face. So that's why I am about to give it a first try as @amaltaro  suggested to the Pr iteself:
cmserb -> reqmgr2ms ->reqmgr2ms-unmerged + gfal2 dependencies from mainstream rpms (non cmsdist defined)
I know this is not the best dependency sequence, this it the most I can afford experimenting with right now.

@vkuznet: It's urgent that something be deployed by next week.  Would a reasonable compromise be that we deploy what @todor-ivanov has, even though you have identified some flaws, and then we create one or more issues as needed that will be included in the Q3 to-do list, assigned to @todor-ivanov, to address those flaws?

@klannon , @todor-ivanov , I provided advice here based on best practices and experience we have now, but I'm not against what you suggested, and you can go ahead with proposed changes.

@vkuznet:  And I appreciate that advice.  @todor-ivanov: Please open one or more (as needed) issues to implement the advice from @vkuznet (and @ericvaandering) as an improvement on the initial version that we deploy next week.

Thanks @klannon I am creating the issues to track this activity on a later stage . @vkuznet  please  take a look at my try to implement the minimal effort image linking chain here [1] in case I totally messed up things - please let me know so that I revert this commit ad we go with the previous ASAP. Thanks you!
[1]
https://github.com/dmwm/CMSKubernetes/pull/615/commits/a1fecf4540234042296c798947a82a535b76e557

And just to keep track of it -  Here [1] is the newly created issue for the Docker images reorganization  based on a properly leayered system of image dependencies.
[1]
https://github.com/dmwm/WMCore/issues/10624


From private repo

The Mac & droid have stopped my systems
On Wed, May 29, 2019 at 12:33 PM huypub<EMAIL_ADDRESS>wrote:

Merged #3848
https://github.com/MicrosoftDocs/windows-itpro-docs/pull/3848 into
master.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/windows-itpro-docs/pull/3848?email_source=notifications&email_token=AIOARWQTH5USCEKG5SCR56TPX2477A5CNFSM4HQPAR42YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGORWOWW6Q#event-2375904122,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIOARWQQW7T362MHFTVLJTLPX2477ANCNFSM4HQPAR4Q
.


Adding all RepeatCard Tests

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 93.248%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
732

Relevant Lines:
785

💛  - Coveralls


ControlNET?

I noticed that outputs generated with ControlNET no longer had the index title. I checked the AppData/Local/Temp folder and sure enough found the tmp images. Would it be possible to have this script also apply to ControlNET? It's no hassle to manually delete them but I'd rather not wear out my SSD if possible.

My sympathies—I know the pain of unnecessary read/writes! Even if realistically, modern SSDs can survive this much just fine.
I'm aware of what ControlNet is but I haven't used it yet. I won't promise anything, but can I get some additional context? "No longer had the index title"? What index title?
And what interface are you using for ControlNet; the base repo—the [https://github.com/lllyasviel/ControlNet](base ControlNet repo)? Or something like an https://github.com/Mikubill/sd-webui-controlnet for the SD webui project?

I appreciate you looking into it. I'm using the SD-webui with that extension you linked.
I should've written name rather than title, sorry. I meant the name of a generated image if you were to save it from the webui browser. Without your script, the image is named something like tmpx8pmlcdf.png. With your script, the images instead are named index.png. I noticed that when saving a few ControlNET test outputs they were saving as tmp again which led me to check the tmp folder.

Good news! It was just that ControlNet extension used one of the formats I didn't have support for in my bare-bones implementation here (ControlNet returns numpy arrays of image data for the supplementary data it generates—like, say, the canny edge detection images—instead of PIL (python image library) data).
It should work now. That said, it's not like I tested every single thing in ControlNet; do let me know if you run into issues!


More concrete recipes for binary conversion of data.

Per @hjellinek's request, concrete commands used to convert the human-readable data into binary formats as used by the reference server.

Please do these are build rules rather than documentation.
If maven can't do it use, just use make.

@diekhans I believe PR #60 satisfies your request - are there any remaining blocks to merging this PR in?


Cannot create COMMUNITY

Describe the bug
On creating a community, I cannot use groups or channels name as bot says I'm not an admin (I'm these groups/channels owner). Tried using group/channel ID and it work all but on dispute channel ID input when bot answer "El porcentaje debe estar entre 0 y 100" (the % has been set 2 steps ago)
Any suggestion?

can you comment here with the screenshot of your input and the bot response?

This is the first error (same occurs with Canales and Disputes. It works with Chat ID:

These groups/channels are private right now (for testing before publish them).

they must be public for now

I'll try again and report (already done before)

Solved. Thanks.


Move to clap

@nagisa has mentioned this a few times

@frewsxcv @nagisa I can do this next week(after work), it should not take me much time. (Wednesday can be a soft deadline :smile: )

Feel free to take it; I got baited towards other stuff.

Fixed.


Regenerate session id on login

I prefer to leave that to the system. You might want to do other stuff after logout and before regeneration or you might to regenerate it yourself.


Line level editing

A user can edit a particular segment of a transcript, which will be saved to a database

How will race conditions be handled when multiple people might edit the same segment of a transcript before one version is saved? what gets saved? what does a transcriber see during transcription? what does a transcriber see after?

Implemented:

concurrent edits to a line are okay; there is no concept of "versioning", just an algorithm that determines the line's status (e.g. "editing", "reviewing", "completed") and what is the "best" edit at any given time
all edits gets saved
each user gets one edit per line of a transcript; they can always "edit" that edit
user always sees their edits (unless a line has reached consensus, then show the final text)
line statuses are communicated via color and icons: "initialized", "editing", "reviewing", "completed"


Restructure respository

Restructure repository to be more consistent with GitHub folder structure conventions (#194) and to facilitate integrating unit testing (#195)
Changes

Renamed lib folder to src
Moved tests folder to tests/benchmarks
Updated paths in existing codebase to reflect new directory structure

Testing

Builld and run cahaba container, run and evaluate FIM successful on personal computer (unable to test on VM)

Recommend waiting to merge this in after #206

Recommend waiting to merge this in after #206

I request that PR #227 gets merged in before this as well.

I request that PR #227 gets merged in before this as well.