ServiceBusProcessor loses lock if processing is going longer than 20 minutes

Description
Somehow ServiceBusProcessor cannot complete messages if processing lasts longer than 20 minutes. The same result is received with ServiceBusReceiver and regular renewal (every 4 minutes).
Here is the code:
using Azure.Messaging.ServiceBus;

internal interface IAnotherVideoFileProcessor : IAsyncDisposable
{
	Task StartProcessingAsync(CancellationToken cancellationToken);
	Task StopProcessingAsync(CancellationToken cancellationToken);
}

internal class AnotherVideoFileProcessor : IAnotherVideoFileProcessor
{
	private readonly MessageSource _messageSource;
	private readonly ISystemLogger _logger;
	private readonly ServiceBusClient _client;
	private readonly ServiceBusProcessor _processor;

	public AnotherVideoFileProcessor(
		ISystemLogger logger,
		MessageSource messageSource)
	{
		_logger = logger;
		_messageSource = messageSource;
		_client = new ServiceBusClient(_messageSource.ConnectionString);
		var options = new ServiceBusProcessorOptions
		{
			AutoCompleteMessages = false,

			MaxConcurrentCalls = 1,
			MaxAutoLockRenewalDuration = TimeSpan.FromMinutes(120),
		};
		_processor = _client.CreateProcessor(_messageSource.EntityName, _messageSource.SubscriptionName, options);
	}

	public async ValueTask DisposeAsync()
	{
		await _processor.DisposeAsync();
		await _client.DisposeAsync();
	}

	public async Task StartProcessingAsync(CancellationToken cancellationToken)
	{
		async Task MessageHandler(ProcessMessageEventArgs args)
		{
			try
			{
				string body = args.Message.Body.ToString();
				_logger.Info($"New message received ({args.Message.MessageId}): {body}");

				await Task.Delay(TimeSpan.FromMinutes(27), cancellationToken);

				_logger.Info($"Completing {args.Message.MessageId}");
				await args.CompleteMessageAsync(args.Message, cancellationToken);
			}
			catch (Exception ex)
			{
				_logger.LogException(ex);
			}
		}

		Task ErrorHandler(ProcessErrorEventArgs args)
		{
			_logger.LogException(args.Exception);
			return Task.CompletedTask;
		}

		_processor.ProcessMessageAsync += MessageHandler;
		_processor.ProcessErrorAsync += ErrorHandler;

		_logger.Debug($"{nameof(AnotherVideoFileProcessor)}: Start processing.");
		await _processor.StartProcessingAsync(cancellationToken);
	}

	public async Task StopProcessingAsync(CancellationToken cancellationToken)
	{
		_logger.Debug($"{nameof(AnotherVideoFileProcessor)}: Stopping processing...");
		await _processor.StopProcessingAsync(cancellationToken);
		_logger.Debug($"{nameof(AnotherVideoFileProcessor)}: Stopped.");
	}
}

Actual Behavior
Logs:
26 Nov 2023 23:44:00.853: New message received (e217c3c1cc1648519dfdbc288c14a46b):
27 Nov 2023 00:11:00.861: Completing e217c3c1cc1648519dfdbc288c14a46b
27 Nov 2023 00:11:00.890: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance. (MessageLockLost). For troubleshooting information, see https://aka.ms/azsdk/net/servicebus/exceptions/troubleshoot.

Expected Behavior
Message should be completed successfully.

@EldertGrootenboer:  Based on the discussion, here and Stack Overflow, it appears that the client is renewing the lock correctly, but the service is rejecting the settlement.   This can also be reproduced when using the receiver and manually calling to renew the lock.
As this requires service investigation, I'm going to transfer this back over to the Service Bus repository.


Menacing fleetd logs

Every 5min or so, something like this will appear in the fleetd logs:
Sep 09 16:47:33 core-01 fleetd[1375]: INFO client.go:278: Failed getting response from http://localhost:4001/: unexpected EOF
Sep 09 16:47:33 core-01 fleetd[1375]: ERROR client.go:200: Unable to get result for {Watch /_coreos.com/fleet/job}, retrying in 100ms

This is due to etcd closing all watcher connections ever 5min. It would be great to handle this error gracefully and not log, as this is a total red herring when debugging real issues.

:+1:

I tried to find out places where such error messages are printed out, but couldn't find any. Probably it has been already gone.
So I'm closing it.


Implement trigonometric functions

Example
x2 = do a <- newFromInterval (-1 ... -0.1)
             return.interval $ cos a
evalAFM x2

Some of the functions have not yet been implemented fully, only in some specific cases.
The issue with cosine is that it can be both concave and convex in the region, so it's not trivial to use min-range approximation in those cases.
I won't be able to work on it for about a month, but it will need to be implemented at some point.

maybe you can check this library to see how they dealt with it
https://github.com/ogay/libaffa


@rom1v Here is the code. Thanks in advance.

@rom1v Here is the code. Thanks in advance.
https://gitlab.com/las2mile/scrcpy-android

Don't working on my devices, when i hit start the app hangs and don't connect.
I tried on android 7, 8 and 10
Originally posted by @sevenleo in https://github.com/Genymobile/scrcpy/issues/100#issuecomment-803649341
It's not working here too. Same issue

No need to open a separate issue, just answer there https://github.com/Genymobile/scrcpy/issues/100#issuecomment-803649341


CLI Search by name giving error

Command used:
pokedex search --name Gloom  OR  pokedex search --name=Gloom  OR  pokedex search --name gloom
Error:
Connected to database sqlite:////home/rluzuriaga/GitHub/pokedex/pokedex/data/pokedex.sqlite (from default)
Traceback (most recent call last):
  File "/home/rluzuriaga/GitHub/pokedex/bin/pokedex", line 11, in <module>
    load_entry_point('Pokedex', 'console_scripts', 'pokedex')()
  File "/home/rluzuriaga/GitHub/pokedex/pokedex/main.py", line 28, in setuptools_entry
    main(*sys.argv)
  File "/home/rluzuriaga/GitHub/pokedex/pokedex/main.py", line 24, in main
    args.func(parser, args)
  File "/home/rluzuriaga/GitHub/pokedex/pokedex/cli/search.py", line 20, in command_search
    results = search(session, **vars(args))
  File "/home/rluzuriaga/GitHub/pokedex/pokedex/search.py", line 65, in search
    return query.all()
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 3244, in all
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 3403, in __iter__
  File "/home/rluzuriaga/GitHub/pokedex/pokedex/db/multilang.py", line 216, in _execute_and_instances
    return super(MultilangQuery, self)._execute_and_instances(*args, **kwargs)
  File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/query.py", line 3428, in _execute_and_instances
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 984, in execute
  File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1103, in _execute_clauseelement
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1288, in _execute_context
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1482, in _handle_dbapi_exception
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1248, in _execute_context
  File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 588, in do_execute
sqlalchemy.exc.InterfaceError: (sqlite3.InterfaceError) Error binding parameter 4 - probably unsupported type.
 LEFT OUTER JOIN pokemon_forms AS pokemon_forms_2 ON pokemon.id = pokemon_forms_2.pokemon_id AND pokemon_forms_2.is_default = 1 LEFT OUTER JOIN pokemon_form_names AS pokemon_form_names_2 ON pokemon_form_names_2.pokemon_form_id = pokemon_forms_2.id AND pokemon_form_names_2.local_language_id = ? JOIN pokemon_species AS pokemon_species_1 ON pokemon_species_1.id = pokemon.species_id LEFT OUTER JOIN pokemon_species_names AS pokemon_species_names_1 ON pokemon_species_names_1.pokemon_species_id = pokemon_species_1.id AND pokemon_species_names_1.local_language_id= ? JOIN pokemon_types AS pokemon_types_1 ON pokemon.id = pokemon_types_1.pokemon_id JOIN types AS types_1 ON types_1.id = pokemon_types_1.type_id LEFT OUTER JOIN type_names AS type_names_1 ON type_names_1.type_id = types_1.id AND type_names_1.local_language_id = ?
WHERE EXISTS (SELECT 1
FROM pokemon_species
WHERE pokemon_species.id = pokemon.species_id AND lower(?) = ?) ORDER BY pokemon_forms_1."order" ASC, pokemon_forms_1.form_identifier ASC, pokemon_types_1.slot ASC]
[parameters: (9, 9, 9, 9, ColumnAssociationProxyInstance(AssociationProxy('names_local', 'name')), 'gloom')]
(Background on this error at: http://sqlalche.me/e/rvf5)

What I am trying to do:
I'm not sure if I am just doing something wrong or if it actually is broken.
I am trying to familiarize myself with this repo because I want to start contributing to it. I started out by doing what is in the Getting Data wiki: https://github.com/veekun/pokedex/wiki/Getting-Data and I didn't have any problems.
I've tried other commands like: pokedex lookup gloom just to see if it was an issue of the data not being loaded but i get the right output.
Connected to database sqlite:////home/rluzuriaga/GitHub/pokedex/pokedex/data/pokedex.sqlite (from default)
Opened lookup index /home/rluzuriaga/GitHub/pokedex/pokedex/data/whoosh-index (from default)
Matched:
pokemon_species: Gloom(Gloom in <pokedex.db.tables.Language object (9): en>)

Edit: I just thought to try that SQL query using SQLite Browser opening the sqlite DB in pokedex/data/pokedex.sqlite and it returned 0 rows.

Confirmed. I also get an error when running that command.
FWIW, aside from load, dump and reindex, the pokedex CLI doesn't do much and isn't really maintained. (I didn't even know we had a search command!) If everything else works, you're probably fine.
If you want to further check your install, you could try running the tests: run py.test in the project directory. Or py.test --engine your://sql/uri if you aren't using the default database location.

Actually... it works for me with an sqlite database and sqlalchemy 0.9.7 or 1.2.5.
When connected to a postgresql database i get a different error than the one you got:

Connected to database postgresql:///pokedex (from command line)
Traceback (most recent call last):
  File "../bin/pokedex", line 9, in
    load_entry_point('Pokedex==0.1', 'console_scripts', 'pokedex')()
  File "/home/andrew/veekun/pokedex/pokedex/main.py", line 28, in setuptools_entry
    main(*sys.argv)
  File "/home/andrew/veekun/pokedex/pokedex/main.py", line 24, in main
    args.func(parser, args)
  File "/home/andrew/veekun/pokedex/pokedex/cli/search.py", line 20, in command_search
    results = search(session, **vars(args))
  File "/home/andrew/veekun/pokedex/pokedex/search.py", line 65, in search
    return query.all()
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2300, in all
    return list(self)
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2412, in __iter__
    return self._execute_and_instances(context)
  File "/home/andrew/veekun/pokedex/pokedex/db/multilang.py", line 216, in _execute_and_instances
    return super(MultilangQuery, self)._execute_and_instances(*args, **kwargs)
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2427, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute
    return meth(self, multiparams, params)
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
    context)
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1160, in _handle_dbapi_exception
    exc_info
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
    context)
  File "/home/andrew/veekun/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 436, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (ProgrammingError) subquery in FROM must have an alias
LINE 4: FROM pokemon_species, (SELECT pokemon_species_names.name AS ...
                              ^
HINT:  For example, FROM (SELECT ...) [AS] foo.

I'm using SQLAlchemy 1.3.15
I'll try downgrading to 1.2.5 and check.

Downgraded to 1.2.5 and search is not working.
Maybe changing the setup.py to specify SQLAlchemy to 1.2.5 so that the setup doesn't install the newest version that doesn't work 100%?


Added HTTP API timeout option

This PR introduces a timeout property that can be configured on TonClient instance that is getting propagated to the HTTP API client.
Signed-off-by: Slava Fomin II<EMAIL_ADDRESS>

Released: v6.4.0
Thank you for you contribution!


Continuous integration with AVR and ARM

I believe that we can setup travis to continuously build on AVR. AVR's toolchain seems to be available on apt-get.

A tutorial for microcontroller firmware CI is given here.

@truell20, there is a branch for AVR or instructions to use Fido with it ?

@patrickelectric, currently Fido doesn't really support AVR to the degree we'd like it to.  We still have reliance on the standard library and file system functionality.  We have a number of open issues to this regard, eg 52.  This is definitely one of our top priorities, and once we feel Fido is in a AVR-ready state we do plan on writing some instructions on using and debugging Fido on AVR.


port: port modules to v12

rename models and fields in product_price_factor to changes in v12.

It stopped works

File "/home/travis/build/it-projects-llc/website-addons/product_price_factor/models/product.py", line 77, in price_compute
for price in value.price_ids.filtered(lambda r: r.product_tmpl_id == product.product_tmpl_id):
AttributeError: 'product.attribute.value' object has no attribute 'price_ids'

https://travis-ci.com/it-projects-llc/website-addons/jobs/219976747
I mark modules as non-installable


Components added by jobs are lost on cloning

Description

Clown and Chaplain receive a component on spawn for their job. The clown is clumsy, and the chaplain is a bible user. This component is put on their mortal body, which means they lose it when they are cloned. Almost all the components I can think of like this (Mime will probably get one eventually) are mental, so they should be put on the mind instead. Similar issue to  https://github.com/space-wizards/space-station-14/issues/6097
Also it'd be nice to be able to have an equivalent for tags, because they're lighter, more easily expandable by non-coders, and stuff like BibleUser would probably be better off as a tag

Oh yeah I fixed this :sunglasses:


Starrred Document Error

If try to star a document in the home screen and unstar it from the starred document it doesn't remove the selected star to unselected one in home
https://user-images.githubusercontent.com/40017559/111680091-dd318e00-8847-11eb-8c43-fdf2f91a7b4d.mp4

The problem here is that the home screen is lying below the starred documents screen, so when we go back the home screen doesn't build up rather the home screen lying below is shown. Going back to home screen (after un-starring the PDF) and then pressing the refresh button in the app bar makes the star icon "un-starred" (this confirms the first sentence). For such a situation we can maybe use streams but I am not so sure.

Yep stream builder has to be used to get updated output from the previous screen to this.


Use pretty print to assist debugging.

Without pretty print, the javascript is all one line, making debugging in the debug build nearly impossible. With pretty print, the javascript is still all in 1 file, but is formatted into multiple lines.
@jiayliu

lgtm


Blank plot using own data

Hi Mohit, thanks for this library. The game of thrones test data works fine but when I use my own data set the plot is blank. The settings panel appears but no nodes or connections. I've attached my node and edge CSVs - though I have tried with just the edge_df and I get the same result. Am I missing something, is there a problem with my data?
status_mapping_edge.csv
status_mapping_node.csv

Solved with a solution found in issue #49 - there were duplicate pairs in the edge csv.


Make Windows version of build script

Currently, the build scripts are written in Unix Bash. It would be ideal for easy build processes available for Windows contributors as well.

Upon further consideration, a containerized approach is a viable option to support a consistent development environment on Windows. This will also remove the need to install varying packages depending on Linux distribution, etc.


pipewire: 0.3.5 -> 0.3.6

should fix bad v0 clients, such as google-chrome-stable etc:
https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/014e48c1f1c47404653f597e500314cbf021056d
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

cc @NixOS/gnome

Changes look okay: https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/b04552b2522a2cdae9a246be6ed77bb5ae31cda3
Though it looks like it omits some things. Full changes:
b04552b2 0.3.6
e7cc0f69 libcamera: fix implicit dependency to libdrm
2c8292be docs: add some more info
8696ad44 docs: add some doc about SPA design
2b7e6e19 doc: annotate code block as c
9fd64283 docs: fix some typos
014e48c1 protocol: add workaround for bad v0 clients
62dd58a6 client: remove busy state from client in resource destroy
8f251fc7 docs: add some docs about SPA POD
a44bea0b tests: check cpu flags
f278ded9 example: use thread loop in example
180f2b0f client-node0: clean up resource
b91a7180 man: Add man pages for pw-jack, pw-pulse
d566c885 port: don't send reuse_buffer on node again
64fc8dda stream: check for NULL context
65798ca5 context: only set TERMINATED when -EPIPE
dac60d4e avoid some leaks and NULL pointer derefs
2fd64f15 connection: dup and close fds
aaaa5417 stream: don't destroy proxy twice
2462cac4 add some more debug
c13c028e bluez5: fix leaks
51ae60b0 portal: fix small leak
e493be38 scripts: Use special ${LIB} token for architecture-independence
96eb6e4d scripts: Don't add an empty entry to LD_LIBRARY_PATH
0efadbb2 client-node: free old mem after setting new mem
972a790c client-node: send error when node_set_io fails
6ee98531 jack: free old mem after we configure new mem
bf4ccc00 stream: destroy proxy
969377b0 client-node: simplify cleanup
ade5bcb5 local-v4l2: fix leaks
ee5d6b44 pulse: add more checks
cecbf319 example: use bound event to find node id
ef770f9c proxy: debug more errors
1b4b8a93 gst: check for NULL stream
1c86ed82 resource: fix errno printout in error message
2ae621c9 fix warning: passing argument 1 of ‘pa_proplist_iterate’ discards ‘const’ qualifier 	in pipewire-pulseaudio/src/proplist.c 	for pulseaudio version < 13.0
77860ef3 add version requirement for vulkan library dependency 	due to usage of VK_ERROR_OUT_OF_POOL_MEMORY, VK_ERROR_INVALID_EXTERNAL_HANDLE, 	VK_API_VERSION_1_1 and VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT 	introduced in version 1.1.69
549431e8 pulse: fix leaks
ed31ca30 media-session: improve cleanup of objects
47ae56df core: improve debug
4abd02be proxy: debug bound id
5d5bb23e proxy: add pw_proxy_ref() private method
5ad84913 pipewire: make pw_init() work after pw_deinit()
eb74b2bf buffers: fix warning
814b7426 jack: stop the thread loop before we destroy objects
253e3041 jack: destroy link proxy
38cfa2d7 jack: fix leaks
d69ff451 tools: fix leaks
49619f2b tools: fix leaks
fa04d2ee profiler: fix some leaks
30571d47 tools: fix some leaks
0ee7d811 core: only warn about leaked proxies
f1e7e86b buffers: add support for multiple data blocks
704d504a media-session: mark enabled modules with *
0b47b47b libcamera: disable by default
39974903 Rebasing
0d8e030c Removed datas field
d93870fd Removed redundant logs
af263187 Reverting redundant changes
9024cc44 Integrating libcamera
ee1b79c4 stream: setting a format always clears buffers
8292c556 client-node: first set param, then clear buffers
33cd2495 examples: add signal handler for clean exit
197ffe32 Makefile: set VALGRIND variable in valgrind
325280cb core: run extra step to warn/destroy leaked proxies
3060c57b examples: add cleanup
6d0fd7b5 remote-node: run _destroy in the _removed handler
a7225090 core: improve cleanup
4405fe4c remote-node: handle deleted client-node
6b313de2 examples: catch signal and clean up
702976ab context: avoid adding duplicate export types
8d0913c9 examples: add audio DSP source
baf3b0a2 pw-cat: zero out info
913ad434 filter: free ports as well
b4b7ba4c remote-node: suspend node before cleanup
e4e03fa7 media-session: always activate DSP nodes
94cbb545 stream: rename variable
203c89da pw-cat: cleanup registry proxy
3890f06c stream: let the node_destroy() destroy the proxy as well
3d22baab node: suspend before emiting destroy
17d7cdc8 spa-node-factory: cleanup node when binding the client resource fails
16c1a7fe spa-device-factory: destroy device when the creator resource is destroyed
0471ba7f Add pw_in_valgrind() to check running in valgrind
fda9e4d5 call _deinit(() in session manager exit
00d61d95 add pw_deinit() to cleanup resources
09913ca4 metadata: free when the global is destroyed
4dfd60f8 add some more debug
c4b87aa5 audioadapter: clear convert handle to avoid leaks
7d88c37e alsa-seq: clear the midi event queue on close to avoid leak
d2bd076a alsa: close output handler to avoid a leak
40609341 client-node: keep track of the client allocated fd
c04a9b54 client-node: refuse the wrong amount of client buffers
bb1cb530 add more debug of fd create and close
803c2860 mem: add flag to suppress notify
6ecbe007 session-manager: emit cached info with CHANGE_MASK_ALL on new resources
1eec6c35 session-manager: do not copy stream link params if they are NULL
6aeaebe2 mem: Clean up stray mappings
47966735 mem: Improve search for existing mapping
34dada9c media-session: install ctrl-C handler
1246116f bluez: free object path
fbaa4dde stream: allow NULL param and 0 buffers in disconnect
7f60ebff logger: handle file without '/' in it
bf483242 alsa-monitor: don't use NULL device reserve
871bb47e gst: handle pause/resume
6eba010d gst: do a sync before disconnect
48dea3d5 gst: handle stream create error
c546167a require meson 0.49
9fa68f6d neon: use regular registers only
60742d75 pw-cli: allocate nodes ids of the right size
08ffc507 midifile: fix CLAMP range of minor/mayor keys
6a689b3c client-node: don't ref destroyed resource
963b00b7 adapter: return an error when we can't find a format
d31c057b Remove some useless checks
8123e271 make sure we call va_end in all cases
c2028a16 Fix some leaks in error cases
8669fd03 initialize variables better
ce98162e Make sure we don't read too much
236ebeeb gst: fix bad unlock when pool acquire fails
78aaf6c0 gst: only pop and adjust choice when we pushed something
eb93f259 avoid some useless assignments
4f60dcad vulkan: check results of stat
f1dc3d97 vulkan: fix macro variables
ff4a3140 use fstat when we can
45c62dfd a2dp: check and log return value of fcntl
459acda6 client-node: use spa_system functions
a19bab4b avoid following NULL pointers
a898f21c improve debug
1b2f6491 format: handle NULL type info
d2c51ed2 ci: publish generated documentation from master on gitlab pages
df818f1e ci: update to use fdo ci-templates, fedora 32 and preserve less artifacts
31a3458c editorconfig: add exception for yml files that use 2-space indentation
98780cdb utils: fix spa_assert_se
9e6a0477 portal: add portal module again
0bb3de0d context: clean up permission check
2fe9c37f core: add some more debug
5aea6252 log: don't re-initialize the log level in pw_init()
f06a884f mididump: dump offset
09b6ef41 alsa: compare the scaled value to decide on xrun
fea16cca channelmix: support Midi volume control
498cb917 channelmix: fix enum_params
b4112968 log: handle logging of NULL object
56730197 alsa: report xrun
fefdc26f plugin: workaround for AVX intrinsic: "_mm256_setr_m128()" missing in GCC
cee1a26b build: skip unsupported compiler option  -Wimplicit-fallthrough for gcc version before 7.x
4000409c channelmix: allow smaller control buffers
055c8d6b channelmix: improve control handling
5e37131c channelmix: add control port to process control sequences
cab645d1 stream: we can get the index before we are ready
8d5571fa audiotestsrc: fix function signature
0a542497 fix error reporting
fdb3985f client-node: handle NULL port
a7d8e805 stream: do some extra checks

Maybe we should package libcamera? It is only disabled by default because distros do not package it: https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/0b47b47b6c5e4fb7c1748625934272ed4583ecdc


[Dynamic Links] Firebase Dynamic Links Offline

Is your feature request related to a problem? Please describe.
I'd like non-shortened Firebase Dynamic Links to properly be handled while the device is offline. Turning my phone on airplane mode and clicking a Firebase Dynamic Link opens my app. But the onLink method is never called with the link's parameters. While I'm aware shortened Firebase Dynamic Link URLs need a network connection, I'd like to be able to handle non-shortened Firebase Dynamic Link URLs in my app while offline.
Describe the solution you'd like
When I have no network connection and tap a non-shortened Firebase Dynamic Link URL, the app should open and the link should be handled by the onLink method, or by the getInitialLink method. I would expect to be able to retrieve the link parameters from the Firebase Dynamic Link which opened the app.
Additional context
Again, while offline, the links will properly open the app, but onLink is never called, and getInitialLink returns null.

Im having the same problem but i have wifi,i dont know why.


New Extention: Crunchbase

Description
Add a new extension, crunchbase that allows searching on the Crunchbase websites thanks to the Meilisearch search engine.
Screencast

https://user-images.githubusercontent.com/6064892/195601833-7dc060f7-5fbc-4b75-89f2-756e11bafde8.mp4
Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Hi @pernielsentikaer, thanks for the review. I've fixed the metadata image and the panel icon that was missing.

Hi, I just added the throttle on search. Thanks again for your review.


How to call a stored procedure and get multiple different result sets

As the title says

@cjon123, hi. Have you tried to just call the procedure and iterate over result sets?
There is a test that uses multi-result procedures, but its purpose is to assert the proper error handling.

@blackbeam Thank you very much, the problem has been solved！


[Sample App] Support deployment of Consul sample app

Enhancement Description
Each service mesh generally has one or more associated sample applications that can be used to demonstrate the power of the given service mesh. Consul uses https://github.com/hashicorp/consul-k8s-l7-obs-guide as a sample application.
Desired Behavior
The Consul adapter currently supports deploying sample applications from other service meshes. It should support the canonical Consul sample app, so that users can get the most thorough tour of Consul's features.

Hey @leecalcote,
Can I take this up? This will help me getting started with adapters.

@kalradev that's great idea. Even higher priority than this sample app is the addition of Learn Layer5 as a sample app to be supported by the adapter for Consul. The addition of that application means that the Consul adapter will support SMI conformance testing. Let's get you going on that app.

Great!!
I will start with adding Learn Layer5.

Good deal. The Meshery adapter for Open Service Mesh and the Meshery adapter for Istio both support Learn Layer5 (SMI Conformance) and make for good references.

The Consul sample app requires quite a bit of additional setup.
I think it would be good if the code could be refactored before that, towards the Kuma-blueprint. I'll add a couple of issues for that, but started with a set of tests (#94).


The UT test of spark is failed. Because there is a test in SQLQuerySuite about creating table “test”

If the tests in "sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala" are  running before CachedTableSuite.scala, the test("Drop cached table") will failed. Because the table test is created in SQLQuerySuite.scala  ,and this table not droped. So when running "drop cached table", table test already exists.
There is error info:
01:18:35.738 ERROR hive.ql.exec.DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: AlreadyExistsException(message:Table test already exists)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:616)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4189)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1503)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1270)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1088)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)test”
And the test about "create table test" in "sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala,is:
test("SPARK-4825 save join to table") {
val testData = sparkContext.parallelize(1 to 10).map(i => TestData(i, i.toString)).toDF()
sql("CREATE TABLE test1 (key INT, value STRING)")
testData.insertInto("test1")
sql("CREATE TABLE test2 (key INT, value STRING)")
testData.insertInto("test2")
testData.insertInto("test2")
sql("CREATE TABLE test AS SELECT COUNT(a.value) FROM test1 a JOIN test2 b ON a.key =   b.key")
checkAnswer(
table("test"),
sql("SELECT COUNT(a.value) FROM test1 a JOIN test2 b ON a.key = b.key").collect().toSeq)
}

Can one of the admins verify this patch?

(Add [SQL] to the title please to sort it properly. It's borderline whether this should have a JIRA, but it's pretty much the same to describe as to fix.)

ok to test

Test build #29133 has started for   PR 5150 at commit 7534b02.

This patch merges cleanly.

Test build #29133 has finished for   PR 5150 at commit 7534b02.

This patch passes all tests.

This patch merges cleanly.

This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29133/
Test PASSed.

Merged to master. Thanks.


UnhandledPromiseRejectionWarning when running the usage example

Hello folks.
I tried to run the usage example located here : https://github.com/graphcool/chromeless#usage
I got
(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: spawn EACCES
(node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Environment: docker image from node:8
Source code is available here : https://github.com/hamadr/chromeless-test
How to reproduce?
Run make

When catching all the unhandled rejection I got: ```Possibly Unhandled Rejection at: Promise  Promise {
 { Error: spawn EACCES
at exports._errnoException (util.js:1024:11)
at ChildProcess.spawn (internal/child_process.js:325:11)
at Launcher.exports.spawn (child_process.js:493:9)
at Launcher. (/usr/src/app/node_modules/chrome-launcher/chrome-launcher.js:139:37)
at Generator.next ()
at /usr/src/app/node_modules/chrome-launcher/chrome-launcher.js:12:71
at Promise ()
at __awaiter (/usr/src/app/node_modules/chrome-launcher/chrome-launcher.js:8:12)
at Promise (/usr/src/app/node_modules/chrome-launcher/chrome-launcher.js:126:59)
at Promise () code: 'EACCES', errno: 'EACCES', syscall: 'spawn' } }  reason:  { Error: spawn EACCES
at exports._errnoException (util.js:1024:11)
at ChildProcess.spawn (internal/child_process.js:325:11)
at Launcher.exports.spawn (child_process.js:493:9)
at Launcher. (/usr/src/app/node_modules/chrome-launcher/chrome-launcher.js:139:37)
at Generator.next ()
at /usr/src/app/node_modules/chrome-launcher/chrome-launcher.js:12:71
at Promise ()
at __awaiter (/usr/src/app/node_modules/chrome-launcher/chrome-launcher.js:8:12)
at Promise (/usr/src/app/node_modules/chrome-launcher/chrome-launcher.js:126:59)
at Promise () code: 'EACCES', errno: 'EACCES', syscall: 'spawn' }

I only took a quick glance, but most likely chrome-launcher  (or, specifically your node process) doesn't have the correct executable permissions to spawn Chromium.

On my Ubuntu box, Chrome is installed in /usr/bin/google-chrome but when I start ChromeLess (remote=false of course), it doesn't spawn it.
How can I tell chrome-launcher where my google-chrome is?
Could we pass that option in the constructor of Chromeless, e.g.:
const chromeless = new Chromeless({ remote: false, chromePath:"/usr/bin/google-chrome" })

Hi @ldenoue — I've opened #184 to track your request.

I get the same error, when I try to run it as a Mocha test in VSTS. It will be spawned if I run the test manually via command line.

I'm dying to know if you found a good workaround for this problem.

@kumorig I believe a decent workaround is to simply launch Chrome before running Chromeless -- if Chrome is running, it won't try to spawn a new process, and you get to pass any flags you need.
e.g. path/to/google-chrome --headless --remote-debugging-port=9222 --disable-gpu && node your-script.js


[FEATURE] Release windows binaries as ZIP not tar.gz

Requirements
This issue proposes that we .zip files for windows instead of .tar.gzing them.

Good point. Right now we perform .tar.gz for all binaries [1]. But, yes it makes sense to release .zip for windows
@toddbaert  any input ? Otherwise, I can work on this through https://github.com/open-feature/flagd/issues/581 as I already will be working on GH releas action.
[1] - https://github.com/open-feature/flagd/blob/main/.github/workflows/release-please.yaml#L192-L198

I haven't used Windows as a daily driver for quite a few years, so I'm probably not the best person to ask. If people aren't using WSL I guess it'd make sense that they'd prefer a .zip?

I used Windows and usually .zip works best there. The alternative here is to rely on an installation like 7-zip but this could be limited in restricted systems.

This is fixed with PR #642


overrides regions in templates with regions in configs

Since we moved to dicts for the regions block, the default in our templates gets put in even if excluded in application configs. This makes sure to only use the regions in the application config. If not provided in the application config, it will use the default in the templates.

This fixes the following scenario:
If you have "regions": { "us-east-1": {} } in the templates as a default, but "regions": { "us-west-2": {} } in the application configs, you end up with "regions": { "us-east-1": {}, "us-west-2": {} } in the rendered configs due to how DeepChainMap works. This makes sure just the default or just the app configs are used

Thank you for clarifying the change in the last comment.
I see no issues with this.


⚠ Bofrost.be has degraded performance

In 91b34a8, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1774 ms

Resolved: Bofrost.be performance has improved in ab239c7 after 1 hour, 5 minutes.


bash: mercury-parser: command not found

Platform: Linux s2e24de3f 2.6.32-042stab134.3 #1 SMP Sun Oct 14 12:26:01 MSK 2018 x86_64 GNU/Linux
Node Version: v10.15.1
I installed yarn and npm 10 versions, then executed the command:
yarn global add @ postlight / mercury-parser
root @ s2e24de3f: ~ / mercury-parser # yarn global add @ postlight / mercury-parser
yarn global v1.13.0
[1/4] Resolving packages ...
[2/4] Fetching packages ...
[3/4] Linking dependencies ...
[4/4] Building fresh packages ...
success Installed "@ postlight / mercury-parser @ 2.0.0" with binaries:
       - mercury-parser
Done in 2.62s.

And then:
mercury-parser https://postlight.com/trackchanges/mercury-goes-open-source
The message appears:
-bash: mercury-parser: command not found

@altblog My best guess is that you haven't added yarn binaries to your path. You might want to check here for a starting place for how to fix that.


rqt_reconfigure: the layout is pretty broken

From @dirk-thomas on August 16, 2013 22:10

[x] left column takes all the space, right column has only minimum width, should be the other way around
[x] the configuration block per node and vertically spread, should have spacing at the bottom
[x] each edit row is horizontally spread, should make use of the full width instead
[ ] the row label is overlapping with the controls when the window is small

Copied from original issue: ros-visualization/rqt_common_plugins#150

First three bullets fixed with pr #151.

Will these fixes resolve the fixed-width space allocated for the tree in the left panel? The cutoff width does not increase if the left panel is made wider. It results in nested names becoming invisible, as in this picture:


type all not working

if I put
type: "all",
in the config {}    than there is:
[ERROR] Unknown error TypeError: Cannot read properties of undefined (reading 'length')
at Class.wikiOnThisDayRequest (/home/ronny/MagicMirror/modules/MMM-OnThisDayWikiApi/node_helper.js:52:62)

at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

I got the same "unknown error" setting type: all

Also, with type "selected" I got the same facts everyday..
I'm going to disable this module until it got some fix.


dns support only routing domain but not search domain

if add a domain with nameserver  in the DNS tab it's add only routing only domain (at least on linux).  eg if i add foo.com to <IP_ADDRESS> then i see something like this on the resolvectl output:
Link 5 (wt0)
    Current Scopes: DNS
         Protocols: +DefaultRoute LLMNR=resolve -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: <IP_ADDRESS>
       DNS Servers: <IP_ADDRESS>
        DNS Domain: netbird.cloud ~foo.com

this result that if i try to resolve a name with:
resolvectl query bar.foo.com
than it's working, BUT if i try with short name only eg:
resolvectl query bar or ping bar
than it's not working. and currently I'm not able to solve it.
it'd be nice if i can add search domains too. The best would be if i can configure whether add routing or search domain in the DNS tab.

Hello, @lfarkas. When Netbird DNS name resolving is enabled, a short domain name search works only for netbird.cloud zone. So you can resolve as short names only peers in the Netbird network. For all other queries, you need to use FQDN domain names.

it seems the situation was not clear enough.
if we got a domain and a dns for it than the given zone probably contains many fqdn hostname in the dns (otherwise we do't use a dns).
we probably have a netbird client in the given network but not all of the host has a netbird client (otherwise the .netbird.cloud zone resolution would be enough).
so when we'd like to manage dns in netbird, then we've a routed network with a gateway netbird clinet inside this  network.
if i'd like to use dns in netbird then i'd like to be able to resolve all host in that domain.
of course as a the routing only domain it's better then nothing since if i use fqdn then it resolve it but if i'd like to resolve short names than it's not working what's more there is no way to resolve short names for these domains.
that's  why i'd propose that by default these domains should have to be search domain or at least it should have to be configurable to be routing only dns  or search dns.

fixed in the latest release


Counts tsv files on experiement level contain only NAs

Hi,
I noticed that on the experiment level the tsv count files contain only NA's, but the counts files at the selection levels and lower are fine. Is this a bug?

Does this also happen if you run the example data (https://github.com/fowlerlab/enrich2-example)?

Just checked, yes. It's not much of a problem since I can just get counts from the lower level tsv files, I just thought I'd mention it.

Hi I have the same problem, I was wondering if the issue was solved.

Hi I have the same problem, I was wondering if the issue was solved.

This may be addressed in a future version of the software, but for now I encourage you to use the above workaround and get the counts from the selection-level TSV files.

Is there any calculation applied to get the combined counts on the experiment level?

Is there any calculation applied to get the combined counts on the experiment level?

There's no calculation applied. The counts are the same from the seqlib level through to the selection and experiment. They're just organized in a single dataframe for convenience.

Clear, thanks!


Cache on Translator saved across languages

I translated a story from English into Norwegian, and now the same story appears in the cache (or whatever) in my browser when I translate using the Norwegian translator, which makes no sense. The browser cache should be cleared for a new language.

Thanks for pointing this out. Your right that the different versions should use different storage/cache.
In any case, I can fix this, but it will wipe out any current translations you have in the Norwegian translator. Is that ok?

Yes, go ahead. Thanks.

Done. Let me know if everything is ok now and I will close this issue.

Yes, it works now.


UI/UX Desigining

I want to contribute to this open-source world-changing project. and I want to work on UI/UX Designing.

Hi @DevenRathod2 in #2 we are talking about a new UI/UX Design

Yeah. i can help you to design a stunning UI
also @getify can you create a slack workspace for discussion about this project?

Hi, i can help too. also have a uterus 😁

Hey! I am frontend developer and a I also have a uterus 😁. I would love to help because I had to delete my period tracking apps with these laws and I need something which protects my identity and data. (PS. I live in Texas and female reproductive rights have gone down the the drain here).

@aditiranka27 I live in Texas too, a big reason motivating this app!

@getify I agree! A huge motivation to contribute to something which would help all the people affected.

I think #2 is tracking the UI design. Do we need a separate thread (this one) just for UX discussions separate from UI, or can we combine our discussion into that other thread?

I think we can combine the two threads.

Great, everyone here, please move over to #2 and jump in! There's a figma project tracking some early design concepts, and plenty more to discuss and decide. Welcome!


line info currentPower shows only 1/3 of the real consumption

hi there,
it looks like, that currentPower only shows the the power of the first phase.
in the EVSEMaster App it shows me 10,27, the currentPower shows only 3491.
if i calculate all three lines like this
let line = evse.getLines().get(1);

line.l1Power = line.l1Voltage * line.l1Electricity;
line.l2Power = line.l2Voltage * line.l2Electricity;
line.l3Power = line.l3Voltage * line.l3Electricity;
line.totalPower = line.l1Power + line.l2Power + line.l3Power;

i got the real value which match the app state.
is this a bug?

jepp, it shows only l1
{
    "lineId": 1,
    "currentPower": 3471,
    "currentAmount": 1307.06,
    "l1Voltage": 229,
    "l1Electricity": 15.16,
    "l2Voltage": 230,
    "l2Electricity": 14.68,
    "l3Voltage": 229,
    "l3Electricity": 14.63,
    "innerTemp": 34.94,
    "outerTemp": 34.94,
    "currentState": "CHARGING",
    "gunState": "CONNECTED_LOCKED",
    "outputState": "CHARGING",
    "errors": [],
    "l1Power": 3471.64,
    "l2Power": 3376.4,
    "l3Power": 3350.27,
    "totalPower": 10198.31
}

I'd say this is partly a bug in the OEM comms board (it actually reports this power field with this value; I'm just passing it through) and partly a misinterpretation of mine: I saw 3 voltage and amperage fields, and just 1 power field, so I assumed it was the power across all phases. Turns out it isn't; I checked the decompiled OEM app and it also does the same calculation (multiplying the volts and amps per phase and summing them) for the power display field. I didn't notice that because my wallbox is still connected on 1 phase only.
I'll fix this in the EmEvseLine: I'll keep using the same currentPower field to preserve the API, but using the summed phase powers as the field value (and ignoring the field we get from the wallbox; or maybe I'll use the maximum of both, in case there are wallboxes which do report it but no per-phase info).

Updated: using the higher value of the reported power and the calculated power across the 3 phases for EmEvseLine.currentPower.


It was not possible to find any compatible framework version - compilation failed

Don't have log any more and for the moment can't install plugin again to regenerate it.
Installed components (apart form standard Unity):

I also have 2 Unity editor versions installed:

Please tell me what other data you would need or what to check?

This isn't an FSR problem. Check out this StackOverflow answer on a possible approach to fix your issue.
https://stackoverflow.com/a/58702215

This isn't an FSR problem. Check out this StackOverflow answer on a possible approach to fix your issue. https://stackoverflow.com/a/58702215

It's not a problem with my setup. I don't have any problems at all with Unity normal workflow and compilation, this problem is only happening when using FSR to compile code. Alternative solution "Hot Reload" also works fine.

Thanks @akarpinski1 - tool should use existing CSC (compiler) that Unity uses and shouldn't throw this exception.
One thing that I though could happen is wrong compiler version picked up (maybe due to multiple versions).
If you can recreate the issue and add a debug breakpoint here: (or log parameters)
FastScriptReload.Editor.Compilation.DotnetExeDynamicCompilation.ExecuteDotnetExeCompilation
I'd be interested to know what's in
string dotnetExePath
string cscDll
and then to see what versions of those files you have

I've seen it before but couldn't reproduce. As @Chizaruu mentioned I think it'll be some specific environemnt issue

Hello,
Both files points to what is shipped with editor in use for given project:

and version of those files:

I also double checked this on second PC. Same editor version (2020.3.44f1), Same behavior - exact same error.

I had the same issue and solved it by installing .NET Core 2.0.0.

I had the same issue and solved it by installing .NET Core 2.0.0. https://dotnet.microsoft.com/en-us/download/dotnet/2.0

same error, i dont know why, but it works


How to unmarshal virtual machines returned from resource graph query

Feature Request

We found the Azure defined virtual machine property structure is like below:
type VirtualMachineProperties struct {
	// HardwareProfile - Specifies the hardware settings for the virtual machine.
	HardwareProfile *HardwareProfile `json:"hardwareProfile,omitempty"`
	// StorageProfile - Specifies the storage settings for the virtual machine disks.
	StorageProfile *StorageProfile `json:"storageProfile,omitempty"`
	// AdditionalCapabilities - Specifies additional capabilities enabled or disabled on the virtual machine.
	AdditionalCapabilities *AdditionalCapabilities `json:"additionalCapabilities,omitempty"`
	// OsProfile - Specifies the operating system settings used while creating the virtual machine. Some of the settings cannot be changed once VM is provisioned.
	OsProfile *OSProfile `json:"osProfile,omitempty"`
	// NetworkProfile - Specifies the network interfaces of the virtual machine.
	NetworkProfile *NetworkProfile `json:"networkProfile,omitempty"`
	// SecurityProfile - Specifies the Security related profile settings for the virtual machine.
	SecurityProfile *SecurityProfile `json:"securityProfile,omitempty"`
	// DiagnosticsProfile - Specifies the boot diagnostic settings state.

Minimum api-version: 2015-06-15.
	DiagnosticsProfile *DiagnosticsProfile `json:"diagnosticsProfile,omitempty"`
	// AvailabilitySet - Specifies information about the availability set that the virtual machine should be assigned to. Virtual machines specified in the same availability set are allocated to different nodes to maximize availability. For more information about availability sets, see [Manage the availability of virtual machines](https://docs.microsoft.com/azure/virtual-machines/virtual-machines-windows-manage-availability?toc=%2fazure%2fvirtual-machines%2fwindows%2ftoc.json).

 For more information on Azure planned maintenance, see [Planned maintenance for virtual machines in Azure](https://docs.microsoft.com/azure/virtual-machines/virtual-machines-windows-planned-maintenance?toc=%2fazure%2fvirtual-machines%2fwindows%2ftoc.json)

 Currently, a VM can only be added to availability set at creation time. The availability set to which the VM is being added should be under the same resource group as the availability set resource. An existing VM cannot be added to an availability set.

This property cannot exist along with a non-null properties.virtualMachineScaleSet reference.
	AvailabilitySet *SubResource `json:"availabilitySet,omitempty"`
	// VirtualMachineScaleSet - Specifies information about the virtual machine scale set that the virtual machine should be assigned to. Virtual machines specified in the same virtual machine scale set are allocated to different nodes to maximize availability. Currently, a VM can only be added to virtual machine scale set at creation time. An existing VM cannot be added to a virtual machine scale set.

This property cannot exist along with a non-null properties.availabilitySet reference.

Minimum api‐version: 2019‐03‐01
	VirtualMachineScaleSet *SubResource `json:"virtualMachineScaleSet,omitempty"`
	// ProximityPlacementGroup - Specifies information about the proximity placement group that the virtual machine should be assigned to.

Minimum api-version: 2018-04-01.
	ProximityPlacementGroup *SubResource `json:"proximityPlacementGroup,omitempty"`
	// Priority - Specifies the priority for the virtual machine.

Minimum api-version: 2019-03-01. Possible values include: 'Regular', 'Low', 'Spot'
	Priority VirtualMachinePriorityTypes `json:"priority,omitempty"`
	// EvictionPolicy - Specifies the eviction policy for the Azure Spot virtual machine and Azure Spot scale set.

For Azure Spot virtual machines, both 'Deallocate' and 'Delete' are supported and the minimum api-version is 2019-03-01.

For Azure Spot scale sets, both 'Deallocate' and 'Delete' are supported and the minimum api-version is 2017-10-30-preview. Possible values include: 'Deallocate', 'Delete'
	EvictionPolicy VirtualMachineEvictionPolicyTypes `json:"evictionPolicy,omitempty"`
	// BillingProfile - Specifies the billing related details of a Azure Spot virtual machine.

Minimum api-version: 2019-03-01.
	BillingProfile *BillingProfile `json:"billingProfile,omitempty"`
	// Host - Specifies information about the dedicated host that the virtual machine resides in.

Minimum api-version: 2018-10-01.
	Host *SubResource `json:"host,omitempty"`
	// HostGroup - Specifies information about the dedicated host group that the virtual machine resides in.

Minimum api-version: 2020-06-01.

NOTE: User cannot specify both host and hostGroup properties.
	HostGroup *SubResource `json:"hostGroup,omitempty"`
	// ProvisioningState - READ-ONLY; The provisioning state, which only appears in the response.
	ProvisioningState *string `json:"provisioningState,omitempty"`
	// InstanceView - READ-ONLY; The virtual machine instance view.
	InstanceView *VirtualMachineInstanceView `json:"instanceView,omitempty"`
	// LicenseType - Specifies that the image or disk that is being used was licensed on-premises. This element is only used for images that contain the Windows Server operating system.

 Possible values are:

 Windows_Client

 Windows_Server

 If this element is included in a request for an update, the value must match the initial value. This value cannot be updated.

 For more information, see [Azure Hybrid Use Benefit for Windows Server](https://docs.microsoft.com/azure/virtual-machines/virtual-machines-windows-hybrid-use-benefit-licensing?toc=%2fazure%2fvirtual-machines%2fwindows%2ftoc.json)

 Minimum api-version: 2015-06-15
	LicenseType *string `json:"licenseType,omitempty"`
	// VMID - READ-ONLY; Specifies the VM unique ID which is a 128-bits identifier that is encoded and stored in all Azure IaaS VMs SMBIOS and can be read using platform BIOS commands.
	VMID *string `json:"vmId,omitempty"`
	// ExtensionsTimeBudget - Specifies the time alloted for all extensions to start. The time duration should be between 15 minutes and 120 minutes (inclusive) and should be specified in ISO 8601 format. The default value is 90 minutes (PT1H30M).

 Minimum api-version: 2020-06-01
	ExtensionsTimeBudget *string `json:"extensionsTimeBudget,omitempty"`
}

We build and run a resource graph query to get a list of virtual machines, and for each of them we want to parse and get the power state. However, one example of the returned virtual machine property is like this:
"properties": {
		"extended": {
			"instanceView": {
				"computerName": <vmName>,
				"hyperVGeneration": "V1",
				"osName": "ubuntu",
				"osVersion": "18.04",
				"powerState": {
					"code": "PowerState/running",
					"displayStatus": "VM running",
					"level": "Info"
				}
			}
		},
		"provisioningState": "Succeeded",
		"vmId": <vmID>
	},

We checked different go sdk versions, but look like we didn't find a structure where we could unmarshal properties --> extended --> instanceView, we only got properties --> instanceView.
Can you please point out if it is possible to use a structure to unmarshal the above data returned from resource graph query so that we can get the virtual machine power state? The sdk versions we've checked included 2020-06-01, 2020-12-01, 2021-03-01, 2021-04-01 and 2021-07-01.

Hi @shaobovaltix thanks for this issue!
This issue looks like that the struct in the SDK does not align with what the service is returning. I just checked the corresponding swagger: https://github.com/Azure/azure-rest-api-specs/blob/main/specification/compute/resource-manager/Microsoft.Compute/stable/2021-07-01/compute.json#L11986
this means that the SDK aligns with the SDK.
Could you please share the code that you do the query? Which function in the SDK are you calling to get the result?

@shaobovaltix, the service behaviour is not controlled by SDK, could you create an issue in https://github.com/Azure/azure-rest-api-specs/issues/new?
We will close this one as the SDK is auto generated from the Swagger and type VirtualMachineProperties correctly mapped to the Swagger definition.


Add NotoColorEmoji font to emoji-picker?

This font allows country flag emojis to work in Windows 10/11: https://github.com/googlefonts/noto-emoji.  But I can't figure out how to apply it to the shadowRoot element where emoji-picker lives (while still relying on code from your cdn).  Any thoughts?  Thanks much!

emoji-picker-element isn't really designed to allow for custom fonts, but you can kind of get it to work if you inject what you want into the shadow root. Here's an example using Twemoji: https://github.com/nolanlawson/emoji-picker-element#examples

And yeah I agree it's too bad Windows doesn't have flag emoji; I really wish they'd fix that.


Automatically manage restic repos

Fixes #577
Builds on #570, ignore first commit.
Replaces ark restic repo init with automatic repository initialization, using a single static non-secret restic key.

@skriss does this look accurate for the EnsureRepo flow?

Looks right to me

Backup flow:

Delete snapshot flow (from backup deletion controller):

Daemonset pod backup flow:

Restore overview:

DaemonSet pod restore flow:

Addressed items discussed; will do squashing post-review.

LGTM

squashed.


Question - Thread pool alternatives

This is just a question.
Does .Net Core provide any classes that implement a "throttled" thread pool concept?
The use case is a situation where a very large number of short lived REST upsert calls are being made, we want to limit the number of concurrent requests to some defined value like 3 or 5 or whatever.
This is to avoid flooding the network with excessive traffic, restrict the network traffic for the entire operation to some controlled value.
The standard thread pool limits the number of concurrent activities based on available CPU core count, such a design would tend to keep the network very busy which is undesirable.
So I just wondered if there have been any additions to .Net over the past few years that address this kind of thing.
Thanks
(I guess one way to do this is to put a Sleep inside every scheduled action and just use the thread pool as is, this will tend to have as many tasks active as there are cores yet prevent network saturation. So if a typical upsert elapsed time was say 50 mS then adding a 50 mS sleep at the end of the operation would reduce network traffic rates by 50%...)

There is ConcurrentExclusiveSchedulerPair, which can give you a TaskScheduler with a specified concurrency level:
TaskScheduler scheduer = new ConcurrentExclusiveSchedulerPair(
    TaskScheduler.Default, maxConcurrencyLevel: threeOrFiveOrWhatever).ConcurrentScheduler;

But note that this limits the number of threads used, which may be different than the number of tasks that are executing, when some of them are async.
To limit concurrent requests, I would probably use SemaphoreSlim, since that can work on the task level.

@svick - Many thanks for this info, great help, I'll delve into this and assess, much appreciated.


Stack overflow with incorrect mapped type as constraint of type parameter

function sum<T extends { [P in T]: number }, K extends keyof T>() {
    n += v[k];
}

This incorrect code puts the compiler in an infinite loop:
     RangeError: Maximum call stack size exceeded
      at Object.contains (src/compiler/core.ts:276:5)
      at getBaseConstraint (src/compiler/checker.ts:5955:21)
      at getResolvedBaseConstraint (src/compiler/checker.ts:5949:36)
      at hasNonCircularBaseConstraint (src/compiler/checker.ts:5936:20)
      at getConstraintOfTypeParameter (src/compiler/checker.ts:5906:20)
      at getConstraintTypeFromMappedType (src/compiler/checker.ts:5761:56)
      at isGenericMappedType (src/compiler/checker.ts:5798:84)
      at computeBaseConstraint (src/compiler/checker.ts:5997:21)
      at getBaseConstraint (src/compiler/checker.ts:5960:32)
      at computeBaseConstraint (src/compiler/checker.ts:5969:38)
      at getBaseConstraint (src/compiler/checker.ts:5960:32)

I'm having a similar issue to this, but can't reproduce the issue in a minimal case.  In the minimal case it works properly, but inside of my project, I have this same stack trace, but my code is:
export type MatchQuery<T> = {
  [P in keyof T]?: T[P] | MatchQuery<T[P]>
};

@arciisine for that small of a repro, there's nothing to cause the compiler to start checking the insides of the mapped type. As soon as you add some code that uses the mapped type you should see the crash.

I actually have a larger minimal case, that doesn't cause it to happen.  It appears to be something along the way, that I'm going to try to narrow down and see if I can hit that a minimal repro.
Btw, I have many versions of the recursive mapped type that do not throw any errors, but this one in specific seems to be problematic.


Unparsable meta.json does not throw an error

Without realizing it, I ended up with a merge conflict in my meta.json file after pulling from a remote. It looked something like this as you might imagine:
{
<<<<<<< remote commit
    "time": "2016-01-28T05:57:18.879Z",
    "version":<PHONE_NUMBER>879,
    "remotes": {
        "repo-1": {
            "time":<PHONE_NUMBER>879,
            "version":<PHONE_NUMBER>879,
            "shasums": {
                "repo-1.thrift": "repo-1.hash"
=======
    "time": "2016-02-09T02:22:18.987Z",
    "version":<PHONE_NUMBER>987,
    "remotes": {
        "repo-2": {
            "time":<PHONE_NUMBER>987,
            "version":<PHONE_NUMBER>987,
            "shasums": {
                "repo-2.thrift": "repo-2.hash"
>>>>>>> my conflicting local commit
            }
        },
        "repo-2": {
            "time":<PHONE_NUMBER>356,
            "version":<PHONE_NUMBER>356,
            "shasums": {
                "repo-2.thrift": "repo-2.hash"
            }
        },
        "repo-3": {
<<<<<<< remote commit
            "time":<PHONE_NUMBER>124,
            "version":<PHONE_NUMBER>124,
=======
            "time":<PHONE_NUMBER>744,
            "version":<PHONE_NUMBER>744,
>>>>>>> my conflicting local commit
            "shasums": {
                "repo-3.thrift": "repo-3.hash"
            }
        }
    }
}

Clearly this is invalid json and no parser would be able to read it. However, since I was not aware meta.json had invalid json, I ran idl update as usually, and it returned without response. This seemed normal to me (I later remembered I don't actually see no response, I generally see n-1 commas returned, where n is the number of thriftfiles idl is managing).
It was not until I put my change up for code review I saw the meta.json was totally invalid, and the managed thriftfiles had not been updated.
idl should definitely be returning an error if meta.json is unparsable. In fact it looks like it used to according to #28, perhaps this is a regression? I was using version 3.1.1

Fixed by https://github.com/uber/idl/pull/37


Remove useless property and add comment

Property broker_url is deleted as env var cannot be used with custom rest action
Added comments for remaining properties to explain where/why they are used

wrt these 2 env variables, one way to configure the actual value is to set the values using another custom CM or Secret, and inject them in the SonataFlow application using the podTemplate.container.envFrom.configMapRef/secretRef

wrt these 2 env variables, one way to configure the actual value is to set the values using another custom CM or Secret, and inject them in the SonataFlow application using the podTemplate.container.envFrom.configMapRef/secretRef

@dmartinol You mean set the env variable in the pod as a reference to the CM/Secret, right? So instead of redefining the props we could do that, is that what you meant? But IIRC, the props CM is generated by the operator so if they are not added in this props CM, they won't be available,  right?

wrt these 2 env variables, one way to configure the actual value is to set the values using another custom CM or Secret, and inject them in the SonataFlow application using the podTemplate.container.envFrom.configMapRef/secretRef

@dmartinol You mean set the env variable in the pod as a reference to the CM/Secret, right? So instead of redefining the props we could do that, is that what you meant? But IIRC, the props CM is generated by the operator so if they are not added in this props CM, they won't be available, right?

I mean to add your own CM/Secret with the deployment values.
Of course, this can't be done in this repo, but we should have another one for the deployment configuration where we can do the hack (using kustomize IMO, so you don't have to edit the manifests generated by this pipeline)


[TS] Background image spec changes in 1.2

Automatically generated task. Any updates to the title and body will be overwritten. Do NOT modify the labels.
TS fix of #2794

Already fixed. Can't put my finger on the commit/PR though.

Oh @dclaux this is for the spec changes we just decided on (rename stretch to cover, rename mode to fillMode, etc). I assume those aren't implemented yet? :P


primitive support?

does this format support more than just triangle primitives??
tri-strips, tri-fans, quad-strips, and polygons reduce a heck of alot of data used by triangles alone.
I'm considering downloading this, but I NEED support for those primitives...
does this have it??

no, only triangles are supported.


GITHUB_URL does not work for BitBucket :upside_down_face:

Well, duh, but there's one simple difference:
GitHub has /commit/<sha>
BitBucket has /commits/<sha>.
There's only a single-character between the two URL schemes.
Could URL support please be extended to include BitBucket as well?

To do it in a backwards compatible way perhaps a new config item in addition to github_url that includes /commit/ or /commits/ in the vcsLink?
https://github.com/justinbarrick/fluxcloud/blob/master/pkg/formatters/default.go#L157

Good catch! You can now set a COMMIT_TEMPLATE: https://github.com/justinbarrick/fluxcloud/#formatting-commit-links

Thanks a bunch!


FAI-226: Score Cards: Refactor the Linear Regression Viewer to use new UI model (UI changes)

See https://issues.redhat.com/browse/FAI-226
This PR moves an old "Linear Regression (model) viewer" from Trusty's "sandbox" to the PMML editor.
I have updated the "PMML Editor migration" document to include the new components added herein.

@jstastny-cz

I don't see any tests here.

Please see the document I have shared with you and accompanying explanation.

@jstastny-cz

I don't see any tests here.

Please see the document I have shared with you and accompanying explanation.

@manstis that explains it - thanks.

@manstis that explains it - thanks.

Thank-you @tiagobento
Many of the properties to build the graph are optional and hence I need most undefined checks to then substitute defaults. I have however refactored a little, moving to anonymous/arrow functions (as seems to be our more common approach) and removed some undefined return values and handling where possible. This is in a new commit.

Thank-you @tiagobento
Many of the properties to build the graph are optional and hence I need most undefined checks to then substitute defaults. I have however refactored a little, moving to anonymous/arrow functions (as seems to be our more common approach) and removed some undefined return values and handling where possible. This is in a new commit.

@jstastny-cz are you able to approve this PR now?

@jstastny-cz are you able to approve this PR now?

@manstis can you share that document with me. I mean the document which explains missing tests.

@manstis can you share that document with me. I mean the document which explains missing tests.


Change vocabulary size during re-run

Hello,
With a vocabulary size of 55K, I have trained the model for 200K steps and saved the latest checkpoint.
Now, I increased my vocabulary size to 70K.

How can I continue training from the saved checkpoint but with this new vocabulary size (70K)?
Simply changing the vocabulary size throws error: InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match
For model fine-tuning, does updating the vocabulary and continue training from previous best checkpoint is the optimal strategy?  Or is there another optimal way to fine-tune the pre-trained model?

The issue 51 has no working solutions yet.

Hi @kmario23,
Is the 55K vocab a subset of the 70K? If so, maybe you can just start training with 70K for the first 200K steps as well. Otherwise, you might want to try init_from_checkpoint to restore non-embedding variables only.

Hi @oahziur,
Thanks for your insights!
The 55K vocabulary can be considered as a subset of 75K. But, All of 75K vocab is not available in single instance. The data samples comes as and when available (usually once in a week). Now, I want to constantly adapt or (increase) the vocabulary size by N as new samples come in. Think of it as constantly improving a baseline model :)
In such a scenario, what would be the ideal way to use the latest checkpoints for further training?
Thanks!


Visualizer: retain parameterized info for types and use getColumnValues method

Changes in this PR:

Retain parameterized info for types.
Use getColumnValues method.
Extract constants into a constants class.
Move more methods from Visualizer to VisualizerRelInfo.
Added "Load" button to visualizer page. The graph now loads when button is clicked, not when scope input changes.


The Party and Location should not be re-usable schemas anymore, and we should remove the Customer value from ALL assignedBy enumeration, with the exception of the Paper structure

Done, see commit f6397df.


Add Example on BLE client + Wifi coex (IDFGH-11907)

Is your feature request related to a problem?
I tried to find sutable example, but there is non.
Describe the solution you'd like.
I will like to see working solution how BLE Client can scan/connect/exchange passcode and coexist with WIFI.
On ESP32 C3 S3 boards.
Describe alternatives you've considered.
My second choice will be Arduino wrapper, but i hope thats can be done.
Additional context.
My main goal is interconnect BLE Device with Wifi Device.
In scenario BLE device - BT - ESP logic - Wifi - Wifi AP - Wifi Clients

Hi @esp-lrh
Thank you so much for your answer.
Your response make sense.
My scenario is
Scan/Connecting/Connected: BLE and WIFI not at the same time - First wifi, after OK, MQTT after OK, BLE.
This should work.
My main question is about if can WIFI and BLE coexists on receiving and sending data (sorry for not clear question at very first post)
The Goal is write BLE to WIFI bridge (just forwarding Buffer data nothing fancy).
I look at almost all examples, but i can not found the right for me.
The example in  examples/bluetooth/nimble/bleprph_wifi_coex is PRPH, but PHRP do not do a scan and do not connect to the Advertised devices. So i think i need CENT example - but i can find any in my scenario.
(If i understand right that CENT is "client" who connect to the advertised device, and adverthise device show passcode on the sceen).
I do this once in past in arduino, but this time i switch to pure IDF only (and i thinks is better approach) but i struggle with basics (like it is CENT what i need?)
Thank you again for your response.

Hi @esp-lrh
Thank you so much for your answer.
Your response make sense.
My scenario is
Scan/Connecting/Connected: BLE and WIFI not at the same time - First wifi, after OK, MQTT after OK, BLE.
This should work.
My main question is about if can WIFI and BLE coexists on receiving and sending data (sorry for not clear question at very first post)
The Goal is write BLE to WIFI bridge (just forwarding Buffer data nothing fancy).
I look at almost all examples, but i can not found the right for me.
The example in  examples/bluetooth/nimble/bleprph_wifi_coex is PRPH, but PHRP do not do a scan and do not connect to the Advertised devices. So i think i need CENT example - but i can find any in my scenario.
(If i understand right that CENT is "client" who connect to the advertised device, and adverthise device show passcode on the screen and client have to type and send the passcode to the adv device).
I do this once in past in arduino, but this time i switch to pure IDF only (and i thinks is better approach) but i struggle with basics (like it is CENT what i need?)
Thank you again for your response.

CENT is "client" who connect to the advertised device you can think so.
From Bluetooth Core Specification:
When entered from the Initiating state, the Connection state shall be in the Central Role.
When entered from the Advertising state, the Connection state shall be in the Peripheral Role.

The Link Layer in the Central Role will communicate with a device in the Peripheral Role and defines the timings of transmissions.

RF time-division multiplexing between BLE and WIFI is implemented in lower level, has almost no impact on the upper-layer APIs. All you need to do is simply receive data from BLE and call the WIFI APIs to send the data, and it will work as if you are receiving data from BLE and sending data using WIFI at the same time.
There is a BLE Central example in examples/bluetooth/nimble/blecent, you can try to merge it's code with example `bleprph_wifi_coex' (remember to remove the origin BLE Peripheral code).

I wil try.
In fact this is exactly what is this issue about. To provide "merged" code for CENT+COEX scenario :)
I hope that i will have success.
If i will have more question i will ask you.
Thank you again.

I have to say that example examples/bluetooth/nimble/blecent/tutorial/blecent_walkthrough.md  have many thing not cleared. I think many functions mentioned in this tutorial is not part of main.c and i was strugling what is about then.
I missing the part when CENT exchange the passcode with PRPH. So i have no clue how to setup and handle Security events.
The esp_central.h is dependency, but is not mentioned in walkthrough.  And is not defined if i need to use "examples" path to include (via idf_component.yml which i did) or is better approach to handle this.
Can you help me with those issues? Thank you.

I m reallz strugling. I cannot move forward, because of PASSKEY.
The default CENT example to not count with this scenario... so i m done...
If you will have some time can you help?
Thank you.

I can not pass this:
    /* Read the supported-new-alert-category characteristic. */
    chr = peer_chr_find_uuid(peer, BLE_UUID16_DECLARE(ServiceUUID), CameraService);
    if (chr == NULL) {
        MODLOG_DFLT(ERROR, "Error: Peer doesn't support CameraManufacturerUUID and CameraModelUUID\n");
        goto err;
    }

According to the protocol of Camera to i want connect
Blackmagic Bluetooth Camera Control
Blackmagic cameras with Bluetooth LE implement a variety of features and commands that
allow users to control their cameras wirelessly. Developers have full access to these features for
their custom applications.
The following services and characteristics describe the full range of communication options that
are available to the developer.
Service: Device Information Service
UUID: 180A
Characteristics
Camera Manufacturer
UUID: 2A29
Read the name of the manufacturer (always “Blackmagic Design”).
Camera Model
UUID: 2A24
Read the name of the camera model (eg. “URSA Mini Pro”).
Service: Blackmagic Camera Service
UUID: 291D567A-6D75-11E6-8B77-86F30CA893D3
Characteristics
Outgoing Camera Control (encrypted)
UUID: 5DD3465F-1AEE-4299-8493-D2ECA2F8E1BB
Send Camera Control messages.
These messages are identical to those described in the Blackmagic SDI Camera Control
Protocol section above. Please read that section for a list of supported messages and required
formatting information.
For an example of how packets are structured, please see the ‘example protocol packets’
section in this document.
Incoming Camera Control (encrypted)
UUID: B864E140-76A0-416A-BF30-5876504537D9
Request notifications for this characteristic to receive Camera Control messages from
the camera.
These messages are identical to those described in the Blackmagic SDI Camera Control
Protocol section above. Please read that section for a list of supported messages and required
formatting information.

I tried think i tried everythink but i m missing something.

Ok i make progress.
The whole issue was with configuration of peer. Needs to be: ble_hs_cfg.sm_io_cap = BLE_HS_IO_KEYBOARD_ONLY;
But this is what examples is good for :) There is no info about SM_IO_CAP.
I found this in esp-idf/components/bt/host/nimble/nimble/nimble/host/include/host/ble_hs.h

@esp-lrh
OK i just give up. I m done.
With no solid example i m only guessing, flashing, guessing, flashing, almost for days now. With no progress.
Please add an example which will contains:

setup CENT device for accepting passkey (i think i have this done correctly)
scan a filter device with target service UUID (0x1800 in my case, i have this done)
establish connection and initiate secure connection (i have this done, kind of)
notify that PHPR needs to send passkey (on_passkey event, i have this done)
receive passkey from PHPR (!!! i was searching for this almost 5 hours!!!) i found ble_sm_inject_io works but if this is correct? I have no clue
create pair and bonded device and store (!!! my PRPH device still force me to enter password, again and again...)
subscribe to the custom characteristics (128) and starts receiving data (on event) NOT DONE

I do not know if my subscription is working... i have no clue if i do not receive data because wrong encryption, wrong subscriptions, wrong
I tried establish connection with PRPH device from my phone and is done with remembering the connection, never ask me again for passkey, and reads data like expected.
Please add example to the Examples.
Thank you.

Hi, @gavalierm
Here are some explanations for your issues:

CONFIG_EXAMPLE_EXTENDED_ADV
This is a feature introduced in BLE 5.0, you can just ignore it

PASSKEY and SM_IO_CAP
SM_IO_CAP  is a properties defined by the Bluetooth protocol stack and it will affect the pairing process.
see https://www.bluetooth.com/blog/bluetooth-pairing-part-1-pairing-feature-exchange/
The default value in blecent example is BLE_HS_IO_NO_INPUT_OUTPUT, which means the device has no ability to input or output anything, the pairing and encryption process will just work.
And that is the reason why you  'missing the part when CENT exchange the passcode with PRPH. So i have no clue how to setup and handle Security events'.
If you still want to entry a passkey when paring, you should change SM_IO_CAP and  listen to GAP event BLE_GAP_EVENT_PASSKEY_ACTION then respond appropriately.
example usage see examples/bluetooth/nimble/bleprph/main/main.c

Add example to the examples
Maybe not now, we don't have much time to add example of such a specific scenario.

I think you could simply set SM_IO_CAP as BLE_HS_IO_NO_INPUT_OUTPUT, it will work and link key will be automatically save in device. Please try again.

Hi @esp-lrh
Thank you for your time.
Yes i figure out the the BLE_GAP_EVENT_PASSKEY_ACTION and SM_IO_CAP.
But:
the SM have to be enabled in menuconfig, store persist in NVS have to be enabled in menuconfig. I figure out those things too.
The connection have to be encrypted. This is my code from Arduino which works, but i not figure out how this i can setup in esp idf nimble.
  // Enable encryption
  _pBLEDevice.setEncryptionLevel(ESP_BLE_SEC_ENCRYPT);  // i do not know how
  BLESecurity *pSecurity = new BLESecurity(); // not relevant
  pSecurity->setAuthenticationMode(ESP_LE_AUTH_REQ_SC_BOND); // i do not know how
  pSecurity->setCapability(ESP_IO_CAP_IN); // done - ble_hs_cfg.sm_io_cap = BLE_HS_IO_KEYBOARD_ONLY;
  pSecurity->setRespEncryptionKey(ESP_BLE_ENC_KEY_MASK | ESP_BLE_ID_KEY_MASK);   // i do not know how

Because connection have to be encrpzted i do not know that i do not receive the nofitifaction because wrong encryption or why.
and missing something like this _incomingCameraControl->registerForNotify(_receiveNotifyCallback, false);
I found in example the "send 2 bytes to seubscribe" so i try to write 2 bytes 0x01 and 0x00 to the CHR but i got Err code 6...

Hi,
The example blecent listen to BLE_GAP_EVENT_CONNECT event and call ble_gap_security_initiate, after that, the encryption of the connection should be enabled.
To register a service notification, you can use esp_ble_gattc_register_for_notify, see
https://docs.espressif.com/projects/esp-idf/en/latest/esp32c3/api-reference/bluetooth/esp_gattc.html

For nimble host, you can refer to the function blecent_read_write_subscribe in blecent.
Actually, the question now is how to use nimble host, maybe you can read nimble official document first.
https://mynewt.apache.org/latest/network/ble_hs/ble_hs.html

I have blecent_read_write_subscribe tried a thousend times :)
It is possible that i need differend write data?
My CHR is write only or notify only, so writing to them cause ERR 6.
I have bluetooth explorer app on my computer and i can see all CHR and roles of CHR. This explorer works as expected, he can read what is readable, he receive notifications from notif only CHR and he can write to the writable CHR.
I m missing something and i do not know what it is.
in fact i think i m connected and secured, because i can receive RX events. But just from one single CHR. I have to receive 3 CHR at the same time.
When i tried send same "subscribe" data to the others events with "no_rsp_write" i do not receive any RX events at all.
Like i said i have old Arduino code and after "Register notify" code i start imediatelly receivin bunch of data ...
So something not working properly with my nimble code.

@esp-lrh  can you tell me please what is error code 6 when i m trying to write_flat ?

ble_gattc_write_flat return error code 6 (BLE_HS_ENOMEM)?
It means no memory to store and send the data, can you provide your code for me to check?

I just try it again (not in repo now)
in function blecent_subscribe_to instead of ble_gattc_write_flat  instead of ble_gattc_write_no_rsp_flat add callback, and NULL and i m geting
W (13414) ESP: Service ERR 6
        const uint8_t notifyOn[] = {1, 0};
        /*** Write 0x00 and 0x01 (The subscription code) to the CCCD ***/

        rc = ble_gattc_write_flat(peer->conn_handle, dsc->dsc.handle, &notifyOn, sizeof(notifyOn), after_write, NULL); //ble_gattc_write_no_rsp_flat
        if (rc != 0) {
            ESP_LOGW(TAG,"Service ERR %d\n",rc);
            return rc;
        }

I want to go step by step.
First i need to send "my name" to the camera.
This CHR is not encrypted at all. So need to work.
This is what i use to to that. (i like i said i know that this is possible, because on Arduino is done by chr->writeValue)
Camera just show up the name of connected device in the menu. Now is ADDR only. To this step is first step to get done.
chr = peer_chr_find_uuid(peer, CameraService_UUID, DeviceName_UUID);
 if (chr == NULL) {
     MODLOG_DFLT(ERROR, "Error: Peer doesn't support DeviceName_UUID\n");
     goto err;
 }else{
     MODLOG_DFLT(WARN, "OK: Peer HAVE DeviceName_UUID\n");
     const char *string_to_write = ble_svc_gap_device_name();
     ESP_LOGW(TAG, "Device name %s",string_to_write);
     uint16_t data_len = strlen(string_to_write);
     rc = ble_gattc_write_no_rsp_flat(peer->conn_handle, chr->chr.val_handle, &string_to_write, sizeof(string_to_write));
     if (rc != 0) {
         MODLOG_DFLT(ERROR, "Error: Failed to WRITE characteristic; DeviceName_UUID");
         goto err;
     }
 }

OH YEAH, after 5 days of trying. I was able to set the name.
The problem was the "reading" part before "write".
So question is: When i perform "read" operation - this somhow block the other steps?
This is ma read part (just read the data and log)

    chr = peer_chr_find_uuid(peer, BLE_UUID16_DECLARE(DeviceInformation_UUID), BLE_UUID16_DECLARE(CameraModel_UUID));
    if (chr == NULL) {
        MODLOG_DFLT(ERROR, "Error: Peer doesn't support CameraModel_UUID\n");
        goto err;
    }else{
        MODLOG_DFLT(WARN, "OK: Peer HAVE CameraModel_UUID\n");
        rc = ble_gattc_read(peer->conn_handle, chr->chr.val_handle, blecent_read_log, NULL);
        if (rc != 0) {
            MODLOG_DFLT(ERROR, "Error: Failed to read characteristic; CameraManufacturer_UUID");
            goto err;
        }
    }

This part failed always with error code 6
    chr = peer_chr_find_uuid(peer, CameraService_UUID, OutgoingCameraControl_UUID);
    if (chr == NULL) {
        MODLOG_DFLT(ERROR, "Error: Peer doesn't support OutgoingCameraControl_UUID\n");
        goto err;
    }else{
        MODLOG_DFLT(WARN, "OK: Peer HAVE OutgoingCameraControl_UUID\n");
        int8_t value = {0xFF,0x05,0x00,0x00,0x01,0x0A,0x01,0x00,0x02,0x00,0x00,0x00};
        rc = ble_gattc_write_flat(peer->conn_handle, chr->chr.val_handle, &value, sizeof(value), NULL, NULL);
        if (rc != 0) {
            MODLOG_DFLT(ERROR, "Error: Failed to WRITE characteristic; OutgoingCameraControl_UUID rc=%d", rc);
            goto err;
        }
    }

Hi, @gavalierm
Pairing and encryption do not always require user to input PASSKEY, just set sm_io_cap to BLE_HS_IO_NO_INPUT_OUTPUT first.
In addition, I recommend to disable other functions, such as wifi and MQTT, when debugging Bluetooth.
I made some small changes and run your code, it can connect to example bleprph and enable encryption, subscribe notification successfully.
W (5102) NimBLE: START: blecent_subscribe_to

I (5102) NimBLE: INFO: Service HAVE DESCRIPTOR

  conn_handle=0
  conn_handle=20

I (5112) NimBLE: GATT procedure initiated: write no rsp;
I (5122) NimBLE: att_handle=20 len=2

Maybe you can connect to a bleprph  and debug first.

code 6

code 6 means no memory, it is strange, disable wifi and try again ?
error code definition see components/bt/host/nimble/nimble/nimble/host/include/host/ble_hs.h

Yes i can disable WIFI an MQTT because i already have PASSKEY exchanged.
But with BLE_HS_IO_NO_INPUT_OUTPUT the camera never fire the PASSKEY and i can not make the encrypted connection.
According to the protocol i think PASSKEY is essential and Camera do not work with BLE_HS_IO_NO_INPUT_OUTPUT.
I think i have Security done. I can WRITE to the camera command which do what i need to do. BUT JUST ONCE.
After i write command second command is returned as ERR=6.
My goal is simple and i do not understand why is so hard to achieve this.
Just read data from MQTT (hex buffer) and PASS it to the CHR on BLE (same buffer, same data, same strucutre, do nothig just pass it).
And vice versa.
READ data from Notification and just PASS it to the MQTT as buffer.
And i hoped that will be simple, with 2 callbacks on both sides for each direction and done. No storing not manipulating just pass the data and free the buffer.
Why is this so hard to do - i m so confused.

code 6

code 6 means no memory, it is strange, disable wifi and try again ? error code definition see components/bt/host/nimble/nimble/nimble/host/include/host/ble_hs.h

I disable WIFI and MQTT in main.c by commenting it out so WIFI MQTT will not start.
And still getting ERR 6 on 4th operation.

I found this in Menuconfig and looks like the 3 is the number what i m looking for.
(3) Maximum number of concurrent connections
(3) Maximum number of bonds to save across reboots
(8) Maximum number of CCC descriptors to save across reboots
(4) Maximum number of connection oriented channels

Event if you do not input a PASSKEY, the connection will also be encrypted after ble_gap_security_initiate
if you got ERR 6 after 3 operation, it means the data of previous 3 operation had not been sent.
I think your connection is stuck encryption process so it can't send data.
When I run your code first time (sm_io_cap=BLE_HS_IO_KEYBOARD_ONLY), I got a encryption timeout, it means you do not response BLE_GAP_EVENT_PASSKEY_ACTION event appropriately.
Bluetooth is not a so simple protocol, Bluetooth Core Specification has more than 3000 pages and there are many other profile, don't guess its behaviors, please read document and search in internet.
The method you response to BLE_GAP_EVENT_PASSKEY_ACTION event is not good, please refer to the example.

When I run your code first time (sm_io_cap=BLE_HS_IO_KEYBOARD_ONLY), I got a encryption timeout, it means you do not response BLE_GAP_EVENT_PASSKEY_ACTION event appropriately.
Can you point me to the EXAMPLE when is BLE_GAP_EVENT_PASSKEY_ACTION on BLECENT done?
Me esp is in CENT mode and the PHPR send the BLE_GAP_EVENT_PASSKEY_ACTION event, i need the catch this event and SEND BACK THE PASSKEY.
This is how i send back the PASSKEY if it is wrong approach i will be really glad to see the right one, can you point me to the exmaple when CENT device response to the PHPR device with the passkey?
void sendBLE_Passkey(uint32_t passkey){
            // Prepare the structure with the passkey
        struct ble_sm_io io;
        io.action = BLE_SM_IOACT_INPUT;
        io.passkey = passkey;

        ble_sm_inject_io(conn_handle, &io);
    //esp_ble_passkey_reply(mqtt_client->ble_security.ble_req.bd_addr, true, passkey);
}

if you got ERR 6 after 3 operation, it means the data of previous 3 operation had not been sent.
Thats not true, because PHPR device CHANGE THE VALUE. So my data was send and PHPR device (in my case Camera) receive the data and change the value (change settings to what i want). So still this confusing me fully.
Camera receive the data, do her job BUT data "was not send?" this make no sense.
Bluetooth is not a so simple protocol, Bluetooth Core Specification has more than 3000 pages and there are many other profile, don't guess its behaviors, please read document and search in internet.
Yes i learn this hardway... but still there is a companioes like ESP IDF or NIMBLE who create the Libs to use it without any "deep" knowledge i think. Just "read" and "write" scenario can not by so hard that i need to learn 3000 pages.... But i got your point.

Ok I make a HUGE progress just with simple BYTE :)
I explore whole Arduino library. So much inspiration here. I found that "Subscribe" message can be:

0x01 0x00 for Notification (without ACK)
0x02 0x00 for Notification (with ACK) a.k.a Indication

Because Camera PROTOCOL do not say anything about 01/02 i tried the 02 and it works! So i update the registry process like Arudino lib have - just add flag for 01/02 scenario.
So i m really close to the finish now.
Just resolve the problem with most 3 write operations (icluding the subscibe writes...)
@esp-lrh if you can help mi figure out this problem i will be done. And i can create some Example for my scenario.

So i update my git repo
I add the blecent_write_to_outgoing function to test how many writes i can perform..
Just 3 including the registrations writes.
I think there is a problem with

my ESP config
Callbacks
om_buf usage (have to be freed or something?)

I realize that the writes without ACK (no_rsp_write) not working at all. So i think the writes get respone with but my callback (NULL) dont do anything and this can be problem?
It is essential to do something after write operation?
Because my CHRs is write only, when i perform READ on this the Camera reject the connection (it looks like that)

I say not good because your method response to BLE_GAP_EVENT_PASSKEY_ACTION only work in specific scenarios
The type of io.action is not up to you, you must keep it as pkey.action = event->passkey.params.action, perform the action pass from the event. Reference example I told you before, blephrp.

Anyway, no matter what io capability you set, just make sure you got the BLE_GAP_EVENT_ENC_CHANGE event and the event status code is 0.
If you still got error code 6, try add CONFIG_BT_NIMBLE_GATT_MAX_PROCS=20 to your sdkconfig.defaults and rebuild, or edit this option in menuconfig, the default value is 4.

Who is initiator and who is responder?
Because i think in my case the initiator is "CENT" device (my code), because i "Initiate security".
My PHPR camera is "Display only" for sure. Because this is written in protocol (camera display the code) there is no other option - on the page what you send is not option for "Display only" for responder so i think is "Display/Keyoard" so in this fact my device can be "display only" and Camera should show up the keyboard to type the pass. But this never happed, camera respond to this type of request with error. So i think the developers of camera jus implement the part "Display Only" and all others (Keyboard) just responde with error.
So my CENT device (initiator) have to be some sord of "passkey INPUT" becase camera is DISPLAY ONLY (or respond only for display type of inititation).
So my cant HAVE to be set to "KeyboardOnly" because whe i set it to "DisplayKeyboard" nothing happned... camera do not show the keyboard and i do not generate the pass ...
Like you said - I got ENC response 0 so all security is not the case for me - its working.
I will try to solve the operations...
Thank you for the tip CONFIG_BT_NIMBLE_GATT_MAX_PROCS i will try.

@esp-lrh
CONFIG_BT_NIMBLE_GATT_MAX_PROCS works!!! Thank you for pointing me.
So i need to try now send the command 100 times? because (logically) i need to write to the camera many times for long long time.

And one more thing (and i hope it will be the last one)
My data is truncated to the 2 block of max 12 bytes you can see the log, that i have one log of 12 bytes and 1 log of remaining 4 bytes...
Can you please ponit me to the code example where i can combine this data before sending int to MQTT? Thank you.
I (23067) DATA: ff 10 00 00 09 07 02 02 c7 02 12 03 00 00 01 00
I (23068) DATA: 10 27 00 00
I (23167) DATA: ff 05 00 00 01 08 01 02 03

@gavalierm  ,
Please check once in your application if the word "DATA" is printed as part of it.  There is no such print with word "DATA" in stack / example.
As for the sequence, i think it may be due to print_mbuf(attr->om) call done from some place in the application.
The data is actually in sequence only, but in chained format. So it gets printed in separate lines in a loop as the print introduces a new line after every chain iteration.  It is being sent as a whole only though.

@rahult-github
i use ESP_LOG_BUFFER_HEX("DATA", data, data_len);
in BLE_onReceive which is called every time i got RX
    case BLE_GAP_EVENT_NOTIFY_RX:
        BLE_onReceive(event->notify_rx.om);
        return 0;

void BLE_onReceive(const struct os_mbuf *om){
    if(esp_mqtt_state != STATE_CONNECTED){
        return;
    }
    uint16_t data_len = OS_MBUF_PKTLEN(om);
    uint8_t *data = malloc(data_len);

    if (data != NULL) {
        os_mbuf_copydata(om, 0, data_len, data);
        if(data[4] == 9 && data[5] == 0){ // 4 means operation type timecode 5 means thick
            //ESP_LOGW(BLE_TAG,"DATA Timecode Thick");
            return;
        }
        ESP_LOG_BUFFER_HEX("DATA", data, data_len);
        // send data to mqtt using enqueue whis is async-like behavior
        // sending from camera to the operator do not have priorty
        esp_mqtt_client_enqueue(mqtt_client, MQTT_DOWNSTREAM_TOPIC, (const char *)data, sizeof(data), 0, 0, true);

        // Free the allocated buffer
        free(data);
    } else {
        ESP_LOGE(BLE_TAG, "Failed to allocate memory for data");
    }
}

So this means that RX have to be triggeret twice with 12 bytes and with 4 remaining bytes...

Closing this issue . Please feel free to reopen in case of further queries.


Update s3 links in README to remove 0.0.5

Used mc to copy data from one location to another to remove 0.0.5 dir incorrectly included in previous links.
E.g.
time ./mc cp -r uk1s3/idr/share/ome2024-ngff-challenge/0.0.5/idr0090/190129.zarr uk1s3/idr/share/ome2024-ngff-challenge/idr0090/
...90129.zarr/zarr.json: 1.01 TiB / 1.01 TiB ┃▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓┃ 1.07 GiB/s 16m6s
real    16m6.120s
user    4m27.611s
sys     1m6.473s

Once this PR is merged, I can delete data from old locations (might break some links in old PRs etc, especially the validator PR at https://github.com/ome/ome-ngff-validator/pull/36 - so I may update some of those links).
cc @joshmoore

@will-moore: can you pre-commit install in this repo to catch the formatting errors?