Disconnect call can hang thermostat

Here's the raw data from PostMan for a connect, disconnect, connect sequence using the same AppID:
POST https://<IP_ADDRESS>/Endpoints/Arzel/Connect
204
82 ms
Warning: Self signed certificate
POST /Endpoints/Arzel/Connect HTTP/1.1
User-Agent: PostmanRuntime/7.32.3
HTTP/1.1 204 No Content
Server: MicroRelayServer-LCC
Connection: Keep-Alive
Content-Type: text/plain
z-conn_id: 0xb1e47484
z-method: POST
z-uri: /Endpoints/Arzel/Connect
z-time: Jun 21 16:52:41
Content-Length: 0
POST https://<IP_ADDRESS>/Endpoints/Arzel/Disconnect
204
94 ms
Warning: Self signed certificate
POST /Endpoints/Arzel/Disconnect HTTP/1.1
User-Agent: PostmanRuntime/7.32.3
Accept: /
Accept-Encoding: gzip, deflate, br
Content-Length: 0
HTTP/1.1 204 No Content
Server: MicroRelayServer-LCC
Connection: Keep-Alive
Content-Type: text/plain
z-conn_id: 0xb1e47404
z-method: POST
z-uri: /Endpoints/Arzel/Disconnect
z-time: Jun 21 16:53:01
Content-Length: 0
POST https://<IP_ADDRESS>/Endpoints/Arzel/Connect
400
68 ms
Warning: Self signed certificate
POST /Endpoints/Arzel/Connect HTTP/1.1
User-Agent: PostmanRuntime/7.32.3
HTTP/1.1 400 Bad Request
Server: MicroRelayServer-LCC
Connection: Keep-Alive
Content-Type: text/plain
Content-Length: 84
Transfer-Encoding: chunked
{"code":4,"message":"Connect Failed!: Tstat is already connected with other mobile"}
Trying any other connect with any other appID fails with the same message as above. Any subsequent disconnects result in an response that isn't even HTTP :(
POST https://<IP_ADDRESS>/Endpoints/Arzel/Disconnect
Error: Parse Error: Expected HTTP/
Request Headers
User-Agent: PostmanRuntime/7.32.3
Accept: /
Accept-Encoding: gzip, deflate, br
Content-Length: 0

I will try this on my S30 tomorrow to see if it behaves the same.
A difference between the sequence you are doing and the sequence that the integration does, is after the Connect call it establishes a subscription.  It is possible that Disconnecting from a Connection without a subscription causes some type of crash in lennox.
This endpoint should be used to establish a subscription:
POST https://{self.ip}/Messages/RequestData
With a message body similar to this.  SenderID should be your app_id "Arzel"
{
    "MessageType": "RequestData",
    "SenderID": "**redacted**",
    "MessageID": "3f30d00b-3282-4d31-9d6c-5bec3e564848",
    "TargetID": "LCC",
    "AdditionalParameters": {
        "JSONPath": "1;/systemControl;/systemController;/reminderSensors;/reminders;/alerts/active;/alerts/meta;/bleProvisionDB;/ble;/indoorAirQuality;/fwm;/rgw;/devices;/zones;/equipments;/schedules;/occupancy;/system"
    }
}

And then again
{
    "MessageType": "RequestData",
    "SenderID": "**redacted**",
    "MessageID": "1b8fdc51-cc67-4df9-acf7-5aeb8a9bea1e",
    "TargetID": "LCC",
    "AdditionalParameters": {
        "JSONPath": "1;/automatedTest;/zoneTestControl;/homes;/reminders;/algorithm;/historyReportFileDetails;/interfaces;/logs"
    }
}

From there get on this
https://{self.ip}/Messages/{self._applicationid}/Retrieve
To get messages, there will be alot of them.

Tried exactly that sequence and when no more messages were retrievable, (204 response with no content), I issued a disconnect. Which succeeded. But then any further attempts to connect (or disconnect) fail and I have to restart the S40.

Interesting.  I would expect every user of this integration to run into that problem when they shutdown and restart HASS. My understanding is you are not using the integration and so have not experienced that issue with Home Assistant?
One other thing to try is to make the app_id all lower-case.  Lennox usually constructs some like dca_nnnnn from the mobile app when performing a direct connect.

I just tested this on my S30  using postman to  POST the connect and disconnect using the app_id = ha_test1 and it works fine.
I did get a weird error on Disconnect when I tried to use a Get instead of Post

Interesting. I would expect every user of this integration to run into that problem when they shutdown and restart HASS. My understanding is you are not using the integration and so have not experienced that issue with Home Assistant?
One other thing to try is to make the app_id all lower-case. Lennox usually constructs some like dca_nnnnn from the mobile app when performing a direct connect.

The integration was working fine for me with my S40's until the last Lennox firmware update which I received June 8th.  Since then it has not worked unless I restart the thermostats.

Very interesting.  And it stops working whenever you re-start the integration?  On restart is does issue the disconnect which @GabeDeBacker says created an issue.

I had removed the thermostats from HA thinking it was something with an HA update or something else.  In attempting to add them I can only get to the initial integration page after a fresh reboot of the thermostats.  After they have been added to integration and reboot the thermostat again, it will show devices it found on the thermostat.
First thermostat reboot

Second thermostat reboot - upstairs only (*.10 IP)

That makes sense, during integration configuration it creates a connection, grabs some info and then Disconnects.
I have a patch for the API for you to test, give me 15 minutes.

I pushed a new version of the API 0.2.8
You'll need to be on the latest release of the integration which is 2023.5.2
In the custom_components/lennoxs30 directory there is a manifest.json file, edit it
replace
  "requirements": ["lennoxs30api==0.2.7"],

with
  "requirements": ["lennoxs30api==0.2.8"],

Then you will need to restart HASS and it will download the updated API.  The API should no longer issue the disconnect to an S40.
It will log a debug message:
            _LOGGER.debug("Skipping logout for S40 thermostat")

To see that you'd need to enable debug logging described here:
https://github.com/PeteRager/lennoxs30#reporting-bugs
logger:
  default: warning
  logs:
    custom_components.lennoxs30: debug
    lennoxs30api: debug

But, I'd say try the update and if the issues goes away - fantastic - if not then let's enable debug to make sure its hitting that line.

Thanks
Try to reload the integration.  That should push it through the Disconnect / Reconnect cycle.

I've verified you are running 04.20.007 same version reported in this issue #245

I reloaded both thermostats.  All seems to be working as expected.  Thank you very much for resolving this so quickly.
home-assistant.log

Thanks for providing the logs and reproduction scenario.
Big call out to @GabeDeBacker for discovering and reporting the error with disconnect.  We'd still be hunting for the bug Lennox introduced.
I'll get this packaged into a new release in the morning, the only change will be the manifest that you already updated.

Log entries verify the disconnect is being ignored for S40.
2023-06-22 17:05:11.395 INFO (MainThread) [lennoxs30api.s30api_async] logout - Entering - [https://<IP_ADDRESS>/Endpoints/homeassistant/Disconnect]
2023-06-22 17:05:11.395 DEBUG (MainThread) [lennoxs30api.s30api_async] Skipping logout for S40 thermostat
2023-06-22 17:05:11.395 DEBUG (MainThread) [lennoxs30api.s30api_async] Closing Session
2023-06-22 17:05:11.399 DEBUG (MainThread) [custom_components.lennoxs30] async_shutdown complete [<IP_ADDRESS>]

Closing this as the new release should resolve.  Open new issues if this is not the case:
https://github.com/PeteRager/lennoxs30/releases/tag/2023.6.1


Add to stackage

Hi. Thanks for that package, works exactly as needed so far.
We are going to use it in https://github.com/fpco/stackage-server/pull/132. And it would be much nicer if barrier itself was present on stackage.
It would be really great if you could add barrier to stackage with yourself as a maintainer. If not, I'd be glad to do it myself.
Please tell what you think.

https://github.com/fpco/stackage/pull/914

Sorry for the delay in answering...
I'm going to be maintainer of barrier on stackage:smile:
Could you close your PR?

That's great. Thanks.


modules: How to specify an option is not optional?

I want to create a module for the isso discussion system.
When the user enables it, he has to set the config via a config option, otherwise isso won’t do anything sensible.
The way I do it now is not set any default and simply use it when enabled, so the module system throws the following error:
error: The option `services.isso.config' is used but not defined.
(use ‘--show-trace’ to show detailed location information)

I think that could be a bit misleading maybe, is it supposed to be done this way?

See nixos/modules/misc/assertions.nix

Sweeeet!
Now, how to document that? It might be sensible to add a writeup of nixos/modules/misc to Chapter 22 of the nixos manual.

Doesn't the error message say exactly what the problem is? I.e. that the user needs to define services.isso.config.

But the user never used the option anywhere, and if one doesn’t know how the module system works (aka that it eats its own dogfood), it might lead to serious self-doubt (“but I didn’t use it anywhere. I checked everything”).
On the other hand that would be trivially resolved by an error message structured like this:
error: The option `services.isso.config' is used at `…/nixpkgs/nixos/modules/…/isso.nix:49 but not defined.

@Profpatsch 22.3 sections sounds good


quicksilver repoducible crash (fast user switching)

Every time my son uses my computer, Quicksilver crashes the next time on launch. Then I have to click "clear caches" then wait for a minute while it restarts. Here is what I think is happening:
Quicksilver is running in my account (Mac os x Lion 10.7.3)
He logs in to his account with fast user switching menu
Computer is shut down by entering an administrator password when prompted
(note: we did not return to the 1st account by fast user switching)
computer is booted
quicksilver crashes on launch

I also have problems with Quicksilver crashing after startup. I always also get the "Clear Caches" warning. Luckily I reboot my machine only once in a while

Hi there,
From reading your report, it seems as if Quicksilver is not actually crashing, but the crash protection mechanism we have put in place is not fully realising that Quicksilver has not actually crashed. We will look at improving the crash protection in a future release.
In the meantime, I suggest you click the 'Do Nothing' button when the pop up appears.

I rebooted my machine today and experienced the same problem.
I couldn''t use Quicksilver (Cmd+Space). But before starting it manually by opening Quicksilver.app, I checked the running processes via ps -ef | grep Quick and it was started but just not working.
If I then proceed and open Quicksilver.app I get the dialog.


statusHeat not good?

I am pretty sure the statusHeat flag is not working (I wanted to have some fun with it). When I look at the webpage source, I think that Control.Model.Property.statusHeat (which the code picks up) has no relation with the heat being on. (1) Control.Model.Property.statusCool is set to 1 at the same time, and (2) I can see that flag being 0 when the graphics shows that the heater is on.
There is an interesting tag  further down on the website. This seems to be followed by  class="" when the heater is on, and class="hidden" when not. Might be a better way to do this

I agree, statusHeat isn't working. Here's the code that determines that on the site-
 updateEquipmentStatus = function() {
        var isFanRunning = model.get(model.Property.isFanRunning);
        var eqOutputStatus = model.get(model.Property.equipOutputStatus);
        var isLost = model.get(model.Property.isLost);

        $(ui.eqStatusFanOn).css('display', !isLost && isFanRunning != undefined && isFanRunning && (eqOutputStatus == undefined || (eqOutputStatus != 1 && eqOutputStatus != 2)) ? "inline-block" : "none");
        $(ui.eqStatusHeating).css('display', !isLost && eqOutputStatus != undefined && (eqOutputStatus == 1) ? "inline-block" : "none");
        $(ui.eqStatusCooling).css('display', !isLost && eqOutputStatus != undefined && (eqOutputStatus == 2) ? "inline-block" : "none");
    };

isFanRunning and equipOutputStatus appear to be set in a different way than the other properties, and aren't present in the source code at all, so I'm not sure if we'll be able to scrape them directly. Your idea of scraping the output would make sense to me.


Coarse-grained SVM has to be mapped before usage!

I have a system with CL_DEVICE_SVM_COARSE_GRAIN_BUFFER only.
SVMVec does not work as expected, since the buffer has to be mapped via CL_MAP_READ / CL_MAP_WRITE before reading / writing and unmapped before executing a kernel. While an SVM is mapped, you cannot grow it. So SVMVec::push() ect. won't work. Also, SVMVec::with_capacity_zeroed() does not zero the values, since it doesn't map.
See https://software.intel.com/content/www/us/en/develop/articles/opencl-20-shared-virtual-memory-overview.html#access
I tested this with the correctly applied mapping game and everything worked just fine.

Here is a modified sample part of the Readme.md that works for me:
        // Copy input data into an OpenCL SVM vector
        let mut test_values = SvmVec::<cl_int>::with_capacity(&context, svm_capability, ARRAY_SIZE);
        unsafe { test_values.set_len(ARRAY_SIZE) };

And then I could either:
        if !is_fine_grained_svm {
            queue.enqueue_svm_map(CL_BLOCKING, CL_MAP_WRITE, &mut test_values, &[]).unwrap();
        }
        test_values.copy_from_slice(&value_array);
        if !is_fine_grained_svm {
            queue.enqueue_svm_unmap(&mut test_values, &[]).unwrap();
        }

Or just:
        queue.enqueue_svm_mem_cpy(
            CL_BLOCKING,
            test_values.as_mut_ptr() as *mut c_void,
            value_array.as_ptr() as *const c_void,
            ARRAY_SIZE * mem::size_of::<cl_int>(),
            &[] ).unwrap();

Which seems to sync the memory automatically.
Either way, it takes away a lot of comfort.

Thank you for taking the time to analyse this @Draghtnod.
Your issue is closely related to issue #32, which also involves CL_DEVICE_SVM_COARSE_GRAIN_BUFFER support.
I would like to fix this issue in SvmVec, but as you point out (and the linked Intel article explains), coarse grained SVM needs to be mapped before it can be grown or initialised.
The easiest solution for me is to ensure that SvmVec can only be used with fine grained SVM; coarse gained SVM would just have to use conventional buffers, as per issue #32 . However, I would like to support coarse grained SVM. Do you have any ideas how best to support it without braking the "comfort" of fine grained SVM?

@Draghtnod there is a test test_opencl_svm_example in integration_test.rs that shows how to use both fine and coarse grained SVM. However, the coarse grained SVM test was incorrect. Fortunately, I've been able to update one of my systems and I've now corrected it to call enqueue_svm_map before setting the values in the input SVM vectors.

@Draghtnod I've just incorporated changes to fix this issue in version 0.5 of the library on crates.io.
If you are satisfied with the changes then please close this issue.

@kenba I read your code changes and I think this is fine. SvmVec may still be useful on systems with CL_DEVICE_SVM_COARSE_GRAIN_BUFFER, so disabling it would be a shame. On the other hand, shifting the hustle of automatically queuing all the mapping/unmapping into SvmVec, so it feels like a fine grained buffer, would take away the flexibility to queue in an efficient manner. Leaving the mapping/unmapping in the hands of the user, like you did, is the right thing to do in my opinion.
Maybe it would get a little more comfy for CL_DEVICE_SVM_COARSE_GRAIN_BUFFER users with some SvmVec functions like:
    pub fn map_read(&mut self, &mut queue: CommandQueue) {
        if !self.is_fine_grained() {
            queue.enqueue_svm_map(CL_BLOCKING, CL_MAP_READ, &mut self, &[]).unwrap();
        }
    }

This way you could utilize SvmVec in an conservative manner (not pushing beyond cap() and so on) and could still get coarse buffer conform with relative high convenance by just adding a few map_read()/map_write()/unmap() without writing a different SVM logic for every buffer type.


Unicity of pair (port, message)

I was wondering about the following case:
If we have 3 things t1, t2, and t3, and a unique message m
configuration testMessageForwarding {
	instance t1 : t1
	instance t2 : t2
	instance t3 : t3

	connector t2.p1 => t1.p //t1 sends m
	connector t3.p => t2.p2 //t2 sends m on p2 whenever it receives m on p1

}

Is this situation acceptable? Because, so far it compile and generate working code in c, but m is referenced by two different codes depending on by which port it has been sent.
It could become problematic when the annotation @code is used to force the choice of a code.
From there, I see several solutions:

say that different messages must be declared for this case,
move the annotation @code at the port declaration level
"fix" the code generator in order to handle this case. (meaning rewrite the processMessageQueue generation)

Either way, I can do it, but if someone has an opinion on the matter, I would be glad to hear it.

fixed


Import Fails: json.decoder.JSONDecodeError

Hi,
Recieving the following message when trying to import on Windows 10:
Traceback (most recent call last):   File "C:\Users\Jamie Adams\bin\elodie\elodie.py", line 345, in <module>     main()   File "C:\Users\Jamie Adams\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 716, in __call__     return self.main(*args, **kwargs)   File "C:\Users\Jamie Adams\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 696, in main     rv = self.invoke(ctx)   File "C:\Users\Jamie Adams\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 1060, in invoke     return _process_result(sub_ctx.command.invoke(sub_ctx))   File "C:\Users\Jamie Adams\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 889, in invoke     return ctx.invoke(self.callback, **ctx.params)   File "C:\Users\Jamie Adams\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 534, in invoke     return callback(*args, **kwargs)   File "C:\Users\Jamie Adams\bin\elodie\elodie.py", line 115, in _import     trash, allow_duplicates)   File "C:\Users\Jamie Adams\bin\elodie\elodie.py", line 64, in import_file     media, allowDuplicate=allow_duplicates, move=False)   File "C:\Users\Jamie Adams\bin\elodie\elodie\filesystem.py", line 336, in process_file     media.set_original_name()   File "C:\Users\Jamie Adams\bin\elodie\elodie\media\media.py", line 246, in set_original_name     if self.get_original_name() is not None:   File "C:\Users\Jamie Adams\bin\elodie\elodie\media\media.py", line 143, in get_original_name     exiftool_attributes = self.get_exiftool_attributes()   File "C:\Users\Jamie Adams\bin\elodie\elodie\media\media.py", line 129, in get_exiftool_attributes     metadata = et.get_metadata(source)   File "C:\Users\Jamie Adams\bin\elodie\elodie\external\pyexiftool.py", line 330, in get_metadata     return self.execute_json(filename)[0]   File "C:\Users\Jamie Adams\bin\elodie\elodie\external\pyexiftool.py", line 314, in execute_json     return json.loads(self.execute(b"-j", *params).decode("latin-1"))   File "C:\Users\Jamie Adams\AppData\Local\Programs\Python\Python36\lib\json\__init__.py", line 354, in loads     return _default_decoder.decode(s)   File "C:\Users\Jamie Adams\AppData\Local\Programs\Python\Python36\lib\json\decoder.py", line 339, in decode     obj, end = self.raw_decode(s, idx=_w(s, 0).end())   File "C:\Users\Jamie Adams\AppData\Local\Programs\Python\Python36\lib\json\decoder.py", line 357, in raw_decode     raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Anything I can provide to help debug? I'm not using any special configuration in config.ini.

I just got the same error, after successfully processing almost 2GB of images. Just died
Traceback (most recent call last):
  File "C:\Users\dixon\elodie\elodie.py", line 345, in <module>
    main()
  File "C:\Users\dixon\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\dixon\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 696, in main
    rv = self.invoke(ctx)
  File "C:\Users\dixon\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\dixon\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\dixon\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\dixon\elodie\elodie.py", line 115, in _import
    trash, allow_duplicates)
  File "C:\Users\dixon\elodie\elodie.py", line 64, in import_file
    media, allowDuplicate=allow_duplicates, move=False)
  File "C:\Users\dixon\elodie\elodie\filesystem.py", line 336, in process_file
    media.set_original_name()
  File "C:\Users\dixon\elodie\elodie\media\media.py", line 246, in set_original_name
    if self.get_original_name() is not None:
  File "C:\Users\dixon\elodie\elodie\media\media.py", line 143, in get_original_name
    exiftool_attributes = self.get_exiftool_attributes()
  File "C:\Users\dixon\elodie\elodie\media\media.py", line 129, in get_exiftool_attributes
    metadata = et.get_metadata(source)
  File "C:\Users\dixon\elodie\elodie\external\pyexiftool.py", line 330, in get_metadata
    return self.execute_json(filename)[0]
  File "C:\Users\dixon\elodie\elodie\external\pyexiftool.py", line 312, in execute_json
    return json.loads(self.execute(b"-j", *params).decode("utf-8"))
  File "C:\Users\dixon\AppData\Local\Programs\Python\Python36\lib\json\__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "C:\Users\dixon\AppData\Local\Programs\Python\Python36\lib\json\decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\dixon\AppData\Local\Programs\Python\Python36\lib\json\decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Do either of you know which file this error occurred? If you could send me the file that may help me reproduce the bug.

This is the last thing in the buffer before the error
F:\Pictures\ONLINE\Social Media\IMG_20170204_093148736_kindlephoto-647753632.jpg -> F:\Pictures\Google Photos Staging\2017-02-Feb\Unknown Location\2017-02-04_15-31-48-img_20170204_093148736_kindlephoto-647753632.jpg

@dixonge looks like that's the last photo which succeeded. It would be helpful to know which one caused an error. Could you add print(_file) to this line and run the import command again?

It will probably be later tonight or tomorrow.
Speaking of files, what determines the file order? It seems to be rather random and across folders...

@dixonge sort order is based on the default of os.walk which is not guaranteed to be of any order. However, there's a pull request to sort them alphabetically which has not been merged yet.

To the ‘move = False’ line? I’m not sure where to put it. Could you show me in context?
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10
From: Jaisen<EMAIL_ADDRESS>Sent: Tuesday, October 10, 2017 1:33 AM
To<EMAIL_ADDRESS>Cc: Glenn<EMAIL_ADDRESS><EMAIL_ADDRESS>Subject: Re: [jmathai/elodie] Import Fails: json.decoder.JSONDecodeError (#251)
@dixongehttps://github.com/dixonge looks like that's the last photo which succeeded. It would be helpful to know which one caused an error. Could you add print(_file) to this linehttps://github.com/jmathai/elodie/blob/master/elodie/filesystem.py#L324 and run the import command again?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/jmathai/elodie/issues/251#issuecomment-335374631, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABgWCgD7EqKva3REj6GpdRvKfOjTMzSaks5sqw-7gaJpZM4Phx7w.

Sure, here you go.
    def process_file(self, _file, destination, media, **kwargs):
        print(_file)
        move = False
        if('move' in kwargs):
            move = kwargs['move']

I am rerunning the import now, but it looks like elodie is remembering the previous import, because my external drive now has a different drive letter, and I am getting notifications like "matched checksum but file not found at F:..." So far all of my imports have been tests using a small group of folders.

There's a local database at ~/.elodie/hash.json. You can delete that file whenever you want to reimport photos. There's also a --allow-duplicates flag you can pass in where it skips the check altogether.

and, of course, it imported 83 files from 6 folders into the target folder inside 8 year-based folders with no errors....

Is it the same corpus of photos that caused an error before and with the same config file?

Same files, but config file has been changed – mainly just dropped the location subfolders and used year-only for dated folders. Still working out the exact folder structure I want.
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10
From: Jaisen<EMAIL_ADDRESS>Sent: Friday, October 13, 2017 3:34 PM
To<EMAIL_ADDRESS>Cc: Glenn<EMAIL_ADDRESS><EMAIL_ADDRESS>Subject: Re: [jmathai/elodie] Import Fails: json.decoder.JSONDecodeError (#251)
Is it the same corpus of photos that caused an error before and with the same config file?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/jmathai/elodie/issues/251#issuecomment-336560439, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABgWCoa-Fd5dGn9I4A8CuZLgUoBiBdb3ks5sr8lRgaJpZM4Phx7w.

@dixonge for the sake of this bug do you mind rerunning the import with the original config file which caused the error and letting me know what file causes it? That will help me reproduce.

Running now, but if it stops like it did last time I don't think I will have any better idea which file was the one being processed. It is currently going through the six folders randomly. Would --debug help?

Yes, I would include the --debug. Also, the print(_file) should  output the file that is being imported before failing.

83 complete, 0 errors
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10
From: Jaisen<EMAIL_ADDRESS>Sent: Friday, October 13, 2017 6:06 PM
To<EMAIL_ADDRESS>Cc: Glenn<EMAIL_ADDRESS><EMAIL_ADDRESS>Subject: Re: [jmathai/elodie] Import Fails: json.decoder.JSONDecodeError (#251)
Yes, I would include the --debug. Also, the print(_file) should output the file that is being imported before failing.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/jmathai/elodie/issues/251#issuecomment-336587339, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABgWCnOokpA1VdQOqOzkeaigQeK9WHIPks5sr-zcgaJpZM4Phx7w.

I get this error too. It happens on different files (which I can tell after adding the print(_file) line in. I've looked at those files and I don't see anything irregular about them.
This is my output, from running this command:
"./elodie.py import --destination="~/2013-elodie" ~/2013 --debug":
/home/evan/2013/combined/combined/06_22_img_6301.jpg
/home/evan/2013/combined/combined/06_22_img_6301.jpg -> /home/evan/2013-elodie/2013-06-Jun/Unknown Location/2013-06-22_02-46-35-06_22_img_6301.jpg
/home/evan/2013/combined/combined/09_24_15689814391.jpg
Traceback (most recent call last):
File "./elodie.py", line 345, in
main()
File "/home/evan/anaconda3/lib/python3.6/site-packages/click/core.py", line 716, in call
return self.main(*args, **kwargs)
File "/home/evan/anaconda3/lib/python3.6/site-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/home/evan/anaconda3/lib/python3.6/site-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/evan/anaconda3/lib/python3.6/site-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/evan/anaconda3/lib/python3.6/site-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "./elodie.py", line 115, in _import
trash, allow_duplicates)
File "./elodie.py", line 64, in import_file
media, allowDuplicate=allow_duplicates, move=False)
File "/home/evan/elodie/elodie/filesystem.py", line 337, in process_file
media.set_original_name()
File "/home/evan/elodie/elodie/media/media.py", line 286, in set_original_name
if self.get_original_name() is not None:
File "/home/evan/elodie/elodie/media/media.py", line 183, in get_original_name
exiftool_attributes = self.get_exiftool_attributes()
File "/home/evan/elodie/elodie/media/media.py", line 131, in get_exiftool_attributes
metadata = et.get_metadata(source)
File "/home/evan/elodie/elodie/external/pyexiftool.py", line 330, in get_metadata
return self.execute_json(filename)[0]
File "/home/evan/elodie/elodie/external/pyexiftool.py", line 314, in execute_json
return json.loads(self.execute(b"-j", *params).decode("latin-1"))
File "/home/evan/anaconda3/lib/python3.6/json/init.py", line 354, in loads
return _default_decoder.decode(s)
File "/home/evan/anaconda3/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/evan/anaconda3/lib/python3.6/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I just found this one year old thread.  Has it been solved by now?
I ran into the same error. Not by using elodie but in my own program which also uses pyexiftool. Scanning many images in varies folders, I noticed the error "Expecting value: line 1 column 1 (char 0)" occurs always, if the path name contains accented characters, such us ä, ö, ü or é.

@WilfriedBl there have been various unicode errors but if you share your photo I can see if it fails.

Thank you for your quick response!
My question is not related to elodie, but only to pyexiftool (which I believe you use too ..?) and I am curious if you experienced (and hopefully solved) a similar problem.
It does not appear only for a single photo, but for all images residing in folders with path name containing accented characters.
Your response  reminded me to do a more isolated test. So I copied one of my pictures to a different folder with the name 'testä'. Running my program which uses pyexiftool, indeed brings the error message mentioned above - renaming the folder to "test" and running it again, works fine.

We made this change to pyexiftool.py which may help.

Thank you! I was hoping for something like that, the change didn't solve the problem. In fact it seems the json,loads in line 312 never raises an exception and I get:
Traceback (most recent call last):
File "D:/PyCharmProjects/Daminion/DAMinspect.py", line 623, in
inspect_each_item()
File "D:/PyCharmProjects/Daminion/DAMinspect.py", line 333, in inspect_each_item
exif_been_read = et.get_tags(exiftool_tags_to_read, str(path_and_file))
File "D:\PyCharmProjects\Daminion\exiftool.py", line 364, in get_tags
return self.get_tags_batch(tags, [filename])[0]
File "D:\PyCharmProjects\Daminion\exiftool.py", line 356, in get_tags_batch
return self.execute_json(*params)
File "D:\PyCharmProjects\Daminion\exiftool.py", line 315, in execute_json
return json.loads(self.execute(b"-j", *params).decode("latin-1"))
File "C:\Program Files\Python36\lib\json_init_.py", line 354, in loads
return _default_decoder.decode(s)
File "C:\Program Files\Python36\lib\json\decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files\Python36\lib\json\decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Seems like the json.loads line does throw the exception. Unsure if it's the same as what the commit I pointed to fixes though.
File "D:\PyCharmProjects\Daminion\exiftool.py", line 315, in execute_json
return json.loads(self.execute(b"-j", *params).decode("latin-1"))

Hello Jaisen, just to let you know the final solution for my problem:
I needed to add .encode('latin-1') to the file name in my code where I call the ExifTool. This solved the problem, if there is any of the special characters éèêáàâÉÈÊÁÀÂäöüßÄÖÜ in the path name: . However, pyexiftool still throws an exception if the path contains 'æ', but since I found only a single path with it, renaming the folder was the easier solution.
I wish you a happy new year!

`  def get_metadata(self, filename):
"""Return meta-data for a single file.
    The returned dictionary has the format described in the
    documentation of :py:meth:`execute_json()`.
    """
    return self.execute_json(filename.encode('latin-1'))[0]

// changed from: return self.execute_json(filename)[0]
`
in here:
https://github.com/jmathai/elodie/blob/4186d4309a7b1c2b9ddca1f27275ed204d14110b/elodie/external/pyexiftool.py#L334
Hi, this indeed fixed it for me.
The problem was for me when special chars are in filename is that exiftool did not read in any JSON. I modified code to print out the parameters passed to exiftool and the returned JSON in the code.
Without the fix these are the outputs:
Filename: á.jpg
python3 ./elodie.py import --destination="A:\elodie-test" "A:\arch\archive-kep2\asztal!\test" --debug --album-from-folder
Parameters given to exiftool:
(b'-j', b'A:\arch\archive-kep2\asztal!\test\\xc3\xa1.jpg')
Returned JSON:
b''
Filename: a.jpg
python3 ./elodie.py import --destination="A:\elodie-test" "A:\arch\archive-kep2\asztal!\test" --debug --album-from-folder
Parameters given to exiftool:
(b'-j', b'A:\arch\archive-kep2\asztal!\test\a.jpg')
Returned JSON:
b'[{\r\n  "SourceFile": "A:/arch/archive-kep2/asztal!/test/a.jpg",\r\n  "ExifTool:ExifToolVersion": 12.10,\r\n  "File:FileName": "a.jpg",\r\n  "File:Directory": "A:/arch/archive-kep2/asztal!/test",\r\n  "File:FileSize": 210976,\r\n  "File:FileModifyDate": "2020:11:20 06:31:09+01:00",\r\n  "File:FileAccessDate": "2020:11:20 06:31:09+01:00",\r\n  "File:FileCreateDate": "2020:11:20 04:58:54+01:00",\r\n  "File:FilePermissions": 666,\r\n  "File:FileType": "JPEG",\r\n  "File:FileTypeExtension": "JPG",\r\n  "File:MIMEType": "image/jpeg",\r\n  "File:ImageWidth": 1137,\r\n  "File:ImageHeight": 674,\r\n  "File:EncodingProcess": 0,\r\n  "File:BitsPerSample": 8,\r\n  "File:ColorComponents": 3,\r\n  "File:YCbCrSubSampling": "2 2",\r\n  "JFIF:JFIFVersion": "1 1",\r\n  "JFIF:ResolutionUnit": 1,\r\n  "JFIF:XResolution": 96,\r\n  "JFIF:YResolution": 96,\r\n  "XMP:XMPToolkit": "Image::ExifTool 12.10",\r\n  "XMP:Album": "test",\r\n  "Composite:ImageSize": "1137 674",\r\n  "Composite:Megapixels": 0.766338\r\n}]\r\n'
A:\arch\archive-kep2\asztal!\test\a.jpg already at A:\elodie-test\2020\test\2020-11-20_03-58-54-nev.jpg.

@vkhobor were you able to make a code that was more generic which worked for you?

I couldn't sorry.


Non-pinned AppBar can result in unreadable status bar on iOS

Steps to Reproduce
Setup:

Open Flutter Gallery example
Open Contacts demo
Set App bar scrolls away in the app bar drop-down menu

Issues:

Top overscroll: drag down; overscroll makes white status bar text unreadable over white background.
Scroll down the page: when the AppBar scrolls offscreen, the status bar is white on white.

For the downward scroll, one option might be to have the last 20 pixels of the app bar pin under the status bar area.

Some parts of the system are aware of the padding passed down from the Engine, presumably this part just needs to take that into account as well?

Is this a regression? Or just a bug in a new feature?

Looks like a already-existing bug revealed by the change in default behaviour in the Contacts demo (previously defaulted to pinned app bar, like the Gallery home).

Alright on further discussion with @Hixie, it seems like there are three bugs:

Overscroll-related: status bar text invisible on same-coloured background: #5718
The contacts overscroll is wrong on iOS. The overscroll should apply to the section under the photo (currently a block with categories and contact items. Filed #8466
We should provide a mode where we do the suggested pin of the status bar height under the status bar. (this bug)

Dupe of https://github.com/flutter/flutter/issues/4630?

Yep 3 is the same issue as this one.


[stable/chaoskube] Support K8S older then 1.9

Add support for API versions for older k8s in the deployment file.

/assign @linki

Verify CLA

Verify CLA

Could you sign the CLA @shaikatz?

/ok-to-test

Verify CLA

Verify CLA

Verify CLA

Verify CLA

@linki I had some issues signing the CLA, it's ready now :)

@davidkarlsen Cool method, wasn't familiar with that.

/lgtm


Random freezes after 4.1.0 upgrade

I confirm (by marking "x" in the [ ] below: [x]):

[x] This is not a troubleshooting question. Troubleshooting questions go here: http://www.mattermost.org/troubleshoot/.
[ ] This doesn't reproduce on web browsers (such as in Chrome). If it does, issue reports go to the Mattermost Server repository.
[x] I have read contributing guidelines.

Summary
After upgrading to version 4.1.0 of the Mattermost desktop app, every now and then when writing messages the app freezes for a while (~15sec). I am not able to enter more text. This happens once or twice a day. I was not able to reproduce it in a web browser, given this doesn't happen regularly on desktop, this could just be a coincidence.

Operating System: Linux (KDE Neon User Edition)
Mattermost Desktop App version: 4.1.0
Mattermost Server version: 4.10.0
Clear steps to reproduce the issue

Hi @shochdoerfer, thank you for reporting this issue!
Has this issue been happening daily? Would there be any logs that you would be able to gather from around the time the freezes occur?

The issue happened twice so far, in both cases I had the app open for hours. I don't have any logs regarding the freeze, will see if I provide those at a later point.

We added a mechanism to clear memory cashes at v4.1.0. https://github.com/mattermost/desktop/pull/746
Possibly it might affect performance. However its interval is 6 hours, so I think you would not see that frequently.

Hmm, not sure if it really is the client. I switched to Rambox a few days ago and also got a freeze once when having a Mattermost tab open.

Just had another freeze and wasn't able to open another application, thus feels more like a KDE / Linux issue rather an an issue with the desktop app. Will close the issue for now. Might re-open it, if I can clearly say that this is Mattermost issue.


Have a local-development-only Docker image for an easy-to-use dev setup

The idea is to ship RIG in a development configuration that's really easy to use. Ideally, all that'd be needed is this:
$ docker run -d -p 4000:4000 -p 4010:4010 accenture/reactive-interaction-gateway:latest-local-development-only

Some ideas around the default settings that come with the image:

JWT validation could be disabled (new feature!) and just log a warning instead
Logging incoming and outgoing messages could be set to a more detailed level

Since the whole point is to have zero configuration on the users' end, it probably doesn't make sense to apply this to the AWS flavor or enable the Kafka integration. Ideas welcome!

Superseded by #90 as JWTs will no longer be mandatory. Therefore, disabling JWT validation is no longer needed. In general, I think our goal should be to make the default image as easy to use as possible, so we really shouldn't need such an image anymore.


Is the CUDA version limited?

Is the only CUDA version 10.2 available?
I want to use 11.1, is it possible?

If you can build the native yolo library (https://github.com/AlexeyAB/darknet) then you can repace the
src\Alturos.Yolo\dependencies\win\yolo_cpp_dll_cpu.dll
and
src\Alturos.Yolo\dependencies\win\yolo_cpp_dll_gpu.dll
files and it should work with newer CUDA.
I tried it, but the build was failing in my machine. I did not spent too much time on it.

If you can build the native yolo library (https://github.com/AlexeyAB/darknet) then you can repace the
src\Alturos.Yolo\dependencies\win\yolo_cpp_dll_cpu.dll
and
src\Alturos.Yolo\dependencies\win\yolo_cpp_dll_gpu.dll
files and it should work with newer CUDA.
I tried it, but the build was failing in my machine. I did not spent too much time on it.

Thanks for answering
I didn't waste any more time and decided to just use CUDA 10.2ver

Thanks for answering
I didn't waste any more time and decided to just use CUDA 10.2ver


Bug: HealTarget payload IsHero is sometimes -1, sometimes 0

needs documentation
  {
    "time": "2017-10-11T18:19:59+0000",
    "type": "HealTarget",
    "payload": {
      "Team": "Right",
      "Actor": "*JungleMinion_TreeEnt*",
      "TargetActor": "*Blackfeather*",
      "TargetTeam": "Right",
      "Source": "Buff_JungleMinion_TreeEnt_Restore",
      "Heal": 31,
      "Healed": 31,
      "IsHero": 0,
      "TargetIsHero": 1
    }
  },

    "time": "2017-10-11T18:20:01+0000",
    "type": "HealTarget",
    "payload": {
      "Team": "Unknown",
      "Actor": "Unknown",
      "TargetActor": "*Blackfeather*",
      "TargetTeam": "Right",
      "Source": "Buff_JungleMinion_TreeEnt_Restore",
      "Heal": 31,
      "Healed": 31,
      "IsHero": -1,
      "TargetIsHero": 1
    }

https://github.com/gamelocker/vainglory-assets/pull/308

To be more clear: This is INDEED a bug. But it was the reason heal target has been blocked for so long. Work around was to set isHero as -1 for all unknown targets. The best fix to that requires refactor of some code that it won't be happening anytime soon

Why isn't it 0 like in every other event?
Why doesn't the api shim this data? A find/replace is what I have to do in my code now, it should happen on a higher level though

Because the actor is unknown. Other events you have the actor which is clearly not a hero, while on this one you don't know

It's clearly not a hero. If it was a hero, the actor was a hero. A boolean only has two states, and if the source is not a hero, the source is not a hero. 🤔 😄
So any overtime heal has the source unknown / IsHero-1?

It's clear to me that it's from a minion — as you can see in the source — so IMHO this should be patched at MadGlory's level if it is not possible to do so at SEMC's. Because otherwise I have to do it, and it's not an API client's responsibility to parse data source bugs.

on SEMC side, isHero is not a boolean. The name is maybe misleading. Its an integer


Zendesk pipeline docs

Please review the documentation on running the Zendesk pipeline
@TyDunn and @adrianbr .
Have a good day ahead.
BR
Aman

@adrianbr has requested some major changes in these docs, and he has asked to convert this doc into three separate docs. So please don't merge it yet. @adrianbr and @TyDunn. Thankx


clone does not change con and tap properties

When cloning a vm, all properties except name are cloned directly.
The con and tap properties should be reinitialized with new values, like when iohyve create is used, to avoid duplicate console and tap interfaces.

That's totally a feature not a bug.
Just kidding. That's actually how I designed the clone function on purpose. At the time, I wanted to go back and take a second look on how iohyve handled command line input but I REALLY wanted cloning. I left "using a command line option to make an exact clone or a replicated clone" until I figured something out. In the mean time, I only change the name and description.
You do make a good point, and iohyve has matured ever so slightly to the point where I should really be giving this option. I'll leave this open as an enhancement request or until I put it in the raodmap.
Thanks for you thoughts on the project!

I only read the docs carefully enough to see that it is intended after creating this bug.
My use case is to create a few template vm's and spin up a bunch of clones and then destroy them, rinse and repeat.
I might just have a look at implementing the functionality myself as I need it sooner rather than later as I want to stop using linux/kvm :)
Any thoughts on whether it should be an option to clone, the default for clone or have a new name?

Feel free! I think we should stick it in the same clone function, but iohyve clone -c(opy) or iohyve clone -r(eplicate) or something. I just want the ability to do either carbon copies or reinitialized copies, kind of like VirtualBox has.

Ok, i've implemented this now, if -c (or anything other than -r in fact) is
used, it's a carbon copy.
If -r is specified, values are reinitialized.
Also updated man and text file with description, along with iohyve help
text.
Attached is a patch created with git diff -p > patch.
On Fri, 15 Jan 2016 at 17:26 pr1ntf<EMAIL_ADDRESS>wrote:

Feel free! I think we should stick it in the same clone function, but iohyve
clone -c(opy) or iohyve clone -r(eplicate) or something. I just want the
ability to do either carbon copies or reinitialized copies, kind of like
VirtualBox has.
—
Reply to this email directly or view it on GitHub
https://github.com/pr1ntf/iohyve/issues/97#issuecomment-172006604.

--
Regards,
Michael

GitHub strips email attachments. :octopus:
You can always submit a Pull Request, though. :space_invader:

Ah ok, now I've learned a little more about git/github :)
On Sat, 16 Jan 2016 at 01:34 pr1ntf<EMAIL_ADDRESS>wrote:

GitHub strips email attachments. [image: :octopus:]
You can always submit a Pull Request
https://github.com/pr1ntf/iohyve/pulls, though. [image: :space_invader:]
—
Reply to this email directly or view it on GitHub
https://github.com/pr1ntf/iohyve/issues/97#issuecomment-172136601.

--
Regards,
Michael

Closing and bringing discussion to #99


B-21667 & B-21487 MAIN

[!IMPORTANT]
Please ensure the upstreams are merged into main first:
B-21374
B-21375
B-21376

This main branch contains commits from all the below integration PRs. I combined B-21667 and B-21487, because the rename of a variable in B-21487 was the cause of some of the issues needing fixed.
B-21667 INT PR
B-21487 INT PR
B-21667 I-13550 INT  PR
B-21667 I-13567 INT PR
B-21667 I-13574 INT PR

Heads up I resolved a lot of these conflicts in the upstream @traskowskycaci , don't worry about resolving them here. Once the upstream is merged the fix will be in the git history

Alternatively you can merge the upstream into your PR now and it'll resolve them like that, I did that just now with https://github.com/transcom/mymove/pull/14342

Alternatively you can merge the upstream B-21374 into your PR now and it'll resolve them like that, I did that just now with #14342

I'll merge in 21374 - just brought it in yesterday to resolve some test fails so shouldn't be too bad! Thanks for the heads up @cameroncaci

upstream to this pr is next, advise merging it in to help resolve the current conflicts


Wiki

Add initial wiki documents

Codecov Report

Merging #363 into scope2 will not change coverage.
The diff coverage is 0%.

@@           Coverage Diff           @@
##           scope2     #363   +/-   ##
=======================================
  Coverage   50.93%   50.93%
=======================================
  Files         118      118
  Lines        8003     8003
=======================================
  Hits         4076     4076
  Misses       3592     3592
  Partials      335      335

Impacted Files
Coverage Δ

poset/apply_genesis.go
57.5% <ø> (ø)
:arrow_up:

poset/event_ordering.go
100% <ø> (ø)
:arrow_up:

poset/poset.go
77.61% <ø> (ø)
:arrow_up:

gossip/ethapi_backend.go
0% <ø> (ø)
:arrow_up:

inter/block.go
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f0c630...c571751. Read the comment docs.

@devintegral @devintegral2 @devintegral3 So what's your intention with the actual wiki then? - https://github.com/Fantom-foundation/go-lachesis/wiki


feat(server): middleware params

closes https://github.com/denoland/fresh/issues/903
supersedes https://github.com/denoland/fresh/pull/1025
I drew inspiration from the deficiency in the comment of 1025. Sylc says:

For a middleware to access the params that are downstream from its level would require, I think, to duplicate in fresh the logic that is currently in the Rutt router.

Fortunately some problems get solved by just sitting around. The router was inlined into Fresh in April via https://github.com/denoland/fresh/pull/1122. So now, as my test confirms, middleware at all layers have access to all the params.
I did this by refactoring the router into two steps:

compute the params and route, based on the request
do the actual routing

If you look at the existing router you can see these two parts are there, but they're all lumped together in one method. Everything for the computation of groups (i.e. params) is done as we're looping over processedRoutes. Presumably this is for speed, so we don't have to loop twice.
What I've done is pull this code out to compute the params and route ahead of time. Then we can set up the MiddlewareHandlerContext with the params. Later on, we can reuse the pre-computed params and route when invoking the FinalHandler like so:
inner(
  req,
  ctx,
  paramsAndRouteResult.params,
  paramsAndRouteResult.route,
)

Obviously some of the naming here is horrible and I'm happy to receive suggestions. Problems: paramsAndRouteResult and getParamsAndRoute.

I see Luca has a handler_2 branch, so maybe this can get closed in lieu of that. Or I can rework this to build on that new version, once it's merged.


Revert to imagemagick

Let's try this temporarily to see if memory use by ImageMagick is still a concern.  I would love to remove our dependence on an in-house solution to resizing images.  Especially given it has a few known limitations concerning treatment of the EXIF header.

Coverage remained the same at 93.029% when pulling 203b8581d2a1279d82a5b2e6d9be5d48a0da1573 on pellaea:revert-to-imagemagick into 572526a5c9c9db6bea92dae397e827ed90f23ce1 on MushroomObserver:master.


more bit？？？？

how to embed more bits into the audio?

Hi, could you elaborate ? Did you mean the bits of the secret messages ?
You can modify the existing model card to have for example 32 instead of 16 bit-message in here. For example:

model = AudioSeal.load_generator("audioseal_wm_16bits", nbits=32)
...

Note that this might not always be optimal as the model is trained with different pertubed / augmented training data while embedding 16-bit messages. So other option is to train a new model (see our  instruction notes) , where you can specify the number of bits during the training. We provide an example recipe here, where this line is where the number of bits used for the training is specified

@DanMerry I closed the issue. Feel free to re-open it if you still face the problem

@antoine-tran using the approach you wrote is not possible as the weights are for 16 bits and changing the number of bits should also change the MsgProcessor.
Is there a way using dora framework to fine-tune it to more bits? for example:

Loading the weights of the 16 bit network and changing the msg processor + last layer to 32 bits
Freezing all layers except those two and fine-tune only them

Or there isn't any "easy" way to do it with the training code given in the repo?


Followup #278 Assert that quarkus-extension.json exists for each exte…

…nsion

FWIW the file is auto generated by the quarkus plugin in any case

FWIW the file is auto generated by the quarkus plugin in any case

My understanding is that it is generated in ${project.build.outputDirectory} unless it exists in src/main/resources. But we definitely want a custom file in src/main/resources because of the labels and name. So we should check that we have it, should we not?

IMHO, the validation is meaningful if it checks that the content has some minimal info, checking only the existence of the file does not add real value

Also all these build checks are a bit painful when developing... Any chance to move them to a Profile ? Similar to what is done in Camel with the sourcecheck profile?

IMHO, the validation is meaningful if it checks that the content has some minimal info, checking only the existence of the file does not add real value

Added some checks in 7203021

Also all these build checks are a bit painful when developing... Any chance to move them to a Profile ? Similar to what is done in Camel with the sourcecheck profile?

I moved it to validate phase so that these issues pop up at the very end. I do not think this should be optional because failing to deliver a valid metadata is a real functional problem.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/camel-quarkus-pr/278/Build result: FAILURE[...truncated 1.48 MB...]    at java.lang.reflect.Method.invoke (Method.java:498)    at hudson.maven.Maven3Builder.call (Maven3Builder.java:139)    at hudson.maven.Maven3Builder.call (Maven3Builder.java:70)    at hudson.remoting.UserRequest.perform (UserRequest.java:212)    at hudson.remoting.UserRequest.perform (UserRequest.java:54)    at hudson.remoting.Request$2.run (Request.java:369)    at hudson.remoting.InterceptingExecutorService$1.call (InterceptingExecutorService.java:72)    at java.util.concurrent.FutureTask.run (FutureTask.java:266)    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)    at java.lang.Thread.run (Thread.java:748)[ERROR] [ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR]   mvn  -rf :camel-quarkus-integration-test-beanchannel stoppedAdding one-line test results to commit status...Setting status of 03ac64f1e5db4a0f7cb4712dd4326647bbed08ee to FAILURE with url https://builds.apache.org/job/camel-quarkus-pr/278/ and message: 'FAILURE 76 tests run, 2 skipped, 0 failed.'Using context: default

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/camel-quarkus-pr/279/Build result: FAILURE[...truncated 1.48 MB...]    at java.lang.reflect.Method.invoke (Method.java:498)    at hudson.maven.Maven3Builder.call (Maven3Builder.java:139)    at hudson.maven.Maven3Builder.call (Maven3Builder.java:70)    at hudson.remoting.UserRequest.perform (UserRequest.java:212)    at hudson.remoting.UserRequest.perform (UserRequest.java:54)    at hudson.remoting.Request$2.run (Request.java:369)    at hudson.remoting.InterceptingExecutorService$1.call (InterceptingExecutorService.java:72)    at java.util.concurrent.FutureTask.run (FutureTask.java:266)    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)    at java.lang.Thread.run (Thread.java:748)[ERROR] [ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR]   mvn  -rf :camel-quarkus-integration-test-beanchannel stoppedAdding one-line test results to commit status...Setting status of 7203021fb02e0c0b2469b20f07b8952ff5102cc2 to FAILURE with url https://builds.apache.org/job/camel-quarkus-pr/279/ and message: 'FAILURE 76 tests run, 2 skipped, 0 failed.'Using context: default

@ppalaga can you rebase?

2ce6dc7 rebased

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/camel-quarkus-pr/284/

0557eee updated based on the discussion with @lburgazzoli

0557eee Added labels to XML and HTML common

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/camel-quarkus-pr/285/

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/camel-quarkus-pr/286/


If you use a Group or Light in more than one flow -> Connection Error

Hello,
great Nodes. One Problem i have switching to huemagic:
If i a Node of one Lamp or Group in more than one flow, i got a connection error.
I would be nice if the Node would output the Lamp/Group config (as name, Lighttype ...)
not only the id.

Hi,
thanks for your feedback! Unfortunately I could not reproduce a connection error when using the same light / group additionally on other flows.

Do you get some more details other than that on your debug tab?
Do you get a connection error on Node-RED or (under) the group / light?
Are you using the most recent version of Node-RED (which is currently 0.17.5)? If not, please upgrade and try again.

Regarding your other feature request of a node with more details on devices – I am already working on that and will release the update soon ;)


ack hangs on an unfinished "|"

Context: searching python code for .split('|'), so I used ack like
$ /usr/local/bin/ack --python 'split..|'

This resulted in ack showing me the name of the first matching file and then hanging. Quickly realised the silly mistake and retried with the | escaped
$ /usr/local/bin/ack --python 'split..\|'

which worked fine. So not really an issue for me, just thought you'd like to know.
For completeness, I tried the following, which found nothing (because I don't have any perl files in that dir), and came back without hanging.
$ /usr/local/bin/ack --perl 'split..|'

Adding a single .pl file, even an empty one, will then cause that command to hang

I'm guessing that the RHS of the | is matching the empty string, so the matching engine doesn't advance.  This looks to be happening with .? as well.

This appears to have been fixed in 80402b4cca94834795210fc6e25261f2d57eb049

This went out in 2.16.

Works for me at verson 2.16


linter: Print debugging message if match fails

This commit makes split_match print a debugging message if the match completely fails, meaning that no groups have been captured.
I ran into this problem while writing a linter plugin for SublimeLinter3 - I had a very trivial (and hard to pick out) mistake in my regular expression, and the lack of a debugging message telling me about a missing match made it difficult to pinpoint the failure.

Thanks and sorry for the delay

Not a problem, thanks for merging!


Cannot turn off oslogin from project module when org policy is set

Since we removed this block and the corresponding variable from the project module, setting oslogin to false via metadata fails when the relevant policy is enabled
resource "google_compute_project_metadata_item" "oslogin_meta" {
  count   = var.oslogin ? 1 : 0
  project = local.project.project_id
  key     = "enable-oslogin"
  value   = "TRUE"
  # depend on services or it will fail on destroy
  depends_on = [google_project_service.project_services]
}

In FAST we should implement the policy with a tag-based condition, so as to do this via resource management tags, In the project module, I see two ways of doing this

bring back the oslogin specific variable and reintroduce the above code block
detect the tag key in metadata and use the metadata item resource when that is present

I prefer the second to keep variable space small, but I hate the fact it adds some magic so let's discuss.

I'm not sure If I understand correctly the goal and second idea here.
Is the goal here, to control the OSLogin only through the policy and do not bother user with compute metadata? Or do not raise errors, when compute_metadata is not matching the policy?
Is one of the solutions to check tag_bindings if it contains a tag attached to condition in org policy for os-login enforcement and then adjust compute_metadata / google_compute_project_metadata_item ?

So, many data services need oslogin off. If it's enabled via the org policy, you cannot disable it via our current project module.
Doing an exception via org policy condition and resource management tags is a clean way of toggling the behaviour, easily controlled via IAM (tag roles on tag value and folder/project) so this we should do, similarly to what we are already doing for DRS.
But allowing the same to be done at the project level via metadata is also needed, as we cannot expect every user to be in an enterprise environment, and we also need to support blueprints where we don't know or care about the org.
In this second case, we have two choices:

bring back the variable and google_compute_project_metadata_item resource, or
infer the use of the resource from the metadata variable (is the key there? use the correct resource and remove the key)

I slightly prefer the second project-level approach. The org policy one we will implement anyway for FAST.

Ok, so the goal is - let allow "non-enterprise users", who do not control org policy by tags, to create a project, where they can create VMs without oslogin, although they have constraints/compute.requireOsLogin enabled on org level?
To make it work at the project level, currently what we need is:
module "project" {
...
  org_policies = {
    "constraints/compute.requireOsLogin" = {
      rules = [{ enforce = false }]
  }
}

Setting the google_compute_project_metadata_item will fail, if org policy doesn't allow for that. If there is no policy, I think that data services will just place metadata on the instances accordingly, so we are good to go?
And IIUC, what was once:
module "project" {
  ...
  oslogin = true
}

Is know represented:
module "project" {
  ...
  compute_metadata = {
    (enable-oslogin) = true
  }
}

?
So totally agree, that we need a tag in FAST, but I'm not sure if there is a need to add magic inside project module, as all toggles are available.

We cannot assume the user is able to contro org policies at the project level, as that role lives at the org level.
So: org policies via tag, project via metadata. It used to work before the last change to project module via item.
Wiktor, we might want to have a chat instead of going in circles. :)

Closing this, I went on a wild chase but the resourse we are using now is exactly the same. @lcaggio feel free to ping me with the actual use case.


Is there support for 2 channel and 4 channel training

In Yolov8 we can train 2 channel images or 4 channel images by adding ch:2 or ch:4 parameter to yolov8s.yaml file . Does Yolo-NAS also have similar support if so where should we change it

Hi @ajithkumarmcw .
It seems like we have a bug in not passing the in_channel parameter on instantiation.
a ticket was opened, and this will be fixed in the coming patch version.
you can work around it by changing the default value in the code of
super_gradients.training.models.detection_models.yolo_nas.yolo_nas_variants.YoloNAS_S
or
super_gradients.training.models.detection_models.yolo_nas.yolo_nas_variants.YoloNAS_M
or
super_gradients.training.models.detection_models.yolo_nas.yolo_nas_variants.YoloNAS_L
simply change in_channels: int = 3 to in_channels: int = 3
please note that changing the input channels will prevent you from using the pre-trained weights, of course.
Hope that helps.

thanks for adding support


No module named 'pytorch_radon'？

When I run the ellipses, can not find the package.

pytorch_radon is a separate package that is a requirement for the ellipses experiment. It has to be manually installed in addition to cloning our repository. You can get it from here: https://github.com/phernst/pytorch_radon

OK, Thank you very much!


Possible impact on composer funding

If I understand this correctly, offering a similar service to private packagist for free will likely reduce the revenue of private packagist. By doing this, the fundings available to the composer project will go down. How does having this service free help support composer and packagist?

Thank you for pointing that out. We created Repman as a solution to our internal needs and decided to release it as open-source so that other developers can use it, too.
Composer is the cornerstone for every PHP and WordPress developer. We know it, we use it, and we love it. We already have a couple of ideas for optimizing Composer in CI/CD, too, with the team ready to actively participate in the development process. Look out for new pull requests in the upcoming weeks.
If you have any ideas developing new or improving existing features for Composer, make sure to link to the issues in this project. We’ll be happy to provide any assistance that we can.

Hi @akondas
Glad to hear that you guys plan to contribute to the project. I'm a PHP developer myself and also relay on that project and I hope they can continue to get funding and improve the project. I am sure it's not your intention to harm the project in any way and I hope that this project doesn't have a negative impact on Private Packagist and the Composer project in general.
Could be worth reaching out to the Packagist team to discuss with them the impact that this might have on them if you wanted to do that 👍


Support RDF export

#16

Would be useful for BARTOC and to better support resuse of vocabularies. See jskos-cli for impementation: https://github.com/gbv/jskos-cli/blob/master/lib/rdf-serializer.js this could be moved to jskos-tools or a new module jskos-rdf

Support for NTriples should be enough, this can be implemented easily as done here in BARTOC. Support only for /data endpoint may be easier (add query parameter format with allowed values jskos (default) and nt (N-Triples).

Support for NTriples should be enough, this can be implemented easily as done here in BARTOC. Support only for /data endpoint may be easier (add query parameter format with allowed values jskos (default) and nt (N-Triples).

What's the difference between application/n-triples and application/n-quads?

What's the difference between application/n-triples and application/n-quads?

application/n-quads is an extension of  application/n-triples used for RDF named graphs. jsonld library also supports the latter and expects this string as format but for our data it only emits N-Triples. Better don't ask.  A method to download plain RDF triples (application/n-triples) is enough to support RDF export.

application/n-quads is an extension of  application/n-triples used for RDF named graphs. jsonld library also supports the latter and expects this string as format but for our data it only emits N-Triples. Better don't ask.  A method to download plain RDF triples (application/n-triples) is enough to support RDF export.

JSKOS to RDF via context document has been implemented in BARTOC: https://github.com/gbv/bartoc.org/blob/dev/src/rdf.js Just copy the code or move it to a reusable library?

Implementing https://github.com/gbv/jskos-proxy instead, parts of it may later be merged into jskos-server.


[Docker] Install oneflow from PyPi

Installing oneflow from the current link (https://release.oneflow.info) seems to be broken as reported in #15754, which is impacting other unrelated changes in CI. This commit attempts to fix the install by using a version from PyPi.

cc @ashutosh-arm @leandron @neildhickey


Version requirements

First round of changes.
We need to discuss the default version for Pg for example
proposed recommended versions:
psql 14, Java 11, Python 3.9, nginx 1.24

@pwalczysko suggested change included


Noisy error message when creating a new file with vim (nvim.coc + coc-ansible)

Summary
(Lazy copy+paste from ansible-lint since it's the third issue report about this :slightly_smiling_face: )
When opening an non-existing ansible file with vim, ansible-lint is triggered on the filename (should probably be the buffer contents instead of the filename but that's not up to ansible-lint at all) and ansible-lint spews a noisy exception.
See yaegassy/coc-ansible#26 for more details.
I think the question is: should ansible-lint be less verbose (again?) about a non-existent file or should Ansible languageserver be smarter with handling non-existent files? (Doing both is also an option :shrug: )
Extension version
0.10.1
VS Code version
1.71 (don't know)
Ansible Version
ansible [core 2.13.3]
  config file = /home/twouters/.ansible.cfg
  configured module search path = ['/home/twouters/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.10/site-packages/ansible
  ansible collection location = /home/twouters/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.10.6 (main, Aug  3 2022, 17:39:45) [GCC 12.1.1 20220730]
  jinja version = 3.1.2
  libyaml = True

OS / Environment
Arch Linux
ansible-lint 6.5.2 using ansible 2.13.3
Relevant log output
Please see yaegassy/coc-ansible#26

Here is a supplemental explanation of this issue.
In Vim/Neovim, when you add a new file, you first create what is called a buffer. Only when you save the buffer is the actual file created.
This results in a situation where ansible-lint is executed on a file that does not actually exist, resulting in an error.

@twouters, @yaegassy, could you please check if the error still exists? If not, feel free to close it.

@priyamsahoo This problem seems to be resolved by upgrading ansible-lint.


[BUG] com.azure.core.exception.ResourceNotFoundException is thrown during ServiceBusAdministrationClient#getQueueExists execution

Describe the bug
The com.azure.messaging.servicebus.administration.ServiceBusAdministrationClient#getQueueExists method is throwing ResourceNotFoundException when the queue doesn't exist instead of returning false.
Exception or Stack Trace
at com.azure.messaging.servicebus.administration.ServiceBusAdministrationClient.getQueueWithResponse(ServiceBusAdministrationClient.java:682) ~[azure-messaging-servicebus-7.12.1.jar!/:7.12.1]\n
at com.azure.messaging.servicebus.administration.ServiceBusAdministrationClient.getQueueExistsWithResponse(ServiceBusAdministrationClient.java:721) ~[azure-messaging-servicebus-7.12.1.jar!/:7.12.1]\n
at com.azure.messaging.servicebus.administration.ServiceBusAdministrationClient.getQueueExists(ServiceBusAdministrationClient.java:703) ~[azure-messaging-servicebus-7.12.1.jar!/:7.12.1]\n
To Reproduce
Invoke ServiceBusAdministrationClient#getQueueExists with the non-existing queue
Expected behavior
False is returned
Setup (please complete the following information):
Affects 7.12.1 version

same as https://github.com/Azure/azure-sdk-for-java/issues/32220

@ki1729 could you take a look at this

@avrilfanomar Thank you for reporting this! This has been fixed in the latest release v7.13.0. Here is th PR that fixed this problem : https://github.com/Azure/azure-sdk-for-java/pull/31804


Fix CurvedAnimation memory leaks

I love the loaders provided by this library! However, during performance-profiling my project I discovered that almost every animation in this library has a memory leak caused by creating CurvedAnimation instances every build (they register a listener on the parent AnimationController, preventing it from being GCed until the parent AnimationController is disposed).
The memory leak isn't a big deal when the whole animation is disposed quickly, however in real life it's not that hard to leave the animation long-running (e.g. in the background). At 60FPS this can quickly increase the application's memory usage (in my - pretty extreme - case it amounted to additional 400MB of just CurvedAnimations after leaving the app open for some time).
This PR fixes all of these leaks by replacing CurvedAnimations by just a direct evaluation of specific Tweens and Intervals.

Thanks, I hope you are right about the memory leak, since I don't write flutter anymore, I can't profile it. And sorry I was very busy.

Thanks for the approval! Will you publish an updated version to pub.dev so users can upgrade?

It's updated to pub.dev now try v1.3.0 or above


[GLUTEN-4125][VL] add scan preload splits metric

What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
Fixes: #4125
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

cc @zhouyuan  and @FelixYBW , thanks.


Remove assertIsPromise() from PlainResult/tryCatchAsync

We close this once by https://github.com/option-t/option-t/issues/1608#issuecomment-1456023611.
We'll reopen this when we conclude about #1608.
If you have some opinions, please comment to #1608


Update to 2.0.15

Closes https://github.com/conda-forge/uwsgi-feedstock/issues/3
I also updated the pinning, version and the use of source_files.
ping @ericmjl

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


Add reference interpreter probe instrumentation

Reopening #1763 due to a branch corruption issue. Original text below:
===
As the number of ML frameworks continues to rapidly grow, it’s becoming increasingly desirable to be able to compare numerical results across different runtimes. This ensures that ML models run equivalently in different environments, and overall helps to instill confidence in ML frameworks. For example, given the recency of the StableHLO representation, it may be desirable to have a way to compare StableHLO interpreter results to other frameworks, such as Tensorflow, or other implementations
This PR proposes the addition of a probe instruction to the new interpreter dialect in StableHLO. The probe instruction will enable us to extract intermediate tensor state from the StableHLO interpreter. Since this new op is intended to initially support debuggability, it will not be added to the StableHLO opset itself, but instead to the new interpreter dialect.
The probe instruction will take precisely 1 shaped tensor input, and produce the same shaped tensor output, without modifying it:
%1 = "interpreter.probe"(%0) {probe_id="probe0"} : (tensor<*xf32) -> tensor<*xf32>

The op will be modeled as having side effects, to prevent it from being moved or removed by the compiler. It will additionally contain a singular, unique string attribute, probe_id, which will allow us to later pair this tensor’s data with other runtimes. At compile time, the probe instruction may be inserted after every StableHLO operation (except constants), allowing runtime data to be serialized to disk.
This PR also adds a new StableHLO interpreter configuration flag, --probe_readout_dir, which will be used to specify the directory in which to store all probed interpreter values. When executed, the probe instruction will associate the contents of its input tensor with the given probe_id, and serialize the tensor data to disk using the portable NumPy format. The filename used will be in the form of probe_readout_dir/probe_id.npy. As the interpreter executes interpreter.probe operations, in addition to serializing tensor data to NumPy files, it will also append serialized filepaths to a metadata index.csv file in the form of:
probe_id0,/probe_readout_dir/probe_id0.npy
probe_id1,/probe_readout_dir/probe_id1.npy
…
probe_id0,/probe_readout_dir/probe_id0_1.npy

The metadata file can then be used by post processing tools to analyze the instrumented tensor data. Note that a probe_id can appear more than once in the metadata file (i.e. if a probe is executed within a loop).
Validation
To validate the new instruction, we propose extending the existing stablehlo::check MLIR dialect with a expect_serialized_eq operation, which can be utilized in the following way:
// Where `T` is a statically shaped tensor
%0 = …
%1 = interpreter.probe %0 { probe_id = "probe0" } : (tensor<T>) -> tensor<T>
check.expect_serialized_eq %3 { probe_id = "probe0", iteration = 0 : i32 } : tensor<T>

The check.expect_serialized_eq is expected to appear after the producing interpreter.probe instruction with the same probe_id. When executed, the instruction will search for an existing index.csv metadata file in the same directory as specified by the --probe_readout_dir flag, and then attempt to find the serialized filepath for the given probe_id and iteration. After which, the interpreter will load the serialized tensor from the NumPy file, and then proceed to compare it with its operand tensor. The operation will assert on an answer mismatch or failure to find/parse the associated NumPy file. The iteration attribute is 0-based and is optional and will default to 0.

Adding link to OpenXLA Discuss RFC:
https://groups.google.com/a/openxla.org/g/openxla-discuss/c/Y1875kCT3Hc

Changes merged! Thanks for the contribution!


feat: additional features for mime types

adds extension based MIME type detection and default MIME types for justyn so he doesn't have to use content sniffing

ur tests are a meme and using goto is kinda sus but LGTM 👍


Various UI improvements

(I'll open this as a draft while I make it so I can compare easier)

Typescript doesn't raise warnings for absolute imports anymore! You can stop using '../../../..../../../../../../' monsters now :D

Also this PR makes sure that warning text is displayed both in the top and bottom of the page when it's applicable

I mean, I didn't even touch anything related to models :smiling_face_with_tear:

My guess is that it's the same problem that I've had with worker list, in which case everything should run fine on your side


add staking release function

set_release_owner_address: Sets the admin address for daily staking release operations, requires root action
set_release_limit_parameter: Sets the maximum amount of releases per release and per day, requires root action
staking_release: execute staking release, requires admin action

Codecov Report

Merging #220 (4243778) into master (99903e6) will increase coverage by 0.18%.
The diff coverage is 84.75%.

@@            Coverage Diff             @@
##           master     #220      +/-   ##
==========================================
+ Coverage   69.65%   69.83%   +0.18%
==========================================
  Files          53       53
  Lines       14390    14552     +162
==========================================
+ Hits        10023    10163     +140
- Misses       4367     4389      +22

Impacted Files
Coverage Δ

pallets/operation/src/weights.rs
0.00% <ø> (ø)

runtime/src/lib.rs
11.49% <ø> (ø)

pallets/operation/src/lib.rs
43.01% <65.75%> (+17.09%)
:arrow_up:

pallets/operation/src/tests.rs
92.80% <100.00%> (+19.27%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 99903e6...4243778. Read the comment docs.


Merge gz-rendering7 ➡  main

➡  Forward port
Port gz-rendering7  ➡  main
Branch comparison: https://github.com/gazebosim/gz-rendering/compare/main...gz-rendering7
Note to maintainers: Remember to Merge with commit (not squash-merge or rebase)

@iche033 GZ_ENGINE_TO_TEST=ogre GZ_ENGINE_BACKEND=gl3plus bin/INTEGRATION_gpu_rays  fails on Jammy locally. Is that a known issue?

@iche033 GZ_ENGINE_TO_TEST=ogre GZ_ENGINE_BACKEND=gl3plus bin/INTEGRATION_gpu_rays  fails on Jammy locally. Is that a known issue?

hmm no I don't this one. I know the same test was failing in ogre2 on Jammy (but not locally) but didn't know about ogre1.x. I can reproduce it. We'll need to investigate


error[internal]: left behind trailing whitespace

I never had that before, but rustfmt fails for me with an error error[internal]: left behind trailing whitespace for this (I tried to break it down to the minimal example code below):
error[internal]: left behind trailing whitespace
  --> \\?\C:\Users\dailyuse\Desktop\repro-rustfmt\repro\src\lib.rs:16:16:1
   |
16 |
   | ^^^^^^^^
   |

warning: rustfmt has failed to format. See previous 1 errors.

A repository with the code can be found here: https://github.com/simonsan/rustfmt_repro
use std::result::Result;

type TestError<T> = Result<T, Box<dyn std::error::Error>>;

pub fn test() -> TestError<Vec<usize>> {
    Ok(vec![1, 2, 3])
}

fn end_all_unfinished_activities(activities: Vec<u64>) -> TestError<Vec<usize>> {
    activities
            .iter()
            .for_each(|_| {
                match "Ok(true)" {
                    "bla" => {

                    },
                    "bli" => {
                        log::warn!(
                            "Activity {} activity ends before it began. That's impossible. Skipping activity.", "test".to_string()
                        )
                    },
                    _ => {}
                };
            });

    Ok(vec![1, 2, 3])
}

In the match arm "bla", between the brackets are some whitespaces. If you remove the log::warn!() it works again, so it's definitely related. I wonder though, if this could be considered a bug in rustfmt?
Related: #6054 (?)

Ah, interesting. I have not experienced that behaviour before, so I was surprised. Thanks for explaining the workarounds, much appreciated!


Firecracker Robot Tests

In order to run firecracker tests in GitHub actions we need a publicly available and stable way to install firecracker on the runner.

I suggest to use a custom container which includes firecracker. Such a container can be found here

https://build.opensuse.org/package/show/home:marcus.schaefer:delta_containers/test-os

I think we can reference this container in a github action like this
jobs:
  test:
    runs-on: ubuntu-latest
    container: registry.opensuse.org/home/marcus.schaefer/delta_containers/images_15.4/flakeos:latest
    steps:
        ...

I haven't tested this but it should pull the container and allows to run the workflow. Missing components, e.g the robot framework can either be added inside of the container or installed by the action inside of the container instance. For performance reasons it might be better to add the framework into the container imho
But there is probably another issue.
Running firecracker requires KVM, meaning the runner must expose /dev/kvm and I doubt that this is done by the github runners. If we cannot have a runner with KVM support we need to host our own runner. This would require some infrastructure to maintain.
Thus this issue can cause some headaches

In addition it might require the containers called on the runners to be privileged such that access to /dev/kvm is granted. So there are some unknowns which needs to be tested at best outside of github to know the pre-conditions better

One option to maintain our own runner would be a metal instance in AWS or GCE supporting nested virtualization. Keeping such an instance open 24x7 will produce costs though. This is the reason why I don't have an always running instance of this type in my AWS account. I think at EB you could ask for this to allow the tests to run. Not sure if the small project here is enough justification. Maybe a topic for Bo to clarify


Write Part - 4 Step - 6 & unit test

Implement Step06Controller and write all unit tests. Verify the display name of the test conforms to the pattern of "Part X Step Y". Follow the standardize naming, comments, etc. to be consistent.

Similar to Part 01 Step 03 (no ticket for it)


Point to java from JRE version which gets defined at the package build time

Connected to #416

The copy and replacement of a variable in a file can be done much more efficiently, portably, and succinctly by using the resources plugin rather than cp/sed through the exec plugin.  You should only need to move the file to the resources area, add the file to the list of filtered files, and change the value to replace with ${your variable} in the file.

Thanks!  Never used it :-/  Trying now.

@loomis Just pushed the changes with using resources plugin (very convenient! :-)).  Please check.

Looks OK to me.  Files are properly changed and included in the packages.  Merging...


cURL can't load Paper

When I use multiarch-latest, I get this error:
ERROR: unknown error while looking up PaperMC version=1.16.4
curl: (28) Timeout was reached
[init] Resolving type given Paper
[init] Resolved version given 1.16.4 into 1.16.4
Even if you use PAPER_DOWNLOAD_URL, there will still be an error.
But if I use multiarch (not latest) then everything works.
docker-compose.yml
version: '2'
services:
  mc:
    image: itzg/minecraft-server:multiarch-latest
    ports:
        - "25565:25565"
    restart: always
    environment:
        TZ: "Europe/Moscow"
        EULA: "True"
        TYPE: "Paper"
        MEMORY: "2G"
        FORCE_REDOWNLOAD: "True"
        VERSION: "1.16.4"
        ONLINE_MODE: "False"
        CONSOLE: "False"
        DISABLE_HEALTHCHECK: "True"
    volumes:
      - mc-paper:/data

volumes:
  mc-paper:
    driver: local

I am not able to recreate the issue using the compose file you provided. I wonder if it was just a temporary networking issue that just happened to coincide with using the multiarch-latest variant; however, it sounds like it was fairly repeatable for you. Perhaps docker-compose pull to ensure you have the latest image and give it another try.

I am not able to recreate the issue using the compose file you provided. I wonder if it was just a temporary networking issue that just happened to coincide with using the multiarch-latest variant; however, it sounds like it was fairly repeatable for you. Perhaps docker-compose pull to ensure you have the latest image and give it another try.

I have updated all the images and may run into the problem again. Maybe I'm doing something wrong? For example:
multiarch-latest:
pi@mercury:~ $ docker run -e EULA=TRUE -e TYPE=PAPER -e VERSION=1.16.4 itzg/minecraft-server:multiarch-latest
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 2 1000 1000 4096 Jan  9  2021 /data'
[init] Resolved version given 1.16.4 into 1.16.4
[init] Resolving type given PAPER
curl: (28) Resolving timed out after<PHONE_NUMBER>961 milliseconds
ERROR: unknown error while looking up PaperMC version=1.16.4

As I see it, everything goes well in this case.
multiarch:
pi@mercury:~ $ docker run -e EULA=TRUE -e TYPE=PAPER -e VERSION=1.16.4 itzg/minecraft-server:multiarch
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 2 1000 1000 4096 Jan  9 18:32 /data'
[init] Resolved version given 1.16.4 into 1.16.4
[init] Resolving type given PAPER
[init] Removing old PaperMC versions ...
[init] Downloading PaperMC 1.16.4 (build 416) ...
[init] Creating server.properties in /data/server.properties
[init] Setting server-name to 'Dedicated Server' in /data/server.properties
[init] Skip setting server-ip
....

I have updated all the images and may run into the problem again. Maybe I'm doing something wrong? For example:
multiarch-latest:
pi@mercury:~ $ docker run -e EULA=TRUE -e TYPE=PAPER -e VERSION=1.16.4 itzg/minecraft-server:multiarch-latest
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 2 1000 1000 4096 Jan  9  2021 /data'
[init] Resolved version given 1.16.4 into 1.16.4
[init] Resolving type given PAPER
curl: (28) Resolving timed out after<PHONE_NUMBER>961 milliseconds
ERROR: unknown error while looking up PaperMC version=1.16.4

As I see it, everything goes well in this case.
multiarch:
pi@mercury:~ $ docker run -e EULA=TRUE -e TYPE=PAPER -e VERSION=1.16.4 itzg/minecraft-server:multiarch
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 2 1000 1000 4096 Jan  9 18:32 /data'
[init] Resolved version given 1.16.4 into 1.16.4
[init] Resolving type given PAPER
[init] Removing old PaperMC versions ...
[init] Downloading PaperMC 1.16.4 (build 416) ...
[init] Creating server.properties in /data/server.properties
[init] Setting server-name to 'Dedicated Server' in /data/server.properties
[init] Skip setting server-ip
....

Yeah, it's definitely consistent for you. The word "resolving" in "curl: (28) Resolving timed out" makes me wonder if it is DNS related; however both tags are ultimately based on Ubuntu, so should both use similar DNS resolving techniques. They are each a different release of Ubuntu though:

multiarch -> based on Ubuntu 18.04
multiarch-latest -> {based on Ubuntu 20.04](https://github.com/AdoptOpenJDK/openjdk-docker/blob/f59c16624216f38cde51772068e2040f659913bb/15/jdk/ubuntu/Dockerfile.hotspot.releases.full#L20)

Since I'm not able to debug myself, you might need to investigate some more yourself. I would recommend entering a shell of the container with
docker run -it --rm --entrypoint bash itzg/minecraft-server:multiarch-latest

and then apt update; apt install dnsutils; dig papermc.io to see if/how that resolves.

Yeah, it's definitely consistent for you. The word "resolving" in "curl: (28) Resolving timed out" makes me wonder if it is DNS related; however both tags are ultimately based on Ubuntu, so should both use similar DNS resolving techniques. They are each a different release of Ubuntu though:

multiarch -> based on Ubuntu 18.04
multiarch-latest -> {based on Ubuntu 20.04](https://github.com/AdoptOpenJDK/openjdk-docker/blob/f59c16624216f38cde51772068e2040f659913bb/15/jdk/ubuntu/Dockerfile.hotspot.releases.full#L20)

Since I'm not able to debug myself, you might need to investigate some more yourself. I would recommend entering a shell of the container with
docker run -it --rm --entrypoint bash itzg/minecraft-server:multiarch-latest

and then apt update; apt install dnsutils; dig papermc.io to see if/how that resolves.

Same issue, not sure what's wrong on first glance.
I was unable to run the DNS diagnostic within the container due to permission issues - not sure how to advance (except using multiarch image

Same issue, not sure what's wrong on first glance.
I was unable to run the DNS diagnostic within the container due to permission issues - not sure how to advance (except using multiarch image