After Update to HomeAssistant 105.5 the webUI is not starting -> serveral py-Errors

The problem:
After upgrading to 105.5 the webUI is not starting.
Environment
Home Assistant 0.105.5
Frontend version: 20200130.3 - latest
Home Assistant release with the issue: 105.5
Last working Home Assistant release (if known): 105.4
Operating environment (Hass.io/Docker/Windows/etc.):
Home assistant on PI3b+
Logs (home assistant log):
hassio homeassistant logs
2020-02-21 15:51:46 ERROR (MainThread) [homeassistant.setup] Error during setup of component default_config
Traceback (most recent call last):
File "/usr/local/lib/python3.7/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
return future.result()
File "/usr/src/homeassistant/homeassistant/main.py", line 249, in setup_and_run_hass
safe_mode=args.safe_mode,
File "/usr/src/homeassistant/homeassistant/bootstrap.py", line 89, in async_setup_hass
await async_from_config_dict(config_dict, hass)
File "/usr/src/homeassistant/homeassistant/bootstrap.py", line 132, in async_from_config_dict
await _async_set_up_integrations(hass, config)
File "/usr/src/homeassistant/homeassistant/bootstrap.py", line 377, in _async_set_up_integrations
*(async_setup_component(hass, domain, config) for domain in domains_to_load)
File "/usr/src/homeassistant/homeassistant/setup.py", line 49, in async_setup_component
return await task  # type: ignore
File "/usr/src/homeassistant/homeassistant/setup.py", line 128, in _async_setup_component
await async_process_deps_reqs(hass, config, integration)
File "/usr/src/homeassistant/homeassistant/setup.py", line 290, in async_process_deps_reqs
hass, integration.domain
File "/usr/src/homeassistant/homeassistant/requirements.py", line 55, in async_get_integration_with_requirements
hass, integration.domain, integration.requirements
File "/usr/src/homeassistant/homeassistant/requirements.py", line 99, in async_process_requirements
if pkg_util.is_installed(req):
File "/usr/src/homeassistant/homeassistant/util/package.py", line 44, in is_installed
return version(req.project_name) in req
File "/usr/local/lib/python3.7/site-packages/pkg_resources/init.py", line 3138, in contains
return self.specifier.contains(item, prereleases=True)
File "/usr/local/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/specifiers.py", line 703, in contains
item = parse(item)
File "/usr/local/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/version.py", line 31, in parse
return Version(version)
File "/usr/local/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/version.py", line 200, in init
match = self._regex.search(version)
TypeError: expected string or bytes-like object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/setup.py", line 170, in _async_setup_component
hass, processed_config
File "/usr/src/homeassistant/homeassistant/components/default_config/init.py", line 17, in async_setup
return await async_setup_component(hass, "stream", config)
File "/usr/src/homeassistant/homeassistant/setup.py", line 49, in async_setup_component
return await task  # type: ignore
concurrent.futures._base.CancelledError
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/src/homeassistant/homeassistant/util/logging.py", line 85, in _process
self._queue.get(), self.loop
File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 433, in result
raise CancelledError()
concurrent.futures._base.CancelledError
Traceback (most recent call last):
File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/src/homeassistant/homeassistant/main.py", line 350, in
sys.exit(main())
File "/usr/src/homeassistant/homeassistant/main.py", line 342, in main
exit_code = asyncio.run(setup_and_run_hass(config_dir, args))
File "/usr/local/lib/python3.7/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
return future.result()
File "/usr/src/homeassistant/homeassistant/main.py", line 249, in setup_and_run_hass
safe_mode=args.safe_mode,
File "/usr/src/homeassistant/homeassistant/bootstrap.py", line 89, in async_setup_hass
await async_from_config_dict(config_dict, hass)
File "/usr/src/homeassistant/homeassistant/bootstrap.py", line 132, in async_from_config_dict
await _async_set_up_integrations(hass, config)
File "/usr/src/homeassistant/homeassistant/bootstrap.py", line 377, in _async_set_up_integrations
*(async_setup_component(hass, domain, config) for domain in domains_to_load)
File "/usr/src/homeassistant/homeassistant/setup.py", line 49, in async_setup_component
return await task  # type: ignore
File "/usr/src/homeassistant/homeassistant/setup.py", line 128, in _async_setup_component
await async_process_deps_reqs(hass, config, integration)
File "/usr/src/homeassistant/homeassistant/setup.py", line 290, in async_process_deps_reqs
hass, integration.domain
File "/usr/src/homeassistant/homeassistant/requirements.py", line 55, in async_get_integration_with_requirements
hass, integration.domain, integration.requirements
File "/usr/src/homeassistant/homeassistant/requirements.py", line 99, in async_process_requirements
if pkg_util.is_installed(req):
File "/usr/src/homeassistant/homeassistant/util/package.py", line 44, in is_installed
return version(req.project_name) in req
File "/usr/local/lib/python3.7/site-packages/pkg_resources/init.py", line 3138, in contains
return self.specifier.contains(item, prereleases=True)
File "/usr/local/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/specifiers.py", line 703, in contains
item = parse(item)
File "/usr/local/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/version.py", line 31, in parse
return Version(version)
File "/usr/local/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/version.py", line 200, in init
match = self._regex.search(version)
TypeError: expected string or bytes-like object

Duplicate of #32062


Adaptive cards don't render the Image dimensions

The IImage interface defined in https://github.com/Microsoft/AdaptiveCards/blob/master/source/nodejs/adaptivecards/src/schema.ts:
inherits the properties of ICardElement
export interface IImage extends ICardElement {
	type: 'Image';
	altText?: string;
	selectAction?: IAction;
	size?: Size;
	style?: ImageStyle;
	url: string;
}

export interface ICardElement {
	id?: string;
	speak?: string;
	horizontalAlignment?: HorizontalAlignment;
	spacing?: Spacing;
	separator?: boolean;
	height?: 'auto' | 'stretch';
	[propName: string]: any;
}

ICardElement has a height property which can be 'auto' or 'stretch'
It does not have a width property
e.g. I take a json which has a column with this Image item:
{
  type: 'Column',
  style: 'default',
  items: [
    {
      type: 'Image',
      horizontalAlignment: 'Right',
      url:
        'http://cdn.onlinewebfonts.com/svg/img_286083.png',
      size: 'Small',
      width: '16px',
      height: '16px',
      altText: '',
    },
  ],
  width: 'auto',
},

The width and height of the Images are not rendered. The image is displayed with the default dimensions.
Can you please help?

Setting the image height and width in an AdaptiveCard seems to working fine in Web Chat. What channel are you using, and what version of AdaptiveCards are you using? Try removing the size attribute from image element.
AdaptiveCard
{
    "type": "AdaptiveCard",
    "body": [
        {
            "type": "ColumnSet",
            "columns": [
                {
                    "type": "Column",
                    "width": "auto",
                    "items": [
                        {
                            "type": "Image",
                            "url": "http://cdn.onlinewebfonts.com/svg/img_286083.png",
                            "width": "50px",
                            "height": "50px",
                            "altText": ""
                        }
                    ]
                }
            ]
        }
    ],
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "version": "1.0"
}

Screenshot

I deleted the size attribute but it still does not work. I used your json also. The version I am using is 1.0 also.

What channel are you using? Can you add screenshots and the code you are using to send the AdaptiveCard?

The image height and width properties were just added to AdaptiveCards in one of their most recent releases - v1.1. Web Chat v4 is currently the only channel that supports the most recent release of AdaptiveCards, so if you try to set those properties in any other channel (Web Chat v3, Test in Web Chat in the Azure Portal, Teams, Skype, Messenger, etc.), they will be ignored by the client's card rendered.  Other channels should hopefully be updated in the near future, but in the meantime, they do not support attributes new to AdaptiveCards.
Closing this issue as AdaptiveCards v1.2 and image height and width are supported in Web Chat v4.


How to use almond with Hadoop 2.8

I'm trying to use almond with Hadoop 2.8.5 (the hadoop version used by recent EMR) and I ran into an error due to incompatible versions of hadoop jars in the classpath.
19/07/23 16:09:47 WARN TaskSetManager: Lost task 1.0 in stage 0.0 (TID 1, ip-10-20-101-239.eu-west-1.compute.internal, executor 2): java.lang.IllegalAccessError: tried to access method org.apache.hadoop.metrics2.lib.MutableCounterLong.<init>(Lorg/apache/hadoop/metrics2/MetricsInfo;J)V from class org.apache.hadoop.fs.s3a.S3AInstrumentation
	at org.apache.hadoop.fs.s3a.S3AInstrumentation.streamCounter(S3AInstrumentation.java:195)

It seems that spark-yarn has transitive dependencies on hadoop 2.6.5
The first idea is to use a profile but unfortunately, there is no hadoop 2.8 profile for Spark.
The second idea is to exclude hadoop jars, it works for the driver but there are still downloaded to the executors.
interp.load.ivy(
  coursier.Dependency(
    module = coursier.Module(coursier.Organization("org.apache.spark"), coursier.ModuleName("spark-yarn_2.11")),
    version = "2.4.3",
    exclusions = Set((coursier.Organization("org.apache.hadoop"), coursier.ModuleName("*")))
  )
)

import $ivy.`sh.almond::almond-spark:0.6.0`
import $ivy.`org.apache.hadoop:hadoop-aws:2.8.5`
import $ivy.`org.apache.hadoop:hadoop-hdfs-client:2.8.5`
import $ivy.`org.apache.hadoop:hadoop-hdfs:2.8.5`
import $ivy.`org.apache.hadoop:hadoop-yarn-api:2.8.5`
import $ivy.`org.apache.hadoop:hadoop-yarn-client:2.8.5`
import $ivy.`org.apache.hadoop:hadoop-mapreduce-client-core:2.8.5`
import $ivy.`org.apache.hadoop:hadoop-yarn-server-web-proxy:2.8.5`
import $ivy.`org.apache.hadoop:hadoop-yarn-common:2.8.5`

The third idea is to have a way to exclude jar from the classpath built in ammonite-spark, but it doesn't seem possible yet.

I guess the resolutionHooks should be passed around here, so that the exclusions can be added to all dependencies that need it via hooks (by calling fetch.dependencies / fetch.withDependencies). So that needs changes in ammonite-spark…

IMO one part of the problem is that ammonite-spark uses vanilla spark version and its spark-yarn is linked with hadoop 2.6.5.
So instead of using transitive version of hadoop, maybe we can let user provide the hadoop/yarn version when creating the spark session.
WDYT?

@darkjh That could be added, yeah.

Good news, it works !
I've built a custom version of almond against ammonite-spark with this PR https://github.com/alexarchambault/ammonite-spark/pull/58.
In the notebook, I've just forced the version of hadoop-client before importing spark-sql
import coursier.core._
interp.resolutionHooks += { fetch =>
   fetch.withResolutionParams(
     fetch.resolutionParams.addForceVersion(
         (Module(Organization("org.apache.hadoop"), ModuleName("hadoop-client"), Map.empty), "2.8.5")
       )
       )
 }

And I'm now able to read a file from s3 with the following config : EMR 5.24.1 (Hadoop 2.8.5), Spark 2.4.3, Scala 2.12.

@YannMoisan would you be able to provide an example of being able to connect to YARN with almond? I've tried a number of things and can't seem to get it to work with jupyter/almond running on an EMR cluster.

@brayellison we are still using our custom version of almond, based on almond 0.6.3. We tried to bump without success but we haven't spent that much time to investigate the error.
import coursier.core._
interp.resolutionHooks += { fetch =>
  fetch.withResolutionParams(
    fetch.resolutionParams.addForceVersion(
      (Module(Organization("org.apache.hadoop"), ModuleName("hadoop-client"), Map.empty), "2.8.5")
    )
  )
}

// This @ is necessary for Ammonite to process the `interp.` before continuing
// cf: https://ammonite.io/#Multi-stageScripts
@

import $ivy.`sh.almond::almond-spark:0.6.3-custom-SNAPSHOT`
import $ivy.`org.apache.hadoop:hadoop-aws:2.8.5`

    NotebookSparkSession.builder()
      .master("yarn")
      .getOrCreate()

I missed this before and came back to it searching for a solution again. Thank you @YannMoisan, I'll give it a shot!


Backward compatibility on session generation broken on 3PAR WSAPI v3

Curl command fails on v3
cho-suse:~ # curl -k -H "Content-type: application/json" -H "Accept: application/json" -X POST -d'{"user": "3paradm", "password":"3pardata"}' https://<IP_ADDRESS>:8080/api/v3/credentials
{"message":"json: cannot unmarshal string into Go struct field LoginRequest.password of type []string","resourceUri":""}
Against v1, api running on the array which supports v3/v1 , the same payload works
cho-suse:~ # curl -k -H "Content-type: application/json" -H "Accept: application/json" -X POST -d'{"user": "3paradm", "password":"3pardata"}' https://<IP_ADDRESS>:8080/api/v1/credentials
{"key":"0-5b3a9c6b96eb33e13268de7a971141df-e77e465b"}
What we have found further is , the password text has to be send in a JSON string array format
cho-suse:~ # curl -k -H "Content-type: application/json" -H "Accept: application/json" -X POST -d'{"user": "3paradm", "password":["3","p","a","r","d","a","t","a"]}' https://<IP_ADDRESS>:8080/api/v3/credentials
{"key":"e1a6dc3c812fde7c34e9e015d60cc76062ef017177b9a5fa"}cho-suse:~ #
cho-suse:~ # curl -k -H "Content-type: application/json" -H "Accept: application/json" -X POST -d'{"user": "3paradm", "password":["3pardata"]}' https://<IP_ADDRESS>:8080/api/v3/credentials
{"key":"527b3234531c01cb5d47fbcc511f046c5baed15c170168b9"}

Left a WSAPI bug for triage bug # 241396


Group bookmarks in widgets

As a user, I want to group Bookmarks in widgets, so that I can summarize them according to different topics.
The widgets should be blocks that have an editable title.
Non-empty widgets cannot be deleted.

almost complete except for #103


Demo 2 Add delete usage example

Demo 2 does not have a use case for deleting widgets; this PR provides that example. The @key directive is used to avoid rendering issues when deleting components.

getCellHeight return type changed from int to double


add alt to img thumbs on home and other aggregation pages

need alts, you have alts on the posts, so probably have the data?
where the thumb image is used as clickable link definitely needs alt--> but could be title of post it links to

Added alts. some a little less descriptive than I'd like but others are good.


Add Player Data with GET Request from Stats Royale

request("https://statsroyale.com/profile/" + id).pipe(fs.createWriteStream('stats.html'));
var wins1 = document.getElementsByClassName('statistics__metricCounter ui__headerExtraSmall statistics__trophyMetric');

i did so and it works


Add linux install script

1. Add linux install  bash script
2. Update readme
This Bash script automates the installation and update process for the "Carton" tool.

Defines the destination directory for the Carton binary.
Checks if the 'swift' command is available, and if not, installs Swift.
Clones the Carton repository from GitHub.
Determines the latest release tag and checks out that release.
Installs necessary dependencies for Carton on an Ubuntu system.
Builds the Carton tool in release mode.
If the build is successful, it updates the Carton binary in the specified destination directory.
Cleans up temporary files and the cloned repository, regardless of success or failure.

Overall, this script ensures that Carton is installed or updated to the latest release on your system, making it ready for use in Swift WebAssembly development.

Because carton is going to be a SwiftPM Plugin, custom CLI installation is no longer needed. Sorry for leaving it for a long time, and thank you for trying to solve the issue :pray:


fix spawn on windows

I get the error on windows: Couldn't start client Idris 2 LSP Client.
I checked the code and found out that it was a problem with the spawn function.
This is a common problem, can refer to this link.
Using shell:true can solve this problem, or a library such as this one.
Here is my modification, now it works fine on my windows.

Could you make the shell: true part conditional to Windows only since the extension works fine on linux and macos?

This is a good idea, I submitted new code, now "shell:true" only works on Windows.

Thanks, @lsby!


AzCopy sync re-copies files even if file already exists and is unchanged

Which version of the AzCopy was used?
10.8.1
Which platform are you using? (ex: Windows, Mac, Linux)
Debian Linux
What command did you run?
azcopy sync --recursive=true --delete-destination=true --mirror-mode=true --preserve-smb-info=true --log-level=INFO
Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.
What problem was encountered?
Every file is re-copied every time I run the command.
How can we reproduce the problem in the simplest way?
Have you found a mitigation/solution?
No
SOme things that might be unique to my setup, the source Debian is mounting a NFS drive which is hosted on a Linux based NAS device(Buffalo Terrastation).
What I notice is the Last Modifed Time on in the Azure container seems to get updated to the date/time when the azcopy runs, it's not keeping the source date/time.
I also tested using --dry-run on a windows server where I mapped the NAS drive and all the files would have been copied over as well.

remove  --mirror-mode=true or mark it as false,  --mirror-mode=false

remove --mirror-mode=true or mark it as false, --mirror-mode=false

That works, thanks for the help.


Playnite 10.1

Extension failed to load properly.
Basically any addon I use that was created by Lacro59 get this error.

fixed with the last release


missing package namespace generates repeated errors on dc: elements

Here's an obscure bug to file away for a rainy day...
If the xmlns declaration is missing on the package element, the invalid element error also gets repeated verbatim for every instance of a dc: element in the metadata:
Validating using EPUB version 3.0 rules.
ERROR(RSC-005): .\WCAG.epub/EPUB/package.opf(3,49): Error while parsing file 'element "package" not allowed anywhere; expected element "package" (with xmlns="http://www.idpf.org/2007/opf")'.
ERROR(RSC-005): .\WCAG.epub/EPUB/package.opf(5,73): Error while parsing file 'element "package" not allowed anywhere; expected element "package" (with xmlns="http://www.idpf.org/2007/opf")'.
ERROR(RSC-005): .\WCAG.epub/EPUB/package.opf(7,67): Error while parsing file 'element "package" not allowed anywhere; expected element "package" (with xmlns="http://www.idpf.org/2007/opf")'.
ERROR(RSC-005): .\WCAG.epub/EPUB/package.opf(8,69): Error while parsing file 'element "package" not allowed anywhere; expected element "package" (with xmlns="http://www.idpf.org/2007/opf")'.
ERROR(RSC-005): .\WCAG.epub/EPUB/package.opf(9,91): Error while parsing file 'element "package" not allowed anywhere; expected element "package" (with xmlns="http://www.idpf.org/2007/opf")'.
ERROR(RSC-005): .\WCAG.epub/EPUB/package.opf(10,41): Error while parsing file 'element "package" not allowed anywhere; expected element "package" (with xmlns="http://www.idpf.org/2007/opf")'.
ERROR(RSC-005): .\WCAG.epub/EPUB/package.opf(14,63): Error while parsing file 'element "package" not allowed anywhere; expected element "package" (with xmlns="http://www.idpf.org/2007/opf")'.

I'm totally whith you that this is bad style, but I think we cannot conditionally suppress parsing errors.
However, we could probably report the missing namespace in a future version.

But if the package element can't be found, shouldn't epubcheck bail entirely on validation at that point and not run any further checks? Seems like a fatal error to not find a package element.
I assume the message is coming from some subsequent test of DC elements, but checking those is pointless.

Okay, got it. Looks like a bug in the validation engine Jing (https://github.com/relaxng/jing-trang). When I try to validate the same OPF file in oXygen (which uses a newer version of Jing for validation) the error message is not thrown multiple times.
Also EpubCheck uses it's own version of Jing (https://mvnrepository.com/artifact/org.daisy.libs/jing/20120724.0.0) which also hasn't gotten any updates from upstream lately and for which I cannot find the source repo, so I'm stuck here... Probably @rdeltour can take over sometime...


Feature Request: Meta Playlists?

Thanks for your app! One feature that I really like in Spotify is the ability to nest playlists in folders. (Up until recently it was also possible to play all the songs in a folder of playlists, but they deprecated that feature on mobile apps. Grr.) For example, I might have a genre folder with playlists of different artists within.
Is it possible to create meta-playlists, by combining multiple playlists according to, say, a common naming convention? In this way you could create a sort of folder of playlists and then shuffle play, etc. For example, if playlists were named "A_Band1", "A_Band2", then all the songs in the two playlists might be grouped through the GPMDP UI as a playlist "A" — even though under the hood the songs were played from the two actual playlists.
Thoughts? I'd be willing to delve into the code, if it were possible. But I'm guessing that without an actual API from Google that this might be more than GPMDP could do.

Judging by the complexity of it, I'm not sure that it's possible. We are already jumping through hoops just to grab the information about the current playlist and even that's not full proof. It's a good idea, but not sure how possible it'd be at the moment.


update tags to be more persistent in time

Codecov Report

Merging #120 (364998d) into main (796addf) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main     #120      +/-   ##
==========================================
- Coverage   54.00%   53.98%   -0.02%
==========================================
  Files          22       22
  Lines        2533     2532       -1
==========================================
- Hits         1368     1367       -1
  Misses       1165     1165

Impacted Files
Coverage Δ

fink_fat/test/cli_test/test_cli.py
100.00% <ø> (ø)

bin/utils_cli.py
100.00% <100.00%> (ø)

fink_fat/others/id_tags.py
100.00% <100.00%> (ø)


Want to add my name in this list

I want to add my name is this list kindly assign this to me

@ShahrozAhmd go ahead!

@shreyamalogi  Thanks 😄


Update SNS get_endpoint_attributes response for Not Found endpoints #3477

Currently the response to get_endpoint_attributes for a non existent endpoint
differ from boto and moto, in the following way (shown in JSON instead of xml
for shortness):

boto: {ErrorResponse:  { Error: { Code: "... ", Type: "... ", Sender: "..." }}
moto:  { ErrorResponse: { Errors: [ { Error: { Code: "... ", Type: "... ", Sender: "..." }}]}}

The template used to generate the NotFound error was the default for RESTError.
Moreover, although RESTError has another template called SINGLE_ERROR_RESPONSE.
That at first sight looks like the right one, it lacks the ErrorResponse node at the root
of the XML, which causes an error when parsing that XML later in the application, and
results in a ClientError instead of a NotFound exception.
This PR takes the approach suggested in the issue conversation.
Although there is one thing I personally do not entirely like, which is the fact there is yet
another variation of this template:

One in moto/core/exceptions.py
One in moto/sns/responses.py
Plus the one added in this commit

An alternative, which avoids creating the third template is not to catch the exception in moto/sns/models.py,
allowing the exception to "bubble" up to moto/sns/responses.py
where it can be caught, and return an error response using the already existing
template in that module.
The issue with this later approach, would be that it differs a bit more from the current
implementation, since moto/sns/models.py does catch all exceptions for all other
functions, and it is also nicer to catch exceptions sooner than later.

Although there is one thing I personally do not entirely like, which is the fact there is yet
another variation of this template
Not a fan of it either, but considering the amount of variation in templates that AWS uses, I think this is the best solution :+1:

Although there is one thing I personally do not entirely like, which is the fact there is yet
another variation of this template
Not a fan of it either, but considering the amount of variation in templates that AWS uses, I think this is the best solution :+1:


Fix for Development proxy ignores path in K8s API URL. Throw error if it has path.

Bug Report
This issue is for temporary fix of #4925. In this fix, we will throw an error once we encounter this sitaution.
What did you do?

set ~/.kube/config to a Rancher-managed cluster's API
(sth. like https://my-domain.tld/k8s/clusters/c-some-id)
kubectl apply […] to deploy CRD
operator-sdk run to run operator

What did you expect to see?
I expected both steps 2 and 3 to target the cluster configured in step 1.
What did you see instead? Under which circumstances?

kubectl apply works as expected
operator-sdk run ignores the path component when setting up the proxy and uses https://my-domain.tld instead
In my particular case this didn't actually surface as an error. Instead the modified API endpoint without a path happened to target the management cluster of my Rancher deployment instead of a development cluster. So it actually ran the operator against a different cluster without any warning at all.

Environment
Operator type:
/language ansible
Kubernetes cluster type:
Rancher, RKE
$ operator-sdk version
operator-sdk version: "v1.7.1-19-g3ce1eeef", […], kubernetes version: "v1.19.4", […]
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"21", […]}
Server Version: version.Info{Major:"1", Minor:"20", […]}
Possible Solution
To prevent people from shooting themselves in the foot I'd suggest that operator-sdk run (and any other commands that might behave similarly) fail with an error message if they encounter an API endpoint with a path component.
Additional context
Kept same description as that of #4925

Released as part of Operator SDK v1.9.0 release.

Fix is working fine and confirmed by the user.


unknown directive "ldap_server"

I'm noticing this after starting nginx even though the ldap auth worked.
2015/07/13 18:13:40 [emerg] 12651#0: unknown directive "ldap_server" in /usr/local/nginx/conf/includes/vhosta.conf:9
Used be8ff8eecb2578304d72e1f4947466ac9ba50295 + nginx 1.8.0.

I ran into this issue because I ran "make install" while nginx was running. Make sure to quit nginx before you try compiling and deploying nginx


Add an optional application_context to Tracker

https://github.com/snowplow/iglu-central/pull/140#issuecomment-136620274

+1 this has come recently on an implementation we're currently doing (separate to Colm's mentioned above), feature exists on iOS but not on Android


Windows flutter/plugins tasks failing

Examples:
https://ci.chromium.org/ui/p/flutter/builders/try/Windows Plugins/4267/overview
https://ci.chromium.org/ui/p/flutter/builders/try/Windows Plugins/4279/overview
The output is:
Checking Dart SDK version...
Downloading Dart SDK from Flutter engine b1385c0df12549b9dce7ce3fee361bd4898f49ae...
Expanding downloaded archive...
Building flutter tool...
Running pub upgrade...
Got TLS error trying to find package archive at https://pub.dartlang.org.
Error (69): Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (9 tries left)
Running pub upgrade...
Terminate batch job (Y/N)?
^CError (-1073741510): Unable to 'pub upgrade' flutter tool. Retrying in five seconds... (8 tries left)
Running pub upgrade...

It looks like this started sometime over the weekend. It may be an out-of-band failure, or it may have been caused somehow by https://github.com/flutter/plugins/pull/3938 which was landed with exactly this problem. @CaseyHillers

@godofredoc do we need to CP the windows cert changes over to the plugins?

Setting to P1 as it's blocking the plugins tree. The plugin pool started to experience this 5/21 with it fully escalating to blocking on 5/23.

This is because the windows image have regressed and the plugins recipe is not installing certificates manually.

Recipes CL to add the dependencies support:
https://flutter-review.googlesource.com/c/recipes/+/13900

Please reopen if it is still failing.


Update default SECRET_KEY in docs

docs: rotate SECRET_KEY without invalid decryption key error
SUMMARY
If someone has superset running with important stuffs and wants to change SECRET_KEY then definitely they need PREVIOUS_SECRET_KEY mentioned along with SECRET_KEY so decryption with PREVIOUS_SECRET_KEY can happen and encryption with latest SECRET_KEY can be performed. But in the documentation a wrong default SECRET_KEY is mentioned that can mislead the reader/user.

@rusackas How can I contribute to docs.

Creating one more pull request on top of this.


Use .NET 6+ built-in time zone data to avoid other native platform dependencies

This is intended to fix https://github.com/datalust/seq-app-htmlemail/issues/105.
The problem causing that failure appears to be that GZipStream having platform and architecture-specific dependencies that the managed Seq app packaging mechanism (using Seq.Apps.dll and seqcli host) can't account for.
This PR moves from TimeZoneConverter (thanks Matt Johnson-Pint!) to .NET 6's built-in time zone data (thanks also Matt Johnson-Pint 😁), which should be correctly deployed thanks to the architecture-specific seqcli host distribution.
Unfortunately this will lift the minimum Seq version supported by this package up to 2022.1, but that should cover the majority of actively-maintained Seq servers out there.
The 4.0.0 app version will remain available for Seq 5.1+ compatibility.

Notes on the .NET 6 APIs: https://devblogs.microsoft.com/dotnet/date-time-and-time-zone-enhancements-in-net-6/#time-zone-conversion-apis

Looks like I spoke too soon, it appears that we'd need to deploy ICU for this to work.

Tests are now passing on .NET 7 (Seq 2023.1); we'll need to adjust the tests to use Windows time zone names for non-UTC time zones on .NET 6 (Seq 2022.1).


fails to compile

Downloading nwjs... CMake Error at /opt/ros/melodic/share/catkin/cmake/safe_execute_process.cmake:11 (message):   execute_process(/home/gal/flexBE_ws/src/flexbe_app/bin/nwjs_install)   returned error code 18 Call Stack (most recent call first):   flexbe_app/CMakeLists.txt:36 (safe_execute_process) during compilation
Could it be cause I'm running Ubuntu 18.04 with ROS Melodic?

any idea why this happens? server maybe?

Can you try to run
rosrun flexbe_app nwjs_install

seperately? Does it fail as well? If yes, what's the error there?
Ubuntu 18.04 with melodic should be fine, I'm using the same at the moment.

Somehow got it to work on melodic and kinetic - thanks. I should have a few more installs to go on new machines so if I have the same issue I'll try your recommendation.


dont run golanglint-ci

it fails on ci because there are no go files
we should add it back when we add integration tests

There is already an integration suite PR open, https://github.com/paketo-community/python/pull/6. Does it make sense to just close this then?

@ryanmoran didnt see that, thanks


the game slows down after showing ads

I created a game using the matter js engine. Everything works great, but after showing ads (admob or unity ads), the game starts to slow down. Even if we remove almost all objects and behavior. Has anyone faced a similar problem? How to cure?

your ads are mining bitcoin


Provide a better report of non-fatal errors to users

The goal of this issue is to improve reporting and management of non-fatal errors:

When encountering too frequent retries, change the base in the exponential backoff to a higher value like 1s.
Add a boolean flag to Error condition to indicate whether re-attempts will be done to fix the error.
Affects: FybrikApplication

@shlomitk1, what's the status of the issue? can it be closed?


Docs say get_tree().physics_frame is signal but await can't detect it as such

Godot version
Godot 4.0 alpha 2
System information
Linux
Issue description
await get_tree().physics_frame's error message says physics_frame is not a signal.
The Docs for Class: SceneTree state it is a signal.
Steps to reproduce
await get_tree().physics_frame
print("done")

instantly prints done without waiting for physics frame with the debug message:
W 0:00:00:0555   "await" keyword not needed in this case, because the expression isn't a coroutine nor a signal. <C++ Error>    REDUNDANT_AWAIT
Minimal reproduction project
No response

this is not limited to physics_frame specifically,
I get
W 0:00:00:0635   "await" keyword not needed in this case, because the expression isn't a coroutine nor a signal.
  <C++ Error>    REDUNDANT_AWAIT

for several signals, including
await tween.finished
await get_tree().process_frame
await get_tree().create_timer(0.4).timeout
This just seems to be an incorrect warning though, the await still seems to work as expected.

Sounds like #56265, except signal should definitely be known in this case...

@h0lley I am not sure which like of code you are referring to but I want to be clear that the line await get_tree().physics_frame is completely skipped and does not work as expected. You can test by putting a print before await get_tree().process_frame, a print in inside the _process function and a print below await get_tree().process_frame. Run it and in your log it should be before, _process, after, (_process * Inf). I incorrectly get before, after, (_process * Inf).

@h0lley Sorry, you are right, it is asyncing, I changed my test code and this worked:
self.call("a")
self.call_deferred("b")
await get_tree().physics_frame
self.call("c")

It prints a, b, then c
If you comment out the await line it will prints a, c, then b.

This also warns on custom signals:
My node:
extends Node

signal built()

func _ready():
    self.emit_signal("built")

Another node:
func _ready():
    var other_node = preload("res://my_node.gd").new()
    await other_node.built


Nvidia drivers fail to build with linuxPackages_latest on 20.09

Describe the bug
When running kernel version 5.11 (through linuxPackages_latest or similar), the nvidia driver will fail to build with this:

Build log
nvidia-x11> make[3]: *** [/nix/store/q1f1bm027060j3ax0gvfl37bkcj3qf7q-linux-5.11.20-dev/lib/modules/5.11.20/source/scripts/Makefile.build:279: /build/kernel/nvidia/nv-mmap.o] Error 1
nvidia-x11> In file included from /build/kernel/common/inc/nv-pci.h:15,
nvidia-x11>                  from /build/kernel/nvidia/nv-pci.c:13:
nvidia-x11> /build/kernel/common/inc/nv-linux.h:119:10: fatal error: asm/kmap_types.h: No such file or directory
nvidia-x11>   119 | #include <asm/kmap_types.h>         /* page table entry lookup          */
nvidia-x11>       |          ^~~~~~~~~~~~~~~~~~
nvidia-x11> compilation terminated.
nvidia-x11> make[3]: *** [/nix/store/q1f1bm027060j3ax0gvfl37bkcj3qf7q-linux-5.11.20-dev/lib/modules/5.11.20/source/scripts/Makefile.build:279: /build/kernel/nvidia/nv-pci.o] Error 1
nvidia-x11> In file included from /build/kernel/nvidia/nv-i2c.c:17:
nvidia-x11> /build/kernel/common/inc/nv-linux.h:119:10: fatal error: asm/kmap_types.h: No such file or directory
nvidia-x11>   119 | #include <asm/kmap_types.h>         /* page table entry lookup          */
nvidia-x11>       |          ^~~~~~~~~~~~~~~~~~
nvidia-x11> compilation terminated.
nvidia-x11> make[3]: *** [/nix/store/q1f1bm027060j3ax0gvfl37bkcj3qf7q-linux-5.11.20-dev/lib/modules/5.11.20/source/scripts/Makefile.build:279: /build/kernel/nvidia/nv-i2c.o] Error 1
nvidia-x11> In file included from /build/kernel/nvidia/nv-procfs.c:15:
nvidia-x11> /build/kernel/common/inc/nv-linux.h:119:10: fatal error: asm/kmap_types.h: No such file or directory
nvidia-x11>   119 | #include <asm/kmap_types.h>         /* page table entry lookup          */
nvidia-x11>       |          ^~~~~~~~~~~~~~~~~~
nvidia-x11> compilation terminated.
nvidia-x11> make[3]: *** [/nix/store/q1f1bm027060j3ax0gvfl37bkcj3qf7q-linux-5.11.20-dev/lib/modules/5.11.20/source/scripts/Makefile.build:279: /build/kernel/nvidia/nv-procfs.o] Error 1
nvidia-x11> In file included from /build/kernel/nvidia/nv-cray.c:15:
nvidia-x11> /build/kernel/common/inc/nv-linux.h:119:10: fatal error: asm/kmap_types.h: No such file or directory
nvidia-x11>   119 | #include <asm/kmap_types.h>         /* page table entry lookup          */
nvidia-x11>       |          ^~~~~~~~~~~~~~~~~~
nvidia-x11> compilation terminated.
nvidia-x11> make[3]: *** [/nix/store/q1f1bm027060j3ax0gvfl37bkcj3qf7q-linux-5.11.20-dev/lib/modules/5.11.20/source/scripts/Makefile.build:279: /build/kernel/nvidia/nv-cray.o] Error 1
nvidia-x11> make[2]: *** [/nix/store/q1f1bm027060j3ax0gvfl37bkcj3qf7q-linux-5.11.20-dev/lib/modules/5.11.20/source/Makefile:1820: /build/kernel] Error 2
nvidia-x11> make[2]: Leaving directory '/nix/store/q1f1bm027060j3ax0gvfl37bkcj3qf7q-linux-5.11.20-dev/lib/modules/5.11.20/build'
nvidia-x11> make[1]: *** [Makefile:185: __sub-make] Error 2
nvidia-x11> make[1]: Leaving directory '/nix/store/q1f1bm027060j3ax0gvfl37bkcj3qf7q-linux-5.11.20-dev/lib/modules/5.11.20/source'
nvidia-x11> make: *** [Makefile:81: modules] Error 2

This is an upstream issue, quoting the release notes of a subsequent driver:

Fixed a driver installation failure on Linux kernel 5.11 release candidates, where the NVIDIA kernel module failed to build with error
"fatal error: asm/kmap_types.h: No such file or directory".

To Reproduce
Steps to reproduce the behavior:

Specify the following in a configuration.nix:

{ pkgs, ... }: {
    nixpkgs.config.allowUnfreePredicate = pkg:
        builtins.elem (pkgs.lib.getName pkg) [
            "nvidia-x11"
            "nvidia-settings"
            "nvidia-persistenced"
        ];

    boot.kernelPackages = pkgs.linuxPackages_latest;
    services.xserver.videoDrivers = "nvidia";
}

Build the config

Expected behavior
Nvidia drivers shouldn't fail to build.
Additional context
Some people are working around this by using the unstable linux package, which made #123972 worse than it should have been.
Notify maintainers

@baracoder

Metadata

system: "x86_64-linux"
host os: Linux 5.11.19, NixOS, 20.09.20210514.21ff930 (Nightingale)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.4pre20210503_6d2553a
channels(root): "nixos-21.05pre279012.ad47284f8b0"
channels(tlater): ""
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

(Note the channels are wrong, see my flake.lock for exact details).
Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

I can confirm having the same issue with both  nvidia and nvidiaBeta when using
boot.kernelPackages = pkgs.linuxPackages_latest;

Same here

For reference, this will be fixed in 21.05, which should be released any time today, after which I believe 20.09 will be unmaintained anyway.
So if you're running into this, just upgrade to the next stable version :)

tested under 21.05 and seems issue is solved

Yep, can confirm as well. Closing the issue with that.


Enable spread-cycle for Head

Head should use spread-cycle by default, because it creates a more predictable output current, which we need during pick-up-tip. Downside is that it is louder than stealth mode, but that's an ok tradeoff.
Should follow #458 so it uses the global-scaler setting.

Head should use spread-cycle by default, because it creates a more predictable output current, which we need during pick-up-tip. Downside is that it is louder than stealth mode, but that's an ok tradeoff.
Should follow #458 so it uses the global-scaler setting.

@andySigler if you retarget this on that pr it'll make the diff  easier to see and should automatically go target on main when merged

Ok I did that and now the diff looks crazy.  I'm hoping #458 can be merged and this will look less messy or I mucked it up somehow


Update doc for vagrant deploy module.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2153/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2156/
Test PASSed.


Add ja locale.

Add Japanese locale.

@tsuyoz Is there any way I could get you to do a Japanese translation for PrimeFaces?
In PrimeFaces we have two files to translate a client side and server side translation. Below are the two English full translations:
Client: Client English JS File
Server: Server English Properties File
I know its asking a lot but the PF community would really appreciate it!

Hi there!
Thank you for contacting, about this work, I'll handle it!
Happy holidays!

Thank you!  You can either paste me the files and I can submit the PR or however you want to do it.


Error parsing lftp job status

2020/06/08 24:41:05 - DEBUG - RemoteScanner - Running a scan
2020/06/08 24:41:05 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'tv'
2020/06/08 24:41:05 - DEBUG - RemoteScanner.RemoteScanner.Sshcp - Command: ssh (redacted)
2020/06/08 24:41:06 - DEBUG - LocalScanner - Running a scan
2020/06/08 24:41:06 - DEBUG - LocalScanner - Scan took 0.056s
2020/06/08 24:41:06 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'tv'
2020/06/08 24:41:07 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'tv'
2020/06/08 24:41:08 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'tv'
2020/06/08 24:41:08 - DEBUG - RemoteScanner.RemoteScanner.Sshcp - Return code: 0
2020/06/08 24:41:08 - DEBUG - RemoteScanner.RemoteScanner.Sshcp - Command took 2.365s
2020/06/08 24:41:08 - DEBUG - RemoteScanner - Scan took 2.366s
2020/06/08 24:41:08 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'tv'
2020/06/08 24:41:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'tv'
2020/06/08 24:41:10 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - LftpJobStateParser error: Missing queue header line 1: 2020-06-08 05:41:10 sftp://(redacted).mkv -> /downloads/tv/(redacted).mkv.lftp 0-100812963 284.0 KiB/s
2020/06/08 24:41:10 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - Status: 2020-06-08 05:41:10 sftp://(redacted).mkv -> /downloads/tv/(redacted).mkv.lftp 0-100812963 284.0 KiB/s [0] queue (sftp://(redacted)) -- 17.77 MiB/s sftp://(redacted) Now executing: [1] mirror -c /home/user/media/tv /downloads/ -- 30G/30G (97%) 17.77 MiB/s [1] mirror -c /home/user/media/tv /downloads/ -- 30G/30G (97%) 17.77 MiB/s \mirror `(redacted)' -- 19G/20G (96%) 17.77 MiB/s \mirror `(redacted)' \transfer `(redacted).mkv' `(redacted).mkv', got<PHONE_NUMBER> of<PHONE_NUMBER> (99%) 249.6K/s eta:20s \chunk 0-71637542 `(redacted).mkv' at 66355200 (3%) 249.6K/s eta:20s [Receiving data] \transfer `(redacted).mkv' `(redacted).mkv', got<PHONE_NUMBER> of<PHONE_NUMBER> (62%) 17.52M/s eta:3m \chunk 0-82293835 `(redacted).mkv' at 39157760 (1%) 537.3K/s eta:79s [Receiving data] \chunk<PHONE_NUMBER>-2057345442 `(redacted).mkv' at<PHONE_NUMBER> (91%) 992.7K/s eta:6s [Receiving data] \chunk<PHONE_NUMBER>-1975051625 `(redacted).mkv' at<PHONE_NUMBER> (28%) 398.3K/s eta:3m [Receiving data] \chunk<PHONE_NUMBER>-1892757808 `(redacted).mkv' at<PHONE_NUMBER> (73%) 952.9K/s eta:26s [Receiving data] \chunk<PHONE_NUMBER>-1810463991 `(redacted).mkv' at<PHONE_NUMBER> (64%) 698.8K/s eta:39s [Receiving data] \chunk<PHONE_NUMBER>-1728170174 `(redacted).mkv' at<PHONE_NUMBER> (66%) 629.6K/s eta:35s [Receiving data] \chunk<PHONE_NUMBER>-1645876357 `(redacted).mkv' at<PHONE_NUMBER> (33%) 367.3K/s eta:2m [Receiving data] \chunk<PHONE_NUMBER>-1563582540 `(redacted).mkv' at<PHONE_NUMBER> (73%) 798.3K/s eta:25s [Receiving data] \chunk<PHONE_NUMBER>-1481288723 `(redacted).mkv' at<PHONE_NUMBER> (59%) 631.1K/s eta:49s [Receiving data] \chunk<PHONE_NUMBER>-1398994906 `(redacted).mkv' at<PHONE_NUMBER> (57%) 594.5K/s eta:52s [Receiving data] \chunk<PHONE_NUMBER>-1316701089 `(redacted).mkv' at<PHONE_NUMBER> (60%) 646.4K/s eta:47s [Receiving data] \chunk<PHONE_NUMBER>-1234407272 `(redacted).mkv' at<PHONE_NUMBER> (46%) 666.6K/s eta:82s [Receiving data] \chunk<PHONE_NUMBER>-1152113455 `(redacted).mkv' at<PHONE_NUMBER> (70%) 700.5K/s eta:29s [Receiving data] \chunk 987525822-1069819638 `(redacted).mkv' at<PHONE_NUMBER> (55%) 560.4K/s eta:56s [Receiving data] \chunk 905232005-987525821 `(redacted).mkv' at 949239429 (53%) 779.0K/s eta:62s [Receiving data] \chunk 822938188-905232004 `(redacted).mkv' at 853838412 (37%) 578.5K/s eta:2m [Receiving data] \chunk 740644371-822938187 `(redacted).mkv' at 781735443 (49%) 558.3K/s eta:71s [Receiving data] \chunk 658350554-740644370 `(redacted).mkv' at 734077402 (92%) 1.06M/s eta:6s [Receiving data] \chunk 576056737-658350553 `(redacted).mkv' at 640445857 (78%) 923.6K/s eta:20s [Receiving data] \chunk 493762920-576056736 `(redacted).mkv' at 547305832 (65%) 731.4K/s eta:38s [Receiving data] \chunk 411469103-493762919 `(redacted).mkv' at 466158895 (66%) 686.8K/s eta:36s [Receiving data] \chunk 329175286-411469102 `(redacted).mkv' at 394809590 (79%) 955.9K/s eta:18s [Receiving data] \chunk 246881469-329175285 `(redacted).mkv' at 312646845 (79%) 1.05M/s eta:18s [Receiving data] \chunk 164587652-246881468 `(redacted).mkv' at 204728452 (48%) 558.6K/s eta:75s [Receiving data] \chunk 82293835-164587651 `(redacted).mkv' at 152286283 (85%) 840.1K/s eta:12s [Receiving data]
2020/06/08 24:41:10 - ERROR - seedsync.ControllerJob - Caught exception in job ControllerJob
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 81, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 486, in __parse_queue
raise ValueError("Missing queue header line 1: {}".format(line))
ValueError: Missing queue header line 1: 2020-06-08 05:41:10 sftp://(redacted).mkv -> /downloads/tv/(redacted).mkv.lftp 0-100812963 284.0 KiB/s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 314, in status
statuses = self.__job_status_parser.parse(statuses_str)
File "/app/python/lftp/job_status_parser.py", line 86, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2020/06/08 24:41:10 - DEBUG - seedsync.ControllerJob - Calling cleanup for ControllerJob
2020/06/08 24:41:10 - DEBUG - seedsync.Controller - Exiting controller
2020/06/08 24:41:10 - ERROR - seedsync - Caught exception
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 81, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 486, in __parse_queue
raise ValueError("Missing queue header line 1: {}".format(line))
ValueError: Missing queue header line 1: 2020-06-08 05:41:10 sftp://(redacted).mkv -> /downloads/tv/(redacted).mkv.lftp 0-100812963 284.0 KiB/s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/python/seedsync.py", line 162, in run
controller_job.propagate_exception()
File "/app/python/common/job.py", line 77, in propagate_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 314, in status
statuses = self.__job_status_parser.parse(statuses_str)
File "/app/python/lftp/job_status_parser.py", line 86, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2020/06/08 24:41:11 - DEBUG - ActiveScanner - Process received terminate flag
2020/06/08 24:41:11 - DEBUG - ActiveScanner - Exiting process
2020/06/08 24:41:14 - DEBUG - ExtractProcess - Process received terminate flag
2020/06/08 24:41:15 - DEBUG - seedsync.Controller.MPLogger - Stopped listener thread
2020/06/08 24:41:15 - INFO - seedsync.Controller - Exited controller

Likely related to #35

Mark, is this on the latest version? This error was fixed in version 0.7.1.
If you still see it, please enable Debug Logs in Settings, and send me the logs including the crash.

Yes, I'm running v0.7.3. I currently have this disabled due to some local issues I'm working through (transferring files, don't need this to mess it up). I'll try to get the debug log when I can. For now, the output I already provided above is all I got.

I've updated the output above to include the debug about before and after the error.

I have the same problem, but it did not start until I updated to 0.7.3. I was previously on 0.7.0 before that so whatever causes Error parsing lftp job status must have been in-between those.
Right now I have to manually use the Restart multiple times a day to get it working again.

Here is my log (with identifying info removed). Looks exactly the same as Mark's.
2020/06/08 12:31:36 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - LftpJobStateParser error: Missing queue header line 1: 2020-06-08 19:31:36<EMAIL_ADDRESS>-> /path/video.mkv.lftp 708761702-945015599 1.87 MiB/s
2020/06/08 12:31:36 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - Status: 2020-06-08 19:31:36<EMAIL_ADDRESS>-> /path/video.mkv.lftp 708761702-945015599 1.87 MiB/s [0] queue<EMAIL_ADDRESS>-- 15.23 MiB/s<EMAIL_ADDRESS>Now executing: [1] mirror -c /path/tv /downloads/ -- 99G/117G (84%) 15.23 MiB/s [1] mirror -c /path/tv /downloads/ -- 99G/117G (84%) 15.23 MiB/s \mirror `directory1' \mirror `directory1/directory2' -- 32G/51G (62%) 15.23 MiB/s \transfer `directory1/directory2/video.mkv' `video.mkv', got<PHONE_NUMBER> of<PHONE_NUMBER> (97%) 15.23M/s eta:13s \chunk 0-236253908 `video.mkv' at 235143168 (8%) 2.01M/s eta:1s [Receiving data] \chunk<PHONE_NUMBER>-2598792877 `video.mkv' at<PHONE_NUMBER> (98%) 2.00M/s eta:2s [Receiving data] \chunk<PHONE_NUMBER>-2126285083 `video.mkv' at<PHONE_NUMBER> (94%) 1.69M/s eta:7s [Receiving data] \chunk<PHONE_NUMBER>-1890031186 `video.mkv' at<PHONE_NUMBER> (95%) 1.87M/s eta:6s [Receiving data] \chunk<PHONE_NUMBER>-1417523392 `video.mkv' at<PHONE_NUMBER> (97%) 1.80M/s eta:4s [Receiving data] \chunk 708761702-945015598 `video.mkv' at 945015599 (100%) 2.06M/s eta:0s [Receiving data] \chunk 472507805-708761701 `video.mkv' at 703686045 (97%) 1.92M/s eta:3s [Receiving data] \chunk 236253908-472507804 `video.mkv' at 449737428 (90%) 1.87M/s eta:13s [Receiving data]
2020/06/08 12:31:36 - ERROR - seedsync.ControllerJob - Caught exception in job ControllerJob
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 81, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 486, in __parse_queue
raise ValueError("Missing queue header line 1: {}".format(line))
ValueError: Missing queue header line 1: 2020-06-08 19:31:36<EMAIL_ADDRESS>-> /path/video.mkv.lftp 708761702-945015599 1.87 MiB/s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 314, in status
statuses = self.__job_status_parser.parse(statuses_str)
File "/app/python/lftp/job_status_parser.py", line 86, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2020/06/08 12:31:36 - DEBUG - seedsync.ControllerJob - Calling cleanup for ControllerJob
2020/06/08 12:31:36 - DEBUG - seedsync.Controller - Exiting controller
2020/06/08 12:31:36 - ERROR - seedsync - Caught exception
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 81, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 486, in __parse_queue
raise ValueError("Missing queue header line 1: {}".format(line))
ValueError: Missing queue header line 1: 2020-06-08 19:31:36<EMAIL_ADDRESS>-> /path/video.mkv.lftp 708761702-945015599 1.87 MiB/s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/python/seedsync.py", line 162, in run
controller_job.propagate_exception()
File "/app/python/common/job.py", line 77, in propagate_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 314, in status
statuses = self.__job_status_parser.parse(statuses_str)
File "/app/python/lftp/job_status_parser.py", line 86, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status

Thanks for the logs guys. It will be fixed soon in the next version.

Please try the newest version v0.8.0.

Nope. Still happening (albeit less frequently):
2020/06/15 09:33:10 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - LftpJobStateParser error: Missing queue header line 1: /downloads/(redacted).mp4.lftp<PHONE_NUMBER>-1369325714 220.3 KiB/s
2020/06/15 09:33:10 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - Status: (redacted) [Receiving data]
2020/06/15 09:33:10 - ERROR - seedsync.ControllerJob - Caught exception in job ControllerJob
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 85, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 490, in __parse_queue
raise ValueError("Missing queue header line 1: {}".format(line))
ValueError: Missing queue header line 1: /downloads/(redacted).mp4.lftp<PHONE_NUMBER>-1369325714 220.3 KiB/s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 309, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 90, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2020/06/15 09:33:11 - ERROR - seedsync - Caught exception
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 85, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 490, in __parse_queue
raise ValueError("Missing queue header line 1: {}".format(line))
ValueError: Missing queue header line 1: /downloads/(redacted).mp4.lftp<PHONE_NUMBER>-1369325714 220.3 KiB/s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/python/seedsync.py", line 162, in run
controller_job.propagate_exception()
File "/app/python/common/job.py", line 77, in propagate_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 309, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 90, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2020/06/15 09:33:14 - INFO - seedsync.Controller - Exited controller

Could you please post the full second line only redacting the filenames.
2020/06/15 09:33:10 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - Status: (redacted) [Receiving data]
I need to see the full status to see what's going wrong.

updated

It seemed like we had the same problem but mine has actually had no problems for the last week. Been smooth sailing since 0.8.0.

Another crash, this had debug enabled though:
2020/06/17 07:42:12 - DEBUG - RemoteScanner - Scan took 2.360s
2020/06/17 07:42:12 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:42:12 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:13 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:13 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:14 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:14 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:15 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:16 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:16 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:17 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:17 - DEBUG - LocalScanner - Running a scan
2020/06/17 07:42:17 - DEBUG - LocalScanner - Scan took 0.058s
2020/06/17 07:42:17 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:18 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:18 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:19 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:20 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file '(redacted)'
2020/06/17 07:42:20 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - LftpJobStateParser error: Missing queue header line 1: /downloads/(redacted).mp4.lftp 914457562-997590066 436.5 KiB/s
2020/06/17 07:42:20 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - Status: /downloads/(redacted).mp4.lftp 914457562-997590066 436.5 KiB/s jobs -v [0] queue (sftp://(redacted)) -- 4.50 MiB/s sftp://(redacted) Now executing: [1] mirror -c (redacted) /downloads/ -- 2.1G/2.1G (97%) 4.50 MiB/s [1] mirror -c (redacted) /downloads/ -- 2.1G/2.1G (97%) 4.50 MiB/s \mirror `Fallen (1998)' \transfer `(redacted).mp4' `(redacted).mp4', got<PHONE_NUMBER> of<PHONE_NUMBER> (97%) 4.50M/s eta:32s \chunk<PHONE_NUMBER>-1579517593 `(redacted).mp4' at<PHONE_NUMBER> (85%) 823.2K/s eta:29s [Receiving data] \chunk<PHONE_NUMBER>-1413252585 `(redacted).mp4' at<PHONE_NUMBER> (84%) 735.7K/s eta:32s [Receiving data] \chunk 748192554-831325057 `(redacted).mp4' at 819200810 (85%) 787.2K/s eta:29s [Receiving data] \chunk 665060050-748192553 `(redacted).mp4' at 736985810 (86%) 891.7K/s eta:26s [Receiving data] \chunk 415662538-498795041 `(redacted).mp4' at 497844682 (98%) 1.33M/s eta:2s [Receiving data]
2020/06/17 07:42:20 - ERROR - seedsync.ControllerJob - Caught exception in job ControllerJob
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 85, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 490, in __parse_queue
raise ValueError("Missing queue header line 1: {}".format(line))
ValueError: Missing queue header line 1: /downloads/(redacted).mp4.lftp 914457562-997590066 436.5 KiB/s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 309, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 90, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2020/06/17 07:42:20 - DEBUG - seedsync.ControllerJob - Calling cleanup for ControllerJob
2020/06/17 07:42:20 - DEBUG - seedsync.Controller - Exiting controller
2020/06/17 07:42:21 - DEBUG - ActiveScanner - Process received terminate flag
2020/06/17 07:42:21 - DEBUG - ActiveScanner - Exiting process
2020/06/17 07:42:21 - ERROR - seedsync - Caught exception
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 85, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 490, in __parse_queue
raise ValueError("Missing queue header line 1: {}".format(line))
ValueError: Missing queue header line 1: /downloads/(redacted).mp4.lftp 914457562-997590066 436.5 KiB/s

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/python/seedsync.py", line 162, in run
controller_job.propagate_exception()
File "/app/python/common/job.py", line 77, in propagate_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 309, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 90, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2020/06/17 07:42:24 - DEBUG - ExtractProcess - Process received terminate flag
2020/06/17 07:42:24 - DEBUG - seedsync.Controller.MPLogger - Stopped listener thread
2020/06/17 07:42:24 - INFO - seedsync.Controller - Exited controller
2020/06/17 07:42:24 - DEBUG - seedsync.ControllerJob - Finished cleanup for ControllerJob
2020/06/17 07:42:24 - DEBUG - seedsync.ControllerJob - Thread ControllerJob stopped
2020/06/17 07:42:42 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:43:17 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 07:43:42 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:44:12 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:44:42 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:45:18 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 07:45:42 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:46:12 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:46:48 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 07:47:13 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:47:43 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:48:16 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 07:48:43 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:49:13 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:49:43 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:50:16 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 07:50:43 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:51:13 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:51:47 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 07:52:13 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:52:43 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:53:17 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 07:53:43 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:54:13 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:54:47 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 07:55:13 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:55:43 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:56:18 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 07:56:43 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:57:13 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:57:44 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:58:18 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 07:58:44 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:59:14 - DEBUG - seedsync - Persisting states to file
2020/06/17 07:59:44 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:00:18 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:00:44 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:01:14 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:01:47 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:02:14 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:02:44 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:03:18 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:03:44 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:04:14 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:04:48 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:05:14 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:05:44 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:06:14 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:06:44 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:07:14 - DEBUG - seedsync.WebApp - Stream connection stopped by client
2020/06/17 08:07:14 - DEBUG - seedsync.Controller.Model - LftpModel: Removing a listener
2020/06/17 08:07:17 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:07:44 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:08:14 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:08:45 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:09:20 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:09:45 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:10:15 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:10:45 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:11:19 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:11:45 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:12:15 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:12:49 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:13:15 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:13:45 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:14:19 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:14:45 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:15:15 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:15:48 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:16:15 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:16:45 - DEBUG - seedsync - Persisting states to file
2020/06/17 08:17:15 - DEBUG - seedsync.WebApp - Stream connection stopped by client
2020/06/17 08:17:15 - DEBUG - seedsync.Controller.Model - LftpModel: Removing a listener
2020/06/17 08:17:18 - DEBUG - seedsync.Controller.Model - LftpModel: Adding a listener
2020/06/17 08:17:45 - DEBUG - seedsync - Persisting states to file

Thanks Mark for the logs. I have never seen this case before but I'll add a handler for it.

Mark try v0.8.1

Ok. Trying now.

I still have some parsing issue here

@Jencryzthers Could you please send me a few of the messages before this error. This screenshot does not include the status line which I need to understand the failure. Thanks.

I reproduced the error and redacted a few line. Let me know if this is enough.
`2020/06/29 18:55:09 - DEBUG - LocalScanner - Scan took 3.306s
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:09 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted'
2020/06/29 18:55:10 - WARNING - ActiveScanner.ActiveScanner - Path does not exist: /downloads/redacted
2020/06/29 18:55:10 - WARNING - ActiveScanner.ActiveScanner - Path does not exist: /downloads/redacted
2020/06/29 18:55:10 - WARNING - ActiveScanner.ActiveScanner - Path does not exist: /downloads/redacted
2020/06/29 18:55:10 - WARNING - ActiveScanner.ActiveScanner - Path does not exist: /downloads/redacted
2020/06/29 18:55:11 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - LftpJobStateParser error: Unable to parse line '[8] pget -c /media/sdq1/home/jencryzthers/private/deluge/data/sync/The.Redacted.Redacted.2020.Multi.10jobs -v'
2020/06/29 18:55:11 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - Status: jobs -v [0] queue (sftp://redacted@feralhosting.com:22) -- 29.12 MiB/s sftp://redacted@feralhosting.com:22/media/sdq1/home/jencryzthers
2020/06/29 18:55:11 - ERROR - seedsync.ControllerJob - Caught exception in job ControllerJob
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 90, in parse
statuses += self.__parse_jobs(lines)
File "/app/python/lftp/job_status_parser.py", line 472, in __parse_jobs
raise ValueError("Unable to parse line '{}'".format(line))
ValueError: Unable to parse line '[8] pget -c /media/sdq1/home/jencryzthers/private/deluge/data/sync/The.Redacted.Redacted.2020.Multi.10jobs -v'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 316, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 94, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2020/06/29 18:55:11 - DEBUG - seedsync.ControllerJob - Calling cleanup for ControllerJob
2020/06/29 18:55:11 - DEBUG - seedsync.Controller - Exiting controller
2020/06/29 18:55:11 - WARNING - ActiveScanner.ActiveScanner - Path does not exist: /downloads/redacted
2020/06/29 18:55:11 - WARNING - ActiveScanner.ActiveScanner - Path does not exist: /downloads/redacted
2020/06/29 18:55:11 - WARNING - ActiveScanner.ActiveScanner - Path does not exist: /downloads/redacted
2020/06/29 18:55:11 - WARNING - ActiveScanner.ActiveScanner - Path does not exist: /downloads/redacted
2020/06/29 18:55:11 - ERROR - seedsync - Caught exception
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 90, in parse
statuses += self.__parse_jobs(lines)
File "/app/python/lftp/job_status_parser.py", line 472, in __parse_jobs
raise ValueError("Unable to parse line '{}'".format(line))
ValueError: Unable to parse line '[8] pget -c /media/sdq1/home/jencryzthers/private/deluge/data/sync/The.Redacted.Redacted.2020.Multi.10jobs -v'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python/seedsync.py", line 162, in run
controller_job.propagate_exception()
File "/app/python/common/job.py", line 77, in propagate_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 316, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 94, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status`

Please try version 0.8.3.

I just updated and now it is unable to retrive anything. I tested using manual lftp command and it works.
Here is the log
2020/06/30 10:29:52 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted.mkv'
2020/06/30 10:29:52 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted.mkv'
2020/06/30 10:29:52 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted.mkv'
2020/06/30 10:29:52 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted.mkv'
2020/06/30 10:29:52 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted.mkv'
2020/06/30 10:29:52 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted.mkv'
2020/06/30 10:29:52 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted.mkv'
2020/06/30 10:29:52 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted.mkv'
2020/06/30 10:29:52 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted.mkv'
2020/06/30 10:29:52 - DEBUG - seedsync.Controller.Model - LftpModel: Updating file 'redacted.mkv'
2020/06/30 10:29:56 - ERROR - seedsync.Controller.Lftp - Lftp timeout exception
Traceback (most recent call last):
File "/app/python/lftp/lftp.py", line 128, in __run_command
self.__process.expect(self.__expect_pattern, timeout=self._timeout)
File "/usr/local/lib/python3.7/dist-packages/pexpect/spawnbase.py", line 344, in expect
timeout, searchwindowsize, async)
File "/usr/local/lib/python3.7/dist-packages/pexpect/spawnbase.py", line 372, in expect_list
return exp.expect_loop(timeout)
File "/usr/local/lib/python3.7/dist-packages/pexpect/expect.py", line 181, in expect_loop
return self.timeout(e)
File "/usr/local/lib/python3.7/dist-packages/pexpect/expect.py", line 144, in timeout
raise exc
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x14fdf0f15a10>
command: /usr/bin/lftp
args: ['/usr/bin/lftp', '-p', '22', '-u', 'jencryzthers,{redacted}', 'sftp://{redacted}.feralhosting.com']
buffer (last 100 chars): b' '
before (last 100 chars): b' '
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 7
child_fd: 5
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: re.compile(b'lftp jencryzthers@{redacted}.feralhosting.com:.*>')
2020/06/30 10:29:57 - DEBUG - RemoteScanner - Running a scan
2020/06/30 10:29:57 - DEBUG - RemoteScanner.RemoteScanner.Sshcp - Command: ssh -p 22 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -o PubkeyAuthentication=no<EMAIL_ADDRESS>"'/tmp/scanfs' '/media/sdq1/home/jencryzthers/private/deluge/data/sync'"
2020/06/30 10:29:57 - DEBUG - LocalScanner - Running a scan
2020/06/30 10:29:57 - DEBUG - seedsync - Persisting states to file
2020/06/30 10:29:57 - DEBUG - LocalScanner - Scan took 0.369s

@Jencryzthers Did you try this again? Are you using the docker image or the debian package? Also what operating system are you installing SeedSync on?

@ipsingh06 I use unRaid with docker
I wiped everything and tried again with the last version.
here is the new error log
2020/07/04 12:41:24 - INFO - seedsync.ServerActionHandler - Received a restart action
2020/07/04 12:41:24 - INFO - seedsync - Exiting Seedsync
2020/07/04 12:41:31 - INFO - RemoteScanner.RemoteScanner - Skipping remote scanfs installation: already installed
2020/07/04 12:41:38 - INFO - seedsync.AutoQueue - Auto queueing 'redacted'
2020/07/04 12:41:45 - INFO - seedsync.Controller - Received command Action.QUEUE for file redacted
2020/07/04 12:42:20 - ERROR - seedsync.Controller.Lftp - Lftp timeout exception
Traceback (most recent call last):
File "/app/python/lftp/lftp.py", line 128, in __run_command
self.__process.expect(self.__expect_pattern, timeout=self._timeout)
File "/usr/local/lib/python3.7/dist-packages/pexpect/spawnbase.py", line 344, in expect
timeout, searchwindowsize, async)
File "/usr/local/lib/python3.7/dist-packages/pexpect/spawnbase.py", line 372, in expect_list
return exp.expect_loop(timeout)
File "/usr/local/lib/python3.7/dist-packages/pexpect/expect.py", line 181, in expect_loop
return self.timeout(e)
File "/usr/local/lib/python3.7/dist-packages/pexpect/expect.py", line 144, in timeout
raise exc
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x15187f492790>
command: /usr/bin/lftp
args: ['/usr/bin/lftp', '-p', '22', '-u', 'jencryzthers,redacted', 'sftp://redacted.feralhosting.com']
buffer (last 100 chars): b' '
before (last 100 chars): b' '
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 20
child_fd: 30
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: re.compile(b'lftp jencryzthers@redacted.feralhosting.com:.*>')
2020/07/04 12:42:24 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - LftpJobStateParser error: Unable to parse line '0 (23%) 1.98M/s eta:2m [Receiving data]'
2020/07/04 12:42:24 - ERROR - seedsync.Controller.Lftp.LftpJobStatusParser - Status: jobs -v [0] queue (sftp://jencryzthers:redacted@redacted.feralhosting.com:22) -- 16.34 MiB/s
2020/07/04 12:42:25 - ERROR - seedsync.ControllerJob - Caught exception in job ControllerJob
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 90, in parse
statuses += self.__parse_jobs(lines)
File "/app/python/lftp/job_status_parser.py", line 472, in __parse_jobs
raise ValueError("Unable to parse line '{}'".format(line))
ValueError: Unable to parse line '2 (23%) 1.86M/s eta:2m [Receiving data]'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 322, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 94, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2020/07/04 12:42:26 - ERROR - seedsync - Caught exception
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 90, in parse
statuses += self.__parse_jobs(lines)
File "/app/python/lftp/job_status_parser.py", line 472, in __parse_jobs
raise ValueError("Unable to parse line '{}'".format(line))
ValueError: Unable to parse line '2 (23%) 1.86M/s eta:2m [Receiving data]'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python/seedsync.py", line 162, in run
controller_job.propagate_exception()
File "/app/python/common/job.py", line 77, in propagate_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 322, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 94, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2020/07/04 12:42:27 - INFO - seedsync.Controller - Exited controller

@Jencryzthers I'm not able to tell what's going wrong from those logs. Please do the following to enable verbose logging.
In your settings.cfg file, change verbose = False to verbose = True. Then run it again and send the logs. Thanks.

I haven't seen any more issues. I think this can be closed now. I think the issue @Jencryzthers is having is seemingly unrelated to this one as it throws a timeout exception prior to the parsing one.

@Jencryzthers Could you please create a new ticket with the verbose logs, thanks.

I have docker 0.8.6 and hitting the same error.
Restarting resolves the ap for a random period of time (could be minutes, could be hours)

Log Dump:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 322, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 94, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2022-06-04 12:59:42,020 - ERROR - seedsync (MainProcess/MainThread) - Caught exception
Traceback (most recent call last):
File "/app/python/lftp/job_status_parser.py", line 89, in parse
statuses += self.__parse_queue(lines)
File "/app/python/lftp/job_status_parser.py", line 482, in __parse_queue
raise ValueError("Unrecognized line '{}'".format(lines[0]))
ValueError: Unrecognized line '/downloads/AudioBook/1984 (George Orwell) - Audio Book/Audio Books - George Orwell - 1984 - 6 of 14.mp3.lftp 7813120-11719680 34.4 KiB/s'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/python/seedsync.py", line 164, in run
controller_job.propagate_exception()
File "/app/python/common/job.py", line 77, in propagate_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/app/python/common/job.py", line 44, in run
self.execute()
File "/app/python/controller/controller_job.py", line 29, in execute
self.__controller.process()
File "/app/python/controller/controller.py", line 202, in process
self.__update_model()
File "/app/python/controller/controller.py", line 295, in __update_model
lftp_statuses = self.__lftp.status()
File "/app/python/lftp/lftp.py", line 322, in status
statuses = self.__job_status_parser.parse(out)
File "/app/python/lftp/job_status_parser.py", line 94, in parse
raise LftpJobStatusParserError("Error parsing lftp job status")
lftp.job_status_parser.LftpJobStatusParserError: Error parsing lftp job status
2022-06-04 12:59:45,037 - ERROR - seedsync.Controller.Lftp (MainProcess/ControllerJob) - Lftp timeout exception
Traceback (most recent call last):
File "/app/python/lftp/lftp.py", line 128, in __run_command
self.__process.expect(self.__expect_pattern, timeout=self.__timeout)
File "/usr/local/lib/python3.8/site-packages/pexpect/spawnbase.py", line 343, in expect
return self.expect_list(compiled_pattern_list,
File "/usr/local/lib/python3.8/site-packages/pexpect/spawnbase.py", line 372, in expect_list
return exp.expect_loop(timeout)
File "/usr/local/lib/python3.8/site-packages/pexpect/expect.py", line 181, in expect_loop
return self.timeout(e)
File "/usr/local/lib/python3.8/site-packages/pexpect/expect.py", line 144, in timeout
raise exc
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x14b62a98ca90>
command: /usr/bin/lftp
args: ['/usr/bin/lftp', '-p', '22', '-u', ‘XXX,XXX’, 'sftp://XXX’]
buffer (last 100 chars): b'll) - Audio Book/Audio Books - George Orwell - 1984 - 6 of 14.mp3.lftp 7813120-11719680 34.4 KiB/s\r\n'
before (last 100 chars): b'll) - Audio Book/Audio Books - George Orwell - 1984 - 6 of 14.mp3.lftp 7813120-11719680 34.4 KiB/s\r\n'
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 714
child_fd: 68
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: re.compile(b'lftp XXX@XXXX:.>’)
2022-06-04 12:59:48,092 - ERROR - seedsync.Controller.Lftp (MainProcess/ControllerJob) - Lftp timeout exception
Traceback (most recent call last):
File "/app/python/lftp/lftp.py", line 128, in __run_command
self.__process.expect(self.__expect_pattern, timeout=self.__timeout)
File "/usr/local/lib/python3.8/site-packages/pexpect/spawnbase.py", line 343, in expect
return self.expect_list(compiled_pattern_list,
File "/usr/local/lib/python3.8/site-packages/pexpect/spawnbase.py", line 372, in expect_list
return exp.expect_loop(timeout)
File "/usr/local/lib/python3.8/site-packages/pexpect/expect.py", line 181, in expect_loop
return self.timeout(e)
File "/usr/local/lib/python3.8/site-packages/pexpect/expect.py", line 144, in timeout
raise exc
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x14b62a98ca90>
command: /usr/bin/lftp
args: ['/usr/bin/lftp', '-p', '22', '-u', ‘XXX,XXX’, 'sftp://XXX’]
buffer (last 100 chars): b'll) - Audio Book/Audio Books - George Orwell - 1984 - 6 of 14.mp3.lftp 7813120-11719680 34.4 KiB/s\r\n'
before (last 100 chars): b'll) - Audio Book/Audio Books - George Orwell - 1984 - 6 of 14.mp3.lftp 7813120-11719680 34.4 KiB/s\r\n'
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 714
child_fd: 68
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: re.compile(b'lftp XXXX@XXXX:.>’)
Exception in thread ControllerJob:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/pexpect/pty_spawn.py", line 23, in _wrap_ptyprocess_err
yield
File "/usr/local/lib/python3.8/site-packages/pexpect/pty_spawn.py", line 328, in close
self.ptyproc.close(force=force)
File "/usr/local/lib/python3.8/site-packages/ptyprocess/ptyprocess.py", line 400, in close
raise PtyProcessError('Could not terminate the child.')
ptyprocess.util.PtyProcessError: Could not terminate the child.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/app/python/common/job.py", line 56, in run
self.cleanup()
File "/app/python/controller/controller_job.py", line 34, in cleanup
self.__controller.exit()
File "/app/python/controller/controller.py", line 207, in exit
self.__lftp.exit()
File "/app/python/lftp/lftp.py", line 404, in exit
self.__process.close(force=True)
File "/usr/local/lib/python3.8/site-packages/pexpect/pty_spawn.py", line 328, in close
self.ptyproc.close(force=force)
File "/usr/local/lib/python3.8/contextlib.py", line 131, in exit
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.8/site-packages/pexpect/pty_spawn.py", line 25, in _wrap_ptyprocess_err
raise ExceptionPexpect(*e.args)
pexpect.exceptions.ExceptionPexpect: Could not terminate the child.


raftstore-v2: fix metrics and perf context

What is changed and how it works?
Issue Number: Ref #12842
What's Changed:
This PR fixes several bugs and metrics:
- Now waterfall timer will be reset in before_write, the goal is to solve
    the confusion that stall writes can pollute the whole waterfall metrics.
- Perf context is changed not to be associated with engine instance. Perf
      context is thread local and instance independent under the hook.
- Fix flushed index advance failure due to suspicious flush.
- Support print long uncommitted logs and fix incorrect commit time

Check List
Tests

Unit test
Integration test

Release note
None

/merge

/merge

/merge


Generics

I couldn't find anything on generics, is that a language concept yet? Generally something that happens later though

Generics are planned, but not implemented yet.
I hope that some day, I will add something like this to stdlib:
class List[T]:
    ptr: byte*
    length: long
    alloc: long

    @inline
    def append(self, elem: T) -> None:
        ...

    def extend(self, elems: T*, count: long) -> None:
        ...

    def end(self) -> T*:
        return &self->ptr[self->length]

    # ...other methods...

With this in stdlib, you could write:
import "stdlib/list.jou"

def main() -> int:
    nums = List[int]{}
    nums.append(1)
    nums.append(2)
    nums.append(3)

    # Output: 1
    # Output: 2
    # Output: 3
    for p = nums.ptr; p < nums.end(); p++:
        printf("%d\n", nums.ptr[i])

    return 0

There is a reason why append() must be inline function. The element being appended has type T and it will be a stack variable. If the compiler generates a function named append() that is not inline, it will use a different amount of stack space depending on what type T is. This is considered bad style in C because it easily leads to stack overflow vulnerabilities. It's not supported at all in Jou, and I don't want to add it just for this.
If append() is not inlined, List.append() needs to take a pointer, so it will be annoying but not impossible to use. Inline functions don't exist yet in Jou, so the initial implementation of generics will probably have this annoyance.

Ok! I was actually asking because I noticed that you use Pointers a lot instead of a List type and after not finding generics supposed that was the reason why :D. How hard would inlining be to implement in jou?

For inlining, the compiler needs to:

know which functions are inlined
generate no code for definitions of inline functions
basically copy/paste (that is, inline) contents of inline functions when they are called, with new dummy variables for everything that is local inside the inline function

"3." is the most difficult, of course, but everything seems doable.

Generics would also be useful for plain old functions. For example, there is currently no sort() function in standard library, although that would also require some kind of callback functions to define how to compare elements.

With generics should probably also come compile time type checking

Types are already checked at compile time. Jou's type checking is very similar to C: you can do dumb things with void* pointers (void* means "any pointer whatsoever"), but it's quite safe otherwise.

I meant it as in
foo = Variant[Bar | Baz](xyz)
if constrained(foo, Variant[Bar]):
    ...
elif constrained(foo, Variant[Baz]):
    ...

It seems like you are talking about tagged unions, like Rust's union, or Python's union types together with isinstance(). In my mind they don't really have anything to do with generics.
At some point I was going to add tagged unions, but now I feel like it's a Rust feature that doesn't really belong in Jou. I might change my mind eventually though.

I assume Jou doesn't do errors as a type then? :D

You're right. Jou is a simple language, not a full-featured language.

Actually, how do you handle exceptions/errors in Jou? I can't find anything about it in the docs

I created #533.
(I'm on IRC and Discord if you want to talk about this elsewhere)
I will go to sleep soon. I might have more time tomorrow... :)


Is this your channel?

https://youtu.be/rscl9VwwnQg?si=HpZAMZO0eQygM56q

Hi gerrardxcc,
Thanks for reaching out! To clarify, that YouTube channel and associated GitHub repository aren't mine. Unfortunately, it seems someone took inspiration from my project and created their own version without proper attribution. 😔

That's no good, you should report it to the YouTube and take it down.