Rancher HA - Failed to open project management: Can not create a stack, check API key Rancher

Rancher Version:
v1.2.0-pre3
Docker Version:
1.10.3
OS and where are the hosts located? (cloud, bare metal, etc):
RHEL 7.3
Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB)
HA Rancher in testing on single node, with external DB
Environment Type: (Cattle/Kubernetes/Swarm/Mesos)
Cattle
Steps to Reproduce:

http://docs.rancher.com/rancher/v1.2/en/installing-rancher/installing-server/multi-nodes/
docker run -d --restart=always --name rancher-server -p 8080:8080 -e CATTLE_DB_CATTLE_MYSQL_HOST=$ip_server -e CATTLE_DB_CATTLE_MYSQL_PORT=3306 -e CATTLE_DB_CATTLE_MYSQL_NAME=cattle -e CATTLE_DB_CATTLE_USERNAME=cattle -e CATTLE_DB_CATTLE_PASSWORD=cattle rancher/server:v1.2.0-pre3
docker run -d --restart=always --name nginx_proxy -p 80:80 -p 443:443 --link=rancher-server nginx_proxy

Results:
command "docker -f logs rancher-ha" shows the error messages:
time = "2016-10-18T09: 31: 13Z" level = error msg = "Failed to open project management: Can not create a stack, check API key [279E88DEA199DEA5E619] is [http://<IP_ADDRESS>:18080/v1 / projects / 1A7 / schemas] "
time = "2016-10-18T09: 31: 13Z" level = fatal msg = "Failed to read project: Can not create a stack, check API key [279E88DEA199DEA5E619] is [http://<IP_ADDRESS>:18080/v1/ projects / 1A7 / schemas] "
time = "2016-10-18T09: 31: 13Z" level = info msg = "Can not launch agent right now: exit status 1"
Expected:
No show fatal errors

I can confirm this issue for:

Upgrading (a fresh) 1.1.4 to 1.2.0-pre3
Installing 1.2.0-pre3 from scratch in ha-mode

Tk's ymc-dabe. I will test the new version and send feedback.

@ymc-dabe has some version (same old) in the HA to work's fine?

@kraucer The current stable (1.1.4) is working just great in ha-mode.
btw.: 1.2.0-pre4-rc4 was failing in ha-mode with different errors. I'll try to check if 1.2.0-pre4-rc5 works (or just wait for 1.1.2 to get released, which should be in a few days).

When I run the script in version 1.1.4 this error occurred:
time="2016-10-19T08:21:50Z" level=info msg="Waiting for server to be available" component=cert
time="2016-10-19T08:21:50Z" level=info msg="Can not launch agent right now: Server not available at http://<IP_ADDRESS>:18080/ping:" component=service
Note that it looks for the agent on the IP <IP_ADDRESS> when it should be the IP <IP_ADDRESS>
[root@dockersrv kraucer]# ifconfig
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
inet <IP_ADDRESS>  netmask <IP_ADDRESS>  broadcast <IP_ADDRESS>
Do you have any tips or additional script to share with me?
Best regards,

@kraucer I followed http://docs.rancher.com/rancher/v1.1/en/installing-rancher/installing-server/multi-nodes/ for a three nodes setup and it worked out of the box with 1.1.4.
In a three node setup you need to run the ha-script on three different servers. The waiting message is normal during startup is normal and will go away once all the zookeeper instances start seeing each other and the admin UI on port 18080 gets available. Once it is reachable you can check the initialization of the "System HA"-Stack on port 18080. After the initialization is done and the "System HA" stack is ready you will be able to access rancher on port 80 (hence 18080 will still remain working).

@ymc-dabe test in version 1.1.4, error:
time="2016-10-19T08:21:50Z" level=info msg="Waiting for server to be available" component=cert
time="2016-10-19T08:21:50Z" level=info msg="Can not launch agent right now: Server not available at http://<IP_ADDRESS>:18080/ping:" component=service
Note that it looks for the agent on the IP <IP_ADDRESS> when it should be the IP <IP_ADDRESS>
Same error reported here: https://forums.rancher.com/t/rancher-ha-can-not-launch-agent-right-now-server-not-available-at-http-172-19-10-2-18080-ping/3109

Closing due to age, and the way HA works was completely changed after 1.1.x.


Access permission issue

Thank you for developing the Twitter bot, which allows me to quickly get started with such a great tool without any programming background. However, it seems that some problems have occurred during my use:
warning: `tweet_bot` (bin "tweet_bot") generated 3 warnings
    Finished release [optimized] target(s) in 5m 08s
     Running `target/release/tweet_bot`
Error: TweetError { kind: TweetError(TwitterError({"perf": "7626143928", "content-type": "application/json;charset=utf-8", "cache-control": "no-cache, no-store, max-age=0", "content-length": "311", "x-transaction-id": "3fccce770c5517c1", "x-response-time": "161", "x-connection-hash": "c35e1e9e80bfb05113707e47427cead188bfcd45", "date": "Wed, 15 Nov 2023 22:29:18 GMT", "server": "tsa_k"}, TwitterErrors { errors: [TwitterErrorCode { message: "You currently have access to a subset of Twitter API v2 endpoints and limited v1.1 endpoints (e.g. media post, oauth) only. If you need access to this endpoint, you may need a different access level. You can learn more here: https://developer.twitter.com/en/portal/product", code: 453 }] })), location: Location { file: "src/utils.rs", line: 6, col: 33 } }

I searched and it seems to be because the free Twitter API V2 I'm using doesn't have this permission. Can I use API V1 instead?

Hi @aihoom. Twitter APIv2 is not supported by this bot it still runs on APIv1. Since Twitter has made their APIs paid I am not really getting interest to update this to APIv2

I took a look at the Twitter API2 documentation, and it seems that free developer accounts require authorization to fetch data from specific users. However, I still appreciate you for developing such a useful application

There is an Oauth file and secret which can be generated on twitter with a free account. This should let tweets be pulled with a free account but this authentication is not included in the .env file. Can the code be modified to accomplish this?


Error when loading the model saved

Hi,
I modified your code to train a model with my own dataset, and I am trying to load the model saved as "latest.pth" to do some tests. However, I am getting this error:
AttributeError: 'DenseNet' object has no attribute 'copy'
The code I use to load the model is:
net.load_state_dict(torch.load(checkpoint, map_location=lambda storage, loc: storage))
where checkpoint is the path to "latest.pth"
Any help would be appreciated.
Thanks

Hi,
I modified your code to train a model with my own dataset, and I am trying to load the model saved as "latest.pth" to do some tests. However, I am getting this error:
AttributeError: 'DenseNet' object has no attribute 'copy'
The code I use to load the model is:
net.load_state_dict(torch.load(checkpoint, map_location=lambda storage, loc: storage))
where checkpoint is the path to "latest.pth"
Any help would be appreciated.
Thanks

I think you should load the model directly instead of the state_dict.
The following worked for me:
model = torch.load("./work/densenet.base/latest.pth")


Random minor cleanups

My only question is whether Uint8Array and ArrayBuffer are mutually interchangeable enough that we don't care about just unioning their types.

Uint8Array is a view on an ArrayBuffer but I don't think the types are mutually interchangeable


[#162475628] Add retry logic in collection and eventually stop the worker

What?
We want to make the collection of metrics more resilient to errors and
failures, including a simple instance reboot and failover.
We add logic to retry the collection of metrics if they fail with exponential
backoff, and to stop the collection worker if the error persists after
several retries.
We also refactor the global scheduler to simplify it by not using a external
scheduler, but normal go routines. This allows us to remove unnecessary locks.
To test resiliance to reboots, we extended the broker to allow restart
the instance using an update configuration.
How to review?
Review first:  https://github.com/alphagov/paas-rds-broker/pull/94
Code review.
Tests should pass.
After https://github.com/alphagov/paas-rds-broker/pull/94 has been merged,
remove the latest commit and do dep ensure -update instead to pull the
latest changes.
Who?
Not me

I fixed the comments above


feat(Meter): new component

🔗 Linked issue

❓ Type of change

[ ] 📖 Documentation (updates to the documentation or readme)
[ ] 🐞 Bug fix (a non-breaking change that fixes an issue)
[ ] 👌 Enhancement (improving an existing functionality)
[x] ✨ New feature (a non-breaking change that adds functionality)
[ ] 🧹 Chore (updates to the build process or auxiliary tools and libraries)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description
This component is basically <meter> with some amphetamines. It follows the same principle of #697:

Allows the use of min and max values.
Percent indicator and its slot
Label and its slot
ABLE TO GROUP THEM

You read that right: YOU CAN GROUP THEM.

Things to note

No optimum|high|low props: you can change the bar behavior with Vue already. Below a value? Turn it red, or change the label.
The <UMeterGroup> reuses the <Meter> but uses the default slot to inject the cloned VNodes (which are edited to meet the group criteria). I could done a normal .vue file but I didn't find how to override the default slot being passed without doing h(...) into oblivion.

📝 Checklist

[x] I have linked an issue or discussion.
[x] I have updated the documentation accordingly.

All deployments are failing and I can't check why. The only clue is that on my machine it's LandingCard.vue the culprit, which is from @nuxt-pro/ui or something like that.

It seems the Vercel build breaks because of this line: https://github.com/nuxt/ui/pull/708/files#diff-041bae31f76c99f1b1872d3aaf8b763b4363e117deb1cad1141d90a8d3639d0fR131. I didn't get what's going on with LandingCard? Would you mind opening a new issue for this?
[09:43:08.894] [error] [nuxt] [request error] [unhandled] [500] can not read a block mapping entry; a multiline key may not be an implicit key (18:1)
[09:43:08.894]
[09:43:08.894]  15 | <UMeter :value="12" color="oran ...
[09:43:08.894]  16 | <UMeter :value="37" color="yell ...
[09:43:08.894]  17 | <UMeter :value="52" color="gree ...
[09:43:08.894]  18 |
[09:43:08.895] ------^

I think I’ll try to reload and reinstall the dev branch all over again.

No luck and I can’t see why it’s failing.

It seems the Vercel build breaks because of this line: https://github.com/nuxt/ui/pull/708/files#diff-041bae31f76c99f1b1872d3aaf8b763b4363e117deb1cad1141d90a8d3639d0fR131. I didn't get what's going on with LandingCard? Would you mind opening a new issue for this?
[09:43:08.894] [error] [nuxt] [request error] [unhandled] [500] can not read a block mapping entry; a multiline key may not be an implicit key (18:1)
[09:43:08.894]
[09:43:08.894]  15 | <UMeter :value="12" color="oran ...
[09:43:08.894]  16 | <UMeter :value="37" color="yell ...
[09:43:08.894]  17 | <UMeter :value="52" color="gree ...
[09:43:08.894]  18 |
[09:43:08.895] ------^

Update: I just forgot to use dev:prepare. Error dissapeared after that. lol.

Going to give this a new change with a new up-to-date PR later.


No module named 'keras.legacy'

Hello Benjamin,
Good afternoon. I'm experiencing ModuleNotFoundError: no module named "keras.legacy" when running a single line command for prediction. Which version of Keras should I use to solve this problem? The latest version of Keras doesn't seem to have keras.legacy anymore.
Best,
Siyan

Hi!
please use the same libraries (keras, etc.) as for SynthSeg https://github.com/BBillot/SynthSeg. This should solve your problem :)

I used the SythSeg requirement_python3.8.txt on a linux system, but there is still an issue during the installation process:
ERROR: Failed cleaning build dir for numpy
Failed to build numpy
ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
Could you help with this? :)
Thanks,

Hey I've updated the installation instructions. Can you please let me know if it works for you for a fresh installation ? Thanks!


Error on serverpod generate after changing project name

Describe the bug
I renamed my serverpod directories and renamed the project, using a global search/replace in all files. I then ran generate_all in the serverpod project (based on git main version).
After that, when I run serverpod generate, I get this:
Analyzing the endpoints.
Yikes! It is possible that this error is caused by an internal issue with the Serverpod tooling. We would appreciate if you filed an issue over at Github. Please include the stack trace below and describe any steps you did to trigger the error.
https://github.com/serverpod/serverpod/issues
Null check operator used on a null value
#0      new TypeDefinition.fromDartType (package:serverpod_cli/src/generator/types.dart:75:53)
#1      EndpointsAnalyzer.analyze (package:serverpod_cli/src/analyzer/dart/endpoints_analyzer.dart:140:42)
<asynchronous suspension>
#2      performGenerate (package:serverpod_cli/src/generator/generator.dart:46:19)
<asynchronous suspension>
#3      _main (file:///home/luke/Work/myproj/vendor/serverpod/tools/serverpod_cli/bin/serverpod_cli.dart:282:7)
<asynchronous suspension>
#4      main.<anonymous closure> (file:///home/luke/Work/myproj/vendor/serverpod/tools/serverpod_cli/bin/serverpod_cli.dart:44:9)
<asynchronous suspension>
#5      main (file:///home/luke/Work/myproj/vendor/serverpod/tools/serverpod_cli/bin/serverpod_cli.dart:41:3)
<asynchronous suspension>

package:serverpod_cli/src/analyzer/dart/endpoints_analyzer.dart:140 is:
                    type: TypeDefinition.fromDartType(param.type),

I changed a bunch of things, and now this command works fine -- no idea how I fixed it, or what the problem was...


fixed scaladoc for  new validation options

Coverage remained the same at 46.671% when pulling 9e1e23083e88ce938afff90250a96a07221f9ecc on more_validation_options into 87376ceb71d857cf41eb6926180db87300badf92 on master.


Request for suggestions to deal with decorrelation in interferogram

Hi there,
After running all the steps to interferogram, I generated correlation and interferogram maps
For phase and correlation, I used the parameters as followed:
topo = sbas.get_topo()
data = sbas.open_data()
intensity = sbas.multilooking(np.square(np.abs(data)), wavelength=400, coarsen=(4,16))
phase = sbas.phasediff(pairs, data, topo)
phase = sbas.multilooking(phase, wavelength=400, coarsen=(4,16))
corr = sbas.correlation(phase, intensity)
phase_goldstein = sbas.goldstein(phase, corr, 32)
intf = sbas.interferogram(phase_goldstein)
decimator = sbas.decimator(resolution=RESOLUTION, grid=intf)
tqdm_dask(result := dask.persist(decimator(corr), decimator(intf)), desc='Compute Phase and Correlation')
corr, intf = [grid[0] for grid in result]
intf
I am wondering if the interferogram could be improved tweaking some of these parameters. Looking forward to your suggestions and directions. Indeed most of the part of this sentinel 1 scene is offshore so thats understandable that we have zero amplitude there but part of the interferogram where the seismic event happen still have some decorrelation around.
Thank you:) dear  Alexey Pechnikov for your patience and advices, finally I have made it to an interferogram...

The interferogram appears usable, with fringes clearly detected. To enhance the results, consider applying a land mask to exclude water surface areas from the interferogram. It can enhance and speedup the unwrapping process.


Conduct gold-standard validation + weight learning for V2G and L2G scoring

Preliminary work is here: https://github.com/opentargets/genetics-scoring-validation

Results of a preliminary analysis are here: https://docs.google.com/presentation/d/1-FRpBmiYnMgxOTJEKdrS16YIQ_uD0oN2SAXyB38AQqs/edit#slide=id.g5482455acf_0_5
Decision was to wait until we have examined the gold standards more thoroughly before changing weights in production.


start fail with error code:10001, message: An unknow error occurred.

after enabling diego on an app, tried starting up the app, got this error messages.
Starting app spring-music in org diego / space diego as admin...
FAILED
Server error, status code: 500, error code: 10001, message: An unknown error occurred.

Hi, @tee33,
Thanks for the report. Unfortunately, that's not a lot of information to go on.

What versions of CF, Diego, Garden-Linux, and ETCD releases do you have deployed?
Could you provide logs from at least the Cloud Controller and the CC-Bridge services that cover the time window in which you observed the error? The app guid of the app you're switching over to run on Diego would also be helpful for identifying the exact error in the logs.

I have seen this generic error myself when CC can't find an nsync-listener service, and have filed https://www.pivotaltracker.com/story/show/104066600 as at least a placeholder to give a more informative error. I'm not sure that that's causing your error, though.
Thanks again,
Eric, CF Runtime Diego PM

Hello Eric,
It's working now.  the cc_bridge wasn't started.   Somehow, when doing deployment, cc_bridge did not get started. I have to do manually, not sure if this is something that automatically startup.

Hi, @tee33,
All the cc_bridge components should start up automatically after BOSH deploys them (as should the components on all the CF and Diego VMs), so that does sound strange. The Monit logs from that VM, located at /var/vcap/monit/monit.log, may contain enough information to determine why they didn't start up as expected. Is there any chance you accidentally ran bosh stop against that cc_bridge VM?
Thanks,
Eric


Replace CentOS Stream 8 with AlmaLinux 8

Looks like all the changes I made were already made in #9, so nothing else needs to be added :)


Pin Dependencies

This Pull Request renovates the package group "Pin Dependencies".

snyk (devDependencies): from ^1.70.2 to 1.70.2
weather-js (dependencies): from ^2.0.0 to 2.0.0
superagent (dependencies): from ^3.8.2 to 3.8.2
sqlite (dependencies): from ^2.9.1 to 2.9.1
firebase (dependencies): from ^4.11.0 to 4.11.0
discord.js-commando (dependencies): from ^0.9.0 to 0.9.0
discord.js (dependencies): from ^11.3.2 to 11.3.2

Important: Renovate will wait until you have merged this Pin request before creating PRs for any upgrades. If you do not wish to pin anything, please update your config accordingly instead of leaving this PR open.

This PR has been generated by Renovate Bot.

Lock file problem
Renovate failed when attempting to generate a lock file. This is usually happens when you have private modules but have not added configuration for private module support. It is strongly recommended that you do not merge this PR as-is.
Renovate will not retry generating a lockfile for this PR unless either (a) the package.json in this branch needs updating, or (b) you manually delete this PR so that it can be regenerated.
The output from stderr is included below:
package-lock.json
'ls' n'est pas reconnu en tant que commande interne
ou externe, un programme ex�cutable ou un fichier de commandes.


Added missing future requirement

This PR adds a missing requirement for ligo-gracedb-2.0.1.

Checklist

[x] Used a fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


FollowPath taking sporadic breaks to respin pokestops

Expected Behavior
Bot would walk from place to place as it used to not dilly dallying. I'm using FollowPath
Actual Behavior
It walks, but incredibly slowly/appears to take breaks at times. I'm not sure where the slow down is. Here are 2 example of stops that it should just collect once and keep walking past(as they have many times before), but for some reason the bot is just stopping for 5 mins to spin it again with the newest version. This doesn't appear to happen with every stop though...
Example 1:

[22:57:41]  Now at Pokestop: STOP1
[22:57:41]  Spinning ...
[22:57:41]  Loot:
[22:57:41]  50 xp
[22:57:41]  - 2x Pokeball (Total: 64)
[22:57:41]  - 1x Razz Berry (Total: 26)
[22:57:41] PokeStop on cooldown. Time left: 4 minutes, 59 seconds
[22:57:55]  -- Discarded 1x Razz Berry (keeps only 25 maximum)
[22:59:45]  [x] Current egg hatches in 2.78 km
[23:02:56]  Now at Pokestop: STOP1
[23:02:56]  Spinning ...
[23:02:56]  Loot:
[23:02:56]  50 xp
[23:02:56]  - 2x Pokeball (Total: 66)
[23:02:56]  - 1x Razz Berry (Total: 26)
[23:02:56] PokeStop on cooldown. Time left: 4 minutes, 59 seconds

Example 2:

[23:17:00]  Now at Pokestop: STOP2
[23:17:00]  Spinning ...
[23:17:00]  Loot:
[23:17:00]  50 xp
[23:17:00]  - 2x Pokeball (Total: 67)
[23:17:00]  - 1x Greatball (Total: 45)
[23:17:00] PokeStop on cooldown. Time left: 4 minutes, 59 seconds
[23:17:15]  -- Discarded 2x Pokeball (keeps only 65 maximum)
[23:18:29] Something rustles nearby!
[23:18:29]  A Wild Weedle appeared! [CP 10] [Potential 0.69]
[23:18:29] IV [Attack/Defense/Stamina] = [4/13/14]
[23:18:33] Using Pokeball (chance: 100.00%)... (64 left!)
[23:18:33]  Captured Weedle! [CP 10] [Potential 0.69] [4/13/14] [+210 exp]
[23:18:49]  Releasing Weedle with CP 10 and IV 0.69. Matching release rule: CP < 200 AND IV < 0.85.
[23:18:49]  Exchanging Weedle [CP 10] [Potential 0.69] for candy!
[23:19:34] Something rustles nearby!
[23:19:35]  A Wild Pidgey appeared! [CP 225] [Potential 0.38]
[23:19:35] IV [Attack/Defense/Stamina] = [6/5/6]
[23:19:37] Using Pokeball (chance: 40.01%)... (63 left!)
[23:19:38]  Captured Pidgey! [CP 225] [Potential 0.38] [6/5/6] [+210 exp]
[23:20:15]  [x] Current egg hatches in 2.57 km
[23:22:18]  Now at Pokestop: STOP 2
[23:22:18]  Spinning ...
[23:22:19]  Loot:
[23:22:19]  50 xp
[23:22:19]  - 3x Pokeball (Total: 66)
[23:22:19] PokeStop on cooldown. Time left: 4 minutes, 59 seconds

Steps to Reproduce
    "tasks": [       {         "type": "HandleSoftBan"       },       {         "type": "CollectLevelUpReward"       },       {         "type": "IncubateEggs",         "config": {           "longer_eggs_first": true         }       },       {         "type": "TransferPokemon"       },       {         "type": "RecycleItems",         "config": {           "item_filter": {             "Pokeball":       { "keep" : 65 },             "Greatball":       { "keep" : 70 },             "Ultraball":       { "keep" : 75 },             "Potion":         { "keep" : 0 },             "Super Potion":   { "keep" : 0 },             "Hyper Potion":   { "keep" : 40 },             "Revive":         { "keep" : 25 },             "Razz Berry":     { "keep" : 25 }           }         }       },       {         "type": "CatchVisiblePokemon"       },       {         "type": "CatchLuredPokemon"       },       {         "type": "SpinFort"       },       {         "type": "FollowPath",         "config": {           "path_mode": "loop",           "path_file": "configs/path.json"         }       }     ],     "map_object_cache_time": 5,     "forts": {       "avoid_circles": true,       "max_circle_size": 10     },     "websocket_server": false,     "walk": 2.2,     "action_wait_min": 1,     "action_wait_max": 4,     "debug": false,     "test": false,     "health_record": true,     "location_cache": true,     "distance_unit": "km",     "reconnecting_timeout": 15,
Only new thing is: "map_object_cache_time": 5 being added, and max_steps being taken out
Other Information
OS: OSX
Git Commit: (run 'git log -n 1 --pretty=format:"%H"' and paste it here)
I believe it was working fine on: https://github.com/PokemonGoF/PokemonGo-Bot/pull/2269
I am now on https://github.com/PokemonGoF/PokemonGo-Bot/pull/2416

To narrow it down further I checked out 1154c083d14e937b044d007fe2c635ef481523d3 (https://github.com/PokemonGoF/PokemonGo-Bot/pull/2342) and its working fine. So something something in those 7 commits

To narrow it down further I checked out 1154c083d14e937b044d007fe2c635ef481523d3 (https://github.com/PokemonGoF/PokemonGo-Bot/pull/2342) and its working fine. So something something in those 7 commits


simple_retry_responses not deallocated

https://github.com/apache/trafficserver/blob/075dcda4ae01338b155846917db6035fd0788d5a/src/proxy/ParentSelection.cc#L869
simple_retry_responses is not deallocated. Should be the same as unavailable_server_retry_responses

Where is simple_retry_responses in the code?

Where is simple_retry_responses in the code?

Sorry, it should be simple_server_retry_responses. I have updated the description now


(3.6.0) NVIDIA GPU nodes fail to start with custom AMI built from DLAMI

Bug description
ParallelCluster version 3.6.0 introduced the installation of the NVIDIA daemon nvidia-persistenced. The daemon has the goal to persist the NVIDIA device file(s) in order to save the latency incurred by repetitive device initialization.
In the DLAMI, since version published from 2022-12-20, the same service was deployed with a different name (nvidia_gpu_settings_service.service).
In a PCluster 3.6.0 custom AMI, built from the DLAMI, the two services have a conflict and they are causing an error in the bootstrap procedure of the nodes. The cluster detects the error as a compute node bootstrap issue and after a number of consecutive nodes failing at bootstrap time, it enters protected status.
When the issue occurs, the nodes are not able to join the cluster and in the CloudWatch LogGroup of the cluster the following lines can be found:
STDERR: Error: 'systemctl start nvidia-persistenced.service' failed with
'Job for nvidia-persistenced.service failed because the control process exited with error code.
See "systemctl status nvidia-persistenced.service" and "journalctl -xe" for details.'.
     ---- End output of "bash"  ----
Ran "bash"  returned 1

Affected versions (OSes, schedulers)
ParallelCluster 3.6.0 and custom AMI built from DLAMI when the instance type has NVIDIA GPU.
Solution
You can find a detailed explanation and the mitigation of the problem here.

The issue has been fixed as part of 3.6.1.


Cannot install PojoMaker 0.3.3 plugin into Apache NetBeans IDE 11.3

Environment:
Product Version: Apache NetBeans IDE 11.3
Updates: Updates available to version NetBeans 8.2 Patch 2
Java: 1.8.0_271; Java HotSpot(TM) 64-Bit Server VM 25.271-b09
Runtime: Java(TM) SE Runtime Environment 1.8.0_271-b09
Description:
I cannot install 0.3.3 version of your PojoMaker plugin into Apache NetBeans IDE 11.3 due to not satisfied dependencies. Installation into Apache NetBeans IDE 12.6 works fine. Can you please compile your plugin against RELEASE110 NetBeans version to make it compatible with 11.3?

please check that release
https://github.com/ranSprd/netbeans-pojomaker/releases/tag/version-0.4.0-nb-11-pre-release

I tried but also this version results in the same dependency errors. :-1:


REFACTOR-#5951: remove code duplication for to_pickle_distributed

What do these changes do?

[x] first commit message and PR title follow format outlined here

NOTE:  If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

[x] passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] signed commit with git commit -s
[x] Resolves #5951
[ ] tests added and passing
[x] module layout described at docs/development/architecture.rst is up-to-date

@dchigarev thanks for review! It's ready for another one.

@YarShev thanks for review! It's ready for another one.

@YarShev could we merge it?


SMA-MOD + SAB = ModuleNotFoundError: No module named 'tmdbsimple'

Hi there,
Updated the container with your mod, went smooth but failed conversion with ModuleNotFoundError: No module named 'tmdbsimple'.

Actually it seems to be all of them lol
ModuleNotFoundError: No module named 'mutagen'

What command are you running? Are you using the virtual environment?

Yes, I'm using the venv. I've tried deleting it and rebuilding the container with the mod. Still the same issue though.
Here is what happens during deployment:
[mod-init] Curl/JQ was not found on this system for Docker mods installing
Get:1 http://ppa.launchpad.net/jcfp/nobetas/ubuntu bionic InRelease [15.4 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:3 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu bionic InRelease [15.4 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:6 http://ppa.launchpad.net/jcfp/nobetas/ubuntu bionic/main Sources [2,747 B]
Get:7 http://ppa.launchpad.net/jcfp/nobetas/ubuntu bionic/main amd64 Packages [2,122 B]
Get:8 http://archive.ubuntu.com/ubuntu bionic/universe Sources [11.5 MB]
Get:9 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu bionic/main Sources [2,306 B]
Get:10 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu bionic/main amd64 Packages [1,898 B]
Get:11 http://archive.ubuntu.com/ubuntu bionic/multiverse Sources [216 kB]
Get:12 http://archive.ubuntu.com/ubuntu bionic/main Sources [1,063 kB]
Get:13 http://archive.ubuntu.com/ubuntu bionic/restricted Sources [5,823 B]
Get:14 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:15 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1,344 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main Sources [641 kB]
Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/universe Sources [573 kB]
Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Sources [25.0 kB]
Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Sources [13.2 kB]
Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [452 kB]
Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2,583 kB]
Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2,183 kB]
Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.6 kB]
Get:26 http://archive.ubuntu.com/ubuntu bionic-security/main Sources [318 kB]
Get:27 http://archive.ubuntu.com/ubuntu bionic-security/universe Sources [347 kB]
Get:28 http://archive.ubuntu.com/ubuntu bionic-security/multiverse Sources [5,571 B]
Get:29 http://archive.ubuntu.com/ubuntu bionic-security/restricted Sources [21.6 kB]
Get:30 http://archive.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,412 kB]
Get:31 http://archive.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [423 kB]
Get:32 http://archive.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,152 kB]
Get:33 http://archive.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [24.7 kB]
Fetched 37.4 MB in 5s (8,013 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
curl is already the newest version (7.58.0-2ubuntu3.13).
The following additional packages will be installed:
  libjq1 libonig4
The following NEW packages will be installed:
  jq libjq1 libonig4
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 276 kB of archives.
After this operation, 930 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB]
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 3.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, <> line 3.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin:
Fetched 276 kB in 0s (1,101 kB/s)
Selecting previously unselected package libonig4:amd64.
(Reading database ...

Preparing to unpack .../libonig4_6.7.0-1_amd64.deb ...
Unpacking libonig4:amd64 (6.7.0-1) ...
Selecting previously unselected package libjq1:amd64.
Preparing to unpack .../libjq1_1.5+dfsg-2_amd64.deb ...
Unpacking libjq1:amd64 (1.5+dfsg-2) ...
Selecting previously unselected package jq.
Preparing to unpack .../jq_1.5+dfsg-2_amd64.deb ...
Unpacking jq (1.5+dfsg-2) ...
Setting up libonig4:amd64 (6.7.0-1) ...
Setting up libjq1:amd64 (1.5+dfsg-2) ...
Setting up jq (1.5+dfsg-2) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
[mod-init] Attempting to run Docker Modification Logic
[mod-init] Applying mdhiggins/sma-mod:latest files to container
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-envfile: executing...
[cont-init.d] 01-envfile: exited 0.
[cont-init.d] 10-adduser: executing...
-------------------------------------
          _         ()
         | |  ___   _    __
         | | / __| | |  /  \
         | | \__ \ | | | () |
         |_| |___/ |_|  \__/
Brought to you by linuxserver.io
-------------------------------------
To support the app dev(s) visit:
SABnzbd: https://sabnzbd.org/donateTo support LSIO projects visit:
https://www.linuxserver.io/donate/
-------------------------------------
GID/UID
-------------------------------------
User uid:    1000
User gid:    1000
-------------------------------------
[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 20-config: executing...
[cont-init.d] 20-config: exited 0.
[cont-init.d] 90-sma-config: executing...
[90-sma-config] Creating python virtual environment and installing requirements
WARNING: The directory '/config/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[90-sma-config] Installing additional deluge/qbittorrent requirements
WARNING: The directory '/config/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[90-sma-config] Downloading and installing FFMpeg
[cont-init.d] 90-sma-config: exited 0.
[cont-init.d] 99-custom-scripts: executing...
[custom-init] no custom files found exiting...
[cont-init.d] 99-custom-scripts: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.

I'm reacting to [90-sma-config] Installing additional deluge/qbittorrent requirements.

What command are you running?

What command are you running?

Sorry, what do you mean?

What is the command you are executing that results in this error

I'm running the SMA-MOD within SAB. So I guess it's the SABPostProcess.py?

I guess show me your SAB configuration then, I'm thinking you are not using the virtual environment

This is in my autoProcess.ini
[Converter]
ffmpeg = /usr/local/bin/ffmpeg
ffprobe = /usr/local/bin/ffprobe

(I'm using the same SMA script folder as the one I'm using for qBit. Better to use separate ones?)
This is the path to the script folder in SAB config: (the same one I bound to in the container)

Was it what you were after?

Gotcha, so yeah you're not using the virtual environment, that's the problem (nothing to do with ffmpeg path)
When you execute a .py file unless you explicitly tell it what version of python you want it uses the operating systems version of python
For all SMA docker containers this wont work since the libraries are installed in a self container virtual python environment, and you need to provide the path
/usr/local/sma/venv/bin/python3 /usr/local/sma/SABPostProcess.py for example
I put together a SABPostProcess.sh file for SMA-MOD just now that does that for you but understand that's what needs to happen
Update and it should show up, point SAB to the .sh file instead of the .py file and it should work

Oh, gotcha! Thanks, mate :) I appreciate it!

Hi,
I did use the SABPostProcess.sh in categories of SAB and I still get this error:

Cat in SAB:

I deleted venv and rebuilt container. Script seems to be working. Then I had a systemexit 1 error and changed my autoProcess.ini config from h264vaapi to h264, x264. But I'm still getting the same error:
]
2021-05-28 08:17:48 - resources.mediaprocessor - INFO - Downloaded Subtitles
2021-05-28 08:17:48 - resources.mediaprocessor - INFO - []
2021-05-28 08:17:48 - resources.mediaprocessor - INFO - Starting conversion.
2021-05-28 08:17:48 - resources.mediaprocessor - INFO - FFmpeg command:
2021-05-28 08:17:48 - resources.mediaprocessor - INFO - ======================
2021-05-28 08:17:48 - resources.mediaprocessor - INFO - /usr/local/bin/ffmpeg -fix_sub_duration -init_hw_device vaapi=sma:/dev/dri/renderD128 -i /mnt/downloads/nzb/radarrMovie/Movie.mkv.original -map_metadata -1 -vcodec h264_vaapi -map 0:0 -field_order progressive -vb 2401k -metadata:s:v BPS=2401000 -metadata:s:v BPS-eng=2401000 -metadata:s:v title=FHD -metadata:s:v handler_name=FHD -filter_hw_device sma -vf format=nv12|vaapi,hwupload -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -filter:a:0 pan=stereo|FL=0.5*FC+0.707*FL+0.707*BL+0.5*LFE|FR=0.5*FC+0.707*FR+0.707*BR+0.5*LFE -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 aac -map 0:1 -ac:a:1 6 -b:a:1 256k -metadata:s:a:1 BPS=256000 -metadata:s:a:1 BPS-eng=256000 -metadata:s:a:1 "title=5.1 Channel" -metadata:s:a:1 "handler_name=5.1 Channel" -metadata:s:a:1 language=eng -disposition:a:1 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:2 copy -map 0:1 -metadata:s:a:2 "title=5.1 Channel" -metadata:s:a:2 "handler_name=5.1 Channel" -metadata:s:a:2 language=eng -disposition:a:2 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:0 srt -map 0:2 -metadata:s:s:0 title= -metadata:s:s:0 handler_name= -metadata:s:s:0 language=eng -disposition:s:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:1 srt -map 0:3 -metadata:s:s:1 title= -metadata:s:s:1 handler_name= -metadata:s:s:1 language=spa -disposition:s:1 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f matroska -threads 0 -metadata:g encoding_tool=SMA -y /mnt/downloads/nzb/radarr/Movie/Movie.mkv
2021-05-28 08:17:48 - resources.mediaprocessor - INFO - ======================
0%
2021-05-28 08:17:48 - resources.mediaprocessor - ERROR - Error converting file, FFMPEG error.
Traceback (most recent call last):
File "/usr/local/sma/resources/mediaprocessor.py", line 1613, in convert
for timecode, debug in conv:
File "/usr/local/sma/converter/__init__.py", line 321, in convert
postopts=postopts):
File "/usr/local/sma/converter/ffmpeg.py", line 724, in convert
total_output, pid=p.pid)
converter.ffmpeg.FFMpegConvertError: <unprintable FFMpegConvertError object>
2021-05-28 08:17:48 - resources.mediaprocessor - ERROR - /usr/local/bin/ffmpeg -fix_sub_duration -init_hw_device vaapi=sma:/dev/dri/renderD128 -i /mnt/downloads/nzb/radarr/Movie/Movie.mkv.original -map_metadata -1 -vcodec h264_vaapi -map 0:0 -field_order progressive -vb 2401k -metadata:s:v BPS=2401000 -metadata:s:v BPS-eng=2401000 -metadata:s:v title=FHD -metadata:s:v handler_name=FHD -filter_hw_device sma -vf format=nv12|vaapi,hwupload -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -filter:a:0 pan=stereo|FL=0.5*FC+0.707*FL+0.707*BL+0.5*LFE|FR=0.5*FC+0.707*FR+0.707*BR+0.5*LFE -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 aac -map 0:1 -ac:a:1 6 -b:a:1 256k -metadata:s:a:1 BPS=256000 -metadata:s:a:1 BPS-eng=256000 -metadata:s:a:1 title=5.1 Channel -metadata:s:a:1 handler_name=5.1 Channel -metadata:s:a:1 language=eng -disposition:a:1 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:2 copy -map 0:1 -metadata:s:a:2 title=5.1 Channel -metadata:s:a:2 handler_name=5.1 Channel -metadata:s:a:2 language=eng -disposition:a:2 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:0 srt -map 0:2 -metadata:s:s:0 title= -metadata:s:s:0 handler_name= -metadata:s:s:0 language=eng -disposition:s:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:1 srt -map 0:3 -metadata:s:s:1 title= -metadata:s:s:1 handler_name= -metadata:s:s:1 language=spa -disposition:s:1 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f matroska -threads 0 -metadata:g encoding_tool=SMA -y /mnt/downloads/nzb/radarr/Movie/Movie.mkv
2021-05-28 08:17:48 - resources.mediaprocessor - ERROR - ffmpeg version 4.4-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100
Device creation failed: -12.
Failed to set value 'vaapi=sma:/dev/dri/renderD128' for option 'init_hw_device': Cannot allocate memory
Error parsing global options: Cannot allocate memory

2021-05-28 08:17:48 - SABPostProcess - ERROR - Converting file failed /mnt/downloads/nzb/radarr/Movie/Movie.mkv.
2021-05-28 08:17:48 - SABPostProcess - ERROR - No valid files found for conversion in download, aborting.
2021-05-28 08:17:48 - SABPostProcess - ERROR - Unexpected exception.
Traceback (most recent call last):
File "/usr/local/sma/SABPostProcess.py", line 81, in <module>
sys.exit(1)
SystemExit: 1

This is still a VAAPI error, looks like  you haven't removed all the VAAPI based options
The fact that you're getting this error
Failed to set value 'vaapi=sma:/dev/dri/renderD128' for option 'init_hw_device': Cannot allocate memory Error parsing global options: Cannot allocate memory is a little surprising, I would think the ffmpeg builds from johnvansickle's site wouldn't support VAAPI at all.
2 paths going forward

Remove the rest of the VAAPI references include hwaccels, hwaccel-decoders, hwdevices, and hwaccel-output-formats. Specifically if you remove the vaapi reference from hwaccels none of the other options should be applied even if present
Try and get VAAPI working. You need to pass your hardware device to the docker container (add /dev/dri/renderD128:/dev/dri/renderD218 to your volumes). I just updated SMA-mod to include a permission script that would be needed to get VAAPI working as well, it was previously not included as I did not believe the build of FFMPEG includes with sma-mod would support it but worth a try. Ultimately VAAPI still may not work and the error you're seeing if corrected will still follow with a more tradiational "VAAPI not supported' error

Thank you for guiding me thorougly. I tried getting the VAAPI working but it fails, it doesn't seem to give me a different error than before. I also noticed that pymediainfo fails.
2021-05-29 13:28:57 - SABPostProcess - INFO - SAB post processing started.
2021-05-29 13:28:57 - resources.readsettings - INFO - /usr/local/sma/venv/bin/python3
2021-05-29 13:28:57 - resources.readsettings - INFO - Loading config file /usr/local/sma/config/autoProcess.ini.
2021-05-29 13:28:57 - SABPostProcess - INFO - Performing conversion
2021-05-29 13:28:57 - resources.mediaprocessor - ERROR - Pymediainfo exception.
Traceback (most recent call last):
File "/usr/local/sma/resources/mediaprocessor.py", line 312, in isValidSource
media_info = MediaInfo.parse(inputfile)
File "/usr/local/sma/venv/lib/python3.6/site-packages/pymediainfo/__init__.py", line 431, in parse
lib, handle, lib_version_str, lib_version = cls._get_library(library_file)
File "/usr/local/sma/venv/lib/python3.6/site-packages/pymediainfo/__init__.py", line 331, in _get_library
", ".join(library_paths), ", ".join(exceptions)
OSError: Failed to load library from libmediainfo.so.0 - libmediainfo.so.0: cannot open shared object file: No such file or directory
2021-05-29 13:28:57 - SABPostProcess - INFO - Processing file /mnt/downloads/nzb/radarr/Video/Video.mkv.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Input Data
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - {
"format": "matroska,webm",
"format-fullname": "Matroska / WebM",
"video": {
"index": 0,
"codec": "h264",
"pix_fmt": "yuv420p",
"profile": "high",
"fps": 23.976023976023978,
"framedata": {
"pix_fmt": "yuv420p",
"color_space": "bt709",
"color_primaries": "bt709",
"color_transfer": "bt709"
},
"dimensions": "1280x720",
"level": 3.1,
"field_order": "progressive"
},
"audio": [
{
"index": 1,
"codec": "eac3",
"bitrate": 224000,
"channels": 2,
"samplerate": 48000,
"language": "eng",
"disposition": "+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired"
}
],
"subtitle": [
{
"index": 2,
"codec": "subrip",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"language": "eng"
}
],
"attachment": []
}
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Reading video stream.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Video codec detected: h264.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Pix Fmt: yuv420p.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Profile: high.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Video codec parameters None.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Creating h264vaapi video stream from source stream 0.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Reading audio streams.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Audio detected for stream 1 - eac3 eng 2 channel.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Creating aac audio stream from source stream 1.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Copying audio stream from source stream 1 format eac3 [audio-copy-original].
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Default audio stream set to eng aac 2 channel stream [default-more-channels: True].
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Reading subtitle streams.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Text-based subtitle detected for stream 2 - subrip eng.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Creating srt subtitle stream from source stream 2.
2021-05-29 13:28:57 - resources.mediaprocessor - ERROR - No valid subtitle download languages detected, subtitles will not be downloaded.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Scanned for external subtitles and found 0 results in your approved languages.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Output Data
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - {
"source": [
"/mnt/downloads/nzb/radarr/Video/Video.mkv"
],
"format": "mkv",
"video": {
"codec": "h264vaapi",
"map": 0,
"bitrate": 4409.329999999999,
"crf": -1,
"maxrate": null,
"bufsize": null,
"level": 0.0,
"profile": null,
"preset": null,
"pix_fmt": null,
"field_order": "progressive",
"width": null,
"filter": null,
"params": null,
"framedata": null,
"title": "HD",
"debug": "video",
"device": "sma"
},
"audio": [
{
"map": 1,
"codec": "aac",
"channels": 2,
"bitrate": 224.0,
"filter": null,
"samplerate": null,
"language": "eng",
"disposition": "+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"bsf": null,
"title": "Stereo",
"debug": "universal-audio"
},
{
"map": 1,
"codec": "copy",
"channels": 2,
"language": "eng",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"title": "Stereo",
"debug": "audio-copy-original"
}
],
"subtitle": [
{
"map": 2,
"codec": "srt",
"language": "eng",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"title": null,
"debug": "subtitle.embed-subs"
}
],
"attachment": []
}
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Preopts
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - [
"-fix_sub_duration",
"-init_hw_device",
"vaapi=sma:/dev/dri/renderD128"
]
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Postopts
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - [
"-threads",
"0",
"-metadata:g",
"encoding_tool=SMA"
]
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Downloaded Subtitles
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - []
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - Starting conversion.
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - FFmpeg command:
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - ======================
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - /usr/local/bin/ffmpeg -fix_sub_duration -init_hw_device vaapi=sma:/dev/dri/renderD128 -i /mnt/downloads/nzb/radarr/Video/Video.mkv.original -map_metadata -1 -vcodec h264_vaapi -map 0:0 -field_order progressive -vb 4409k -metadata:s:v BPS=4409000 -metadata:s:v BPS-eng=4409000 -metadata:s:v title=HD -metadata:s:v handler_name=HD -filter_hw_device sma -vf format=nv12|vaapi,hwupload -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 224k -metadata:s:a:0 BPS=224000 -metadata:s:a:0 BPS-eng=224000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 copy -map 0:1 -metadata:s:a:1 title=Stereo -metadata:s:a:1 handler_name=Stereo -metadata:s:a:1 language=eng -disposition:a:1 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:0 srt -map 0:2 -metadata:s:s:0 title= -metadata:s:s:0 handler_name= -metadata:s:s:0 language=eng -disposition:s:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f matroska -threads 0 -metadata:g encoding_tool=SMA -y /mnt/downloads/nzb/radarr/Video/Video.mkv
2021-05-29 13:28:57 - resources.mediaprocessor - INFO - ======================
0%
2021-05-29 13:28:57 - resources.mediaprocessor - ERROR - Error converting file, FFMPEG error.
Traceback (most recent call last):
File "/usr/local/sma/resources/mediaprocessor.py", line 1613, in convert
for timecode, debug in conv:
File "/usr/local/sma/converter/__init__.py", line 321, in convert
postopts=postopts):
File "/usr/local/sma/converter/ffmpeg.py", line 724, in convert
total_output, pid=p.pid)
converter.ffmpeg.FFMpegConvertError: <unprintable FFMpegConvertError object>
2021-05-29 13:28:57 - resources.mediaprocessor - ERROR - /usr/local/bin/ffmpeg -fix_sub_duration -init_hw_device vaapi=sma:/dev/dri/renderD128 -i /mnt/downloads/nzb/radarr/Video/Video.mkv.original -map_metadata -1 -vcodec h264_vaapi -map 0:0 -field_order progressive -vb 4409k -metadata:s:v BPS=4409000 -metadata:s:v BPS-eng=4409000 -metadata:s:v title=HD -metadata:s:v handler_name=HD -filter_hw_device sma -vf format=nv12|vaapi,hwupload -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 224k -metadata:s:a:0 BPS=224000 -metadata:s:a:0 BPS-eng=224000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 copy -map 0:1 -metadata:s:a:1 title=Stereo -metadata:s:a:1 handler_name=Stereo -metadata:s:a:1 language=eng -disposition:a:1 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:0 srt -map 0:2 -metadata:s:s:0 title= -metadata:s:s:0 handler_name= -metadata:s:s:0 language=eng -disposition:s:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f matroska -threads 0 -metadata:g encoding_tool=SMA -y /mnt/downloads/nzb/radarr/Video/Video.mkv
2021-05-29 13:28:57 - resources.mediaprocessor - ERROR - ffmpeg version 4.4-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100
Device creation failed: -12.
Failed to set value 'vaapi=sma:/dev/dri/renderD128' for option 'init_hw_device': Cannot allocate memory
Error parsing global options: Cannot allocate memory

2021-05-29 13:28:57 - SABPostProcess - ERROR - Converting file failed /mnt/downloads/nzb/radarr/Video/Video.mkv.
2021-05-29 13:28:57 - SABPostProcess - ERROR - No valid files found for conversion in download, aborting.
2021-05-29 13:28:57 - SABPostProcess - ERROR - Unexpected exception.
Traceback (most recent call last):
File "/usr/local/sma/SABPostProcess.py", line 81, in <module>
sys.exit(1)
SystemExit: 1

Does it work without VAAPI?
If you open your docker container and navigate to the /dev/dri directory do you have a device named renderD128? What are the permissions on said device
And finally, pymediainfo may fail, that doesn't impact much though sma-mod should try to install the needed packages to allow it to work

It's there
root@08fefb5c3b68:/dev/dri# ls -lah
total 0
drwxr-xr-x 2 root root        80 May 29 13:40 .
drwxr-xr-x 6 root root       360 May 29 13:40 ..
crw-rw---- 1 root video 226,   0 May 29 13:40 card0
crw-rw---- 1 root video 226, 128 May 29 13:40 renderD128

Permissions look ok, maybe try passing it as a device instead of a volume, think maybe I gave bad advice there

That's what the line from my config looks like (its not in volumes)

I actually did put it in devices and not volumes haha :)

If you run ffmpeg -hwaccels does VAAPI show up under supported?

Hmm weird, I got it working now. I just rebuilt it like 50 times haha.
I run on an EX42-NVMe from Hetzner, I think it's an i7 3.60GHz.

Make sure its one of the newer gen ones that supports VAAPI, probably fine especially since you said you got it working

Make sure its one of the newer gen ones that supports VAAPI, probably fine especially since you said you got it working

Yeah, I've been encoding with the same machine sort and VAAPI over a year, no problem :)

Hi mate, just a heads up! Everytime I make a new container with your mod it get these errors, no matter what I do. I end up having to install them manually everytime.
ModuleNotFoundError: No module named 'tmdbsimple'

And when trying to install requirements manually I get this:
ModuleNotFoundError: No module named 'setuptools'

What command are you running that's generating that error?

I'm running the SABPostProcess.sh. Tried deleting the venv and rebuilding it. Doesn't seem as the container install the dependencies, or that they get deleted after creation of container somehow.
Best,
Morgan
On 8 Jun 2021, 12:07 +0200, Michael Higgins @.***>, wrote:

What command are you running that's generating that error?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or unsubscribe.

What command are you using to install the missing dependencies? One example should be fine
The command that handles the requirement install is as follows
# check virtual python environment
if [[ -f "${SMA_PATH}/venv/bin/python3" ]]; then
    :
else
    echo "[90-sma-config] Creating python virtual environment and installing requirements"
    python3 -m venv ${SMA_PATH}/venv
    ${SMA_PATH}/venv/bin/python3 -m pip install -q --upgrade pip > /dev/null
    ${SMA_PATH}/venv/bin/pip install -q \
        -r ${SMA_PATH}/setup/requirements.txt > /dev/null
    if [[ -z "${SMA_RS}" && -f /usr/bin/apt ]]; then
    echo "[90-sma-config] Installing additional deluge/qbittorrent requirements"
    ${SMA_PATH}/venv/bin/pip install -q \
        -r ${SMA_PATH}/setup/requirements-deluge.txt \
        -r ${SMA_PATH}/setup/requirements-qbittorrent.txt > /dev/null
    fi
fi

Its not particularly complicated but it won't attempt to create the venv or install the dependencies if there already is a python executable at ${SMA_PATH}/venv/bin/python3. Is there any reason that executable is present on the containers you're building? I haven't had this issue nor can I reproduce it. I also haven't made any changes recently to sma-mod that should have changed this behavior
Can you post the docker start up logs from a container on first launch after creation to see if there's any errors being thrown during 90-sma-config's run

For example if I rebuild my deluge container and then run docker logs deluge I get the following output
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
[mod-init] Attempting to run Docker Modification Logic
[mod-init] Applying mdhiggins/sma-mod:latest files to container
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-envfile: executing...
[cont-init.d] 01-envfile: exited 0.
[cont-init.d] 10-adduser: executing...

-------------------------------------
          _         ()
         | |  ___   _    __
         | | / __| | |  /  \
         | | \__ \ | | | () |
         |_| |___/ |_|  \__/

Brought to you by linuxserver.io
-------------------------------------

To support LSIO projects visit:
https://www.linuxserver.io/donate/
-------------------------------------
GID/UID
-------------------------------------

User uid:    1001
User gid:    1001
-------------------------------------

[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 30-config: executing...
[cont-init.d] 30-config: exited 0.
[cont-init.d] 50-gid-video: executing...
[cont-init.d] 50-gid-video: exited 0.
[cont-init.d] 90-sma-config: executing...
[90-sma-config] Creating python virtual environment and installing requirements
[90-sma-config] Installing additional deluge/qbittorrent requirements
[90-sma-config] Downloading and installing FFMpeg
Config file not found, creating /usr/local/sma/config/autoProcess.ini.
[cont-init.d] 90-sma-config: exited 0.
[cont-init.d] 99-custom-scripts: executing...
[custom-init] no custom files found exiting...
[cont-init.d] 99-custom-scripts: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.

The important lines being early on when the mod is applied and then the lines starting with [90-sma-config] indicated that the venv and extra requirements are installed and FFMPEG is downloaded

What command are you using to install the missing dependencies? One example should be fine

When installing them manully I bash in to container and cd /usr/local/sma/setup and then pip3 install setuptools && pip3 install -r requirements.txt

The important lines being early on when the mod is applied and then the lines starting with [90-sma-config] indicated that the venv and extra requirements are installed and FFMPEG is downloaded

This is what I get on first start up on a completely new container:
[mod-init] Curl/JQ was not found on this system for Docker mods installing
Get:1 http://ppa.launchpad.net/jcfp/nobetas/ubuntu bionic InRelease [15.4 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:3 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu bionic InRelease [15.4 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:6 http://ppa.launchpad.net/jcfp/nobetas/ubuntu bionic/main Sources [2,752 B]
Get:7 http://ppa.launchpad.net/jcfp/nobetas/ubuntu bionic/main amd64 Packages [2,123 B]
Get:8 http://archive.ubuntu.com/ubuntu bionic/main Sources [1,063 kB]
Get:9 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu bionic/main Sources [2,306 B]
Get:10 http://ppa.launchpad.net/jcfp/sab-addons/ubuntu bionic/main amd64 Packages [1,898 B]
Get:11 http://archive.ubuntu.com/ubuntu bionic/universe Sources [11.5 MB]
Get:12 http://archive.ubuntu.com/ubuntu bionic/restricted Sources [5,823 B]
Get:13 http://archive.ubuntu.com/ubuntu bionic/multiverse Sources [216 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1,344 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:17 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Sources [16.4 kB]
Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Sources [26.0 kB]
Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/universe Sources [573 kB]
Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main Sources [642 kB]
Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2,184 kB]
Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [480 kB]
Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2,616 kB]
Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [33.5 kB]
Get:26 http://archive.ubuntu.com/ubuntu bionic-security/main Sources [320 kB]
Get:27 http://archive.ubuntu.com/ubuntu bionic-security/multiverse Sources [5,571 B]
Get:28 http://archive.ubuntu.com/ubuntu bionic-security/restricted Sources [22.6 kB]
Get:29 http://archive.ubuntu.com/ubuntu bionic-security/universe Sources [347 kB]
Get:30 http://archive.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [24.7 kB]
Get:31 http://archive.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [450 kB]
Get:32 http://archive.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,414 kB]
Get:33 http://archive.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,184 kB]
Fetched 37.5 MB in 3s (11.1 MB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
curl is already the newest version (7.58.0-2ubuntu3.13).
The following additional packages will be installed:
  libjq1 libonig4
The following NEW packages will be installed:
  jq libjq1 libonig4
0 upgraded, 3 newly installed, 0 to remove and 5 not upgraded.
Need to get 276 kB of archives.
After this operation, 930 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB]
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 3.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, <> line 3.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin:
Fetched 276 kB in 0s (1,170 kB/s)
Selecting previously unselected package libonig4:amd64.
(Reading database ...

Preparing to unpack .../libonig4_6.7.0-1_amd64.deb ...
Unpacking libonig4:amd64 (6.7.0-1) ...
Selecting previously unselected package libjq1:amd64.
Preparing to unpack .../libjq1_1.5+dfsg-2_amd64.deb ...
Unpacking libjq1:amd64 (1.5+dfsg-2) ...
Selecting previously unselected package jq.
Preparing to unpack .../jq_1.5+dfsg-2_amd64.deb ...
Unpacking jq (1.5+dfsg-2) ...
Setting up libonig4:amd64 (6.7.0-1) ...
Setting up libjq1:amd64 (1.5+dfsg-2) ...
Setting up jq (1.5+dfsg-2) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
[mod-init] Attempting to run Docker Modification Logic
[mod-init] Applying mdhiggins/sma-mod:latest files to container
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-envfile: executing...
[cont-init.d] 01-envfile: exited 0.
[cont-init.d] 10-adduser: executing...
-------------------------------------
          _         ()
         | |  ___   _    __
         | | / __| | |  /  \
         | | \__ \ | | | () |
         |_| |___/ |_|  \__/
Brought to you by linuxserver.io
-------------------------------------
To support the app dev(s) visit:
SABnzbd: https://sabnzbd.org/donateTo support LSIO projects visit:
https://www.linuxserver.io/donate/
-------------------------------------
GID/UID
-------------------------------------
User uid:    1000
User gid:    1000
-------------------------------------
[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 20-config: executing...
[cont-init.d] 20-config: exited 0.
[cont-init.d] 50-gid-video: executing...
[cont-init.d] 50-gid-video: exited 0.
[cont-init.d] 90-sma-config: executing...
[90-sma-config] Downloading and installing FFMpeg
[cont-init.d] 90-sma-config: exited 0.
[cont-init.d] 99-custom-scripts: executing...
[custom-init] no custom files found exiting...
[cont-init.d] 99-custom-scripts: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.

As you can see it doesn't say that it's installing the Python VE.

Reading package lists...
Building dependency tree...
Reading state information...
curl is already the newest version (7.58.0-2ubuntu3.13).
The following additional packages will be installed:
  libjq1 libonig4
The following NEW packages will be installed:
  jq libjq1 libonig4
0 upgraded, 3 newly installed, 0 to remove and 5 not upgraded.
Need to get 276 kB of archives.
After this operation, 930 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB]
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 3.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, <> line 3.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin:
Fetched 276 kB in 0s (627 kB/s)
Selecting previously unselected package libonig4:amd64.
(Reading database ...

Preparing to unpack .../libonig4_6.7.0-1_amd64.deb ...
Unpacking libonig4:amd64 (6.7.0-1) ...
Selecting previously unselected package libjq1:amd64.
Preparing to unpack .../libjq1_1.5+dfsg-2_amd64.deb ...
Unpacking libjq1:amd64 (1.5+dfsg-2) ...
Selecting previously unselected package jq.
Preparing to unpack .../jq_1.5+dfsg-2_amd64.deb ...
Unpacking jq (1.5+dfsg-2) ...
Setting up libonig4:amd64 (6.7.0-1) ...
Setting up libjq1:amd64 (1.5+dfsg-2) ...
Setting up jq (1.5+dfsg-2) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
[mod-init] Attempting to run Docker Modification Logic
[mod-init] Applying mdhiggins/sma-mod:latest files to container
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-envfile: executing...
[cont-init.d] 01-envfile: exited 0.
[cont-init.d] 10-adduser: executing...
-------------------------------------
          _         ()
         | |  ___   _    __
         | | / __| | |  /  \
         | | \__ \ | | | () |
         |_| |___/ |_|  \__/
Brought to you by linuxserver.io
-------------------------------------
To support the app dev(s) visit:
SABnzbd: https://sabnzbd.org/donateTo support LSIO projects visit:
https://www.linuxserver.io/donate/
-------------------------------------
GID/UID
-------------------------------------
User uid:    1000
User gid:    1000
-------------------------------------
[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 20-config: executing...
[cont-init.d] 20-config: exited 0.
[cont-init.d] 50-gid-video: executing...
[cont-init.d] 50-gid-video: exited 0.
[cont-init.d] 90-sma-config: executing...
[90-sma-config] Creating python virtual environment and installing requirements
WARNING: The directory '/config/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[90-sma-config] Installing additional deluge/qbittorrent requirements
WARNING: The directory '/config/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[90-sma-config] Downloading and installing FFMpeg
[cont-init.d] 90-sma-config: exited 0.
[cont-init.d] 99-custom-scripts: executing...
[custom-init] no custom files found exiting...
[cont-init.d] 99-custom-scripts: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.

The main difference I find is this [90-sma-config] Creating python virtual environment and installing requirements

It looks like from the command to install the requirements you're using that you're installing them on the main python library on the OS and not the virtual environment, which means you're having the dependencies fail because you're not using the virtual environment

Why do I need to install them manually in the first place?
If I create a new sab container with your mod and use the PostProcess.sh, I then get an error saying that I'm missing tmdbsimple.
I thought all requirements were installed upon container creation.

That's what we need to figure out, but if installing them on the main python instance fixes it then it means for some reason things are being executed from the main python instance and not the virtual environment
On my tests I'm not able to reproduce your issue and and I don't have to manually install these extra dependencies so I'm not sure what's going wrong
Maybe post some screenshots of your SAB configuration and I can take a look
If you do a fresh container creation, can you run manual.py and see if you get the same errors?
/usr/local/sma/venv/bin/python3 /usr/local/sma/manual.py
Just see if it starts, you don't need to process a file to test the dependencies

Sure, I'll post the results here in a sec. Just going to create another container and start the manual.py.
By SAB configuration, what do you mean?

Reading package lists...
Building dependency tree...
Reading state information...
curl is already the newest version (7.58.0-2ubuntu3.13).
The following additional packages will be installed:
libjq1 libonig4
The following NEW packages will be installed:
jq libjq1 libonig4
0 upgraded, 3 newly installed, 0 to remove and 5 not upgraded.
Need to get 276 kB of archives.
After this operation, 930 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB]
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 3.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, <> line 3.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin:
Fetched 276 kB in 0s (1,166 kB/s)
Selecting previously unselected package libonig4:amd64.
(Reading database ...

Preparing to unpack .../libonig4_6.7.0-1_amd64.deb ...
Unpacking libonig4:amd64 (6.7.0-1) ...
Selecting previously unselected package libjq1:amd64.
Preparing to unpack .../libjq1_1.5+dfsg-2_amd64.deb ...
Unpacking libjq1:amd64 (1.5+dfsg-2) ...
Selecting previously unselected package jq.
Preparing to unpack .../jq_1.5+dfsg-2_amd64.deb ...
Unpacking jq (1.5+dfsg-2) ...
Setting up libonig4:amd64 (6.7.0-1) ...
Setting up libjq1:amd64 (1.5+dfsg-2) ...
Setting up jq (1.5+dfsg-2) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
[mod-init] Attempting to run Docker Modification Logic
[mod-init] Applying mdhiggins/sma-mod:latest files to container
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-envfile: executing...
[cont-init.d] 01-envfile: exited 0.
[cont-init.d] 10-adduser: executing...
-------------------------------------
    _         ()
   | |  ___   _    __
   | | / __| | |  /  \
   | | \__ \ | | | () |
   |_| |___/ |_|  \__/
Brought to you by linuxserver.io
-------------------------------------
To support the app dev(s) visit:
SABnzbd: https://sabnzbd.org/donateTo support LSIO projects visit:
https://www.linuxserver.io/donate/
-------------------------------------
GID/UID
-------------------------------------
User uid:    1000
User gid:    1000
-------------------------------------
[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 20-config: executing...
[cont-init.d] 20-config: exited 0.
[cont-init.d] 50-gid-video: executing...
[cont-init.d] 50-gid-video: exited 0.
[cont-init.d] 90-sma-config: executing...
[90-sma-config] Downloading and installing FFMpeg
[cont-init.d] 90-sma-config: exited 0.
[cont-init.d] 99-custom-scripts: executing...
[custom-init] no custom files found exiting...
[cont-init.d] 99-custom-scripts: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
2021-06-10 10:24:04,873::INFO::[SABnzbd:1160] --------------------------------
2021-06-10 10:24:04,878::INFO::[SABnzbd:1161] SABnzbd.py-3.3.0
2021-06-10 10:24:04,878::INFO::[SABnzbd:1171] Commit = e8206fbdd921b72636f6d1705624493ef907938c
2021-06-10 10:24:04,881::INFO::[SABnzbd:1173] Full executable path = /app/sabnzbd/SABnzbd.py
2021-06-10 10:24:04,881::INFO::[SABnzbd:1174] Arguments = "/app/sabnzbd/SABnzbd.py" "--config-file" "/config" "--server" "<IP_ADDRESS>:8080"
2021-06-10 10:24:04,881::INFO::[SABnzbd:1175] Python-version = 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0]
2021-06-10 10:24:04,881::INFO::[SABnzbd:1176] Dockerized = True
2021-06-10 10:24:04,882::INFO::[SABnzbd:1177] CPU architecture = x86_64
2021-06-10 10:24:04,907::INFO::[SABnzbd:1180] Platform = posix - Linux-4.15.0-143-generic-x86_64-with-Ubuntu-18.04-bionic
2021-06-10 10:24:04,908::INFO::[SABnzbd:1186] Preferred encoding = UTF-8
2021-06-10 10:24:04,908::INFO::[SABnzbd:1198] SSL version = OpenSSL 1.1.1  11 Sep 2018
2021-06-10 10:24:04,909::INFO::[SABnzbd:1207] Certifi version = 2021.05.30
2021-06-10 10:24:04,909::INFO::[SABnzbd:1208] Loaded additional certificates from /usr/local/lib/python3.6/dist-packages/certifi/cacert.pem
2021-06-10 10:24:04,909::INFO::[SABnzbd:1228] Using INI file /config/sabnzbd.ini
2021-06-10 10:24:04,910::INFO::[filesystem:364] download_dir directory: /config/Downloads/incomplete does not exist, try to create it
2021-06-10 10:24:04,910::INFO::[filesystem:651] Creating directories: /config/Downloads/incomplete
2021-06-10 10:24:04,934::INFO::[__init__:924] [N/A] /config/admin/totals10.sab missing
2021-06-10 10:24:06,832::INFO::[postproc:139] Loading postproc queue
2021-06-10 10:24:06,832::INFO::[__init__:924] [N/A] /config/admin/postproc2.sab missing
2021-06-10 10:24:06,832::INFO::[__init__:924] [N/A] /config/admin/watched_data2.sab missing
2021-06-10 10:24:06,832::INFO::[__init__:924] [N/A] /config/admin/Rating.sab missing
2021-06-10 10:24:06,833::INFO::[__init__:924] [N/A] /config/admin/rss_data.sab missing
2021-06-10 10:24:06,833::INFO::[scheduler:190] Scheduling RSS interval task every 60 min (delay=53)
2021-06-10 10:24:06,833::INFO::[scheduler:201] Scheduling VersionCheck on day 2 at 3:3
2021-06-10 10:24:06,833::INFO::[scheduler:215] Setting schedule for midnight BPS reset
2021-06-10 10:24:06,833::INFO::[scheduler:218] Setting schedule for server expiration check
2021-06-10 10:24:06,833::INFO::[scheduler:223] Setting scheduler for server quota check
2021-06-10 10:24:06,834::INFO::[__init__:924] [N/A] /config/admin/queue10.sab missing
2021-06-10 10:24:06,836::INFO::[config:905] Writing settings to INI file /config/sabnzbd.ini
2021-06-10 10:24:06,840::INFO::[__init__:334] All processes started
2021-06-10 10:24:06,840::INFO::[SABnzbd:324] Template location for Glitter is /app/sabnzbd/interfaces/Glitter
2021-06-10 10:24:06,841::INFO::[SABnzbd:324] Template location for Config is /app/sabnzbd/interfaces/Config
2021-06-10 10:24:06,859::INFO::[misc:1066] [N/A] Running external command: ['/usr/bin/unrar']
2021-06-10 10:24:06,898::INFO::[misc:1066] [N/A] Running external command: ['/usr/bin/par2', '-h']
2021-06-10 10:24:06,968::INFO::[SABnzbd:411] SABYenc module (v4.0.2)... found!
2021-06-10 10:24:06,969::INFO::[SABnzbd:430] Cryptography module (v3.4.7)... found!
2021-06-10 10:24:06,969::INFO::[SABnzbd:435] par2 binary... found (/usr/bin/par2)
2021-06-10 10:24:06,969::INFO::[SABnzbd:442] UNRAR binary... found (/usr/bin/unrar)
2021-06-10 10:24:06,969::INFO::[SABnzbd:452] UNRAR binary version 6.00
2021-06-10 10:24:06,970::INFO::[SABnzbd:460] 7za binary... found (/usr/bin/7za)
2021-06-10 10:24:06,970::INFO::[SABnzbd:471] nice binary... found (/usr/bin/nice)
2021-06-10 10:24:06,970::INFO::[SABnzbd:475] ionice binary... found (/usr/bin/ionice)
2021-06-10 10:24:06,971::INFO::[SABnzbd:1417] Starting web-interface on <IP_ADDRESS>:8080
2021-06-10 10:24:06,972::INFO::[_cplogging:216] [10/Jun/2021:10:24:06] ENGINE Bus STARTING
2021-06-10 10:24:07,110::INFO::[_cplogging:216] [10/Jun/2021:10:24:07] ENGINE Serving on http://<IP_ADDRESS>:8080
2021-06-10 10:24:07,110::INFO::[_cplogging:216] [10/Jun/2021:10:24:07] ENGINE Bus STARTED
2021-06-10 10:24:07,110::INFO::[SABnzbd:1454] Starting SABnzbd.py-3.3.0
2021-06-10 10:24:07,118::INFO::[postproc:237] Completed Download Folder /config/Downloads/complete is not on FAT
2021-06-10 10:24:07,186::INFO::[dirscanner:117] Dirscanner starting up
2021-06-10 10:24:07,218::INFO::[panic:239] Launching browser with http://<IP_ADDRESS>:8080/sabnzbd
2021-06-10 10:24:07,218::INFO::[notifier:122] Sending notification: SABnzbd - SABnzbd 3.3.0 started (type=startup, job_cat=None)
2021-06-10 10:24:07,426::INFO::[zconfig:62] No bonjour/zeroconf support installed
2021-06-10 10:24:07,428::INFO::[ssdp:108] Serving SSDP on <IP_ADDRESS> as SABnzbd
2021-06-10 10:24:37,444::INFO::[config:905] Writing settings to INI file /config/sabnzbd.ini

And here is the output from manual.py after I ran /usr/local/sma/venv/bin/python3 /usr/local/sma/manual.py
root@0edeae04db17:/# /usr/local/sma/venv/bin/python3 /usr/local/sma/manual.py
Manual processor started.
Python 64-bit 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0].
Guessit version: 3.3.1.
/usr/local/sma/venv/bin/python3
Loading config file /usr/local/sma/config/autoProcess.ini.
Enter path to file:
#:

Looks to me like it's running without issues.
If I run /usr/local/sma/venv/bin/pip3 install -r /usr/local/sma/setup/requirements.txt I get Requirement already satisfied on all dependencies. But if I run the PostProcess.sh from SAB to process a file I will get ModuleNotFoundError: No module named 'tmdbsimple'

Can you post a screenshot of your script config from within the SAB web interface?

Do you mean these?

If you modify SABPostProcess.py to look like this at the top
#!/usr/bin/env python3

import os
import sys
print(sys.executable)

Adding the print(sys.executable) line that should print out the path to python that's being used before the import error occurs. Can you do that and report back where it's launching from? I suspect this is the issue but can't quite figure out why its doing this

Alright I just decided to make my own SAB container and test this with SMA mod, made no modifications and it worked fine, no dependency errors
  sabnzbd:
    image: ghcr.io/linuxserver/sabnzbd
    container_name: sabnzbd
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - DOCKER_MODS=mdhiggins/sma-mod:latest
    volumes:
      - /opt/appdata/sabnzbd:/config
      - /mnt/storage/downloads:/downloads
      - /opt/appdata/sma:/usr/local/sma/config

/usr/local/sma/venv/bin/python3
2021-06-11 15:08:28 - SABPostProcess - INFO - SAB post processing started.
2021-06-11 15:08:28 - resources.readsettings - INFO - /usr/local/sma/venv/bin/python3
2021-06-11 15:08:28 - resources.readsettings - INFO - Loading config file /usr/local/sma/config/autoProcess.ini.
2021-06-11 15:08:28 - SABPostProcess - INFO - Performing conversion
2021-06-11 15:08:31 - resources.mediaprocessor - ERROR - Pymediainfo exception.
Traceback (most recent call last):
File "/usr/local/sma/resources/mediaprocessor.py", line 312, in isValidSource
media_info = MediaInfo.parse(inputfile)
File "/usr/local/sma/venv/lib/python3.6/site-packages/pymediainfo/__init__.py", line 431, in parse
lib, handle, lib_version_str, lib_version = cls._get_library(library_file)
File "/usr/local/sma/venv/lib/python3.6/site-packages/pymediainfo/__init__.py", line 331, in _get_library
", ".join(library_paths), ", ".join(exceptions)
OSError: Failed to load library from libmediainfo.so.0 - libmediainfo.so.0: cannot open shared object file: No such file or directory
2021-06-11 15:08:31 - SABPostProcess - INFO - Processing file /config/Downloads/complete/Mythic.Quest.Ravens.Banquet.S02E07.MULTi.1080p.WEB.H264-HERC.nzb.queued.1/9038c5b0cc094fb2b19198a6ef13a9bd.mkv.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Input Data
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - {
"format": "matroska,webm",
"format-fullname": "Matroska / WebM",
"video": {
"index": 0,
"codec": "h264",
"pix_fmt": "yuv420p",
"profile": "high",
"fps": 23.976023976023978,
"framedata": {
"pix_fmt": "yuv420p",
"color_space": "bt709",
"color_primaries": "bt709",
"color_transfer": "bt709",
"side_data_list": [
{
"side_data_type": "H.26[45] User Data Unregistered SEI message"
}
]
},
"dimensions": "1920x960",
"level": 4.0,
"field_order": "progressive"
},
"audio": [
{
"index": 1,
"codec": "eac3",
"bitrate": 768000,
"channels": 6,
"samplerate": 48000,
"language": "fra",
"disposition": "+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired"
},
{
"index": 2,
"codec": "eac3",
"bitrate": 768000,
"channels": 6,
"samplerate": 48000,
"language": "eng",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired"
}
],
"subtitle": [
{
"index": 3,
"codec": "subrip",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"language": "fra"
},
{
"index": 4,
"codec": "subrip",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"language": "fra"
},
{
"index": 5,
"codec": "subrip",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"language": "eng"
},
{
"index": 6,
"codec": "subrip",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"language": "eng"
}
],
"attachment": []
}
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Reading video stream.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Video codec detected: h264.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Pix Fmt: yuv420p.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Profile: high.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Video codec parameters None.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Creating copy video stream from source stream 0.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Reading audio streams.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Audio detected for stream 2 - eac3 eng 6 channel.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Creating aac audio stream source audio stream 2 [universal-audio].
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Creating ac3 audio stream from source stream 2.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Audio detected for stream 1 - eac3 fra 6 channel.
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Default audio stream set to eng aac 2 channel stream [default-more-channels: False].
2021-06-11 15:08:31 - resources.mediaprocessor - INFO - Reading subtitle streams.
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Text-based subtitle detected for stream 3 - subrip fra.
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Text-based subtitle detected for stream 4 - subrip fra.
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Text-based subtitle detected for stream 5 - subrip eng.
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Creating mov_text subtitle stream from source stream 5.
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Text-based subtitle detected for stream 6 - subrip eng.
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Creating mov_text subtitle stream from source stream 6.
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Scanned for external subtitles and found 0 results in your approved languages.
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Output Data
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - {
"source": [
"/config/Downloads/complete/Mythic.Quest.Ravens.Banquet.S02E07.MULTi.1080p.WEB.H264-HERC.nzb.queued.1/9038c5b0cc094fb2b19198a6ef13a9bd.mkv"
],
"format": "mp4",
"video": {
"codec": "copy",
"map": 0,
"bitrate": 9452.979749999999,
"crf": 24,
"maxrate": null,
"bufsize": null,
"level": 0.0,
"profile": null,
"preset": null,
"pix_fmt": null,
"field_order": "progressive",
"width": null,
"filter": null,
"params": null,
"framedata": null,
"title": "FHD",
"debug": "video"
},
"audio": [
{
"map": 2,
"codec": "aac",
"channels": 2,
"bitrate": 256,
"samplerate": null,
"filter": "pan=stereo|FL=0.5*FC+0.707*FL+0.707*BL+0.5*LFE|FR=0.5*FC+0.707*FR+0.707*BR+0.5*LFE",
"language": "eng",
"disposition": "+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"title": "Stereo",
"debug": "universal-audio"
},
{
"map": 2,
"codec": "ac3",
"channels": 6,
"bitrate": 1536,
"filter": null,
"samplerate": null,
"language": "eng",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"bsf": null,
"title": "5.1 Channel",
"debug": "audio"
}
],
"subtitle": [
{
"map": 5,
"codec": "mov_text",
"language": "eng",
"disposition": "+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"title": null,
"debug": "subtitle.embed-subs"
},
{
"map": 6,
"codec": "mov_text",
"language": "eng",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"title": null,
"debug": "subtitle.embed-subs"
}
],
"attachment": []
}
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Preopts
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - [
"-fix_sub_duration"
]
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Postopts
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - [
"-threads",
"0",
"-metadata:g",
"encoding_tool=SMA"
]
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - Starting conversion.
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - FFmpeg command:
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - ======================
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - /usr/local/bin/ffmpeg -fix_sub_duration -i /config/Downloads/complete/Mythic.Quest.Ravens.Banquet.S02E07.MULTi.1080p.WEB.H264-HERC.nzb.queued.1/9038c5b0cc094fb2b19198a6ef13a9bd.mkv -vcodec copy -map 0:0 -metadata:s:v title=FHD -metadata:s:v handler_name=FHD -c:a:0 aac -map 0:2 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -filter:a:0 pan=stereo|FL=0.5*FC+0.707*FL+0.707*BL+0.5*LFE|FR=0.5*FC+0.707*FR+0.707*BR+0.5*LFE -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 ac3 -map 0:2 -ac:a:1 6 -b:a:1 1536k -metadata:s:a:1 BPS=1536000 -metadata:s:a:1 BPS-eng=1536000 -metadata:s:a:1 "title=5.1 Channel" -metadata:s:a:1 "handler_name=5.1 Channel" -metadata:s:a:1 language=eng -disposition:a:1 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:0 mov_text -map 0:5 -metadata:s:s:0 title= -metadata:s:s:0 handler_name= -metadata:s:s:0 language=eng -disposition:s:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:s:1 mov_text -map 0:6 -metadata:s:s:1 title= -metadata:s:s:1 handler_name= -metadata:s:s:1 language=eng -disposition:s:1 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -y /config/Downloads/complete/Mythic.Quest.Ravens.Banquet.S02E07.MULTi.1080p.WEB.H264-HERC.nzb.queued.1/9038c5b0cc094fb2b19198a6ef13a9bd.mp4
2021-06-11 15:08:34 - resources.mediaprocessor - INFO - ======================
0%
...
100%
2021-06-11 15:09:18 - resources.mediaprocessor - INFO - /config/Downloads/complete/Mythic.Quest.Ravens.Banquet.S02E07.MULTi.1080p.WEB.H264-HERC.nzb.queued.1/9038c5b0cc094fb2b19198a6ef13a9bd.mp4 created.
2021-06-11 15:09:19 - SABPostProcess - INFO - Successfully processed /config/Downloads/complete/Mythic.Quest.Ravens.Banquet.S02E07.MULTi.1080p.WEB.H264-HERC.nzb.queued.1/9038c5b0cc094fb2b19198a6ef13a9bd.mkv.
2021-06-11 15:09:19 - SABPostProcess - INFO - Bypassing any further processing as per category.

Pymediainfo fails (which is optional anyway) but everything else is fine

I just noticed you're mounting the entire script directory -v /opt/scripts/sma/nzb:/usr/local/sma \ to a persistent volume, is it possible you have some old or messed up version of SABPostProcess.sh in there? You really should only be mounting the sma config directory, see my example

Sorry, I was away for a couple of days.
Ah, that's just me having different SMA for different clients. I have one for torrent and one for nzb.
Here's my SABPostProcess.sh:
#!/usr/bin/env bash

# env check
if [[ -z "${SMA_PATH}" ]]; then
    export SMA_PATH="/usr/local/sma"
fi

$SMA_PATH/venv/bin/python3 $SMA_PATH/SABPostProcess.py "$@"

You want me to add the print(sys.executable) line still? :)

Still no reason to be mounting the whole script folder and that could definitely create problems
Something in your setup is messed up since I'm not having this issue on a fresh container
So yeah try adding that line

What should I mount? This is what's inside /opt/scripts/sma/nzb/:

Just the config folder

/opt/appdata/sma:/usr/local/sma/config

Oh!

/opt/scripts/sma/nzb/config:/usr/local/sma/config
Use that

The fact that the venv folder is also included in the persistent mount directory might also be creating conflicts that's screwing up the dependencies too, I think adjusting this and maybe doing a fresh container will fix your issue

/opt/scripts/sma/nzb/config:/usr/local/sma/config
Use that

Should I change the path in my clients and the unrar helper script from /usr/local/sma to /usr/local/sma/config/ as well?

Only if you move those scripts there
You could map a custom sub directory for those custom solutions without mapping the root

/opt/scripts/sma/nzb/config:/usr/local/sma/config
Use that

Thanks @mdhiggins, this worked like a charm! I haven't had any errors ever since I deploy containers with /config/ at the end of the SMA path.
Thanks for your help and your work as always!

Hi mate, Using this thread instead of opening a new one.
Currently, I have multiple SMA directories for different clients that do different stuff. Is it possible to use 1 master SMA folder with different autoProcess's for specific clients?
E.G:
I have one for anime, those I keep H265/HEVC.
I have one that I only use for manual encoding, with a specific output path.
And two others...

Yeah this is doable. For docker you can just use volumes to map different config files
Beyond that just set your SMA_CONFIG environment variable before calling the script to use different configs
https://github.com/mdhiggins/sickbeard_mp4_automator#override-configuration-path
Just make a simple wrapper script and set that value before things launch to the path of the config you want to apply


Homework-14-modules-routing

GitHub Pages

Love the design! Well done

Love the design! Well done